Every higher version of OS is designed for new hardwares. So using latest OS with old hardware, there will be some slowness. Software cannot take full advantage of the required hardware. iPhone 4s has only 512mb RAM, and runs on A5 chip. And main difference is on graphics rendering.
Same thing happened with my Macbook pro when I upgraded last week to Yesomite. I have 2011 Macbook pro i7, which was running great until I upgraded. Now my 8gb ram will be completely used up even when I am not using any heavy apps.

So bottom line, if you don't want to buy new hardware, don't upgrade to new software. They are designed to work with new hardware.