... if it is just school web or blog hosting, fine. But if you try something compititive and once your users cross 2000 then there is security threats from every corner. Security is the main challange in home based web server. You know what I mean once you expose your system to the external world..

One more thing, dont expect any financial gain unless your site is drawing really good traffic... However, it is really rewarding in terms of technical knowledge.. You will discover lots of new techy stuffs with every single error message..  Believe me, there is no better way to learn..

You dont need to go for special server machines at start.. just use normal current-days desktop machines (but dont use less than  7200 rpm hard drive ,, cuz memory speed is the main bottleneck for server's performance).

There is no question which software/OS should you use.. it is LINUX/Apache web server (i.e. total software cost $0).. My preference is Ubuntu/LAMP server. Though SUSE and FC are nice, some how i feel debian packages are stable .. easy to update and manage as compared with rpm. ..

Next, 'read terms of use' of your ISP provider.. make sure that it is OK to run web servers.

Regarding technical set up guide, there should be lots of online resources/books avialable... If you need any specific help.. feel free to contact me. And yeah, I presume you are comfortable with basic linux commands..

go ahead.. learn with fun.