Stay clear of the cheapest hosts. They're cheap for a reason.
Without getting into it too much (

) be clear on what "reliable" means to you! You may think it's what it means to other people but...
How much traffic (bandwidth) do you think you're going to need? If you're offering galleries or downloads you'll need quite a bit. It all depends on your needs. Are you going to need a certificate to offer secure credit card transactions etc? Are you going to need hand-holding from tech support or do you know what you're doing? So many variables, so many unreliable (

) hosts...
I do hosting (as does just about everyone!) but It's not a money maker for me. I usually help charities or people starting out. I don't have a flashy site etc because I don't need one. Finding a good host is a real chore! There is so much biased help out there which tries to steer you to an affiliate program with top rated host this, or 99% uptime that -pathetic. Think of hosting a one big pyramid scheme with hosts passing themselves off as the guys at the top of the pyramid (the guys in the data center) when they're most likely to be many, many levels down the food chain renting space on someone else's machine.
The trick is to find someone who's close to the top AND truly has a reliable technical infrastructure. Now, realistically, that quality of hosting is inexpensive -but it's not cheap
I'm delighted with my host and could set you up as a client of mine, depending on your needs. Like I said, I haven't bothered with the smoke and mirrors of "Live Tech Support" etc. You'll just send me email if you're having trouble installing something. I got into hosting because sometimes, if you want a job done right you have to just do it your damn self
I guess I should add that things like cpanel, Fantastico, spam assassin, subdomains etc are standard. No chat, IRC, warez etc. Adult ok (anything legal in the USA). Audio/video streaming ok. No shoutcast. I'm not going to sell you XGB of bandwidth hoping you won't use it all and then sell/say the same thing to 20 other people... My host does not "oversell" server resources or allow anyone else (like me) to screw people like that. The games people play. Like I said, I don't do this for money, it's not a business for me.
Good luck.