I currently have www.hollenback.net hosted on totalchoicehosting.com. That costs $4 a month (starterhosting package) for 10GB disk space, 300GB bandwidth. My site currently (April 2011) hits a max of about 4.5GB/mo. bandwidth and I'm using 1.3GB of my allowed disk space.
UPDATE As of February 2012 I am in the process of moving everything to a pgrmr.com VPS.
UPDATE As of March 2012 I've moved everything over to pgrmr.com and shut down my Total Choice account. I'm very happy I made the switch.
One annoyance with Total Choice is that I don't get a shell account. Doing all my administration through cpanel or ftp is somewhat tedious.
I have registered the neckbeard.us domain and am trying to decide what to do with it. I think I'd like to run a pastebin and ideally imagebin.
None of these things make me any money, so I want to find the cheapest possible way to host a few websites with the maximum amount of flexibility. I don't want to spend more than $10/mo. for hosting.
Both my domains are registered through Hover which I really like. Super minimal, no Godaddy upsell pressure (and nobody shooting elephants). Everyone should be using Hover for domain registration.
Several people recommended Dreamhost for hosting. For $11 a month @solarce says you get all the basics and a shell account. Dreamhost and Total Choice seem similar in their approach - crappy magazine ad style websites, very generic. Note I don't have any particular complaints about Total Choice. They're cheap and reliable.
csoft was another hosting provider recommendation. I like the look of their website but that's as far as I got.
I poked a bit at SDF. Looks like an old shell account company that has branched out. They do offer various hosting solutions. Like their very unix-oriented style. I plan on at least getting a shell account through them so I can run twitter on the command line. :)
Nearly Free Speech is also attractive. They seem to have a very minimal approach, basically along the lines of "here's your server space, that's all we give you". That works well for me as a very technical user.
All of Csoft, Nearly Free Speech, and SDF look like good options for very technical users who just want hosting space without any hand holding or annoying advertising.
In the last couple of years, virtual private servers have gotten very cheap. It looks like if I want to host more than one hobby domain, the price works out about the same as basic traditional web hosting.
Two VPS services that came up in the discussion were Linode and prmgr.com. Both have similar offerings (Xen-based), although Linode seems more polished (and more expensive). One comment on twitter was that there can be delays in getting prgmr.com virtual servers as it is a very small company that doesn't oversubscribe service.
If I use my LOPSA membership perk, I can get the $12/mo prgmr.com deal for $8/mo. That's 512MB ram, 12GB disk, 80GB network. I'd probably go with a CentOS or Debian disk image. The advantage there is I own the server and can host as many domains, etc. as I like. Given the current size of my site, I could probably support quite a few more random hobbyist sites.
The smallest Linode offering is 512MB ram, 16GB disk, 200GB network, for $19.95 a month. Unless I plan on running much more popular sites, I don't see much of an advantage to going that route.
Conclusion: I should probably get the prgmr.com setup and manage my own server and virtual hosts. I should also probably go with the Debian image since I'm mostly a CentOS guy these days and more debian experience would be good for me.
I'd be very interested to hear if anyone has any other thoughts on companies or services I might have missed.
Update: @lusis and @gallifreyan are both big fans of the VPS service from John Companies. The basic linux server package from that company is 256MB ram, 15GB disk, 200GB network, for $20/month. That's beyond my price range, but note that it does include some nice extra features, like automated daily backups.
Update 2: Last night my website on total choice went down. Turns out the upgraded php to 5.3, which broke my crappy old phpwiki website. Yet another excellent reason for me to run a VPS instead of relying on a hosting provider. Also, this.