Comfy Host has some cheap VPS plans that fit into our low-end-box criteria. For $6/month you can get the following OpenVZ virtual server:
128MB guaranteed memory
256MB burstable memory
10GB storage
75GB monthly transfer
Los Angeles, CA servers
No info on which Linux distribution is available. As it offers CPanel so possible only CentOS is provided.
Update 2008-03-30: Comfy Host has raised [...]