Almost any PC can be left on 24/7 for years on end as long as they have clean power and enough cooling - in fact, it's probably easier on them because startup puts alot of stress on the hardware. My last PC was on continuously for 8 years except for power outages and it's still working.
I'm still not comfortable with solid state storage - cost and speed are still not quite there. Give it another year or two for the price comes down. You should also be aware that flash has a specific number of times it can be written to - the manufacturers give you the average MTBF for the entire drive, but some areas of the drive get written to almost continuously.
Instant obsolescence is unavoidable - whatever you buy today is obsolete tomorrow. The best you can do is get the hottest machine you can afford to make it worth something as long as possible.
Base specs - 3 GHz or better, Intel or AMD is no longer significant, 2 GB RAM mimimum, 4 GB or more preferred, decent graphics card with at least 256 MB RAM - if possible, get one that does not need a cooling fan (I've seen too many fail because the fan failed). Big case fans, front and back, good processor fan, power supply more than adequate, lots of drive bays, SATA controller and drives, IDE controller for older drives is sometimes handy. It's nice to have lots of USB ports, especially on the front of the case. Firewire ports only if you think you need them - the only time I've used one was to control a CD Duplicator that cranked out 100 copies at a time.
Many people are mirroring their hard drives or using RAID arrays so a drive failure doesn't mean everything's gone - you might want to look up RAID on Wikipedia. This is not a backup solution - your data is still on hard drives that can die. If you can afford it, tape may be the way to go (because of capacity). Blu-Ray or similar for backups may be better, DVD if that's all you have. Most home users don't back up at all, but most home users don't have terabytes of data on their hard drives like we do.
Hard drives are cheap - around $80 to $90 for 500 GB. I'd get several smaller drives instead of one very large one - don't put all your eggs in one basket.
Is your PC less than 3 years old? Maybe you don't need a new PC, just a revamp of the mechanical stuff like hard drives and fans, and add more RAM. RAM and a good video card can speed things up more than you might think. If you do opt for a new PC, think about buying the parts and putting it together yourself - you can get a lot more PC for the same money and it's not that difficult.