What is a good stereo system for classical music?

I listen to a lot of classical music myself (I have to add that I dislike that term but that's another discussion altogether.) I went to a fairly prestigious music school and played in every conceivable ensemble from playing the sackbut in early music groups to a full blown orchestra to choirs to electronic music programs so I know what music sounds like live and not Memorex. In a nutshell forget about "shelf systems" or any other "all in one" type of stereo. When it boils right down to it; what you are looking for is a real "high fidelity" system. And to get real high fidelity you don't need tube amplifiers, horn speakers or even a turntable. A good CD player, good receiver and good speakers are all you need. I'll take them one by one: I have several Sony CD players. I've heard a lot of different CD players and quite frankly there really isn't enough of a difference between them to justify the high cost of some of the more expensive ones. True, a really cheap one won't sound as good but I find the Sony's reliable and acceptable. They and Phillips invented the things and they seem to know what they are doing. In the early days I went through quite a few that sounded fine but had problems with mis-tracking. The slightest scratch or speck of dust sent it into a tizzy. Of course other brands are fine too and some will say that they sound better than a Sony, such as Marantz, Luxman, Krell, but I don't think the $500 price difference warrants the 5% improvement on the sound, even if there is one. Pioneer and Onkyo are also brands that will sound good, work well and won't break the bank.
Personally I prefer Marantz receivers. I like the sound and the user friendly controls. Onkyo is often sited as a good sounding and cost effective brand for receivers. Yamaha have been making quality receivers for years and Pioneer has once again started making quality receivers at a decent price. For a while the only good Pioneer receivers were their high priced, Elite line. Stay away from cheap brands like Sherwood, Teac and "off brands" - anything with the words; "Technical," "Digital" or "Pro" as part of their name. Sony receivers are not all good. Some seem to sound OK, others not and they seem to have reliability problems. There is also the consideration that you may not even want a receiver. You may want to get an "integrated amplifier" instead. It's the same thing as a receiver but without the radio tuner. Integrated amplifiers tend to be of a higher quality than the average receiver as they have a smaller, more discriminating, market.
Speakers are where you loose most of your fidelity when less than stellar products are obtained. I have two big factors when it comes to this: 1. Purchase speakers made by firms that specialize in speakers and, 2. Listen to them first. Despite the fact that they might make other products well firms like Sony, Yamaha and Bose don't make very good speakers. Some firms that specialize in speakers are Klipsch, Polk, B&W, and Energy. Actually there are a lot of companies that make only speakers but a lot of them are rather pricey, like OHM and Martin - Logan. As speakers add the most coloration they are a rather personal choice. They should be auditioned, preferably with the receiver that will be driving them and your own music. As far as the style of speakers, you have many options; a couple small speakers and a subwoofer or a pair of large, floor standing speakers and no subwoofer and options in between. As stated, the audition is the important thing. Find a shop that will let you listen to before purchasing. You might spend a bit more than mail order but you are more likely to get something that will have excellent sound, suit your lifestyle and last for many years.

How much are you willing to spend?

A truly high-end hi-fi system could easily set you back over £100,000. More modest systems can be put together for a thousand pounds or so. You budget will dictate how good a system you can actually buy.

Classical music sounds amazing when played though a valve amplifier and horn-loaded speakers - but they are not cheap components...

The best advice I can give you is to visit a high end audio shop. You need a cd player, receiver with a phono input, a turntable & speakers.

The more money you spend, the more good sound you heard