The downsides: lacking in deep bass impact and high frequency transparency. Classical music is generally fine but rock music lacks passion. The image is largely inside my head, though this is a an problem with nearly all headphones (I haven't tried binaural recordings). By comparison, the troublesome Acoustat image is full sized and glorious, the whole soundstage is laid out in front of me from far right to far left.
At one time, I had Infinity ES-1 electrostatic headphones which I recall (though this was long ago, and I could be wrong, especially because my standards were much lower, I was using large Advent speakers) sounded both more transparent in the highs and less in-your-head. They blew away the sound quality I had from any speakers at the time. However, the Infinity's (actually made by Mechano Electric in Japan) were an on-the-ear design which was far less comfortable (hard plastic rested right on my ear flesh) and they often made a crackling sound. I was very sad (and still am) to have sold them, but actually never used them much anyway when I had them.
While I still owned and occasionally used the ES-1's, I worked at Audio Dimensions (parent of Audio Directions) for awhile. The store sold Stax electrostatic phones, and I had an opportunity to try all of the ones we sold in 1978. The smaller ones with transformers sucked, I thought, sounding no where near as transparent as my Infinity's. Some even had hard to ignore resonant colorations. Only the large Lambda phones with a transistor amplifier was competitive, and I even thought it sounded a bit wooly by comparison with the ES-1's (though they had deeper bass, and were far more comfortable, and no crackling, and I certainly wouldn't have turned them down as a gift). One of the notable sales technicians at the store had a personal pair of electrostatic phones whose very obscure name I can't remember. I borrowed them for awhile, and thought singularly they were even more transparent than my Infinity's. I rediscovered the name on The Audio Circuit awhile back, and looked up the company history. They didn't make headphones for very long unfortunately. After doing that research, I promptly forgot the name again.
It is commonly believed (and I believe it to be true) that the Koss system is limited mainly by the very small and lightweight E90 amplifier. Power by a wall wart, weighing about a pound, it is the antithesis of the current generation of top Stax electrostatic headphone amplifiers which can be as heavy as a home theater receiver and consume even more continuous power. A good number of people have gone out of their way to adapt ESP950 phones for use with Stax amplifiers and are pleased with the result. According to many reviewers, Koss's ESP950's same rank as Stax's best headphones when used with Stax amplifiers. (Some like the Koss better than any Stax even with the E90.) And the Stax large size Omega phones by themselves (without the amplifiers) cost 5 times as much as the complete Koss system. The Stax amplifiers cost nearly that much also. Though many find the Koss lacking in high frequency transparency compared with the Stax, the tables may be turned when the Stax amplifiers are used for the Koss, with the Koss having greater high frequency transparency still.
I still lust after the top Stax phones, and though I haven't heard any of the recent top models, I wouldn't be surprised if a Stax Omega 007 with top Stax amplifier would sound better than my Koss. I'd love to have the Stax Omega 007's (or 009's) with their top of the line balanced class A transistor headphone amplifier. The minimum price for that package with 007's is a cool $4500. (The 009's cost that much just for the phones.) It's hard to plunk down that kind of change for something I don't even know I'm going to use very much. I got the ESP950 package for $632 with free shipping. I could get a smaller cheaper Stax setup (say, 303 phones with 323 amplifier) for as low as $1280. But the smaller Stax phones rest on (not around) the ear, and people complain about an etched high frequency sound. I'm worried that while the smaller Stax phones might have greater transparency than my Koss, they might have some unpleasant aspects as well. I wouldn't mind trying them, but I don't want to waste the time of my local by-appointment-only dealer for something I'm not that likely to buy anyway, because I don't know if my limited usage would justify paying so much.
Another largely unappreciated factor may be the pedestrian cable the Koss uses to connect the phones to the energizer, but quite likely the Stax could benefit from better cables as well. A proper cable would use Teflon insulation with small gauge silver plated solid core or litz wire. It would not be nice and flexible, but it would maximize audio performance. I recall my Infinity phones had a fairly stiff cloth wrapped cable. Under the cloth, there was likely polypropylene coated wire, far better than what appears to be the soft vinyl used by Koss and maybe even most Stax. That could be a large part of why the ES-1's sounded more transparent than even Stax phones at the time. The cables connecting energizer to phones could have a substantial fraction of the overall capacitance in the high voltage circuit, and the poor dielectric absorption (if not outright leakage) of vinyl coated wire could easily do a lot of high frequency smearing, far more so than in the far lower impedances of interconnect circuits.
Here's a review comparing several pre-Omega top-of-the-line Stax phones with the ESP950's, using Stax amplifiers in all cases. He doesn't even bother to review the Koss phones with the E90 amplifier.
Here's a general discussion of the Koss phones, some pictures of the electrostatic elements, and discussion toward making an adapter to run the Koss phones from a Stax amplifier.
Even the very fine Stax amplifiers are not perfect, and a properly engineered electrostatic phone amplifier might do even better still. I am especially convinced that the very popular Stax tube amplifier is far from an optimal tube design. It is commonly believed to soften the sound and reduce transparency compared with the top Stax transistor amplifier. A friend of mine might step up to the challenge. His last project was a direct drive tube amplifier for Quad ESL-63 type electrostatic speakers.
Here is a very interesting review and measurement of dozens of headphones, in which the Koss ESP950 is among the top few in flat response, if not the outright flattest. The author doesn't select the ESP950 as his new reference phone because he needs a sealed back unit for his job. While the Koss has more extended HF response than all but a few, (a tiny bar at the very top which most don't even have) it does show a couple dB of average HF rolloff above 4k which some others don't show (while they may show more HF roughness instead of an average rolloff). That makes me think a couple dB of boost above 2-4k might help. The deepest bass also shows a tiny drop, but actually most others (even his reference) shows a larger decline, but preceded by more upper bass boost. Roller coaster bass boost and hole in the mid treble are par for many other headphones.
I've found one company, APureSound, which will make an ESP950 to Stax Amplifier adapter for $140 (that seems too high), or recable the ESP950 with a Stax connector for $150 (sounds better). The recable includes cleaning out the dust inside (great for older phones) and removing the foam pad on the ear side (which they say improves transparency). I believe "recable" also includes different better cable of their choosing. They also offer custom modifications, that might permit me to choose an even better cable, say with Teflon dielectric.
Here is a truly amazing thread (but unfortunately filled with OT rants and flames) on the Koss ESP950, with lots of people using Stax amps, other amps like KGSS, homebuilt amps, experiencing an infamous "squeal" problem (may be dust, hair, or terminal arcing, Koss changed terminal design in 2008 apparently in response), Stax gets squeal also but less frequently. In this thread you will see pictures of the inside of the E90 energizer and more.
Here is a great site with homebuilt E.S. headphone amps, and the following response curve (don't link due to web design).