Friday, December 26, 2014

Why not to bother with listening tests

Peter Aczel suggests not bothering with listening tests on well performing electronic equipment, and now even speakers (not to mention special cables and tweaks, which he says have no effect).  Measurements and basic design tell you pretty much all you need to know.  There is no scientific evidence (such as with double blind tests and proper statistics) that shows otherwise (wrt electronics anyway--Peter goes by his own measuring/listening experience wrt speakers).  Box speakers with lumpy response will sound like box speakers with lumpy response.  (Last I read, Peter had gone on/back to dipolar speakers…which have some key advantages such as not exciting room modes and spurious room response generally.)

Peter can be a bit over the top.  But against the matra, "listen for yourself," I think he is almost entirely correct.  There is most often little or even negative value in "listening for yourself."  And I can say different things than he does.

Listening to speakers remains a good thing to do, actually, with an open mind not prejudiced by previous listening experience, and with great disregard for the consistency of your listening results.  But not necessarily so much to each and every little modification to speakers, electronics, or anything else.   Unless there are likely to be large differences in measurable amplitude, such as 1dB or greater, listening tests are likely to be wrong.  And even if there are large differences, they may be misinterpreted. Listening tests to things that have no explanation acceptable to audio objectivists (like Peter Aczel) shouldn't even be bothered with for most people--except people who thing strongly otherwise, and they should concentrate on getting definitive double blind test results to prove their position rather than engaging in endless polemics.

The principle reason why you should not even bother with listening tests (except, I'd concede, on speakers and phonograph systems--if you bother with phonograph systems) is that the outcomes of listening tests are untrustworthy.  There is huge variation in auditory experience ever under identical and controlled circumstances.  This variation likely explains all the differences allegedly heard in comparisons which have no explanation Peter Aczel and audio objectivists at Hydrogen Audio would accept (and they're pretty tough).  And probably a lot more, most untrained listeners (and I'd consider myself barely trained if at all) don't necessarily even reliably hear differences at the scientific thresholds of human ability, such as level or frequency response differences of 0.1dB.

As a first order of approximation, if the very serious listeners at Hydrogen Audio don't believe there is a difference they can hear, you and I, less serious, should probably not even waste our time.  We'd need to spend months training our ears to reliably hear and identify small differences as serious DBT aficionados or academic audio engineers (who are more interested in truth than having a competitive angle).

To rise above the untrustworthiness, if you believe otherwise, prove it to yourself and others with a double blind test.  I tested myself in 1983, and since then I have become far less serious about audio tweakery (although, as you can see reading these pages, I have hardly given up on tweako-explorations of the kinds that make sense to me, generally involving physically identifiable and in-principle measurable factors, just beyond the normally accepted limits).

I find fancy cables and capacitors and digital converters fun to think about.  I doubt I could prove scientifically that I hear differences, even after a long struggle, though perhaps could with actual lifetime training.

But if I find them fun to think about, why not have them in play anyway?  Or why should I deprive myself the fun of buying and deploying fancy gizmos regardless of my doubt over their efficacy?  Why should people with foggy thinking on this subject have all the fun?  I temper my lust for gleaming machinery with the serious knowledge that it's all really bs anyway, meaning basically I can spend only what I want and on the things I want--not the things other golden ears have come to believe in and insist I must have.  My more critical thinking only advances my freedom to do anything I want.

Now given very high unreliability, frequently hearing differences that can't actually be heard, the psychological effect of performing listening tests routinely is to develop audio superstitions.  Audio superstitions are acceptable to a certain degree, just like all psychological disorders, we all have a little of them, and perhaps life would be boring if we didn't.  But for some people, superstitions become very costly in a variety of ways, not necessarily even including money.  Superstitions can become costly in behaviors, including not-listening-to-music (the well known audiophilia nervosa), but into many other obsessive and anti-social behaviors.  People may begin summarily dismissing their closest friends or associates because of petty disagreements based ultimately on dueling superstitions.  Obsession may lead people to abandon their friends and gainful employment, in order, usually nowadays, to conduct endless internet flame warfare.  And there can be various kinds of paranoia, grandiose delusions, and insomnia.

It's not as though, necessarily, doing listening tests regarding audio minutae will lead immediately to psychosis.  In fact, you could do all the listening tests you like, either if you proceed scientifically with dbt, or, universally, simply fail to take yourself and your perceptions at any given time very seriously.

Put then if you don't take your momentary perceptions very seriously, why would you even bother with listening tests?  Listening tests, especially serious listening tests, take serious time and effort and generally speaking are a big boring head scratcher.

A standard methodology for non-listening-test audiophiles like me is basically buy stuff and hook it up.  If it sounds bad, replace it with something else.  If something new reads interesting to my collection of audio knowledge, superstitions and lusts, repeat.

Lack of believing in listening tests doesn't mean I can't buy and play with cool stuff, just based on what I know about them through other means than personal listening tests.  It doesn't mean I need only restrict Myself to that which I know as provably audible.  I need only be conservative and reasonable in my expectations and findings to avoid madness, and stay merely eccentric.