Big Sound 2015 Wrap: What I Learned
When I came to the conclusion that the considerable amount of gear assembled to test the latest high-end headphones must be shared and began inviting people to join me, I really had no idea of how to conduct the experience of participants. As I thought about it, I came to the conclusion that providing some sort of mental reset button to tamp down expectation bias and allow participants to evaluate the gear with an open mind would be cool. I wondered how to do that.
As I said on the home page teaser for this post, "WARNING: This is not headphone science. This is just a very, very complicated headphone anecdote." I've got nothing to prove here, not because I wouldn't like to prove a few things, but because "proof" is a heavy burdenone that can only be addressed by real science. I'm simply not qualified or equipped to offer proof positive. What I can do is hold a somewhat structured listening event and report on my observations. These are my subjective observations and amount only to my opinion and what I personally learned and took away from the experience. I hope the objectivists among you will read this article in that light.
My ulertior motive for including some blind testing of participants was to shake people up a bit and show that the objective magnitude of difference between a lot of this gear was very small. There are a couple of reasons for doing this: One was to give people confidence as they evaluated the wide array of headphones that the amps and DACs up-stream (within reason) would have litte effect on the natural sound quality of the various headphones. The transducers themselves are, by far, the biggest variable, and the fact that they were driven by different DACs and amps would not substantially alter the inherent character of the headphones. The other reason was to disabuse folks of the idea that amps and DACS made a huge difference in sound qualityand much more so, the differences due to cable and power conditioning/regeneration. (After participating in the initial blind test I would often offer folks the opportunity to blind test the differences between cables. They would often laugh and say, "No way! If I can't tell the differences in amps with the HE-1000, I'm certainly not going to be able to test the differences between cables.") It is very important for entusiasts not to over-enthusiastically claim huge differences in these areasit makes objectivists crazy...and rightly so.
For example, one excited participant came in obviously having read a lot about the gear and had a number of pre-conceived notions about what he was likely to hear and the supposedly large differences between various products. After an hour of blind testing he openly commented that his preconceived ideas needed to be thrown outthe differences in the gear was much smaller than he had previously believed. Eggselent!!!
The other thing I learned about blind testing was that it essentially made you blind to the small differences between gear. Objectivists will claim blind testing removes biases, but I think you just trade one set of biases for another. For example, most folks experience significant anxiety when faced with the small differences during blind testsa lot of self-doubt can apear in the listener's mind, which leads to confusion and indecision. Also, when a listener begins to believe they've identified one of the selections, they then begin to superimpose a bias on that selection. The only way to combat these issues, it seems to me, is long experience with the technique of blind testingand even then it's very hard. If you ever read some pop-press article about how they blind tested a bunch of people and found there was no difference between an iPod and a $1000 digital audio player, don't believe it. Joe Blow off the street is going to be completely lost with this kind of evaluation. So, the Big Sound blind tests demonstrated that the differences between gear is small.
Despite the difficult blind listening experience, the Big Sound participants went on to sighted listening tests, and over the course of the remainder of the day would express fairly clearly their preferences and reasons for preferring one amp over the other. An objectivist might say that a biased opinion is easy to come by, but I have to note that even though the sample size is too small to be conclusive, there was a reasonably strong similarity of impressions among the participants. Sure, the bias against the Burson might be chalked up to the loose volume knob, but there was fairly broad agreement that the TTVJ Teton was a great ampdespite its homely looks next to the Woo mono-blocks. Many participants observed the Eddie Current Black Widow sounded somewhat "tube-like" and was quite pleasant to hear, even though it was placed off to one side and wasn't particularly cool or impressive looking.
There were some notable disagreements, but often they could be chalked up to personal preference. For example, Bob Katz described the SimAudio Moon 430HA as "overly soft and fake-tubey", but I hear it as neutral and even-handed; he found the HeadAmp GS-X Mk2 "excellent transient clarity without any high end exaggeration", where I found it a bit tipped-up and a tad brighter than I prefer. Though we disagree from our respective perspectives, we clearly agreed on which amp was faster sounding.
My point here is that blind testing fundamentally showed that people found it extremely difficult to tell the differences between gear; while sighted evaluation allowed people to develop preferencesalbeit in accordance to their personal taste. Why is that?
I think humans want to develop relationships with things. "Hey, that's a chair. I can sit in it. It's light enough for me to move. It doesn't look comfortable." and on, and on. Humans make a coat hook in their head for an object, and then start hanging opinions about it on the hook. When we do blind tests we are without the hook. We have a hard time relating withevaluatingthe object. With sighted listening we get the hook back and can begin to develop a relationship with the gear making the evaluation more natural.
Small vs. Subtle
Still, I remain convinced that the differences between amps, and more-so DACs, is objectively small. If so, how are people able to relatively quickly and easily develop preferences? Here's my thoughts:
I remember as a teen walking through the Museum of Modern Art with my father and rounding a corner to see a large Picasso. It took my breath away on first sighting...stunningly beautiful. I'll observe that not everyone would have this responsesome will have taste that runs counter to that particular esthetic, and some simply won't have developed sensitivities along those lines in their visual cognition. But for me it was striking.
The thing about great art is that the artist has extremely well developed technical skills and is able to create nuanced and subtle detail. But the thousand subtle details that go into the painting come together as the coherent whole that speaks to our being. The impression we receive occurs without having to observe each detail, but rather comes to us in an instantaneous grok. We relate directly with the character of the complete painting. Yes, we can dig deeper into the painting and come to appreciate the details, but the more important relationship is to the artwork as a whole.
Similarly, I think when we listen for pleasure to a piece of gear we are neither attracted or off-put by the individual details, but rather, experience the sound as a whole. But the sound, as a whole, is made up of all it's constituent small, but subtle, details. My point here is that while individual subtle details may be difficult to consciously discern, the mind will subconsciously sum the myriad subtle but important details and sense the overall character of the product. It is that experience which allows us to come to personally valid impressions.
I need to add that there is also a time dimension to the relationship with a piece of gear. Over time our mind accommodates to the sound of something. The brain modifies its perception to normalize the sound it hears. Because no headphone is perfectheck, it's actually a completely artificial way to listenour mind must always make an accommodation. Sometimes the accommodation is easy, and we find ourselves with a piece of gear we like perpetually; and sometimes the accommodation is not so easy, and we find ourselves with an acoustic burr under the saddle...and gear is soon to be sold.
My point in all this is: Small details add to the whole in ways that make them more important than their objective magnitude.
(As a side note, I highly recommend reading this post by Mike Moffat (Baldr), of Schiit Audio, regarding his experience with blind testing.)
A Special Note About Cables
While I think that cables can make a small difference in sound, and while I think small, subtle differences can provide an improved magnitude of enjoyment larger than the objective differences might indicate, I do think that on the whole far too much emphasis is placed on cables by audio enthusiasts. The reason for this is understandable.
Enthusiasts want to tweak their gear; they want to participate in the creation of the equipment that delivers them so much satisfaction. Nothing wrong with that. But it amounts to an insult to the engineers who design audio electronics when enthusiasts place so much attention on cables without regard for the complexities inside the box. Most enthusiasts are not aren't qualified to open the lid and start modifying the gear, so the only thing left for them to do is swap cables and such.
The big problem for me and my objectivist friends is when this cable swapping activity (and all manner of other tweaks) becomes excessively emphasized as an important method for improving the sound of ones gear. Worse yet, this mindset is often encouraged by cable manufacturers claiming dramatic improvement. I very much wish cable makers and enthusiasts alike would dial back the rhetoric on the benefits of expensive cables. Yes, they can make a small but worthy difference, but choosing electronics and transducers that satisfy is the more important driver of sound quality by far. Once the gear is sorted, time spent with folks like The Cable Companywho have a program to ship you loaner cables for auditionis a reasonable last step in your journey toward building a satisfying listening system.
Personally, I would look at the low cost lines from AudioQuest, Blue Jeans Cable, or Kimber Kable for your initial cabling needs, and then wait until your system is complete to audition more expensive conductors.
Big Sound 2015 taught me that blind tests are really only useful for professionals evaluating small details, and really shouldn't be used by casual listeners or enthusiast n00bs. It's better for most folks to keep an open mind about gear; reduce your desire to produce pre-conceived notions from your forum reading; and resist the temptation to jump to any thoughts that there are huge differences between amps, DACs, cables, break-in, and the like. Please be careful about exaggerating your experience in listening comparisons; they're usually not that big; they'll often be specific to your tastes and sensibilities; and they may skew the experience of others and lead them astray.
I also learned that while we may be hearing the same piece of gear, each individual's impression of it may be different based on both physiological variance and in matters of taste. Just because someone's carefully formed opinion is different than yours doesn't make one of you right and the other wrongat least, insofar as relatively good kit goes. However, there are junk products out there that should be known as such to all.
Next, and last, Big Sound 2015 post: My take on the gear itself.