Are Objective Headphone Measurements Relevant to Audiophile's Subjective Experience?
Ed. Note: I talk with Steve Guttenberg on the phone a few times a month, and I very much enjoy our chats. When Steve suggested we have a conversation about measurements and their relevance to audiophiles and write it up as an article, I was all-in immediately.
I hope you enjoy our chat as much as I did.
Steve Guttenberg: Let's get down to the nitty-gritty, do you really think measurements can be used to predict sound quality preferences? And are measurements in any way more reliable predictors than subjective listening tests? We all know that tube electronics don't measure as well as solid-state gear, but that doesn't seem to bother a lot of audiophiles (and musicians) who crave tube sound. So what do measurements bring to the party that subjective evaluations miss?
Tyll Hertsens: There's a tension between subjective and objective evaluation. It seems to me they both attempt to evaluate headphones, but operate in two different domains. Essentially, I think measurements can tell you how close something is to neutral, and to what extent it deviates from neutral. Subjective testing allows you to know how pleasing something is, and in what ways. I don’t think our brain and perceptual system is like an Audio Precision tester at all; I don’t think we’re good at objectively analyzing what we hear. Likewise, my AP tester isn’t conscious, and it doesn’t experience what it’s hearing. So I don’t think it can tell us much about the pleasurable experiences of listening to a pair of headphones.
The trick here is knowing that objective and subjective evaluation happen in two different domains, and one has to have an open mind to bring in information from both without feeling conflicted. If a person falls too strongly into one camp or the other, their viewpoint bias may override the ability to see valid points of the other viewpoint. The question is how to keep both valid viewpoints at the same time.
SG: I see your point, and I want to say upfront that measurements are valuable tools for engineers, I'm just not convinced laymen can use them to predict whether they would prefer headphone A over headphone B based on looking a wiggly lines on a graph, as opposed to reading a reviewer's impressions of the two headphones. I recall that when we were both on a panel at the 2011 Rocky Mountain Audio Fest you said the Sennheiser HD 800 measured better than any headphone you tested to date, but it wasn't your favorite headphone to listen to. On another occasion you mentioned the Bowers & Wilkins P5 didn't measure all that well, but you enjoyed its sound.
TH: Well … it’s complicated. There are a whole variety of measurements that I take, and when I say a headphone measures well it’s really a summary of a variety of discrete measurements. The HD 800, for example, measures excellently in terms of the speed of the initial transient of a 300 Hz square wave, and has little ringing after the initial transient. This makes for a headphone with very good imaging as transients are clearly identifiable, providing accurate sound arrival time information for the brain to use to accurately localize sound and build a sonic image. But the initial transient also overshoots a little too far, and makes the headphones sound a little piercing, which is a characteristic I don’t like.
The subjective experience is complex too. The B&W P5 doesn’t measure well---having a somewhat excessive bass, and rolled-off highs---but I remain very attracted to them by their look and feel, for movie listening and phone calls they’re very easy on the ears and fun to hear, and their isolation is astonishing. There are a lot of things I really like about them, and I care very little about their inaccuracies faced with my satisfaction in other ways. Basically, I’m not listening for their faults in normal every-day use, so don’t hear them.
I feel things are very complicated for both subjective and objective evaluation. “Measures well” and “sounds good” are both gross oversimplifications.
SG: So there's subjective analysis of the objective measurements. That's cool, and then there's the stuff that can't be measured and quantified -- like comfort and the feel of the thing in your hands – which heavily influence preferences. The intimacy of headphones puts them in another category compared with other audio components. Measurements can't measure that, and in some ways it's more important for a headphone to feel good than sound good.
TH: I would certainly say the look and feel of headphones has far more relevance to user satisfaction then the look and feel of speakers. I mean, you’re not wearing your speakers on your head while you commute on the subway.
The value system for personal audio is clearly different than other types of audio. Esthetics, comfort, and convenience play a much more intimate role in your experience with them than with home audio. And satisfaction in those areas is going to bleed over into your aural satisfaction as confirmation bias causes you to hear them as better than they otherwise might be.
So, yes, feeling good about the look and feel of a personal audio product is probably more important than it is with a home product.
SG: It's interesting, John Grado told me last year that he never measures prototypes when designing new headphones. He's been at it a long time, and his ears tell him what he needs to know. Grado's working method is at one extreme and I guess Sennheiser is at the other. Sennheiser takes a more hard engineering, more measurement oriented approach to advancing the state-of-the-art of headphone design.
John Grado is a "seat of the pants" designer, so why would his sound be far more consistent than Sennheiser's? I can't identify a Sennheiser sound per se, it's all over the place. If its engineers were really using a scientific approach, wouldn't all Sennheisers of a certain type, say open-back for example, have an identifiable Sennheiser sound? They don't, so even for a company like Sennheiser other factors come into play.
TH: Well … Grado’s sound doesn’t change much because their headphones are all nearly identical in design. Sennheiser makes maybe a dozen full-size open cans, but the designs change significantly and tend to run in families. For example: Sennheiser makes the HD 518, HD 558, and HD 598. This is a family of headphones that share a similar look. Sennheiser probably does some things like use common parts in these headphones to gain economies of scale. So they really aren’t completely free to optimize the top product in the line as it may have some parts in common with the lowest priced product in the line.
You’ve also got a big company with corporate goals and timelines. There comes a point where management will “shoot the engineer” and put the product into production. Their job is WAY more complex than Grado’s, so you get more variance in the end sound.
SG: I still think it's odd that Sennheiser doesn't have a Sennheiser sound. The reasons behind consumer preferences are complex, and all I'm saying is the better measuring headphone isn't guaranteed success. Grado owners don't live in a vacuum, and many of them are being exposed to other brands' sounds. Some stick with Grado, some move on. Some Sennheiser owners cross over to Grado or Beyerdynamic or Hifiman. People eventually wind up with the sound they like, how well the 'phones measure has little to do with it.