Why InnerFidelity Measures Headphones

InnerFidelity reader scharfsj wrote in my recent "A New Push for Headphone Measurements" article:

All well and good, but, I'm really not particularly interested in measurements, and really don't know why there is such a preoccupation with this opposed to creating more interesting content. What I am interested is what they sound like.

There are a lot of reasons really, maybe most fundamental is the fact that it's simply the way I look at things. I do measurements because I like to; it satisfies my curiosity. I used to work on scanning electron microscopes. I like meters and probes. When I started working with headphones the desire to measure them was simply an in-built primal urge to know something as completely as I can. Okay, that's the personal side, but that doesn't matter one whit when it come to what I should do professionally for InnerFidelity readers.

In my "Mission Trumps Bias" article, I state my mission as:

"To help people better access the art of music by finding and describing the benefits of quality personal audio gear."

That word "finding" in that statement above is essentially about measuring. Even if I didn't publish the measurements, having them makes finding good headphones a much more efficient process. But the original comment was more directed at the ratio of measurement content vs. review content.

Well, publishing measurements is a very precise description, and therefore a review of sorts. At the end of the comment was the statement "What I am interested is what they sound like." I would contend that the measurements may be the most compact and efficient form for doing just that. No, it doesn't give you the fine details and the gestalt of the listening experience, but measurements do tell you a lot of information about the sound quality of the headphones...if you know how to interpret them. And that's exactly why I'm writing the "Headphone 101" and "Interpreting Headphone Measurements" articles. They help readers learn enough about headphones and measurements to use the measurement datasheets to sort through product quickly for things they might like. I'll add it only takes maybe two hours to measure a headphone (about an hour to measure it, and then various bits of time acquiring, receiving, formatting, and publishing), but takes about three days to review a headphone. So , I can measure a lot more headphones than I can review.

I know that's not quite a satisfactory answer. What about folks who don't want to, or know how to, read the measurements? Or you may simply want to hear the opinion of myself or the other InnerFidelity writers. Yes, I do have to write reviews. Let me make the comment that nine out of ten headphones I measure don't, in my opinion, warrant a review. It is sometimes a pretty significant struggle to find enough worthy headphones to review. I feel that writing only about the really good headphones I find provides a good cut-off for me, and it provides a nice high bar that manufacturers will recognize (as it's significantly objective in nature) and may try to attain. I will add though that I measure every headphone that comes through the lab and if you read the updates I do usually give a one or two line impression of the headphone along with the measurements. In my head, those one liners are a big tool. Gives me a quiet place to say negative things without stirring the pot too much.

I think there's also a much larger but low-key way in which Innerfidelity measurements help people find good gear. In just about any forum conversation between a dozen members or so, one will be familiar with measurements. If one person says, "Hey that new Noontec ZoroII sounds awesome!" The guy who's familiar with measurements will look it up in the InnerFidelity measurements to see how well it measures. (Very well, in this case; ZoroII measurements will go up next Update.) Then he'll go back to the conversation, maybe post a link to the .pdf, and exclaim, "Sure does look like a great headphone in terms of measurements." In that exchange is born a strong opinion that can carry within the headphone enthusiast community. Sounds great + measures great = is actually a great headphone. The reverse is true as well, where a poor headphone is heard as having oddities and measurements show significant problems that opinion can firm up and propagate quickly. Without the measurements the "community take" on a particular headphone will take longer to develop and more people will make poor purchases in the interim. I think measurements help enthusiasts form their opinions more quickly and accurately, and make better purchasing decisions.

That last paragraph addresses measurements producing an intelligent demand for headphones where the consumers spend more wisely, but hot on its heals follows the next and maybe most important influence the measurements have: Manufacturers have to sell to smart consumers. Measurements hold manufacturer's feet to the fire. When there were no measurements out there it was a "my word vs. your word" world; manufacturers would print "Digital Ready" on the side of their headphone as proof they were good. In a world with independent measurements we can verify that the Audio Technica ATH-MSR7 is as clean and resolving as it's "Hi-Res Audio" sticker promises...which it is.

Additionally, and also very importantly, this intelligent demand is not based on something willy-nilly, it's based on objective measurements. Headphone measurements are a clear stake in the ground for all to see. Sure, there will be people who like more or less bass, but the bounds of "normal" will be more closely defined. Manufacturers who are aware, motivated, and smart, will start producing headphones that more closely align with "neutral"—not that we know what that is exactly, but we know a lot more now than we did 15 years ago.

So, yes, the content here at InnerFidelity at the moment is weighted a bit heavily to the objective side, and might not be quite your cup of tea, but I really do feel that InnerFidelity's headphone measurement mission is improving the world of headphones enthusiasts get to play in, and does provide (at least for those that have taken the time to learn how to read the measurements) a good way to get a first glimps how many, many headphones sound.

COMMENTS
Seth195208's picture

Well put!

tdherbert's picture

I really value the measurements, because it makes me think about what qualities in a headphone I appreciate. It's not that the numbers will tell everyone what they should buy- but it gives context for the subjective "what it sounds like". Anyone could decide that they like a certain type of headphone that doesn't match the "ideal"- whatever that is- but you would have a more educated view of what kind of headphones YOU like.

zobel's picture

Good on ya Tyll! I think at this point along the evolution of making relevant and useful measurements, you have done a good job of following the scientific method. You have a standardized, repeatable procedure that you follow, keeping all the controls constant; such as headphone mounting procedure, standardized and identical level matching in inputting the test signals, and a controlled environment that remains a constant for each test.
Therefore, your data is good. The way the data is displayed will be made much more relevant and readable when eventually you can come up with a much more representative HRTF to make the corrected raw frequency response curves appear more like the straight lines we hear on he best cans.

That is so much easier to achieve when graphing loudspeakers SPL/freq. charts, since measuring loudspeakers in anechoic conditions gives a representative curve to what we hear. You have an enormous number of variables measuring headphones in comparison, but with Sean Olive's work and your adaptation of it on your test gear you've come up with a pretty good general target curve to achieve when looking at the measured SPL at the ear drum in your dummy head. It looks like the data just raw is very representative below 1000 Hz. Now I suppose the next step in producing a graph that is representative to what we actually hear in our brain, is to make computer analysis of as many samples of proper sounding cans raw data, and see what the best average correction curve needs to be applied to produce the correct subjective response curve.

Wouldn't it be great to have model, ideal SPL and square waves, and isolation curves to compare to? I think JA does a good job testing gear for Stereophile, and the info he comes up with is solid, readable, and accurate. It also is much, much easier to come by than what you can expect when measuring headphones.

More power to you Tyll, I say, Hear! Hear!

mikeaj's picture

Measuring headphones isn't easy, even for an experienced expert, and it takes some time, but I imagine it's a lot less than just the listening required for more than a cursory impression, never mind a full review. It's an efficient use of time to provide information about more headphones—furthermore, information that largely cannot be gotten elsewhere. Most people don't have the gear, and even results from similar (but not virtually identical) setups may be tricky to compare.

I really appreciate the short blurbs on headphones for the monthly updates, for what it's worth, too.

Stefraki's picture

I find the measurements you provide extremely useful, and I find your subjective reflections really useful as well but for different reasons.

The similarity is a reference point though. Having read your reviews of many headphones I know well or have owned, I have a very clear idea of what we have in common when it comes to tastes, and where we differ (you prefer a slightly darker sound than me, but I know that, so I have a reference point).

It's the same with measurements. I can look at the measurements of all the headphones I really love and see some commonalities between them all - for instance I have identified that I can be fairly open minded about frequency response, but if distortion is too high then I won't like them, and if it's very low they will sound good to me, even if I don't want to live with them. I know how high the bass can be shelved before I find it unrealistic and distraction, how high a treble peak I find acceptable.

You can't look at a set of measurements and know exactly how headphones will sound, but by having them available for such a broad range of headphones, you can definitely set some reference points for yourself.

jeckyll's picture

Stefraki's reply very much mirrors my own feelings. Nothing will take the place of finding an unbiased reviewer whose tastes you understand to help you pick a new set of headphones (aside from spending a few hours really listening to them of course, which isn't always possible).
Having the measurements to back this up is excellent an an approach that is fairly novel. Can't really get that anywhere else.

Keep up the good work Tyll!

Three Toes of Fury's picture

Thanks Tyll, for yet another great look deeper into what you do and why you do what you do.

To me it comes down to the fact that headphones are a tricky beast..a wonderful tricky beast. One that cant be easily qualified by a single approach. That's why i love how you tackle the challenge: 1) Science: using consistent testing methods and data gathering techniques you build a great way to start to quantify headphone output. 2) Feel: Because of huge amount of unique variables (head size, ear size, personal response to sound waves, personal "taste" for sound types, etc), there is the personal experience with headphones. Its the balance of these two, which you provide in your regular reviews, which provides great comprehensive information to the readers.

Additionally you are building a great database of information. This information may someday help identify unique characteristics, trends, test methods, etc to improve the overall process.

Keep up the great work!

Peace .n. Living in Stereo

3ToF.

tony's picture

Measurements end up being of great value!
A couple of Decades ago Mecerdes Cars & F150 Pick-up trucks were the safest cars a person could buy.
Car manufacurers were mandated to measure Crash Worthiness resulting in today's cars having better safety than those F150s, much better than the old Gold Standard of 5 Fatalities per 100 Million miles ( today all Cars are scoring around 2 Fatalities per 100 million miles ).
Measurements corelate facts and end up being critically helpful in creating improved designs.
It takes an Analytical type person to do this type of work, a rare breed in Audiophile Worlds.

Tony in Michigan

ps. I'm hoping for someone to develop a measurement system to evaluate future wives with a full set of graphs:

1). distortion measurements curves,
2). a loudness graph ,
3). an efficiency graph,

Wow, that would be helpful!

Suraki's picture

This article is the perfect explanation, why this is my favourite headphone related website. The more subjective sites and forums are useful for additional informations but Tyll's approach is much more reliable as first stage filter.

castleofargh's picture

I can't read, so stop publishing books!
oh and I don't like mushrooms, so please let's get rid of them worldwide!
^_^

your headphone list is a very necessary source of intel, and the more headphones there are, the easier it is to zero in on the specs we love, and know which to avoid like a plague.
with headroom^_^, golden ears and a few cool guys like rin choi, we end up with almost everything covered and we can often check if a problem was only a one pair accident or if it comes up on all the measurements. and that's fracking great!

Long time listener's picture

It's fantastic that we can access so many measurement charts here. Once you learn to read them, they really do tell you what a headphone will sound like--in general terms. Unfortunately, learning to read them effectively often means buying a lot of headphones, and then correlating what you like or don't like with the measurements. That involves a lot of money.

joneson's picture

Tyll, what about a post where your describe the correlation between measurements and sonic qualities (or lack of)?

Jazz Casual's picture

and that's here. ;) But seriously folks! I look at them with interest but they have never influenced my purchasing decisions. That continues to be an entirely subjective choice.

Shalow's picture

Apart from the fact that I do value your measurements, I can, somehow, understand people who just didn't worked out on your explanations about measurements. I did it by the way.
I got to say that, some people are more concerned by quick statement such as : "This is a good can". I'm more interested by a "This is a good can because..." but having spent some time with friends, relatives and other people, they usually want quick answer. I feel that, this fact, may lead to some issues when you want to value measurements.
I don't want to say that it is good or bad, I just wanted to add this little extra info which, I feel, could help understand why others don't see the true power of measurements.
From my point of view, zobel expressed everything about why your approach is completly viable and, I would say, more valuable. But community is what it is :).

Keep going on dude and thank you for all great stuff you did, people from France is reading you :).

Realsoundisgood's picture

Couldn't agree with you more, Tyll. To me, researching how headphones sound, without researching the objective measurements, is like looking for a car to buy based on how someone else thinks it rides without also knowing the type of engine and transmission, the MPG, the safety features, etc. I like to have as much information as I can gather - subjective and objective.

Bobs Your Uncle's picture

To draw from the fount of insight that is George Orwell: “Journalism is printing what someone else does not want printed: everything else is public relations.”

The demands on our time & attention increase constantly, as do the pressures resulting from such demands. As we strive to negotiate these evermore demanding environments with resources that are continuously eroded (or at best remain constant) our longing for "quick & easy answers" becomes more passionate.

A significant problem, however, lies in the knowledge that:
1.) "quick & easy answers" are few in the "real world" (if they exist at all)
2.) we are surrounded & constantly solicited by an overabundance of people & institutions eager to provide us with all the "quick & easy answers" that simply do not exist in the "real world"

If we remain mindful in our pursuit of positive, productive lives, a great place to start is by stipulating to reality. Given that we cannot possess perfect knowledge, & that we cannot command control over all possible eventualities & outcomes, we can at least avoid hobbling ourselves through leveraging that which can be known.

It's kind of a drag sometimes, but life does require some effort. It's a participatory sport.

    Tangential Opining
      : I truly lament what I perceive in a significant segment of society as an orchestrated push toward the rejection of objectivity, the disparagement of critical thought & an insistence upon (or the unyielding support for) an unquestioning embrace of dogma & ideology. But hey; Sociopathy for fun & profit!
forkboy1965's picture

I will echo some of others... in shopping for any product one needs to establish a starting point. While measurements only convey a portion of the overall picture they at least provide a jumping off point. A means by which to potentially compare and contrast what are similar products at least from a measurements standpoint.

From there comes the subjective cues and, eventually, a purchase decision.

skris88's picture

Audio is a learning process. Our brains are designed recognise a baby in distress whether that sound is real, hi-fi, or lo-fi. If lo-fi is all that one is used to, hearing a hi-fi system can actually be off-putting!

So I prefer measurements to wordy reviews.

In anything if you want to be lazy and not learn about the technology of a technical device, then you'll get cheated - that is your choice.

Of course items measured similar can still sound different, and that's where the reviewer helps by her or his subjective comments.

Knowledge is power.

Keep those headphone measurement graphs coming!

ysyung's picture

All the measurement graphs are valuable to us, great reference to the industry and consumers.

Journeyman's picture

Tyll this article shows why you are by far the best headphone/gear reviewer on/off the internet.
Keep doing what you do best! Objective measurements are a great way to understand the gear if you know how to read them, if not just skip them.

MarlenesMusings's picture

Articles like these are the reasons why this place is my favourite place to read about any headphone. Headphones are electronic devices, they are machines that are engineered according to pre-set rules. The effects of those "rules" can be measured. To do so is vital IMO, without measurements any review about any device concerning itself with audio playback is only half complete without those measurements.

They complete the picture one has of a certain piece of gear. People ignoring measurements are - from my point of view - stupid and limited. Furthermore, they can be fooled easily by any kind of "audiophoolery".

For my blog, I like to measure myself every piece I review even though I´m only able to perform incomplete half-assed measurements. But I still do them because I myself have the desire to know everything about anything (if I can).

Very well written, lovely article. As always, Tyll.

sszorin's picture

Silly man whose life is ruined by encountering headphones' measurements. Some people do not read books, they only watch movies so there is no need for books. And some people do not eat pizza.
I found headphones' measurements, especially FR graph, a pretty good indicator of headphones' sound. Not to a 100%, maybe only to a 90%, but close enough. There are some peculiar exceptions though. Beyerdynamic T1 has worse measurements than measurements of Sennheiser HD800, some of them as FR graph of T1 looks better, but I find T1 to sound better than HD800.

X