Researchers conclude, "No correlation between headphone frequency response and retail price."

Just a few days ago in my Sony MDR-Z1R review I said:

When I'd rather listen to the Audio Technica ATH-M50x we've got problems. I don't want to misslead here, it's an okay sounding headphone, but 'okay' just doesn't cut it at $2299.

(Please note this is a statement of preference not performance.)

Boy, did that cause a stir. You can check it out yourself in the thread starting about here. I'll save you the time with a few quotes:

"Tyll does not like the Z1R, prefers lowly M50. OUCH!"

"Yeah sure it has a "tuning" but come on M50's over these, I wouldn't pick the M50's over the Z7 let alone the Z1R"

"Man tyll just pooped all over these headphones lol, i do agree with him compairing to the mr speakers but that part in the end with the athm50x compairson killed me."

"I haven't heard the MDR-Z1R. BUT! Tyll's comment was that he would more preferably listen the ATH-M50Xs is hard to swallow."

It seems to me that part of the astonishment folks express is that it just doesn't make sense for a cheap headphone to sound as good as an expensive one. Okay, that's a rational way to think. How could a $100 Huffy be anywhere near as good as a $2000 Cannondale? It absolutely won't be.

Sadly, in my experience, the headphone world just isn't like that. For a start there's no accepted standardized measurement target for manufacturers to shoot for—the Harman curve is gaining traction, but it's far from a standard.

Subjective evaluation of sound quality is hard enough given biases and psychological accommodation. Headphones are a completely artificial way to hear things, and therefore, I think subjective evaluation is even harder than when evaluating speakers, which uses your OEM listening equipment in a normal way.

Further, people buy and are satisfied more with comfort and styling than they are with sound quality. It's no wonder manufacturers are all over the map with sound quality.

Well, now there's researched evidence of that. In a recent paper published by The Journal of the Acoustical Society of America titled, "No correlation between headphone frequency response and retail price," researchers found no evidence that price has any effect on frequency response performance. They used the Harman response as their baseline, but it really doesn't matter. What the paper shows is that there is no evidence showing expensive headphones narrow in on any target or common response.

Here's a couple of quotes from the paper:

This study quantifies variability of measured headphone response patterns and aims to uncover any correlations between headphone type, retail price, and frequency response.

The market for headphones and earphones has been growing steadily over the last decade, both in value and volume. Market reports indicate a world-wide aftermarket reaching over 11 × 109 units sold per year (Iyer and Jelisejeva, 2016), with growth rates of up to 55% in the 25–49 USD segment. Research suggests that factors influencing consumers' choice as to which model to purchase are mostly based on wireless functionality (Iyer and Jelisejeva, 2016) and attributes such as shape, design, and comfort (Jensen et al., 2016). Interestingly, sound quality does not seem to be a major attribute for purchase decisions.

Based on the evaluation of the mean, variance, PCA, and mean square error with respect to a target function, no correlation could be observed between the measured magnitude response and retail price of headphones. However, the variance in low-frequency response seems to decrease with increasing price, indicating an improved bass response measurement consistency across headphones in the higher price range. [Ed note: in other words, expensive cans have better quality and seal on their ear pads.]

Nevertheless, assuming that the perceived audio quality is largely determined by the spectral magnitude response of headphones, there are plenty of relatively cheap models that match the assumed target function, as well as very expensive ones that deviate significantly from an assumed ideal response.

*sigh*

I do think headphones have generally gotten better over the last five years, but I absolutely agree with the paper's findings that price has little to do with sonic performance.

I gotta get back on the hunt for good, cheap headphones I reckon.

COMMENTS
thefitz's picture

You're paying out the wazoo for leather, carbon fibre, and low batch numbers anyway. THD levels are a different story.

I mean, there's ---> ZERO! <--- quantitative evidence that cables do a damn thing... but here we are.

MRC01's picture

How about a new Wall - the Wall of Accuracy? It represents the headphones having the best measurements.
Score a few key measurements (FR, distortion, Sq wave response, impulse response) from 1 to 100. Score represents how close they are to "perfect".
Combine the scores with weights representing typical audibility, to generate the rank. FR has highest weight because if it's not right, nothing else matters.
Publish the scores & weights so readers can re-compute ranks using their own weights. Or make that an interactive feature of the page.
Price is not part of the score.

Bansaku's picture

Tell that to the electronic engineer I worked with, working out of a multi-million dollar test lab, who show'd me first hand using gear I guarantee no tech review site has access to. I made a joking comment " What difference does 26ga wire make over 28ga? " after I accidentally brought him the wrong spool of the wrong gauge and wire type, and man oh man was I schooled! If my ears weren't enough validation, seeing real numbers on a Xeon workstation's screen in real time convinced me.

thefitz's picture

I'm sure there are several scenarios where using bonkers-thin wire like 28ga over long lengths make a difference.

Headphones is not one of these scenarios. Even speakers have some people rolling their eyes, insofar as the wire impedance should be no more than 5% of the minimum speaker impedance. And we're talking dozens of feet of wire with 4-8 ohm drivers. 10 feet with 20 ohm drivers? 300 ohm drivers? 600 ohms!? Come on!

(And as an aside... how come upgraded cables always seem to "open up the mids" and cure a headphone of whatever flaw it's known for? Cables cure an HD800's 6kHz peak... make Audeze treble clearer... extends HD650 bass... all of this with more open mids of course!)

Bansaku's picture

Because of NDA I cannot disclose too much (I kid you not, even though I am no longer employed at the company). The length of wire I am talking about measured no more than 1m, usually a lot shorter, and the products in question are similar in design to a dynamic driver (minus the membrane) and their power requirements are very similar to headphones. The readings I saw in regards to both length and material (copper, silver, hybrids, and platinum) measured in the 0.XX range. How does this relate to headphones, heck if I know, but my point is material, thickness, and length (as well as insolation and temperature) do play a huge role in how well that signal is passed from the transmitter to receiver and are measurable.

I will agree, 100%, that people's reactions and expectations regarding how well an "upgrade" cable will perform is laughable. In my experience, using a better quality copper cable v.s. stock does alter the sound for the better, but the effects are

    extremely subtle
      in regards to changes in the frequency response and/or soundstage; Noticeable to my ears, but not night and day. While a good cable like 99.99% OFC or OCC will increase transparency (/clarity/smoothness), a bad cable's potential to introduce grain is very real. I tend to think this is why people are hearing what they hear. It's not that x cable increases say bass, or soundstage (because a cable is passive and simply cannot do that), no it's that the cheap cable is masking the signal and therefore cleaning up the signal path makes it appear as an enhancement/increase.

      One last blurb that's going to trigger people. Silver for use with headphone/interconnects in no way enhances anything. Copper is king, period! In real world applications the only reason silver (and platinum) is used is because of factors such as thickness vs signal transmission, and the effects of environmental conditions such as moisture and temperature. In terms of performance, copper slaughters everything. It's only weakness is the fact that it corrodes a lot easier than silver, gold, and platinum.

thefitz's picture

Tyll actually tested the FR and impulse response of several cables on an HD650 and found no appreciable change. One reading of one stock cable was out of whack... but another stock cable was almost identical to all the other cables.

There's a theory postulated that better cables yield better THD. Fascinating concept, truly. However, will one be able to detect a drop of THD when it's less than 1% with the stock cable?

Cats say that THD is stupid, and you need to look at harmonic distortion. Apparently stock cables just so happen to lower the unpleasant harmonic distortion. Fascinating theory again.

But all these are as measurable as the FR and impulse responses were when objectively proving cables do not impact those things.

Morphemes's picture

If you look at the data, the better cables had slightly better impulse response. You could say he found "no appreciable change" and he said it too, but there was indeed a measurable change. Even if it's just a fraction of a dB extra on the initial impulse (which it was), that could correlate to slightly better acceleration of the driver overall. You wouldn't really see it in a sweep, but your average listening situation isn't a sweep. Not that I think a "good" cable should cost you an arm and a leg, but there are "bad" cables out there...

MRC01's picture

What made me think of this is perusing the headphone measurements here realizing there's no easy to see which headphones have closest to ideal FR.
If such a thing existed it would be a good way to build a short list of headphones to audition, having whatever attributes one values most.

steaxauce's picture

This is pretty annoying and definitely matches my experience. Frequency response isn't all that matters, though.

When using Sonarworks calibration, the ATH-M50x, HD650 and HD800 all have very similar frequency responses as far as I can tell. In terms of overall sound quality, though, post-calibration the HD800 is much better than the HD650, which in turn is much better than the ATH-M50x. At each step you get a better sense of space and more clarity and impact.

More expensive headphones usually come with better drivers, housing and pads. If the designer doesn't completely screw the pooch, those things ought to lead to performance improvements like lower distortion, flatter wavefront, etc. My theory is that frequency response is different, though, because headphones have to shoot for a target curve that isn't flat, and better parts don't reduce the need for tuning to hit that target. That takes significant R&D, which often doesn't make sense for expensive headphones because of poor economy of scale.

Calibrating high-end headphones can lead to really good performance, though.

The Z1R is an exception in that it performs poorly all around. Sony really made me feel like an ass for preordering those.

Bern L.'s picture

I am interested to know if you sent your Z1R's in for calibration to Sonarworks.

steaxauce's picture

I haven't tried that yet, but was thinking about it. Maybe it's too soon to say whether they'll sound good post-calibration.

Bern L.'s picture

Thanks for the reply. Agree with your assessment of Sonarowrks...it saved the HD800 for me. I EQ the mid/upper bass down a bit on the ZR1....I have found them to be the most "speaker like" experience yet in a headphone.

ELPCU's picture

I believe frequency response takes very big role in sound quality, but I do not thing it is everything.

Though I wanna say that I won't buy any headphone which costs 1000 bucks or more without good FR curve. Simply not worth it.

AstralStorm's picture

Please define "much better post calibration" between HD800 and HD650. Even after equalization and mods they each keep their own basic character and neither is clearly better, they have different strengths and weaknesses.
If you mean "I don't care about x in HD650 sound but I do care about y in HD800 sound" please state so.

Even then, "much" is not a quantifier I'd ever use when comparing the two.

One thing that "much" applies to, is that HD800 is much more expensive and materials are much nicer.

steaxauce's picture

Thanks for the reply AstralStorm. Have you tried Sonarworks? EQing and modding by ear it's hard to get two headphones to have exactly the same frequency response. Usually the focus is on correcting easily identifiable flaws, while leaving the overall character of the headphones in place. (For example, the EQ profiles Tyll has published for the headphones he's tested haven't flattened them entirely.)

Sonarworks isn't really like this. To my ears, using their EQ profile, the "character" of the HD800 is completely removed. The HD800 is stereotyped as a lean, bright, analytical-sounding headphone. I don't feel there's any of this sound left after calibration. Similarly, the HD650 loses its mid-bass hump and isn't a warm-sounding headphone anymore. From a purely FR perspective they sound the same.

Obviously the phrase "much better" doesn't have any objective meaning, and whether the HD800 is much better than the HD650 with calibration depends how much you care about the differences. I'd personally describe the HD800 as sounding more natural, spacious and impactful. By contrast, the HD650's famous "veil" doesn't seem to be merely a frequency response thing and sticks around post-calibration. The HD800 made a pretty big difference for me, so for me it's "much better", but to you and others it may not be, which is fine. It's honestly hard for me to imagine someone thinking the HD650 sounds better than the HD800 when listening in this way, though. I do think the HD800 is clearly the better of the two. I haven't experimented with any mods, though, and can't comment on that.

If you haven't tried Sonarworks, I recommend you check out the trial, but be sure to remove any mods beforehand because their profiles are set up for stock headphones.

AstralStorm's picture

There's no way to do "the same".

For starters there are things that are not equalizable with any amount of DSP as you get a huge mechanical resonant null or peak, these tend to be very sharp and move around. Place/replace and pad variance is pretty high in terms of positioning the peaks and nulls in highs.

Additionally one thing that cannot really be fixed is the bandwidth really. While you can, say, boost subbass on HD650 or HD800, it won't get you much as the excursion will be exceeded and you just get distortion.
Likewise with highs, you will get nothing as membrane is too stiff or heavy to move given any amount of input. On this front though, both are in equalizable territory.

I do my own eq that is noticeably better than Sonarworks one as it is tweaked to the specific headphone SN and thus I can fix some (but not all) specific nulls. Plus I have an experimental tunable phase equalizer to improve on decay in places which need it. The latter has to be done very carefully to not be screwy when placement variance is involved.
The general gist of mine vs Sonarworks eq is similar on HD650 and mine is much better at HD800 though not perfect.

It would be probably best to have a closed loop circuit for it, like in active noise cancellation. Very similar to how room cancellation is done in speakers, but easier. Sadly those designs have long been patented. (Apple for IEMs, Bose for headphones.) Don't expect them in anything high fidelity.

As to HD800 vs HD650.
The former needs a more advanced eq to get rid of the horn effect in the lower highs (~5k) which shows up on CSD, is in a sensitive range, and cannot be fixed with either linear phase (Sonarworks) or minimum phase (CALF, analog etc.) equalizer.
Nor even with a mixed phase eq unless you make it super long impulse perhaps.

Nor can it be changed with a mod - unless you replace the whole earcup. It can only slightly be reduced by dampening unless you want to lose all the highs including extension.

On the other hand, HD650 has longer decay in bass which also require such equalizer. Or you can install a mechanical bass dampener which brings it to next level. Then there's a dip around 6k which required a bit of trickery combining linear phase and minimum phase eqs together. (Or very specific mixed phase weirdness.)

Have fun fixing things. :)

AstralStorm's picture

As to using someone else's curves, my HD650 SN has different placement of the one important peak. Not at 6k which Sonarworks corrects but at 7.3kHz. This is attributable to different pad thickness.

Here's the link to that eq, since minimum phase alone get real close. For HD800 I'd have to release the research equalizer that is not ready for publication yet.

https://www.dropbox.com/s/t22ml3mt9b385l6/2017-07-29-164921_1099x657_scr...

BrooklynNick's picture

I can't seem to find any list of what headphones were tested in that study. Do you know? The STAX SR-009 looks quite a bit better than most in your measurements. I see that the top price was 5120, which is probably the Abyss.

Type35's picture

Ultrasone anyone? Anyone?
And let's not even talk price vs manufacturing cost.
The margins on headphones would make an investment banker blush with envy.

Jazz Casual's picture

The research findings come as no surprise to me whatsoever and nor did the responses from Head-Fi'ers to your MDR-Z1R review.

--------------'s picture

Harman made a similar study with identical results. They only tested IEMs though.

http://seanolive.blogspot.com/2017/02/twirt-337-predicting-headphone-sou...

inks00's picture

The MDR-Z1R rings like a bell at 3KHz and 10KHz. Just look at the impulse response and square wave measurements.

The magnitude of the problem is can be seen on CSD plots... see SBAF. http://superbestaudiofriends.org/index.php?threads/sony-mdr-z1r-measurem...

Expensive headphones that misbehave with significant ringing and resonances should be named and shamed. If you like the literally extra-musical 3KHz sass and 10KHz tizz shooting arrows directly into your skull, then the MDR-Z1R is for you.

thefitz's picture

It's nice to actually see a link to SBAF on a thread without moderators crying ;-)

Johan B's picture

Lets get back to analysis of the drivers. Fast responding, fast dampening and neutral drivers are key to good performance. Then after that the earpads design and "can" design are second order effects. If I may draw the analogy to speaker drivers where frequency response, distortion, resonance frequency, impuls response and in particular the waterfall plots are excellent means to see quality. Have you ever heard high quality speakers with a bad waterfall plot? What measurements are we missing to distil good from bad drivers?

AstralStorm's picture

Open air 3 cm distance driver measurements of course, but these are destructive and few want to do them.

thefitz's picture

Doesn't it cost $250 to replace the drivers on Audeze's ~$2k headphones? People, you're paying for the bling.

And THD readings.

MRC01's picture

Actually Audeze quoted me $400 a couple of months ago. Yet even though this is still relatively cheap compared to the LCD-4, it's the same price for all models and almost half the price of the LCD-2. Even if it were only $100, this is not necessarily a good analogy. For repairs or upgrades to existing customers they are probably billing marginal cost of the next unit produced which doesn't cover R&D and other costs. Most companies recover their full cost and become profitable from new sales, not upgrades & repairs.

The high prices of rare models (whether headphones or anything else) are due in large part to low production numbers. If the price difference between LCD-2 and LCD-4 (or Focal Elear vs. Utopia) were purely the mechanical/electrical differences, they would be much closer in price. But they sell a lot fewer LCD-4s and Utopias. So the R&D and other design work unique to those models has to be amortized over far fewer units.

That's how cheaper "flagship" models give you something like 95% of the objectively measurable accuracy for less than half the price. Plus over time they trickle some of the R&D of the TOTL down to these cheaper models. You mention THD... incidentally, THD of the LCD-4 is admirably low, but not any lower than the LCD-2.

thefitz's picture

I should have definitely elaborated more - you do hit a "saturation point" with THD, but that's a major distinction when claiming price has no impact on FR.

My point is - if you produced 25,000,000 units of an LCD-4 drivers with pleather and aluminum instead of carbon fiber on an assembly line, you'd end up with something much, much cheaper what would sound identical to the $4k LCD-4... assuming you can make two $4k LCD-4s sound identical :D

cspirou's picture

I learned this lesson recently with IEMs. I've been using the re400s for awhile and I enjoy them quite a bit. I was looking for an upgrade and I found the nuforce primo 8 for $250. I thought it was a good buy because it's reviewed on a lot of 'important' sites and it had an original MSRP of $500. Immeadiately I noticed that it had good resolution but the highs were a bit harsh. This went away at lower volumes, but paying 3x the price for this sort of limitation is inexcusable. Comparing directly with the re400 had me enjoying the re400 far more.

While there are certainly $250 IEMs that are worth it, not all are necessarily better.

So far the Innerfidelity Wall of Fame is spot on. While the list isn't definitive, I do think it's a safe list. The re-400 and the HD600s I bought are great and fairly priced.

Three Toes of Fury's picture

This subject is near and dear to my heart...Cost VS Sound Quality.
Its one that i wrestle with all the time...my primary questions are: 1) Couldnt a manufacturer create low/mid cost (lets say $100-$200) headphones that exhibit sound quality of the uber expensive models?? 2) I believe they easily could but dont due to product differentiation. 3) There are, indeed, "diamonds in the rough" out there which are lower cost headphones that give above value sound...thats my favorite thing to find on this site.

Im not negating costs associated with specific materials or process', however when it comes to headphones, lets be real, there are VERY few components. Sure there's bound to be production challenges in making the fancier pairs, but imagine if a manufacturer came out with a pair universally embraced by the community as the high-price-killer? They'd get gobbled up! I suppose the issue here is missing piece from all reviews...individual subjectivity and preference...there is no universal standard for acceptance. Waverforms are one piece of the puzzle.

Anyway...enough of my babbling...Tyll, one thing i think about and appreciate often is the photo of you and your wall of fame. I know, sitting behind you, are countless pairs of >$1k headphones. And yet you regularly provide good, critical, opinions and reviews of great sounding, lower cost cans. I certainly appreciate reading about your experiences with higher cost items but love that you still look for those diamonds in the rough....sooo..in your quest for "good, cheap headphones"...happy hunting!!!

Peace .n. Living in Stereo

Three Toes of Fury

Pages

X