Head-Fi Contradicts InnerFidelity Sony MDR-Z1R Measurements

Out on my morning web surf and I stumble onto a bombshell on the front page of Head-Fi:

"Sony MDR-Z1R Measurements: Head-Fi HQ’s compared to InnerFidelity’s"

That article links to a much more comprehensive post in the Z1R thread by Jude Mansilla, Head-Fi's Founder. It seems Jude's measurements are somewhat different than mine, particularly in terms of the peak in response around 10kHz. Here are the two plots in question:

InnerFidelity Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_InnerFidelity

Head-Fi's Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_HeadFi

It's important to note here that the Head-Fi plots are compensated plots and are best compared with the top blue and red traces in the InnerFidelity plot. It's also important to note that it is somewhat unwise to compare plots between different systems as they will vary. And lastly it's also worth noting that Head-Fi's system is of excellent quality. There's a description in the linked post above and I'm jealous...very nice gear.

In Jude's post he concludes:

What's our conclusion? Right now, we'll assume that perhaps Tyll has an outlier Sony MDR-Z1R. n=3 across two different measurement setups isn't enough to come to a conclusion. If Tyll will allow us to measure the MDR-Z1R he has, that would certainly be helpful. We'll also be receiving at least one other production MDR-Z1R, and we'll measure that, too. Based on what we've heard here (and also on hearing other MDR-Z1R's)--and based on many reviews/impressions that predated the publishing of Tyll's measurements--we'll also assume for the moment that our impressions and measurements are more representative of the MDR-Z1R's that are out in the wild.

Bit by bit:

"Right now, we'll assume that perhaps Tyll has an outlier Sony MDR-Z1R."

I've found assuming a poor idea. Usually I'll try to think of a number of possibilities and then move toward gathering evidence that points towards a conclusion. It could be that I have an outlier, but it might be that Jude's compensation curve has a big bump at or near 10kHz. I'd certainly like to see the compensation curve used, and I'd really like to see the raw measurements from his HATS (head and torso simulator).

"If Tyll will allow us to measure the MDR-Z1R he has, that would certainly be helpful."

Absolutely, and back attcha. Let's trade headphones and re-measure. I'll shoot Jude an email.

"...we'll also assume for the moment that our impressions and measurements are more representative of the MDR-Z1R's that are out in the wild."

Yeah...that assume stuff. Given other measurements available on the web, I think there is a significant peak at 10kHz. Let's take a look:

From Speakerphone:

Raw Frequency Response Measurements 170616_Blog_HeadFiMeasurements_Measurements_SpeakerphoneRaw

Cumulative Spectral Decay 170616_Blog_HeadFiMeasurements_Measurements_SpeakerphoneCSD

From PMR Reviews:

Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_thatonenoob

From Superbestaudiofriends.org:

Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_SBAFFR

CSD Left
170616_Blog_HeadFiMeasurements_Measurements_SBAFCSDL

CSD Right
170616_Blog_HeadFiMeasurements_Measurements_SBAFCSDR

CSD Foam Coupler
170616_Blog_HeadFiMeasurements_Measurements_SBAFCSDfoam

From SBAF Member spawth:

Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_SBAFspawth

Again, comparing measurements from a number of rigs is rife with potential miss-assumptions, but it does look like there is a significant feature at 10kHz in all of them.

Since Jude has measured three Z1R headphones on his system that measure similar to each other and without such a feature, I'm lead to believe his system is measuring repeatably. Possible sources of this error might be (in order of likelyhood from my experience):

  1. The geometry of his head may be different than mine and may not measure the 10kHz area the same as mine. It's clear from Audeze's comments about measurements on various systems that this might be the case.
  2. His compensation curve has a feature at 10kHz that is reducing the peak in the headphones. Peaks at or around 10kHz are common in these curves. Here are commonly used DF and FF curves, both show elevated high frequency energy or peaks:
    170616_Blog_HeadFiMeasurements_Graph_DFCurveA
  3. Frequency response repeatability at high frequencies is know to be riddled with error. Small differences in headphone position can make all the difference. It's possible that Head-Fi's method for positioning the headphone on the head for measurement is generally different than mine and may result in differences in high frequency measurements.
  4. The headphones that i have are indeed not representative of currently produced product.

Personally, I'm inclined to believe #1 above, but I'd really need to see the raw data to move forward on that thought. Jude willing, I'd be very happy to pursue this further.

I'd love to hear your thoughts in the comments below!

COMMENTS
sarin's picture

next time get your fact right in first place..don't try to cover up like this.Or apologize perhaps??

tony's picture

what am I to apologize for ???

Tony in Michigan

AstralStorm's picture

"Mid" fi and is such a silly term. It is devoid of meaning because everyone moves the goalpoast.

There are no out of the box "hi-fi" headphones out there. They don't exist. Not even boutique top of the line stuff. (This includes Stax, ZMF, Audeze, Hifiman and top Sennheisers.)
Best ones can be made high fidelity either with a mod or equalizer or both - and then a spatial simulator on top.
In fact out of the box the closest seems to be Sennheiser HD 650/6xx or ZMF Eikon. Note few thousand dollar difference between these two, and neither are the most expensive.

The definition of high fidelity involves a played back recording being true to life. There are no tiers in it, only smooth comparisons.

MDR-Z1 are decent but definitely nowhere near the best bang for buck you can get in these terms. The resonance is audible but not getting to terrible levels. However, such issues are very hard to get rid of, even with mechanical modifications. The FR needs eq to sound lifelike.
About the only place they excel is at bass (except subbass) reproduction.

AstralStorm's picture

They are definitely better than anything Beyerdynamic or AKG released ever.

Luigi's picture

in my experience, having extensively tried almost every hp under 2500 bucks, i can certainly say that the akg models, from k712 to the new K872, are in another world in comparison

darkswordsman17's picture

This (the Sony as well as the issues seen with the Audeze and Focal) might be just some fairly minor hiccups in early production runs or even fairly anomalous units, but I think this might be a symptom of an issue that seems like more and more people are starting to notice. Companies are definitely listening to feedback (and especially when it can be pointed to objectively, they know what to address), and often release products that "fix" the issue. But, that is becoming an issue itself, in that has consistently come with a substantial price increase. So its like they get to have people beta test a production run, tweak it, and then since they fixed some issue they can justify a price hike.

ultrabike's picture

I think there is some evidence that Audeze had (and may continue to have) issues with their drivers.

I think Focal send a Utopia headphone to Tyll that was not very representative of the final production unit. But what is out there in the field does seem very consistent. I measured two Utopias, and both units measured remarkably the same. This BTW was repeated and shown by Tyll on a separate article in which he measured way more than two Utopia cans, and those measurements were fairly similar.

I don't think Sony had some hiccups though (aside from apparently producing an overpriced lack luster product).

Argyris's picture

We'll see how the measurements compare when Jude's headphone makes its way to IF (and visa versa).

However, does anybody else think it's a bit crappy how Tyll wasn't contacted about that article before it hit the front page of HF? Normally when you have a big discrepency like that and want to make a story about it, it's common courtesy to reach out beforehand to the person you're referencing and trade notes, or at least let them know what you're planning to. Not to mention, the article itself sounds vaguely accusatory and incredulous. It's as though Jude is questioning how Tyll could have come to the conclusions he did and is using his (Jude's) own measurements to prove a point, rather than highlighting a measurement and impressions discrepancy and suggesting possible causes, as Tyll has done in this article.

Tyll Hertsens's picture
Seriously,I've got no problem what so ever with Jude posting as he did. I might have done the same in his position. When you have a soap box, that's where you talk from. Really, it keeps the dialog out in the open and that's a good thing.
donunus's picture

I agree with this comment. It's best to put everything out there to get to the bottom of things. Everybody learns more in the end.

maelob's picture

I would definetly like to see a standard in the way Headphones are measured across the board. At the end is a good thing that Tyll and Jude collaborate on this issue. also sorry to say that I like the Z1r lol

donlin's picture

Rather than Tyll and Jude exchanging headphones for measurements, I'd much prefer they listen to each one side by side and compare how they sound. In the end that's all that matters.

Tyll Hertsens's picture
Hm. Good point. I'll see if I can work that out.
raulromanjr's picture

I like Head-Fi and I like Innerfidelity. I appreciate how you keep it professional and don't get trapped into what some people want to see (a bit of drama). If I read your you right through what you publish, you probably find this interesting and want to explore it to reach some better understanding for you and your audience. I'm sure Jude would find it equally beneficial for you two to work together on this. Looking forward to the process and I hope you both document your interactions, research and conclusions on this matter.

me klasse's picture

Despite not being a 100% reliable method, especially when ear simulators are involved, one can always compare the measurements of other (known to be consistent from pair to pair) headphones in both rigs (Innefidelity and HeadFi). If they differ similarly in the 9-10kHz region then the rigs/compensations are responsible for the exposed differences on Z1r measurements.

Here we have Innerfidelity (thicker trace) and HeadFi(thinner trace) measurements of Z1r scaled and overlapped so the difference is clearer:
https://s23.postimg.org/rwovvbme3/Z1r_Tyll_Jude.png

Here we have measurements for the consistent HD650 on both rigs for the sake of comparison (IF:darkblue-red / HF:lighter blue): https://s23.postimg.org/nl0a6bfh7/hd650_Tyll_Jude.png
We can see a ~10dB difference in the 9kHz region here.

Here's also the same kind of graph comparing Utopia measurements on both rigs (IF:darkblue-red / HF:lighter blue):
https://s23.postimg.org/nz1m5wzkr/Utopia_Tyll_Jude.png
We can see a ~10dB difference in the 9kHz region here as well.

Me x3

ultrabike's picture
This IMO is a very helpful comment. It comparatively shows how the different rigs produce different results in the absolute sense. I don't think it's easy to compare results from two different measurement rigs.
ADU's picture
Quote:

It comparatively shows how the different rigs produce different results in the absolute sense.

Not sure I agree with this.

I believe the plots being compared in me klasse's post above are both compensated/corrected.

To get a better feel for the difference in "absolute" measurements between the two rigs, I think you'd have to compare the raw _uncorrected_ FR plots of both systems. Or possibly FR plots corrected with the same _type_ of compensation curve.

Diffuse Field is the most common. But I believe Tyll still uses an Independent of Direction HRTF for all his compensated plots.

I don't know what Jude is using for his compensated plots, but I sort of doubt that it's an IoD function.

It has been stated here before that the Head Acoustics IoD correction curve (which is what Tyll uses) results in more prominent peaks in the 9 to 10 kHz range on compensated plots than the Diffuse Field curve. That's because the IoD curve has a "dip" in that frequency range, rather than a peak...

https://www.innerfidelity.com/images/110401_measure_graph_headacousticsh...

Quote:

I don't think it's easy to compare results from two different measurement rigs.

I'd agree it's a difficult proposition under any circumstances, unless you really know your way around both systems. Using the same type of compensation curve should theoretically make it somewhat easier though.

ADU's picture
Quote:

I don't know what Jude is using for his compensated plots, but I sort of doubt that it's an IoD function.

After looking at the fine print in Jude's post, it appears that he is indeed using some sort of Diffuse Field compensation...

Quote:

The frequency response measurements reflect diffuse field correction applied via the APx555's input EQ.

That may account for some of the difference, since Tyll is using a different type of compensation curve, namely the Independent of Direction HRTF, which is probably not as forgiving in the 9 kHz range as the DF curve.

Jude is also using a different head & torso simulator, the G.R.A.S. 45BB-12 KEMAR, while Tyll is using one by Head Acoustics. The differences between the two HATS systems could perhaps be somewhat mitigated though if they were both using comparable DF compensation curves appropriate to each system.

And as others have pointed out, it looks like Jude is probably using more smoothing on his compensated plots.

Since they're using different HATS systems (and probably also different smoothing), I would not expect Jude's raw/uncorrected plots to look the same in the treble as the raw Inner Fidelity plots. There could be more of peak at 9 to 10 KHz on Jude's raw G.R.A.S. plot than on his compensated plot. Or it could just be smoothed away.

Now, about those Audeze and Focal HPs... ; )

ADU's picture
Quote:

To get a better feel for the difference in "absolute" measurements between the two rigs, I think you'd have to compare the raw _uncorrected_ FR plots of both systems. Or possibly FR plots corrected with the same _type_ of compensation curve.

I need to correct my above statement.

RAW/NO COMPENSATION

If there's no difference in sound quality between the two headphones being measured, then the raw plots with no compensation should show the difference (if any) between the two HATS systems.

DIFFUSE FIELD COMPENSATION

Performed correctly, Diffuse Field compensation should show the difference (if any) between the two headphones, irrespective of any differences between the two HATS systems.

Sorry for any confusion on this.

AstralStorm's picture

Diffuse field compensation is not quite a standard you make it. Unless you have the actual compensation curve, you're essentially guessing as to what it is.

Three Toes of Fury's picture

.and post opinions...good and bad.

It appears there's a bit of a trend forming where folks get defensive when something critical is posted lately..especially involving waveforms..and then question the process to defend the gear they like, or perhaps have vested interest in.

Absolutely the science behind measurements..and methods used..should be examined and understood. And provide comparisons where/when possible.

However the reviewers that we seek input from...and I would consider Tyll and Jude tops on that list...should be able to post their: analysis, findings, data, review, suggestion, or criticism, without worrying about back-and-forth questioning of their review or stance.

The talkbacks on this site do a great job of both: agreeing with tyll in times that he's critical as well as offering differening opinions when folks own and love the gear that he didnt connect with.

Peace .n. Living in Stero

3ToF

HiFiArt's picture

Let's look at that SBAF Graph.

From my eye, the frequency reponse (FR) looks ok, from 20 Hz to 70 Hz. (+/- 3dB, ok +/- 1). Then it begins to fall off rapidly until 2K.

From 60 to 10 kHz there seems to be a major suck-out, and the 10K I see as a recovery in FR. Ok, there's a bigger suck out from 2K to 10K.

It reminds me of the frequency response of the Sure SM-58 Vocal Microphone.

http://cdn.shure.com/specification_sheet/upload/82/sm58-specification-sh...

Only the FR of the '58 is better!

What's up with headphones?

You wish an transducer quality that is at least equivalent to a microphone.

Let's look at a Earthworks M50 (not a great one but ok) :

http://www.earthworksaudio.com/wp-content/uploads/2016/08/M50-Data-Sheet...

3Hz to 50kHz ± 1/-3dB (Frequency Response)

140dB SPL (Max Acoustic Input / Playback SPL with low relevant THD) so you can do 24-bit (23.25)

audio

tinyaudio's picture

you're really comparing apples and oranges here.

You can not - ABSOLUTELY NOT - compare frequency responses of microphones and loudspeakers, let a lone microphones and headphones.

They are not measured the same way, and represent very different qualities of the specimen.
The only thing they have in common is that they can accurately be called "frequency response".
But this term describes different things for headphones/speakers and microphones.

AstralStorm's picture

I do not agree with the statement at all.
Guess what you measure headphones or loudspeakers with? Is it perhaps a microphone?

In the olden days, the measurement rigs compensated their capsules more mechanically than digitally, but in the end, there's always a microphone.

Now headphones vs loudspeakers, we have additional variables, like the room, distance, positioning of the microphone and so on.

HiFiArt's picture

For comparison, let's look at the Magico M5 Loudsepaker:

http://www.theabsolutesound.com/articles/tas-196-magico-m5-loudspeaker/

Ok, it's got a bit of a bass suckout (+/- 2 dB) at around 40, which bass heads won't prefer.

But we're literally talking about deviations of about 1 dB (or at most 2.5 dB) from a reference line for the entire frequency spectrum (20 Hz to 20 kHz).

Let's look at the Magico Q5 Loudspeaker now:

http://www.theabsolutesound.com/articles/magico-q5-loudspeaker-tas-214/

The graphs are equivalent to Tyll's now, visually.

Shouldn't headphone FR be like this?

http://media.tas.zeitpress.com/articles/images/rta(1).jpg

At least to sound representative of the recorded frequency spectrum.

tinyaudio's picture

Your graph shows a roughly linear frequency response in the audible band (20 Hz - 20 kHz).
This is roughly the target for loudspeakers, yes. (There's a bit more to it to make a speaker that sounds good in a room, but a linear frequency response as measured in an anechoic measuring chamber is a good starting point)

However the target for headphones is very, very different. Mostly because you can't really measure headphones the way you measure speakers - you measure headphones on artificial heads, and not with a measurement microphone in an anechoic chamber.
This means that the various resonances of the ear canal etc play a great role.

Long story short:
The target FR for headphones as measured on a dummy head (professionally called: coupler) is very much not linear.

For further research read up on Harman's latest research concerning this very topic (ideal frequency response for headphones, conducted by Dr. Sean Olive and Todd Welti)

AstralStorm's picture

There is no "target" for headphones, even if we're talking average HRTF over big populations which nobody has measured ever.

All alleged "targets" are guesswork by someone who thought "it sounds right" or more scientifically, derived by comparison with reference speakers in a reference room.

AstralStorm's picture

Unless you're talking IEC curves which are known to be a mediocre guess and there are a few revisions thereof for various measurements too.
Even then, the exact compensation curve and measurement technique has to be specified. There's an IEC standard for that!

HiFiArt's picture

Ok, I'll disagree with you on the first point ("you are really comparing") and take up your second.

Shouldn't there then be a frequency response chart that shows deviation?

http://www.moogaudio.com/images/thumbnails/162_images_neumann_KU100_Z.jpg

I don't have a Neuman KU81i or KU 100, so I don't know how far down the microphone capsules go on a typical binaural head used for headphone measuring - the best I've got is a Sennheiser MKE 2002 which has the microphone capsules, externally.

It's a pretty shit device, but was a pretty fun toy.

Others might have much better binaural heads for headphone measuring : it's typically used for binaural enthusiast (re: nature) recording.

I gave a pretty shit microphone comparison.

For example, if you want to record 100K, you get something like a Sanken.

http://www.sanken-mic.com/upload/prd/img/1000400_frequency_01.jpg

It's FR is pretty shit, like headphones, but you can at least digitally introduce a compensation (given you have a recorded reference).

It's got a wider FR than a Neuman, though.

Do you really want more than a +/- 0.5 dB variation?

I'm not sure I'm really into Harmans headphone response curve.

For playback their M2 Monitor, recommended by Franco Filipetti, is apparently a pretty decent bargain.

http://www.jblpro.com/www/products/recording-broadcast/m2/m2-master-refe...

The reason I'm bringing up speakers and microphones on the topic of deviation is that I believe it brings something to offer to the conversation.

If the microphone capsule is not too deep on the binaural head (not a full represenation of a ear, full canal, drum, etc.) then you will at least get some direct recorded sound.

Then, like a loudspeaker played back in a room, you just have to deal with refelctions, etc.

Most Stereophile measurements you will find (I know this is InnerFidelity) are done in home garages and such (and not anechoic chambers) with microphones like the ol' DPA (now B&K) etc.

As a consumer, I don't expect a lot of variation, unit to unit (as do not most, here), and I would like a flat FR.

I am a big collector of headphones (as they're cheaper than speakers, and easier to store: less space), but was most disappointed when I heard the Focal Utopia headphones.

Actually, completely disappointed.

I was thinking of purchasing the Focal Grande Utopia III EM Loudpeakers for front and Focal Scala Utopia III 2's for rears for a home theatre, and couldn't believe how shitty the Utopia headphones sounded by comparison.

Perhaps that's why I got to comparing headphones and speakers. Same manufacturer, same product name. :)

As to headphone FR, in particular, a pair of Atomic Floyd Power Jax headpones which I purchased, which I found quite durably made - a great in-ear headphone feature - had absolutely magnificently poor FR, in terms of deviation.

Oh well, life is life.

Since microphones are used to record headphone FR, I think it is fair to compare microphone FR with headphone FR.

Since headphones are often touted as an inexpensive replacement (the same quality for less money) for speakers, I think it is fair to bring up speaker FR as opposed to headphone.

Sennheiser for example, made really wonderful microphones before making a decent consumer headphone.

Their MKH 800 48 is really nice, today.

I had some of their headphones like the 560 and 560 II Ovation, and while they were beautiful, and luxuriously comfortable: the sound was shit.

It wasn't really until their 580 that the sound of their headphones got good, and comparable to a Rotel / NHT (non Michi) system.

In terms of binaural playback their 800 is stunning!

The full 360 sphere is properly played back (from a recording with a Neuman KU81i or 100) unless every other headphone.

I love my stax, but I'll give it to Sennheiser for making both a great collection of microphones, and a decent headphone too.

Likely, I'll pick up this Sony, but I sure wish headphone FR with respect to flatness would be improved.

It's such a simple system. +/- 0.5 dB should be achievable.

ADU's picture

You echo the frustrations of many of us.

With proper correction, I believe a flat frequency response target for headphones is possible. And it's one of the things I'm working toward for my equipment. But I'm an optimist. And only do this as a hobby.

To really be useful, a headphone target probably needs to be adaptable for different measurement systems (HATS), and also different designs (in-ear, on-ear, over-ear). And maybe also different room simulations.

Harman's headphone target is certainly a step in the right direction. But its application is limited, because it was only designed for one measurement system (G.R.A.S.). And some of us feel that its modeling of room FX is not accurate enough, and too subjective.

I haven't heard the new Focal Utopia btw. It employs a fairly radical new design though (based on what I've read here). And there are bound to be some growing pains associated with that.

Pages

X