Head-Fi Contradicts InnerFidelity Sony MDR-Z1R Measurements

Out on my morning web surf and I stumble onto a bombshell on the front page of Head-Fi:

"Sony MDR-Z1R Measurements: Head-Fi HQ’s compared to InnerFidelity’s"

That article links to a much more comprehensive post in the Z1R thread by Jude Mansilla, Head-Fi's Founder. It seems Jude's measurements are somewhat different than mine, particularly in terms of the peak in response around 10kHz. Here are the two plots in question:

InnerFidelity Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_InnerFidelity

Head-Fi's Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_HeadFi

It's important to note here that the Head-Fi plots are compensated plots and are best compared with the top blue and red traces in the InnerFidelity plot. It's also important to note that it is somewhat unwise to compare plots between different systems as they will vary. And lastly it's also worth noting that Head-Fi's system is of excellent quality. There's a description in the linked post above and I'm jealous...very nice gear.

In Jude's post he concludes:

What's our conclusion? Right now, we'll assume that perhaps Tyll has an outlier Sony MDR-Z1R. n=3 across two different measurement setups isn't enough to come to a conclusion. If Tyll will allow us to measure the MDR-Z1R he has, that would certainly be helpful. We'll also be receiving at least one other production MDR-Z1R, and we'll measure that, too. Based on what we've heard here (and also on hearing other MDR-Z1R's)--and based on many reviews/impressions that predated the publishing of Tyll's measurements--we'll also assume for the moment that our impressions and measurements are more representative of the MDR-Z1R's that are out in the wild.

Bit by bit:

"Right now, we'll assume that perhaps Tyll has an outlier Sony MDR-Z1R."

I've found assuming a poor idea. Usually I'll try to think of a number of possibilities and then move toward gathering evidence that points towards a conclusion. It could be that I have an outlier, but it might be that Jude's compensation curve has a big bump at or near 10kHz. I'd certainly like to see the compensation curve used, and I'd really like to see the raw measurements from his HATS (head and torso simulator).

"If Tyll will allow us to measure the MDR-Z1R he has, that would certainly be helpful."

Absolutely, and back attcha. Let's trade headphones and re-measure. I'll shoot Jude an email.

"...we'll also assume for the moment that our impressions and measurements are more representative of the MDR-Z1R's that are out in the wild."

Yeah...that assume stuff. Given other measurements available on the web, I think there is a significant peak at 10kHz. Let's take a look:

From Speakerphone:

Raw Frequency Response Measurements 170616_Blog_HeadFiMeasurements_Measurements_SpeakerphoneRaw

Cumulative Spectral Decay 170616_Blog_HeadFiMeasurements_Measurements_SpeakerphoneCSD

From PMR Reviews:

Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_thatonenoob

From Superbestaudiofriends.org:

Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_SBAFFR

CSD Left
170616_Blog_HeadFiMeasurements_Measurements_SBAFCSDL

CSD Right
170616_Blog_HeadFiMeasurements_Measurements_SBAFCSDR

CSD Foam Coupler
170616_Blog_HeadFiMeasurements_Measurements_SBAFCSDfoam

From SBAF Member spawth:

Frequency Response
170616_Blog_HeadFiMeasurements_Measurements_SBAFspawth

Again, comparing measurements from a number of rigs is rife with potential miss-assumptions, but it does look like there is a significant feature at 10kHz in all of them.

Since Jude has measured three Z1R headphones on his system that measure similar to each other and without such a feature, I'm lead to believe his system is measuring repeatably. Possible sources of this error might be (in order of likelyhood from my experience):

  1. The geometry of his head may be different than mine and may not measure the 10kHz area the same as mine. It's clear from Audeze's comments about measurements on various systems that this might be the case.
  2. His compensation curve has a feature at 10kHz that is reducing the peak in the headphones. Peaks at or around 10kHz are common in these curves. Here are commonly used DF and FF curves, both show elevated high frequency energy or peaks:
    170616_Blog_HeadFiMeasurements_Graph_DFCurveA
  3. Frequency response repeatability at high frequencies is know to be riddled with error. Small differences in headphone position can make all the difference. It's possible that Head-Fi's method for positioning the headphone on the head for measurement is generally different than mine and may result in differences in high frequency measurements.
  4. The headphones that i have are indeed not representative of currently produced product.

Personally, I'm inclined to believe #1 above, but I'd really need to see the raw data to move forward on that thought. Jude willing, I'd be very happy to pursue this further.

I'd love to hear your thoughts in the comments below!

COMMENTS
thefitz's picture

Even Jude's measurements show a spike at 10kHz, but none of them (except maybe the last one) show a spike that's like 6db higher than the 3kHz spike. In fact, in most graphs, the 10kHz spike is a db or two lower than the 3kHz one.

Maybe's picture

The Speakerphone graph is uncompensated. Diffuse-field compensated it shows quite a peak at 10kHz.

See: https://lh3.googleusercontent.com/-tviKM8RJVsY/WKm3RCV4M8I/AAAAAAAAFh4/H...

I'd say this specific measurement is the most comparable to Tyll's and Jude's setup, as the others are all flat plate couplers whereas Speakerphone uses a Cheek and Pinna Simulator.

Additionally the CSD plots suggest ringing at that frequency across all the systems.

spwath's picture

I made the last measurement. That spike is much bigger than others. I would not necessarily trust mine completely. My measurement rig is a piece of leather and some felt tapped to a broken lamp. But it does seem relatively accurate for most measurements.

thefitz's picture

... as evidenced by being so insecure that his mods remove references to SBAF. I mean, that's insane.

Magoo's picture

I think all of us would just like some consistency in Measurement AND Production. These TOTL hp's cost waaaay too much money for these differences to show up.

It's will interesting to discover where the inconsistencies lay....maybe they are in both Production and Measurement???

ultrabike's picture

I don't think there is a consistency problem with these headphones.

Every measurement rig will have discrepancies when compared to another measurement rig. But several headphones of the same make and model should measure very similar in the same rig.

Based on what we are seeing from even different measurement setups, I think it's clear that the headphones Tyll has do represent the product well, and Jude's measurement rig paints a better picture than most every other measurement rig from a variety of different sites.

darkswordsman17's picture

I can't agree simply because we have a distinct lack of hard data to support things either way. I would guess things have improved, especially in the past just few years it seems like there's been a movement towards tangible objective improvements across the industry, and I'd be willing to bet that has helped improve quality consistency. There's likely to be outliers though, and some of these companies treat things more like a boutique production, where variability almost seems to be a desired trait.

I really liked how the HD-800 came with frequency response plot of each individual pair for the owner. I wondered how many pairs were tested and compared to the Sennheiser measurements.

ultrabike's picture

Plenty of HD800s have been tested. The HD800 in particular is a very consistent headphone across samples when measured.

Same could be said about HD6x0, PortaPros, KSC75s, HD202, Beyerdynamic classic line, ...

Measurement setup variation can be dramatic however. Specially when using a body and head model and when lacking experience, which unfortunately I think is Jude's case.

darkswordsman17's picture

How well did they matchup with the Sennheiser provided plots?

Definitely there have been some that are the model of consistency, which is why I don't know that it is a big issue, although in this sorta odd high end boutique niche they've always seemed a bit...off. Some definitely were better (one reason why the HD-800 is so lauded even though plenty of people say it isn't quite to their liking) than others.

AstralStorm's picture

They do match up with regards to peak locations and general tilt.

There is a plenty of high frequency magnitude variability due to pads being "broken in" or not as well.

All of the above is why I personally measure the headphones by ear as a check and recommend everyone to do that. It's a reasonably quick process involving sine sweeps and results often match ones from good measurement rigs almost to a point. With minor wideband differences in compensation. (Less than 3 dB.)

halcyon's picture

and results beyond 8K are infamously prone to small adjustments and changes from HATS, mic positioning, etc.

They might be all right.

The final proof is in the (eating of the) pudding.

spwath's picture

Do note my measurements are of the same pair as the SBAF pair, but different measurement rig

kais's picture

I always thought Innerfidelity's distortion measurements are spoiled by system or external noise.
It's very unlikely that higher levels show lower distortions for almost all headphones measured.
In most measurements the distortion goes down by the same amount that the signal level goes up, specially at those that have quite low distortions
Strong evidence for my assumption.
For my own measurements I use the Bruel&Kjaer artificial ear 4153.
My results are much better then Innerfidelity's for same models of headphones.

Tyll Hertsens's picture
This is true and I've stated it many times. My distortion measurements are noise limited.
kais's picture

This needs to be improved by at least 10 dB.
I suggest to find the source of noise and quiet it. I guess it's something from outside your measurement system, if you look at the Stax 4070's measurement you see it's mostly around 0.2%.
This results from it's good isolation, and the curve kind of follows the isolation.
Something in your house is noisy I guess (air condition, motor or heating pump vibrations or whatever).
Try to listen to the noise, this might help to identify the problem.
When I do my measurements in my recording studio I do get much better results without an isolation box.
Even in my living room (in the night past 12 ) it's better, and I live in the city!
It doesn't make too much sense to publish measurements that are not representing the reality.
The average reader will not understand the problem and take the measurements for real.

ultrabike's picture

The main issue discussed in this article has little to do with noise and distortion, and much more to do with linear frequency response magnitude.

In question are the severe octave averaging, likely unrealistic compensation curve, left/right channel variation, and plot graphic widening of the frequency axis on Jude's plots.

The relatively large left/right channel variation in Jude's plots above 4 to 5 kHz maybe due to inaccuracies in headphone positioning. This is probably due to either inexperience, or effort to paint a better picture by positioning the headphone unnaturally on the head. Or both.

Tyll has explained that to avoid positional issues he uses multiple clearly labeled locations, and averages the results. He is also very careful about the compensation curve and has a full article on it. Obviously Tyll has a lot of experience in headphone characterization. Jude probably not so much.

As far as noise and distortion, past a certain point, distortion start to become academic. Tyll's plots show clearly when a headphone is severely limited by distortion, and when it isn't. On headphones where distortion is indeed a limitation, Tyll's plots do show distortion tracking SPL level. When this is not the case, particularly since Tyll's distortion plots are in %, the results may indeed show the limitations of the measurement rig. But again, this happens when distortion levels are so low, that the issue is academic. The reader should take from this that the headphones in question are not limited by distortion.

kais's picture

Criticism on Tylls distortion measurement is mentioned in the linked article:
https://www.head-fi.org/f/threads/the-official-sony-mdr-z1r-flagship-hea...
What's audible and what's not is a very personal thing. Sometimes people neglect the audibility of 1% distortion, while the same people tell cable A sounds better than cable B.
I by myself cannot distinguish between different cables (as long as they are used correctly), but can hear distortion, noise and other artifacts down to -80 dB in blind A/B testing (this would be 0,01% distortion). I half to admit that I'm not talking about acoustic transducers, but comparing digital audio recordings. Doing this is part of my daily job, so I'm trained with it.
Distortion in headphones may not only consists of "nice" harmonics, but all kinds of noises and buzzes.
As headphones are usually full range systems, intermodulation distortions are quite important.
This means the level of higher frequencies is modulated by bass frequencies, which can be quite annoying if you listen at higher levels.
Intermodulation distortions are always connected to the distortion figures in the low-frequency area.
The Doppler effect is another source of signal deterioration, it's connected to diaphragma excursion.
Higher frequencies are tone modulated by bass frequencies. Even though not directly connected, higher distortions in the base range points to the possibility of those to appear.

francisk's picture

Do take into account of the graph scaling which obviously has a great effect on how the graph looks. It's not a very good idea to compare graphs that's scaled differently.

ultrabike's picture

Were is the like button when you need it.

This is a great observation. Graph scaling is a tool used by some to make things look better or worse.

Journeyman's picture

IMO the best way to show off a new measuring rig is comparing it to standard used by most enthusiasts, in this case Innerfidelity.
Jude needed something to call attention to his new fancy rig and that's about it.
Anyway lets asume I'm biased because I don't really like or want to like Jude and what Head-Fi represents, glad at least Sony is not their sponsor, some companies don't really need all that hype train.
This will end with something a long the lines of "Oh we made a little mistake and we are sorry for all the commotion but look we promise we will use our new rig better next time"...

tony's picture

Jude says he's a fan of the Sony.

I'd say he's a big fan of everything, he only uses superlatives.

Tyll & Co. are the Scientists in the Room.

Has Sony Hi-Fi ever been anything more than Mid-Fi?

Sony's A7 Camera is quite good by Canon 5d Standards, their Pro Video stuff is pretty good ( not quite a RED ).

People buy Sony because they want Sony, they've never heard of Audeze or Focal or probably Stax for god's sake.

I'll bet these Sonys are better than those pretty Beats.

Tony in Michigan

ultrabike's picture

Jude is a Fan of Head-Fi

darkswordsman17's picture

Sony has made some pretty boutique audio gear over the years. Sure a lot of their stuff is fairly generic consumer stuff and I've found it to be somewhat lacking.

But even in headphones the R10s and Qualias have legendary status. Something interesting to note is that there was even talk about bass light/heavy versions of I think the R10s, so it seems that there were production differences. I always found it odd that people seemed ok with that, when to me it seemed to indicate an issue, and on such an expensive item.

ultrabike's picture

I've hear a "legendary" R10. I have a lot of respect for the owner, but it was not just bass lite. It was a complete mess. I was told the driver degrades over time. If that is the case, it is likely that this was not a production difference issue. It was a production longevity issue.

I don't think that's the problem in the MRD-Z1R case discussed here. Those things just came out.

Again, I could be wrong but I don't think this is a Sony production variability problem. I think it's more likely that Jude lacks experience in audio characterization, and may lack some critical listening skills. He is pretty experienced in running audio forums for commercial purposes though.

John Grandberg's picture
I've owned two different sets of "bass heavy" R10s over the years. They were beautifully built and sounded... "interesting"... but I think nostalgia and rarity have elevated them to a status they don't deserve. That said, Sony can make some good stuff - the ZX2 player is killer.
ultrabike's picture

Yup. I think R10's will be all over the place probably due to driver degradation. Here are my impressions when I heard them:

http://www.changstar.com/www.changstar.com/index.php/topic%2c2247.msg615...

I didn't measure them, but it seems Bill-P did:

http://www.changstar.com/www.changstar.com/index.php/topic%2c2251.msg616...

AJ's picture

@Tony in Michigan,

I always enjoy reading your comments.

Wanted to clarify on the Sony comment. I do own stuff from Audeze, Hifiman, Sennheiser, JH Audio, Stax, Shure, Cypher Labs, Alo Audio, Chord and so on and the Z1Rs are my first Sony audio purchase in a long time. Yes they are largely a midfi company but from time to time companies do surprise with innovation (Focal was a surprise recently with their headphones while they have been good with larger drivers). While this thread is about measurements and the discussion is both good and important, I am curious where this topic goes because I am impressed with the Z1Rs. I enjoy listening to music on them and was surprised by how negatively they have been discussed here. This hasn't happened for the longish time I have been following this site. I think Sony has come up with a good product and feel sad about them getting this negative buzz.

AJ

tony's picture

I'm certain that most folks will be delighted with their Sony Gear, just like folks being satisfied with their Toyota cars. ( I've always wished that my GM could build as well as Toyota )

If I were Sony, I'd ignore the high end of headphones and push for a Global acceptance.

Measurement data is a wonderful catalyst for deep rooted argumentation that only more measurement will begin to resolve. I'll trust Tyll & JA when it comes to measurement but I'll buy on sound quality and probably ignore the graphs. ( kinda like Katz )

This topic goes to Insanity unless there is repeatable consistency which seems unlikely because these transducers are working with such low energy levels. ( just a tiny increase in energy will yield 3db difference).

I'll simply apply digital signal processing to accommodate my personal Ear's response characteristics and live happily ever after.

These latest high end headphones are secondary to the much larger Format change that we're about to embrace : "Streaming", we're about to have access to much higher quality sound sources.

Tony in Michigan

HiFiArt's picture

Hiya Tony in Michagan:

What about...

1) Sony MDR-R10 "King" Headphone

2) Sony MDR-7520 Headphone (and other Sony Pro headphones used is broadcast studios worldwide and headphone entusiasts also)

3) Sony SS-R10 Electrostatic Speakers

4) Sony SS-M9ED Loudspeakers

5) Sony TA-NR1 Power Amplifier

6) Sony TA-NR10 Power Amplifier

7) Sony SCD-1 SACD Player

8) Sony DAS-R10 DAC / Matching CD Drive

http://www.thevintageknob.org/sony-SS-R10.html
https://www.cnet.com/news/sony-mdr-r10-the-worlds-best-headphone/

How's everything in Michegan?

tony's picture

I finally got round to looking up all these Sonys on Google images.

This is nice stuff, indeed.

But it's not High-End like the stuff you'd see at LAAS 2017!

True HighEnd stuff sits on the floor, not in Racks.

There are no $50,000 Sony interconnects or $25,000 to $125,000 Sony Reference Speaker Cables that need 300 to 400 hours of proper break-in. ( maybe considerably more )

I didn't see a Sony Record Player, hmm, can't quite be HighEnd without a proper Vinyl playback with $15,000 MC phono pick-up, can it?

To top it all off, Sony has way too much distribution, high-end stuff like darTZeel have only "One" dealer in the entire USA ( for gods sake ), it can't be high end if anyone can buy the stuff. (can it?)

Sony is nice stuff for "Everyman" not quite twitchy enough for true High-end believers.

Tony in Michigan

ps. Although, a person might get "special" dispensation if they happen to buy some of those Sony SSR-R10 Electrostatic Speakers.

Pages

X