In response to the cacophony: Tyll' Stance on Reviewing

Quite the tumult over my Sony MDR-Z1R review. The forums churned. I reckon a little re-cap is in order:

  1. On June 5th, I Published a review of the Z1R and found it to have incredible build quality, styling, and comfort. I also found it intolerably uneven and stated I'd prefer to listen to the ATH-M50X. This is a statement of preference not absolute performance and price is factored into this comparison.
  2. I was heartily rebuffed and, in some cases, reviled and insulted as a result starting about here in the Head-Fi thread.
  3. On June 10th, Jude published measurements contradicting the InnerFidelity measurements showing a significant peak at around 10kHz. Evidencing listening experiences, and frequency response and distortion measurements, Jude concluded:
    Based on what we've heard here (and also on hearing other MDR-Z1R's)--and based on many reviews/impressions that predated the publishing of Tyll's measurements--we'll also assume for the moment that our impressions and measurements are more representative of the MDR-Z1R's that are out in the wild.
  4. This poured fuel on the firestorm of posts in the Head-Fi thread, and spurred a critical SuperBestAudioFriends.org thread. I found the counter-viewpoints of the two threads interesting, entertaining, and enlightening.

  5. On June 16th, I published my response to his findings, in which I show numerous other enthusiast measurements showing a similar 10kHz elevated feature. More gasoline.

While the ensuing mayhem was fun, I walked away concerned: There is a pervasive idea among, what I'll call, the hard-core subjectivists that any peculiar "sound signature" is not flawed if you like it. I have no problem with people liking what they like, but I do have a problem with abandoning a reasonable middle ground from which to evaluate headphones. As I said at SBAF:

Other Enthusiast: The thing is, if we are evaluating the merits of audio gear, as compared to each other, different signatures are fundamental.

Tyll:Every time you say "different sound signatures" I hear "the astonishingly wide array of ways things can go wrong."

Other Enthusiast: Headphones that exist outside of standard models are not necessarily wrong, but admitedly are also probably not going to be well received by the majority of people.

Tyll: "Not well received by the majority of people" sounds like a pretty good definition of "wrong" considering the subjective nature of audio evaluation. It's my job to get it right for "the majority of people." I can't possibly be right for everyone, so I try to give good advice for most.

*snip*

I can tell you in the world of speakers that Sean Olive often sums up the tens of millions of dollars he's spent on researching subjective opinion by saying, "With measurements we can tell with 87% accuracy how people will feel about the sound of a pair of speakers."

These aren't just your average Joe listener either, it's a well rounded sampling of all demographic groups. He's even said there's not much difference when you compare things group to group. With trained listeners, he says his prediction accuracy is much higher.

Now I think Sean tends significantly more to the objective side than I, and I sometimes cringe when he says some things, but he's done the no shit-research to prove the things he says. And what he says is that listeners will basically prefer speakers (in a properly treated room) that: measure flat in an anechoic chamber; are even in their response (no big bumps and wiggles in frequency response); and have well controlled dispersion.

For example, I don't like his headphone target curve in some ways (over emphasizes 600-2.5kHz and 4-8kHz by a few dB) but I do understand why it's drawn the way it is. I'll also note that that target curve is just a preliminary curve they developed to put in the mix of curves to test. That I'm aware of, he hasn't done testing to finesse it.

At any rate, I've got no problem with folks wearing rose colored granny glasses, or high tech, multi-layered, micro-coated, prescription motorcycling glasses (I have some and love them), strokes for folks. The problem arises when the hippy chick in the sun dress looks at you dead serious over her little spectacles and says, "You really need to do some ear candleing now while the moon is in Virgo."

There is a Neutral
Sean's experiments correlating objective measurements and subjective listening satisfaction work for a simple reason: well designed speakers that measure well and do a good job of reproducing audio are what people generally like. As speakers or headphones become neutral and transparent you can better hear the music in its most natural way. The original artist/producer/engineer are driving after something...it often sounds "right." Anytime you add, subtract, or otherwise deviate too far from neutral you run the risk of producing coloration that taints the original musical expression.

I like the word neutral because it doesn't say it's good. But it does say something. Neutral means: Most like real listening. Now we could write a book defining that, but let's just agree it means the reproduced version sounds quite like the cello sounds in front of you. High-fidelity = highly truthful to the original thing. I would generally claim that this is the basis of high-end, reference quality audio equipment, and that some/many aspects of this is measurable.

Let me stipulate, at this point, that there is a good few dB margin for deviation from "perfectly neutral" within which a manufacturer can create an aurally pleasurable experience. Things like second harmonic distortion in singe ended tube amps and simple device circuits a-la Nelson Pass. These things are basically unmeasurable in their gestalt producing effect, as I know it. I'll add that I'm not sure we're able to, or know what to, measure relative to that far more complex interior experience of music listening. So, I reckon there's a good margin for slop around technical measures and still remain pleasurable.

However!

There are measurable performance errors that clearly indicate deviations from the norm that are known to disturb the listening experience for most. These would include: rough, uneven frequency response; significant increases in bass distortion as level increases; significantly noisy impulse response blurring leading edge transients; and inordinate level deviations from known standards (free-field; diffuse field; Harman taget curves).

Now, I have no problem with people liking what they like. I like anchovies on pizza...but I rarely order it if I'm sharing with others. I get it, salty fish on pizza is weird. Similarly, I don't have a problem with people enjoying a "V" shaped response and will even recommend then if tastefully done. (Shure SRH1540) What I do have a problem with is a headphone response that is so far away from neutral that I have no way of enjoying them or figure out how to recommend them to someone with out-of-the-ordinary tastes. If I try to please everyone, I'll please no one as a result.

As a reviewer I feel I have to speak to the norm. It's important that people know where I'm coming from so that I can be used as a benchmark of sorts. People know I tend to like things a tad on the warm side and I try to remain consistant so folks can interpret what I'm saying and put it into their own context. (I'll note here as we evolve toward something like a Harman Target Curve with a bit extra bass and a slight warm tilt I find myself fitting in quite nicely to that norm.) Also, I try not to over-evaluate a headphone to noodle out small, subtle details because I know that people may hear those small things differently than I given variations in taste and physiology. I try to keep it simple, relevant, and, most importantly, straight down the middle.

When I find myself listening to something like the Sony MDR-Z1R with significant deviations from neutral in excessive upper-bass/low-mid emphasis and lack of tight time domain control with large resonances at 3kHz and 10kHz, I'm going to call it flawed regardless of whether or not it was Sony's intention to tune the headphone in that way. It's simply not going to please most listeners in the long run.

So, I promise:

  • I'll keep reviewing headphones the same way I always have. The bar may move sometimes as headphones get better, but my tastes and evaluative criteria will remain consistant.
  • I'll keep measuring headphones the same way so that InnerFidelity's database of measurements will remain self-referentially consistant.
  • The more expensive a headphone is, the more critical I will be. The same modest departure from neutral may be acceptable on a $200 headphones but unacceptable on a $2000 headphone.

Thanks for following along. I'll keep working at it.

COMMENTS
GimmeCans's picture

It isn't 'good' just because it costs a lot or because some advertiser's ox is being gored if you say it isn't (not the first time such a thing has happened on head-fi). For what these cans cost, it shouldn't have flaws that other headphones at a much lower cost avoid. This is Sony's attempt to tap into the 'money-to-burn' market, and if the result is a headphone that doesn't 'walk the walk', too bad for them. Right or wrong, you can at least count on integrity from inner-fi. Stick to your guns.

Visigoth's picture

Well f*cking said!

skris88's picture

Yes. Well said Tyll. Stick to neutral. Natural.

Let those who like sharp, bassy or forward sounds go elsewhere for their learning.

pbarach's picture

I just want to know what you were mumbling to yourself while your head was in your hands...

inventionlws's picture

and keep it up!

GNagus's picture

You would have really made us laugh if they were Grado or Ultrasone headphones

jim in cheyenne's picture

For me it is incredibly important to have a reviewer that evaluates relative to 'neutral'. Otherwise one has no firm base. Along with that you provide measurements. Yes those vary a bit depending on equipment, but as you note, consistency is the issue here. With consistency in the measurements, one gets a good idea what to expect. One element you can't control is variations in the phones from the manufacturer, but you do a good job of reviewing other reviews to see if that is the source.

Keep up the good work!

100VoltTube's picture

They have an audeze carbon fiber headband, and the cups appear to be black, so I guess they're the new LCD-pro. Unless he has a pair of LCD-X's with a carbon fiber headband, or a pair of very dark LCD-4's.

windcar's picture

Good job, keep it up. Ignore the head-fi shrills. Over the years, I found that site loves to hype up plenty of stuff depending on which is FOTM. There are so many noises over there it's hard to get accurate information.

sdecker's picture

Seems most of this Sony tiff could be mitigated if we knew about sample-to-sample variation. Otherwise every reviewer could be hearing varying sonics from their pair and their reporting on what they're hearing is largely accurate. This is especially crucial when a high-price low-volume headphone first comes on the market and the QC consistency just isn't there (yet). They've made a zillion HD600s over 20 years and they all sound the same. Much harder to manufacture planars with much smaller fabrication budgets and much higher price tags deserve special scrutiny in this regard, especially on early production units, even as there are fewer on the market to evaluate.

BlackWolf's picture

At this price point, I would personally view it as Quality Control Problem vs sample-to-sample variation. Variation sounds too passive for this type of money.

My problem is that only a few professionals have a meaningful way measure a frequency curb. What concerns me is that there must be many out there who shell out more than 2k for faulty units and they do not know at all.

At this price point, Sony should have put every units in a through testing before selling to a customer. How much would it have costed Sony to test every MDR Z1r? $20-30 at max? Sony decided to put that extra few bucks in their pocket rather being faithful to their customers who shell out more than 2k... Unacceptable in my opinion.

sdecker's picture

You're right, I was too generous in suggesting sample-to-sample variation when at these prices (and a lot less expensive models too) they better have a solid final product test suite established and reviewed before each phone is packaged. Set a target max deviation for a number of parameters and reject the outliers. Micro mechanical machining is imprecise with a small run of expensive 'audiophile' headphones barely out of development with incomplete manufacturing methods still being optimized, if they ever are or effectively can be (looking at you Audeze).

Ironic that I'm a career electro-mechanical test/quality engineer and I didn't call out a FAIL for clearly half-assed final test results!

wiinippongamer's picture

Audio reproduction is a science. No more no less.

romaz's picture

There will always only be one Tyll. Don't change a thing. Much respect.

headwhacker's picture

I appreciate what innerfidelity is doing with the combination of objective measurements and subjective personal preference evaluating headphones. I may not find myself agreeing with Tyll's subjective preferences all the time but it helps me to find the headphones I like. Like you said just be consistent so that you become a reference point for everyone evaluating headphones according to their personal preferences.

Thanks to your collection of measurements, I was able to find my gems in GR07, Blue Lola and even the tascam TH-02 (TEAC CTH-02) without having to audition them prior to a purchase. Price indeed has no correlation with the headphones SQ.

sunnydaler's picture

a forum full of pro-market idiots.

RPGWiZaRD's picture

The issue with this community boils down to the many various factors:

- Different reviewers have different methods; some take a more objective standpoint, others more subjective approach. We as humans typically are bad at accepting different ways of approaches and opinions as we like to believe "I am the right one", it's one of those bad natures of humanity that lead to wars, especially with people with strong subjective opinions.

- The consistency of manufacturing of headphones is often not given enough analysis, I'm glad Tyll lately have brought this issue forward quite clearly. We may simply be arguing over two headphones that happens to measure very differently so the persons hears a quite different sound and then throwing sticks at each other.

- The human ear and the ear canal is invidual for every person, in speakers, it's more about the room threatment as well as the listeners hearing capability and interpretability but with headphones, the percieved sound from a headphone will always be slightly different so even if it measures perfect in a set of standard "doll head", it's not exactly the same for every person.

- Money & sponsorship. Different sources may be more or less exposed to that. It's often hard for the reader to be able to judge where the reviewer stands as there's often lack of transparency on this part which quickly leads to assumptions. With better transparency this could be improved.

- Psychoacoustics, ie. the science studying the psychological and physiological responses associated with sound. Especially the more subjective oriented the person is, the more important this topic becomes since subjectively oriented reviewers tend to more often go by "what I enjoy, must be good"-sentiments. But when you get into psychoacoustics, you'll quickly discover based on several researches that our brain simply release the "feel good" dopamine based on sounds we enjoy and what triggers this "feel good" response is highly invidual and therefore I personally think it's very important to at least to some extent, be able to separate from personal tastes as well as objective analysis.

- The complex task of translating a personal experience and findings into a well interpretable context for the masses which comes from vary different backgrounds. Being able to write in a way that gives an accurate presentation of one's findings isn't always the easiest task. We often get caught up in our personal experience too deeply and might forget to present several factors which may or may not affect the end result.

What I like about Tyll's approach is his consistency and objective standpoint but still sees value in the subjective analysis. For me personally I find his approach works best for using as guidance to find the headphone which matches your personal tastes. Everyone has to find their preferred sound themselves as I wrote earlier we all have different tastes (due psychology, how our brain associates with certain sounds) as well as different structured ears. It's impossible for anyone else to tell what headphone is right for you, you'll have to do your own research! But once you got a good grasp what sound you are looking for, that's when Tyll's efforts and approach to his way of reviewing becomes extremely useful.

castleofargh's picture

is known to be very effective in relaxing your eyes.

there seem to be a fairly universal idea of neutral speaker nowadays a great deal thanks to the Harman crew. listening levels would probably shake that a little but in a very predictable way following the equal loudness contour of the listener. so it's all good.
meaning that despite physiological variations, our brain finds a way(no idea how but it's amazing)to calibrate the response well enough for most humans to apparently end up hearing about the same thing.

but that logic is exactly why I don't agree with a universal neutral response on headphones. if the brain compensates for the body, then when the sounds come from a different angle and don't get affected by the body and head, what happens? IMO we still have the brain compensation messing up the response and based on our very own body.
and based on HRTF measurements, some of those variations can go well beyond 2 or 3dB. to me they're significant enough to say that 1 neutral doesn't work.

maybe after a long time when we spend a lot of hours with a headphone, the clever brain starts working its magic on that headphone. after all I can walk, I can use roller blades, a bicycle, a car, windsurf... and end up with references and god reflexes for all those different systems. the brains knows to switch when moving from one to the other. so why not the same brain plasticity for headphones after some time? but then IMO it could be an even bigger problem as right now, we all get used to BS responses from headphone responses all over the place.
what if the calibration on those responses is the reason why the next headphone sounds good if rather close in some aspects, or like crap if not? it would mean that people using a similar headphone could tend to like the same new ones more than people who have been used to a different signature.
if that was real, because for all I know it's just my wild hypothesis, then it's a chicken and the egg problem. to help get a universal neutral we might need to first set a universal neutral.

of course getting responses based on our own HRTF data would bypass all that crap and make so much more sense. the answer is something like the Smyth realiser IMO.

as for keeping your calibrations and way of doing things, I'm 133.7% with you. the value of your work comes from a common reference. bith in measurements and in subjective opinions. if tomorrow you started saying that your preference goes with stuff Joker likes most, I would be very confused. he likes colder stuff than I do. you like warmer stuff than I do. I know it and can work things out on my own like a big boy.

castleofargh's picture

wall of words!!!!
even the white walkers can't possibly get past this.

timmyw's picture

Keep it up mate. Your work has helped me a lot over the years.

thefitz's picture

10kHz spike or not - that 300Hz step response was a joke. If a headphone measures with that 300Hz step response, it's going to make your ears bleed. End of story.

JimL's picture

I'd give you three thumbs up but then I'd be a monster.

Three Toes of Fury's picture

THIS is why Innerfidelity is, bar none, my go-to favorite site for headphone and headphone related reviews, discussions, and information.

Thanks Tyll for the brain dump on the subject and your promises to keep doing what you do so well...much appreciated.

Thanks Talkbackers for the great discussion and laughs. There's lots of both today. By and large, y'all have good, open, sometimes critical/conflicting, but most often professional and productive discussions about a subject near and dear to ya.

Peace .n. "Without music, life would be a mistake" -nietzsche

3ToF

skyblazer's picture

First, this is my first comment on this website which I've been reading for a few years now, great website.

Now for the heart of the matter, I feel like I've seen this being talked about a lot of times. Almost ever since I've been reading about headphones (~2011 I'd say, where I've bought my first higher-end headphones), I always see this debate between objectivity (which in itself is already a massive debate as to what is actually neutral for a pair of headphones) and subjectivity ("what's the matter as long as it sounds good to me ?").

And honestly, I think I'm more confused now than ever, since I've started to read reviews of headphones I own. I have read pretty much anything and its opposite depending on when I read it and where. I fear that there is a lot of "bandwagonning" when it comes to appreciating how something feels, rather than actual correct subjective impression. It's probably especially bad now that headphones which used to be top-end (Sennheiser HD650/AKG K701/Beyerdynamic DT880) are now considered middle-of-the-road when it comes to headphones, with so much more higher-end options, and I fear that people really want to justify the added cost of their higher-end headphones without using the most probable law of diminishing returns which probably strikes especially hard past that price point of 200/250$.

stalepie's picture

"I can't possibly be right for everyone, so I try to give good advice for most."

The problem there is that the Z1R is not going to be sold to "most." It's not even available in many stores. It is a specialty item appealing to someone who wants a fancy sound, who is well off and probably already bored with some other established choices (Senn, Audeze, Grado, etc). It's comparing a Jaguar to a Civic or Camry. If someone wants Neutral from Sony without paying a grand or more they buy their pro line (7506, 7520)? They know they're not getting neutral. They're deliberately spending more to avoid accuracy, to get colorization. A flavor, a style, some distinction. To hear their old recordings again in a slightly different way. That last issue is key because people (now with today's perfect modern equipment) are hearing their audio identically over and over again, so it helps to buy something different to hear it again new, a little fresh, with some surprises and so forth.

wiinippongamer's picture

If you wanted "surprises" in your old recordings, in the form of coloration, wouldn't it be easier and cheaper to just use an equalizer? That's a laughable excuse to justify a $2000 purchase.

Besides, if you need coloration to enjoy recordings you've already listened to, you're simply not listening to the right music.

Martin.'s picture

You beat me to this. If you have all that money, just fix it with an equalizer or get something cheap. Buy 10 different headphones for 200 each and then you'll get all the coloration you need. No, not everyone's opinion means as much.

Tyll, the reason I come to innerfidelity is because of this. Good to know you'll keep going as you have.

skris88's picture

Agreed! Why switch 'phones when all you need is a tweak of the EQ??

On my iOS devices I use EQu app, great equaliser. On Android "Music Volume EQ" is a simple, non-CPU hog.

The key is, I use my HD-600s to "calibrate" my ears. Yes, my ears. At Flat, zero EQ. Then, when I switch to any one of a half dozen or so more portable/convenient 'phones I try and EQ it to match the HD-600s. That way I can travel with a cheap, throw away pair, yet not have to put up with shrill, thin, boomy or dull sound as the case my be.

halcyon's picture

.... I'd add that after having conversed with S. Olive more than 10 years ago and watched his work, I'd take anything he says over all the psychoacoustic text books on my shelf.

As for measurements... tricky.

Not saying there isn't the 10kHz peak on the cans.

It's just that beyond c. 8Khz things get easily really wonky in measuring. So some variance is to be expected.

But after seeing so many different measurements - there is clearly a pronounced peak at 10kHz.

Thanks for all the great work Tyll. And keeping it open.

darkswordsman17's picture

I'll echo the sentiments, please don't change Tyll, unless you genuinely think it is for the better, but most importantly keep being open about it.

A couple of thoughts I had about the Sonys. What has Sony said about all of this? Was this very deliberate on their part? Did they voice it with other particular equipment (or music?) in mind, that may have changed the perceived sound? I know they were showing off a high end DAC/amp at around the same time, and I've heard varying things about their portables, that makes me wonder if there was some weird "voicing" going on that caused them to tweak the sound a certain way. I think this has happened before (ATH-W5000 and HA-5000 amp by Audio-Technica) where the headphone gets fairly polarizing (but fairly consistent) opinions but most seemed to find it sounded its best when paired with the correlating amp (which allegedly helped to minimize a lot of the issues people had with it otherwise). I think that seems to be more extreme in how polarizing the sound is, and I know you've had similar issues with it, liking the looks and materials, and I think even aspects of the sound but finding the sound too flawed.

Not that I'm saying that's a good defense, but perhaps something more to study that can explain things? But the headphone community can be weird, as I saw people saying the HD-600/650 are picky about source and amp as well (some claiming they didn't like them until used on a fairly specific source/amp setup for instance) when I think that is grossly exaggerated, and some people were going to great lengths to try to rationalize that they only sometimes enjoyed what is one of the objectively best headphones ever made. The HD-800 had similar claims although people seemed more open about it trying to deal with the ~8K (think that's right?) peak. I actually think the Sony's FR kinda resemble the HD-800, just with a boosted low-end and a 10K vs 8K issue. I'd guess that was deliberate (since plenty have wanted more low end from the HD-800, but perhaps they achieved it with some manner that gives up too much quality for quantity).

Oh, I just noticed, you didn't delve into any equalizing on the Sony, would be curious how much tweaking it would take for you to find it enjoyable. I wonder if that might be fueling some of the backlash as well, since the Audeze, Focals, and HD-800S all had some bit about that (so people might be feeling like you were being impartial by not doing so on the Sony?). Maybe some modding could help as well. (Not that that is something that should be necessary on such a high end headphone, but wouldn't be the first, just kinda curious if there might be some things that might be possible for Sony to tweak).

One last bit, would love if you could get some of the older Sony's in for measuring and a listen. Looking at the MDRV-CD3000, I can't help but think that if not for the bass rolloff and that aggressive initial response, that you'd find that to be a pretty good FR. Looking at some of the other Sonys you've measured, I'm noticing some similarities, and also with many of the Audio-Technicas (guess it'd be interesting for you to get a lineup of theirs, well the special edition woodies, too), where they seem to do pretty decent between about 100Hz and 2KHz, but pretty steep bass/sub-bass rolloff and get downright bizarre above 2K. Them both being Japanese brands, really makes me wonder, especially since I believe they use somewhat different drivers (so it wouldn't seem to be that they just are sourcing drivers from the same company, like how several were sourcing from Fostex). I feel like the...Bavarians (? - AKG, Beyerdynamic, Sennheiser) also seem to kinda exhibit a similar "wavelength" in some ways.

tinyaudio's picture

AKG is Austrian, thank you very much :D (although only until the end of this week when their main headquarters will be shut down and the remaining high-end production will move to Hungary)

Also: Sony and Audio Technica build and design their own drivers. Sony is one of the three big companies that produce Balanced Armature drivers (the others being Knowles and Sonion), they also have a great deal of experience with dynamic drivers.
Audio Technica are an absolute pro company, they build professional microphones and have a great deal of experience designing diaphragms and the like, they use that knowledge to design headphone drivers as well.

skris88's picture

There are some things EQ can't fix. Loose bass being a clear example. If it's "wobbly", it'll stay that way.

I had that problem with one well known Classic pair of highly-rated headphones. Nice, neutral frequency response too.

So I switched to another pair that needs some EQ to sound neutral. But well worth it, for that super tight bass!

:-)

Jazz Casual's picture

Looks like you've struck the right balance as a reviewer. Keep up the great work Tyll.

punit's picture

Keep up the good work.

tony's picture

is the "High Integrity" site,

far and above everything else.

Without Tyll, headphone journalism would deflate like a Garage Sale Balloon.

Tyll is the Tyll Group of: Joker, Grandberg, Katz and a few others.

Five Stars plus

Tony in Michigan

Jim Tavegia's picture

Do not fret as most of us trust your opinions to be truthful and as accurate as one can be. Your experience on all things headphones is way beyond what any of us have.

Since all of our own hearing response is all over the place how can one set of can suit everyone? Yet, this is what everyone expects? Not me. I have listened to many and keep in my studio what sounds right to me, but when mixing for clients I do what THEY want, not what sounds right to me. I may make suggestions, but they finally decide.

As to a comment about Sony's 7506's...I find them to be a great vocal tracking set of headphones as the vocals are a little more up front and can give the singer and better window to their performance, but I would never master anything on them, but for all of $99 I have two pair for studio use with singers. Might I like the 7510's?

I finally did sell my Focal Spirit Pros which just did not work for me, but that is MY problem. I will take my two pair of AKG K271's (closed) and my pair of K701's (open), and even my Sennheiser HD-380s over them any day of the week. There are just too many cans and no way for any of us to audition them all. That is the real problem.

We are on your side.

iMatt's picture

Keep doing what you're doing. Measurements are fine, they may vary between reviewers depending on method of testing. Whoopdie-do. I look forward to your future content.

-Matt

Peragulator's picture

Facepalm with Audeze (LCD-X's?) will cause a slight variation in the overall mid/upper response.

The Federalist's picture

Headphone measurements are interesting to me, but I've never really been able to look at a graph and correlate it directly to how a headphone sounds... I'm sure to some professionally trained and educated eyes, the graph's tell a certain tale. But ultimately I trust Tyll's impressions because they've stood the test of time.

This hobby is proliferated by armchair scientists without formal education pontificating on extraordinarily complex theories of physics like electro-magnetism, and acoustics. Often with the unremarkable findings that we need to spend more money because THIS NEW THING... This Is It!

Every few months a new evolutionary step takes place in audio.... if you believe the hyperbole of the audio press and the audio forums based on the supposed exponential improvements they are constantly awarding to the latest hot product release. We should all be in a perpetual state of sonic Nirvana. That is of course, if we can afford the $2500 to $5000 entry fee needed every six months to stay current.

It doesn't surprise me that a small army of hobbyists is storming the gates to burn Tyll down... I mean it is Sony after all. There are a lot of amateur people with a lot of professional ambition in the audio "hobby" and playing Captain Save-A-Ho with Sony makes perfect sense in those terms.

However I trust Tyll's ears regardless of how high someone else measured the peak at 10K....

I think the write up on the Z1R is proof positive that Tyll rolls with absolutely impeccable character... Could've just as easily not written anything, would've been much easier to punt. But he was compelled to inform the buying public because his methods told him the Z1R falls short.

Good on Ya!

Wmcmanus's picture

I missed Tyll's original review and everything else that followed. This article is the ONLY thing I've ever read about the Sony (whatever whatever) Z1R. Now I know that I'm not interested in them.

Thanks, Tyll; keep doing it the way you do it. As Howard Cosell used to say, "I just tell it like it is."

OMGLadyGaga's picture

I've been reading this website since 2011 when I was on the hunt for a great headphone, not wanting to settle for the bose and beats that littered retail stores at the time because they were just too expensive and because I knew nothing about headphones. As fate would have it, Tyll reviewed exactly what I was looking for a closed, all purpose headphone under $100: The Sony ZX700.

I bought the headphone immediately, and after a couple listens I was extremely disappointed: Where's the bass? These look goofy on me, beats look prettier and probably have the bass I desired. Bose are so comfy too. WTF Tyll! I then discovered Head Fi around the same time, and got all kinds of suggestions for what I wanted. I got recommended Ultrasone since they were the best hip hop cans with omg bass, I got recommended Grados because it makes the music come alive, I got recommended in ears because better sound quality for the price. Being as naive as I was, I listened to their suggestions. Big Mistake, after a few months I knew what flavor of the month finally meant.

Fast forward now 6 years later, after trying out so many headphones from the Koss KSC75 all the way to the Sennheiser HD800 and learning more about tonal balance, dry mids, imaging, warmth, resonances, etc. What do I see on the Head Fi forums? The very same people that recommended me Ultrasones now laugh at those cans and call them garbage, hmmmm weird Tyll has said that since before it was cool. The people whose advice I took wholeheartedly, now like entirely different sounding headphones and call the previous ones they liked and recommended, awful and rubbish. I'm looking at you mcadamiahero.

After giving the ZX700 to my brother, I went to his place the other day and there they were! Still in pretty damn good condition despite some pretty obvious abuse over the years, still his daily driver headphone. I put them on and guess what? They are damn good. They were back then too, I just didn't realize it. Warm, with excellent mids, and subdued treble. Exactly how Tyll likes his sound. His tastes never change. His equipment never changes and measurements stay consistent after all these years. Head Fi on the other hand, flavor of the month mentality with every new product being the best ever psshh and fancy new equipment that will probably change to fit some weird new compensation curve. Tyll likes the Harman curve but guess what he's not switching to it for his curve because it will keep all headphones consistent and because he cares about giving the best advice possible and letting us make a decison, Jude and company will recommend a treble cannon one week and a dark smoke tunnel the next.

It's gonna be hilarious when Sony comes out with the Z1R MK2 and announce that they have smoothed the treble of the first one and tightened up the bass.

RPGWiZaRD's picture

I couldn't help but smile a bit for myself reading this, I know where this come from as I've walked the same path. :) It's so spot on.

The thing is about Tyll, he just explains how it sounds like (as in comparison to other headphones). I couldn't care less about what a reviewer likes, I'm wondering what kind of sound the headphones have so I can judge whether or not it's a worthy candidate for me. Both Tyll's measurement as well as subjective analysis helps me with this.

mr.m's picture

Many of the commenters don't seem to fully understand Tyll's and other commenters point about the strong resonances at 3.5KHz and 10KHz. The waterfall plots clearly show the Z1R rings like a bell for quite awhile at both frequencies and both resonances are almost certainly the result of driver breakup modes; which means the two response peaks aren't due to sample to sample variation (they are inherent to a driver design using a large rigid diaphragm). Considering the waterfall plots obtained by the other testers, I find the Head-Fi frequency response measurements highly suspect, to say the least.

I find strong higher frequency resonances very irritating, while some people don't. That's why I really like checking the frequency response versus time waterfall plots, before I even bother auditioning a particular loudspeaker. Some of the first metal dome tweeters had their first break-up mode as low as 10KHz and consequently many critical listeners were so annoyed by this they wrote off all metal dome tweeters, at that time. Much time and effort was subsequently devoted to raising the frequency of the breakup modes of rigid dome tweeters (including the use of beryllium and diamond diaphragms), to eliminate this criticism. I'd be surprised if these same design considerations don't also apply to headphones.

RPGWiZaRD's picture

Yea, based on multiple cases, I've come to the conclusion my own ears are particularly sensitive to ringing around 3.5kHz and 10kHz area. I've heard multiple lower priced Sonys that have this particular issue so I'm not totally suprised if Z1R also has some issues here. Like you I also tend to have a lot of use from waterfall and frequency response plots as they help me avoid those problems that I know will be an issue for me.

Sean_S's picture

The amount of useful information at Head-Fi makes up about .01% of the site. For InnerFidelity it is more like 25%.

Johan B's picture

Good sound is not a function of the application of expensive alcantara ear cushions. The WoF has helped me to appreciate the differences between (4) good headphones. The WoF also made me a better listener. (I however conclude that no headphone will be better than my KEF speakers with uniQ drivers). I therefore think that spending >$500 on a headphone is a waste of cash. And to top that ... my favourite headphones are $65 Sony in ear phones. I admire Tyll's superb work making a difference through dead-honest measurement and comparisons. I am happy that his work supports my opinion. None the less I do feel that it is important to keep testing top bracket headphones to keep improving the market quality.

Lawk's picture

From my experience with InnerFidelity reviews is that I mostly agree. Actually I often consider cans that got a moderate recommendation to be even better than the review would have indicated (at least for me).

Naturally I never bought the ones that got bad reviews (there are only a few anyway) but the QC35 for example got some criticism for the treble. And I ended up thinking it is a honky harsh trash can for 300€ made entirely of plastic.

I mean best judgement will always be your own audition due to hearing sensitivity differences and taste, but for the most part if innerfidelity concludes a 2000€ can sucks, guess what, it probably does!

But then again I never go over 400 bucks budget anyway.

Also consider that people spening that amount on headphones are probably overly protective of their investment.

Personally I think starting with the MDR1R a few years back, a lot of the consumer products are a little too flavored in the upper midrange low treble for my taste.

jerseyd's picture

It is not an unreasonable stance to take in saying that a very expensive headphone should not be as "flavored" away from neutral as an inexpensive headphone might be. If, however, it was Sony's intent to craft a very expensive headphone that was colored in this way, that would make the conversation much more interesting!

Johan B's picture

But that will be similar to Ferrari starting to build F1 cars with truck tires.

LytleSound's picture

The way that you measure headphone responses is perfectly fine and I think that it is important to continue to take those measurements. If anything, keep what you're doing add something new if you find a need to do so. As to Sean Olive's headphone response, it hasn't had the amount of testing as he applied to loudspeakers. But, it is reasonable reference; if a headphone's response differed from it by 6 dB or so, you'd probably judge it as flawed when you listened to it.

As to the smoothness of the response, in general listener will prefer an earphone or IEM with the smoothest response. You might want to consider calculating an index of irregularity. What is involved is calculating a least-squares fit curve for the response of the earphone. This will be a smoothed curve, but it doesn't distort as a moving linear average will. I have found that a 7-point LSF curve is fine. Then simply count the number of peaks and dips in the raw earphone response that deviate from the LSF by 3 dB. You'll find that the higher number,the less satisfied the listening experience will be. This is a variation of a method that the Veterans Administration once used for selecting what hearing aids to take on contract. It was dropped because in the days of analogue measurements it was too labor intensive. With a digital measurement system it is easier. I suggest using the center frequencies of 1/12 octave bands.

wink's picture

Well aid..... well done...!!!!!

wink's picture

Well said..... well done.

(this time without the spelling faux pas)

geargeeksp's picture

And had similar impression~of them to urs

I don't always agree with ur reviews, but god knows if u do enough of them , nobody can like everyone of them~ but I do respect the dedication, standing tall for whatever we believe is right and in the same time~ holding faithful reproduction ahead of brand price tag~

I wish we can have more of guys like u in China, not to judge book by its cover nor it price tag~

cheers

Lee

Dadracer's picture

I like your blog and your reviews. I certainly don't agree with some of your conclusions but I certainly do appreciate your hard work and the professional way in which you review and present your results. Keep doing what you do as it is good (even when we don't agree) and let those who cannot see this a hobby which has no place for abusive comments go their own way back to where they came from.

bogdanb's picture

Sound signature = someone's inability to stay on target
it is something like this searching on google on images Monalisa. Guess which is the right representation.
If there should be more base, mids or highs, it the artist's choice not the listener!

Equalizer -Freestalker.fr-'s picture

Sorry but in my own experience with measurement graph, I can say nothing can be sure.
He used a GRAS KEMAR HATS and you used another one, there is also B&K HATS, and every time I compare measurement in other website, they are NEVER the same! It's really strange. Compare Innerfidelity, GoldenEars (English and Korea), Rtings, RAA, DigitalVersus, KopfHoerer.de, etc; you will see they are not the same, for instance one has more bass and another has less bass. The only thing that can allow to hear the true frequency response based on your own HRTF is to use pure tone.

Another example is when I was searching for the Superlux HD681 freq resp I used your website as reference and I searched another Freq Resp site and I was so shocked there is so much variations between measurement that after that I didn't trust any measurements. I had to torment my hears with pure tone and even with that, the result isn't always concluding but the results are close to my own HRTF flatness. Now I can't trust measurements where everything can interfere with accuracy, but I have to be honest, I can't trust your Superlux measurements because of rifts after 10KHz that I haven't detect with pure tone. Sorry if I've hurt you Tyll.

Equalizer -Freestalker.fr-'s picture

I am not a subjectiviste too. In audio everything have to be carefully analysed.

I was badly surprised with Astrotec AM 90 too. Your meas shows a soft Diffuse Field, I searched on Head Fi and someone said that it has weird sound, I was saying, he isn't pleased with flat Freq resp. I bought it and... +6 or 7 dB at 2500 2700 Hz (pure tone certified) the sound is really aggressive, wrong manipulation, I dunno, but it has come to do measurements. I used the mic of the AM90 and turn the in ear to the mic and the trebles at 2500 2700 Hz were 7 dB too high compared to your meas. After I pasted the in ear to the mic and I obtained your measurements... and for the deep 6 kHz, they are higher without pasting and like yours with pasting, so it's really hard to have accurate measurements.

To come again with Superlux Headphones, Superlux's measurements are also not so accurate in particular in the trebles.

Bob Katz's picture

I just got to read your "state of the state". Looks like you and I are largely in agreement. I'm also feeling that Sean's curve is a bit too aggressive in the high presence ranges. To refine it, however, you'd need to know the curve of the reference loudspeakers the listeners are comparing the headphones against. AND I would use professional, trained listeners, not average consumers, at least for the initial go round that would refine the curve.

X