To Review, or Not to Review?

This is a question I find myself pondering weekly, if not daily. I thought it would be a good idea to lay out my thought process on the subject for InnerFidelity readers and open it for discussion. Sometimes it's a tough decision and I think InnerFidelity readers should have some input. I'm willing to listen.

I find headphones fall into one of four categories in the decision of whether or not to review them:

  1. A headphone is really good and people should know about it.
  2. A headphone is a significantly poor value, but it's not a remarkably visible product for consumers or enthusiasts.
  3. A headphones is a significantly poor value, and has a very high profile in the market and is likely to sell well.
  4. A headphone is good, but not great. Or might have very interesting features, but fails to deliver across the board so remains a mediocre value. Or maybe it's just outside my taste by being a bit too bright for me and anything I would honestly write would appear overly negative to readers.

Item one is a no-brainer, this is what I'm here for: To recommend good headphones. In large measure I see my job as filling up and maintaining the "Wall of Fame". If there's any one thing I should do well, it's create a list of headphones that sound good, feel good, and look good. I think we can all agree on that...with the understanding that it's just the InnerFidelity team doing its best and that the "Wall of Fame" is not necessarily gospel. Also, there may be good headphones out there that don't quite beat out the WoF cans, but remain a solid value. Those get the "Stuff We Like" badge.

141217_Blog_ToReviewOrNot_Photo_UrbanearsPlattanItem two is also pretty clear-cut. Take, for example, the Urbanears Plattan shown at right. This is a $69 headphone that's fairly widely distributed. I doubt any headphone enthusiast expects it to sound good...and it doesn't. It's a pure fashion headphone. People buy them for the wide variety of pretty colors. I don't see a need to review them at all. I've measured them here, had that measurement been done six months later I would have made a short paragraph statement about them in one of the "InnerFidelity Updates", and never bothered with them again. So, it's important to point out that all headphones I receive are measured, and if not reviewed, I mention them and my basic thoughts in the updates.

Item three is a bit tougher call. Sometimes I receive a headphone that has a fairly high profile in the market or a significant buzz in the enthusiast world. If I find I dont like it, sometimes I feel compelled to warn people off due to a widespread, but unwarranted in my opinion, appeal. The most glaring example of that my review of the original Monster Beats Solo headphone. What a piece of junk...that was marketed HUGE to the public...I needed to say something. I don't have too much trouble making those calls.

Item four is the tough one. Right here at home, right now, I have three pairs of headphones in this category. People aren't getting "ripped-off" if they buy them; they all have attractive features. They are the: Master and Dynamic MH40; Sony MDR-Z7; and B&W P5 Series 2. All three of these headphones are attractive to me in various ways, and all three have problems for me. Enough of a problem that I can't recommend them without significant caveats. (Well, still haven't quite made my mind up on the Sony.)

Since a lot of this boils down to personal taste, I'm very concerned that a negative review doesn't do justice to people whose tastes are different than mine and might like the product. It seems to me that sometimes it's best for me to keep my mouth shut and let the churn on the forums figure it out. You do get a wider range of opinion there—I am, after all, only one guy. And, I'll reiterate, if I choose not to review something, I always measure it and make a comment in the updates on the problems I found.

Lastly, if I do review cans at this "okay, but not really good" level, I'll have less time for "Headphone 101" posts. I think those posts are important information for enthusiasts entering the hobby. Time, for me, is a significant limitation.

There you have it. Is the current mix with reviews for superior gear, notes in "InnerFidelity Updates" for the also-rans, and a continued focus on solid basic informations on headphones and related technologies reasonably well balanced? Or, is there a better trade-off to be made? I'm interested in your view, please comment if you've got some thoughts you'd like to share.

COMMENTS
Hifihedgehog's picture

The HiFiMAN planars and Grado SR, RS, GS and PS line have never gotten actual reviews, which is quite surprising when they are moderately to highly popular in the headphone community. Some people say it is because of your warm-leaning preferences in sound signature, which opinion I respect but I beg to differ on, and I and many others think these headphones should get at least a short or combined review of sorts. Even if your opinion is they are mediocre, professional reviewers (like your colleague, Steve Guttenburg) and prominent hobbyists like or even love them so at least a review could at least allow us to see into your mind and help us to understand why they are not up to snuff for most people.

jjgr's picture

It would be interesting to read why the cans fall in category #4 for you - maybe just a bullet list of pros / cons / interesting features / deal breakers. Would rather read more about what's actually more interesting from your perspective (#1, #3 and other more topical/ educational pieces).
Enjoy the site!

DiRo's picture

If this is truly an open invitation for ideas and constructive criticisms, I wouldn’t mind chiming in as this is one of my go to sites.

As you stated review of good headphones is important, but just as important is headphones to avoid (2&3) and why to avoid. This also gives incentive for the manufactures out there to not make crap products. Not sure how your site gets its revenue, doing what I stated above could be an issue? Time wise if this is your full time job, I do not see how it is a problem reviewing (1) and (4). As for “headphone 101” interesting as it is, there is much more detailed information on the net if someone wants to research technical information. For example no point in allocating time to restate available information such as Wikipedia (not always right source but there are others). If indeed there is not enough time in the day, or this is just a part time thing. I do not see the point in concentrating review efforts into unobtainable products, I say that because I do not see more than a handful of people buying headphones over 1g and audio chain setup costing upwards of 5g(makes up majority of your wall of fame). Focusing reviews on more mainstream products would also generate more site traffic. It is easy for someone like yourself or anyone that is deep into this hobby to become desensitized to the $$$ money and cost of things. Going back to not enough time, for number (4) headphone types, perhaps cherry pick ones that there is not a lot of reviews on. Would generate site traffic as you have the monopoly on being one of few if not the only review out there. For other (4) headphones you can do reviews in point form, just so readers know you covered it and nothing major of note.

... with all do respect.

mkozlows's picture

I very much want to see negative/not-to-your-taste reviews. Because after reading your opinions for years now, I have a good idea of what you like, and how your tastes match up to mine, and if you say that a headphone is too bright/harsh, I know that I'll probably find it the same way.

So if there's a headphone out there that's getting good comments, and I look on InnerFidelity to see what you're saying about it and there's nothing here... now I have to wonder, what does that mean? Does it mean that you didn't like it and politely said nothing, or does it mean that you just haven't gotten to it? I don't want to have to wonder, I want to read what you say, even if it's not great.

And note that this is not only useful for people who share your tastes. Marco Arment has on his blog a roundup of closed headphones (http://www.marco.org/headphones-closed-portable), in which he notes his preference for brighter headphones and lists an AKG as his favorite closed headphone -- but adds that people who like a more laidback sound will prefer the NAD HP50. Even though he doesn't love it himself, that gives me meaningful information about how I might. And similarly, people who like brighter headphones can still take useful information away from a review of a bright headphone that you didn't love.

Argyris's picture

This. mkozlows has done a nice job articulating exactly how I feel.

Every reviewer of anything has personal preferences which might skew their opinion of an otherwise worthy product. The important thing is for that reviewer to know when it's happening and to make it clear where they're coming from, so people can interpret the review accordingly.

I think you've done a terrific job of this thus far, Tyll, in the instances where you've had to review something you didn't love, like, for instance, the AKG K812. I still felt like I got a lot of information from that review, even though you didn't give the headphone a strong recommendation.

I know it can be difficult to put up a less than stellar review, since (as you said) it can feel like everything is coming out too negatively. Most people don't enjoy putting down others or their efforts, and it can feel like we're doing just that when we basically say something is mediocre or significantly flawed. But it's still useful information. In my experience, the most useful feedback I receive is constructive criticism. It never feels very good being on the receiving end, I'll admit, but if something isn't as good as it could be, at some point that needs to be said is there is to be any improvement.

hanshopf's picture

My impression is that there could be more reviews on a site like this which is dedicated to headphones. If you do not review products which do not match your taste but are likely to be highly regarded by many others, than - I think - you could work on a way of being more objective. For example I tend to believe that only with classical music (or at least acoustic instruments) it is possible to safely say what is neutral. AKG K701/702 for example are definitively on the bright side, knowing the actual sound of acoustic instruments, likewise the Audeze's are on the dark side and not neutral either. Even HD 800, in my opinion, needs some EQ to become more or less neutral. So, if you make clear, that Audeze's are not on your wall of fame because they are headphones which are well suited for giving a clear and natural picture of what is going on in an orchestra but because their color matches your tastes and the music you are listening to (which I assume is mainly not classical), than you could also review the Hifimans and write, how their QUALITY compares with the Audeze's and others.
In Headfi they mainly compare headphones of similar price categories, which I think is restrictive. You could make a clearer difference and tell us how far away is Fidelio X2 from HD800 for example. I tend to believe there are only two relevant categories: quality and naturalness. I'd be happy with many more reviews, because you are the most competent reviewer of headphones I came across!!

Dreyka's picture

In one of your Inner Fidelity Updates (Here) you talked about you didn't want to review the Sennheiser HD8 DJ as it had a very odd frequency response. You said that you didn't want to review them because you're not a DJ and therefore don't understand the thought process behind that frequency response.

In the latest Home Theater Geeks episode you mentioned that you talked to a Sennheiser engineer about that frequency response and they responded about how mids weren't very important to a DJ and they were intended to be used in very loud conditions. It's a shame that you never updated that old update to add this clarification because it shows that Sennheiser did have a reason for such an odd frequency response. It's helpful for people to realise when frequency responses significantly deviate from what we consider the norm or "ideal" that there can sometimes be a reason for this.

The problem with not reviewing mediocre headphones such as the AKG K812 and MDR-Z7 is that by remaining silent the overwhelmingly positive praise from people, whom I'll put politely aren't experienced enough to write reviews, dominate the discussion and New Toy Syndrome colors peoples expectations.

A negative or mediocre review is useful simply to damp the breaks on the hype train that happen every single time a new and noteworthy headphone comes out. The thread for the Z7 and Alpha Dog Prime on Head-Fi is simply out of control and this happens every single time something new and of audiophile interest comes out.

I don't ask that you write a review for every bad headphone but please do write reviews for the Z7 like you did the K812. Disappointment and clearly explaining what you didn't like is useful because at the moment peoples purchasing decisions are being biased towards headphones that are mediocre and of significant cost.

People who have distinguished voices and distinguished ears, such as yours, shouldn't be cowering away from reviewing these significant headphones because you personally didn't like or found to just be mediocre. A single negative or middling review in a sea of positive reviews can help temper peoples expectations and give people the time to have a second thought about what they are about to spend several hundred dollars on. This is something that is sorely needed in this hobby and there are just too many headphone makers that are getting away with under-engineered, over-priced headphones. The Z7 doesn't even have damping material in it which is just ridiculous when a lot of it's flaw can be fixed by rather simple modifications and that's true for other flagships such as the Sennheiser HD800.(I really don't know how a mostly handmade headphone such as the HD800 hasn't had the Anax mod incorporated into it yet. Doesn't seem that challenging for Sennheiser to do.)

On another point I can't help but think back to the Focal Spirit reviews and your comments about comfort or generally lack of. In the context that it has been widely agreed that the pads are simply far too small to ever be considered circumaural for the average ear. Unless of course you've tiny ears, or a significantly high pain threshold, there is no way you couldn't have found those headphones becoming uncomfortable after 30 minutes. One thing that Zeos Pantera does and I wish more reviewers would do is simply wear the headphones for an hour or more without music playing and then consider how comfortable they really are.

Beagle's picture

"The problem with not reviewing mediocre headphones such as the AKG K812 and MDR-Z7 is that by remaining silent the overwhelmingly positive praise from people, whom I'll put politely aren't experienced enough to write reviews, dominate the discussion and New Toy Syndrome colors peoples expectations"

Who's to say who is 'experienced' and who is or is not capable of reviewing? How would you know what another's experience is anyway? And is one persons experience better or worse than someone elses?

If you value other peoples opinions more than your own, that's bad. If you think other's opinions are not valid because you think they are not experienced, then don't bother reading them. Don't tune in at all.

grawk's picture
I generally agree with the choices you've made. Stick to what's good, or what's popular and bad, things that will drive traffic. Every once in a while, have an overview post of headphones that are popular but mediocre, and ask if people who like them can explain in comments what they like. That, and more video reviews of horrible headphones...
Lumos's picture

I generally agree that headphones which have mediocre sound does not deserve the full review, just keep commenting what's wrong on them.

But please make review on HE400i and HE560.

ednaz's picture

Life is too short to spend writing negative or indifferent reviews of every headphone that isn't really good.

AustinValentine's picture

I feel like there's a reasonable solution to the problem of reviewing more of #4 while not cutting into #1, #3, and your broader Headphone 101 informational pieces: have a monthly poll where you list 10 headphones from category #4 and let your readership sort it out for what one they want to read about more. You could start an "InnerFidelity Flavor of the Month" Review, so to speak. (I don't mean that sarcastically, and obviously it could be named something else to remove the negative FOTM connotations).

--------------------

I also wonder if your taxonomy doesn't omit another category of headphones: those between "extremely poor value" and "mediocre value". Obviously, it's your ethical responsibility to produce a negative review of headphones in category #3. (Beats Solo, Ultrasone Edition 10, I'm looking at you both.) But there has to be something between the nearly criminally bad and blase. If any spot in the market should demand more of your time, it should probably be that spot.

For example: the going price on the Sennheiser HD700 has fallen substantially as of late. (They recently were on sale at the Sennheiser Outlet for $350 with discount and coupon code.) At their release price, with that horrific 6k peak, I think that they fell pretty firmly into category #3. But, at $350 - or even more regularly $500 - the price change forces a reassessment of value. They now occupy that space in-between the two dipoles. I can't help but think that there are more headphones that *start* right here as well.

I think that more of these negative reviews, done with characteristic humor, tact, and supported by your measurements, would fill an aporia in the headphone review space. I also think that they would drive site traffic. Your Beats Solo and ED10 videos are above 200,000 views each. Obviously, Beats would be #1 given its market share alone...but for the ED10 to almost match it suggests to me that people are willing to click for this.

That kind of review prioritization might run counter to your pretty damn positive ethos. But, ask any professional wrestler: sometimes it's fun to be the heel. If it seems a disservice to manufacturers to spend time producing negative reviews...well, Oscar Wilde said it best, "It is difficult not to be unjust to what one loves."

wiinippongamer's picture

^Winner

veggieboy2001's picture

to expand on what mkozlows said, I thing there are a great many people who respect your ...ears? (opinion). Part of why I enjoy reading many different reviewers is because I become familiar with their particular tastes, and I can often make value judgements from there (with the proverbial grain of salt). For example, I find it quite helpful how ljokerl categorizes and compares, and grades different headphones, not that you should change your review style. It's just that if I read what you do or don't like, It may give me an idea of what I might think given my perception of your past reviews.

Either way, thanks for all you do!

veggieboy2001's picture

"I THINK there are a great many..."

incidentflux's picture

Would love to see a review series on Pro Headphones for professional reasons. Those being used in broadcast television and radio stations for decades. Must be some good reason they keep using the same models.

ultrabike's picture

Perhaps one other option is to add:

-A headphone of significant popularity given certain unverified characteristics assumed by consumers through visual association and manufacturer claims/line-position.

Visual association:

The AKG K612 ($150 - $200) bares a remarkable resemblance to the AKG-7XX cans, but for less $$$. A value minded consumer may assume that these are just as good by visual association... or just be curious about it.

Manufacturer claims/line-position:

It seems a lot of folks are curious about the Sony MDR-7520 ($500 MSRP) here in IF. And I can see to some extent why. The MDR-7520 seems like a well regarded professional headphone and Sony's pro/studio TOTL. It looks remarkably similar to the MDR-7510 which is perhaps a more value proposition. A consumer may assume that these are good value not only due to their price (in some cases, perhaps not this one), but also given their pro audio line status.

Anyhow, one could also take a look at all those Turtle Beach gamer can stuffs...

ears_not_of_dog's picture

I've recently seen some effort to improve the sound quality of 'headsets' recently.

The dearth of detail in the sound quality department for any of the reviews I've seen makes it difficult to know if Plantronics or Sennheiser would make me more happy for my all day in the office headset.

These are clearly not meant to replace your Stax or or Audeze cans, but many folks put a lot of mileage on these headsets due to office requirements / constraints. I have some really great Sennheiser Momentums that I use to listen to my music with, but I end up wearing some rather lousy Sennheiser PC-38 USB headset due to my need to be often on the phone all day (and its mic. is superior to the Momentums in every way...which is required.)

It would be great if somebody made something with decent SQ for voice and unified comms folks. -I would trust few others to put these headsets through their paces from a sound quality perspective.

I know your time is scarce and you don't exacly have a lack of different cans to review. -Hence the article & thread in question.

I still can't help wondering what the market share must be for office users and decent headsets and whether it's worth having a go.

Cheers.

BTW, here's a review I saw for Headsets:

http://voip-headset-review.toptenreviews.com/

tony's picture

I don't know how many pix you had to go thru to capture the essence of the question but this one does it quite well.
I'll just keep with the Sennheiser stuff, I ain't trying to fix sumptin that ain't broke!

Ohhhhhhhh, Pleeeeeeaaasssseeee, Sir , can you help?, I'm on my aching old knees in supplication here, praying for some guiding light:
Can I flatten out my HD580s with a 31 Band EQ.?
I'll pay nearly anything you ask for a reasoned and experienced 85% accurate & truthful response!!

Tony in Michigan

ps. I'll even fly out to the Frozen North to sharpen your kitchen knives and cook you up some wonderful Cabbage.
It's not the best I can do but it's good for starting negociations, hint: I can bring Cubans & Scotch

Three Toes of Fury's picture

....wait.strike that..reverse it" w. wonka.

Thanks captain T for reaching out and presenting your approach, philosophy, and asking for input.

I'll focus on the "ok but not really good"...i think that rating and opinion is good and important to share with readers but not at the expense of investing extra time on them (ie: comprehensive review and measurements). The key thing is that readers like myself are looking for intel from you and your team but not requiring comprehensive at the loss of more pertinent content. One possible compromise is that you could post abbreviated writes ups on such headphones..quick and dirty and a brief summary of what you liked (if any) and what didnt float your boat. This is also accomplished in your current method of bullet-point-updates postings.

In the end i honestly think you should post based on what works for, and excites you, as i think that will (and has) result in the best content and balance. Im really enjoying the balance of headphones 101 articles with the reviews of all around good performers (with other misc entries such as show reports, misc surveys/content, contests, etc).

Thanks, as always, for the great information and education. Keep rockin in a free world!

Peace .n. Living in Stereo

3ToF

24bitbob's picture

Keep the reviews coming just as you do.
It's because your opinions are so well constructed, and backed up by objective analysis too, that your site has become a 'go to' web site for me. Nobody does it better in my view, so don't fret.
Bob

Claritas's picture

I suggest adding a Stuff We Like section. The badge is tacked on at the end of some reviews but, unlike the WOF, the information isn't collected in one place so it gets lost or overlooked.

Negative reviews are unnecessary in obvious cases. But negative or mixed reviews are very useful for headphones we hope will be good such as K812 or ones we'll end up disagreeing about such as the new Hifimans. I'm interested in your analyses of measurements and your personal observations regardless of whether I agree with them.

Last, I hope you add a note about how P5 series 2 compares with the original because you liked that one despite its coloration. If you think the sequel doesn't live up to the original, it's worth knowing why.

DaveinSM's picture

Wait-- the old B&W P5 is on your wall of fame, and by all accounts the new P5 S2 has improved sound and less bass heavy with more balanced mids and highs. Yet you won't even review the new one?

Kalrykh's picture

A coffee mug with the header image on it.

KG_Jag's picture

Reading your (Tyll's) work over the years, I have concluded that you often do not favor cans that you find to be a bit bright or not quite to your taste. Perhaps this is best illustrated in the Audeze vs. HiFiMan discussions. Although they are included in articles here, some great HiFiMan products have escaped stand alone reviews. If a can does not fall within you sound signature preferences, you can have another reviewer on your staff, with a complementary sound signature (or other type of) preference, provide a second opinion in the same review.

I also believe that you (the staff) should devote your limited review resources to cover the flagships and worthy second bananas (often representing the best quality to value point) of the "serious" and established headphone companies. In addition to high value use of your available reviewing resources, this often tells us about the results achieved (and not achieved) through the best efforts of the company. It may also give us insight into their design and "the sound" for which they are reaching. Over time a moving picture (or at least a plot dots) will be created for each company.

drm870's picture

...that A) the products in item 4 should be reveiwed; and B) theHifiMan planars should be added list #4 (for the sake of quenching our collective curiosity, if nothing else :P ).

Seriously, though, while it may put you in an awkward position from a relations standpoint with some of these companies, it really is reviews like the one you gave for the AKG K812 that are most interesting for many of us.

Just think about it! :)

bernardperu's picture

Tyll, how do advertisers react to negative reviews of their products?

What would the average advertiser do if a product of them had a bad review? What is your guess?

I don't mean to be a cynic and i really love your web site, but the fact is that the hi-fi media has some financial constraints and it is collectively choosing not to hurt advertisers. This has nothing to do with being willingly dishonest. On the other hand, every subjective approach to finding truths is to be accepted. For instance, comparing cables without blind tests. A long and comfortable blind test could be done with the assistance of a high school student working for minimum wage. Yet, these blind tests are not performed when comparing cables. Also, the opinions of major manufactures who claim cables make no difference are systematically ignored.

The above is just an example of an element that helps depict a broader picture: bad reviews are to be avoided because they hurt advertisers. As a reader and a business man, i can live with it. I just happen to appreciate honesty.

SevenPlus's picture

- In depth reviews for 1 & 3

- Mini Review for everything else you tried, no more than 1-2 paragraphs, no video, no measurements necessary. This is important because we might believe that something is not in the Wall of Fame simply because you didn't try it.

- Skip Headphone 101. This is a review website. Lots of info on the web / books for those who want to learn those things. At most you can write a mini-post with books/websites you read and found interesting.

ManiaC's picture

Please make review on HE400i and HE560!

Mauro's picture

I like the way you choose to review headphones. I feel it relaxed and not aggressive, which is a common attitude on blogs. I appreciate it a lot!
I would just expand monthly updates maybe with bullet points with pros and cons to allow us to understand your opinion without being too 'tranchant'/sharp on headphones you are not planning to review...

Cheers!

Currawong's picture

...is a tough one, I agree. I've started facing much the same kinds of issues. It does help to read people's impressions and compare them to how new they are to the hobby (what they've owned before) and what music they like. I've found that someone buying a controversial pair of headphones coming from a lesser pair may like the better pair, but someone such as myself who has a lot more experience and a lot more expensive headphones on hand may be more aware of their flaws. So when I review now, I try and consider what I would feel from different perspectives, especially different types of music and volume levels.

The new Ultrasone Performance series are similar: I like them better than the regular closed-back Ultrasones, but I know people who tried them and didn't like them who prefer the Edition and Signature-series (which are basically the same headphones with or without fancy metal).

What really put me on to this was the first time I tried V-MODA headphones. They don't really match the type of music I like, but when Val put a pair on my head and turned up the volume on some club music, I finally understood what his aims were.

So ultimately I find that patterns are emerging amongst people who like or dislike certain headphones and empathising with those different types of people, based primarily on the music they like and what their level of experience is has really helped get a good picture when evaluating a pair of headphones.

Jazz Casual's picture

Currawong, how do you acquire the headphones that you review?

Currawong's picture

I either buy them (eg: Sony, Audio Technica) or ask for a loaner set to review them because I'm specifically interested (eg: JHAudio, Oppo, Ultrasone). The latter usually because they are getting a lot of discussion and Head-Fi members ask if I can review them. Occasionally someone from a company will ask if I'll review their headphones (eg: Pendulumic).

Jazz Casual's picture

Thanks.

mikemercer's picture

Tyll,

I'd LOVE to read your in-depth review of the Master & Dynamic MH40s! I got a pair in gun-metal and black (LOVE em compared to the light-leather/popular color scheme) and I love em! I love the sound, their industrial design, even the mute button!

With better pads, they're be EXCELLENT IMO -

But they're well-worth the dough.

mikemercer's picture

but, you can scratch the B&Ws, IMO...

jeckyll's picture

You can't please everyone and focusing on the 'big hits & misses' is important.

However, I think there's nothing wrong with a short 'blurb' on the other categories, especially if you list your own personal bias, so people who have different biases can take that into account.

Generally, it's tough to get useful feedback, there is so much pure noise out there that having consistent, trusted feedback can make a huge difference and that's why I personally would rather have a bit of feedback than none at all.

BobMc's picture

I think a reviewer's job is to review everything...good, bad, or mediocre. Where are the HiFiMANs and Grados?

roscoeiii's picture

Hi Tyll,

Great work here for sure, but there are also many reviewers who have different listening priorities than you do. If brightness is a real dealbreaker for you personally and may result in you not giving treble-leaning headphones a fair shake, then get someone on staff who can give an evaluation from someone who likes flavors of headphones you do not. That would certainly help keep this site from having a skew to it. Stereophile and other sources of reviews seem to have a stable of folks with different priorities and preferences. And reviews seem to be assigned to reviewers that might most appreciate the gear due to what can be made out about the gear's characteristics, in conjunction with the gear's fit or lack of fit with the reviewer's system.

Lastly, thank you for being so upfront about all of this. I think it is key for reviewers to be clear about their preferences.

Tyll Hertsens's picture
....and I think I like that idea a lot. I think Joker likes the HE560i, I'll drop him a note.
FizzlePop's picture

It's a bummer Joker doesn't do video reviews. Next time you send him some IEMS, throw in a few Hawaiian shirts and a video camera.

catscratch's picture

It's a tough question for sure.

First of all, you need to wear your biases on your sleeve, and be totally upfront and transparent about why you choose to review certain headphones and not review others. It's very easy to start asking questions like "does he review only headphones he likes so as not to offend sponsors?" and being upfront helps diffuse potentially nasty thoughts like this that readers will often have. Also, I don't think you need to be totally impartial, as long as, once again, you are clear about your preferences. I think trying to hire someone that has different views from your own detracts from the fact that this is YOUR website and people come here for YOUR thoughts primarily. You don't want to send a confused message, you want to tell people what YOU think honestly, and remind then of what your preferences are so they know how to judge your opinion, and that's pretty much all of it.

Second, I think you need to let your enthusiasm for the content you create dictate what you choose to do and what you don't. Obviously, I don't know you and this advice is probably off-the-wall and overly general, but... If you're enthusiastic about the content you create, it is immediately obvious and it results in better quality content. If you're blaze about a pair of headphones and aren't sure whether or not it's worth your time to review them, then don't review them. Think not only of whether or not the headphones need a review, but of how you can use your time to provide the highest possible quality content for your site.

Personally, I find things like the articles on the fundamentals of headphone audio invaluable, a great read, and a great addition to the site. I'd rather see more of that, than a more complete inventory of reviews for every single headphone out there.

Lastly, the short updates you do are also pretty valuable. If you're concerned over the amount of coverage that you can reasonably do, perhaps an article with a very short summary on your takes on various headphones that don't otherwise warrant the time needed for a full review could be really good.

guerillaw's picture

Thanks for all the work and the free edutainment you have provided for our community.
I think you are asking the wrong people. As I comment the main interest I have is in the site's survival and prosperity.
As such, I would say you need to review two categories of cans to drive traffic: popular cans people shop for and widely available cans that for some reason have not been reviewed. This is all about traffic.

The folks commenting here and I would guess most regular users of this site appreciate all the other information but we WILL find it wherever it is and consume most of the site because of our interest in the hobby. The traffic gains are to be had by appealing to the folks who are shopping for themselves or others and will only read the site for that purpose and likely never come back. For example, someone looking to buy a nice set of headphones for the first time. Surfs the web, listens to a pair or three, makes an investment, and likely will not be reading about headphones for years. Put another way, not every purchaser of an HD800 becomes a Head-fi member and starts stuffing cotton in the cans to tame reflections. In fact, MOST do not. They look for the "best they can afford" buy it and get back to their lives.

Now, as part of the minority that views this as a hobby and is interested from that perspective I love your priorities and especially "headphones 101". I am surprised by the many folks who claim that the "information is available elsewhere" because I have not found it. I would love to see links or cites from those people. Headphones are a booming and developing industry and the fact is that the measuring and science of headphones is still developing and it is wonderful that Innerfidelity keeps us in the loop of that development and in some ways contributes to it.

Thanks again, wishing you a prosperous 2015!

MrO's picture

I think I speak for a lot of people when I say, we are depending on you to guide us through the murky waters of the headphone market.
That being said, the more headphones you review, the better. Do you know how many times I've come across a pair of headphones and thought "what would Tyll think of those?"
Case in point: the Sony MDR-1A. I happen to have their predecessor and I've noticed they've gone and improved certain things that were wrong with them, like those holes on the back of the "arms" that created a lot of air noise when outside. But before I decide to replace the 1R, I'd really like to know what you think of them.

So in all, I suppose reviewing more stuff would not only make existing fans happier, it would also attract way more people to innerfidelity. Educate the masses, save us from bad sound! :D

vertical2's picture

Tyll, thanks for another great year of reviews, informative articles, and pieces that get me thinking...

OK, back to the festivities ;)

Merry Christmas!

RPGWiZaRD's picture

For my taste I find this site lacks the thing I keep coming to this site to read; ie. headphone reviews. All those 3 you listed in this article are headphones which in your shoes I had probably reviewed because they are markable in one way or another. Sony for its "affordable" higher end headphone with a bit better build than your typical mainstream headphone or the MH40 for its interesting newcomer in the market with a rather oldschool design but really promissing entry product in the market and P5 Series 2 for the build quality, design and well appealing-to-mainstream sound. It's easy to write good articles around all three.

RPGWiZaRD's picture

How I'm missing the edit button! I think you should forget about focusing too much from your own perspective and rather look at the headphones from a third person perspective and more objectively point out the good and bad and try focus on "which person does this headphone fit for" because you and me both know there's no single "this is right" for everyone in this business.

theaudioanalyst's picture

Tyll, I COMPLETELY get your dilemma... Might I suggest that your thoughts about weighing your time doing positive works should CLEARLY outweigh worrying about such things is an example of right thinking? Honestly, the simple fact that a product does not get reviewed has the power to speak volumes. The glaring omission of a review of a “popular” headphone can be as scathing as any actual negative review, or even a mild warning, that you might write, no? Your time should be focused on accomplishing your mission; sharing information to allow your readers to divine the exceptional from the merely good or acceptable entrants in the field. Just my two-hundredths of a buck…

Mitch's picture

I used to love going into a wine store that was owned by my grandfathers friend.
One topic that would often one up is "I'm going to bust a friend who drinks wine in the $100 per bottle class, but as I wasn't capable at that time of spending more than 25.00 what is a $25 bottle that would pleasantly surprise people who have more costly tastes. Something that I can afford but really performs well for that price range.

I have Grado RS-1s and the Grado Headphone amp. They probably will always be my top of the line setup
Every now and then I purchase something that far outperforms its cost. Most can't hold a candle to my Grado cans, but at a significantly lower price, I'm not expecting them too.

I bought a lot this year. The Blu mofi is my favorite purchase. Beats Studio 2 is better than expected but even with the Black Friday Amazon deal, isn't very impressive (I'd like it better if you could turn off the noise canceling). Westone Pro 40s I'm happy with, but the cost a bunch and I feel like I got what I paid for. Sol Republic often surprises me. The cost vs. performance scale is often better than I expected. The $14 Panasonic in ear that often tops lists of surprises was pretty good once you had a decent in ear seal, but the cable/wiring is so tangle prone, they are a constant source of frustration.

So to answer your question (almost) my favorite reviews are the less expensive pairs that really outperform anything in their price group.

Basstwo's picture

Tyll

First of all (and most important), I think you are able to garner such a big following because you seem to really enjoy your work (hobby). Therefore, whist it's good to see that you also like to receive feedback, my suggestion is that you just continue to do what you love. Review what you want, when you want. I am sure the rest of us are more than happy to learn from you whatever is it that you choose to review. It's not your job to be Guardian of the Headphone Galaxy so don't fall into that trap.

If you do run out of ideas on what to review, you have plenty of eager readers ready to give you suggestions so you can just pose this question from time to time.

Cheers and please continue to enjoy your work. Look forward to your next review!

jerseyd's picture

Would love to have mini-capsule profiles of the headphones you deem good-but-not-great. A brief paragraph that describes the sound, strengths and weaknesses of these cans would be very welcome. For example, after reading this very story I am now itching to find out what is holding back the Master&Dynamic, B&W and Sony models from greatness. Please do not be afraid to hurt advertisers (and potential advertisers). You may just spur them to improve their products.

zobel's picture

You do a good job measuring and reviewing cans, and it has to be very time consuming. I think you are wise not to bother with headphones that are for special purposes, such as gaming, or DJs, or for the hearing impaired, or any other application other than normal music. You are smart to have Joker do IEMs too, as that is his bag, and not really yours as much.

What that leaves you is still a big pile of cans to deal with. I think you should adopt a quick survey mode for for the headphones you find non-competitive for reasons of sound quality, fit and/or build quality, and price point in the market. A good yardstick would be price/performance in each category of can. Most of us appreciate your down to earth value ratings, and benefit by your recommendations in affordable cans. We definitely need your insight when things are over-priced in their category, and its good to know why some cans don't make the A list. You know how Stereophile does equipment reviews. A bunch of guys send in reviews (mostly subjective) of the expensive gear they have been given by a manufacturer for their review, and the press that that implies. JA does some measurements and comments. They then collect these reviews and publish their recommended components which are organised by cost mostly. Letter grades are awarded and almost without fail, The grade A go to gear nobody can afford, except for those with more money than sense.

Your system is much better. One reviewer who readers can relate to in terms of tastes, who also does the measurements, allows for much better consistency, and reliability, and usefulness in the reviews. Also, thank you for not weighting headphone ratings by their weight in gold. It is one reason I don't read Stereophile, but like Innerfidelity. Headphones for the most part haven't gone off the deep end in cost, and there is tremendous "bang for the buck" value in many very affordable cans.

So I think you should give these lesser models a quick mention, and no measurements. But advising us about them in this way would be very useful. Probably your biggest problem is just getting all these headphones in your studio to review. Readers should continue to submit cans of theirs (on loan) to you for your reviews.

Lunatique's picture

Two very good examples of headphones you didn't review but I wondered why, were the Westone 4, and Shure SE846.

The Westone 4 was Westone's flagship universal IEM, and quite popular among head-fiers, often being mentioned in the same breath as Shure SE535, yet you seem to not care about it, and it makes me wonder why, since it's not too bright and has some of the qualities you do like in headphones.

The Shure SE846--well it's the next step to the SE535, and I would think you'd review it as soon as it became available, since it's one of the most prominent and talked about new headphones for serious head-fiers. Maybe you have it on your to-do list and it's coming soon?

EarSonics SM3 is another one I wondered why you never reviewed, since it's so highly regarded.

Another thing I want to mention, is that I think more emphasis can be placed on universal IEM's instead of customs, because a lot more people are interested in universal and they are a lot less problematic to deal with. Customs are niche products even in the headphone community, and although I understand you want to give them more exposure, I think for the large number of people who would never want to deal with customs, it feels a bit alienating.

X