Subjectivist: "Man, I got my headphones last week and they're breaking in nicely."
Obectivist: "Yer nuts, dude, it's your head breaking in to the sound of your new headphones."
Subjectivist: "Leave me alone, troll, take your objectivism to 'Sound Science.' We have the minds of Gods and poets, and don't need your weights and measures to know what we know what we know."
Objectivist: "What can I say to someone who's their own placebo?"
Subjectivist: "Break-in exists ... I've heard it ... I stamp my feet three times and you will go away."
Objectivist: "Lol ... you couldn't blind test your way out of a paper bag!"
And so it goes.
Let's try to clear a bit of this up, eh?
Break-In Testing So Far
If you've been following along, you'll know I had three brand new Quincy Jones Q701 headphones that I'm using (green, black, and white) to see if we can measure the effects of break-in.
I did the first exploratory break-in test on the green pair some time ago, in which we saw some changes and learned what we might look for in subsequent testing.
I designed a second more complex test based on what we learned on the first test, and we saw the changes over time more clearly.
Now, we have an avenue to do a break-in test on the third pair and really run it through the ringer. But first, I thought it would be good to do a subjective test to see if I could hear the difference between a brand new pair and one that's been broken in considerably.
Subjective Break-In Testing
The green Q701 that was used in the first test has been on my bench playing pink noise at about 90dB for well beyond 1000 hours at this point. The white Q701 remains sealed, brand spanking new in its box. I thought it's important for our exploration into break-in to find out if your could hear some difference between the two.
So, I called my buddy Brian (screen name "NA Blur" on Head-Fi) here in Bozeman, and arranged a time for him to come over and help me out with the test. I made some score sheets, colored a coin, and set up the gear on my dining room table.
The Test
This was a single blind test where I did not know which headphone was being placed on my head, but Brian did know. (In double blind tests, Brian wouldn't know which was which either.) In order to take some influence out of Brian's hands, a coin was flipped at the beginning of each test that would indicate which headphone he would place on my head. At no time during the test would Brian indicate whether I was guessing correctly or not, and I would not know the score until the end of each trial.
I did notice that the music playing computer in front of me did provide some reflections in which I could potentially see the headphones on my head, so the screen was tilted way back to prevent seeing any reflection. As Brian was putting headphones on my head, there was no way to see the headphones, or feel the difference between the headphones. I had no idea at all which headphone was on my head.
Before testing began, I listened to both headphones to try to perceive what the differences were between the two. I thought it was fairly clear that the broken-in pair was smoother sounding then the new pair. Brian did the same and thought he heard a difference as well.
Then we began testing. I went first and had a somewhat difficult time. I was using a Tiger Okoshi track with a strong trumpet solo that I knew could sound harsh if not well reproduced. As we progressed through this test, Brian said maybe I should try another track ... which was a bit of a hint that I wasn't doing so well, but we were on our first trial so I figured I'd switch the music I was using mid-stream. I switched to a Pinback driving rock track that was very dense with sound, and which could sound harsh and pinched when poorly presented.
By the time we finished 19 guesses, I called it quits and found that I had gotten 13 out of 19 correct, which is statistically significant, but I thought I could do better.
Brian tried it as well. At this point I hadn't told him which headphone was the broken-in pair, and he didn't want to know, he also took only a very short (probably too short) listen before starting his trial. When he was done he had gotten about 65% wrong. We think he had gotten mixed up with the sounds and which color headphone made what sound. It seemed evident he was hearing differences, but misidentifying which headphone was which.
Then I sat down for my second trial with a pretty good sense of what I was listening for and what music to play.
Blind testing is not easy. Even when there is a fairly clear difference it can be quite disorienting not knowing whether you are guessing correctly or not. Opportunity is rife for self-doubt, anxiety, and second-guessing oneself. But I've done a fair bit of blind testing of prototype amplifiers, so I knew what I was in for. As the second test progressed, I relaxed and relied on my previous experience, and used a technique I think works very well for this sort of thing.
I don't actually try to listen for the sound for a problem, or differences in sound. I relax and listen to the music normally, as if for enjoyment, then I pay attention to and monitor how I feel about the music. So I'm not trying to be critically aware of the sound as much as I'm critically aware of my reaction to the sound. It's worked very well for me in the past.
 | |||||||||
|
 Article Contents
| ||||||||
Comments
"Have we absolutely proven that break-in is an audible phenomenon? No. All I've proven is that I could tell one headphone from another."
To rule out that you were just discerning manufacturing tolerances, you could break-in the new Q701 for a comparable amount of time and then re-do the blind test again. If -- after an honest effort -- you then failed to discern both headphones, you could be sure that the current audible differences are (mainly) due to break-in.
Manufacturing tolerances, my thought exactly.
I really respect what you're doing but that's a big problem in a test where you try to show that burn-in exists. You've seen yourself that different headphone samples (even if they have the same color 
and probably even if they're from the same batch) have slightly different channel balance, impedance, sensitivity/efficiency (-> different SPL).
What I'm trying to say is that I think you could achieve the same test results with two brand new Q701s or K701s.
When you see the headphone being manufactured and coming off the assembly line and you grab it and take it with you, then you know the mfr didn't pre-burn it on an automated rig. Other than that, you don't know. You could assume no pre-burn for cheapies, but for mfrs' pride items, I wouldn't assume that.
I will, however, make an attempt to find out.
Are you planning to do a similar test with an in-ear model?
What about the heat and humidity of the skin transmitted to the earpads? It could be a clue because the most recent headphones worn could be hotter or had a feeling of recently wear. A good way to solve this is to heat the earpads equally, or wearing the same earpads for both. Furthermore, earpads virtually identical could sound different.
There are many factors like those but I've no doubt that break-in is subjective noticeable, of course, that's my personal experience, like many.
Also because it's been asked elsewhere, the green headphones had about 200 hours of head time on the dummy, and were on the shelf burning in for the remainder. Pad wear is very likely not a factor.
Even if there are differences in manufacturing with some models being slightly bassier and older pads making the sound more intimate and richer, the difference where one is harsh and makes one wince vs the other is a typical burn in related thing. I'm inclined to believe that these new cans will be as smooth as the one with 1000 hours on them after burn in but what I am not sure about is if they will end up with the same exact tonal balance.
There have been reports of k701s sounding different in the bass for instance even when both have been well used. These conclusions were made after comparing two k701s side by side by the way.
I don't think we should ban them from the hobby though ... they should be made to fetch cigars and burbon at meets, and massage my feet at trade shows. Yeah, that'll teach 'em.
Also, more trials would not have gone any further toward proving the audibility of burn-in, so making the test more stringent wouldn't have added any info, I think.
If my memory is correct, you have the measurements of the new green Q701 somewhere in your database.
Including the measurements of the new green Q701, its measurements in its broken in state and the measurements of the still new white Q701, should give us some information about whether the differences you heard are more likely due to break in or manufacturing tolerance.
Really, looking at this issue is best done by putting a pair of headphones in the chamber and doing the long testing over time without moveing the head ... which was done in previous tests, and did show small but measurable difference.
Thanks Tyll. More great work here.
I used to be fairly skeptical of break-in... and I still am, somewhat. But when I reviewed Maximo's iM-590, I noticed they had a tag that said they should be burned in for 20 hours or so. So I listened to them, found them harsh, gave them a 24 hour burn-in, listened again, and although I still found them somewhat bright, most of the harshness was gone.
So now I burn everything I review in for 24 hours, usually before I even try them, and certainly before I do any formal evaluation. Although judging by your data, maybe I should do more...
Great article, I love your approach, and I love it that you assume NOTHING even after hearing an audible differences.
The amount of variables is obviously astounding but even if you show proved audible differences that's a great step forward IMO.
In the previous articles you said that based on the measurements you don't think the differences would be significant i.e. cans would have the same SQ out of the box as well as after burn in. How do you judge that now that you have heard the difference? (how big is the difference?)
I also fully support the suggestion in the first comment above. It would have also been good if there was a blind test conducted on the headphones when they were new as well - to show that (possibly) statistically you couldn't tell them apart. Maybe worth doing one before burning in the third pair?
One last thing, in one of the previous articles someone suggested an experiment which I think would be very interesting: record the same track's output through the new and burned in cans and then do a comparison of the wave lengths (hopefully that's the right terms).
I think what I've said in the past is the there differences out of the box and after burn-in are small, and what you her out of the box is pretty much what you'll hear 1000 hours later. What I'm saying is comments like "OMG! My cans are like night-and -day different not that I've burned them in" are largly balony.
On all headphones I have had including the k701s, I have noticed the effects of burn in were mostly subtle and admittedly sometimes due to psychological acclimation/ getting used to a certain sound but the hd555s I had before were a totally different phoenomenon. It was really weird in that out of the box they were echoey as hell and was much brighter than the grados I had. They also had zero bass. Listening to the same 555s now which a cousin of mine now owns, they are warm sounding and even slightly dull in the highs. I actually thought they were damaged when I first listened to them, out of the box. My 555s were an Irish made 50ohm model by the way in case that matters.
One of the most useful points I feel you made was how, with practice, you could improve your blind test score. I see a lot of people feel that any DBT is absolute in the truth it conveys regardless of anything else, but don't understand that such a thing is far from what a properly conceived and executed scientific experiment requires if it is to produce meaningful results.
Most importantly, you've pointed out that there are no absolutes regarding audio and you're doing very well demonstrating the complexity with what goes on when we put headphones on and listen to music.
Why? Because they're lazy, and don't want to do something that's uncomfortable. It's so much easier to say that blind testing isn't informative, and so much easier to blather about their own experience as if it's objectively factual.
I think the best part of this article is the intro, which strives to show the closed mindedness of both "sides."
I really agree with this post here. Some people just say this and that is much better like flac vs aac, etc for example when they won't even do a foobar abx test to prove it. Then there are people that also won't listen when you say that with a foobar abx, you can also not really make sure that one file sounds exactly the same as another unless you had the best equipment connected to your system at the time of testing.
With an hd600 for example, I can tell a flac from an mp3 in a foobar abx way more easily than if I were using a px100-II for example. These details don't seem to matter to a lot of the objectivist camp and its sad. I think one has to be a combination of both subjectivist and objectivist to really know for sure about their hearing skills and the truth in a lot of things in audio.
I'm finding it easy and informative to compare music tracks now, using the same player, using different players, or using multiple copies of the same player so I can have both running and switch instantly. Comparing headphones is not so easy, having to remove one and plant the other. Very distracting. Most of that can be tuned out, but still very stressing when you're concentrating as hard as you can. Speakers were easier since you could just flip a switch.
For many years I too was 'caught' in the VS. trap. And then I realized it is very artificial and trying to stay in only one camp or the other divorced me from the reality I was actually immersed within.
Both have value and both provide their own types of results. And neither is 'complete' without the other. And in reality they can't be mixed together, like oil and water. They can however be used to examine and discern our ability to enjoy music to a greater or lesser degree. To help us hone and refine and increase our ability to discover more and more of what is already there just waiting to be revealed.
Numbers will never capture the enjoyment of listening to music nor the visceral and nearly (at times) palpable experience of being immersed in the music. And conversely we wouldn't be able to listen into those very same inner details and structure of the music unless it were capable of delivering a faithful re-creation of the original. It is this ability to accurately re-create the original where the numbers can give us the ability to peer into the finer structure of how the music is 'put back together' and then presented for our enjoyment.
And I suspect that as we gain in our ability to resolve ever more complex relationships between the numbers and our experience of what the numbers 'mean' (to the direct experience itself) both sides of the VS. syndrome will gain.
But even more importantly, our appreciation and enjoyment of the experience not only of listening but also striving to grasp and understand the complexity of the relationship between the numbers and our enjoyment, which is usually kept apart by the VS. syndrome itself, provides even further levels of fascination and challenge.
Well at least for me it does.
JJ
I think the tricky thing for most to get is the concept of "domains." Measurement is in one domain, the listening experience is in another. The twain meet in places, but don't in most.
Both domains exist, though. Both are valid. Oil and water indeed. A little balsamic vinigar may help.
:)
The biggest problem I see is that you didn't consider pads as the reason for the difference. At a meet there was a guy with brand new K701 and another person's old K701. They could tell the difference in an AB test but they switched the pads and came to the conclusion that the broken in pads were providing a better seal, therefor better sound.
Thanks for a fascinating exercise in non-rancorous self-experimentation. Indeed, you’ve boldly tackled a topic that has fanned many a flame war. But this kind of testing is too rare and too often taboo in audio journalism, and it provides a refreshing break from the customary oenophile-like audio evaluation based on subjective sound-tasting and the reviewer’s bona fides in “trained listening.”
But I still think we end up in the old familiar objectivist/subjectivist hall of mirrors. I have to admit that my heart sank a bit when I got to this proviso: "Before testing began, I listened to both headphones to try to perceive what the differences were between the two. I thought it was fairly clear that the broken-in pair was smoother sounding then the new pair. Brian did the same and thought he heard a difference as well.” The potential effect this had in terms of providing cues and generally priming expectations and identifying audible distinctiveness leading into the experiment is to disqualify the test on the basis of prejudicial prep work. You took a sneak peek at the answers before taking the test! A key pitfall in the ideas behind blind testing and component break-in is that you may be training your ear, not hearing meaningful differences between different pieces of gear at different stages of break-in.
And of course, the possibility that inarc and xnor raise in the first comments above — that the differences between the headphones may exist already before any “breaking in” — adds another degree of uncertainty to the project.
Part of my skepticism about break-in also relates to the unvarying conclusion that long break-in not only changes the sound, but inevitably IMPROVES it. The fact that 100% of the reported changes after break-in result in a component sounding better should be a significant warning that the fog of subjectivity is thick on the ground.
Still, as much I’d like to dismiss your outcome for such reasons, it’s difficult to avoid the conclusion that you’re on to something. Again, thanks for a highly suggestive effort.
|
|
|
|
Site Map / Direct Links| Headphones | |
Headphone Amplifiers | |
Portable Devices Home Devices | |
Blogs Guides Show Reports | |
Community Buyers Guides | |
News Resources |
