Testing the Audibility of Break-in Effects
Subjectivist: "Man, I got my headphones last week and they're breaking in nicely."
Obectivist: "Yer nuts, dude, it's your head breaking in to the sound of your new headphones."
Subjectivist: "Leave me alone, troll, take your objectivism to 'Sound Science.' We have the minds of Gods and poets, and don't need your weights and measures to know what we know what we know."
Objectivist: "What can I say to someone who's their own placebo?"
Subjectivist: "Break-in exists ... I've heard it ... I stamp my feet three times and you will go away."
Objectivist: "Lol ... you couldn't blind test your way out of a paper bag!"
And so it goes.
Let's try to clear a bit of this up, eh?
Break-In Testing So Far
If you've been following along, you'll know I had three brand new Quincy Jones Q701 headphones that I'm using (green, black, and white) to see if we can measure the effects of break-in.
I did the first exploratory break-in test on the green pair some time ago, in which we saw some changes and learned what we might look for in subsequent testing.
I designed a second more complex test based on what we learned on the first test, and we saw the changes over time more clearly.
Now, we have an avenue to do a break-in test on the third pair and really run it through the ringer. But first, I thought it would be good to do a subjective test to see if I could hear the difference between a brand new pair and one that's been broken in considerably.
Subjective Break-In Testing
The green Q701 that was used in the first test has been on my bench playing pink noise at about 90dB for well beyond 1000 hours at this point. The white Q701 remains sealed, brand spanking new in its box. I thought it's important for our exploration into break-in to find out if your could hear some difference between the two.
So, I called my buddy Brian (screen name "NA Blur" on Head-Fi) here in Bozeman, and arranged a time for him to come over and help me out with the test. I made some score sheets, colored a coin, and set up the gear on my dining room table.
This was a single blind test where I did not know which headphone was being placed on my head, but Brian did know. (In double blind tests, Brian wouldn't know which was which either.) In order to take some influence out of Brian's hands, a coin was flipped at the beginning of each test that would indicate which headphone he would place on my head. At no time during the test would Brian indicate whether I was guessing correctly or not, and I would not know the score until the end of each trial.
I did notice that the music playing computer in front of me did provide some reflections in which I could potentially see the headphones on my head, so the screen was tilted way back to prevent seeing any reflection. As Brian was putting headphones on my head, there was no way to see the headphones, or feel the difference between the headphones. I had no idea at all which headphone was on my head.
Before testing began, I listened to both headphones to try to perceive what the differences were between the two. I thought it was fairly clear that the broken-in pair was smoother sounding then the new pair. Brian did the same and thought he heard a difference as well.
Then we began testing. I went first and had a somewhat difficult time. I was using a Tiger Okoshi track with a strong trumpet solo that I knew could sound harsh if not well reproduced. As we progressed through this test, Brian said maybe I should try another track ... which was a bit of a hint that I wasn't doing so well, but we were on our first trial so I figured I'd switch the music I was using mid-stream. I switched to a Pinback driving rock track that was very dense with sound, and which could sound harsh and pinched when poorly presented.
By the time we finished 19 guesses, I called it quits and found that I had gotten 13 out of 19 correct, which is statistically significant, but I thought I could do better.
Brian tried it as well. At this point I hadn't told him which headphone was the broken-in pair, and he didn't want to know, he also took only a very short (probably too short) listen before starting his trial. When he was done he had gotten about 65% wrong. We think he had gotten mixed up with the sounds and which color headphone made what sound. It seemed evident he was hearing differences, but misidentifying which headphone was which.
Then I sat down for my second trial with a pretty good sense of what I was listening for and what music to play.
Blind testing is not easy. Even when there is a fairly clear difference it can be quite disorienting not knowing whether you are guessing correctly or not. Opportunity is rife for self-doubt, anxiety, and second-guessing oneself. But I've done a fair bit of blind testing of prototype amplifiers, so I knew what I was in for. As the second test progressed, I relaxed and relied on my previous experience, and used a technique I think works very well for this sort of thing.
I don't actually try to listen for the sound for a problem, or differences in sound. I relax and listen to the music normally, as if for enjoyment, then I pay attention to and monitor how I feel about the music. So I'm not trying to be critically aware of the sound as much as I'm critically aware of my reaction to the sound. It's worked very well for me in the past.