InnerFidelity Headphone Amp Measurement Program Update July 2012

Image of the computer screen while setting up the THD+Noise Vs. Output Voltage test.

The headphone amp measurement routine consists basically of two things: (1) the main program to run the various tests and (2) the tests themselves. The individual tests (Frequency Response, THD+Noise Vs Frequency, SMPTE IMD, etc.) are configured with a set of panels on a computer that runs the Audio Precision tester. Then a program is written that calls the various tests in sequence, prompts the operator to do various things like change loads or switch cables, and pokes all the data into an Excel spreadsheet as it is gathered. So far, I've been working on the tests, and Brian (NA Blur on has been working on the main program to execute the tests, and the spreadsheet that accumulates and displays the data.

Over the last couple of weeks, we've both been working on our respective tasks. Brian has managed to set up the software on a computer in his home; has started looking at the headphone measurement test to see how the programming language works; and has started to configure the Excel spreadsheet template in which the data is stored. There is a mode in the software that allows it to simulate being hooked up to the Audio Precision tester, and that will allow Brian to do the basic software development at his home. I can't express how grateful I am to Brian for voluntarily taking on this task (well, he'll get some headphones out of it) and helping me get this ball rolling. I can manage the programming, but I'm sure he'll be able to do it much quicker than I. Feel free to thank Brian in the comments, or NA Blur on Head-Fi if you bump into him there.

My task over the last couple of weeks has been to run and tweak the tests I've been slowly configuring on NwAvGuy's O2 headphone amp, and compare my measurements with his measurements with the hope that they match fairly well. If you've not heard of the O2 amplifier, it is NwAvGuy's attempt to design a very inexpensive amp that performs very well and can be built by DIYers. My experience with this amp so far is that it's very clean sounding, and does indeed measure very well. You can find out about it here.

The tests I'm configuring are designed to match up with NwAvGuy's recommendations in his InnerFidelity post here. In that article he recommends producing a datasheet that looks like this mock-up:

Let's have a look at the O2 measurements I've done in the last fortnight, and how well they match up with NwAvGuy's measurements. Graphs with black backgrounds are NwAvGuy's and mine have white backgrounds:

Frequency Response
This is essentially a measure of Gain vs. Frequency. Ideally, a headphone amplifier should have the same gain at all frequencies producing a "flat" frequency response plot. Apart from the fact that NwAvGuy's measurement gear can go down to 5Hz and my AP can only go down to 10Hz, you can see that the two measurements are quite similar.

Noise Spectra

NwAvGuy's noise spectra measurement with the O2 under battery power.

NwAvGuy's noise spectra measurement with the O2 under AC power.

My noise spectra measurement with both battery and AC power.

This is a plot of how much energy is at each frequency without any input signal. What you're seeing in this plot is spectral density of noise in the amp. It is in this plot that you'll typically observe AC power hum with peaks at the AC fundamental at 60Hz and some harmonics at 120Hz and 180Hz.

As you can see, my measurements have some additional artifact peaks between 50Hz and 600Hz, and about a 4dB higher noise floor. It's not bad, but I'd like to do better. Unfortunately, these low-level problems usually amount to things like grounding and cable lengths and shielding. Usually not too much fun to hunt down. That's life, I guess.

Total Harmonic Distortion + Noise vs. Frequency
120716_blog_HeadAmpUpdate1_graph_nwavguyTHDnoiseVsFreq 120716_blog_HeadAmpUpdate1_graph_InnerfidelityTHDnoiseVsFreq

This plot measures the amount of distortion vs. frequency with various loads on the output. As the load resistance gets lower and lower, the amp has a harder and harder time swinging the amount of current needed. Amps have more difficulty at the frequency extremes here, which is why the curve is "U" shaped; the drop-off in THD+noise above 7kHz is actually a measurement artifact as the harmonic distortions being measured begin to exceed the measurement system's bandwidth setting. For example, at 7kHz the third harmonic is 21kHz, and because the system bandwidth is set to 20kHz, the harmonics begin to be attenuated when the fundamental tone rises above 7kHz.

These two plots match up pretty nicely! In fact, my measurements show slightly less THD+noise, which on first blush might indicate my system has less noise in its measurements. I will try to spend some time talking with NwAvGuy about these data--his help will likely be invaluable in interpreting the measurements in order to troubleshoot potential problems.

Total Harmonic Distortion + Noise vs. Output Level
120716_blog_HeadAmpUpdate1_graph_NwAvGuyTHDnoiseVsOutputV 120716_blog_HeadAmpUpdate1_graph_InnerfidelityTHDnoiseVsOutputV

This plot shows the amount of THD+noise from a 1kHz signal as output voltage rises. The steep slope at the extreme left of the plot occurs because as the output gets lower and lower the ratio of noise to output signal gets higher and higher. In other words, at the low end of output voltage, the measurement is dominated by the amplifier's noise.

As soon as the output signal gets strong enough the THD+noise measurement becomes dominated by the harmonic distortion and flattens out. As the output voltage rises at some point, the amplifier begins to "clip" and the distortion begins to rise dramatically. By placing various resistive loads across the amp and running this test, we can begin to see how good the amp is at delivering clean power to headphones of differing impedance. By calculating the power being delivered at the "knee" of the curves (the point to the right at which each curve begins to rise dramatically) we can get a look at the output power limit of the amp into different loads.

  • With 600 Ohm and the knee at 6.8 Vrms indicates 77 mWatt.
  • With 150 Ohm and the knee at 6.6 Vrms indicates 290 mWatt.
  • With 32 Ohm and the knee at 3.8 Vrms indicates 451 mWatt.
  • With 16 Ohm and the knee at 2.1 Vrms indicates 275 mWatt.
As you can see, the output power handling capability of an amp will be different depending upon the load resistance. I've often heard people ask what the output power of an amp is. From this chart you can see that a single number to describe an amp's output power can be quite misleading because it changes with the load impedance.

Bottom line here is that both sets of plots are remarkably similar...and that makes me quite happy!

CCIF Intermodulation Distortion

Intermodulation distortion comes from two signals non-linearly mixing, which produces sum and difference signals. In the case of the CCIF IMD (CCIF is a French standards setting organization: International Telephone Consultative Committee), the two tones used are at 19kHz and 20kHz. When these two tones are mixed together, you will get a difference tone at 1kHz. This difference tone will also intermodulate with the two probe tones to produce distortion peaks with 1kHz spacing around the probe tones.

NwAvGuy's measurement gear goes higher in frequency than mine, and my measurement is set to show down to -140dBr (his can go that low, but for whatever reason it was set to -130 for his measurement), otherwise the two plots are quite similar. Differences could be chalked up to very slight differences in system noise and grounding. Again, I'm going to have to chat with him to fully asses my measurements and system performance.

SMPTE Intermodulation Distortion

This is another standardized method for testing IMD, this time from the SMPTE (Society of Motion Picture and Television Engineers). In this case, the probe tones are at 60Hz and 7kHz, with the 7kHz signal at 1/4 the power of the 60 Hz tone.

As with the CCIF test the plots are similar enough to make me happy, but contain enough differences that I want to get a little education from NwAvGuy on the results.

Woot! We're making progress here! The above plots show much progress has been made in getting a headphone amplifier characterization capability in place for InnerFidelity. There's still some room to tweak grounding and cables to see if the measurements can be improved--and that's the name of the game with this type of endeavor--but it seems to me that the measurements are so close that the differences might simply be chalked up to different measurement systems in different environments. Nonetheless, my next step is getting together with NwAvGuy and working on getting any little gremlins out of the measurement system and methods.

I gave Brian the data from these measurements and he will be working on a mock-up of the spreadsheet that produces the graphical datasheets for the amps. And he'll also be working on the program to automate the tests. I suspect we'll see some mock-up datasheets sometime next month.

Damn glad to see progress here!

TMRaven's picture

Woot! Can't wait to see some abominations from some subjectively good audiophile amps. Here's to hoping, anyways. Otherwise it wouldn't be so fun!

Tyll Hertsens's picture
Not really looking to "out" amplifiers ... though there will likely be surprises.
zobel's picture

Good work, looks like you guys are very close to getting the charting underway!

Could IM distortion be measured on headphones as well? Would that test provide a significant amount of information to help illustrate overall quality in headphones?

Tyll Hertsens's picture
I reckon it could. But it would be a long slog back-filling already measured headphones.
br777's picture

i mean if you really do this, start measuring headphone amps, and essentially picking up where NwAvGuy left off, i believe you are going to turn the headphone amp industry upsie-down.. b/c as he has found, you're probably going to find alot of amps are not what they are cracked up to be. At the very least we're all going to find out that we're paying way too much to achieve less than the O2 is already achieving.

frankly I'm all for it. I ditched all my gear in favor of the O2/Odac as soon as it was possible to do so. I'll never look back.

Tyll Hertsens's picture
What do you mean by "where NwAvGuy left off"? Has he stopped measuring amps. (Sorry, haven't kept up with his postings.)
Willakan's picture

NwAvGuy hasn't stopped doing anything as such, he just lacks the time or money to buy and measure lots of (expensive) audiophile amps.

The hope (presumption?) is that you will be in a position to get your hands on a lot more amps, especially the more expensive ones, that might pass through your hands in the course of a review/investigation/sample/whatever. An example would be the Burson HA-160, which features briefly in one of your earliest posts on headphone amp measurements.

As far as your intention not to "out" amps: if you post bad measurements for a given product you'll find those who view comprehensive measurements as all you really need to know about the SQ of an amp (myself included) will do the outing for you.

As far as I'm concerned, choosing an amplifier should be an *incredibly boring* affair. If all the energy that is expended on flogging bizarre electrical monstrosities to those who should know better goes into digital-domain audio manipulation (for psychoacoustically-derived quality improvements and precise colouration to taste), we could see *genuine* improvements in SQ in areas other than transducer design.

frenchbat's picture

Thank you Tyll, this is really appreciated and I hope this will make people understand the hobby better, and by consequence make the offering from the manufacturers better too.

Looks like you got you work planned for the next few years. Keep on the good work !

Tyll Hertsens's picture
"I hope this will make people understand the hobby better, and by consequence make the offering from the manufacturers better too."

My motives are more along those lines. Take data > publish data > everybody learns something new > cross fingers > write about good product (mostly) > products get better. That's what I hope anyway.

Currawong's picture

It can't be emphasized enough that measurements (and science) are to help us understand things better. I think one of the problems that comes about from measurements is a handful of people try to shove them down people's throats (along with the idea that they are wrong for enjoying their hobby with expensive gear) despite only having a casual, and possibly wrong understanding of their meaning.

FLAudioGuy's picture

Tyll & NABlur,
Congratulations on a very fine article. I wish you great success in your newest efforts here and I know it will come.

Never having doubted NwAvGuy's measurements, it is still nice to finally have another open authority with proper test gear to show his integrity stands untarnished. I have said before that he would be practically insane to have faked any measurement. It would have instantly killed anything he was trying to accomplish. Seeing your work-in-progress programming come so very close, where every graph nearly overlays with his is a tremendous thing. That alone should should fend off the rabid nay-sayers and poseurs du jour.

Some people will always "like what they like" and that is ok. I have heard people say that a $300.00/pr Sony APM-22 speakers were the best they ever heard. Dr. Phil says "perception is reality" and that is what theirs was. I never cared for Duntech Sovereigns. Perhaps it was the environments they were in, IDK. I certainly do know first-hand they were very well built with high quality of construction and materials. Headphones will always be the more subjective piece in the chain, as it should be. Headphones/loudspeakers will always have much more distortion, compression, ringing and more than any good amp design. At least with an O2, you can be sure it is out of the way of your music. "Synergy" be gone!

Again, great work guys and looking forward to much more! Cheers!

Tyll Hertsens's picture
Really? People thought he was faking his measurements? Got a linky?

It's patently obvious to me that the guy knew what he was doing with the measurements. His "crusade" attitude might have been a bit touch and go, but I never doubted his numbers.

FWIW, that's the tricky thing as an audio journalist/critic: How to report such that what you do is more constructive than destructive of the things that are good. Focusing on the poor performers doesn't do much for the good ones. I try to applaud the good stuff loudly, and let the measurements and little note in updates speak about the poor ones.

I'll pen bad reviews on things that might be wildly popular though. To bad that happens so often.

frenchbat's picture

Not everyone on HF has taken the measurements very well. By order of appearance the critics were :

1. Who's this guy and how does he dare criticizing our beloved gear ? I think this one is mostly linked to the AMB product that have been reviewed negatively.

2. How does he dare say we can't hear the difference between amps ? Linked to the article on threshold of audibility and such.

3. This thing is just a publicity stunt to get HIS product sold (seriously, some said he was doing all this for the money).

4. Anyway why should we trust his measurements ? After all it's possible it's all fake, and done to draw attention to his blog.

I think you get the idea. If only people could take the best out of it (measurements and basic electric engineering knowledge) and leave the rest...

Willakan's picture

I don't think anyone's denying people hear differences between amps/DACs/ect. They're just for reasons that are tediously obvious to those who don't throw the laws of physics down the nearest well at the first available opportunity :D

funkmeister's picture

If you want to preserve the open-mindedness that should prevail then store up the heart-breakers for one batch on a new way of understanding measurements and position it as a revelation that will have some scratching their heads trying to understand how a set of poor performers can still have a good interaction with the music and output.

Anyway, it's all in the wordsmithing and craftiness... otherwise you'll get into the middle of a flame war that could ruin you in the end. I guess if you were like that then you'd only have a personal blog and not the sponsorship you enjoy today. Overall, you write good stuff and totally not bi-polar in mood, which is why I keep coming back.

Steve Guttenberg's picture
Just wondering, does a headphone's impedance vary with the frequency being played? If so, can that be incorporated into the testing routine? Are other factors changing when playing music that don't occur with test tones?
frenchbat's picture

Tyll will most likely explain it to you much better than me, but yes headphones usually have a varying impedance (except for orthos and some Denon units, AFAIK), check the impedance curve in Tyll's headphones Data Sheets.

As to whether it changes the frequency response, it depends. With a low impedance output (roughly 0 ohms to 10 ohms, the lower the better), changes are minimal to null. With high impedance output, all bets are off.

FLAudioGuy's picture

Yes, just like any electro-dynamic motor system, there is a change of impedance with frequency. Headphones, except a few, are like loudspeakers inasmuch as they have mass, compliance, electrical (magnetic) and mechanical damping (not "dampening"), voice coils, suspensions and motor structures. Add in a closed/tuned air quantity behind the drive unit and you also have pneumatic damping, as well. All of this interacts with the acoustic space the driver/enclosure is located in.

For headphones, the air cavity in front of the driver couples with the air in and around the ear forming an additional compliance and acoustic space. This is why running an impedance plot on a bare drive unit "free air" will show one plot and running the same sweep with the drive unit installed in an enclosure or on a mannequin head will show up differently. You can see the effects of the enclosure on the drive unit and use that information to further characterize a system.

You can see from some plots what current is required to produce any given SPL from a driver/enclosure system. Amplifiers with 0Ω output impedance do not affect the FR of the system and essentially behave as a perfect voltage source. This let's the amplifier maintain proper electrical damping of the electro-mechanical system. When the amplifier has high output Z, then the voltage begins to follow the impedance of the driver/enclosure, hence affecting the FR (and many other points as well, such as time coherence) of the acoustic system. This is the crux of "synergy". When an amplifier affects the natural FR of the acoustic system, then in essence you have an unnatural EQ that is dependent and ever-changing with whatever you attach to it. It is rarely reproducible even if the pairing happen to produce an FR that the particular listener enjoys.

Truly, it is a complex subject but the pro sound industry and much of home audio knows these things, I just wonder why it has not filtered down to the headphone arena. I can't find a QSC, Crown, Yamaha, L'Acoustic, Nexo, Samson, Rane or even a Behringer amp that has a high output impedance. Keep in mind that these amps routinely drive speaker systems with complex impedances between 2-64Ω's to very high levels.

Amplifiers can be tested with both static or dynamic loads. Static loads (16, 32, 80, 300, 600Ω) characterize the amplifiers output into various loads and can show frequency segments where diminished current capability, either in the PS or output devices, exists. These areas would show up in the acoustic space as lower dBSPL. Regardless of whether music (or any complex waveform) or sine waves are used the results would be the same.

I hope this helps. Cheers!

mikeaj's picture

For standardization, repeatability, and so on, everybody seems to test with resistors. After all, different headphones have different impedance vs. frequency curves, so if you wanted to use a "real-world" load (i.e. headphones), which ones would you use? Also note that many headphones wouldn't react too kindly to running an amp's maximum output into them. That said, many headphones actually have pretty resistive impedance, pretty much flat across the audio spectrum, as seen in the headphone data sheets here.

If an amp has low enough output impedance, there shouldn't be much difference when it drives a resistor as opposed to headphones or something else with wilder impedance vs. frequency plots.

As for what happens driving headphones without low output impedance, Benchmark Media had a short white paper on that:

But note that some headphone amps have much higher than 30 ohms output impedance, some headphones (particularly multi-BA IEMs) have much more extreme impedance variations, but also keep in mind that any differences the amplifier imparts on the signal may be masked by the distortion and frequency response variations of the headphones themselves. Also, there are clearly some diminishing returns, so 0.01 ohms is nice but probably not really much of a selling point.

When playing music, the inputs will contain multiple frequencies. There's really much that distinguishes music from test tones, in terms of what audio equipment has to be able to handle. You can see some of those multi-tone interactions in the IMD tests though. Chances are, if all manner of non-linear distortion in the tests are low for the test tones (evidence of high linearity), then the device shouldn't be adding much to music either (which is what highly linear devices do). If they aren't, then it's more complicated to predict what kind of coloration the device in question could be adding.

FLAudioGuy's picture

"There's really NOT much that distinguishes music from test tones..." I am sure what you meant to say, mikeaj

A lot of headphone amps have higher than 30Ω output impedance. NwAvGuy measured the Behringer UCA202 at 47Ω and FiiO E9's 3.5mm jack at 43Ω. Avid's MBox line all have 33Ω, making them practically unsuitable according the 1/8 rule-of-thumb to phones less than 300Ω. Cheers!

mikeaj's picture

That's what I meant to say. Good catch, thanks.

By the way, near the bottom here is a list of the output impedances of a few dozen devices:

It's interesting to see that most of the portable products have lower output impedance, at least out of those. It makes sense, as they're more likely to be used with IEMs, and you don't want to be wasting power. What doesn't make sense is all sorts of desktop equipment having high output Z (that isn't a switchable option).

Willakan's picture

Headphone impedance does vary with frequency, depending on the headphone (some, such as the orthodynamics, exhibiting very little variance). Either way, the amp being tested with a variety of impedances accounts for this.

As for other factors that we can't measure with test tones, null testing shows such factors to be nigh nonexistent, and a null test can be conducted with whatever signal you like. NwAvGuy has an article on this if memory serves.

In short, these issues do not present barriers to the measurements capturing the entirety of an amp's performance.

aravind's picture

Thanks Tyll and Brian for your efforts...these measurements will be of great help to the hobbyists in choosing the right equipments for their headphone system. For the end user,especially for those who don't have the luxury of personally auditioning different gear, these measurements provide a different perspective to understand and compare the behavior and performance of the equipments, in comparison to a reviewer's/another person's listening impressions. Also they provide a much better UNIFORM baseline for comparison than listening impressions can provide which have lot of inter-individual variations making interpretations difficult...
besides..this objective perspective of audio gear adds more fun and involvement to the hobby...i seem to be spending as much time reading and learning as i spend listening to music..nothing wrong with that..and more fun is always(err.. mostly)good..

yuriv's picture

Please measure the headphone outputs of some popular mainstream products--gear that many of us actually have. Those will serve as a useful baseline for future measurements. It will also give us more information on when a portable amp might improve the sound for a particular pair of headphones. There sure are a lot of folks strapping various DACs and amps to their players. In several of these cases, I wouldn't be surprised if the performance is actually worse than just using the phone's headphone output.

It's often hard to get a proper audition of some of the headphone amplifiers reviewed here, and that's why measurements of phones, DAPs, tablets, and notebook computers aren't useless. Sometimes they can help us determine when or how much a DAC or an amp can improve the performance with the particular headphones we intend to use.

Portable players are getting better every year. Some of them are much improved over products that came out just only a few years ago. For the test bench, may I suggest the new iPad and the Samsung Galaxy S III. Incomplete measurements at other sites show good numbers so far.

Tyll Hertsens's picture

Thanks for all the comments above, I've been off on vacation so haven't responded to them all, but, wow, what a great set of thoughts. Thanks!

Brian and I have been talking about the future of the measurement program. Once we get routines that measure the analog ins and outs of balanced and unbalanced headphone amps, we will be moving forward toward measuring the digital USB/SPDIF input performance of the amps. Putting these tests into place will likely take the remainder of this year.

Then we will start developing test routines for portable players, smartphones, tablets, and like devices. So, yupper, we'll get around to it.

FLAudioGuy's picture

Hope your vacay was nice and refreshing! Lots of work ahead I look forward to reading on. This site, NwAvGuy's blog and Dr. Earl Geddes site/forum are my favorite spots to relax at.

Hats off to your efforts here!

aravind's picture

Also, it will be interesting to measure and compare single ended vs balanced amplifiers, look for the claimed differences in noise, crosstalk, etc., and see if they correlate with the much debated auditory benefits/differences

frenchbat's picture

I second this suggestion. That would make for a very interesting and instructive article