Yay!!! Measurements!!!
Okay, they're not complete yet, but it is automated! Brian (NA Blur on Head-Fi) finished his first pass as best he could without a running system in front of him, and we met Wednesday of last week to go over it. He structured the code and subroutines of the program that runs the test, and built the spreadsheet template. Now it was handed off to me to get it to run on the Audio Precision tester.
Brian did a great job, but there were all sorts of limits to what he could do without the hardware in his home to run the test, so there were plenty of bugs to fix. But there were two real problems facing me as I got it up and running. The first was getting the THD+Noise vs. Output Voltage to stop when it reached 1%. Try as I might, I just couldn't figure it out, so I called Audio Precision tech support for help. OMG! If only everybody had tech support this good! First, you don't need to pay for it, and second, the guys answering the phone are amazingly good at helping...AMAZINGLY GOOD. They told me how to go about it, and in 20 minutes I had the test up and running. The second problem was a bit weirder having to do with the FFT sampling rates. The tech support guys again came to the rescue and talked me through panels that I didn't even know were on the machine, and again, the problem was solved within minutes.
I won't bore you with the minute details, but by Saturday evening I had a running test. Now, this isn't the complete test. My goal was to get the program running well enough to take all the graphs and spit them into the Excel spreadsheet with one load resistance, and then to measure a sampling of solid state amps to determine if the graph ranges were reasonably set. By Sunday afternoon I had measured seven amps, and then last night and this morning I measured them again to determine if it was reliably getting the same results. The bottom line? I think it looks great.
So, for your viewing pleasure, I've created this .pdf booklet of all fourteen datasheets, two for each amp. I'm somewhat familiar with the various measurements, so I'll talk you through a few of them. I must admit I'm not nearly as fluent with interpreting amp measurements as I am with headphone measurements. I can help you with the basics, but I'd love to hear from some of our more technical readers in the comments about what you see in these data. As a refresher and guide, you may want to go back to this article NwAvGuy wrote for InnerFidelity to help me develop the tests. He's got some in-depth information there.
The Graphs
Basic Parameters - NwAvGuy suggested to me that all measurements should either be at unity gain or with the volume control at the 12 o'clock position. I decided to use unity gain as it would make the voltage levels the same on all measurements and would aid in direct comparisons. I picked 0dBu (0.774Vrms) as the reference input level as it's a voltage that should work on most amps, and is a hot enough signal to get rational noise ratios. I may have problems with some portable amps at that level. If so, I'll may make a second test routine for portable amps that uses a lower voltage standard.
Frequency Response - FR is measured from 10Hz to 100kHz. I'm thinking about extending this to maybe 300kHz, but unfortunately Excel will only do log scales in powers of ten and would cause the graph to go out to 1mHz, which would reduce the size of the plot in the graph. Quite a few of the amps measured were flat to 100kHz, so it would be nice to extend this plot.
THD+Noise vs. Frequency - This is a measure of harmonic distortion at various frequencies. The AP starts this measurement at 20Hz, and once it reaches 7kHz, the third harmonic is at 21kHz and needs to stop as the AP has a filter to limit distortion products above 20kHz, so measurements beyond 7kHz become misleading as it filters out the third harmonic.
Noise Spectrum - This test is taken with no signal present and shorting plugs inserted into the inputs of the amp. It shows the spectral energy content of the amp's self generated noise.
THD+Noise vs Output Voltage - This shows the amount of distortion present as the output voltage rises. Solid state amps typically clip hard at some point and the rapid rise at the right-hand side of the plot is an indication of this clipping distorting the signal. You can look at the right-hand near vertical clipping point and basically see how much voltage the amp can drive. When you see the left-hand part of the curve descending linearly, you can assume the measurement is limited by system noise. As it begins to curve, you can assume distortion is beginning to become more dominant.
SMPTE IMD - (Oops, sorry for the misspelling on the graph, still lots of things to tidy up.) With this test, two tones are played into the amp: 60Hz at 0dBu and 7kHz at 1/4 the level. You can see these as two spikes in the curve at those frequencies. These tests are used to look at non-linearities in the higher frequencies of the amp that cannot be seen in the THD+noise measurement due to the bandwidth limitations I mentioned in the THD+Noise vs. Frequency section above. Basically, what you're looking for is a clean graph with as few spikes as possible and for the noise floor to remain constant. Remember, what you see in the "Noise Spectrum" plot will also appear in the SMPTE IMD plot.
CCIF IMD - similar to the SMPTE IMD plot, but with more sensitivity to errors in the higher frequencies. Here again, you're looking for the noise floor to remain flat, and for intermodulation products to be as narrow and short as possible.
Single Point Measurements - The small table in the bottom right corner are for single point measurements and are fake. This table will be filled in on the next iteration with data measured with four different loads (16, 32, 150, and 470 Ohms).
The Amps
I've started reading seriously John Atkinson's pre-amp measurement sections and NwAvGuy's site to try to become more expert at interpreting the measurements, so I'm not going to attempt a blow-by-blow analysis of the measurements here, but I will make a few comments. I'm going to preface them, though, with the observation that almost all these amps are ruler flat from 20Hz to 20kHz. Headphone amps are MUCH closer to each other in sound quality than headphones. The best measuring amp is not necessarily the best sounding amp. I think the most important thing amp measurements can do is isolate amps that might have glaring issues. After that, I reckon we will all be on a learning curve here to find some correlations between measurements and sound quality.
NwAvGuy's O2 as built by JPS Labs is clearly a top performer. This amp was measured with AC power to put it on a more even keel with the other amps, but it still showed an extraordinarily low noise floor and very good THD measures. THD+noise and Noise Spectrum plots are stunningly low. The O2 did suffer some spreading of the CCIF IMD peak at 20kHz, where Meier Corda Rock had the best results of the group to my eyes.
I felt the repeatability of the test was pretty good--I'm very happy to see that--so I thought I'd test the sensitivity a bit with the HeadRoom Desktop. I was using it with the Desktop Power Supply on which it stacks. I knew that stacking them does allow a little AC cross-talk into the amp as the transformer radiates some AC into the amp. So for the second test I moved the Desktop Amp from atop the power supply and separated them by a foot or two. You can see in the two Noise Spectrum plots that the second test has virtually no power supply 60Hz harmonics. You will note a little blip at about 43Hz. This blip can be seen in all plots with somewhat differing amplitude. I think this is something coming from the measurement system and I'm going to have to do some gremlin hunting to see if I can find its source. Ah, well.
The Lake People G109 seemed to have quite a bit of CCIF IMD distortion around the 20kHz peak. The Burson HA-160D had considerable noise. It seemed to me the O2, Corda Rock, and Butte fared very well indeed.
What do you see?
Update Update
I did measure some headphones this month, but felt I should get this post done and up as quickly as I could, so headphones measured this month will appear in the December update.
Upcoming headphone reviews will be the V-Moda M-100, Sennheiser Momentum, and a review of the Sennheiser HD580/HD600/HD650 family. I've also started receiving a bunch of high-end electrostatic amps and headphones for a large comparative review I'm planning for right before Christmas.
Have a great turkey day!
Comments
This is great ! I'm looking forward to seeing the tests.
Just 2 quick questions before the pros come to comment : All the amps are given for .2 ohms output impedance, which I don't think is the case. Is it something you're planning to iron out later on ?
Also, are you planning to use a real load for your measurements ? (I'll go back to your previous articles in the meantime, maybe you've already addressed that question)
The numbers on the bottome are fake...just place holders for later.
Just resistive loads. Better I think for long term consistancy of tests. It's likely that as time goes on and I get better at looking at these things that when I see something unusual happening with the amps I'll do some specific test to check things out that might be part of a review article, but not a part of the standardized test.
That's what I was thinking for the output impedance. Last details anyway, the whole thing looks better and better.
As to the real load thing, it's probably a bit nitpicky, but that helps in some cases I think. Here is an interesting example that compares an O2, a Leckerton UHA6S MKII, and an Iqube : http://scientistsaudio.blogspot.jp/
That doesn't show particularly interesting IMHO: unless I'm much mistaken it's just showing the effect of the O2's marginally higher output impedance versus some of the other options with some very low impedance IEMs.
That was an example Willakan. Nothing against the O2 or the Leckerton in particular, but I have no particular interest in the flat lines that are sometimes used as marketing material. I'd rather know what's the worst possible case's results.
Granted, it doesn't not always translate into audible effects with headphones, but some of them have wild impedance peaks that might change the FR (I'm looking at you HD800). And multi BA IEMs are affected in more ways than just the output impedance.
However, I'm not sure a difference of only .4 ohms (assuming the O2 is .5 ohms and the Leckerton .1 ohm) can translate into almost 1dB of variation in the FR (as seen for the O2 with the UERM). Especially with such low power, since the test was reportedly done at 100 mVrms. So there must be something else at play here.
BA IEMs have an enormous output variation across the FR range, including dropping into single-figure impedances, so with the O2's output impedance at 0.5ohms, and the IEMs being extremely sensitive, I don't think we're seeing anything out of the ordinary, although I haven't done the maths - couldn't find an impedance curve for the UERM...
When you're talking about something like the HD800, the damping factor with the O2 is so high throughout the impedance range that you're not going to get a problem, period, let alone an audible one.
Jeez, did I say there was something wrong with the O2 ? Don't think so either.
I was asking Tyll, whether he'd do some loaded tests, because loaded tests can show differences. I prefer to see what can actually happen, instead of dreaming on flat lines. Fact is, Multi BA IEMs are part of the equation in the actual market, and they're not going to disappear.
As to the UERM, they are given for 21 ohms at 1kHz. Assuming they dip until 8 ohms like the TF10, the difference between the FR for the given 100 mVrms output is only about .3dB, if my math is right.
My points are:
1) The results don't show any real "problems" as you implied they did in your first response.
2) There's nothing there which couldn't be predicted mathematically without going to all the hassle of running FR tests.
Never mind...
We're going to have to agree to disagree here.
1. I haven't implied anything, but the fact that a real load can show a different result compared to unloaded. I actually own the Leckerton and the O2, which I both appreciate. Doesn't mean I don't want to know what are their limits and understand the reasons behind it.
2. Mathematically, the results are lower (.3db) than the graph results (.7db), assuming we take the output impedance as the sole reason for this variation. That leaves half a dB unaccounted for. Unless you'd to go there and show how my calculation is wrong (entirely possible, but show me the money first).
3. Mikeaj is also asking for real load results down this very page. I don't see you jumping on him.
Your move.
I feel that simply knowing the output impedance is enough. As was mentioned, after that it's all about the headphone's impedance plot. Someday it might be nice to have the ability to look at a headphone's FR and then dial in an output impedance to show a modified FR, but again, it's on the headphone side more than the amp.
Anxiously waiting for the m-100 review and charts, interested to hear your opinion on them. Also happy you're able to give a review on the Momentum's, they seem like a cool can. Hope the sound is as good as the looks. 
Thanks for all the hard work you do.
Great to see the measurements taking shape! I'm pleasantly surprised by just how all the amps turn out. Sure, some of the amps are pretty mediocre, but there's no real glaring failures. Admittedly, the tube amps were conspicuously absent, but nobody expected them to do well in a bench test anyway 
Also, on the subject of the O2: to make the O2 win at every benchmark, simply replace the NJM2068 in the voltage gain stage with an NE5532: the latter chip isn't the default option for battery life reasons, but it has much lower HF IMD.
Not that anybody's going to hear this anyway, but it's nice to see the option is there!
I think I've got some 5532s here. I'll give it a shot.
Yeah, tube amps are going to need a different test with different scales.
Out of curiosity, is that 43Hz blip there when testing the AP into itself? I don't envy your task, trying to do your best for these precision measurements.
By the way, JPS Labs is a different operation from JDS Labs.
Interesting tidbit about AP tech support. I think this is what happens when you don't have dedicated support staff, and it's the developers that do support as part of their job. Either that, or they're trained very well or have a lot of experience.
I think 100 kHz is enough on the frequency, unless some people are using headphone amps for something other than audio purposes... 100 kHz is enough to listen to whatever bat noises are on 192 kHz recordings, anyhow!
I did a loopback test a while ago and it looked good. But that's surely where I'll start my Gremlin hunt.
Yeah, I figured out I was sideways on the JDS name about half way through, but didn't clean it up. I'll get it right by the time we get to publishing final measurements.
Great to see progress and all looking good, but I'm a little bit disappointed because THD+N vs Output Vrms graph is still incorrect for O2 amp, misleading for people that are not familiar with O2 design (or doesn't understand it) and misuse of the amp ie. in contrary to what NwAvGuy warns several times at his blog. Clipping point at ~2.8Vrms as it could be seen on your graph is not correct! ~4.6Vrms on battery and ~7.1Vrms on AC power are correct values, so much higher than your measurement suggests. Actually, with even lower volume pot position then you selected you would get event lower output Vrms clipping point. As O2 has no voltage gain of output stage (Av=1 after volume pot) it basically means that volume pot attenuate output signal which is in contrary to input signal attenuation common to all other amps. With correct use you could "reach heaven" but do it wrong way and it would perform as worst amp ever. So that's the reason why everyone should be extra careful when using O2. IT'S REALLY GREAT AMP BUT IT'S NOT AS PLUG&PLAY AS OTHER AMPS. Measuring O2's THD+N vs Output Vrms like you did is like measuring max power on the wheels of a car with the handbrake engaged!
Cheers!
Yup. You're right. I think the HeadRoom Portable Micro Amp is clipping due to a similar problem as well. I think when I do that test I'll have to readjust the volume to full volume.
Exactly, full volume only for THD+N vs. Output Vrms measurements.
I'm thinking about extending this to maybe 300kHz, but unfortunately Excel will only do log scales in powers of ten and would cause the graph to go out to 1mHz which would reduce the size of the plot in the graph.
Hint: Don't use automatic scaling in Excel but fixed (min 10 and max 300000) and the graph would stop at 300kHz.
That's still useful to know though. Sometimes, you want the knob to stay at one position and control volume e.g. digitally. Just a remark on the page whether the useful voltage range stays the same with the pot positions or not is welcome.
I'm thinking that there's room for another graph on the page and I'm considering additional tests.
I'm thinking either crosstalk or transient intermodulation distortion tests. Bothmight be telling of design quality. Any suggestions?
NwAvGuy suggests having Phase response superimposed on the frequency response plot. I think I'll do that.
Very promissing results Tyll!
Superimposing the FR phase with the magnitude would be nice. Maybe a light gray much like what its done with the un-averaged headphones FR magnitude plots.
I'd like to think that very few amps have problems with slew rate limiting: crosstalk, on the other hand, could be quite interesting. One measurement I'd particularly like to see is crossover distortion: I suspect that could be very interesting...
How would you like to see crossover distortion displayed?
Crosstalk often varies with frequency, but a single number (using a certain load) is fine by me. I mean, most amps are probably excessively good at crosstalk. Speaker room / position interactions and resulting L speaker -> R ear and R speaker -> L ear crossings are slightly time delayed, yes, and filtered by the shape of the ears and head, so that's not really crosstalk like you'd see in an amplifier... but come on, the levels we're talking about are many magnitudes above the levels of crosstalk we see in amplifiers. A little L signal in the R ear is not going to kill you. I don't think you need to dedicate a plot to that.
I'd like to see an IMD test (maybe a different one than SMPTE and CCIF, why not?)) done with a load that's not just a resistor. Maybe use actual headphones or just an RLC network. Actually, you can model some headphones' weird impedance graphs decently with R and L in series with R / L / C all in parallel. e.g. Sennheiser HD5xx and above, with the resonant bass hump. But is there anything here going on that's interesting other than the amp output impedance interaction? I don't know.
Particularly when you include tube amps, it may be interesting to show the harmonic distortion separately: for one load and output level, show 2nd / 3rd / 4th / 5th harmonics vs. frequency. For many of the other amps, each curve is just going to be low anyway (none are going to be higher than the total THD+N), probably not that interesting...
I'm pretty sure all modern amps have no or very little TIM - just because they don't overshoot and don't soft-clip. It's very correlated with step response ringing, slope or sagging.
Currently, most amps are stable, non-ringy, not overdamped and non-clipping enough to not have much if any issue with this before they have one with normal IMD and/or THD+N.
Could be quite useful for headphones and IEMs though, these are far further from perfect there. Would be a good measure of "confusion" or "compression".
Of note, CCIF-ID test already measures some of dynamic IM behavior.
EDIT: Meant to be a reply, screwed up...
Hmm: I quite liked NwAvGuy's approach, superimposing the distortion waveform over the sine wave used for testing, but that doesn't turn out much in the way of quantifiable data.
An alternative (and possibly easier approach) would be using the ITU-R 468 noise weighting standard, applied to a 1khz tone with the fundamental nulled out. This emphasises any problems with crossover distortion.
See:
http://en.wikipedia.org/wiki/ITU-R_468_noise_weighting (esp. "Present Usage" section)
http://www.lindos.co.uk/test_and_measurement/SOURCE=Articles/SOURCE=Arti...
Really looking forward the M-100 review! Appreciate your work.
Getting closer and closer to finally being able to see some meaningful and comparable amp #'s.
I'm stoked in seeing this.
JJ 
loved the T-shirt
Site Map / Direct Links| Headphones |  |
Headphone Amplifiers | |
Portable Devices Home Devices | |
Blogs Guides Show Reports | |
Community Buyers Guides | |
News Resources |
