CES 2016 Showstopper: Sennheiser Demonstrates 3D Audio on Headphones and Describes First Commercial Uses

Sennheiser's front-of-booth display was all about 3D audio on headphones. Sebastian Chafe, Sennheiser's Strategic Innovation Manager, gave me the full rundown on what might be their first commercial immersive audio application: Concert simulation for DJs.

The idea here is that DJs will be able to produce their music more efficiently if they're able to hear the concert as if immersed in the sound just like their audience will be at the show. Sennheiser is able to produce this effect by recording special calibration signals played through the concert hall speakers at each particular venue with a special multi-capsule 360 degree microphone. The sound of that particular venue can then be computer modeled, and any music or audio can be played back and heard on head-tracking headphones as if you are there.

One of the demo systems in the booth allowed you to hear music and then switch between three different dance clubs in Germany. Though frontal localization was still not perfectly convincing, the head tracking headphone did provide a pretty sufficiently immersive experience. Take this one step further, and you can imagine DJs producing virtual concerts where people might listen virtually at their venue of choice, all from the comfort of home.

But...as good as it sounded and as interesting as the idea is, that's not what really stopped me in my tracks and had Sennheiser in the back of my mind for the rest of the show. The question burning in my mind was: Why was Sennheiser showing prototypes front and center? I can't remember them doing that ever before. (Though one might argue the new Orpheus is basically a prototype.)

I'll quote just a few words from Sebastian Chafe in the video below as he begins to introduce me to the 3D audio demo:

"These are new concepts in the prototype phase that we're showing to get some feedback on our ideas; show what we're working on; and kind of get people excited about immersive audio...if that makes sense."

Well, no, it still doesn't quite make sense. Feedback on ideas and showing what your working on can be done in more controlled and useful ways than the CES mad-house. But that last bit about "get[ting] people excited about immersive audio", that does make sense for CES. But still, why do it in the prototype stage? Why is Sennheiser using one of the most visible PR events of the year to get people excited about something it doesn't even sell yet? Why the urgency?

Let me refer you to this article containing my thoughts about the purchase of Beats by Apple written one and a half years ago. Basically, I think Apple inked the deal because with the acquisition of Beats, Apple now has all the pieces in-house to develop the technological ecosystem to commercialize immersive audio.

3D audio on headphones is technologically very complex. For example, with 11.2 surround you will need to perform head related transfer function DSP from head tracker and HRTF data set inputs to create 11 virtual speakers in the headphones. How do you do that without latency problems? Where does the processing take place—in the hand-held device, or in the headphones? How do you get that much bandwidth to the headphones? Video displays have standards for manufacturers to build to, but no such standards exist for aural displays. These are tough problems.

The new iPhone7 is rumored to be without a 3.5mm headphone jack. Beats will almost certainly be producing Lightning connector headphones for roll-out with the new iPhone. That's going to shake things up a bit, but it's not a terribly difficult or time consuming development. I think it borders on an intentional misdirection when compared to what they're more likely working feverishly on: Apple's soup-to-nuts immersive audio technology platform that will solve all those pesky problems. They don't have to wait for industry standards to develop, they are their own standard.

I reckon this is a big enough problem that it will probably take them 3-5 years to to put together a hardware solution—but remember, they're already a year into it. We'll have one little blip here soon where Beats will produce Lightning cabled headphones and maybe some wireless stuff. Maybe we'll see a line of Apple branded headphones. But the big change, the one that is going to blow peoples mind, will trail by a year or two. If I had to guess, I'd say Apple/Beats first 3D audio headphone product will show up in 2017, but the big roll-out of crazy stuff will be 2018.

Obviously, there a lot of pulling numbers out of my butt going on here, but there is a point: Immersive audio is coming, and Apple has skewed the odds in its favor. Sennheiser (and Harman and Sony, for that matter) know it and need to get things moving along. And that's why they produced a technological demo at CES, to wake people up and get their creative and competitive juices flowing. The only thing that's going to stop Beats >60% market share of premium headphones from getting bigger is a lot of collaborative hard work on the technological platforms and aural display standards needed for immersive audio on headphones.

Now, I don't want to come off sounding like I think Sennheiser is in a panic about this. The CES display itself is evidence they are already working very hard on the technology, and, no doubt, have a larger strategic plan to get the job done. Here's a bit more evidence that not just Sennheiser but the industry as a whole are surging towards solutions: The Audio Engineering Society International Conference on Headphone Technology. As only one of four currently scheduled conferences this year, the AES leadership obviously believes smart headphones demand attention. From their conference description:

Therefore, this conference will focus on technologies for headphones with a special emphasis on the emerging fields of Mobile Spatial Audio, Personal Assistive Listening, and Augmented Reality. This conference will assemble scientists, developers, and practitioners who are involved in any head-worn hearing technology, be it in theory, technical design, application or evaluation. The conference will enable an interdisciplinary dialogue across the headphone and hearing aid industries.

Want to know something else that's cool? Because of all this focus, attention, and sense of urgency around smart headphones, I've been approved to go and report on this conference! You haven't heard the last from me on this subject, that's for sure. I will, however, wrap up this post for now, and invite you back to planet Earth with Sebastian Chafe's description of immersive audio for DJs.

(And I want to know why DJs get this stuff first? Those Germans must love their dance music, I guess.)

Jim Tavegia's picture

I just completed my first Decca-Tree concert recording for a school system and the reason was to be able to provide a better spatial representation of the band in their 180 degree semi-circle layout and recreate what the audience heard at the concert. The full Left-Center-Right perspective was interesting and all who heard the recording liked it. We have all heard great Decca-Tree recordings of great orchestras over the years.

This could be interesting with a single array of mics of whatever one wanted into a multichannel recorder, played back in in a Home Theater setting and changed to the listeners liking. You could record the music group in a circle and record in 360 degree perspective and recreate the dance floor with the listener in the center perspective. This could be very interesting.

I may not mess with stereo snobs like me who just want to hear and record more, but for the masses this could be a home run for Sennheiser. I use a lot of their recording mics and cans love them.

Hi-Reality's picture

I have followed Tyll and a few other experts for quite some time in preparation of my Hi-Reality project.

Tyll's reported topic, his reflections and speculations here are significant stuff when it comes to the future of sound capture and reproduction (stuff that have been, in general, broadly neglected by the audio industry and also the typical audiophiles).

Regarding Apple, I think Tyll's speculation makes sense. Apple often meets industry standards but sometimes makes industry standards. True immersive and head-tracking 3D Audio would be one of those things Apple could refine and make de-facto standard for its ecosystem of devices and services. Why? well, for one thing, because this is an important part of the Virtual Reality experience. And this one thing isn't yet quite solved by the Facebook/Oculus Rift folks.

Babak, Founder
Irvine, CA

Ps. Looking forward to Tyll's reflections after:

2016 AES International Conference on Headphone Technology
Dates: August 24-26, 2016
Location: Aalborg, Denmark