Last fall, I got an email from my friend Galen Brandt, connecting me with the great composer and producer Richard Einhorn, whose amazing Voices of Light I saw at BAM back in 1995. In recent years, Richard suffered an extreme hearing loss, losing almost all of his hearing in his right ear and most of his hearing in the left. Richard hadn't abused his hearing; the problems literally ocurred overnight, and while the cause isn't precisely known, it's likely that it was some sort of viral infection. Richard made a terrifying simulation to show what his world now sounds like; you can hear it for yourself at about the 5:45 minute mark from his October appearance on Jon Schaefer's excellent WNYC show Soundcheck (which I listen to regularly) here:
Richard now wears an earplug in his right ear to keep from triggering the distortion in his damaged cochlea, and a powerful and expensive hearing aid in the left.
Richard is incredibly affable and articulate, and, as it turns out, tenacious--he has since become a strong advocate for improving the quality of life for those with hearing loss. He has followed the traditional and non-traditional routes through audiologists and insurance companies, and, as an artist, has also found the hearing assistance systems in many theatres and other similar venues to be sorely lacking. To address that issue, he has been strongly advocating for the increased use of hearing loops in theatres and other public spaces, since these loops deliver sound directly to hearing aids via a magnetic field; the hearing aid, which has been customized for the wearer, then takes the direct signal and delivers it to the listener. Richard first heard one of these loop systems at a touring production of Wicked in DC; he has since been on a crusade about the issue, which is well covered in the Soundcheck piece embedded above, and in this New York Times article. (I also put Richard in touch with my friend Tony Meola, the sound designer for Wicked.)
Richard contacted Galen looking for help in another battleground for those with hearing loss--the restaurant. Restaurants can be a challenge even for those with excellent hearing; reverberant acoustics, lots of people talking, and background music can lead to a difficult environment for anyone. For me, the challenge in these environments is working hard to focus on my dinner companions and not be distracted by being drawn into the conversations around me; I can't imagine what this environment must be like for those with hearing loss. There have been a few commercial systems made to address this problem, but, according to Richard, the best of those systems isn't very good, and is currently not being made; the rest aren't worth the money. His hearing aid alone (especially listening in mono, where the cocktail party effect is negated) is simply inadequate in this difficult environment, and Richard's homebrew solution of a high quality directional microphone and an earphone is cumbersome, and makes him look like a reporter doing an interview, which isn't really conducive to comfortable, casual conversation. Richard had been assured by an array of experts that a better solution to this situation was simply not possible.
This was surprising to me, since I've spent many years designing and working with complex sound and other systems; it seemed to me that the technical solution was pretty straightforward (while the real challenge is making such a solution affordable and useable by non-technical people). I suggested that simply putting mics as close as possible to each talker and using a high quality automatic mixing system should do the trick. An automatic mixing system or "automixer" automatically raises (or maintains) each talker's volume when they are speaking, and lowers the volume of the other channels, drastically reducing the background noise, which all leads to increased intelligibility (and better gain before feedback and a number of other technical advantages). While these systems aren't really up to the creative job of mixing something like a band or a Broadway musical, they are great for panel discussions, TV broadcasts, and other similar applications. In my industry, the best known automixing system has long been made by Dan Dugan--when people want one of these they often ask for a "Dugan". I've met Mr. Dugan a few times over the years, and he seemed like a nice guy who is interested in solving challenging problems, so Richard contacted Mr. Dugan and within a couple weeks we had a shiny new Dugan-MY 16 digital auto mixing card and a date scheduled for a practical test at CityTech (where I teach).
Proof of Concept
For the test, I used a professional Yamaha 01V96CM mixer, fitted with the Dugan-MY 16 card, four hard-wired Sennheiser MKE2 omnidirectional lavalier mics, and a laptop to control the Dugan system--about $7500 worth of stuff. One of my students, Adam Agarrat helped Richard and I set up, and my colleague Charles Scott came in to be one of the "talkers".
For Richard to listen, we used his Ampertronic CLD1 Loop Driver:
We made a temporary loop inside which Richard sat:
The Dugan card is controlled by a little Java program connecting over a network:
More pictures here.
I also set up a CD player and a Meyer UPA-1P speaker playing background music to simulate a restaurant, and the space we chose is reverberant. The result? When we engaged the Dugan automixer, the intelligibility increased dramatically. Richard absolutely loved it.
The Dugan system balanced everything amazingly well. So well, in fact, that when Adam and I had a side conversation, it was literally impossible for Richard to concentrate on his own conversation with Chip. With the Dugan system's mixing interface, Richard was able to simply de-weight talkers not involved in his conversation. It seems that no matter how good the automixer, giving the listener (Richard in this case) at least some control over the mix is critical, although this professional interface, obviously, is likely too complicated for use by the general public.
Richard also brought along a loop listener, which allows the use of headphones to monitor the signal. Though Richard referred to this as a high quality send, listening for myself I was surprised at how noisy the system was, with lots of background electrical noise and hiss. Additionally, we discovered that the loop sender must have quite a bit of audio compression in it, since Richard, who has years of critical listening under his belt, didn't even notice some of the things (turning on and off phantom power to the mics, etc) that really bothered me in monitoring the system directly out of the mixer feed (upstream of the loop driver).
Probably the most amazing thing to me was how Richard seemed like a different person (himself) with the system on, but as soon as we turned it off, he immediately became less engaged, and conversation with him took a lot more energy. So, as a proof of concept, this auto-mixing system came through with flying colors, and the next step was to try it in a real restaurant environment, which we did last Tuesday.
Real World Trial
Richard had also contacted Howard Kaufman, the New York rep of the well-regarded wireless audio system manufacturer Lectrosonics, and Howard offered to bring a portable system to Richard's excellent restaurant choice: ICI restaurant here in Brooklyn (Richard knows the owner and made arrangements so we wouldn't be freaking anyone out showing up with a rack of equipment). The all-Lectrosonics system was based around their ASPEN SPN1624 DSP automixer, with feeds in from LMA Digital Hybrid belt pack transmitters and HM172 headworn microphones. Monitoring was done through Quadra Wireless multi-channel IEM (multi-channel in-ear monitor) receivers. Howard, Richard, and I were joined at dinner by my friend Ellen Juhlin of Meyer Sound, who just happened to be in town on vacation.
Interesting, aside from the rack of gear, the system was actually pretty unobtrusive (all of us have the mics on in these photos):
(Sorry for the photo quality, even with my Nikon D7000 it was hard to get shots in that low light; more here.)
The system worked out great, and again, the automixing was the key to success. When we turned it off, it became very hard to pick the voices out of the background noise; when we turned it on, it gave better intelligibility than if we were listening acoustically without the system. We also experimented with the automixer's noise reduction function, but that seemed to have a hard time with the dynamic background of the restaurant (noise reduction generally works better on constant, unchanging sources of unwanted noise, like airplane engines). We were using quite a few wireless frequencies in our restaurant test, and we got a bit lucky since the frequencies Howard picked worked well in our location. That wouldn't always be the case, but of course using hard-wired mics and earphones would solve this problem.
Being sound geeks, we all wanted to listen to the feed (which wasn't really necessary for the talkers), and with headphones on, we did look a little funny (see the first photo above). Maybe because it's in NYC, but no one really seemed to care what we were doing. A couple sat down next to us and I told them what was going on and they thought it was really cool.
Interestingly, with the omnidirectional mics, the automixer mixed the waitress just fine, so hearing her without a dedicated microphone was not a problem. Richard did find that he needed a mic for himself to make it comfortable to talk, and Howard provided a remote button panel that allowed Richard to mute talkers having a separate conversation. Even when eating ICI's delicious pork chop, I didn't find the tiny mics to be in the way at all, and, amongst the background noise, chewing sounds--even with crunchy foods--were not a problem. Wearing them was a minor inconvenience that was well worth it to fully include Richard in the conversation. All in all this system too worked great.
Where Do We Go From Here?
So, we have I think now definitely proved that there is indeed a technological solution to this problem. If you are rich and wanted to do this for a night, you could probably rent this gear and a technician for probably less than a grand (and it's actually a pretty fun and intimate dining experience if everyone wears headphones). If you're technically inclined, you could buy this gear and operate it yourself, and even $7000-10,000 isn't (relatively) a huge amount of money when a hearing aid can cost $3000-4000. But obviously, this is not a solution for the general public. But while we were using high-end, heavy-duty, expensive professional audio gear, a manufactuer could likely integrate all this stuff into a compact system at a much, much lower cost. But the price would likely have to remain somewhat high, since the level of technical support, at least at the outset, would have to be pretty high as well, and the interface for the selected muting of talkers would have to be thought through in a way that would make sense to non-technical people.
We have just started the process, but since the market for hearing assistance technologies--especially with an aging population--is massive, I'm hopeful that some one will pick up this ball and run with it!
John Huntington
