February 2018
I’d thought that the measuring of headphones was improving, but some recent developments have made me not so sure. Last year, the introduction of some excellent new lab gear made it possible for experienced engineers and technicians to make more useful measurements than ever before. But the introduction of some very cheap, nonstandard measurement gear has made it possible for almost anyone to make headphone measurements that are less useful than ever before.
On the positive side are the recent introductions of the first truly new headphone-measuring rigs in three decades. For me, the most exciting development is G.R.A.S. Sound and Vibration’s KB5000, a new, far more realistic artificial pinna, or human outer ear, that has allowed me to make far more accurate assessments of, for example, the isolation properties of earphones and earplugs.
G.R.A.S. and two competitors, Brüel & Kjær and Head Acoustics, also launched new ear simulators -- the component that mimics the acoustics of the average human ear, and incorporates a measurement microphone -- designed to deliver more accurate measurements above 10kHz. Because existing headphone-measuring products were designed largely for testing hearing aids, their performance with higher frequencies wasn’t of much concern, and they were inaccurate above 10kHz. New “high resolution” ear simulators, such as the G.R.A.S. RA0401 and RA0402, are designed with additional damping to reduce high-frequency resonance and deliver more consistent results above 10kHz without significantly affecting results below 10kHz. We don’t yet know much about how the performance of headphones in the top octave of human hearing affects their perceived sound quality, but with this new equipment we can learn.
The negative developments in headphone-measuring gear were summed up by Head-Fi founder Jude Mansilla in his presentation at the recent Association of Loudspeaker and Manufacturing Acoustics (ALMA) International Symposium and Expo (AISE) 2018, which took place in Las Vegas in early January, two days before the 2018 Consumer Electronics Show. In Mansilla’s talk, “Audio Engineering as Consumer Content,” he lamented the fact that the availability of cheap measurement microphones and free measurement software such as Room EQ Wizard has inspired many headphone enthusiasts to jury-rig their own measuring systems.
Most of these systems are little more than a consumer-grade measurement mike surrounded by a bit of foam and mounted in an improvised apparatus to hold the headphone under test. Little or no attempt is made to replicate the acoustics of the human ear according to the international standards adhered to by the companies that make professional headphone measurement gear. As a result, measurements taken with these systems look radically different from measurements taken using professional gear. The charts are wildly inaccurate, and impossible to compare meaningfully with anyone else’s measurements. Yet some enthusiasts are drawing conclusions about headphone performance based on these measurements.
Another development that had some audio professionals cringing was the introduction of the Earphone Audio Response System (EARS, $179), a new headphone-measuring rig from miniDSP, best known for its affordable digital-signal-processing (DSP) audio processors. EARS appears to be little more than a couple of cheap measurement mikes mounted in a couple of plastic ears. Considering that a comparable professional product incorporating ear simulators and standardized simulated pinnae would typically cost around $12,000, it’s highly unlikely that the EARS conforms to any established standards, and it doesn’t seem to have been designed by anyone familiar with those standards.
miniDSP seems to intend EARS primarily as a way for enthusiasts to fine-tune physical mods or the equalization settings they use for their headphones, and it might be somewhat useful for those tasks -- while it doesn’t appear to be useful for absolute measurements, it might help gauge the effect of a change. But on the EARS webpage miniDSP promises that it “brings headphone measurement capability to ‘the masses.’” Considering the increasing popularity of amateur headphone measurements, it’s hard to believe that this device won’t be snapped up by enthusiasts -- even a few reviewers -- who, unfamiliar with the established standards of headphone measuring, will likely proclaim it to be “not as good as a pro system, but close enough.”
Before I bought my headphone-measuring gear about six years ago, I experimented with making my own, using techniques similar to the ones Mansilla described. I can tell you that the results of such efforts were far from “close enough” to professional gear. They weren’t even remotely close.
In his presentation at AISE, Mansilla begged headphone manufacturers to publish their own measurements when they introduce their products, and ended by saying (I paraphrase) that if manufacturers don’t publish measurements, consumers will. But in talking with fellow attendees afterward, I wasn’t convinced that most of them got the message.
Meanwhile, I strongly encourage SoundStage! readers to ignore any headphone measurements produced using nonstandardized test gear. Not only are the results misleading, the expertise of those using this homemade gear is questionable, to say the least. I’ve been performing audio measurements professionally for more than 20 years, during which I’ve learned to measure speakers, subwoofers, amplifiers, and, most recently, headphones. Each of these skills was added to my repertoire through laborious experimentation, countless late nights wrestling with finicky analyzers and digging through decades-old research, and long discussions with fellow professionals -- and I know that the handful of professional audio reviewers who do their own measuring have all endured the same test. In my experience, it takes any technician about two years of performing a particular measurement before he or she becomes truly proficient at it.
The bottom line is, owning measurement gear -- even the very best professional, standardized measuring system -- doesn’t make you an expert measurer, any more than owning a trumpet makes you a trumpeter.
. . . Brent Butterworth
brentb@soundstagenetwork.com