VR audio started with military simulations such as DARPA’s SIMNET, the $240 million project in the 1990s that pioneered VR and massively multiplayer online games. At a time when graphics processors were still low-resolution and very expensive, designers discovered that they could trade off high-quality audio for visuals to enable the immersion their virtual environments required. Dr. Robert Jacobs, a VR audio expert, said, “We discovered that a dollar’s worth of audio was worth ten dollars’ worth of almost anything else.” In contrast, audio for films at the time was often under-appreciated, as represented in the famous quote, “They can fix the sound in post-production.” Jacobs pioneered immersive techniques to heighten impressions of danger, including creation of high-resolution sounds appearing behind the player as well as the addition of subliminal tiger growls.
“Realistic audio is critical to VR. Any glitches disrupt the immersive feeling and suspension of disbelief that VR depends on,” said Tim Gedemer, CEO of Source Sound. Gedemer has worked with Jaunt, a leading developer of VR content, on projects including the Paul McCartney and Jack White concert experiences. Jaunt received $66 million recently from investors led by the Walt Disney Company, and has received more than $100 million in total, to develop its end-to-end cinematic VR content-creation platform. A major market targeted by Jaunt for its VR is concerts, and the company is concentrating on audio technology to create a “you are there” feel of being at a live event. For the time being, most of Jaunt’s productions are short, until there is more customer feedback on how long people are willing to wear VR headsets.
Many of the rules for VR audio are very different from that of normal films or games. “You will normally use mono sounds instead of stereo because you need to use 3D spatialization all the time in VR,” said Tristan Panniers, an expert VR speech and music consultant, at the after-hours Audio Party at the GDC. “In traditional games, you might use a mono source of a bird in a tree, but then couple this with stereo sounds to deepen the immersion of the player in the game world and assure there’s a constant bed of sound there. In VR, however, if you have a looping stereo sound of birds in a forest, the player quickly notices that a bird call will always enter the ear in the same way, no matter how the head is turned. In VR, this will break the immersion. So, mono sounds must be placed very precisely in the game to allow correct 3D spatialization.”
Instead of using traditional microphones, VR recording is often done with setups that mimic the human head and ears. High-rez headphones are also important. “Closed-back headphones with HRTF (Head Related Transfer Function) filters that mimic the human head are optimal for VR, to block out outside sounds and enable fuller immersion,” according to Panniers, who adds, “Some sources can still be stereo, such as narration or music that is not part of the virtual world.”
“VR audio can be counter-intuitive,” said Eric Darnell, co-founder of VR studio Baobab and director of Madagascar and Antz — who is also a voice actor. He continued, “For instance, in a regular film, the volume of a speaker approaching the viewer rises gradually. However, in VR, audio is so important for both directing the viewer’s attention and indicating closeness that you may have to quickly boost the volume as the person or critter gets real close.”
VR audio was not the only sound topic at the GDC, as there was an entire track with many presentations dedicated to improving the sound quality of videogames, with experts from indies and major studios sharing their hard-won knowledge. It appears that audio engineers and experts are finally getting the respect that they’ve always deserved.
