Most people agree that the future of entertainment, education and learning involves virtual reality. But at the same time, most people generally associate VR primarily with seeing, not hearing. But in fact, the audio component of VR is what makes it truly immersive; in order to really believe what we’re seeing, we have to hear it. That’s why one of the most exciting areas of innovation in the VR space right now is the way that sound technology is being used to create ever more immersive VR experiences.
All the bells and whistles
As engadget recently wrote, without truly advanced audio, the visual VR experience doesn’t quite work: “The premise of VR is to create an alternate reality, but without the right audio cues to match the visuals, the brain doesn't buy into the illusion. For the trickery to succeed, the immersive graphics need equally immersive 3D audio that replicates the natural listening experience.”
One of the teams working hardest to pioneer what we can do with audio and VR is Oculus’s engineering team. They are working on new spatialized audio technology, which provides huge gains in the quest to develop how we view and experience VR content in simulated 360 degree spaces. Thanks to the team, VR developers now have two new tools in their toolbox when it comes to audio: “Near-Field HRTF” (head-related transfer function) and “Volumetric Sound Sources.”
It’s about space and time
GeekWire got a preview of what the engineers at Oculus are doing and wrote: “With ‘Near-Field HRTF,’ developers are able to control sound as it moves closer and farther from someone’s head with great precision. Previous technology only allowed developers to control sounds more than one meter away from a user.”
This means that a VR user will feel sounds and voices move from ear to ear, mimicking how one might experience sound if they were moving dynamically around a room, with the elements around them moving as well. The other tool, Volumetric sound sources mimics the variance of volume levels coming from different sources, or how “a bee buzzing past your ear will sound different than a train rumbling past you.” This gives a sense of the rich texture of sound that’s available to VR experiences.
One of the developers on Oculus’s team explained to GeekWire how audio functions in the grand scheme of immersive experiences: “In VR, audio has become more of a multiplier. If the audio is done correctly, it makes the whole experience two or three times better than it was before.”
In addition, The Verge notes that the rise of consumer-facing VR—like the Oculus Rift, Sony’s Morpheus, and Samsung’s Gear—is majorly expanding the area of research and innovation around binaural audio. Similar to X, this more closely mimics how humans hear sound (from two sources, one on each side of the head).
“Binaural recording systems are unique because they emulate the workings of the human head. If a dog barks by our left ear, it takes a few extra microseconds for the bark to reach the right ear; the sound will also be louder in one ear than the other. In addition, sound waves interact with the physical constitution of the listener — the pinna (or outer ear), the head, and the torso — and the surrounding space, creating listener-specific variations otherwise known as head-related transfer function. The brain scrutinizes these miniscule interaural differences of time and strength in order to localize sound with immaculate precision.”
Indeed, anyone interested in the world of VR and its developments should definitely pay attention to the audio side of things in the next year, as the technology is clearly accelerating quickly.