In VR, No One Can Hear You Scream

The further we go into virtual reality, the more we understand how little we actually know about reality. Whatever you make of inventor/entrepreneur Elon Musk’s claim there is a “one in a billion” chance that we are not living in a simulation, there’s no doubt that virtual reality and augmented reality are evolving so rapidly, ultimately VR will become indistinguishable from reality.

Once we reach full immersion with all five physical senses, we will encounter Schrödinger’s dilemma: am I a man dreaming of a butterfly or am I a butterfly dreaming of a man? Are we spiritual beings having a physical experience or are we physical beings having a spiritual experience?

The sound of God asking for light

So, what role does sound play in this immersion? In Genesis, the first act of creation starts with light: “And God said: let there be light.” But light is, in fact, preceded by sound. That is, he said: “Let there be light.”

In VR, the role of sound is to create a persistent universe in terms of quality and consistency that maintains the suspension of disbelief. The world we are portraying via sensory input or implied physics needs to be above a certain threshold to be convincing. Virtual reality is an unforgiving medium, and spatial sound plays a critical role.

Working on a VR project is a very different process to working on a film or a broadcast commercial. As the sound partner we perform the role of an audio director, working very closely with the VR director to use sound to direct the viewer’s attention. Sound is a unique tool because it is malleable. It can be realistic, hyperreal or fantastical. It can distract, misdirect or direct attention and is therefore a great device for moving the viewer through a story where you cannot control the experience in the same way you do in film.

When Q Department began working on virtual reality projects there were no standards or best practices, and while the medium has come a long way, there is still much to be figured out. As Robert Stromberg, the director of The Martian VR experience put it, the process is akin to flying a plane while you are building it.

Taking spatial audio to Mars

As a music studio focussed on both original music and sound design, we have always been obsessed with immersing the audience in a film’s narrative through sound. When virtual reality arrived on our scene with The Martian VR experience, it prompted us to transform into a kind of music-lab, identifying techniques and processes to bring high-fidelity spatial audio into virtual reality.

Earlier last year, through our spatial audio technology company Mach1, we succeeded in designing a new spatial sound format specifically for the VR medium, which launched on The New York Times NYTVR App for iOS Android Daydream and Gear VR which we built in collaboration with Secret Location.

The only problem is that once you have left Plato’s cave, and seen true reality, there is no going back. Sound that is not 360° suddenly feels inadequate.

 We want to create visuals with sounds that make people feel like they have never used their eyes and ears before. As we move towards increasing levels of technological mediation, we need to make sure that our storytelling still touches the core of what it means to be human, rather than creating barriers that distance us and disconnect us from who and what we really are.

Am I an alien dreaming I'm a man or…

In the future, we will be creating permanent universes for an audience to inhabit, but what responsibilities will come with that power? As we enter worlds within worlds that seem real, retaining a sense of self in a disorientating, unfamiliar environment will be more important than ever – as Q Department’s recent work on the Alien: Covenant VR experience has shown. After all, in VR, no one can hear you scream…