Spatial sound creates the illusion that sounds come from a 3D space rather than from speakers located in a fixed position. In game development, spatial sound provides audio that seems as though it is emanating from points in space within a game world. Spatial sound follows characters as they move through the game space, allowing players to experience sounds as though they are coming from specific directions. For example, a sound made by an object changes as the object moves away from the player character or camera, immersing the player in the sound space. This effect is known as anchoring persistence to video objects. The sound comes from the object no matter where it is in the game world, even when it is offscreen, outside the player’s visual field. In addition to providing a wider field of awareness and a far more immersive experience, spatial sound also improves accessibility for the visually impaired through the inclusion of sound design with accurate spatial positioning. Spatial sound allows game developers and designers to enrich worldbuilding and provide more accurate directional cues, taking players’ gaming experience to a new level of realism.
Why Spatial Sound?
Traditional audio, even 5.1 and 7.1 surround sound, is flat. It only provides audible cues in a 2D environment (left/right, front/back). Spatial audio creates the illusion of sounds coming directly from an object in a three-dimensional space. Think of it this way: in the real world, when you hear an airplane fly over you, the sound doesn’t come from around you but from above. The sound changes as the airplane moves away from you or climbs higher into the sky. While surround sound attempts to capture it, the sound changes as it moves and still comes from fixed speakers, limiting what the sound engineer can accomplish.
Spatial audio delivered through headphones creates a more immersive sound experience than fixed speakers. Spatial sound can take advantage of the headphones’ sound space to deliver sound from any point in space. Further, that point can be engineered to change dynamically with an object’s movements on a screen through anchoring persistence.
For the gamer, this creates a broader field of awareness than the visuals alone. A sound coming from an unseen source — such as behind a wall or camouflaged in the terrain — provides advanced warning of impending danger and allows the gamer to pinpoint its location with much greater accuracy. Whether seen or unseen, anchoring and persistence make it possible for sound to move with objects, giving the gamer the same cues and awareness they would have in a real-life situation.
These features help the game designer tell their story more interactively. Not only does the player move through the visual space of the game, but they also move through the auditory space. Sound cues make the game more realistic, adding to the experience of play.
Let’s look at some of the features of enhanced audio.
Immersion
Spatial sound improves immersion, deepening a player’s sense of presence in the world of a game. Let’s take a rainstorm, for example. With the traditional sound experience, rain falls around you. Thunder may emanate from front to back or vice versa, but it comes around you with surround sound. Consider the real world, where sound interacts with space. Rain makes a sound when it strikes an object. This sound might come from a tin roof to your right and above or from the ground near your feet. Thunder originates from above and descends to you in a 3D setting, echoing off of objects around you. Spatial sound allows you to hear these differences, creating a greater sense of immersion in a game.
Storytelling
Sound helps us tell a story. Characters in a story don't just stand there. They move and interact in ways that cannot be fully captured by traditional left, right, front, back sound design. Spatial sound allows players to follow characters through space. Players can hear two characters approaching each other in a way that static speakers cannot emulate. Theme music for characters can follow them as they move. When an object is hidden from view, its sound can change as the distance between the object and player character changes, providing clues to the object’s location. This feature is incredibly powerful when used in a visually complex environment where corners and angles impede vision in the space.
Gameplay
Sound location becomes part of gameplay, adding more complexity to the traditional left/right, front/back auditory positioning. Since spatial sound can come from a specific direction, it can improve reaction time and accuracy of response. Being able to identify where noise is coming from allows players to prepare for their next action. A player’s decision to react to the danger of a cougar in a tree versus a snake on the ground could make a big difference. Location of sound is particularly important when a hidden threat jumps out at the player. For many games, play has been primarily visual due to limitations of traditional sound, but spatial sound adds a stronger auditory component, bringing gameplay closer to the real-world sensory experience.
Spectacle
When it comes to a game’s “wow” moment, we want the sound effects to match the visual effects. Consider an explosion of an ammunition depot on a distant mountaintop a little left of the center of the screen from the player’s position. With spatial sound, the designer can create the illusion that the sounds are spreading out from the upper left location and enveloping the player, as debris drops around them from above, hitting the ground below. This adds dimensionality to the flat sound provided by fixed speakers.
Clarity and Mix Space
Spatial sound allows you to open up space and create depth by situating sounds positionally. If a commander addresses troops from an elevated platform, the voice emanates from above, rather than just the front. Similarly, ambient noises — such as the movement of people or objects within a scene — can be more clearly depicted with the flexibility of spatial sound, as this technique makes it possible to distinguish where the sound is coming from in the mix space.
Accessibility
For the visually impaired, spatial sound can take the place of, or supplement the auditory descriptions that accompany a scene. You don’t need to add an auditory description that says, “a door opens to the left” when the player can hear where the sound originated from in the game space. Spatial sound design improves the accessibility of gaming to visually impaired players by allowing informative auditory cues to be directly incorporated into game design. Directional sound clues from a threat allow visually impaired players to locate the threat and take action.
The Future of Sound Design
Adding spatial sound allows game developers to move beyond the limitations of flat sound. Spatial sound offers a new dimension of realism by creating the illusion of sound in three-dimensional space. These features can enhance storytelling and incorporate a stronger sensory experience into the gameplay. The use of spatial sound also enhances the accessibility of gaming for visually impaired players.
For more information on spatial sound, you can view the presentation, Enhanced Audio Content with Spatial Sound.
