Setting up the 3D Audio Listener in Unreal Engine and FMOD
Si prefieres leer una versión en Español, da click en este botón =>
Unreal Engine Version: 5.5.4
FMOD Version: 2.03.06
This blog post shows you how to setup the 3D audio listener in Unreal Engine and FMOD. In this post I show you how to configure the listener on a side scroller, first person, third person and a top-down view game perspectives.
Here is the list of essential tools and keywords used for this project:
Not everything required for these setups is exposed to Blueprint, so I used a combination of Blueprint visual scripting and native C++. Additionally, I slightly modified the FMOD - Unreal integration to be able to handle the third person, top - down and side scroll examples.
Important:
Please consider that this guide does not intend to teach creative sound design. Instead, it exclusively focuses on game audio's implementation and programming aspects.
Download the project here:
IMPORTANT!
〰️
IMPORTANT! 〰️
Prerequisites:
For this tutorial to make sense, you need to know how to integrate the FMOD plugin into Unreal Engine. Additionally, consider reading and referencing my previous post about UPROPERTY & UFUNCTION:
IMPORTANT!
〰️
IMPORTANT! 〰️
Project Overview
The project has four distinct levels with their own game modes. All inheriting from BP_HGameMode and HGameMode class in C++. I am set up the 3D listener on these game modes for simplicity.
IMPORTANT:
Its important to mention that if you are creating a multiplayer game, using the game mode for this might not be a good idea because this class only exists on the server or host, not on clients. If this is the case, consider using the game state or the player controller instead.
Game Modes
Maps
The FMOD project has three sets of audio events with their own spatializer shared effect. _2DTD is used in the side scroller and top down view maps, _FP is used in the first person map, and _TP in the third person map. All these are assigned to the MASTER bank for simplicity.
How the 3D audio listener works?
The listener position needs to be updated on every tick, which allows the audio engine to update its state and to calculate how far emitters are from it. It also updates its rotation which allows the engine to calculate the emitters orientation in 3D space.
Here is how this is handled in FMOD and Unreal Engine:
Set Listener Attributes / Set Listener Position
The FMOD Unreal Integration calls FMOD::Studio::System::setListenerAttributes() on the FMOD Studio API passing a listener index and the necessary 3D attributes like its location and front and right vectors.
Line 1005 is where the FMOD API setListenerAttributes() is finally called.
FFMODStudioModule::SetListenerPosition() is called by FFMODStudioModule::UpdateWorldListeners(). Here is where the integration finds all the local players and retrieves their listener positions.
Notice that on line 923, the function uses Unreal Engine’s own listener handler on the player controller to get the listener transform.
FFMODStudioModule::UpdateWorldListeners() is called by FFMODStudioModule::UpdateListeners(). Here is where the integration handles whether the viewport world is on a “Play In Editor” context or a build.
This update process begins when the FMOD Studio Module is created and its bounded to the Unreal Engine Media Clock Sink interface. This means that the audio engine’s update is being handled by the FMOD Studio Module internally and there is no need to update it from outside this class.
First Person
For the first person perspective, the default listener setup works well. The “ears” of the listener are placed at the player character’s head and follow its rotation. The distance to the emitters is also calculated from the character’s head. This is what we want and no extra setup is required.
Attenuation Override
The default setup won’t work for the rest of the perspectives. The location we need to use for distance calculations is not the same location where panning should happen. So to address this, its important to detach these two parts of the 3D listener and place them in the correct location.
Unfortunately, the FMOD - Unreal integration assumes that we only want to make first person games so it does not have this option exposed by default.
Code Review!
This is how I modified the plugins’ code to support the rest of the perspectives. I added “<ANS - HVALDIVIESO>“ comments to wrap the code I modified to help you identify the differences.
First, I added a new virtual function declaration that has the listener position and also the attenuation override position as arguments
Afterwards, I moved most of the code inside FFMODStudioModule::SetListenerPosition() to the new FFMODStudioModule::SetListenerPositionWithOverride(). Here I made sure that if we pass an attenuation override location the correct overload of FMOD::Studio::System::setListenerAttributes() is called.
Finally, in FFMODStudioModule::UpdateWorldListeners() I checked if the player controller has an attenuation override position set and called the FFMODStudioModule::SetListenerPositionWithOverride() if that is the case.
In the HGameMode class I created a function called AHGameMode::SetupAudioListenerCPP() to handle the listener and attenuation setup. I also created a BP function in the BP_HGameMode class that does the same: SetupAudioListenerBP
These functions are called from the game mode for each example map on the HandleStartingNewPlayer event.
Finally, on each player character, I added a scene component with the tag “AttenuationOverride“ attached to the “head” socket on the skeletal mesh or to the character’s sprite. This is how the new functions can find the precise location for the attenuation overrides.
2D Side Scroller
On a 2D game, the camera usually does not rotate, but we still need to calculate panning from that location. Attenuation is calculated from the player character’s location
The white sphere is where panning is handled, the green sphere is on the Player Character where distance calculations are handled
Top Down View
On a Top Down view game, the camera has limited rotation but its enough to affect panning. Attenuation is calculated from the player character’s location. This assumes that the camera will always follow the player character, if that is not the case, you might have to come up with a different solution like clamping this position on the floor at the center of the screen.
Third Person
On a third person perspective, panning is calculated from the camera and distance from the player character. Some projects implement minor variations of this technique, like placing the listener slightly in front of the camera, but the main idea persists: separate where the attenuation is calculated from where the panning is calculated.
Extra: Debugging
On the HGameMode class I created a function called AHGameMode::DrawAudioListenerDebugCPP() to retrieve the current location of the listener and attenuation override, and then draw them on the viewport. Additionally, I declared a console variable to quickly activate or deactivate these debug spheres.
The function is invoked on the BP_HGameMode tick function.
The white sphere is the listener (panning) and the green sphere is the attenuation override (distance)
Next…
On the next post I’ll show how to configure the Spatial Audio Listener in Audiokinetic’s Wwise.
