In order to improve and demonstrate my sound design and implementation skills, after a year studying at Leeds Beckett University. I have begun working on designing sounds and implementing them through Wwise and UE4. I am using Nuendo with Ableton live 10 for my sound design DAW's along with iZotope, Waves, Native Instruments and other plug-ins. These are great tools for the job I have built up over time, but there are many others out there. Dave Raybould has a list of free to use plug-ins as well as a number of implementation tutorials that are worth checking out. Most notably the book Game Audio Implementation Guide co-authored with Richard Stevens, which takes you through implementing audio natively in UE4. Other resources such as Audiokinetic and Epic's Unreal Engine YouTube channels and the GDC vault are great places to start to learn about sound design and implementation for games.
For my sound design project, I’m using the Unreal Engine example game action RPG as the foundation to build my sound design. The free to download project (which can be found on the learning tab of the Epic games client ) has some typical systems found in many third person action RPG games, such as: combat, abilities, inventory and UI as well as short cut scenes and environmental cues.
In this post I’ll take you through the project set up as well as how I went about designing and implementing the footstep audio for the main player character. So you can get an idea of what I am covering, here's a video demonstration of the surface switching footsteps you should be able to create by the end.
As Action RPG already has sound in it, it makes sense to use the existing cues as a basis which can be later added upon as you see fit bearing in mind the media count of 200 available in Wwise. Below is a screenshot of the blank hierarchy I laid out for the main character.
I found it easier to find all the existing audio cues by looking at the character animation page, which can be found by double-clicking on the character model found in the characters folder in Unreal Engine. If you then open the animations tab in the top right, you’ll see a list of the character animation montages and animations sequences in the bottom right. Clicking on the sequences in green will show a preview of those animations. For each one of the animations you’ll see the ‘animation notifies’ for that particular animation in the centre of the screen, as shown in the screenshot below.
The animation notifies can be used as a guide to show you what you'll require in your Wwise hierarchy for the player character. However, if you choose to you could also use the sound cues found in the assets folder also shown below.
You will need to replace the original notifies with the Aknotify by right-clicking in the area notifies are shown, you then can select your footstep event in the right hand menu.
So, once I had the blank structure for the characters setup, I then needed to create the events that are required for UE4 to trigger the sounds. This is done by selecting the switch container in the hierarchy right-clicking and then selecting new event/play, I did this for all of the events I would require as shown below, you can make multiple selections and assignments.
You then need to generate the sound bank in the sound bank layout F7 if you don't have one then you must create a new on in the default work unit. You then drag all the events you wish to associate with the sound bank across from the event viewer into the sound bank editor. Once done you can go ahead and generate your sound bank.
You then need to go into UE4 and create a sound bank by right-clicking in the content browser. It’s a good idea to have a specific folder for sound banks and one for events. You then select 'Audiokinetic' from the drop down menu, then Audiokinetic bank. You need to name this exactly the same as the name of the sound bank you created in Wwise. Once done save everything.
Now open the Wwise picker from the windows tab in UE4, you should see all the events you created and associated with the sound bank. Drag them all into a folder in the contents browser, then double-click on the Audiokinetic events. A page will pop up for the event, set the required sound bank, do this for all events and save. You now have all the structures setup ready for sounds!
After the setup was completed, I then went looking for inspiration in the third person fantasy genre. I came across games such as Conan, Witcher, Dragon Age and Torchlight as well as many others, I looked for video footage of the games being played without commentary to use as reference. I also did some research on how other teams had implemented character sounds into their games and came across a great article by Lost Chocolate Lab specifically on footsteps, which seemed a logical place to start my own sound design.
After looking at the Action RPG map I could see that the terrain has a lot of different surface types for such a small map. These include: rock, mud, water, leaves and dirt. To add these to the game I went into the project settings in UE4 and added them into the Physical Materials found in the Physics tab. This allows us to create a switching system which will work from physical materials applied to textures in the game world, by clicking on a surface or object and applying the Phys mat to it in the collisions tab.
There was a complication with this particular map, which stems from the fact that it was created using a landscape. This meant that only one Phys mat can be applied to the landscape as an object. To go around this you need to search for landscape layer in the content browser of UE4. This will show you the layers used to create the landscape, the Phys mats can be applied to each layer.
Although this was better than one footstep type for the whole map, it didn’t give me as much control as I wanted, and the water aspects of the map didn’t seem to work as I had hoped. To work around this, I applied box triggers to the map in the water and leaves areas and wrote a simple blueprint (shown further below) to trigger the correct surface type for those areas. The rest of the map utilised the ray tracing method of switching surfaces.
It basically uses a ray trace aimed at the players feet to detect what Phys mat is assigned to the surface material, it then switches the surface type based on what it detects. It's important to get a few little things correct for this setup to work correctly. First, the event you create for the footsteps, in my case Play_Character_Step must be attached to the player character before you add the Akanim notifies for the animations otherwise you will get a warning telling you to do it. Likewise, you want to attach your blueprint for ray tracing to the player character blueprint.
You can then use a print node at the end of the chain to print the surface type to the screen, which is useful for debugging, as is the options in the line trace node to make the trace visible.
I then used the desktop recording software OBS to record the animations of the various characters, to use as reference when designing the sounds in Nuendo. This will really help you with getting the timing of sonic events down nicely with the animations. There are a number of great features in Nuendo and other DAW's for this type of work. I like to enable the cross-hairs in preferences and make them a bold colour to help me line things up. The scrub picture to event option helps you to see frame by frame where you are placing things and Nuendo's macro capabilities are second to none in my opinion and can make repetitive tasks a simple one button action.
From this point you will want to start designing your sounds. The footsteps you require depend on the surfaces of your game and the type of platform its intended for. As this game is intended for multiple platforms including mobile, it's not a good idea to get too fancy with the implementation, we don’t need Mirrors Edge level of details. Also, if you are using the evaluation licence for Wwise then you are limited to 200 media items and they are better used elsewhere.
With this in mind I opted to use 5 wavs for each surface type 25 wavs in total. These came from my library as I'd already recorded a number of footsteps sessions, on top of this there are some excellent recordings in the Odyssey Pro Sound Effects library as well as other independent libraries at websites such as Soniss or aSoundEffect most of which have some free content which will help get you started.
For searching through the libraries, I've always found Steinberg’s media bay to be excellent, the better the meta data, the better it is, so if you are recording your own sounds it’s good practice to keep on top of data tagging them. Also you can make selections in the preview window and add only that content to the project, super useful for picking pieces from larger files.
I also used isotope’s Trash 2 in multi-band mode to saturate with subtle distortion different portions of the frequency spectrum and to utilise the dynamic module for some compression. This helped to beef up the recorded source, as well as helping to add presence and tonally balance the sounds. I then used the render in place to combine the layers into one wav through some bus compression and a small amount of tape distortion by setting up mono groups for each footstep type. It's worth noting that if you want to render in place your files in mono you'll need to set up a mono output bus, otherwise render in place will render your files in stereo. Once I was happy with the results, it was then time to export them into Wwise.
Steinberg have made this process super-efficient, there are several options for exporting, and I recommend that you experiment with all of them to find what works best for you. For me, I find that it works best by first opening game audio connect, then selecting the container I want to export into in Wwise, i.e. character step/dirt, then going back to Nuendo, selecting all of the material I want to export then file/export/selected events. This allows you to add custom naming conventions and set the bit depth of the files upon export which is a real time saver. The tail mode allows you to make sure you are exporting tails on time based effects and you have options to export the files from the channel, groups or master outs.
Once you’re happy with your selections you hit export and the files will appear in Wwise with an import dialogue, where further options can be applied as required. You can then add some randomisation to the containers in order for the sounds to feel a little less repetitive. There is a small circle like icon next to pitch, volume, low-pass and high-pass filters, in the general tab of the Designer view F5, click these and tweak to your taste.
You can go ahead and generate the sound bank in Wwise and then also in UE4 by selecting generate sound bank in the build drop down menu. If everything has gone to plan you should now be able to test your audio in game by hitting the play button.
It's sometimes useful to switch off some of the other sounds in the game to isolate what you are working on. You can do this by going into the Assets/Sounds/AudioClasses folder in UE4. Double-clicking one of the orange sound classes will take you to the mixing structure, click on the nodes and reduce the volume multiplier to 0 in the inspector on the left for whatever sounds you wish to mute.
The process of creating and implementing game audio is an iterative one, you may decide that you don’t like how it sounds, the sounds are too loud, too soft, maybe the samples don’t fit as well as you thought they would in DAW. This is fine and normal, go ahead and make your adjustments in DAW and then reimport, regenerate sound banks in Wwise and UE4 and rinse and repeat until you're happy. One other thing you might want to look into in order to save time is, you can also change the destination of the exported sound banks to the UE4 "WwiseAudio" folder, meaning that you don't need to re-generate your banks, just a quick refresh works. Thanks to Wilfried Nass for the tip.
I hope that this has been useful for some of you starting out, as I move through the different aspects of the game, I will add new blog entries giving an explanation of my process. They will be in the form of brief demonstrations of what I am currently working on and or more detailed tutorials on specific elements of sound design or implementation.
Cheers,
Rory