BOOM app: Spatial Audio AR app design, via We Are Listen agency and the client X-Ambassadors band.
Contracted by We Are Listen in partnership with Microsoft, I sound designed the end-to-end Audio AR experience for the band X-ambassadors’ new “audio-only music video” app for their song “Boom.” Happily, the project went on to win the 2020 Clio Bronze Award for Audio Craft / Sound Design! I encourage you to download it for free from your App Store of choice to give it a spin for yourself.
Intro screens give context and instructions
Here we cleverly avoid using the phone’s compass and instead rely on the user correctly performing this task, which then logs a heading value for the app which keys the audio spatializations “true north” value, or 0 degree marker.
Next we briefly demonstrate the 3D audio, priming the user to understand the UX audio interaction involved.
The band features a pair of brothers, one of whom is blind, and so the band wanted to create a new type of music video experience that essentially had the same richness for both the visually impaired and the sighted. This authentic vision manifested into telling a pair of audio-only stories with overlaid ambisonic field recordings that capture the actual scenes and characters that the band grew up around in their hometown of Ithaca NY and their current home in Brooklyn. I traveled to Ithaca and around Brooklyn to produce these recordings that would then become the scenes that bring to life the song within the app.
Pictured here to the right is the home screen, which also now features the “Director’s Cut” for each of the two scenarios. These are comprised of an additional ambisonic layer that positions the two bandmate brothers in conversation on opposite sides of your headphones soundfield (allowing you to look towards and away from them while they speak), and they provide charismatic reminiscing and extra context for the environmental sounds and scenes.
Recording Ambisonics at park with AMBEO VR mic thru Zoom F8, as well as Binaural via the AMBEO SmartHeadset thru iPhone.
Recording Ambisonics at night of insects and owls in rural setting
Design, Prototyping and Workflow
I took the Boom song stems and spatialized them dynamically, so that through the course of one song - although the experience is essentially on rails from a linear perspective - the sound sources can move past you. This was done in Unity, by keyframing the transform positions of the various stems (lead vocal, guitar, percussion, etc..) to moments in the song when we’d like to build spatial excitement (i.e. the beginning of a chorus, a bridge). The lead vocal might travel from by your right side to straight in front of you, also suggestive of the singer accompanying you on this walk through their old neighborhoods.
The ambisonic field recordings and mono sound efx were then laid on top of the song in the Unity Timeline, and their position transforms were similarly keyframed so that characters could appear to walk past you, or a car could do a tire-screeching donut around you.
I worked closely with Josh Vaughn - the developer on this project, in a collaborative Unity doc that enabled us to fluidly develop the audio experience as he assembled the visual UI elements and core functionality of the app.
We went through a couple rounds of revision with agency and band, and then we were in a great place to launch the app to the iOS, Android, Windows app stores to line up with the band’s upcoming US tour dates.
Launch and Press
Billboard magazine link / Microsoft In Culture blog link / I Heart Radio link