What is the future of spatial audio?

Gareth Davies: 00:00:32

What happens when sound surrounds you not just from left to right, but from every angle, when it stops being something you hear and becomes something you feel?

I'm Gareth, a composer and audio producer, and in this episode of the Sound Session, I visit the School of Digital Arts in Manchester to meet composer and sound designer Dr. Neil Bruce. We talk about immersive sound. Not just the buzzword, but the practice, how it works, where it doesn't and why.

It's not always about more speakers or clever software, but about paying deeper attention. In this sound session, we're going beyond stereo to answer the question, what's the future of spatial audio? Welcome, soundmaker.

This is The Sound Session where we explore how sound shapes our world and how we shape sound.

If you'd like to share your thoughts or your favourite sound, you can do that at thesoundsession.uk and with that in mind, here's a recording that sound maker and listener Adam Forum made in Admiral's Cove, Cape Broyle in Newfoundland, Canada. He used to live just above the beach there, and this is his favourite sound. Enjoy.

Isn't that amazing?

So we start, as always, with a favourite sound, but back to the School of Digital Arts in Manchester, or SODA, as it's known. Neil had to really think about his answer.

Dr Neil Bruce: 00:02:43

This is probably the hardest question to answer, and my immediate sound goes to the sound of my daughter laughing and giggling. If I think of something practical in the real world, this is a remembered sound, something I'm still trying to track down.

They're definitely not in the UK anymore, but it was the sound of clacker boards at railway stations and airports. I used to love that sound, the first sound.

Gareth Davies: 00:03:09

Well, that's obvious why Neil likes it, but what about the second sound? Is that a childhood thing?

Dr Neil Bruce: 00:03:15

I think if we're going on the kind of therapist couch here, if you look at a lot of my work, there's a sense of space, sense of journey in there. So I'm. This is my cod psychology answer. I think I just like traveling and moving and this idea of, like stations and playing.

You know, airports are this place of transition and movement, those kind of liminal spaces that are quite interesting. It's like a happy place. So I think it's a memo. It used to be happy, but I think maybe it's the nostalgia for, for traveling and movement.

Gareth Davies: 00:03:43

It's easy to call that nostalgia, but Neil's early relationship with sound wasn't just sentimental, it. It was transformational.

Dr Neil Bruce: 00:03:50

Wanting to be a musician at school and going through, as I've discovered, Recently, a process that many kids at school unfortunately go through where I wasn't necessarily supported by certain music teachers.

It was a new music teacher who came in and actually introduced us to the works of people like John Cage and Steve Reich and Pierre Schaer, that suddenly this light bulb went off on my head like, oh, music is sound and sound is music. And it took me on this whole other avenue of discovery in that I didn't have to be able to play, you know, Bach's sonatas or concertos perfectly.

I could do something else. And I think instinctively this always tied in with something that was inherent in the music that I liked.

And a lot of that came down to the sense of space and creating what I'm going to refer to as sound worlds. It's always been this kind of exploratory avenue to what I wanted to do.

Gareth Davies: 00:04:46

For Neil, space is more than physical, it's emotional. And nothing embodies that like reverb.

Dr Neil Bruce: 00:04:52

Part of that is my fascination with sound in space. And maybe that goes back to, you know, listening to early Pink Floyd records and the use of reverb.

But something about playing a note and hearing those tales and the way that sound interacts with space, which has also been a part, I think of, you know, when you reflect on your own practice, part of your own compositional practice and the things that you like to do, even though I'm a guitarist, do like to drench the guitar sometimes in a lot of reverb.

Gareth Davies: 00:05:19

But in his teaching and research at the School of Digital Arts, Neil takes that exploration even further.

Dr Neil Bruce: 00:05:25

What we do here quite a lot is actually capturing impulse responses in real world spaces and of course, using tools in the computer.

But it's the actual physicality of going out into a space, either doing a sweet sign or either popping a balloon if you're doing the quick and dirty version. But there's something about that which intrinsically gives some form of character to the music that you play through it that excites me.

Gareth Davies: 00:05:46

So let's talk about when spatial audio works and when it doesn't. Spatial audio promises immersion, but not every format welcomes it the same way.

Dr Neil Bruce: 00:05:54

I think it all boils down to the context of what the material is.

So if we briefly delineate it perhaps into musical content, visual content, film, television, etc, and maybe more modern content such as gaming, there are various different applications within that.

Probably I should also add a fourth category, and that would be something like immersive experiences, maybe in gallery spaces or in theme rides or something along those so there's these. These kind of delineation. And each of those requires a different way of thinking about how you approach the use of spatial.

Gareth Davies: 00:06:31

And in some cases, immersion can work against the experience.

Dr Neil Bruce: 00:06:35

If someone clicks and I'll demonstrate, I'm clicking my fingers behind my head. If I do that, I will turn my head. It's just human nature.

I will try to localise the sound, in which case I am no longer immersed in the world that the filmmaker is trying to create. I am in a cinema looking at someone eating popcorn. So I am taken out of the experience.

Gareth Davies: 00:06:55

That tension between experience and distraction is the key.

Dr Neil Bruce: 00:06:59

So what we've always tended to see, perhaps in film sound design, is how the earlier days, when these newer technologies are happening, all these things happening, but it ends up being that we tend to just put reverb in the back. So you feel a sense of wash and immense immersion.

Perhaps quicker events occur, creepy events that are not necessarily diegetic sounds, but something that adds a sense of emotional content to the film, but not direct sounds, because you will just be removed from the story. The narrative always has to be key.

And perhaps again, I would argue from a stereo perspective, if you listen to a lot of earlier classic records, and they don't even have to be classic records, did anyone ever complain that the Pink Floyd was not immersive enough? Not in the slightest. Not in the slightest. And I think with music, we have to be a bit wary of what we're trying to do.

Film, I think, has already got a handle on how it handles this. And a lot of that immersion is done through, again, like I say, reverb and kind of non diegetic sounds to give a sense of wash.

Gaming, though key, because you are moving, you are locating. So everything has to move and rotate with you. So you have to sound like you're completely present in that world. And that becomes even more important.

If you're playing games through VR goggles, the sound world has to move with you.

However, if you're playing Tetris, perhaps less important, then you get onto maybe the immersive world of say, a gallery installation or a theme park. And again, that is more of.

Not necessarily a typical, like a Dolby Atmos type setup that wouldn't necessarily work in there, but you have to kind of plan. And that, again, can be quite a creative process to create an immersive narrative.

Gareth Davies: 00:08:46

So what do Neil's students think about immersive audio, especially in music mixes?

Dr Neil Bruce: 00:08:50

Yeah, so this is not a rigorous academic study.

This is more anecdotally but we teach several years modules on immersive audio and whilst playing a lot of music mixes through them, the general feedback tends to be irritating. That's the word that comes up quite a lot.

mixes that more current, say: 2025

And again, you've got, as you heard, like maybe some backing vocals or something like that, but they're mixed in a way that you feel present, but you're still in the experience of seeing that. That performance. It's the.

It's the mix and it's the creativity when the mix in the idea that we just have, you know, things whizzing about all over the place. And maybe you put the keyboard behind because again, it feels weird. You kind of keep moving.

But when you're listening to music, perhaps you want to be.

Gareth Davies: 00:09:48

Yeah.

Dr Neil Bruce: 00:09:49

Even if you're feeling it's a music that's going to get you fired up and even aggressive in some regards, you don't necessarily want to be turning your head around all the time.

Gareth Davies: 00:09:57

So with all this potential, why hasn't immersive audio taken over?

Dr Neil Bruce: 00:10:01

I think the obstacles haven't really changed over the years and it depends on the system. So we still have a variety of different companies putting out different codecs and different ways of decoders and encoders.

So there's no kind of universality about it as there is. It's quite complex in that regard.

I don't think that's necessarily a reason why the public shouldn't engage with it, but they have to make that choice and again, it can be quite an abstracted choice to try. And what's the difference between Dolby and Sony's version of it?

Yet within the house, if you want to do it practically, do you have the space for all of those speakers that have to be positioned? Precisely. There's a lot more precision in immersive audio. Of course, now we're getting technology through things like sound bars which do throw and.

But again, it's set up and calibrated maybe for ideal spaces, and most home spaces are not ideal.

Of course, we're seeing a massive trend now to people engaging with content through earbuds or earphones, and that is probably where it does work more effectively. So there isn't a barrier in that sense.

I think that the main reason for uptake is how it's being used creatively versus some of those discussions we had earlier in the earlier question about what does it add to the narrative. I just realized there was something I forgot to mention and this is an area where I feel like it's also being underused.

It is in kind of audio books feels like a massive area. But again it's this idea of how does it contribute?

And that also seems like the ideal kind of medium where people listen to their audiobooks through earbuds. So there isn't a barrier to entry there.

The other barrier is from a production side of it because in the best willing world, many studios are struggling. I was going to say budgets for audiobooks are not. They're not good. But budget for music, budget for all things are all being cut.

So as a studio, do you want to invest in another kind of 10 or so speakers? And it's not just the cost of the speakers, it's all the infrastructure that has to go into position them, wiring them, cabling them and doing that.

Is it worth it? And again, the jury is perhaps still out because I think some studios have been burnt in the past.

Gareth Davies: 00:12:33

So it's not just the cost of the speakers, it's all the infrastructure around it. But even when systems are in place, Neil sees a deeper issue.

Dr Neil Bruce: 00:12:41

Now some of the big streaming services are all requiring that you're doing Atmos mixes and things like that. So there is no escape.

But we're just in an interesting time and like 3D cinema, there weren't people out on the streets saying make it more immersive. It's not really consideration.

And the sad thing is with sound in particular and just a cheap plug here, I've just finished a book on come on called Reevaluating Soundscape. But one of the things is sound is often a sense that's not thought about or not considered. And you see this across much of the production industry.

Sound is always a thing left to the end. And any way that it can be cut, it will be cut.

But also within the general public, when it comes to my other work with the public and research in the public, sound is something that again is not considered.

So perhaps that's a barrier because no one's asking for it because most people take what they are given yet with the caveat that if the sound is bad, the sound is problematic, they will probably turn the program off. But that's, you know, that's a different thing if it's glitching or it's dropping out. Picture. Can you know, drop out a little bit.

It can get blocky, will carry on. As soon as the sound goes, people are switched off.

Gareth Davies: 00:14:35

If you didn't catch that. Neil has a book out later this year.

His recent work explores how spatial sound can support more than storytelling. It can shape environments and how people feel in them.

Dr Neil Bruce: 00:14:47

k to when I did my PhD around: 2010

And then from my initial project, it was a tool that urban planners and architects and designers could use, for want of a better word, to mock up how a space may sound. So the importance there was to have something which was immersive in that regard. So early ones used Ambisonics.

I've used Ambisonics quite a lot throughout this to create a sense of a world which doesn't necessarily have such a tight sweet spot as a Dolby system surround system will have, so you can move around freely within. Within the cube of speakers.

Gareth Davies: 00:15:39

According to Wikipedia, Ambisonics is a full sphere surround sound format. In addition to the horizontal plane, it covers sound sources above and below the listener. We're all learning here.

Dr Neil Bruce: 00:15:48

But you're saying about barriers. What a pain to set it up. It's a massive pain. So again, that's a. In a sense, it's a way of putting people off.

So the idea would be that you can create spaces you can design. Roughly, I equate it a little bit to an old...I'm not sure if it's still out there, but Google Sketchup.

So this is not an exact precision rendering of a space. But it's this rough idea for again, something which is often well overlooked or overheard most of the time.

The ability for an architect to actually kind of get a rough idea of what something might sound like. Oh, I can put a cafe here. This is what the cafe might contribute. A later project. I worked on a project called Project DESTRESS.

Now DESTRESS, like all things academic has is an acronym and I can never for life. For me it's such a long one. It's about basically restorative environments and sustainability.

But the idea that we could take, we were looking at the importance of green space in city centres. So case studies were in Sheffield, Brighton and Edinburgh. I went out for weeks on end and made lots of recordings of these spaces.

And then we did a visual mock up so you could have a visual walkthrough. And I use the word fudge a lot in this, again, this is not a precision acoustics.

It's a fudge attempt where you could change, say, the stone facade to glass or grass wall and see the effect, the rough effect it had on how the environment would sound with those kind of interventions. We have moved on a bit since then, but immersion is still the key. It has to feel real.

And I would also argue now, having built a lot of these things and I'm still building variations of them, actually it's better to be in situ. And it's trying to think of ways that we can have the same kind of simulation process but actually be in the real world.

And I've got a few ideas, like NDA at the moment, but trying to think of ways that you could actually do this process, but in real time, in an actual space. Because the one thing I have learned is, apart from us who are involved in this industry, no one listens.

So actually, the key skill is not giving urban planners necessarily a tool or architects a tool. It's a good thing. But actually, the first stage of that is getting them to go out and listen.

And then we can give you tools that can kind of work with that. There are some other commercial tools available, but they maybe go to the other extreme where they're using extreme precision.

But to me, that also defeats some of the object because I think, again, listening is the skill. And then from a listening process, I think the next stage actually is a capture process.

So a big thing I encourage our students to do is actually go out and capture, learning how to, you know, not necessarily being the world's best field recordist or the be... but actually thinking about space and capturing and coming back and listening to it and thinking, then what would you do? There's. There's your little bit of material to work with. How would you improve this? How would you make it different?

And then ask the question, why, why?

Why do you think that matters through to perhaps the more engaging pedagogical process of creating, you know, environments for, you know, school kids to start to learn to listen.

Because that's a whole other area, which maybe a whole nother topic for another podcast by getting us learning to listen to our environment an earlier age.

I know there's some elements on the curriculum now that do occur, but I think it could be more important because that leads into, you know, thinking about the environment and other other issues. But I think using tools which are engaging and narrative forming, again, for.

For the public to just be engaged with outside of going, oh, this is a Dolby Atmos set. And they have to be in this room. We have to thinking about installation, thinking about again museums, thinking about places where people go.

Let's bring sound into it a bit more. And on top of that, above all of that, you say, well, that does kind of happen.

But the key thing, let's do it right, because that is a whole other hobby horse of mine. I also find it. There's a big irony for me, you go to academic conferences on sound or just anything in presence. Sound never works. Oh, it's terrible.

Or people are giving PowerPoint presentations. It's like to talk about sound.

Gareth Davies: 00:20:09

Neil mentioned Ambisonics back there. So I asked him to run through what the differences between Stereo, Binaural and Ambisonics are. You might want to jot these down.

Dr Neil Bruce: 00:20:19

So the differences very simply boil down to in some regards, the kind of channel setup. So stereo just has two speakers, two channels of audio, typically positioned left and right. You could go one step back where you have mono.

It's important to mention that because why we don't necessarily go with much mono stuff anymore. Although the biggest selling speaker device at the moment is a mono speaker, ironically, you have no positional information within a mono setup.

You can't tell where it is. You have some element of position front to back. And that will be done on volume rather than left and right.

So there's no left and right information as you move up through various surround formats. And I'll come to Binaural in a second. The most obvious one was quad then, which was four speakers.

Once you start positioning more speakers, you have to deal with more technical problems such as phasing. Quad was around in the 70s in Pink Floyd, who mentioned earlier tried to do it. Apocalypse Now was mixed in in Quad. Again, never really took off.

Then we went to 5.1. What this means is that you also have a bass channel as well and five speakers.

As you do away with that, you then get into Ambisonics, which is more of a computational form of audio, where you have a special microphone which has four capsules in it which capture XYZ and what's known as the W component, which is a kind of omni component to the sound.

Which means that if you do a lot of clever mathematics, basically relying on phase differentials between the microphones and the W signal, you get the positional information of the sound field. So it's not. It's not tied specifically to channels.

Gareth Davies: 00:22:10

Yeah.

Dr Neil Bruce: 00:22:11

You then go sideways perhaps to Dolby Atmos, which is both a channel based system, but is also has what's known as objects in it. And the clever thing about Dolby Atmos is it's technically speaker agnostic.

So it doesn't matter what setup you have in it, it can detect what setup you have and it will adjust your object's positioning through metadata. So it's basically a stream of metadata rather than again, if you don't have those two rear speakers, you don't hear those sounds.

If you've got them as set as objects in Dolby Atmos, it will again cleverly adjust it so it to stereo or to 5.1 or to 5.7, 1.4. It's quite clever in that regard. Which brings us back to the, you know, the other one, which is binaural.

So binaural is based on the way that we hear. So the way it tends to be recorded is through, ideally through microphones that are placed in your ears.

And this is currently a reason why it's very personal because it's based on the shape of your pinna, your, your earlobe. It's really important to the way it sounds.

Also your face mass, the way your nose is, and what's known as the interaural time differences between your ears all play a way into how we localise sound. So again, I'm doing the clicking thing on this side. There is the click arrives at my right ear.

Soon that arrives at my left ear, but it's also being filtered out by my head and my hair. This is all known as the head related transfer functional HRTF which is again, it's mine. It's like my finger, my acoustic fingerprint.

So ultimately if I made a binaural recording, the person it sounds best to when I play it back is me.

Gareth Davies: 00:23:50

Got that? You can skip back to listen to that again if you like. Next up, when is spatial audio the most impressive?

Dr Neil Bruce: 00:23:57

I think the simple version is some of our students work who have really been kind of again, pushing the creative boundaries of what they can do, but using it again, not so it's necessarily irritating, but just cleverly that you feel this has really enhanced the narrative of the piece that we are exploring.

Gareth Davies: 00:24:20

I found it interesting that Neil spoke about his students work in that way. So I asked him if the work his students are creating inspire him or give him pause for thought in his own practice.

Dr Neil Bruce: 00:24:30

Oh, 100%.

I think that's one of the things that, you know, keeps me going with this is students are really good at kicking your backside, but also giving you, you know, just ideas and you know, collaborative ideas and ways of Thinking about things and, you know, not wanting to sound too old, you know, But I've had a good run at this, getting on a bit now. It's good to see, you know, it's good to see a freshness and a different undertaking because I've spent a lot of time in this world, no pun intended.

This is new. So it's seeing new explorations, like their kind of, you know, first steps into this world and how they're approaching it.

I think that's really an interesting thing to consider.

Gareth Davies: 00:25:17

So one piece of gear every sound maker should try.

Dr Neil Bruce: 00:25:19

For anyone who's wanting to start out or think about it, or anyone who's evolving, just goes back to binaural recording.

I think binaural recording is just as a great insight to what interests you as a practitioner as well, because you can freely explore spaces, because, again, like I say, you're not tethered to anything. You can listen back to that. And it kind of, you know, it's an insightful tool to see what interests you and what you're actually trying to recreate.

Because I think that's no matter what we're doing professionally or artistically, there's something intrinsically in all of us that we're trying to recreate and explore. And I think this, from a recording perspective, it offers a good insight.

I also think that it provides you with a kind of audio diary that's very personal. But there are inherent problems in the recording method that need to be overcome or at least considered. So I think it's also a good.

It's a practice as a recording tool because there are considerations. Anything that makes you think about it is. I think it's good. So, yeah, I think I'm just slightly biased. Biased towards it. That's great.

But in terms of thinking about the sound world, again, it's something you could sit.

You can make some recordings and then go back and listen to those worlds you've recorded and really, you know, map them out, whether it's in your head or on paper, and really think about the space, be it an urban park, be it, you know, the Tate Modern, doesn't matter where it is. You can really start to think about, what is this space doing? Can I visualise, like, sonically visualise it in my head whilst I'm listening to it?

And then maybe I can. Then help me think about. I'll translate this over onto an Atmos system, perhaps. But I know now where my world sits.

If you just want a one word answer, binaural every time.

Gareth Davies: 00:27:08

What's the future of spatial audio. For Dr. Neil Bruce, it's not about louder, fuller or even fancier. It's about listening better and creating spaces that feel as rich as they sound.

If you've got thoughts on immersive audio or a sound you love, you can share it at thesoundsession.uk I'm Gareth. Thanks for listening.