Hello, my name is Rob Oldfield and I am the co-founder and CEO of Salsa Sound. We are a part of the 5G Edge-XR project, demonstrating how 5G networks—coupled with cloud graphics processing units—can enable people to view immersive sporting events from all angles and access additional data a video feeds across a broader range of devices including smartphones, tablets, AR and VR headsets and TVs. At Salsa Sound, we are developing the audio tools for the project that ensure that immersive visual content is accompanied by dynamic, personalised and immersive audio.
Our audio tools are used by broadcasters and sports clubs alike to create content that better engages with fans. Built on an object-based audio framework, our automixing tools also provide a real-time metadata output stream describing the content (source type, location and timestamp) that can assist in next-generation content and add value to current production workflows.
As a football fan, I’m particularly proud to say that with the 5G Edge-XR project we have already successfully trialled a 360-degree viewing of a match by way of 5G with 4 8k 360-degree cameras with adaptive immersive audio at each location. Viewing on a tablet or AR/VR headset, The stadium perspective was incredible: I was able to customise my viewpoint and even zoom in to check in on the goalkeeper. Immersive audio is essential here and at Salsa Sound we have developed AI applications to isolate the on-pitch sounds and also the immersive crowd at each location, rendering 360 reality sound. If, for example, somebody is clapping behind you, when you turn around the sound will rotate accordingly. The low latency and high bandwidth of 5G help make this a reality for live content by doing the complex audio rendering in the cloud and streaming the individualised audio to the end-users based on their personal viewpoint.
Another forward-thinking use case is DanceEast: a live-streamed AR dance class, hosted by a remote dance artist who is presented as a video realistic avatar using a real-time volumetric video. Building on the current model of live streaming to schools to enable artists and teachers to co-deliver a dance lesson, the approach hopes to increase levels of pupil engagement and active participation.
Democratising XR experiences is really important and being able to remove the barrier-to-entry using 5G streaming to do complicated rendering in the cloud is a real privilege. People can now engage in these experiences from their bedrooms! This would not be possible with 4G, though. 5G provides crucial additional bandwidth and speed, along with the ability for network slicing which ensures the dependable delivery that is vital, especially in a stadium context.
1. What do you enjoy most about working on the project?
Innovation! I enjoy improving technology, carrying out new methods and bringing exciting experiences to market.
2. What’s the most ridiculous thing you’ve done in the name of 5G?
Definitely climbing up a wobbling scaffolding tower on the stage of the Royal Albert Hall to rig our immersive microphones to capture the sound of a boxing match…
3. What excites you most about the project?
I love both sports and technology so combining the two creates a real sweet spot for me.
4. What would you like people to know about the project?
We use AI to automate audio mixing for live sports; this makes it more immersive and engaging for viewers!
5. What’s the biggest benefit you think 5G will bring to the UK?
New experiences. 5G will remove constraints to innovation, which will open up a whole new array of creativity.