VR and AR are the buzzwords of the moment and 360 video is often seen as a poor imitation, so why go to the trouble of capturing a Super Furry Animals concert and ? 大象传媒 Connected Studio's Robin Moore explains.
While 360 video is proving crucial for developing audience awareness of VR (mainly through clones sold in your local garage), and it has been , it hardly gets a mention at VR conferences compared with computer generated (CG) 鈥榯rue VR鈥�. Is it really worthy of further R&D?
We have an ulterior motive, however. We鈥檝e been working on . We鈥檝e run pilots on Radio, such as , and on TV, , but we (and those speaking at VR conferences) think it is VR where spacial audio can be most powerful. So we need VR projects to hone our skills and develop the art.
, are the perfect way to trial this, but they are few and far between. How could it work on the content we more routinely make in the 大象传媒? We want to trial binaural on different genre and content formats, such as we built the binaural sound for last year. And 360 video is the easiest way to do this.
The 大象传媒 has a huge portfolio of Music content and as have shown, this is an interesting opportunity for binaural. So, when the chance arose to capture a Super Furry Animals concert with full access to mic the audience and get all of the tracks from the mixing desk, we jumped at the opportunity. I鈥檒l let my colleague and binaural audio expert, Cathy Robinson, take up the story:
'We had the opportunity to take multi 360 camera positions at the Super Furry Animals gig at Cardiff Motorpoint arena, allowing the viewer to attend a virtual gig and experience what it would be like to be in the audience and on the stage with the band. This presented itself as an interesting challenge to try and represent the sound from each of those locations that were both pleasing on the ear and able to be immersive and rotate it with the 360 image. It was a fantastic opportunity for us to experiment with a live music scenario in a large concert venue.
'Each location was filmed using a 360 camera, so why not just put a 360 mic at each of the camera positions and capture the actual sound happening in that place? Seems logical, but what actually happens Is you would not have much control in post-production of the mix and you would be stuck with one sound position or everything sounding off mic, woolly, with no definition.
'What we were able to do was take a multi- track recording of each of the sources on stage, which could then be mixed to sound like a representation of the location being filmed. Each was mixed to give a nice blend of all the sources, but with each source given its own location in the 360 sphere as an 鈥榦bject鈥�. Once the sound object has its own position this can then be moved with the 360 image giving you a much greater immersive audio effect. We had complete control over the sound mix in levels and space. During the concert I also recorded lots of separate locations of the audience, which I used to create that live feel and sense of distance from the stage during the shots within the audience.
It was amazing to get such great access to the concert visually and sonically and has proved to be an incredibly useful audio experiment paving the way forward for creating truly immersive live music virtual reality experiences in the future.鈥�
That鈥檚 not to say this pilot didn鈥檛 also help us improve our approach to 360 filming. While 360 cameras are becoming cheaper and the content widespread, the art of shooting professional 360 is still immature. This pilot gave us the chance to compare six different camera rigs, with ten camera positions in what is a challenging environment, with lots of vibration, high temperatures, noisy and with lighting extremes.
We鈥檝e also used this footage as an opportunity to test a new 360 player for the 大象传媒. Oliver Spall, Senior Product Manager for 大象传媒 Taster, worked with the Standard Media Player team led by Oli Freke :
'This first iteration displays the 360 video in the 鈥榤agic window鈥� style, allowing the viewing window to be moved in all directions using a mouse, or finger for touch devices. Future versions will include full headset support and improved mobile support (including iPhone and Safari desktop which are not currently supported).
The team building the Standard Media Player 360 features will also aim to support 鈥榙ynamic audio鈥� in which audio responds to the direction a headset is pointing in. This requires support for 4, 8 or even 16 channels of sound, but can be very powerful in the creation of a fully immersive space.
The advantage for the 大象传媒 to have its own 360 player is that we can more easily embed the content on our own websites and apps instead of linking out to another service, such as YouTube or Facebook, which reduces clicks and makes for a more coherent experience.'
360 video may be less cool than 鈥榯rue鈥� VR, but we think the immersive experience that this sort of trial can deliver is still immensely enjoyable, particularly with the addition of binaural audio. It is also clear that this could offer a useful addition to our coverage of big events, particularly allowing you to get access to places you could otherwise never visit. However, we are still learning and the feedback we gain through 大象传媒 Taster will help us determine how to make the most of this new medium.