Fire, plague... and bees! A recap of the 360 Video work we've carried out so far which is helping us to explore a new way of telling stories and make films. By Zillah Watson and Peter Boyd Maclean.
These films are designed to be viewed through a VR headset with headphones to fully immerse the viewer in the worlds they evoke. For this reason, they are not all publically available yet, but you can watch clips below, but they are being shown at events and festivals (including and ).Creating films optimised for tablet/mobile viewing (rather than the more immersive experience provided by VR headsets) demands different production considerations to maintain viewer interest : for example, the pace would be quicker, and because nausea isn鈥檛 an issue, they allow for more camera movements.
We鈥檝e approached this project as film-makers, and our focus was on story, rather than achieving technical perfection in each shot. The films are not steroscopic (3D) - they were shot on a standard with six GoPros,and post-production budgets were limited, so the stitching is not always perfect. Here are some the things we have learnt:
- Story is all. We tried to select stories that showed off location-based/spatial storytelling and engendered presence to show off the possibilities of 360. We .
- We still believe that the unique value of 360 films is to take people to places they cannot otherwise go.
- Simple narratives still work best for now. For example, the Resistance of Honey is a profile of a beekeeper with a quirk.
- Because these films are intended to watch through a VR headset we have generally avoided camera movements to prevent nausea for viewers. Any camera movements must be approached with care, generally only moving forward and at a slow and even pace, with a clear, stable horizon in view.
- To 鈥渄irect attention鈥� in these 360 films, we have generally kept the focal point of shots in front of the viewer, to make watching more comfortable. But there are other ways to direct attention, .
- You can create sequences of 360 shots (without fading to black) if you carefully cut from focal point to focal point. But transitioning between 鈥渨orlds鈥� needs to be done with care, or you鈥檒l create a jarring experience for the viewer.
- Pacing is very different for 360 immersive films - it needs to be slow. To give people a chance to look around each shot needs to be at least 30 seconds long. You also need to give people time to allow the user to become acquainted with the environment before you begin the story.
- Having a voice-over seems to allow people to look around more than having a presenter/reporter talking to you. (We have some early audience research to back this up - )
- Shot composition - if you can鈥檛 get the camera in the right place you are not going to achieve powerful 360. You also need to evaluate how difficult the shot will be to stitch when you are shooting (eg a drone shot over a graveyard in The Tragic Story of Betty Corrigall is a great shot for the story, but on our budget we couldn鈥檛 achieve perfect stitching on our budget. So you need to make a judgement call.) Planning and good recces are essential.
- The secret of good 360 often lies in post-production - the time spent 鈥渟titching鈥� footage together and grading it. 大象传媒flow was as follows: we used to rough stitch and Premiere Pro to edit. Then, once we had a picture lock we went back and fixed the stitching and began object removal (for tripods etc) in Premiere After Effects. Then the footage was graded. Then we worked on sound.
- You don鈥檛 have to hide the crew to keep them out of shot as much as we thought. For news films, we left the crew in shot because it felt more transparent to do so. In the Resistance of Honey we hid behind trees, or dressed up as beekeepers. But as we progressed we used object removal to erase the crew and tripod from shots. We used the suite and tried the beta plug-in too on some tricky moving shots which made this easier. How much digital manipulation of footage is done in post-production may give rise to trust issues in some genres.
Sound designer Catherine Robinson hides behind a rock while filming in Orkney
- First person narrative - we haven鈥檛 found this to work in the powerful way some people imagine. Sticking a camera on the head of the fireman would not make you feel like the fireman - it could leave you feeling sick and confused. But you can show what it鈥檚 like to be with them,as we have done in Fire Rescue 360. We鈥檝e generally placed the camera on a tripod at the centre of the action to achieve this.
- Imaginative use of effects can enable you to achieve things you couldn鈥檛 actually film in 360. For example, we created an artistic representation of the inside of a beehive in the Resistance of Honey. In Fire Rescue 360 we reconstructed a house fire with effects. Similar editorial issues arise in relation to journalistic responsibility and reconstruction as those arising in CGI based VR, discussed in a blog by Frontline and Emblematic.
- Sound - this is a whole new creative field for 360 which still hasn鈥檛 been explored properly. For most films use stereo mixes. For Betty Corrigall we used spatialized sound, using R&D鈥檚 sound production system, previously used for The Turning Forest - .
Notes on the films Fire Rescue 360
Breathe the smoke and smell the fear as two London firefighters put their lives on the line to rescue six children from a blazing house fire. Experience what it鈥檚 like to be a firefighter in a life and death emergency in this true story with 360 reconstruction.
Filming interview with firefighter of the year, Paul Rich, at Southall Fire Station
The film combines interviews with the firemen, filmed at Southall Fire Station, with a reconstruction of the fire. Commissioner, Ron Dobson, was impressed with the results.
"This film is the most realistic representation of being at a real fire that I have seen. We never been able to show people the dangers and difficulties our firefighters are faced with before."
Ron Dobson, London Fire Brigade Commissioner
Our challenge was to see if we could use the qualities of 360 film-making to take the viewer into the shoes of a firefighter. This was a technical challenge as well as an exercise in using empathy to see the world from another鈥檚 point of view.
We experimented with putting the camera on the face of the firefighter to get his point of view and make the viewer feel in the moment. But we couldn鈥檛 make this work, because sudden movement that you are not in control of as a viewer is confusing at best, and at worst creates motion sickness. However we did use a POV (point of view) shot to show the fireman going up the stairs - we used a short shot and slowed it down to avoid nausea.With clever use of special effects, we were able to create a strong sense of what it would be like to be alongside the firefighters through reconstructing a fire scene in 360. This film was made with .
The fire reconstruction is created through multi-layer compositing. To create the fire effects we filmed multiple shots, for example on the stairs in a real house - clean, with smoke and with the firefighter. We then re-shot the scene at the Fire Service College in Gloucestershire, where we could add the fire effects safely. We used a fire special effects team with a gas controlled fire which burned on different areas on the stairs so we could composite the scene later. The final film had over 500 layers of effects.
Crossrail 360: The MusicalTravel deep below the streets of London to - Europe's biggest construction project - for an unforgettable musical tour in 360. From the ghosts of the plague pit to the platforms of future stations, this provides a unique glimpse of the tunnels that will carry 200 million people across London in 2018.
Our challenge here was to demonstrate how 360 could give you a unique tour of places that you wouldn鈥檛 otherwise be able to visit. And for us, this meant dressing up in lots of safety gear. It鈥檚 a short film, intended also to work for audiences watching it on YouTube with .The song about was composed by and brilliantly strings together a series of engineering facts about the construction project. This allowed us to cut a range of shots to the music and create a sense of story, which included shots of being excavated. We鈥檇 originally attempted to create a tour of the tunnels led by an apprentice engineer, but didn鈥檛 feel a straight 360 tour was strong enough to engage viewers.We deliberately introduced a low camera angle to film the train driving over the camera in an homage to the Lumiere brothers and to create a dramatic effect. Although low angle shots in 360 are generally avoided because it makes the viewer feel like a giant, here it works because of the context. Some viewers have reported that it鈥檚 given them the sense of being 鈥榯ied to the tracks鈥�.We also experimented with using controlled camera movements. If the camera is moving directly forward at an even pace, and the horizon doesn鈥檛 change, then sickness can generally be avoided. But individual reactions can vary.Filming in the tunnels was more straightforward. The camera was on a tripod, which we later removed in post-production.
Step inside the world of Bioni Samp, an urban beekeeper who makes honey - and music - from his bees. .
This film demonstrated how a 360 profile could help you understand someone鈥檚 world, and . It has been featured at , the , and other documentary festivals.
The Tragic Story of Betty CorrigallOne of the saddest sights on the Scottish island of Orkney must be the lonely grave of . Gather round the smoky peat fire to hear storyteller Tom Muir recall her tragic fate as virtual reality meets the centuries old tradition of oral storytelling.Our challenge here was to use effectively and to create a more passive viewing experience.
Storyteller Tom Muir creates 360 performance of George Mackay Brown鈥檚 story of Betty Corrigall in an Orkney crofter鈥檚 cottage
The itself film experiments with two 360 techniques. The first calls upon the Orcadian tradition of oral storytelling in which people gathered around peat fires to tell stories passed down for centuries. The storyteller, Tom Muir introduces the story of Betty Corrigall (by Orkney poet George Mackay Brown) by addressing the viewer directly as you stand beside him in a crofter鈥檚 cottage by the fire. It鈥檚 an intense experience - a personal performance by a master storyteller.But as the story begins, the viewer is transported to the landscapes of the island of Orkney, which are used to evoke this emotionally intense tale. The story continues in voiceover and works upon the viewer鈥檚 imagination (rather like a radio drama). The film uses compositing and dissolves and fades for artistic effect - for example the waves from the sea are layered over the grass.
Evoking the landscapes of Orkney in 360
We aimed to show how landscape, colour and sound can transport the viewer to the story world. It鈥檚 proved a powerful technique, and is one that could be repeated to create visual immersive formats for poetry and music.
Peter Boyd Maclean filming 360 by Betty Corrigall鈥檚 grave in Orkney
Sound designer has created a unique dynamic binaural soundtrack using 大象传媒 Research & Development's software, and will describe this process in more detail elsewhere.
ConclusionNearly two years on from creating our first advanced 360 film - (a profile of Martin Mossop, a printer), we have achieved many of the challenges we set out to understand. We鈥檝e created engaging 360 stories with flowing sequences of shots, without fading to black. And we鈥檝e begun to explore the possibilities of spatial audio. But there is still much more work to be done to realise the full possibilities of this new medium. The on-going developments in technology - such as 360 cameras, and production work tools - will not only increase the quality of 360 films, but allow new storytelling techniques. It鈥檚 an area that will continue to develop rapidly.The films we鈥檝e created have been the starting point for several audience research projects - notably by (大象传媒), (Middlesex University) and Andrew MacQuarrie (UCL), which will soon be published.Meanwhile, other 大象传媒 R&D research (for example by , Maxine Glancy and ) continues to explore other long term possibilities of 360 including performance in 360 and 360 for Web VR. This research has continued apace as the wider 大象传媒 has also begun to explore the more immediate editorial possibilities of 360 films too - many of which can be seen on .Over the next few years we can expect better stereoscopic 360 video, improved spatial sound and more fusion with CGI based techniques allowing a range of fantastic effects. We鈥檒l also see more volumetric video (video with depth) and more interactivity built into 360 experiences. But story will remain at the heart of what makes these experiences truly memorable.We would like to thank all the people who have help us on our journey to understand storytelling in 360. Especially Peter Passmore from Middlesex University, Andrew McQuarrie from UCL, and Kim Wigger, and Kofi Sekyere from the 大象传媒. And to everyone from 大象传媒 R&D, and who has helped support us. And to for inspiring and advising us never to stop thinking about the story.
- -
- More on Virtual Reality and 360 Video:
- from day