European Framework 7 collaborative project to revolutionise the production of realistic animated characters by developing an automated process to extract and represent actors from performance capture in a multiple camera studio
Project from 2011 - 2014
What we are doing
Recent advances in graphics hardware have produced interactive video games with photo-realistic scenes. However, interactive characters still lack the visual appeal and subtle details of real actor performance as captured on film. In addition, existing production pipelines for authoring animated characters are highly labour-intensive.
RE@CT set out to revolutionise the production of realistic characters and significantly reduce costs by developing an automated process to extract and represent animated characters from actor performance capture in a multiple camera studio. The key innovation is the development of methods for analysis and representation of 3D video to allow reuse for real-time interactive animation. This enables efficient authoring of interactive characters with video quality appearance and motion.
The project started in January 2011 and finished in November 2014
Further details can be found on the
Our Goals
One of our technical goals in the project is to develop production technology for character modelling and animation that can fit in with the way that TV programmes are made today (such as use of conventional HD cameras in a TV studio, rather than special motion capture cameras), yet provide content to support new media forms such as interactive games containing actors from a TV programme. We developed a multi-camera capture and processing system to allow 3D reconstruction to be checked during or shortly after capture, in the same way as a conventional recording can be reviewed before the actors leave the studio.
We organised several test shoots to allow the technology to be tested in a TV studio, one based around medieval soldiers and another with a ballet dancer.
We also developed technology for authoring user-controllable interactive annotations that can be overlaid on video or rendered content, building on our work on the Augmented Video Player.
We were the lead partner in this FP7 project.
Project Team
Project Partners
-
German research institute in the fields of mobile broadband communications, photonic networks, electronic imaging and multimedia
-
French public science and technology institution, dedicated to computational sciences
-
Based at the University of Surrey
-
French SME specialising in virtual reality, augmented reality, serious gaming
-
Group of companies (including Vicon) working in motion capture and computer vision
-
Immersive and Interactive Content section
IIC section is a group of around 25 researchers, investigating ways of capturing and creating new kinds of audio-visual content, with a particular focus on immersion and interactivity.