´óÏó´«Ã½

A commercially-available sports graphics system based on our computer vision work.

Project from 2004 - present

What we are doing

Piero is a system for producing 3D graphics to help analyse and explain sports events, for use by TV presenters and sports pundits. It has been a familiar sight to viewers of Match of the Day on ´óÏó´«Ã½ One since 2004, where it is used to highlight and track football players, draw off-side lines on the pitch, and generate a unique ‘virtual view’ of the action from a viewpoint where a real camera could never be placed.

Piero uses our innovative image processing algorithms to compute the location of the camera with respect to the pitch, and track the camera as it moves. This provides the data needed to allow the system to draw graphics so as to appear to be on the pitch, including things like accurate distance makings. It can also extract the players from the pitch, allowing them to be placed in a ‘virtual stadium’ model, which can then be viewed from any location, to give the best viewpoint for explaining the action. Our tracking software can also cope with non-pitch-based sports, such as athletics.

The system was named after the Italian painter and pioneer of perspective, .

The project is a good example of how we work with industry to turn our research prototypes into commercially-available systems for use by the ´óÏó´«Ã½ and the wider broadcast industry.

Outcomes

The ground-breaking nature of the system led to it winning the , and the 2006 Cable and Satellite Magazine Award for the Best Outside Broadcast Technology. It was also nominated for a . In 2011, ´óÏó´«Ã½ R&D and Red Bee Media jointly won a Queen’s Award for Innovation for Piero.

As of 2011, Piero was in use in over 40 countries around the world. At the 2010 World Cup in South Africa over 85% of all analysis effects used in the world feed came from the Piero system.

We are continuing our work on enhancing sports coverage using computer vision techniques in the . We have also been developing approaches to capturing full 3D models of players using multiple cameras, in projects including iview, i3DLive, and RE@CT.

How it works

The roots of the Piero system date back to around 2002, when we were carrying out some work on technology that could form the basis of a future 3DTV system. A to build a rough 3D model of a scene by automatically analysing the image from a single TV camera, using knowledge of the camera position and the assumption that people were standing on the ground plane. Through discussions with ´óÏó´«Ã½ Sport, it was realised that this could be applied to work out the positions of football players, and create a simplified model of each player, thereby allowing the match to be recreated as a 3D model. This would allow a view of the match to be simulated from an arbitrary viewpoint, making a radical new kind of analysis tool available to sports pundits and commentators.

We developed a proof-of-concept system, and conducted various trials with ´óÏó´«Ã½ Sport, both to refine the technology and to identify the kind of graphics that made sense from an editorial perspective. We licensed the core technology to , and on September 25th 2004 the system they produced was used for the first time on ‘Match of the Day’. Since then, we have continued to work with Red Bee Media to add additional innovative features to the system:

When launched in 2004, the system initially relied on pan/tilt data from sensors on special camera mounts, which were difficult and costly to install. In 2005, the system was upgraded to use image processing techniques we developed to track the camera movement by . In the same year, Red Bee upgraded the system for High Definition TV.

In 2006, Red Bee added a touch-screen user interface, called Piero Sketch, allowing a presenter to operate the system live on-air in a highly-intuitive manner.

In 2007, we started work on a new that could use arbitrary image features (not just lines) to compute the camera position and orientation, allowing the system to be used on a wider range of sports such as athletics, for example to draw a virtual world record line on a long-jump sandpit or a 100m track.

In 2008, Red Bee developed customisable 3D models of players, complementing the original image-based player modelling system to allow a wider range of virtual camera moves.

In 2009, our image-based camera tracker was integrated into Piero, and was used live on-air for athletics coverage for the first time in July 2009.

In 2010, we developed an intelligent keying system, to improve the ability of the system to distinguish the colour of athletes and background. This was particularly useful for distinguishing between sand and skin when placing graphics on a long-jump pit or in beach volleyball.

In 2012 we developed the to enhance diving coverage.

In 2014 we developed a tool for tracking arbitrary objects, for applications such as tracking the head of a cyclist in a race so a label with their name can be overlaid. This was a spin-off of more general work we were doing on tracking objects to to them.