iPlayer3D update
Post categories: distribution,Ìýtelevision technology
| 16:18 UK time, Wednesday, 21 December 2011
The Digital Service Development group led by Phil Layton in ´óÏó´«Ã½ R&D was involved in the previous trial of 3D at the Wimbledon Tennis Championships this year and also the recently broadcast Strictly Come Dancing Grand Final. In this post Dr Peter Cherriman and Paul Gorley outline the work they did to determine if it was possible to put 3D content onto the Freeview and Freesat versions of TV iPlayer.
Ìý
Generally 3D requires a high bitrate to achieve good stereoscopy. If the video bitrate is too low, depth cues are lost and the 3D becomes tiring to watch. However, due to varying Internet speeds, the higher the bitrate on iPlayer, the less people that are able to watch it. So we had the challenging task of trying to producing high quality 3D at as low a bitrate as possible.
Our 3D television broadcasts on Freeview, Freesat, Sky and Virgin all use a side-by-side frame-compatible format. This combines the Left and Right eye views into a single HD signal, by anamorphically squashing horizontally each eye's view into half of the HD frame, so they appear side-by-side. This HD signal was compressed at resolution of 1920x1080i25, which means 25 interlaced frames per second each of which is comprised of 1920 pixels across and 1080 lines. Each interlaced frame comprised of two fields, each field is 1920x540 pixels, and the fields are captured 1/50th of a second apart.
In order to produce the best quality 3D we decided to use a recording made in Blackpool, rather than use the broadcast feed received via satellite. This had a number of advantages, it meant we weren't limited to the side-by-side 3D format and the recording would have less compression artefacts.
We did a number of experiments with different resolutions and determined the best compromise for bitrate and quality was to convert the recorded 1920x1080i25 interlaced signal for each eye into a non-interlaced 1280x720p50 signal using a professional cross-converter. The intermediate signal created is at 50 frame per second, where each frame is 1280x720 pixels for each eye.
Ìý
We then needed to convert the pair of 1280x720p50 signals into a standardised frame-compatible format. The preferred frame-compatible format for 1280x720p50 signals is to anamorphically squash vertically each eye signal into half the HD frame, the so-called top-bottom or over-under format. This results in 1280 by 360 pixels per eye per frame.
Alternative methods for compressing two stereoscopic elements into a single HD image.
Our broadcasts use side-by-side format, however this requires horizontal squashing of the picture which degrades the stereoscopic depth cues. These would be further degraded for a 1280x720 image format.Ìý Vertical squashing used in the top-bottom format preserves more of this depth information, but the rescaling required in a receiver is much harder for the interlaced broadcast format which is why it's not used.
We found that our broadcast video encoders were much more efficient at encoding this 1280x720p50 signal than the existing software encoders. The bespoke workflow of this experiment allowed us to trial the use of broadcast encoders for iPlayer content.Ìý We tested with a wide range of 3D material, but the most challenging of which was the Strictly Come Dancing 3D footage, shown in cinemas, for last year's Children in Need. By using HE-AAC audio encoding we were able to minimise the audio bitrate required. This enabled us to create good quality 3D at a constant total bitrate of less than 5Mbit/s. You should notice the improved quality of the iPlayer 3D pictures in terms of less compression artefacts, and smoother motion due to the 50 frames per second, which is twice the framerate of standard iPlayer.
The next challenge was to modify the file to be suitable to upload to the iPlayer platform. The Freesat receivers required a MPEG transport stream (ISO/IEC 13818-1), which is produced directly by the broadcast video encoders.
However, FreeviewHD receivers require a mp4 file. When we used our standard software tool to create the mp4 files we found that the audio and video was not in-sync on some receivers. The mp4 files contain metadata to indicate how to synchronise the video and audio. However, it seems some receivers assume the first frame of video should be synchronised with the first frame of audio and don't make use of this metadata. Using a alternative tool, we were able to create mp4 files which played in-sync on all receivers available to us, including those which previously seemed to ignore this synchronisation metadata.
We don't yet know what the future of 3D will be, but these experiments have demonstrated another platform on which 3D content can be delivered to viewers.
Ìý
Comment number 1.
At 21st Dec 2011, Kit Green wrote:...it seems some receivers assume the first frame of video should be synchronised with the first frame of audio
This is of course the logical conclusion. Digital audio includes digital silence, just as a black video frame is not a lack of video, even in analogue video, so why would the start point of video and audio ever be different at source?
Complain about this comment (Comment number 1)
Comment number 2.
At 22nd Dec 2011, technologist wrote:In a perfect world you may assume that the two first frames are synchronized - But there can be all sorts of reasons in the real-world when one or the other stream suffers packet loss and thus comes out of sync.
that is why there is metadata ins the stream (PTS/PCR) to ensure that all components are kept in sync regularly.
Complain about this comment (Comment number 2)
Comment number 3.
At 22nd Dec 2011, Andy Quested wrote:Agreed technologist - even digital broadcast streams have stamps to maintain sync and they should have exactly the same path and contention issues
Complain about this comment (Comment number 3)
Comment number 4.
At 22nd Dec 2011, Kit Green wrote:As "digital broadcast streams have stamps to maintain sync" why is so much received badly out of sync?
Why is it usually the worst way round (sound early)? (Or is it just that small errors are not as noticeable when the sound is delayed?)
In a similar vein, iPlayer (PC) currently has a sync issue after pausing a downloaded file. This was not the same a couple of updates ago. Only way to get sync back is to scrub up and down the programme and all comes good.
Complain about this comment (Comment number 4)
Comment number 5.
At 27th Dec 2011, xyzbird wrote:Hi, I am trying to reach Andy Quested as his 3D iPlayer blog has closed. This may not be the right place to do this but I feel that an ongoing problem being experienced with various Samsung TV's needs to be brought to his attention. I copy a response received from Samsung today:
"We are currently aware that 3D does not function with the ´óÏó´«Ã½ I Player app. This app is developed and maintained by the ´óÏó´«Ã½ who must maintain their app to function with our TV's at all times and I would strongly suggest that you contact the ´óÏó´«Ã½ and refer them to this error with their application if it is not functioning within the required parameters of the TV". [Helen McCrickerd [NEW]
Online Support Team
SAMSUNG Customer Support Centre]
Complain about this comment (Comment number 5)
Comment number 6.
At 28th Dec 2011, Phil Layton wrote:As expected the 3D iPlayer experiment has identified a small number of compatibility issues with receivers. Where possible we will investigate these and consider, in cross-industry working groups, appropriate changes. We tested a number of Freeview and Freesat products and I am glad to say the vast majority worked fine. However we did observe issues with some products and clearly some further work is needed. Whether the problems are with the iPlayer application or the receiver remains subject to further investigation.
On the issue around synchronisation the point Pete and Paul were making is that there is metadata (in addition to the PTS/PCR mechanism) within the mp4 container which accurately describes the delay between video and audio components. Not all implementations seem to use this delay to align the video and audio, hence the need to re-align the components.
Complain about this comment (Comment number 6)
Comment number 7.
At 28th Dec 2011, Andy Quested wrote:Dear all
I am away over Christmas - the blog closed before I could post a note (too efficient!) I will do a round up of the comments and issues when I am back in the new year.
In the meantime - and with apologies to R&D for using this to post - thanks for all the feedback and Happy New Year
Andy
Complain about this comment (Comment number 7)
Comment number 8.
At 13th Oct 2012, U14179821 wrote:All this user's posts have been removed.Why?
Complain about this comment (Comment number 8)