Coyopa's guts
Coyopa, as any regular Radio Labs reader will know, is the new system for encoding ´óÏó´«Ã½ national radio stations for the iPlayer and internet media devices - both simulcast and on-demand. It's been running well in production since November, and is now producing all listen-again and most simulcast streams across our services.
If you're deeply interested in the technology we use, here's a quick delve into rather more detail about Coyopa. This isn't for the faint-hearted, particularly if your tolerance for TLAs is low. (What's a TLA? A Three Letter Acronym. LOL. FTW!)
We have used broadcast-engineering principles to implement the new encoding system - such as resilient system design with backups, multiple power supplies, keeping digital audio linear all the way to the encoder.
For simulcast - live streaming - we take the audio directly from the broadcast chain which feeds DSat/DTT. This is passed through an audio router (for resilience) and then directly into an AES3 sound card. The sound card implements protection limiting in a DSP and carries out other functions such as detecting audio silence (failure alarms). Up to four radio station versions (eg "Radio 2 UK") are encoded on each machine.
For Listen Again, the process is divided into two functions. Linear audio is recorded from the broadcast chain for each radio station, non-stop, and kept for up to seven days. Producers can schedule their radio programmes to be available for Listen Again, by selecting the times/days and repeat patterns (rather like a professional version of a consumer ). The system will let you schedule "in the past" too, using its internal store of audio. Users can also select which regions will hear the audio (UK/International) and allow programmes to be automatically published after the show (some programmes have to be edited before being allowed through for compliance or rights reasons). There are different rights restrictions for download (podcast) files, which Coyopa will begin producing later this year, and for streaming files. Producers can also re-edit the programme, particularly the start and end times, at any time to tidy up the audio that listeners hear.
When a programme is ready for encoding, it is sent to a set of sixteen encoding machines, allocated according to the current workload: Radio 3's Through the Night (nearly 6 hours) takes longer to encode than a fifteen-minute news bulletin, after all. Up to four files can be sent for encoding for each radio programme (UK/International, Streamed/Download), and the encoding system knows which formats/bitrates to use for encoding each file. After encoding, the many different encoded files are passed, using , into a large store, which then clones the files to our content delivery partners.
The schedule of radio programmes compounds the encoding workload by having many programmes across the radio stations ending at about the same time; nevertheless, the whole process is designed to be automatic and fast.
The server hardware uses HP "Blade systems" - where each PC server is a plug-in module, with up to 16 in a, very heavy, chassis. Six power supplies share the load of the whole rack, fed from two different sources of mains power. These servers are used because as well as offering the performance, they are reliable, easy to maintain and allow a very high packing density. Each server has two 4-core processor chips which are needed to achieve the throughput for listen again encoding.
As you can see, Coyopa is rather more than a little machine with a sound-card!
I hope all that was interesting to some; and I'm indebted to my colleague David for working on this blog post with me.
Comment number 1.
At 24th Mar 2009, Neil wrote:This is all well & good but all I want is 128kbit/s aac stream or above for ´óÏó´«Ã½ Local radio & other stations too. Radio 2 is 128k aac and I want the others to be too.
I ask is this likely to happen in 2009?
Complain about this comment (Comment number 1)
Comment number 2.
At 24th Mar 2009, juuxjuux wrote:I'd be interested in the bitrates selected for each channel/programme - is this information published anywhere? How were the various bit rates selected?
Also, the post infers that the bit rate might vary between programme types, e.g. 128kbps for music and 64kbps for news bulletins on the same channel. Is this the case?
Complain about this comment (Comment number 2)
Comment number 3.
At 25th Mar 2009, Pete wrote:Interesting to read the inner workings of your (fairly heavy-duty) encoding systems. Looks like you're ready to cope with a fairly major throughput!
Never heard of AES3 before, but I was at home with the other TLAs :)
Complain about this comment (Comment number 3)
Comment number 4.
At 25th Mar 2009, 2Bdecided wrote:It's great to hear it's a serious resilient broadcast system - I'm sure the day will come when your on-line delivery becomes more important than your traditional broadcast delivery.
However, if there's so much fault resilience, how come days of radio content go missing on the iPhone?
Don't believe me? Try listening on-demand to anything from Friday. I've only tried Radio 4, but several things just aren't there: Feedback, Something Understood... I had to visit my PC(!) to catch the last two episodes of book of the week (from the foothills).
Is Coyopa responsible for this? Or does it go somewhere else?
btw, the iPlayer on-demand Radio 4 audio sounds quite rough on the iPhone, whereas Radio 3 programmes sound good (sometimes very good), so it's not faulty hardware at this end.
Cheers,
David.
Complain about this comment (Comment number 4)
Comment number 5.
At 1st Apr 2009, Ed Lyons wrote:Is this used for archiving as well, i.e. beyond the 7 days...
Complain about this comment (Comment number 5)
Comment number 6.
At 11th Apr 2009, Society wrote:"how come days of radio content go missing on the iPhone?"
The incidence of this seems much higher of late.
Wednesday 1st had some bits missing, Friday 3rd April had larger chunks missing and Monday 6th seems little better.
It's not just the iPhone encodings. Book of the Week isn't quite the same when the opening episode just isn't available in "Listen Again" (at a time when there are "3 days left to listen" for Episode 2)! Even unofficial ra files don't seem to be present.
Given that "Producers can schedule their radio programmes to be available for Listen Again, by selecting the times/days and repeat patterns" it seems that it can all go wrong due to human error/carelessness at a very early stage with no obvious feedback mechanism for frustrated listeners when stuff is simply missing.
Complain about this comment (Comment number 6)
Comment number 7.
At 12th Apr 2009, Society wrote:"Book of the Week isn't quite the same when the opening episode just isn't available in "Listen Again""
Actually, I note that a substitute is presented at: /programmes/b006qftk
However, I'm not convinced that Episode 3 of "The Decisive Moment" from 4th March is an entirely acceptable substitute.
I can cope with the changes to R4's website style but had rather hoped that content wouldn't be trashed along the way.
Complain about this comment (Comment number 7)
Comment number 8.
At 22nd Apr 2009, Tim Regan wrote:I love this kind of post - it's great when the engineering detail is talked about in the open. One question though (which may show my ignorance). What does "keeping digital audio linear all the way to the encoder" mean? Linear in what sense?
Complain about this comment (Comment number 8)
Comment number 9.
At 23rd Apr 2009, tristanf wrote:@dumbledad - I think in this context, linear audio means uncompressed (in terms of bitrate, not loudness).
Complain about this comment (Comment number 9)