Why did we build ´óÏó´«Ã½ iD?
Post categories: ´óÏó´«Ã½ iD
| 15:40 UK time, Friday, 12 March 2010
You may have noticed that slowly but surely, we're moving all our existing services to a new sign in system, called . You might also notice that anything we build from now on uses ´óÏó´«Ã½ iD from the start. So far we've migrated all our blogs, nearly all our messageboards, and our three big communities: , and H2G2.
A few people have posted blog comments asking why we've done this, and what what it means for the future. I thought I'd write this to help explain what we're doing and why.
So, why did we build ´óÏó´«Ã½ iD?
The simple answer is that our old system - called 'Single Sign On', or SSO - needed replacing. It had been around for nearly 6 years, skillfully powering all the ´óÏó´«Ã½'s online services which required authentication, but 6 years is a long time on the web. SSO has been showing its age in some very specific ways:
The technology platform
SSO was built on Perl and MySQL. Good technologies for their time, but the ´óÏó´«Ã½ is moving towards a new online architecture (internally called 'Forge') which uses Java and PHP on top of MySQL, Apache and Memcached. Soon, the old Perl-based system will be turned off. SSO would have to have been ported to Forge anyway, so it was a good time to completely refresh it from the ground up.
Performance
SSO used a single MySQL database instance. Forge allows applications to have multiple partitioned databases - which helps to make it horizontally scalable. This means that as ´óÏó´«Ã½ iD gets used more and more, we can make it perform simply by adding more servers. Until recently, you only signed in to small pockets of the ´óÏó´«Ã½ - the odd messageboard here, a one-off application there.
However, with the advent of ´óÏó´«Ã½ iD, nearly every page on ´óÏó´«Ã½ Online will know if you're signed in or not, and will be able to adjust itself accordingly. This new level of personalisation will allow ´óÏó´«Ã½ Online to grow and personalise around you in ways that were never before possible. But this level of integration, and load, will needed a totally new architecture which made heavy use of partitioned (sharded) databases, Memcache, and load balancing.
Internationalisation
´óÏó´«Ã½ Online continues to grow its audience internationally, and has a staggering number of language sites. As these sites want to do things like personalisation, they need sign in features in their native language. Adding features like these retroactively to a product is really hard - they have to be built in from the start. One more reason why we knew SSO had to be replaced.
Although the first versions of ´óÏó´«Ã½ iD are english-only, under the hood, it's been designed with internationalisation in mind. For example, every bit of text you see isn't embedded into the code, it comes from a language specific package. We're now working on increasing the number of supported locales. This will eventualy include not only the main UK languages like Welsh and Gaelic, but languages with different characters (like cyrillic in Russian) and right-to-left text (persian etc) - in fact, anything you can throw at Unicode.
Security
Since SSO was developed, security techniques and technologies have moved on a lot. For example, a while back it was impossible to support the loads we needed to support and encrypt data both in transit and on disk. Now, that's possible. As such, ´óÏó´«Ã½ iD has been built from the ground up with very secure architecture in mind. All personal data is stored on disk encrypted, all personal data is transferred over https, and inside the ´óÏó´«Ã½ there are strict access controls put in place to make sure only the staff who are authorised have access to it. While SSO was good for its time, the security model had to be thoroughly rethought.
But why build your own sign-in system at all?
, , - the modern web is full of distributed, decentralised identity systems. We could have just forgotten about building our own system, and just implemented one, or all, of these.
Well, the good news is they're on their way! ´óÏó´«Ã½ iD was built from the ground up to be compatible with and other distributed authentication systems and later this year, we'll be introducing the ability for you to sign in to ´óÏó´«Ã½ Online using your Facebook login via Facebook Connect, and your Google and Yahoo logins (and more) via OpenID.
However, we still felt we needed our own base-level sign in system, both for those users who don't have external logins they want to use, and also for those who just don't want these things linked together. As the ´óÏó´«Ã½ has a mandate to serve all licence fee payers, building our own standalone system was a necessary evil.
Truly, Single Sign On
The biggest problem with the old SSO system was that, although it was actually a bbc-wide sign on system, almost none of our users realised this. It was mainly down to some user-experience descisions within the SSO interface. While a tiny percentage did use their SSO account for more than one service, nearly everyone created a new SSO account for each ´óÏó´«Ã½ service they registered for. We're trying to move ´óÏó´«Ã½ Online to become a more social, more coherent website. As such, it's essential that our users realise they're signing into the whole ´óÏó´«Ã½ site - not just a part of it.
With the old SSO model, we had ghettos of interactivity which didn't connect with each other or the rest of the site; each had their own users, their own rules and their own user interfaces. This made it impossible to represent users on every part of ´óÏó´«Ã½ Online consistently.
´óÏó´«Ã½ iD solves this problem in two ways.
Firstly, you can only have one ´óÏó´«Ã½ iD per email address. This is made clear as soon as you try and create a second ´óÏó´«Ã½ iD with the same email address. A single ´óÏó´«Ã½ iD can be used across ´óÏó´«Ã½ Online and a person can have more than one ´óÏó´«Ã½ iD, but they'll need a separate personal email address to register with for each one. Contrary to some comments on our blogs, ´óÏó´«Ã½ iDs are not limited by IP address, so you can have more than one per household. The email address is the important unique field.
Secondly, we created a 'brand' for our login. We're not the first to do this, , , all do it. And remember ? We'd rather not have called it anything, but we did lots of testing that showed that people didn't realise their login was global across our site unless we branded it. We've been careful to keep is a 'soft' brand though. It's represented by colour, language and iconography. This consistent message should remind users where ever they see the 'Cid' symbol (Cid's the bod on the badges pictured above, derived from BBC iD) and the words 'sign in', that they can use the same sign in details they use elsewhere on ´óÏó´«Ã½ Online.
By contrast, SSO's sign in and register pages were branded to match the service you came from - further reinforcing the impression that SSO was service-specific sign in.
But it's a pain to upgrade
Yes it is. Transitioning users from the old system to the new system is not easy. We could have just copied all the old user data from SSO into our new system, but that would have meant millions, literally millions, of old, dead unused accounts in our nice, clean, new system. Instead, we chose to allow our users to 'upgrade' their old SSO accounts to ´óÏó´«Ã½ iD. While this is a little annoying for some users, it is a one-time only process, and means the users we have in ´óÏó´«Ã½ iD have new, clean data - and best of all, it means people can register with sensible usernames again. With 13 million accounts created over 8 years, SSO was full of old, bad data.
We take our users' experiences very seriously, so we've done all we can to make the upgrade process simple, reliable and quick. There will always be some people who experience problems, but we monitor our stats and our help email addresses very closely and try and help each and every one of our users who has problems.
Will it be worth it?
The short answer is, yes.
Change is often disruptive, but necessary. The rollout of ´óÏó´«Ã½ iD across ´óÏó´«Ã½ Online will allow our site to do incredible new things - more personalisation, better interactivity and provide more security to our users. Without this move to use ´óÏó´«Ã½ iD, ´óÏó´«Ã½ Online would not be able to build, grow and become a properly modern interactive, coherent site.
Simon Cross is the Product Manager for ´óÏó´«Ã½ iD.
Comment number 1.
At 12th Mar 2010, Mina - waiting for the gift of sound and vision wrote:I don't understand the stuff about facebook, and I don't want to be signed in to any bbc pages other than the community I use. I don't want you spying on me when I read the news!
And out of interest, the site is called ´óÏó´«Ã½ Online again? How things stay the same when they change...
Complain about this comment (Comment number 1)
Comment number 2.
At 12th Mar 2010, Mo McRoberts wrote:iD will be extended to support OpenID logins? Ace!
(Presumably I’ll be able to attach an OpenID to my existing iD, though… right?)
Complain about this comment (Comment number 2)
Comment number 3.
At 12th Mar 2010, Hyperstar wrote:Should just stick to the ´óÏó´«Ã½ ID for all of the ´óÏó´«Ã½ site. Facebook causes problems
Complain about this comment (Comment number 3)
Comment number 4.
At 13th Mar 2010, Mark Stickley wrote:@Mina: Facebook connect allows you to sign in to sites other than Facebook using your Facebook sign in details. The other sites do not know or store your Facebook sign in details but it means you have to remember one less set of credentials. This is what we're planning on adding to ´óÏó´«Ã½ iD in the near future. Signing in to ´óÏó´«Ã½ iD does mean that you're signing in across the whole of the ´óÏó´«Ã½ website, not just a single community, but don't worry, we don't spy on you while you're reading the news! It just means that if you want to leave a comment on Have Your Say then you'll already be signed in from when you were talking in your community.
@Mo: Yep, OpenID will be supported around the same time as Facebook connect. You certainly will be able to link your OpenID account as well as your Facebook account. As was mentioned in the article, we don't like the idea of people having multiple accounts so it only seems natural we should let people do this.
@Hypestar: It'll still be ´óÏó´«Ã½ iD but people will be able to sign in to it with Facebook. Don't worry, we have some awesome developers here with loads of Faceboko Connect experience so we shouldn't have too many problems...
Complain about this comment (Comment number 4)
Comment number 5.
At 13th Mar 2010, Paul Livingstone wrote:Will the ´óÏó´«Ã½ iD ever be used to restrict content viewing? The scenario I'm imagining is: your ´óÏó´«Ã½ iD is linked with your TV license details and if you've not payed your dues, then you wont be able to use the iPlayer service.
Is that part of the plan, I wonder?
Complain about this comment (Comment number 5)
Comment number 6.
At 13th Mar 2010, TV Licence fee payer against ´óÏó´«Ã½ censorship wrote:#5. At 11:35am on 13 Mar 2010, Paul Livingstone wrote:
"The scenario I'm imagining is: your ´óÏó´«Ã½ iD is linked with your TV license details and if you've not payed your dues, then you wont be able to use the iPlayer service."
Well I suppose anything is possible, technically speaking, but until the ´óÏó´«Ã½ iD for your TVL number, name and address (none of which they do at the moment, unless things have changed radically since I created my iD) there will be no way of linking peoples iD with an address - even more so when one can sign-up using 'chuck-away' email addresses such as Gmail and Hotmail.
Complain about this comment (Comment number 6)
Comment number 7.
At 13th Mar 2010, Mark Stickley wrote:@Paul: No, there are definitely no plans for anything quite so nefarious! The closest we'll get to content restriction is preventing children from accessing mature or unsuitable content and likewise preventing adults from accessing children's message boards etc.
Complain about this comment (Comment number 7)
Comment number 8.
At 13th Mar 2010, TV Licence fee payer against ´óÏó´«Ã½ censorship wrote:#7. At 3:44pm on 13 Mar 2010, Mark wrote:
"@Paul: No, there are definitely no plans for anything quite so nefarious! The closest we'll get to content restriction is preventing children from accessing mature or unsuitable content and likewise preventing adults from accessing children's message boards etc."
How the heck are the ´óÏó´«Ã½ going to police such a measure, how will you know that little Jimmy signing up for his ´óÏó´«Ã½-iD is actually just 10 and not 25 as stated, how are you going to know that Little Jane isn't actually a 40 year old bloke in a dirty raincoat, also, assuming that you did mean accessing and not posting-to children's message boards etc, you do realise that that you will be preventing parents (or other adults who might have legitimate reasons to access such message boards etc.) from checking on what their kids/charges are reading and/or contributing to - short of the adults logging-on with the kids computers/accounts, but hang on, we're back to adults pretending to be kids again...
The ´óÏó´«Ã½ should NOT be playing the "net-nanny", for one thing it lulls both parent and child into a fail sense of security were the internet is concerned. No one wants to see kids get hurt but wrapping them up in cotton-wool doesn't protect them when they find they don't have the cotton-wool coat on, it's as crazy as teaching a kid how to cross the road by only ever using a one-way street, as soon as the kid comes across the more usual two-way street...
Complain about this comment (Comment number 8)
Comment number 9.
At 13th Mar 2010, JoeAD wrote:So will iPlayer also be migrated from Pearl to Forge?
Complain about this comment (Comment number 9)
Comment number 10.
At 14th Mar 2010, Ed Lyons wrote:Boilerplated: I believe the ´óÏó´«Ã½ already do this - there was some kid-orientated page I came across a few months ago and it asked me to log in, then complained I was too old. I had to create a new account with a fake DOB to see the site. No idea what page it was though.
I'm never entirely sure how such a system can be enforced - it's fairly easy to enforce drinking regulation as you're checking people are over an age - and expecting ID, but kids don't have ID (and shouldn't have to!)
You can bet the government has some scheme planed to give all kids ID cards to help solve this problem.
Complain about this comment (Comment number 10)
Comment number 11.
At 14th Mar 2010, Russ wrote:Simon - thanks for your explanation of the new system (to which I had no problem migrating, in fact it was seamless in my case). It is a bit of a surprise to hear about the perception of the old system not being '´óÏó´«Ã½-wide' - I had always regarded it as such.
I have a question regarding the 'old, dead, unused accounts'. In the current messageboard system, my entries are shown as "Russ (U2360818)". On the assumption that the new ID system is essentially an 'opt-in' one (in the sense that a user has to confirm current details), what will be shown for those users who have not upgraded to the new system, either because they are not around anymore, or because they may have ditched a previous identity?
I ask this question because I do often look back over old messageboard threads, and it would be a shame if the identity of many 'old' users was wiped.
Russ
Complain about this comment (Comment number 11)
Comment number 12.
At 14th Mar 2010, Mark Stickley wrote:@Boilerplated: You're right of course - there's no way to be totally sure that someone hasn't created another account with a different date of birth. This is something we can't solve completely and will not attempt to do so. What we have created is a system that goes as far as it can to address the problem. It's not watertight in that respect but then it doesn't claim to be so, nor does it promote itself in that way. In fact the only time anyone would come across this feature is when trying to access something they are not permitted to access.
For the casual browser this will be enough to stop them. For someone slightly more interested, they may try registering another account but most people would stop when it complained that they had already registered with that email address.
As for accessing as opposed to posting, that is at the discretion of the message board or service in question.
Complain about this comment (Comment number 12)
Comment number 13.
At 14th Mar 2010, TV Licence fee payer against ´óÏó´«Ã½ censorship wrote:12. At 5:56pm on 14 Mar 2010, Mark wrote:
"@Boilerplated: You're right of course - there's no way to be totally sure that someone hasn't created another account with a different date of birth. This is something we can't solve completely and will not attempt to do so. What we have created is a system that goes as far as it can to address the problem. It's not watertight in that respect but then it doesn't claim to be so, nor does it promote itself in that way."
So why do it, the fact is that SOME parents WILL think it's water tight - it is the ´óÏó´«Ã½ after all, a trusted brand...
What you have created is like one of those houses that has a dummy burglar alarm boxes on the front wall, no passers-by bother about suspicious activity as they assume that the property is protected by said alarm...
"In fact the only time anyone would come across this feature is when trying to access something they are not permitted to access."
Who says they are not permitted to enter, I though we were talking about the public social media areas of the ´óÏó´«Ã½ not internal Intranet areas, who are you to tell a parent that they can't access (on their own accounts) the areas were their kids visit.
"For the casual browser this will be enough to stop them. For someone slightly more interested, they may try registering another account but most people would stop when it complained that they had already registered with that email address.
As for accessing as opposed to posting, that is at the discretion of the message board or service in question."
So again the parent or legitimate adult gets rejected whilst the 40 year old man in the old raincoat and that nice fresh throw-away hotmail or gmail account gets accepted!
Unbelievable cluelessness, how long and how much money did this cost us...
Complain about this comment (Comment number 13)
Comment number 14.
At 15th Mar 2010, Paul Murphy wrote:#8 #13
In a previous post and its comments (/blogs/bbcinternet/2009/10/bbc_trusts_good_news_for_cbbc.html) Marc Goodchild, Head of Interactive and On Demand for ´óÏó´«Ã½ Childrens, has talked about many of the issues brought up here around child safety on ´óÏó´«Ã½ sites, and in particular why things have been done the way they have.
Complain about this comment (Comment number 14)
Comment number 15.
At 15th Mar 2010, TV Licence fee payer against ´óÏó´«Ã½ censorship wrote:In reply to Paul @ 14:
Thanks for reminding me of that other blog were I (also) asked difficult questions about the ´óÏó´«Ã½'s policy in respect of on-line security, the C´óÏó´«Ã½ "brand" and were the host decided to walk away after posting the one follow up, basically repeating official policy position rather than entering any sort of debate...
Complain about this comment (Comment number 15)
Comment number 16.
At 15th Mar 2010, DarkStar111S wrote:Sign in to sign to sign in, to What ?
So, it says my under 16 year-old son to 'protect' him must have a ´óÏó´«Ã½ 'sign-in' ID to play a game on the C´óÏó´«Ã½ web-site. Oh and once he's signed in I, his parent must give my consent, by e-mail. Yes, he doesn't have an e-mail address, because he is 8 years-old, he doesn't go near a keyboard without me at his shoulder. So, I enter my e-mail address and 'no this e-mail address is already being used' yes by ME - BECAUSE I'M SIGNED IN !!!!!
25,000 staff and not one of them capable of understanding a log-in script. I give up, but then £3.5Bn in 'unique funding' can't be wrong.
Complain about this comment (Comment number 16)
Comment number 17.
At 16th Mar 2010, Mark Stickley wrote:@DarkStar: Really sorry you're having so many problems. If you could put your problem into an email (and include the URL for the game you are trying to sign your son up for) and send it to membership@bbc.co.uk we will work together to fix this issue.
Complain about this comment (Comment number 17)
Comment number 18.
At 19th Mar 2010, CBers wrote:So, with a ´óÏó´«Ã½iD, will it be possible to link your TV licence to it, so that when abroad, it's still possible to view iPlayer content as though you were in the UK ??
This is one of the biggest problems with iPlayer when on holiday - I can't watch anything as I am "not in the UK", even though I am a UK rewsident on holiday with a valid TV licence.
Complain about this comment (Comment number 18)
Comment number 19.
At 19th Mar 2010, Simon Cross wrote:@CBers no, we've got no plans to connect your ´óÏó´«Ã½ iD to your TV Licence.
Sadly, the iPlayer and most other AV service will continue to be limited by IP address - i.e. WHERE you are accessing it from.
Complain about this comment (Comment number 19)
Comment number 20.
At 20th Mar 2010, Sesli Chat wrote:iD will be extended to support OpenID logins? Ace!
Complain about this comment (Comment number 20)
Comment number 21.
At 22nd Mar 2010, Simon Cross wrote:@Sesli Chat Yep - that's what we're working on right now!
Complain about this comment (Comment number 21)
Comment number 22.
At 12th Apr 2010, charlie wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 22)
Comment number 23.
At 12th May 2010, U14460911 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 23)
Comment number 24.
At 12th May 2010, U14460911 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 24)
Comment number 25.
At 20th May 2010, talat wrote:All this user's posts have been removed.Why?
Complain about this comment (Comment number 25)
Comment number 26.
At 24th May 2010, hd2010 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 26)
Comment number 27.
At 28th May 2010, Manu wrote:It’s a good idea to launch the ´óÏó´«Ã½ iD. In my opinion, the PHP and MYSQL languages are best than Perl and MySQL.
´óÏó´«Ã½ iD was built from the ground up to be compatible with OpenID. This is a good idea.
[Unsuitable/Broken URL removed by Moderator]
Complain about this comment (Comment number 27)
Comment number 28.
At 2nd Jun 2010, softdinamic wrote:I saw above that will be available for open ID... is it ready?
Complain about this comment (Comment number 28)
Comment number 29.
At 30th Jun 2010, suzyanne wrote:I promised my 10 year old and 6 year old boys that i would like to express joy with the programme 'Tinga Tales' currently on Cbeebies daily at 07.40. The only problem is that you probably have not that many and we see the same ones again and again.
My boys love both the questions and the answers. The beautiful African voices given to the animals are wonderful and sound very authentic. Even Lenny Henry sounds Africa.
So please, quickly make another series with more questions to answer!
suzy-anne lees
xx
Complain about this comment (Comment number 29)
Comment number 30.
At 10th Sep 2010, munindra wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 30)
Comment number 31.
At 10th Sep 2010, munindra wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 31)
Comment number 32.
At 10th Sep 2010, munindra wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 32)
Comment number 33.
At 25th Oct 2010, U14661414 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 33)