Still Playing in the Cloud
Post categories: R&D
| 16:54 UK time, Thursday, 27 March 2008
Matt Wood (we really need to sort him out his own login) gives an update on his XMPP experiements...
I've made a little time recently to move my previous XMPP experiments forward a little. Here are the results.
Live Text
I've started echoing the stream of text that passes across the little screens of DAB radios into XMPP space. We call it 'live text' and it's frequently entered by production staff in a studio as a show is actually being broadcast, so it's timely but with a very short half-life. Seemed like a good fit for XMPP delivery!
You can find the live text at grouphug.hellomatty.com: point your client at the group chats at radio1@grouphug.hellomatty.com, radio2@grouphug.hellomatty.com, radio3@grouphug.hellomatty.com... Feel free to hook this into your own applications...
The backend was again written with which I'm still finding very easy to work with.
We currently use a very lightweight but proprietary protocol and server to get this text into the Flash widgets for iPlayer and the radio network sites. On the assumption that it would be just better to use an open protocol for this I'm hoping to spend some time with soon to see whether we can make XMPP scale comparably. If you've any scaling war stories I'd really like to hear them - to be competitive we'll need to support at least 100,000 concurrent clients without breaking a sweat.
Federation
By popular request I've switched federation on at hug.hellomatty.com - this means you no longer need a hug.hellomatty.com account to subscribe to the now playing and live text streems - just use your regular jid or gmail account.
XMPP over HTTP and Javascript
Paul asked me whether XMPP could go through firewalls and the answer is . This is important if you've a large client base any number of whom may be behind a corporate firewall that only allows 'safe' traffic through. The BOSH spec allows you to tunnel XMPP over HTTP in a way that doesn't hammer your server in a polling frenzy but which still respects timely content deliver and which plays nicely with proxies.
I've hacked together a couple of small demos to show how this works, and how easy it can be to put together XMPP apps with Javascript. They're built on top of , a compact and robust XMPP client library which handles the BOSH mechanics for you - your code sees a single, 'persistent' connection object. These implementations are pretty naive and I'm going to maintain that this is to make the code as easy to follow as possible.
Very Live Text!
The pulls the live text for Radio 4 into your browser and hopefully shows how little code you need to script XMPP from your browser. It also shows you how to do 'anonymous' logins, and how to work with JSJaC's callback-based programming model.
Choon!
The tracks the 'now playing' feed I presented last time. Here you can see basic support for the XMPP extensions you need to get 'user tunes' working (rosters, entity capabilities and personal eventing via pubsub) layered on top of JSJaC.
I also noticed that the tune spec was updated to allow you to publish your rating of the tune that's currently playing. So with this app you can click 'Pop!' or 'Drop!' buttons to publish your (dis-)pleasure with our playlists. From a public service perspective a federated standard for scrobbling certainly seems kind of interesting, especially if you worry about who owns your attention profile.
Next...
Next up is some benchmarking with Tsung, and I've a couple more toys in the pipeline. More here as it happens...
Your comments are, of course, very welcome!
Hi,
I know of a couple of implementations above 100k users, concurrent.
But if you use Federation, you would not have this problem, just thousands of connections to the servers that have the users.
Best regards,
PS: You should join the XMPP Social mailing list. Details here:
Are you sure federation is working? I've tried adding radio4@grouphug.hellomatty.com as a buddy in 2 different XMPP accounts, but it doesn't authorize.
@scott - yup, i think it's working but my instructions obviously aren't clear :-( The 'grouphug...' jids (such as radio4@grouphug.hellomatty.com) are actually multi-user chatrooms into which I'm pumping the live text. So you don't need to add them as a buddy - you should join them as a 'groupchat' in your client. The 'hug...' jids (such as radio1@hug.hellomatty.com) publish the 'now playing' feed and you can add these as buddies. Hope this helps!
@scot - yup, i think it's working but my instructions obviously aren't clear :-( The 'grouphug...' jids (such as radio4@grouphug.hellomatty.com) are actually multi-user chatrooms into which I'm pumping the live text. So you don't need to add them as a buddy - you should join them as a 'groupchat' in your client. The 'hug...' jids (such as radio1@hug.hellomatty.com) publish the 'now playing' feed and you can add these as buddies. Hope this helps!
interesting like always.
thanks for the tipoff about JSJaC, too.