´óÏó´«Ã½ launches Enhanced Search
Earlier this week we began a phased roll-out of our new site search. New features include:
- A brand-new search which intelligently tailors the display to the specific query the user enters
- The addition of featured content to search result pages
- Improved links to non-´óÏó´«Ã½ content
- More accurate results through improved meta-data creation
Our aim in designing this new search experience has been to present search results in a way that makes sense of the huge variety of content available on bbc.co.uk. It should be effortless to find a specific piece of content and enjoyable to explore everything that we have on a subject.
The richness and diversity of the ´óÏó´«Ã½'s internet content (News, Blogs, iPlayer, Weather, Sport, Recipes and so on) places demands on site search that are different from more focused websites and this led us to explore original solutions to enterprise search.
As you will see in this post, our solution is a major departure from traditional designs. So, for the rest of this post, I'd like to give you some insight into how we reached our decision to make such a significant change to the way we deliver our search results.
As always we would love to hear your views on these changes.
The challenge of site search
Last year, Erik Huggers challenged our team to "deliver a step-change" in the quality of site search.This was a daunting challenge, but as always the first thing to do was to define the problem. This meant analysing what users are searching for and which results they are clicking on.
To explain what we found, consider this list of the 50 most popular searches in December 2009.
The third column shows the click-through rate (%CTR) for each term, i.e. what percentage of people who searched for that term actually decided to click on a result. The higher the %CTR the more satisfied we believe users to be.
Notice that I have colour coded the different search terms. Green are searches where the user is looking for something specific: usually a part of the ´óÏó´«Ã½ website (iPlayer, the Weather website, Bitesize) or a named programme (EastEnders, Top Gear, Doctor Who). Red ones are searches where the user is looking for something about a topic - a particular person, a country or city, a subject or news event.
Look what happens when you average the %CTR for these two types of search:
Looking for something specific 83.86%
Looking for something about a topic 63.91%
As you can see when you want to get something specific, the ´óÏó´«Ã½ site search is pretty effective. But if you want to find things about a person, place, subject or event the experience is patchy.
So we began to re-develop site search to improve this type of search, which I call Topic Searches.
To start with we asked "why are Topic Searches less successful?" The answer is surely that the intentions of the person doing this type of search are less clear. When you search for merlin or top gear or travel news the intended results are easy to predict. But when you search for iran, what do you want? The latest news about Iran? Some background information on the country? A programme that's available on iPlayer?
What about searches for Delia Smith? Do you want to know when her next programme is being broadcast? Do you want to follow one of her recipes?
Results display: the problem with search result lists
We then looked at how the results were being presented and whether this was hindering users' understanding and therefore depressing the click through rate.The traditional way to present search results is as a single list ordered by relevance to the search term. This has been our approach up until now. But it has never been great for topic searches. Here is what one interviewee said during some research we carried out:
"The search just throws everything at you, you would have to spend ages just looking through the pages to find what you need."
There are two big problems that make a single list approach very difficult:
- You need to understand what the user means to know what relevance scores to give to their results. Two searchers may want two different things but use the same search term - one user may search for swine flu looking for a description of the symptoms whereas another may use exactly the same search term but be interested in news about the latest outbreaks.
- Different types of content are relevant for different reasons. The most relevant news article is the most recent one (unless of course someone is looking for a specific article). Whereas for archive programmes or background articles date is much less important, the most relevant thing is the one "most about" the subject.
This means that when you try to produce a homogenised, "relevance ranked" list of results you need to add biases and boosts to different types of content to get the right results to the top of the list. With the diverse content on offer on bbc.co.uk, this is nearly impossible.
(It's worth noting that the major search engines have also been moving from a single list view to a more modular set of results over the last few years. Google for instance include modules of results from their verticals - News, Video, Blogs, Maps and now Tweets - blended into their Web results. Google's Marissa Mayer laid out , Danny Sullivan has too.)
So our solution rejects the single list approach and instead introduces "Smart Zones".
The New, Enhanced ´óÏó´«Ã½ Site Search: Smart Zones
This is a close-up taken from the results page I got when I searched the new system for David Cameron on Friday.
As you can see, the results are split up into sections or "zones" - in this case: News, iPlayer and Knowledge (the zone which holds background and in-depth content). The content in each zone is ordered in the most appropriate way - so that you see the latest News but the most relevant Knowledge items.
Of course zones are nothing new in search result pages but - this is the clever bit - these zones order themselves on the page depending on the search query, the matching content available and what we think users find most important. So every time you search you will see the best possible ordering of available results.
For example, compare the results above with a search for David Cameron I did on Sunday morning just after he had been interviewed on The Andrew Marr Show. You can see that the News and iPlayer zones have swapped places as the system has calculated that the interview is likely to be the most popular result.
When I searched for Ryan Giggs, the Sport zone appears in the results.
Searchers can see all zones or focus in on one particular zone. Here I have searched for Jonathan Ross and I have selected to expand the News zone to see more News content (note all the other zones are still available on the right of the screen reminding you that other relevant content is available).
Other Enhancements to Search: Highlighted Content
This is a search for George Orwell. As well as search results we also show profiles for certain people. In this case it is from the ´óÏó´«Ã½ History site.
Likewise, a search for India returns the ´óÏó´«Ã½ Country Profile and the latest weather forecast. (Note that no new content is being produced here, we are just finding new ways to draw attention to our best web pages). We will be working to increase the amount of highlighted ´óÏó´«Ã½ content over the coming months and highlighting the best content from other websites.
Other Enhancements to Search: Extra results from around the web
Here I have highlighted another zone that appears for most searches - Around the Web. This shows relevant content from other news providers. In the image below the search was for Simon Cowell.
Other Enhancements to Search: Extended query terms
Here is a close-up of a section of the results for a search for Prince Charles. Some journalists will refer to the Prince as Prince Charles, others as The Prince of Wales. Our system knows this and automatically includes synonyms of your search terms (as shown in the detail below).
Other Enhancements to Search: Better results through added content structure
One of the advantages that site searches have over internet web search engines is that we can influence the way content is produced to improve the quality of our search results. We have put a system in place that uses search to suggest the most relevant tags for ´óÏó´«Ã½ content. Content producers accept or reject these tags before they are committed to the system. The tags are then used to influence the results that are returned when you search the website.Note this system is in its infancy so it will take some time before the improvements in result relevance are apparent for every search.
Other Enhancements to Search: Addressable search results
All search result pages have unique and persistent urls. This makes it very easy for people to link to everything that the ´óÏó´«Ã½ has about a topic. In time public APIs will allow other systems to request feeds of content about a topic.
Phased Roll-out
The roll-out will be phased. We'll be monitoring our metrics (usage, click-throughs etc) and the feedback we receive to tune the product during the deployment cycle.At the moment only about 11,000 search terms will return the enhanced results detailed above.
Over the coming weeks more and more searches will return the enhancements until every search works in this way. There will be additional zones added into the results and we have a slate of new features that are in development.
We will also be using the technologies we have built here to produce exciting new applications. One of the objectives we are focusing on is how we can use these tools to make you aware that there is new content about things that are of interest to you as soon as it is published. Watch out for something special when the ´óÏó´«Ã½ Homepage re-launches in the very near future.
Matthew McDonnell is Head of Search, ´óÏó´«Ã½.
Comment number 1.
At 13th Jan 2010, SheffTim wrote:‘Google’ and ‘Facebook’ may be in your list from user attempts to reach those sites.
Would you put links to external sites in your results?
Complain about this comment (Comment number 1)
Comment number 2.
At 13th Jan 2010, Matthew McDonnell wrote:You are quite right SheffTim, a lot of people search for external sites and we should give them links to those sites as the first result. We will be rolling out this feature soon and have chosen not to enhance results for searches for 'Google' and 'Facebook' until it is in place (the existing search does carry external website links though not as prominently as I would like).
Complain about this comment (Comment number 2)
Comment number 3.
At 13th Jan 2010, Josh wrote:@SheffTim: Could it not be that people were searching for, say, news articles about Google and Facebook, rather than the sites themselves?
Complain about this comment (Comment number 3)
Comment number 4.
At 13th Jan 2010, Matthew McDonnell wrote:@saxsux: You are also right. This is the idea that we are trying to bring out in the new design - if you are just looking to navigate to Facebook then here is the link; if you want to read about facebook then here is the related news, programmes and so on.
Complain about this comment (Comment number 4)
Comment number 5.
At 13th Jan 2010, Frankie Roberto wrote:Do you take into account the page that the user is searching *from*? For example, if I'm searching from the ´óÏó´«Ã½ News website, then I'm more likely to be interested in ´óÏó´«Ã½ News articles, and so you should show that module more prominently, in the same way that if you're searching from iPlayer, you're more likely to be interested in iPlayer content (which is currently all you get).
Complain about this comment (Comment number 5)
Comment number 6.
At 13th Jan 2010, Matthew McDonnell wrote:Frankie Roberto - it currently doesn't but it almost certainly will in the future. It is too valuable a clue to the user's intention to ignore. The question is *how much* should the location you search from influence the result mix? We need to balance it right so that if the most interesting thing we have is not from the section of the site you have come from, it is not lost in the mix.
Complain about this comment (Comment number 6)
Comment number 7.
At 13th Jan 2010, Sesli Sohbet Sesli Chat wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 7)
Comment number 8.
At 13th Jan 2010, Sesli Sohbet Sesli Chat wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 8)
Comment number 9.
At 13th Jan 2010, Sesli Sohbet Sesli Chat wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 9)
Comment number 10.
At 14th Jan 2010, Charles Knight wrote:Requesting permission to republish this article with full attribution.
House rules prohibit giving my blog URL or my contact information?
Contact Us page was uneventful.
Complain about this comment (Comment number 10)
Comment number 11.
At 14th Jan 2010, Matthew McDonnell wrote:Charles Knight: republishing with attribution is fine. Sorry you had such trouble.
Complain about this comment (Comment number 11)
Comment number 12.
At 14th Jan 2010, Nick Reynolds wrote:Hi Charles - the house rules shouldn't stop you linking to your blog in a comment. It would depend on whether you were spamming or not!
Why not simply use an extract from Matt's blog and then link to the full text?
Complain about this comment (Comment number 12)
Comment number 13.
At 14th Jan 2010, Keith wrote:Hopefully some of the URLs will tidied up so that eventually they all use /search, as some URLs still refer to the /topics beta. It's nice to see more of the site are getting integrated into the search. Would be nice if in time it could eliminate the secondary search box which appears on some sections of the ´óÏó´«Ã½ (eg. h2g2).
Complain about this comment (Comment number 13)
Comment number 14.
At 14th Jan 2010, Matthew McDonnell wrote:Keith: The /topics URLs and the the equivalent /search URLs return the same results. This is to retain the integrity of any links people might have made to the Topic Pages we created last year. In time we will probably retire the /topics URLs and redirect them to /search.
The next aim of this project is to harmonise all the different search experiences on the ´óÏó´«Ã½ website. This will address the two search boxes issue among other things. I will post in more detail once our plans are firmed up.
Complain about this comment (Comment number 14)
Comment number 15.
At 14th Jan 2010, John99 wrote:It allways helps to give the user more options to customise a search, I guess the difficult part is trying to make it easy enough for the majority of users to be able to see and utilise the additional features.
It does seem to be an improvement from a cursory glance at the blog and a quick trial of some search terms.
You compare with google, they of course also have an 'advanced search' option from a link adjacent to the search field expanding the customisation even more for those making the effort, will you be considering anything similar
Complain about this comment (Comment number 15)
Comment number 16.
At 14th Jan 2010, Charles Knight wrote:Matthew and Nick:
Thank you. Using an excerpt and linking to the original article is indeed the way to go.
Cheers,
Charles Knight, editor
www.AltSearchEngines.com
Complain about this comment (Comment number 16)
Comment number 17.
At 14th Jan 2010, John99 wrote:Your top ranked search term is 'iPlayer'
Some users may want to search the iPlayer MessageBoards. Is the search facility of that going to be improved. (either within the messageboard or within the main ´óÏó´«Ã½ search) For example how about the ability to search the message board thread titles, surely a useful idea.
Currently the messageboard host suggests using an external search engine ! if searching the threads. The help/faq section of the iPlayer messageboard is searchable but it is pretty basic.
Complain about this comment (Comment number 17)
Comment number 18.
At 14th Jan 2010, John99 wrote:On the subject of iPlayer this time the programme search.
This often seems to give slightly unexpected results, is this going to be made to work well, and/or enhanced also.
Complain about this comment (Comment number 18)
Comment number 19.
At 14th Jan 2010, Matthew McDonnell wrote:John99: Three good points.
You are right about getting the balance right between catering for the casual user and the more active user. Our plan is to enable zone-specific filters to allow active users to specify exactly what they want.
Messageboard searching is on the roadmap for development.
Have you an example of the unexpected iplayer results you were seeing? iPlayer search is something we are always aiming to improve so any specific problems are very good to know about.
Complain about this comment (Comment number 19)
Comment number 20.
At 14th Jan 2010, John99 wrote:Matthew: Thanks for the reply
"Have you an example of the unexpected iplayer results you were seeing?"
Not sure I remember last one that I cam e across, but one I did post a reply about was the difficulty of finding the R4 programme called Today. Which is not returned when 'today' is typed in (I just tried it it still does not work) see thread where you will note the original poster was complaining that this is a re-occurring problem.
I am sure you will find many other threads about similar problems, but we need to use search engines to try to find them at present!
PS
How do I put quotes in, 'quote' tag used in iPlayer mb does not work neither did 'q' or 'blockquote', or maybe I got syntax wrong.
Complain about this comment (Comment number 20)
Comment number 21.
At 15th Jan 2010, Matthew McDonnell wrote:John99: Re Today search in iPlayer. I am looking into this. In the meantime a search on the main ´óÏó´«Ã½ site brings back the right result -
Nick Reynolds may be able to help you with the iPlayer messageboard problem.
Complain about this comment (Comment number 21)
Comment number 22.
At 15th Jan 2010, John99 wrote:Anyone : general enquiry re use of this blog
Are quotations from other comments used on these blogs, if so how do I enter them? (Is any method standard across most ´óÏó´«Ã½ blogs now ?)
I asked above [msg # 20]
"How do I put quotes in, 'quote' tag used in iPlayer mb does not work neither did 'q' or 'blockquote', or maybe I got syntax wrong."
My wording was rather clumsy. On the iPlayer mesageboard the recommended method as in this link often used by posters on that mb is the 'quote' tag. I only know how to to use bold or very small on this blog to try to make a quote stand out.
@Matthew [msg #21]
Thanks for reply, I was giving an example as you requested, & chose a current problem.
Complain about this comment (Comment number 22)
Comment number 23.
At 16th Jan 2010, John99 wrote:Typo msg#24 corrected link
Complain about this comment (Comment number 23)
Comment number 24.
At 17th Jan 2010, mikehill33 wrote:Matthew,
Kudos on an extremely innovative approach to search. Your ideas of relevance and content grouping are light years ahead of your time.
would love to know what is on your bookshelf and who is in your Twitter feed.
I am an Information Architect and am just embarking on proposing a radical search experience, and am definitely using your site as a solid example of a site that "gets it".
Good luck on your data tuning, I will definitely be watching!
Complain about this comment (Comment number 24)
Comment number 25.
At 18th Jan 2010, onlinewill wrote:I searched there, but for the terms that I am searching, I see good results, however, it says for UK visitors only. And it does not give any RSS opportunities as google search does.
Complain about this comment (Comment number 25)
Comment number 26.
At 18th Jan 2010, Matthew McDonnell wrote:John99: I have been meeting this morning about the problem with the Today Programme not appearing in iPlayer search results. Our engineers have a fix in place but rolling it out may take some time due to the pressures the iPlayer team are currently under. Rest assured, "Today" will be at the top of the results in iPlayer in the near future.
mikehill33: Thanks so much for your comments - it is great to get positive reactions from someone who works in this area and clearly knows the challenges of site search. I hope your radical search experience gets adopted by your clients and implemented. I think site searches around the web are in desperate need of some fresh ideas.
onlinewill: Where did it say "for UK visitors only"? There are some types of ´óÏó´«Ã½ content that are not cleared for international use (some iPlayer programmes for instance", did you see that message when you clicked on an iPlayer result? We are planning to adjust to take account of international rights limitations in the future.
RSS feeds of search results is a top priority that you should start to see pretty soon.
Complain about this comment (Comment number 26)
Comment number 27.
At 25th Jan 2010, SheffTim wrote:The new search makes it harder to locate the ´óÏó´«Ã½ Webwise online course materials. (I work in a learning centre.)
The first link to anything actually related to the Webwise course is #9, near the bottom of the page.
/webwise/course/tutors/qualifications.shtml
Even then it takes a further click to get to the course start page, if you know where to look.
It seems strange that Webwise isn't the top result (try it); it would certainly help those new to the Web if it were (hint), which it should be.
Complain about this comment (Comment number 27)
Comment number 28.
At 25th Jan 2010, Matthew McDonnell wrote:SheffTim: What search were you running?
Learning content (including webwise) isnt yet included in these search results. we are working on this at the moment and the results will cover all ´óÏó´«Ã½ output very soon. In the meantime traditional search is still available.
Complain about this comment (Comment number 28)
Comment number 29.
At 27th Mar 2010, U14390976 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 29)
Comment number 30.
At 2nd Apr 2010, Kim Gjerstad wrote:A happy reader is one that complains: please setup RSS for search results. We've been waiting. :-)
Complain about this comment (Comment number 30)
Comment number 31.
At 16th Apr 2010, Matthew McDonnell wrote:Kim: sorry for the delayed response. RSS feeds are in development and on their way soon. I'll post when they are available.
Complain about this comment (Comment number 31)
Comment number 32.
At 12th May 2010, U14460911 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 32)
Comment number 33.
At 20th May 2010, talat wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 33)
Comment number 34.
At 24th May 2010, hd2010 wrote:This comment was removed because the moderators found it broke the house rules. Explain.
Complain about this comment (Comment number 34)