´óÏó´«Ã½

« Previous | Main | Next »

´óÏó´«Ã½ launches Enhanced Search

Post categories:

Matthew McDonnell | 17:15 UK time, Wednesday, 13 January 2010

Earlier this week we began a phased roll-out of our new site search. New features include:

  • A brand-new search which intelligently tailors the display to the specific query the user enters
  • The addition of featured content to search result pages
  • Improved links to non-´óÏó´«Ã½ content
  • More accurate results through improved meta-data creation

Our aim in designing this new search experience has been to present search results in a way that makes sense of the huge variety of content available on bbc.co.uk. It should be effortless to find a specific piece of content and enjoyable to explore everything that we have on a subject.

The richness and diversity of the ´óÏó´«Ã½'s internet content (News, Blogs, iPlayer, Weather, Sport, Recipes and so on) places demands on site search that are different from more focused websites and this led us to explore original solutions to enterprise search.

As you will see in this post, our solution is a major departure from traditional designs. So, for the rest of this post, I'd like to give you some insight into how we reached our decision to make such a significant change to the way we deliver our search results.

As always we would love to hear your views on these changes.


The challenge of site search

Last year, Erik Huggers challenged our team to "deliver a step-change" in the quality of site search.

This was a daunting challenge, but as always the first thing to do was to define the problem. This meant analysing what users are searching for and which results they are clicking on.

To explain what we found, consider this list of the 50 most popular searches in December 2009.

The third column shows the click-through rate (%CTR) for each term, i.e. what percentage of people who searched for that term actually decided to click on a result. The higher the %CTR the more satisfied we believe users to be.

table_search_601.png

Notice that I have colour coded the different search terms. Green are searches where the user is looking for something specific: usually a part of the ´óÏó´«Ã½ website (iPlayer, the Weather website, Bitesize) or a named programme (EastEnders, Top Gear, Doctor Who). Red ones are searches where the user is looking for something about a topic - a particular person, a country or city, a subject or news event.

Look what happens when you average the %CTR for these two types of search:

Looking for something specific 83.86%
Looking for something about a topic 63.91%

As you can see when you want to get something specific, the ´óÏó´«Ã½ site search is pretty effective. But if you want to find things about a person, place, subject or event the experience is patchy.

So we began to re-develop site search to improve this type of search, which I call Topic Searches.

To start with we asked "why are Topic Searches less successful?" The answer is surely that the intentions of the person doing this type of search are less clear. When you search for merlin or top gear or travel news the intended results are easy to predict. But when you search for iran, what do you want? The latest news about Iran? Some background information on the country? A programme that's available on iPlayer?

What about searches for Delia Smith? Do you want to know when her next programme is being broadcast? Do you want to follow one of her recipes?


Results display: the problem with search result lists

We then looked at how the results were being presented and whether this was hindering users' understanding and therefore depressing the click through rate.
The traditional way to present search results is as a single list ordered by relevance to the search term. This has been our approach up until now. But it has never been great for topic searches. Here is what one interviewee said during some research we carried out:
"The search just throws everything at you, you would have to spend ages just looking through the pages to find what you need."

There are two big problems that make a single list approach very difficult:

  1. You need to understand what the user means to know what relevance scores to give to their results. Two searchers may want two different things but use the same search term - one user may search for swine flu looking for a description of the symptoms whereas another may use exactly the same search term but be interested in news about the latest outbreaks.

  2. Different types of content are relevant for different reasons. The most relevant news article is the most recent one (unless of course someone is looking for a specific article). Whereas for archive programmes or background articles date is much less important, the most relevant thing is the one "most about" the subject.


This means that when you try to produce a homogenised, "relevance ranked" list of results you need to add biases and boosts to different types of content to get the right results to the top of the list. With the diverse content on offer on bbc.co.uk, this is nearly impossible.

(It's worth noting that the major search engines have also been moving from a single list view to a more modular set of results over the last few years. Google for instance include modules of results from their verticals - News, Video, Blogs, Maps and now Tweets - blended into their Web results. Google's Marissa Mayer laid out , Danny Sullivan has too.)

So our solution rejects the single list approach and instead introduces "Smart Zones".


The New, Enhanced ´óÏó´«Ã½ Site Search: Smart Zones

This is a close-up taken from the results page I got when I searched the new system for David Cameron on Friday.


cam_600_01.png

As you can see, the results are split up into sections or "zones" - in this case: News, iPlayer and Knowledge (the zone which holds background and in-depth content). The content in each zone is ordered in the most appropriate way - so that you see the latest News but the most relevant Knowledge items.

Of course zones are nothing new in search result pages but - this is the clever bit - these zones order themselves on the page depending on the search query, the matching content available and what we think users find most important. So every time you search you will see the best possible ordering of available results.

For example, compare the results above with a search for David Cameron I did on Sunday morning just after he had been interviewed on The Andrew Marr Show. You can see that the News and iPlayer zones have swapped places as the system has calculated that the interview is likely to be the most popular result.

cam_600_02.png


When I searched for Ryan Giggs, the Sport zone appears in the results.

giggs_600.png

Searchers can see all zones or focus in on one particular zone. Here I have searched for Jonathan Ross and I have selected to expand the News zone to see more News content (note all the other zones are still available on the right of the screen reminding you that other relevant content is available).

ross_600.png


Other Enhancements to Search: Highlighted Content

This is a search for George Orwell. As well as search results we also show profiles for certain people. In this case it is from the ´óÏó´«Ã½ History site.

orwell_600.png


Likewise, a search for India returns the ´óÏó´«Ã½ Country Profile and the latest weather forecast. (Note that no new content is being produced here, we are just finding new ways to draw attention to our best web pages). We will be working to increase the amount of highlighted ´óÏó´«Ã½ content over the coming months and highlighting the best content from other websites.

india_600.png

Other Enhancements to Search: Extra results from around the web

Here I have highlighted another zone that appears for most searches - Around the Web. This shows relevant content from other news providers. In the image below the search was for Simon Cowell.


aroundweb_new.png

Other Enhancements to Search: Extended query terms

Here is a close-up of a section of the results for a search for Prince Charles. Some journalists will refer to the Prince as Prince Charles, others as The Prince of Wales. Our system knows this and automatically includes synonyms of your search terms (as shown in the detail below).

charles_600.png

Other Enhancements to Search: Better results through added content structure

One of the advantages that site searches have over internet web search engines is that we can influence the way content is produced to improve the quality of our search results. We have put a system in place that uses search to suggest the most relevant tags for ´óÏó´«Ã½ content. Content producers accept or reject these tags before they are committed to the system. The tags are then used to influence the results that are returned when you search the website.

Note this system is in its infancy so it will take some time before the improvements in result relevance are apparent for every search.


Other Enhancements to Search: Addressable search results

All search result pages have unique and persistent urls. This makes it very easy for people to link to everything that the ´óÏó´«Ã½ has about a topic. In time public APIs will allow other systems to request feeds of content about a topic.


address.png


Phased Roll-out

The roll-out will be phased. We'll be monitoring our metrics (usage, click-throughs etc) and the feedback we receive to tune the product during the deployment cycle.

At the moment only about 11,000 search terms will return the enhanced results detailed above.

Over the coming weeks more and more searches will return the enhancements until every search works in this way. There will be additional zones added into the results and we have a slate of new features that are in development.

We will also be using the technologies we have built here to produce exciting new applications. One of the objectives we are focusing on is how we can use these tools to make you aware that there is new content about things that are of interest to you as soon as it is published. Watch out for something special when the ´óÏó´«Ã½ Homepage re-launches in the very near future.

Matthew McDonnell is Head of Search, ´óÏó´«Ã½.

Comments

  • Comment number 1.

    ‘Google’ and ‘Facebook’ may be in your list from user attempts to reach those sites.
    Would you put links to external sites in your results?

  • Comment number 2.

    You are quite right SheffTim, a lot of people search for external sites and we should give them links to those sites as the first result. We will be rolling out this feature soon and have chosen not to enhance results for searches for 'Google' and 'Facebook' until it is in place (the existing search does carry external website links though not as prominently as I would like).

  • Comment number 3.

    @SheffTim: Could it not be that people were searching for, say, news articles about Google and Facebook, rather than the sites themselves?

  • Comment number 4.

    @saxsux: You are also right. This is the idea that we are trying to bring out in the new design - if you are just looking to navigate to Facebook then here is the link; if you want to read about facebook then here is the related news, programmes and so on.

  • Comment number 5.

    Do you take into account the page that the user is searching *from*? For example, if I'm searching from the ´óÏó´«Ã½ News website, then I'm more likely to be interested in ´óÏó´«Ã½ News articles, and so you should show that module more prominently, in the same way that if you're searching from iPlayer, you're more likely to be interested in iPlayer content (which is currently all you get).

  • Comment number 6.

    Frankie Roberto - it currently doesn't but it almost certainly will in the future. It is too valuable a clue to the user's intention to ignore. The question is *how much* should the location you search from influence the result mix? We need to balance it right so that if the most interesting thing we have is not from the section of the site you have come from, it is not lost in the mix.

  • Comment number 7.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 8.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 9.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 10.

    Requesting permission to republish this article with full attribution.

    House rules prohibit giving my blog URL or my contact information?

    Contact Us page was uneventful.

  • Comment number 11.

    Charles Knight: republishing with attribution is fine. Sorry you had such trouble.

  • Comment number 12.

    Hi Charles - the house rules shouldn't stop you linking to your blog in a comment. It would depend on whether you were spamming or not!

    Why not simply use an extract from Matt's blog and then link to the full text?

  • Comment number 13.

    Hopefully some of the URLs will tidied up so that eventually they all use /search, as some URLs still refer to the /topics beta. It's nice to see more of the site are getting integrated into the search. Would be nice if in time it could eliminate the secondary search box which appears on some sections of the ´óÏó´«Ã½ (eg. h2g2).

  • Comment number 14.

    Keith: The /topics URLs and the the equivalent /search URLs return the same results. This is to retain the integrity of any links people might have made to the Topic Pages we created last year. In time we will probably retire the /topics URLs and redirect them to /search.

    The next aim of this project is to harmonise all the different search experiences on the ´óÏó´«Ã½ website. This will address the two search boxes issue among other things. I will post in more detail once our plans are firmed up.

  • Comment number 15.

    It allways helps to give the user more options to customise a search, I guess the difficult part is trying to make it easy enough for the majority of users to be able to see and utilise the additional features.

    It does seem to be an improvement from a cursory glance at the blog and a quick trial of some search terms.

    You compare with google, they of course also have an 'advanced search' option from a link adjacent to the search field expanding the customisation even more for those making the effort, will you be considering anything similar

  • Comment number 16.

    Matthew and Nick:

    Thank you. Using an excerpt and linking to the original article is indeed the way to go.

    Cheers,

    Charles Knight, editor
    www.AltSearchEngines.com

  • Comment number 17.

    Your top ranked search term is 'iPlayer'

    Some users may want to search the iPlayer MessageBoards. Is the search facility of that going to be improved. (either within the messageboard or within the main ´óÏó´«Ã½ search) For example how about the ability to search the message board thread titles, surely a useful idea.
    Currently the messageboard host suggests using an external search engine ! if searching the threads. The help/faq section of the iPlayer messageboard is searchable but it is pretty basic.

  • Comment number 18.

    On the subject of iPlayer this time the programme search.

    This often seems to give slightly unexpected results, is this going to be made to work well, and/or enhanced also.

  • Comment number 19.

    John99: Three good points.

    You are right about getting the balance right between catering for the casual user and the more active user. Our plan is to enable zone-specific filters to allow active users to specify exactly what they want.

    Messageboard searching is on the roadmap for development.

    Have you an example of the unexpected iplayer results you were seeing? iPlayer search is something we are always aiming to improve so any specific problems are very good to know about.

  • Comment number 20.

    Matthew: Thanks for the reply
    "Have you an example of the unexpected iplayer results you were seeing?"

    Not sure I remember last one that I cam e across, but one I did post a reply about was the difficulty of finding the R4 programme called Today. Which is not returned when 'today' is typed in (I just tried it it still does not work) see thread where you will note the original poster was complaining that this is a re-occurring problem.

    I am sure you will find many other threads about similar problems, but we need to use search engines to try to find them at present!

    PS
    How do I put quotes in, 'quote' tag used in iPlayer mb does not work neither did 'q' or 'blockquote', or maybe I got syntax wrong.

  • Comment number 21.

    John99: Re Today search in iPlayer. I am looking into this. In the meantime a search on the main ´óÏó´«Ã½ site brings back the right result -

    Nick Reynolds may be able to help you with the iPlayer messageboard problem.

  • Comment number 22.

    Anyone : general enquiry re use of this blog

    Are quotations from other comments used on these blogs, if so how do I enter them? (Is any method standard across most ´óÏó´«Ã½ blogs now ?)

    I asked above [msg # 20]
    "How do I put quotes in, 'quote' tag used in iPlayer mb does not work neither did 'q' or 'blockquote', or maybe I got syntax wrong."
    My wording was rather clumsy. On the iPlayer mesageboard the recommended method as in this link often used by posters on that mb is the 'quote' tag. I only know how to to use bold or very small on this blog to try to make a quote stand out.

    @Matthew [msg #21]
    Thanks for reply, I was giving an example as you requested, & chose a current problem.

  • Comment number 23.

    Typo msg#24 corrected link

  • Comment number 24.

    Matthew,

    Kudos on an extremely innovative approach to search. Your ideas of relevance and content grouping are light years ahead of your time.

    would love to know what is on your bookshelf and who is in your Twitter feed.

    I am an Information Architect and am just embarking on proposing a radical search experience, and am definitely using your site as a solid example of a site that "gets it".

    Good luck on your data tuning, I will definitely be watching!

  • Comment number 25.

    I searched there, but for the terms that I am searching, I see good results, however, it says for UK visitors only. And it does not give any RSS opportunities as google search does.

  • Comment number 26.

    John99: I have been meeting this morning about the problem with the Today Programme not appearing in iPlayer search results. Our engineers have a fix in place but rolling it out may take some time due to the pressures the iPlayer team are currently under. Rest assured, "Today" will be at the top of the results in iPlayer in the near future.

    mikehill33: Thanks so much for your comments - it is great to get positive reactions from someone who works in this area and clearly knows the challenges of site search. I hope your radical search experience gets adopted by your clients and implemented. I think site searches around the web are in desperate need of some fresh ideas.

    onlinewill: Where did it say "for UK visitors only"? There are some types of ´óÏó´«Ã½ content that are not cleared for international use (some iPlayer programmes for instance", did you see that message when you clicked on an iPlayer result? We are planning to adjust to take account of international rights limitations in the future.

    RSS feeds of search results is a top priority that you should start to see pretty soon.

  • Comment number 27.

    The new search makes it harder to locate the ´óÏó´«Ã½ Webwise online course materials. (I work in a learning centre.)
    The first link to anything actually related to the Webwise course is #9, near the bottom of the page.
    /webwise/course/tutors/qualifications.shtml
    Even then it takes a further click to get to the course start page, if you know where to look.

    It seems strange that Webwise isn't the top result (try it); it would certainly help those new to the Web if it were (hint), which it should be.

  • Comment number 28.

    SheffTim: What search were you running?

    Learning content (including webwise) isnt yet included in these search results. we are working on this at the moment and the results will cover all ´óÏó´«Ã½ output very soon. In the meantime traditional search is still available.

  • Comment number 29.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 30.

    A happy reader is one that complains: please setup RSS for search results. We've been waiting. :-)

  • Comment number 31.

    Kim: sorry for the delayed response. RSS feeds are in development and on their way soon. I'll post when they are available.

  • Comment number 32.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 33.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 34.

    This comment was removed because the moderators found it broke the house rules. Explain.

Ìý

More from this blog...

´óÏó´«Ã½ iD

´óÏó´«Ã½ navigation

´óÏó´«Ã½ © 2014 The ´óÏó´«Ã½ is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.