´óÏó´«Ã½

When VAR (video assistant referee) was introduced into top-flight world football in 2018, many hoped that it would put an end to the uncertainty around offside and penalty decisions. While it may well have cut out many bad decisions, supporters still feel aggrieved by the controversial decisions that don’t go their way, and VAR may even question the idea of truth itself.

This notion was posited by Daisy Christodoulou, a director of education for a company measuring educational attainment. Daisy was watching West Ham versus Brighton at the London Stadium in February 2020. During a lengthy stoppage due to a VAR check, Daisy started a Twitter thread about the questions the video refereeing system throws up. Science writer Tom Chivers has taken up this thread to find out what is certainty – an especially pertinent question during the Coronavirus pandemic - and can we live without it.

One of her Tweets added that VAR “complicates the very notion of 'right'. What is a handball? What is offside? What is 'clear and obvious'? Ultimately, what is truth?”

How do we measure uncertainty?

As Professor David Spiegelhalter, author of The Art Of Statistics, explains, the main role of statistical science is to say how uncertain we are about something, for example using a poll or results from a medical trial, and arrive at a “quantifiable margin of error.”

As Tom Chivers acknowledges, we want “more right decisions” but we’re also looking for better outcomes. In the case of football, our overriding focus is not the offside that VAR is supposed to help us detect, we want the flow of the game to improve.

So, sometimes the method used to control uncertainty gets in the way of the intended result. Daisy Chistodoulou paraphrases Goodhart’s Law to explain this: “when a measure becomes a target, it ceases to be a good measure.”

We need common sense, but it can’t act alone. “There are huge problems with human judgement,’ says Daisy Christodoulou, “that it’s inconsistent, that it’s biased, often in quite unpleasant ways, so, often, using a statistical judgement will give you a more consistent understanding.”

Tom Chivers warns: “it’s easy to lie with statistics, but it’s even easier to lie without them.”

Despite Tetlock’s assessment of the majority of forecasters, there are those who excel. Michael Story from consultancy Maby remarks that “most of the time, people tend to be over-confident and they predict things at 90% probability that probably don’t happen 90% of the time.” Story says that you have to start with a broad view of probabilities before looking at the narrower detail of a forecast. Keeping track of what you got wrong is vital to making better forecasts in the future, he says.

Of course, the stories that are selected need specific scrutiny too. Jennifer Rodgers recalls seeing a ´óÏó´«Ã½ News story about a 28,000 fall in UK unemployment. The survey had a margin of error of 79,000 either way!

Even scientific publishing is problematic. Michael Blastland believes we have forced people to provide “exciting findings” – i.e. by “suppressing degrees of uncertainty around their findings” - so that they can further their careers in terms of research grants and getting published. “We’ve created a culture around the very people who should be the most resistant around that kind of culture.”

Although we strive for tangibles, we’re actually quite good at living with uncertainty, epitomised by something as simple as crossing the road. In fact, as Spiegelhalter notes, sometimes we actively seek uncertainty – for example, we don’t want to know what presents we’ve got for Christmas and we don’t want to know when we’re going to die!

With all that has gone on with Coronavirus and the ever-changing information around it, Tom Chivers suggests “if you’re not epistemically humble now, you never will be.” However, can we relax about knowing things absolutely? Jennifer Rodgers thinks that we need to do better at “communicating that uncertainty isn’t a bad thing, it’s a natural thing.”

Michael Blastland goes further. He calls time on the idea that confidence and uncertainty are the only ways to be persuasive. He believes that we need to “stop demanding certainty” and sums up the argument with a quote on Voltaire – “doubt is uncomfortable, but certainty is absurd.”

One can only wonder what Voltaire would have made of VAR.