On the importance of knowing your audience in data storytelling
Press conferences held at Downing Street during the pandemic will provide future generations of researchers with a rich vein of evidence on which to base journal articles, dissertations, and theses.
Interested in how to appear to apologise for 90 minutes but not say sorry once? Look no further than Cummings’ lawyerly masterclass in wriggling and squirming from last May.
Keen to see how to mangle and make nonsense out of data at a time when experts suddenly came back into favour? Priti Patel created a whole new class of number – 300,094 974,000 – in one of her rare podium appearances.
Fancy standing back and watching a catastrophic, hastily-cobbled-together mish-mash of hardcore stats to a lay audience, in a ham-fisted attempt to justify a second lockdown? My blog from last Halloween – “A nightmare on Downing Street” – picks over the entrails of that catastrophe, one in which the PM’s usually-reliable side-kick, Chief Medical Officer Chris Whitty, must take some blame. His car crash slides are eloquently picked apart and beautifully reconstructed here by my friend Dave Henson, the Slide Presentation Man.
Time and again, bluff over-optimist Johnson – a man claiming to be leading a government that’s led by the data – has mispresented and misrepresented data and shown himself to be a very poor data storyteller indeed. Blending the rational and the emotional is a fire-and-ice juggling act, and the British public have been routinely let down most days at the 5pm press conference. And one of the areas where the Downing Street shower have failed most often is in one of the golden rules of data storytelling: knowing their audience.
Last Friday 22 January was no different. Britain is in the grip of one of the worst second waves of coronavirus of any nation, with deaths in four figures for most days last week. Infections are falling – they’ve halved since the turn of the year, indeed – and the R number is below one, largely thanks to Lockdown 3.0; not a pub or restaurant, theatre or sporting venue open for business, and much of UK plc that can work once again working from home. Nevertheless infections are still running in the tens of thousands each day, a figure unimaginable even two months ago.
The trouble, we are told, is that the British mutation of COVID-19 is more infectious than 2020’s dominant strain. More transmissible and, so Johnson said, more deadly. Up to 30% more deadly. That sounds like a serious blow to the rapidly roll-out of the growing army of vaccines at our disposal. Johnson said: “In addition to spreading more quickly, it also now appears that there is some evidence that the new variant – the variant that was first identified in London and the South East – may be associated with a higher degree of mortality.”
Chief Scientific Officer Vallance reported on the tentative findings of a mathematical modelling paper which gives hints that new improved British COVID may be 30% more deadly than old COVID. Vallance did have the good grace to give an example that explained the relative risk: if 1,000, sixty-year-olds are infected by old COVID, 10 would be expected to die. That figure may rise to 13 for the new strain. Hence the 30% increase in relative risk.
The absolute risk is 1.0 in 100 for over-60s with old COVID, and perhaps 1.3 in 100 for new COVID. But he didn’t say that explicitly, and this is a consistent failing in the reporting of risk and odds ratios in epidemiology. The absolute classic regularly comes from the global, intergovernmental, anti-cancer body IARC, the International Agency for Research on Cancer. Several times in recent years, research reported by IARC has shown around a 20% increased risk – increased relative risk – in bowel cancer from daily consumption of processed meat. The UK media often likes to caricature this as a daily bacon sandwich.
An increase of 20% sounds serious. And any increase in avoidable, lifestyle-related mortality should of course be taken seriously. But when you dig into the data and identify what the 20% (18% in the 2015 study) means, you find it means this. On average, six people in 100 get bowel cancer in the UK at some point during their lifetime. Of those who don’t eat processed meat, just over five in 100 get bowel cancer at some point in their lifetime. The 18% difference is because just over 5/100 non-processed meat eaters get bowel cancer, which 6/100 bacon buttie fans do.
The unofficial chief rock star statistician of the pandemic, Professor Sir David Spiegelhalter, has used this particularly egregious and quite deliberate confusion of relative and absolute risk as a case study for many years. Here’s a brilliant video of him explaining this in detail, and here he is in The Independent pouring cold water on IARC’s conclusions: “There may be good evidence for there being an increased risk in developing cancer after eating meat, but the magnitude needs to be put into perspective,” he said. The same is very definitely true of the Vallance and Johnson show last week.
In his book How Spies Think, the former head of GCHQ, David Omand, says: “Forecasts of risk need to be communicated effectively to those who can make use of the information.” I believe that Vallance spelled out the relative risk – and made no mention of the absolute risk – with two audiences firmly in mind: the public and the media. The public would hear the “30% increased risk” if they were tuned in to the press conference and, he hoped, be panicked into better behaviour and closer observation of lockdown. For the most common interpretation – and misinterpretation – is “my own personal risk of dying has gone up by a third”. And if they didn’t hear the stat live, Vallance could guarantee that the media report this compelling, killer statistic; do excuse the pun. And report it they did, as the montage of Saturday’s screaming headlines makes clear.
In a very few hours after the Johnson and Vallance show, however, the scientists behind the tentative findings poured cold water on the way it had been hijacked for political purposes. This summary on the BBC News website quotes:
- The study’s co-author saying evidence of the new variant’s deadlines remains “an open question”
- A Johnson aide expressing surprise he and Vallance had used it because the data is preliminary and “not particularly strong”
- A “top medic” saying it is “too early to be absolutely clear”
As the co-author of the preliminary study – Graham Medley, professor of infectious disease modelling at the London School of Hygiene & Tropical Medicine – told Radio 4: “In terms of making the situation worse it is not a game changer. It is a very bad thing that is slightly worse.” Quite so.
Overexaggerating absolute risk by confusing and conflating it with relative risk doesn’t work. It is literally incredible and a phenomenon that has been dubbed “the megaphone solution”. It’s a none-too-subtle variant of being the boy (he, him) who cried wolf.
The long-overdue departure of Cummings and Cain – the Mitchell brothers clones responsible for the nightmare on Downing Street last Halloween – had appeared to have improved the way Downing Street press conferences used data in their storytelling. Last Friday has shattered that illusion. The irony is that, horrific as the daily and cumulative data are – numbers stoked and inflamed by successive failures in government policy and failure to properly listen to the science – Johnson et al. may well get out of the pandemic smelling of roses. The rapid approval and deployment of vaccines – almost six million first doses and rising 400K+ per day – may just get (or keep) them out of jail. But that won’t be down for one second to good or improving data storytelling, and certainly not the crucial importance of knowing and understanding your audience.
Sam Knowles is a master data storyteller and the Founder & MD of the consultancy Insight Agents. His purpose is to help organisations make smarter use of data, talk Human, and sound like people. An established and sought-after trainer, keynote speaker, and podcaster, he is the founder and host of Data Malarkey podcast and chair of I-COM’s Data Storytelling Council. He’s a Fellow of the Market Research Society, the RSA, and the Professional Speaking Association.
Sam is the author of the ‘Using Data Better’ trilogy of books, all published by Routledge. These include the 2022’s critically-acclaimed Asking Smarter Questions, 2020’s award-winning How To Be Insightful, and the 2018 best-seller Narrative by Numbers. In 2023, Insight Agents launched Using Data Smarter, a comprehensive, online training course based on all three books.
Find out more about Sam’s approach to data storytelling in this 15-minute video.
