Why yes/no questions can yield surprising answers

Was the Iraq war caused by the West’s desire to control access to Middle Eastern oil?

Do you want this man or that woman as your leader for the next four to eight years?

Should we stay or should we go?

There’s been a lot of soul searching in the past month about the wisdom or otherwise of asking simple yes/no, binary questions. As the holiday season begins and David Cameron – for one – enjoys a more relaxed summer than he might have planned, it’s worth taking time out to reflect on the choices we make. As consumers, as individual citizens, and as collective electorates.

As essentially reductionist cognitive creatures, humans like to understand, in simple terms, what caused what. We seek clarity and understanding to reassure ourselves that the world makes sense and we understand the relationship between agency, actions and outcomes.

Bright as we are, we’re attracted to simple answers with as few interacting variables as possible. The more contingencies we have to hold in our minds simultaneously, the harder it is. Even super-smart scientists, who are paid to understand the complexity of interacting factors for a living, start to get confused when they have to consider more than four or five at a time.

The trouble is, issues are rarely as simple as we’d like them to be. While politicians may force us to make binary choices – Brexit or Remain, Trump or Clinton – the truth is often very much more complex and nuanced than such rigid decision-making trees allow. Our decisions about who or what deserves our vote or our consumer choice sit along a spectrum, but the final choice is unable to reflect that.

To be trivial for a moment, let’s consider my choice of phone. There may be many things that I admire about Samsung or Android as an OS, but I’d never have anything other than an iPhone. I’ve owned almost every model from 2-6S. It’s about style. It’s about form AND function. It’s about elegance. It’s about the Apple ecosystem. It’s about Jobs and Ive and Cook. It’s about colours called Space Grey and Rose Gold. It’s about the Apple Store experience – particularly Covent Garden and Central Park. It’s about seamless, integrated functionality. It’s about a lifestyle. I could go on.

The multiplicity of factors underpinning binary decisions and the spectrum of opinion that binary decisions fail to capture is reflected in the types of stories we like to consume and tell. Yes, we might think we like black and white stories. Stories of good and evil, of heroes and villains. And yet the hero’s journey is an incredibly rich and complex brew of interlocking parts. At any point on his or her journey, our heroes face adversity and near-fatal reversals of fortune. They travel from the status quo through a world of peril and daemons and back to the original world, transformed. The stories aren’t as simple as we first thought. Be they Red Riding Hood, Star Wars or The Hunger Games.

Modern analytical methods enable us to harness and blend all manner of different, apparently contradictory data sources to understand the richness and diversity of opinion and fact – yes, fact – that go to make up complex stories and the spectra on which they sit. And even if pollsters and now bookies are failing in their ability to make effective predictions – as the 2015 General Election and the 2016 EU Referendum showed in spades, not to mention Leicester City’s triumph in the Premiership last season – now is not a time to descend into a trough of analytical despond.

No, now is the time to embrace the opportunities that the Big Data revolution offers us to make sense of the world. And perhaps, if we’re politicians looking to take the temperature of the electorate, we look to ask questions that don’t insult the intelligence of the electorate by forcing them down simple, yes/no blind alleys. At least they shouldn’t be surprised to get answers they don’t expect or their metropolitan and media echo chamber bubbles suggest at first, second and thirty-fourth glance.

These and other thoughts were the topics of the second in a new series of podcasts for the Small Data Forum, convened and chaired by Thomas Stoeckle of LexisNexis BIS, and featuring IBM Watson’s Neville Hobson and me. Check out “The role of data in the Brexit campaigns”. Find it here on iTunes, where else?