Opinion polls exude an aura of scientific truth. Those numbers and percentages are so reassuringly solid, especially when generated by one of the well-known names of the polling world, that for many people they represent hard evidence of the state of British public opinion.
So when the Daily Telegraph announces in a stark front page headline that “Public backs Johnson to shut down Parliament for Brexit”, majority public support for such a huge constitutional risk seems assured. And when this headline is reiterated by Newsnight’s Emma Barnett – who rarely lets anything remotely contentious slip by – without so much as a raised eyebrow, it receives an implicit nod of authority from the nation’s impartial public broadcaster.
In fact, it is complete nonsense: at best, a misleading interpretation of a simplistic poll attempting to tap public feelings on a complex and controversial issue; at worst, a deliberate and malign distortion of a poorly designed poll, with a headline calculated to set the news agenda and serve the interests of Johnson and his strategy sidekick Dominic Cummings.
Let’s take the poll first. Anyone who understands questionnaire design will tell you that both question wording and question order can easily bias responses if not carefully drafted.
There was little sign of careful drafting. The headline was based on a question which invited respondents to agree or disagree with the statement “Boris needs to deliver Brexit by any means, including suspending parliament if necessary, in order to prevent MPs from stopping it”.
It doesn’t take a research methods expert to understand that there are at least four problems here. First, there is “Boris”: not Mr Johnson or the PM but that nice uncle “Boris” with whom we are all intimately familiar. Wouldn’t hurt a fly.
Second there is the double question, breaking all the usual rules of simplicity, which buries the suspension of Parliament inside a statement about delivering Brexit “by any means”.
Third, there is what academics grandly call “acquiescence bias” which simply means we’re all more prone to agreeing with statements than disagreeing with them.
Fourth, and most egregious of all, there is the rider that this is all about preventing MPs from stopping Brexit rather than allowing MPs to prevent a no-deal Brexit (which would almost certainly command a higher level of support).
That rider is even more problematic given the order of questions, because it followed almost immediately after two bluntly anti Parliament agree/disagree statements. One read: “On Brexit, most MPs seem to ignore the wishes of voters and push their own agendas”, the other “Parliament is out of touch with the British public”.
In the art of questionnaire design, respondent-priming is a well-known phenomenon and generally avoided. By the time they got to the uncle Boris question, respondents would hardly be well disposed to MPs. Had the Parliament suspension question come first, unqualified by cuddly Boris or those dastardly MPs, the results would almost certainly have been very different.
Then there was the Telegraph reporting, which deliberately distorted weak data to produce the headline they desperately wanted in pursuit of their unadulterated pro-Johnson agenda.
That suspending Parliament has “the support of more than half of the public” is a straightforward lie, derived from ignoring the one in five “don’t knows”. In fact, 44% agreed with the dodgy statement, 37% disagreed, and 19% said they did not know. However much you try to slice up the statistical cake, 44% is not half the British public.
There has been at least one accuracy complaint to the puppet press “regulator” IPSO which, if past behaviour is any guide, will now take several months to determine that everything is absolutely fine; or that perhaps a tiny correction might be warranted sometime next decade at the bottom of page 17. By which time either Brexit or Boris (and quite possibly both) will be ancient history.
If this was a single lapse – either by polling company or by publisher – it might be less worrying. But our Brexit-loving press will be using every conceivable device over the next 10 weeks to persuade MPs that a PM with no democratic mandate of his own is reflecting the Will of the People in his determination to crash us out of the EU.
Not only will they be exploiting polling companies who really ought to know better. They will also be exploiting broadcasters who routinely feature the next day’s press headlines in their programmes, as if these come from trusted bi-partisan seekers after truth rather than propaganda machines for Johnson and his band of hardline Brexiteers.
With declining circulations, the Telegraph, Sun, Mail and Express should be less powerful than during their halcyon days of last century’s elections. But their ability to influence agendas in other news organisations, and to provide content for TV programmes that are cash-strapped and seemingly addicted to newspaper headlines whatever their circulations, remain as potent as ever.
It will therefore be down to the producers of those programmes on the BBC News Channel and Sky News, as well as Newsnight, Marr, Peston and all the other political programmes that still love to feature our daily papers, to treat every poll and every statement of “public opinion” with a huge dose of scepticism. They need to remember that there are newspaper journalists and editors who, whether deliberately or not, are spreading misleading information under the cloak of “objective” opinion research.
As for the pollsters, it is time they took responsibility for how their work is reported as well as conducted. The Market Research Society used to insist that its members take steps “to check and where necessary amend any client-prepared materials prior to publication”. But many (including ComRes who conducted the Telegraph poll) do not belong to the MRS, and therefore allow themselves to become accomplices to newspaper propaganda.
They, along with responsible broadcasters, academics, and ordinary members of the public, should be watching carefully and calling out bogus claims about public opinion.
This post originally appeared on openDemocracy and is reproduced with the permission of the author