Exit poll denial (and a handy tip for the pollsters)

Channel flicking between BBC and ITN on election night, I was astonished by the cavalier over-confidence with which the assembled politicians and pundits wrote off the 'obviously' misleading exit poll that had been commissioned from NOP and Ipsos MORI by the BBC, ITN and Sky News.

When presenters employed by the said news outlets joined in the chorus of scorn, it raised the question of why their bosses had bothered to commission such pointless polls in the first place.

Time and again, we were treated to glib reminders that "It's only an exit poll", "exit polls are notoriously unreliable" and "they don't take postal votes into account" from pretty well everyone in the studios and on location around the country - all of whom had been afflicted by a collective amnesia about the awesome precision with which the same polling companies had used the same polling procedures to predict the outcome of the last general election.

Exit poll 2005

As the polling booths closed five years ago, a headline had come up on the TV screens of the nation telling us that the exit polls predicted a Labour majority of 51 seats. When the all the votes had been counted, the actual figure was a Labour majority of 51 seats.

Yes, they had slightly overestimated the number of Conservative seats (predicting 209 against the actual 198) and underestimated the number of Liberal Democrat seats (predicting 53against the actual 62). But they were spot on both with Labour's overall majority of 51 and their number of 356 seats.

Exit poll 2010

With all that in mind, you'd have thought that the chatterers might have thought twice before writing off a poll conducted by the same companies using the same well-proven methodology of previous years. But not a bit of it. They knew best and trotted out the same repetitive refrains.

Meanwhile, as the results came in, it gradually became clear that we could all have gone to bed a lot earlier if only we'd been allowed to believe the news from combined forces of NOP and Ipsos MORI.

For the record, here are their hopelessly flawed predictions and the actual results:

Con: 306/305

Lab 255/258

LD: 59/57

In the words of John Rentoul of the Independent on Sunday: 'a crowning triumph of the opinion research business'.

A more reliable exit poll?

In the Wells constituency, where I live, an interesting new predictive measure emerged this year. It became clear that the Liberal Democrat candidate had won when we learnt that far fewer of her posters had been ripped down this year than in 2005 (when she'd lost by 3,000 to David Heathcote Amory).

I've advised the IpsosMORI high command that they might like to take this into account in any future exit polls they do.

The good news is that they've agreed to consider building in a 'defaced poster count' next time.

The bad news is that, with their fastidious methodological caution, they're worried about how to control for whether or not poster removal results from the actions of a lone ripper or many rippers - and, if the latter, there would then be the question of how representative they are of the electorate at large.