Thursday 18 September 2008

Polls Apart

When an opinion poll comes along that is out of line with previous surveys, there's a good chance that two things will happen. First, it will attract a huge amount of publicity. Secondly, it will turn out to be wrong.

The latest Ipsos-MORI survey shows the Tories, for the first time since... well, since decades ago, above 50%. To be precise, 52%. Labour is on 24%, and the Liberal Democrats down on just 12%. It was widely covered in Thursday's papers.

Could MORI's figures be right? Yes. Are they likely to be right? No.

As the Americans say, do the math. MORI's total sample was 1,017. But MORI's published figures relate only to those people who say they are certain to vote and who declare their allegiance. According to the tables published on MORI's website, there are just 511 of these people. The margin of error on a sample of 511 is around 5 percentage points.

Even that may be an underestimate. That 5 point calculation assumes that the sample is as good as a pure random sample with a 100% response rate. No poll in the real world meets those conditions.

All the other regular pollsters - ComRes, ICM, Populus and YouGov - take steps to ensure that their data are politically, and not just demographically, representative. Their - our - methods vary, but they - we - all believe that demographic representativeness is not enough. MORI takes no such steps. They have never done so; their folk advance a perfectly reputable case in statistical theory for keeping clear of such measures and weighting by demographics alone; and they had a fantastic record of accuracy in elections in the Seventies and Eighties.

However, they are vulnerable to the occasional politically-distorted sample, especially when the effective sample for their headline voting figures is barely 500. My guess is that they have been hit by the curse to which all pollsters, even the best, are occasionally prone: a rogue sample. If I am right, polls over the next few days will show the Tories with somewhat less than 52% support, and the Lib Dems with substantially more than 12%.

We'll see. If I am right, remember you read it here first. If I am wrong, then I shall email my congratulations to Bob Worcester - just as he emailed me his congratulations in May, when YouGov got London's Mayoral result spot on.

No comments: