BES reveals longstanding flaw in opinion polls
- Flaws which made the opinion polls wrong in 2015 General Election also explain errors in 2010
- Flaw also likely to have been present in the 2001 and 2005 General Elections
- Liberal Democrats were over-estimated in 2010
Flaws which made the opinion polls wrong in 2015 also explain errors in 2010, new research from the British Election Study (BES) reveals.
BES researchers Dr Chris Prosser from The University of Manchester and Dr Jon Mellon from The University of Oxford last year revealed that sampling was the reason for the 2015 polling miss, confirmed in Tuesday's Independent Inquiry preliminary report.
Now they show the same mechanism of over-representing groups such as young people who were less likely to vote, is also likely to have been at least partially responsible for the 2010 over-estimation of the Liberal Democrats’ vote share.
The flaw, they say, was also likely to have been present in the 2001 and 2005 general elections, but the electoral maths meant the polls correctly forecast which party would form the Government, despite overestimating Labour’s share.
Dr Mellon said, “We have shown the 2015 polls were fundamentally unrepresentative of the electorate and that Labour leaning respondents were the most over-represented as voters in the sample.
“Our new research shows that this problem is a longstanding issue: in 2010 these over-represented respondents were Liberal Democrat voters, which could potentially explain the 2010 polls’ over-estimation.”
In 2010, the polls over-estimated the Liberal Democrats by around 4.0 percentage points, but no reason was found for the error.
The BES findings now suggest the error was down to sampling voters from demographic groups with low turnout demographics, who tended to support the Liberal Democrats.
Dr Chris Prosser said: “The surprising surge of support for the Liberal Democrats among low-turnout demographic groups, such as students, may in fact have saved the pollsters from a 2015 polling miss in 2010.”
The academics have devised a way of correcting the flaw, so pollsters can in future create more accurate forecasts by factoring in additional weighting to account for expected proportions of non-voters.
By adding an additional weighting factor of the expected proportion of non-voters, respondents from demographics which vote less would be more represented in the non-voting section of the general population sample.
That means they will be less represented as voters – improving accuracy.
“While we do not know exactly what the turnout will be in a given election, we do know that it is likely to be reasonably close to the previous election’s turnout,” said Dr Mellon.
“But further work is needed to determine whether this method will be sufficient.”
The surprising surge of support for the Liberal Democrats among low-turnout demographic groups, such as students, may in fact have saved the pollsters from a 2015 polling miss in 2010
The BES face-to-face survey, carried out by GfK between May 2015 and September 2015, was a random probability sample of 2,987 people.
It showed a closer match to the 2015 general election Conservative lead over Labour than the pre-election polls, which failed to forecast the scale of the Conservative victory.
Analysis of the data by the BES team suggests that this closer match was made possible by using probability-based sampling methods and successful targeting of hard-to-reach groups, including non-voters.
Dr Mellon said: “We believe these findings have serious implications for pollsters both in the UK and in other countries.
“Weighting to population targets and then removing non-voters is by far the most dominant election polling approach, and any pollster that uses it risks the same kind of polling miss that Britain saw in 2015.
“Given the increasing number of polling misses throughout the world, we may already be seeing the results of this methodological choice.”
NOTES FOR EDITORS
Paper is available on request
In-person survey interviews are conducted face to face, rather than by phone or internet. The most important feature of the BES in-person survey is its sampling method based on probability (constituency types are first selected, then respondents are selected by chance and only those selected are targeted for interview). This differs from samples achieved by samples of people signed up to take surveys, or by the filling of quotas according to target populations based on demographics.
The Economic and Social Research Council funded BES is one of the world’s oldest studies of electoral behaviour and run by a consortium of the Universities of Manchester, Oxford and Nottingham.
For media interviews contact:
Communications and Marketing Manager
British Election Study
University of Manchester
0161 306 6901
Follow us on Twitter @BESResearch