BES data closer match to May’s result than pre-election polls

British Election Study in-person survey data has cast new light on how sampling impacted on the 2015 general election polling miss.

BES data closer match to May's result than pre-election polls

The data, released today, reveals a closer match to the 2015 general election Conservative lead over Labour than the pre-election polls, which failed to forecast the scale of the Conservative victory.

The British Election Study 2015 is managed by a consortium of The University of Manchester, The University of Oxford and The University of Nottingham.

Analysis of the new data by the BES team suggests that this closer match was made possible by using probability-based sampling methods and successful targeting of hard-to-reach groups, including non-voters.

The results of the in-person survey have been much awaited, because they have implications for understanding what went wrong with the polls in 2015.

The BES conducts an online panel study following the same 30,000 respondents over time and also an in-person probability based sample of 2,987 people in 300 constituencies.

BES Director Professor Edward Fieldhouse, from The University of Manchester, said: “If sampling and representativeness are important for understanding the 2015 polling miss, this should appear in the in-person post-election BES data.

“Our preliminary analysis points to this as the best explanation for why we get a relatively large Conservative lead over Labour in their sample, unlike the pre-election polls.”

In the BES in-person survey, 40.3% of the voters said they voted Conservative, 32.75% Labour, 7.05% LibDem, 10.61% UKIP and 4.69% SNP.

Professor Fieldhouse added: “We don’t have a perfect sample of the whole population, but our data performs well with respect to lower reported turnout, and a good distribution of age. We think these two factors help us get much closer to the election result, although further research is also needed”.

He continued, “We do not think the results are due to people knowing the outcome of the election either. For this ‘rationalisation’ to occur, we would expect to find more Conservatives the longer from the election the person was interviewed. Our analysis suggests this is not a feature which explains the result”.

The BES team found no evidence for popular explanations, including Shy Tories, late swing or systematically different preferences among “don’t knows”, to explain why the polls were wrong before the election.

The new analysis now points to unrepresentative samples as an important explanation for the polling miss, a problem which is largely offset by the methods used in the BES in-person survey.

BES Research Fellow Dr Jon Mellon, from The University of Oxford, said: “Opinion pollsters are very good at making their samples reflect the general population, using a system of weighting.

“But the general population and the electorate are very different things, because around 40% of adults don’t vote.”

“

If sampling and representativeness are important for understanding the 2015 polling miss, this should appear in the in-person post-election BES data. Our preliminary analysis points to this as the best explanation for why we get a relatively large Conservative lead over Labour in their sample, unlike the pre-election polls.

BES Director Professor Edward Fieldhouse, University of Manchester

„

BES Research Associate Chris Prosser, from The University of Manchester, said: “The 40% who don't vote are also less likely to answer political polls

“This means there are too many voters in polls among low turnout groups such as young people.”

In the BES in-person survey, 43% of the under 30s said that they didn't vote. However, polls will often find less than 15% of them saying they didn't vote.

Dr Mellon added: “There are far too many young voters in polling samples and not enough young non-voters.

“Because young people tend to be more Labour leaning, this means we end up with too many Labour voters in the polls, as happened before the election in May.

“This appears to be part of the reason why there was a bias in favour of the Labour party in May."

Notes for editors

In-person survey interviews are conducted face to face, rather than by phone or internet. The most important feature of the BES in-person survey is its sampling method based on probability (constituency types are first selected, then respondents are selected by chance and only those selected are targeted for interview). This differs from samples achieved by samples of people signed up to take surveys, or by the filling of quotas according to target populations based on demographics.