Are polling companies in crisis?

Major wrong call yet again

Once again the polling companies have got it wrong. And not just slightly wrong, but very wrong.

This time they predicted a Hillary Clinton win. Even 24 hours before the polls they predicted that Hillary would win.

Just a few months ago they predicted a “Remain” win in the Brexit poll.

Last year, there performance would deserve an “F” for the UK general elections. None of the 92 polls accurately predicted the 7% lead the Conservatives would actually achieve and a majority predicted a Labour victory.

% gap between Conservative and LabourAll pollsConservative leadsLabour leads
Total923342
017
1321517
222715
31037
4752
511
6321

In 2013, the pollsters were wrong with the British Columbia provincial elections.

Why do the pollsters repeatedly get it so wrong? Why don’t they learn?

Well, actually, the pollsters have been getting wrong for much longer.

Gallup has recorded the history of its own accuracy back to 1936 (this had not been updated with today’s results when VoxSapiens included it in this post). A 6.8% error in both directions in 1936. A 5% error and a 4.4% error in 1948. Another 4.4% error, this time in both directions, in 1952. Even in more modern times, a 5.7% error in 1992. And a wrong call for who would win in 2012.

fb7l0hgt5kgmzyn3mlgsvg

Given this history, why do we continue to believe the pollsters? Are they in crisis? Or is our ability to believe anything the actual problem?

And is there anything that they can do about it? Is there a responders’ bias that cannot be overcome (e.g. impossible to get a truly representative sample, or pollees just lie and/or say what they believe that they are expected to say)? Or must the pollsters just work smarter?

Next year there are some more interesting elections. Will the pollsters predict the fate of Geert Wilders and Marine Le Pen? Will British Columbia surprise us again?

One single comment

  1. TheVoice says:

    Update 2017/01/21: https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy is a very interesting article by British newspaper The Guardian. It covers sampling and prediction but it covers a lot more – such as public distrust in government statistics. It also gives credence to our 2016 Word of the Year.

Post a comment

*

Copyright © Vox Sapiens
Intelligent Commentary on Society and Business

Built on Notes Blog Core
Powered by WordPress