Can polls accurately predict future outcomes of elections? Or are they meaningless numbers often disconnected from reality?
The latter sentiment is often tied to conversations around the 2016 U.S. presidential election. The story goes that polls were showing Hillary Clinton as a near-certain victor, and so Donald Trump winning the presidency proves that polls are useless. Right?
It’s more complicated than that. For one thing, credible polls were showing Clinton only several percentage points ahead, a far cry from a certain victory especially when you consider margins of error. Ahead of the election, poll aggregator FiveThirtyEight upgraded Trump’s chance to winning the election, estimating that he had a 1-in-3 chance; many commentators actually criticized this for ‘inflating’ Trump’s likelihood of winning. We all know how things actually turned out.
There’s also the fact that Clinton won the popular vote by about 3 million votes, further complicating the narrative that the polls were dramatically wrong.
There’s also the question of what exactly we’re talking about when we refer to “the polls” – are we including things like Breitbart’s impeachment poll that showed almost 98% of Americans “stand with President Trump”? Are we talking about actual pollsters that are credible and not spawned from the depths of a dumpster fire propaganda site, like Marist College, Quinnipiac University, Abacus Data, or EKOS? Or are we talking about poll aggregators, like FiveThirtyEight or 338Canada?
Another problem is the way that people often interpret the outcome of an event. Let’s say the polls are predicting a close race, with a small edge to one candidate. That person ultimately wins by a huge margin. This is actually case where people might give polls too much credit, because they predicted the ‘outcome’ – yet the actual numbers are quite different.
There’s the issue of probabilistic versus deterministic questions. If I say that something has an 80% chance of happening, that doesn’t mean it’s definitely going to happen. It means… it should have an 80% chance of happening. It means that if I make 10 predictions and each time I say there’s an 80% of Outcome A, then Outcome A should have come true 8 times. If it actually came true all ten times, people might feel satisfied because 80% sounds extremely likely – but my predictions are actually a bit off, because again, Outcome A should have come true 80% of the time.
In the context of the upcoming Canadian federal election, we see the Conservatives often ahead with a popular vote lead. Of course, what really matters in our first past the post system is the number of seats that a party acquires (imagine if the Conservatives win the popular vote but they don’t get the most seats, and then become advocates for electoral reform? Just imagine that).
If we look at poll aggregator 338Canada’s seat projections, we see a real rollercoaster: before September 18th, the Liberals supposedly had an 81% chance of winning the most seats; just six days later on the 24th, this had dramatically dropped to a 51.4% chance. As of October 12th, they supposedly have a 56% chance of winning the most seats.
What does this tell us about the upcoming election, and about political polls more generally? We obviously shouldn’t look at polls as some kind of carved-in-stone accurate prediction of the future. But we also shouldn’t write them off as ‘useless’ or meaningless noise. Rather, we can look at polls as snapshots of the current public sentiment, giving us some insight by reflecting the reactions to current events (why did the seat projection for the Liberals drop so dramatically around September 18th? Oh…)
I predict that 100% of people who read this will love it and think I’m incredibly insightful.