Polls and Political Astrology
By: Justin Stapley
Tags: Polls And Surveys, Politics, Polling, Political Science, Opinion Polls

Even accurate polls fall far short of being the crystal balls we seem to think they are.

The last four years saw a lot of talk about the 2016 polls being wrong, and many observers and pundits are going to spend the next four years talking about how the 2020 polls were pretty far off as well. But personally, I think we’re looking at polls the wrong way.

The scientific process of conducting a poll, when done correctly, gives us a snapshot of admitted attitudes at a given moment. It can’t read the minds of participants nor divine the next moment, let alone tell us how voters will actually act when their attitudes meet the ballot box, nor what issues could alter attitudes days, weeks, or months down the road.

A poll can both be 100% accurate when it was taken yet completely fail to reflect what ended up being most important to voters when they step into the ballot box.

If the 2016 election had happened days after the Access Hollywood tape came to light, the story would be how generous the polls had made Trump’s chances appear before his crushing defeat.

Instead, a month’s time softened the blow, and Comey’s public reopening of the investigation into Hillary’s emails created a completely different set of attitudes and considerations for voters.

Polls are tools of analysis that grant insight into what people are caring about and how that translates to who they think they’ll vote for on election day. Polls are an important part of an election’s story and can chart the evolution of public attitudes as the story plays out. But, as we just saw, polls are not ultimately forecasts of what will happen on election day.

A poll is a thermometer, and a thermometer only tells us what the temperature is one moment at a time. It has no way to tell us how hot it will be beyond that single moment. If a thermometer tells us we’ve had a four-day heatwave, the cold front that hits on day five doesn’t make the thermometer wrong on the days it told us it was hot as hell.

Sure, a string of hot days can lead us to assume the trend will continue, but that’s us making an inferred judgment, not the thermometer. When ol’ mother nature surprises us, it’s our forecast that was wrong, not the thermometer.

Really, what we’re doing with the way we look at polls is looking at them like fortune tellers and expecting them to forecast an unknowable future. The variables that stand between even just one moment to the next are so vast as to defy any hope of total accuracy in any forecast of events.

If Biden had shown up to the final debate without wearing any clothes or if Trump actually had shot someone on Fifth Avenue, would polls conducted before such game-changing events have been of any use for forecasting the election’s result? It would be unreasonable and unrealistic to expect them to.

Yet, equally consequential, if not quite as farfetched, things happened between any given moment along the road to election day and election day itself. Yet, we unreasonably and unrealistically expected polls conducted leading up to election day to tell us what would happen on the other side of those unknowable events.

So really, polls are not right or wrong in what they forecast because they’re not forecasting anything. They’re accurate or inaccurate in whether they properly reflect attitudes when they’re conducted.

Treating even a conclusively accurate poll as anything more than a snapshot in time is really just political astrology.

This article is expanded from a section of the October 12th issue of the Self-Evident newsletter.