South Carolina Senator Lindsey Graham just beat his
Democratic opponent 57%-42%, when the polls gave him only a 3% advantage.
Graham gave a belligerent speech on November 4 attacking the press and making
fun of the polls:
“I’ve
had two calls already, one from President Trump, he’s gonna win. He’s gonna win.”
Graham
added:
“So all the
pollsters out there, you have no idea what you’re doing. And to all the
liberals in California and New York, you wasted a lot of money. This is the
worst return on investment in the history of American politics.”
It does sound as if the 2020 polls were wildly
wrong. Well, they were wrong, but most of them were not wrong as much, or in
the ways, that either Graham or media talking heads seem to think. Still, in
2020, as in 2016, political opinion polls undercounted Republican voters. This
is despite the fact the pollsters were on the lookout for that problem and
tried to compensate statistically.
Poll aggregator Nate Silver’s last prediction for
the 2020 presidential election posited that Joe Biden would beat Donald Trump in
the popular vote by about 8%. Votes are still being counted, but it seems
likely that Biden will end up with about a 3% popular vote advantage over Trump,
or maybe slightly less. The Blue Wave that optimistic Democrats predicted never
happened. (Note that pollsters did not predict a blue wave.) As I write
this, it seems probable but not certain that Biden will have more than the 270
electoral college votes that he needs to win the presidency. The Democrats will
probably keep control of the House of Representatives, but will not, barring a
real surprise, gain the Senate.
Some people think that polls are a conspiracy
against our republic. The truth, as always, is more complicated. National polls
were less accurate than usual in two major elections in a row. Worse, they missed in
the same direction, underestimating Republican turnout. Social scientists don’t
like coincidences any more than TV detectives. So, let’s look more deeply.
First, opinion polls are just estimates based on a
sample. The quality of the poll depends on how good the sample is. No one is
going to survey hundreds of millions of voters. Instead, a poll will survey a
few thousand voters, at the most, using those results to estimate the outcome.
Let’s suppose that a poll samples a thousand voters. Let’s suppose that half support
Biden and half support Trump. But do
they fairly represent the population at large? Maybe yes, maybe no.
It is always possible that, by sheer bad luck, the
pollsters accidentally surveyed people who are not typical of the population.
After all, expecting a couple thousand people to speak for the entire nation is
kind of tricky. That’s why pollsters calculate the chance that their sample was
wrong. Opinion polls typically run with a 3% to 5% predicted error rate. What that means is
that there is no more than a 5% chance that the poll’s findings will be within
3% to 5% of the entire population. So:
(a) A 3% to 5% error predicting the voting results isn't surprising, and
(b) There is one chance in twenty that the error will be even more than 5%.
Since the
polls predicted an 8% win for Biden, and he only won (we think) by 3%, that
doesn’t prove that the polls were wrong. It might only show bad luck. Pollsters
can reduce the risk of sampling error by surveying more people, but, strangely
enough, that doesn’t help as much as you might think. Not unless the sample is enormous, and that causes other statistical problems.
Second, poll aggregators like Nate Silver look at
many polls, not just one. That makes up for the danger that one or two polls
might be wrong. Silver gives more weight to quality polls and statewide polls.
That usually helps, but there are no guarantees. Pollsters all read the same statistics books, and they might have all made the same sampling errors.
Third, people don’t always cooperate with polls.
The election polls coincided with healthcare open enrollment, so most of us are
sick and tired of harassing phone calls. I accidentally hung up on a legitimate
pollster last summer, just because she called at the end of a string of robocalls.
I realized my mistake a second later, but it was too late. Pollsters have
massive databases that help them to compensate for uncooperative respondents,
but that’s never precise.
Fourth, the exact way that a question is asked makes
a big difference in how people respond. Suppose that one pollster asks, “who do
you plan to vote for in the upcoming presidential election?” Another pollster
asks, “do you plan to vote for Trump or Biden in the next election?” Even if
they survey the same people, they won’t get exactly the same responses because
they did not ask identical questions. When you hear, for example that Rasmussen
tends to get more conservative results than the Washington Post poll, part of
the explanation is probably slight differences in the way they ask questions. Tiny
differences in the question can produce big changes in the poll’s
results.
Fifth, it matters what order questions are asked
in. If a pollster asks a series of positive questions, the respondent could be
in a more favorable frame of mind when asked a hard question. Again, tiny differences
in the way the questions are organized can cause huge changes in poll results.
Good pollsters work hard to organize their questions well, but it’s an art as
much as a science.
Sixth, people respond differently to different poll
formats. In-person polls, like what Harris and Gallup used to do, are often the
most accurate. But that’s an expensive way to gather information, and it would
be unsafe during the pandemic. Telephone polls with a live operator are the
next best, but, as I said, people often hang up on them. Robocall polls are
much cheaper – and therefore can get larger sample sizes – but no one feels bad
when they hang up on a computer. Again, pollsters try to compensate for that
statistically, but that’s never as good as getting accurate answers. Internet
polls are always suspicious. Polls that you see on Twitter or Facebook are
totally useless. Polls issued by politicians are worse than useless.
Think about the obvious. Telephone polls only work
if people answer their phones. Nowadays, many people screen their calls.
Internet polls only work for people who have Internet access.
Seventh, people sometimes flat-out lie to pollsters.
There are Internet rumors that some Trump voters are embarrassed to support him
and lie to pollsters. Other rumors say some Trump voters lie to pollsters because
they want to “own liberals” or “make liberals heads explode.” I don’t know
whether that’s true, but pollsters have known for decades that people do not
always tell them the truth. (The technical term for this is “demand
characteristics,” which means that some people respond to polls socially
instead of truthfully.)
Eighth, there is what statisticians call regression
toward the mean. In simple terms, as election day gets close, most people
shrug their shoulders and vote for the political party they’ve always voted
for. No matter who the candidate is.
Earlier Post: Trump's Polarizing Rally Was All About Getting People to Vote
Ninth, and this could be the most important
problem: it’s hard to predict who is going to vote. Most polls ask whether you
plan to vote, but those responses are not always right. People might feel sick
or discouraged and stay home. Trump voters might get excited by a Trump boat parade
and show up in force. That’s important because the #1 factor in elections is
voter turnout. Few voters change their minds during the campaign. Disgruntled Republicans
who switched and voted for Biden were few and far between – and vice versa. Most
people choose candidates by party loyalty. But people don't always submit a ballot. Enthusiasm is hard to measure,
but pollsters probably need to work harder to predict who will and will not
actually vote.
Also, voter suppression states, like Texas, where I live, or Lindsey Graham’s South Carolina, make it harder for people to vote. They make it especially hard to vote in Democratic communities. It’s possible that some people who planned to vote got discouraged and never submitted a ballot.
Earlier Post: Kimberly Guilfoyle Tried to Drum up Enthusiasm in her 2020 RNC Speech
In decades past, presidential opinion polls rarely
missed by more than 3%-4%. This year, as in 2016, they seemed to be off by 5%,
or even a little more. Some statewide polls missed by more than that. Of the
hundreds of polls, some came close to the mark and some missed by a country
mile.
So, overall, polltaking is not an exact science.
Sampling error, errors in technique, and problems with the respondents themselves
all affect the accuracy of opinion polls. Pollsters spent the last four years
trying to avoid the mistakes of 2016. It appears they only partly succeeded. At
the same time, the polls were – for the
most part – more accurate than what many people might think. Biden and Trump mostly carried the states that the polls predicted.
So, yes, the polls missed Graham’s landslide
victory. But Graham also griped that the polls were wrong about Trump, and yet
Trump is losing. In the long run, the only count that matters is the official
election.
