9 things to know about election polling data, according to Stanford pollster David Brady

As the U.S. general election draws closer, a topic of heated discussion is the latest polling data – which came under scrutiny in 2016 after so many polls failed to accurately predict the election of Donald Trump as the 45th president of the United States. Where did pollsters go wrong in 2016 and what are they doing differently in 2020?

Here, Stanford political scientist David Brady spoke with Stanford News Service about some of the lessons learned in 2016 and what to know about tracking election forecasts – including why it might be more useful to look at data from battleground states to get a better picture of an election’s outcome rather than focusing on national polls that only reflect the popular vote, not how the Electoral College will ultimately decide who is the victor of the presidential race.

Brady is a professor emeritus of political economy at Stanford Graduate School of Business and a senior fellow at the Hoover Institution and at the Stanford Institute for Economic Policy Research. He has contributed regularly for Real Clear Politics, a website that aggregates data from national and state polls.

Below are nine takeaways from Brady about election polls.

1. Different polls can show different results.

“Polls can vary widely,” said Brady. “One way to look at polls is to turn to something like the Real Clear Politics average, which takes the average of eight or nine different polls that they believe to have some level of accuracy and integrity. This can help average out the mistakes.”

2. Polls can be biased.

“Sometimes polls can also skew to the left or the right – like the Rasmussen Report, for example,” said Brady. “But sometimes those polls can have value. If Rasmussen has Biden up by 9 points, that tells you something.”

3. Polls don’t predict voter turnout.

“One thing polls are unable to accurately measure is voter turnout,” Brady said. “Another key variable is voter turnout. We don’t have great turnout models and nobody has solved this problem. There are a lot of ways to try to do that. For example, you can ask people if they are registered to vote but more people say they are registered than actually are.

“Readers need to look carefully at how many people did the poll sample and is it a sample of registered voters or is it likely voters? Then you have to dig pretty deep to see how the poll defined likely voters versus and so on – and that gets complicated.”

4. Who is sampled matters.

Sometimes, a poll’s sample is incorrectly “weighted” – in other words, the demographics of the voting population was not reflected in the pool of people surveyed.

“The biggest problem in 2016 was the weighting of the education variable,” Brady said. “In the past, the less well-educated voted Democratic, but in the 2016 and 2018 elections, as they are in 2020, that group of voters is supporting President Trump. They were not weighted properly; in other words, we didn’t have enough of them.”

5. When people are polled also matters.

“Another mistake made in 2016 is when you do a telephone poll right after an event like a [presidential] debate, the people who are most likely to answer the call are people who thought their candidate did very well, and that skews the distribution.”

6. Also important: the number of people polled.

“Readers need to look carefully at how many people did the poll sample,” Brady said. “Here, too, people will need to look at how polls define the voters they interviewed.”

7. It usually takes a few days to gather data and interpret polling results.

Another consideration is the time it takes to generate meaningful data.

“At the Presidential Debate [Sept. 29], there were only two polls that came out that night, both of which were fairly reasonable. But generally, it takes three to five days – sometimes a week, depending on when you get the poll up – before you get the full effect of the debate,” Brady said.

8. National polls do not necessarily reflect how the Electoral College – the formal body that elects the president and vice president of the United States – will unfold.

While Donald Trump lost the popular vote to Democratic nominee Hillary Clinton in 2016, he won more seats from the Electoral College, which actually determines the election’s outcome. National polls typically do not fully reflect this process in close elections like 2000 and 2016, Brady said.

“If it’s a national sample, then it’s just the national vote that the poll is predicting. For example, in 2016, Doug Rivers’ YouGov polls said Hilary Clinton would win by 2.7 percent and that was about what she won by [2.1 percent]. So, the polls were very close – the difference was the Electoral College.”

9. To better understand the outcome of an election and how the Electoral College will vote, pay attention to state polls, particularly those in battleground states.

Because of the Electoral College, Brady suggests paying attention to the polls to emerge from battleground states.

“When you also talk about the Electoral College, what you are talking about is going into the battleground states and to aggregate results that way. I think when it comes to the Electoral College, the best thing to look at is individual state polls and the states that went for Trump in 2016 – Michigan, Wisconsin and Pennsylvania – which essentially gave him the Electoral College win.”