Live Science Plus
Survey Says: Polls Have Problems
Election polls showing John McCain ahead one day, Barack Obama the next, then some neck-and-neck results the next day, are seriously flawed, according to one pollster. Another pollster begs to differ, saying polls provide valuable information about public opinion on candidates and about which issues are pushing the electorate.
"Right now polls don't tell the truth about the electorate and they don’t tell the truth about the American public," said David Moore, founder of the University of New Hampshire Survey Center and former managing editor of the Gallup Poll.
Moore's main issue involves the wording of a standard poll question, which asks who a person would vote for if elections were held today. Rather than giving voters a chance to report mixed feelings or just not knowing, polls tend to "force" a definitive answer, said Moore, author of "The Opinion Makers: An Insider Exposes the Truth Behind the Polls" (Beacon Press, 2008).
Other shortcomings include the lack of cell-phone users polled as well as the natural variability that occurs in voter opinions months before the election.
To some, however, rejecting all polls seems a little extreme. "I think that's vastly overblown if it's an attempt to discredit virtually all polling because of this issue," said Charles Franklin, a political scientist at the University of Wisconsin, Madison. "It rests on a fundamentally correct fact. Absolutely the way you word questions affects the answers that you get. But for anyone to claim there's one right way to ask the question and every other way is flawed, I think is a vast overreach."
In the end, polls can be analyzed after the fact. The truth: Polls done months before an election don't turn out to have been very predictive of real outcomes.
Track record
Get the world’s most fascinating discoveries delivered straight to your inbox.
Though polls have occasionally failed to predict who will win an election, most notably in the 2008 Democratic primary in New Hampshire in which Hillary Clinton won, the polling track record is "very good," according to the Pew Research Center.
This is particularly true for polls taken close to an election. For example, in 2004 the average of several major national polls from the days leading up to the presidential election showed President Bush with a 1.6 percentage point advantage over Sen. John Kerry. Bush ended up winning the election by 2.4 percentage points.
Election polls taken early in a race, during the first quarter of the year prior to the presidential election, have shown a poor track record in predicting the winner, according to a review of polls between 1959 and 2003 by the Pew Research Center.
"Polls conducted early in an election season should be taken as snapshots in time, and obviously cannot capture the impact of the campaign and its events to come," according to Pew analysts.
For instance, a Pew analysis of polling done early in campaigns found that in February 1995, several early readings showed Sen. Bob Dole leading President Clinton by as many as 6 percentage points. Then, 21 months later, Clinton won by 8 percentage points.
"If you take all previous presidential elections, the polls vary a lot over time and they all end up basically where the election results are," said Gary King, a political scientist at Harvard University.
So as the election gets closer the polls all tend to narrow down and point to the right candidate.
"By the time you get to the night before the election that's pretty much what the election results are going to be," he said, adding that political scientists are pretty accurate at forecasting the election outcome at the time of the conventions.
As for why the polls are so variable and possibly inaccurate months and months before the elections, King said, it's "natural variability" in part. "People don't really know who the candidates are yet. There's no reason for them to decide who they're really going to vote for months before the election. They only really have to know by November," King said during a telephone interview.
Missing cell-phone users
Natural variability is just part of the problem. Many Americans are ditching their landlines for cell phones, a trend that can wreak havoc on election polls.
While some polls are starting to include cell-phone users, others aren't.
Surveys conducted by Pew in June, July and September showed that including cell-phone interviews led to results showing more support for Obama and slightly less for McCain.
For instance, the September poll involved more than 2,500 registered voters including nearly 550 individuals reached by cell phone. The combined phone-type results showed 46 percent backed Obama and 44 percent backed McCain. Among just the landline respondents, candidates were tied each with 45 percent support.
The difference between cell-only and landline individuals is age, with the cell-only sample being younger than 30, Pew analysts suggest. Young people as a group, according to Pew, have consistently backed Obama this year.
King sees the cell-phone issue as a big problem.
"There's real reason to worry about that because of the rise in cell phones and non-response," King said, referring to the ability of the polls to predict the public sentiment on the day of polling.
In addition to cell-only individuals, the pollsters don't grab a real random sample of the American public, King said.
"Nine out of ten people that pollsters call don't answer the phone or they can't reach the person," King said. The people who are home and do decide to answer the polling call, he said, are probably not representative of the people who will vote on Election Day.
Who would you vote for today?
Moore calls for polling reforms, including measuring and reporting the percentage of undecided voters, and recognizing bias in question wording and other question features.
Other political scientists disagree about the forced-question issue.
Franklin said research has shown this "forced" type question doesn't skew the results.
"If you predict the answer to that question by your political ideology, your partisanship, how you feel abut the environment, your age, education, the usual suspects, you get the same structure for people who were pushed to give an answer as for people who were not," Franklin told LiveScience.
"If there was a serious flaw to asking the question with this push for how do you lean, then we ought to see polls consistently missing the right answer of the outcome," Franklin said. "We don't see that."
He added that individual polls can be off the mark, but on average, they get it right.
- Video – Presidential Marketing Power
- Quiz: Bizarre U.S. Presidential Elections
- Top 10 Ailing Presidents
Jeanna Bryner is managing editor of Scientific American. Previously she was editor in chief of Live Science and, prior to that, an editor at Scholastic's Science World magazine. Bryner has an English degree from Salisbury University, a master's degree in biogeochemistry and environmental sciences from the University of Maryland and a graduate science journalism degree from New York University. She has worked as a biologist in Florida, where she monitored wetlands and did field surveys for endangered species, including the gorgeous Florida Scrub Jay. She also received an ocean sciences journalism fellowship from the Woods Hole Oceanographic Institution. She is a firm believer that science is for everyone and that just about everything can be viewed through the lens of science.