Who Will Believe the Exit Polls in 2008?
This article is from the archive of The New York Sun before the launch of its new website in 2022. The Sun has neither altered nor updated such articles but will seek to correct any errors, mis-categorizations or other problems introduced during transfer.
Early Tuesday afternoon, many Kerry supporters around the country thought they were sitting pretty, counting the minutes until the end of the Bush administration.
A few hours later the exit polls that were circulating on Internet blogs – which had Mr. Kerry leading 51% to 49% in Florida and Ohio, two of the crucial states in the election – seemed to Democrats to have been a cruel prank.
Now, Kerry backers, press critics, statisticians, election lawyers, and pollsters are asking the same questions: Where did the numbers come from, and why were they so wrong?
“The problem is that the samples are not good samples,” an election law professor at Washington and Lee University in Virginia, Lewis LaRue, said yesterday.
“If you say, as the famous Literary Digest poll did, ‘Send in your cards and tell us who you voted for,’ you are only polling readers of the Digest and only using those who choose to participate,” he added, referring to the 1936 magazine poll that confidently predicted Alf Landon would beat Franklin Roosevelt 57% to 43%.
The problem, Mr. LaRue said, is that exit polls are not random and, therefore, not statistically sound. Tainted samples come from, among other things, interviewers questioning voters who are more willing to, well, be questioned. Those surveys fail to capture voters who slip out of the polling booths with a polite “no thanks.”
In close races, pollsters said, it is outright dangerous to rely on exit poll data without offsetting it by a sample of raw voter tallies.
“Suppose your exit polls says Bush is down 2%, then you get returns from precincts and he is actually up 4%, that tells you your margin is 6%,” Mr. LaRue said. The numbers are then “calibrated” to compensate for the margin, he said. Without a benchmark from reporting precincts, the exit poll is misleading.
On Tuesday, women, who leaned more heavily toward Mr. Kerry, were apparently over-sampled in battleground states. The problem was not rectified until after numbers had already gotten out on Web sites and blogs.
The director of the Polling Institute at Quinnipiac University, Maurice Carroll, said another problem is that exit polls are misused. Polls are not meant to forecast election results, particularly in close elections.
The value of exit polls, Mr. Carroll said, is in breaking out demographic voting trends. Questioning voters as they leave polling sites is the only means for collecting information on how many women or how many African-Americans, for example, vote for a certain candidate.
The surveys are also useful in gauging why voters opt for one candidate. That’s how we find out what percentage of voters deemed Mr. Bush “more likely to improve the economy” or what percentage of voters saw Mr. Kerry as a “strong leader.”
The National Election Pool, a consortium of six news companies, was formed to avoid a repeat of the 2000 Florida debacle that led to networks’ initially calling the state for Vice President Gore. Though the NEP agreed to hold exit poll results until voting was closed on Tuesday, Web sites and blogs did not. Those entities are now being blamed.
In a conference call with reporters yesterday, the Kerry campaign manager, Mary Beth Cahill, said: “My initial sense of this is that the exit polls were all wrong and once again the model really didn’t work.”
“It smelled to high heaven,” said a political science professor at the University of South Florida, Susan Mac-Manus. “This is the second time in a row this has happened in Florida. I’m just glad that the networks didn’t use them and wreak havoc throughout the state.” Of the bloggers, she said: “They now have egg on their face.”
A Slate writer, Jack Shafer, who reported the polls in three installments Tuesday, defended the decision in a column yesterday, saying: “We wanted to expose the hypocrisy of networks that simultaneously embargoed exit-poll data and broadcast their essence without saying how they come by that knowledge.”
Polling methodology is now being dissected in a way it wasn’t even after the 2000 election. The opinion polls leading up to the elections, which employ a more statistically reliable methodology than do exit polls, were off, too, predicting close races nationwide. Only Pew Research Center for the People & the Press accurately predicted the Bush-Cheney ticket’s 51% to 48% victory.