A few good links on the accuracy of McCain/Obama polling

In August I posted some questions about the accuracy of opinion polls on the presidential race. I wondered whether any of the following factors might introduce distortions in the polling:

-the growing number of voters who use only cell phones;

-the practice of polling on weekends (when certain demographic groups are less likely to be at home);

-the varying estimates of the partisan and demographic breakdown of the electorate;

-the enormous disparity in the two campaigns’ ground games (which is even more obvious now than it was in the summer).

Most of the factors I mentioned would lead polls to understate support for Barack Obama. However, some political analysts have also questioned whether polls might be overstating Obama’s support because of the “Bradley effect” (or “Bradley-Wilder effect”), whereby white people tell pollsters they plan to vote for a black candidate but act differently in the voting booth. Here at Bleeding Heartland, American007 has expressed concern about this possibility.

We won’t know how accurate the polls were until November 5, and even then we won’t be able to prove how much of the difference was related to last-minute external events and how much was related to pollsters’ errors in weighting their samples, or respondents lying about their intentions.

However, here are some pieces worth your time if you enjoy this kind of speculation.

The “mystery pollster” Mark Blumenthal doubts there will be a “perfect storm” leading to wildly inaccurate polling, because

the potential polling foibles may work in opposite directions and “cancel each other out.” A return of the Bradley-Wilder effect would work to McCain’s benefit, while an underrepresentation of younger, African American or “cell-phone-only” voters will likely benefit Obama.

Embedded in that piece is a link to a research paper (pdf file) by Daniel J. Hopkins, a post-doctoral fellow at Harvard who analyzed data from 133 gubernatorial and Senate elections from 1989 to 2006. He found a Bradley effect in the early 1990s but no evidence that it still existed in more recent elections. If you don’t want to download the whole file, Sam Wang of the Princeton Election Consortium summarized Hopkins’ findings in this post on “The disappearing Bradley effect.”

Sam Wang looked at the evidence about cell phone users here and believes this factor is probably only understating Obama’s support by about 1 percent.

But Nate Silver of fivethirtyeight.com compared the McCain-Obama numbers in many polls and found that Obama does 2 to 3 percent better in surveys by pollsters that call cell phone numbers in addition to landlines.

Silver is not concerned about the Bradley effect after analyzing the primary results. Obama did better than his pre-election polling numbers in more states than he underperformed.

I wonder whether the kind of person who would lie to a pollster about being willing to vote for a black candidate is more likely to vote in a general election than in a Democratic primary. That said, I do find Hopkins’ analysis persuasive, so I have decided not to worry about the Bradley effect either.

I’m not a pollster or a statistician, but my hunch is that the greatest potential for pollster error is in the assumptions made about relative turnout by certain demographic groups. Should we assume the proportion of Democrats, blacks and young voters will be about the same as 2004, or should we assume higher turnout in those groups? Being wrong in one direction or another could significantly skew the results, especially in states with large black populations.

The Research 2000 tracking poll for Daily Kos is assuming a higher proportion of Democrats in the electorate than Gallup and Rasmussen, for instance. I assume Democrats will increase their share of the electorate because of the trends in voter registration over the past year as well as the enthusiasm gap. However, Jerome Armstrong is among the skeptics who think the partisan turnout will look very much like 2004.

I would question any poll that assumes African-American voters will make up the same proportion of the electorate this year as they did in 2004, especially in states where Obama has a massive voter turnout operation and John Kerry did not compete (such as North Carolina, Virginia and Missouri). Even in Georgia, where Obama has significantly reduced staff since the summer, we can expect to see much higher black turnout if voter registration trends and early voting are any indication.

I am less confident about a surge in young voter turnout, but if that did happen, pre-election polls weighted according to the 2004 figures would understate Obama’s support.

If Latino turnout is higher than in 2004, Obama will benefit because McCain does quite poorly among Latinos, far worse than George Bush did in 2004.

What do you think? Are you counting on polls to be mostly accurate this year, or significantly off the mark in one direction?

About the Author(s)

desmoinesdem

Comments