UPDATE: Nate Silver followed up here, comparing patterns in Strategic Vision poll findings to those from Quinnipac.
Strategic Vision released a number of Iowa polls during 2007, sampling Democratic and Republican would-be caucus-goers on the presidential candidates.
Did any Iowa journalist or political analyst receive cross-tabs or any details about the methodology from these polls? Does anyone remember talking to any Iowan who had been surveyed by Strategic Vision?
I’m asking because incredibly, polling experts are now questioning whether Strategic Vision has been conducting polls at all. More on that story is after the jump.
I’ve never paid much attention to Strategic Vision polls, because I assumed that a Republican PR firm would produce results slanted toward Republican candidates. Even so, it didn’t occur to me that Strategic Vision might be fabricating results.
The American Association for Public Opinion Research has criticized Strategic Vision for refusing “to release essential facts about polls it published prior to the 2008 presidential primaries in New Hampshire and Wisconsin.”
Strategic Vision also refuses to release cross-tabs for any of its polls and doesn’t seem to have any physical offices. (The addresses listed on its website are UPS offices.)
Nate Silver of FiveThirtyEight.com came right out and said it (emphasis in original):
One of the questions, in light of Strategic Vision LLC’s repeated failure to disclose even basic details about its polling methodology, is whether the firm is in fact conducting polling at all, or rather, is creating fake but plausible-looking results in order to increase traffic and attention to its core business as a PR and literary firm.
I posed that question largely as a hypothetical yesterday. But today, I pose it much more literally. Certain statistical properties of the results reported by Strategic Vision, LLC suggest, perhaps strongly, the possibility of fraud, although they certainly do not prove it and further investigation will be required.
Silver looked at results for all questions in more than 100 Strategic Vision polls and found that some numbers were much more likely to be the final digit than other numbers. That is, the percentage of respondents answering any given question a certain way was much more likely to end in a 7, 8 or 9 than a 1 or 2.
Even without considering Silver’s “trailing digit” analysis, Mark Blumenthal of Pollster.com finds enough conflicting statements and troubling facts to question whether Strategic Vision’s polls “deserve our trust.”
Ben Smith’s piece for Politico on this controversy is also worth reading.
Whether or not Strategic Vision has been faking its polls, it’s disturbing that so little is known about their methodology. Blumenthal is right to call for scoring “the quality of disclosure of every public survey release” and “publish[ing] the scores alongside polling results.”
If you’ve been a respondent for any Strategic Vision poll during the last four years, Nate Silver wants to hear from you. For future reference, always ask who’s conducting the survey if you agree to participate in any telephone poll.
3 Comments
Ok, this is nuts...Or not..
The USA Today polls always look like somebody is “stepping the graph”, but you can always get their crosstabs and break it down accordingly.
Saying that crosstabs are “proprietary”, and only available to paying customers?
Well, I dunno. I tried to establish a mailing list to caucus precinct captains once for a push at a platform plank. Iowa Dem Party only wanted $70,000 for the list.
eltondavis Sun 27 Sep 7:36 PM
Strategic Vision doesn't make them available to anyone
It’s common to restrict crosstabs to subscribers. Read Mark Blumenthal’s piece. It’s not clear whether anyone has ever seen crosstabs for one of their polls. They won’t release the most basic information to AAPOR. You would think that Nate Silver’s post would prompt them to release some proof that they’ve been conducting surveys, even if it’s only payroll to the people who worked the phones.
desmoinesdem Sun 27 Sep 8:13 PM
The last comment I saw from Strategic Visions
is that even though they are not a member of AAPOR, and felt no obligation whatsoever to provide crosstabs, they would start doing so. Maybe that’s just getting their hand caught in the cookie jar talking, who knows?
Strategic Visions also said that litigation would commence against those alleging impropriety. We’ll see, but if it is the case that they were fabricating poll data, gads.
eltondavis Sun 27 Sep 10:34 PM