crossposted at MyDD
About 2 months ago, there seemed to be a daily oversupply of poll diaries. Now…not so much. This shortage was even noted by the Jerome himself, just this morning.
I try to keep in mind that single polls are meaningless. It’s trends and rolling averages that are meaningful, not single data points. But we all like polls. They give us a feeling that we truly understand what is going on.
How do we know that looking at a single poll is mostly meaningless? Mark Blumenthal of the National Journal has an excellent piece this morning that takes on this subject, specifically with respect to Instant Response Polls:
What do those “instant response” polls tells us about what voters think about John McCain ‘s selection of Sarah Palin as his running mate? It may depend on which poll we look at.
The Gallup Organization called 898 registered voters Friday night and found that most voters had “never heard” of Palin (51 percent) or could not offer an opinion (20 percent). The few that had heard of Palin were mostly positive: 22 percent rated her favorably, 7 percent unfavorably.
On the same night, Rasmussen Reports called 1,000 “likely voters” using its automated methodology and obtained a very different result. Four of five voters were able to offer an opinion: 53 percent rated Palin favorably, 28 percent rated her unfavorably, only 18 percent said they were unfamiliar, and 2 percent were unsure.
So, as of Friday night, the percentage of Americans who knew Palin well enough to rate her was either 29 or 71 percent.
The link to the entire article is here:
http://www.msnbc.msn.com/id/26…
To close, Mark makes the case for restraint in reading too much into these “Rapid Reaction” polls below:
Thus, as the National Council of Public Polls warns, the response rates for one night surveys “are generally much lower” than for more conventional surveys. As such, and given the possibility that those at home might be different, one night polls “are much more likely to have substantial biases than polls with multiple callbacks over several days.”
Is it possible that Rasmussen’s automated methodology obtained a significantly lower response rate than the live interviewers used by Gallup? Since neither organization disclosed its response rate, we have no way to know for certain, but live interviewers are usually more adept at securing cooperation than recorded messages. Advantage Gallup on this score.
In any event, the wide differences between these two polls ought to cast some doubt on the wisdom of these instant-reaction polls. Better to set aside the one-night wonders and wait for more reliable data that will be available once the dust of the Republican convention clears.
Couple this with the fact that it’s Labor Day Weekend, means that any poll taken over the last couple of days is not reliable, in any real fashion.
Polls are not going away, and we will not stop discussing them. I am certainly not making the case that we should ignore them. Rather, we should try to view polls as trending indicators, and be willing to understand that the style of polling has a huge impact on it’s reliability.
Cheers!