BY JIM KANE
Two national polls this past week had completely different results on the Obama/Romney race.
Gallup had Romney up by 2 points (47% to 45%) while CNN/ORC had Obama up by 9 points (52% to 43%).
Both polls were done within the same period (April 11-15) and both were large samples of registered voters with sampling errors of +/- 3% (Gallup) and +/-3.5% (CNN).
Obviously, the differences cannot be blamed on sampling error.
Curious, I decided to have my University of Florida students look at both questions to see if somehow the wording of the question may have caused the differences in results. (We call this context effects.)
I thought that unlikely since both polling firms have a long history of doing excellent surveys, but these are graduate students studying political surveys and I thought the exercise would be beneficial.
At first glance, it appeared that both “trial ballot” questions used were nearly identical.
A closer look revealed something that surprised me and my students.
The Gallop poll used the phrase “If Barack Obama were the Democratic Party’s candidate and former Massachusetts Gov. Mitt Romney were the Republican Party’s candidate, who would you vote for?”
The CNN/ORC question only identified the “Barack Obama as the Democratic Parry’s candidate, and Mitt Romney as the Republican Party’s candidate.”
In survey research when you identify a person’s office or position, it evokes in some people a sense of additional importance. This is called the “prestige effect” and virtually every experienced polling firm avoids this mistake unless they are intending to bias the outcome.
Although most folks know Romney in name, most average voters would be hard pressed to know his previous role as Massachusetts governor. Additionally, this added reference can add an untended image of Romney linked to Massachusetts more progressive (liberal) policies.
Perhaps some voters (mainly ambivalent voters) have an initial concern that Romney may be “too extreme” on social issues due to the hard right positions he took in the primary. The reference to Massachusetts could possibly mitigate this concern.
But that’s the problem with unintended effects. You don’t always know how thought processes are changed, if at all.
I do not think Gallup intended to bias the outcome, but did this inadvertently, probably thinking voters needed to know more about Romney’s past experiences. I have never, however, seen this before in any of Gallup’s previous polls.
(Browardbeat.com pollster Jim Kane has conducted polls for candidates, sponsors of referendums, state political parties, media companies such as the South Florida Sun-Sentinel and business groups. He was the first Democratic pollster to work for the Republican Party of Florida. In addition to his political work, Kane currently teaches graduate seminars on the Gainesville campus of the University of Florida in survey research, political behavior, political campaigning and political parties and interest groups. He is frequently quoted in national publications and is a commentator on national television.)