Florida’s 2018 Polls: Fair Or Foul?

 

BY JIM KANE  

 

 

 

After Trump’s surprising victory, many media pundits questioned the use of polls in the modern age of the internet and the increasing non-responses in today’s political surveys. The mantra the “political poll is dead” filled the airwaves after the “stunning” Trump victory. 

But perhaps the announced death of polling was a little premature. 

Nate Silver, the statistical guru of the FiveThirtyEight blog, looked at the historical accuracy of surveys (The Polls Are All Right, May 30, 2018) and found, that although the polls often get the results wrong, “they’re about as accurate as they’ve always been.”

Don’t get me wrong. Several of the state polls taken in the “Blue Wall” were significantly off the mark. National surveys, however, were mostly in line with the actual results, with the final Real Clear Politics average having Clinton winning by 3.2%%. On election night, Clinton won the national popular vote by 2.1%. And as we all know, the Electoral College gave the victory to Trump.

With Florida having just completed its election, I thought it would be interesting to see how accurate the polls were just prior to the 2018 Florida election, specifically the Rick Scott, Bill Nelson race and  the Andrew Gillium and Ron DeSantis contest.  

My methodology is simple: Gather every public poll conducted in these two races that occurred between October 1 and November 5. This time frame ensures that most voters have some understanding who the candidates are and to some degree, the issues proposed by the candidates’ campaign. In other words, polling during this period should be more accurate than if the surveys occurred earlier. Only polls, however, not sponsored by either a candidate, party or special interest group were included.

Although Florida is known for close elections, the two 2018 races set the standard for how narrow a victory can be. 

On election night, Rick Scott had defeated incumbent Bill Nelson by only two-tenths of a percent. It was almost as close in the Governor’s contest, where DeSantis defeated Andrew Gillum by six tenths of a percent. Consequently, most polls frontrunners bounced back and forth between the candidates. 

During this period, some 18 different firms conducted 74 surveys for either the Senate or Governor’s race (or both). But how accurate were they? 

The simplest and best way to measure a poll’s accuracy is how far the poll’s results were from the actual election outcome. This statistic is called the Simple Percent Average Error. 

Using the average error, the accuracy of different polling firms can be compared to each other using the same metric, without the consideration the firm’s polling methodology. In other words, the poll with the least amount of error in the table below wins.  

As Table 1 shows, the average error for all polls conducted during this period was 3.8 percent. The average error for individual surveys ranged from a low of 1.5 percent (Suffolk University) to a high of 7.5 percent (Survey/USA).

TABLE 1

FIRM  #

POLLS

AVERAGE ERROR RANK
CNN 3 4.3 9*
HARRIS/INTERACTIVE 13 3.0 5*
GRAVIS 4 2.3 4*
QUINNIPIAC 3 6.7 12
IPSOS 4 5.0 11*
MARIST 3 4.3 9*
ST.PETE POLLS 9 1.9 2
EMERSON 2 5.0 11*
TRAFALGAR 4 2.3 4*
FAU 2 3.5 6*
KAISER FOUNDATION 2 4.5 10
SCHROTH-EDISON 2 4.0 7
SURVEY/USA 2 7.5 13
SIENNA/NYT 2 4.1 8
UNIV N. FLA 7 3.5 6*
FLA SOUTHERN  2 2.0 3
SUFFOLK UNIV 3 1.5 1
VOX 2 3.0 5*
TOTAL AVERAGE ERROR 3.8

*Tied ranking

The average error for each poll, however, varied from a low of 1.5% to 7.5%. That’s a big difference considering the time frame is only one month. 

But how does this compare to other states’ state-wide polls? In surveys conducted in the last 21 days of the 2018 election cycle in other states (Governor and Senate combined), had an average percent error of 5.1.

In other words, the Florida polls did well, at least compared to other polls.  

It’s important to note that not all these surveys use probability samples. Many are hybrids of different modes, such as Harris Interactive, which combines online samples with an IVR poll.  In a future post, I’ll explain the differences between survey methods and their effect on results. 

Finally, what I found interesting was how accurate Florida based surveys performed.

Florida Southern College, a small school located in Lakeland, had an average error of only 2 percent and ranked third (admittedly, with only two polls). The St. Pete Poll had an average error of only 1.9 percent and ranked second. FAU and the University of North Florida both had errors significantly below the average as well. Perhaps local knowledge does help even in polls!

 

XXXXXX

 

One of South Florida’s premier pollsters and political strategists, James G. Kane is a graduate of George Washington University’s political science program, Kane worked on more than three dozen Republican and Democratic campaigns. Candidates have used his skills for campaign management, message design, voter contact, opposition research and polling.

Kane has conducted polls for candidates, sponsors of referendums, state political parties, media companies such as the South Florida Sun-Sentinel and business groups.  He was the first Democratic pollster to work for the Republican Party of Florida.

In addition to his political work, Kane has taught graduate seminars at the University of Florida in survey research, political behavior, political campaigning and political parties and interest groups.   He also is a regular political commentator on local and national television.



7 Responses to “Florida’s 2018 Polls: Fair Or Foul?”

  1. City activist Robert Walsh says:

    Polls mean nothing.A tool for these polling companies and political operatives( hi Judy) to get a cut($) of the price.

    Just look at Trump.Every poll that was conducted stated he would lose and lose big.

    He prevailed.

    What they do( polls) is they get a list of say if it’s a Democrat running will call registered Demos and wow your gonna win just look at the polls.

    Vice versa if a Republican hires poller co.Than the canidates staff use these favorable polls to say see look how beautiful are canidates is doing.What is effective is TV.Get that canidates out there.With their families( money shot) get with esp.their pets( don.t have one,get one).Also the fitter u look the better reception u will get.Look at Jeb Bush those glasses make him look weird etc.My take get to the gym.Get a dog.Not married get a buffer( I have plenty) . Lastly dress to the nine.Thats how he win.

    Polls waste of money…

  2. Herman Faulk says:

    Polls always have a margin of error. They never are 100 percent predictive. A lot of people forget that.

  3. Humpy Dump says:

    Kane is obviously in a bar. typical. pollsters seem to get their results out of the bottom on a glass. I don’t trust polls.

    FROM BUDDY:

    The picture was taken in one of Jacksonville airport’s restaurants.

  4. Count. LF Chodkiewicz Chudzikiewicz says:

    #1 WRONG AS USUAL. NO KNOWLEDGE.
    a. Polls were RIGHT ABOUT THE POPULAR VOTE because POLLS SHOW COBSERVATIVE WHITE VOTERS UNDER ACKNOWLEDGE CONSERVATIVE VOTES IN ESPECIALLY COMPETIVE STATES .
    b THE POLLS DID NOT PREDICT THE ELECTORAL COLLEGE ALIGHNMENT WITH STATE BY STATE POLLS but MEDIA WRONGLY COVERED NATIONAL POLLS THAT WERE CORRECT.
    c. Polls DO NOT predict TURNOUT IF EARLIER THAN 2 WEEKS or LESS BEFORE ELEXTION DAY.
    d. POLLS CANNOT PREDICT COMEY WOULD BREAK JUSTICE DEPARTMENT RULES

  5. Why says:

    Interesting that the fabled QUINNIPIAC, which is constantly repeated in the media, did so poorly while places no one has heard of like Florida Southern did well. What do you attribute that to , Mr. Kane? It can’t just be familiarity with Florida since it involves numbers.

    FROM JIM KANE:

    I reacted the same way when I saw Quinnipiac Florida poll’s average error was 6.7 percent, significantly higher than all but one other polling firm (including little Florida Southern College). Quinnipiac has an excellent reputation among the survey research community. Nate Silver (FiveThirtyEight Blog) gives Quinnipiac an A rating due to its methodology and accuracy.

    So why did they miss the mark so badly? In line with Quinnipiac’s rating, they employ live interviews derived from a random sample of telephone numbers (RDD). As methodologically pure as this method is, it relies on the respondent’s truthfulness about their previous voting habits and, most importantly, their likelihood of voting on election day.

    My experience with this method suggests the respondent almost always says they are certain to vote on election day. In the social sciences, this is called “social desirability bias.” People tell the interviewer what the socially acceptable answer is: “I’m definitely voting!”

    Experienced commercial pollsters don’t rely on self-reports. Instead they use voter election data, that shows how often the respondent votes in past elections. Depending on the expected turnout, the pollster then assigns a “likely voter” label.

    Florida is difficult state to survey, due to its demographics and the transiency of the population. Likely voters are more settled and have lived in the state for some period. Using election data with a history of the respondents’ voting, helps prevent a reliance on inaccurate self-reports.

    So how did Quinnipiac mess up? In my opinion, their reliance on voters’ self-reports likely caused an oversample of less likely voters. Consequently, the predicted winning candidate derived from the survey results are inaccurate.

  6. Gerry Lindscom says:

    Voters and especially the media must remember that polls have a margin of error and are often, as Kane indicated, even more off the mark than the margin.

    The Age Old axiom is more important today than at any time in the past. The Only Poll That Counts is On Election Day.

  7. City activist Robert Walsh says:

    Bottom line canidates should not rely on polls or polling etc.Best bet.Get out there.Knock on doors.Go to neighborhood HOA.Get with these condo commandos.Learn their issues and concerns.What can u do for the residents (voters) and how can u make their lives easier.Look nice.Be presentable.Smile even if it seems like campaigning will never end.Lastly,never,never let them( voters) see u sweat….That is how u win.Andnu better believe I know how this whole system,process works.. Will see what lies ahead for u Robert…(hmm)….