Articles Comments

New Mexico Telegram » Featured, Polling » Explaining the PPP gap

Explaining the PPP gap

New Mexico Watchdog, apparently based on the gripes of an anonymous Republican, took a look at the numbers in the latest Public Policy Polling polls — specifically the approval ratings of Gov. Susana Martinez.

Specifically, the Republican questions the polling firm on Martinez’s approval ratings in two separate polls conducted near each other and suggests that reporting on the governor’s approval ratings are more important than the presidential numbers comparing President Barack Obama to Mitt Romney .

“They’re not so much a polling organization as they are a group of people trying to establish a narrative,” one statewide Republican official who preferred not to have his name used told Capitol Report New Mexico this week.

The report goes on to ignore the sample size and omits mention of margin of error in the polls.

One poll is a Project New West poll, reported on by Buzzfeed a week ago, as found with a simple Google search.

The poll, conducted by PPP for Project New West, of 587 Hispanic voters found Gov. Susana Martinez with a 57 percent approval rating among Hispanic voters, with 36 percent disapproving of Martinez’s job approval.

A separate poll, conducted between April 19 and 22 found that 51 percent of Hispanics approved of Martinez’s job approval and 41 percent disapproved.

The Republican who was granted anonymity continued his attack on PPP, while ignoring basic polling questions:

“But there are a couple problems with this thing,” a New Mexico Republican said. “First, why didn’t they highlight the poll that showed her [Gov. Martinez] at 57 percent? That’s pretty noteworthy, don’t you think? And second, what does it say about their pollling that the numbers shift six percentage points in the space of just a couple days?”

First of all, the topline of the poll in question was not Martinez’s approval ratings, but rather if Martinez’s presence on the Republican ticket would help Mitt Romney in New Mexico against Barack Obama.

The answer was no.

Secondly, the anonymous Republican fails to realize that the numbers in the second poll are from a much smaller sample size — and therefore have a much larger margin of error. The second poll was of 526 New Mexico voters — only 38 percent of whom were Hispanics. This means the sample size in the Hispanics in the second poll is just 200 (rounded up from 199.8) — and would as such have a much higher margin of error.

Even if the two polls had the same sample size and margin of error (the margin of error is not available for either sample, though the poll of 547 voters would likely be somewhere near 4 percent), it wouldn’t necessarily mean the polls were cooked or inaccurate.

The National Council on Public Polls explains.

What happens if another carefully done poll of 1,000 adults gives slightly different results from the first survey? Neither of the polls is “wrong.” This range of possible results is called the error due to sampling, often called the margin of error.

If the margin of error is, say, 4 percent, that means that each number in the poll can be adjust 4 percent either way.

There is another practical example in the form of Gallup’s polling in recent days. Take a look at Gallup’s presidential job approval numbers:

Even at a position as high information as the President of the United States, there can be dramatic shift in just a few days.

And that is without factoring in sample size or margin of error.

One last thing — the report notes a Republican conspiracy theory that PPP gins up their numbers early in the cycle before moving towards accuracy as the election nears. To use Occam’s razor, there is a simpler explanation.

As elections near, the likely voters become more apparent and likely voter screens are more accurate. Also, polls are not predictive, rather they are snapshots of the thought of voters at the time.

So just because a poll from two months ago shows something different than the poll right before the election does not mean the poll two months ago was wrong — it likely means that public opinion changed in the intervening time.

If one polling firm’s numbers differ from the others months out but gradually converge on another (as some polling firms do, if you look at the numbers at Talking Points Memo’s Polltracker or Huffington Post’s polling aggregator) then there could be some questions.

But there just aren’t enough public polls tracking Martinez’s job approval ratings to find out if this is the case with PPP in New Mexico.

Either the anonymous Republican has a flawed understanding of basic polling conventions or he hid behind anonymity to cast doubt on the Democratic pollster.

Written by

Matthew Reichbach has blogged about New Mexico politics since 2006. Matthew was a co-founder of New Mexico FBIHOP with his brother and part of the original hirings at the groundbreaking website the New Mexico Independent. Matthew has covered events such as the Democratic National Convention and Netroots Nation. In addition to politics, Matthew is an avid sports fan, especially of the Los Angeles Dodgers, and TV fan.

Filed under: Featured, Polling · Tags:

2 Responses to "Explaining the PPP gap"

  1. Ironically, what the R’s allege about PPP is *exactly* what Rasmussen Reports does, except in the Republicans’ favor. Project much, Republican concern trolls?

  2. Matt —

    I just read your post about my post. You’ve got your take, I’ve got mine. Fair enough. Maybe the Republicans are whining. Or maybe PPP is trying to create a narrative. Or it’s something different or something in between. Who knows.

    But the only thing I object to in your piece is the line “One poll is a Project New West poll, reported on by Buzzfeed a week ago, as found with a simple Google search,” which insinuates that I was lazy in trying to track down the poll.

    I wasn’t.

    After seeing the Scribd.com document, I could see it came from someone named Zeke Miller and from there it was easy to see he wrote for Buzzfeed.

    Maybe your Internet skills are more advanced than mine but when I got to the Buzzfeed website I didn’t see the post you linked to.

    I went to the site’s search box and typed in zeke+miller and found a lot of his posts but after clicking through three pages couldn’t find anything about a poll of New Mexico Hispanics (although I did find a post of video of a Montana congressman picking his nose while on camera, but I digress).

    I then typed in zeke+miller+new mexico and got, “Sorry, we couldn’t find any hot results that match that search.”

    Same result when I typed in zeke+miller+susana+martinez

    So I’ve got a poll on Scribd but but at that point I can’t confirm it. It’s not on the PPPP site, I can’t find it on the Buzzfeed or anywhere else.

    So I made a good faith effort.

    If Mr. Jensen at PPP had bothered to return my calls, it could have been confirmed.

    Rob