Murray bashes methodology of Stockton poll

A Stockton University poll that put the New Jersey U.S. Senate race into a statistical dead heat is being panned by other pollsters as deeply flawed.

Patrick Murray, the director of the Monmouth University poll and widely recognized as the predominant New Jersey pollster, said that the Stockton poll released Monday morning has serious problems that raise concerns about its reliability.

“Pollsters can have honest differences about methodological choices, especially in likely voter polls — which require more professional discretion on the part of the pollster — but there are times when clear methodological problems raise concerns about the enterprise,” Murray said in a statement he released on Twitter.

Murray dismissed Stockton data suggesting that 92% of registered voters are likely midterm election voters and said that the bigger issue for the poll is their unweighted sample.

“Only 4% of the Stockton poll’s sample are voters age 18-29.   Based on my own polling of New Jersey using random samples, I would expect that number to be closer to 12%.  Stockton’s sample was only 8% black and 4% Hispanic.  I would expect those unweighted numbers to be closer to 14% and 11% respectively,” Murray said.  “And even the party id number – which admittedly can be a moving target – has been about a 15-point unweighted advantage for Dems in my recent polling but is only half that in Stockton’s poll.”

John Froonjian, a senior research associate at Stockton, said he respected Murray but defended his own methodology.

“While we recognize the demographics in the poll are not entirely reflective of the population, I believe it provides a snapshot of where the Senate race is at this point,” Froonjian told the New Jersey Globe.

The poll had ahead Menendez 45%-43%.

“Rather than having excessive weighting skew the results, I trimmed the weights, meaning reduced the size of the weight on age and Hispanic ethnicity so as not to magnify any under-representativeness,” Froonjian explained. “Because we screened for likely voters and sampled a good number of them, I felt this had value regarding the state of the race. We put out all our numbers online so that anyone could see them.”

Murray acknowledged that no single poll draws a complete and accurate sample, especially ones that utilize random dialing instead of voter lists to draw a sample.  He says that’s the reason pollsters use weighting to allow the demographic composition of a sample to match the actual electorate.

“When a sample is so far off in the unweighted data, the design effect — how much the weighting scheme itself changes the results — can have an erratic impact,” Murray said.  “For example, the few 18-29 years-olds who were interviewed in this poll can be “upweighted” to their proper share of the electorate, but if those few cases are skewed in a way that is not representative of the views of all 18-29 year-olds, you are simply upweighting the skew.”

He said this could be the case with the Stockton poll.

Menendez’s pollster, Joel Benenson, also used Twitter to suggest caution in interpreting the results of the Stockton poll.

He said that the 578 likely voter sample only had 24 people ages 18-29, and 24 Latinos, both key Democratic constituencies that are too small to assign a proper weight.

Murray said he can’t tell how Stockton weighed their data because they don’t include it in the publicly released polling memo.

“Because Stockton does not subscribe to American Association for Public Opinion Research’s transparency standards, there is no way to know this impact from what has been publicly released by Stockton,” said Murray.  “A poll which carries the imprimatur of an institution of higher education has a greater obligation than other pollsters to engage in sound research practices and to be transparent about those practices.”

Froonjian strongly defended Stockton’s track record, noting that the poll accurately predicted all of the winners and the spread within the margin of error un the last New Jersey presidential, gubernatorial and legislative races they polled in.

The results of the Stockton poll resulted in a shift of the Senate race from Leans Democratic to Toss-Up by Real Clear politics, one of the non-partisan organizations that rates the status of key campaigns.

“We go pretty much strictly off of poll averages for Senate races, and if a lead drops below five points, it’s categorized as a Tossup,” said Sean Trende, the senior elections analyst for Real Clear Politics.

Trende said that two of the last three polls for the New Jersey race fell within the tossup range, with the other barely outside of it.

“Whatever the internals of this poll may say, it’s not out of whack with what others are saying. And in the end, the nice thing about averages is that if we get a bad poll, the other polls will balance them out,” Trende told the New Jersey Globe.  “As we like to say, in a well-polled race, the averages take care of themselves.”

Benenson called the flaws in the Stockton poll significant.

“Time for all editors, producers, etc. to end the addiction to every poll that pops up,” Benenson tweeted.  “Treat them as you would any other story and report them out and verify validity first.”