I am a retired lawyer, not a statistician, but it occurs to me there could be an error that is tainting your results. Let's suppose for the sake of argument that polling is accurate and selects the respondents which actually predict the outcome. Then let's add in the impact of gerrymandering. If it actually perverts the outcome of the election, wouldn't it fully explain the errors in polling. If the poll accurately predicts the outcome, but gerrymandering kept one political party from voting, they wouldn't be able to vote as predicted, would they.
As long as candidates keep campaigning, the election will be what it will be. If polls don't help candidates win or lose elections, then how do they differ from watching a telenovela? Polls might change money coming into a campaign, they might change campaign worker enthusiasm or voter turnout. Maybe the right question to ask about polling isn't how well do they predict outcome (winner) but when do the polls predict futility? A dangerous question for anyone who remembers Patriot's Superbowl comeback from 28-3. Or if you prefer the other form of football, Leicester City (2016) with preseason odds of 5,000-to-1, a team that narrowly avoided relegation the year prior won the English Premier League.
Would love to see 95% confidence limits for polls predicting winners. If polling keeps most of the races in the lean or tossup, not 95% confidence of predicting a winner, then let's just campaign and not worry about the noise.
I have not finished your article yet, my bad, but will be looking for how polls compensate for gerrymandered states where the results are not dependent on voter turnout out or party preference.
In Ruffini's poll centric way of looking at campaigns, he inherently discounts the fact that the reason for Republicans outperforming summer polling is the GOP running more effective fall campaigns - betters ads, better mail, better field and turnout - in favor of simple "bias" in summer polling.
If elections come down to persuading the small group of Independents who hate both parties and would not say they are voting Republican in an August poll, and Republicans keep winning them in November, I would celebrate that. The problem is that his president is polling in the high teens, low 20s with them on approval with 2/3rds disapproving. Much better to throw out some "poll bias" chum on Twitter instead.
In the aggregate, the over/under predictions are within the range of measurement error. The unskewing probably just magnifies that because it is error variance. Even though you have some idea of what accounts for this variance, it's based on intentions that have limited predictive validity. Being a "likely voter" in July is probably a pretty imperfect predictor of being an actual voter in November. Aside from face valid anecdotal explanations, the simple fact is that intentions are lousy predictors of behavior the more distal they are (lots of social psych research on this). Also, the act of voting sounds simple, but can become a complex social behavior depending on numerous factors--people moving, the weather, the intensity of the desire to vote, how easy it is to vote, etc. Most of those sources of variance are dynamic and unstable.
The one proxy for actual voting might be past voting behavior. There will still be a lot of variance, but it's probably a stable behavior and vivid enough to limit recall bias. People who consistently vote or always vote in mid-terms might provide better prediction or at least allow you to introduce alternative data points for comparison that might prove useful over time, esp. in the aggregate.
I am a retired lawyer, not a statistician, but it occurs to me there could be an error that is tainting your results. Let's suppose for the sake of argument that polling is accurate and selects the respondents which actually predict the outcome. Then let's add in the impact of gerrymandering. If it actually perverts the outcome of the election, wouldn't it fully explain the errors in polling. If the poll accurately predicts the outcome, but gerrymandering kept one political party from voting, they wouldn't be able to vote as predicted, would they.
Yes! More yogi berra quotes :)
As long as candidates keep campaigning, the election will be what it will be. If polls don't help candidates win or lose elections, then how do they differ from watching a telenovela? Polls might change money coming into a campaign, they might change campaign worker enthusiasm or voter turnout. Maybe the right question to ask about polling isn't how well do they predict outcome (winner) but when do the polls predict futility? A dangerous question for anyone who remembers Patriot's Superbowl comeback from 28-3. Or if you prefer the other form of football, Leicester City (2016) with preseason odds of 5,000-to-1, a team that narrowly avoided relegation the year prior won the English Premier League.
Would love to see 95% confidence limits for polls predicting winners. If polling keeps most of the races in the lean or tossup, not 95% confidence of predicting a winner, then let's just campaign and not worry about the noise.
I have not finished your article yet, my bad, but will be looking for how polls compensate for gerrymandered states where the results are not dependent on voter turnout out or party preference.
As usual, stellar analysis.
In Ruffini's poll centric way of looking at campaigns, he inherently discounts the fact that the reason for Republicans outperforming summer polling is the GOP running more effective fall campaigns - betters ads, better mail, better field and turnout - in favor of simple "bias" in summer polling.
If elections come down to persuading the small group of Independents who hate both parties and would not say they are voting Republican in an August poll, and Republicans keep winning them in November, I would celebrate that. The problem is that his president is polling in the high teens, low 20s with them on approval with 2/3rds disapproving. Much better to throw out some "poll bias" chum on Twitter instead.
Thanks for always educating us as you explore the nagging questions surrounding polling.
Patrick Ruffini made a farcical argument that America is secretly a right-wing country??? Nooooooooooo I'm so shocked!! 😱😱😱
In the aggregate, the over/under predictions are within the range of measurement error. The unskewing probably just magnifies that because it is error variance. Even though you have some idea of what accounts for this variance, it's based on intentions that have limited predictive validity. Being a "likely voter" in July is probably a pretty imperfect predictor of being an actual voter in November. Aside from face valid anecdotal explanations, the simple fact is that intentions are lousy predictors of behavior the more distal they are (lots of social psych research on this). Also, the act of voting sounds simple, but can become a complex social behavior depending on numerous factors--people moving, the weather, the intensity of the desire to vote, how easy it is to vote, etc. Most of those sources of variance are dynamic and unstable.
The one proxy for actual voting might be past voting behavior. There will still be a lot of variance, but it's probably a stable behavior and vivid enough to limit recall bias. People who consistently vote or always vote in mid-terms might provide better prediction or at least allow you to introduce alternative data points for comparison that might prove useful over time, esp. in the aggregate.