Also wondering how advertising moves the polls as we get closer to election day. What someone thinks about a candidate in August may be shaped by the opposing candidate pointing out unpopular positions or unfavorable incidents in fall ads, and most people have only a passing knowledge of most candidate's positions. As a resident in a swing state here in WI, I can't overstate the amount of advertising we're exposed to, the vast majority of which is negative.
My standing reminder that every assessment of polling "bias" or "misses" -- at least any coming from the adult table -- is predicated strictly and faithfully on the assumption that vote counts are honest and accurate and therefore reflect the true disposition of the electorate, from which baseline we can measure the success or failure of the polls.
If we consider the thumbs on the electoral scales, some (gerrymandering, disinformation, e.g.) should be fairly accurately reflected in polling, but others (voter purges and other forms of suppression, electronic vote count manipulation, e.g.) will not be.
When looking at a consistent, unidirectional pattern of putative polling/sampling "bias" -- one that pollsters' best efforts at correction over multiple election cycles can't seem to eliminate -- perhaps it is not absurd to also consider the possible "bias" in the data the polling is attempting to model. I.e., the tabulations producing the official results.
It's reasonable to assume that public opinion of Trump, and his numbers are in the gutter, are bound to negatively affect the Republican vote and raise it for Democrats. It's a factor that cannot be ignored by citing what happened in previous elections. With tyranny raising its ugly head, we're in new territory right now.
I wonder if the "August bias" might have any correlation with either the presumption of anti-party-in-power voting given difficulty in predicting turnout in November based on statements in August? Also, if there's any correlation between it and the historical context that the electorate tends to move left in a series of swing then backlash moves because of the structure of our party / political systems?
I am a retired lawyer, not a statistician, but it occurs to me there could be an error that is tainting your results. Let's suppose for the sake of argument that polling is accurate and selects the respondents which actually predict the outcome. Then let's add in the impact of gerrymandering. If it actually perverts the outcome of the election, wouldn't it fully explain the errors in polling. If the poll accurately predicts the outcome, but gerrymandering kept one political party from voting, they wouldn't be able to vote as predicted, would they.
Elliott’s model accounts for gerrymandering, as do the other major models out there (Silver, Split Ticket etc.). And the post is primarily about Senate races, which are statewide.
As long as candidates keep campaigning, the election will be what it will be. If polls don't help candidates win or lose elections, then how do they differ from watching a telenovela? Polls might change money coming into a campaign, they might change campaign worker enthusiasm or voter turnout. Maybe the right question to ask about polling isn't how well do they predict outcome (winner) but when do the polls predict futility? A dangerous question for anyone who remembers Patriot's Superbowl comeback from 28-3. Or if you prefer the other form of football, Leicester City (2016) with preseason odds of 5,000-to-1, a team that narrowly avoided relegation the year prior won the English Premier League.
Would love to see 95% confidence limits for polls predicting winners. If polling keeps most of the races in the lean or tossup, not 95% confidence of predicting a winner, then let's just campaign and not worry about the noise.
I have not finished your article yet, my bad, but will be looking for how polls compensate for gerrymandered states where the results are not dependent on voter turnout out or party preference.
In Ruffini's poll centric way of looking at campaigns, he inherently discounts the fact that the reason for Republicans outperforming summer polling is the GOP running more effective fall campaigns - betters ads, better mail, better field and turnout - in favor of simple "bias" in summer polling.
If elections come down to persuading the small group of Independents who hate both parties and would not say they are voting Republican in an August poll, and Republicans keep winning them in November, I would celebrate that. The problem is that his president is polling in the high teens, low 20s with them on approval with 2/3rds disapproving. Much better to throw out some "poll bias" chum on Twitter instead.
In the aggregate, the over/under predictions are within the range of measurement error. The unskewing probably just magnifies that because it is error variance. Even though you have some idea of what accounts for this variance, it's based on intentions that have limited predictive validity. Being a "likely voter" in July is probably a pretty imperfect predictor of being an actual voter in November. Aside from face valid anecdotal explanations, the simple fact is that intentions are lousy predictors of behavior the more distal they are (lots of social psych research on this). Also, the act of voting sounds simple, but can become a complex social behavior depending on numerous factors--people moving, the weather, the intensity of the desire to vote, how easy it is to vote, etc. Most of those sources of variance are dynamic and unstable.
The one proxy for actual voting might be past voting behavior. There will still be a lot of variance, but it's probably a stable behavior and vivid enough to limit recall bias. People who consistently vote or always vote in mid-terms might provide better prediction or at least allow you to introduce alternative data points for comparison that might prove useful over time, esp. in the aggregate.
Also wondering how advertising moves the polls as we get closer to election day. What someone thinks about a candidate in August may be shaped by the opposing candidate pointing out unpopular positions or unfavorable incidents in fall ads, and most people have only a passing knowledge of most candidate's positions. As a resident in a swing state here in WI, I can't overstate the amount of advertising we're exposed to, the vast majority of which is negative.
It's way easier to predict the past! (former history professor speaking here)
My standing reminder that every assessment of polling "bias" or "misses" -- at least any coming from the adult table -- is predicated strictly and faithfully on the assumption that vote counts are honest and accurate and therefore reflect the true disposition of the electorate, from which baseline we can measure the success or failure of the polls.
In the age of computerized voting, counting, and reporting, there is some unsettling evidence suggesting that this is a dubious assumption. Some of the work collecting and analyzing that evidence is my own: https://www.amazon.com/CODE-RED-Computerized-Elections-Democracy/dp/B087H83JCR/ref and https://codered2014.com/wp-content/uploads/2023/11/TheRealSteal-IntroAnalysisCombinedUpdated-js8_WWW-2.pdf. More recently, Election Truth Alliance has done serious work on Election 2024.
If we consider the thumbs on the electoral scales, some (gerrymandering, disinformation, e.g.) should be fairly accurately reflected in polling, but others (voter purges and other forms of suppression, electronic vote count manipulation, e.g.) will not be.
When looking at a consistent, unidirectional pattern of putative polling/sampling "bias" -- one that pollsters' best efforts at correction over multiple election cycles can't seem to eliminate -- perhaps it is not absurd to also consider the possible "bias" in the data the polling is attempting to model. I.e., the tabulations producing the official results.
It's reasonable to assume that public opinion of Trump, and his numbers are in the gutter, are bound to negatively affect the Republican vote and raise it for Democrats. It's a factor that cannot be ignored by citing what happened in previous elections. With tyranny raising its ugly head, we're in new territory right now.
Joseph in Fairport, NY
I wonder if the "August bias" might have any correlation with either the presumption of anti-party-in-power voting given difficulty in predicting turnout in November based on statements in August? Also, if there's any correlation between it and the historical context that the electorate tends to move left in a series of swing then backlash moves because of the structure of our party / political systems?
I am a retired lawyer, not a statistician, but it occurs to me there could be an error that is tainting your results. Let's suppose for the sake of argument that polling is accurate and selects the respondents which actually predict the outcome. Then let's add in the impact of gerrymandering. If it actually perverts the outcome of the election, wouldn't it fully explain the errors in polling. If the poll accurately predicts the outcome, but gerrymandering kept one political party from voting, they wouldn't be able to vote as predicted, would they.
Elliott’s model accounts for gerrymandering, as do the other major models out there (Silver, Split Ticket etc.). And the post is primarily about Senate races, which are statewide.
I think that's a good observation.
Joseph in Fairport, NY
Yes! More yogi berra quotes :)
As long as candidates keep campaigning, the election will be what it will be. If polls don't help candidates win or lose elections, then how do they differ from watching a telenovela? Polls might change money coming into a campaign, they might change campaign worker enthusiasm or voter turnout. Maybe the right question to ask about polling isn't how well do they predict outcome (winner) but when do the polls predict futility? A dangerous question for anyone who remembers Patriot's Superbowl comeback from 28-3. Or if you prefer the other form of football, Leicester City (2016) with preseason odds of 5,000-to-1, a team that narrowly avoided relegation the year prior won the English Premier League.
Would love to see 95% confidence limits for polls predicting winners. If polling keeps most of the races in the lean or tossup, not 95% confidence of predicting a winner, then let's just campaign and not worry about the noise.
I have not finished your article yet, my bad, but will be looking for how polls compensate for gerrymandered states where the results are not dependent on voter turnout out or party preference.
As usual, stellar analysis.
In Ruffini's poll centric way of looking at campaigns, he inherently discounts the fact that the reason for Republicans outperforming summer polling is the GOP running more effective fall campaigns - betters ads, better mail, better field and turnout - in favor of simple "bias" in summer polling.
If elections come down to persuading the small group of Independents who hate both parties and would not say they are voting Republican in an August poll, and Republicans keep winning them in November, I would celebrate that. The problem is that his president is polling in the high teens, low 20s with them on approval with 2/3rds disapproving. Much better to throw out some "poll bias" chum on Twitter instead.
Thanks for always educating us as you explore the nagging questions surrounding polling.
Patrick Ruffini made a farcical argument that America is secretly a right-wing country??? Nooooooooooo I'm so shocked!! 😱😱😱
In the aggregate, the over/under predictions are within the range of measurement error. The unskewing probably just magnifies that because it is error variance. Even though you have some idea of what accounts for this variance, it's based on intentions that have limited predictive validity. Being a "likely voter" in July is probably a pretty imperfect predictor of being an actual voter in November. Aside from face valid anecdotal explanations, the simple fact is that intentions are lousy predictors of behavior the more distal they are (lots of social psych research on this). Also, the act of voting sounds simple, but can become a complex social behavior depending on numerous factors--people moving, the weather, the intensity of the desire to vote, how easy it is to vote, etc. Most of those sources of variance are dynamic and unstable.
The one proxy for actual voting might be past voting behavior. There will still be a lot of variance, but it's probably a stable behavior and vivid enough to limit recall bias. People who consistently vote or always vote in mid-terms might provide better prediction or at least allow you to introduce alternative data points for comparison that might prove useful over time, esp. in the aggregate.