Finally had time to caught up on this. Great piece!
Something that I have been wondering about house effects is that in models like yours, they are essentially based on the average of all polls, right? But what if this average happened to be more left- or right-leaning for whatever reason? Wouldn't this house effect be impacted by that?
Good piece. Helped me understand polling and aggregation a little better. As far as RCP goes they seem borderline deceptive. I was looking at their site today and they had the Yahoo News / YouGov poll at 45% approval for Trump but if you drill down to the actual data from YouGov they say it was 40%. I am sure I am missing some convoluted method for getting this from the data but it definitely skews their average favorably for Trump
Very interesting reading, so much more as I’ve noticed this difference myself and didn’t know how to interpret it. however I can still see a caveat before throwing right wing polls out: we’ve seen Trump clearly overperforming the polls in three consecutive PE cycles. Those polls were conducted by the same firms that couduct approval polls, and right wing assotiateds seem to be better. A coincidence? Very unlikely. Is there a convincing explanation for why these seemingly more biased pollsters actually did better? Could it be due to the fact that presidential approval is never tested by actual voting?
Trump isn’t always overperforming the polls (in 2024 primaries he underperformed), but he did overperform in all three GE cycles. Looking only at the polls conducted by right wing pollsters (or paid by the campaign or other right wing circles) this overperformance is not that clear. In all three GE cycles, these polls saw him a couple of points or so so further than other polls, so these “right wing” polls appear to be better. Were they really better or were just broken clocks right, I have no idea. poll reflected presidential preferences are ultimately proven at the ballot boxes, while poll measured presidential approval are not, so there’s no definitive answer to that and that’s why I appreciated this article so much more.
You’re better informed about polling than I am. As a result, I’m confused about what your response means. You referred to PE cycles in your July 3 comment and to GE cycles in your reply to me. I don’t know what either of those things might be. “Presidential Election cycles” and “General Election cycles?”
You might have a point about presidential preference polling being confirmed at the ballot box while presidential preference is not. I’ll reread Morris’s article to see if that clears up my confusion. Thanks for this discussion.
another super helpful article. The only thing that kind of confuses me: if the pollsters in question are clearly putting their thumb on the scale, why put them in the average? Are their data still reflective of some reality? Do the controls insulate the worst effects?
Hey Jack. That's a good question. One reason we might want the data is that we can infer trends even in biased polls — so long as the data is similarly biased across time. Another is that I don't want to be the polls police, deciding where the red line is that determines when a pollster is "too partisan" to trust. And while we are suspicious of these polls being so pro-Trump, we do not have proof that the data are intentionally rigged or otherwise manipulated. We have a set of standards for which polls we aggregate, and they aren't in violation of those.
I'm a new subscriber and very much like your data-based approach to the news despite my having little knowledge of statistics. I appreciate your use of data to discover what's true and factual in the news.
I think Jack's question is a good one, too. While I understand your reasons for including Republican pollsters, another reason to think it's unwise is that manipulating their own data would be small potatoes compared to the other things they are doing to abuse and subjugate the American people. The other things amount to another insurrection. They intend to replace our democracy with a far-Right autocracy. They've shown themselves ready to do any cruel, duplicitous thing to achieve that goal, most certainly including releasing false polling data.
Finally had time to caught up on this. Great piece!
Something that I have been wondering about house effects is that in models like yours, they are essentially based on the average of all polls, right? But what if this average happened to be more left- or right-leaning for whatever reason? Wouldn't this house effect be impacted by that?
Good piece. Helped me understand polling and aggregation a little better. As far as RCP goes they seem borderline deceptive. I was looking at their site today and they had the Yahoo News / YouGov poll at 45% approval for Trump but if you drill down to the actual data from YouGov they say it was 40%. I am sure I am missing some convoluted method for getting this from the data but it definitely skews their average favorably for Trump
Very interesting reading, so much more as I’ve noticed this difference myself and didn’t know how to interpret it. however I can still see a caveat before throwing right wing polls out: we’ve seen Trump clearly overperforming the polls in three consecutive PE cycles. Those polls were conducted by the same firms that couduct approval polls, and right wing assotiateds seem to be better. A coincidence? Very unlikely. Is there a convincing explanation for why these seemingly more biased pollsters actually did better? Could it be due to the fact that presidential approval is never tested by actual voting?
Your comment is interesting, but I don’t agree that Trump is over performing or that right wing polls are better. Can you say why you think so?
Trump isn’t always overperforming the polls (in 2024 primaries he underperformed), but he did overperform in all three GE cycles. Looking only at the polls conducted by right wing pollsters (or paid by the campaign or other right wing circles) this overperformance is not that clear. In all three GE cycles, these polls saw him a couple of points or so so further than other polls, so these “right wing” polls appear to be better. Were they really better or were just broken clocks right, I have no idea. poll reflected presidential preferences are ultimately proven at the ballot boxes, while poll measured presidential approval are not, so there’s no definitive answer to that and that’s why I appreciated this article so much more.
You’re better informed about polling than I am. As a result, I’m confused about what your response means. You referred to PE cycles in your July 3 comment and to GE cycles in your reply to me. I don’t know what either of those things might be. “Presidential Election cycles” and “General Election cycles?”
You might have a point about presidential preference polling being confirmed at the ballot box while presidential preference is not. I’ll reread Morris’s article to see if that clears up my confusion. Thanks for this discussion.
Another great piece. I learn something new in most of your newsletters!
The splines!
another super helpful article. The only thing that kind of confuses me: if the pollsters in question are clearly putting their thumb on the scale, why put them in the average? Are their data still reflective of some reality? Do the controls insulate the worst effects?
Hey Jack. That's a good question. One reason we might want the data is that we can infer trends even in biased polls — so long as the data is similarly biased across time. Another is that I don't want to be the polls police, deciding where the red line is that determines when a pollster is "too partisan" to trust. And while we are suspicious of these polls being so pro-Trump, we do not have proof that the data are intentionally rigged or otherwise manipulated. We have a set of standards for which polls we aggregate, and they aren't in violation of those.
I'm a new subscriber and very much like your data-based approach to the news despite my having little knowledge of statistics. I appreciate your use of data to discover what's true and factual in the news.
I think Jack's question is a good one, too. While I understand your reasons for including Republican pollsters, another reason to think it's unwise is that manipulating their own data would be small potatoes compared to the other things they are doing to abuse and subjugate the American people. The other things amount to another insurrection. They intend to replace our democracy with a far-Right autocracy. They've shown themselves ready to do any cruel, duplicitous thing to achieve that goal, most certainly including releasing false polling data.