On this week’s Deep Dive episode of the Strength In Numbers podcast, Elliott sits down with Peter Enns — Cornell University political scientist, political methodologist, and co-founder of the polling firm Verasight — to talk through a huge new paper he has published, "How a Seemingly Innocuous and Intuitive Methodological Choice Confused a Generation of Research on Policy Responsiveness." We get into why one of the most famous papers in political science appears to be fatally flawed, and why the conventional wisdom about democratic responsiveness is outdated as a result. Democracy, Enns argues, might be working better than you think.
This is a wonky episode, but one with profound implications for anyone who cares about what the public wants and how — or if — the government responds to that.
Here are the big takeaways:
One of the most famous results in modern political science looks like a statistical fluke. From 2004-2014, Martín Gilens and Ben Page took thousands of opinion polls and broke down preferences for public policy among rich, middle-income and poor Americans. Then they identified policies where the rich and poor disagreed, and calculated how how often either group got what it wanted. The problem is that middle and the affluent agreed about 94 percent of the time — so Gilens and Page set aside every policy where the two groups weren’t at least 10 points apart, which meant throwing out 80 to 90 percent of 2,000 policies. The way the scholars analyzed the remaining subset produced a statistical paradox called “Simpson’s Paradox,” which obscured an underlying relationship of responsiveness to the poor even when the rich wanted something different.
The paper looks at income differences among the wrong groups. One misfire of Gilens’ approach is it pits the top 20% of earners ($150k) against the bottom. But, e.g., two public school teachers in Bakersfield make enough to land in the top 10 percent of American households — a comfortable income, but not the world of private fundraisers and super PACs that influence politics. Page and Bartels once ran a special survey of the genuinely wealthy precisely because a normal 1,000-person poll turns up no multimillionaires at all, and using it Enns shows that the top 10 percent’s opinions track the middle class far more closely than they track the very rich. But they didn’t look at policy responsiveness.
The system responds to the public as a whole. When Enns measures how policy tracks overall public opinion, while accounting for the influence of organized interest groups, he finds Congress does respond to what Americans want, and at least as strongly as it responds to those groups. A separate 2017 reanalysis of the Gilens data by Branham, Soroka and Wlezien reached a similar place, finding something close to a coin flip when the middle class and the rich disagree. The unexciting answer may be the correct one: policy does loosely follow public opinion, and most groups (excluding the super-rich) want the same things.
This matters because telling people their government ignores them, when it doesn’t, discourages participation. Enns concludes by saying Americans agree with each other far more than they realize, that policy does follow them when they do agree, and that the right reaction to seeing unrepresentative slices of the electorate or society get their way is to get more involved, not less.
For more of the methods details, see this blog post from Peter about his new paper, and the study link here.
If you value this work and want to help keep it going, please consider becoming a premium subscriber to Strength In Numbers. Paid subscriptions support this podcast, the newsletter, and the time it takes to do this kind of data-driven analysis of politics and elections. Paying subscribers also gain the ability to send in questions during our live streams, so you can directly shape the conversations we’re having on the podcast.
You can also subscribe to this podcast on your favorite podcast app to listen on your own time. And if you do listen via one of those apps, please drop us a five-star rating and review if you feel we’ve earned it — it really helps people discover the show!
A reminder that paid subscribers to Strength In Numbers get to participate in our live Q&A!
You can also read the transcript of our conversation by clicking the headline of this article to take you to the web version of the podcast, then clicking the button just below the byline that looks either like a piece of paper or is labeled “Transcript,” like so:
