Good and very important post. It's also worth noting that even the 4% estimate is almost certainly a sizable overestimate because of misclassification bias. The short version of this argument is that surveys are bad at estimating the size of extremely small groups because even a small measurement error (e.g. clicking yes when you meant to click no, misreading the question as being against violent felonies instead of in favor of violent felonies), multiplied by the vast majority of respondents who are not in the population, will still result in greatly overstating the percentage of people in the very small group. Removing obviously inattentive respondents probably reduces this bias, but does not eliminate it.
I personally find this figure hard to believe. I live in a deep blue part of a deep blue state and work in a deep blue occupation and have mostly deep blue friends and associates, and I can't think of anyone I know who would endorse murder. And yet, I can't see anything obviously methodologically problematic about the survey.
This is exactly the kind of thoughtful analysis missing from most "news." Who wants to read a story heaadlined "Less Than 5% of Country Supports Political Violence" vs "Over 30% Say Political Violence Can Be Justified"? How many news subscriptions or TV viewers does that get you? Sheesh!
Good and very important post. It's also worth noting that even the 4% estimate is almost certainly a sizable overestimate because of misclassification bias. The short version of this argument is that surveys are bad at estimating the size of extremely small groups because even a small measurement error (e.g. clicking yes when you meant to click no, misreading the question as being against violent felonies instead of in favor of violent felonies), multiplied by the vast majority of respondents who are not in the population, will still result in greatly overstating the percentage of people in the very small group. Removing obviously inattentive respondents probably reduces this bias, but does not eliminate it.
An example of this error being used to erroneously claim that the CCES shows non-citizen voting: https://cces.gov.harvard.edu/news/perils-cherry-picking-low-frequency-events-large-sample-surveys#:~:text=Hence%2C%20a%200.1%20percent%20rate,no%20non%2Dcitizens%20actually%20voted.
If you prefer an example from the opposite political valence, here is a paper overestimating the percentage of Americans who have witnessed a mass shooting: https://statmodeling.stat.columbia.edu/2025/09/22/its-jama-time-junk-science-presented-as-public-health-research/
thanks for links Dan!
Good post. It seems worth pointing out, though, that Bright Line found more recently that 12% of Democrats approved of Kirk's killing: https://brightlinewatch.org/violence-redistricting-and-democratic-norms-in-trumps-america/
I personally find this figure hard to believe. I live in a deep blue part of a deep blue state and work in a deep blue occupation and have mostly deep blue friends and associates, and I can't think of anyone I know who would endorse murder. And yet, I can't see anything obviously methodologically problematic about the survey.
This is exactly the kind of thoughtful analysis missing from most "news." Who wants to read a story heaadlined "Less Than 5% of Country Supports Political Violence" vs "Over 30% Say Political Violence Can Be Justified"? How many news subscriptions or TV viewers does that get you? Sheesh!