Every survey is about yourself

Like all successful modern marriages, my wife and I have a running text message thread that stretches back to the earliest phase of our relationship. It’s where I successfully courted her (I’m more charming in writing), and it’s where we have exchanged many of the seismic announcements and trivial banalities that comprise a loving lifelong relationship. It’s where where we continually touch-in and reconnect as a couple, no matter what we’re both up to throughout the day. And in recent years, it’s where we discuss, at never-ending length, everything that’s wrong with the world, and especially our peers, the libtards.

Online, she inhabits an almost entirely disjoint space from me (she’s a woman, uses different platforms, and isn’t terminally online). This is a tremendous boon to our relationship, as the things she shares comprise a rich seam of grist for our never-ending discourse. Here’s one from the other day, put on her screen by Zuck’s algorithmists.

whitehouse

A post shared by @whitehouse

The White House’s official Instagram account is crowing about the deportation of a convicted pedophile. Everybody, regardless of their political leanings, should be able to agree that’s a good thing, right?

lol, no.

There are lots more comments in this vein, but you get the idea.

Not a good look bro

This one stuck in my craw a bit, because I had just tangled with some very decent folks “on our side” for defending another White House missive on deportation, this one with an admittedly distasteful caption that really rubbed a lot of people in my scene the wrong way.

People were upset that I stuck my neck out. Some of the usual suspects said some very mean things about my character, which is expected and uninteresting. More interesting, some of Our Guys trotted out the Not a Good Look argument, which I don’t agree with but I think is worth discussing. Paraphrasing only a little: “there’s no need to be cute or cocky about this, just post a list of the criminals being deported with the crimes they committed, people would love that.” Basically: this kind of imagery and presentation scares the hoes, but the marginal voter endorses deporting criminal aliens as long as you don’t make it look mean or gross.

Well, here’s how that marginal voter responds to a list of mugshots and crimes announcing deportations.

I’m being cheeky — I don’t think these are marginal voters, I think they’re rabid partisans, just like most of the people who bother to write comments online. But I still think they’re telling us something interesting about themselves.

The steak and the avocado

If you hang out on certain parts of The Platform Formerly Known as Twitter, you’ve probably run across this meme, which purports to encapsulate all the important differences in how conservatives versus liberals view the world. It has become a shorthand in-group rejoinder to when someone on the left says something that seems to come from another moral universe, like objecting to deporting a child rapist.

The steak and the avocado are a second-generation meme that reference a study about how self-described liberals and conservatives assign moral worth in a thought experiment. The study assignment was to allocate imaginary “care points” to a series of concentric rings, where each ring expanded the moral universe of concern — so the inner circle is one’s self, then immediate family, then extended family, then friends, then community members, then fellow citizens of one’s nation, then citizens of other countries, and so on until the outer bands begin to encompass non-human animals, plants, and non-living things like rocks.

This study hit right-wing twitter like a tactical mimetic nuke, obliterating many otherwise intelligent people’s basic sense of plausibility and skepticism. Here’s a typical specimen.

The center of moral concern, the hot-spot chosen by the most people, is centered on friends and family for conservatives, and rocks and trees for liberals.

I’ve had drawn-out arguments about how to interpret these results, diving into details of the methodology and the specific wording of the questions posed to the experimental subjects. Are subjects responding to the question as posed and allocating scarce imaginary “care tokens”, or simply selecting the outermost extent of their moral concern? At the end of the day, it’s mostly moot. What it comes down to is what you think happens in the following scenario.

Jigsaw appears before terrified test subjects with two platforms poised over a pit of lava. On one platform is the subject’s grandma. On the other is a potted ficus. Subjects have thirty seconds to choose which platform gets incinerated, or they both do. If you take the study results seriously, at face value, you’re forced to conclude the teary-eyed liberal says goodbye to grandma to save the ficus. Just look at the graph, it’s objective data!

This is an absurd conclusion, yet you’ll find plenty of people in the new online right eager to bite this bullet. Similar thought experiments (mom v. random foreigner, childhood friend v. wild yak) reveal just how implausible the supposed case against the libs really is. It should go without saying, but it’s simply not the case that liberals, however they place their imaginary care tokens on the ring for a grad student, care more about geodes than their dads.

Ok fine. So what does it mean? We’ll get there in a minute.

Everything is objectively much better now that we’re in charge

Gallup has been collecting opinion polling split out by partisan affiliation for decades, and their biggest feature is the comical inflection points that occur like clockwork whenever a new party takes the White House. The graph below is about the economy, but it doesn’t matter the particular issue in question, every single one exhibits this pattern to varying degrees.

Note the exact wording of the question above: “do you think that economic conditions in the country as a whole are getting better or worse”. “Getting better”, present continuous tense, i.e. what is happening at this moment. Republicans started getting rosier about current conditions immediately following the election, democrats the opposite. Then after inauguration, a total swap of outlooks. Incredible that Trump was able to completely transform the entire economy in less than a month.

Here one is forced to conclude that present economic conditions are not what partisans are responding to in this poll. Actual economic conditions are a lagging indicator of any policy changes, and these swings coincide exactly with change in control of the executive branch. Charitably, respondents are thinking past the sale: not what things are like now, but they’ll be like going forward. But you have to square this with the clear partisan leanings in perception before the election, and the reactions of the poor lonely, exhausted independents, which are effectively noise centered around “+40 getting worse”.

What’s going on here? If respondents aren’t answering the questions, what do their answers even mean?

Every survey is equally terrible, but some are more equally terrible than others

If you haven’t read it yet, I cannot recommend strongly enough this takedown of the science of surveys by the inimitable Science Banana. Banana has much to say on this topic, most importantly interrogating whether the idea of measuring abstract concepts through the muddy and imprecise mechanism of language passes muster as an evidentiary standard. There is no such thing as a happiness ruler that you can hold up next to a test subject to objectively measure how contented they are with life. Instead, you must ask them, with slippery and subjective language, how they feel on the inside, and they must consider and respond, again with that slippery and subjective loss on the way out.

But the point I’m most interested in, for the purposes of this essay, is about how those surveyed interpret the task of answering.

A related adversarial motivation is making a point. In the normal course of conversation, in ordinary language use, one forms opinions about why the speaker is saying what she is saying, and prepares a reply based as much on this as on the words actually said. In surveys, survey takers may form an opinion about the hypothesis the instrument is investigating, and conform his answers to what he thinks is the right answer. It’s a bit subtle, but it’s easy to see in the communicative form of twitter polls. When you see a poll, and your true answer doesn’t make your point as well as another answer, do you answer truthfully or try to make a point? What do you think all the other survey respondents are doing? This is not cheating except in a vary narrow sense. This is ordinary language use—making guesses about the reasons underlying a communication, and communicating back with that information in mind. It’s the survey form that’s artificial, offered as if it can preclude this kind of communication. And even when a survey manages to hide its true hypothesis, survey takers still may be guessing other hypotheses, and responding based on factors other than their own innocent truths.

It’s clear that the partisans in the economic survey above are not answering the question, but making a point. That point is, alternately, that Trump is good and therefore good for the economy, or that Trump is terrible and terrible for the economy. The actual purported measurement of “how the economy is doing” isn’t just drowned out by this point-making, it’s subsumed by it, buried, replaced wholesale. (At least among self-identified partisans. Somebody light a candle for those brave and lonely Independents).

What’s remarkable about this is that despite the question not mentioning current White House leadership, and the objectively tenuous link between such White House occupants and macroeconomic movements, nearly every partisan respondent correctly intuited the “true” assignment of the survey and responded to the “true” question: “will Donald Trump be good for the economy?” This happens because “the economy” is widely understood to be a nebulous but vital political indicator, every bit as mysterious and all-portending as the Mandate of Heaven. Whether one believes that the President can actually affect it or not, everyone understands that he will take the blame or the credit for it.

So while every survey should be treated with an almost scornful skepticism by default, those that touch on one’s political identity should be understood to be responding to a completely different question than the one asked, not in part as a subjective coloration of responses, but in whole, a total replacement of the purported semantic content of the question.

Nor is intelligence a defense against the mind-killing effect of partisanship. In a widely reported 2013 study, participants were asked to interpret a table of numbers about the efficacy of an intervention.

In the case of a neutral topic (whether a skin cream works, top graph), greater numeracy skills were strongly associated with coming to the correct logical conclusion independent of partisan affiliation. But on a partisan topic (whether gun control increases or decreases crime, bottom graph), greater numeracy had next to no protective effect if the numbers contradicted one’s political priors (bottom two lines). In this example, liberals could not bring themselves to conclude that the (invented) numbers showed that a gun ban increased crime; likewise, conservatives just couldn’t conclude that such a ban could decrease crime. This condition of partisan contradiction effectively unmade the previous relationship between stronger numeracy and a correct quantitative interpretation. Being smarter didn’t help you reason better, not when the correct conclusion would prove you wrong on an issue you identify with.

Every survey question is about you

We come at last the to the crux of the topic: what do all these things have in common? And what, really, is wrong with the libtards? What makes them tell researchers they care more about rocks than their dads and respond to the deportation of literal child rapists with “what about Trump?”

The answer to all these questions is identity. In responding to a researcher or a pollster, or reacting to a news feed on social media that will be read by their friends and family, there is always an underlying ur-question that goes unstated: “what kind of person am I?” Only the most banal and neutral topics (skin cream) manage to slide past this filter. Every topic even tangentially related to identity — and the list keeps growing longer — swerves headfirst into it.

It’s helpful to consider the missing context in these responses, the interior monologue, the unspoken and mostly unexamined assumptions that underlie them.

On the economy:

[What kind of person am I? I’m the kind of person who values responsible government by educated experts, not political appointees chosen for their loyalty. What Trump and Musk are doing with DOGE and tariffs is reckless and it will probably destroy the economy if it’s not stopped.] The economy is definitely getting worse.

On the avocado graph:

[What kind of person am I? I’m the kind of person who cares about other people, all other people, and even more, I care about the entire planet, which humans are threatening with global warming. Everyone needs to pay more attention to these issues, which could destroy our way of life.] I value rocks and trees at least as much as my dad.

On deporting child rapists:

[What kind of person am I? I’m the kind of person who gives people the benefit of a doubt and forgives them. Obviously I don’t think child rapist illegal aliens should be allowed to stay in the country, but Trump is a racist who wants to deport every brown immigrant, so the more important consideration here is showing opposition to that racism] What about the white men who do this daily?

This behavior isn’t limited to the libtards, bless their hearts. You could easily do this exercise for conservative partisans as well — or, at least, you could if you were a conservative or a moderate. For reasons that I plan to write more about later, liberals have a much harder time understanding and explaining the values of conservatives than vice versa.

The impulse to treat any survey question that is even a little subjective as a mirror that we hold up to our values is a human universal. People are not perfectly rational policy-outcome-maximizing machines capable of autistic feeling-death as demanded (or assumed) by a shocking number of people online, especially on the right.

People aren’t rational. But they’re irrational in relatively predictable and understandable ways (unless you’re a liberal, in which case your enemies probably remain mysterious and unpredictable).

When someone tells you what they think about something, they’re telling you about themselves and their values, not about what you asked them. Keep this in mind the next time you read a survey or the comments on instagram.

Thanks for reading, and stay safe out there.