Given the recent discussion of media bias here, I wanted to bring up Alyssa Vance’s “Chinese robber fallacy”, which she describes as:

..where you use a generic problem to attack a specific person or group, even though other groups have the problem just as much (or even more so)

For example, if you don’t like Chinese people, you can find some story of a Chinese person robbing someone, and claim that means there’s a big social problem with Chinese people being robbers.

I originally didn’t find this too interesting. It sounds like the same idea as plain old stereotyping, something we think about often and are carefully warned to avoid.

But after re-reading the post, I think the argument is more complex. There are over a billion Chinese people. If even one in a thousand is a robber, you can provide one million examples of Chinese robbers to appease the doubters. Most people think of stereotyping as “Here’s one example I heard of where the out-group does something bad,” and then you correct it with “But we can’t generalize about an entire group just from one example!” It’s less obvious that you may be able to provide literally one million examples of your false stereotype and still have it be a false stereotype. If you spend twelve hours a day on the task and can describe one crime every ten seconds, you can spend four months doing nothing but providing examples of burglarous Chinese – and still have absolutely no point.

If we’re really concerned about media bias, we need to think about Chinese Robber Fallacy as one of the media’s strongest weapons. There are lots of people – 300 million in America alone. No matter what point the media wants to make, there will be hundreds of salient examples. No matter how low-probability their outcome of interest is, they will never have to stop covering it if they don’t want to.

A fantastic and important post by Scott Alexander of the great Slate Star Codex: Cardiologists and Chinese Robbers.

This is why I'm so suspicious of anecdote-based journalism, especially when it comes from an outlet with a hallowed reputation. Think back to the piece on Amazon working conditions in the NYTimes, and see how much actual data backs up some of the generalizations made in the piece. I'm not saying that the individual stories of terrible managers don't matter, because each of those in and of themselves was terrible and worth deep investigation.

Many people I know just take it for granted that it's like that throughout the company, though. Take this op-ed from Joe Nocera. He felt comfortable enough, after reading that piece, to make sweeping statements like this:

It’s an enormously adversarial place. Employees who face difficult life moments, such as dealing with a serious illness, are offered not empathy and time off but rebukes that they are not focused enough on work. A normal workweek is 80 to 85 hours, in an unrelenting pressure-cooker atmosphere.

I will bet Joe Nocera his net worth that the average workweek at Amazon is not 80 to 85 hours. I don't think any company in the world with over 170,000 employees has an average work week approaching anywhere near 80 to 85 hours. But hey, it's just a NYTimes op-ed, let's just throw a crazy fact like that out there with no sourcing whatsoever, who's going to fact-check an op-ed anyhow?

What 170,000 employees and who knows how many former employees provides a reporter is a lot of people to mine for Chinese robbers.

[Incidentally, that large a sample should also provide plenty of counter-examples, but Amazon's restrictive, and in my opinion, short-sighted social media policy prevents folks like that from speaking out. One employee couldn't take the piece lying down and wrote a rebuttal on LinkedIn, and later other former employees came out in the company's defense, including one who felt her story was used in the piece in a misleading way. It doesn't have to work just in the company's favor, other stories like this one have come and added to some of the terrible anecdotes in the original NYTimes piece. However, since the social media policy restricts current employees from speaking out, it likely mutes the largest population of people who enjoy working there.]

I don't mean to wade back into the Amazon debate with this piece, and parts of it, even if rhetorically framed with bias, struck me as reasonably accurate. It just happens to be the most prominent recent example of Chinese robber fallacy that came to mind. Anyone who's been the subject of an anecdote-based journalistic piece should be suspicious of such pieces, yet so many people in and outside of tech took the Amazon piece as gospel.

The fact is, the Chinese robber fallacy really works. It must be so satisfying, as a reporter, to come across a source willing to go on the record with a dramatic narrative, even if it isn't statistically significant. That source also has spent their life looking for narrative patterns, and soon it's Chinese robbers all the way down.

Humans are wired to respond to narratives, to draw conclusions based on insufficient data. We're all looking for narrative shortcuts to the truth. When reporters give us a few carefully chosen examples, it's game over, regardless of whether or not it's a statistically significant sample, or whether or not the sample was plagued by selection bias.

Such journalism can be moving and hugely important. It can move people's hearts, and that's often what's needed to change the world. But it's also a dangerous weapon. Recall Janet Malcolm's opening line to her classic piece “The Journalist and the Murderer”:

Every journalist who is not too stupid or full of himself to notice what is going on knows that what he does is morally indefensible.

She meant it in a different context, but it echoes here.

Journalism with lots of data and statistics aren't sexy. They may not even require as much legwork as interviewing lots of people over long period of time, and it's not the type of journalism that gets dramatized in the movies. But there's a reason that science isn't based on a few good stories.