I think this misses how the contrarian dynamic works. It isn't that controversial topics draw contrarians, it's that all topics do. When people see something they dislike or feel they've spotted a flaw, they rush to point it out. Threads fill up initially with negative responses, not because the community is unusually negative or disproportionately disagrees with article X, but because contrarian responses are the fastest ones to form in the brain. They don't require much processing time or writing time. They're reflexive, not reflective. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

After that first wave of comments, we frequently see a second wave of objections that react in a reflexive way to the first wave. Although they take the opposing position—defending the article and criticizing the comments—it's the same contrarian dynamic, the same mechanism, driving them. Usually the second wave gets upvoted the most, leading to the paradox of the top comment in a thread expressing how bad the thread is, or the most popular comment expressing how wrong the populace is.

There's a place for such comments, and it wouldn't work to exclude them anyhow. But their rapidity means that they add less information. That's not because they're shorter (though it does take longer to add more information). The problem is how predictable they are. They're fast because they're largely precomputed—they're really responses to past things, stored up and ready to fire when a similar-enough signal arises. Comments like that make for repeats of past discussions, which are less interesting. They don't gratify intellectual curiosity. https://news.ycombinator.com/newsguidelines.html

Comments that add new information, rather than repeating what has been thought or felt or said before, tend to be reflective rather than reflexive. This happens when a topic sparks a new thought or feeling, or touches an unusual experience one has had. These comments add new information and encourage discussions that aren't just repetitions of old ones. But they require processing time: they have to be computed from scratch, not just fetched from cache. It isn't just that they take longer to write, but that it takes longer to form new pathways. It also consumes more energy, which means that reflective comments tend to come out quieter and more delicate than reflexive ones do. And they take more time and energy to read, too, so replies are more likely to be reflective in their own right.

The task of HN can be summed up as: how can we nurture reflective responses and give them priority over reflexive ones, given that the latter are both faster and stronger? How can we get into a reflective-reflective cycle rather than a reflexive-reflexive one?

I agree with you about race relations being a force multiplier on this (and also [1]). But I'm insisting on the more general issue of contrarian dynamics because it shows why you can't conclude that negative comments, which spring up like mushrooms when a thread is fresh, represent the community. It's a non sequitur, for example, to imply/conclude as you did in https://news.ycombinator.com/item?id=21474980 that HN's community must be racist because the comments found fault with the article. That phenomenon is general: the comments always find fault with the article [2]. The fact that the community upvoted your comment to the top (where it was sitting before I moderated it) suggests that much of it is aligned with your view. This interplay between votes and comments is typical.

Most of what we see on HN is so determined by the basic structure of the internet and the initial conditions of this forum that I appreciate McLuhan's "the medium is the message" more every day. It is far more deeply true than I ever realized.

[1] You're right that the context of American society and history makes the topic of race especially charged. But it's even worse than that here. HN's audience is at least half outside the U.S., leading to additional externalities and conflicts that look like something other than what they are, and so are particularly difficult to understand. This overlaps with the American context in inflammatory ways that make the threads both worse and harder to recover from. Basically everybody is fighting from their own, often very different, context and history without even realizing that they're doing it.

[2] That phenomenon is also far stronger than it appears to be, because as moderators we're constantly doing things to counteract it. If we didn't mark shallow dismissals and reflexive negations as off topic, which lowers their rank on the page, they would probably dominate every thread.