# Free Speech and Triskaidekaphobic Calculators: A Reply to Hubinger on the Relevance of Public Online Discussion to Existential Risk

In response to [Wei Dai's claim that](https://www.lesswrong.com/posts/bFv8soRx6HB94p5Pg/against-premature-abstraction-of-political-issues#mp4uYsP9cfukFhMSd) a multi-post 2009 _Less Wrong_ discussion on [gender](https://www.lesswrong.com/posts/gsL6CLqjujPNSLL2o/sayeth-the-girl) [issues](https://www.lesswrong.com/posts/MyqGb24pM54rJhDpb/of-exclusionary-speech-and-gender-politics) and [offensive](https://www.lesswrong.com/posts/oZb6AqZKBG4gmLdAZ/outside-analysis-and-blind-spots) [speech](https://www.lesswrong.com/posts/QPqm5aj2meRmE7kR8/the-nature-of-offense) went well, [MIRI researcher Evan Hubinger writes](https://www.lesswrong.com/posts/bFv8soRx6HB94p5Pg/against-premature-abstraction-of-political-issues#zEm8daGbL6fhTzFwh)—

> Do you think having that debate online was something that needed to happen for AI safety/x-risk? Do you think it benefited AI safety at all? I'm genuinely curious. My bet would be the opposite—that it caused AI safety to be more associated with political drama that helped further taint it.

Okay, but the _reason_ you think AI safety/x-risk is important is _because_ twenty years ago, people like Eliezer Yudkowsky and Nick Bostrom were trying to do _systematically correct reasoning_ about the future, noticed that the alignment problem looked really important, and _followed that line of reasoning where it took them_—even though it probably looked "tainted" to the serious academics of the time. (The robot apocalypse is nigh? Pftt, sounds like science fiction.)

The [_cognitive algorithm_](https://www.lesswrong.com/posts/HcCpvYLoSFP4iAqSz/rationality-appreciating-cognitive-algorithms) of "Assume my current agenda is the most important thing, and then execute whatever political strategies are required to protect its social status, funding, power, un-taintedness, _&c._" wouldn't have led us to _noticing_ the alignment problem, and I would be pretty surprised if it were sufficient to solve it (although that would be very convenient).

An analogy: it's actually _easier_ to build a calculator that does correct arithmetic than it is to build a "[triskaidekaphobic](https://en.wikipedia.org/wiki/Triskaidekaphobia) calculator" that does "correct arithmetic, _except_ that it never displays the result 13", because the simplest implementation of the latter is just a calculator _plus_ an extra conditional that puts something else on the screen when the real answer would have been 13.

If you don't actually understand how arithmetic works, but you feel intense social pressure to produce a machine that never displays the number 13, I don't think you actually succeed at building a triskaidekaphobic calculator: you're trying to solve a problem under constraints that make it impossible to solve a _strictly_ easier problem.

Similarly, I conjecture that it's actually easier to build a rationality/alignment research community that does systematically correct reasoning, than it is to build a Catholic rationality/alignment research community that does "systematically correct reasoning, _except_ never saying anything the Pope disagrees with." The latter is a strictly harder problem: you have to somehow _both_ get the right answer, _and_ throw out all of the steps of your reasoning that the Pope doesn't want you to say.

You're absolutely right that figuring out how politics and the psychology of offense work doesn't _directly_ help increase the power and prestige of the "AI safety" research agenda. It's just that the _caliber of thinkers_ who can solve AGI alignment should _also_ be able to solve politics and the psychology of offense, much as how a calculator that can compute `1423 + 1389` should _also_ be able to compute `6 + 7`.