# Beginnings: An Introduction

This, the final book of _Rationality: From AI to Zombies_, is less a conclusion than a call to action. In keeping with _Becoming Stronger_’s function as a jumping-off point for further investigation, I’ll conclude by citing resources the reader can use to move beyond these sequences and seek out a fuller understanding of Bayesianism.

This text’s definition of normative rationality in terms of Bayesian probability theory and decision theory is standard in cognitive science. For an introduction to the heuristics and biases approach, see Baron’s _Thinking and Deciding_.\[1\] For a general introduction to the field, see the _Oxford Handbook of Thinking and Reasoning_.\[2\]

The arguments made in these pages about the _philosophy_ of rationality are more controversial. Yudkowsky argues, for example, that a rational agent should one-box in Newcomb’s Problem—a minority position among working decision theorists.\[3\] (See [Holt](http://www.slate.com/articles/arts/egghead/2002/02/thinkinginside_the_boxes.single.html) for a nontechnical description of Newcomb’s Problem.\[4\]) Gary Drescher’s _Good and Real_ independently comes to many of the same conclusions as Yudkowsky on philosophy of science and decision theory.\[5\] As such, it serves as an excellent book-length treatment of the core philosophical content of _Rationality: From AI to Zombies_.

[Talbott](https://plato.stanford.edu/archives/fall2013/entries/epistemology-bayesian/) distinguishes several views in Bayesian epistemology, including E. T. Jaynes’s position that not all possible priors are equally reasonable.\[6,7\] Like Jaynes, Yudkowsky is interested in supplementing the Bayesian optimality criterion for belief revision with an optimality criterion for priors. This aligns Yudkowsky with researchers who hope to better understand general-purpose AI via an improved theory of ideal reasoning, such as Marcus Hutter.\[8\] For a broader discussion of philosophical efforts to naturalize theories of knowledge, see [Feldman](https://plato.stanford.edu/archives/sum2012/entries/epistemology-naturalized/).\[9\]

“Bayesianism” is often contrasted with “frequentism.” Some frequentists criticize Bayesians for treating probabilities as subjective states of belief, rather than as objective frequencies of events. [Kruschke](http://www.indiana.edu/~kruschke/articles/Kruschke2010TiCS.pdf) and [Yudkowsky](http://lesswrong.com/lw/1gc/frequentist_statistics_are_frequently_subjective/) have replied that frequentism is even more “subjective” than Bayesianism, because frequentism’s probability assignments depend on the intentions of the experimenter.\[10\]

Importantly, this philosophical disagreement shouldn’t be conflated with the distinction between Bayesian and frequentist data analysis methods, which can both be useful when employed correctly. Bayesian statistical tools have become cheaper to use since the 1980s, and their informativeness, intuitiveness, and generality have come to be more widely appreciated, resulting in “Bayesian revolutions” in many sciences. However, traditional frequentist methods re- main more popular, and in some contexts they are still clearly superior to Bayesian approaches. Kruschke’s _Doing Bayesian Data Analysis_ is a fun and accessible introduction to the topic.\[11\]

In light of evidence that training in statistics—and some other fields, such as psychology—improves reasoning skills outside the classroom, statistical literacy is directly relevant to the project of overcoming bias. (Classes in formal logic and informal fallacies have not proven similarly useful.)\[12,13\]

### An Art in its Infancy

We conclude with three sequences on individual and collective self- improvement. “Yudkowsky’s Coming of Age” provides a last in-depth illustration of the dynamics of irrational belief, this time spotlighting the author’s own intellectual history. “Challenging the Difficult” asks what it takes to solve a truly difficult problem—including demands that go beyond epistemic rationality. Finally, “The Craft and the Community” discusses rationality groups and group rationality, raising the questions:

*   Can rationality be learned and taught?
    
*   If so, how much improvement is possible?
    

*   How can we be confident we’re seeing a real effect in a rationality intervention, and picking out the right cause?
    
*   What community norms would make this process of bettering ourselves easier?
    
*   Can we effectively collaborate on large-scale problems without sacrificing our freedom of thought and conduct?
    

Above all: What’s missing? What should be in the next generation of rationality primers—the ones that replace this text, improve on its style, test its prescriptions, supplement its content, and branch out in altogether new directions?

Though Yudkowsky was moved to write these essays by his own philosophical mistakes and professional difficulties in AI theory, the resultant material has proven useful to a much wider audience. The original blog posts inspired the growth of _Less Wrong_, a community of intellectuals and life hackers with shared interests in cognitive science, computer science, and philosophy. Yudkowsky and other writers on _Less Wrong_ have helped seed the effective altruism movement, a vibrant and audacious effort to identify the most high-impact humanitarian charities and causes. These writings also sparked the establishment of the Center for Applied Rationality, a nonprofit organization that attempts to translate results from the science of rationality into useable techniques for self-improvement.

I don’t know what’s next—what other unconventional projects or ideas might draw inspiration from these pages. We certainly face no shortage of global challenges, and the art of applied rationality is a new and half-formed thing. There are not many rationalists, and there are many things left undone.

But wherever you’re headed next, reader—may you serve your purpose well.

* * *

###### 1\. Jonathan Baron, _Thinking and Deciding_ (Cambridge University Press, 2007).

###### 2\. Keith J. Holyoak and Robert G. Morrison, _The Oxford Handbook of Thinking and Reasoning_ (Oxford University Press, 2013).

###### 3\. Bourget and Chalmers, “What Do Philosophers Believe?”

###### 4\. Holt, “Thinking Inside the Boxes.”

###### 5\. Gary L. Drescher, _Good and Real: Demystifying Paradoxes from Physics to Ethics_ (Cambridge, MA: MIT Press, 2006).

###### 6\. William Talbott, “Bayesian Epistemology,” in _The Stanford Encyclopedia of Philosophy_, Fall 2013, ed. Edward N. Zalta.

###### 7\. Jaynes, _Probability Theory_.

###### 8\. Marcus Hutter, _Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probability_ (Berlin: Springer, 2005), doi:[10.1007/b138233](http://dx.doi.org/10.1007/b138233).

###### 9\. Richard Feldman, “Naturalized Epistemology,” in _The Stanford Encyclopedia of Philosophy_, Summer 2012, ed. Edward N. Zalta.

###### 10\. John K. Kruschke, “What to Believe: Bayesian Methods for Data Analysis,” _Trends in Cognitive Sciences_ 14, no. 7 (2010): 293–300.

###### 11\. John K. Kruschke, _Doing Bayesian Data Analysis, Second Edition: A Tutorial with R, JAGS, and Stan_ (Academic Press, 2014).

###### 12\. Geoffrey T. Fong, David H. Krantz, and Richard E. Nisbett, “The Effects of Statistical Training on Thinking about Everyday Problems,” _Cognitive Psychology_ 18, no. 3 (1986): 253–292, doi:[10.1016/0010- 0285(86)90001- 0](https://doi.org/10.1016/0010-0285(86)90001-0).

###### 13\. Paul J. H. Schoemaker, “The Role of Statistical Knowledge in Gambling Decisions: Moment vs. Risk Dimension Approaches,” _Organizational Behavior and Human Performance_ 24, no. 1 (1979): 1–17.