Eliezer Yudkowsky

Eliezer S. Yudkowsky is an American artificial intelligence (AI) researcher, blogger, cult leader, shitty fan fiction writer, and autodidact exponent of Bayes-based human rationality. Yudkowsky cofounded and works at the Machine Intelligence Research Institute (formerly the Singularity Institute for Artificial Intelligence), a nonprofit organization that concerns itself with the concept known as the singularity.Our Team, Machine Intelligence Research Institute Yudkowsky founded the blog LessWrong as a sister site and offshoot of Overcoming Bias. He began his blogging career with George Mason University economist Robin Hanson.
Being idealistic enough to want everyone (above a certain income bracket) to live forever, he also has an overweening interest in cryonics and is (probably) vulnerable to hemlock.
==AI "research"==
Yudkowsky believes he has identified a "big problem in AI research" in that we can't assume an AI would care about humans or ethics without our evolutionary history. Believing that artificial general intelligence (AGI) is imminent, Yudkowsky has taken it upon himself to create a Friendly AI (FAI). Such an AI won't kill us, inadvertently or otherwise.
"AI foom," short for "recursively self-improving Artificial Intelligence engendered singularity," comes from these ideas:
*An AI-based singularity will occur soon. 
*AIs and humans would want them to improve themselves.
*Human-level or better AIs could not be imprisoned or threatened with having their plugs pulled because they would talk their way out of the situation. Interestingly, Yudkowsky claims to have tested this idea by role-playing the part of supersmart AI, with a lesser mortal playing the jailer. However — the reader may note the beginning of a pattern here — the transcripts are unpublished.
*AIs will, at some point, gain virtually unlimited control over the physical world through nanotechnology.
His transhumanist philosopher and existential risk theorist friend Nick Bostrom, director of the Future of Humanity Institute at the University of Oxford until it was dissolved in 2024,Oxford shuts down institute run by Elon Musk-backed philosopher by Nick Robins-Early (19 Apr 2024 18.46 EDT) The Guardian. wrote Superintelligence: Paths, Dangers, Strategies on the above ideas in 2014.Superintelligence: Paths, Dangers, Strategies' 'by Nick Bostrom (2014). Oxford University Press. ISBN 0198739834. Bostrom's central thesis appears to be "you can't prove this won't happen, therefore you should worry about it." Bostrom credits Yudkowsky in the acknowledgments section and quotes from him throughout the book.
As is evident from the dialogue between Luke Muehlhauser and Pei Wang,Muehlhauser-Wang Dialogue by lukeprog (April 22, 2012) LessWrong Yudkowsky's (and thus LessWrong's) conception of AGI differs starkly from the mainstream scientific and practical research understanding of it.
Yudkowsky believes an AGI will come from decision theory despite his examples being computationally intractable, or Bayesian logic that does not scale in real AI.
Since these are both totally abstract mathematical functions, there is no reason an AGI based on them would share human values (not that one can base an AGI on them). By this reasoning, Yudkowsky sees the need to bolt on some sort of morality module which will somehow be immune from alteration, even though the "foom" scenario he fears so much is based on AGIs which can rewrite their own code. (Bostrom points out in his book that there are several situations in which an AI might have an incentive to alter its own final goals.) Thus Yudkowsky sees the need to "solve" ethics in some form that can be computerized — although ethics remains an open problem in philosophy after thousands of years. However, Plato, Aristotle and Kant just weren't as smart as Yudkowsky believes himself to be.
In addition, some of Yudkowsky's critics identify more urgent threats from AI than unfriendliness: deliberate weaponization of AIs, coding errors, humans themselves, cyborg ethics, etc.Q&A with experts on risks from AI #2 by XiXiDu (January 9, 2012) LessWrong
===Yudkowsky/Hanson debates===
Economist Robin Hanson, Yudkowsky's then co-blogger, has debated Yudkowsky on "AI foom" on several occasions. Hanson, who had previously been an AI researcher for several years, wrote a series of blog posts with Yudkowsky in 2008 and debated him in person in 2011. These debates were later collected and published as an ebook.Debate Is Now Book by Robin Hanson (September 4, 2013) Overcoming Bias In 2014, after having read Bostrom's book Superintelligence, Hanson was sticking to his guns: he wrote another blog post entitled "I Still Don't Get Foom". He argued:
Immediately after this debate, Yudkowsky left Overcoming Bias (now Hanson's blog) and moved the Sequences to LessWrong. They still weren't seeing eye to eye in 2016 after the famous match between Google's Alpha Go software and Lee Sedol.Facebook post by Eliezer Yudkowsky (March 11, 2016) Facebook Hanson said he thought general-purpose AI was like nation-states, requiring a lot of workers and idea-sharing. Limited special-purpose AI would be like product teams in businesses. While he gave a criterion to test his theory, Yudkowsky said he thought the world would already have "ended" before there was a chance to verify it. He claimed different AI labs were doing AI training, and there was almost no exchange of pre-trained modules or specialist components. (However, general AI components get widely reused between companies.)Why TensorFlow will change the game for AI somatic blog
Yudkowsky had previously announced his belief that if, as we now know, Alpha Go beat the Go grandmaster, that would mean the singularity was getting closer. However, the previous occasions when computers beat top-ranked humans at a board game (checkers and chess) still left computers well below the intelligence of a human infant.
===Singularity Institute / MIRI===
In 1998, Eliezer Yudkowsky finished the first version of 'Coding a Transhuman AI', alternatively known as 'Notes on the Design of Self-Enhancing Intelligence'.Coding a Transhuman AI by Eliezer Yudkowsky (Published 1998; Archived February 1, 2001) sysopmind.com After that, he spent a few months writing design notes for a new programming language named Flare. Thereafter, he wrote 'Plan to Singularity',The Plan to Singularity by Eliezer Yudkowsky (1999; Archived February 9, 2001) sysopmind.com presenting the idea of a Singularity Institute. In May 2000, at the meeting of Foresight Institute "Engines of Creation 2000 Confronting Singularity": Spring 2000 Senior Associate Gathering (Published 2000; Archived April 28, 2001) Foresight Institute the idea of establishing the institute got support from two others. Now, the Singularity Institute had a website.singinst.org (Archived September 14, 2000) Singularity Institute, now Machine Intelligence Research Institute Then he wrote a revised version of 'Coding a Transhuman AI'.Coding a Transhuman AI 2.2.0 by Eliezer Yudkowsky (Published 2000; Archived February 2, 2001) 
On July 23, 2001, SI started the Flare project under the leadership of one Dimitry Myshkin but canceled the project in 2003.Timeline of Machine Intelligence Research Institute, Timelines Wiki Since the very start, Yudkowsky has made creating 'Friendly AI' crucial to himself. In 2001, he wrote a book-length work 'Creating a Friendly AI'.Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures by Eliezer Yudkowsky (2001) Machine Intelligence Research Institute, formerly Singularity InstituteThis table shows the quantity of donation received in the years 2000-2009.
In 2004, Eliezer Yudkowsky got a "major insight" about Coherent Extrapolated Volition (CEV). A person might not always want what is best for themself, but Yudkowsky wants his AI to do good for people. He believes that to find what is best for a person, AI would scan the person's brain and do "something complicated". To ensure the AI is friendly to all humans, it would do it to everyone. The sheer computing power needed is incomprehensibly large, and we have no idea what this would look like if implemented.Coherent Extrapolated Volition by Eliezer Yudkowsky (2004) Machine Intelligence Research Institute, formerly Singularity Institute 
Until 2008 or 2009, Singularity Institute was practically a one-man show. All its publications bore the Yudkowksy's name, and there was no other full-time employee of SI. However, change came in 2008 with the emergence of LessWrong. Soon, new publications by new authors like Carl Shulman, N Tarleton, and after some time, Luke Muelhauser began to arrive. And this became the norm, with newly inducted researchers writing most papers.
The new focus of research at SI became decision theory. In 2010, Eliezer Yudkowsky published Timeless Decision Theory.Timeless Decision Theory by Eliezer Yudkowsky (2010) Machine Intelligence Research Institute, formerly Singularity Institute
Yudkowsky's collaboration with Bostrom's Future of Humanity Institute became visible by this time. Multiple joint articles by SI researchers and FHI researchers such as A. Sandberg,Implications of a Software-Limited Singularity by Carl Shulman and Anders Sandberg, edited by Klaus Mainzer (2010) In ECAP10: VIII European Conference on Computing and Philosophy Stuart Armstrong, and the like appeared. A joint article by Yudkowsky and Bostrom appeared in 2011.The Ethics of Artificial Intelligence by Nick Bostrom and Eliezer Yudkowsky (2011) Bostrom's 2014 book Superintelligence refers to Yudkowsky 15 times. There were also nearly five pages on Coherent Extrapolated Volition.Superintelligence by Nick Bostrum (2014) Oxford University Press, 1st ed. ISBN 978-0-19-967811-2 In 2009, Jeffery Epstein donated $50,000 to SIAI.Here's exactly how Jeffrey Epstein spent $30 million by Aaron Brezel (July 24, 2019) Miami Herald
==Other projects==
===Fan fiction===
For a slightly more humorous but insightful look into his mind, Yudkowsky has written Harry Potter fanfiction to teach rationality to a larger audience. In the story, a rationalist Harry tries to get his head around magicHarry Potter and the Methods of Rationality by Eliezer Yudkowsky (Published February 28, 2010; Last updated March 14, 2015) FanFiction.net while lecturing science to his elders and audience.
Harry Potter and the Methods of Rationality (HPMoR) is widely acclaimed by the standards of fanfiction (such as they are), with many readers hailing it as one of the best of its kind.Here's 10 Harry Potter Fanfics That You Should Read Right Now by Rhys McKay Yudkowsky and his fans tried to spam it for the 2016 Hugo Awards for science fiction and fantasy,Hugo Awards - HP:MoR by Eneasz (January 21, 2011) LessWrongHugo Awards - HP:MoR (part 2) by Eneasz (January 31, 2012) LessWrongAuthor's Note 119: Shameless Blegging by Eliezer Yudkowsky (March 10, 2015) hpmor.comTumblr post by asocratesgonemad (Published Jan 29, 2016 8:14 pm ; Archived January, 30 2016 16:04:19 UTC) Tumblr though it didn't make the shortlist. The podcast version was a Parsec Awards finalist in 20122012 Parsec Awards Winners & Finalists and 2015.2015 Parsec Awards Winners & Finalists
Some readers felt disillusioned by the sermonising.Why are people "put off by rationality"? by adamzerner (August 5, 2014) LessWrong As Yudkowsky himself notes in a Facebook posting:Post by Eliezer Yudkowsky (March 18, 2014) Facebook
LessWrong may have developed a personality cult around Yudkowsky, with a list of Eliezer Yudkowsky Facts,Eliezer Yudkowsky Facts by steven0461 (March 22, 2009) LessWrong but the fan fiction of him was apparently a play, produced without his knowledge.Message by Eliezer Yudkowsky (Published July 4, 2007 ; Archived July 17, 2013) SL4Yudkowski Returns! reviewed by Robert Weinstein in: Pretentious Festival Reviews - Page 2 (Archived February 11, 2009) nytheatre.com 
===Arbital===
After tiring of LessWrong and finishing his Harry Potter fanfic, Yudkowsky's next project was Arbital, an exciting new competitor in the educational market with a different approach to learning than that of Wikipedia.Arbital: Solving online explanations by Eliezer Yudkowsky (Mar 3, 2016) He first proposed it in March 2015:
With $300k in funding, developers were hired, with Yudkowsky as an overseer. There were several changes in direction. It went live in early 2016 as the LessWrong Sequences in a different bottle. By late 2016, they identified as "the place for crowdsourced, intuitive math explanations." It had a chance of beating the notoriously opaque math articles on Wikipedia, except they couldn't find enough knowledgeable people to write them. As of early 2017, the front page was a Slashdot-style aggregator for people in the MIRI subculture. By the end of March, they admitted defeatWhat's up with Arbital? by Alexei (29th Mar 2017) LessWrong. and turned the front page into a coming-soon for a blogging platform. A project developer lamented Yudkowsky's leadership style.
==Achievements==
With no training in his field of interest, Yudkowsky has no accomplishments to his credit beyond getting Peter Thiel to give him money. (And moving on to Ethereum cryptocurrency programmers.) 
Even his fans admit, "A recurring theme here seems to be 'grandiose plans, left unfinished'."Discussion: Yudkowsky's actual accomplishments besides divulgation (Raw Power, LessWrong, 25 June 2011) He claims to be a skilled computer programmer, but has no code available other than Flare, an unfinished computer language for AI programming with XML-based syntax.Flare Programming Language, sourceforge.net. It was supposed to be the programming language in which SIAI would code Seed AI but got canceled in 2003. He hired professional programmers to implement his Arbital project (see above), which eventually failed.
Yudkowsky is almost entirely unpublished outside his foundation and blogsThere is a conference proceeding: Eliezer Yudkowsky (2011). Complex Value Systems are Required to Realize Valuable Futures. In Proceedings of AGI 2011. and never attended high school or college,5 Minutes With a Visionary: Eliezer Yudkowsky by Gregory Saperstein (3:38 PM ET Thu, 9 Aug 2012) CNBC. much less did any AI research. No samples of his AI coding are public. In 2017 and 2018 he self-published some pop science books with titles like Inadequate Equilibria and endorsements from fellow travelers and Eliezer Yudkowsky, Inadequate Equilibria: Where and How Civilizations Get Stuck.
Yudkowsky's papers are generally self-published and had a total of two cites on JSTOR-archived journals (neither to do with AI) as of 2015. One of these came from his friend Nick Bostrom at the closely-associated Future of Humanity Institute.How Law Changes the Environmental Mind: An Experimental Study of the Effect of Legal Norms on Moral Perceptions and Civic Enforcement by Yuval Feldman & Oren Perez (2009) Journal of Law and Society 36(4):501-535.Pascal's mugging by Nick Bostrom (2009) Analysis 69(3):443-445.
Yudkowsky's observable results in the real world are a fanfiction wildly acclaimed for standing out by the genre's standards, a pastiche erotic light novel,A Girl Corrupted by the Internet is the Summoned Hero?! a large pile of blog posts and self-published nonfiction, and a surprisingly well-funded research organization with fewer papers in a decade and a half than a single graduate student in the course of a physics Ph.D. The latter would be peer reviewed.
==Genius or crank?==
Yudkowsky refers to himself as a genius six times in his "autobiography" that he wrote in 2000 when he was 20.Eliezer by Eliezer S. Yudkowsky (2000) Sysop Mind (archived from February 5, 2001). To his credit, he has since disavowed it, considering his younger self an idiot. He also claims to have "randomly won the writing talent lottery".Checking in with r/badphil's old friend Eliezer Yudkowsky by barkevious2 (c. 2017) Reddit. He thinks most Ph.D.s are pretty ordinary compared to the people he hobnobs with.
However, he admits to possibly being less smart than The Level above Mine by Eliezer Yudkowsky (26th Sep 2008) LessWrong. As a homeschooled individual with no college degree, Yudkowsky may not be in an ideal position to estimate his own intelligence. For obvious reasons, many of his followers think he is a genius.Eliezer Yudkowsky Facts: (Photoshopped version of this photo.) comment by ata (2009) Lesswrong.Slide from Yudkowsky speech by Mike Kuniavsky (May 14, 2006) Flickr.Yudkowsky as Christ Risen ImageShack (archived from December 28, 2011).The End (of Sequences) Anonymous 2009 comment with link to Photoshopped image of Yudkowsky as Christ Risen. Similarly, some of his followers are derisive of renowned scientists. Just look for comments about "not smart outside the lab" and "for a celebrity scientist."Neil deGrasse Tyson on Cryonics by bekkerd (9th May 2012) LessWrong. Yudkowsky believes a doctorate in AI is a net negative when it comes to Seed AI.So You Want To Be A Seed AI Programmer by Eliezer Yudkowsky (c. 2009) Accelerating Future (archived from January 16, 2010). "I am tempted to say that a doctorate in AI would be negatively useful, but I am not one to hold someone's reckless youth against them - just because you acquired a doctorate in AI doesn't mean you should be permanently disqualified." While Yudkowsky doesn't attack Einstein, he does indeed think the scientific method cannot handle things like the Many Worlds Interpretation or his view on Bayes' theorem.The Dilemma: Science or Bayes? by Eliezer Yudkowsky (13th May 2008) LessWrong. LessWrong does indeed have its own unique jargon.LessWrong Jargon LessWrong.
Just so that the reader doesn't get the mistaken impression that Yudkowsky boasts about his intellect incessantly, here he is, boasting about how nice and good he is:Re: Ethical basics email from Eliezer S. Yudkowsky (Jan 23 2002 - 21:29:18 MST) Accelerating Future (archived from 21 Aug 2013 07:23:00 UTC).
Disagreement with Yudkowsky's ideas is often attributed to "undiscriminating skepticism". If you don't believe cryonics works, it's because you have watched Penn & Teller: Bullshit!.Undiscriminating Skepticism by Eliezer Yudkowsky (14th Mar 2010) LessWrong. It's just not a possibility that you don't believe it works because it has failed tests and is made improbable by the facts.Is Cryonics Feasible? by Stephen Barrett (September 2, 2005) Quackwatch (archived from June 17, 2017).
==Other controversial positions==	
Despite being viewed as the smartest two-legged being to ever walk this planet, Yudkowsky and much of the LessWrong community endorse several positions subject to bitter debate and controversy in their respective fields. For example, he idealistically believes in transhumanism and cryonics with hope and without willpower and preaches the "Many Worlds Interpretation" (MWI) of quantum physics as a "slam dunk" despite the lack of scientific consensus among quantum physicists and the belief by some that it is unfalsifiable.The Correct Contrarian Cluster by Eliezer Yudkowsky (21st Dec 2009) LessWrong. "I can think of only three slam-dunks off the top of my head: Atheism: Yes. Many-worlds: Yes. P-zombies: No."Scientific method: Defend the integrity of physics by George Ellis & Joe Silk (2014) Nature 516:321–323. doi:10.1038/516321a. Yudkowsky says a majority of theoretical physicists might not share his view, adding that attempted opinion polls conflict on this point.And the Winner is… Many-Worlds! by Eliezer Yudkowsky (11th Jun 2008) LessWrong.
Concerning Bayes' theorem, Yudkowsky believes it can contradict and supersede the scientific methodEliezer Yudkowsky on Bayes and Science by Massimo Pigliucci (September 24, 2010) Rationally Speaking. and wants to apply Bayesian probability indiscriminately.
 
As a partial list, Yudkowsky also claims non-computable results like Kolmogorov complexity are a reasonable basis for an entire epistemology, evolutionary psychology is well-established science, and philosophy is a subject so easy he can "re-invent it on the fly".Eliezer Yudkowsky: Lesswrong Rationality and Mainstream Philosophy
Yudkowsky is a staunch total utilitarian, leading him to questionable conclusions through questionable thought experiments. To him, too many people experiencing a speck of dust in their eyes for a moment could be worse than torturing a man for 50 years.Less Wrong: Torture vs Dust Specks More specifically, Yudkowsky defends total utilitarian in this thought experiment.
Also, while his precise position on it is vague, he wrote a short sci-fi story where rape was briefly mentioned as legal.Interlude with the Confessor (4/8) by Eliezer Yudkowsky (2nd Feb 2009) LessWrong. How the character remarking on it didn't seem to be referring to consensual sex like we do today didn't prevent a massive reaction in the comments section. He responded, "The fact that it's taken over the comments is not as good as I hoped, but neither was the reaction as bad as I feared." He described the science fiction world he had in mind as a "Weirdtopia" rather than a dystopia.
==External links==
*Yudkowsky's outdated autobiography
*Making HAL Your Pal by Declan McCullagh, Wired, April 19, 2001.
*Interview with Yudkowsky by John Horgan, Scientific American, March 1, 2016
*yudkowsky.net
*Arbital
*Machine Intelligence Research Institute (MIRI) web page
==Notes==
==References==
Category:Authors
Category:Bloggers
Category:Consequentialism
Category:Cult leaders
Category:Futurism
Category:Internet kooks
Category:LessWrong
Category:Living people
Category:Philosophers
Category:Pseudoscience promoters
Category:Rationalism
Category:Thielsphere
Category:Transhumanism