December 2012 Newsletter

Greetings from the Executive Director

Dear friends of the Singularity Institute,
This month marks the biggest shift in our operations since the Singularity Summit was founded in 2006. Now that Singularity University has acquired the Singularity Summit (details below), and SI’s interests in rationality training are being developed by the now-separate Center for Applied Rationality, the Singularity Institute is making a major transition. For 12 years we’ve largely focused on movement-building — through the Singularity Summit, Less Wrong, and other programs. This work was needed to build up a community of support for our mission and a pool of potential researchers for our unique interdisciplinary work.
Now, the time has come to say “Mission Accomplished Well Enough to Pivot to Research.” Our community of supporters is now large enough that qualified researchers are available for us to hire, if we can afford to hire them. Having published 30+ research papers and dozens more original research articles on Less Wrong, we certainly haven’t neglected research. But in 2013 we plan to pivot so that a much larger share of the funds we raise is spent on research. If you’d like to help with that, please contribute to our ongoing fundraising drive.
Onward and upward,
Luke

Singularity Summit Conference Acquired by Singularity University

The Singularity Summit conference, founded by SI in 2006, has been acquired from SI by Singularity University. As part of the agreement, the Singularity Institute will change its name (to reduce brand confusion), but will remain as co-producers of the Singularity Summit in some succeeding years. We are pleased that we can transition the conference to an organization with a strong commitment to maintaining its quality as it grows.
Most of the funds from the Summit acquisition will be placed in a separate fund for a “Friendly AI team,” and therefore does not support our daily operations or other programs.
We wish to thank everyone who participated in making the Singularity Summit a success, especially past SI president Michael Vassar (now with Panacea Research and Summit organizer Amy Willey.

2012 Winter Matching Challenge!

We’re excited to announce our 2012 Winter Matching Challenge. Thanks to the generosity of several major donors,† every donation to the Singularity Institute made now until January 5th, 2013 will be matched dollar-for-dollar, up to a total of $115,000!
Now is your chance to double your impact while helping us raise up to $230,000 to help fund our research program.
Note that the new Center for Applied Rationality (CFAR) will be running a separate fundraiser soon.
Please read our blog post for the challenge for more details, including our accomplishments throughout the last year and our plans for the next 6 months. Please support our transition from a movement-building phase into a research phase.
† $115,000 of total matching funds has been provided by Edwin Evans, Mihály Barasz, Rob Zahra, Alexei Andreev, Jeff Bone, Michael Blume, Guy Srinivasan, and Kevin Fischer.

A Week of Friendly AI Math at the Singularity Institute

From Nov. 11th-17th, SI held a Friendly AI math workshop at our headquarters in Berkeley, California. The participants — Eliezer Yudkowsky, Marcello Herreshoff, Paul Christiano, and Mihály Barasz — tackled a particular problem related to Friendly AI. We held the workshop mostly to test hypotheses about ideal team size and the problem’s tractability, while allowing there was some small chance the team would achieve a significant result in just one week.
Happily, it seems the team did achieve a significant result, which the participants estimate would be the equivalent of 1-3 papers if published. More details are forthcoming.

SI’s Turing Prize Awarded to Bill Hibbard for “Avoiding Unintended AI Behaviors”

This year’s AGI-12 conference, held in Oxford UK, included a special track on AGI Impacts. A selection of papers from this track will be published in a special volume of the Journal of Experimental & Theoretical Artificial Intelligence in 2013.
The Singularity Institute had previously announced a $1000 prize for the best paper from AGI-12 or AGI Impacts on the question of how to develop safe architectures or goals for AGI. At the event, the prize was awarded to Bill Hibbard for his paper Avoiding Unintended AI Behaviors.
SI’s Turing Prize is awarded in honor of Alan Turing, who not only discovered some of the key ideas of machine intelligence, but also grasped its importance, writing that “…it seems probable that once [human-level machine thinking] has started, it would not take long to outstrip our feeble powers… At some stage therefore we should have to expect the machines to take control…” The prize is awarded for work that not only increases awareness of this important problem, but also makes technical progress in addressing it.
SI researcher Carl Shulman also presented at AGI Impacts. You can read the abstract of his talk, “Could we use untrustworthy human brain emulations to make trustworthy ones?”, here. If video of the talk becomes available later, we’ll link to it in a future newsletter.

New Paper: How We’re Predicting AI — or Failing To

Note: The findings in this paper are based on a dataset error. For details, see https://aiimpacts.org/error-in-armstrong-and-sotala-2012/.

The new paper by Stuart Armstrong (FHI) and Kaj Sotala (SI) has now been published (PDF) as part of the Beyond AI conference proceedings. Some of these results were previously discussed here. The original predictions data are available here. The Less Wrong thread is here. We thank Stuart and Kaj for this valuable meta-study of AI predictions.
For the study, Stuart Armstrong and Kaj Sotala examined a database of 257 AI predictions, made in a period spanning from the 1950s to the present day. This database was assembled by researchers from the Singularity Institute (Jonathan Wang and Brian Potter) systematically searching though the literature. 95 of these are considered AI timeline predictions.
The paper examines a couple folk theories of AI prediction, the “Maes-Garreau law” (that people predict AI happening near the end of their own lifetime) and the prediction that “AI is always 15-25 years into the future”. Systematic analysis of the database of AI predictions revealed support for the second theory but not the first. Many of the predictions were concentrated around 15-25 years in the future, and this trend held whether the predictions were being made in the 1950s or the 2000s. Predictions were not observed to cluster around the expected end of lifetime of the predictors, a result which contradicts the Maes-Garreau hypothesis. It was also found that the predictions of experts do not correlate in any distinct way relative to non-experts; i.e., there seems to be little evidence that experts make better AI predictions than non-experts. At Singularity Summit 2012, Stuart Armstrong summarized the results of the study in an onstage talk.
The major take-away from this study is that predicting the arrival of human-level AI is such a fuzzy endeavor that we should take any prediction with a large grain of salt. The rational approach is to widen our confidence intervals — that is, recognize that we don’t really know when human-level AI will be developed, and make plans accordingly. Just as we cannot confidently state that AI is near, we can’t confidently state that AI is far off. (This was also the conclusion of an earlier Singulairty Institute publication, “Intelligence Explosion: Evidence and Import.”)

Michael Anissimov Publishes Responses to P.Z. Myers and Kevin Kelly

SI media director Michael Anissimov has published blog posts responding to biologist P.Z. Myers on whole brain emulation and bestselling author Kevin Kelly on AI takeoff.
In a blog post, P.Z. Myers rejected the idea of whole brain emulation in general, stating “It won’t work. It can’t.” However, his response focuses on the scanning of a live brain with current technology. In response, Anissimov concedes that Myers’ criticisms make sense in the narrow context in which he makes them, but his response misunderstands that whole brain emulation refers to a wide range of possible scanning approaches, not just the reductionistic straw man of “scan in, emulation out”. So, while Myers’ critique applies to certain types of brain scanning approaches, it does not apply to whole brain emulation in general.
Anissimov’s response to Kevin Kelly is a response to a blog post from four years ago, “Thinkism”. Kelly’s blog post is notable as one of the most substantive critiques of the fast AI takeoff idea by a prominent intellectual. Kelly argues that the idea of scientific and technological research and development occurring more rapidly than “calendar time” is incredulous, because there are inherently time-limited processes, such as cellular metabolism, which limit progress on difficult research problems, namely indefinitely extending human healthspans. Anissimov argues that faster-than-human, smarter-than-human intelligence could overcome the human-characteristic rate of innovation through superior insight, breaking problems into their constituent parts, and by making experimentation massively accelerated and parallel.

Original Research on Less Wrong

SI executive director Luke Muehlhauser, with help from our research team, has compiled a list of original research produced by the web community Less Wrong. Though many of the posts on Less Wrong are summaries of previously published research, there is also a substantial amount of original expert material in philosophy, decision theory, mathematical logic, and other fields.
Examples of original research on Less Wrong include Eliezer Yudkowsky’s “Highly Advanced Epistemology 101 for Beginners” sequence, Wei Dai’s posts on his original decision theory (UDT), Vladimir Nesov’s thoughts on counterfactual mugging, and Benja Fallstein’s investigation of the problem of logical uncertainty. In all, the list compiles over 50 examples of original research on Less Wrong stretching from 2008 to the present. Of particular interest is original research that contributes towards solving open problems in Friendly AI.
Original research on Less Wrong continues to be pursued in several discussion threads on the site.

How Can I Reduce Existential Risk From AI?

Another recent Less Wrong post of note by Luke Muehlhauser is “How can I reduce existential risk from AI?” “Existential risk”, a term coined by Oxford philosopher Nick Bostrom, specifically refers to risks to the survival of the human species. The Singularity Institute argues that advanced artificial intelligence is an existential risk to the future of the human species. “Existential risk” generally refers to the total destruction of the human species, rather than risks which threaten 90% or 99% of the population, which would constitute global catastrophic risks but not true existential risks.
Since our founding in 2000, the Singularity Institute has argued that smarter-than-human, self-improving Artificial Intelligence is an existential risk to humanity. The concern is that, at some point over the next hundred years, advanced AI will be created that can manufacture its own sophisticated robotics and threaten to displace human civilization, not necessarily through deliberate action but merely as a side-effect of the exploitation of resources required for our survival, such as carbon, oxygen, or physical space. For additional background, please read our concise summary, “Reducing Long-Term Catastrophic Risks from Artificial Intelligence”.
The post outlines three major categories of work towards reducing AI risk: (1) meta-work, such as making money to contribute to Friendly AI research; (2) strategic work, work towards a better strategic understanding of the challenges we face; and (3) direct work, such as technical research, political action, or particular kinds of technological development. All three are crucial to building the “existential risk mitigation ecosystem,” a cooperative effort of hundreds of people to better understand AI risk and do something about it.

New Research Associates

The Singularity Institute is pleased to announce four new research associates, Benja Fallenstein, Marcello Herreshoff, Mihály Barasz, and Bill Hibbard.
Benja Fallenstein is interested in the basic research necessary for the development of safe AI goals, especially from the perspective of mathematical models of evolutionary psychology, and also in anthropic reasoning, decision theory, game theory, reflective mathematics, and programming languages with integrated proof checkers. Benja is a mathematics student at University of Vienna, with a focus in biomathematics.
Marcello Herreshoff has worked with the Singularity Institute on the math of Friendly AI, from time to time since 2007. In High School, he was a two time USACO finalist and he published a novel combinatorics result, which he presented at the Twelfth International Conference on Fibonacci Numbers and Their Applications. He holds a BA in Math from Stanford University. At Stanford he was awarded two honorable mentions on the Putnam mathematics competition, and submitted his honors thesis for publication in the Logic Journal of the IGPL. His research interests include mathematical logic, and in its use in formalizing coherent goal systems.
Mihály Barasz is interested in functional languages and type theory and their application in formal proof systems. He cares deeply about reducing existential risks. He has an M.Sc. summa cum laude in Mathematics from Eotvos Lorand University, Budapest and currently works at Google.
Bill Hibbard is an Emeritus Senior Scientist at the University of Wisconsin-Madison Space Science and Engineering Center, currently working on issues of AI safety and unintended behaviors. He has a BA in Mathematics and MS and PhD in Computer Sciences, all from the University of Wisconsin-Madison.

AI Risk-Related Improvements to the LW Wiki

The Singularity Institute has greatly improved the Less Wrong wiki with new entries, featuring topics from Seed AI to moral uncertainty and more. Over 120 pages were updated in total. The improvements to the wiki were prompted by earlier proposals for a dedicated scholarly AI risk wiki. The improvements to the wiki enable more background knowledge for publishing short, clear, scholarly articles on AI risk.
Some articles of interest include the 5-and-10 problem, AGI skepticism, AGI Sputnik moment, AI advantages, AI takeoff, basic AI drives, benevolence, biological cognitive enhancement, Coherent Extrapolated Volition, computing overhang, computronium, differential intellectual progress, economic consequences of AI and whole brain emulation, Eliezer Yudkowsky, emulation argument for human-level AI, extensibility argument for greater-than-human intelligence, evolutionary argument for human-level AI, complexity of human value, Friendly artificial intelligence, Future of Humanity Institute, history of AI risk thought, intelligence explosion, moral divergence, moral uncertainty, Nick Bostrom, optimal philanthropy, optimization process, Oracle AI, orthogonality thesis, paperclip maximizer, Pascal’s mugging, recursive self-improvement, reflective decision theory, singleton, Singularitarianism, Singularity, subgoal stomp, superintelligence, terminal value, timeless decision theory, tool AI, utility extraction, value extrapolation, value learning, and whole brain emulation.

Featured Volunteer: Ethan Dickinson

This month, we thank Ethan Dickinson for his volunteer work transcribing videos from the Singularity Summit 2012 conference. When we talked with Ethan about his work, he mentioned how one talk, Julia Galef’s, (embedded below) even inspired a teary-eyed emotional climax. Ethan became involved in volunteer work for SI after several years of developing an increasing interest in rationality, originally introduced via Harry Potter and the Methods of Rationality. Today, he feels he is using the full powers of
his imagination to understand the Singularity. Paraphrasing Jaan Tallin, he has used his own reading of science fiction such as William Gibson’s Neuromancer and Isaac Asimov’s novels to imagine worlds “much more optimistic and much more pessimistic” than many of the middle ground scenarios for the future widely assumed today.

Featured Summit Video: Julia Galef on Rationality

Four decades of cognitive science have confirmed that Homo sapiens are far from “rational animals.” Scientists have amassed a daunting list of ways that our brain’s fast-and-frugal judgment heuristics fail in modern contexts for which they weren’t adapted, or stymie our attempts to be happy and effective. Hence the project we’re undertaking at the new Center for Applied Rationality (CFAR) — training human brains to run algorithms that optimize for our interests as autonomous beings in the modern world, not for the interests of ancient replicators. This talk explores what we’ve learned from that process so far, and why training smart people to be rational decision-makers is crucial to a better future.

News Items

Killer robots? Cambridge brains to assess AI risk
CNET, November 26, 2012

DARPA’s Pet-Proto Robot Navigates Obstacles
YouTube, October 24, 2012

The post December 2012 Newsletter appeared first on Machine Intelligence Research Institute.

December 2012 Newsletter


2012 Winter Matching Challenge!

Thanks to the generosity of several major donors,† every donation to the Machine Intelligence Research Institute made now until January 5th, 2013 will be matched dollar-for-dollar, up to a total of $115,000!
Donate Now!

$0

$28.75K

$57.5K

$86.25K

$115K

Now is your chance to double your impact while helping us raise up to $230,000 to help fund our research program.

(If you’re unfamiliar with our mission, please see our press kit and read our short research summary: Reducing Long-Term Catastrophic Risks from Artificial Intelligence.)
Now that Singularity University has acquired the Singularity Summit, and SI’s interests in rationality training are being developed by the now-separate CFAR, the Machine Intelligence Research Institute is making a profound transition. (Note that most of the money from the acquisition is being placed in a separate fund for Friendly AI researchers, and therefore does not support our daily operations or other programs.)
For 12 years we’ve largely focused on movement-building — through the Singularity Summit, Less Wrong, and other programs. This work was needed to build up a community of support for our mission and a pool of potential researchers for our unique interdisciplinary work.
Now, the time has come to say “Mission Accomplished.” Or at least, “Mission Accomplished Well Enough to Pivot to Research.” Our community of supporters is now large enough that many qualified researchers are available for us to hire, if we can afford to hire them.
Having published 30+ research papers and dozens more original research articles on Less Wrong, we certainly haven’t neglected research. But in 2013 we plan to pivot so that a much larger share of the funds we raise is spent on research.
Accomplishments in 2012

Held a one-week research workshop on one of the open problems in Friendly AI research, and got progress that participants estimate would be the equivalent of 1-3 papers if published. (Details forthcoming. The workshop participants were Eliezer Yudkowsky, Paul Christiano, Marcello Herreshoff, and Mihály Bárász.)
Produced our annual Singularity Summit in San Francisco. Speakers included Ray Kurzweil, Steven Pinker, Daniel Kahneman, Temple Grandin, Peter Norvig, and many others.
Launched the new Center for Applied Rationality, which ran 5 workshops in 2012, including Rationality for Entrepreneurs and SPARC (for young math geniuses), and also published one (early-version) smartphone app, The Credence Game.
Launched the redesigned, updated, and reorganized Singularity.org website.
Achieved most of the goals from our August 2011 strategic plan.
11 new research publications.
Eliezer published the first 12 posts in his sequence Highly Advanced Epistemology 101 for Beginners, the precursor to his forthcoming sequence, Open Problems in Friendly AI.
SI staff members published many other substantive articles on Less Wrong, including How to Purchase AI Risk Reduction, How to Run a Successful Less Wrong Meetup, a Solomonoff Induction tutorial, The Human’s Hidden Utility Function (Maybe), How can I reduce existential risk from AI?, AI Risk and Opportunity: A Strategic Analysis, and Checklist of Rationality Habits.
Launched our new volunteers platform, SingularityVolunteers.org.
Hired two new researchers, Kaj Sotala and Alex Altair.
Published our press kit to make journalists’ lives easier.
And of course much more.

Future Plans You Can Help Support
In the coming months, we plan to do the following:

As part of Singularity University’s acquisition of the Singularity Summit, we will be changing our name and launching a new website.
Eliezer will publish his sequence Open Problems in Friendly AI.
We will publish nicely-edited ebooks (Kindle, iBooks, and PDF) for many of our core materials, to make them more accessible: The Sequences, 2006-2009, Facing the Singularity, and The Hanson-Yudkowsky AI Foom Debate.
We will publish several more research papers, including “Responses to Catastrophic AGI Risk: A Survey” and a short, technical introduction to timeless decision theory.
We will set up the infrastructure required to host a productive Friendly AI team and try hard to recruit enough top-level math talent to launch it.

(Other projects are still being surveyed for likely cost and strategic impact.)
We appreciate your support for our high-impact work! Donate now, and seize a better than usual chance to move our work forward. Credit card transactions are securely processed using either PayPal or Google Checkout. If you have questions about donating, please contact Louie Helm at (510) 717-1477 or louie@intelligence.org.
† $115,000 of total matching funds has been provided by Edwin Evans, Mihály Bárász, Rob Zahra, Alexei Andreev, Jeff Bone, Michael Blume, Guy Srinivasan, and Kevin Fischer.
The post 2012 Winter Matching Challenge! appeared first on Machine Intelligence Research Institute.

2012 Winter Matching Challenge!


Once again, a reporter thinks our positions are the opposite of what they are

Perhaps the most accurate media coverage the Machine Intelligence Research Institute (MIRI) has yet received was a piece by legendary science author Carl Zimmer in Playboy. To give you a sense of how inaccurate most of our media coverage is, here’s a (translated) quote from some coverage of MIRI in a Franco-German documentary:
In San Francisco however, a society of young voluntary scientists believes in the good in robots. How naive! Here at [MIRI]…
Such a quote is amusing because, of course, the Machine Intelligence Research Institute has been saying for a decade that AI will by default be harmful to humanity, that it is extremely difficult to design a “moral machine,” that neither we nor anyone else knows how to do it yet, and that dozens of approaches proposed by others are naive and insufficient for one reason or another.
Now, in a new piece for The Sunday Times, Bryan Appleyard writes:
Yudkowsky [from MIRI] seemed to me simplistic in his understanding of moral norms. “You would not kill a baby,” he said to me, implying that was one norm that could easily be programmed into a machine.
“Some people do,” I pointed out, but he didn’t see the full significance. SS officers killed babies routinely because of an adjustment in the society from which they sprang in the form of Nazism. Machines would be much more radically adjusted away from human social norms, however we programmed them.
Again: MIRI has been saying for a decade that it is extremely difficult to program a machine to not kill a baby. Indeed, our position is that directly programming moral norms won’t work because our values are more complex than we realize. The direct programming of moral norms is something that others have proposed, and a position we criticize. For example, here is a quote from the concise summary of our research program:
Since we have no introspective access to the details of human values, the solution to this problem probably involves designing an AI to learn human values by looking at humans, asking questions, scanning human brains, etc., rather than an AI preprogrammed with a fixed set of imperatives that sounded like good ideas at the time.
But this doesn’t mean we can simply let some advanced machine learning algorithms observe human behavior to learn our moral values, because:
The explicit moral values of human civilization have changed over time, and we regard this change as progress, and extrapolate that progress may continue in the future. An AI programmed with the explicit values of 1800 might now be fighting to reestablish slavery… Possible bootstrapping algorithms include “do what we would have told you to do if we knew everything you knew,” “do what we would’ve told you to do if we thought as fast as you did and could consider many more possible lines of moral argument,” and “do what we would tell you to do if we had your ability to reflect on and modify ourselves.” In moral philosophy, this notion of moral progress is known as reflective equilibrium.
Moving on… Appleyard’s point that “Machines would be much more radically adjusted away from human social norms, however we programmed them” is another point MIRI been making from the very beginning. See the warnings against anthropomorphism in “Creating Friendly AI” (2001) and “Artificial Intelligence as a Positive and Negative Factor in Global Risk” (written in 2005, published in 2008). AI mind designs will be far more “alien” to us than the minds of aliens appearing in movies.
Appleyard goes on to say:
[Compared to MIRI,] the Cambridge group has a much more sophisticated grasp of these issues. Price, in particular, is aware that machines will not be subject to social pressure to behave well.
“When you think of the forms intelligence might take,” Price says, “it seems reasonable to think we occupy some tiny corner of that space and there are many ways in which something might be intelligent in ways that are nothing like our minds at all.”
What Appleyard doesn’t seem to realize, here, is that Price is basically quoting a point long stressed by MIRI researcher Eliezer Yudkowsky — that there is a huge space of possible minds, and humans only occupy a tiny corner of that space. In fact, that point is associated with MIRI and Eliezer Yudkowsky more than with anyone else! Here’s another quote from the paper Yudkowsky wrote in 2005:
The term “Artificial Intelligence” refers to a vastly greater space of possibilities than does the term “Homo sapiens.” When we talk about “AIs” we are really talking about minds-in-general, or optimization processes in general. Imagine a map of mind design space. In one corner, a tiny little circle contains all humans; within a larger tiny circle containing all biological life; and all the rest of the huge map is the space of minds in general. The entire map floats in a still vaster space, the space of optimization processes… It is this enormous space of possibilities which outlaws anthropomorphism as legitimate reasoning.
So once again, a reporter thinks MIRI’s positions are the opposite of what they are.
Beware what you read in the popular media!
Update: I told Appleyard of his mistake, and he simply denied that his article has made a mistake on this matter.
The post Once again, a reporter thinks our positions are the opposite of what they are appeared first on Machine Intelligence Research Institute.


November 2012 Newsletter

November 2012 Newsletter

Greetings from the Executive Director

Dear friends of the Machine Intelligence Research Institute,
My thanks to the dozens of staff members, contractors, and volunteers who helped make this year’s Singularity Summit our most professional and exciting Summit yet! Videos of the talks are now online, but I pity those who missed out on the live event and the killer lobby scene. We made more room in the schedule this year for mingling and networking, and everyone seemed to love it. After all, the future won’t be created merely by information and information technologies, but by the communities of people who decide to create the future together.
The Summit is a tremendous amount of work each year, and so it felt great to have so many people approach me to say, unprompted, “Wow, this is the best Summit yet!” and “You guys really took it to the next level this year; this is great!” I replayed those moments in my head on Sunday night as I drifted into the blissful coma that would repay several weeks of sleep debt.
Luke Muehlhauser

Singularity Summit Rocks San Francisco

The Singularity Summit 2012 was held at the Masonic Center in San Francisco on October 13-14, with an attendance of over 600 scientists, entrepreneurs, and thought leaders. We received media coverage from the BBC (online later this month), the Wall Street Journal, The Verge, and PolicyMic, and several other media outlets. The full program is online for your viewing enjoyment  at Fora.tv. (If you like to watch talks at an accelerated playback speed, you can sign up for a free trial of Fora.tv, download the HD video, and play it with a program like VLC that allows you to adjust playback speed.)
We would like to thank everyone who participated this year, including all the speakers, attendees, and hard-working staff. Everyone we heard from was very impressed by the program and the quality of lobby networking and discussion. Stay tuned for announcements about the next Singularity Summit!

Workshop: Rationality for Entrepreneurs

On November 16-18, the Machine Intelligence Research Institute’s sister organization Center for Applied Rationality (CFAR) will be running an immersive rationality workshop in the San Francisco Bay Area for a select group of 25 entrepreneurs.
The workshop builds on CFAR’s previous rationality training retreats to present a curriculum that addresses the highest-priority improvements to reasoning for entrepreneurs. Small class sizes, interactive workshop activities, personalized attention inside and outside of class, and six weeks of regular followup are designed to help participants learn to actually use the techniques, rather than just know about them academically. The curriculum includes:

What ideal decision-making looks like: when to trust your gut, and when to trust your head.
How to learn about your own motivations and goals using principled thought experiments.
How to make more accurate everyday predictions using Bayes’ Rule, a simple but powerful intuitive tool from probability theory.
The science behind stress reactions, and how to make it easier to ask VCs for investments, customers for money, and your employees to go for equity.

Applications are still open. To fill out an application, or to find out more about the workshop, go to the CFAR website.

New Less Wrong Sequence from Eliezer Yudkowsky

Eliezer Yudkowsky returns to Less Wrong with a new sequence of articles, Highly Advanced Epistemology 101 for Beginners, which sets the stage for his next sequence, “Open Problems in Friendly AI.” The latter sequence will outline the mathematical and philosophical problems which need to be solved to make concrete progress on Friendly AI.
For an earlier article outlining some open problems in Friendly Artificial Intelligence, see Luke Muehlhauser’s So You Want to Save the World. These open problems include: developing a reflective decision theory, selecting ideal Bayesian priors, and ensuring that an AI’s utility function remains stable even under fundamental changes to the AI’s ontology.

New Volunteer Platform Launched!

Sign up here: www.singularityvolunteers.org
Over the past couple of months we thought hard about how to improve our volunteer program, with the goal of finding a system that makes it easier to engage volunteers, create a sense of community, and quantify volunteer contributions. After evaluating several different volunteer management platforms, we decided to partner with Youtopia — a young company with a lot of promise — and make heavy use of Google Docs.
Youtopia structures volunteer opportunities into challenges with associated activities. Completing activities earn volunteers points — which allows them to measure how much they are contributing relative to other volunteers (friendly competition encouraged) — and awards that showcase specific accomplishments. Leveraging Google Docs allows volunteers to work together more smoothly — in real-time or asynchronously.
It used to be that most volunteers were isolated from each other as they worked with different SI staff members on various projects. This made generating a sense of community difficult. We think a sense of community is important for long term volunteer engagement. Also, it was difficult to quantify the contributions made by our volunteers. Now, volunteers can see what their peers are working on, compete on challenges, and work on activities together — all while Youtopia makes it easy for us to quantify their contributions.
Here is a quote from Project Manager Malo Bourgon: “I’d strongly encourage everyone to head over to singularityvolunteers.org, register as a volunteer, and explore the challenges we currently have posted. As a small nonprofit, we literally have hundreds of hours of work we just can’t afford to do each month. Because of this, volunteers are really important to us; they really do make a meaningful impact.”

Singularity Rising Published

Singularity Rising, a new book by Smith College economics professor James D. Miller (author of Principles of Microeconomics), is now available for purchase. Here are some of the scenarios that Professor Miller considers in his new book:

A merger of man and machine making society fantastically wealthy and nearly immortal.
Competition with billions of cheap AIs drive human wages to almost nothing while making investors rich.
Businesses rethink investment decisions to take into account an expected future period of intense creative destruction.
Inequality drops worldwide as technologies mitigate the cognitive cost of living in impoverished environments.
Drugs designed to fight Alzheimer’s disease and keep soldiers alert on battlefields have the fortunate side effect of increasing all of their users’ IQs, which, in turn, adds a percentage points to worldwide economic growth.

Miller’s book has received glowing endorsements from Luke Muehlhauser, Paypal co-founder Peter Thiel, SENS foundation Chief Science Officer Aubrey de Grey, Humanity+ Chairman Natasha Vita-More, and novelist Vernor Vinge.

Register Now for Tickets to AGI-12!

The Fifth Conference on Artificial General Intelligence will be held at Oxford University this year, from December 8-11. AGI researchers will present and discuss their results from the last year. Register here.
Some of the speakers include David Hanson, CEO of Hanson Robotics, who will speak on humanoid robots and AGI; Angelo Cangelosi, professor of AI and cognition, who will speak on cognitive robotics; professor of cognitive science Margaret Boden, who will speak on creativity and AGI; and Nick Bostrom, professor of philosophy, who will speak on the future evolution of advanced AGIs and the dynamics of AGI goal systems.
Immediately following AGI-12 will be the first conference on AGI Impacts, organized and hosted by the Future of Humanity Institute. The keynote speakers will be Steve Omohundro and Bruce Schneier.
The Singularity Institute is sponsoring a $1,000 prize, the 2012 Turing Prize for Best AGI Safety Paper, for exceptional research on the question of how to develop safe architectures or goals for AGI.
We hope to see you in Oxford for these important conferences!

Michael Anissimov and Louie Helm to Speak at Humanity+ @ San Francisco

SI staffers Michael Anissimov and Louie Helm will give talks at the upcoming Humanity+ @ San Francisco conference on December 1-2, speaking alongside distinguished presenters such as Aubrey de Grey and David Pearce.
Michael Anissimov will speak on “The Media Performance of Transhumanism” while Louie Helm will speak on “The Mainstream Academic Publishing We Need”. Here is Michael Anissimov’s abstract:
Since 2005 or so, transhumanism and transhumanist ideas have had a rising profile in the media. What have been our greatest successes of the past few years and how can we repeat them? Which memes are getting the most airtime, and which are being ignored? Is more media exposure always better? What can we do to ensure that we and our organizations are media-savvy? How do we leverage technology to maximize the impact of social media? This talk by the media director of the Singularity Institute will examine these questions and come to concrete conclusions.
Here is Louie Helm’s abstract:
It may seem obvious to you that progress in fields like AGI and life extension will have the most long-lasting and far-reaching impact of any work you could possibly be doing right now. But what enabled you to realize that? Your path to understanding probably passed through a period of several months of independent self-study that required you to evaluate many informal arguments of varying quality, scattered across the internet. But if we want to attract more and better researchers, especially domain experts who don’t have hundreds of hours to study outside their field, we need to formally summarize our best ideas in standard academic style to help erase this enormous barrier to entry. Contributing to this effort by publishing current research is accessible to many and support is available for those who are motivated.
See other abstracts on the conference abstracts page. Tickets for the conference are available now.

Featured Volunteer: Tim Oertel

Every newsletter, we like to recognize a volunteer who has made a special contribution to the Singularity Institute. This month, we honor Tim Oertel for his proofreading work.
With the launch of the new website, the Singularity Institute has also republished many of it’s existing publications into SI’s new article template. Moreover, this year has also been a productive one in terms of new publications from SI staff and research associates. As such, volunteer proofreaders have never been more important to SI. Tim Oertel is one of SI’s leading proofreaders. Thank you for your excellent work, Tim!

Featured Summit Video: Luke Muehlhauser on
“The Singularity, Promise and Peril”

In  “The Singularity: Promise and Peril“, SI Executive Director Luke Muehlhauser explains the Singularity, its potential risks and benefits, and what we can do about it. Muehlhauser emphasizes that intelligence lies at the root of all technology, updating the familiar Arthur C. Clarke quote “Any sufficiently advanced technology is indistinguishable from magic”, with his new version, “Any sufficiently advanced intelligence is indistinguishable from magic.” For more of Luke’s views on the Singularity, we encourage you to read his e-book Facing the Singularity.

Featured Research Paper: “Learning What to Value”

In this research paper from 2011, SI research associate Daniel Dewey (now also a Research Fellow at the Future of Humanity Institute at Oxford University) outlines his concept of “value learners.” Here is a snippet from the paper’s abstract:
Reinforcement learning can only be used in the real world to deŀne agents whose goal is to maximize expected rewards, and since this goal does not match with human goals, AGIs based on reinforcement learning will often work at cross-purposes to us. To solve this problem, we deŀne value learners, agents that can be designed to learn and maximize any initially unknown utility function so long as we provide them with an idea of what constitutes evidence about that utility function.

News Items

First Genome Study to Sequence 1000+ Human Genomes
The Guardian, October 31, 2012

Self-driving cars now legal in California
CNN, October 30, 2012

Swarm Robots Cooperate with a Flying Drone
Wimp.com, October 23, 2012

IBM’s Watson Expands Commercial Applications, Plans to Go Mobile
Singularity Hub, October 14, 2012

Google Puts Its Virtual Brain Technology to Work
Technology Review, October 5, 2012

‘Green Brain’ Project to Create an Autonomous Flying Robot With a Honey Bee Brain
ScienceDaily, October 2, 2012

Thank You for Reading!
The post November 2012 Newsletter appeared first on Machine Intelligence Research Institute.

November 2012 Newsletter


September 2012 Newsletter

September 2012 Newsletter

Greetings from the Executive Director

August was a busy month for the Machine Intelligence Research Institute. Thanks to our successful summer fundraiser, we are running full steam ahead on all fronts: Singularity Summit 2012, the launch of CFAR, increased research output (see below), and improving organizational efficiency in literally dozens of ways.
Thank you for your continued support as we work toward a positive Singularity.
Luke Muehlhauser

Register Now for Singularity Summit 2012

Our annual conference, the Singularity Summit, is back on the west coast this year! The Summit is October 13-14 at the Nob Hill Masonic Auditorium in San Francisco. Thought leaders from all over the world converge on this event, the leading conference on transformative emerging technologies, providing fantastic opportunities for networking.
The full schedule of events for Singularity Summit was recently announced. Featured presenters include futurist Ray Kurzweil, Nobel Prize winner Daniel Kahneman, cognitive scientist Steven Pinker, co-founder of genomics company 23andMe Linda Avey, animal welfare and autism advocate Temple Grandin, mathematician and sci-fi great Vernor Vinge, Google Director of Research Peter Norvig, and other thought leaders.
For regular updates on the Summit as it approaches, follow the Singularity Summit blog. There are still discounted rooms available at the Mark Hopkins hotel, a two minute walk from the Summit.

SI Hires Two New Researchers

The Machine Intelligence Research Institute has hired two new full-time Research Fellows. Alex Altair, based out of our Berkeley headquarters, is focused on making the investigation of Friendly Artificial Intelligence more mathematically rigorous. He recently wrote “An Intuitive Explanation of Solomonoff Induction” with Luke Muehlhauser, and is now working on a paper on timeless decision theory. Kaj Sotala, based in Helsinki, Finland, is developing several papers on AI risk strategy. Kaj co-authored two SI papers this year, on the advantages of artificial intelligences, uploads, and digital minds and brain uploading-related group mind scenarios. We congratulate Alex and Kaj on their new positions and look forward to encouraging their future efforts.

New Evidence About AI Predictions

Kaj Sotala, our new Research Fellow, recently analyzed a large set of AI predictions (compiled previously by other SI researchers). The results are summarized in the Less Wrong post “AI timeline predictions: are we getting better?” Kaj collaborated with Stuart Armstrong of the Future of Humanity Institute at Oxford for this project. 257 predictions were retained, and the key results are summarized in the graph above.
Of particular interest was whether predictions had a tendency of falling within the lifetime of the predictor — an alleged phenomenon coined as the “Maes-Garreau law” by Kevin Kelly. This study revealed, for the first time in a scientific way, that there is no substantial tendency for predictions to expect AI within their own lifetimes. The clustering effect you would expect if this tendency existed is nowhere to be found.
Anders Sandberg, a researcher at the Future of Humanity Institute, blogged about the study, writing, “Stuart Armstrong and Kaj Sotala… produced an excellent post analyzing a set of predictions about the future of AI. Among other things, they looked for evidence of the Maes-Garreau law, that people predict AI somewhere about when they retire. Somewhat surprisingly, they found that this was not true. Instead, over a third of predictors claim AI will happen 16-25 years in the future, irrespective of age. There was no strong correlation between age and expected distance into the future.”

Machine Intelligence Research Institute Upgrades Its Research Output

The Machine Intelligence Research Institute’s research fellows and research associates have more peer-reviewed publications forthcoming in 2012 than they had published in all past years combined.
We thank Eliezer Yudkowsky, Daniel Dewey, Luke Muehlhauser, Anna Salamon, Louie Helm, Joshua Fox, Carl Shulman, and Kaj Sotala for their recent research efforts and encourage them to continue this valuable work.

Press Kit Published

In anticipation of the upcoming Singularity Summit, we have published an online press kit here. This press kit compiles a letter from our executive director, a quick overview of our research, essential bios of staff and supporters, and a selection of media articles. Journalists interested in writing about the Machine Intelligence Research Institute or Singularity Summit are encouraged to get in touch with our Media Director Michael Anissimov at admin@intelligence.org.

Summer Program on Rationality and Cognition

In early August, SI co-sponsored the Summer Program on Rationality and Cognition (SPARC), a week-long summer camp for young mathematicians held at UC Berkeley. The Center for Applied Rationality (CFAR), a non-profit currently in the process is spinning off from SI, also sponsored the program. Paul Christiano, a primary organizer of the program, said, “SPARC drew many of the very best high school mathematicians, and introduced them to a roughly even mix of LessWrong-style rationality and technical foundations for probabilistic reasoning. Students were enthusiastic and the program was very well received (comparable to other CFAR programs). The apparent success of SPARC will facilitate similar programs in future years, helping CFAR understand how to reach a wider audience and establishing a positive reputation in a wider community.”

Luke Muehlhauser’s Reddit AMA a Success

Luke Muehlhauser conducted an “Ask Me Anything” on the social media website Reddit.com, under /r/Futurology. He said “I am Luke Muehlhauser, CEO of the Machine Intelligence Research Institute. Ask me anything about the Singularity, AI progress, technological forecasting, and researching Friendly AI!” The result was a total of 2,150 comments and 1,329 upvotes! For a selection of Luke’s responses, read the feed for his Reddit account. There was also coverage at WIRED.

Featured Singularity Summit Video: Max Tegmark

In this presentation at Singularity Summit 2011, cosmologist and MIT professor Max Tegmark explores what could happen to us billions of years in the future. Entitled “The Future of Life: a Cosmic Perspective”, Tegmark summarizes what direction cosmologists currently believe the universe will take in coming billions of years and analyzes the implications for the future of life and civilization. He questions the common perception of humans as being the pinnacle of life and foresees beings “as far ahead of us as we are from bacteria”.

Featured Volunteer: Alton Sun

Alton Sun, a photographer from the San Francisco Bay Area, has volunteered for the Machine Intelligence Research Institute (MIRI) and our sponsored organization CFAR (Center for Applied Rationality) on many occasions now. Alton likes orchestrating photo shoots in some cool warehouses with shadows and orange light, or a grassy field with bright sun and beautiful trees. Alton finds, however, that the component of beauty normally found in the physical surroundings of his work is found in the minds of the people who work here, where their ethically oriented and intellectual personalities hold new sources of inspiration.
Since to Alton “inaction is inexcusable”, it simply becomes a question of upon what one should act. For him, his contribution of photography to SI is a means of addressing some of the world’s most challenging problems and answering some of the largest questions for what is coming in the future of humanity.
To get to know his work, find his portfolio here: altonsunphoto.com

Thank You for Reading Our Newsletter!

The post September 2012 Newsletter appeared first on Machine Intelligence Research Institute.

September 2012 Newsletter


August 2012 Newsletter

This newsletter was sent to newsletter subscribers in early August, 2012.

Greetings from the Executive Director

The big news this month is that we surpassed our fundraising goal of raising $300,000 in the month of July. My thanks to everyone who donated! Your contributions will help us finish launching CFAR and begin to build a larger and more productive research team working on some of the most important research problems in the world. Luke Muehlhauser

Singularity Summit Prices Will Increase on August 15th!

Two-day tickets prices for the Singularity Summit 2012 are still only $635, but will increase again on August 15th! For anyone interested in hearing some of the foremost speakers on science, technology, and the future of humanity, buy your ticket today to our international conference at the Nob Hill Masonic Center, SF on October 13-14th!

2012 Summer Singularity Challenge Success!

Thanks to the effort of our donors, the 2012 Summer Singularity Challenge has been met! All $150,000 contributed will be matched dollar for dollar by our matching backers, raising a total of $300,000 to fund the Machine Intelligence Research Institute’s operations. We reached our goal near 6pm on July 29th. On behalf of our staff, volunteers, and entire community, I want to personally thank everyone who donated. Your dollars make the difference. Here’s to a better future for the human species.

Facing the Singularity Finished

Luke Muehlhauser has now published the final chapters of his introductory blog on the coming of AI, Facing the Singularity. The penultimate chapter explains what can be done to improve our odds of a positive singularity, and the final chapter outlines what benefits we can expect from a positive singularity.

Comparison of 2011 August strategic plan to today

Progress updates are nice, but without a previously defined metric for success it’s hard to know whether an organization’s achievements are noteworthy or not. Is the Machine Intelligence Research Institute making good progress, or underwhelming progress? Luckily, in August 2011 we published a strategic plan that outlined lots of specific goals. It’s now August 2012, so we can check our progress against the standard set nearly one year ago. The full comparison is available here, and the final section is excerpted below: Now let’s check in on what we said our top priorities for 2011-2012 were:

Public-facing research on creating a positive singularity. Check. SI has more peer-reviewed publications in 2012 than in all past years combined.
Outreach / education / fundraising. Check. Especially, through CFAR.
Improved organizational effectiveness. Check. Lots of good progress on this.
Singularity Summit. Check.

In summary, I think SI is a bit behind where I hoped we’d be by now, though this is largely because we’ve poured so much into launching CFAR, and as a result, CFAR has turned out to be significantly more cool at launch than I had anticipated.

SI Publishes Solomonoff Induction Tutorial

Visiting Fellow Alex Altair worked with Luke Muehlhauser to publish An Intuitive Explanation of Solomonoff Induction, a sequel to Eliezer Yudkowsky’s Intuitive Explanation of Bayes’ Theorem. Whereas Bayes’ Theorem is a key idea in probability theory, Solomonoff Induction is a key idea in the study of universal, automated inference.It begins:
People disagree about things. Some say that television makes you dumber; other say it makes you smarter.  Some scientists believe life must exist elsewhere in the universe; others believe it must not. Some say that complicated financial derivatives are essential to a modern competitive economy; others think a nation’s economy will do better without them.  It’s hard to know what is true.
And it’s hard to know how to figure out what is true.  Some argue that you should assume the things you are most certain about and then deduce all other beliefs from your original beliefs. Others think you should accept at face value the most intuitive explanations of personal experience. Still others think you should generally agree with the scientific consensus until it is disproved.
Wouldn’t it be nice if determining what is true was like baking a cake? What if there was a recipe for finding out what is true? All you’d have to do is follow the written directions exactly, and after the last instruction you’d inevitably find yourself with some sweet, tasty truth!
In this tutorial, we’ll explain the closest thing we’ve found so far to a recipe for finding truth: Solomonoff induction.

Dialogue with Bill Hibbard about AGI

Luke Muehlhauser has published a dialogue between himself and computer scientist Bill Hibbard, author of Super-Intelligent Machines, about AI safety. The dialogue is part of Luke’s series of interviews about AI safety.

Featured Donor: Robin Powell

Below is an interview with this month’s featured donor, Robin Powell.Luke Muehlhauser: Robin, you’ve been donating $200 a month since August 2004. That adds up to more than $20,000, making you our 8th largest publicly listed donor! Why do you support the Machine Intelligence Research Institute like this?Robin Powell: I honestly believe that a beneficial Singularity is the best hope that humanity has for long-term survival. Having spent hundreds of hours researching the various people and groups that are actively working on Singularity-related issues, the Machine Intelligence Research Institute is the only one that I really feel has their eyes on the right ball, which is the Friendly AI problem. I feel confident that my donations are the most effective way I can possibly aid in the best possible future for humanity.
Luke: What do you give up each month in order to donate $200/month to the Machine Intelligence Research Institute?
Robin: Mostly I’ve been able to get by when things got complicated by re-budgeting, but I’ve had to do that rather a lot more often than I would have had to otherwise.
Luke: What challenges have you faced since August 2004, while continuing to donate $200 a month?
Robin: The time that I took off a couple of months to help my aging father, without pay, was by far the hardest; the extra money would really have helped then. But for me it’s about expected return: when the future of the human race is in the balance, having to borrow from friends briefly or similar hardships seem pretty inconsequential.
Luke: What one thought would you most like to share with the community of people who care about reducing existential risks?
Robin: AI is coming, relatively soon. There is no more important task for humanity than to prevent our extinction and preserve a better version of our values. Now is the time to spend time and money protecting the future of humanity. Please help us.
Luke: Thanks for your time, Robin, and thanks for your continued support!

Featured Summit Video

This month we are featuring a video from the 2006 Singularity Summit: Eliezer Yudkowsky’s “The Human Importance of the Intelligence Explosion“. Eliezer’s talk discusses I.J. Good’s concept of an “intelligence explosion,” and its central importance for the human species.

Use Good Search, support the Machine Intelligence Research Institute

GoodSearch, which allows you to donate to a cause merely by using their search engine, now has a donation option for the Machine Intelligence Research Institute. Use GoodSearch to donate every day without opening your wallet!

The post August 2012 Newsletter appeared first on Machine Intelligence Research Institute.

August 2012 Newsletter


July 2012 Newsletter

This newsletter was sent out to Machine Intelligence Research Institute newsletter subscribers in July 2012
Greetings from the Executive Director

Friends of the Machine Intelligence Research Institute,
Greetings! Our new monthly newsletter will bring you the latest updates from the Machine Intelligence Research Institute (MIRI). (You can read earlier monthly progress updates here.)
These are exciting times at MIRI. We just launched our new website, and also the website for the Center for Applied Rationality. We have several research papers under development, and after a long hiatus from AI research, researcher Eliezer Yudkowsky is planning a new sequence of articles on “Open Problems in Friendly AI.”
We have also secured $150,000 in matching funds for a new fundraising drive. To help support us in our work toward a positive Singularity, please donate today and have your gift doubled!
Luke Muehlhauser
Machine Intelligence Research Institute Executive Director

Donate Today to Double Your Impact!

Our summer 2012 matching drive is on! Now is your chance to double your impact while helping us raise up to $300,000 to help fund our research program and stage the upcoming Singularity Summit… which you can register for now!Since we published our strategic plan in August 2011, we have achieved most of the near-term goals outlined therein. Here are just a few examples:

We outlined the open research problems related to our work (Section 1.1).
We recruited several more research associates and about a dozen remote researchers (Section 1.2e).
We held our annual Singularity Summit and gained corporate sponsors for it (Section 2.1).
We made progress in decision theory (example) via LessWrong.com and our research associates (Section 2.2b).
We published How to Run a Successful Less Wrong Meetup Group (Section 2.2d).
We released pre-prints of several forthcoming research articles, including How Hard is Artificial Intelligence?, Intelligence Explosion: Evidence and Import, and The Singularity and Machine Ethics (Section 2.3).
We redesigned our primary website (Section 2.6).
We acquired $40,000/month in free Google Adwords advertising, to drive traffic to websites operated by the Machine Intelligence Research Institute (Section 2.6c).
We began publishing monthly progress reports (Section 2.9b).
We built up the Center for Applied Rationality such that it should be able to spin off from the Machine Intelligence Research Institute later this year (Section 3.1).
We created a transparency section on our website, where visitors can find our IRS 990 forms, and also several standard organizational policies, e.g. a conflict of interest policy, non-discrimination policy, etc (Section 3.2).

In the coming year, the Machine Intelligence Research Institute plans to do the following:

Hold our annual Singularity Summit, this year in San Francisco! Speakers this year include Ray Kurzweil, Steven Pinker, Tyler Cowen, Temple Grandin, Peter Norvig, Robin Hanson, and Vernor Vinge.
Spin off the Center for Applied Rationality as a separate organization focused on rationality training, so that the Machine Intelligence Research Institute can be focused more exclusively on Singularity research and outreach.
Publish additional research on AI risk and Friendly AI.
Eliezer will write an “Open Problems in Friendly AI” sequence for Less Wrong.
Finish Facing the Singularity and publish ebook versions of Facing the Singularity and The Sequences, 2006-2009.
And much more! For details on what we might do with additional funding, see How to Purchase AI Risk Reduction.

We appreciate your support for our high-impact work! Donate now, and seize a better than usual chance to move our work forward.

Eliezer to Write “Open Problems in Friendly AI”

After a long hiatus from AI research to help with movement-building, Eliezer Yudkowsky is now planning a sequence of articles on open problems in Friendly AI research. These articles will help to explain the technical research that can be done today to help ensure a positive Singularity. The articles will be initially published at Less Wrong, where his earlier sequences of articleswere published.What about Eliezer’s in-progress rationality books? They are on hold for now while Eliezer works on other projects. We have signed a retainer with a professional author who has written at least one best-selling science book, and he will work on Eliezer’s books once he finishes his current project, probably late this fall.

Visit Our New Website

SI Media Director Michael Anissimov and many others have worked hard to create the new look and feel for our online web presence, at Singularity.org. The site has been reorganized into six top-level pages: About, What We Do, Research, Media, Get Involved, and Donate.The single greatest update to our website is to our researchpage and our research papers, which are now organized into three categories. Most of our research papers have been ported to a clean new template, and every reference has been manually checked and updated.
We’ve also created a transparency page which features Q&As, policy documents, and our tax forms dating back to our founding in 2000.
For casual reading, there is also a new tech summaries page that features short articles on emerging technologies such as the Berkeley Brain-Computer Interface, regenerative medicine, and AI in automated science and discovery.
Be sure to subscribe to our blog for regular updates!

Center for Applied Rationality (CFAR)

Per our August 2011 strategic plan, SI is helping to launch a separate organization devoted to rationality skills training. That organization is the Center for Applied Rationality (CFAR), which was recently granted 501c3 status and has a new website at AppliedRationality.org.In an age when our economic, political, and technological choices can spark both amazing progress and unprecedented devastation, it’s crucial that decision-makers are not only aware of the many near-universal cognitive biases, but also well-trained in avoiding them, and in using thinking habits based in probability and logic that outperform our brains’ innate algorithms. And that’s the goal of the Center for Applied Rationality (CFAR): to turn decades of research in cognitive science into a set of practicable techniques people can actually use to become better at reasoning and making decisions.
To think rationally about the future, we must be able to weigh different levels of risk and uncertainty, compare expected values, avoid over-weighting short term outcomes at the expense of the long term, avoid the “narrative fallacy” (in which vividly imagined outcomes appear more likely), have well-calibrated levels of confidence in their judgments, and think clearly even about emotionally fraught decisions — and much, much more.
CFAR is devoted to teaching those techniques, and the math and science behind them, to adults and exceptional youth. In the process, CFAR will be breaking new ground in studying the long-term effects of rationality on life outcomes using randomized controlled trials, to help us improve our material and to contribute to the body of knowledge in applied rationality. And we’ll building a real-life community of tens of thousands of students, entrepreneurs, researchers, programmers, philanthropists, and other people who are passionate about using rationality to improve their decisions for themselves and for the rest of the world.
Read more about what CFAR does, and send your friends to the new What is Rationality? page, which may now be the best short introduction to rationality available.

Featured Donor: Jesse Liptrap

Each month, our newsletter will share the views of a featured donor.Jesse Liptrap, a bioinformatician at the University of California, Berkeley was led to the SI’s broader community via a Less Wrong meetup in Spring of 2009. With a longstanding interest in transhumanism and the philosophy of moving beyond the frailty of human hardware, Jesse took quickly to the idea that applied rationality offers a set of tools for more accurately assessing the risks and opportunities of powerful emerging technologies.
Over the past few years, Jesse has climbed our Top Contributors list. He also serves as SI’s non-staff Treasurer.

Featured Summit Video

This month we are featuring a video from the 2010 Singularity Summit: Anna Salamon’s “How Much it Matters to Know What Matters: A Back of the Envelope Calculation.” Anna’s talk shows that research about the Singularity has extremely high value of information, and should therefore be supported over and above many other kinds of research currently being conducted.

$1000 Prize for Best AGI Safety Paper

This year’s AGI-12 conference will include a special track about the impacts of artificial general intelligence, called “AGI Impacts. “The Machine Intelligence Research Institute is sponsoring a $1000 prize for the best AGI safety contribution to ‘AGI-12′ and ‘AGI-Impacts 2012.’ The winner will be decided by a jury from SI and announced by Louie Helm at the end of the AGI Impacts conference.
The award is given in honor of Alan Turing, who not only discovered some of the key ideas in machine intelligence, but also grasped its importance, writing that “…it seems probable that once [human-level machine thinking] has started, it would not take long to outstrip our feeble powers… At some stage therefore we should have to expect the machines to take control…”
The prize is awarded for work that not only increases awareness of this important problem, but also makes technical progress in addressing it.
The deadline for paper submission is August 31st. For details, see the AGI-12 Call for Papers.

Ioven Fables Hired

The Machine Intelligence Research Institute has hired Ioven Fables to help with organizational operations and development. Ioven is a business person with a long dedication to big-picture understanding, philosophy, and the future.Ioven has worked for years at the business end of technology, and has B.A. in Philosophy from Boston College.

How to Get Involved

Would you like to be more involved in our work at MIRI? There are so many opportunities available, there may be a role for everyone. Here’s what you could do:

Donate. There are projects we’d like to launch, and people we’d like to hire, as soon as we can raise the funds to do so. Financial support is perhaps the clearest, most direct way to contribute to our mission.
Volunteer. We have dozens of opportunities for skilled volunteering work. Visit our volunteering page to see how you can help.
Share your expertise. Have expertise in economics, maths, computer science, cognitive science, physics, law, non-profit development, marketing, event planning, executive coaching, publishing? Please sign up as a Machine Intelligence Research Institute Volunteer Advisor!
Intern with us. Apply to work with us in Berkeley as an intern.
Apply to be a visiting fellow. Visiting fellows work with us for short periods on research projects. Gain valuable experience and work directly with our researchers!
Apply for a job. We are currently seeking research fellows, a communications director, and a grants manager. You can also apply to be a remote researcher, a job you can do from anywhere in the world!

Thank you for your continued interest and support. Don’t hesitate to get in contact with me at louie@intelligence.org.
Louie Helm
Machine Intelligence Research Institute Director of Development
In Other News

Cambridge University to Launch Centre for the Study of Existential Risk (CSER)
Humanoid Robot Works Side by Side With People
New Robot Outperforms Humans in Identifying Natural Materials by Their Textures
New Device Allows Paralyzed People to Type Words Using Only Their Thoughts
Scientists Develop Most Realistic Robot Legs Yet
Sophisticated Robotic Hand Doubles As A Human Exoskeleton

The post July 2012 Newsletter appeared first on Machine Intelligence Research Institute.

July 2012 Newsletter


2012 Summer Singularity Challenge Success!

Thanks to the effort of our donors, the 2012 Summer Singularity Challenge has been met! All $150,000 contributed will be matched dollar for dollar by our matching backers, raising a total of $300,000 to fund the Machine Intelligence Research Institute’€™s operations. We reached our goal near 6pm on July 29th.
On behalf of our staff, volunteers, and entire community, I want to personally thank everyone who donated. Your dollars make the difference.
Here’€™s to a better future for the human species.
The post 2012 Summer Singularity Challenge Success! appeared first on Machine Intelligence Research Institute.


2012 Summer Singularity Challenge

Thanks to the generosity of several major donors,† every donation to the Machine Intelligence Research Institute made now until July 31, 2012 will be matched dollar-for-dollar, up to a total of $150,000!
Donate Now!

$0

$37.5K

$75K

$112.5K

$150K

Now is your chance to double your impact while helping us raise up to $300,000 to help fund our research program and stage the upcoming Singularity Summit… which you can register for now!
Note: If you prefer to support rationality training, you are welcome to earmark your donations for “CFAR” (Center for Applied Rationality). Donations earmarked for CFAR will only be used for CFAR, and donations not earmarked for CFAR will only be used for Singularity research and outreach.

Since we published our strategic plan in August 2011, we have achieved most of the near-term goals outlined therein. Here are just a few examples:

We outlined the open research problems related to our work (Section 1.1).
We recruited several more research associates and about a dozen remote researchers (Section 1.2e).
We held our annual Singularity Summit and gained corporate sponsors for it (Section 2.1).
We made progress in decision theory (example) via LessWrong.com and our research associates (Section 2.2b).
We published How to Run a Successful Less Wrong Meetup Group (Section 2.2d).
We released pre-prints of several forthcoming research articles, including How Hard is Artificial Intelligence?, Intelligence Explosion: Evidence and Import, and The Singularity and Machine Ethics (Section 2.3).
We redesigned our primary website (Section 2.6).
We acquired $40,000/month in free Google Adwords advertising, to drive traffic to websites operated by the Machine Intelligence Research Institute (Section 2.6c).
We began publishing monthly progress reports (Section 2.9b).
We built up the Center for Applied Rationality such that it should be able to spin off from the Machine Intelligence Research Institute later this year (Section 3.1).
We created a transparency section on our website, where visitors can find our IRS 990 forms, and also several standard organizational policies, e.g. a conflict of interest policy, non-discrimination policy, etc (Section 3.2).

In the coming year, the Machine Intelligence Research Institute plans to do the following:

Hold our annual Singularity Summit, this year in San Francisco!
Spin off the Center for Applied Rationality as a separate organization focused on rationality training, so that the Machine Intelligence Research Institute can be focused more exclusively on Singularity research and outreach.
Publish additional research on AI risk and Friendly AI.
Eliezer will write an “Open Problems in Friendly AI” sequence for Less Wrong. (For news on his rationality books, see here.)
Finish Facing the Singularity and publish ebook versions of Facing the Singularity and The Sequences, 2006-2009.
And much more! For details on what we might do with additional funding, see How to Purchase AI Risk Reduction.

If you’re planning to earmark your donation to CFAR (Center for Applied Rationality), here’s a preview of what CFAR plans to do in the next year:

Develop additional lessons teaching the most important and useful parts of rationality. CFAR has already developed and tested over 18 hours of lessons so far, including classes on how to evaluate evidence using Bayesianism, how to make more accurate predictions, how to be more efficient using economics, how to use thought experiments to better understand your own motivations, and much more.
Run immersive rationality retreats to teach from our curriculum and to connect aspiring rationalists with each other. CFAR ran pilot retreats in May and June. Participants in the May retreat called it “transformative” and “astonishing,” and the average response on the survey question, “Are you glad you came? (1-10)” was a 9.4. (We don’t have the June data yet, but people were similarly enthusiastic about that one.)
Run SPARC, a camp on the advanced math of rationality for mathematically gifted high school students. CFAR has a stellar first-year class for SPARC 2012; most students admitted to the program placed in the top 50 on the USA Math Olympiad (or performed equivalently in a similar contest).
Collect longitudinal data on the effects of rationality training, to improve our curriculum and to generate promising hypotheses to test and publish, in collaboration with other researchers. CFAR has already launched a one-year randomized controlled study tracking reasoning ability and various metrics of life success, using participants in our June minicamp and a control group.
Develop apps and games about rationality, with the dual goals of (a) helping aspiring rationalists practice essential skills, and (b) making rationality fun and intriguing to a much wider audience. CFAR has two apps in beta testing: one training players to update their own beliefs the right amount after hearing other people’s beliefs, and another training players to calibrate their level of confidence in their own beliefs. CFAR is working with a developer on several more games training people to avoid cognitive biases.
And more!

We appreciate your support for our high-impact work! Donate now, and seize a better than usual chance to move our work forward. Credit card transactions are securely processed using either PayPal or Google Checkout. If you have questions about donating, please contact Louie Helm at (510) 717-1477 or louie@intelligence.org.
† $150,000 of total matching funds has been provided by Jaan Tallinn, Tomer Kagan, Alexei Andreev, and Brandon Reinhart.
The post 2012 Summer Singularity Challenge appeared first on Machine Intelligence Research Institute.

2012 Summer Singularity Challenge


Machine Intelligence Research Institute Progress Report, May 2012

Past progress reports: April 2012, March 2012, February 2012, January 2012, December 2011.
Here’s what the Machine Intelligence Research Institute did in May 2012:

How to Purchase AI Risk Reduction: Luke wrote a series of posts on how to purchase AI risk reduction, with cost estimates for many specific projects. Some projects are currently in place at SI; others can be launched if we are able to raise sufficient funding.
Research articles: Luke continued to work with about a dozen collaborators on several developing research articles, including “Responses to Catastrophic AGI Risk,” mentioned here.
Other writings: Kaj Sotala, with help from Luke and many others, published How to Run a Successful Less Wrong Meetup Group. Carl published several articles: (1) Utilitarianism, contractualism, and self-sacrifice, (2) Philosophers vs. economists on discounting, (3) Economic growth: more costly disasters, better prevention, and (4) What to eat during impact winter? Eliezer wrote Avoid Motivated Cognition. Luke posted part 2 of his dialogue with Ben Goertzel about AGI.
Ongoing long-term projects: Amy continued to work on Singularity Summit 2012. Michael continued to work on the Machine Intelligence Research Institute’s new primary website, new annual report, and new newsletter design. Louie and SI’s new executive assistant Ioven Fables are hard at work on organizational development and transparency (some of which will be apparent when the new website launches).
Center for Applied Rationality (CFAR): The CFAR team continued to make progress toward spinning off this rationality-centric organization, in keeping with SI’s strategic plan. We also held the first summer minicamp, which surpassed our expectations and was very positively received. (More details on this will be compiled later.)
Meetings with advisors, supporters, and potential researchers: As usual, various SI staff met or spoke with dozens of advisors, supporters, and collaborators about how to build the existential risk community, how to mitigate AI risk, how to improve the Machine Intelligence Research Institute’s effectiveness, and other topics.
And of course much more than is listed here!

Finally, we’d like to recognize our most active volunteers in May 2012: Matthew Fallshaw, Gerard McCusker, Frank Adamek, David Althaus, Tim Oertel, and Casey Pfluger. Thanks everyone! (And, our apologies if we forgot to name you!)
The post Machine Intelligence Research Institute Progress Report, May 2012 appeared first on Machine Intelligence Research Institute.


Machine Intelligence Research Institute Progress Report, April 2012

Past progress reports: March 2012, February 2012, January 2012, December 2011.
Here’s what the Machine Intelligence Research Institute did in April 2012:

SPARC: Several MIRI staff members are working in collaboration with SI research associate Paul Christiano and a few others to develop a rationality camp for high school students with exceptional mathematical ability (SPARC). This is related to our efforts to spin off a new rationality-focused organization, and it is also a major step forward in our efforts to locate elite young math talent that may be useful in our research efforts.
Research articles: Luke published AI Risk Bibliography 2012. He is currently developing nearly a dozen other papers with a variety of co-authors. New SI research associate Kaj Sotala has two papers forthcoming in the International Journal of Machine Consciousness: Advantages of Artificial Intelligences, Uploads, and Digital Minds and Coalescing Minds: Brain Uploading-Related Group Mind Scenarios.
Other articles: Luke published a dialogue with AGI researcher Pei Wang and several more posts in the AI Risk and Opportunity series. Luke also worked with Kaj Sotala to develop an instructional booklet for Less Wrong meetup group organizers, which is nearly complete.
Ongoing long-term projects: Amy continued to work on Singularity Summit 2012. Michael launched the new Singularity Summit website, continued to work on the Machine Intelligence Research Institute’s new primary website, new annual report, and new newsletter design. Luke uploaded several more volunteer-prepared translations of Facing the Singularity. Luke also continued to build the Machine Intelligence Research Institute’s set of remote collaborators, who are hard at work converting the Machine Intelligence Research Institute’s research articles to a new template, hunting down predictions of AI, writing literature summaries on heuristics and biases, and more.
Center for Applied Rationality (CFAR): “Rationality Group” now has a final name: the Center for Applied Rationality (CFAR). The CFAR team has been hard at work preparing for the upcoming rationality minicamps, as well as continuing to develop the overall strategy for the emerging organization.
Meetings with advisors, supporters, and potential researchers: As usual, various SI staff met or spoke with dozens of advisors, supporters, and collaborators about how to build the existential risk community, how to mitigate AI risk, how to improve the Machine Intelligence Research Institute’s effectiveness, and other topics. Quixey co-founder and CEO Liron Shapira was added as an advisor.
And of course much more than is listed here!

Finally, we’d like to recognize our most active volunteers in April 2012: Matthew Fallshaw, Gerard McCusker, Frank Adamek, David Althaus, Tim Oertel, Casey Pfluger, Paul Gentemann, and John Maxwell. Thanks everyone! (And, our apologies if we forgot to name you!)
The post Machine Intelligence Research Institute Progress Report, April 2012 appeared first on Machine Intelligence Research Institute.


Machine Intelligence Research Institute Progress Report, March 2012

Past progress reports: February 2012, January 2012, December 2011.
Fun fact of the day: The Machine Intelligence Research Institute’s research fellows and research associates have more peer-reviewed publications forthcoming in 2012 than they had published in all past years combined.
Here’s what the Machine Intelligence Research Institute did in March 2012:

Research articles: Luke and Anna released an updated draft of Intelligence Explosion: Evidence and Import, and Luke and Louie released an updated draft of The Singularity and Machine Ethics. Luke submitted an article (co-authored with Nick Bostrom) to Communications of the ACM — an article on Friendly AI. Machine Intelligence Research Institute research associate Joshua Fox released two forthcoming articles co-authored with (past Machine Intelligence Research Institute Visiting Fellow) Roman Yampolskiy: Safety Engineering for Artificial General Intelligence and Artificial General Intelligence and the Human Mental Model.
Other articles: Luke published The AI Problem, with Solutions, How to Fix Science, Muehlhauser-Goertzel Dialogue Part 1, a list of journals that may publish articles on AI risk, and the first three posts in his series AI Risk and Opportunity: A Strategic Analysis. The Machine Intelligence Research Institute paid (past Visiting Fellow) Kaj Sotala to write most of a new instructional booklet for Less Wrong meetup group organizers, which should be published in the next month or two. Eliezer continued work on his new Bayes’ Theorem tutorial and other writing projects. Carl published Using degrees of freedom to change the past for fun and profit and Are pain and pleasure equally energy efficient?
Ongoing long-term projects: Amy continued to work on Singularity Summit 2012. Michael continued to work on the Machine Intelligence Research Institute’s new primary website, new Summit website, new annual report, and new newsletter design. Louie continued to improve our accounting processes and also handled several legal and tax issues. Luke uploaded several more volunteer-prepared translations of Facing the Singularity. Luke also continued to build the Machine Intelligence Research Institute’s set of remote collaborators, who are hard at work converting the Machine Intelligence Research Institute’s research articles to a new template, hunting down predictions of AI, writing literature summaries on heuristics and biases, and more.
Rationality Group: Per our strategic plan, we will launch this new “Rationality Group” organization soon, so that the Machine Intelligence Research Institute can focus on its efforts on activities related to AI risk. In March, Rationality Group (led by Anna) contracted with Julia Galef and Michael Smith to work toward launching the organization. Eliezer continued to help Rationality Group develop and test its lessons. Rationality Group has begun offering prizes for suggesting exercises for developing rationality skills, starting with the skills of “Be Specific” and “Check Consequentialism.” Rationality Group has also announced three Minicamps on Rationality and Awesomeness, for May 11-13, June 22-24, and July 21-28. Apply now.
Meetings with advisors, supporters, and potential researchers: As usual, various SI staff met or spoke with dozens of advisors, supporters, and collaborators about how to build the existential risk community, how to mitigate AI risk, how to improve the Machine Intelligence Research Institute’s effectiveness, and other topics. This included a two-week visit by Nick Beckstead, who worked with us on AI risk reduction strategy.
And of course much more than is listed here!

Finally, we’d like to recognize our most active volunteers in March 2012: Matthew Fallshaw, Gerard McCusker, Frank Adamek, and David Althaus. Thanks everyone! (And, our apologies if we forgot to name you!)
The post Machine Intelligence Research Institute Progress Report, March 2012 appeared first on Machine Intelligence Research Institute.


Machine Intelligence Research Institute Progress Report, February 2012

Past progress reports: January 2012, December 2011.
Here’s what the Machine Intelligence Research Institute did in February 2012:

Winter fundraiser completed: Thanks to the generous contributions of our supporters, our latest winter fundraiser was a success, raising much more than our target of $100,000!
Research articles: Luke and Anna published the Singularity Summit 2011 Workshop Report and released a draft of their article Intelligence Explosion: Evidence and Import, forthcoming in Springer’s The Singularity Hypothesis. Luke also worked on an article forthcoming in Communications of the ACM.
Other articles: Luke published a continuously updated list of Forthcoming and desired articles on AI risk. For Less Wrong, Carl published Feed the Spinoff Heuristic, and Luke published My Algorithm for Beating Procrastination, A brief tutorial on preferences in AI, and Get Curious. Carl also published 4 articles on ethical careers for the 80,000 Hours blog (later posts will discuss optimal philanthropy and existential risks): How hard is it to become the Prime Minister of the United Kingdom?, Entrepreneurship: a game of poker, not roulette, Software engineering: Britain vs. Silicon Valley, and 5 ways to be misled by salary rankings.
Ongoing long-term projects: Amy continued to work on Singularity Summit 2012. Michael continued to work on the Machine Intelligence Research Institute’s new website, and uploaded all past Singularity Summit videos to YouTube. Louie continued to improve our accounting processes and also handled several legal and tax issues. Luke uploaded several volunteer-prepared translations of Facing the Singularity, and also a podcast for this online mini-book.
Grant awarded: The Machine Intelligence Research Institute awarded philosopher Rachael Briggs a $20,000 grant to write a paper on Eliezer Yudkowsky’s timeless decision theory. Two of Rachael’s papers — Distorted Reflection and Decision-Theoretic Paradoxes as Voting Paradoxes — have previously been selected as among the 10 best philosophy papers of the year by The Philosopher’s Annual.
Rationality Group: Anna and Eliezer continued to lead the development of a new rationality education organization, temporarily called “Rationality Group.” Per our strategic plan, we will launch this new organization soon, so that the Machine Intelligence Research Institute can focus on its efforts on activities related to AI risk. In February our Rationality Group team worked on curriculum development with several potential long-term hires, developed several rationality lessons which they tested (weekly) on small groups and iterated in response to feedback, spoke to advisors about how to build the organization and gather fundraising, and much more. The team also produced one example rationality lesson on sunk costs, including a presentation and exercise booklets. Note that Rationality Group is currently hiring curriculum developers, a remote executive assistant, and others, so apply here if you’re interested!
Meetings with advisors, supporters, and potential researchers: As usual, various SI staff met or spoke with dozens of advisors, supporters, and collaborators about how to build the existential risk community, how to mitigate AI risk, how to improve the Machine Intelligence Research Institute’s effectiveness, and other topics. We also met with several potential researchers to gauge their interest and abilities. Carl spent two weeks in Oxford visiting the Future of Humanity Institute and working with the researchers there.
Outsourcing: On Louie’s (sound) advice, the Machine Intelligence Research Institute is undergoing a labor transition such that most of the work we do (in hours) will eventually be performed not by our core staff but by (mostly remote) hourly contractors and volunteers, for example remote researchers, remote LaTeX workers, remote editors, and remote assistants. This shift provides numerous benefits, including (1) involving the broader community more directly in our work, (2) providing jobs for aspiring rationalists, and (3) freeing up our core staff to do the things that, due to accumulated rare expertise, only they can do.
And of course much more than is listed here!

Finally, we’d like to recognize our most active volunteers in February 2012: Brian Rabkin, Cameron Taylor, Mitchell Owen, Gerard McCusker, Alex Richard, Andrew Homan, Vincent Vu, Gabriel Sztorc, Paul Gentemann, John Maxwell, and David Althaus. Thanks everyone! (And, our apologies if we forgot to name you!)
The post Machine Intelligence Research Institute Progress Report, February 2012 appeared first on Machine Intelligence Research Institute.


2011-2012 Winter Fundraiser Completed

Thanks to our dedicated supporters, we met our goal for our 2011-2012 Winter Fundraiser. Thank you!
The fundraiser ran for 56 days, from December 27, 2011 to February 20, 2012.
We exceeded our $100K goal, raising a total of $143,048.84 from 101 individual donors.
Every donation that the Machine Intelligence Research Institute receives is powerful support for our mission — ensuring that the creation of smarter-than-human intelligence (superintelligence) benefits human society. We welcome donors contacting us to learn more about our pursuit of this mission and our continued expansion.
Keep your eye on this blog for regular progress reports from our executive director.
The post 2011-2012 Winter Fundraiser Completed appeared first on Machine Intelligence Research Institute.

2011-2012 Winter Fundraiser Completed


Machine Intelligence Research Institute Progress Report, January 2012

Past progress report: December 2011.
Here’s what the Machine Intelligence Research Institute did in January 2012:

Winter fundraiser: We continued raising funds in January, but we still have about $30,000 left to go in our winter fundraiser before the deadline of February 20th. Please support our recent efforts toward greater transparency, efficiency, and productivity by donating now!
Strategic discussions: In January we held a long and ongoing series of discussions concerning Machine Intelligence Research Institute strategy. Which scenarios are the most probable “desirable” futures for humanity, which ones can our species influence most significantly, and which ones should the Machine Intelligence Research Institute work to influence? Which tactical moves should the Machine Intelligence Research Institute make right now? How can our efforts best create synergies with other organizations focused on existential risks? These are complex questions, and in January, Machine Intelligence Research Institute staff members spent dozens of hours sharing their own evidence and arguments. (At one point, we also called upon the expertise of more than a dozen elite mathematicians in our circle.) These discussions continue today, and our opinions on strategy appear to be more unified than they were at the beginning of the month. But there is more evidence to gather and more strategic analysis to be done.
Ongoing long-term projects: Amy continued her preparations for Singularity Summit 2012. Michael Anissimov and others continued work on the Machine Intelligence Research Institute’s new website, which will feature loads of new content and a cleaner design. As part of our transparency efforts, Luke gave a second Q&A about the Machine Intelligence Research Institute, an interview at 80,000 Hours, and another interview at Singularity 1 on 1. Louie continued to work on improving our book-keeping and accounting practices. Anissimov finished thanking all donors who gave during 2011. (If you donated in 2011 and were not thanked, please contact michael@singularity.org!)
Articles: Luke and Anna continued writing “Intelligence Explosion: Evidence and Import,” and Carl continued working with Stuart Armstrong of FHI on “Arms Races and Intelligence Explosions.” Luke began adding non-English translations at Facing the Singularity, and published No God to Save Us and Value is Complex and Fragile there. Carl, with co-author Nick Bostrom, submitted a final version of “How Hard is Artificial Intelligence?” to the Journal of Consciousness Studies. For Less Wrong, Luke published What Curiosity Looks Like, Can the Chain Still Hold You?, Leveling Up in Rationality, and The Human’s Hidden Utility Function (Maybe); Anna published Urges vs. Goals. Eliezer continued work on his new Bayes Tutorial. Luke and Anna wrote a report on the workshops that followed Singularity Summit 2011, which should be published soon.
Rationality Group: Anna continued to lead the development of a new rationality education organization, temporarily called “Rationality Group.” Per our strategic plan, we will launch this new organization soon, so that the Machine Intelligence Research Institute can focus on its efforts on activities related to AI risk. In January we made one trial-hire for the new organization, and reached out to dozens of other potential team members. We also published a draft of one rationality lesson as a sample (PowerPoint slides + booklet PDFs).
New team members: Kevin Fischer of GK International joined our board of directors. We also added several new research associates: Paul Christiano, Tyrrell McAllister, János Kramar, and Mihaly Barasz (at Google Switzerland). Luke hired an executive assistant, Denise Simard. Michael Vassar officially left his role as President to work for his new company, Personalized Medicine.
Meetings with advisors, supporters, and potential researchers: As usual, various SI staff met or spoke with dozens of advisors, supporters, and collaborators about how to build the existential risk community, how to mitigate AI risk, how to improve the Machine Intelligence Research Institute’s effectiveness, and other topics. We also met with several potential researchers to gauge their interest and abilities.
Relaunched the Visiting Fellows program: In January we relaunched our Visiting Fellows program. Instead of hosting many visiting fellows at once, we will now host only 1-2 fellows at a time, for a limited duration unique to each visiting fellow. Our visiting fellow for the last week of January was Princeton philosophy undergraduate Jake Nebel. If you’re interested, please apply to our Visiting Fellows program here.
Much more: Launched a redesign of HPMoR.com, continued work in the optimal philanthropy movement, continued work on our first annual report, and much more.

Finally, we’d like to recognize our most active volunteers in January 2012: Mitchell Owen, Brian Rabkin, Huon Wilson, David Althaus, Florent Berthet, Sergio Terrero, “Lightwave,” Emile Kroeger, and Giles Edkins. Thanks everyone! (And, our apologies if we forgot to name you!)
The post Machine Intelligence Research Institute Progress Report, January 2012 appeared first on Machine Intelligence Research Institute.