# "Can we know what to do about AI?": An Introduction

I'm currently working on a research project for MIRI, and I would welcome feedback on my research as I proceed. In this post, I describe the project.

As a part of an effort to [steel-man](/lw/85h/better_disagreement/) objections to MIRI's mission, MIRI Executive Director Luke Muehlhauser has asked me to develop the following objection:

> "Even if AI is somewhat likely to arrive during the latter half of this century, how on earth can we know what to do about it _now_, so far in advance?"

In Luke's initial email to me, he wrote:

> I think there are plausibly [many weak arguments](/lw/hmb/many_weak_arguments_vs_one_relatively_strong/) and historical examples suggesting that P: "it's very hard to nudge specific distant events in a positive direction through highly targeted actions or policies undertaken today." Targeted actions might have no lasting effect, or they might completely miss their mark, or they might backfire.
> 
> If P is true, this would weigh against the view that a highly targeted intervention today (e.g. Yudkowsky's Friendly AI math research) is likely to positively affect the future creation of AI, and might instead weigh in favor of the view that all we can do about [AGI](http://en.wikipedia.org/wiki/Strong_AI#Artificial_General_Intelligence_research) from this distance is to engage in broad interventions likely to improve our odds of wisely handling future crises in general — e.g. improving decision-making institutions, spreading rationality, etc.
> 
> I'm interested in abstract arguments for P, but I'm even more interested in historical data. What can we learn from seemingly analogous cases, and are those cases analogous in the relevant ways? What sorts of [counterfactual history](http://www.amazon.com/Unmaking-West-What-If-Scenarios-Rewrite/dp/0472031430/) can we do to clarify our picture?

Luke and I brainstormed a list of potential historical examples of people predicting the future 10+ years out, and using the predictions to inform their actions. We came up with the following potential examples, which I've listed in chronological order by approximate year:

*   1896: Svante Arrhenius's [prediction of anthropogenic climate change](http://en.wikipedia.org/wiki/Svante_Arrhenius#Greenhouse_effect).
*   1935: Leo Szilard's ~1935 [attempts to keep his patent of the atomic bomb secret from Germany](http://www.cccoe.k12.ca.us/abomb/popups/patent.htm).
*   1950-1980: Efforts to win the [Cold War](http://en.wikipedia.org/wiki/Cold_War) decades later, such as increasing education for gifted children.
*   1960: Norbert Weiner [highlighting the dangers of artificial intelligence](http://www.itu.dk/people/cmmm/Wiener.pdf).
*   1972: The circle of ideas and actions around the [The Limits to Growth](http://en.wikipedia.org/wiki/The_Limits_to_Growth), a book about the consequences of unchecked population growth and economic growth.
*   1975: The [WASH-1400 reactor safety study](http://en.wikipedia.org/wiki/WASH-1400), which attempted to assess the risks associated with nuclear reactors. 
*   1975: The [Asilomar Conference on Recombinant DNA](http://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA), which set up guidelines to ensure the safety of recombinant DNA technology.
*   1978: China's [one-child policy](https://en.wikipedia.org/wiki/One-child_policy) to reduce population growth.
*   1980: The [Ford Foundation setting up a policy think](http://www.givewell.org/files/conversations/Lant%20Pritchet%2006-18-12%20final%20for%20upload.pdf) in India that helped India recover from [its 1991 financial crisis](http://en.wikipedia.org/wiki/1991_India_economic_crisis) 
*   1988: [Early climate change mitigation efforts](http://www.ipcc.ch/organization/organization_history.shtml)
*   1992+: [Asteroid strike deflection efforts](http://en.wikipedia.org/wiki/Asteroid_impact_avoidance#History_of_government_mandates) 
*   ???: Possible deliberate long term efforts to produce revolutionary scientific technologies.
*   ????: Long term computer security research

In addition, we selected

*   [The Signal and the Noise: Why So Many Predictions Fail — but Some Don't](http://www.amazon.com/The-Signal-Noise-Many-Predictions/dp/159420411X) by Nate Silver
*   [Expert Political Judgment: How Good Is It? How Can We Know?](http://www.amazon.com/books/dp/0691128715) by Philip Tetlock

as background reading.

  

I would greatly appreciate any ideas from the Less Wrong community concerning potential historical examples and relevant background reading.

  

Over the coming weeks, I'll be making a series of discussion board posts on Less Wrong reporting on my findings, and linking these posts here.