# Building the AI Risk Research Community

Series: [How to Purchase AI Risk Reduction](/r/discussion/lw/cs6/how_to_purchase_ai_risk_reduction/)

[Yet](/lw/cnj/a_scholarly_ai_risk_wiki/) [another](/r/discussion/lw/cok/funding_good_research/) [way](/lw/cr0/short_primers_on_crucial_topics/) [to](/lw/cr7/proposal_for_open_problems_in_friendly_ai/) purchase reductions in AI risk may be to grow the AI risk research community.

The AI risk research community is pretty small. It currently consists of:

*   4-ish AI risk researchers at the Singularity Institute. (Eliezer is helping to launch CFAR before he goes back to AI risk research. The AI risk research done at SI right now is: about 40% of Carl, 25% of me, plus large and small fractions of various remote researchers, most significantly about 90% of [Kaj Sotala](http://www.xuenay.net/).)
*   4-ish AI risk researchers at the Future of Humanity Institute: Nick Bostrom, Anders Sandberg, Stuart Armstrong, Vincent Mueller. (This number might be wrong. It seems that Nick and Stuart are working basically full-time on AI risk right now, but I'm not sure about Anders and Vincent. Also, FHI should be hiring someone shortly with the [Tamas Research Fellowship](http://data.ox.ac.uk/doc/vacancy/102060) money. Finally, note that FHI has a broader mission than AI risk, so while they are focusing on AI risk while Nick works on his _Superintelligence_ book, they will probably return to other subjects sometime thereafter.)
*   0.6-ish AI risk researchers at [Leverage Research](http://www.leverageresearch.org/), maybe?
*   0.2-ish AI risk researchers at [GCRi](http://www.bmsis.org/gcri), maybe?
*   Nobody yet at [CSER](http://cser.org/), but maybe 1-2 people in the relatively near future?
*   Occasionally, something useful might come from mainstream [machine ethics](http://en.wikipedia.org/wiki/Machine_ethics), but they mostly aren't focused on problems of machine superintelligence (yet).
*   Small fractions of some people in the broader AI risk community, e.g. [Ben Goertzel](http://jetpress.org/v22/goertzel-pitt.htm), [David Chalmers](http://consc.net/papers/singreply.pdf), and [Wei Dai](/user/Wei_Dai/submitted/).

Obviously, a larger AI risk research community could be more productive. (It could also grow to include more people but fail to do actually _useful_ work, like so many academic disciplines. But there are ways to push such a small field in useful directions as it grows.)

So, how would one grow the AI risk research community? Here are some methods:

1.  Make it easier for AI risk researchers to do their work, by providing a well-organized platform of work from which they can build. Nick's _Superintelligence_ book will help with that. So would a [scholarly AI risk wiki](/lw/cnj/a_scholarly_ai_risk_wiki/). So do sites like [Existential-Risk.org](http://www.existential-risk.org/), [IntelligenceExplosion.com](http://intelligenceexplosion.com/), [Friendly-AI.com](http://friendly-ai.com/), [Friendly AI Research](http://friendlyai.tumblr.com/), and SI's [Singularity FAQ](http://intelligence.org/singularityfaq). So does my [AI Risk Bibliography 2012](http://intelligence.org/upload/AI%20Risk%20Bibliography%202012.pdf), and so do "basics" or "survey" articles like [Artificial Intelligence as a Positive and Negative Factor in Global Risk](http://intelligence.org/upload/artificial-intelligence-risk.pdf), [The Singularity: A Philosophical Analysis](http://www.consc.net/papers/singularityjcs.pdf), and [Intelligence Explosion: Evidence and Import](http://commonsenseatheism.com/wp-content/uploads/2012/02/Muehlhauser-Salamon-Intelligence-Explosion-Evidence-and-Import.pdf). So do helpful lists like [journals that may publish articles related to AI risk](https://docs.google.com/spreadsheet/pub?key=0AvoX2xCTgYnWdFhzanBvSllRaHROVW90eDctZVhPMFE&output=html). (If you don't think such things are useful, then you probably don't know what it's like to be a researcher trying to develop papers in the field. When I send my AI risk bibliography and my list of "journals that may publish articles related to AI risk" to AI risk researchers, I get back emails that say "thank you" with multiple exclamation points.)
2.  Run an annual conference for researchers, put out a call for papers, etc. This brings researchers together and creates a community. The [AGI conference series](http://agi-conf.org/) did this for AGI. The new [AGI Impacts sessions](http://agi-conf.org/2012/call-for-papers/) at AGI-12 could potentially be grown into an AGI Impacts conference series that would effectively be an AI Risk conference series.
3.  _Maybe_ launch a journal for AI risk papers. Like a conference, this can to some degree bring the community closer. It can also provide a place to publish articles that don't fit within the scope of any other existing journals. I say "maybe" on this one because it can be costly to run a journal _well_, and there are [plenty of journals](https://docs.google.com/spreadsheet/pub?key=0AvoX2xCTgYnWdFhzanBvSllRaHROVW90eDctZVhPMFE&output=html) already that will publish papers on AI risk.
4.  Give out [grants](/r/discussion/lw/cok/funding_good_research/) for AI risk research.

Here's just _one example_ of what SI is _currently_ doing to help grow the AI risk research community.

Writing "**Responses to Catastrophic AGI Risk**": A journal-bound summary of the AI risk problem, and a taxonomy of the societal proposals (e.g. denial of the risk, no action, legal and economic controls, differential technological development) and AI design proposals (e.g. AI confinement, chaining, Oracle AI, FAI) that have been made.

_Estimated final cost_: $5,000 for Kaj's time, $500 for other remote research, 30 hours of Luke's time.

Now, here's a list of things SI _could_ be doing to help grow the AI risk research community:

*   Creating a [scholarly AI risk wiki](/lw/cnj/a_scholarly_ai_risk_wiki/). Estimated cost: 1,920 hours of SI staff time (over two years), $384,000 for remote researchers and writers, and $30,000 for wiki design, development, and hosting costs.
*   Helping to grow the AGI Impacts sessions at AGI-12 into an AGI Impacts conference. (No cost estimate yet.)
*   Writing _[Open Problems in Friendly AI](/lw/cr7/proposal_for_open_problems_in_friendly_ai/)_. Estimated cost: 2 months of Eliezer's time, 250 hours of Luke's time, $40,000 for internal and external researchers.
*   Writing more "basics" and "survey" articles on AI risk topics.
*   Giving out [grants](/r/discussion/lw/cok/funding_good_research/) for AI risk research.