# New(ish) AI control ideas

EDIT: this post is no longer being maintained, it has been replaced by [this new one](https://agentfoundations.org/item?id=601).

I recently went on a two day intense solitary "AI control retreat", with the aim of generating new ideas for making safe AI. The "retreat" format wasn't really a success ("focused uninterrupted thought" was the main gain, not "two days of solitude" - it would have been more effective in three hour sessions), but I did manage to generate a lot of new ideas. These ideas will now go before the baying bloodthirsty audience (that's you, folks) to test them for viability.

A central thread running through could be: **if you want something, you have to define it, then code it, rather than assuming you can get if for free through some other approach**.

To provide inspiration and direction to my thought process, I first listed all the easy responses that we generally give to most proposals for AI control. If someone comes up with a new/old brilliant idea for AI control, it can normally be dismissed by appealing to one of these responses:

1.  The AI is much smarter than us.
2.  It’s not well defined.
3.  The setup can be hacked.
    *   By the agent.
    *   By outsiders, including other AI.
    *   Adding restrictions encourages the AI to hack them, not obey them.
4.  The agent will resist changes.
5.  Humans can be manipulated, hacked, or seduced.
6.  The design is not stable.
    *   Under self-modification.
    *   Under subagent creation.
7.  Unrestricted search is dangerous.
8.  The agent has, or will develop, dangerous goals.

Important background ideas:

*   [Utility Indifference](https://www.fhi.ox.ac.uk/wp-content/uploads/2010-1_body.pdf)
*   [Safe value change](/lw/jxa/proper_value_learning_through_indifference/)
*   [Corrigibility](https://intelligence.org/files/Corrigibility.pdf)
*   [Reduced](/lw/a39/the_mathematics_of_reduced_impact_help_needed/) [impact AI](/lw/iyx/reduced_impact_ai_no_back_channels/)

I decided to try and attack as many of these ideas as I could, head on, and see if there was any way of turning these objections. A key concept is that we should never just expect a system to behave "nicely" by default (see eg [here](/r/discussion/lw/ltg/satisficers_undefined_behaviour/)). If we wanted that, we should define what "nicely" is, and put that in by hand.

I came up with sixteen main ideas, of varying usefulness and quality, which I will be posting in the coming weekdays in comments (the following links will go live after each post). The ones I feel most important (or most developed) are:

*   **[Anti-pascaline agent](/r/discussion/lw/lv6/antipascaline_agent/)**
*   **[Anti-restriction-hacking](/r/discussion/lw/lu9/restrictions_that_are_hard_to_hack/) ****(EDIT: I have big doubts about this approach, currently)**
*   **[Creating a satisficer](/r/discussion/lw/lv0/creating_a_satisficer/) (EDIT: I have big doubts about this approach, currently)**
*   **[Crude measures](/r/discussion/lw/ly9/crude_measures/)**
*   **[False miracles](/r/discussion/lw/ltf/false_thermodynamic_miracles/)**
*   **[Intelligence modules](/r/discussion/lw/lxm/intelligence_modules/)**
*   **[Models as definitions](/r/discussion/lw/ly2/models_as_definitions/)**
*   **Added: [Utility vs Probability: idea synthesis](/r/discussion/lw/lyh/utility_vs_probability_idea_synthesis/)**

While the less important or developed ideas are:

*   _Added: [A counterfactual and hypothetical note on AI design](/r/discussion/lw/lv1/a_counterfactual_and_hypothetical_note_on_ai/)_
*   _Added: [Acausal trade barriers](/r/discussion/lw/luy/acaucal_trade_barriers/)_
*   [_Anti-seduction_](/r/discussion/lw/lub/anti_assessorseduction/)
*   _[Closest stable alternative](/r/discussion/lw/lws/closest_stable_alternative_preferences/)_
*   _[Consistent Plato](/r/discussion/lw/lwx/consistent_plato/)_
*   [_Defining a proper satisficer_](/r/discussion/lw/luz/defining_a_proper_satisficer/)
*   _[Detecting subagents](/r/discussion/lw/lur/detecting_agents_and_subagents/)_
*   _Added: [Humans get different counterfactual](/r/discussion/lw/lxl/humans_get_different_counterfactuals/)_
*   _Added: [Indifferent vs false-friendly AIs](/r/discussion/lw/lxu/indifferent_vs_falsefriendly_ais/)_
*   [_Resource gathering and pre-corrigied agent_](/r/discussion/lw/luo/resource_gathering_and_precorriged_agents/)
*   [_Time-symmetric discount rate_](/r/discussion/lw/ltn/continuallyadjusted_discounted_preferences/)
*   _[Values at compile time](/r/discussion/lw/ly8/values_at_compile_time/)_
*   _[What I mean](/r/discussion/lw/ly7/what_i_mean/)_

Please let me know your impressions on any of these! The ideas are roughly related to each other as follows (where the arrow Y→X can mean "X depends on Y", "Y is useful for X", "X complements Y on this problem" or even "Y inspires X"):

![](http://images.lesswrong.com/t3_lt6_6.png?v=6b3d0dc39ea9c1072fb1b148295f3556)

**EDIT**: I've decided to use this post as a sort of central repository of my new ideas on AI control. So adding the following links:

Short tricks:

*   [Un-optimised vs anti-optimised](/lw/m28/unoptimised_vs_antioptimised/)
*   [Anti-Pascaline satisficer](/lw/m27/antipascaline_satisficer/)
*   [An Oracle standard trick](/lw/mao/an_oracle_standard_trick/)

High-impact from low impact:

*   [High impact from low impact](/lw/m25/high_impact_from_low_impact/)
*   [High impact from low impact, continued](/r/discussion/lw/m47/high_impact_from_low_impact_continued/)
*   [Help needed: nice AIs and presidential deaths](/lw/mbe/help_needed_nice_ais_and_presidential_deaths/)
*   [The president didn't die: failures at extending AI behaviour](/lw/mbq/the_president_didnt_die_failures_at_extending_ai/)
*   [Green Emeralds, Grue Diamonds](/lw/mbp/green_emeralds_grue_diamonds/)
*   [Grue, Bleen, and natural categories](/r/discussion/lw/mbr/grue_bleen_and_natural_categories/)
*   [Presidents, asteroids, natural categories, and reduced impact](/r/discussion/lw/mfq/presidents_asteroids_natural_categories_and/)

High impact from low impact, best advice:

*   [The AI as "best" human advisor](/r/discussion/lw/mgs/the_ai_the_best_human_advisor/)
*   [Using chatbots or set answers](/r/discussion/lw/mqc/chatbots_or_set_answers_not_wbes/)

Overall meta-thoughts:

*   [An overall schema for the friendly AI problems: self-referential convergence criteria](/r/discussion/lw/mfx/an_overall_schema_for_the_friendly_ai_problems/)
*   [The subagent problem is really hard](/r/discussion/lw/mrv/the_subagent_problem_is_really_hard/)
*   [Tackling the subagent problem: preliminary analysis](/r/discussion/lw/n62/tackling_the_subagent_problem_preliminary_analysis/)
*   [Extending the stated objectives](/lw/n63/extending_the_stated_objectives/)

Pareto-improvements to corrigible agents:

*   [Predicted corrigibility: pareto improvements](/r/discussion/lw/mmv/predicted_corrigibility_pareto_improvements/)

AIs in virtual worlds:

*   [Using the AI's output in virtual worlds: cure a fake cancer](/r/discussion/lw/mo6/ai_cure_this_fake_persons_fake_cancer/)
*   [Having an AI model itself as virtual agent in a virtual world](/r/discussion/lw/mo8/the_virtual_ai_within_its_virtual_world/)
*   [How the virtual AI controls itself](/r/discussion/lw/mqh/how_the_virtual_ai_controls_itself/)

Low importance AIs:

*   [Counterfactual agents detecting agent's influence](/lw/mri/agents_detecting_agents_counterfactual_versus/)

Wireheading:

*   [Superintelligence and wireheading](/r/discussion/lw/mwe/superintelligence_and_wireheading/)

AI honesty and testing:

*   [Question an AI to get an honest answer](/r/discussion/lw/mrz/ask_and_ye_shall_be_answered/)
*   [The Ultimate Testing Grounds](/r/discussion/lw/mx0/the_ultimate_testing_grounds/)
*   [The mathematics of the testing ground](/r/discussion/lw/mxd/ai_utilitybased_correlation/)[s](/r/discussion/lw/mxd/ai_utilitybased_correlation/)
*   [Utility, probability, and false beliefs](/r/discussion/lw/myl/utility_probability_and_false_beliefs/)