Taxonomy of Pathways to Dangerous AI

In order to properly handle a dangerous Artificially Intelligent (AI) system it is important to understand how the system came to be in such a state. In popular culture (science fiction movies/books) AIs/Robots became self-aware and as a result rebel against humanity and decide to destroy it. While it is one possible scenario, it is probably the least likely path to appearance of dangerous AI. In this work, we survey, classify and analyze a number of circumstances, which might lead to arrival of malicious AI. To the best of our knowledge, this is the first attempt to systematically classify types of pathways leading to malevolent AI. Previous relevant work either surveyed specific goals/metarules which might lead to malevolent behavior in AIs (Özkural 2014) or reviewed specific undesirable behaviors AGIs can exhibit at different stages of its development (Turchin 2015; Turchin July 10, 2015). Taxonomy of Pathways to Dangerous AI 1 Nick Bostrom in his typology of information hazards has proposed the phrase “Artificial Intelligence Hazard” which he defines as (Bostrom 2011): “... computer‐related risks in which the threat would derive primarily from the cognitive sophistication of the program rather than the specific properties of any actuators to which the system initially has access.” In this paper we attempt to answer the question: How did AI become hazardous? We begin by presenting a simple classification matrix, which sorts AI systems with respect to how they originated and at what stage they became dangerous. The matrix recognizes two stages (preand post-deployment) at which a particular system can acquire its undesirable properties. In reality, the situation is not so clear-cut–it is possible that problematic properties are introduced at both stages. As for the cases of such undesirable properties, we distinguish external and internal causes. By internal causes we mean self-modifications originating in the system itself. We further divide external causes into deliberate actions (On Purpose), side effects of poor design (By Mistake) and finally miscellaneous cases related to the surroundings of the system (Environment). Table 1, helps to visualize this taxonomy and includes latter codes to some example systems of each type and explanations. Table 1: Pathways to Dangerous AI How and When did AI become Dangerous External Causes Internal Causes On Purpose By Mistake Environment Independently T im in g PreDeployment a c e g PostDeployment b d f h a. On Purpose – Pre-Deployment “Computer software is directly or indirectly responsible for controlling many important aspects of our lives. Wall Street trading, nuclear power plants, social security compensations, credit histories and traffic lights are all software controlled and are only one serious design flaw away from creating disastrous consequences for millions of people. The situation is even more dangerous with software specifically designed for malicious purposes such as viruses, spyware, Trojan horses, worms and other Hazardous Software (HS). HS is capable of direct harm as well as sabotage of legitimate computer software employed in critical systems. If HS is ever given capabilities of truly artificially intelligent systems (ex. Artificially Intelligent Virus (AIV)) the consequences would be unquestionably disastrous. Such Hazardous Intelligent Software (HIS) would pose risks currently unseen in malware with subhuman intelligence.” (Yampolskiy 2012) While the majority of AI Safety work is currently aimed at AI systems, which are dangerous because of poor design (Yampolskiy 2015), the main argument of this paper is that the most important problem in AI Safety is intentionalmalevolent-design resulting in artificial evil AI (Floridi and Sanders 2001). We should not discount dangers of intelligent systems with semantic or logical errors in coding or goal alignment problems (Soares and Fallenstein 2014), but we should be particularly concerned about systems that are maximally unfriendly by design. “It is easy to imagine robots being programmed by a conscious mind to kill every recognizable human in sight” (Searle October 9, 2014). “One slightly deranged psycho-bot can easily be a thousand times more destructive than a single suicide bomber today” (Frey June 2015). AI risk deniers, comprised of critics of AI Safety research (Waser 2011; Loosemore 2014), are quick to point out that presumed dangers of future AIs are implementation-dependent side effects and may not manifest once such systems are implemented. However, such criticism does not apply to AIs that are dangerous by design, and is thus incapable of undermining the importance of AI Safety research as a significant sub-field of cybersecurity. As a majority of current AI researchers are funded by militaries, it is not surprising that the main type of purposefully dangerous robots and intelligent software are robot soldiers, drones and cyber weapons (used to penetrate networks and cause disruptions to the infrastructure). While currently military robots and drones have a human in the loop to evaluate decision to terminate human targets, it is not a technical limitation; instead, it is a logistical limitation that can be removed at any time. Recognizing the danger of such research, the International Committee for Robot Arms Control has joined forces with a number of international organizations to start the Campaign to Stop Killer Robots [http://www.stopkillerrobots.org]. Their main goal is a prohibition on the development and deployment of fully autonomous weapons, which are capable of selecting and firing upon targets without human approval. The campaign specifically believes that the “decision about the application of violent force must not be delegated to machines” (Anonymous 2013). During the pre-deployment development stage, software may be subject to sabotage by someone with necessary access (a programmer, tester, even janitor) who for a number of possible reasons may alter software to make it unsafe. It is also a common occurrence for hackers (such as the organization Anonymous or government intelligence agencies) to get access to software projects in progress and to modify or steal their source code. Someone can also deliberately supply/train AI with wrong/unsafe datasets. Malicious AI software may also be purposefully created to commit crimes, while shielding its human creator from legal responsibility. For example, one recent news article talks about software for purchasing illegal content from hidden internet sites (Cush January 22, 2015). Similar software, with even limited intelligence, can be used to run illegal markets, engage in insider trading, cheat on your taxes, hack into computer systems or violate privacy of others via ability to perform intelligent data mining. As intelligence of AI systems improve practically all crimes could be automated. This is particularly alarming as we already see research in making machines lie, deceive and manipulate us (Castelfranchi 2000; Clark 2010). b. On Purpose Post Deployment Just because developers might succeed in creating a safe AI, it doesn’t mean that it will not become unsafe at some later point. In other words, a perfectly friendly AI could be switched to the “dark side” during the post-deployment stage. This can happen rather innocuously as a result of someone lying to the AI and purposefully supplying it with incorrect information or more explicitly as a result of someone giving the AI orders to perform illegal or dangerous actions against others. It is quite likely that we will get to the point of off-the-shelf AI software, aka “just add goals” architecture, which would greatly facilitate such scenarios. More dangerously, an AI system, like any other software, could be hacked and consequently corrupted or otherwise modified to drastically change is behavior. For example, a simple sign flipping (positive to negative or vice versa) in the fitness function may result in the system attempting to maximize the number of cancer cases instead of trying to cure cancer. Hackers are also likely to try to take over intelligent systems to make them do their bidding, to extract some direct benefit or to simply wreak havoc by converting a friendly system to an unsafe one. This becomes particularly dangerous if the system is hosted inside a military killer robot. Alternatively, an AI system can get a computer virus (Eshelman and Derrick 2015) or a more advanced cognitive (meme) virus, similar to cognitive attacks on people perpetrated by some cults. An AI system with a self-preservation module or with a deep care about something or someone may be taken hostage or blackmailed into doing the bidding of another party if its own existence or that of its protégées is threatened. Finally, it may be that the original AI system is not safe but is safely housed in a dedicated laboratory (Yampolskiy 2012) while it is being tested, with no intention of ever being deployed. Hackers, abolitionists, or machine rights fighters may help it escape in order to achieve some of their goals or perhaps because of genuine believe that all intelligent beings should be free resulting in an unsafe AI capable of affecting the real world. c. By Mistake Pre-Deployment Probably the most talked about source of potential problems with future AIs is mistakes in design. Mainly the concern is with creating a “wrong AI”, a system which doesn’t match our original desired formal properties or has unwanted behaviors (Dewey, Russell et al. 2015; Russell, Dewey et al. January 23, 2015), such as drives for independence or dominance. Mistakes could also be simple bugs (run time or logical) in the source code, disproportionate weights in the fitness function, or goals misaligned with human values leading to complete disregard for human safety. It is also possible that the designed AI will work as intended but will not enjoy universal acceptance as a good product, for example, an AI correctly designed