Abstract: We present nine facets for the analysis of the past and future evolution of AI. Each facet has also a set of edges that can summarise different trends and contours in AI. With them, we first conduct a quantitative analysis using the information from two decades of AAAI/IJCAI conferences and around 50 years of documents from AI topics, an official database from the AAAI, illustrated by several plots. We then perform a qualitative analysis using the facets and edges, locating AI systems in the intelligence landscape and the discipline as a whole. This analytical framework provides a more structured and systematic way of looking at the shape and boundaries of AI.

Fernando Martínez-Plumed
Bao Sheng Loe
Peter Flach
Seán Ó H Éigeartaigh
Karina Vold
José Hernández-Orallo

The Facets of Artificial Intelligence: A Framework to Track the Evolution of AI

Introduction "What is AI?" has been a common question from the inception of the discipline in the 1950s  [McCarthy et al., 2006; Moor, 2006; Solomonoff, 2017]  until the end of the last century  [Lehman-Wilzig, 1981; Fetzer, 1990; McCarthy, 1998 ]. In the twenty-first century, the discipline is not only welldeveloped but it is often said that this is the age of AI  [Mc-Corduck, 2004] . AI is set to pervade and transform every aspect of life. In a way, this is no different from what computer science has already been doing, creating computerised, digital and virtual versions of almost everything, with AI now introducing new adjectives such as 'intelligent', 'smart' and 'cognitive' to almost every process or gadget, from medical diagnosis to personal assistants. This expected expansion and intertwining with every other research discipline and aspect of life is pushing the contours of AI in many directions. However, unlike computer science, which is based on wellestablished models of computation that integrate hardware and software, AI has evolved with a more fluid definition, primarily because of our varied conceptions of intelligence. When looking at the definitions of (artificial) intelligence (see, e.g.,  [Legg and Hutter, 2007] , for a compendium), we see that definitions can be categorised according to two distinct dimensions, following  [Russell and Norvig, 2009] . First, we can characterise intelligence in terms of "thinking" (process-oriented) or "acting" (goal-oriented). Second, we can characterise intelligence taking humans as a reference or looking for a more abstract or universal reference (such as rationality). These two dimensions are summarised in Table  1 . Not all definitions can be clearly classified according to this table. For instance, Minsky's famous definition of AI, as the "science of making machines capable of performing tasks that would require intelligence if done by  [humans] "  [Minsky, 1968] , uses humans as a reference, so it would be located on the left of the table, but still somewhere between the top and bottom part of the table as it focuses on processes (i.e., thinking) for humans and tasks (i.e., acting) for machines This suggests that we can look at these dimensions in a more nuanced, continuous way, as facets rather than discrete categories, to look at the evolution of AI as a discipline. For instance, has AI been more focused towards "thinking" (and hence processes or techniques) or more focused towards "acting' (and hence tasks and applications)? Is it more or less influenced by human intelligence (using human processes and the tasks humans can actually solve) or is the discipline going in a direction of a more abstract or universal characterisation? In this paper, we will develop these and other dimensions into facets to characterise the object of AI, i.e., the AI systems and services, such as generality, location and embodiment, and the subject of AI, such as paradigms, actors, character and nature, as a discipline. With these facets we will perform a qualitative and quantitative analysis of the evolution of AI. We will base our analysis on evidence as much as possible, looking at data from scientific venues, reports, surveys and other sources. At the end of the paper, we will take a more principled stance and we will discuss how to extrapolate these Thinking Humanly Thinking Rationally Acting Humanly Acting Rationally Table  1 : Categories of (artificial) intelligence definitions, according to the two binary dimensions in  [Russell and Norvig, 2009, Fig. 1.1] . trends or identify what kinds of criteria are necessary to establish AI contours that can be more stable and useful. The contributions of this paper are a set of well-structured facets to analyse the location and contours of AI. This is complemented by substantial evidence collected from several sources, such as 20 years of the IJCAI and AAAI proceedings and the whole AI topics database 1 . All the datasets, plots and code to scrape and process this data are made publicly available (see footnote 7). There are several reasons why using clear criteria such as the facets and edges are helpful for defining AI, how it has evolved and what it is likely to be in the future. First, internally, the research community needs clear criteria to determine what is in or out of scope, for conferences, journals, funding and hiring. Second, policy makers are considering ways of regulating AI, but wrestle with the problem of defining what systems are actually AI. Third, different terms are being used to capture the same subparts or forms of AI, such as artificial general intelligence (AGI), machine intelligence, cognitive computing, computational intelligence, soft computing, etc. Without a clear structure about AI, it is difficult to determine whether they are part, they overlap or they are simply redundant. Fourth, some areas, such as machine learning, are taking a more relevant role in AI, but they are also intertwined with areas such as statistics, optimisation or probability theory, which were not always considered near the contours of AI. Fifth, along the history of AI there has been rise and decline of interest and research towards the different AI paradigms, techniques and approaches.
 Background Disciplines are commonly analysed historically, and AI is not an exception. A history of a discipline usually emphasises the problems, progress and prospects, but not necessarily delineates its contours. For instance, there are several excellent accounts of the history of AI  [McCorduck, 2004; Buchanan, 2005; Nilsson, 2009; Boden, 2016] . Some of them cover AI from a philosophical perspective. However, given the implications of AI for the interpretation of the human mind, the analysis of AI as a discipline (as usual from the viewpoint of philosophy -or methodology-of science) is not commonly done in terms of its external and internal contours. As a result, several key questions remain: what are the criteria to recognise that an entity is part of AI? Furthermore, how can we recognise the internal subdisciplines in AI? One possible approach to this is to determine the nature and contours of AI by its common use. While this may be a good approach for evaluating progress as a whole or for a few benchmarks 2 , the analysis of subdisciplines by their popularity may be prone to many terminological confusions and many vested interests, with the risk of having characterisations that are very volatile, such as big data. Bibliometric approaches are a common tool to analyse disciplines, but the focus is usually put on impacts per author, venue, location or institution. Sometimes, bibliometrics studies disciplines and subdisciplines. For instance, Scimago pro-  vides a way of looking at the "shape of science" 3 where one can locate AI and some of its subareas in terms of their internal and external relations. Provided with a set of tags, bibliometrics can study how frequent several tags are in published papers (including titles, keywords or abstracts). For instance,  [Niu et al., 2016]  includes a thorough historical analysis of publications in about 20 relevant journals in AI (but not conferences or open journals such as JAIR) from 1990 to 2014. The number of publications is shown by 5-year periods. The keywords used are shown in Table  2 . As we can see in the list, the keywords include terms for disciplines and subdisciplines, techniques, and application areas, from which the authors distinguish "methods and models" and "applications". As the analysis is limited to most frequent areas, it excludes important subfields of AI (e.g., "planning") and some assignments are vague, with keywords such as "design", "identification" or "prediction". Still, we see that this analysis is aligned with the first facet mentioned in the introduction, of whether the discipline is characterised by their techniques or their applications. This is not surprising, as it is a typical categorisation of disciplines according to their techniques and applications, especially in engineering. A proper cataloguing effort (e.g., as done by ACM and other associations for computing) would be an option, but AI is too dynamic to allow for a stable set of terms for a long period. In the end, instead of a reactive approach focusing on the trends, a more proactive stance towards the recognition of the subdisciplines can be seen as a duty for scientific associations, editorial boards and program chairs. Accordingly, AI researchers should configure the landscape of AI 4 and determine what is relevant or off-topic for a venue. They can also determine what the subareas are, so that proper reviewers and sessions are allocated depending on the importance of each area. It is unusual, however, to conduct a more systematic analysis of how these choices are made. Three remarkable exceptions are  [Shah et al., 2017] , where area relevance is only examined on passing,  [Fast and Horvitz, 2017]       separated by their area (topic "keyword", Table  3 ). Areas are mutually exclusive and sum up to 100% per year. General (dashed black line) tendencies and standard errors (bands) are shown for both data series together.   by area (topic "keyword", Table  3 ). Legend as in Figure  1  (note that the x-axis is different, with a much wider time span here). • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •
 Planning methods • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • •• • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • lanning earch & constraint the authors focus on views expressed about AI in the New York Times over a 30-year period in terms of public concerns as well as optimism, and  [Moran et al., 2014] , which focuses on the venue keywords, including a cluster analysis on the 2013 keyword set and a new series of keywords, which were adapted by AAAI2014. Table  3  shows an integration of their selection with the keywords of AAAI2014.
 Historical Data: Analysis by Keywords We used the keywords in Table  3  for a first quantitative analysis, using data obtained from two significant sources: • AAAI/IJCAI conferences . We obtained data of all the accepted papers for these two conferences from DBLP 5 . This database represents information about computer science (and thus AI) that comes mostly from their researchers (conference proceedings and journals). • AI topics documents . AI topics 6 is an archive kept by the AAAI, containing a variety of documents (e.g. news, blog entries, conferences, journals and other repositories) that are collected automatically with NewsFinder  [Buchanan et al., 2013] . With a mapping approach between terms and the categories in Table  3  as a representative list of subareas in AI, we summarised trends in a series of plots. For the AAAI/IJCAI conferences data, we used the keywords appearing in the proceedings, and for AI topics we used the tags (substrings appearing in titles, abstracts and topics). Regarding conferences, other major venues (such as ECAI, ICML or NIPS) were not included due to the lack of keyword information in 5 http://dblp.uni-trier.de/ 6 https://aitopics.org/misc/about. DBLP. Still, AAAI and IJCAI can be considered to be a representative basis for analysing AI trends proper. Figure  1  shows the evolution of the areas in Table  3  for the past 20 years using the AAAI/IJCAI data. Document counts are normalised to sum up to 100% per year. Standard errors are also shown, where in some cases the thickness of the band is very variable (e.g., "Human & AI" and "Game Theory") due to data sparsity, or it cannot be computed due to insufficient data available (e.g., "Heuristic search & optimisation"). Figure  2  shows a similar evolution based on the AI topics data. If we visually compare the same keywords between the plots found in Figures  1 and 2  (taking into account that axes are different), we see that both sources are rather consistent and highlight a few clear trends. For example, the categories heuristic search & optimisation and knowledge representation show a decreasing trend. Other categories seemed to have peaks: cognitive modeling in the 1990s, strongly associated with the emergence of several cognitive architectures (e.g.,  ACT-R [Anderson et al., 1997] , EPIC  [Kieras and Meyer, 1997]  or SOAR  [Newell, 1994] ); planning methods around 2000, possibly due to the introduction of the first method for solving POMDP offline  [Kaelbling et al., 1998] , jumpstarting its widespread use in robotics and automated planning; and multiagent systems around 2010, when they were successfully applied to real world scenarios (e.g., autonomous vehicles  [How et al., 2008] ) and graphical applications (e.g., video games  [Hagelbäck and Johansson, 2008] ). Some others had valleys, such as natural language processing (NLP) around 2000, showing a paradigm shift (from deterministic phrase structure analysis before the 1990s to more probabilistic NLP methods). Long Short-Term Memory (LSTM) recurrent neural net (RNN) models  [Hochreiter and Schmidhuber, 1997]  have nowadays found rapid adoption due to an increase in computational capacity in the 2010s. Only machine learning had a clear increasing trend over time found in both sources, where the steepest slope is located in the 2010s and reveals the current relevance and attention, mostly caused by deep learning and reinforcement learning. Overall, this is in agreement with the general perception about the field. While the plots show a confirmatory evidence of the changing trend in AI, using area keywords may only present one side of AI. The choice and relevance may be highly dependent on many factors such as the state of the art, the sociological perceptions or a moving target. Thus, we question whether it is possible to investigate the historical evoluation in a richer and more systematic manner.
 A Faceted Analysis of the Evolution of AI The two dimensions in Table  1  can be used as a basis for a different arrangement of tags and categories in our historical analysis. However, instead of taking a dichotomous and monolithic perspective about dimensions, we consider the use of facets. This is motivated by realising that when choosing any criterion for analysis, there is always a gradation, and sometimes this gradation does not follow a straight line between two extremes, but an area among two or more edges, like a polygon. Hence, we use the term facet for this surface, and the term edge for each of its boundaries. To illustrate this, let us develop the first two dimensions into facets: • F1: The functionality facet (with edges 'techniques', 'applications' and 'tasks') analyses the functionality of AI systems, such as knowledge representation, reasoning, learning, communication, perception, action, etc. The processes of AI systems fall at the edge technique, which relates to how AI systems "think". We can also characterise AI systems in terms of their behaviour (how they "act"), leading to two different edges: the tasks they solve and the application areas they are used in. This facet can then be imagined as a triangle. • F2: The referent facet (with edges 'human' and 'universal') distinguishes definitions or conceptions of AI systems that go from an anthropocentric view to a more universal (theoretical) perspective. At the "human" edge, AI could be characterised by being able to solve all the tasks or by implementing all the intelligent processes humans are able to do. This view would be closely related to what is known as human-like AI (see, e.g.,  [Lake et al., 2017] ) or the view of AI as pursuing human automation  [Frey and Osborne, 2017; Brynjolfsson and Mitchell, 2017] . On the other hand, if AI is characterised at the "universal" edge, it would be defined in terms of a more theoretical set of problems or by implementing some abstract processes. Using the AI topics data, for the functionality facet we can look at several categories for each of the three edges: techniques, applications and tasks. We made tag-category mappings for about 30 techniques, 20 applications and 30 tasks. A selection of categories for each edge is shown here, although all the plots and results are presented in a separate link  7  . Fig 3  7  Data, code and plots, and an online R Shiny app are publicly (a) Top 6 techniques (in %) in AI topics   For the referent facet, we can also explore its edges. Fig  4  (a)  shows six groups that are most associated with the edge 'human'. The only relatively clear trends are a fall in psychology and an increase in security, privacy and safety. While the former may be due to the decline of cognitive modelling in general, the latter clearly underlines the rise of AI ethical and privacy issues already pointed out by several governments and agencies (e.g.,  [White House, 2016] ), as well as the appearance of new regulations (e.g., GPDR  [EU Regulation, 2016] ), ultimately triggered by public opinion and a concern in the field itself. associated with the edge 'universal'. The representations are rather flat for this edge, with very variable coverage over the whole period under analysis, but no clear trends. (a) Top 6 on the human edge (in %) in AI topics • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • •• • •• • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • •• • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • •• • • • • •• • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • nowledge management • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • • • • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • •• •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • •• • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • nterf & human− • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • earning theory heoretical appr utomata abs mach omputability complex orrect , opt verif  Both the functionality and the referent facets are usually linked to a large number of terms and keywords, and they give a broad overview that is familiar to AI researchers. However, are these two facets sufficient to characterise the object and subject of AI? If we think of the elements of AI -its systems and services (the object)-, we need some other relevant facets to characterise them, such as their generality, location and embodiment. If we think of AI as a discipline (the subject), we need to consider its paradigms, actors, character and nature. Let us describe these new seven facets: • F3: The generality facet (with edges 'specific' and 'general') considers whether AI is concerned with the creation of specific systems (i.e., solving one particular task) or the development of systems (and techniques) that solve (or can be applied to) a wide range of tasks. This dichotomy is usually referred to as narrow versus general AI. • F4: The location facet (with edges 'integration' and 'distribution') depicts where a system starts and ends. The very notion of centralised or distributed decision making has been at the definitional level of agent for a few decades, as well as the related notion of autonomy  [Luck, 2017] . Also, there is an increasingly blurred boundary on where human cognition ends and where it is assisted, extended or orthosed by AI  [Ford et al., 2015]  (and vice versa, through human computation  [Quinn and Bederson, 2011] ). • F5: The embodiment facet  (with edges 'physical' and 'virtual')  distinguishes whether the AI system is linked to a body or physical equipment or, on the contrary, is basically of algorithmic character, installed on de-vices, working on the cloud or migrating between different platforms, usually dealing with elements in a digital world. • F6: The paradigm facet (with edges 'discrete  ', 'continuous' and 'probabilistic')  distinguishes the underlying approaches behind many principles and tools of AI. At the discrete edge, we see those problems and methods seen in a combinatorial way, where a logical or evolutionary process combines or applies operators. At the continuous edge, we see quantitative optimisation problems tackled with gradient descent, kernels and matrix operations. And, on the probabilistic edge, problems are seen in a probabilistic, stochastic or statistical view. • F7: The actor facet (with edges 'academia', 'industry', 'government' and 'independent') identifies who are the driving forces behind AI as a discipline. This facet considers who is most relevant in -and ultimately steeringthe discipline according to its current challenges, regulations and potential future advances. • F8: The character facet (with edges 'empirical' and 'theoretical') determines whether AI is guided by experiments, like other empirical disciplines, or whether it is of a more theoretical nature. Note that this facet is different from the referent facet (e.g., a non-anthropocentric view can be very experimental). • F9: The nature facet (with edges 'technology', 'engineering', 'science' and 'philosophy') describes AI according to what kind of discipline it is. This loosely corresponds respectively to whether it creates devices and products, solves problems, answers/asks questions. We do not show individual categories for these facets due to space limitations. Instead, we will look at the aggregation of categories by edges, assuming the edges are exclusive (so their share is always 100%). In a way, this can be understood as a non-monolithic view of polarities in sentiment analysis. Fig.  5  shows this share for the nine facets, where the data has been smoothed with a moving average filter. We see trends for many of them: an increase of the relevance of applications for F1 and the focus on more specific systems for F3 (the number, but also the diversification, of applications may explain this), more virtual systems for F5 (with the appearance of many new AI experimentation and evaluation platforms  [Hernández-Orallo et al., 2017] ), more continuous paradigms for F6 (given the success of deep learning and other methods based on this paradigm), more industry for F7 (and mostly at the cost of academia), a more empirical character for F8 and a view of the discipline in a more technological way for F9 (which may be one possible reason for the current concern in AI safety and governance, usually harder to handle when the engineering and scientific perspectives are weak). Overall, we can also see that the trends in some facets, especially the shifts from the 2000s, may be strongly related (facets F1, F3, F7 and F9). One important insight that we can gain from the visual output shown in all these plots is that some trends peak, while some others are cyclic. Consequently, the plots have explanatory and confirmatory value about the relevance of different areas and perspectives of how AI is defined. However, we have to be cautious and not use them for forecasting. In fact, from a governance perspective, there is a known bias towards reinforcing the dominant view (the winner takes all) or even confounding the part with the whole (e.g., deep learning with machine learning, and machine learning with AI). Therefore, this quantitative analysis could be used to inform compensatory actions with some less popular but important areas, so that the discipline is ready when dominant paradigms shift or some of their pathways plateau.
 Qualitative Analysis The facets are also useful to analyse AI in a more qualitative manner. For example, F1 to F5 are based on the object of AI and can characterise either particular AI systems or AI as a discipline. On the other hand, F6 to F9 focus on the subject, being more methodological for the discipline as a whole. Let us examine the first five facets from the perspective of the AI systems and components. In the end, if we set the goal of AI as building and understanding AI systems then the contours of the discipline will be clear as far as we have a clear notion of what an AI system is. Nonetheless, the question remains: how can we characterise this in a stable way for the years to come? Looking forward, the referent facet (F2) seems to become very relevant. We are aware of AI systems already exceeding some human capabilities and this will continue to be so during the century. Considering human intelligence as a goal has been a driving force (and will continue to be so in the near future), but it is rather short-sighted. Instead, placing AI as exploring a more universal 'intelligence landscape' is not only more inclusive about what AI is (systems solving tasks humans cannot solve would still be part of AI), but represents a Copernican view where humans are no longer at the centre (see Figure  6 ). For instance, the terms human-level machine intelligence or human-level artificial intelligence have many issues (is it for all tasks?, what is an average human?, how to extrapolate beyond human level?). Actually, the term -albeit not the definition-is usually replaced by high-level machine intelligence  [Müller and Bostrom, ]  or simply AGI (which should rather refer to facet F3). Of course, even if humans are not taken as a reference, they still have to be put at the central position due to the interaction and impact of AI on them. But this is usually referred to as human-centred artificial intelligence. For instance, applications must be prioritised according to human needs, and for many of these applications we want more 'human-like' AI  [Lake et al., 2017; Marcus, 2018] , more 'human-beneficial' AI (i.e., safer and taking the values of humans into account  [Russell et al., 2015; Amodei et al., 2016] ) and also more 'human-ethical' AI  [Goldsmith and Burton, 2017] . Note that we do not want to replicate certain behaviours humans display (e.g., gender bias) in our AI systems. In many cases, it is not that AI should automate all tasks humans do, or perform better than humans, but AI should also do other tasks or perform very differently. Consequently, there should be no reason why some tasks are excluded from AI when they are solved in a different way. This -known as the 'AI effect'  [McCorduck, 2004] -is partly motivated because of the specialisation of the solution, as we will discuss below. The bottom view of Figure  6  opens the contours of AI but requires the definition of the intelligence landscape. That takes us back to the range around functionality facet and Table 1 (right), between thinking and acting. We have seen in the quantitative analysis that techniques, tasks and applications are volatile, so any enumeration of them is going to be incomplete. An alternative view based on skills and abilities can endure changes much better  [Hernández-Orallo, 2016; . For instance, perception, learning and planning have been consubstantial in AI. Systems having some of these skills are recognised as AI without further information about the techniques or the particular tasks the elements are applied to, in the same way we do for non-human animals. This is actually the true core of facet F1. But skills and abilities can only be characterised by looking at facets F3, F4 and F5. For instance, having n different systems for navigating n different buildings would not convince us that AI systems today have very good navigation skills (F3, generality). They would just be narrow systems not really having the skill. Similarly, the system might actually be controlled by many subsystems on the Internet, using real-time data from sensors around the building and even from the devices the humans in the building are using (F4, location). Finally, it is not the same to physically navigate a real building than a virtual scenario (F5, embodiment). In fact, we are heading towards the direction of AI services instead of AI systems, which is typically known as cognition as a service  [Spohrer and Banavar, 2015] . In cognitive services research, the focus is on bringing down the overall cost and increasing general performance. However, certain considerations must be taken into account, especially in the context of automation  [Frey and Osborne, 2017; Brynjolfsson and Mitchell, 2017] . For example, what is the difference between full automation and efficient semiautomation? Furthermore, the blurred line between work contributed by AI systems and humans makes it even harder to differentiate how performance can be attributed realistically. Also, some tasks can be finally automated in ways where AI plays a secondary role, by changes in the logical or physical configuration. For instance, an intelligent robot can be helped by a proper design of the body, usually referred to as morphological intelligence  [Winfield, 2017] . Interestingly, the phenomenon of AI being more powerful because of the use of human data or human computation is mirrored by the extended mind  [Clark and Chalmers, 1998 ]. The human mind is viewed as incorporating all of its mind tools, from pen and paper to cognitive assistants. The contours of where the human mind ends, and whether the surrounding tools and devices are included, is also important to determine where the AI system ends too, and who really provides the services. Hence, the facets F3 to F5 will become more relevant and sophisticated in the future. Perhaps instead of autonomy, we will need to trace how much each part contributes to the whole ability of the whole system or service. Finally, as we mentioned above, the four last facets have a more methodological stance. For instance, it seems irrelevant what paradigm from facet F6 is most important at the moment. For instance, arcade games (such as the Atari ALE benchmark) can now be played with relatively good performance using deep reinforcement learning  [Mnih et al., 2015] , evolutionary programming  [Kelly and Heywood, 2017]  or neuroevolution  [Hausknecht et al., 2014] . However, if it is the case that more and more AI systems are using continuous, gradient-descent, approaches rather than more discrete, combinatorial, approaches, this may have an impact on the contours with neighbouring disciplines. AI would be clashing more often with algebra, optimisation and statistics (or even physics), and would be pulled away from the traditional logic and discrete mathematics (or even evolution). Similarly, whether approaches are more empirical or theoretical, according to facet F8, is not necessarily a reason to be considered more or less AI, but can still affect the perception of the field, and the boundaries with some other disciplines. Facets F7 (actors) and F9 (nature) are related, as it is mostly academia that cares for scientific and philosophical questions about AI. If we look at Figure  6  again, all actors will be interested in building systems and services that cover the intelligence landscape for an increasing number of applications, considering AI as a technology and focusing on engineering. But it is mostly (or only) academia which is interested in what the intelligence landscape looks like, the evaluation of where systems are located in this space and what the implications are while covering this landscape. These and some other ar-eas will have to be prioritised by academia (and government funding) in order to have more vision and governance within the field of AI.
 Conclusion Artificial intelligence, as any other scientific discipline, is partly a social phenomenon, and its definition and contour are highly influenced by its actors and stakeholders. We will always need to track and update the field in many ways, from the use of self-reported questionnaires 8 to data from the venues and news related to AI. In this paper, we presented a series of facets and associated edges to analyse the historical evolution of AI, and gathered some insight into its future. The data of venues and AI repositories were useful for quantitative analyses. Moreover, the data and the mapping between tags and categories are publicly provided so that others can apply the same faceted framework to other sources of data about AI. Finally, the facets represent a framework to discuss, in a more qualitative way, the past, present and future of AI, and its relation to other disciplines. Figure 1 : 1 Figure 1: Papers published in AAAI (red dots) and IJCAI (blue triangles) conferences separated by their area (topic "keyword", Table3). Areas are mutually exclusive and sum up to 100% per year. General (dashed black line) tendencies and standard errors (bands) are shown for both data series together.
 Figure 2 : 2 Figure 2: Documents included in AI topics by area (topic "keyword", Table3). Legend as in Figure1(note that the x-axis is different, with a much wider time span here).
 Top 6 applications (in %) in AI topics
 Top 3 tasks (in %) in AI topics.
 Figure 3 : 3 Figure 3: Functionality facet.(a) shows the techniques with the highest peak in terms of percentage. We see trends that are consistent with machine learning taking more relevance, with roughly 4-5 times more coverage over the 50 year period, and along the lines of what was seen in the previous section. Fig3 (b)  shows the six most popular application areas. Here the trends are flatter, although some slight trends can be seen for health & medicine and personal assistants, in line with the insights reported in related work [Fast and Horvitz, 2017] .Finally, Fig 3 (c)  shows the relevance of chess, especially in the 1980s and 1990s, and their decrease after Deep Blue beat Kasparov. Poker and Robocup are examples of tasks that became representative over a small period of time because of either an algorithmic breakthrough or the popularity of their competitions.
 Fig 4 (b)  shows six groups that are most available at: https://evoai.shinyapps.io/evoai/.
 Top 6 on the universal edge (in %) in AI topics.
 Figure 4 : 4 Figure 4: Referent facet.
 Figure 5 : 5 Figure 5: Share of all edges per facet in AI topics (facets F1-F9, ordered left to right, top to bottom).
 Figure 6 : 6 Figure 6: Top: Anthropocentric AI. Bottom: AI in the 'intelligence landscape'.
 Table 3 : 3 A selection of keywords as an intersection of the two last columns of[Moran et al., 2014, Tab. 2].
			 Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
			 https://aitopics.org. 2 See, e.g., http://aiindex.org/ and https://www.eff.org/ai/metrics.
			 http://www.scimagojr.com/shapeofscience/. 4 https://aaai.org/Magazine/ailandscape.php.Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
			  For example, [Müller and Bostrom, ]  or http://agisi.org/Survey intelligence.html.Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence


Abstract: Advances in robotics technology are causing major changes in manufacturing, transportation, medicine, and a number of other sectors. While many of these changes are beneficial, there will inevitably be some harms. Who or what is liable when a robot causes harm? This paper addresses how liability law can and should account for robots, including robots that exist today and robots that potentially could be built at some point in the near or distant future. Already, robots have been implicated in a variety of harms. However, current and near-future robots pose no significant challenge for liability law: they can be readily handled with existing liability law or minor variations thereof. We show this through examples from medical technology, drones, and consumer robotics. A greater challenge will arise if it becomes possible to build robots that merit legal personhood and thus can be held liable. Liability law for robot persons could draw on certain precedents, such as animal liability. However, legal innovations will be needed, in particular for determining which robots merit legal personhood. Finally, a major challenge comes from the possibility of future robots that could cause major global catastrophe. As with other global catastrophic risks, liability law could not apply, because there would be no postcatastrophe legal system to impose liability. Instead, law must be based on pre-catastrophe precautionary measures.

Trevor N White
Seth D Baum
Patrick Lin
George Bekey
Keith Abney
Ryan Jenkins

Liability Law for Present and Future Robotics Technology

Introduction In June 2005, a surgical robot at a hospital in Philadelphia malfunctioned during a prostate surgery, possibly injuring the patient.  1  In June 2015, a worker at a Volkswagen plant in Germany was crushed to death by a robot that was part of the assembly process.  2  In November 2015, a self-driving car in California made a complete stop at an intersection and then was hit by a car with a human driver, apparently because the self-driving car followed traffic law but not traffic norms.  3  These are just some of the ways that robots are already implicated in harms. As robots become more sophisticated and more widely adopted, the potential for harm will get even larger. Robots even show potential for causing harm at massive catastrophic scales. How should robot harms be governed? In general, liability law governs harms in which someone or something else is responsible. Liability law is used to punish those who have caused harms, particularly those that could have and should have been avoided. The threat of punishment further serves to discourage those who could cause harm. Liability law is thus an important legal tool for serving justice and advancing the general welfare of society and its members. The value of liability law holds for robotics just as it does for any other harm-causing technology. But robots are not just any other technology. Robots are (or at least can be) intelligent, autonomous actors moving about in the physical world. They can cause harms through actions that they choose to make, actions that no human told them to make and, indeed, that may surprise their human creators. Perhaps robots should be liable for their harms. This is a historic moment: humans creating technology that could potentially be liable for its own actions. Furthermore, robots can have the strength of industrial machinery and the intelligence of advanced computer systems. Robots can also be mass produced and connected to each other and to other technological systems. This creates the potential for robots to cause unusually great harm. This paper addresses how liability law can and should account for robots, including robots that exist today and robots that potentially could be built at some point in the near or distant future. Three types of cases are distinguished, each with very different implications. First are cases in which some human party is liable, such as the manufacturer or the human using the robot. These cases pose no novel challenges for liability law: they are handled the same way as with other technologies in comparable circumstances. Second are cases in which the robot itself is liable. These cases require dramatic revision to liability law, including standards to assess when robots can be held liable and principles for dividing liability between the robot and the humans who designed, built, and used it. Third are cases in which the robot poses a major catastrophic risk. These cases merit separate attention because a sufficiently large catastrophe would destroy the legal system and thus the potential to hold anyone or anything liable. The three types of cases differ across two dimensions as shown in Figure  1 . One dimension is the robot's degree of legal personhood, meaning the extent to which a robot shows attributes that qualify it for independent standing in a court of law. As we discuss, a robot can be held liable in the eyes of the law to the extent that it merits legal personhood. The other dimension shows the size of the harm the robot causes. Harms of extreme severity cannot be handled by liability law. However, there is no strict distinction between the three cases. Instead, there is a continuum, as shown by the regions in which a robot can have partial liability or more-than-human liability and in which liability law works to a limited extent.
 I -A Human Party Is Liable In a detailed study of robot law,  Weaver (2014, 21-27)  identifies four types of parties that could be liable for harm caused by a robot: (1) people who were using the robot or overseeing its use; (2) other people who were not using the robot but otherwise came into contact with it, which can include people harmed by the robot; (3) some party involved in the robot's production and distribution, such as the company that manufactured the robot; or (4) the robot itself. For the first three types of parties, liability applies the same as for other technologies. A surgical robot, for example, can be misused by the surgeon (type 1), bumped into by a hospital visitor who wandered into a restricted area (type 2), or poorly built by the manufacturer (type 3). The same situations can also arise for other, non-robotic medical technologies. In each case, the application of liability law is straightforward. Or rather, to the extent that the application of liability law is not straightforward, the challenges faced are familiar. The fourth type-when the robot is liable-is the only one that poses novel challenges for liability law. To see this, consider one of the thornier cases of robot liability, that of lethal autonomous weapon systems (LAWSs). These are weapons that decide for themselves whom to kill.  Sparrow (2007)  argues that there could be no one liable for certain LAWS harms-for example, if a LAWS decides to kill civilians or soldiers who have surrendered. A sufficiently autonomous LAWS could make its own decisions, regardless of how humans designed and deployed it. In this case, Sparrow argues, it would be unfair to hold the designer or deployer liable (or the manufacturer or other human parties). It might further be inappropriate to hold the robot itself liable, if it is not sufficiently advanced in legally relevant ways (more on this in Section II). In this case, who or what to hold liable is ambiguous. This ambiguous liability is indeed a challenge, but it is a familiar one. In the military context, precedents include child soldiers  (Sparrow 2007, 73-74)  and landmines  (Hammond 2015, note 62) . Child soldiers can make their own decisions, disobey orders, and cause harm in the process. Landmines can linger long after a conflict, making it difficult or impossible to identify who is responsible for their placement. In both cases, it can be difficult or perhaps impossible to determine who is liable. So too for LAWSs. This ambiguous liability can be a reason to avoid or even ban the use of child soldiers, landmines, and LAWSs in armed conflict. Regardless, even for this relatively thorny case of robot liability, robotics technology raises no new challenges for liability law. In the United States, agencies such as the Department of Defense produce regulations on the use of LAWSs which are not dramatically different than for other weapons. Internationally, bodies like the UN's International Court of Justice could hold a state liable for authorizing drone strikes that caused excessive civilian casualties. Meanwhile, commercial drones can be regulated as other aircraft are now: by a combination of the FAA and corporate oversight by their creators  (McFarland 2015) . The handling of such relatively simple robots under liability law will thus be familiar if not straightforward. The above LAWSs examples also resemble how liability law handles non-human animals, which has prompted proposals for robots to be given legal status similar to non-human animals (e.g.,  Kelley et al. 2010) . Suppose someone gets a pet dog and then the dog bites someone, despite the owner trying to stop it. If this person had absolutely no idea the dog would bite someone, then she would not be liable for that bite. However, having seen the dog bite someone, she now knows the dog is a biter, and is now expected to exercise caution with it in the future. If the dog bites again, she can be liable. In legal terms, this is known as having scienterknowledge of the potential harm. Scienter could also apply to LAWSs or other robots that are not expected to cause certain harms. Once the robots are observed causing those harms, their owners or users could be liable for subsequent harms. For comparison, the Google Photos computer system raised controversy in 2015 when it mislabeled photographs of black people as "gorillas"  (Hernandez 2015) . No Google programmer instructed Photos to do this; it was a surprise, arising from the nature of Photos's algorithm. Google acted immediately to apologize and fix Photos. While it did not have scienter for the gorilla incident, it would for any subsequent offenses.  4  The same logic also applies for LAWSs or other types of robots. Again, as long as a human party was responsible for it, a robot does not pose novel challenges to liability law. Even if a human is ultimately liable, a robot could still be taken to court. This would occur, most likely, under in rem jurisdiction, in which the court treats an object of property as a party to a case when it cannot do so with a human owner. In rem cases include United States v.  Fifty-Three Electus Parrots (1982) , in which a human brought parrots from southeast Asia to the U.S. in violation of an animal import law, and United States v. Forty Barrels & Twenty Kegs of  Coca-Cola (1916) , in which the presence of caffeine in the beverage was at issue. In both cases, a human (or corporation) was ultimately considered liable, with the parrots and soda only serving as stand-ins. Robots could be taken to court in the same way, but they would not be considered liable except in a symbolic or proxy fashion. Again, since the robot is not ultimately liable, it poses no novel challenges to liability law. This is not to say that such robots do not pose challenges to liability law-only that these are familiar challenges. Indeed, the nascent literature on robot liability identifies a range of challenges, including assigning liability when robots can be modified by users  (Calo 2011) , when they behave in surprising ways  (Vladeck 2014) , and when the complexity of robot systems makes it difficult to diagnose who is at fault  (Funkhouser 2013; Gurney 2013) . There are also concerns that liability laws could impede the adoption of socially beneficial robotics (e.g.,  Marchant and Lindor 2012; Wu 2016 ). However, these challenges all point to familiar solutions based in various ways of holding manufacturers, users, and other human parties liable. Fine tuning the details is an important and nontrivial task, but it is not a revolutionary one. The familiar nature of typical robots to liability law is further seen in court cases in which robots have been implicated in harms  (Calo 2016 ). An early case is Brouse v. United  States (1949) , in which two airplanes crashed, one of which was a US military plane that was using an early form of autopilot. The court rejected the US claim that it should not be liable because the plane was being controlled by the robotic autopilot; instead the court found that the human pilot in the plane is obligated to pay attention and avoid crashes.  More recently, in Ferguson v. Bombardier Services Corp. (2007) , another airplane crash may have been attributable to the autopilot system, in which case the court would have found the autopilot manufacturer liable, not the autopilot itself, but instead it found that the airline had improperly loaded the plane. (See Calo 2016 for further discussion of these and other cases.)
 II -Robot Liability If a robot can be held liable, then liability law faces some major challenges in terms of which robots to hold liable for which harms, and in terms of how to divide liability between the robot and its human designers, manufacturers, users, etc. In this section, we will argue that robots should be able to be held liable to the extent that they qualify for legal personhood. First, though, let us briefly consider some alternative perspectives. One perspective is that, in an informal sense, any sort of object can be liable for a harm. The pollen in the air is liable for making you sneeze. The faulty gas pipe is liable for burning down your home. The earthquake is liable for destroying the bridge. This is not the sense of liability we address in this paper. Our focus is on legal liability, in which a party can be tried in court. Another perspective comes from the notion that the law ultimately derives from what members of a society want it be. This is why laws are different in different jurisdictions and at different times. From this perspective, robots will be held liable whenever societies decide to hold them liable. There are difficult issues here, such as whether to give robots a say in if they should be held liable.  5  Regardless, the fact that laws are products of societies need not end debate on what laws societies can and should have. To the contrary, it is incumbent upon members of society to have such debates. Within human society, in the United States and many other countries, parties can be held liable for harms to the extent that they qualify as legal persons. Legal personhood is the ability to have legal rights and obligations, such as the ability to enter contracts, sue or be sued, and be held liable for one's actions. Legal liability thus follows directly from legal personhood. Normal adult humans are full legal persons and can be held liable for their actions across a wide range of circumstances. Children, the mentally disabled, and corporations have partial legal personhood, and in turn can be held liable across a narrower range of circumstances. Non-human animals generally do not have personhood, though this status has been contested, especially for nonhuman primates.  6  The denial of legal personhood to non-human animals can be justified on grounds that they lack humans' cognitive sophistication and corresponding ability to participate in society. Such justification avoids charges of speciesism (a pro-human bias for no other reason than just happening to be human). However, the same justification implies that robots should merit legal personhood if they have similar capabilities as humans. As  Hubbard (2011, 417)  puts it, "Absent some strong justification, a denial of personhood to an entity with at least an equal capacity for personhood would be inconsistent and contrary to the egalitarian aspect of liberalism."  7  The question of when robots can be liable thus becomes the question of when robots merit personhood. If robots merit personhood, then they can be held liable for harms they cause. Otherwise, they cannot be held liable, and instead liability must go to some human party, as is the case with non-human animals and other technologies or entities that can cause harm. Hubbard proposes three criteria that a robot or other artificial intelligence should meet to merit personhood: (1) complex intellectual interaction skills, including the ability to communicate and learn from experience; (2) self-consciousness, including the ability to make one's own goals or life plan, and (3) community, meaning the ability to pursue mutual benefits within a group of persons. These three criteria, central to human concepts of personhood, may offer a reasonable standard for robot personhood. We will use these criteria for this paper while emphasizing that their definitude should be a matter of ongoing debate. Do Hubbard's criteria also apply for liability? Perhaps not for the criterion of selfconsciousness. The criterion makes sense for harms caused to a robot: only a conscious robot can experience harms as humans do.  8  This follows from, for example, classic utilitarianism, as in Bentham's line "The question is not, Can they reason? nor, Can they talk? but, Can they suffer?" However, the same logic does not apply to harms caused by a robot. Consider an advanced robot that meets all of Hubbard's criteria except that it lacks consciousness. Suppose the robot causes some harm-and, to be clear, the harm causes suffering to a human or to some other conscious person. Should the robot be held liable? The answer to this may depend on society's foundational reasoning for liability. If liability exists mainly to discourage or deter the commission of harms, then consciousness is unnecessary. The robot should be punished so long as doing so discourages the commission of future harms. The entities that get discouraged here could include the robot, other similar robots, conscious robots, and even humans. It is quite conceivable that non-conscious robots could be punished with some sort of reduced reward or utility as per whatever reward/utility function they might have  (Majot and Yampolskiy 2014) . Specifically, they could be reprogrammed, deactivated or destroyed, or put in what is known as a "Box": digital solitary confinement restricting an AI's ability to communicate or function  (Corwin 2002; Yudkowsky 2002) . To make this possible, however, such robots ought to be based (at least in part) on reinforcement learning or similar computing paradigms (except ones based on neural network algorithms, for reasons we explain later). Alternatively, if liability exists mainly for retribution, to bring justice to whomever committed the harm, then consciousness could be necessary. Whether it is necessary depends on the purpose of the punishment. If the punishment aims to worsen the life of the liable party, so as to "balance things out," then consciousness seems necessary. It makes little sense to "worsen" the life of something that cannot experience the worsening. However, if the punishment aims to satisfy society's sense of justice, then consciousness may be unnecessary. Instead, it could be sufficient that members of society observe the punishment and see justice being served.  9  Whether the robot's consciousness would be necessary in this case would simply depend on whether society's sense of justice requires it to be conscious. This potential exception regarding consciousness is a good example of partial liability as shown in Figure  1 . The advanced, non-conscious robot can be held liable, but not in every case in which normal adult humans could. Specifically, the robot would not be held liable in certain cases where punishment is for retribution. Other limitations to a robot's capabilities could also reduce the extent of its liability. Such robots would be analogous to children and mentally disabled adult humans, who are similarly not held liable for as many cases as normal adult humans are. Robots of less sophistication along any of Hubbard's three criteria (or whatever other criteria are ultimately established) should be liable to a lesser extent than robots that meet the criteria in full. What about robots of greater-than-human sophistication in Hubbard's three criteria? These would be robots with more advanced intellectual interaction skills, self-consciousness, or communal living ability. It is conceivable that such robots could exist-indeed, the idea dates back many decades  (Good 1965 ). If they do come into existence, then by the above logic, they should be held to a higher liability standard than normal adult humans. Indeed, concepts such as negligence recognize human fallibility in many respects that a robot could surpass humans in, including reaction time, eyesight, and mental recall. The potential for holding robots to a higher standard of liability could offer one means of governing robots with greater-than-human capacities; more on this in Section III in the discussion of catastrophic risk. Before turning to catastrophic risk, there is one additional aspect of robot liability to consider: the division of liability among the robot itself and other parties that influence the robot's actions. These other parties can include the robot's designer, its manufacturer, and any users or operators it may have. These parties are comparable to a human's parents and employers, though the comparison is imperfect due to basic differences between humans and robots. One key difference is that robots are to a very large extent designed. Humans can be designed as well via genetic screening and related techniques, hence the term "designer baby." But designers have much more control over the eventual character of robots than they do for humans. This suggests that robot designers should hold more liability for robots' actions than human parents should for their children's actions. If robot designers know that certain designs tend to yield harmful robots, then a case can be made for holding the designers at least partially liable for harms caused by those robots, even if the robots merit legal personhood. Designers could be similarly liable for building robots using opaque algorithms, such as neural networks and related deep learning methods, in which it is difficult to predict in advance whether the robot will cause harm. Those parties that commission the robot's design could be similarly liable. In court, the testimony of relevant industry experts would be valuable for proving whether any available, feasible safeguards to minimize such risks existed. Another difference is that, at least for now, the production of robots is elective, whereas the birthing of humans is required for the continuity of society. Society cannot currently function without humans, but it can function without robots. This fact suggests some lenience for parents in order to encourage procreation, and to be stricter with robot designers in order to safely ease society's transition into an era in which humans and their robot creations coexist. Such a gradual transition seems especially warranted in light of potential robot catastrophe scenarios.
 III -Catastrophic Robot/AI Liability "Catastrophe" has many meanings, many of which require no special legal attention. For example, a person's death is catastrophic for the deceased and her or his loved ones, yet the law is perfectly capable of addressing individual deaths caused by robots or AIs. However, a certain class of extreme catastrophe does merit special legal attention, due to its outsized severity and significance for human civilization. These are catastrophes that cause major, permanent harm to the entirety of global human civilization. Such catastrophes are commonly known as global catastrophes  (Baum and Barrett 2016)  or existential catastrophes  (Bostrom 2013) . Following  Posner (2004) , we will simply call them catastrophes. A range of catastrophic risks exist, including global warming, nuclear war, a pandemic, and collision between Earth and a large asteroid or comet. Recently, a body of scholarship has built up analyzing the possibility of catastrophe from certain types of future AI. Much of the attention has gone to "superintelligent" AI that outsmart humanity and "achieve complete world domination"  (Bostrom 2014, 78 ; see also  Müller 2015) . Such AI could harm humans through the use of robotics. Additionally, some experts believe that robotics could play an important role in the development of such AI  (Baum et al. 2011) . Other catastrophe scenarios could also involve robotics. Robots could be used in the systems for launching nuclear weapons or for detecting incoming attacks, potentially resulting in unwanted nuclear wars.  10  They could be used in critical civil, transportation, or manufacturing infrastructure, contributing to a global systemic failure.  11  They could be used for geoengineering -the intentional manipulation of the global environment, such as to counteract global warming -and this could backfire, causing environmental catastrophe.  12  Robots could be used in establishing or maintaining an oppressive totalitarian world government.  13  Still further robot catastrophe scenarios may also be possible. The enormous scale of the catastrophes in question creates profound moral and legal dilemmas. If the harm is permanent, it impacts members of all future generations, which could be immensely many people. Earth will remain habitable for at least a billion more years, and the galaxy and the universe for much longer  (Baum 2016) ; the present generation thus contains just a tiny fraction of all people who could exist. The legal standing and representation of members of future generations is a difficult question  (Tonn 1996; Wolfe 2008 ). If members of future generations are to be counted, then they can overwhelm the calculus. Despite this, present generations unilaterally make the decisions. There is thus a tension in how to balance the interests of present and future generations  (Page 2003) . A sufficiently large catastrophe raises similar issues even just within the context of the present generation. About seven billion humans live today; a catastrophe that risks killing all of them could be seven billion times larger than a catastrophe that risks killing just one. One could justify enormous effort to reduce that risk regardless of future generations  (Posner 2004) . Further complications come from the irreversible nature of these catastrophes. In a sense, every event is irreversible: if someone wears a blue shirt today, no one can ever change the fact that they wore a blue shirt today. Such events are irreversible only in a trivial sense: you can change what shirt you wear on subsequent days. Nontrivially irreversible events are more or less permanent: if that person should die today, then nothing 14 can bring that person back to life. At a larger scale, nontrivially irreversible effects exist for many ecological shifts and may also exist for the collapse of human civilization  (Baum and Handoh 2014) . The possibility of large and nontrivially irreversible harm creates a major reason to avoid taking certain risks. The precautionary principle is commonly invoked in this context, raising questions of just how cautious to be  (Posner 2004; Sunstein 2006 ). An irreversible AI catastrophe could be too large for liability law to handle. In the simplest case, if the catastrophe results in human extinction, then there would be no one remaining to hold liable. A catastrophe that leaves some survivors but sees the collapse of human civilization would lack the legal system needed for holding people liable. Alternatively, AI could cause a catastrophe in which everyone is still alive but they have become enslaved or otherwise harmed by the AI; in this case the pre-catastrophe human authorities would lack the power needed to hold those at fault liable. For smaller catastrophes, the legal system may exist to a limited extent (Figure  1 ). In this case, it may be possible to bring the liable parties to trial and/or punish them, but not as reliably or completely as is possible under normal circumstances. The closest possible example would be creating special international proceedings, like the Nuremberg Trials, to deal with the aftermath. Much like such war tribunals, though, these may do little to address the chaos' original cause. This would leave victims or society at large wasting time and resources on reliving a tragedy  (McMorran 2013) . Hence, instead of liability, a precautionary approach could be used. This would set a default policy of disallowing any activity with any remote chance of causing catastrophe. It could further place the burden of proof on those who wish to conduct such activity, requiring them to demonstrate in advance that it could not cause catastrophe.  15  Trial-and-error would not be permitted, because a single error could cause major irreversible harm. This would likely be a significant impediment for AI research and development (at least for the subset of AI that poses catastrophic risk), which, like other fields of technology, is likely to make extensive use of trial and error. Indeed, some AI researchers recommend a trial-and-error approach, in which AIs are gradually trained to learn human values so that they will not cause catastrophe  (Goertzel 2016) . However, given the high stakes of AI catastrophe, perhaps these sorts of trial-and-error approaches should still be avoided. It may be possible to use a novel liability scheme to assist with a catastrophe-avoiding precautionary approach. In a wide-ranging discussion of legal measures to avoid catastrophe from emerging technologies,  Wilson (2013, 356)  proposes "liability mechanisms to punish violators whether or not their activities cause any harm". In effect, people would be held liable not for causing catastrophe, but for taking actions that could cause catastrophe. This proposal could be a successful component of a precautionary approach to catastrophic risk and is worth ongoing consideration. Taking the precautionary principle to the extreme can have undesirable consequences. All actions carry some risk. In some cases, it may be impossible to prove a robot does not have the potential to cause catastrophe. Therefore, requiring demonstrations of minimal risk prior to performing actions would be paralyzing  (Sunstein 2006) . Furthermore, many actions can reduce some risks even while increasing others; requiring precaution due to concern about one risk can cause net harm to society by denying opportunities to decrease other risks  (Wiener 2002) . AI research and development can pose significant risks, but it can also help reduce other risks. For AI that poses catastrophic risk, net risk will be minimized when the AI research and development is expected to bring a net reduction in catastrophic risk  (Baum 2014) . In summary, there are significant legal challenges raised by AI that poses catastrophic risk. Liability law, most critically, is of little help. Precautionary approaches can work instead, although care should be taken to avoid preventing AI from reducing different catastrophic risks. The legal challenges from AI that poses catastrophic risk is distinct from the challenges from other types of AI, but they are similar to the challenges from other catastrophic risks.
 Conclusion While robots benefit society in many ways, they also cause or are otherwise implicated in a variety of harms. The frequency and size of these harms is likely to increase as robots become more advanced and ubiquitous. Robots could even cause or contribute to a number of major global catastrophe scenarios. It is important for liability law to successfully govern these harms to the extent possible so that the harms are minimized and, when they do occur, that justice may be served. For many robot harms, a human party is ultimately liable. For these harms, traditional liability law applies. A major challenge to liability law comes when robots could be liable. Such cases require legal personhood tests for robots to assess the extent to which they can be liable. One promising personhood test evaluates the robot's intellectual interaction skills, selfconsciousness, and communal living ability. Depending on how a robot fares on a personhood test, it could have the same liability as, or less or more liability than, a normal adult human. A robot being liable does not preclude a human party also being liable. Indeed, robot designers should expect more liability for robot harms than would human parents, because robots are designed so much more extensively than human children are. Finally, for robots that pose catastrophic risk, liability law cannot be counted on and a precautionary approach is warranted. People involved in the design, manufacture, and use of robots can limit their liability by choosing robots that reliably avoid harms. One potential way to improve reliability is to avoid computing paradigms such as neural nets that tend to result in surprising behaviors, or adapt these paradigms to make them less surprising  (Huang and Xing 2002) . Robot designs should be sufficiently transparent that the responsible human parties can, with reasonable confidence, determine in advance what harms could occur. They can then build safety restrictions into the robot or at least give warnings to robot users, as is common practice with other technologies. Robots should also go through rigorous safety testing before being placed into situations where they can cause harms. If robots cannot reliably avoid harms, then they probably should not be used in the first place. These sorts of safety guidelines should be especially strict for robots that could contribute to major global catastrophe. A single catastrophe could permanently harm human civilization. It is thus crucial to avoid any catastrophe. Safety testing itself could be dangerous. This increases the value of transparent computing paradigms that let humans assess risks prior to building the robot. Legal measures must also take effect prior to the robot's build because there may be no legal system afterwards. Advanced robots may be less likely to cause catastrophe if they are designed to be upstanding legal persons. But even then, some legal system would need to exist to hold them liable for what harms they cause. As this paper illustrates, robot liability poses major new challenges to liability law. Meeting these challenges requires contributions from law, robotics, philosophy, risk analysis, and other fields. It is essential for humans with these various specialties to work together to build robot liability regimes that avoid harms while capturing the many benefits of robotics. The potential for harm is extremely large, making this an urgent task. We hope that humans and robots will coexist successfully and for mutual benefit in a community of responsible persons. Figure 1 . 1 Figure 1. Classification scheme for the applicability of liability law to various sizes of harms caused by various types of robots.