Defining Explanation in an AI Context (2020)

Abstract: With the increase in the use of AI systems, a need for explanation systems arises. Building an explanation system requires a definition of explanation. However, the natural language term explanation is difficult to define formally as it includes multiple perspectives from different domains such as psychology, philosophy, and cognitive sciences. We study multiple perspectives and aspects of explainability of recommendations or predictions made by AI systems, and provide a generic definition of explanation. The proposed definition is ambitious and challenging to apply. With the intention to bridge the gap between theory and application, we also propose a possible architecture of an automated explanation system based on our definition of explanation.

## introduction definitions of explanation have been proposed by many researchers from the perspective of various fields of science, however the scope of our research is focused on computer science and cognitive science perspectives, as we are looking for automatically generated explanations for people affected by recommendations or predictions made by ai systems. explanation is often described as a phenomenon which enables transfer of specific information. philosophically, acceptance of a statement as an explanation is subject to acceptance by the recipient @xcite . however, according to computer scientists explanations must be complete, i.e. encapsulate all factors of complex internal workings, and accurate, i.e. hold high fidelity to the ai model in question @xcite . we can find many definitions in different domains, often conflicting. some choose to focus only on selected aspects of explanation. for generating explanations automatically, a strong understanding of how people define, generate, select, evaluate, and present explanations is essential. in order to design and implement intelligent agents that are truly capable of providing appropriate explanations to people, analyzing how humans explain decisions and behavior to each other is a good starting point. in the past, researchers have defined models and conducted social experiments, many of which uncovered novel and essential aspects of human explanations @xcite @xcite . specifically social experiments help us understand the impact of unquantifiable attributes of human behavior which include belief, desire, social norms, intention to understand, emotions, personality traits, etc. these attributes are aptly defined by folk psychology, or commonsense psychology @xcite . folk psychology does not describe our thought process or behavior but our expected thought process or behavior, which makes it an important aspect of explanation. as mentioned earlier, most of the past research is focused on selective aspects of explanation. this is because building a generic explanation system is a tedious task. with the intention to take a step towards realization of this task, we assume the ai system to be a black-box i.e. all internal structure and/or processes are hidden and only input-output pairs are perceivable. no restriction is imposed on inputs and outputs. in this paper we will review cognitive and social models, experiments and definitions which may assist us in writing a more generic definition of explanation. in the background section we will discuss the need for explanation in detail, which will enable us to define necessary conditions and possible quality measures. we will also highlight important models, theories and conflicting definitions. in later sections, we argue which aspects of explanation are most important and propose a generic definition of explanation. lastly, we introduce a possible architecture of a generic automated explanation system based on our proposed definition. ## background in this section, we will discuss why explanation is required, the theoretical advances made towards building explanation models and insights about conflicts in the definitions proposed so far. ## proposed definition in this section we present a generic definition, break down its components which can be interpreted differently depending on perspectives and discuss them from the ai perspective. here, if an explanation is requested for a certain prediction/recommendation made via a particular blackbox, we will represent the combination of input to the black-box, the black-box itself and the prediction/recommendation as a scenario. we argue that explanation requires pre-existing information about three major aspects: the posed question, the recipient's knowledge, and the given scenario. with respect to the aforementioned three aspects, we propose the following definition of explanation: an explanation is a representation of fair and accurate assessments made by an explainer to transfer relevant knowledge (about a given scenario) from the explainer to a recipient. in ai perspective, each component of the definition can be translated as follows: explanation as a statement represents a passive form of knowledge transfer based on only the given scenario. on the other hand, explanation as a response to a question posed by a recipient represents an active form of knowledge transfer based on the posed question, the recipient's knowledge, and the given scenario. ## conclusion in this paper, we review multiple perspectives and aspects of explanation. based on this, we summarize the most important aspects of explanation and propose a generic definition. the definition and examples of explanation in different scenarios guide us towards building an explanation system. we propose an architecture for one such generic explanation system based on the proposed definition of explanation. it is capable of producing multiple explanations depending on the given recipient, the posed question and a given scenario. we hope that this paper paves the way to build generic explanation systems.