“Mama Always Had a Way of Explaining Things So I Could Understand”: A Dialogue Corpus for Learning to Construct Explanations (2022)

Abstract: As AI is more and more pervasive in everyday life, humans have an increasing demand to understand its behavior and decisions. Most research on explainable AI builds on the premise that there is one ideal explanation to be found. In fact, however, everyday explanations are co-constructed in a dialogue between the person explaining (the explainer) and the specific person being explained to (the explainee). In this paper, we introduce a first corpus of dialogical explanations to enable NLP research on how humans explain as well as on how AI can learn to imitate this process. The corpus consists of 65 transcribed English dialogues from the Wired video series 5 Levels, explaining 13 topics to five explainees of different proficiency. All 1550 dialogue turns have been manually labeled by five independent professionals for the topic discussed as well as for the dialogue act and the explanation move performed. We analyze linguistic patterns of explainers and explainees, and we explore differences across proficiency levels. BERT-based baseline results indicate that sequence information helps predicting topics, acts, and moves effectively.

## introduction explaining is one of the most pervasive communicative processes in everyday life, aiming for mutual understanding of the two sides involved. parents explain to children, doctors to patients, teachers to students, seniors to juniors-or all the other way round. in explaining dialogues, one side takes the role of the explainer, the other the role of the explainee. explainers seek to enable explainees to comprehend a given topic to a certain extent or to perform some action related to it @xcite . this usually implies a series of dialogue turns where both sides request and provide different information about the topic. in line with the quote from the movie "forrest gump" in the title, * both authors contributed equally to this paper. explaining dialogue on the main topic "blockchain" explainer (expert) (child) explainee do you know what we're gonna talk about today? it's called blockchain. what's blockchain? that's a really good question. it's actually a way that we can trade. do you know what trade is? mmm-hmm, it's when you take turns doing something. it's when you give up most of what you want, right? when you give up most of what you want? well, sometimes that definitely happens for sure. what if i told you that this is the kind of technology that i work on that means you could trade with any kid all over the world? if i could trade with any kid, i would trade, well, i would trade something i don't like so much. that's probably a good idea, maybe somebody else likes it more than you do. so normally, when people trade, they have to go to the store, or they have to know the person so they can get what they asked for. with blockchain, you can make that exact same trade, but you don't need the store, and you don't even necessarily need to know the other person. really. how an explaining dialogue looks like is strongly affected by the specific explainer and explainee as well as by their interaction. consider the dialogue in figure 1 , where a technology expert explains the basic idea of blockchain to a 5-year old in a controlled setting. beyond the explanations of the main topic (turns 05 and 09), the dialogue contains an explanation request (02), a test of prior knowledge (03), explanations from the explainee (04), and more. we observe that the explainer's explanations depend on the reaction of the explainee and that their level of depth is most likely adjusted to the explainee's proficiency. the importance of studying how to explain has become apparent with the rise of research on explainable artificial intelligence, xai @xcite . as ai finds its way into various aspects of work and private life, humans interacting with respective systems, or being affected by them, have an increasing demand to understand their behavior and decisions. this demand has also been manifested in a right to explanation within the eu's general data protection regulation @xcite . prior work on xai largely starts from the premise that an ideal (monological) explanation exists for any behavior or decision, possibly dependent on the explainee at hand @xcite . according to @xcite , however, real explainability must account for the co-constructive nature of explaining emerging from interaction. in natural language processing, early work modeled discourse structure of monological explanations @xcite , and a number of recent approaches generate respective explanations for xai @xcite and recommendation @xcite . in contrast, the language of dialogical explanations is still understudied (details in section 2). we argue that a better understanding of how humans explain in dialogues is needed, so that xai can learn to interact with humans. in this paper, we present a first corpus for computational research on how to explain in dialogues (section 3). the corpus has been created as part of a big interdisplinary research project dealing with the construction of explainability. 1 it consists of 65 transcribed dialogical explanations from the american video series 5 levels freely published by the wired magazine. 2 five dialogues each refer to one of 13 science-related topics (e.g., "blockchain" or "machine learning"). they have the same explainer (an expert on the topic), but differ in the explainee's proficiency (from child to colleague). to enable xai to mimic human explainers, it has to learn what turn to make at any point in a dialogue. in discussion with humanities researchers, we model a turn for this purpose by the relation of its topic to the main topic (e.g., subtopic or related topic), its dialogue act (e.g., check question or informing statement), and its explanation move (e.g., testing prior knowledge or providing an explanation). we segmented the dialogues into a total of 1550 turns, and we let five independent professionals annotate each turn for these three dimensions. in section 4, we analyze linguistic patterns of explaining dialogues in the annotated corpus. we find clear signals for the explainer's alignment to 1 constructing explainability, https://trr318.upb.de/en 2 5 levels, https://www.wired.com/video/series/5-levels the explainee's proficiency, such as the avoidance of deviating to related topics towards children. the roles of explainer and explainee are reflected in the varying use of dialogue acts and explanation moves, possibly stressed by the given setting. to obtain baselines for the prediction of the three annotated dimensions, we evaluate three variants of bert @xcite in 13-topic crossvalidation on the corpus (section 5). our results reveal that modeling sequential dialogue interaction helps predicting a turn's topic, act, and move effectively. improvements seem still possible, calling for more sophisticated approaches as well as for more explaining dialogue data in the future. 3 in summary, the contributions of our paper are: explainable ai (xai) largely focuses on the interpretability of learned models from the perspective of scientific completeness @xcite . even though recent works tackle cognitive aspects, such as the trade-off between completeness and compactness @xcite , @xcite pointed out that this perspective is far away from the understanding of everyday explanations in the social sciences. @xcite argues that the key is to sort out what the explainer should actually explain, and barredo arrieta et al. (2020) stressed the importance of who is the explainee for xai. @xcite built on these works, but reasoned that explanations can only be successful in general, if they are co-constructed in interaction between explainer and explainee. the rationale is that explainees vary in their motives and needs, and they face different challenges @xcite . the corpus we present serves as a basis for studying the linguistic aspects of the explainer-explainee interaction computationally. natural language language processing (nlp) has notably dealt with the related genre of instructional texts, modeling their structure @xcite , extracting knowledge @xcite , comprehending some meaning @xcite , or generating them @xcite . however, instructional text has a clear procedural style with distinctive surface features @xcite , unlike explanations in general. for tutorial applications, @xcite extracted concepts from explanation sentences, whereas @xcite studied the knowledge needed for scientific explanations, and @xcite identified causal explanations in social media. towards a computational understanding of explaining, bourse and saint-dizier (2012) modeled explanation structure with discourse relations @xcite . in xai and recommendation contexts, the generation of respective explanations is explored increasingly @xcite . however, our main goal is not to understand how to generate an explanation, but to model how people interact in an explanation process. for annotation, we thus rely on the widely accepted concept of dialogue acts @xcite . similar has been done for deliberative dialogues by al @xcite . in addition, we model the moves that explainers and explainees make in their interaction, adapting the idea of rhetorical moves, in terms of communicative functions of text segments used to support the communicative objective of a full text @xcite . @xcite proposed taskspecific moves for monological arguments, but we are not aware of any work on moves for explanations, nor for dialogical settings. hence, we start by compiling data in this paper. existing related corpora contain tutorial feedback for explanation questions @xcite , answers to non-factoid questions @xcite , and pairs of questions and responses from community question answering platforms @xcite . finally, the corpus of @xcite includes 270k threads from the reddit forum explain like i'm five where participants explain a concept asked for in simple ways. while all these allow for in-depth analyses of linguistic aspects of explanations, none of them include explaining dialogues with multiple turns on each side. this is the gap we fill with the corpus that we introduce. ## data this section introduces the corpus that we created to enable computational research on dialogical explanation processes of humans. we discuss our design choices with respect to the source and annotation, and we present detailed corpus statistics. ## analysis one main goal of the presented corpus is to learn how humans explain in dialogical settings. this section analyzes commonalities and differences regarding meta-information available in the corpus. ## experiments the second goal of the corpus is to serve the creation of xai systems that mimic human explainers. as an initial endeavor, this section reports on baseline experiments on the computational prediction of topics, dialogue acts, and explanation moves. ## ethical statement we do not see any immediate ethical concerns with respect to the research in this paper. the data included in the corpus is freely available. all participants involved in the given dialogues gave their consent to be recorded and received expense allowances, as far as perceivable from the wired web resources. as discussed in the paper, the three freelancers in our annotation study were paid about $13 per hour, which exceeds the minimum wage in most us states and is also conform to the standards in the regions of our host institution. in our view, the provided prediction models target dimensions of dialogue turns that are not prone to be misused for ethically doubtful applications.