CIDER : Commonsense Inference for Dialogue Explanation and Reasoning (2021)

Abstract: Commonsense inference to understand and explain human language is a fundamental research problem in natural language processing. Explaining human conversations poses a great challenge as it requires contextual understanding, planning, inference, and several aspects of reasoning including causal, temporal, and commonsense reasoning. In this work, we introduce CIDER – a manually curated dataset that contains dyadic dialogue explanations in the form of implicit and explicit knowledge triplets inferred using contextual commonsense inference. Extracting such rich explanations from conversations can be conducive to improving several downstream applications. The annotated triplets are categorized by the type of commonsense knowledge present (e.g., causal, conditional, temporal). We set up three different tasks conditioned on the annotated dataset: Dialogue-level Natural Language Inference, Span Extraction, and Multi-choice Span Selection. Baseline results obtained with transformer-based models reveal that the tasks are difficult, paving the way for promising future research. The dataset and the baseline implementations are publicly available at https://github.com/declare-lab/CIDER .

## introduction understanding and explaining a conversation requires the decomposition of dialogue concepts -entities, events and actions, and also connecting them through definitive relations. the process of breaking down dialogues into such explanations is grounded in the conversational context and often requires commonsense inference. such explanations, when expressed in the form of structured knowledge triplets foot_0 (fig. 1 ), can describe the exact commonsense relation (causal/temporal/conditional/others) through which the concepts are related in the particular conversational context. establishing such concept links that help explain the dialogue demands two distinct forms of commonsense inference: i) explicit -the explanation is verbatim in the triplet. such triplets can be easily extracted out by a parser (e.g., syntactic, pattern matching). these triplets are also prevalent in existing commonsense knowledge graphs @xcite ; and ii) implicit -the explanation is entirely contextual, making it more difficult for machines to infer as it requires complex multi-hop commonsense rea-soning skills. our goal is to explain a dialogue by the means of these commonsense inferred triplets. this form of explanation may not be complete, but can give a substantial understanding of the dialogue by breaking it down into contextual triplets. the key element of the dialogue explanation using such triplets is the aspect of contextuality. the triplets extracted from a dialogue using commonsense inference are contextual and are grounded exclusively in that particular dialogue. from our world knowledge, we know that missing a bus could cause being late, but (missed the bus, causes, late) is grounded and definitive only in the dialogue illustrated in fig. 1 . this particular triplet may not be valid in a different dialogue, where the cause of being late could be something different. similarly, losing wallet could cause a different consequence (apart from being late) e.g., getting anxious in the context of another dialogue. it is also important to highlight that some extracted triplets could be persona-specific. for instance, (tardiness, causes, embarrassment) is grounded in the conversation of fig. 1 , but tardiness may not cause embarrassment for every listener. in literature, there has been much work on extracting structured knowledge triplets from natural language text. however, there has been only little research to distinguish implicit triplets from explicit triplets present in the text. explicit triplets can be relatively easily parsed out using semantic parsing @xcite and simple co-reference resolution @xcite . implicit triplets, however, involve non-trivial inference, which becomes even more challenging on dialogue data due to the contextual interplay and latent background knowledge shared between the speakers. extraction of both explicit and implicit triplets can be conducive to improved dialogue understanding leading to better question-answering systems and richer knowledge bases. to this end, we construct a dataset of commonsense inference for dialogue explanation and reasoning (cider) -as illustrated in fig. 1 -which captures the relations between textual concepts or spans appearing in a dialogue. a concept or span can constitute one or multiple entities, objects, actions, states, or events that can be extracted from the dialogue. the relations are commonsense based, as elaborated in §3.2. each triplet is tagged as explicit or implicit. through this dataset, we aim to evaluate whether state-of-the-art natural language processing models can really read, understand, and comprehend the conversational context of dialogues. we define three tasks on this dataset that require dialoguelevel contextual commonsense reasoning -(i) dialogue-level natural language inference, (ii) span extraction, and (iii) multi-choice span selection. all three tasks require an overall contextual understanding of the dialogue with commonsense reasoning and inference. we setup different state-ofthe-art transformer language models as baselines and found that the tasks are challenging to solve. the importance of this dataset: the immediate aim of this research is to develop a rich corpus of dialogues with structured explanations in the form of implicit and explicit triplets, and then use this corpus to perform commonsense inference and reasoning. we formulate non-trivial natural language inference (nli) and question answering (qa) tasks that can be used to benchmark such reasoning capabilities of natural language processing models. ## related work recently, language models have been scaled up a lot and have seen a performance improvement on various tasks @xcite . however, it has been proved that declarative knowledge is still valuable, especially implicit relationships that are hardly acquired by the state of the art models @xcite . widely used commonsense knowledge bases such as conceptnet @xcite and atomic @xcite are mainly based on crowd-sourced effort. conceptnet is a semantic network with nodes composed of common words or phrases in their natural language form. it contains 34 relations, including taxonomic, temporal, and causal ones, such as motivatedbygoal and causes. however, the knowledge in conceptnet is annotated solely based on the first entity without any other context, making it difficult to capture the long-tail knowledge outside of the most common ones. atomic focus on inferential knowledge and consists of nine relations, such as xintent (the intent for personx's action) and xeffect (the effect of the event on personx). it covers knowledge around agents involved in the event for if-then reasoning, including subsequent events, mental state, and persona. however, it ignores causal relationships between events not carried out by a person. in contrast, our work captures relationships between spans across multiple turns in dialogues. as a result of the dialogue aspect of our data, we also manage to cover implicit knowledge that requires context from conversations to make sense. this tardiness causes embarrassment every time. ## background the primary impetus behind this dataset is the contextualized structured explanation of a dialogue in the form of concept triplets that can be inferred only through commonsense reasoning. the triplets are considered to be the commonsense explanations of different aspects and events that occur in the dialogue. such aspects would include attributional, comparative, temporal knowledge, and the events may range from physical events involving physical entities, conditional and causal chains, social interactions, persona, etc. we focus on conversations as our data source, with the choice being motivated by the fact that part of the context in conversations is naturally implicit and interlocutor dependent @xcite . commonsense knowledge is considered to be the set of all facts and knowledge about the everyday world which is assumed to be known by all humans @xcite . for this very reason, human-tohuman dialogues -typically guided by the gricean maxims of human interactions -tend to avoid explicit mentions of commonsense knowledge and the associated reasoning steps. it is thus reasonable to assume that conversations are generally likely to hold more context-specific inferable implicit knowledge than other genres. this ensures a rich dataset with plenty of contextual implicit triplets and a reasonable amount of explicit triplets. two distinct spans (e.g., events, entities) in a dialogue may have an implicit connection that can be trivial for humans to interpret using commonsense reasoning and contextual understanding, but can be challenging for machines. uncovering implicit explanations has the potential to enable many important tasks, which we focus on later on. in this work, we propose a dataset that contains manually labeled implicit explanations present in dyadic dialogues that require commonsense reasoning to infer. we use this dataset to evaluate the ability of pre-trained language models' to perform commonsense-based implicit reasoning tasks. the extracted triplets or explanations, of the form (h, r, t) or alternatively h r -→ t, consist of a head (h) and a tail (t) span and the directed relation (r) between them. these spans are representative of some events, actions, objects, entities, and so on. the directed relation r comes from a predefined set of relations r that explain or describe the relationship between the head and tail spans within the context of the conversation -illustrated in fig. 1 with the arrows between spans. notably, the relation set r is intended to be generic in nature, rather than specifically factual or taxonomic, so as to accommodate wide categories of knowledge ( §3.2) inferred from the context of the conversation. ## experimental setup and results we formulate three tasks on the cider dataset: 1) dialogue-level natural language inference; 2) span extraction; 3) multi-choice span selection. ## conclusion in this work, we introduced cider-a new dataset that focuses on commonsense-based implicit explanation extraction from dialogues. the dataset consists of more than 4,500 manually annotated triplets from over 800 dialogues. we also introduced dialogue-level nli and qa tasks, along with pre-trained transformer-based baselines to evaluate their inference and reasoning capabilities.