CICERO : A Dataset for Contextualized Commonsense Inference in Dialogues (2022)

Abstract: This paper addresses the problem of dialogue reasoning with contextualized commonsense inference. We curate CICERO, a dataset of dyadic conversations with five types of utterance-level reasoning-based inferences: cause, subsequent event, prerequisite, motivation, and emotional reaction. The dataset contains 53,105 of such inferences from 5,672 dialogues. We use this dataset to solve relevant generative and discriminative tasks: generation of cause and subsequent event; generation of prerequisite, motivation, and listener’s emotional reaction; and selection of plausible alternatives. Our results ascertain the value of such dialogue-centric commonsense knowledge datasets. It is our hope that CICERO will open new research avenues into commonsense-based dialogue reasoning.

## introduction conversational content on the internet is quickly growing, and such content holds valuable knowledge about how information exchange takes place among speakers. a key step towards understanding such dialogues is gaining the ability to reason with the information shared in the dialogue. to this end, we curate a dataset of dyadic conversations named cicero (contextualized commonsense inference in dialogues) , which contains inferences around the utterances in the dialogues. the dataset focuses on five types of reasoning-based inferences for a given utterance in a dialogue: cause, subsequent event, prerequisite, motivation, and emotional reaction. arguably, making such reasoning-based inferences often demands commonsense knowledge, especially when the inference is implicit. fig. 1a shows such a case where the cause behind the target utterance is not explicit in the context. however, applying the commonsense knowledge worn gloves allowed the annotator to infer a probable cause of the utterance. on the other hand, commonsense can be crucial in sifting relevant information from the context. fig. 1b depicts an instance where the cause behind the target utterance is inferred from the context. this inference can be explained by commonsense knowledge (see fig. 3 ) such as repetitive consumption of the same food causes -----→ boredom dispelled by ----------→ changing food achieved by ## construction of cicero we create cicero -a large dataset of english dyadic conversations annotated with five types of inferences with the help of human annotators, who are instructed with a carefully crafted set of guidelines. ## commonsense inference on cicero we design generative and multi-choice question answering tasks on cicero to evaluate dialoguelevel commonsense-based reasoning capabilities of language models. ## cicero tasks: experimental results we split our dataset in dialogue level where the training, validation and test instances are obtained from a total of 3477, 1097, 1098 distinct dialogues respectively. this results in a 60:20:20 proportion of total annotation instances. the three sets have 17365, 5370, and 5331 unique target utterances respectively. we tune on the validation dataset and report results on the test dataset (average of 5 runs). for the sake of brevity, the detailed hyperparameters are given in the supplementary material. we use the following questions (q) for the five inference types for all the tasks: cause: what is or could be the cause of target? subsequent event: what subsequent event happens or could happen following the target? prerequisite: what is or could be the prerequisite of target? motivation: what is or could be the motivation of target? reaction: what is the possible emotional reaction of the listener in response to target? 4.1 baseline models cicero nlg -(1.1-1.2). we use large versions of t5 @xcite and glucose-t5 @xcite as our models. glucose-t5 is a t5 large model that is pre-trained on the glucose dataset. we concatenate q, u t , and the context c with separators to form the input to the model: @xmath0. the context c is formed by concatenating utterances of d :u t (subsequent event clipped) or d (all other tasks). for the chained generation task, we additionally provide the cause/subsequent event as input. the inputs are @xmath1: e @xmath2: c @xmath3, respectively. the objective is to generate the answer as output in the sequenceto-sequence setup. we use teacher forcing during training and beam search during inference. ## related work commonsense knowledge has received more attention compared with factual knowledge, as it is usually not mentioned explicitly in the context. it is demonstrated to be essential in openended generation tasks, such as story explanation generation @xcite , story end generation @xcite and abductive reasoning @xcite . to infuse commonsense knowledge in nlp models, several approaches to tasks like sentence ordering @xcite , emotion recognition @xcite , story generation @xcite and dialogue generation @xcite use prevalent commonsense knowledge bases (cskb) like conceptnet @xcite or atomic @xcite . however, conceptnet is context-free, meaning that they only capture relationships around a selected set of entities, without paying attention to the context where the entity occurs. moreover, inference is often needed in discourse level, which do not always align with the entities in knowledge bases. knowledge models such as comet @xcite is a way to circumvent this issue and make inferences on an utterance (sentence) level. but the generated knowledge still lacks the detail from the dialogue, as it is trained on the aforementioned knowledge base. our approach, instead, centers on the dialogue dataset and provides more detailed commonsense inference at an utterance level. ## conclusion we introduced cicero, a new dataset for dialogue reasoning with contextualized commonsense inference. it contains ∼53k inferences for five commonsense dimensions -cause, subsequent event, prerequisite, motivation, and emotional reactioncollected from ∼5.6k dialogues. to show the usefulness of cicero for dialogue reasoning, we design several challenging generative and multichoice answer selection tasks for state-of-the-art nlp models to solve. a: hello, old english restaurant. can i help you? b: yes. i'd like to book a table for tonight. a: yes, sir. what time? b: eight o'clock. a: certainly. for how many people? b: there are ten of us. a: ten of you! but we don't usually accept large parties, sir. b: i know, but we are regular customers. a: what's your name please, sir? b: michael peterson. a: mr. peterson...of course! that'll be all right. we'll put two tables together. b: thanks. a: what food would you prefer for today, sir? we have fresh seafood tonight. b: no. we like to eat vegetables. question: what is or could be cause of the target? the speaker is eager to know about the preference of the listener and his friends for the dinner. the speaker is eager to know about the preferences of the listener and his friends for the party. the speaker is eager to know about the preferences of the listener and his friends for the breakfast. the speaker is eager to know about the preference of the listener and his friends for the lunch. the speaker is eager to know about the preference of the listener and his friends for the movie. target: what food would you prefer for today, sir? we have fresh seafood tonight. figure 5 : a data sample of cicero for the cicero mcq task. here, commonsense is required to infer the following events -booking a table at night implies the intention of having dinner. question: what is or could be cause of the target? b: rageh , is that you ? b: wow , you look different . b: i have n't seen you in 10 years . a: hi , mimi , it 's great to see you . a: we have n't seen each other since college . b: do you live in boston now ? a: yes , i work in boston . target: wow , you look different . rageh and mimi met after a long time. mimi and rageh met for the first time. rageh and mimi met when they were both still very young. rageh and mimi didn't meet after a long time. rageh and mimi were unable to meet after a long time. question: what subsequent event happens or could happen following the target? a: i'm trying to get on flight fa2028. am i on time? b: not exactly. it's 6:20 pm now. but lucky for you, that flight has been delayed. a: i never thought i'd be happier about a delay. but hey, that's great news. b: ok, may i check your luggage and tickets, please? a: here you are. which gate do i leave from? b: gate 36, but i'm sorry to tell you that it's been changed to gate 7 and your plane is taking off in 20 minutes. target: gate 36, but i'm sorry to tell you that it's been changed to gate 7 and your planeis taking off in 20 minutes. the speaker and her family rushed towards the gate number 7 to catch the flight. the speaker and her family rushed towards the gate number 7 and they missed the flight. the family rushed towards gate number 8 to catch the flight. the speaker and her family rushed towards the gate number 7 after their flight had been canceled. the speaker and her family rushed towards the gate number 7 but could not catch the flight. figure 6 : instances of temporal commonsense in ci-cero.