Social Commonsense Reasoning with Multi-Head Knowledge Attention (2020)

Abstract: Social Commonsense Reasoning requires understanding of text, knowledge about social events and their pragmatic implications, as well as commonsense reasoning skills. In this work we propose a novel multi-head knowledge attention model that encodes semi-structured commonsense inference rules and learns to incorporate them in a transformer-based reasoning cell. We assess the model’s performance on two tasks that require different reasoning skills: Abductive Natural Language Inference and Counterfactual Invariance Prediction as a new task. We show that our proposed model improves performance over strong state-of-the-art models (i.e., RoBERTa) across both reasoning tasks. Notably we are, to the best of our knowledge, the first to demonstrate that a model that learns to perform counterfactual reasoning helps predicting the best explanation in an abductive reasoning task. We validate the robustness of the model’s reasoning capabilities by perturbing the knowledge and provide qualitative analysis on the model’s knowledge incorporation capabilities.

## introduction humans are able to understand natural language text about everyday situations effortlessly, by relying on commonsense knowledge and making inferences. for example in figure 1 , given two observations: dotty was being very grumpy and she felt much better afterwards -we can come up with a plausible explanation about what could have provoked the change in dotty's emotion. we can also construct alternative hypotheses that will not change dotty's emotion. in order to judge the plausibility of such explanations, we need to have information about mental states and social norms, i.e., a form of commonsense knowledge. such information includes that calling a close friend, in dotty was being very grumpy. observation1: she felt much better afterwards. observation2 : ## semantic & commonsense knowledge this section details the steps we follow to generate social commonsense knowledge about events mentioned in a narrative. see figure 2 for illustration. understanding a narrative text requires the ability to identify events and to reason about their 2 more details about the data are given in the supplement. in a first step we apply srl to extract the basic structure "who did what to whom, when and where" from each sentence in the context, using state-of-the-art srl @xcite . in a second step, we use commonsense transformer (comet2.0, 3 bosselut et al. (2019)) to extract social commonsense knowledge about the extracted events. comet2.0 is trained on the atomic @xcite inferential knowledge resource which consists of 877k everyday events, each characterized by nine relation types (xintent, xneed, xreact, etc.) which we call dimensions. these dimensions connect the event in question with manifold properties, emotions, as well as other states or events. in the last processing step we generate, for each event in each sentence from our datasets, all dimensions defined for it using comet2.0. for example, for: dotty ate something bad we generate (among others) foot_1 the tuple: hpersonx, xreact, sicki and derive hdotty, feels, sicki by substituting personx with the logical subject, the filler of the role arg0. ## a multi-head knowledge attention (mhka) model for social reasoning in this section we introduce the mhka model and discuss some key differences in how mhka works for the two different social commonsense reasoning tasks. for a model overview see figure 3 . ## experimental results this section describes the experiments and results of our proposed model in different configurations. results on ↵nli. our experiment results for the ↵nli task are summarized in ## analysis up to now, we have focused on performance analysis with different experimental settings and model ablations to analyze our model's capacities. now, we turn to leveraging the fact that our model works with semi-structured knowledge in order to obtain deeper insight into its inner workings. ## related work social commonsense knowledge teaching machines to reason about daily events with commonsense knowledge has been an important component for natural language understanding @xcite @xcite . given the growth of interest among researchers in commonsense reasoning, a large body of work has been focused on learning commonsense knowl-edge representations @xcite @xcite . in this work, we address social commonsense reasoning, where knowledge about events and its implications is crucial. rashkin et al. (2018) (event2mind) proposed a knowledge resource for commonsense inference about people's intentions and reactions in everyday events. later, sap et al. (2019) (atomic) extended the event2mind resource with substantially more events, and with nine dimensions (if-then relation types) per event. there has also been work on automatically acquiring commonsense knowledge @xcite @xcite . recently, nasrin mostafazadeh (2020) introduced a large-scale dataset (glucose) capturing ten dimensions of causal explanation (implicit commonsense knowledge) in a narrative context. however, learning to reason over such event-based semistructured knowledge is still a challenging task. in this work, we propose a model which learns to imitate reasoning using such structured knowledge. ## conclusion in this work, we propose a new multi-head knowledge attention model to incorporate semi-structured social commonsense knowledge. we show that our model improves over state-of-the-art lms on two complex commonsense inference tasks. besides the improvement i) we demonstrate a correlation between abduction and counterfactual reasoning in a narrative context, based on the newly proposed task of counterfactual invariance prediction, which we apply to support abductive inference. importantly, ii) we confirm the reasoning capacity of our model by perturbing and adding noise to the knowledge, and performing model inspection using manually validated knowledge rules. in future work, we aim to deeper investigate compositional effects of inferencing, such as the interaction of socially grounded and general inferential knowledge.