Lifelong Knowledge-Enriched Social Event Representation Learning (2021)

Abstract: The ability of humans to symbolically represent social events and situations is crucial for various interactions in everyday life. Several studies in cognitive psychology have established the role of mental state attributions in effectively representing variable aspects of these social events. In the past, NLP research on learning event representations often focuses on construing syntactic and semantic information from language. However, they fail to consider the importance of pragmatic aspects and the need to consistently update new social situational information without forgetting the accumulated experiences. In this work, we propose a representation learning framework to directly address these shortcomings by integrating social commonsense knowledge with recent advancements in the space of lifelong language learning. First, we investigate methods to incorporate pragmatic aspects into our social event embeddings by leveraging social commonsense knowledge. Next, we introduce continual learning strategies that allow for incremental consolidation of new knowledge while retaining and promoting efficient usage of prior knowledge. Experimental results on event similarity, reasoning, and paraphrase detection tasks prove the efficacy of our social event embeddings.

## introduction everyday life comprises the ways in which people typically act, think, and feel on a daily basis. our life experiences unfold naturally into temporally extended daily events. the event descriptions can be packaged in various ways depending on several factors like speaker's perspective or the related domain. interpretation of event descriptions will be incomplete without understanding multiple entities involved in the events and even more so when the focus is primarily on "social events", s1: student goes to class s2: student goes to wedding s3: student takes course s4: teacher takes course s5: professor teaches subject x s1 figure 1 : illustration of functioning of our lifelong representation learning approach that produces incrementally richer social event representations. event texts are given in the top green box. with more knowledge, social event embeddings move beyond high lexical overlap @xcite ] and learn to integrate semantic and pragmatic properties @xcite , (c)] of event texts along with social role information @xcite ]. understanding the pragmatic implications of social events is non-trivial for machines as they are not explicitly found in the event texts. prior studies @xcite @xcite often extract the syntactic and semantic information from the event descriptions but ignore the pragmatic aspects of language. in this work, we address this shortcoming, and aim to (a) disentangle semantic and pragmatic attributes from social event descriptions and (b) encapsulate these attributes into an embedding that can move beyond simple linguistic structures and dispel apparent ambiguities in the real sense of their context and meaning. towards this goal, we propose to train our models with social commonsense knowledge about events focusing specifically on intents and emotional reactions of people. such commonsense understanding can be obtained from existing knowledge bases like conceptnet @xcite , event2mind/atomic @xcite or by collecting more noisy commonsense knowledge using data mining techniques. as new domain sources emerge, each containing different knowledge assertions, it is essential that the representation models for social events keep evolving with this growing knowledge. since it is generally infeasible to retrain models from scratch for every new knowledge source, we consider the need to employ prominent continual learning practices @xcite @xcite and enable semantic and pragmatic enrichment of social event representations. this problem can be addressed from the perspective of incremental domain adaptation @xcite , which quickly adapts to new domain knowledge without interfering with existing ones. figure 1 presents a sample functioning scenario producing incrementally richer social event embeddings. as the model gains more knowledge from different sources, it learns to discern events based on semantic and pragmatic properties, including social roles. for example, "student takes course", and "teacher takes course" has significant lexical and semantic relatedness. however, the social role information changes the meaning as depicted in figure 1 (d) with the introduction of our in-house dataset (sb-sck). in this paper, we develop a lifelong representation learning approach for embedding social events from their free-form textual descriptions. our model augments a growing set of knowledge obtained from various domain sources to allow for positive knowledge transfer across these domains. our contributions are as follows: ## problem formalization formally, we assume that our learning framework has access to streams of social commonsense knowledge data obtained from n different domains, denoted by @xmath0, d 2 , ..., d n }. at a particular point in time, we extract knowledge from the current domain d i . we produce an embedding of social events by consolidating the accumulated knowledge across the modeled domains @xmath1. data from each domain source contains source-specific textual descriptions of social situations and their intuitive commonsense information such as intents and emotions. training samples, drawn from a domain dataset d i , could contain either a significant overlap or an entirely new set of knowledge when compared with the previously processed domains d 1:i . given such a setup, we aim to generate incrementally richer social event representations using our continual learning framework. ## datasets for our representation learning task, we aggregate social commonsense knowledge data from various domain sources. this knowledge contains details about pragmatic aspects like intents and emotional reactions. we create a continual learning benchmark based on these commonsense data sources foot_0 . ## experiments we conduct several experiments to study the power of our learned embeddings. our experiments are designed to answer the following questions: rq1: how well does our model perform in comparison to other continual learning approaches for intent-emotion prediction? rq2: to what extent do our modeling choices impact the results in predicting intents & emotions? rq3: does our model outperform existing stateof-the-art methods in hard similarity task that evaluates the effectiveness of the learned embeddings? rq4: do the learned embeddings demonstrate transfer capability to downstream tasks -paraphrase detection & social iqa reasoning? ## conclusion humans rely upon commonsense knowledge about social contexts to ascribe meaning to everyday events. in this paper, we introduce a lifelong learning approach to effectively embed social events with the help of a growing set of social commonsense knowledge assertions acquired from different domains. first, we leverage social commonsense knowledge to sharpen social event embeddings with semantic and pragmatic attributes. next, we employ domain-representative episodic memory replay (dr-emr) to overcome catastrophic forgetting and enable positive knowledge transfer with the emergence of new domain knowledge. by evaluating on a corpus of social events aggregated from multiple sources, we establish that our model is able to outperform several baselines. experimental results on downstream tasks like event similarity, reasoning, and paraphrase detection demonstrate the capabilities of our social event embeddings. we hope that our work will motivate further exploration into lifelong representation learning of social events and advance the research in inferring pragmatic dimensions from texts.