Causal Discovery Inspired Unsupervised Domain Adaptation for Emotion-Cause Pair Extraction (2024)

Abstract: This paper tackles the task of emotion-cause pair extraction in the unsupervised domain adaptation setting.The problem is challenging as the distributions of the events causing emotions in target domains are dramatically different than those in source domains, despite the distributions of emotional expressions between domains are overlapped. Inspired by causal discovery,we propose a novel deep latent model in the variational autoencoder (VAE) framework, which not only captures the underlying latent structures of data but also utilizes the easily transferable knowledge of emotions as the bridge to link the distributions of events in different domains. To facilitate knowledge transfer across domains, we also propose a novel variational posterior regularization technique to disentangle the latent representations of emotions from those of events in order to mitigate the damage caused by the spurious correlations related to the events in source domains. Through extensive experiments, we demonstrate that our model outperforms the strongest baseline by approximately 11.05% on a Chinese benchmark and 2.45% on a English benchmark in terms of weighted-average F1 score. We have released our source code and the generated dataset publicly at: https://github.com/tk1363704/CAREL-VAE.

## introduction emotion-cause pair extraction (ecpe) aims to extract emotions and the events causing such emotions mentioned in a document @xcite . the task has potential applications in a number of areas, such as affective computing, market analysis, and intelligent agents for customer support. however, there are only a small number of labeled training corpora available in a * equal contribution. multi-class or multi-label classification dominates in conventional uda tasks. uda-ecpe is more challenging because the events causing the same emotion are barely the same across domains, despite the knowledge of emotional expressions is easier to transfer across domains using the uda methods @xcite . for example, the reason for "i feel so happy today" can be "i have received a grant from the government" in the society domain and "i found that the stock i bought went up" in the finance domain. there are usually no explicit keywords such as "because" showing their causal relations. however, current uda methods assume that there are small discrepancies between source and target distributions @xcite . we show in sec. 4.2 that the state-of-the-art (sota) uda methods indeed have limited capabilities to improve the performance of the sota ecpe models. it is a common practice to project texts into latent representations for improving language understanding @xcite . existing techniques disentangle different types of latent representations by applying regularization terms to enforce independence between the corresponding random variables @xcite . however, the independence assumption contradicts the fact that emotions and the events causing them are statically dependent. furthermore, large language models (llms), as a powerful general-purpose natural language processing tool, should be wellsuited for the uda-ecpe task. however, in many related studies @xcite @xcite @xcite @xcite , researchers have found that llms are not particularly effective at solving causal discovery tasks. to tackle the above challenges, we take the transferable knowledge of emotional expressions as the bridge between a source domain and a target domain. in a single domain, we identify causal relations between emotions and domain-specific events, which can be viewed as a causal discovery problem between the corresponding random variables. in the vae framework @xcite , we propose a novel model, coined carel-vae, to map inputs texts into latent emotion representations and latent event representations and detect their causal relations. herein, we propose a novel variational posterior regularizer to disentangle those representations by maximizing the divergences between the posteriors without assuming independence. in a target domain, we improve the self-training algorithm @xcite for discovering domain-specific causal relations, referred to as cd-selftrain. instead of incrementally updating a training set, we improve the original algorithm by producing a new pseudo-labeled training set in each epoch. as a result, our method outperforms the sota ecpe models trained with the sota uda methods by a wide margin. to sum up, our contributions are the following: the task ecpe is concerned with recognizing causal relations between the events causing emotions and the corresponding emotional expressions mentioned in a document. all prior studies on the ecpe task employ a (deep) learning-based classifier to detect mentions of causal relations based on an input text. they often choose an input text that mentions an event and an emotional expression. then those classifiers determine whether the event causes the emotional expression by investigating if i) the event and the emotional expression are correlated and ii) there is a linguistic pattern indicating their relation is causal, e.g. using a key phrase "leads to". formally, given an input text x, we extract an event embedding z c and an emotion embedding z e , which are the values sampled from the corresponding latent random variable vectors z c and z e . in a source domain, a model learns a distribution z c ,z e p(y |z c , z e , x)p(z c , z e |x), where y denotes a binary random variable indicating if there is a causal relation between z c and z e . the key challenge is that both p(y |z c , z e , x) and p(z c , z e |x) are significantly different in target domains. although prior studies show that p(z e |x) can be easily transferred from source domains to target domains @xcite , the correlations between z c and z e are almost not transferable, because p(z c ) are dramatically different between domains. therefore, when adapting a model trained in a source domain to a target domain, the model needs to forget the correlations between emotions and events from the source domain, followed by learning new correlations in the target domain. to provide an intuitive understanding of the above challenges in the uda setting, we visualize the clause embeddings, namely p(z c ), for ground-truth emotion and emotion causes respectively on ch-ecpe and en-ecpe, and compare them with the sentence embeddings for a widely used domain adaptation corpus amazon reviews @xcite using t-sne. as the original ch-ecpe are not partitioned based on domains, we manually assign each data point in the corpus with the corresponding domain label. further details are provided in sec. 4.1. as shown in figure 2 , the data points of chinese emotion clauses from various ch-ecpe's domains are strongly overlapped, the domain divergences are far smaller than those of the embeddings of the emotion causes. it is thus challenging for existing uda methods, which work only in the cases that the distribution shift from a source domain to a target domain is small, as illustrated in fig. 2a @xcite . in addition, we employ two different datasets as different domains for english. the english corpora similar tendency can be found in a.1. as shown in fig. 4 , regardless if a clause mentions an emotion or an emotion cause, there is a very clear boundary between the two domains. their domain differences are largely caused by the differences between the two datasets. ## methodology the uda-ecpe task is concerned with identifying causal relations between mentions of events and emotional expressions in target domains, which do not have labeled data. in the source domain, there is a set of labeled documents and is annotated with a set of labeled emotion-cause pairs r s @xmath0, y c i , y e j )} i,j , where y r ij is a binary label indicating if x i is an event mention causing an emotion expressed in x j , y c i denotes whether x i is an event or not, and y e @xmath1. in this work, we consider the widely used six basic emotion categories: happiness, sadness, fear, disgust, anger, and surprise. then the task is to identify a set of such causal relations and emotion categories r t @xmath2, y e j )} i,j from each unlabeled document k in target domains. in contrast, the prior studies @xcite assume the training and test distributions are identi-cal and emotional expressions are not categorized. hence, our setting is more difficult and practical by considering emotion categories and distribution discrepancies between domains. ## conclusion we propose a novel causal discovery inspired vae model and a customized self-training algorithm for the uda-ecpe task. herein, we propose to disentangle the latent representations of emotions from those of events by a novel variational posterior regularization technique that does not enforce independence between the corresponding latent random variables. this work also sheds the light on the connections between the task of causal relation identification in the nlp community and the causal discovery theory, paves the way for theoretically grounded approaches to comprehensively analyzing causal structures in texts.