Joint Alignment of Multi-Task Feature and Label Spaces for Emotion Cause Pair Extraction (2022)

Abstract: Emotion cause pair extraction (ECPE), as one of the derived subtasks of emotion cause analysis (ECA), shares rich inter-related features with emotion extraction (EE) and cause extraction (CE). Therefore EE and CE are frequently utilized as auxiliary tasks for better feature learning, modeled via multi-task learning (MTL) framework by prior works to achieve state-of-the-art (SoTA) ECPE results. However, existing MTL-based methods either fail to simultaneously model the specific features and the interactive feature in between, or suffer from the inconsistency of label prediction. In this work, we consider addressing the above challenges for improving ECPE by performing two alignment mechanisms with a novel Aˆ2Net model. We first propose a feature-task alignment to explicitly model the specific emotion-&cause-specific features and the shared interactive feature. Besides, an inter-task alignment is implemented, in which the label distance between the ECPE and the combinations of EE&CE are learned to be narrowed for better label consistency. Evaluations of benchmarks show that our methods outperform current best-performing systems on all ECA subtasks. Further analysis proves the importance of our proposed alignment mechanisms for the task.

## introduction emotion cause analysis (eca), detecting potential causes for certain emotion expressions in a document, has been a hot research topic in natural language processing (nlp) community @xcite @xcite . eca has derived three associated tasks: ee, ce and ecpe. as illustrated in figure 1 (a), ee and ce detects the emotion and cause clauses respectively, while ecpe identifies both the emotion and cause clauses as there are close correlations among ee, ce and ecpe, existing ecpe works extensively treat ee and ce as auxiliary tasks for additional feature supports, and mostly adopt the multi-task learning framework to explicitly model the interdependency in between, thus achieving current sota performances @xcite @xcite . from the view of feature encoding, there are two major categories of mtl-based ecpe methods: shared feature encoding method and parallel feature encoding method. as shown in figure 1(b) , shared methods only learn mixed features via one encoder without distinguishing specific features for individual subtasks @xcite . in contrast, parallel methods @xcite use two encoders to learn emotionand cause-specific features separate (cf. figure 1(c )), where unfortunately, the interaction among these tasks are overlooked. we argue that both the private emotion-specific and cause-specific features and the shared interactive feature are important to the final performance, which should be explicitly modeled in the mtl framework. to this end, we in this work propose a feature-task alignment (fta) scheme of mtl for ecpe (cf. figure 1(d) ), in which we explicitly split three parts of features, and align them to ee, ce and ecpe respectively. meanwhile, aligning the label space in a mtl framework is crucial to overall ecpe performance, because intuitively all modules in the mtl process should reach a consensus. for example as in figure 1 (a), once "c1" is recognized as a cause clause by ecpe module, it should not be further predicted as a non-cause clause by ce module. we notice that such label consistency is not guaranteed in existing mtl-based ecpe methods, which could inevitably hurt the prediction. therefore we further introduce a inter-task alignment (ita) mechanism (cf. 1(e)) that learns to pull closer the label distance between the ecpe and the combinations of ee&ce, ensuring label consistency. we implement the above ideas of feature-task alignment and inter-task alignment by developing a novel neural network, namely a 2 net, as shown in figure 2 . first, we employ the bert @xcite as document encoder for producing clauses representations. we then leverage the partition filter network (pfn) @xcite to implement the feature-task alignment, generating emotion-specific features, cause-specific features and interaction features, respectively. afterwards, we apply emotion-specific and interaction features for ee, cause-specific and interaction features for ce, and all features for ecpe. finally, we reach the goal of inter-task alignment by minimizing the bidirectional kl-divergence between the output distributions of ecpe and ee×ce, thus maintaining the consistency of label spaces among all tasks. our a 2 net framework is evaluated on the eca benchmark @xcite , where our system achieves new sota results on ee, ce and ecpe. further analyses demonstrate that our method learns better consistency in the predictions of all subtasks than existing baselines. overall, this work contributes to three major aspects: ## related work in nlp area, the analysis on sentiment and opinion is a long-standing research topic @xcite @xcite , including detecting of the sentiment polarities @xcite @xcite and the emotion categories @xcite . one of the recent trend on the emotion detection has been upgraded to the emotion cause analysis (eca). centered on the topic of eca, there are several subordinated tasks according to the extracting elements of emotion and cause, such as emotion cause extraction (ece) @xcite @xcite and ecpe @xcite @xcite . @xcite pioneer the ece task, in which the task is formulated as a word-level cause labeling problem. following this work, initial research constructs rule-based methods @xcite and machine learning methods @xcite on their own corpus. deep learning based methods greatly facilitate the line of this research @xcite @xcite . recently, @xcite release a public corpus and re-formalize ece as a clause-level classification problem, where the goal is to detect cause clauses for a given emotion in the text. the framework has received much attention in recent years and the corpus has become a benchmark eca dataset @xcite @xcite @xcite @xcite . however, as @xcite points out, the ece task is limited to the task definition, i.e., emotion needs to be manually marked in advance. they thus introduce the ecpe task that simultaneously extract both the emotion and cause clauses as well as determining their relations, which has a better utility in real-world applications @xcite . thereafter, a line of subsequent research efforts are paid to ecpe within the last years @xcite @xcite . recent ecpe methods mostly employ the multitask learning for simultaneously modeling the ee and ce as auxiliary tasks for making use of the shared features, and thus realize sota ecpe performances @xcite . existing mtl-based ecpe works can largely be divided into two categories: parallel encoding and shared encoding methods. parallel methods mostly learn the emotion/cause feature representations in mutually independent ways @xcite @xcite . @xcite ; @xcite uses auxiliary task prediction to aid the interactions between emotion and cause features. however, the prediction values are limited to only a two-dimensional vector, leading to insufficient interaction between emotion and cause features. shared methods learn the mixed features by only one encoder without distinguishing between features of different tasks @xcite @xcite . in this work, we use the pfn @xcite to generate emotionspecific features, cause-specific features and interaction features and implement the feature-task alignment. moreover, we use an inter-task alignment module to reduce the gap between ee, ce and ecpe, maintaining the consistency of the label space. ## methodology task formulation given a document consisting of n clauses @xmath0, c 2 , . . . , c n }, and each c i denotes a subsection of a sentence separated by a comma. the goals of ee and ce task are extracting emotion clauses c e @xmath1, respectively, while ecpe task identifies the emotion-cause clause pair (c e i , c c j ) that has causal relationship between emotion and cause clauses. as illustrated in figure 2 , the overall architecture of our a 2 net consists of four tiers, including the encoder layer, feature-task alignment layer, prediction layer and inter-task alignment mechanism. first, following the previous work and we use bert @xcite as encoder to yield contextualized clause representations from input documents. then, to explicitly model task-specific features and task-shared features. we leverage a pfn @xcite to capture emotion-and cause-specific features and the interaction between them. afterward, a prediction layer is used to align three combinations of three kinds of features from pfn with three tasks, and predict the emotion clauses, cause clauses and emotion-cause pair for ee, ce and ecpe, respectively. finally, considering that there should be a consensus among all tasks. we propose an inter-task alignment mechanism to enhance the consistency between ecpe and ee×ce. ## conclusion existing best-performing ecpe works extensively leverage ee and ce as auxiliary tasks for better feature learning via multi-task learning (mtl). in this paper, we further enhance the existing best-performing mtl-based ecpe by proposing feature-task alignment and inter-task alignment mechanisms. at the feature space, the feature-task alignment mechanism aligns the task-specific features and the shared interactive feature with corre-sponding tasks. at the label space, the inter-task alignment mechanism reduces the inconsistency among the predicted labels of ee, ce and ecpe. experimental results on the benchmark ecpe data demonstrate the effectiveness of our methods. further analysis shows that our system achieves better consistency than existing baselines, which explains the improvements of our model. the idea to align the feature space and label space in mtl framework is promising. in the future work, we consider further constructing intra-clause relations, inter-clause relations, and relations among pairs of clauses for ecpe.