How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning (2023)

Abstract: Our investigation into the Affective Reasoning in Conversation (ARC) task highlights the challenge of causal discrimination. Almost all existing models, including large language models (LLMs), excel at capturing semantic correlations within utterance embeddings but fall short in determining the specific causal relationships. To overcome this limitation, we propose the incorporation of i.i.d. noise terms into the conversation process, thereby constructing a structural causal model (SCM). It explores how distinct causal relationships of fitted embeddings can be discerned through independent conditions. To facilitate the implementation of deep learning, we introduce the cogn frameworks to handle unstructured conversation data, and employ an autoencoder architecture to regard the unobservable noise as learnable “implicit causes.” Moreover, we curate a synthetic dataset that includes i.i.d. noise. Through comprehensive experiments, we validate the effectiveness and interpretability of our approach. Our code is available in https://github.com/Zodiark-ch/mater-of-our-EMNLP2023-paper.

## introduction nowadays, numerous conversation recognition tasks (such as emotion recognition in conversation (erc) task @xcite , intent recognition (ir) task @xcite and dialogue act recognition (dar) task @xcite ) have shown promising performance in specialized supervised and unsupervised methods. considering the roberta pretrained model @xcite as the examples, "my eyelids are fighting" and "i want to sleep," which have similar semantics but different tokens can be well fitted within embeddings. (i.e., these two embeddings exhibit a strong resemblance via certain metrics such as cosine similarity.) however, when it comes to the relationship between two utterances, denoted as a and b, wherein their embeddings can be fitted, various possible relationships exist: a acts as the cause of b (@xmath0, a acts as the outcome of b (a ← b), or more complex, a and b are both influenced by a common cause (a ← @xmath1, and so on. particularly in reasoning tasks @xcite , it is crucial for these methods to transcend the mere fitting of embeddings and possess the capacity to discriminate diverse causal relationships. (i.e., the ability of causal discrimination) @xcite . to specifically investigate the causal discrimination capability of existing methods in conversation, we narrow down our research to a particular task: affective reasoning in conversation (arc), which has included emotion-cause pair extraction (ecpe) @xcite and emotion-cause span recognition (ecsr) @xcite . we begin with conducting tests to evaluate the causal discrimination of existing methods including the large language models (llms) @xcite . one typical evaluation involves the causal reversal test: for emotion-cause utterance pairs with true labels (a, b) representing a causal relationship of @xmath2, we scrutinize the predictions generated by the existing methods using both positive pairs (a, b) and negative pairs (b, a). the results reveal that all the examined methods performed similarly across the two sample types. as we are concerned, they lacked causal discriminability. (details are shown in section 2.3) in order to discriminate different causal relationships between two similar embeddings, we construct the dialogue process as a structural causal model (scm). many endeavors @xcite supporting that i.i.d. noise of scm could facilitate the discrimination of causal relationships when fitting two variables. under the presence of noise, each utterance is not only explicitly influenced by the other utterances but also implicitly influenced by the i.i.d. exoge-nous noise. consequently, this framework ensures that two fitted embeddings result in diverse causal relationships, which are determined by corresponding independent conditions between the residual terms and embeddings. for simplicity, we refer to other utterances as explicit causes and exogenous noise as implicit causes. furthermore, to enable the learnability of such causal discrimination within embeddings, we propose a common skeleton, named centering one graph node (cogn) skeleton for each utterance derived from some broadly accepted prior hypotheses. it can address the challenges arising from variable-length and unstructured dialogue samples. subsequently, we develop an autoencoder architecture to learn the unobservable implicit causes. specifically, we consider the implicit causes as latent variables and utilize a graph attention network (gat) @xcite to encode its representation. additionally, the decoder leverages the inverse matrix of the causal strength, ensuring an accurate retrieval of the causal relationships. finally, we conduct extensive experimental evaluations: 1) our approach significantly outperforms existing methods including prominent llms (gpt-3.5 and gpt-4) in two affective reasoning tasks (ecpe and ecsr) and one emotion recognition task (erc), demonstrating its effectiveness in affective reasoning. 2) our method exhibits a significant reduction in false predictions for negative samples across three causal discrimination scenarios. 3) we curate a synthetic dataset with implicit causes to visualize the latent variable in our implementation. our contribution is four-fold: ## methodology in this section, we begin by outlining incorporating i.i.d. noise terms into a dialogue model to construct an scm in section 3.1, demonstrating independent residual allowing for the identification of more specific causal relations within pairs of fitted utterances. next, to mitigate conflicts between scm models and dialogue data, we designed cogn skeletons with six instantiations in section 3.2. finally, we propose a deep learning implementation to tackle the issue of noise being unknown in dialogue data in section 3.3. ## experiments in this section, we conduct extensive experiments to answer the 3 research questions: rq1: how effective is our method in affective reasoning tasks? rq2: how do we justify the causal discriminability of our method? rq3: how do we gauge the difference between the latent variable e and designed implicit causes? ## conclusion the results of testing prevalent approaches on the arc task have demonstrated that almost all approaches are unable to determine the specific causal relationship that leads to the association of two well-fitted embeddings. in order to enhance the causal discrimination of existing methods, we constructed a scm with i.i.d. noise terms, and analyzed the independent conditions that can identify the causal relationships between two fitted utterances. moreover, we proposed the cogn framework to address the unstructured nature of conversation data, designed an autoencoder implementation to make implicit cause learnable, and created a synthetic dataset with noise labels for comprehensive experimental evaluation. while our method still has some limitations, such as confounders and the inability to scale to all methods, we hope that our theory, design, and model can provide valuable insights for the broader exploration of this problem to demonstrate that our work is de facto need for identifying causal relationships. mao ye, haitao wang, and zheqian chen. 2023. msmix: an interpolation-based text data augmentation method manifold swap mixup. arxiv preprint arxiv:2305.19617. sayyed zahiri and jinho d. choi. 2018. emotion detection on tv show transcripts with sequence-based convolutional neural networks. in proceedings of the aaai workshop on affective content analysis, affcon'18, pages 44-51, new orleans, la. dong zhang, liangqing wu, changlong sun, shoushan li, qiaoming zhu, and guodong zhou. 2019. modeling both context-and speaker-sensitive dependence for emotion detection in multi-speaker conversations. in ijcai, pages 5415-5421.