In-context Contrastive Learning for Event Causality Identification (2024)

Abstract: Event Causality Identification (ECI) aims at determining the existence of a causal relation between two events. Although recent prompt learning-based approaches have shown promising improvements on the ECI task, their performance are often subject to the delicate design of multiple prompts and the positive correlations between the main task and derivate tasks. The in-context learning paradigm provides explicit guidance for label prediction in the prompt learning paradigm, alleviating its reliance on complex prompts and derivative tasks. However, it does not distinguish between positive and negative demonstrations for analogy learning. Motivated from such considerations, this paper proposes an **I**n-**C**ontext **C**ontrastive **L**earning (ICCL) model that utilizes contrastive learning to enhance the effectiveness of both positive and negative demonstrations. Additionally, we apply contrastive learning to event pairs to better facilitate event causality identification. Our ICCL is evaluated on the widely used corpora, including the EventStoryLine and Causal-TimeBank, and results show significant performance improvements over the state-of-the-art algorithms.

## introduction event causality identification (eci) is to detect whether there exists a causal relation between two event mentions in a document. it is of great importance for many natural language processing (nlp) applications, such as question answer @xcite , machine reading comprehension @xcite , and etc. furthermore, it also has many practical applications in real-world scenarios, such as event prediction @xcite and strategy optimization @xcite example from the event storyline corpus (esc). we concatenated two causal demonstrations and two non-causal demonstrations before the query to be predicted, and enhanced the analogy between the query and demonstrations through contrastive. ultimately, our iccl model determined the causality between the two events, "died" and "shield", in the query. some graph-based methods have been proposed for the eci task @xcite @xcite , which apply a graph structure to represent events and their potential relations. for example, @xcite initialize event nodes' embeddings using a documentlevel encoder and employ a graph inference mechanism to update their embeddings. @xcite incorporate causal label information and event pair interaction information to enhance the representation learning for event nodes in the graph. these methods follow the traditional representation learning for classification yet on a graph structure. recently the prompt learning paradigm @xcite has shown its great successes in many nlp tasks, as it can well leverage the potentials of a pre-trained language model (plm). some researchers have applied the prompt learning for the eci task @xcite . for example, the dpjl model @xcite designs a main cloze task but also designs two derivative prompt tasks. although the dpjl has achieved new state-of-the-art performance, it involves the delicate design of multiple prompts and relies on the positive correlations between the main task and derivative tasks. the in-context learning paradigm @xcite includes some demonstrations with their ground-truth labels into the query prompt to learn some patterns hidden in demostrations when making its prediction. however, it does not distinguish between positive and negative demonstrations for analogy. motivated from such considerations, we propose to use contrastive learning on the in-context demonstrations to enhance the effectiveness of analogy, as illustrated in fig. 1 . besides, we also argue that the semantic of event mentions are the most important for the causal relation identification between them. as such we apply contrastive learning to the representation of event mentions in in-context demonstrations, so as to distinguishing the semantic between causal and non-causal event pairs and facilitating event causality predictions. in this paper, we propose an in-context contrastive learning (iccl) model for the eci task. the iccl model contains three modules. the prompt learning module reformulates an input event pair and some retrieved demonstrations into a prompt template, as the input for plm encoding. the in-context contrastive module optimizes the representation of event mention by simultaneously maximizing its agreement with positive demonstrations and minimizing with negative ones, via a contrastive loss. the causality prediction module predicts answer word to identify causal relations. experiments are conducted on the widely used eventstoryline and causal-timebank corpora, and results have shown that our iccl achieves the new state-of-the-art performance for the eci task. ## method fig. 2 illustrates our iccl model, including the prompt learning module, the in-context contrastive module, and the causality prediction module. ## concluding remarks in this paper, we propose an iccl model and apply it on the eci task. we leverage the causality knowledge of plm by introducing explicit guidance through the inclusion of demonstrations, rather than relying on the design of complex prompts. meanwhile, we employ contrastive learning with event pairs as input to enhance the plm's attention to event pairs and strengthen the analogy between query and demonstrations. experiments on the esc and ctb corpus have validated that our iccl can significantly outperform the state-of-the-art algorithms. in future, we will try to undertake experiments to apply our proposed framework to other nlp tasks in order to explore whether it can exhibit favorable adaptability when applied to different tasks.