DICP : Deep In-Context Prompt for Event Causality Identification (2025)

Abstract: Event causality identification (ECI) is a challenging task that involves predicting causal relationships between events in text. Existing prompt-learning-based methods typically concatenate in-context examples only at the input layer, this shallow integration limits the model’s ability to capture the abstract semantic cues necessary for identifying complex causal relationships. To address this limitation, we propose a novel model called Deep In-Context Prompt (DICP), which injects in-context examples into the deeper layer of a pre-trained language model (PLM). This strategy enables the model to leverage the hierarchical semantic representations formed in deeper layers, thereby enhancing its capacity to learn high-level causal abstractions. Moreover, DICP introduces a multi-layer prompt injection mechanism, distributing diverse in-context examples across multiple transformer layers. This design allows the model to recognize a broader range of causal patterns and improves its generalization across different contexts. We evaluate the DICP model through extensive experiments on two widely used datasets, demonstrating its significant improvement in ECI performance compared to existing approaches. Furthermore, we explore the impact of varying the number of deep layers on performance, providing valuable insights into the optimal layer configuration for ECI tasks.

## introduction event causality identification (eci) involves predicting the causal relationships between events mentioned in the text. for instance, consider the sentence s 1 : "john won the competition because of his practice." here, the two event mentions practice and won, are linked through the explicit cue phrase "because of", which directly signals that * corresponding author practice causes won. in contrast, in sentence s 2 : "the reporter was killed during the attack.", the causal relationship between attack and killed is implicit, as it lacks explicit connective phrases. this highlights a key challenge in eci: while explicit causality may be detected through lexical signals, recognizing implicit causality requires deeper contextual and semantic understanding. accurate identification of such causal relationships is foundational for a wide range of downstream natural language processing (nlp) applications, such as question answering @xcite and machine reading comprehension @xcite . recently, researchers have focused on finetuning pre-trained language models (plms) to tackle eci tasks, resulting in significant performance improvements. among these approaches, the prompt learning paradigm has emerged as particularly promising @xcite . this approach fine-tunes plms by concatenating taskspecific prompts with input, guiding the model to generate more accurate predictions. a notable example is the iccl model @xcite , which incorporates the concept of in-context learning @xcite . iccl enhances the model's causal reasoning by concatenating task-specific examples to the input as prompts, allowing the model to infer patterns from relevant prior instances. this method has achieved state-of-the-art performance, particularly in cases involving implicit causality. however, existing prompt-based eci models @xcite suffer from a key limitation: they restrict prompt information to the input layer only. this architectural constraint prevents the model from fully leveraging the rich semantic information embedded in taskspecific prompts. recent insights into plm internals suggest that deeper layers capture more abstract and semantically complex concepts @xcite . motivated by this observation, we conducted a preliminary experiment on the esc @xcite dataset, evaluating the effects of injecting in-context examples at various layers of plms. as illustrated in figure 1 , injecting in-context examples into deeper layers led to substantial performance improvements over input-level-only in-context examples concatenation. these findings underscore the importance of aligning the semantic complexity of the task with the representational depth of the model. building on this insight, we propose a novel model called deep in-context prompt (dicp) for the eci task. dicp overcomes the limitations of shallow prompt injection by embedding diverse in-context examples directly into deeper layers of a plm, thereby enabling the model to capture higher-order causal abstractions. furthermore, dicp enhances this process by incorporating diverse in-context examples across various deeper layers. this strategy allows the model to efficiently capture diverse causal patterns from these in-context examples, further refining its ability to recognize complex causal relationships. additionally, we employ an abstract meaning representation (amr) parser to comprehend semantic structures and extract event-related information from the text. in summary, our contributions are as follows: ## methodology to address the challenges of recognizing complex causality patterns in text, we propose the deep in-context prompt (dicp) framework, which integrates three synergistic components: the encoding in-context module, the deep prompt learning module, and the semantic structure module, as shown in figure 2 . these components are designed to enhance the capabilities of pre-trained language models (plms) by incorporating semantic and contextual information tailored for event causality identification (eci) tasks. ## conclusion in this paper, we propose dicp for the eci task. dicp integrates three synergistic components to enhance its performance. first, dicp selects a set of diverse in-context examples and encodes them into semantic embeddings that reflect a wide range of causal scenarios. second, these embeddings are strategically injected across multiple deeper layers of a plm, enabling the model to learn abstract, high-level causal representations that go beyond surface-level patterns. third, dicp incorporates an amr parser to extract and integrate structured semantic information, thereby further enriching the model's ability to infer complex, implicit causal relationships. we validate dicp through comprehensive experiments on two widely used eci datasets. results show that dicp consistently outperforms existing methods, particularly in capturing nuanced and context-dependent causal patterns. by address-ing the limitations of input-only prompt concatenation, dicp leverages the semantic depth of plms to achieve state-of-the-art performance in eci.