Event Causality Identification with Synthetic Control (2024)

Abstract: Event causality identification (ECI), a process that extracts causal relations between events from text, is crucial for distinguishing causation from correlation. Traditional approaches to ECI have primarily utilized linguistic patterns and multi-hop relational inference, risking false causality identification due to informal usage of causality and specious graphical inference. In this paper, we adopt the Rubin Causal Model to identify event causality: given two temporally ordered events, we see the first event as the treatment and the second one as the observed outcome. Determining their causality involves manipulating the treatment and estimating the resultant change in the likelihood of the outcome. Given that it is only possible to implement manipulation conceptually in the text domain, as a work-around, we try to find a twin for the protagonist from existing corpora. This twin should have identical life experiences with the protagonist before the treatment but undergoes an intervention of treatment. However, the practical difficulty of locating such a match limits its feasibility. Addressing this issue, we use the synthetic control method to generate such a twin’ from relevant historical data, leveraging text embedding synthesis and inversion techniques. This approach allows us to identify causal relations more robustly than previous methods, including GPT-4, which is demonstrated on a causality benchmark, COPES-hard.

## introduction previous endeavours in event causality identification in text have, to a large extent, depended on feature-based approaches where linguistic patterns serve as a crucial role @xcite @xcite . these patterns can roughly indicate causal relations in that causal language is widely used in an informal way in everyday life @xcite . without proper manipulation of the potential cause, and comparison between the observed outcome and the intervened outcome, these approaches often identify specious causal relations. for example, "because" is often considered as a causal indicator @xcite , yet it might not be rigorous as in the case of "she got a nice job because she graduated from one of the top universities." it is very possible that the employer paid more attention to the candidate's ability, rather than just the educational background in offering a job. regardless of how these linguistic features are obtained -whether extracted from causal keywords and semantic indications, or obtained from language models -any featurebased approach may be predisposed to bias due to their unreliable causality foundations. thus, highly sophisticated methods that rely on multihop reasoning on graphs for eci @xcite @xcite @xcite , also risk being fundamentally flawed in their conclusions. if we want to reliably discover event causality, say whether there exists a causal relation between a pair of temporally ordered events (e 1 , e 2 ), we need to manipulate e 1 and see if e 2 still happens in a 'parallel universe' in which e 1 does not happen. in other words, we want to find a 'twin' for the protagonist p in the events, who has identical life experiences with p (i.e., a sequence of events) up to the point when e 1 takes place, but instead undergoes an intervention of e 1 . however, it is almost impossible to implement this idea in the text domain (e.g., stories, narratives, and news reports): for any event in text, it is very rare that its protagonist has 'twins' that satisfy the aforementioned requirements. in this work, as a workaround, we attempt to synthesize 'twins' by merging relevant event sequences retrieved from a corpus, inspired by a causal inference method, called synthetic control, used in economics and social sciences @xcite @xcite . such event sequence merging is also seen in recent effort in event schema induction in information extraction studies @xcite @xcite . specifically, our approach 1 consists of three components: (1) noncontemporary control group retrieval, (2) control unit synthesis, and (3) treatment effect estimation. given a pair of events with certain context, the first component retrieves relevant event sequences from historical data that can be leveraged to synthesize 'twins' via text embedding and inversion techniques @xcite in the second component. and this is followed by the third component which calculates a causal estimand to determine whether there exists a causal relation between the two events. the proposed methodology fundamentally shifts from conventional eci methods by introducing the concept of synthetic control to the text domain. this allows the inherent linguistic bias which is prevalent in data-driven eci approaches to be significantly mitigated. moreover, by introducing full-context matching in a continuous space, we overcome the limitation of discrete temporal propensity matching proposed in previous attempts @xcite of solving eci with the potential-outcome framework. our approach demonstrates significantly improved results on the copes-hard dataset, a commonly used causality benchmark, by at least 9% (relatively) over existing methods and gpt-4. the contribution of this paper is threefold: ## preliminaries in this section, we present the fundamentals of our method, which is grounded on the rubin causal model @xcite and discuss its previous application to the problem of event causality identification in the text domain. and then we discuss the limitation of previous attempts and introduce why we adopt synthetic control in this work. ## method for a pair of events e 1 and e 2 mentioned in some context that we consider as the study unit, we want to (1) find relevant stories (event sequences) from historical data that can be considered as noncontemporary control group, and (2) merge them to create a synthetic control unit, and then (3) calculate the causal estimand. ## experiments we conduct experiments to demonstrate the effectiveness of our proposed approach. ## conclusion our work shows that creating counterfactuals with synthetic control, a concept that has been widely adopted in other disciplines such as economics, can be effectively applied to event causality identification under zero-shot settings. this retrievalbased method instills more confidence in the result, offering more robust performance in tasks at which state-of-the-art llms might fail. our re-sults also open up opporunities for research with more complex datasets and causal relationships.