Honey or Poison? Solving the Trigger Curse in Few-shot Event Detection via Causal Intervention (2021)

Abstract: Event detection has long been troubled by the trigger curse: overfitting the trigger will harm the generalization ability while underfitting it will hurt the detection performance. This problem is even more severe in few-shot scenario. In this paper, we identify and solve the trigger curse problem in few-shot event detection (FSED) from a causal view. By formulating FSED with a structural causal model (SCM), we found that the trigger is a confounder of the context and the result, which makes previous FSED methods much easier to overfit triggers. To resolve this problem, we propose to intervene on the context via backdoor adjustment during training. Experiments show that our method significantly improves the FSED on both ACE05 and MAVEN datasets.

## introduction event detection (ed) aims to identify and classify event triggers in a sentence, e.g., detecting an attack event triggered by fire in "they killed by hostile fire in iraqi". recently, supervised ed approaches have achieved promising performance @xcite @xcite @xcite @xcite , but when adapting to new event types and domains, a large number of manually annotated event data is required which is expensive. by contrast, fewshot event detection (fsed) aims to build effective event detectors that are able to detect new events from instances (query) with a few labeled instances (support set). due to their ability to classify novel types, many few-shot algorithms have been used in fsed, e.g., metric-based methods like prototypical network @xcite @xcite . unfortunately, there has long been a "trigger curse" which troubles the learning of event detec-an attack event in iraqi. they were killed by hostile @xcite in iraqi. forces, fire, attack, ... they were killed by hostile @xcite in iraqi. they were killed by hostile tion models, especially in few-shot scenario @xcite @xcite @xcite . for many event types, their triggers are dominated by several popular words, e.g., the attack event type is dominated by war, attack, fight, fire, bomb in ace05. and we found the top 5 triggers of each event type cover 78% of event occurrences in ace05. due to the trigger curse, event detection models nearly degenerate to a trigger matcher, ignore the majority of contextual information and mainly rely on whether the candidate word matches the dominant triggers. this problem is more severe in fsed: since the given support instances are very sparse and lack diversity, it is much easier to overfit the trigger of the support instances. an intuitive solution for the trigger curse is to erase the trigger information in instances and forces the model to focus more on the context. unfortunately, due to the decisive role of triggers, directly wiping out the trigger information commonly hurts the performance @xcite . some previous approaches try to tackle this problem by introducing more di-versified context information like event argument information @xcite @xcite and document-level information @xcite @xcite . however, rich context information is commonly not available for fsed, and therefore these methods can not be directly applied. in this paper, we revisit the trigger curse in fsed from a causal view. specifically, we formulate the data distribution of fsed using a trigger-centric structural causal model (scm) @xcite shown in figure 1(a ). such trigger-centric formulation is based on the fact that, given the event type, contexts have a much lower impact on triggers, compared with the impact of triggers on contexts. this results in the decisive role of triggers in event extraction, and therefore conventional event extraction approaches commonly follow the triggercentric procedure (i.e., identifying triggers first and then using triggers as an indicator to find arguments in contexts). furthermore, the case grammar theory in linguistics @xcite also formulate the language using such trigger/predicate-centric assumption, and have been widely exploited in many nlp tasks like semantic role labeling @xcite and abstract meaning representation @xcite . from the scm, we found that t (trigger set) is a confounder of the c(context set) and the y (result), and therefore there exists a backdoor path c ← @xmath0. the backdoor path explains why previous fsed models disregard contextual information: it misleads the conventional learning procedure to mistakenly regard effects of triggers as the effects of contexts. consequently, the learning criteria of conventional fsed methods are optimized towards spurious correlation, rather than capturing causality between c and y . to address this issue, we propose to intervene on context to block the information from trigger to context. specifically, we apply backdoor adjustment to estimate the interventional distribution that is used for optimizing causality. furthermore, because backdoor adjustment relies on the unknown prior confounder (trigger) distribution, we also propose to estimate it based on contextualized word prediction. we conducted experiments on ace05 foot_0 , maven foot_1 and kbp17 foot_2 datasets. experiments show that causal intervention can significantly alleviate trigger curse, and therefore the proposed method significantly outperforms previous fsed methods. ## structural causal model for fsed this section describes the structural causal model (scm) for fsed, illustrated in figure 1(a) . note that, we omit the causal structure of the query for simplicity since it is the same as the support set. concretely, the scm formulates the data distribution of fsed: 1) starting from an event e we want to describe (in figure 1(a ) is an attack in iraqi). conventional learning criteria for fsed directly optimize towards the conditional distribution p (y |s, q). however, from the scm, we found that the backdoor path c ← @xmath1. consequently, the learning procedure towards p (y |s, q) will mistakenly regard the effects of triggers as the effects of contexts, and therefore overfit the trigger information. ## causal intervention for trigger curse based on the scm, this section describes how to resolve the trigger curse via causal intervention. context intervention. to block the backdoor path, we intervene on the context c and the new context-intervened scm is shown in figure 1(b) . given support set s, event set e of s, context set c of s and query instance q, we optimize the interventional distribution p (y |do(@xmath2, @xmath3, @xmath4, @xmath5, where do(•) denotes causal intervention operation. by interven-ing, the learning objective of models changes from optimizing correlation to optimizing causality. backdoor adjustment. backdoor adjustment is used to estimate the interventional distribution 4 : estimating p (t|e) via contextualized prediction. the confounder distribution p (t|e) is unknown because e is a hidden variable. since the event argument information is contained in c, we argue that p (t|e) ∝ m (t|c) where m (•|c) indicates a masked token prediction task @xcite which is constructed by masking triggers in the support set. in this paper, we use masked language model to calculate p (t|e) by first generating a set of candidate triggers through the context: optimizing via representation learning. given the interventional distribution, fsed model can be learned by minimizing the loss function on it: however, the optimization of l(θ) needs to calculate every p (y |s, q; θ), which is quite time-consuming. to this end, we propose a surrogate learning criteria l sg (θ) to optimize the causal relation based on representation learning: 4 the proof is shown in appendix lsg(θ) = -@xmath6; θ), @xmath7, t)p (t|e)r(s; θ)) here r is a representation model which inputs s or q and outputs a dense representation. g(•, •) is a distance metric measuring the similarity between two representations. such loss function is widely used in many metric-based methods (e.g., prototypical networks and relation networks). in the appendix, we prove l sg (θ) is equivalent to l(θ). ## conclusions this paper proposes to revisit the trigger curse in fsed from a causal view. specifically, we identify the cause of the trigger curse problem from a structural causal model, and then solve the problem through casual intervention via backdoor adjustment. experimental results demonstrate the effectiveness and robustness of our methods. when evaluating, we treat the labels as io tagging schemes, and adjacent i are considered to be the same trigger so that we can handle a trigger with multiple tokens.