Saliency as Evidence: Event Detection with Trigger Saliency Attribution (2022)

Abstract: Event detection (ED) is a critical subtask of event extraction that seeks to identify event triggers of certain types in texts. Despite significant advances in ED, existing methods typically follow a “one model fits all types” approach, which sees no differences between event types and often results in a quite skewed performance. Finding the causes of skewed performance is crucial for the robustness of an ED model, but to date there has been little exploration of this problem. This research examines the issue in depth and presents a new concept termed trigger salience attribution, which can explicitly quantify the underlying patterns of events. On this foundation, we develop a new training mechanism for ED, which can distinguish between trigger-dependent and context-dependent types and achieve promising performance on two benchmarks. Finally, by highlighting many distinct characteristics of trigger-dependent and context-dependent types, our work may promote more research into this problem.

## introduction event detection (ed) is the first and a crucial step of event extraction, which aims to identify events of certain types in plain texts @xcite @xcite . previous methods to ed typically adopt a "one model fits all types" approach, seeing no difference between event types and using a single model to address them all @xcite @xcite . however, such approaches produce quite skewed performance on different types. tasking the ace benchmark as an example, we note the state-of-the-art ed model @xcite can strike 90% in f1 for the type divorce, yet only 50% for the type start-position, and it is more surprising that the training set of divorce is eight times smaller than that of @xcite finding the causes underlying s1: the couple divorced four years later. s2: he became the first us minister to england. in this study we take a fresh look at above problem and for the first time attribute the skewed performance to the contextual patterns of events. let consider the two typical instances of divorce and start-position shown in figure 1 . intuitively, they demonstrate distinct patterns: the di-vorce event is more trigger-dependent, and the trigger word (i.e., "divorced") is very indicative of the event's occurrence; by contrast, the start-position event is more context-dependent -the event semantic is primarily expressed by contexts rather than the trigger "become", which is a merely light verb. we hypothesize an ed model performs poorly on context-dependent types because capturing context semantics is challenging @xcite . with the above intuitions, two questions rise: (i) can we estimate an event's pattern quantitatively? (ii)) how to robustify an ed model by characterizing such patterns? to address the first question, we introduce a brandy new concept called trigger saliency attribution, which can explicitly quantify an event's contextual pattern. figure 2 illustrates the key idea: to determine how much an event is trigger-dependent or context-dependent, we measure the trigger's contribution to expressing overall the event semantic. specifically, we first assign each sentence a global event label that represents the overall event semantic. then, inspired by the feature attribution method @xcite , we regard each word as a feature and compute its contribution (i.e., saliency value) for predicting the global event label. finally, by examining the ground-truth trigger's saliency value, we can tell how much an event depends on triggers or contexts: a higher value, for example, indicates that the trigger contributes more to the event, implying the event is more trigger-dependent. to answer the second question, we develop a new training mechanism based on trigger saliency attribution, which uses saliency as evidence to enhance learning. our method is simple and straightforward -instead of using a single model to detect all event types, we group event types with similar patterns together (assessed by trigger saliency attribution) and develop separate models for each group. this strategy enables different models to capture distinct patterns -for example, the model for context-dependent type can focus on mining contextual information for learning. to further boost learning, we also propose two saliency-exploration strategy to augment the above framework, which can explicitly integrate saliency information into learning and produce improved performance particularly for context-dependent types ( § 6.2). to verify the effectiveness of our approach, we have conducted extensive experiments on two ed benchmarks (i.e., @xcite and maven @xcite ). according to the results: (i) our trigger saliency attribution method can capture the underlying pattern and well explain the skewed performance, obtaining spearman's correlation coefficients of 0.72 and 0.61 with per-type f1 on ace 2005 and maven respectively; (ii) our new training regime based on saliency demonstrates improved results on the two benchmarks. on ace 2005, for example, it produces a 2% absolute gain in f1 over methods training different event types jointly. finally, in ablation studies, we compare and highlight many significant characteristics (e.g., linguistic and lexical patterns) of triggerdependent and context-dependent event types; our work may inspire future research into their patterns. to summarize, our contributions are three-fold: s2: he became the first minister to england. feature attribution. the goal of feature attribution (fa) is to assess how important an input feature for model prediction, which has sparked a lot of interest in interpreting model decisions @xcite . formally, suppose we have an input vector @xmath0, x 2 , ..., x n ) ∈ r n and a function f: r @xmath1, 1] representing a model. the attribution value of x, with respect to the output f(x), is defined as a vector a f (x) = (a 1 , a 2 , ..., a n ) ∈ r n , where a i measures the contribution of x i to f(x). the existing fa methods are classified as gradient-based methods, which consider the gradient of the output to the input as the attribution value @xcite , and reference-based methods, which consider the difference between the model's output and some "reference" output, in terms of the difference between the input and some "reference" input, as the attribution value @xcite . fa have been used to interpret model predictions in applications including image classification @xcite , machine translation @xcite ), text classification @xcite , and others @xcite . to the best of our knowledge, this is the first work introducing fa to ed for quantifying the underlying event patterns. integrated gradient. integrated gradient @xcite is a specific (referencebased) fa method that views the feature attribution value as the accumulated gradient along the line between the model's input x and a reference input x , which denotes the lack of a feature 1 . particularly, the attribution value of x i (i.e., the i th dimension of x) with respect to an output f(x) is defined as: ## trigger saliency attribution algorithm 1 provides an overview of our trigger saliency attribution method, which consists of three major steps: (i) sentence-level event classification, (ii) word-level saliency estimation, and (iii) typelevel saliency estimation. let @xmath2, w 2 , • • • , w n ] be a sentence of n words, and the ed task corresponds to predicting an event label sequence sentence-level event classification. we start by giving s a sentence-level event label g s , which represents the overall event semantic. let the label be g ## saliency enhanced ed based on trigger saliency attribution, we devise a new training paradigm for ed, which can distinguish event types with similar patterns for learning and achieves promising results. the overview is shown in figure 3 , and the technical details follow. ## conclusion in this study, we analyze the origins of an ed model's skewed performance and introduce a new notion called trigger saliency attribution to quantify the pattern of events. we devise a new training paradigm for ed that can distinguish between trigger-dependent and context-dependent types for learning, yielding promising results on two benchmarks. we also examine the differences between the two types extensively, and our work may promote future research on this problem. in the future, we would apply our method to other tasks (e.g., relation extraction) where contextual patterns matter.