Constrained Multi-Task Learning for Event Coreference Resolution (2021)

Abstract: We propose a neural event coreference model in which event coreference is jointly trained with five tasks: trigger detection, entity coreference, anaphoricity determination, realis detection, and argument extraction. To guide the learning of this complex model, we incorporate cross-task consistency constraints into the learning process as soft constraints via designing penalty functions. In addition, we propose the novel idea of viewing entity coreference and event coreference as a single coreference task, which we believe is a step towards a unified model of coreference resolution. The resulting model achieves state-of-the-art results on the KBP 2017 event coreference dataset.

## introduction event coreference resolution is the task of determining whether two event mentions in a document refer to the same real-world event. for two event mentions to be coreferent, their triggers (i.e., the words realizing the occurrence of the events) should have the same subtype and their corresponding arguments (e.g., the times, places, and people involved) have to be entity-coreferent. however, identifying potential arguments (which is performed by an entity extraction system), linking arguments to their event mentions (which is also performed by an event extraction system), and determining whether two event arguments are coreferent (which is the job of an entity coreference resolver), are all nontrivial tasks. hence, a key challenge in designing an event coreference resolver involves determining how to integrate these noisy components. one of the most common approaches to event coreference resolution is pipelined approaches, where a trigger detection component, which identifies triggers and assigns event subtypes to them, is followed by an event coreference component, which clusters coreferent event mentions. it should therefore not be surprising that errors propagate from the trigger detection component to the event coreference component. to avoid aggravating this error propagation problem, knowledge provided by other information extraction (ie) components (e.g., entity coreference, event arguments) is typically employed as features for training event coreference models @xcite @xcite @xcite @xcite . oftentimes, these features provide limited improvements to event coreference models as they are too noisy to be useful. though less popular than pipelined approaches, bootstrapping approaches have been used for event coreference resolution, where an event coreference model is bootstrapped with models trained for one or more related ie tasks. for instance, @xcite incrementally build clusters of coreferent event and entity mentions by iteratively bootstrapping event coreference output using entity coreference output and vice versa. while in pipelined approaches only upstream tasks can influence downstream tasks, in bootstrapping approaches different tasks can influence each other. nevertheless, errors made in earlier iterations of the bootstrapping process cannot be undone in later iterations. joint learning approaches have recently emerged as promising approaches to event coreference owing to their ability to address error propagation. in these approaches, two or more tasks are jointly trained. for instance, @xcite learn a joint model for trigger detection and event coreference using a structured perceptron, and @xcite learn a joint model for trigger detection, event coreference, and anaphoricity determination using a structured conditional random field. the key advantage of these models is that the tasks involved can benefit from each other during training. however, since a jointly learned model involves multiple tasks, it is typically complex. in fact, it is by no means easy to scale such a model to a large number of tasks because of the high computational complexity involved in learning. joint inference approaches have also been applied to event coreference resolution. for instance, @xcite and @xcite first train separate models for entity coreference, trigger detection, argument extraction, and event coreference, then use integer linear programming or markov logic networks to jointly infer the outputs of these tasks subject to (mostly) hard cross-task consistency constraints. for instance, one such hard constraint says that two coreferent event mentions should have the same event subtype. since the models are trained independently, they cannot benefit from each other and could be noisy. worse still, performing joint inference using hard constraints over (very) noisy outputs could do more harm than good. for instance, if two event mentions are correctly classified as coreferent but one of their subtypes is misclassified, then enforcing the aforementioned constraint might cause the joint inference procedure to incorrectly infer that the two are not coreferent. this explains why joint inference approaches have become less popular than joint learning approaches in recent years. in light of the above discussion, we seek to advance the state of the art in event coreference resolution by proposing a model that jointly learns six tasks: trigger detection, event coreference, entity coreference, anaphoricity determination, argument extraction, and realis detection. as noted above, joint learning typically presents a serious computational challenge, and training a complex joint model involving six tasks would not have been possible without the advent of the neural nlp era. while multi-task learning in a neural network typically allows the different tasks involved to benefit from each other via learning shared representations, we hypothesize that the model would benefit additional guidance given that the learning task, which involves six tasks, is so complex. consequently, we propose to guide the learning process by exploiting cross-task consistency constraints. as mentioned above, such consistency constraints are typically employed in joint inference and rarely in joint learning. moreover, unlike in joint inference where such constraints are typically implemented as hard constraints, we provide flexibility by implementing them as soft constraints. specifically, we design penalty functions for penalizing outputs that violate a constraint, where the degree of penalty depends on the extent of the violation. another contribution of our work involves proposing the idea of a unified coreference model. so far, entity and event coreference have always been viewed as two separate tasks, where links between entity mentions are distinguished from links between event mentions. however, their similarity has led us to hypothesize that they could be viewed as a single task, where coreference links are established between a set of mentions without distinguishing between entity and event mentions. ## definitions before formally defining the six tasks in the next section, we introduce several related definitions. to better understand these definitions, consider the example in ## model we design a span-based neural model for event coreference resolution owing to its ability to effectively learn representations of text spans. while span-based models have been successfully applied to a variety of entity-based ie tasks such as entity coreference @xcite and relation extraction @xcite , they have not been applied to event coreference. more formally, our model takes as input a document d represented as a sequence of word tokens, from which we extract all possible intra-sentence spans of up to length l. it simultaneously learns six tasks, which we define below. the trigger detection task aims to assign each span i a subtype label y i . each y i takes a value in a subtype inventory or none, which indicates that i is not a trigger. the model predicts i's subtype to be y * @xmath0, y t ), where s t is a scoring function suggesting i's likelihood of having y i as its subtype. the event coreference resolution task aims to assign span i an antecedent y c , where y @xmath1, . . . , i -1, }. in other words, the value of y c is the id of i's antecedent, which can be one of the preceding spans or a dummy antecedent (if the event mention underlying i starts a new cluster). we define the following scoring function: the entity coreference resolution task involves identifying entity mentions that refer to the same real-world entity. intuitively, entity coreference is useful for event coreference: two event mentions are not likely to be coreferent if there exists an argument role (e.g., attacker) for which the corresponding arguments in the two event mentions are not entity-coreferent. in our model, it is defined in the same way as the event coreference resolution task except that it operates on the spans identified by the entity mention detection component rather than the trigger detection component. the entity mention detection task is defined in the same way as the trigger detection task except that it aims to assign each span i an entity type label. the anaphoricity determination task aims to assign each span i an anaphoricity label y a , where y a can be anaphoric, which indicates that the mention having span i is coreferent with a preceding mention, or non-anaphoric. the model s a (i) predicts the mention having span i as anaphoric if and only if s a (i) ≥ 0. to train this model, we set the target value to 1 for anaphoric mentions and -1 for non-anaphoric mentions. anaphoricity is useful for coreference: it prevents non-anaphoric mentions from being resolved. the realis detection task aims to assign each span i a realis label y r , where y @xmath2, generic, other, entity, and none}. as mentioned in section 3, actual, generic, and other are labels used for event mention spans. to enable every span i to be assigned a realis label, we augment the realis label set to include entity and none. specifically, entity is a label that is exclusively reserved for spans that correspond to entity mentions, and none indicates that i does not correspond to a mention. the model predicts the realis type of i to be y * @xmath3, y r ), where s r is a scoring function suggesting i's likelihood of having realis type y r . realis detection is useful for event coreference: two event mentions cannot be coreferent if their realis labels are different. the argument extraction task aims to assign an argument role label y o to a candidate argument k of a candidate event mention span i, where (1) k is a candidate entity mention span, and (2) y o is a role taken from an argument role inventory or none, which indicates that the token is not an argument of i. we consider (1) k to be a candidate argument of i if and only if it appears within the same sentence as i; and (2) a span to be a candidate event/entity mention span if it is assigned a non-none event/entity type by the mention prediction layer, which we will describe shortly. for each candidate argument k of i, the model predicts its role in i to be y * @xmath4, k, y o ), where s o is a scoring function suggesting token k's likelihood of being an argument of i having role y o . arguments, when combined with entity coreference chains, would be useful for event coreference. ## conclusion we proposed the first neural model for event coreference resolution that (1) jointly learned six tasks, (2) used consistency constraints to guide learning, and (3) viewed entity and event coreference as a single task. our model outperformed several strong baselines and achieved state-of-the-art results on the kbp 2017 event coreference dataset.