Hierarchical Graph Convolutional Networks for Jointly Resolving Cross-document Coreference of Entity and Event Mentions (2021)

Abstract: This paper studies the problem of cross-document event coreference resolution (CDECR) that seeks to determine if event mentions across multiple documents refer to the same real-world events. Prior work has demonstrated the benefits of the predicate-argument information and document context for resolving the coreference of event mentions. However, such information has not been captured effectively in prior work for CDECR. To address these limitations, we propose a novel deep learning model for CDECR that introduces hierarchical graph convolutional neural networks (GCN) to jointly resolve entity and event mentions. As such, sentence-level GCNs enable the encoding of important context words for event mentions and their arguments while the document-level GCN leverages the interaction structures of event mentions and arguments to compute document representations to perform CDECR. Extensive experiments are conducted to demonstrate the effectiveness of the proposed model.

## introduction event coreference resolution (ecr) aims to cluster event-triggering expressions in text such that all event mentions in a group refer to the same unique event in real world. we are interested in cross-document ecr (cdecr) where event mentions might appear in the same or different documents. for instance, consider the following sentences (event mentions) s1 and s2 that involve "leaving" and "left" (respectively) as event trigger words (i.e., the predicates): s1: o'brien was forced into the drastic step of leaving the 76ers. s2: jim o'brien left the 76ers after one season as coach. an cdecr system in this case would need to recognize that both event mentions in s1 and s2 refer to the same event. a major challenge in cdecr involves the necessity to model entity mentions (e.g., "jim o'brien") that participate into events and reveal their spatio-temporal information @xcite (called event arguments). in particular, as event mentions might be presented in different sentences/documents, an important evidence for predicting the coreference of two event mentions is to realize that the two event mentions have the same participants in the real world and/or occur at the same location and time (i.e., same arguments). motivated by this intuition, prior work for cdecr has attempted to jointly resolve crossdocument coreference for entities and events so the two tasks can mutually benefit from each other (iterative clustering) @xcite . in fact, this iterative and joint modeling approach has recently led to the state-of-the-art performance for cdecr @xcite . our model for cdecr follows this joint coreference resolution method; however, we advance it by introducing novel techniques to address two major limitations from previous work @xcite @xcite , i.e., the inadequate mechanisms to capture the argument-related information for representing event mentions and the use of only lexical features to represent input documents. as the first limitation with the event argumentrelated evidence, existing methods for cdecr have mainly captured the direct information of event arguments for event mention representations, thus failing to explicitly encode other important context words in the sentences to reveal fine-grained nature of relations between arguments and triggers for ecr @xcite . for instance, consider the coreference prediction between the event mentions in s1 and the following sentence s3 (with "leave" as the event trigger word): s3: the baseball coach jim o'brien decided to leave the the arguments for the event mention in s3 involves "the baseball coach jim o'brien", "the club", and "monday". if an entity coreference resolution system considers the entity mention pairs ("o'brien" in s1, "the baseball coach jim o'brien" in s3) and ("the 76ers" in s1 and "the club" in s3) as being coreferred, a cdecr system, which only concerns event arguments and their coreference information, would incorrectly predict the coreference between the event mentions in s1 and s3 in this case. however, if the cdecr system further models the important words for the relations between event triggers and arguments (i.e., the words "was forced" in s1 and "decided" in s3), it can realize the unwillingness of the subject for the position ending event in s1 and the self intent to leave the position for the event in s3. as such, this difference can help the system to reject the event coreference for s1 and s3. to this end, we propose to explicitly identify and capture important context words for event triggers and arguments in representation learning for cdecr. in particular, our motivation is based on the shortest dependency paths between event triggers and arguments that have been used to reveal important context words for their relations @xcite @xcite . as an example, figure 1 shows the dependency tree of s1 where the shortest dependency path between "o'brien" and "leaving" can successfully include the important context word "forced". as such, for each event mention, we leverage the shortest dependency paths to build a pruned and argumentcustomized dependency tree to simultaneously contain event triggers, arguments and the important words in a single structure. afterward, the structure will be exploited to learn richer representation vectors for cdecr. second, for document representations, previous work on cdecr has proved that input documents also provide useful context information for event mentions (e.g., document topics) to enhance the clustering performance @xcite . however, the document information is only captured via lexical features in prior work, e.g., tf-idf vectors @xcite , leading to the poor generalization to unseen words/tokens and inability to encapsulate latent semantic information for cdecr. to this end, we propose to learn distributed representation vectors for input documents to enrich event mention representations and improve the generalization of the models for cdecr. in particular, as entity and event mentions are the main objects of interest for cdecr, our motivation is to focus on the context information from these objects to induce document representations for the models. to implement this idea, we propose to represent input documents via interaction graphs between their entity and event mentions, serving as the structures to generate document representation vectors. based on those motivations, we introduce a novel hierarchical graph convolutional neural network (gcn) that involves two levels of gcn models to learn representation vectors for the iterative and joint model for cdecr. in particular, sentence-level gcns will consume the pruned dependency trees to obtain context-enriched representation vectors for event and entity mentions while a document-level gcn will be run over the entityevent interaction graphs, leveraging the mention representations from the sentence-level gcns as the inputs to generate document representations for cdecr. extensive experiments show that the proposed model achieves the state-of-the-art resolution performance for both entities and events on the ecb+ dataset. to our knowledge, this is the first work that utilizes gcns and entity-event interaction graphs for coreference resolution. finally, regarding the modeling perspective, our work is related to the models that use gcns to learn representation vectors for different nlp tasks, e.g., event detection @xcite and target opinion word extraction (veyseh et al., 2020b), applying both sentence-and document-level graphs @xcite @xcite @xcite . however, to our knowledge, none of the prior work has employed gcns for ecr. ## model given a set of input documents d, the goal of cdecr is to cluster event mentions in the documents of d according to their coreference. our model for cdecr follows @xcite that simultaneously clusters entity mentions in d to benefit from the inter-dependencies between entities and events for coreference resolution. in this section, we will first describe the overall framework of our iterative method for joint entity and event coreference resolution based on @xcite (a summary of the framework is given in algorithm 1). the novel hierarchical gcn model for inducing mention foot_0 representation vectors will be discussed afterward. iterative clustering for cdecr: following @xcite , we first cluster the input document set d into different topics to improve the coreference performance (the set of document topics is called t ). as we use the ecb+ dataset @xcite to evaluate the models in this work, the training phase directly utilizes the golden topics of the documents while the test phase applies the k-mean algorithm for document clustering as in @xcite . afterward, given a topic t 2 t with the corresponding document subset d t ⇢ d, our cdecr model initializes the entity and event cluster configurations e 0 t and v 0 t (respectively) where: e 0 t involves withindocument clusters of the entity mentions in the documents in d t , and v 0 t simply puts each event mention presented in d t into its own cluster (lines 2 and 3 in algorithm 1). in the training phase, e 0 t is obtained from the golden within-document coreference information of the entity mentions (to reduce noise) while the within-document entity mention clusters returned by stanford corenlp @xcite are used for e 0 t in the test phase, following @xcite . for convenience, the sets of entity and event mentions in d t are called m e t and m v t respectively. train rv and sv using the gold entity mention clusters 15: finally, we note that the agglomerative clustering in each iteration of our model also starts with the initial clusters as in e 0 t and v 0 t , and greedily merges multiple cluster pairs with the high-est cluster-pair scores until all the scores are below a predefined threshold . in this way, the algorithm first focuses on high-precision merging operations and postpones less precise ones until more information is available. the cluster-pair score s c (c i , c j ) for two clusters c i and c j (mention sets) at some algorithm step is based on averaging mention linkage coreference scores: s c (c i , c j ) = @xcite where ⇤ can be e or v depending on whether c i and c j are entity or event clusters (respectively). mention representations: let m be a mention (event or entity) in a sentence @xmath0, w 2 , . . . , w n of n words (w i is the i-th word) where w a is the head word of m. to prepare w for the mention representation computation and achieve a fair comparison with @xcite , we first convert each word w i 2 w into a vector x i using the elmo embeddings @xcite . here, x i is obtained by running the pre-trained elmo model over w and averaging the hidden vectors for w i at the three layers in elmo. this transforms w into a sequence of vectors @xmath1, x 2 , . . . , x n for the next steps. the mention representations in our work are based on two major elements, i.e., the modeling of important context words for event triggers and arguments, and the induction of document presentations. finally, the mean squared difference between h ent and h env is introduced into the overall loss function to regularize the models: as this loss is specific to mention m, it will be computed independently for event and entity mentions and added into the corresponding training loss (i.e., lines 9 or 14 in algorithm 1). in particular, the overall training loss for entity mention coreference resolution in line 9 of algorithm 1 is: here, ↵ ent and ↵ env are the trade-off parameters. (iv) document representation: as motivated in the introduction, we propose to learn representation vectors for input documents to enrich mention representations and improve the generalization of the models (over the lexical features for documents). as such, our principle is to employ entity and event mentions (the main objects of interest in cdecr) and their interactions/structures to represent input documents (i.e., documents as interaction graphs of entity and event mentions). given the mention m of interest and its correspond-ing document d 2 d t , we start by building an interaction graph g @xmath2, e doc } where the node set n doc involves all the entity and event mentions in d. for the edge set e doc , we leverage two types of information to connect the nodes in n doc : (i) predicate-argument information: we establish a link between the nodes for an event mention x and an entity mention y in n doc if y is an argument of x for one of the four semantic roles, i.e., arg0, arg1, location, and time (identified by the srl system), and (ii) entity mention coreference: we connect any pairs of nodes in n doc that correspond to two coreferring entity mentions in d (using the gold within-document entity coreference in training and the predicted one in testing, as in e 0 t ). in this way, g doc helps to emphasize on important objects of d and enables intra-and intersentence interactions between event mentions (via entity mentions/arguments) to produce effective document representations for cdecr. in the next step, we feed the interaction graph g doc into a gcn model g doc (called the documentlevel gcn) using the sentence-level gcn-based representations of the entity and event mentions in n doc (i.e., sent(m)) as the initial vectors for the nodes (thus called a hierarchical gcn model). the hidden vectors produced by the last layer of g doc for the nodes in g doc is called {h d u |u 2 n doc }. finally, we obtain the representation vector doc(m) for d based on the max-pooling: doc(m) = max_pool(h d u |u 2 n doc ). (v) final representation: given the representation vectors learned so far, we form the final representation vector for the cluster-based representation cluster(m) = [arg0 m , arg1 m , location m , time m ]. taking arg0 m as an example, it is computed by considering the mention m 0 that is associated with m via the semantic role arg0 in w using the srl system. here, m 0 is an event mention if m is an entity mention and vice versa (@xmath3. as such, let c be the cluster in the current configuration (i.e., v k 1 t or e k t ) that contain m 0 . we then obtain arg0 m by averaging the elmo-based vectors (i.e., @xmath4, . . . , x n ) of the head words of the mentions in c: arg0 @xmath5. note that as the current entity cluster configuration e k t is used to generate the cluster-based representations if m is an event mention (and vice verse), it serves as the main mechanism to enable the two coreference tasks to interact and benefit from each other. these vectors are inherited from @xcite for a fair comparison. finally, given two mentions m 1 and m 2 , and their corresponding representation vectors r(m 1 ) and r(m 2 ) (as computed above), the coreference score functions s e and s v send the concatenated vector [r(m 1 ), r(m 2 ), r(m 1 ) r(m 2 )] to twolayer feed-forward networks (separate ones for s e and s v ) that involve the sigmoid function in the end to produce coreference score for m 1 and m 2 . here, is the element-wise product. this completes the description of our cdecr model. ## experiments dataset: we use the ecb+ dataset @xcite to evaluate the cdecr models in this work. note that ecb+ is the largest dataset with both within-and cross-document annotation for the coreference of entity and event mentions so far. we follow the setup and split for this dataset in prior work to ensure a fair comparison @xcite @xcite . in particular, this setup employs the annotation subset that has been validated for correctness by @xcite and involves a larger portion of the dataset for training. in ecb+, only a part of the mentions are annotated. this setup thus utilizes gold-standard event and entity mentions in the evaluation and does not require special treatment for unannotated mentions @xcite . note that there is a different setup for ecb+ that is applied in @xcite and @xcite . in this setup, the full ecb+ dataset is employed, including the portions with known annotation errors. in test time, such prior work utilizes the predicted mentions from a mention extraction tool @xcite . to handle the partial annotation in ecb+, those prior work only evaluates the systems on the predicted mentions that are also annotated as the gold mentions. however, as shown by @xcite , this ecb+ setup has several limitations (e.g., the ignorance of clusters with a single mention and the separate evaluation for each sub-topic). following @xcite , we thus do not evaluate the systems on this setup, i.e., not comparing our model with those models in @xcite and @xcite due to the incompatibility. hyper-parameters: to achieve a fair comparison, we utilize the preprocessed data and extend the implementation for the model in @xcite to include our novel hierarchical gcn model. the development dataset of ecb+ is used to tune the hyper-parameters of the proposed model (called hgcn). the suggested values and the resources for our model are reported in appendix a. comparison: following @xcite , we compare hgcn with the following baselines: for cross-document entity coreference resolution, we compare our model with the lemma and bse models in @xcite , the only works that report the performance for event mentions on ecb+ so far. following @xcite , we use the common coreference resolution metrics to evaluate the models in this work, including muc @xcite , b 3 , ceaf-e @xcite , and conll f1 (average of three previous metrics). the official conll scorer in @xcite is employed to compute these metrics. tables 1 and 2 show the performance (f1 scores) of the models for cross-document resolution for entity and event mentions (respectively). note that we also report the performance of a variant (called hgcn-dj) of the proposed hgcn model where the cluster-based representations cluster(m) are excluded (thus separately doing event and entity resolution as bse-dj). ## conclusion we present a model to jointly resolve the crossdocument coreference of entity and event mentions. our model introduces a novel hierarchical gcn that captures both sentence and document context for the representations of entity and event mentions. in particular, we design pruned dependency trees to capture important context words for sentencelevel gcns while interaction graphs between entity and event mentions are employed for documentlevel gcn. in the future, we plan to explore better mechanisms to identify important context words for cdecr.