End-to-End AMR Coreference Resolution (2021)

Abstract: Although parsing to Abstract Meaning Representation (AMR) has become very popular and AMR has been shown effective on the many sentence-level downstream tasks, little work has studied how to generate AMRs that can represent multi-sentence information. We introduce the first end-to-end AMR coreference resolution model in order to build multi-sentence AMRs. Compared with the previous pipeline and rule-based approaches, our model alleviates error propagation and it is more robust for both in-domain and out-domain situations. Besides, the document-level AMRs obtained by our model can significantly improve over the AMRs generated by a rule-based method (Liu et al., 2015) on text summarization.

## introduction abstract meaning representation (amr) @xcite is a semantic formalism for natural language understanding. it represents a sentence as a rooted, directed and acyclic graph, where nodes (e.g., "bill" in figure 1 ) represents concepts and edges (e.g., ":arg0") are the semantic relations. encompassing knowledge of named entities, semantic roles and coreference structures, amr has been proven effective for downstream tasks, including information extraction @xcite ), text summarization @xcite @xcite , paraphrase detection @xcite , event detection @xcite , machine translation @xcite and dialogue understanding @xcite . existing work on amr mainly focuses on individual sentences @xcite @xcite @xcite . on the other hand, with the advance of neural networks in nlp, tasks involving multiple sentences with cross-sentence reasoning (e.g., text summarization, reading comprehension and dialogue response generation) have received increasing research attention. given the effectiveness of amr on sentencelevel tasks @xcite @xcite , it is important to extend sentence-level amrs into the multi-sentence level. to this end, a prerequisite step is amr coreference resolution, which aims to find the amr components referring to the same entity. figure 1 shows the amr graphs of two consecutive sentences in a document. an amr coreference resolution model need to identify two coreference cases: "he" refers to "bill" in the first graph, and "arrive-01" omits an argument ":arg3" that refers to "paris". relatively little research has been done on amr coreference resolution. initial attempts @xcite merge the nodes that have the same surface string. to minimize noise, only named entities and date entities are considered, and they do not consider merging non-identical nodes (e.g., "bill" and "he" in figure 1 ) that are also frequent in reallife situation. subsequent work considers more co-reference cases by either manually annotating amr coreference information @xcite or taking a pipeline system @xcite consisting of a textual coreference resolution model @xcite and an amr-to-text aligner @xcite . yet there is little research on automatically resolving coreference ambiguities directly on amr, making use of amr graph-structural features. in this work, we formulate amr coreference resolution as a missing-link prediction problem over amr graphs, where the input consists of multiple sentence-level amrs, and the goal is to recover the missing coreference links connecting the amr nodes that represent to the same entity. there are two types of links. the first type corresponds to the standard situation, where the edge connects two entity nodes (e.g., "bill" and "he" in figure 1 ) that refer to the same entity. the second type is the implicit role coreference, where one node (e.g., "paris" in figure 1 ) is a dropped argument (":arg3") of other predicate node ("arrive-01"). we propose an amr coreference resolution model by extending an end-to-end text-based coreference resolution model @xcite . in particular, we use a graph neural network to represent input amrs for inducing expressive features. to enable cross-sentence information exchange, we make connections between sentence-level amrs by linking their root nodes. besides, we introduce a concept identification module to distinguish functional graph nodes (non-concept nodes, e.g., "person" in figure 1 ), entity nodes (e.g., "bill"), verbal nodes with implicit role (e.g., "arrive-01") and other regular nodes (e.g., "leave-11") to help improve the performance. the final antecedent prediction is conducted between the selected nodes and all their possible antecedent candidates, following previous work on textual coreference resolution @xcite . experiments on the ms-amr benchmark 1 (o' @xcite show that our model outperforms competitive baselines by a large margin. to verify the effectiveness and generalization of our proposed model, we annotate an out-of-domain test set over the gold amr little prince 3.0 data following the guidelines of @xcite , and the corresponding results show that our model is consistently more robust than the baselines in domain-transfer scenarios. finally, results on docu-1 it consists gold coreference links on gold amrs. ment abstractive summarization show that our document amrs lead to much better summary quality compared to the document amrs by @xcite . this further verifies the practical value of our approach. our code and data is available at ## experiments we conduct experiments on the ms-amr dataset foot_1 (o'gorman et al., 2018), which is annotated over a previous gold amr corpus (ldc2017t10). it has 293 annotated documents in total with an average of 27.4 amrs per document, covering roughly 10% of the total amr corpus. we split a dev data with the same size as the test set from the training set. following the annotation guidelines of ms-amr, we manually annotate the amr coreference resolution information over the development and test data of the little prince (lp) amr corpus foot_2 and use it as an out-of-domain test set. for this dataset, we consider each chapter as a document. the data statistics are shown in ## related work multi-sentence amr although some previous work @xcite explore the coreference phenomena of amr, they mainly focus on the situation within a sentence. on the other hand, previous work on multi-sentence amr primarily focuses on data annotation. @xcite annotate dropped pronouns over chinese amr but only deals with implicit roles in specific constructions. @xcite provide implicit role annotations, but the resources were limited to a small inventory of 5-10 predicate types rather than all implicit arguments. @xcite annotated the ms-amr dataset by simultaneously considering coreference, implicit role coreference and bridging relations. we consider coreference resolution as the prerequisite for creating multi-sentence amrs, proposing the first end-to-end model for this task. coreference resolution coreference resolution is a fundamental problem in natural language processing. neural network models have shown promising results over the years. recent work @xcite @xcite tackled the problem end-to-end by jointly detecting mentions and predicting coreference. @xcite build a complete end-to-end system with the span-ranking architecture and higher-order inference technique. while previous work considers only text-level coreference, we investigate amr co-reference resolution. amr representation using gnn to encode amr graphs, many variants of gnns such as grns @xcite , gcns @xcite and gats @xcite @xcite have been introduced. we choose a classic grn model following @xcite to represent our document-level amr graph and leave the exploiting on a more efficient gnn structure for future work. ## conclusion we investigated a novel end-to-end multi-sentence amr coreference resolution model using a graph neural network. compared with previous rulebased and pipeline methods, our model better captures multi-sentence semantic information. results on ms-amr (in-domain) and lp (out-of-domain) datasets show the superiority and robustness of our model. in addition, experiments on the downstream text summarization task further demonstrate the effectiveness of the document-level amrs produced by our model. in future work, we plan to resolve both the cross-amr coreference links and the sentence-level ones together with our model.