The Role of Reentrancies in A bstract M eaning R epresentation Parsing (2020)

Abstract: Abstract Meaning Representation (AMR) parsing aims at converting sentences into AMR representations. These are graphs and not trees because AMR supports reentrancies (nodes with more than one parent). Following previous findings on the importance of reen- trancies for AMR, we empirically find and discuss several linguistic phenomena respon- sible for reentrancies in AMR, some of which have not received attention before. We cate- gorize the types of errors AMR parsers make with respect to reentrancies. Furthermore, we find that correcting these errors provides an in- crease of up to 5% Smatch in parsing perfor- mance and 20% in reentrancy prediction

## introduction abstract meaning representation (amr) is a semantic formalism used to annotate natural language sentences as graphs. the task of amr parsing is to convert sentences into amr graphs @xcite -rooted and directed acyclic graphs where nodes represent concepts and edges represent semantic relations between them. the amr for the sentence i want you to believe me is shown in figure 1 . one of the main properties of amr, and the reason why sentences are represented as graphs rather than trees, is the presence of nodes with multiple parents, called reentrancies, as demonstrated in figure 1 , where the node i has two parents. reentrancies complicate amr parsing and require the addition of specific transitions in transition-based parsing @xcite or of pre-and post-processing steps in sequence-to-sequence parsing @xcite . enabling amr parsers to predict reentrancy structures correctly is of particular importance because it separates amr parsing from semantic parsing based on tree structures @xcite @xcite . reentrancy is however not an amr-specific problem @xcite , and other formalisms can benefit from a better understanding of how to parse such structures. nevertheless, to our knowledge, the amr literature lacks any detailed discussion of the types and linguistic causes of reentrant structures. we aim to fill the gap by describing the phenomena causing reentrancies and quantifying their prevalence in the amr corpus. we identify sources of reentrancy which have not been acknowledged in the amr literature such as adjunct control, verbalization, and pragmatics. amr parsers are evaluated using smatch @xcite , which however does not explicitly assess the parsers' ability to recover reentrancies. @xcite introduced a measure of reentrancy prediction, which computes the smatch score of the amr subgraphs containing reentrancies. it was observed that the performance of parsers at recovering reentrancy structures is generally poor. we analyze errors made by the parsers and use an oracle to demonstrate that correcting reentrancy-related errors leads to parsing score improvement. our contributions are as follows: in the analysis that follows we define an amr node as reentrant if it is a child of more than one other node in the penman linearization of the graph provided in the corpus. because of the frequent use of inverse roles in amr graphs, the directionality of the edges is not obvious. normalizing inverse roles reverses the edge direction, which changes the parent-child relations between nodes and thus influences which nodes are reentrant, i.e. have more than one parent. as @xcite report, the percentage of reentrant nodes in the amr corpus increases from 5% to 19% when inverse roles are normalized. for instance, in the relative clause example in with that in mind, we classify reentrancy triggers into three broad types: syntactic, pragmatic, and amr-specific. ## quantifying reentrancy causes in order to assess the prevalence of the various reentrancy triggers, we designed heuristics to assign each reentrancy in the amr corpus to one of the above phenomena. we automatically align amr graphs to their source sentences using jamr @xcite and identify the spans of words associated with re-entrant nodes. 3 heuristics based on universal dependency (ud) parses @xcite and automatic coreference resolution are applied to the spans and the amr subgraphs containing the reentrancy to classify the cause. 4 we use the neuralcoref project for coreference resolution. 5 we recognize syntactic reentrancy triggers primarily with ud-based heuristics. for prototypical cases of control we look for common con- the heuristics column reports automatically detected frequencies for the whole training set. the total column reports frequencies estimated by combining automatic and manual annotation. "unclassified" are all reentrancies for which our heuristics fail to detect the cause. finally, for verbalization, we look for nouns or adjectives aligned with ontonotes predicates in the amr graph. we tried to identify nominal control-like structures by looking for nominals with an acl dependent infinitive or gerund subjectless verb. however, as the precision of the rule is low, and most examples uncovered by this heuristic also fall into the verbalization category, we do not include it in our statistics. the results of this analysis are in we note that our heuristics cannot find the cause for 46% of all reentrancies. this can happen for several reasons. there are sources of reentrancy (ellipsis, nominal control-like structures) for which we do not have heuristics due to the difficulty of defining them in terms of ud parses. the heuristics we do define are of high precision if provided with correct input, but all of the systems we use to provide that input -amr aligner, pos tagger, ud parse, and coreference resolution system -are in fact noisy. moreover, what is considered to co-refer in amr does not necessarily agree with the notion implicit in the coreference resolution system. consider the following sentence: the coreference resolution system does not follow the looser definition of coreference used in the amr annotation guidelines, where the countries and the signatories are labeled as coreferential. finaly, some of the reentrancies unaccounted for by the heuristics are due to annotation mistakes. for example in the sentence a nuclear team will make a visit to inspect the nuclear site. the amr for this sentence contains a reentrancy for the nucleus node, which is used to modify both the team and the site, while there should be two separate nucleus nodes. to estimate the overall prevalence of reentrancy triggers, including cases for which the heuristics do not work, we manually annotated causes of unaccounted for reentrancies (79 cases) in a sample of 50 sentences. we combine the results of that manual analysis with the frequencies obtained through the use of heuristics to obtain the overall trigger frequency estimate. the results are shown in we find that triggers not covered by heuristics account for estimated 4% of total cases, and 34% of unclassified triggers belong to categories for which we do have heuristics, which illustrated the noisiness of the systems used for the heuristic analysis. the final 3% consist of examples of what we consider to be amr annotators overreaching in their pragmatic interpretation of the sentence. consider the sentence: in its amr, the node insult-01 takes group as its :arg1, making an arguably unwarranted assumption that the behavior is insulting to the group. we note that the inclusion of this type of reentrancies in amr is controversial as it annotates be-yond what semantics should represent. finally, 5% of the unaccounted reentrancies were due to mistakes in the amr annotations. in the following sentence, the annotator redundantly created both an edge expressing that make-19 is the purpose of remove-01, as well as an edge showing that remove-01 is :arg0 of make-19, leading to an unnecessary reentrancy for the remove-01 node. ## reentrancy-related parsing errors we propose a method, independent from the amr parser used, to classify the errors that amr parsers typically make when predicting such structures. in order to identify the errors, we compare the predicted amr graphs with the gold standard. we use smatch to find the best alignments between variables of the predicted and gold graph. we can then find cases where the predicted graph is either missing a reentrancy or contains an unnecessary one. due to the aforementioned noise in the heuristics of section 3, we did not follow the finegrained classification of linguistic causes. we instead follow a coarser structural classification of the errors. a typical reentrancy error involves the parser generating two nodes in place of one in the gold standard. this is often the case for reentrancies caused by coreference, as shown in figure 2 . the parser may not realize that two entity corefer, hence erroneously generating two different nodes. the opposite is also possible, where two nodes are erroneously collapsed. re-entrant edges can also occur between siblings. this is often the case for reentrancies caused by control verbs, as shown in figure 3 . ## automatic error correction we further provide baseline systems that learn when to apply add, the most impactful action. first, we experiment with a system that randomly selects two nodes in the predicted graph that are not connected by any edge and add an edge with arg0, the most frequent label. we also train a opennmt-py @xcite sequence-tosequence model @xcite with a copy mechanism @xcite . the input sequence is the predicted graph and the output sequence is the sequence of edges to add. for each edge, the output contains three tokens: the parent node, the child node, and the edge label. ## related work our classification of phenomena causing reentrancies extends previous work in this direction @xcite . van noord and bos (2017) previously attempted to improve the prediction of reentrancies in a neural parser. they experiment with several pre-and post-processing techniques and showed that co-indexing reentrancies nodes in the amr annotations yields the best results. transformation-based learning @xcite inspired the idea of correcting existing parses. this approach has been mostly used for tagging @xcite @xcite but it has also shown promises for semantic parsing @xcite . a similar approach has been also used to add empty nodes in constituent parses @xcite , with considerable success. the seq2seq baseline is an adaptation of the popular sequenceto-sequence modeling @xcite . an alternative approach to reduce reentrancy errors is to better inform training so that the errors are avoided in the first place. a recent amr parser @xcite outperforms the previous state of the art @xcite by implementing a copy mechanism aimed at recovering reentrancies, confirming that reentrancies are critical for achieving good amr parsing performance. ## conclusions building upon previous observations that amr parsers do not perform well at recovering reentrancies, we analyzed the linguistic phenomena responsible for reentrancies in amr. we found sources of reentrancies which have not been acknowledged in the amr literature such as adjunct control, verbalization, and pragmatics. the inclusion of reentrancies due to pragmatics is controversial; we hope that this work can spur new discussions on the role of reentrancies. our heuristics fail to detect the causes of many reentrancies. for a more precise estimate of the most common causes of reentrancies, it is necessary to manually annotate the reentrancies in the amr corpora. our oracle experiments show that there is room for improvement in predicting reentrancies, which in turn can translate to better parsing results. stronger baselines that can learn how to correct the errors automatically are left to future work. while the parser we experimented with no longer gives state-of-the-art results (but also not far from them), newer parsers @xcite also report relatively low accuracy on reentrancies (using the metrics from @xcite , and as such we believe our work is relevant to these parsers.