Normalizing Compositional Structures Across Graphbanks (2020)

Abstract: The emergence of a variety of graph-based meaning representations (MRs) has sparked an important conversation about how to adequately represent semantic structure. MRs exhibit structural differences that reflect different theoretical and design considerations, presenting challenges to uniform linguistic analysis and cross-framework semantic parsing. Here, we ask the question of which design differences between MRs are meaningful and semantically-rooted, and which are superficial. We present a methodology for normalizing discrepancies between MRs at the compositional level (Lindemann et al., 2019), finding that we can normalize the majority of divergent phenomena using linguistically-grounded rules. Our work significantly increases the match in compositional structure between MRs and improves multi-task learning (MTL) in a low-resource setting, serving as a proof of concept for future broad-scale cross-MR normalization.

## introduction graph-based representations of sentence meaning offer an expressive and flexible means of modeling natural language semantics. in recent years, a number of different graphbanks have annotated large corpora with graph-based semantic representations of various types @xcite @xcite . because of differences in graphbank design principles, individual graphs can differ greatly and often in fundamental strategies @xcite . fig. 1 illustrates distinct graphs from the three graphbanks of the semeval 2015 shared task on semantic dependency parsing, which we take as our focus in this paper. the graphs visibly differ with respect to edge labels, edge directions, the treatment of a periphrastic verb construction, and coordination. in this paper, we develop a methodology to (i) understand the nature of the differences across graphbanks and (ii) normalize annotations from different graphbanks at the level of their compositional structures. this approach allows us to identify which design differences between graphbanks are linguistically meaningful and important for the design of future corpora; as well as to identify more unified approaches to certain structures to potentially facilitate multi-task learning (mtl). in section 2 we build upon @xcite , who used am dependency trees to represent the compositional structure of graph-based meaning representations (mrs) based on the am algebra @xcite . we detail a new methodology for identifying and quantifying mismatches between the compositional structures assigned to the same sentence by different graphbanks; we then present our extended am+ algebra, with which we can systematically reshape these compositional structures to align them across graphbanks (section 3). finally, we provide key examples of how our methodology normalizes specific linguistic phenomena that pose challenges to mr design and differ in representation across graphbanks (section 4). using our methods, we increase the match between the am dependency trees (compositional structure) of the different graphbanks to 76.3 (dm-pas), 78.8 (dm-psd), and 82.0 (pas-psd) directed unlabeled f-score, compared to 63.5, 55.7, and 57.0 for lindemann et al.'s am dependency trees (section 5). this is a drastic improvement over the unlabeled f-scores of 64.2, 26.1, and 29.6 for the original graphs @xcite . we additionally demonstrate that when training a graph parser on a tiny graphbank combined with larger graphbanks using multi-task learning (mtl), our methods improve parsing accuracy by up to 2.6 points f-score over mtl without normalized am dependency trees. our work serves as a proof of concept that linguistically-informed methods to normalize compositional structure across graphbanks is successful and can be applied at broad-scale and potentially to more complex graphbanks (section 6). ## quantifying and normalizing compositional mismatches the am dependency trees of l19 (henceforth, l19 am trees) naturally normalize some of the differences of the original mrs we consider in this paper (fig. 3 ). here, we go further and ask: how far can this normalization be pushed? given our corpus, if we can find a single compositional structure for each sentence that corresponds to all three mr graphs, this suggests that the designs and annotators of the three graphbanks tacitly agree on compositional structure; surface differences between the graphs can then be explained by low-level representations of individual words. in this work, we take a first step towards this goal, devising linguistically motivated changes to the compositional structures that we can apply automatically to all sentences in the corpus to normalize them. fig. 3(g , h , i ) shows a successful normalization by our method: the edges of the am trees are the same across all three mrs; all differences are delegated to the graph constants. take for example the determiner "the", which is originally ignored in psd but not in dm or pas. in the normalized psd am tree (fig. 3i ), it has a constant that consists of an unlabeled root marked with a det-source, and has an incoming mod det edge from "cat". when this semantically empty graph modifies the graph for "cat" g cat (the mod det operation), it does not in fact change that graph g cat but just merges with its root without leaving a trace. this way, the normalized am tree in fig. 3i still evaluates to the original graph in fig. 3c but includes the added mod det edge to match dm and pas. the trees also normalize the copula and negation, which requires more subtle methods detailed in section 4. the cat is not lazy root bv arg1 neg (a) dm graph the cat is not lazy root det_arg1 adj_arg1 adj_arg1 verb_arg1 verb_arg2 (b) pas graph the cat is not lazy root rhem act-arg pat-arg (c) psd graph the cat is not lazy root moddet apps appm the det bv cat ⊥ not m neg lazy s arg1 (d) l19 am tree for dm the cat is not lazy root moddet apps appo modm the det det_arg1 cat is s verb_arg1 o[s] verb_arg2 not m adj_arg1 lazy s adj_arg1 (e) l19 am tree for pas the cat is not lazy root apps appo modm ⊥ cat is s act-arg o pat-arg not m rhem lazy (f) l19 am tree for psd the cat is not lazy root moddet apps appo appm the det bv cat o[s] not m neg lazy s arg1 (g) normalized tree for dm the cat is not lazy root moddet apps appo appm the det det_arg1 cat is s verb_arg1 o[s] cats and cute dogs are playing in the house root apps modcoord mods mods appo moddet cat ⊥ cute s arg1 dog coord _and_c ⊥ play s arg1 in s arg1 o arg2 the det bv house (a) l19 am tree for dm cats and cute dogs are playing in the house root apps appop appop2 modm modm cat and op conj.member op2 conj.member cute m rstr dog ⊥ play s act-arg ⊥ ⊥ house m loc (b) l19 am tree for psd cats and cute dogs are playing in the house root apps modo appop1 appop2 mods modo apps moddet cat and op1 coord_arg1 op2 coord_arg2 cute s adj_arg1 dog are s aux_arg1 o aux_arg2 play s verb_arg1 in o prep_arg1 s prep_arg2 the det det_arg1 house (c) l19 am tree is equal to normalized tree for pas cats and cute dogs are playing in the house root apps modaux appop1 appop2 mods mods appo moddet cat op1 op2 _and_c cute s arg1 dog aux play s arg1 in s arg1 o arg2 the det bv house (d) normalized tree for dm cats and cute dogs are playing in the house root apps modaux appop1 appop2 modm modo apps moddet cat and op1 conj.member op2 conj.member cute m rstr dog aux play s act-arg o s loc det house (e) normalized tree for psd figure 4: l19 am trees and normalized trees for sdp graphs presented in fig. 1. figure 5 : the set of local patterns for w we distinguish based on their structures in l19 am trees. ## normalizing key compositional structures across graphbanks examining the pattern signatures of distinct tokens, we make a key finding: about half of the nodes exhibit the same pattern in all three graphbanks. these aligned patterns are typically in the form of open-class words and predicate-argument structure, core components of mr design. for example, adjectives often display the signature [mod / mod / mod], and nouns [basic / basic / basic]. uniform pattern signatures represent a priori shared compositional structure across graphbanks that we do not need to normalize. for unaligned pattern signatures, i.e. ones that display a pattern mismatch across graphbanks, our goal is to align the pattern signature so the patterns for each graphbank are equivalent. this normalization process follows a three-step process: (i) identify the pattern signatures with distinct local patterns for the three mrs; (ii) match these patterns to meaningful linguistic phenomena; and (iii) manipulate the compositional structures to align the pattern signature across graphbanks, using the am+ algebra. a close investigation of the unaligned pattern signatures reveals three main disparities between the graphbanks that we can now address with the am+ algebra: (i) 'ignored' elements; (ii) disagreement in headedness of constructions; and (iii) relational constructions. here we detail representative examples for each case; appendix c has additional examples. for all normalizations, we apply transformations using the am+ algebra directly to the am dependency trees of l19. the transformations are carefully designed not to change the graph the am dependency trees evaluate to. we do this for all am dependency trees by iterating over the sentences in the graphbanks and applying a transformation whenever its corresponding pattern signature matches. the transformations are always applied in the same order ( ## evaluation we evaluate the quality of the normalized am dependency trees by measuring cross-formalism similarity and suitability for parsing on the sdp corpora of the 2015 shared task @xcite . we apply l19's preprocessing of coordination for psd. due to the similarities of the graphbanks, it is appealing to use multi-task learning (mtl), which shares a subset of the model weights across graphbanks to allow training data for one graphbank influence the prediction on another. we expect this to help particularly in scenarios where there is limited data for one graphbank but large amounts for others. ## discussion the methodology we detail in sections 3 and 4 allows us to normalize roughly 60% of differences across compositional structures of the sdp graphbanks. our work establishes that a small set of linguisticallygrounded transformations are quite powerful towards creating uniform compositional structure; in fact, we find that all of the divergent patterns across graphbanks that we detect are 'superficial' in the sense that, given proper attention and methodology, we can normalize them. the individual phenomena that create inconsistencies across graph representations and their corresponding l19 am trees predictably adhere to a zipfian distribution (appendix b). to manually develop automatic normalizations for 100% of the unaligned patterns may be infeasible. these observations seem to indicate that the remaining differences between our normalized am trees are due to practical, implementation concerns rather than conceptual ones. a possible approach to solving the practical issues would be a fully automated method with statistical rather than human guidance. in a low resource scenario, manual annotations could also be a feasible solution. it is to note though that the am+ algebra is not all-powerful; the l19 preprocessing step for coordination in psd remains necessary. despite the fact that all phenomena we examine can be normalized, how they are normalized provides insights into the differences among the underlying mrs. while structural differences like those seen in negation and coordination essentially reduce to mildly different graph constants, ignored words or underspecified attachment that may create ambiguity are a different story. namely, they make explicit the different kinds of information that some mrs include and others do not, forcing us to choose the most informative compositional structure (such as including determiners for psd). curiously, our parser does not seem to care much about this added, more-explicit information, but we believe future work in cross-framework parsing with more distinct graphbanks may provide more meaningful insight on this fact. extending our work to other graphbanks will shed light on the effect of graph flavours; all three sdp formalisms investigated in this paper share the same graph 'flavour' possibly leading to very uniform graph constants and facilitating normalization. adding other formalisms with different graph flavours is therefore an interesting future research direction, but limited by the availability of parallel texts. ## conclusion we have shown how annotations from different graphbanks (specifically, dm, pas, and psd) can be normalized at the level of their compositional structure using principled linguistic reasoning. we achieve this by updating the am algebra to the am+ algebra, which allows more flexibility in adapting compositional structures across mrs. by working at the compositional level of the graphs using am dependency trees, we are able to quantify mismatches between different graphbanks, systematically reshape these mismatches to make them more uniform across graphbanks, and thereby increase the match between the compositional structures for the three graphbanks. such work serves as a proof of concept that normalization is possible and contributes to a broader effort to increase parallel mr parsing accuracy. mice run and hide root arg1 arg1 and c (a) dm graph mice run and hide root verb arg1 verb arg1 coord arg1 coord arg2 (b) pas graph mice run and hide root app s app op1 app op2 mouse run s arg1 op1[s] op2[s] _and_c hide s arg1 (c) normalized am dependency tree for dm mice run and hide root app s app op1 app op2 mouse run s verb_arg1 and op1[s] coord_arg1 op2[s] pas is the only sdp mr to broadly annotate tokens that are mostly syntactic markers. this pattern includes punctuation, infinitival "to", passive "by", and particles of particle verbs. as with determiners, we add empty modifiers, here to dm and psd. we restrict our fix to pos tags that indicate punctuation or those of in, to or rp, which covers 97% of pattern. temporal and aspectual auxiliary verbs. these adhere to the [empty / cmod / empty] pattern. an example of this is "are" in fig. 4 . these auxiliaries are annotated only in pas, where the l19 am trees (fig. 4c ) use complex modification to create the reentrancy observed in fig. 1b . while the cmod pattern is different from the mod pattern as far as the resulting graph is concerned, both patterns simply have an incoming mod edge when only looking at the am tree edges. thus, we can apply the same fix here as for the punctuation case. we restrict our fix to pos tags that start with v, which covers 82% of pattern.