Coordinate Constructions in E nglish Enhanced U niversal D ependencies: Analysis and Computational Modeling (2021)

Abstract: In this paper, we address the representation of coordinate constructions in Enhanced Universal Dependencies (UD), where relevant dependency links are propagated from conjunction heads to other conjuncts. English treebanks for enhanced UD have been created from gold basic dependencies using a heuristic rule-based converter, which propagates only core arguments. With the aim of determining which set of links should be propagated from a semantic perspective, we create a large-scale dataset of manually edited syntax graphs. We identify several systematic errors in the original data, and propose to also propagate adjuncts. We observe high inter-annotator agreement for this semantic annotation task. Using our new manually verified dataset, we perform the first principled comparison of rule-based and (partially novel) machine-learning based methods for conjunction propagation for English. We show that learning propagation rules is more effective than hand-designing heuristic rules. When using automatic parses, our neural graph-parser based edge predictor outperforms the currently predominant pipelines using a basic-layer tree parser plus converters.

## introduction the universal dependencies (ud) formalism @xcite is a framework for representing syntactic dependencies between words, prioritizing links between content words. ud parses provide two levels of analysis. basic dependencies form standard syntactic dependency trees in which each node has exactly one governor (black links on top in figure 1 ). enhanced dependencies @xcite are extensions of these trees including additional relations (blue links below sentence) with the aim of representing linguistic phenomena such as coordination, control, or relative clauses. they have been shown to provide valuable in 1594 perez wrote and published a book . input for information extraction tasks @xcite . one of the most frequent phenomena addressed by enhanced ud is coordination. in the english web treebank (ewt), more than 15% of all sentences contain conjoined verbs. hence, a good representation of coordination clearly is crucial for downstream tasks. for example, in figure 1 , the enhanced layer explicitly captures that the arguments of the predicate "wrote" also fill the corresponding slots of "published," which is highly relevant for natural language understanding tasks. in many cases, enhanced representations can be derived from the gold basic layer in a rule-based fashion @xcite . the currently available english enhanced ud treebanks have been created by applying such a converter. however, we are not aware of a large study regarding their correctness and completeness. focusing on precision, the converter only propagates core arguments. in this paper, we take a complementary approach, performing a large-scale annotation study in order to determine which set of links should be propagated from a semantic perspective. on a new dataset of 1,417 sentences from the ewt containing conjoined verbs, we verify and if necessary modify/extend the links involved in coordinate constructions. we argue that adjuncts such as obliques should in fact be propagated at times, e.g., in figure 1 , the additional (green dotted) link that we propose to add facilitates answering questions like "when was the book published?". to the best of our knowledge, our work constitutes the first large-scale annotation effort of this kind. on the basis of our new dataset, we make the following contributions. first, we estimate the degree of correctness and completeness of the rulebased converter/existing treebanks. we find that the converter usually generates correct graphs when applied on gold basic trees, with some notable exceptions involving non-parallel syntactic constructions (e.g., conjuncts having different voice or mood). in addition, the converter does not propagate links correctly in presence of multiple interacting conjunctions. our inter-anntotator agreement study shows high overlap for propagation decisions, with f1 between pairs of annotators of about 0.9 on average and around 0.75 for obliques. second, we address the question of how to create high-quality treebanks for enhanced ud from gold basic dependencies, again focusing on coordinate constructions. based on the findings of our corpus study, we improve the rule-based converter by @xcite . we also compare machine-learning (ml) based conjunction propagation classifiers in the form of (a) svmbased classifiers as previously used for finnish, swedish and italian @xcite , and (b) a novel neural approach integrating tree-and roberta-based features. we find that all systems mostly rely on tree-based features, but contextual embeddings also provide useful information. performance on propagation decisions has promising f1 around 0.9, already similar to human agreement. ml-based classifiers outperform the rule-based converters on the ewt test set. third, we compare methods for extracting propagated dependencies in an automatic parsing setting. the currently predominant approach is to run a basic-layer tree parser and then the same converter that has been used for gold standard construction. we propose to use a neural graph-parser based edge predictor with an architecture similar to @xcite instead, and show that this approach outperforms pipelines by around 9 points f1 on propagating links in conjunctions. in sum, our contributions include: (1) a manually curated large-scale dataset of 1,417 sentences addressing semantically motivated correct and complete conjunction propagation in enhanced ud; (2) the proposal of novel neural approaches to conjunc-tion propagation; and (3) experimental evidence that these models outperform rule-and pipelinebased approaches in both gold standard treebank enhancing and automatic parsing settings. to the best of our knowledge, our work constitutes the first principled comparison of various approaches to propagating conjunctions in enhanced ud on manually corrected gold standard data for english. both our model implementations and the dataset are freely available. 1 we will contribute our changes to the ewt corpus to the next ud release. conjunction propagation classifiers. @xcite present an svm-based approach for enhancing finnish syntax trees. they observe high performance on conjunction propagation when operating on gold basic trees, but markedly worse results when using automatic parser output. @xcite evaluate a similar approach for swedish and italian. we show that their approach also works well for english, and extend it with neural models and contextualized word embeddings. @xcite de-lexicalize their rule-based converter developed for italian, showing that their language-independent system also correctly produces most of the propagations for english. however, they evaluate on ewt, which is itself the result of a rule-based system. in contrast, we evaluate on manually checked gold data. for the related task of dealing with gapping constructions such as "paul likes coffee and mary tea," @xcite reconstruct elided predicates by first parsing into an intermediate representation and then applying either a rule-based or an ml-based algorithm to copy over lexical material. we here focus on dependency propagation and operate on gold tokens as annotated in the enhanced ud treebanks, which already include traces. other related work exists in the area of manual and rule-based error correction on ud treebanks @xcite . there is still little published work regarding fully automatic enhanced ud parsing, however, the topic has recently been addressed by the iwpt 2020 shared task @xcite . among the top-performing systems, several approaches first parse into basic ud and then added transformation rules (e.g., @xcite . others directly employ graph parsing techniques (e.g., @xcite @xcite . the overall winner turkunlp @xcite transforms enhanced ud into a tree format and then makes use of udify @xcite . in addition, much work exists on semantic dependency parsing (sdp, @xcite @xcite . these works differ from udbased approaches as the respective formalisms represent meaning less close to syntactic structure, thus not requiring propagation. from a modeling point of view, our work is most similar to that of @xcite , who also use a graph-based biaffine architecture for enhanced ud parsing, and to that of @xcite , who achieve state-of-the-art results for sdp. ## coordinate constructions dataset in this section, we describe our creation of our manually created dataset and analyse the results. ## modeling in this section, we describe three approaches to generating links propagated due to coordination: ## experiments in this section, we describe our experiments on creating enhanced ud representations for coordinate constructions. analogous to @xcite , we measure precision, recall and f1 on enhanced links that are the result of propagation in coordinate constructions. for all experiments, we use gold sentence segmentation and tokenization, and evaluate on our manually corrected sentences from the dev and test sets of the ewt corpus. ## conclusion and outlook we have presented a large-scale manually curated dataset for conjunction propagation in english. in contrast to previous work focusing on highprecision rule-based propagation, we propagate links in all cases that semantically suggest argument or adjunct sharing. in the gold standard treebank enhancing setting, we found ml-based models to outperform the de-facto standard rule-based converter by learning to exploit mostly structural features. however, one of our main insights is that neither rule-based nor ml-based classifiers work well on noisy parser output precisely because of this reliance on structural information. we propose to use a graph-parser based edge predictor instead and show that it outperforms pipeline-based models by a large margin. our model reaches f1 scores between 0.75 and 0.78 with a precision of more than 0.82, a level of performance that may already be useful in downstream tasks. our models could be used for creating highquality enhanced-level representations of conjunctions for the remaining english data, and could thus help in a ud community effort to continuously improve the ud treebanks. future work also includes the study of conjunction propagation methods for further languages. our in-depth study on english data provides several insights that we expect to be transferable cross-linguistically. first, conjunction propagation can to some extent be addressed using heuristic rules, but capturing the full semantic nature of the task requires manual annotation. second, given appropriate training data, our machine-learning based approaches are also applicable to other languages. in addition, it would be interesting to see if manually annotated data for coordinate constructions may be useful in natural language understanding tasks such as natural language inference (nli). this is especially true for "stress test" datasets such as conjnli @xcite , which are designed to specifically test models' capabilities to process coordination. finally, as morphological features are generally important for this task, improving their automatic prediction (see e.g., @xcite as well as ud's gold standard seems to be a promising way to go. our work has demonstrated the value of a linguistically motivated corpus study of a syntactic-semantic phenomenon, and shown that given manually curated data, rules for conjunction propagation can be learned effectively.