Unifying the Treatment of Preposition-Determiner Contractions in G erman U niversal D ependencies Treebanks (2020)

Abstract: HDT-UD, the largest German UD treebank by a large margin, as well as the German-LIT treebank, currently do not analyze preposition-determiner contractions such as zum (= zu dem, “to the”) as multi-word tokens, which is inconsistent both with UD guidelines as well as other German UD corpora (GSD and PUD). In this paper, we show that harmonizing corpora with regard to this highly frequent phenomenon using a lookup-table based approach leads to a considerable increase in automatic parsing performance.

## introduction universal dependencies (ud) are a cross-linguistic dependency grammar framework driven by a largescale multi-lingual community effort @xcite . in general, ud prioritizes relations between content words. the treatment of function words, being a rather language-specific issue, is to date sometimes inconsistent even across the treebanks of a single language. function words including prepositions or negation words are often contracted with other words, which requires to decide whether to keep them as a fused unit or split them up during tokenization -a non-trivial problem. 1 the german language allows to fuse certain combinations of preposition+determiner into a single token, resulting in what ud refers to as a multiword token, i.e., a single token that contains more than one syntactic word. examples include zum (= zu dem, "to the") and ins (= in das, "into the"). these constructions can even be regarded as lexicalized, i.e., as belonging to the inventory of the language's lexicon @xcite . expanding such contractions into several tokens does not depend on the context, hence, treating them as multi-word tokens is rather straightforward. the only caveat is assigning the correct morphological features to the determiner, but these can easily be retrieved from the head noun. the current ud annotation guidelines for german 2 suggest treating preposition-determiner contractions as multi-word tokens in the way outlined above, and their treatment is implemented accordingly in the german gsd treebank (292k tokens) as well as in german pud (21k tokens). however, hdt-ud @xcite , the largest german ud treebank (with 190k sentences and 3.4 million tokens currently the largest available ud treebank overall), as well as the small german-lit treebank (40k tokens, salomoni (2017)), do not expand these tokens. this may lead to inconsistency-based parsing errors in cross-treebank experiments or when training a parser on several treebanks. in this paper, we analyze the extent of the problem, finding that in hdt-ud, 25% of all sentences contain such contractions. our contributions are (i) the development of a simple lookup-based script for splitting german preposition-determiner contractions into several tokens, and (ii) a set of experiments showing that on the relevant sentences, the increased consistency leads to an increase in parsing accuracy by up to 0.8 points in terms of las f1. hence, this paper constitutes a case study of the improvements we can expect from careful linguistic data analysis and corpus harmonization. we contribute our conversion script, as well as the fixed versions of the hdt-ud and german-lit corpora, for the next ud release. contraction expansion count % sents im in dem 26236 12.8 am an dem 7764 3.9 zum zu dem 7584 3.9 zur zu der 6149 3.1 vom von dem 3404 1.8 beim bei dem 2795 1.4 ins in das 1422 0.7 fürs für das 233 0.1 ans an das 160 0.1 übers über das 147 0.1 total 56150 25.0 (a) german-hdt-ud contraction expansion count % sents im in dem 89 4.2 zur zu der 44 2.0 zum zu dem 27 1.4 vom von dem 17 0.9 am an dem 17 0.9 ins in das 8 0.4 aufs auf das 7 0.3 beim bei dem 5 0.3 fürs für das 5 0.3 beym bey dem 3 0.1 total 222 9.8 (b) german-lit ## expanding contractions based on the above discussion, we propose a simple lookup-based method for expanding prepositiondeterminer contractions in german ud corpora into multi-word tokens in order to make the data consistent with the ud annotation guidelines. figure 2 shows our algorithm, which operates on files of the conll-u format. foot_1 note that we do not make changes for the token z. from z. b. = zum @xmath0," as well as in several other infrequent special cases resulting from annotation errors. since prepositions and determiners are closed word classes and the mapping from contractions to their expansions is unambiguous in german, we use a simple lookup table (see we ensure the correctness of the output by manually inspecting some of the resulting annotated sentences as well as running the official ud validation script. foot_2 ## parser evaluation to evaluate how our changes to the corpora affect parsing accuracy, we train the state-of-the-art udify parser @xcite on the existing gsd corpus as well as the original and modified versions of the hdt-ud corpus, and report scores on various versions of the test sets. (lit and pud only provide test data.) we keep all hyperparameters the same as in the original setup, except that (a) we use the german bert model by deepset foot_3 to initialize bert weights; and (b) we only train on hdt-ud for 25 epochs due to the extremely large size of the corpus. we report results for gold tokens. an analysis by label type confirms that increases in accuracy are mainly caused by case and det. for example, when training on gsd, las f1 of case increases from 96.28 (hdt -exp ) to 96.60 (hdt +exp ); det increases from 96.74 (hdt -exp ) to 97.72 (hdt +exp ). we also observe modest improvements in parsing accuracy on a number of other dependency labels such as obl, nmod, and ccomp, indicating that surrounding syntactic constructions also benefit from the consistent handling of contractions. interestingly, our results also show that the parser trained on gsd, despite not having encountered contractions during training, still attaches the vast majority of contractions in hdt orig /lit orig correctly. we suspect that this may be owed to bert's ability to generalize from simple prepositions to contractions because of their similar distribution in the pre-training data. within the context of surface-syntactic universal dependencies (sud, gerdes et al. ( 2018 )), it has been proposed to treat propositions as heads, moving away from ud's focus on content words and applying distributional criteria for the units instead. in fact, in sud, the question of how to deal with these contractions is even more pressing: because the preposition part of the contraction would be the head of a noun, but the determiner part would be a dependent, it is not quite clear what the overall syntactic relation of the contraction to the noun should be if left as one syntactic word. french amalgames. a similar issue arises in the word segmentation of french amalgames (contractions). here, the situation is slightly more complicated due to the ambiguity of du/des, which may occur either as indefinite determiners as in des enfants jouent ("(some) kids play"), in partitive constructions such as je bois du lait (= de le, "i drink (some) milk"), or in possessive constructions such as la lettre des filles (de les, "the letter of the girls"). currently, the major french treebanks annotate infinite determiner des as det, while they split the other two cases into two tokens. however, a context-dependent treatment of tokenization bears technical difficulties for automatic processing; a discussion by the treebank maintainers suggests to always split these contractions and annotate the indefinite determiner case using a fixed relation between the two components. 7 to date, this does not seem to have been implemented. ## conclusion in this paper, we have carefully analysed the treatment of preposition-determiner contractions in german ud corpora. harmonizing representations lead to increases in las f1 of up to 0.8, indicating that a unified treatment of these frequent construction is essential. we here have presented a case study of how to unify word segmentation for the relatively simple case of german preposition-determiner contractions. future work includes addressing similar phenomena in more difficult situations such as