Cross-lingual Inflection as a Data Augmentation Method for Parsing (2022)

Abstract: We propose a morphology-based method for low-resource (LR) dependency parsing. We train a morphological inflector for target LR languages, and apply it to related rich-resource (RR) treebanks to create cross-lingual (x-inflected) treebanks that resemble the target LR language. We use such inflected treebanks to train parsers in zero- (training on x-inflected treebanks) and few-shot (training on x-inflected and target language treebanks) setups. The results show that the method sometimes improves the baselines, but not consistently.

## introduction dependency parsers @xcite @xcite already achieve accurate results for certain setups @xcite . yet, they require large amounts of data to work, which hurts low-resource (lr) scenarios. in this line, authors have studied how to overcome this problem. on data augmentation, recent approaches have replaced subtrees of sentences to generate new ones @xcite . on cross-lingual learning, authors have explored delexicalized approaches from rich-resource (rr) treebanks. @xcite . @xcite permuted constituents of distant treebanks to generate synthetic ones that resembled the target language. @xcite ; @xcite merged treebanks to train multilingual parsers that sometimes could outperform the equivalent monolingual version, which has applications for less-resourced parsing. in the context of multilingual representations, @xcite trained a zero-shot parser on top of a polyglot language model, relying on merged rr treebanks too. in other matters, morphological inflection @xcite generates words from lemmas and morphological feats (e.g. @xmath0. also, it is known that morphology helps parsing and that morphological complexity relates to the magnitude of the improvements @xcite . yet, as far as we know, there is no work on cross-lingual morphological inflection as a data augmentation method for parsing. here, we propose a technique that lies in the intersection between data augmentation, cross-lingual learning, and morphological inflection. contribution we introduce a method that uses cross-lingual morphological inflection to generate 'synthetic creole' treebanks, which we call x-inflected treebanks. to do so, we require a source language treebank from a closely-related language (for which lemmas and morphological feats are available), and a morphological inflection system trained for the target language. this way, we expect to generate x-inflected treebanks that should resemble to a certain extent the target language (see figure 1 ). the goal is to improve the parser's performance for languages for which little or no annotated data are available, but for which we can train an accurate morphological inflection system that can be later applied to a related rr treebank and resemble the target language. the code is available at https://github.com/ amunozo/x-inflection. ## preliminaries we now describe the basics of our work: we use unimorph (um; mccarthy et al., 2020) for morphology and universal dependencies (ud; zeman et al., 2020) ## x-inflection as data augmentation character-level models, such as the ones used for morphological inflection, identify shared morphemes across languages with overlapping alphabets @xcite . thus, if two languages share a significant amount of lemmas, n-grams or inflections, an inflector for the first language could maybe produce noisy-but-useful inflected forms for lemmas and feats available for the second language. we hypothesize that this idea can be used for syntactic data augmentation in lr scenarios. under the assumption that an inflector is available for our target lr language (easier than annotating syntactic data), we could use it to transform a related rr treebank, obtaining silver syntactic data that, despite lexical and grammatical imperfections, could help boost performance. our method consists of three steps: (i, §3.1) training an inflector for a given target language using um data, (ii, §3.2) x-inflecting the source ud treebank, cross-lingually applying the inflector trained in (i), and (iii, §3.3) training the x-inflected parsers. we summarized the process in figure 1 . ## experiments we test both (i, §4.1) zero-shot and (ii, §4.2) fewshot setups. for evaluation, we use unlabeled (uas) and labeled attachment scores (las). appendix e reports the hardware and costs. data we use 10 lr and 21 rr treebanks. although our method can be applied to any pair of treebanks, the availability of um and ud resources (in the sense of having lr languages in um and related rr languages in ud) restricts our empirical analysis to indo-european and uralic languages (see ## discussion the results show that the proposed method is able to improve parsing results for some treebank pairs under both zero-and few-shot setups, but it also obtains decreases for other pairs. due to the high number of factors involved, we were unable to clearly isolate those that are beneficial and those that are harmful. however, we identified some reasons that could partially explain the behaviour of the method: ## conclusion by cross-inflecting a rich-resource ud treebank using an inflector from a low-resource related language, we can obtain silver, syntactically annotated data to train dependency parsers. although containing noise and grammatical imperfections, we aimed to test whether the approach could boost performance. the results show that it is possible to obtain improvements (but also decreases) both for zero-and few-shot setups. about this, we could not clearly identify what aspects make the approach succeed or fail. although we identified moderate correlations between scores and the amount of available um data for the target language, we hypothesize that other aspects that are hard to measure could be playing a role: (i) incorrect/incomplete feature conversion from um to ud schemata that might make the cross-lingual inflections carry different information that the inflections in the original language, or (ii) unknown input features for a given inflector due to differences in exhaustiveness between the um and ud annotations.