Cross-Lingual Dependency Parsing via Self-Training (2020)

Abstract: Recent advances of multilingual word representations weaken the input divergences across languages, making cross-lingual transfer similar to the monolingual cross-domain and semi-supervised settings. Thus self-training, which is effective for these settings, could be possibly beneficial to cross-lingual as well. This paper presents the first comprehensive study for self-training in cross-lingual dependency parsing. Three instance selection strategies are investigated, where two of which are based on the baseline dependency parsing model, and the third one adopts an auxiliary cross-lingual POS tagging model as evidence. We conduct experiments on the universal dependencies for eleven languages. Results show that self-training can boost the dependency parsing performances on the target languages. In addition, the POS tagger assistant instance selection can achieve further improvements consistently. Detailed analysis is conducted to examine the potentiality of self-training in-depth.

## introduction cross-lingual dependency parsing has received increasing attention in recent years @xcite @xcite @xcite @xcite @xcite , which aims to parse target low-resource language with the supervision of resource-rich language. in this paper, we focus on the unsupervised setting @xcite @xcite @xcite @xcite , where no targeted dependency treebank is given. recent advances of multilingual word representations @xcite @xcite @xcite @xcite has substantially promoted cross-lingual dependency parsing, especially serving as the basic input features for model transfer methods @xcite @xcite . they reduce the input divergences between languages significantly. as a result, the cross-lingual transfer learning setting can be considered highly similar to the monolingual semi-supervised and cross-domain settings. in light of this, the self-training strategy, which is widely adopted for cross-domain parsing @xcite @xcite @xcite , can be potentially applicable for cross-lingual dependency parsing as well. however, relatively little work has demonstrated the effects of this potential method. instance selection for the next-round training is the key to self-training @xcite @xcite @xcite , which requires a certain criterion to rank the automatic outputs from the baseline model @xcite @xcite . such criteria are typically derived from the baseline model directly, for example, the prediction probability @xcite , and the delta probability between the final output and the second-best candidate output @xcite . here we hypothesize that we can improve the performance of self-training by an auxiliary task which is highly corrective with the target task. a natural auxiliary task for cross-lingual dependency parsing is universal part-of-speech (pos) tagging. ## models in this section, we describe the dependency parsing and pos tagging models, and the key details which would be used in the self-training. ## self-training the self-training framework for cross-lingual dependency parsing is as follows. first, a cross-lingual dependency parser (section 3.1) trained on a source language corpus is used to parse the raw corpus of a target language. in particular, pos tags of the raw corpus are produced by a supervised cross-lingual pos tagger (section 3.2). next, we select a number of auto-parsed dependency trees from the outputs, and use them as the extra corpus to enhance the dependency parser. instance selection is a key factor to the performance of self-training. we investigate two instance selection strategies based on the baseline dependency parser, and further suggest another alternative by using the cross-lingual pos tagger. 2 we do not exploit crf as its final impact on self-training is marginal while introduces addition calculation cost. ## conclusions we investigated self-training for unsupervised cross-lingual dependency parsing. a baseline dependency parser with multilingual bert representations is trained and used to parse sentences of a target language and a set of the resulting dependency trees are selected to help training a target language dependency parser. we studied three different instance selection strategies, including two criteria by using the baseline dependency parser, and one criterion guided by a multilingual pos tagger. results showed that selftraining is effective in general for cross-lingual parsing. with the pos-assistant strategy, our final model brings the largest improvements, demonstrating the effectiveness of the method.