Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders (2020)

Abstract: We consider the task of cross-lingual adaptation of dependency parsers without annotated target corpora and parallel corpora. Previous work either directly applies a discriminative source parser to the target language, ignoring unannotated target corpora, or employs an unsupervised generative parser that can leverage unannotated target data but has weaker representational power than discriminative parsers. In this paper, we propose to utilize unsupervised discriminative parsers based on the CRF autoencoder framework for this task. We train a source parser and use it to initialize and regularize a target parser that is trained on unannotated target data. We conduct experiments that transfer an English parser to 20 target languages. The results show that our method significantly outperforms previous methods.

## introduction supervised learning of dependency parsing is difficult for low-resource languages because of the lack of large treebanks. on the other hand, crosslingual adaptation of dependency parsers from richresource languages to low-resource languages has shown a lot of promise @xcite @xcite @xcite @xcite , especially with the help of cross-lingual word representation @xcite or part-of-speech (pos) tags @xcite . in this paper, we consider the scenario in which there is only unannotated data for the target language that is not parallel to the source language treebank. a simple strategy is zero-shot transfer or direct transfer, which trains a parser on the source treebank and then directly applies it to the target language @xcite . in order to leverage unannotated target data, @xcite propose to employ an unsupervised generative parser that can be trained on the target data while also regularized via soft parameter tying by a source parser. however, generative parsers are known to underperform discriminative parsers in rich-resource scenarios, mostly because of the unrealistic independence assumptions typically made by generative parsers. in fact, @xcite show that when they use multilingual bert @xcite as the cross-lingual word representation, their method underperforms direct transfer of a strong discriminative parser. in this paper, we propose to instead use an unsupervised discriminative parser based on the crf autoencoder framework @xcite for cross-lingual parser adaptation. we perform supervised training of the source parser with the source treebank and then use it to initialize the target parser. the target parser is then trained on the unannotated target data in an unsupervised way while being regularized by the source parser. we employ three regularization methods proposed by @xcite that encourage similarity between model parameters and edge scores respectively of the source and target parsers. our experiments of transferring from english to 20 target languages show that our method significantly outperforms previous methods. ## conclusion in this paper, we employ unsupervised discriminative parsers based on the crf autoencoder framework for unsupervised cross-lingual adaptation of dependency parsers. we initialize the target model using the source model and train it on unannotated target data in an unsupervised way, with three forms of regularization that encourage its similarity