End-to-end m BERT based Seq2seq Enhanced Dependency Parser with Linguistic Typology knowledge (2021)

Abstract: We describe the NUIG solution for IWPT 2021 Shared Task of Enhanced Dependency (ED) parsing in multiple languages. For this shared task, we propose and evaluate an End-to-end Seq2seq mBERT-based ED parser which predicts the ED-parse tree of a given input sentence as a relative head-position tag-sequence. Our proposed model is a multitasking neural-network which performs five key tasks simultaneously namely UPOS tagging, UFeat tagging, Lemmatization, Dependency-parsing and ED-parsing. Furthermore we utilise the linguistic typology available in the WALS database to improve the ability of our proposed end-to-end parser to transfer across languages. Results show that our proposed Seq2seq ED-parser performs on par with state-of-the-art ED-parser despite having a much simpler de- sign.

## introduction the enhanced universal dependency (eud) parsing @xcite framework is an interesting extension of the standard dependency parsing framework, which provides additional significant syntactic and semantic knowledge, that is missing in a standard dependency parse-tree. such additional knowledge can be crucial for numerous downstream nlp tasks. the iwpt 2021 shared task @xcite requires the participants to perform the enhanced dependency parsing of the given testsentences, in addition to predicting the sentenceboundaries, token-boundaries, lemmatization, postags, morphological features and the basic dependency relations. the participants are provided with the blind test-corpora in 17 languages, and are expected to perform the enhanced dependency parsing on each sentence within these test corpora and submit the results (in the conllu format). for this iwpt 2021 shared task @xcite we propose and evaluate the performance an end-to-end mbert based se2seq ed-parser which performs five key tasks namely upos-tagging, ufeats-prediction, lemmatization, dependency-parsing and enhanced dependencyparsing in multi-tasking settings. our proposed model is an extension of the popular udify model @xcite which is the state-of-the-art mbert based multilingual dependency parser, and is inspired by @xcite which is an end-to-end seq2seq dependency-parser. we describe the udify model in section 2. we trained our proposed ed-parser on a large joint polyglot corpus created by concatenating all the treebanks in the provided training dataset for iwpt 2021 shared task, and evaluated it on eight of the 17 provided blind test-corpora. furthermore, similar to previous approaches @xcite , we utilized the linguistic typology knowledge available in world atlas of language system (wals) database (haspelmath, 2009) to improve the cross-lingual transferring ability of our proposed ed-parser. we fed these typology features together with token-ids into the proposed ed-parser. we describe the architecture of our end-to-end mbert based se2seq ed-parser in detail in section 3. ## mbert based seq2seq ed parser figure 2b depicts the architecture of the proposed ed parser. our proposed end-to-end ed parser is an extension of the udify (kondratyuk and straka, 2019) model described in section 2.2, with one additional component namely the relative head sequence predictor which predicts the relative headposition of the tag-sequence representing the unlabelled enhanced-dependency parse-tree of the input sentence (as the fifth auxiliary task in the multitasking udify model). ## experiments as already explained, our proposed end-to-end seq2seq ed-parser is trained on a large joint polyglot corpus created by concatenating all the treebanks in the provided training dataset for iwpt 2021 shared task. we evaluated our parser on test corpora provided for the iwpt 2021 shared task in eight distinct languages namely bulgarian, estonian, english, latvian, lithuanian, russian, slovak and swedish. we outline the results achieved by our proposed model in detail in section 5. ## results and conclusion appendix a compares the results achieved by our ed-parser with the results achieved by the other participants of iwpt 2021 shared tasks.