Enhanced U niversal D ependency Parsing with Second-Order Inference and Mixture of Training Data (2020)

Abstract: This paper presents the system used in our submission to the IWPT 2020 Shared Task . Our system is a graph-based parser with second-order inference. For the low-resource Tamil corpora, we specially mixed the training data of Tamil with other languages and significantly improved the performance of Tamil. Due to our misunderstanding of the submission requirements, we submitted graphs that are not connected, which makes our system only rank 6th over 10 teams. However, after we fixed this problem, our system is 0.6 ELAS higher than the team that ranked 1st in the official results.

## introduction based on the universal dependencies (ud) @xcite , the enhanced universal dependencies (eud) @xcite foot_0 are non-tree graphs with reentrancies, cycles, and empty nodes to deal with the problem that purely rooted trees cannot adequately represent grammatical relations. we found that we can reduce parsing such a graph to parsing bi-lexical structures like semantic dependency parsing (sdp) @xcite by reducing reentrancies and empty nodes into new labels. @xcite is a state-of-the-art approach for the semantic dependency parsing tasks that use second-order inference methods with mean-field variational inference. we adopt their approach for decoding and encode the sentences with strong pretrained token representations: xlmr @xcite , flair @xcite and fast-text @xcite . among the datasets, the tamil language only contains 400 labeled sentences for training, which makes the performance of the model for tamil low. to further improve the performance for the low resource language, we propose a new approach that we train the tamil model with a mixture of datasets with tamil and a rich resource language. empirical results show that such an approach can improve 2.44 elas on the test set of tamil. due to our misconceptions on the submission format, we submitted invalid unconnected graphs to the submission site. thanks to the help of the organizers, they fixed these graphs with simple scripts, and our system is ranked 6th over 10 teams in the official results. however, the submitted graphs can be easily connected if we apply tree algorithms in the decoding. in the postevaluation, we submitted our system outputs again and found that our system is 0.56 elas higher than the team ranked 1st in the official results. ## conclusion our system is a parser with strong contextual embeddings and second-order inference. for the lowresource language, we propose to train the model with a mixture of datasets. empirical results show that the second-order inference is stronger than the first-order one, and mixing data improves the performance of parser significantly for the lowresource language. after we fix the graph connectivity issue, our system outperforms the system ranked 1st by 0.56 elas in the official results. we also show that the non-connected graphs are practically useful for its higher performance and faster speed. our code is available at https://github .