COMBO : A New Module for EUD Parsing (2021)

Abstract: We introduce the COMBO-based approach for EUD parsing and its implementation, which took part in the IWPT 2021 EUD shared task. The goal of this task is to parse raw texts in 17 languages into Enhanced Universal Dependencies (EUD). The proposed approach uses COMBO to predict UD trees and EUD graphs. These structures are then merged into the final EUD graphs. Some EUD edge labels are extended with case information using a single language-independent expansion rule. In the official evaluation, the solution ranked fourth, achieving an average ELAS of 83.79%. The source code is available at https://gitlab.clarin-pl.eu/syntactic-tools/combo .

## introduction data-driven dependency parsers achieve high parsing performance for languages representing different language families. the state-of-the-art dependency parsers are trained with supervised learning methods on large correctly annotated treebanks, e.g. from universal dependencies (ud, @xcite . ud is an international initiative aimed at developing a cross-linguistically consistent annotation schema and at building a large multilingual collection of dependency treebanks annotated according to this schema. a relatively small subset of ud treebanks is annotated with higher-order syntactic-semantic representations that encode various linguistic phenomena and are called enhanced universal dependencies @xcite . dependency treebanks, especially the uniformly annotated ud treebanks, are used for multilingual system development, e.g. within multiple shared tasks on dependency parsing @xcite @xcite @xcite . in particular, the iwpt 2020 shared task on parsing into enhanced universal dependencies @xcite is worth mentioning, because it is the predecessor of the current iwpt 2021 shared task @xcite . all shared tasks contributed to rapid advancement of language parsing technology, inter alia, the formulation of groundbreaking parsing algorithms and their publicly available implementations (e.g. @xcite @xcite @xcite . dependency parsing is an important issue in various sophisticated downstream tasks, including but not limited to sentiment analysis @xcite , relation extraction @xcite @xcite , semantic role labelling @xcite , or question answering @xcite . on the other hand, even if eud parsing aims at predicting semantically informed structures, which seem to be appropriate in advanced nlp tasks, it is not yet used in solving these tasks. an obstacle can be the availability of the state-of-the-art eud parsers, e.g. two top systems at the iwpt 2020 eud shared task (i.e. @xcite are not publicly available and therefore difficult to integrate into nlu systems without having to implement them from scratch. meeting the potential expectations of nlu system architects, the source code of combo with the new eud parsing module and the pre-trained models developed as part of our solution submitted to this shared task are publicly available. the proposed solution to eud parsing is based on (1) stanza tokeniser @xcite , (2) combo @xcite , a data-driven language-independent system for morphosyntactic prediction, i.e. part-of-speech tagging, morphological analysis, lemmatisation, dependency parsing, and eud parsing (see section 3.3), (3) an algorithm that merges predicted labelled dependency arcs and predicted eud arcs, and builds the final eud graphs (see section 3.4), and (4) two linguistically motivated languageindependent rules that improve the final eud graphs (see section 3.5). the first expansion rule adds case information sublabels to eud modifiers, and the second one amends enhanced arcs coming into the function words. these two rules are integrated into the proposed eud parsing system. in the official evaluation, our eud parser ranked 4th, obtaining an average elas of 83.79% and eulas of 85.20%. 1 it is worth emphasising that combo predicts labelled dependency trees with an average las of 88.91%, only being slightly outperformed by the robertnlp system. ## shared task description the iwpt 2021 eud shared task consists in evaluating systems for parsing raw texts into enhanced universal dependencies. the systems are trained and evaluated on data supplied by the organisers. ## system overview the eud parsing system is built of the following components: a data encoder boosted with a contextual language model (see section 3.1), morphosyntactic predictors (see section 3.2), an eud predictor (see section 3.3), an algorithm merging predicted labelled dependency arcs and enhanced dependency arcs (see section 3.4), and a postprocessing module (see section 3.5). ## results the shared task submissions are evaluated with two evaluation metrics: elas -las 6 on enhanced de-pendencies, and eulas -las on enhanced dependencies where labels are restricted to the ud relation types, i.e. sublabels are ignored. combo ranks 4th, achieving 84.71% elas in the qualitative evaluation (an average over treebanks), and 83.79% elas in the coarse evaluation (an average over languages). in terms of eulas, it ranks 4th achieving 86.30% in the qualitative evaluation, and 5th achieving 85.20% in the coarse evaluation. in addition to elas and eulas metrics, the systems are also compared in terms of quality of predicting labelled dependency trees measured with las (the secondary evaluation measure). in the las ranking, combo takes second place achieving 88.91% in the qualitative evaluation, and 87.84% in the coarse evaluation, being slightly overcome by the robertnlp system (89.25% in the qualitative evaluation, and 89.18% in the coarse evaluation). post-processing impact we measure the impact of the post-processing step (i.e. extending graph labels with case information and correcting edges coming into the function words) on the development data per language (see if a language has multiple treebanks. the second rule modifies the graph structure. however, as the eulas scores are almost negligible, using this rule seems questionable. the first rule, in turn, does not modify the structure of eud graphs, but only their edge labels, and its impact on improving elas scores is significant. ## conclusion we presented the combo-based solution to eud parsing which took part in the iwpt 2021 eud shared task. the proposed approach is hybrid, i.e. based on machine learning and rule-based algorithms. first, ud trees and eud graphs (and also morphosyntactic features of tokens, i.e. parts of speech, morphological features, and lemmata) are automatically predicted with the data-driven combo system. then, the predicted structures are combined into the eud graphs using the developed rule-based merge algorithm. finally, the labels of modifiers and conjuncts in the merged eud graphs are extended with case information using an expansion rule. the proposed solution is simple and language-independent. we recognise that we could still improve the results, e.g. by defining languagespecific correction rules. however, our objective was to build an easy-to-use system for predicting eud graphs that is publicly available and can be efficiently use to solve sophisticated nlu tasks.