A Joint Multitask Model for Morpho-Syntactic Parsing (2025)

Abstract: We present a joint multitask model for the Uni-Dive 2025 Morpho-Syntactic Parsing shared task, where systems predict both morphological and syntactic analyses following novel UD annotation scheme. Our system uses a shared XLM-RoBERTa encoder with three specialized decoders for content word identification, dependency parsing, and morphosyntactic feature prediction. Our model achieves the best overall performance on the shared task’s leaderboard covering nine typologically diverse languages, with an average MSLAS score of 78.7%, LAS of 80.1%, and Feats F1 of 90.3%. Our ablation studies show that matching the task’s gold tokenization and content word identification are crucial to model performance. Error analysis reveals that our model struggles with core grammatical cases (particularly Nom–Acc) and nominal features across languages.

## introduction the unidive 2025 morpho-syntactic parsing shared task @xcite introduces a novel framework for dependency parsing that seeks to bridge the traditional divide between morphological and syntactic analysis. in conventional universal dependencies @xcite , morphology and syntax are treated as distinct modules operating at different linguistic levels, with word boundaries serving as the interface between them. however, this separation has led to significant inconsistencies in how different languages and even different treebanks for the same language handle word segmentation and grammatical analysis. the shared task proposes to address these long-standing challenges by reorganizing grammatical representation around the content-function distinction rather than relying on theoretically problematic word boundaries, proposing a more typologically consistent approach to multi-linguistic parsing. for instance, in the sentence 'from the ap comes this story' shown in the task requires systems to predict both labeled dependency arcs and morphosyntactic features, but with a difference from standard universal dependencies parsing: the dependency tree consists only of content words (lexical words carrying semantic meaning like nouns, verbs, and adjectives), while function words (grammatical elements like adpositions, articles, and auxiliaries) contribute their grammatical information as features on related content words. while the content-function distinction is explicit in the training data, systems must identify this distinction themselves at test time from raw text. this identification determines which words participate in the dependency tree and which contribute features to other words. additionally, the multi-label nature of features, where a content word can have multiple feature values for a given feature class, like @xmath1;atr, 2 requires models to learn intricate morphosyntactic patterns. we present a joint multitask model (figure 1 ) that explicitly addresses these challenges through three specialized decoders sharing a common xlm-roberta encoder @xcite , initialized from pre-trained multilingual representations. we design content word identification as an explicit task to be learned by the model rather than relying on intuition-driven heuristics. we participate in the multilingual track, training separate models for each of the nine languages, allowing us to tune hyperparameters specifically for each language's characteristics while still benefiting from multilingual pretrained representations. on the shared task's results, our system achieves the best overall performance with average scores of 78.7% mslas, 80.1% las, and 90.3% feats f1 across all languages. additionally, our model ranks first on each individual language, demonstrating the effectiveness of multitask learning for this task. our error analysis yields three main observations: (1) errors in tokenization and content word identification cascade through the pipeline, with gold annotations improving mslas by up to 12 points; (2) the majority of residual errors lie in nominal morphology-gender, number, and case-with 2 @xmath2, "inside an enclosed area"; @xmath3, attribute". both definitions come from the official case inventory supplied by the shared-task organisers. ## experimental setup the shared task provided training and development sets for multiple languages. to simulate a realistic evaluation scenario, we split the official training data into 90% for training and 10% for development, using the official development set as our local test set. this allowed us to tune hyperparameters and select models before the official test release. the languages included in our experiments were english, turkish, hebrew, czech, polish, portuguese, italian, serbian, and swedish, with training sizes ranging from approximately 3,000 to 10,000 sentences depending on the language. we develop a custom data loader to handle the modified conll-u format used in the shared task. the loader automatically extracts content words by examining the feats column, where '_' indicates function words and any other value indicates content words. as mentioned before, we filter out abstract nodes during loading. all models are trained using adamw optimizer @xcite with an initial learning rate of 2 × 10 -5 and batch size of 16 for 25 epochs. we employ early stopping with patience of 1 epoch and learning rate reduction by factor 0.5 when validation loss plateaus. training is performed on a nvidia a100 gpu with 32gb ram on a high-performance computing cluster, with each model taking approximately 1-5 hours to converge. we perform grid search over task-specific loss weights on our development split. the optimal weights varied by language-for example, turkish benefited from weighting parsing and morphological feature losses twice as much as content word identification (2.0:2.0:1.5), while english performed better with parsing weighted most heavily, followed by morphological features and content word identification (2.0:1.5:1.0). for each language, we train three models with different random seeds using the same hyperparameter configuration to verify training stability and robustness. all three models are evaluated on our local test set (the official development set) using the shared task's official evaluation script. once hyperparameters are selected, we retrain a single model for each language using the complete official training and development data combined. these final models use the same hyperparameters determined during development. these models are used to generate predictions on the official covered test set, which contains only raw text without annotations. evaluation is performed using the official script which computes three metrics: mslas (morphosyntactic features f1 only on correctly parsed tokens), las (labeled attachment score), and feats f1 (morphosyntactic features f1). ## results this section is divided into two parts: first, we present official test results from models trained on all available data (official train + dev combined) and evaluated on the covered test set; second, we report development results using our local data splits (90% train, 10% dev, official dev as test) to analyze design choices and hyperparameter impact. ## error analysis we performed error analysis on the models trained with our local data splits (90% train, 10% dev, official dev as test). we analyzed only the first seed model for each language, as the low standard deviations indicate minimal variation across seeds. the analysis uses scripts that replicate the official evaluation logic to ensure our error categorization matches the scoring methodology. ## conclusions we present a joint multitask architecture for unified morphosyntactic parsing that achieves first place in the unidive 2025 shared task. our key contribution is explicitly modeling content word identification as a classification task, creating a robust cascade where the identification determines parsing and feature assignment. our analysis reveals systematic error patterns pointing to specific improvement opportunities. case confusions concentrate on core grammatical distinctions (nom-acc), while dependency errors reflect the expected challenges at the nmod-obl boundary. while these patterns are linguistically understandable, they suggest potential room for improvement through weighted training or specialized handling of frequently confused categories, though such optimizations may yield only incremental gains. a more substantial enhancement to the annotation scheme could be making explicit which function words contribute features to which content words. currently, function words are marked with '_' and their grammatical information is incorporated into "related" content words, but these relationships remain implicit. an indexing system could explicitly link each function word to its target content word. this would not only reduce ambiguity in feature assignment but also make the annotation more transparent for researchers unfamiliar with specific languages, as they could trace exactly how morphosyntactic information flows from function words to content words in the unified representation. finally, the 30-point performance gap between portuguese and turkish highlights fundamental challenges in handling typologically diverse languages within a unified framework. while the parser excels at the predominantly projective structures, turkish's agglutinative morphology and flexible word order might be introducing some difficulties. the dispersed error patterns observed for turkish-contrasting with the concentrated confusions in other languages-suggest that the current architecture may not be optimal for highly nonprojective languages. future work could explore specialized parsing algorithms designed for nonprojective structures or alternative architectures that better handle long-distance dependencies and flexible word order. despite these challenges, our results across nine languages demonstrate the viability of joint morphosyntactic modeling for the task. by the university of melbourne's research computing services and the petascale campus initiative. demian is funded by the graduate research scholarship from the faculty of engineering and information technology, university of melbourne. cost (european cooperation in science and technology) kindly provided funding for travel to the event.