Findings of the U ni D ive 2025 shared task on multilingual Morpho-Syntactic Parsing (2025)

Abstract: This paper details the findings of the 2025 UniDive shared task on multilingual morphosyntactic parsing. It introduces a new representation in which morphology and syntax are modelled jointly to form dependency trees of contentful elements, each characterized by features determined by grammatical words and morphemes. This schema allows bypassing the theoretical debate over the definition of “words” and it encourages development of parsers for typologically diverse languages. The data for the task, spanning 9 languages, was annotated based on existing Universal Dependencies (UD) treebanks that were adapted to the new format. We accompany the data with a new metric, MSLAS, that combines syntactic LAS with F1 over grammatical features. The task received two submissions, which together with three baselines give a detailed view on the ability of multi-task encoder models to cope with the task at hand. The best performing system, UM, achieved 78.7 MSLAS macro-averaged over all languages, improving by 31.4 points over the few-shot prompting baseline.

## introduction syntactic and morphological tasks, such as parsing @xcite and linearization @xcite , analysis @xcite and inflection @xcite , have a long history in nlp research. collectively, these tasks aim to provide structured representations of free text to facilitate further research and applications. however, the distinction between morphology and syntax, and hence the definitions of these tasks, rely on the definition of a "word" (dixon and aikhenvald, 2002) -a unit that is notoriously ill-defined from a cross-lingual perspective @xcite . this shared task draws on previous works which attempt to avoid relying on words, in either syntax @xcite or morphology @xcite , and presents a new representation that models morphosyntax in a unified and harmonized fashion. the representation used here closely resembles dependency trees from universal dependencies (ud; de @xcite , but instead of the distinction between words and morphemes, it adopts the distinction between content and function. in this structure, every sentence is represented by a dependency tree whose nodes are content-bearing elements. on the other hand, function elements, words and morphemes, are represented as grammatical features on the nodes, in a manner similar to morphological features in ud. the shared task includes data in 9 languages: czech, english, hebrew, italian, polish, brazilian portuguese, serbian, swedish, and turkish, with several thousands of sentences as training data for each. the data was converted from ud dependency trees in a semi-automatic fashion. submitted systems were tested on a held-out test set for which participants got only the raw text as input. the systems' predictions were evaluated using three metrics. las, as defined by @xcite , was used to measure the systems' success in connecting the nodes to their correct parent with the correct relation. f1 over morpho-syntactic features measured the systems' ability to correctly characterize the functions relating to each node. but as the main metric, we defined mslas, which combines both metrics to measure overall success in modelling both node relations and node content. this report includes the results of five systems, three baselines and two submitted systems. both submitted systems and one of the baselines utilize multi-task training, where several classifiers are trained on top of a frozen encoder-only model such as bert @xcite or xlm-r @xcite . the systems differ mostly in the way they incorporate the prediction of contentfulness into the system, and whether they were trained jointly or separately across languages. in addition, two baselines prompted gemini 2.5 @xcite in a few-shot setting to output the entire prediction as a string. the best performing system, um (inostroza améstica et al., 2025), achieved 78.7 mslas over the covered test set macro-averaged over all languages. this score is a 31.4-point improvement over the best baseline. although impressive, the results show that there is still some headroom, for training better models to excel in the task. specifically, non-indo-european languages, like turkish and hebrew, seem to pose a greater challenge to the submitted systems that are based on dependency parsing models. all in all, this shared task introduces a more holistic representation that takes into account the typological variety in word definitions across languages, and shows that predicting this structure poses a challenge for parsing technology and to large language models alike. ## task description the concept of "word" is known to have a highly disputed definition, ever since (at least) the formation of modern linguistics as a field @xcite . despite that, words have had a crucial role in delineating syntax and morphology, both from a theoretical @xcite and practical @xcite standpoint. the result in ud is that words are defined inconsistently across languages, so the ability to compare structures across languages is severely hindered. on the other hand, the mere attempt to cross-lingually define words entails a tokenization process that is unnatural to some languages that differ greatly from the prototypical western language. morpho-syntactic parsing is designed to bypass the issue by modelling syntax and morphology in tandem, getting rid of the necessity to define words in order to separate the two. in this task, systems are required to predict a morpho-syntactic dependency tree for each sentence, where nodes are contentful elements. these nodes include lexemes from open-class parts of speech, such as nouns and verbs, and their arguments. grammatical, or functional, elements do not appear as nodes in the tree, even if they are written as independent words. instead, they contribute morpho-syntactic features that characterize the content nodes, akin to morphological features in ud. all predicates and arguments are included in the morpho-syntactic tree, including arguments in the form of pronouns, clitics and agreement morphemes. 1 on the other hand, elements that signify relations between content nodes are deemed function nodes, as well as elements that specify some grammatical additional meaning to content nodes. the result is a dataset in which sentences in different languages are structured much more similarly, owing their differences to structural linguistic variations between languages, and not to variations in orthography or grammatical tradition. systems solving morpho-syntactic parsing must then detect contentful elements in a similar fashion across all languages, without the benefit of a more natural tokenization process given to languages that are more similar to english and a few other western languages. figure 1 has an example tree. it shows several function words in gray that are not part of the morpho-syntactic tree, but contribute features to the content nodes, like the conjunction so and the preposition in that contribute a case feature, and the copula am that contributes verbal features to the nominal predicate office. ## languages the data for each language was prepared by individuals or teams who are speakers of that language (see ## baseline systems we provide three baseline systems, of which one is a fine-tuned mbert model and two are on the basis of prompting gemini 2.5. ## results we present overall evaluation results in considering the results per language in ## conclusion this paper presents the results of the unidive 2025 shared task on multilingual morpho-syntactic parsing, introducing a novel representation and evaluation metric designed to be more inclusive of typologically diverse languages. by shifting the focus from traditional distinction between "words" and "morphemes" to a distinction between content and function elements, the task encourages the development of new parsing technologies that operate in a more equitable fashion across all languages. the results from the five evaluated systems demonstrate that the task is challenging yet solvable for modern multi-task learning models. the top-performing system, um, achieved an impressive mslas score of 78.7, significantly outperforming all baselines and showcasing the potential of dedicated architectures for this problem. nevertheless, challenges remain, particularly in handling abstract nodes and parsing non-indo-european languages like turkish and hebrew, indicating clear directions for future research. overall, this shared task successfully established a new benchmark for morpho-syntactic analysis and paved the way for more linguistically comprehensive parsing models. future iterations of the shared task will have the opportunity to cover a more diverse set of languages to allow a better evaluation, as well as better applications for studies in computational typology and morpho-syntax.