VMWE identification with models trained on GUD (a UD v.2 treebank of Standard M odern G reek) (2025)

Abstract: UD_Greek-GUD (GUD) is the most recent Universal Dependencies (UD) treebank for Standard Modern Greek (SMG) and the first SMG UD treebank to annotate Verbal Multiword Expressions (VMWEs). GUD contains material from fiction texts and various sites that use colloquial SMG. We describe the special annotation decisions we implemented with GUD, the pipeline we developed to facilitate the active annotation of new material, and we report on the method we designed to evaluate the performance of models trained on GUD as regards VMWE identification tasks.

## introduction multiword expressions (mwes) pose significant challenges in both linguistic annotation and computational processing due to their semantic and structural idiosyncratic properties. previous research on mwes in modern greek has explored their theoretical properties (2024) and led to the development of lexical resources documenting their semantic and syntactic behavior @xcite . lexicographic and annotation studies have examined various semantic, pragmatic, and methodological aspects of mwe @xcite . computational approaches have also contributed to mwe processing, focusing on mwe extraction @xcite and multilingual parsing @xcite , as well as the evaluation of mwe discovery methods @xcite . however, despite these advancements, systematic treatments of mwes within syntactic parsing adapted to modern greek remain relatively rare, with existing studies employing symbolic frameworks such as lexical functional grammar (lfg; @xcite . to address this gap, we introduce ud_greek-gud (gud)-a new universal dependencies (ud v2) treebank @xcite for smg. gud integrates rich morphological and syntactic annotations with explicit verbal mwes (vmwes) annotation, in the spirit of the parseme guidelines @xcite . we outline specific linguistic decisions regarding tokenization, contractions, functional words, and diminutives/augmentatives, and propose a novel annotation strategy that integrates vmwes information directly into the syntactic layer of the conll-u format. building on this, we experiment with an annotation method that eventually encodes vmwes as dependency sub-relations, facilitating automatic identification through syntactic parsing. the approach is shown to be promising for computational processing; open issues are the identification of nested, or overlapping expressions and of discontinuous mwes @xcite . this paper explores these challenges, evaluates their implications, and outlines ongoing efforts toward improved evaluation and their practical integration into syntactic parsing frameworks, offering new perspectives for linguistic annotation and computational processing in greek and beyond. ## materials and annotation method ud_greek-gdt (henceforth gdt; prokopidis and papageorgiou ( 2017 )) is the first ud treebank for smg. both gud and gdt have been manually annotated for morphology and syntax, with gud additionally annotated for vmwes. to develop gud, a total of 1,807 sentences (25,493 tokens) were randomly selected from fiction texts in smg. additionally, 723 sentences (13,111 tokens), annotated specifically for vmwes, were retrieved from idion @xcite , an open-source web database of smg vmwes, resulting in a combined corpus of 2,530 sentences. these vmwe usage examples have been collected over the past 15 years through google searches from social media, football sites, and other sources where colloquial smg is used. the arboratorgrew tool 1 was used to implement the annotation. the annotation of gud was carried out by graduate students in language technology @xcite @xcite under the supervision of two of the authors. it proceeded in three rounds during this period. in the first round, students edited morphological and syntactic annotations obtained from models trained on gdt, developed morphological guidelines from scratch, and revised and enriched the syntactic guidelines originally produced by the gdt annotators. in the second round, one of the authors reviewed all annotated material and unified the guidelines. in the third round, the authors re-edited gud and refined the material exemplifying vmwes based on the established guidelines. annotation decisions were reached through discussions and consensus among the annotators. ## verbal mwe annotation gud contains material from fiction texts and additional 723 sentences (28% of the total gud sentences) featuring 100 vmwes, primarily of the verbal idiom type, along with some light verb constructions and verb-particle combinations. this classification of vmwes follows the parseme typology @xcite . the sentences exemplify flexible usages of vmwes in terms of morphology, word order permutations of the lexicalized parts of the vmwes, insertion, and lexical variant pairs. our vmwe annotation strategy is in the spirit of the guidelines proposed by @xcite and incorporates suggestions from d. zeman. in our setup, vmwe annotations are initially integrated into the misc column (10th column) of the standard conll-u format. although the parseme project utilizes an additional 11th column in alignment with conll-u plus specifications, foot_1 widelyused annotation tools such as arboratorgrew and parsing frameworks like stanza currently support only the standard ten-column conll-u format. notably, the deps (9th) and misc (10th) columns are generally excluded from dependency parsing training procedures @xcite . to effectively bridge this gap, we created a preprocessing script, move_mwes.py, which transfers vmwe annotations from the misc column to the deprel (8th) column by appending them as subrelation labels to the existing syntactic dependency labels. this transformation allows models such as stanza to directly predict vmwe subrelations as part of their syntactic parsing task. the vmwe annotation pipeline is clearly illustrated in examples 1-3 (see appendix): additionally, the move_mwes.py script supports reversing the annotation transformation, enabling the removal of vmwe subrelation labels from the deprel column and their restoration back into the misc column. for systematic evaluation of vmwe predictions, we employ the evalu-ate_mwes.py script, which computes performance metrics detailed in section 5. in this evaluation procedure, the annotations in the misc column (10th column) serve as the gold standard reference against which the model predictions (encoded in the 8th column) are compared. our integrated pipeline-[move_mwes.@xmath0.py (reversal) → eval-uate_mwes.py]-facilitates the efficient integration of vmwe predictions into active annotation workflows, thereby promoting continuous improvement in annotation accuracy and model performance. 4 ## experiments and evaluation we trained stanza models in four experimental settings: three models combined the full gud corpus (1,807 sentences) with additional subsets of 723, 500, and 300 sentences from idion, each @xmath1; the fourth model was trained exclusively on the 723 idion sentences, each @xmath2, without including the original gud corpus. (embeddings: gud+greekbert). we used only one test set, consisting of 200 sentences, each @xmath3, with a total of 242 vmwe occurrences. importantly, while many test vmwes were not identical to those in the training set, a large portion were lexical variants of seen vmwes. to ensure a diverse test set, sentences were selected to include different morphological forms of the head verb, as well as variations in word order and lexicalized component distance. since vmwes are encoded as subrelations within the syntactic structure, their correct identification depends on the model's ability to accurately recover syntactic dependencies (as reflected in the uas and las measures). we provide a targeted evaluation of vmwe identification. in conll-u, a sentence is represented as a table with 10 columns and a set of rows numbered from 1 to m, @xmath4. the representation of a vmwe with l, @xmath5: vmwe c8 we measure recall (@xmath6: 1. per-token. taking advantage of the tabular format of conll-u, we use the following definitions (see also @xcite : ## future plans we intend to expand our experiments by using larger test sets and corpora that encompass a wider variety of mwe types. another direction for future research and experimentation is exploring the dissociation of mwe subrelations from syntactic annotation, potentially by encoding them in the (currently empty) xpos column. additionally, we aim to develop more informative evaluation metrics to better assess system performance. the gud treebank remains a valuable linguistic resource for facilitating knowledge transfer across greek dialects, contributing to an ongoing contrastive study of low-resource language varieties. furthermore, the integration of mwe into the treebank could prove beneficial for various downstream applications that rely heavily on idiomatic expressions, such as offensive language detection. ## limitations a key limitation of our approach is that the indexes encoded in the misc column are not interpretable by the model, as they indicate vmwe units rather than pos tags, morphological features, or dependency relations. this results in at least two major consequences: moreover, per-token evaluation is not entirely informative, as it does not indicate how many lexicalized elements of a vmwe unit are correctly recognized. we are currently exploring evaluation methods that better capture these nuances. the test set included both seen and partially unseen vmwes. the unseen instances shared only their fixed components with the seen ones but contained different verbal elements. in other words, the test set included lexical variants of the seen vmwes. ideally, our evaluation methods should differentiate between identification and discovery performance; however, this distinction is not currently made. we plan to address this issue in future work.