Second language K orean U niversal D ependency treebank v1.2: Focus on Data Augmentation and Annotation Scheme Refinement (2025)

Abstract: We expand the second language (L2) Korean Universal Dependencies (UD) treebank with 5,454 manually annotated sentences. The annotation guidelines are also revised to better align with the UD framework. Using this enhanced treebank, we fine-tune three Korean language models—Stanza, spaCy, and Trankit—and evaluate their performance on in-domain and out-of-domain L2-Korean datasets. The results show that fine-tuning significantly improves their performance across various metrics, thus highlighting the importance of using well-tailored L2 datasets for fine-tuning first-language-based, general-purpose language models for the morphosyntactic analysis of L2 data.

## introduction the universal dependencies (ud) framework, designed to facilitate accessible morphosyntactic annotations @xcite , has been applied increasingly in linguistics, particularly to annotate learner corpora. this approach supports tasks such as modeling the trajectories of second language (l2) acquisition, which often require treebanks for fine-tuning language models or evaluating their performance on l2 data. such data are typically characterized by simpler and/or nontargetlike lexico-grammatical usages compared to those produced by first-language speakers, although these characteristics vary across l2 proficiency. previous research has increasingly adopted the ud framework to automatically handle learner corpora in various languages, including english @xcite @xcite , chinese @xcite , italian @xcite , russian @xcite , and swedish @xcite @xcite , demonstrating its utility in l2 studies. among these efforts, recent studies in korean have developed l2-korean ud treebanks with language-specific morphemes and dependency tags @xcite . however, two research gaps remain. first, while continuing to expand the amount of data, annotation guidelines should be iteratively updated to balance crosslinguistic standardization with the preservation of language-specific features @xcite . second, the effectiveness of l2-korean-optimized models should be assessed using out-of-domain data to improve their reliability in broader contexts for which they are designed @xcite . the present study addresses these gaps with three key contributions: (1) augmenting the existing l2-korean ud treebank (v1.1, 7,530 sentences) by adding 5,454 manually annotated sentences with korean-specific morphemes and ud annotations; ## related works a line of studies have established approaches for morpheme and dependency annotations in l2 korean. sung and shin (2023b) provided preliminary guidelines for korean morpheme annotations, addressing the need to parse morphemes taking into account the agglutinative nature of korean morphosyntax, where a single word often combines lexical morphemes (e.g., noun, verb) and func-tional morphemes (e.g., postpositions, tense-aspectmodality markers). expanding this work, @xcite introduced detailed ud annotation guidelines to handle korean-specific dependency cases such as particles and coordination. sung and shin (2023a) fine-tuned morpheme parsers optimized for l2 korean and evaluated them on in-domain and out-of-domain datasets, demonstrating the importance of high-quality input for fine-tuning l2-korean language models. however, those studies did not include training or evaluating dependency tags. additionally, their fine-tuning strategy was relatively simple, relying solely on one korean pre-trained model. ## dataset building upon the previous l2-korean ud annotation projects @xcite , we continued annotating l2-korean sentences using a subset of data from the same source @xcite . 1 for the out-of-domain testing, we annotated additional data from the kolla dataset @xcite , which was designed to analyze korean learner language with a focus on particle error annotations. 2 along with the annotations, we refined the annotation guidelines, implementing major revisions to better align with the language-general ud annotation scheme and minor adjustments to morpheme annotations. together, the updated l2-korean ud treebank (v1.2) comprises (# @xmath0,984): (1) additional data augmented and annotated using the revised scheme (# @xmath1,532); (2) revised data from the previous project @xcite , updated with the new annotation scheme (# @xmath2,530); (3) data sourced from the kolla dataset @xcite , annotated with the revised scheme for the out-of-domain testing (# @xmath3. ## discussion and future directions we expanded the l2-korean ud treebank with refined annotation schemes to improve model performance after fine-tuning. using this treebank, we fine-tuned three models-stanza, spacy, and trankit-and evaluated their performance in both in-domain and out-of-domain contexts. the evaluation results showed significant performance improvements across various metrics, underscoring the value of using an l2 dataset for fine-tuning. among the models, trankit's transformer-based architecture outperformed the others in xpos, uas, and las across both test datasets, demonstrating its effectiveness of capturing morphosyntactic features in l2-korean data. the fine-tuned models and relevant documentations are available at https: //github.com/nlpxl2korean/ud-ksl . the treebank will be updated at https://github.com/ universaldependencies/ud_korean-ksl. although both trankit and stanza employ a character-based seq2seq model @xcite , stanza's superior lemmatization performance compared to trankit can be attributed to two primary factors. first, stanza includes a dictionarybased lemmatizer @xcite , which may have strengthened its ability to handle a wide variety of morphological patterns. second, as noted earlier, stanza uniquely leverages a model that was pretrained on l1 data (ud-korean gsd) before being fine-tuned on the current l2 data, which appears to enable it to capitalize on prior lemmatization knowledge for more accurate predictions. to fully harness the potential of transformerbased architectures in fine-tuning l2-korean mod-els, future l2-korean ud treebanks could adopt two complementary strategies. one approach involves combining l2-korean data drawn from various genres or diverse learner backgrounds. the other centers on refining the match between universal upos tags and language-specific xpos tags through expert revisions to enhance upos to boost their effectiveness for lemmatization within the seq2seq framework.