Parsing the Switch: LLM -Based UD Annotation for Complex Code-Switched and Low-Resource Languages (2025)

Abstract: Code-switching presents a complex challenge for syntactic analysis, especially in low-resource language settings where annotated data is scarce. While recent work has explored the use of large language models (LLMs) for sequence-level tagging, few approaches systematically investigate how well these models capture syntactic structure in code-switched contexts. Moreover, existing parsers trained on monolingual treebanks often fail to generalize to multilingual and mixed-language input. To address this gap, we introduce the BiLingua Pipeline, an LLM-based annotation pipeline designed to produce Universal Dependencies (UD) annotations for code-switched text. First, we develop a prompt-based framework for Spanish-English and Spanish-Guaraní data, combining few-shot LLM prompting with expert review. Second, we release two annotated datasets, including the first Spanish-Guaraní UD-parsed corpus. Third, we conduct a detailed syntactic analysis of switch points across language pairs and communicative contexts. Experimental results show that BiLingua Pipeline achieves up to 95.29% LAS after expert revision, significantly outperforming prior baselines and multilingual parsers. These results show that LLMs, when carefully guided, can serve as practical tools for bootstrapping syntactic resources in under-resourced, code-switched environments.

## introduction code-switching (csw) is a widespread linguistic phenomenon observed in multilingual communities around the world. despite its prevalence in spoken and informal digital communication, it remains a complex challenge for natural language processing (nlp), particularly for syntactic parsing. one of the central issues is that most state-of-the-art parsing models are trained on monolingual treebanks and thus lack robustness when applied to mixed-language data @xcite . previous works of @xcite ; @xcite ; @xcite took an important step toward addressing this gap by proposing, for instance, a semi-supervised dependency parsing framework that augments training with auxiliary sequence labeling tasks @xcite . their model improved parsing accuracy on turkish-german spoken corpus by learning better representations of syntactic structure in a multilingual setting. however, even with such enhancements, existing models often rely on large amounts of annotated data, which is particularly limiting for under-resourced language pairs. motivated by this lack of resources, we introduce bilingua pipeline, a bilingual syntactic parser pipeline based on large language models (llms), specifically the gpt-4.1 model, to generate syntactically annotated csw datasets. figure 1 illustrates figure 2 : overview of bilingua pipeline for spanish-english code-switching. left: conversation transcripts from the miami-bilingual corpus are processed through token-level segmentation, language id tagging, and filtering of intra-sentential code-switches. center: the pipeline assigns ud tags to csw sentences, handling contractions, repetitions, ellipsis, and clausal structure. right: the resulting annotated dataset is reviewed by linguistic experts and enables downstream tasks such as pos/deprel analysis and extension to low-resource settings, including spanish-guaraní. in addition to developing bilingua pipeline, we also examine the limitations of current syntactic parsing evaluation metrics. to this end, we introduce additional methods for assessing the performance of our parser with the help of linguistic experts. moreover, the annotated datasets we create also support a wide range of linguistic analyses involving bilingual speakers, including understanding of fine-grained switch-point behavior. while most previous structural studies in nlp on csw have focused on part-of-speech (pos) tags @xcite @xcite , our analysis using dependency parsing shows that syntactic subjects (nsubj) are among the most frequent switch points in both language pairs and that spanish-guarani csw exhibits higher variation in switch points. this finding underscores the value of dependency parsing for analyzing switch points across languages. the main contributions of our work are as follows: overall, we believe our findings and released resources will support future work in both compu- study language pair pos accuracy las (parsing) solorio et al. (2014) spanish-english 85.1% -solorio et al. (2014) hindi-english 83.3% -rijhwani et al. (2017) spanish-english 80.0% -özateş et al. (2022) hindi-english -71.93% özateş et al. (2022) turkish-german -66.30% bhat et al. (2018) hindi-english -71.03% ## related work parsing csw text is considerably more challenging than monolingual parsing due to structural variability, mixed grammar rules, and limited annotated corpora. prior work @xcite @xcite has demonstrated that models trained exclusively on monolingual data perform poorly on csw without specific adaptation. özateş et al. (2022) addressed this gap by proposing a semisupervised parsing framework that incorporates auxiliary sequence labeling tasks. their approach improved parsing performance on turkish-german csw with labeled attachment scores (las (buchholz and marsi, 2006)) reaching up to 73%. similarly, bhat et al. (2018) and rijhwani et al. (2017) reported las scores of about 70-72% for hindi-english data using adapted models ( most prior research focuses on part-of-speech tagging rather than full syntactic parsing, and available resources remain limited to a few language pairs or l2 languages @xcite . @xcite introduced lince, a centralized benchmark for linguistic code-switching covering tasks such as pos-tagging for various language pairs, including spanish-english. a draft ud treebank exists for spanish-english code-switching, but it is not publicly released, further illustrating the scarcity of syntactically annotated csw data. efforts to increase parsing speed, such as recasting dependency parsing as a sequence labeling task @xcite , have improved runtime, but still depend heavily on monolingual training data. our work builds on these foundations but shifts toward llm-based annotation. llms like openai gpt can be prompted with examples and linguistic rules to produce annotations without requiring extensive supervised training. we apply this technique to generate and evaluate new csw datasets, including one for spanish-guaraní, thus expanding the reach of syntactic tools to underserved language communities. ## conclusion this work introduces bilingua pipeline, a novel pipeline for syntactic annotation of code-switched data using llms, supported by expert human validation. by leveraging gpt-4.1 and linguistically informed prompting, we produced high-quality ud annotations for spanish-english and spanish-guaraní code-switching. our results show that llm-based annotations outperform conventional parsers in syntactic accuracy, particularly at switch points where monolingual models typically fail. this performance gap is especially pronounced under our second evaluation method, which compares llm outputs against human-revised annotations and does not penalize linguistically plausible variation. by incorporating groups of semantically similar dependency labels, this evaluation provides a more realistic benchmark for parsing in multilingual settings. importantly, we release the first publicly available ud-annotated datasets for spanish-english and spanish-guaraní csw, addressing a critical gap in multilingual nlp resources. these datasets and our annotation methodology not only enable fine-grained analysis of code-switching behavior but also provide a foundation for advancing low-resource dependency parsing.