Fine-Tuning Cross-Lingual LLM s for POS Tagging in Code-Switched Contexts (2025)

Abstract: Code-switching (CS) involves speakers switching between two (or potentially more) languages during conversation and is a common phenomenon in bilingual communities. The majority of NLP research has been devoted to mono-lingual language modelling. Consequentially, most models perform poorly on code-switched data. This paper investigates the effectiveness of Cross-Lingual Large Language Models on the task of POS (Part-of-Speech) tagging in code-switched contexts, once they have undergone a fine-tuning process. The models are trained on code-switched combinations of Indian languages and English. This paper also seeks to investigate whether fine-tuned models are able to generalise and POS tag code-switched combinations that were not a part of the fine-tuning dataset. Additionally, this paper presents a new metric, the S-index (Switching-Index), for measuring the level of code-switching within an utterance.

## introduction code-switching in digital and face-to-face communication can arise for a multitude of reasons including quoting someone, excluding a particular person or group from a conversation and emphasising group identity @xcite . ## results the fine-tuned models were tested on a portion of the dataset. the results are shown in ## conclusion and future work although the performance of the models trained here is not comparable to those of today's state-ofthe-art pos taggers, we feel that our models are highly promising. the ability of models to pos-tag in unseen code-switched combinations is evident and more research needs to be performed to analyse whether this property extends to other code-switched language combinations that are not so closely related. additionally, the ability of multilingual models to be fine-tuned to perform other nlp tasks such as sentiment analysis and named entity recognition is also an area that needs to be researched.