Can Code-Switched Texts Activate a Knowledge Switch in LLM s? A Case Study on E nglish- K orean Code-Switching (2025)

Abstract: Recent large language models (LLMs) demonstrate multilingual abilities, yet they are English-centric due to dominance of English in training corpora. The limited resource for low-resource languages remains a crucial challenge. Code-switching (CS), a phenomenon where multilingual speakers alternate between languages in a discourse, can convey subtle cultural and linguistic nuances that can be otherwise lost in translation and elicits language-specific knowledge in human communications. In light of this, we investigate whether code-switching can activate, or identify and leverage knowledge for reasoning when LLMs solve low-resource language tasks. To facilitate the research, we first present EnKoQA, a synthetic English-Korean CS question-answering dataset. We provide comprehensive analysis on a variety of multilingual LLMs by subdividing activation process into knowledge identification and knowledge leveraging. Our results demonstrate that compared to English text, CS can faithfully activate knowledge inside LLMs especially on language-specific domains, suggesting the potential of code-switching on low-resource language tasks.

## introduction large language models (llms) have continuously evolved through time to exhibit advanced multilingual capabilities, enabled by training on massive datasets that include text in many different languages. however, these sources are typically skewed toward english, creating an inconsistent performance across different languages @xcite . the limited availability for real-world user queries in low-resource languages remains a crucial challenge for achieving robust multilingual models. prior works attempt to mitigate this issue through machine translation @xcite , but ## preliminaries & related work in this section, we provide preliminary knowledge about code-switching, and explore relevant studies from conventional and computational linguistics. ## enkoqa: english-korean code-switching qa testset to compare the effectiveness of code-switching with dominant language and translation in target language when performing language-specific tasks, we introduce enkoqa, a synthetic english-korean code-switching dataset that is designed based on mlf model. in this section, we first discuss the details of data construction ( § 3.1), and evaluate performances of llms on the dataset ( § 3.2, 3.3). ## can code-switched questions activate a "knowledge switch" in llms? from section 3.3, we observe that most llms are able to answer correctly to questions in cs than in other baselines. to further investigate on the ef-fectiveness of cs in activating language-specific knowledge, we formulate two tasks: knowledge identification and knowledge leveraging. we evaluate the tasks in cs and english questions, the two baselines that share the same matrix language. ## analysis on knowledge leveraging we present the visualized results of accuracy in both cs and english settings in figure 5 , with detailed scores reported in main observations. consistent with the results in section 3.3, all models demonstrate generally higher performances for cs questions compared to english questions. the results indicate that cs effectively activates knowledge across various domains while activating in dominant english language is suboptimal. gpt-4o, claude, and solar exhibit higher cs performance than english across all domains. these models not only identify faithful and helpful knowledge ( § 5.1), but also answer questions by accurately grounding on that knowledge; this shows that cs questions robustly activate essential knowledge in these models. on the contrary, llama3 and gemma2 families show poor performance in both cs and english settings in several domains, such as geography and law. taking into account that these domains require domain-specific expertise, it is likely that their lack of understanding contributes to low accuracy, let alone cs failing to activate korea-focused knowledge. knowledge identification and leveraging both matters. we demonstrate that qualified knowledge identification is prerequisite for knowledge activation of cs. the win ratio of history knowledge by gpt-3.5 was relatively poor compared to others (figure 4 ), leading gpt-3.5 to be the only model that did not benefit from cs. similarly in law, figure 2 and 3 show that helpfulness and pairwise scores for knowledge by gemma2 9b and 27b are lower than others, which are responsible for their suboptimal performance in cs. ## conclusion and future work we explore the efficacy of code-switching in activating language-specific knowledge embedded in llms. utilizing two korean-centric qa datasets, we synthesize enkoqa, a qualified english-korean code-switching qa dataset and conduct experiments on various multilingual llms. our analyses demonstrate that llms can simulate a similar code-switching effect with human communications of facilitating low-resource knowledge within llms, particularly in language-specific domains. regarding this finding, we suggest that codeswitching can be an effective strategy for solving low-resource language tasks. also, augmenting low-resource datasets into code-switching text can amplify resource and mitigate data scarcity challenge. future directions of this work involve a more comprehensive investigation into the potential of code-switching across different aspects of multilingual modeling, including its role in pretraining and instruction-tuning, as well extending our work to a wider range of languages. by first addressing the relatively understudied korean-english pair, our research contributes to a relatively unexplored area and serves as a stepping stone for subsequent studies on code-switching. in particular, our data synthesis approach, grounded in the universally applicable mlf theory, extends beyond korean and is generalizable to all languages. through our work, we encourage the nlp community to further explore the capacity of code-switching to effectively enhance and harness the potential of multilingual llms.