Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning (2025)

Abstract: While multilingual large language models generally perform adequately, and sometimes even rival English performance on high-resource languages (HRLs), they often significantly underperform on low-resource languages (LRLs). Among several prompting strategies aiming at bridging the gap, multilingual in-context learning (ICL) has been particularly effective when demonstration in target languages is unavailable. However, there lacks a systematic understanding when and why it works well.In this work, we systematically analyze multilingual ICL, using demonstrations in HRLs to enhance cross-lingual transfer. We show that demonstrations in mixed HRLs consistently outperform English-only ones across the board, particularly for tasks written in LRLs. Surprisingly, our ablation study show that the presence of irrelevant non-English sentences in the prompt yields measurable gains, suggesting the effectiveness of multilingual exposure itself. Our results highlight the potential of strategically leveraging multilingual resources to bridge the performance gap for underrepresented languages.

## introduction in-context learning (icl; @xcite has become a widely adopted technique in natural language processing with large language models (llms; @xcite @xcite , which enables llms to learn to solve problems by analogy from a few input-output examples (i.e., demonstrations) without updating model parameters. as a generic method, icl has also been effective in improving the cross-lingual performance of multilingual llms (mllms; winata michael had 58 golf balls. on tuesday, he lost 23 golf balls. on wednesday, he lost 2 more. how many golf balls did he have at the end of wednesday? michael started with 58 golf balls and lost 23, so he has 58 -23 = 35. after he lost 2 more, he has 35 -2 = 33 balls. olivia has $23. ⋯ how much does she have left? et @xcite @xcite @xcite . prior work has introduced two common icl strategies for non-english languages: (i) translating the target question into english and performing @xcite @xcite , and (ii) providing demonstrations in the target language (in-language demonstrations; @xcite @xcite . both strategies have critical shortcomings: (i) may suffer from information loss in translation due to nuanced language gap @xcite or the unavailability of high-quality translation systems for extremely low-resource languages (lrls), whereas (ii) may become infeasible due to data scarcity in lrls. as alternatives, when presenting llms with problems in the target language, english demonstrations (fig. 1a ) lead to poor performance on lrls, whereas demonstrations in multiple high-resource languages (hrls; fig. 1b ) can be more effective, even when there is little alphabetical overlap between the demonstration and target languages @xcite ; however, the underlying reasons on why it works remain unclear. in this work, we systematically analyze multilingual icl through a set of controlled experiments. each test question is paired with a set of semantically equivalent demonstrations, while the demonstration languages vary according to different icl modes ( § 3). we compare the performance differences across four icl modes ( § § 4.1 and 4.2): english, individual-hrl, multilingual (i.e., mixed hrls), and in-language demonstrations (fig. 2 ). to disentangle the impact of demonstration language from other confounding factors (e.g., interactions between demonstration languages and indomain demonstrations), we conduct additional control experiments by adding irrelevant sentences in various languages into english-only in-domain demonstrations ( §4.3). we find that • in-context demonstrations in hrls, especially in languages with non-latin writing systems such as chinese and japanese, can more effectively transfer knowledge compared to english icl mode, leading to performance improvement in answering questions in all languages, especially in lrls. this finding is generalizable enough across different domains and various llms. ## background: multilingual icl in this section, we review the basic approaches of icl with instruction-tuned llms ( § 2.1) and introduce the multilingual prompting modes ( §2.2) that we evaluate in this work. ## conclusion and discussion this work systematically analyzes multiple icl strategies for mllms, and confirms that the presence of multiple languages is an effective strategy across multiple mllms. this improvement is partially due to the inclusion of non-english languages in the prompting, and partially due to the in-topic demonstrations in non-english languages, which together strengthen the models' cross-lingual transfer capabilities, particularly the capability to process lrls. our work echoes with @xcite who suggest that hrls other than english excel in the pretraining-finetuning framework-in the in-context learning framework, highlighting the importance of language inclusivity. we are in agreement with costa-jussà et al. (2022) and liu et al. (2024) that an ideal languageuniversal llm should be equally capable in all languages. beyond this belief, we found that non-english languages may better elicit the potential of mllms. although these observations remain in using hrls for lrl processing, our results strongly support the call for greater research investment in enhancing mllm capabilities for a broader range of languages.