Few-Shot Multilingual Open-Domain QA from Five Examples (2025)

Abstract: Recent approaches to multilingual open- domain question answering (MLODQA) have achieved promising results given abundant language-specific training data. However, the considerable annotation cost limits the application of these methods for underrepresented languages. We introduce a few-shot learning approach to synthesize large-scale multilingual data from large language models (LLMs). Our method begins with large-scale self-supervised pre-training using WikiData, followed by training on high-quality synthetic multilingual data generated by prompting LLMs with few-shot supervision. The final model, FsModQA, significantly outperforms existing few-shot and supervised baselines in MLODQA and cross-lingual and monolingual retrieval. We further show our method can be extended for effective zero-shot adaptation to new languages through a cross-lingual prompting strategy with only English-supervised data, making it a general and applicable solution for MLODQA tasks without costly large-scale annotation.

## introduction open-domain qa has demonstrated impressive performance by employing the retrieve-then-read (figure 1(a) ) pipeline @xcite , which is built upon dense retrievers @xcite and efficient generative readers @xcite . however, this success has been primarily limited to english, leaving the multilingual setting under-explored. this limitation is mainly due to the difficulty and costs of creating high-quality and balanced human-supervised training data for languages other than english. moreover, multilingual open-domain qa introduces additional challenges with retrieving evidence from multilingual corpora, requiring the underlying retrieval system to be capable of * also at google. both cross-lingual and monolingual retrieval @xcite . more recently, efforts have been made to create multilingual open-domain qa benchmarks from existing multilingual machine reading comprehension tasks (e.g., xor-tydi qa @xcite ) and by translating english datasets (e.g., mkqa @xcite ). these datasets have enabled various approaches to address multilingual open-domain qa problems, including iterative data augmentation @xcite and extensive additional pre-training on wikipedia texts @xcite . however, these methods still heavily depend on abundant high-quality language-specific data for fine-tuning, making them less effective solutions when language resources are limited. therefore, a more generalizable approach to multilingual open-domain qa should aim to mitigate this reliance and be capable of facilitating language adaptation with minimally supervised samples. in this paper, we present fsmodqa, a method for few-shot multilingual open-domain qa using minimally-sized supervised data (i.e., up to 5 per language). 1 our approach consists of two core components: a self-supervised pre-training objective on multilingual corpora; and a synthetic data generation pipeline that prompts a large language model (llm) using few-shot supervised examples. concretely, we generate question-answer pairs from wikidata triples by leveraging llms' in-context learning (icl) ability. to facilitate icl prompts, we incorporate chatbots to generate curated input-output pairs, which serve as examples for prompting llms to generate millions of questions from wikidata triples across various languages. after generating these question-answer pairs, we identify the supported wikipedia passages through answer string matching. we further gather cross-lingual answers and evidence passages through wikipedia language links to facilitate cross-lingual retrieval. employing this generated data, we train a multilingual model with a joint objective for retrieval and qa, producing a promising pre-trained model (figure 1(c )) for subsequent few-shot learning. in few-shot learning, we employ llms for data generation from few-shot examples. for each target language, we feed the few-shot examples to an llm and prompt it to generate question-answer pairs from a given document. the few-shot examples are assumed to encapsulate the qa style and distribution of the target dataset, enforcing the llm to generate synthetic data with similar characteristics. with this abundant synthetic data, the pre-trained model can be further fine-tuned to achieve superior results (figure 1(c )). as an unsupervised alternative, we explore a zero-shot cross-lingual prompting strategy that uses data from other languages as prompts for data gener-ation, and we show this is almost on par with few-shot prompting (figure 1(c) ). we evaluate fsmodqa on various datasets, including cross-lingual and monolingual retrieval, and multilingual open-domain qa. we observe notable improvements over competitive few-shot baselines, with +5.1% gain on retrieval and +8.4% gain on multilingual open-domain qa. to further test fsmodqa language adaptation ability, we conduct zero-shot adaptation experiments using our cross-lingual prompting strategy on 15 languages. this adaptation improves performance in both monolingual retrieval and multilingual qa significantly, achieving results that are superior or comparable to strong translation-based methods. foot_2 ## fsmodqa figure 2 presents the full pipeline for generating self-supervised pre-training and fine-tuning data. figure 3 : pre-training data construction pipeline: (1) transform wikidata triples into qas using llms for each target language l, and (2) identify in-language and cross-lingual positive passages from the head entity's wikipedia page and through language links. english translations are added for readability. ## zero-shot language adaptation in §2.2.1, we propose a zero-shot prompting strategy that uses few-shot examples from other languages to generate synthetic data for a distinct target language. the effectiveness of this ar bn fi ja ko ru te avg. ar bn fi ja ko ru te avg. ## conclusion and limitation in this work, we propose fsmodqa, a few-shot learning approach for multilingual open-domain retrieval tasks. we present a novel self-supervised pre-training framework that exploits wikidata to effectively initialize both multilingual retrieval and qa capabilities. this process is followed by few-shot synthetic multilingual qa generation from llms using only five human-annotated examples. we demonstrate that the resulting model achieves competitive multilingual retrieval and qa performance through fine-tuning on the high-quality synthetic data. we further show that this few-shot approach generalizes to zero-shot settings that only require english-supervised data. this mechanism serves as an effective approach for language adaptation, enabling the adapted model to achieve both boosted retrieval and end-to-end qa performance across fifteen previously unseen languages. this work uses llms for synthetic data generation, which may propagate undesirable biases to generated data. we believe such biases will not be amplified as we sample prompts from xor-tydi qa, a dataset annotated with strict guidelines. our preliminary safety analysis also reveals that only less than 1% data contains potentially harmful queries, as identified by llama-guard-2.