Cross-Lingual Extractive Question Answering with Unanswerable Questions (2025)

Abstract: Cross-lingual Extractive Question Answering (EQA) extends standard EQA by requiring models to find answers in passages written in languages different from the questions. The Generalized Cross-Lingual Transfer (G-XLT) task evaluates models’ zero-shot ability to transfer question answering capabilities across languages using only English training data. While previous research has primarily focused on scenarios where answers are always present, real-world applications often encounter situations where no answer exists within the given context. This paper introduces an enhanced G-XLT task definition that explicitly handles unanswerable questions, bridging a critical gap in current research. To address this challenge, we present two new datasets: miXQuAD and MLQA-IDK, which address both answerable and unanswerable questions and respectively cover 12 and 7 language pairs. Our study evaluates state-of-the-art large language models using fine-tuning, parameter-efficient techniques, and in-context learning approaches, revealing interesting trade-offs between a smaller fine-tuned model’s performance on answerable questions versus a larger in-context learning model’s capability on unanswerable questions. We also examine language similarity patterns based on model performance, finding alignments with known language families.

## introduction extractive question answering (eqa) is the task of finding text spans within given contexts that answer given natural language questions. this field was formalized with the stanford question answering dataset @xcite , which set a key benchmark for eqa. recent advances in large language models @xcite have significantly improved eqa performance, marking important progress in natural language understanding. cross-lingual eqa is a task where the question and its corresponding context are presented in different languages foot_1 , addressing a critical need in today's globalized world. for instance, an english-speaking user might need to query content available only in chinese or arabic, challenging the monolingual assumptions in traditional qa systems. recent research has demonstrated that eqa techniques can be effectively applied to downstream tasks, showing promising results for example in zero-shot event extraction @xcite and summarization evaluation @xcite . while these studies focused on english, their qabased approaches could naturally extend to crosslingual scenarios such as cross-lingual summarization @xcite and cross-lingual event extraction @xcite . @xcite introduced the generalized cross-lingual transfer (g-xlt) task, addressing cross-lingual qa in the case where models are trained on english and evaluated on multiple language pairs, assuming that all questions are answerable. however, the ability to identify when a question cannot be answered is crucial for real-world applications. in this paper, we expand the g-xlt framework by introducing unanswerable questions, as illustrated in figure 1 . this extension reflects real-world scenarios where answers may not exist within the given context. our approach also broadens the evaluation of @xcite , limited to encoder-based models, to include state-ofthe-art models, encompassing large language models (llms) and various transformer architectures, providing a more comprehensive assessment of cross-lingual qa capabilities. to address the limitations in current datasets and to provide a robust benchmark for this extended task, we have developed two test datasets: mixquad and mlqa-idk. the mixquad dataset combines elements from squad v2.0 and xquad @xcite , integrating unanswerable questions into a multilingual framework. it covers 12 languages and includes a total of 2,072 examples per language, balancing answerable and unanswerable questions. the mlqa-idk dataset, derived from mlqa @xcite , specifically focuses on cross-lingual unanswerability, covering 7 language pairs and employing techniques such as antonym and entity augmentation to generate unanswerable questions. our cross-lingual qa analysis reveals a trade-off between fine-tuned small models and large models with in-context learning: mt5-large (1.2b parameters) with fine-tuning excelled at detecting unanswerable questions, while aya-101 (13b parameters) with hint prompting, where the unanswerability option is mentioned in the prompt, performed better on answerable questions. hint prompting significantly improved unanswerable question detection across models while maintaining performance on answerable questions. fine-tuned aya-101 achieved the best performance, outperforming both its regular prompt version and mt5-large across both answerable and unanswerable questions. out-of-domain testing on mlqa-idk and open-domain evaluation on xtreme-up @xcite demonstrate model robustness across diverse qa scenarios, including low-resource languages. we also examine the dependence of the results on the specific languages and on linguistic relationships. first, comparing between cases where the question is in english (english-questions) to those where the context is in english (english-contexts), we observe that models performed better when contexts are in english, indicating that processing questions in various languages while keeping english contexts is more manageable. second, language clustering analysis revealed three groups that align to some extent with language typology-suggesting linguistic relationships influence model behavior. in an advanced analysis, we examine answerability-related error patterns, test the models' reliance on parametric knowledge, and explore their uncertainty in the different types of prediction. in particular, we observe that hint prompting reduces uncertainty when classifying unanswerable questions, while fine-tuning improves overall certainty but reduces the confidence gap between correct and incorrect predictions. our main contributions are the following. first, we expand the generalized cross-lingual transfer (g-xlt) task to explicitly handle unanswerable questions. second, we introduce two novel test sets, mixquad and mlqa-idk for the extended task. third, through the analysis of state-of-the-art models with varying architectures and parameter sizes, we provide insights into performance patterns and language dependency, and reveal tradeoffs between model size and training approaches, advancing cross-lingual qa understanding. ## methodology all our experiments were conducted in a zero-shot setting, where the training data (for fine-tuning) and provided examples (for in-context learning) are exclusively in english from the squad v2.0 dataset. as detailed in appendix b, we experimented with a diverse set of models ranging from 168m to 13b parameters, using either fine-tuning or in-context learning approaches depending on model architecture and size. ## conclusion our study advances cross-lingual eqa through several key findings. fine-tuning large language models proved most effective, with aya-101 outperforming both smaller fine-tuned models and large models using in-context learning. hint prompting enhanced unanswerable question detection without compromising answerable performance. models