Beyond E nglish: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLM s (2025)

Abstract: Despite advances in the multilingual capabilities of Large Language Models (LLMs) across diverse tasks, English remains the dominant language for LLM research and development. So, when working with a different language, this has led to the widespread practice of pre-translation, i.e., translating the task prompt into English before inference. Selective pre-translation, a more surgical approach, focuses on translating specific prompt components. However, its current use is sporagic and lacks a systematic research foundation. Consequently, the optimal pre-translation strategy for various multilingual settings and tasks remains unclear. In this work, we aim to uncover the optimal setup for pre-translation by systematically assessing its use. Specifically, we view the prompt as a modular entity, composed of four functional parts: instruction, context, examples, and output, either of which could be translated or not. We evaluate pre-translation strategies across 35 languages covering both low and high-resource languages, on various tasks including Question Answering (QA), Natural Language Inference (NLI), Named Entity Recognition (NER), and Abstractive Summarization. Our experiments show the impact of factors as similarity to English, translation quality and the size of pre-trained data, on the model performance with pre-translation. We suggest practical guidelines for choosing optimal strategies in various multilingual settings

## introduction large language models (llms) demonstrate impressive capabilities across various natural language processing tasks, including machine translation @xcite , natural language understanding (saba, 2024) and complex reasoning tasks @xcite . these exceptional capabilities of llms stem, to a large extent, from the vast amounts of data they were trained on @xcite . current llms are primarily trained on english data but also include data from other languages, i.e., gpt-3 was trained on 119 languages, but only 7% of the tokens are from non-english languages. 2 with over 7,000 languages spoken worldwide @xcite , the increasing pace of globalization has amplified the need for llms that understand and respond in diverse languages. one common strategy to respond to a task presented in a language different than english is pretranslation, which involves translating the complete prompt into english before querying the model @xcite , allowing to leverage the robust capabilities acquired in english across different languages. at the same time, this approach introduces complexities and risks of information loss @xcite . also, it is unclear whether this approach is uniformly effective across languages and tasks, especially tasks requiring region-specific or culturally-apropriate knowledge. in contrast to pre-translation, recent studies show that direct inference, i.e., prompting the model directly in the (non-english) source language spoken by the user, outperforms pre-translation for tasks like qa @xcite . however, it is unclear whether this approach is optimal, considering that the model was trained on limited data in the source language. it is also unclear how much information is shared across languages during pretraining. be that as it may, as we show in sec. 3.2.1, direct inference results still remain suboptimal. in view of these shortcomings, various studies propose to use selective pre-translation, a more nuanced method compared to the de-facto standard pre-translation approach, which calls for translating only specific parts of the prompt @xcite @xcite . for example, @xcite show that translating only the context to english outperforms direct-inference in summarization and nli. @xcite translated few-shot examples to english while keeping the context in the source language. @xcite used the selective approach when prompting different cross-lingual compositions of in-context examples. however, the selective approach lacks a systematic evaluation of more complex setups, e.g., instruction in english and output in the source language. consequently, the efficacy of selective pretranslation and the optimal prompt configurations for various multilingual settings and tasks remain unclear. to fill this gap, in this paper we set out to examine the impact of selective pre-translation, a commonly used method, across diverse tasks, in order to devise effective prompting strategies for multilingual llms. concretely, we define a formal configuration for a prompt -consisting of four functional parts: instruction, context, examples, and output -either of which could be selectively pre-translated or not (see also @xcite ; @xcite ). we exhaustively assess all configurations of cross-lingual prompt translation into english from different source languages. figure 1 presents an overview of our approach, demonstrating the various selective pre-translation strategies compared against pre-translation and direct inference in the source language. through a comprehensive evaluation involving 35 languages, four tasks, six dataset collections, and three models, our results demonstrate that se-lective pre-translation consistently outperforms both pre-translation and direct inference in the source language, establishing the efficacy of selective pre-translation strategies (section 3). additionally, we analyze the considerations in determining which component to translate, and illustrate the optimal strategies across tasks and languages with varying resource levels. moreover, we examine how factors such as language similarity to english, training size, and language script affect task performance, and show the effectiveness of selective pre-translation method in mitigating various translation issues, by choosing which prompt components to translate (section 4). more specifically, our findings demonstrate that in extractive tasks such as qa or ner, where the output overlaps with the provided context and no generation is needed, the model is either agnostic to the context language in the case of high-resource languages or prefers context in the source language in the case of low-resource languages. surprisingly, we have discovered that low-resource languages yield better results even when the model's output is required in english, e.g., in ner (section 3). moreover, we show that translation quality significantly affects model performance and that the selective pre-translation approach essentially mitigates the negative effect of suboptimal translations, which are in turn specifically problematic in lower-resourced languages (section 4). all in all, our extensive and systematic evaluation of pre-translation strategies facilitates generalization across a broader range of languages and tasks, beyond the specific ones herein, towards more robust llm-use in multilingual settings. ## translation: key to pre-translation in the previous section we examined various factors affecting model performance for selective pretranslation strategies. since translation forms the foundation of selective pre-translation, this section 17 see appendix e for additional results. focuses on a key question: are these factors primarily due to the limitations of llms, or are due to the quality of the pre-translations themselves? to address this, we first isolate the impact of these factors on translation quality through a controlled experiment. subsequently, we investigate how translation quality, independently of other factors, influences downstream tasks in our setup. ## related work with over 7,000 languages spoken globally (anderson, 2010), the growing use of diverse languages have fueled the demand for multilingual llms. progress in this field stems from two primary efforts: (1) developing dedicated monolingual models for low-to-medium-resource languages @xcite @xcite , and (2) creating multilingual llms with pre-trained data encompassing multiple languages @xcite . the ability of multilingual llms to operate in different languages @xcite @xcite comes from two sources: (1) training or fine-tuning on multilingual data in order to achieve multilingual proficiency @xcite @xcite @xcite , and (2) utilizing prompting techniques to harness the model's inherent multilingual capabilities without modifying parameters during inference @xcite . this latter approach has gained popularity due to its efficiency and applicability to a wider range of use cases. for the latter, to improve the multilingual capabilities of llms researchers developed various prompting methods. @xcite introduced xlt, a cross-lingual prompt that directs llms to function as experts in a specific language through a process involving problem-solving and cross-lingual thinking. zhao and schütze (2021) employed discrete and soft prompting techniques and showed that few-shot non-english prompts outperform finetuning in cross-lingual transfer. @xcite found that chain-of-thought (cot) prompts lead to multilingual reasoning abilities in llms, even in under-represented languages. an-other strategy is pre-transaltion which translates the entire prompt to english @xcite @xcite . a more nuanced approach, selective pre-translation, translates part of the prompt into english, for instance, @xcite translated only the instruction, and @xcite translated the few shot examples. while these use cases lack a systematic research foundation, in this study, we systematically study pre-translation configurations to provide evidencebased recommendations for optimal use. ## conclusion in this work we formalize and comprehensively assess selective pre-translation prompting strategies for llms in multilingual settings. with four tasks, six dataset collections, three models, and 35 languages, we deliver the first systematic evaluation, to our knowledge, of all existing prompt configurations of pre-translation. we demonstrate that selective pre-translation consistently outperforms both pre-translation of the entire prompt and diret-inference in the source language, establishing the efficacy of selective pre-translation in both the high-and low-resource cases. additionally, we show that translation quality significantly affects performance and that selective pre-translation can mitigate the negative effects of suboptimal translations.