PROM : Pivoted and Regulated Optimization for Multilingual Instruction Learning (2025)

Abstract: Large language models (LLMs) have become standard for natural language generation tasks, with instruction-tuning enhancing their capabilities. However, the lack of instruction-tuning datasets in languages other than English limits their application to diverse languages. To address this, researchers have adapted English-centric LLMs to other languages by appending English tuning data with its translated pair, from which we observe negative interference between the two. To resolve this, our contribution is identifying English as an internal pivot language, based on which we disentangle the roles of English and target language data in training. Specifically, we first design two roles as pivoted objectives, and also propose to regulate between the two, to better generalize for under-represented languages. Experiments across various languages demonstrate the effectiveness of our approach on multiple benchmarks. The code is publicly available for further exploration.

## introduction recently, large language models (llms) became a de-facto standard for various natural language generation tasks @xcite @xcite . moreover, careful instructiontuning @xcite improves the llms to be more powerful. however, due to the lack of instruction tuning datasets in other languages, most of instructiontuned llms remain english-centric, hindering the application to 6500+ existing languages @xcite . existing solutions thus propose to adapt english-centric llms into a monolingual target language model: instructions in the target language are either unseen, or underrepresented in pretraining, for which the existing solution translates a high-quality english instruction tuning, to pair with its translation in the target language @xcite . despite the expected performance gain from expanding the training set, our first contribution is observing otherwise, that negative interference @xcite exists between the original and translated pair. (section 3.2). to overcome this, we devise a pivoted objective that disentangles english and target language data in training, to alleviate such interference. specifically, we are inspired by a recent finding that english-centric llms generate in english first and then convert the output into the target language @xcite . this implies that we can design two separate objectives, the first objective using english data for generating the representation corresponding to the english version of the next token at the middle of the layers, then another objective using target data for gradually converting into the representation for the target language. while such disentangled objectives are effective in many languages, we find they fail to generalize well to under-represented languages, where we observe the pivoted behavior reported by @xcite may not hold. to illustrate, figure 1 contrasts language where pivoted assumption holds (left) and not (right), selected for illustration from our empirical studies reported in appendix: following @xcite , the x-axis in the figure represents layer index, from each of which, the y-axis shows the probability (according to logits) of correct target language next token (blue) or english as pivot (orange). while the left figure shows english pivot probability higher than the target token in greek, such behavior is not observed in the right (arabic). inspired, we propose a regulated version, classifying between the two cases, to selectively apply pivoted objective. our proposed method, prom (pivoted and regulated optimization) is shown to be effective on mgsm, xquad, mlqa, indicqa across 9 languages. prom dominates the baselines in most cases, improving the qa exact match score by 50% overall. the code is publicly available. 1 2 pivoted and regulated optimization preliminaries: adapting llm to the target language we first formalize the training of llm architecture as follows: to enhance the set s for adaptation to the target language, we typically augment each existing english instruction and response s @xmath0. moreover, an additional english to target language translation task sample s @xmath1. ## related work instruction-tuned llms for non-english to extend the capabilities of instruction-tuned llms to languages other than english, early attempts involved human annotation of instruction-tuning datasets @xcite , which lacks scalability. @xcite ; @xcite leverage llms to generate synthetic data for instruction-tuning, however the quality would plummet as the generation ability of llm for that language decreases than english. alternatively, machine-translated instructiontune datasets @xcite @xcite paired with higher-quality english instruction-tune data @xcite gained popularity. our distinction is observing a possible negative interference between english and target data, and mitigating it by disentangling the roles of the two. english as a pivot language @xcite explicitly observed pivoting behavior in llama2, an english-centric llm that the llm first generates representations for the next token in english at the middle layer before converting them to representations of the target language at the final layer. our work is inspired by this observation but goes beyond passive observation by (1) recognizing the limitations of their findings for under-represented languages and (2) extending into optimization objectives to mitigate negative interferences. ## conclusion in this paper, we found that appending the english instruction sets along with its translated pairs is not always beneficial, for instruction-tuning in multiple languages. to overcome this, we proposed prom, where we devised pivoted objective and regulated objective. experimental results across 9 languages show the effectiveness of our proposal.