Multilingual Reasoning via Self-training (2025)

Abstract: Although reasoning is innately language-agnostic, the multilingual capacities remains a significant challenge for large language models (LLMs). Their ability to generate structured, step-wise explanations is constantly restricted to dominant languages in pre-training data, making cross-lingual generalisation difficult and hindering broader global adoption. Recent works have introduced eclectic strategies to improve reasoning beyond English; however, these methods remain related to specific language that is not always optimal for reasoning.To improve LLMs’ multilingual reasoning abilities, we propose a modular approach that instructs the models to structure reasoning passages in a different problem space and then self-refine their capabilities to deliver step-wise reasoning passages that lead to the solution. Experiments show that our approach stably achieves significant improvements in the multilingual reasoning of various models and task, with improved reasoning consistency across languages.

## introduction reasoning is innately language-agnostic; indeed, multiple evidences show that human language is optimised for communication rather than reasoning @xcite . yet, some complex reasoning requires step-wise passages to use extralinguistic constructions, such as formulas and symbols, as a tool for structuring and externalising thought processes (amalric and dehaene, 2016). however, in the era of large language models (llms), approaches such as chain-of-thought (cot) et inter alia aim to emulate human reasoning and thought via language generation, which should not be conditioned by the spoken languages. nevertheless, several works demonstrate that, due to the disparity in pre-training data across languages, the llms' reasoning capability differs between languages and dominant languages, such as english, which is much more performant than the others. research advances in multilingual reasoning are increasingly aimed at completing the performance differences among languages, enhancing the models' capabilities through in-context learning interventions @xcite @xcite , supervised fine-tuning (sft) strategies that differ from language-specific augmentation @xcite to task-oriented tuning @xcite , preference optimisation @xcite and encoderbased strategies @xcite . although these approaches enabled the production of performant solutions for transferring and aligning multilingual reasoning capabilities, we argue that some crucial challenges remain hindering further progress. firstly, the actual benefits of in-context interventions are limited to large-scale llms, which are able to follow provided instructions systematically, but at the same time, they must have robust multilingual proficiency; therefore, many works rely on sft techniques that maintain reduced costs when used to specialised smaller-scale llms. secondly, they require huge amounts of complex reasoning annotations and tremendous tuning efforts to get multilingual llms capable of delivering reasoning through sft and preference optimisation techniques. finally, encoder-based approaches are a convenient solution for aligning multilingual representations. still, they map the input into a latent space represented in symbols by limiting it to a single deterministic decoding path. to improve llms' multilingual reasoning capabilities, we propose a modular approach that first instructs the models to formalise the problem abstractly and then deliver step-wise reasoning trajectories that lead to the reasoning in the same way for all languages. our approach breaks down problem solutions into a sequence of formal, languageagnostic sub-problems, which are solved sequentially and can be more effectively utilised by models. the decomposition consists of two high-level 一组 200 名学生有不同的爱好。50 名学生喜欢阅读，29 名学生喜欢 打板球，其余的喜欢跳舞或烘焙。如果喜欢烘焙的学生人数比喜欢打板 球的学生人数的两倍少 2 人，那么有多少学生喜欢跳舞？ "a group of 200 students has various hobbies. 50 like to read, 29 like to play cricket, and the rest like to either dance or bake. how many like to dance if the number that like to bake is 2 less than twice the number that prefer playing cricket?" modules: formalisation and reasoning execution. as shown in figure 1 , we instruct the models to: (i) identify the relevant information in the given problems, formalising the variable and predicates and delivering symbolic transformations; (ii) generate a reasoning execution where the transformations are solved using symbolic representations that explicitly exemplify the final solution, producing an answer where a final solution is delivered in the same query language. previous works proposed english-based strategies that operate via logical formalisms coupled with external symbolic solvers @xcite . however, entirely symbolic approaches have the bottleneck of requiring a complete translation from natural to formal languages, which may negatively impact efficiency and flexibility, and add further obstacles. to achieve a better trade-off, we treat formalisations in an eclectic manner and propose methods to disentangle content from logical reasoning without introducing rigorous formalisms. to this end, we instruct larger llms to generate synthetic demonstrations using step-wise abstractive thought (swath); then we use the generated demonstrations for self-training smaller llms. as classical tuning strategies behind a standard warmup phase, we propose different alignment methods ranging from supervised fine-tuning (instructiontuning) to preference optimisation techniques (reinforcement learning). we performed an extensive empirical evaluation by observing the impact of different tuning and alignment methods. in multilingual reasoning tasks, our approaches demonstrated significant improvements, resulting in an overall increase in exact matching in proposed tasks, which led to the following results and conclusions: ## method to enable models to do this without the constraints of sequential instructions, we propose a self-training approach that, in addition to the loss functions of the fine-tuning, employs different preference optimisation policies §3.1 to self-improve the models. we iteratively apply preference optimisation algorithms (rl) and fine-tuning (sft), training the model to abstract the problem and deliver a step-wise formal solution ( §3.2). the iterative process ends when the model performance converges or reaches the maximum iteration. a formal description of the proposed method is illustrated in algorithm 1. ## results reasoning can be performed via language-agnostic formalisms that large language models (llms) can employ to enhance performance in multilingual tasks. swath that leads to a formal symbolic reasoned resolution, directing llms to deliver robust answers across different languages. while swath is effective in gpt-4o, in smaller models, it does not have the same benefits. self-training allows smaller models to deliver formal reasoning trajectories while gaining the same benefits without operating through instructions and achieving more consistent results than gpt-4o ( §5.1). self-training is definitely more performant than single sft or rl and allows the models to achieve better results with significantly fewer training data ( §5.2). in indepth studies, we analyse the emerging dynamics between languages ( §5.3) and demonstrate the scalability of our method operating the self-training in further models (appendix k). ## conclusion although reasoning is language-agnostic, the llms' generations are unbalanced towards dominant languages in pre-training. in fact, while llms have a good level of proficiency in different languages and tasks, the ability to deliver step-wise reasoned responses remains unstable. to make multilingual reasoning more equitable and robust across languages, we propose a modular approach that first instructs the model to formalise the problem in an abstractive problem space and then solve it using step-wise passages. in order to align performance across languages, we operate through a self-training approach and show that this substantially improves performance by making multilingual reasoning trajectories more correct.