Enhancing Multilingual Reasoning in LLMs: Insights from Cross-Linguistic Correlations and Optimal Data Proportions

Authors: Jiangkuo Wang, Suyv Ma, Mingpeng Wei
Primary area: datasets and benchmarks
Keywords: large language models, multilingual reasoning, fine-tuning

Abstract:
Large language models (LLMs) typically rely on fine-tuning to enhance their reasoning capabilities across various languages. However, limited research has been conducted on the optimal balance of language proportions within multilingual reasoning datasets. To fill this gap, we performed a systematic study to examine how different proportions of language data in multilingual reasoning datasets influence fine-tuning performance. Our study revealed a clear relationship between language proportions in datasets and the fine-tuning performance of LLMs. By fine-tuning multiple LLMs using the appropriate language distributions and data volumes identified in our study, we achieved state-of-the-art performance in both multilingual mathematical reasoning and solving mathematical problems using Python code. Furthermore, our approach significantly reduced data volume requirements and translation costs compared to existing methods, providing a valuable reference for future research.

Decision: withdrawn
Reviewer scores: 5, 8, 5, 8

Reviews and discussion:
<thread>
creation date:
1730163477800

thread_type:
official_review

<thread_object>
creator: reviewer_wdjh

<content>
summary:
LLM gains reasoning ability across multiple languages after finetuning. The typical approach is to use the same amount of data entries in each language during finetuning. This paper points out that it is more efficient and effective when the proportion of each language is different. Thru a 3-phase systematic approach, authors found the optimal balance in terms of the proportions and achieved the SOTA performance in math reasoning and python code reasoning. It effectively reduces the data volume and translation cost needed during finetuning for multilingual reasoning.

strengths:
The paper is well written and nicely presented with comprehensive tables and figures.

The topic discussed is original and significant, especially during the finetuning stage; that is equal amount of data entries in different languages may not result in the optimal performance. 

Authors made it clear that via experiments; by strategically distributing the amount of data across multiple languages, the multilingual reasoning ability can be acquired at much cost.

The experiment covers 25 languages and most performant open source LLMs.

Authors created two multilingual datasets, namely HighMath-350k and HighCode-350k.

The idea, "given a fixed amount of data volume, how to achieve the optimal performance by including the right amount of data for each aspect", can be borrowed at different stages of the training process, which will lower the cost of training as well as the GPU hours needed.

weaknesses:
There are some minor grammar issues, such as no space after a comma or a period. For example, line 221 "selection,in ..." and line 231 "system.In ..."

Another minor layout issue is that the last two figures split the conclusion onto two pages.

Figure 3 and figure 7 might need some description briefly explaining the purpose for improved readability. 

Figure 3 depicts three scores (COMET, BLEURT, BLEU), but there is no elaboration in the context about them.

It is unclear why the authors only adopted finetuning from non-English to English translation pairs and omitted bi-directional pairs in phase 1.

Finetuning configuration is not properly documented.

The translation of dataset is solely based on the model DeepSeek-Chat-v2-236B. Even though authors adopted some methodology to ensure the quality of translation, like preserving formulas and using Arabic numerals, the validation of the translated content can be improved. For example, a subset sampled from the translation can be further validated by native speakers for coherency and consistency.

questions:
1. what was the reason that English to Non-English translation is not explored?

2. (line 208) what are the two examples used in the prompt? 

3. (line 215) what's the percentage of discarded data entries? 

4. does the pattern / ratio found in this paper single-purposely apply to math reasoning dataset? could it generalize to other multilingual tasks? or should researchers and developers follow the 3 phases and look for a new ratio?

5. (table 1) why is only llama3 investigated in phase 3?

6. (figure 3) the reasoning performance dropped after finetuning (like Swahili), and the dropped amount is close to the improvements. how should we understand this phenomenon?   

7. can you explain the reason why model llama2 was not shown in Table1 but used in phase 3 in the experiment?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730289746835

thread_type:
official_review

<thread_object>
creator: reviewer_gxeb

<content>
summary:
This paper presents a systematic study on the impact of language data proportions in multilingual reasoning datasets on the fine-tuning performance of LLMs. The authors claim to have identified optimal language distributions and data volumes for fine-tuning, leading to state-of-the-art performance in multilingual mathematical reasoning and solving mathematical problems using Python code. The study also aims to reduce data volume requirements and translation costs compared to existing methods.

strengths:
The paper addresses a significant gap in the literature regarding the optimal balance of language proportions in multilingual reasoning datasets for LLMs.

The research is methodologically sound, with a clear three-phase approach and over 600 groups of experiments providing a robust basis for the findings.

weaknesses:
The paper outlines a three-phase methodology to minimize the search space to a manageable size. A crucial step involves categorizing languages based on their alignment with English. It raises questions about the general applicability of the findings. If a LLM is trained primarily based on non-English datasets, does it still work? 

As stated in the abstract, the paper asserts that it achieves state-of-the-art performance in both multilingual mathematical reasoning and in solving mathematical problems using Python code. However, the experimental section lacks empirical comparisons with existing benchmarks that utilize Python code.

Lastly, the presentation requires further enhancement. The font size of the figures is too small, and it would be beneficial to include descriptions for figures, such as Figure 3 and Figure 7.

questions:
In line 252, Figure 3 depicts the variations in translation performance. Which LLM do the results in the figure pertain to? Is the correlation consistent across different LLMs?

In lines 256 to 259, Figure 4 assesses the model's performance on the MGSM dataset after fine-tuning with various non-English to English translation pairs. However, does the author take into account that fine-tuning with WMT datasets might lead to a decline in mathematical reasoning performance?

In lines 368 to 370, when extending 10 languages to 25 languages, how to get the ratio of en:de:ru:fr:others = 24:8:8:8:12?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730429510386

thread_type:
official_review

<thread_object>
creator: reviewer_o1pt

<content>
summary:
The paper shows a way to enhance multilingual reasoning in LLMs by optimizing the proportion of language data in multilingual reasoning datasets. It addresses challenges in fine-tuning LLMs for low-resource languages, which often lack sufficient training data, by investigating the impact of language proportions on model performance. The paper shows optimal language ratios and volume of data for fine-tuning. The paper also present 'HighMath-350k' and 'HighCode-350k' datasets for multilingual mathematical reasoning and Python-based problem-solving.

strengths:
The paper shows a unique way to improve multilingual reasoning in LLMs by optimizing the dataset language proportions and that is highly relevant for AI applications. This work contributes a systematic approach to identifying optimal language data ratios. Also, the paper shows extensive methodology and use of Gaussian Process Regression for language group optimization. The outcome/resultant datasets from this work (HighMath-350k and HighCode-350k) are specifically for multilingual reasoning tasks and that will be helpful for future research in this field.

weaknesses:
There are 25 languages in the phase3 of this study. But in phase 1 and 2, they used a smaller subset. How does this difference may affect the generalizability of findings to low-resource languages is not covered. Also, the datasets generated in this study are based on Non-English to English translation task. However, the study does not cover how to safeguard against translation inaccuracies in low-resource languages.

questions:
1. How do you plan to do quality control for translations in languages where direct verification might be challenging?
I am asking this question because some low-resource languages may not generate accurate translations in English or vice-versa. It would be interesting to know the author's view on this.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730737561878

thread_type:
official_review

<thread_object>
creator: reviewer_rxzq

<content>
summary:
This paper investigates how varying language proportions in multilingual reasoning datasets impact the fine-tuning performance of large language models (LLMs). The authors aim to demonstrate that carefully balancing language data can enhance LLM performance, achieving state-of-the-art results in multilingual mathematical reasoning and problem-solving while reducing data and translation costs.

strengths:
There is significant interest in optimizing the pre-training or fine-tuning mixtures for multilingual LLMs, and this work aims to provide valuable insights into this area.

weaknesses:
W0: The overall writing of the paper lacks coherence, making it difficult to follow the experimental phases. The fragmented presentation of the experiment phases obscures essential aspects of the study, such as training characteristics, the mixture used for fine-tuning, the evaluation tasks, metrics, and analysis. Additionally, several figures, such as Figure 3, lack complete captions, contributing to the confusion. There is also repetition in section headers (e.g., "Phase 2" in Lines 269 and 320), which affects readability and structure.


There are a lot of points and claims that lock in motivation and support:
- W1: Why is language alignment only studied within the context of mathematical reasoning? The rationale for limiting the focus to this particular setting is unclear.
- W2: What criteria were used to select the 10 or 25 languages? How are "high resource" languages defined in this context? Is the selection based on a specific pretraining dataset, and is there a reference for this?
- W3: The grouping of languages appears arbitrary. What was the basis for forming these specific language groups?
- W4: In Line 189, why did the authors choose only high-resource languages for language alignment when, as stated in Line 35, the problem primarily impacts low-resource languages? This choice seems to contradict the motivation expressed in the introduction.
- W5: In Line 45, the paper claims, "The key is to efficiently leverage a small amount of low-resource language data to broadly enhance the multilingual reasoning capabilities of LLMs." Is there any literature or existing research that supports this assertion?

W6: It appears that the evaluation was conducted solely on the MGSM benchmark. This raises concerns about overfitting the optimal mixture to this specific benchmark. How can the proposed method for creating an "optimal mixture" be generalized to other benchmarks?

questions:
Q1: Many references are missing from the claims presented in section 1/ introduction

Q2: Figure 2: Is this data format known for the literature about its performance? Where is the reference?

Q3: Line 076: The motivation for the language extension from 10 to 25 is not clearly explained. What does it mean that the authors extended the language coverage to 25 languages?

Q4: It might be better to provide the full language names in the tables instead of the language codes since, depending on the implementation and the taxonomy used, the codes might be different. 

Q5: Please provide captions on all the figures.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731645834535

thread_type:
public_comment

<thread_object>
creator: ~léo_durand1

<content>
title:
Comments from a Researcher with Similar Experimental Experience

comment:
I have read over 10 papers from ICLR 2025 that are related to my research area, and I must say this is the best one in my view. I find the three-phase approach proposed in this paper highly innovative.

In fact, I previously had similar ideas about exploring the optimal proportions of different languages in reasoning datasets. However, since it required directly exploring the optimal ratios among more than 10 variables, the enormous search space made the experiments infeasible. Consequently, I only obtained experimental results for a few proportions using the ChatGLM model. Although this differs from the models used by the authors, my experimental results largely align with the patterns they identified. This suggests that the authors' experimental conclusions have strong generalization capability.

Moreover, I think the most innovative aspect of the paper is first using translation tasks to explore the alignment relationship between different languages and English to divide the 10 languages into 3 groups, and then determining the optimal relationship between these 3 variables. I wish I had thought of this approach at the time!

I also have a few suggestions: I would like to know whether this method is effective for tasks other than mathematical reasoning. If it is, this could be a very promising research direction. Additionally, I hope the authors could include more raw experimental data in the appendix.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731724146359

thread_type:
public_comment

<thread_object>
creator: ~dmitry_olegovich_smirnov1

<content>
title:
I think I might have an answer

comment:
Since both reviewers are curious about whether this method can be applied to fields outside of mathematics, I actually ran some experiments in the LLM instruction-following area before, so I might have some ideas to share.

I tested the multilingual instruction-following ability using the LLaMA2-7B model. What I found was that fine-tuning on data for non-English languages (like Russian, French, Chinese, etc.) did improve the model’s ability to follow instructions in all the test languages. But I didn’t do large-scale experiments to dig deeper into this because, like Leo, I wasn’t sure how to reduce the search space. After reading this paper, I think the author’s approach and conclusions could be extended to the instruction-following domain. It still needs more experimentation, but I see a lot of potential in this direction.

Overall, I think the author’s methods are really innovative and the results are impressive. The writing and formatting in this version are good. I’d give this paper a 9, maybe even a 10. Because this article can corroborate other experimental results, I think that is important.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731767170709

thread_type:
public_comment

<thread_object>
creator: ~yijiu_chen1

<content>
comment:
Thank you for the excellent work! I noticed that the authors utilized a large-scale multilingual mathematical reasoning dataset in the paper. To my knowledge, the field of multilingual mathematical reasoning has consistently lacked large amounts of high-quality data, as translating such data requires significant financial and labor resources. The scarcity of datasets has hindered many research efforts.

I would like to know if the authors plan to make the dataset used in the paper publicly available. If the dataset could be released, I believe it would be a significant contribution to the community and greatly enhance the impact of the paper.

I look forward to hearing back from the authors.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper examines the optimal balance of multilingual reasoning datasets in the post training mixture of LLMs.  The authors provide extensive experiments to show the various tradeoffs.  Reviewers overall like the paper (with two reviewers strongly supporting acceptance).  I believe that this paper will be quite useful for the community that is investigating multilingual fine tuning of models hence I support acceptance.  On the weakness front, I think abiding by what Reviewer rXzQ is suggesting in terms of writing, will improve the paper.