MindMerger: Efficiently Boosting LLM Reasoning in non-English Languages

Authors: zixian  huang
wenhao  zhu
gong  cheng
lei  li
fei  yuan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 11:42:33

Abstract: Reasoning capabilities are crucial for Large Language Models~(LLMs), yet a notable gap exists between English and non-English languages. To bridge this disparity, some works fine-tune LLMs to relearn reasoning capabilities in non-English languages, while others replace non-English inputs with an external model's outputs such as English translation text to circumvent the challenge of LLM understanding non-English. Unfortunately, these methods often underutilize the built-in skilled reasoning and useful language understanding capabilities of LLMs. In order to better utilize the minds of reasoning and language understanding in LLMs, we propose a new method, namely MergeMinds, which merges LLMs with the external language understanding capabilities from multilingual models to boost the multilingual reasoning performance. Furthermore, a two-step training scheme is introduced to first train to embeded the external capabilities into LLMs and then train the collaborative utilization of the external capabilities and the built-in capabilities in LLMs. Experiments on three multilingual reasoning datasets and a language understanding dataset demonstrate that MergeMinds consistently outperforms all baselines, especially in low-resource languages.  Without updating the parameters of LLMs, the average accuracy improved by 6.7 and 8.0 across all languages and low-resource languages on the MGSM dataset, respectively.

Main paper: 
 introduction: 
 One of the primary focuses of Artificial Intelligence research currently revolves around improving its reasoning capabilities [Ahn et al., 2024 , Minaee et al., 2024] , which is derived from the need to enable Large Language Models (LLMs) [Ouyang et al., 2022 , Touvron et al., 2023 , Jiang et al., 2023a] to think rationally and perform functions like humans [Imani et al., 2023 , Jiang et al., 2023b] . Substantial progress has been made in English reasoning [Yu et al., 2023 , Yuan et al., 2023a] , but the performance in non-English, especially low-resource languages, still lags behind [Shi et al., 2023] due to the scarce of multilingual training data [Touvron et al., 2023] .Existing work tries to use external models to compensate for the deficiencies of LLM in multilingual reasoning. Some works use the relearning-based strategy, which uses translation models to generate multilingual training data for fine-tuning LLMs to relearn reasoning in each language [Chen et al., 2023 , Chai et al., 2024] . Some other works use the replacement-based strategy, which use translation models to translate queries from non-English to English text for replacing the non-English input of LLM [Shi et al., 2023] . Both strategies try to use the translation model to help LLMs master new capabilities, but the insufficient translation quality constrains the performance of these methods.Moreover, certain capabilities, such as reasoning and language understanding, are built-in in LLMs and should be utilized without the need to develop them from scratch. For the reasoning capabilities, as illustrated in Figure 1 , regardless of the language in which the same mathematical question is posed, the correct reasoning process remains consistent. It shows that the reasoning capabilities is universal rather than language-specific [Brannon, 2005] . For language understanding capabilities, while the Chinese question in the first example of Figure 1 may cause a reasoning error by failing in understanding the English-origin term "dozen of", the LLM successfully differentiated between "the week" and "the weekdays" in the Chinese question, contrasting with the failure of the English question in the second example. It shows that although the proficiency may not match that of English, expressions in non-English languages remain valuable to LLMs.Considering that the built-in reasoning and language understanding capabilities of LLMs need to be better utilized, in this paper, we propose a new method MindMerger, which preserves the minds of reasoning and language understanding capabilities in LLMs, and merges the external language understanding capabilities from pre-trained multilingual models to boost the multilingual reasoning effects. To address the challenge of insufficient generation quality of external models, MindMerger feeds the LLM the undecoded query representation from the multilingual model rather than text. Additionally, it uses an augmentation strategy that combines the encoded query representation with the input of LLM to utilize both external and built-in language understanding capabilities.Given the inconsistency in the representation space, understanding the query representation encoded by multilingual model is not trivial for LLMs. To address this, we propose a two-stage training scheme including the mapping stage and the augmentation stage. In the mapping stage, we train MindMerger to embed the language capabilities of the multilingual model into the LLM by using accessible general bilingual pairs such as translation data. In the augmentation stage, MindMerger is further trained to collaboratively utilize the built-in capabilities of LLM and the embedded external language capabilities by using query translation task data generated from translation model. Throughout the two stages, only a small number of parameters that connect two models are trained, while both the multilingual model and the LLM are frozen to prevent their built-in capabilities from forgetting.Extensive experiments are conducted on three multilingual reasoning datasets and a language understanding dataset. Taking on the MGSM dataset [Shi et al., 2023] as an example, MindMerger outperforms all baselines and achieves a lead of at least 6.7% on the average accuracy across all languages, and its performance in low-resource languages is even more significant leading by at least 8.0% ( § 4.3) . Compared with the replacement-based method that translates non-English text into English as the LLM input, MindMerger can lead by at least 6.6% in average accuracy based on the same translation model ( § 5.1). Benefiting from the accessible general bilingual pairs used in the mapping stage, the average accuracy across low-resource languages increased by 14.9% ( § 5.2).Our contributions can be summarized as follows:• We propose a new method MindMerger to boost the multilingual reasoning of LLMs, which preserves the built-in reasoning and language understanding capabilities of LLMs while augmenting them with the external language understanding capabilities from multilingual models. • We propose a two-stage training scheme to help MindMerger sequentially learn to embed external capabilities and collaboratively utilize internal and external capabilities.• MindMerger achieves the best results on four dataset about multilingual reasoning or understanding datasets, notably improving all languages, including low-resource languages, with an average accuracy increase of at least 6.7% and 8.0% on the MGSM datasets. 
related work: 
 Multilingual Reasoning. There have been some attempts to improve LLM's performance on multilingual reasoning. Several works design crafted prompts to assist LLMs in reasoning [Huang et al., 2023 , Qin et al., 2023] , but their effectiveness diminishes when used with open-source LLMs like Llama [Touvron et al., 2023] due to limited capacity for multi-step instruction-following [Huang et al., 2023] . Supervised fine-tuning LLMs is another effective way, where some works use translation models to translate query-response [Chen et al., 2023] , or query-only [Zhu et al., 2024] to build multilingual task datasets, enabling LLMs to relearn reasoning or language understanding. Some other works utilize external models to generate English text [Shi et al., 2023] to replace the non-English input of LLMs, aiming to utilize the reasoning capabilities of LLMs directly. However, the above methods are limited by the generation quality of the translation model, and the built-in reasoning or language understanding capabilities of LLM are neglected. In contrast, MindMerger avoid the loss introduced by autoregressive decoding and employs an augmentation-based strategy that utilizes the built-in capabilities of LLMs to enhance multilingual reasoning performance.Model Merging. Model Merging is a popular topic in recent LLM research. Generally, it aims to combine an external module to strengthen the capabilities that LLM lacks, such as multi-modal vision capabilities [Liu et al., 2023 , Li et al., 2023 , Zhu et al., 2023] . Some works [Sun et al., 2021 , Bansal et al., 2024] find that interpolating models with multilingual capabilities using cross-attention can improve the performance of multilingual tasks, but research on merging multilingual models and LLMs with English reasoning capabilities is still scarce. Recently, Yoon et al. [2024] merge the encoder of multilingual model and the LLM to improve the multilingual reasoning performance. However, the input of LLMs is completely replaced by the output of a multilingual encoder, making its built-in multilingual capabilities underutilized. In addition, only using English task data for training limits the performance of model merging. Instead of replacing the input of LLMs, we collaboratively utilize the features of the multilingual model and use the available general bilingual pair to train to obtain multilingual reasoning capabilities.3 Approach Given an LLM with skilled reasoning capabilities and useful language understanding capabilities, our target is to maintain the built-in capabilities and compensate for its shortcomings in non-English language understanding capabilities with an external multilingual model. To this end, as shown in Figure 2 , we propose MindMerger that uses the output of the multilingual model as an augmentation complementing to the original input ( § 3.1). We further design a two-stage training scheme to learn the collaborative utilizing both the external and built-in capabilities ( § 3.2). 
model structure: 
 Given a query q with l tokens, we first utilize the multilingual model to understand and encode it into a representation X that is more general and reduces the challenge of multilingual understanding: X = Encoder(q) , (1) where Encoder(•) is a pre-trained multilingual model, typically using its encoder part, and X ∈ R l×d1 is the hidden state output of the multilingual model with a dimension of d 1 . The representation X resides in the multilingual model space, which is separate from the LLM space and cannot be used directly. Therefore, we introduce a mapping layer:EQUATIONwhere X ∈ R l×d2 is the projection of X on the space of LLM. Unless otherwise stated, the implementation of Mapping(•) is a two-layer multi-layer perceptron (MLP).The acquisition of X utilizes the language understanding capabilities of the external multilingual model. In order to take advantage of the built-in capabilities of the LLM, we prompt it directly:EQUATIONwhere T ∈ R l×d2 is the representation of query q in the space of LLM and Embedding(•) is the embedding layer of LLM.Then, we concatenate the query representation from the multilingual model and the LLM for the collaborative utilizing of LLM's capabilities:( X, T ) = [⟨bos⟩; X; ⟨sep⟩; T ] ,(4) where ⟨bos⟩ ∈ R d2 is the representation of the start token of LLM and ⟨sep⟩ ∈ R d2 is a trainable variable that denotes the boundary of X. Finally, ( X, T ) is fed to LLM to generate the response.The training scheme of MindMerger is divided into two stages: mapping stage and augmentation stage. The former helps LLM learn to use the capabilities of multilingual model, and the latter helps LLM collaboratively utilize its own and the capabilities from multilingual model. Examples of training data for each stage are shown in Appendix D.Mapping Stage. Given the representation spaces between multilingual model and LLM are distant different, it is not trivial for LLM to understand and utilize the external capabilities provided by multilingual model. In order to better learn to utilize external capabilities, we force MindMerger to focus on input from the multilingual model using the replacement-based strategy during this stage. Specifically, we simplify the input of Equation (4) as follows: M = [⟨bos⟩; X; ⟨sep⟩] .(5) Since general bilingual pairs such as translation data is in vast availability, we use it from various languages to English to train MindMerger in this stage. The loss function is outlined as follows:-arg minEQUATION)where Y is the text of training target, θ and ϕ are the parameters of the multilingual model and LLM respectively, which are frozen during the training to prevent forgetting, and σ contains the trainable parameters of the mapping layer in Equation ( 2) and the boundary token ⟨sep⟩.Augmentation Stage. Although MindMerger have learned to utilize external capabilities in the mapping stage, the replacement-based strategy may cause LLM to neglect the use of its own built-in capabilities. To help MindMerger further learn to merge capabilities from external and built-in LLM, in this stage we use the augmentation-based strategy as described in the Equation (4). The loss function is calculated as follows:-arg minEQUATION)where σ is initiated from the checkpoint of σ trained at the mapping stage, and the parameters of the multilingual model and LLM represented as θ and ϕ, respectively, are kept constant during training. In this stage, the query translation task data generated from public translation models is used as the training data to adapt MindMerger to downstream task. 
compared methods: 
 Our Methods. Two variants of MindMerger were implemented. (1) The implementation described in § 3.1 is denoted as MindMerger-Soft.(2) MindMerger-Hard augments the prompts of LLM with the translated query given by the translation model (Appendix D for the details of prompts).Baselines. We compared our methods with three categories of baselines. (1) One basic method MonoReason [Yu et al., 2023 , Zhu et al., 2024] which is fine-tuned on the English task dataset.(2) Three relearning-based methods that use task data with query translation, including the fullparameter fine-tuning model MultiReason-Full [Zhu et al., 2024] , parameter-efficient fine-tuning model MultiReason-Lora [Hu et al., 2022] , and the state-of-the-art method QAlign [Zhu et al., 2024] which first learns language understanding by training LLM on query translation data and then further fine-tunes LLM as MonoReason.(3) Two replacement-based methods that introduce external models, including Translate-En [Shi et al., 2023] and LangBridge [Yoon et al., 2024] . Translate-En is a hard replacement-based which translates the query into English to replace the prompt of LLM. LangBridge is a soft replacement-based method which replaces the input of LLMs with the hidden states output by mT5-xl [Xue et al., 2021] . The prompts of each baselines are presented in Appendix D.Details. Unless otherwise stated, we used the encoder part of mT5-xl [Xue et al., 2021] as the multilingual model in our methods, used the NLLB200-3.3B as the translation model for baselines, and used the Llama 2-7B as the LLM for all methods. The influence of different multilingual models including encoder-only, decoder-only and encoder-decoder architectures will be analyzed in § 5.1.Both MindMerger and all the baselines, except QAlign, are based on the same MonoReason model. Additionally, MonoReason and QAlign are trained based on the same LLM. Specifically, we used the publicly available checkpoint given by Yu et al. [2023] as MonoReason model for mathematical reasoning task. For all models, we set learning rate=2e-5, batch size=128, max length=512, and epoch=3 and used 8 NVIDIA A100 GPUs for training.Evaluation Datasets. We experimented MindMerger with the latest multilingual mathematical reasoning MGSM [Shi et al., 2023] and MSVAMP [Chen et al., 2023] , where MSVAMP serves as an out-of-domain test set. In order to evaluate the diverse multilingual reasoning capabilities of the models, a challenging multilingual dataset X-CSQA [Lin et al., 2021] in commonsense reasoning task and a language understanding dataset XNLI [Conneau et al., 2018] in natural language inference (NLI) task were used. The statistics of these datasets are presented in Appendix D.Training Datasets. Three categories of training data were used in our methods and baselines. (1) General bilingual pairs. We used the translation data from the multilingual language to English and randomly sampled 100K of data for each language (except English) from the Lego-MT [Yuan et al., 2023b] dataset, which is a large-scale translation dateset that contains all the languages that involved in our experiments.(2) English task data. We used MetaMathQA [Yu et al., 2023] and MultiNLI [Williams et al., 2018] datasets for mathematical reasoning and NLI task, respectively. Similar to Huang et al. [2022] , we unified the training set of X-CSQA, OpenBookQA [Mihaylov et al., 2018] , ARC [Bhakthavatsalam et al., 2021] and QASC [Khot et al., 2020] to train commonsense reasoning task more fully.(3) Query translation task data. We used the translated results given by Chen et al. [2023] and the official dev set of XNLI for mathematical reasoning and the NLI task, respectively, and translated the X-CSQA training set based on M2M100-1.2B [Fan et al., 2021] .MindMerger improves LLM performance on all datasets, especially benefiting low-resource languages. MindMerger-Soft has an average accuracy better than all other baselines at least 6.7%, 3.2% on the MGSM and MSVAMP datasets in Table 1 , demonstrating its remarkable multilingual reasoning capabilities. For the commonsense reasoning task in Table 2 , MindMerger-Soft also significantly leads all baselines by at least 4.1%. In Table 3 , MindMerger-Soft demonstrates advantages in language understanding, which significantly outperformed all the baselines (with p-value < 0.01). MindMerger-Hard achieves the best results except MindMerger-Soft on three out of four datasets, demonstrating the advantages of augmentation-based methods over other categories of methods.Table 1 : Experimental results on MGSM and MSVAMP datasets. Lrl., Hrl., and Avg. represent the average accuracy across low-resource languages, high-resource languages, and all languages, respectively. Referring to Shi et al. [2023] , we regard Bn, Th, and Sw as low-resourse languages, and regard the remaining languages as high-resource languages. We used the checkpoints given by Yu et al. [2023] as the MonoReason models. MindMerger-Soft stands out notably in enhancing the reasoning capabilities of low-resource languages. Compared to all baselines, it delivers an average accuracy improvement of at least 8.0% and 5.0% across low-resource languages on the MGSM, MSVAMP datasets.Utilizing the built-in reasoning capabilities of the LLM rather than relearning for non-English is a more effective way to boost multilingual reasoning capabilities. It is not easy for relearning-based methods to improve the performance of high-resource and low-resource languages simultaneously.For instance, MultiReason-Full improves its performance in low-resource languages from an average of 6.9% to 38.4% on the MGSM dataset, but its performance in high-resource languages declines from an average of 51.8% to 45.6%. Compared with the three relearning-based baselines, MindMerger-Soft achieves a lead on both low-resource and high-resource languages and outperforms them by at least 6.9% on the MGSM datasets for the average accuracy across all languages.Utilizing the built-in language understanding capabilities of LLM rather than replacing them boosts multilingual reasoning capabilities more significantly. Comparing Translate-En and MindMerger-Hard which are completely consistent except for the input strategy, the augmentationbased MindMerger-Hard has a higher average accuracy than the replacement-based Translate-En of 4.4% and 11.8% across high-resource languages in the MGSM and MSVAMP datasets respectively, showing that non-English input can help LLM enhance query understanding. Although the understanding capabilities of LLM in low-resource languages are limited, MindMerger-Hard still leads Translate-En by 2.3% on the MSGM dataset. Larger boost comes from MindMerger-Soft, which leads all replacement-based methods with an average accuracy of at least 6.7% across all languages. MindMerger can flexibly interpolate among various multilingual models. We experimented with a wide variety of multilingual models, including decoder-only models mGPT [Shliazhko et al., 2022] , encoder-only models mBERT [Devlin et al., 2018] and XLM-RoBERTa-large [Conneau et al., 2019] , MonoReason [Yu et al., 2023 , Zhu et al., 2024] and the encoder part of encoder-decoder models M2M100 [Fan et al., 2021] , NLLB-200 [Costa-jussà et al., 2022] and mT5 [Xue et al., 2021] . The experimental results in Table 4 show that the encoder part of the encoder-decoder model and the encoder-only model are more suitable interpolate into MindMerger-Soft than the decoder-only model, which can achieve better performance than mGPT with a smaller number of parameters. M2M100 is the most cost-effective model, reaching or even exceeding the performance of XLM-RoBERTa-large and mT5-large while using only half of the parameters of XLM-RoBERTa-large and mT5-large.Our augmentation-based strategy outperforms the translate-then-replace strategy. Comparing Translate-En and MindMerger-Hard which only differ in input strategy, augmentation-based MindMerger-Hard, based on the same translation model, consistently exceeds Translate-En by 5.0%, 5.9%, 4.7%, and 4.1% in the average accuracy. MindMerger-Soft further expands its lead with an increment of the average accuracy by at least 6.6% based on the same translation model.MindMerger-Soft is a better utilization of existing multilingual model. As shown in Table 4 , the performance of MindMerger-Soft consistently exceeds MindMerger-Hard under the same multilingual model with increases of average accuracy of 5.5% and 3.3% on two versions of M2M100, and 3.6% and 2.5% on two versions of NLLB-200. Although only the encoder part of the multilingual model is used, MindMerger-Soft merges LLM with a dense representation rather than decoded text based on sparse bag-of-words, enhancing the effectiveness of utilizing multilingual model.A more powerful multilingual model can better enhance the multilingual capabilities. We compared the different sizes of each encoder-decoder models and consistently observed that the larger version outperformed the smaller one in Table 4 . With the help of the larger model size, the average accuracy is improved to 1.1%, 1.0%, and 3.7% on M2M100, NLLB-200, and mT5, respectively. The improvement in low-resource languages is even more obvious with an average increase in accuracy of 1.1%, 1.0% and 3.7% in M2M100, NLLB-200, and mT5, respectively. This underscores the greater imperative to enhance language understanding in low-resource languages.Lrl. represent the average accuracy across low-resource languages, high-resource languages, and all languages, respectively. Referring to Shi et al. [2023] , we regard Bn, Th, and Sw as low-resourse languages, and regard the remaining languages as high-resource languages. 
ablation studies: 
 Mapping Stage. In Figure 3a , we ablated the mapping stage to observe its necessity. A significant drop in the performance of low-resource languages can be observed when ablating the mapping stage, which shows that the accessible general bilingual pairs are beneficial to help MindMerger-Soft understand the low-resource language information that exists in the multilingual model representation space. More detailed are reported in Appendix B.1.Augmentation Stage. In Figure 3b , we ablated the augmentation stage to observe its necessity. It can be observed that after ablating the augmentation stage, even without using any task-related data, MindMerger-Soft can still outperform MonoReason on low-resource languages, which demonstrates the generalization of MindMerger-Soft that benefits from using accessible general bilingual pairs in the mapping stage. Furthermore, when the augmentation stage is added, MindMerger-Soft exhibits a significant improvement, demonstrating its effectiveness in learning the utilization of both external and built-in capabilities. More detailed are reported in Appendix B.2.Replacement vs. Augmentation. MindMerger uses the representation X outputted by multilingual model to augment the LLM's original representation T rather than replace it to better utilize the builtin capabilities of LLMs. To verify the advantages of the augmentation-based strategy, we removed T to make MindMerger-Soft a replacement-based method. As shown in Figure 3c . although with exactly the same training data and process, the performance of the replacement-based MindMerger-Soft drops significantly, indicating that it is valuable to use the built-in capabilities of LLM to understand the original input. More detailed are reported in Appendix B.3.Table 5 : Results on the MGSM dataset based on MetaMath-Llama-13B and MetaMath-Mistral-7B. Lrl., Hrl., and Avg. represent the average accuracy across low-resource languages, high-resource languages, and all languages, respectively. Referring to Shi et al. [2023] , we regard Bn, Th, and Sw as low-resourse languages, and regard the remaining languages as high-resource languages. MindMerger can be flexibly integrated with different LLMs. To verify this, we conducted experiments on a different type of LLM, MetaMath-Mistral-7B [Jiang et al., 2023a , Yu et al., 2023] , and a larger size, MetaMath-Llama-13B [Touvron et al., 2023 , Yu et al., 2023] . The experimental results are shown in Table 5 , MindMerger-Soft has achieved superior performance across various baselines.The average accuracy of MindMerger-Soft on the MetaMath-Llama-13B and MetaMath-Mistral-7B versions is at least 4.1 and 4.7 higher than all baselines, respectively, demonstrating the potential of extending MindMerger to a wider range of LLMs. For each language, we selected the same 100 texts from the Flores-101 [Fan et al., 2021] dataset, used the mean pooling operation to obtain the representation vectors of the LLM embedding and the hidden states output by and the mapping layer, and visualized it based on T-SNE [Van der Maaten and Hinton, 2008] . As shown in Figure 4 , the representation spaces of non-English on the LLM embedding are independent and away from English, especially in low-resource languages, which leads to understanding challenges for non-English and the inability to use built-in reasoning capabilities. By contrast, as shown in Figure 4b , the representations of all languages outputted by the mapping layer almost overlap with English, which reduces the difficulty for LLM to understand non-English languages and enables various languages to utilize the built-in reasoning capabilities.We experimented with several supplementary experiments, including the influence of training dataset size used in augmentation stage (Appendix A.1), the selection of mapping layers structure (Appendix A.2), the usage of encoder-decoder model in MindMerger-Soft (Appendix A.3), the quantitative analysis on representation space changes (Appendix A.4), and the translation performance of MindMerger-Soft after mapping stage (Appendix A.5).Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: Our code and data are included in the supplementary material and will be made publicly available after the paper is accepted. Guidelines:• The answer NA means that paper does not include experiments requiring code. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 This paper explores a way to more fully utilize the built-in capabilities of LLMs to improve multilingual reasoning effects. We proposed MindMerger to merge the expert multilingual capabilities in the multilingual model with the skilled reasoning and not very proficient but useful multilingual capabilities in the LLM. Through more fully utilizing the potential of LLM and more effective fusion of multilingual models, the performance of MindMerger exceeds all baselines on three reasoning datasets and a language understanding dataset. In the future, we will explore the possibility of MindMerger empowering more professional skills besides reasoning, such as code generation.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We report the detail of experimental setting in § 4.1. And our code and data are included in the supplementary material and will be made publicly available after the paper is accepted.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We calculate the statistical significance for the main experimental results in § 4.3, our method significance outperform all compared baselines.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We report the compute resources in § 4.1.Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer:[NA] Justification: Our work does not involve crowdsourcing nor research with human subjects. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A. Supplementary Experiments We randomly sampled different size of training set to train MindMerger-Soft on the mathematical reasoning task and reported the results on MGSM in Table 6 . Despite only 100 samples per language is used, MindMerger-Soft can be improved by 3.0% in low-resource languages compared to without augmentation stage. The training data size has a significant impact on the performance of MindMerger within the data size range of 10,000. MindMerger-Soft achieves the best result when using 30,000 samples per language, and better results may require enhancing the model's built-in capabilities.. Considering the query training data for various tasks can be easily obtained from public translation models, and training on a scale of tens of thousands is cost-effective, MindMerger can be readily adapted to a wider range of tasks. We experimented with the performance of MindMerger-Soft when using different mapping layers in Table 7 . The 2 layers MLP used in the main experiment achieved the best results, while the effect of Linear will be lower in comparison. This may be the insufficient space transfer capability of Linear due to the small number of parameters. As for QFormer [Li et al., 2023] , which converts variable-length input into fixed-length hidden states output, is a popular model for model merging in computer vision, but it does not work well on MindMerger-Soft. This could be attributed to the multilingual capabilities of LLMs, which may render features diverging too much from the original expression less readily accepted. In all other experiments, for the encoder-decoder model, we only use its encoder part. In this section, we compared the performance of MindMerger-Soft using the decoder. Specifically, we use the translation model M2M100-1.2B autoregressively to generate English text, and take the hidden states corresponding to the generated text, and input them into the mapping layer alone or after concatenating with the output of the encoder. The experimental results are shown in Figure 8 , where the performance of MindMerger-Soft dropped after adding the hidden states output by the decoder.Table 9 : The changes in the representation space within MindMerger as well as its constituent models the multilingual model (mT5-xl), and the LLM (MetaMath-Llama-7B). Lrl., Hrl., and Avg. represent the average accuracy across low-resource languages, high-resource languages, and all languages, respectively. Referring to Shi et al. [2023] , we regard Bn, Th, and Sw as low-resourse languages, and regard the remaining languages as high-resource languages.Bn→En Th→En Sw→En Ja→En Zh→En De→En Fr→En Ru→En Es→En Lrl. Hrl. Avg. We further quantitatively analyzed the changes in the representation space of the models. Two metrics were employ to quantify space relationships: (1) Cosine similarity, indicating the similarity between the same query expressed in different languages, and (2) Recall@1, representing the percentage of top-1 retrieve results from a pool of 1000 English query that correspond to the same query.The experimental results are shown in Table 9 . multilingual model demonstrates the capability to map inputs in different languages to a language-agnostic space. Within this space, the average cosine similarity and Recall@1 from various languages to English reach as high as 0.93 and 99.6. On the contrary, LLM lacks this capability. The average cosine similarity and Recall@1 in the output space of LLM are only 0.07 and 16.8, respectively. The significant disparity in the capabilities to construct language-agnostic representations suggests the potential for LLM to be complemented by multilingual model.For MindMerger-Soft, consisting of multilingual model and LLM, the mapping layer generates a representation with an average cosine similarity of and an average Recall@1 of 99.6. This highly language-agnostic representation as input to LLM effectively helps LLM understand different languages. Compared with the huge difference between the language representations output when using only LLM, the average cosine similarity of LLM output in MindMerger-Soft increased from 0.07 to 0.47, and the average Recall@1 increased from 16.8 to 85.0.Table 10 : Translation results on Flores-101 dataset. spBLEU is used as evaluation metric.Bn→En Th→En Sw→En Ja→En Zh→En De→En Fr→En Ru→En Es→En Avg. In Table 10 , we used the translation dataset Flores-101 [Fan et al., 2021] as the evaluation set and presented the spBLEU performance of various versions of MindMerger-Soft (using MetaMath-Llama-7B as backbone) trained on mapping stage. In contrast to the poor performance of the same LLM, MetaMath-Llama, MindMerger-Soft greatly boost it translation capabilities. Remarkably, the performance of MindMerger, based on the M2M100-418M encoder part, even surpasses the M2M100-418M translation model itself. Although the encoder-only models mBERT and XLM-RoBERTa-large do not have translation capabilities, MindMerger can still learn translation capabilities based on them.We also compared the performance of MindMerger with the full fine-tuning method. We conducted experiments across four translation directions using two different sizes of training data, each covering translations between nine pairs of languages, with the comparison of average scores presented in Figure 5 . It can be observed that MindMerger has significantly enhanced the translation performance across all settings, including X-En, En-X, X-Zh, and Zh-X, resulting in improvements ranging from 3.7% to 7.5%. Although MindMerger primarily enhances the understanding capabilities of LLMs without directly improving their generation capabilities, its performance on translation task demonstrates that it can also benefit generation tasks. Given that understanding is the basis for generation, MindMerger represents a crucial step towards enhancing the multilingual generation capabilities of LLMs.En-X X-Zh Zh-X The benefits of translation data for MindMerger-Soft are significantly greater than full fine-tuning methods. Specifically, with the assistance of translation data, MindMerger-Soft improved the average accuracy by 5.5% (refer to B.1), while the improvements for full fine-tuning are 0.8%, 3.8%, and 1.0%, respectively. We compared the inference speed of MindMerger and the single LLM model QAlign on the MGSM dataset with a batch size of 1. As shown in the upper part of Table 12 , the average inference speed for MindMerger-Soft is even faster than QAlign, where MindMerger taking an average of 3.7s to infer a sample, compared to QAlign's 3.8s. This discrepancy arises because the text generated by QAlign is slightly longer than that produced by MindMerger-Soft, as indicated in the lower part of Table 12 . The complete experimental results are shown in Table 15 . It can be observed that the augmentation strategy is more significantly helpful in improving high-resource language reasoning capabilities than low-resource languages. Compared with the replacement version of MindMerger-Soft, the augmentation strategy improves the average accuracy by 1.1%, 2.1%, 3.2%, and 4.6% across highresource languages in the four experimental results. This suggests that, in contrast to the replacement strategy that neglects these capabilities, the augmentation strategy effectively takes advantage of them, thereby unlocking the full potential of LLMs. For low-resource languages, the augmentation strategy also brings improvements of 3.0% and 4.0% for MindMerger-Soft based on M2M100-1.2B, but on par with the replacement strategy for MindMerger-Soft based on mT5-xl. It could be the multilingual capabilities of mT5-xl on low-resource languages are significantly stronger than LLMs that it can completely replace LLMs, while the slightly weaker multilingual capabilities of M2M100-1.2B still need to be complementary to LLMs.Table 16 : Results on the MGSM dataset. Lrl., Hrl., and Avg. represent the average accuracy across low-resource languages, high-resource languages, and all languages, respectively. We regard Te, Bn, Th, and Sw as low-resourse languages, and regard the remaining languages as high-resource. In the above, we followed the previous work [Chen et al., 2023 , Zhu et al., 2024] and experimented with ten languages on the MGSM dataset. However, it is important to note that the MGSM dataset contains a total of eleven languages, including Telugu (Te). Therefore, we presented the evaluation results of MindMerger on the complete MGSM dataset in Table 16 . It can be observation that MindMerger-Soft is still the best model on Te, achieving a 10.0% improvement over xCoT [Chai et al., 2024 ] and a 20.0% improvement over LangBridge on that language, as well as an average 10.4% improvement over xCoT and an 8.8% improvement over LangBridge across all languages.Although our experimental results show that merging external and built-in language understanding capabilities can help improve the model's multilingual reasoning capabilities, it is unclear the effective boundaries of this mechanism. On one hand, a multilingual model with significantly weaker language capabilities compared to LLMs might not be able to assist LLMs effectively. On the other hand, LLMs may not be able to provide additional built-in knowledge for a multilingual model whose language capabilities are much stronger. Therefore, more experiments are needed to explore the effective boundaries of MindMerger. (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 


Decision: The paper presents a method to improve multilingual capability of LLMs by providing backbone LLMs the undecided query representation from multilingual model to LLM that is more optimized for reasoning. The proposed merging recipe includes two stages (1) mapping multilingual model into the backbone LLM and (2) teaching models to perform task given mapped multilingual model’s representation and backbone LLM’s representation. 

The reviewer (iF4J) have asked for more thorough evaluation against with the baselines, making input setting more equivalent and trying different base LLMs, and the proposed method consistently shows gains in these new settings. The approach is evaluated throughly on various benchmarks, including generation benchmark (suggested by reviewer q66P) and efficiency comparison (suggested by reviewer VBLh). These results strengthens the value of the proposed approach.

Reviews: review: paper_summary: paper_summary: The paper suggests incorporating an embedding block using external multilingual models to improve the models' understanding. Additionally, comprehensive experiments are conducted to demonstrate its efficacy.
strength_weakness: strength_weakness: The paper lacks a clear understanding of why the method can work. 
1. To my understanding, the Embedding Layer of the model merely converts the query from text space to the token/embedding space. This transformation may not be interpreted as "understanding the input" since the information hasn't been operated and combined by the attention layer. However, $X$ after the multilingual model has been "understood" by the model. Therefore, it does not make much sense to concatenate these two embeddings.
2. As $\tilde{X}$ contains the information of query q, $T$ includes duplicated information. This may be why the method can surpass baselines such as QAlign and Translate-En. It would be more convincing if baselines also input duplicated information.
3. This is further proved by Figure (4b), which illustrates that the mapping layer tends to obtain a unified representation for different languages and this is the same as QAlign. Therefore, it is confusing why this method can outperform QAlign.
4. The paper uses Llama2 as the base model. However, its multilingual performance is far from sota. Until submission, I believe Mistral, Gemma, and Llama3 have been released, whose multilingual performance is much better.
questions: questions: The details of the two training stages lack clarity, specifically regarding the utilization of translation and parallel data, as well as the precise training tasks involved.
limitations: limitations: The authors suggest a limitation on the integration of external multilingual models with the model's inherent understanding capabilities, a concern partly explored through the ablation study.
paper_summary: paper_summary: The paper propose a way to improve multilingual reasoning in LLM in own native languages without relying on pivoting methods such as translating to English. The method assume a hypothesis that LLMs have built in knowledge and reasoning abilities in a lower-resource language and not just common language like English. The method plugs in LLMs with the external multilingual models as embedding layers and introduce a training scheme build those new knowledge it. Experiments on MGSM show better accuracy across languages.
strength_weakness: strength_weakness: * The method is incomplete and ambiguous on the generation side. It makes sense that injecting multilingual embeddings at the bottom for the input may help the LLM understand the context better, but that does not necessarily make the LLM **generate** in those native language better, especially if the LLMs are weak in producing content in native languages.
   * It is no surprise that the tasks presented in the paper are all "understanding" tasks where the generation workload is light and trivial. Other "generation" tasks, such as summarization, writing, translation... may not fair well with this method. I would highlight if the authors could show good results for the method for these tasks.

* Missing baseline comparison: https://arxiv.org/abs/2306.11372 , https://aclanthology.org/2023.findings-emnlp.826.pdf

* Confusing word choices, such as "mapping stage", "merging" and "augmentation stage". These words often lead to different understanding rather than what are described in the paper.

* "Query translation task" and "query translation data" and many parts of the methodology are not explained clearly. It would be better if there are explicit visual examples or diagrams.
questions: questions: NA
limitations: limitations: The paper discussed limitations
paper_summary: paper_summary: This paper proposes a framework which merges LLMs with the external language understanding capabilities from multilingual models to improve multilingual reasoning performance. Specifically, the authors introduce a two-step training scheme:  they (i) train the framework to embed the multilingual model into the LLM by using translation data, and (ii) further train the framework to collaboratively utilize the built-in capabilities of LLM and the embedded external language capabilities.
strength_weakness: strength_weakness: - In Section 3.2, the authors mention that they use translation data and query translation task data generated from public translation models for the two-stage training; while in lines 182-192, it shows that they also use several English task datasets, such as MetaMathQA. It is unclear how they use these data. 
- Although the author mentioned that only a small number of parameters are trained during two-stage training,  the paper lacks a comparison with baselines in this apsect. Particularly, the proposed method requires the external multilingual model, which is expected to affect the inference efficiency.
- I know asking for new results might be unrealistic at this stage, but I am curious what would happen if you only used two-stage training for LLM without including an external multilingual model.
questions: questions: See above.
limitations: limitations: See above.
paper_summary: paper_summary: The paper introduces MergeMinds, a method that integrates LLMs with multilingual models to enhance reasoning capabilities across multiple languages. Through a two-step training scheme, MergeMinds effectively boosts performance in multilingual reasoning and language understanding tasks, particularly excelling in low-resource languages, as demonstrated by significant accuracy improvements on various datasets.
strength_weakness: strength_weakness: 1. From my perspective, the motivation of MergeMinds is extremely similar with LangBridge, both of which ultilize the mapping layer to map multilingual encoder to exsisting MLLM.
2. In fact, the work of this paper has already been explored on multi-modal large models a long time ago, such as InstructBlip's Qformer and LLaVA's two-stage alignment training.
questions: questions: 1. It appears that the Te language is not included in the MGSM. Could you explain the reason for this omission?
2. I noticed that some work similar to xcot was referenced but not included in the comparative table. What was the rationale behind this decision?
3. In my view, does this work adapt the paradigm from InstructBlip's Qformer and LLaVA's two-stage alignment training to a multilingual setting?
4. Missing Reference [1-3]

[1] MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization

[2] The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights

[3] Multilingual large language model: A survey of resources, taxonomy and frontiers
limitations: limitations: The authors have adequately addressed the limitations.

Comments: title: Thanks for the rebuttal
Reviewer Response to Authors' Rebuttal
(2/2) Response to Reviewer VBLh
(3/3) Response to Reviewer q66P
(2/3) Response to Reviewer q66P
(3/3) Response to Reviewer iF4J
(2/3) Response to Reviewer iF4J
comment: Thanks so much for so detailed reply in rebuttal to address my concern. Thay addressed my concerns, and I have raised my score accordingly.
Thanks for the author response, I pump up the scores for the effort!
Dear reviewer, 
could you check at the authors new experimental results and see whether it address your concerns, and update the score accordingly if they do?
Dear authors,

Thanks for your response and update with the results! I decided to raise the soundness score.
Dear reviewer -- could you see whether the author response addressed your concern, especially about your comment on how their method can improve *generation* of LLMs? they provided new experiments, as well as a comparison with baseline you proposed. If you can modify the scores based on the response, that'd be very helpful!
Thanks for your great response. I have no additional problems.
**W3. I know asking for new results might be unrealistic at this stage, but I am curious what would happen if you only used two-stage training for LLM without including an external multilingual model.**

- There are three types of training data in the paper: **(a)** translation data, **(b)** English task data, and **(c)** query translation task data. 
- We experimented with four dataset usage settings (b, c, b->c, c->b) to fully fine-tune Llama and compare the influence of using translation data for training in the first stage (a->b, a->c, a->b->c, a->c->b). 
- The experimental results are shown in the table below. Regardless of the way the training data is used, MindMerger has an average accuracy that is at least 6.2 higher than the fully fine-tuned method.
- Using translation data as the first stage training data can help improve performance on 3 out of 4 fully fine-tuning settings (a->c, a->b, and a->c->b).
- The gain of translation data for MindMerger is significantly greater than that of fully fine-tuning Llama. With the help of translation data, MindMerger improved the average accuracy rate by 5.5%, while the improvements for fully fine-tuning Llama were 0.8%, 3.8% and 1.0% respectively.


|  | Bn    | Th    | Sw    | Ja    | Zh    | De    | Fr    | Ru    | Es    | En    | Lrl.  | Hrl.  | Avg.   |
|--------------------------------------------------------------------------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|--------|
| c                                                                                    | 33.2  | 36.8  | 34.8  | 36.0  | 37.2  | 39.6  | 39.6  | 38.4  | 37.6  | 40.4  | 34.9  | 38.4  | 37.4   |
| a->c                                                                                 | 36.4  | 36.0  | 37.2  | 31.6  | 34.4  | 42.4  | 40.4  | 40.4  | 42.4  | 41.2  | 36.5  | 39.0  | 38.2   |
| b (MonoReason)                                                                       | 6.8   | 7.2   | 6.8   | 36.4  | 38.4  | 55.2  | 54.4  | 52.0  | 57.2  | 68.8  | 6.9   | 51.8  | 38.3   |
| a->b                                                                                 | 21.2  | 21.2  | 24.4  | 38.8  | 34.4  | 52.8  | 56.8  | 52.4  | 57.6  | 61.6  | 22.3  | 50.6  | 42.1   |
| b->c (MultiRason)                                                                    | 33.2  | 40.0  | 42.0  | 42.0  | 42.0  | 45.2  | 44.8  | 45.2  | 48.0  | 52.0  | 38.4  | 45.6  | 43.4   |
| a->b->c                                                                              | 33.6  | 40.4  | 40.4  | 41.2  | 37.6  | 38.4  | 41.2  | 44.4  | 41.6  | 47.6  | 38.1  | 41.7  | 40.6   |
| c->b                                                                                 | 40.8  | 46.4  | 45.2  | 44.8  | 50.4  | 52.8  | 52.8  | 52.0  | 56.4  | 59.6  | 44.1  | 52.7  | 50.1   |
| a->c->b                                                                              | 40.8  | 50.0  | 50.4  | 47.6  | 52.8  | 54.8  | 54.0  | 56.0  | 50.0  | 54.8  | 47.1  | 52.9  | 51.1   |


Thanks again to the reviewer for the experimental suggestions. We believe that these additional experiments will help further enhance the impact of our work.

Best wishes.
**W3. Confusing word choices, such as "mapping stage", "merging" and "augmentation stage". These words often lead to different understanding rather than what are described in the paper.**

- Thank you for your feedback, we will fine-tune the wording and description in the next version based on your comments.


**W4. "Query translation task" and "query translation data" and many parts of the methodology are not explained clearly. It would be better if there are explicit visual examples or diagrams.**

- Thank you for your feedback again. We followed the settings of QAlign to translate the query from English to non-English automatically as training data. Below are three examples of query translation task data:

|             | Augmentation stage (Query translation task data)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
|-------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Math        | **Input (Zh):** 伯特每天都填报纸上的每日填字游戏。他每两周就用完一支铅笔。平均而言，他用完一支铅笔需要1050个字。每个填字游戏平均有多少个字？ <br> **Input (En):** Bert fills out the daily crossword puzzle in the newspaper every day. He uses a pencil to fill out the puzzles every two weeks. On average, it takes him 1050 words to use up a pencil. How many words are in each crossword puzzle on average? <br> **Training Target:** If Bert uses up a pencil to fill out the puzzles every two weeks and it takes him 1050 words to use up a pencil ... 1050/14 = 75 words in each daily crossword puzzle on average. #### The answer is: 75 |
| Commonsense | **Input (Zh):** 酒后驾车的人可能会被指控什么? (A)惩罚 (B)逮捕 (C)车祸 (D)胡言乱语 (E)酒后驾驶 <br> **Input (En):** What is someone operating a vehicle likely to be accused of after becoming inebriated? (A) punish (B) arrest (C) automobile accidents (D) talking nonsense (E) drunk driving <br> **Training Target:** E                                                                                                                                                                                                                                                                                          |
| NLI         | **Input (Zh):** Premise: 她不太明白。Hypothesis: 事实上，她没有理解 <br> Input (En): Premise: She doesn’t really understand. Hypothesis: Actually, she doesn’t get it. <br> **Output:** Entailment                                                                                                                                                                                                                                                                                                                                                                                                      |


If you have any questions about our work, we look forward to discussing with you and, if possible, hope that you can improve our score appropriately.

Best wishes.
**W2. Missing baseline comparison XLT.**

- **We have discussed XLT in related work, lines 80-81**, which is a prompt-based approach that brings limited improvements over open source models such as Llama through carefully crafted prompts.
- Our model and the baselines we compared are all supervised fine-tuning methods. Although the work of XLT is inspiring, prompt-based methods are not in the same category works as our model.
- We added a comparison between XLT and our model. The experimental results are shown in the following two tables. Under the same base model, MindMerger has a huge lead over XLT.
- Due to page limitations, we consider adding the experimental results of XLT to the appendix.

|                     | Bn    | Th    | Sw    | Ja    | Zh    | De    | Fr    | Ru    | Es    | En    | Lrl   | Hrl   | Avg.   |
|---------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|--------|
| XLT (Llama2-chat)   | 3.2   | 5.6   | 4.0   | 9.2   | 12.8  | 16.0  | 17.6  | 14.0  | 19.6  | 19.6  | 4.3   | 15.5  | 12.2   |
| MindMerger (Llama2) | 50.4  | 52.8  | 57.2  | 54.4  | 53.6  | 61.2  | 57.6  | 60.8  | 58.4  | 66.8  | 53.5  | 59.0  | 57.3   |


|                       | Bn    | Th    | Sw    | Ja    | Zh    | De    | Fr    | Ru    | Es    | En    | Lrl   | Hrl   | Avg.   |
|-----------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|--------|
| XLT (Llama3-Instruct) | 27.6  | 53.6  | 32.4  | 52.0  | 58.0  | 53.6  | 39.6  | 50.8  | 53.2  | 52.8  | 37.9  | 51.4  | 47.4   |
| MindMerger (Llama3)   | 64.4  | 65.6  | 66.4  | 62.0  | 68.0  | 71.6  | 72.4  | 73.2  | 72.8  | 75.2  | 65.5  | 70.7  | 69.2   |
**Q1. The details of the two training stages lack clarity, specifically regarding the utilization of translation and parallel data, as well as the precise training tasks involved.**

- Thank you for your feedback, we will improve the description of training data in the next version of our paper.
- In the first stage (mapping stage), all tasks including Math, Commonsense, and NLI are trained using non-English to English translation data. The following is an example:

| Mapping stage (Translation data)                                                                                                           |
|--------------------------------------------------------------------------------------------------------------------------------------------|
| **Input (Zh):** 文档中没有元素属于该组时，该名称会被作为根结点显示在结构树上。                                                                                            |
| **Training Target:** It will be shown in the structure tree as a top node when there are no elements belonging to this group in the document. |

- In the second stage (augmentation stage), the input query of each task is translated into non-English to construct multilingual training samples, both English and non-English samples are used to train our model. Below are examples of training samples for three tasks:

|             | Augmentation stage (Query translation task data)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
|-------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Math        | **Input (Zh):** 伯特每天都填报纸上的每日填字游戏。他每两周就用完一支铅笔。平均而言，他用完一支铅笔需要1050个字。每个填字游戏平均有多少个字？ <br> **Input (En):** Bert fills out the daily crossword puzzle in the newspaper every day. He uses a pencil to fill out the puzzles every two weeks. On average, it takes him 1050 words to use up a pencil. How many words are in each crossword puzzle on average? <br> **Training Target:** If Bert uses up a pencil to fill out the puzzles every two weeks and it takes him 1050 words to use up a pencil ... 1050/14 = 75 words in each daily crossword puzzle on average. #### The answer is: 75 |
| Commonsense | **Input (Zh):** 酒后驾车的人可能会被指控什么? (A)惩罚 (B)逮捕 (C)车祸 (D)胡言乱语 (E)酒后驾驶 <br> **Input (En):** What is someone operating a vehicle likely to be accused of after becoming inebriated? (A) punish (B) arrest (C) automobile accidents (D) talking nonsense (E) drunk driving <br> **Training Target:** E                                                                                                                                                                                                                                                                                          |
| NLI         | **Input (Zh):** Premise: 她不太明白。Hypothesis: 事实上，她没有理解 <br> Input (En): Premise: She doesn’t really understand. Hypothesis: Actually, she doesn’t get it. <br> **Output:** Entailment                                                                                                                                                                                                                                                                                                                                                                                                      |



Thank you again for your comments. We hope our responses can resolve your concerns. We hope you can improve our score appropriately and are willing to have more discussions with you.

Best wishes.
**W3. This is further proved by Figure (4b), which illustrates that the mapping layer tends to obtain a unified representation for different languages and this is the same as QAlign. Therefore, it is confusing why this method can outperform QAlign.**

- This reflects that **our model has better capability to obtain unified representation than QAlign**.
- In Table 9 of the paper, we calculated the cosine similarity and top-1 retrieval recall between representations of the same query expressed in different languages to quantify the model’s capability to obtain a unified representation. The results show that our model significantly improves LLM’s capability to obtain a unified representation.
- We further compare the last layer pooling representation of QAlign and our method in the following table. It can be observed that although QAlign slightly outperforms MonoReason, the score of cosine similarity and recall@1 are still far behind our model, which proves that our model is better able to obtain unified representations.

 | Cosine Similarity | Bn->En | Th->En | Sw->En | Ja->En | Zh->En | De->En | Fr->En | Ru->En | Es->En | Lrl   | Hrl   | Avg.   |
|-------------------|--------|--------|--------|--------|--------|--------|--------|--------|--------|-------|-------|--------|
| MonoReason        | 0.08   | 0.07   | -0.08  | 0.11   | 0.13   | 0.04   | 0.12   | 0.05   | 0.09   | 0.02  | 0.09  | 0.07   |
| QAlign            | 0.07   | 0.04   | -0.06  | 0.11   | 0.14   | 0.10   | 0.18   | 0.06   | 0.15   | 0.02  | 0.12  | 0.09   |
| MindMerger        | **0.32**   | **0.32**   | **0.46**   | **0.46**   | **0.46**   | **0.55**   | **0.62**   | **0.52**   | **0.56**   | **0.37**  | **0.53**  | **0.48**   |


| Recall@1 (%) | Bn->En | Th->En | Sw->En | Ja->En | Zh->En | De->En | Fr->En | Ru->En | Es->En | Lrl   | Hrl   | Avg.   |
|--------------|--------|--------|--------|--------|--------|--------|--------|--------|--------|-------|-------|--------|
| MonoReason   | 0.4    | 1.8    | 0.2    | 5.8    | 12.6   | 11.5   | 47.3   | 44.9   | 27.0   | 0.8   | 24.9  | 16.8   |
| QAlign       | 2.8    | 4.9    | 0.5    | 12.3   | 26.1   | 21.2   | 61.7   | 37.3   | 41.0   | 2.7   | 33.3  | 23.1   |
| MindMerger   | **52.8**   | **76.0**   | **64.3**   | **91.6**   | **96.4**   | **98.0**   | **99.2**   | **98.1**   | **88.9**   | **64.4**  | **95.4**  | **85.0**   |



**W4. The paper uses Llama2 as the base model. However, its multilingual performance is far from sota. Until submission, I believe Mistral, Gemma, and Llama3 have been released, whose multilingual performance is much better.**

- In fact, **we have compared Mistral as well as the larger MetaMath-Llama-13B in Table 5**. The experimental results show that our method is also consistently effective on LLM with stronger multilingual capabilities.
- We further compare our method with baselines based on Llama3 in the following table. Our method still outperforms all baselines.

| Llama3, MGSM     | Bn    | Th    | Sw    | Ja    | Zh    | De    | Fr    | Ru    | Es    | En    | Lrl   | Hrl   | Avg.   |
|------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|--------|
| MonoReason       | 40.4  | 53.2  | 32.0  | 53.2  | 58.0  | 64.4  | 67.2  | 67.6  | 69.2  | 76.4  | 41.9  | 65.1  | 58.2   |
| MonoReason       | 41.2  | 55.6  | 29.2  | 50.0  | 56.8  | 62.4  | 64.4  | 66.4  | 66.0  | 74.8  | 42.0  | 63.0  | 56.7   |
| MultiReason-Lora | 52.0  | 62.0  | 52.4  | 58.8  | 64.8  | 66.0  | 68.8  | 72.4  | **74.0**  | 75.2  | 55.5  | 68.6  | 64.6   |
| MultiReason-SFT  | 39.2  | 47.6  | 48.0  | 48.0  | 46.4  | 52.8  | 48.0  | 47.2  | 52.8  | 57.2  | 44.9  | 50.3  | 48.7   |
| QAlign           | 50.0  | 59.6  | 56.0  | 54.0  | 58.4  | 62.4  | 63.6  | 69.6  | 70.8  | 73.2  | 55.2  | 64.6  | 61.8   |
| LangBridge       | 49.2  | 51.6  | 56.4  | 48.0  | 54.8  | 69.6  | 68.4  | 67.6  | 69.6  | **78.0**  | 52.4  | 65.1  | 61.3   |
| Translate-En     | 52.0  | 41.6  | 58.0  | 54.8  | 53.2  | 63.6  | 60.4  | 55.6  | 67.6  | 76.4  | 50.5  | 61.7  | 58.3   |
| **MergeMinds**       | **64.4**  | **65.6**  | **66.4**  | **62.0**  | **68.0**  | **71.6**  | **72.4**  | **73.2**  | 72.8  | 75.2  | **65.5**  | **70.7**  | **69.2**   |