Why Not Transform Chat Large Language Models to Non-English?

Authors: Xiang Geng, Ming Zhu, Jiahuan Li, Zhejian Lai, Wei Zou, Shuaijie She, Jiaxin GUO, Xiaofeng Zhao, Yinglu Li, Yuang Li, Chang Su, Yanqing Zhao, Xinglin Lyu, Min Zhang, Jiajun Chen, Hao Yang, Shujian Huang
Primary area: foundation or frontier models, including LLMs
Keywords: Large Language Model, Low Resource Languages, Knowledge Transfer, Catastrophic Forgetting

Abstract:
Large language models (LLMs) excel in various tasks, but their performance in non-English languages remains limited due to imbalanced training data. To address this limitation, we explore how to transform chat LLMs to non-English. Chat LLMs offer more advanced capabilities than base LLMs, such as multi-turn conversation and alignment with human preferences. However, transforming chat LLMs presents greater challenges than base LLMs. First, how can we effectively transfer advanced capabilities without their supervised data in target languages? Second, how can we prevent the original capabilities from catastrophic forgetting without replaying their training procedure in English? We target these issues by introducing a simple framework called TransLLM. TransLLM divides the transfer problem into some common sub-tasks with the translation chain-of-thought, eliminating the need for complex training data. More importantly, TransLLM uses two key strategies to prevent catastrophic forgetting: Low-rank adaptation, which preserves the original LLM parameters during training, and recovery KD, which utilizes data generated by the chat LLM itself to recover the original knowledge from the frozen parameters. Experiments conducted across five languages and three LLMs demonstrate the superiority of TransLLM. Notably, TransLLM outperforms GPT-4 in Thai, demonstrating higher levels of helpfulness and safety, using just 8B parameters and publicly accessible data. Our analysis demonstrates how recovery KD combined with LoRA helps mitigate catastrophic forgetting.

Decision: reject
Reviewer scores: 8, 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730464859605

thread_type:
official_review

<thread_object>
creator: reviewer_t1tu

<content>
summary:
This paper focus how to transform a chat LLM from English to non-dominant languages without supervised data. The authors propose a framework named TransLLM,  which incorporate two stage training. In the pretraining stage, the proposed method pretrain the model using monolingual data and parallel translation data to improve the model capabilities on target language. Then, the authors introduce translation COT to transfer advanced chat capabilities from original LLM through SFT, utilizing recovery KD combined with LoRA to mitigate catastrophic forgetting during the transformation.

strengths:
1. **Superior Performance** The experimental results on multiple benchmarks demonstrate that the proposed method outperforms ChatGPT and other strong baselines in both human evaluations and assessments using LLMs as judge.
2. **Solid Experiments** The authors conduct extensive experiments across various languages and LLMs, and validate the effectiveness of each module through tailored ablation studies.

weaknesses:
1. **Lack of a baseline that directly employs multilingual supervised fine-tuning (MSFT)** The authors use google translation to construct TCOT data. Therefore, it is necessary to compare the performance of the baseline that directly conducts MSFT using google-translated SFT data with the proposed method. Compared to this simple baseline, TCOT clearly requires higher inference overhead, so its performance improvements must be significant to be acceptable.
2. **Lack of an ablation experiment to demonstrate the necessity of maintaining the original model parameters until the entire training process is complete**
There are two stages in the entire training process: pretraining and transfer fine-tuning. According to the description in lines 418-423 (Table7 Line4), the authors merged LoRA parameters after the pretraining stage and then performed full-parameter tuning during transfer stage. This ablation experiment only shows that LoRA technologies can mitigate catastrophic forgetting to some extent, which has been discussed in previous work, but it does not prove the necessity of preserving the original LLM parameters throughout the training process. To support this point, the authors should conduct an additional ablation experiment where parameters are merged after the pretraining stage and then fine-tuned using LoRA in transfer stage, comparing it with a version that does not combine parameters until the end of the two-stage training.
3.  I provide some minor concerns and suggestions in the following section.

questions:
# Questions
1. **Lack of implementation details about baseline *NLLB-bridge (Line 353)*** Specifically, how dose NLLB-3B as the bridge between Llama2 and the TH language? In the training stage or the inference stage? Does it utilize TCOT, or does it simply translate the response?

# Suggestions
1.  **Consider using the term "original chat LLM" or a similar alternative instead of "target chat LLM"(like in line 185)** While I understand that the authors use this term to refer to the chat LLM they aim to transform, the term "target" is often associated with "target language" in multilingual and translation scenarios, which refers to the language into which content is being transformed or translated. This may lead to potential misunderstandings and confusion.
2. **Consider conducting experiments with other open-source LLMs like Mistral and Gemma** Although the authors have conducted extensive experiments on Llama2/3/3.1, adding additional experiments with different series of LLMs could further demonstrate the generalizability of the proposed framework.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730701328303

thread_type:
official_review

<thread_object>
creator: reviewer_pycd

<content>
summary:
The paper introduces two strategies for transforming existing English models to include more languages: 1) low-rank adaptation and 2) recovery knowledge distillation. By experimenting with Llama2/3-8b, they show models have good performances on translated version of AlpacaEval, xCOPA and AdvBench.

strengths:
The strategies introduced in this paper provide a helpful guide on how to transform a pretrained English-centric chat model to include an additional language by leveraging existing multilingual data, additional translation training, LoRA and knowledge distillation to obtain good performances on standardized benchmarks.

weaknesses:
- The evaluation benchmark is mainly obtained from translation. The training strategies involved decide that the chat models will mainly rely on their reasoning and generation capabilities in English, this is even further enhanced by training on translation tasks, therefore they can consequently perform well on the translated benchmarks. I wonder if the authors have tested on non-translated benchmarks?
- The evaluation is done by comparing with GPT4/ChatGPT via using GPT4 as a judge despite the authors mentioning that GPT4 and human annotators exhibit high consistency in the rating process. I wonder if there are accuracy measures for evaluation datasets like XCOPA instead of comparison with the GPT4 to more accurately observe the model’s performance. It would also be helpful to provide example generations to observe models’ responses more qualitatively after adaption for more thorough error analysis.

questions:
Since the strategies mentioned in the paper can effectively reduce catastrophic forgetting. Have the authors tried to transforming the chat models to more than one languages at the same time and compare their performances across languages? Also does the model still maintain high performance in English after the transformation?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730720692192

thread_type:
official_review

<thread_object>
creator: reviewer_gsyr

<content>
summary:
The paper introduces TransLLM, a framework for adapting large language models (LLMs) that is predominantly trained on English data to other langages.

TransLLM use low-rank adaptation (LoRA) to train monolingual LLMs through three steps: (1) target language pre-training, (2) translation pre-training, and (3) transfer fine-tuning.
The last step uses multiple data format including recovery knowledge distillation (KD) data, translated chain-of-thought (TCOT) data, and bi-direction translation to update LoRA parameters.

The approach is evaluated across five languages and different benchmarks, demonstrating improvements over baselines, surpassing GPT-4 in Thai for helpfulness and safety.

strengths:
+ The paper addresses an important challenge in adapting LLMs to non-English languages, which is crucial for addressing the multilingual generalization problem of LLMs.

weaknesses:
+ The paper is an engineering-focused LLMs training project without introducing any novelty or new methodology for the target problem.
+ The paper is poorly written with contextually unclear terminologies (chat LLMs, Knowledge Distillation, Chain-of-thought).
+ The paper focus on GPT-4 as the main baseline, while GPT-4 multilingual capacity is not properly documented. A multilingual LLM baseline would be more appropriate.
+ The main evaluation is heavily focused on Thai, with limited results for other languages.

questions:
+ Can the proposed method applied to any LLM instead of chat LLM in particular?
+ Is there any distillation of knowledge involved in the training process?
+ Can you elaborate on the "chain-of-thought" in TCOT data format?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730788411857

thread_type:
official_review

<thread_object>
creator: reviewer_vjuz

<content>
summary:
This work proposes a method to equip Chat LLMs with target language capabilities, such that the chat capability in English can be transferred to target languages that the original LLM is not good at or completely lacking. To achieve this, this work introduces a training scheme, aiming at objectives: 1) learn to use translation in the chain-of-thought process (TCOT), serving as the anchor for reasoning in target languages; 2) preservation of original knowledge to avoid catastrophic forgetting.

Specifically, this work builds QA pairs in target languages by translating from English training resources, which are used for TCOT finetuning and translation finetuning. To avoid catastrophic forgetting, the training also includes original QA pairs in English, combined with LoRA finetuning instead of full finetuning, to preserve the original knowledge. Additionally, this work also observes that pretraining on target languages before any finetuning could contribute significantly.

Experiments on conducted on multiple instruction following, reasoning and safety datasets, examining five languages to transfer: Thai, Arabic, Portuguese, Telugu, Turkish. Results show that the transferred LLMs could outperform ChatGPT and other baselines, including Chat Vector (Huang et al., 2024) and translation-as-a-bridge.

strengths:
- The idea to use translation in CoT for target language transfer is intuitive and well-motivated. Experiments show that the overall training scheme is effective, outperforming all baselines except for GPT-4.
  
- Ablation studies and analysis are performed, providing insights on detailed decisions, e.g. distilling answers from the LLM itself rather than more powerful GPT-4, the importance of target language pretraining, and the effectiveness on preserving original knowledge in the source language.
  
- Human studies are performed to evaluate the resulting chat quality in target languages.

weaknesses:
Since this work builds new training data by translating from English resources to target languages, it would be important to decouple the improvement from the new data vs. the new training scheme. Several simple baselines could be added:

- Removing TCOT, only performing: 1) target language pretraining; 2) finetuning directly on target language QA pairs. The resulting LLMs may forget original knowledge, but it would be insightful to see the performance on target languages directly.
  
- For the above process, also add QA pairs in English during finetuning to preserve original knowledge, so to examine how much TCOT really contributes in the transfer process.

The above analysis could increase the significance of this work.

questions:
For the English training resources, is the translation performed for each target languages, with the same amount of translated pairs?
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents a method to enhance Chat LLMs with capabilities in target languages that are not well-supported by the original English-centric models. This is achieved through a framework called TransLLM, which employs a strategic training scheme involving several key components. However, the limitations of the paper are 1)  The paper uses GPT-4 as the primary baseline, despite the lack of comprehensive documentation on its multilingual capabilities. It would be more suitable to compare against a multilingual LLM baseline. 2) The main evaluation is heavily focused on Thai, providing limited results for other languages. 3)The proposed training approach offers limited novelty in addressing the target problem. 4) The formatting of the text and tables needs to be improved to make it easier to follow.