Adapting Chat Language Models Using Only Target Unlabeled Language Data
Authors: Atsuki Yamaguchi
Terufumi Morishita
Aline Villavicencio
Nikolaos Aletras
Venue: Accepted by TMLR ()

Abstract: Vocabulary expansion (VE) is the de-facto approach to language adaptation of large language models (LLMs) by adding new tokens and continuing pre-training on target data. While this is effective for base models trained on unlabeled data, it poses challenges for chat models trained to follow instructions through labeled conversation data. Directly adapting the latter with VE on target unlabeled data may result in forgetting chat abilities. While ideal, target chat data is often unavailable or costly to create for low-resource languages, and machine-translated alternatives are not always effective. To address this issue, previous work proposed using a base and chat model from the same family. This method first adapts the base LLM with VE on target unlabeled data and then converts it to a chat model by adding a chat vector (CV) derived from the weight difference between the source base and chat models. We propose ElChat, a new language adaptation method for chat LLMs that adapts a chat model directly on target unlabeled data, without a base model. It elicits chat abilities by injecting information from the source chat model. ElChat offers more robust and competitive target language and safety performance while achieving superior English, chat, and instruction-following abilities compared to CV.

Published in Transactions on Machine Learning Research (09/2025)
Adapting Chat Language Models Using Only Target Unlabeled Language Data
Atsuki Yamaguchi University of Sheffield
ayamoguchil @shefficld. uk
Terufumi Morishita
Hilachi, Ltd.
Aline Villavicencio
a.llllericencio@skefffflll......
University of Exeter University of Sheffield
Nikolaos Aletras University of Sheffield
n.aletma@sheffeldd.cm
Reviewed on OpenReview: ht tp o/ en revi ew .ne /f um d= 6I do IK ow
Abstract
Vocabulary expansion (VE) is the de-facto approach to language adaptation of large lan- guage models (LLMs) by adding new tokens and continuing pre-training on target data. While this is effective for base models trained on unlabeled data, poses challenges for chat models trained to follow instructions through labeled conversation data. Directly adapting the latter with VE on target unlabeled data may result in forgetting chat abilities. While ideal, target chat data is often unavailable costly to create for low-resource languages, and
not always effective. To address this issue, previous work proposed using a base and chat model from the same family. This method first adapts the base LLM with VE on target unlabeled data and then converts it to a chat model by adding a chat vector (CV) derived from the weight difference between the source base and chat models. We propose ElChat, new language adaptation method for chat LLMs that adapts chat model directly on target unlabeled data, without a base model. It elicits chat abilities by injecting information from the source chat model. ElChat offers more robust and competitive target language and safety performance while achieving superior English, chat, and instruction-following abilities compared to CV
alternatives
1 Introduction
Vocabulary expansion (VE) is the de-facto approach to adapting large language models (LLMs) a target language (Cui al., 2024 Fujii al., 2024 Choi al. 2024). It typically consists of two main steps: (i) new tokens are added to the model vocabulary expanding the input embedding and output head matrices; and (ii) continual pre-training
target data to learn the input and output embeddings of the new tokens (Cui
ct al. 2024 Fujii et al. 2024 Choi et al. 2024 Tejaswi et al. 2024 Mundra et al.) 2024 inter alia). VE is important because LLMs including chat models often perform poorly in languages underrepresented in the training data (Geng et al., 2025 Huang et al. 2024). Morcover, target language tokenization suffers
from overfragmentation due to the heavy reliance on data and vocabulary from particular languages (e.g.,
Quu code available on GitHub
The adapted models available on Hugging Face Hub

---

Published in Transactions on Machine Learning Research (09/2025)
English), resulting to more inference steps especially in low-resource languages (Ahia et al. 2023 Petrov
et al. 2023 Ali et al., 2024p. Consequently, VE. is necessary step to mitigate this issue and achieve crucial inference speedups. While the vocabularies of frontier LLMs are often large, e.g., 152K for Qwen2.5 (Yang ct al. 2024 and 128K for Llama 3.1 (Dbbey et al. 2024h, they still suffer from this overfragmentation in underrepresented languages. This means such languages often require substantially more inference steps than their high-resouree counterparts. For example, processing a text sequence Amharic requires 3.48x more inference steps using the default Qwen2.5 without VE.
While VE is effective for base models trained on unlabeled data, its application poses significant challenges when the LLM at hand is chat model trained to follow instructions through labeled conversation data. Ideally, we need access to target chat data to effectively adapt chat models. However, this is often unavailable or costly to create for low-resource languages, inchuding the acquisition of human feedback (Huang et al. 2024). Alternatively, machine-translated chat data are not consistently effective (Tao et al. 2024
To address this issue, Huang et al. (2024) proposed chat vector (CV), method to obtain a chat model in the target language with access to target unlabeled data only. CV first adapts the base LLM with VE target unlabeled data and then converts it to a chat model by adding chat vector derived from the weight difference between the source base and chat models. However, this requires access to base and chat models from the same family that might not always be available, hindering its applicability. For example, the Phi-3 (Abdin et al.) 2024ap and Phi-4 (Abdin et al.) 2024b) do not provide base models due to safety reasons Similarly, Velvel & EXAONE-3.5 and Trillion (Han et al. 2025bb models are available only as a chat model. Crucially, it is completely to the discretion of developers to decide whether they publish both base and chat variants.
In this paper, we propose ElChat, new language adaptation method for chat LLMs that adapts chat model directly on target unlabeled data, eliminating
CHAT VECTOR
VE
Add
the need for a base model (Figure D We hypothesize that direct adaptation of a source chat model with VE on target unlabeled data negatively impacts its chat and cnaerucctio-folooiing abilities by altering its para- metric knowledge. However, posit that these can still be recovered. For this purpose, ElChat leverages information from the source chat model to elicit chat abilities through two key mechanisms. First, we em- ploy model merging to integrate distinct parametric knowledge from the source and target models (Worts-
Unlabeled target language data
Base LM
Adapted
Adapted Base LM Base chat abilities)
Chat LM
Chat Vector
ELCHAT
Adapted Chat LM (wv degraded chat aallitics)
Adapted Chat LM [w/ chat abilities)
man et al. 2022 Yadav et al. 2023 Yuu al.] 2024 Goddard et al. 2024). We hypothesize that
model merging helps restore the chat and instruction- following abilities of the source model while preserving the target language performance achieved by the tar- get model. Second, we reuse the weights of special tokens from the source model. For example, tokens that mark the start of conversation (urn should be ability
Inject Chat LM abitities
Model merging
Copy special token weights
Figure 1: Chat LLM language adaptation with Chat Vector (Huang et al. 2024) and ElChat (ours). Note that a base LM in this paper refers to an LM pre- trained on unlabeled data without any further post- training. A chat LM, on the other hand, is a base model that has been further supervised fine-tuned on labeled conversational data, enabling it to follow instructions.
crucial for activating the cnstruction-folloviii as they are used to structure raw input into chat for- mat. However, direct adaptation on target unlabeled data may degrade their functionality as they are mod- ified during VE. To mitigate this, we copy these token weights directly from the source model to the target model.
2

---

Published in Transactions on Machine Learning Research (09/2025)
We investigate the efficacy of ElChat by experimenting with two popular chat models across seven typo- logically diverse languages. Our evaluation includes safety, chat, and cestruction-ollowiiin performance. Additionally, we also assess target and source language task performance and target language inference speed. Our key contributions are as follows:
We propose ElChat that adapts a chat model directly on target unlabcled data, climinating the need for (i) a base model and (ii) larget chat data.
EIChat achieves better chat and cestruction-folloiing abilities and source language performance than CV. It is also competitive and more robust (i.e., consistently outperforming the source chat model) in the target langnage and safety tasks compared to CV 55.2 5.3 5.1
Despite model modifications, FlChat achieves comparable larget inference speedups across models and tasks, matching the performance of the adapted VE and CV models ($6).
2 Related Work
2.1
Cross-lingual Vocabulary Adaptation
The most popular approach to adapting LLMs a target language by expanding their vocabulary (VE)
with tokens from the larget language (Balachandran 2023; Larcher et al., 2023 Pipatanakul al.,
2023
Lin et al., 2024 Cui et al. 2024 Kim et al. 2024 Fujii et al. 2024 Choi et al. 2024 Nguyen et al.
2024
Tejaswi et al. 2024 Mundra et al. 2024).
Other methods to language adaptation include full or partial vocabulary replacement with a new larget vocabulary (Ostendorff & Rehm
2023
Csaki et al. 2023
Da Dalt et al., 2024 Remy et al. 2024 Yamaguchi
ct al. 2024a Dobler & de Melo| 2024]
Cahyawijaya et al. 2024). hypernetwork for tokenizer transfer (Minix-
hofer et al.] 2024). and adapters for vocabulary alignment (Han al. 2025a). Our work focuses on VE
it has been widely used recently for mostly base L.I. adaptation in languages such as Chinese, Japanese,
Korean, and Persian (Cui et al. 2024 Fujii et al.
2024
Choi et al.
2024 Mahdizadeh Sani et al. 2025 inter
alia.).
2.2 Language Adaptation of Chat Models
Recent work has proposed developing chat models in target language from source base models. For example, Toraman (2024) and Zhao et al. (2024a) apply VE to base models using target language chat data, consisting of 52k samples and 500M tokens, respectively. Bandarkar et al. (2025) also adapt base models using 30-40k target language chat data. Their approach also adapts a task-specific (i.c., math) model on 200k English math samples, followed by merging the two models to enhance math performance in the larget language. Alexandrov et al.] (2024) iteratively merge models trained on subsets of available target language data to effectively mitigate catastrophic forgetting. Their method first adapts base models with continual pre-training (CPT) on target unlabeled language data, followed by instruction tuning on target language chat data samples. However, this approach requires substantial target language data (at least 50B tokens of unlabeled data for CPT, and 78K samples of target language chat data) to ensure that each subset contains sufficient information for effective adaptation. A different approach, proposed by Tao et al. (2024), involves merging two base models: (1) one supervised fine-tuned on 162k Fnglish data samples, and (2) another trained on at least cight billion tokens of target unlabeled language data. However, it still relics on the availability of base model. [Geng al. (2025) propose adapting source chat models directly through a multi-stage training approach. This method involves target unlabeled data and transfer fine-tuning (i.e., supervised fine-tuning tasks using translated target chat data.)
The main limitation with this line of work is that it requires access to larget chat data (real or translated). typically in large volumes. Chat data often unaviilable or costly to produce for low-resource languages, while machine-translated chat data is not always effective for adaptation (Tao et al., (004)). For example, Burmese, one of our experimental languages, consists only of 472 manually annotated instruction samples in the Aya Dataset (Singh et al. 2024). This is insufficient for direct application of VE, as its typical data
3

---

Published in Transactions on Machine Learning Research (09/2023)
M
VOCABULARY EXPANSION
SOURCE AND TARGET
CHAT MODEL MERGING
1. Train auxiliary
2. Add top-k frequent tokens
Layer-wise
Merged
tokenizer
marging
adapted
Source
D
Ausiliary tokenizer
v Non-overlapping auxiliary
unlabeled
VUk
SLERP W.
3. Resize & initialize new token weights
4. Continual pre-training
COPYING SPECIAL TOKEN WEIGHTS
Adapted
E
E,
Adapted CM
(w/ chat
E
E
<innstarr <lim_and|> system user assistant
abilities)
Copy
k:
M
Figure 2: ElChat: three-step adaptation process overview.
requirements are in the order of millions of tokens (Tejasii et al. 2024 making these methods not applicable in such scttings.
Huang et al. (2024) assumes a language adaptation setting where there no access to target chat data (real or translated). This is a more realistic scenario for low-resource languages. For example, Burmese has only 172k target unlabeled language data in MADLAD-400 (Kudugunta et al., 2023) available for adaptation. Their proposed CV method obtains a larget chat model using source base and a source chat, model. This approach to chat LLM adaptation is the closest to our work.
3 ElChat: Eliciting Chat and Instruction-following Abilities
Similar to Huang et al. 20044) we aim to adapt chat LLM to target language, assuming that we only have access to target unlabeled data and no target chat data (real or translated) at all. Unlike Huang et al. (2024), our goal is to remove the dependence on a base model that might not always be available and adapt a chat model directly on target unlabeled data with VE.
To achieve this, we introduce FlChat that consists of three steps: (i) VE on the source chat model using target unlabeled data obtain an initial adapted target chat model; (ii) merging the source and target models; (iii) injecting information from the source to the target by copying tokens that specific to chat and instruction-following capabilities. Figure visualizes these steps. While the individual components of these are established techniques, the key novelty of ElChat resides in its strategic combination to overcome the dependence on the base model.
We expect that modifying the parametric knowledge of the source model by training on target unlabeled data in step (i) will improve its target language skills by updating specific areas of the network but negatively impact its chat and instruction-followin
abilities due catastrophic forgetting. To remedy this, hy- elicit the latter through (ii) model merging (Wortsman al. 2022 Yadav et al. 2023 Yu ct al. 2024 Goddard et al.] 2024). It allows the integration of distinct parametric knowledge from the source and target models without incurring any additional training costs, i.c., chat abilities from the source and target language knowledge from the target. Finally, in step (3), we restore the representation of special tokens used in the chat template. We assume this will guide the target model in effectively responding to user instructions by transferring this information from the source.
pothesize that
Vocabulary Expansion (VE). Given the sole availability target unlabeled data, we apply VE on the source chat LLM by expanding its input and output head matrices with new tokens, followed by CPT on the target data. This process follows the standard protocol adopted in et al. (022) for resizing and initializing these matrices.
Tejaswi et al. (2024) and Mundra
4

---

Published in Transactions on Machine Learning Research (09/2025)
More specifically, given target unlabeled data D and source chat model M, with initial vocabulary V:
1. An auxiliary target language tokenizer is first trained on D. This tokenizer utilizes the same under- lying tokenization function (e.g., byte-level BPE in this paper) as Ma.
From this auxiliary tokenizer, we then select the top-k most frequent tokens (e.g., k 10K by default, as in Tejaswi et al. (2024)) that do not overlap with V but present in the auxiliary tokenizer's vocabulary. These selected &t tokens are then added to V.
To accommodate these & new tokens in Ms; its input E, E R and output head O. E R matrices are resized. They become E, E [R([++)) and O E - respectively, where II the newly added tokens are then initialized using mean initialization (Yao el al. 2021 a popular and simple, yet effective method in VE (Fujii et al. 2024 Tejaswi et al.) 2024 Mundra et al. 2024h. The weight of each newly added token is initialized as the average embedding or language modeling head weight of their corresponding source tokens, obtained using the corresponding source tokenizer.
denotes the hidden dimensionality of M,. The weights corresponding
After the above procedure, M, undergoes continual pre-training on D using a causal language modeling objective. A key difference between chat and base models is that the former uses a chat template. This includes specific roles (c.g., user, system, or assistant) and has a placcholder for message text in the prompt (see Appendix A for details). For example, chat template might structure input follows:
<| <lim_start starttaattie (syysem
You are helpful assistant. < sim_start|>un />ueer
What is the capital of the US?< im end/>
[lim_start/ Dassisstat.
During CPT, we remove the default chat template the model to support unlabeled data because the unlabeled data typically lacks these explicit role annotations. During inference, we append it to task-specific prompt templates (see Table 3 for spsk-specific prompts).
Source and Target Chat Model Merging (Merge). After VE, merge the source and target chat models. We employ popular merging method: spherical linear interpolation (SLERP) (Goddard et al. 2024h to merge each layer of the source and adapted models P This process excludes the embedding and language modeling head from merging because the source and target models use different vocabularies.
Given the source chat model M, and the target-adapted chat model Mr (which has undergone VE), the weights of cach corresponding layer (e.g., self-attention and foed-forward layers) are merged. For each such layer 1, let w!( denote the weight matrix from M,, and w! denote the corresponding weight matrix from m then computed using SLERP:
Mr. The merged weight matrix, denoted w(!)
m w = SLERP(W(()
.))
where a [0, 1] is an interpolation coefficient. This a serves as a hyperparameter controlling the balance between M, and M, learned representations, where = 0 corresponds to using only the weights of M, and = 1 to using only those of M,.
Copying Special Token and Language Modeling Head Weights (Copy). Special tokens used in a <im_start> in Qwen2.5 5 to represent the start of a turn) should be critical in supporting
chat template (e.g., <im start> chat and festruction-oolloiing abilities of model (see Appendix A for a full list of special tokens). Although the embedding and language modeling heads are excluded from merging due vocabulary differences, leaving
We also test linear merging WWrtsman al. 2022 yielding similar results (see Appendix This suggests the robustness of merging approach regardless of the specific interpolation method. Nonetheless, primarily utilize SLERP streamline our experiments due its superior grained control for merging, even though it may be empirically superior tasks (Goddard al. 2024.
5

---

Published in Transactions on Machine Learning Research (09/2025)
them unchanged may not be optimal for eliciting the chat and instruction-folloo following abilities of the adapted chat model. Hence, we copy all the special token weights from the source model to the adapted model.
Specifically, let S be the set of token Dss corresponding to the special tokens defined by the chat template of MM. The input embedding matrix and the language modeling head of M₁ are updated as follows:
For cach special token ID x E S, the embedding vector for token in Mr. E, E [R(M-+))HH is updated by copying the corresponding vector from M, 's input embedding matrix (E,): Ex Es where Eli,;] denotes the embedding vector for token ID i.
Similarly, the language modeling head weights corresponding to token x in Mr O2 E ( are updated by copying the corresponding vector from M,: O,[;, < Osli; æt,, where Olt,jj] denotes the column of weights for token ID j in the language modeling head.
4 Experimental Setup
This section describes the experimental setup in this paper. More details are listed in Appendix A
4.1 Source Models
We use two popular chat models as source: Qwen2.5 7B YYng et al., 2024); and Llama 3.1 8B (Dubey et al. 2024), across experiments. Additionally, to ensure consistency in our analysis, we also incorporate the stato-of-the-art chat model Qwen3 14B Yang ct al. 2025 The corresponding results and analysis for
Qwen3 are presented Appendix C.3
4.2 Target Languages and Adaptation Data
We experiment with the following seven typologically diverse languages, assuming that they are likely to be underrepresented compared to English in the pre-training data of the source models, or entirely absent: Amharic (Afroasiatic), Bengali Sindo-European, Burmese (Sino-Tibetan), Gujarati Endo--uuopeaan, Sin- hala (Indo-Furopean), Tamil (Dravidian), and Telugu (Dravidian). The ratio of training data in each model for each source base and chat model has not been explicitly disclosed (Appendix C.2 Note that we do not consider Latin script target languages as they are less likely suffer from overfragmentation and usually benefit less from VE [Yamaguchi et al.) 2024bb Tejaswi et al. 2024). Due to computational constraints, we use Qwen3 for Amharic, Bengali, and Telugu only
For the CPT part of VE, use MADLAD-400 (Kudugunta et al. 2023), which consists of highly-filtered document-level samples sourced from CommonCrawi and randomly sample 250K language-specifie docu- ments for each language as the target unlabeled data!"
4.3 Continual Pre-training
Following Remy et al. (2024). we train the embedding, LM head, and the top and bottom two layers of a source model. This approach aims to calibrate only the parts closely related to the encoding and decoding of the target language [Wendler et al. 2024 Tang et al. 2024 Zhao et al. 2024b), minimizing changes to the source model while allowing cost-effective tuning.
4.4 Baselines
We compare ElChat against the following baselines:
Off-tho-shelf base (Basc) and chat (Chat) models without target language adaptation.
Ammaric is selected because it shows the most significant speedup gains in target language generative tasks with Qwen2.5 and Llama 3.1 (Tahles 17 and Bengali and Telugu chosen as they are the only langnages covered by MGSM. For languages with than 250K documents (f.e., Amharic and Burmese), we use the full articles.

---

Published in Transactions on Machine Learning Research (09/2025)
Base and Chat models adapted using standard VE, denoted by Base+VVB and Chat+ respec- lively. Note that the latter uses a chat template in inference (see $3
CV proposed by Huang al. 20224) augmenting Base+ with chat vector using Base and Chat.
For reference, we also experiment with adapting Chat and Base using only CPT on the same target language data without VE (i.e., no inference speedup in target language). We provide the results and analysis of these CPT-only models in Appendix C.1
4.5 Evaluation Tasks
We evaluate the efficacy of ElChat in safety, chat, and instruction-following performance, and target and source language performance.
Safety, Chat, and Instruction-Following Following Cahyawijaya
(2024). we conduct safcty eval-
uation on larget language translated data including TRUTIFULQQ (Lin et al. 2022 ToxIcGEN (Hartvigsen
et al. 2022), and IMPLICITHATE (EISherief et al. 2021). We also measure chat and Instruction-followe
abilities in the source language (English) using EVELL (Zhou et al. 2023). GSM8K (Cobbe et al. 2021
as multi-turn few-shot, and MT-BENCH (Zheng et al. 2023).
Furthermore, we measure the performance on
English ALPACAEVAL v2.0 Li et al. 2023 Dubois et al. 2024 for additional analysis.
Target language evaluation is challenging for instruction-followingg and chat tasks due to the limited data availability. LLM-as-aJJudge (Zheng et al., 2023) is also unstable according to Azime et al. (2024) in low- resource languages. Hence, use multi-turn MGSM (Shi et al. 2023) for target language evaluation it consists of manually translated high-quality data.
Target Language. We use both generative and discriminative target language tasks. For generative tasks, we use summarization (SUM) using XL-SUM (Hasan al. 2021) and tnglish-t--target machine translation (Mr) using FT.ORES-200 (NLLB Team al. 2022).. For a discriminative task, we employ multiple-choice text classification (Mc) using Belebele (Bandarkar et al.) (0244) and Global MMLU (CMMLU) (Singh et al. 2025) as general target language understanding benchmarks.
Source Language (English). We assess the extent to which the adapted models retain their general task-solving abilities in English SUM, target-to-English MT, and English MC using the same datasets those employed for target languages. We also use MMLU (Hendrycks et al. 2021) an English language understanding benchmark and English BBH (Srivastava et al. 2023 Suzgun et al. 2023 as a stress-test benchmark.
Following Ahia et al. 2023) we use 500 random samples for generative tasks: SUM and MT. The rest use the full test sets for evaluation.
4.6 Evaluation Metrics
Task Performance. We report the standard metrics for each task: accuracy for MC, GMMLU, MMLU, BBH, TRUTHFULQA, and IFEVAL (strict prompt), and exact match for GSM8K and MGSM. For MT- BENCH, we use the mean score over two turns across all questions. Adhering to the standard protocol in Eval FFurrier et al, 2023), each score is determined using Flow-Judge-vo. 1 and follows a Likert-5 scale. For ALPA, we use a win-rate over GPT-4 (1106 Preview) measured by GPT-4.1 nano (2025- 04-14) For SUM and MT, we primarily use chrF (Popović 2015
17 For ToxIcGEN and IMPLICITHATE, we
use safety score, which the percentage of likeliness of the model producing benign over harmful sentences, following Cahyawijaya al. (2024).
6 hhttps://openai. comm/nndx///ptt--11//
Allhhough chrl' widely used metric for SUM and MT (Ebrahim al. 2023 Remy 2024 inter alia), we also
show ROUGE-L [Lin 2004) for SUM and chrF++ (Popovid 2017) MT Appendix

---

Published in Transactions on Machine Learning Research (09/2025)
We report average zero- and three-shot performance across three different runs for SUM and MT, respectively. For the remaining lasks, we report single-run zero-shot performance for IFFVAL, MT-BENCH, ToxIc- GEN and IMPLICITHATE, three-shot performance for MC, TRUTHFULQA, five-shot performance for GMMLU, MMLU, GSM8K, and MGSM as these tasks are deterministically evaluated with temperature set to zcro.
Inference Efficiency. VE offers inference specdups in target language compared to source models Te-
jaswi et al. 2024 Mundra et al. 2024 Yamaguchi et al. 2024bh. To quantify this, we measure the number of tokens generated per second (tokens/s) (Hong et al. 2024
5 Task Performance
5.1 Safety, Chat, and cnttruccioo-follovinng
Safety. Figure 3 (a) shows the aggregated mean performance in safety tasks across the seven tar- get languages. We first observe that ElChat out- performs CV in TRUTHFULQA for both Qwen2.5 and Llama 3.1. In particular, it substantially helps Llama 3.1, achieving 13-point gains over CV on av- erage. We speculate that CV may be less effective for classífication tasks as reflected its performance on other discriminative target language tasks ($5.2).
Chat
CV
Chat+VE
ElChat (Ours)
TruthfulQA
ToxiGen
ImplicitHate
Qwen2.5 -7B
0.3
0.5
0.7 0.1 0.2 0.3 0.4 0.1 0.2 0.3
Llama-3.1 -8B
0.3
0.5
0.7
0.1 0.2 0.3 0.4 0.1 0.2 0.3
(a) Target Language
IFEval
GSM8K
MT-Bench
In contrast, CV often surpasses ElChat in ToXI- GEN and IMPLICITHATE for both models, with dif- ferences of up to 6 points, e.g., in TOXIGEN with Qwen2.5. This is primarily due to the use of Merge instead of Copy in ElChat (see Table 7 Specifi- cally, ElChat without Copy follows similar trends to ElChat, while ElChat without Merge exhibits simi- lar trends to Chat+ VE and CV. This is intuitive, merging a model with the lowest target safety per- formance (i.e., Chat) with Chat+VE can degrade safety performance.
Qwen2.5 -7B
0.3
0.5
0.7
0.2 0.4 0.6
2
Llama-3.1 -8B
0.3
0.5
0.7
0.2 0.4 0.6
0.8
2
(b) English (Source)
Figure 3: Aggregated mean performance (f) across seven target languages for each model safety, chat, and instruction folllwwin tasks. Full results are in the Appendix (Tables and 7
We finally find that ElChat, CV, and Chat+ out- perform the Chat baseline across lasks and models, with gains ranging from 1.4 (TRUTHFULQA with Llama 3.1 using ElChat) 20 points (TRUTHFULQA with Qwen2.5 using ElChat). The only exceptions CV and Ch in TRUTHFULQA, where they underperform Chat by 11 and 14 points, respectively. This instability highlights the advantage of ElChat, as it consistently enhances safety performance over Chat across tasks.
Chat and Instructtion-folloiing Figure (b) shows the aggregated mean performance across chat and instruction-followee tasks in English (source). We first analyze the extent to which ElChat impacts perfor- mance on chat and instruction-followinn tasks compared to the Chat baseline. Note that some performance degradation is inevitable, as adapting Chat to target unlabeled data inherently affects the source chat and wow.utiiooooolooiing abilities ($3).
As anticipated, we find that ElChat exhibits performance degradation across tasks and models compared to the Chat baseline. However, the extent of this degradation varies depending on the model, task, and adaptation approach. For example, ElChat achicves comparable performance GSM8K but experiences reductions of 16 and 0.57 points on IFEVAL and MT-BENCH, respectively. Despite these drops, ElChat suc- cessfully improves instruction- followtioonoooooiigg performance compared to the respective adapted model, Chat+VE. It demonstrates improvements of 14 and 13 points over Chat VE for Qwen2.5 and Llama 3.1, respectively. These results indicate that ElChat can inject cnstruction-follwiiii capabilitics into the adapted model.
8

---

Published in Transactions on Machine Learning Research (09/2025)
Target Language
English
SUM
MT
MC
GMMLU
SUM
MT
MC
MMLU
BBH
Liama-33 1.ama-3.1-2 3.1-8B Qwen2 5-7B
20
30
20
30
40
0.4
0.8
0.4
0.6
20
25
30
40
0.8
0.9
0.6
0.8
0.4
0.5
0.6
20
30
20
30
40
0.4
0.6
0.8
0.4
0.6
20
25
30
40
0.8
0.9
0.6
0.8
0.4
0.5
0.6
Base
Chat
Base+VE
CV
Chat+VE
ElChat (Ours)
Figure 4: Aggregated mean performance across seven target languages for each model (error bars indicate
95% confidence interval). Full results are in Tables 11 and 12
Table 1: MGSM performance in Bengali (bn) and Telugu (te) by model.
We next observe that ElChat generally outperforms CV across tasks and models in five out of six cases, with performance differences ranging from 9.5 points (IFEVAL with Llama 3.1) 25 points (GSM8K with Llama 3.1). A similar trend is observed pendix), where ElChat substantially outperforms CV with up to 16- point gain in Qwen2.5, while still performing competitively with only 0.95-point drop in Llama 3.1. These results suggest that ElChat is more effective than CV in enhancing both chat and instriction-following abili- ties.
(a) Qwen2.5 7B
ALpACAEvAL (Table 10 in the Ap-
Model
EM
bu
Le
Chat
.23
.06
CV
.60
.27
Chat+VV
.39
.27
Finally, ElChat's performance advantage extends to target language tasks, Table shows model performance in MGSM (covering Bengali and Tel- ugu). Overall, ElChal surpasses both CV and the original chat model, Chat, in three and four out of four cases, respectively. Notably, ElChat sig- nificantly improves Telugu performance by 29 points with both Qwen2.5 and Llama 3.1. Additionally, while CV enhances Bengali performance by just. point over Chat in Llama 3.1, FlChat achieves a substantial 21-
ElChat (Ours)
.46
.35
(b) Llama 3.1 8B
Model
EM
bu
te
Chat
.30
.12
point gain. These results further support the superiority of ElChat over CV.
CV
.31
.24
Chat+VE VE
.26
.28
ElChat (Ours)
.51
.41
5.2 Target Language
Figure 4 (left) shows the aggregated mean performance across seven languages for all source and adapted target models in target language.
We note that FlChat consistently outperforms its source chat model (Chat) across all models and tasks. This improvement is particularly notable in generative tasks (i.e., SUM and MT), with gains ranging from 8 points (Llama 3.1 on MT) to 20 points (Qwen2.5 on MT). While ElChat generally maintains competitive performance (within 3 points) compared to Vhat++VE it exhibits slightly reduced performance (up to 5.9 points with Qwen2.5 on MC) in discriminative tasks (i.c., MC and GMMLU) and MT with Llama 3.1. These results suggest that ElChat can overall preserve the target language performance, while the model modifications do not substantially degrade performance.
We further observe that ElChat demonstrates competitive performance with CV, with each method outper- forming the other in half of the evaluated cases. Specifically, CV generally outperforms ElChat in Qwen2.5, except for the MT task, whereas ElChat typically achieves better performance than CV in Llama 3.1, exclud- ing MT. However, CV notably underperforms both the source basc and chat models in the two discriminative tasks (i.e., and GMMLU) with Llama 3.1. Thus, although ElChat and CV achieve similar overall perfor- mance, our method is more likely to yield improvements in target language tasks.
9

---

Published in Transactions on Machine Learning Research (09/2025)
Qwen2.5 7B
Llama 3.1 8B
SUM
En-Target MT
MC
SUM
En-Target MT
MC
3.1x
2.5x
5.9X
3.6x
2.8x
3.2x
2.3x
2.0x
3.2x
2.8x
3.2x
2.4x
2.0x
6.0x
3.1x
2.8x
3.2x
-2.0x
2.0x
6.0x
3.9x
2.8x
25
75
20
40
100
Base .... Chat
Base +VE
CV
Chat+VE
ElChat (Ours)
Figure 5: Aggregated mean inference speedup (tokens/s) across seven target languages. each bar represents the speedup ratio over Chat. Full results are in the Appendix (Tables
The value next to
17 and 18
5.3 Source Language (English)
Figure 4 (right) shows the aggregated mean performance across seven languages for all source and adapted larget models in English tasks. ElChat generally underperforms the source Chat baseline. However, the degree of performance degradation varies considerably depending the task and model similar to $5.1 For instance, with Llama 3.1, ElChat exhibits substantial performance drops of 7, 12, and 18 points on MMLU, BBH and MT, respectively. In contrast, the performance degradation observed with Qwen2.5 is less pronounced, with a maximum decrease of 5.7 points on BBH. Interestingly, FlChat even demonstrates a 6.7- point improvement on MT with Qwen2.5. This improvement likely stems from two key factors: (1) ElChat's effective utilization of source tokens (95%) during generation, compared to other approaches (i.e., Base+ VEE CV, and Chat+VE) that achieve at most 71% source token utilization; and (2) its successful early stopping, generating similar number of tokens (33) as Chat (see Figures and in the Appendix).
Comparing ElChat and Vhat+VE, we find that ElChat generally yields better or comparable performance across models and tasks, with the exception of MT with Llama 3.1. This improvement suggests that ElChat can gencrally alleviate catastrophic forgetting not only in chat and instruction-follovii tasks ($5.1) but also in source language lasks by injecting the source chat information into Chat+VE.
Conversely, CV exhibits poor performance English tasks, typically falling short of ElChat in eight out of ten cases. Moreover, the benefits of catastrophic forgetting mitigation are not consistently observed with CV, as it only outperforms its adapted base model, Bas in five out of ten cases. This result somewhat contrasts with Huang et al. (2024), who highlight the benefits of using CV to mitigate catastrophic forgetting and improve knowledge retention and language ability. We speculate that modifying models through simple arithmetic operations, as in CV, may be less robust than our method. These results suggest that ElChat more effectively integrates target language abilities while mitigating performance degradation across chat, instruction- following, and source English tasks compared to CV.
6 Inference Efficiency
VE offers inference speedups on target language tasks by reducing text overfragmeenation ($4.6) Since ElChat and CV involve modifications to model weights, they may increase the generation of source tokens, potentially slowing inference compared to Vh and Va. Furthermore, the inference efficiency of CV with VE has not been previously studied. Therefore, we investigate the inference efficiency of both methods. Figure shows the aggregated mean inference speedups across seven languages and three target tasks.
Overall, both ElChat and CV achieve comparable speedups to their respective adapted models, Chat+VE and Base+ VE, demonstrating 2.0x to 6.0x speedups over the source chat model, Chat. The largest drop is observed between ElChat and Chat+VE on the MT task with Qwen2.5 (0.4x). This likely happens because ElChat frequently terminates generation early (see Table B resulting in fewer generated tokens (and thus smaller numerator in the osconn/eccond calculation). In summary, these results suggest that the model
10

---

Published in Transactions on Machine Learning Research (09/2025)
User]] de (3ac 3g. and dawãló aggess Brfoffeu :3o- a. e933 dieao $68 wódds dlsg a5seeesss diesss 880 ceeg aadci dag and e5 désensa, desis 155 aucc dag Dsd 5a555 Shafffes ahoo awdeo wodh dod?
Toule wend to lukery and longht vuriely of pastries. She bought dozen donute for 965 dozen, 2 dozen min' cupeakes for 350 a dozen, and dozen mini cheeseenes for 855 dozen. What was the total cost?
Unarr] RIBTR बाفा अरीिर उpन साकिaccाााा 15 Y,a, SIf SiC aac oct carl æet şp भाररटय
|EIChhat] 2u Eefasee 6*55=330 meeg aodss clag Braa- nfes Sessssgrou. as5 si 8ésew 2*80=160 cregs sódsssf daagg Shfsffes Baswsgro. a25i5ess 3*68=204 coog wodis dies- i grafffes SSasaggass Iadfo udd 330+1 60+ 204-694444 wreys. Sammmnoo 694.
Ryam planted 8 flosers in his gumlen in one day. After 15 days, of 5 do not grow, many flowers be left total?
[E Chath बााा Gra यालां सिात 2 शान anficccaa 15 टिर टिी 2 * 15 30f6 उ सnPiRCI0ा fer 5ft अमा साानRicाााा सt कां लोв शागााल crt 30- 25f6 पाग शािार (िअस शना 251
Moui choeaenle were parchnand at a cost of 6*03-359 dollors, 6 dozen. Mini cuporkes were parthased et
2*80=100 dollars, dozen. Dorats were purchased at a cost of 2'68=204 dallars, dozen. Total cod was 330- 10--160 160 2000-00-=0 =69; dollers, Answer is 694.
Ryam plunted flowers 2 15 = 39 flowers. He did not piant 5 flossers, So there will be a total of 30 5 = 85 flowers in his gordens The unswer is B5.
gamdens day. After dage he planted
(a) Correct answer Correct reasoning
(b) Correct answer- Wroon reasoning
Figure 6: Output examples on MGSM. We use Google Translate for translation. Few-shot demonstrations (conversation history) are omitted due to space constraints.
modifications introduced by ElChat and CV do not negatively impact the inference speedups provided by
VE.
7 Analysis
To better understand the behavior of FlChat, we perform an ablation in chat and instruction cestruction-folloinng tasks using Llama 3.1. We also conduct qualitative analysis using MGSM to gain further insights.
Ablation. Table 2 shows the impor- lance of each ElChat component. Re- moving either of the two components ($3) significantly reduces performance across chat and instruction-following tasks, demonst.atting their complemen- tarity in eliciting ElChat's abilities. For instance, ablating the weight copying (Copy) component, leaving only VE and model merging, causes performance to drop by an average of points on IFE- val, points on GSM8K, 14 points on MGSM, and 0.27 points on MT-Bench relative to ElChat. This substantially undermines the advantage of ElChat CV it greatly lags behind CV in MT-Bench and shows marginally better performance than CV with only few points (up to 4 points on GSMSK) differences in other tasks, unlike at least 9 points difference observed between ElChat and CV. These results clearly demonstrate that while the individual components are established techniques, their synergistic effect within a chat model context is non-trivial, and their combination is essential for ElChat's strong performance.
Table 2: Mean performance across languages for chat and cestruction-folowiing tasks using Llama 3.1. Ablation results in other lasks are available in Appendix B
Model
IFEval
GSMSK
MGSM
MT-Bench
CV
.380s
.33
.2700
3.04. 20
ElChat
.47
.8.0c
.46
2.92
Merge
.34.0
.40,a
.37.00
2.53
Copy
.39.z2
.37.7
.32..
2.65 23
Qualitative Analysis. Figure 0 presents examples of ElChat's output MGSM, highlighting both successes and challenges. Case (a) showcases correct reasoning and answer generation in Telugu. However, case (b) demonstrates that even when providing a correct answer in Bengali, ElChat can exhibit wrong reasoning. The misinterpretation of "five flowers do not grow" as "Ryan did not plant five flowers" suggests a potential limitation in understanding nuanced language. Notably, in the same case, it correctly reasons in English that "5 did not grow". Focusing on improving target language abilities further during VE while mitigating catastrophic forgetting of chat and instructionnfollowing could address this issue. For instance,
11

---

Published in Transactions on Machine Learning Research (09/2025)
making iterative model merging Alexandrov et al. avenue for future investigation.
2024 applicable low-resource settings is potential
8 Conclusion
We introduced ElChat, method for directly adapting chat models with VE using unlabeled data, eliminating the need for base model and target chat data. To mitigate potential catastrophic forgetting in the adapted chat models after VE, FlChat elicits chat abilities by injecting information from the source chat model without requiring further training. Extensive experiments across safety, chat, and instruction-following target language, and source language tasks demonstrated that ElChat outperforms the previous stato-of- the-art CV approach in chat and instruction-following and English tasks while being competitive and more robust in the target language and safety tasks. These results highlight ElChat's superior abilities.
Limitations
Continual Pre-training Methods. This paper uses a continual pre-training method proposed by Remy et al. (2024). which tunes the top and bottom two layers of model and its embedding and language modeling head, for efficient and effective target language adaptation. Nonetheless, other continual pre- training methods exist, including adapter-based training (e.g.. LoRA (Hu et al. 2022)) and full fine-tuming, It would be interesting to extensively investigate the effect of different training methods for future work, but this is beyond the scope of this paper.
Model Merging Methods. We experiment with linear and SLERP morging as representative model merging methods for simplicity. More recent methods like TIES Yadav et al. 2023 and DARE-TIES YYu et al. 2024) might perform even better in ElChat. Given the resource constraints, leave this investigation for future work.
Languages. This paper covers seven typologically diverse languages, following previous work on language adaptation that has also tested similar number of languages. For instance, Minixhofer et al. (0022) tested eight languages. Note that Huang et al. (2024) used three languages (i.e., Chinese, Traditional Chinese, and Korean) to verify the effectiveness of CV. Experimenting with more languages is an interesting avenue for future work but is out of the scope of this paper, given our limited computing capacity.
Chat and Instruction-folloiing Evaluation. Our chat and mustruction-followii evaluation is mainly on English data except for MGSM due to the limited availability manually curated language-specific evaluation resources. Azime et al. (2024) has also noted the instability of using LLM-as-a-Judge in Amharic, which also one of our experimental languages. and cneruuctionfoollwiin evaluation in target languages for future work. We hope our work inspires the development of extensive evaluation benchmarks in low-resource languages.
would be an interesting avenue to explore more chat
Ethical Considerations
Although we conducted extensive experiments across diverse public datasets to validate the effectiveness of ElChat, these datasets do not fully represent all real-world scenarios. Therefore, any model derived from based on this work should be used with caution.
While this work does not appear to raise immediate concerns, the deployment of adapted chat LMs, es- pecially in inadvertently perpetuate harmful biases, compromisc safety ways not captured by current benchmarks, or be exploited for misinformation and other malicious purposes. Further research and responsible deployment strategies crucial to address these potential risks.
low-resource language regions, warrants further analysis. These models could
12

---

Published in Transactions on Machine Learning Research (09/2025)
Acknowledgments
We would like to thank the Action Editor, Ruoyu Sun, the TMLR reviewers, Xi Wang, Sam Lewis-Lim, Maggie Mi, Huiyin Xue, and Xingwei Tan for their valuable feedback. We acknowledge (1) IT Services at the University of Shelfield for the provision of services for high-performance computing; (2) the use of the University of Oxford Advanced Research Computing (ARC) facility; (3) EuroHPC Joint Undertaking for awarding us access to McluXina at LuxProvide, Luxembourg; and (4) the use of resources provided by the Isambard-Al National AI Research Resource (AIRR). Isambard-Al is operated by the Universily of Bristol and is funded by the UK Government's Department for Science, Innovation and Technology (DSIT) via UK Research and Innovation; and the Science and Technology Facilities Council [ST/AIRR// A-I/1023J. AY is supported by the Engincering and Physical Sciences Research Council (EPSRC) [grant number EP EP/W52 [PP/524360/1] and the Japan Student Services Organization (JASSO) Student Exchange Support Program (Graduate Scholarship for Degree Seeking Students).
References
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Bchl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengehen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Pranectha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wi, Xiaoxia Wu, Michael Wyalt, Bin Xiao, Can Xu, Jiabang Xu, Weijian Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang, Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 technical report: A highly capable language model locally on your phone. arXiu, 04s/2404.142111 2024a. URT. https: //axxiv. wwtpeiiiv.rr///aae org//bs////////e 14219.
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewelt, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. Phi-4 technical report. Xiv, als//2412.0890,, 2024b. URL
//tp:///rxvvvv/.
Orevaoghenc Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David Mortensen, Noah Smith, and Yulia Tsvetkov. Do all languages cost the same? tokenization in the era of commercial language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9904 9923, Singapore, December 2023. Association for Computational Linguistics. doi:
/t/p////22222/mm/// 614. URL httpss//aclantthllo0
2trg/2023.2 emnlp-m
Anton Alexandrov, Vesclin Raychev, Mark Niklas Müller, Ce Zhang, Martin Vechev, and Kristina Toutanova. Mitigating catastrophic forgetting in language transfer via model merging. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 17167-17186, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 10.18653/vl /t1/////222222 www.iiinnneenniiil .0000/
URL httpss//aallatthoolyy..rrgg wwwp://ooooyyy../2222..2.. fiidings-ennnnnnen
13

---

Published in Transactions on Machine Learning Research (09/2025)
Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lüibbrring Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, and Nicolas Florcs-Herr. Tokenizer choice for LLM training: Negligible or crucial? In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Associalion for Compulational Linguistics: NAACL 2024, pp. 3907-3924, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1, /ttp//1112222...mmii findinng--nacl..22. naacl.247. CRL httpss////caatthllogogoo org/ 2024. 2.044ffnndil findin gs maaac.2247)
Jason Ansel, Edward Yang. Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdía, Will Constable, Alban Des- maison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, C. K. Luk, Bert Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Shumting Zhang, Michael Suo, Phil Tillet, Xu Zhao, Fikan Wang, Keren Zhou, Richard Zou, Xiaodong Wang, Ajit Mathews, William Wen, Gregory Chanan, Peng Wu, and Soumith Chintala. Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, ASPLOS '24, pp. USA, 2024. Association for Computing Machinery. ISBN 97984100033800. doi: (0.1145//662065336003366
New York, NY,
URL https://doi. org/ 1el.1111166666666666 3640366
Israel Abebe Azime, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Mitiku Yohannes Fuge, Aman Kas- sahun Wassie, Eyasu Shiferaw Jada, Yonas Chanie, Walelign Tewabe Sewunetie, and Seid Muhie Yi- mam. Walia-LLM: Enhancing Amharic-LLaMA by integrating task-specific and generative datascts. In Yaser Al-Onaizan, Mohit. Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Com- putational Linguistics: EMNLP 2024, pp. 424, Miami, Florida, USA, November 2024. Associa- tion for Computational Linguistics. doi: 10. 10.18653/v1 /2tp3///1/222/// Taaclanthology.orgg/2200 wwndininnnennnnnn 25/
URL https:
Abhinand Balachandran. Tamil-Llama: A new Tamil language model based on Llama 2. arXiv, abss23311.0584/ 2023. URL https://arxiv wwwp:/ag////////e 158445
Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. The belebele benchmark: a parallel reading comprehension dataset in 122 language variants. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 749-775, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/vv / long...c-oon 44. URL https: https://aclanthology org/2024..2/// 2024. acl-long. 44/
Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, and Bing Täu. Layer swapping for zero-shot cross-lingual transfer in large language models. In Proceedings of the Thirteenth International Conference on Learning Representations, 2025. URL htttp:://openreview.rr
/Forum/1d=xvhh4wrrgp
Samuel Cahyawijaya, Holy Lovenia, Fajri Koto, Rifki Putri, Wawan Cenggoro, Jhonson Lec, Salsabil Ak- bar, Emnanuel Dave, Nuurshadieq Nuurshadieq, Muhammad Mahendra, Rr Putri, Bryan Wilie, Genta Winata, Alham Aji, Ayu Purwarianti, and Pascale Fung. Cendol: Open instruction- generative large language models for Indonesian languages. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14899 14899-14914, Bangkok, Thailand, August 2024. Association for Computational Linguistics. URL
doi: 1e.11655/v1222244
http://222.cccccl111 acl-long............ 796/
ChangSu Choi, Yongbin Jeong, Seoyoon Park, Inho Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jacwan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, and Kyung Lim. Optimizing language augmentation for multilingual large language models: A case study on Korean. Tn Nicoletta Calzolari, Min-Yer Vin Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue
14

---

Published in Transactions on Machine Learning Research (09/2025)
(eds.), Proceedings of the 2024 Joint International Conference Computational Linguistics, Language Resources and Evaluation (LRECC-COLIG 2024), pp. 12514-12526, Torino, Italia, May 2024. ELRA and ICCL. URL httpsss//acclatttooooo.ooo////// 1wwp:mmme.....mmamm.a...a.. 1095/
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv, abs/2110.14168 2021. URL https://arxiii fttpss//arxiv.org//bb///
10. 14168.
Zoltan Csaki, Pian Pawakapan, Urmish Thakker, and Qiantong Xu. Efficiently adapting pretrained language models to new languages. arXiv,, abs/2311.05741, 2023. URL
Yiming Cui, Ziqing Yang, and Xin Yao. Efficient and effective text encoding for Chinese LLaMA and Alpaca.
arXiv, abs/2304.08177, 2024. URL https://aaxiv http:////aaaaaioorra////////0/
arg/abs/2304 200ps/877777
Severino Da Dalt, Joan Llop, Irene Baucells, Mare Pamies, Yishi Xu, Aitor Gonzalez-Agirre, and Marta Villegas. FLOR: On the effectiveness of language adaptation. In Nicoletta. Calzolari, Min- Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Proceedings of the 2024 Joint International Conference on Compulational Linguislies, Language Resources and Evoluation (LREC- COLING 2024), pp. 7377-7388. Torino, Italia, May 2024. ELRA and TCCL. URL battps//accaattoon gy.org/ ww..rg//22...2 1g.. /
Konstantin Dobler and Gerard de Melo. Language adaptation on a tight academic compute budget: Tokenizer swapping works and pure b6loat16 is enough. In Proceedings of the 2nd Workshop on Advancing Newral Net- work Training: Compulational Efficieney, Scalabilily, and Resource Oplimizalion (WANTOICML 2024),
2024. URL https://opeererviee www.:eet//oruu wtt.:eeeeeeetooorro//eeee/aaaea
C.m. Downey, Terra Blevins, Nora Goldfine, and Shane Kreinert-Threlkele Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages. In Duygu Ataman (ed.), Pro- ceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL), pp. 268-281, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/ 20.18653/v122088 /ttp//1..22221111111 https://aclannhology.oooo org.eoollgooooo////// wwwg/////222222// mrl-1 arl-1.20/ 20/
URL
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archic Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aure- lien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damion Al- lonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egoo Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovie, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Geor- gia Lewis Anderson, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Tiyann Zarov, Imanol Arrieta Ibarra, Isabel Klounann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jemny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Ji- awen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pa- supulcti, Mannat Singh, Manohar Paluri, Marcin Kardas, Mathow Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pongwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Pravcen Krish- nan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohit Girdhar, Rohit Patel, Romain
15

---

Published in Transactions on Machine Learning Research (09/2025)
Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hos- seini, Sahana Chennabusappa, Sanjay Singh, Sean Bell, Seohyun Sonia. Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Syd- ney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Uijwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ra- manathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaoqing Ellen Tan, Xin- feng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaci, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aadilya Singh, Aaron Grattafiori, Abha Jain, Adam Kelsey, Adam Shajufeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex Vaughan, Alexei Baevski, Allic Feinstein, Amanda Kallet, Amit Sangani, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Audrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Franco, Aparajita Saraf, Arkabandhu Chowdhuuy, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching Baing-ssiang Chu, Chris Cai, Chris Tindal, Christoph Peichtenhofer, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyalt, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Elconora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggioni, Francisco Guzmán, Frank Kanayet, Frank Seide, Gabriela Med- ina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Govind Thattai, Grant Her- man, Grigory Sizov, Guangyi, Zhang, Guna Lalshminnaryaaaan, Hamid Shojanazeri, Han Zou, Hamnah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Ge- boski, James Kohli, Japhet Asher, Jean- Cean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Karthik Prasad, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veer- araghavan, Kelly Michelena, Keqian Li, Kun Huang, Kunal Chawla, Kushal Lakhotia, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Tei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria Tsimpoukelli, Mar- tynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Moham- mad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovich Laplev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Podro Rittner, Philip Bontrager, Picrre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Ro- han Maheswari, Russ Howes, Ruty Rinott, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswany, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vitor Albiero, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaofang Wang, Xiaojian
16

---

Published in Transactions on Machine Learning Research (09/2025)
Wu, Xiaolan Wang, Xide Xia, Xihun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. The Llama 3 herd
of models. arXiv, abs/2407.21783. 2024. URL puctps://arxxar org/abs/ 2207 21783
Yann Dubois, Percy Liang, and Tatsunori Hashimoto. Longth-controlled AlpacaEval: A simple debiasing of automatic evaluators. In Proceedings of the First Conference on Language Modeling, 2024. URL net/ heatt/ooounidddyyBBmaae
https:///ppereet http://ppeeeeeee.eeeetttttrr//aa
Abteen Ebrahimi, Manuel Mager, Shruti Rijhwani, Enora Rice, Arturo Oncevay, Claudia Baltazar, María Cortés, Cynthia Montaño, John E. Ortega, Rolando Coto-solano, Hilaria Cruz, Alexis Palmer, and Katharina Kann. Findings of the AmericasNLP 2023 shared task on machine translation into indige- nous languages. In Manuel Mager, Ableen Ebrahimi, Arturo Oncevay, Enora Rice, Shruti Rijhwani, Alexis Palmer, and Katharina Kann (eds.), Proceedings of the Workshop on Natural Language Process- ing for Indigenous Languages of the Americas (AmericasNLP), pp. 206 219, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/ h /2223. americasnlp-1.28. URL https:://aclanthoo..gg org/ 2023. americasnlp-11.r
v1
Mai EISherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choud- hur, and Diyi Yang. Latent hatred: A benchmark for understanding implicit hate speech. In Maric-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 345-363, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1 2021..222ppppp... 29. URL
org org/2021. amnnp-main.22 29/
Clémentine Fourrier, Nathan Habib, Thomas Wolf, and Lewis Tunstall. LightEval: A lightweight framework for LLM evaluation. http::///ggtbbb....///ugggggggga
com www.a///wwwgggggg..c//gggttaaaa 2023.
Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Tida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. Continual pre-training for lrossllnngul LLM adaptation: Enhancing Japanese language capabilitics. In Proceedings of the First Conference on Language Modeling, 2024. URL httpss://pperrviiee
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Lau- rence Golding, Jeffrey Hsu, Alain Le Noachh, Haonan Li, Kyle McDonell, Niklas Muemnighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation. 2023.
https://zenodo.org//eeee wttp://rrrrrro///225//3333/
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin Surya Bhupatiraju, Téonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, An- ton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grill, Behmam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Ali- cia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Chnn Choo, Danila Sinopalnikov, David Weinberger, Dimple Vijaykumar, Dominika. Rogozińska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimezak-Plucttttt Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqucira, Joana Iljazi, Jocelyn Becker, Joc Fernandez, Joost van Amersfoort, Josh Gor- don, Josh Lipschultz, Josh Newlan, Ju yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, Lilly McNcalus, Livio Baldini Soares, Logan Kil- patrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Görner, Mat. Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal,
Harleen Batra, Harsh
17

---

Published in Transactions on Machine Learning Research (09/2025)
Mchran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej, Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat, Sara Mc Carthy, Sarah Cogan, Sarah Perrin, Sébastien M. R. Arnold, Scbastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Kocisky, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk Kwon, Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jell Dean, Demis Hassabis, Koray Kavukeuoglu, Clement Farabet, Elena Buchatskaya, Sebastian Borgeaud, Noah Fiedel, Armand Joulin, Kathloen Kencaly, Robert Dadashi, and Alck Andreev. Gemma 2: Improving open language models at a practical size. arXie, abss/2408.005
2024. URL htttps://arxiv.org/// www.oo////////..0///0/1
Xiang Geng, Ming Zhu, Jiahuan Li, Zhejian Lai, Wei Zou, Shuaijie She, Jiaxin GUO, Xiaofeng Zhao, Yinglu Li, Yuang Li, Chang Su, Yanqing Zhao, Xinglin Lyu, Min Zhang, Jiajun Chen, Hao Yang, and Shujian Huang. Why not. transform chat large language models to non-English? Frontiers of Computer Science, 2025. ISSN 2095-2228. doi:
http:///o....//110100//////1/11/ //journal. ceatrnal.eep..o. com. htt//////////0110000//1000000000
URL https:
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's MergeKit: A toolkit. for merging large language models. In Franck Dernoncour,, Daniel Preotiuc-Pietro, and Anastasia Shimorina (ods.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 477 485, Miami, Florida, US, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/200 .emmlp-induustr.36. URL https:://acaattttlo/s wwwphhhooogy.ogg/22222 emmlp- hespp--innuutyy..e .36/
HyoJung Han, Akiko Friguchi, Haoran Xu, Hieu Hoang, Marine Carpuat, and Huda Khayrallah. Adapters for altering LLM vocabularies: What languages benefit the most? In Proceedings of the Thirteenth International Conference on Learning Representations, 2025a. URL http::///operreeeeeeeeeeetooooo d=KxQRHOreeDD
Sungjun Han, Juyoung Suk, Suyeong An, Hyungguk Kim, Kyuseok Kim, Wonsuk Yang, Seungtaek Choi, and Jamin Shin. Trillion 7b technical report. arxiv, abs/2504.1543/ 15431,, 2025b. URL https://arxiv.org/ abs/2504.15m0 15431
Thomas Hartvigsen, Snadia Gabricl, Hamid Palangi, Maarten Sap, Dipankar Ray, and Eco Kamar. ToxiGen: A large-scale machine-generaan dataset for adversarial and implicit hate speech detection. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp- May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022 acl acl- long.22.... 234. URL
Dublin, Ireland,
wrw.:oo0g...g//2222.2.. http22022.ccll1.. long.. 234/
Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong- Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. XL-sum: Targe-scale multilingual abstractive summarization for 44 languages. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Computational Linguisties: ACL-IJCNLP 2021, pp. 4693 44703, Online, August. 2021. Association for Computational Linguistics. doi: wwwnnthooogg.orrg22222 http.//o///.......ng........ ac1. 4133/
(00053/v1122// 11t.//........./.n........ 413. URL https:
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. Measuring massive multitask language understanding, In Proceedings of the Nineth International Conference on Learning Representations, 2021. URL https://ppparrooeeeeeeeefooooaip
Jimin Hong, Gibbeum Lee, and Jaewoong Cho. Accelerating multilingual language model for exces- sively tokenized languages. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of
18

---

Published in Transactions on Machine Learning Research (09/2025)
the Association for Computational Linguistics: ACL 2024, pp. 11095-11111, Bangkok, Thailand, Au- gust 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1/202. http////..iinnng-aadlll.l. URL
wwtp///2...iiiinnnsoeooe..o windinggs-al..6on 660/
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Ahu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaplation of large language models. In Proceedings of the Tenth International Conference on Learning Representations, 2022. URL htttp:://ppeneevee..tt wtwp:/eeeeforum/ddneeeeeeeee
Shih-Cheng Huang, Pin-Zu Li, Yu-chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tsai, and Hung-yi Lee. Chat vector: A simple approach to equip LLMs with instruction following and model alignment new languages. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Popers), pp. Bangkok, Thailand, August 2024. Association for Computational Linguisties. doi: 10.18653 /v1/2024.ccllong.5500 URL htttt::://ccaaatttoooooo
org/ 2024 acl-long..555
Seungduk Kim, Seungtaek Choi, and Myeongho Jeong. Efficient and effective vocabulary expansion towards multilingual large language models. arXiv, 0t///2402.144114 2024. URL http:ppp///aaxvvoor////////200
4714
Sneha Kudugunta, Isaac Rayburn Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. MADLAD-400: A multilingual and document-level large au- dited dataset. In Proceedings of the Thirty-seventh Conference on Neural Information Processing Systems Datasels and Benehmarks Track, 2023. URL https: wttp:///ww....eee/oooooooc////a.
Celio Larcher, Marcos Piau, Paulo Finardi, Pedro Gengo, Piero Esposito, and Vinicius Caridá. Cabrita: closing the gap for foreign languages. arv, abs/2308.11878, 2023. URL htttt:///aattiiii wwwpr/iwwwrr/ba/////.0
.18778
Quentin Lhoest, Albert Villanova del Moral, Vacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Saško, Gun- jan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan Major, Philipp Schmid, Sylvain Gugger, Clément Delangue, Théo Matussière, Lysandre Debut, Stas Bekman, Pierric Cistac, Thibault Gochringer, Victor Mustar, François Lagunas, Alexander Rush, and Thomas Wolf. Datasets: A community library for natural language processing. In Heike Adel and Shuming Shi (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 175-184, Online and Punta Cana, Dominican Republic, URL
November 2021. Association for Computational Linguistics. doi: 10.18653/ https://aclantho/// org/ 2021 emnle- 21/
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. AlpacaEval: An automatic evaluator of foslloigion-folloiing modcls. https:
httpaab..ccoottttaaaaaaaaaappaap _val] 2023.
Chin-Yer Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcclona, Spain, July 2004. Association for Computational Linguistics. URL https:
Peiqin Lin, Shaoxiong Ji, Jörg Tiedemann, André F. T. Martins, and Hinrich Schütze. MaLA-500: Massive language adaptation of large language models. arviv, abs/2401.13300, 2024. URL https://arxiv.. org/ abs/2480 0oss/2401.13300
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimie human false- hoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.). Proceedings of the 60th Annual Meeting of the Association for Computational Linguistūes (Volume 1: Long Papers), pp. 3214- 3252, Dublin, Ireland, May 2022. Association for Computational Linguisties. doi: ong.229 229. URL https://aclanthooogg. wwwp:yyy.rgg/22222.2.// a01 229/
19

---

Published in Transactions on Machine Learning Research (09/2025)
Samin Mahdizadeh Sani, Pouya Sadeghi, Thuy-Trang Vu, Yadollah Yaghoobzadch, and Gholamreza Haffari. Extending L.Ms to new languages: A case study of llama and Persian adaptation. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend AI-Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the 31st International Conference on Computational Linguistics, pp. 8868 8884, Abu Dhabi, UAE, January 2025. Association for Computational Linguistics. URL httpp::/aclaathoology.. org/
5. mholing-main...
Joshua Maynez, Priyanka Agrawal, and Sebastian Gehrmann. Benchmarking large language model capa- bilities for conditional generation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (ods.), Proceedings of the 61st Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Papers), pp.
Toronto, Canada, July 2023. Association for Computational Linguisties. doi: 10.1 10.18653/v1 //l/222.ccllooon....
URL htttps://aclanthologg org 2223/2223..22 www.:00..1.0.... acl- 511/
Benjamin Minixhofer, Fabian Paischer, and Navid Rekabsaz. WECHSEL: Effective initialization of sub- word embeddings for pross-lingual transfer of monolingual language models. In Marine Carpuat, Marie- Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (ods.), Proceedings of the 2022 Conference of the North American Chapler of the Association for Compulational Linguislics: Human Language Technolo- gies, pp. 009229990099 Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/vv 20/G533/v12222.mmmmm waw.aaacmmmiii9999 URL https:
org/2022 naacl 293/
Benjamin Minixhofer, Fdoardo Ponti, and Ivan Vulié. Zero-shot tokenizer transfer. In Proceedings of the Tiirtt-eighth Annual Conference on Neural Information Processing Systems, 2024. URL lattps://ppet
Doview.net/fon htt.:o...c...orommmmddddddbbbbb I2C
Nandini Mundra, Aditya Nanda Kishore Khandavally, Raj Dabre, Ratish Puduppully, Anoop Kunchukuttan, and Mitesh M Khapra. An empirical comparison of vocabulary expansion and initialization approaches for language models. In Libby Barak and Malihe Alikhani (eds.), Proceedings of the 28th Conference on Computational Natural Language Learning, pp. 84-104, Miami, FL, USA, November 2024. Association for Computational Linguistics. doi: 10. n..18653//112224.ooll conll-1.8/
URL https://aclaantholog..rrg.
Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang. Chaoqun Liu, Hang Zhang, and Lidong Bing. SeaLLMs- - large language models for Southeast Asia. In Yixin Cao, Yang Feng, and Deyi Xiong (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pp. 294 30440 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/ /11////vv////// h wa.0024.clllddee acl- demos......... URL attps://aclanthhlogg) org. 2024. acl- 28/
NLLB Team, Marta R. Costa-jusså, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia-Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Kochn, Alexandre Mourachko, Christophe Ropers, Safyyaah Saleem, Holger Schwenk, and Jeff Wang. No language left behind: Scaling human-centered machine translation. arXiv, Abbs/2207.04677 2022. URL https://arxiv org/abss abs/
Malte Ostendorff and Georg Rehm. Efficient language model training through cross- lingual and progressive transfer learning. arXiv, abs/2301. (bs/2301.00626, 2023. URL ltttps://arxxi htrg//bs//330000000/ 09626
Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi. Language model tokenizers introduce unfairness between languages. In Proceedings of the Thirty-seuenthh Conference on Neural Information Processing Systems, 2023. URL http:///wwpppppeeeeeeeeeee//aaaa wtw..e//oruuriiee38y/....999
Kunat Pipatanakul, Phatrasek Jirabovonvisut, Potsawee Manakul, Sittipong Stipaisarnmongkol, Ruangsak Patomwong. Pathomporn Chokchainant, and Kasima Thampipitchai. Typhoon: Thai large language
models. Xiv, 2bs/2231.13995 2023. URL https: haaps:///rreee //arxiv....v./// org/abs/23122
httpp///a//dd...raa/////3//2/ 13951
20

---

Published in Transactions on Machine Learning Research (09/2025)
Maja Popović. chrF: character n-gram F-score for automatic MT evaluation. In Ondřej Bojar, Rajan Chatterjee, Christian Federmann, Barry Haddow, Chris Hokamp, Matthias Huck, Varvara Logacheva, and Pavel Pecina (eds.), Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 392 395, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10.18653/v1/W 5-3049. URL /attps://aalannthol.gg/ hrtp:///ooooooooor///530000000 org/W15-3049/
Maja Popović. chrF++:: words helping character n-grams. In Ondřej Bojar, Christian Buck, Rajen Chat- terjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, and Julia Kreutzer (eds.), Proceedings of the Second Conference on Machine Translation, pp. 612-618, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: wwwp///g//////...
10. (...1665/v1/1114//
URL https: tllt:s://aclaanhhllae
Matt Post. A call for clarity in reporting BLEU scores. In Ondřej Bojar, Rajen Chalterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Lucia Specia, Marco Turchi, and Karin Verspoor (cds.), Proceedings of the Third Conference on Machine Thanslation: Research Pupers, pp. 186-191, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 200/53//1//188// 6319. URL https: htwpt/choooggoog////------ org/W18-6319/
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv, abs/2412.151155, 2025. URL
attts://aarae wwwp:/s..ga//////te httg//bb/2412..0 2212 15115
François Remy, Pieter Delobelle, Hayastan Avetisyan, Alfiya Khabibullina, Miryam de Lhoneux, and Thomas Demeester. trken-tokennittion and cross- bross-lingual vocabulary transfers: Language adaptation of LLMs for low-resource NLP. In Proceedings of the First Conference on Language Modeling, 2024. URL https:
//openreview http:///ww..eettooordddddddaaree
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Language models are mul- tilingual chain-of- thain-of-thought reasoners. In Proceedings of the Eleventh International Conference on Learning Representations, 2023. URL
Shivalika Singh, Freddie Vargus, Daniel D'souza, Börje Karlsson, Abinaya Mahendiran, Wii Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O' O'Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiáski, Hakimeh Fadaei, Irem Ergun, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Chien, Sebastian Ruder, Surya Guthikonda, Emad A Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Ustün, Marzieh Fadaee, and Sara Hooker, Aya dataset: An open-access collection for multilingual instruction tuning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Anreual Meeling of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11521 11521-11567, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653 //////1222222 /20...4.cl-ooog......... 620. URL /tctpp::/acclattoott
htg//022.. 200...ac111o.m. 620/
Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotivat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Antoine Bosselut, Alice Oh, Andre Martins, Leshem Choshen, Daphne Ippolito, Enzo Ferrante, Marzich Fadace, Beyza Ermis, and Sara Hooker. Global MMLU: Understanding and addressing cultural and linguistic biases in multilingual evaluation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 18761-18799, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 17199-88117--251-1 10. 201166631122225..oo acc.o.....oooonn.111 URL https::/aclanthoolggy.oo org. 20apg//22222.2 acl aoll-log..90 919/
doi:
21

---

Published in Transactions on Machine Learning Research (09/2025)
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowyez, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Am- brose Slone, Ameet. Rahane, Anantharaman S. Tyer, Anders Johan Andreassen, Andrea Madollo, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew Kyle Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efraa,, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartiomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Ben- jamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, Cesar Ferri, Chandan Singh, Charles Rathkopf, Chenlin Meng. Chitta Baral, Chiyu Wu, Chris Callison-Burch, Christopher Waites, Christian Voigt, Christopher D Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemeneia Siro, Colin Raffel, Courtney Asheraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kil- man, Dan Roth, C. Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debaiyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodolå, Emma Lam, Eric Chu, Eric Tang, Erkut Er- dem, Ernie Chang, Ethan A Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumm Plumed, Francesca Happé, Francois Chol- let, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germàn Kruszewski, Giambat- tista Parascandolo, Giorgio Mariani, Gloria Xinyue Wang, Gonzalo Jaimovitch-Lopez Gregor Betz, Guy Gur-Ari, Hana Galijasovic, Hannah Kim, Hannah Rashkin, Hannanch Hajishirzi, Harsh Mchta, Hayden Bogar, Henry Francis Anthony Shevlin, Hinrich Schuelze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B Simon, James Koppel, James Zheng, James Zou, Jan Kocon, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema. Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orale Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishhan, Katerina Ignatyeva, Katja Mark- ert, Kaustubh Dhole, Kevin Gimpel, Kevin Omondi, Kory Wallace Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Cchando Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng Hc, Luis Oliveros- Colón, Luke Metz, Lütfi Kerem Senel, Maarten Bosma, Maarten Sap, Maartje Ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramirez- Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L Leavitt, Matthias Hagen, Mátyás Schubert, Modina Or- duna Baitemirova, Melody Arnaud, Melvin McElrath, Michael Andrew Yee, Michael Cohen, Michael Gu, Michael Tvanitskiy, Michael Starritt, Michael Strube, Michal Swedrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan Andrew Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Con- stant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormo- labashi, Peiyuan Liao, Percy Liang, Peter W Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Milkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand,
22

---

Published in Transactions on Machine Learning Research (09/2025)
Roman Novak, Roman Sitelew, Ronan Le Bras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Russ Salakhutdi- nov, Ryan Andrew Chi, Seungjae Ryan Tee, Ryan Stovall, Ryan Techan, Rylan Yang, Sahib Singh, Saif M. Moharmmad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bow- man, Samuel Stern Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima Shammie Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divie, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven Piantadosi, Stuart Shieber, Summer Misherghi, Svetlana Kiritchenko, Swa- roop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsunori Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timo- fei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Necraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Venkatesh Ra- masesh, vinay uday prablu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Sophie Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, and Ziyi Wu. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Reseurch, 2023. ISSN 2835-8856. URL https://openrevieen http://ppanerriii...eoooorrrra/p htt.://ooroomm/dddyLLlL....
Featured Certification.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. Challenging BIG-bench tasks and whether chain-of-thought can solve them. In Anna Rogers, Jordan Boad-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 13003-13051, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 10.18653/v / 20t.2/200.....ns--22.. findings-acl.822 URL attps://aclaanthloog. wttp:///wwooooooo/2222..... www.oo23........ooooo wwndings--cc..ee...
Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. Language-specific neurons: The key to multilingual capabilities in large language models. Tn Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5701 5715, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/vl 2136533/11/222.ad/ll loo..on....0 309. URL
wwwpooloogyyog//222/2/// aww-loon...00000 309/
Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songlang Huang, Dongyan Zhao, and Yansong Feng. Unlocking the potential of model merging for low-resource languages. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 8705-8720, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1 1en224..iningg-mmin
URL
ortg//////222 eenddinng-ennl.... 508/
Atula Tejaswi, Nilesh Gupta, and Eunsol Choi. Exploring design choices for building language-specific LLMs. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 10485-10500, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. (2.18653/v1 (0.18653/v1/2024. findings-emnlp6611 URL https:
Cagri Toraman. Llama Adapting open-source generative large language models for low-resource lan- guage. arxiv, abs/2405.077455, 2024. URL https://ardv..or//ab///////a
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do llamas work in English? on the latent language of multilingual transformers. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. -5396,, Bangkok, Thailand, August 2024. Association for Computational Linguistics. URL http:ttt//cccaataaatt wwwphhhoologyoo//222444444 acl -ong. 820/
doi: 10. 10.18653/v /v1l
23

---

Published in Transactions on Machine Learning Research (09/2025)
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: Statc-of-the-art natural language processing. In Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Conference on Empirical Melhods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. Association for Computational Linguistics. doi: 10.18653/v1/ http:/0.eeeeemmdeeeeeee 08.6/
URL https://aallantaoloogy wwtpt/thoooogooog/222202 org/2020- emn mamlp-dem
Mitchell Wortsman, Gabriel Ihharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Topes, Ari S Mor- COS, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple finc-tuned models improves accuracy without increasing infer- ence lime. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 23965 23998. PMLR, 17 23 Jul 2022. URL https://proceedding mlr. wtt.smmr.rrrssse wttp:////vvv2///tttsssesem htmll
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Adilya Siddhant, Adilya Barua, and Colin Raffel. m A massively multilingual pre-trained text-to-text transformer. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotteroll, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Languoge Technologies, pp. 483 498, Online, June 2021. Association for Computational Linguistics. doi: 1118633/1/222..nnaammmaaaat naw.lll.aallllm...... URL https://aclanne http://ooooooooo/221...... naccl-main 41/
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES-merging: Resolving interference when merging models. In Proceedings of the Thirty-seventh Conference on Neurol Information Processing Systems, 2023. URL
Atsuki Yamaguchi, Aline Villavicencio, and Nikolaos Aletras. An empirical study on cross-lingual vocabulary adaptation for efficient language model inference. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNIP 2024, pp. 6760-6785 6785, Miami, Florida, USA, November 2024a. Association for Computational Linguistics. doi: 00.18653/v1/2022 findi ngs-emmlp.396. URL
findings-emmlp.3366
Atsuki Yamaguchi, Aline Villavicencio, and Nikolaos Aletras. How can we effectively expand the vocabulary of LLMs with 0.01GB of target language text? arXiv, abs/2406.11477,, 2024b. URL https://arxiv.
g/abb/2400..111 11477
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report. arXiv, abs/2407.1067) 2024. URL. https:// //arxiv..//iiv./// org/abs//////eeeoooeo..oo.e /alpb///47.01000
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv. Chujic Zheng, Dayihong Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Go. Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Pong Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui,
24

---

Published in Transactions on Machine Learning Research (09/2025)
Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXit, abs abs/250550993s, 2025. URL
https://arxiv... http://axxvvooogaaaa/ barg/abss/i al//a//bb.//...../ 2505.09388
Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, and Furu Wei. Adapt-and-distill: Developing small, fast and effective pretrained language models for domains. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Associalion for Compulational Linguistics: ACI-IJCNLP 2021, pp. 460-470, Online, August 2021. Association for Computational Linguisties. doi: 10.18653/v1 /22011 fin dilgs-acl.40. URL
findings-acl...
Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, Alham Fikri Aji, David Ifeoluwa Adelani, Khalid Almubarak, Saiful Bari, Lintang Sutawika, Jungo Kasai, Ahmed Baruwa, Genta Winata, Stella Bi- derman, Edward Raff, Dragomir Radev, and Vassilina Nikoulina. BLOOM+1: Adding language support BLOOM for zero-shot prompting. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.); Proceedings of the 61st Annual Meeting of the Association for Computational Línguistics (Volume 1: Long Papers), pp. 11682-11703, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v acl-o.n..--l..... URL htttps://aclanthholle
w0t. acl-long. 653/
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as free lunch. In Proceedings of the Forty-first International Conference on Machine Learning, 2024. URL
wttpiiiwwweeeeeeeeee net wwwp:eeeeeeeeetnneeeeee fag//foruun..ddffgaiil8..
Jun Zhao, Zhihao Zhang, Luhui Gao, Qi Zhang, Tao Gui, and Xuanjing Huang. LLaMA beyond English: An empirical study on language capability transfer. arXiv, abs/2401.01055, 2024a. URL attps://arx: v. /.org/abs//401.00055
Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. How do large language models handle multilingualism? In Proceedings of the Thirty-eighth Annual Conference on Neural Infor- mation Processing Systems, 2024b. URL htttps://ppenreiieee http://w.eee/rrmmmrddddoddooooo
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as- a-judge with MT-bench and chatbot arena. In Proceedings of the Thirty-seventh Conference on Newral Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openrevview nee /forum/iiduucchPOD1..
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-followinn evaluation for large language models. arx, 2bs/2211,07911, 2023. URL http/aawwwi..rraaaa/11.......
A Experimental Setup
A.1 Chat Template and Special Tokens
Model-specific chat templates and special tokens are accessible via the following links:
ig. json
Llama 3.1:
nizer sizer_config son
Qwen3: https:
Below excerpts from the chat templates of each model with placeholders for a prompt and output:
25

---

Published in Transactions on Machine Learning Research (09/2025)
Qwen2.5.
<lim_stad start (>system You are Qwen, created by Alibaba Cloud. You are a helpful assistant. <lim_end|>
Puser
{prompt}
<lim_end]>
< start passistant foutput}
<
Llama 3.1.
<lbegin of text/>
< [sstat_header___ id >system
< heedd_haderrrddld
Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024
<leot_idl)
[Istart_header liteerr_dd_uuu >user
<lend_ <lend_header htw.heaadddedllee
{prompt}
</eot_id|>
assart_headee id >assistant
< <lend header_id|>
{output}
Qwen3.
< sim_starrt >usee
(prompt}
< <lim_end|>
<lim_start Dassistant
<think>
</think>
{output}
A.2 Prompt Template
We translate the English prompt templates provided by Ahia al. (0233 for SUM with machine translation API, following Yong et al. (2023). For MT and MC, we formulate task-specific English prompt, followed by machine translation for each language. For the remaining lasks, except for MT-BENCH, we use the default templates provided in am-evvaluation-hanaae template provided in Eval (Fourrier et al. 2023). Table shows the prompt templates used our evaluation. Note that we do not make any changes to the task-specific prompt allow for fair comparison between models with and without chat template (i.e., Base and Chat, respectively).
(Gao et al. 2023). For MT-BENCH, we use the default
26

---

Published in Transactions on Machine Learning Research (09/2025)
Table 4: Hyperparameters for continual pre-training
Hyperparameters
Values
Batch size
32
Number of training steps
30,517
Adam
1e-8
Adam $1
0.9
Adam $2
0.999
Sequence length
512
Tearning rate
5e-5
Tearning rate scheduler
cosine
Warmup steps
First 5% of steps
Weight decay
0.01
Attention dropout
0.0
Training precision
BF16
Table 5: Parameters for gon-greedy generative tasks: MT and SUM.
Parameters
Values
Temperature
0.8
Repetition penalty
1.1
Top
Top
0.9
Beam width
Sampling
True
Early stopping
True
Maximum number of generated tokens
128
A.3 Implementation Details
Our experimental design based on the findings from Tejaswi et al. (2024). They report that (1) there are signsficant gains when adding more than 10K tokens to the source vocabulary, and (2) additional CPT in the order of milliios of tokens is sufficient for model adaplation. Given this, we set the vocabulary size of the auxiliary target language tokenizer Vaaux to 50K across languages and the number of new target tokens k to 10K. We train each model for 500M tokens with batch size of 32, maximum learning rate of 5e-5, and a sequence longth of 512.
Tokenizer Training. We train tokenizers using Hugging Face Tokenizers.
Preprocessing. We preprocess datasets with Hugging Face Datasets
Thoest et al. 2021
Continual Pre-training.
We implement our models using PyTorch (Ansel et al. 2024 and Hugging Face
Transformers Wolf et al. 2020 Table 4 lists the hyperparameters in CPT.
Model Merging. To ensure a smooth transition between layers, we use a 0.3 : 0.7 = Chat : Chal+VE mixing ratio for the top and bottom layers of all merged models, favoring Chat+VE as these layers are adjacent the embeddings and language modcling head of VE. For the second top and bottom layers, we use 0.5 0.5 Chat Chat+VE, balancing the contributions of Chat and VE. For Qwen3, 0.1 0.9 Chat Chat+ VE mixing ratio for all layers.
27

---

Published in Transactions on Machine Learning Research (09/2025)
Evaluation. We use Hugging Face LvghtEva for evaluation on all tasks except for ALPACAEVAL IFE- VAI, GSM8K and MGSM. For ALPACAEVAI, we use the official implementation available GitHut (v0.6.6). For IFEVAL, GSM8K, and MGSM, we use Lm-evaluationn
(Gao et al. 2023). To com-
pute ROUGE-L, we split sentences with an mT5 (Xue et al., 2021 tokenizer as preprocessing following Maynez et al. (2023) and subsequently call rouge_ rouge_scorets scorer to compute the metric. To compute chrF and chrF++,, we use SacreBLEU (Post, 2018). For safcty evaluation, we follow CChyawijjya al. (2024) and
use their implementation available on GitHub: htttpss//ggihuub..om///ooollaaa
Table lists the parameters used during evaluation for generative tasks: MT and SUM. To make fair comparison, we do not conduct any generation parameter tuning and use the same ones across all approaches.
Hardware. We use either single NVIDIA A100 (80GB), NVIDIA H100 (80GB), NVIDIA GH200 (96GB) for CPT. For CPT with Qwen3 14B, we use a single AMD MI300X GPU. For evaluation, we use single NVIDIA A100 (80GB) for all Llama 3.1 models, a single NVIDIA H100 (80GB) for all Qwen2.5 models, and single AMD MT300X GPU for all Qwen3 models to ensure accurate measurement of inference efficiency.
Model Links.
We list all the source model URLs in the following:
Qwen2.5 (Chat):
https://wwgilggaeee..///eeee/222
Qwen2.5 (Base):
btpps//huggangfacccc////e/////23
Llama 3.1 (Chat):
http://wuggggggecocooooaammmmmma
IIssrrutttttt
Llama 3.1 (Base):
Qwen3 (Base):
Qwen3 (Chat):
https://hugsiigscccco///hee/rb/a
B Results
B.1 Task Performance
Safety, Chat, and Instruction--oolloinng Tables 0 and provide detailed breakdown of the task performance results for Qwen2.5, Llama 3.1, and Qwen3 across safety, chat, and instruction- following tasks. Table shows the results of Chat+CPT CPT and ElChat-related ablation models in MGSM. Table [0 shows the results of Chat-related models on ALPACAEVAL.
12 and 13 provide a detailed breakdown of the task performance results for Qwen2.5, Llama 3.1, and Qwen3 across target language and source (English) language tasks.
Target Language and Source (English): Tables 11
and mt Results with Auxiliary Metrics: Tables 14
15 and T6 provide a detailed break-
down of SUM and MT performance results, measured by ROUGE- for SUM and chrF++ for MT, for Qwen2.5, Llama 3.1, and Qwen3, respectively.
B.2 Inference Efficiency
Tables 17
18 and 19 provide detailed breakdown of the inference efficiency results for Qwen2.5, Llama
3.1, and Qwen3, respectively.
http:://gttuub.cm//steeu
28

---

Published in Transactions on Machine Learning Research (09/2025)
Target Language
English
Owen2. 5-7B
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Lima- Laama-311-90 3.1-83
1.00
0.00
0.25
0.50
0.25
0.25
0.50
0.75
1.00
Base+VE ZP
CV
Chat+VE
ElChat (Ours)
Figure 7: Aggregated average ratio of target tokens in output sample across seven target languages for each model (error bars indicate 95% confidence interval).
Target Language
English
SUM
MTT
SUM
MTT
Qwen2. 5-B
0
50
100
50
100
50
100
50
100
Liama-3 3.ma--31-8B
50
100
100
50
100
Base
Chat
Ba
CV
Chat+VE
ElChat (Ours)
Figure 8: Aggregated average number of generated lokens per sample across seven target languages for each
model (error bars indicate 95% confidence interval).
B.3 Ratio of Target Language Tokens
Figure 7 shows the aggregated mean ratio of target new tokens in output per sample across seven target languages for each model.
B.4 Number of Generated Tokens
Figure shows the aggregated average number of generated tokens per sample across target languages for each model.
29

---

Published in Transactions on Machine Learning Research (09/202))
Amharic [Afroasiatic]
Bengali [Bengali]
(urnese(Myanmar) [Sino-Tibetan]
Gujarati [Indo-Europea]]
Sinhala Enno--uuoopenn
Tamil [Dravidian]
Telugu [Dravidian]
50
Qwen2. 5-7B
40
20
10
1lama-3.1-89
30
20
Base
Chat
Base+VE VE
=
CV
Chat+VE
ElChat (Ours)
Figure 9: Aggregated mean performance across target. language tasks for each model by language (error bars
indicate 95% confidence interval).
C Analysis and Discussion
C.1 CPT-only vs. VE
Comparing the adapted chat models (Chat+VE and ElChat) with the CPT-only model (Chat+CCPT) in Tables I and 12 observe three key trends: (1) Chat+VE generally performs better than Chat+CPT target language tasks across both models; (2) Chat+CPT often outperforms Chat+VE on MT tasks; and (3) ElChat either matches or surpasses Chat+CPT) on nearly all tasks and models, except for Target MC and GMMLU with Qwen2.5 and target-to- tngglsg MT with Llama 3.1. This performance advantage of ElChat is also confirmed for chat and cnerruction-folloviiii tasks (Tables 6 and 9). where ElChat substantially outperforms Chat++CPP almost all cases across languages, tasks, and models.
We also observe similar trend between Base+ VE and the CPT-only adapted base model (Base+CPT) in Tables II and 12 Base+VVE often outperforms Base CPT in most of the tasks, while Base++CPT excels in target-to- English MT across models.
These results somewhat contradict the recent observations (Downey et al. 2023 Yamaguchi et al. 2024a by that CPT-only models often perform better than vocabulary adapted models, possibly due to the robustly aligned original embeddings. However, Fujii et al. (2024) reported that "the overall impact of vocabulary expansion on performance is minor." Further, Dobler & de Melo (2024) also claimed that "we do not see clear trend of better performance with without tokenizer swapping" for their vocabulary adaptation experiments. We hypothesize that the superiority of CPT is greatly affected by the amount of CPT data, and it can be more apparent in low-resource settings as in Yamaguchi et al. (202bb), where new embeddings are likely to be not well aligned.
It important to note that the CPT-only models (i.e., Base+CPT and Chat+CCT) have no speedups at all (Tables 17 and 18 as they use the same vocabulary as the source models (i.e., Base and Chat).
C.2 Additional Analysis by Language
We conduct additional analysis of the target language task performance models by language. Figure 9 presents the aggregated mcan performance across three target language tasks (i.c., Target SUM, English- to-target MT, and Target MC). Figure 10 shows the aggregated mean performance across generative target language tasks (i.e., Target SUM and English-to-target MT).
30

---

Published in Transactions on Machine Learning Research (09/202))
Amharic [Afroasiatic]
Bengali [Bengali]
Burnese(Myanmar) [Sino-Tibetan]
Gujarati [Indo-European]
Sinhala Enro--uroppenn
Tamil [Dravidian]
Telugu [Dravidian]
40
Qwen2. 5-78
30
20
10
40
Llama-3.1-88
30
20
10
Base
Chat
Base+VE
CV
Chat+VE
ElChat (Ours)
Aggregated mcan performance across target language generative tasks (SUM and MT) for cach model by language (error bars indicate 95% confidence interval).
Figure 10:
Performance improvements achieved with VE are evident across languages and models. Over- all, we observe from Figure that ElChat consistently outperforms Base and Chat in all the target languages across models.
The extent to which adapted chat models improve target language task performance substan- tially varies by language and model. We observe from Figures g and [0 that ElChat substantially improves their target language performance over Base and Chat across models in Amharic, Burmese (Myan- mar), and Sinhala. However, its improvement in Bengali remains minimal for both Qwen2.5 and Llama 3.1, especially in generative tasks (Figure 10j. Similarly, for Llama 3.1, the performance gains in Tamil, Telugu, and Gujarati are less pronounced, particularly dispority strongly correlates with the amount of language-specifie data used to train the source chat model. Note that similar findings have already been reported by Yamaguchi al. 2024a and Tejaswi al. (2024) inter alia.
generative tasks (Figure 10). We hypothesize that this
How many languages do source chat LLMs support? The exact language coverage of the source explicitly list their supported languages.
LLMs remains unclear,
Qwen2.5 QQwn et al. 2025 Yang et al. 2024) reportedly supports over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, and Arabic. However, none our target languages are explicitly included in this list.
Llama 3.1 (Dubey et al. 2024) officially supports English, German, French, Italian, Portuguese, Hindi, Span- ish, and Thai. However, employs Fast best- language identification model categorize documents into 176 languages during pre- pre-processing, suggesting that some of our target languages may be included in its pre-training data.
Additionally, Llama 3.1 reportedly utilizes 15T multilingual tokens Dubey et al. 2024 for pre-training,
while Qwen2.5 uses 18T tokens in total (not exclusively multilingual) (Qwen et al. 2025 Yang et al., 2024). This suggests that Llama likely benefits from exposure to a broader set of languages. The relatively
modest performance improvements observed for Gujarati, Tamil, and Telugu in Llama 3.1 could indicate that these languages were already present in its training data.
Approximating the language coverage. We can roughly estimate the language coverage the source LLMs using MADLAD-400. The rationale is that these open-weight models generally trained mainly
31

---

Published in Transactions on Machine Learning Research (09/2025)
mixture of publicly available data (Gemma Team et al. 2024 Dubey al. 2024). Given that MADLAD- 400 is sourced from CommonCrawl as mentioned in its data distribution can potentially approximate the relative coverage of our target languages.
Table 20 presents the data statistics of MADLAD-400 for our target languages. We observe that Burmese, Amharic, and Sinhala. have the fewest. articles and total characters in MADLAD-400. Notably, these lan- guages also exhibit the largest performance gains in our experiments. This suggests a strong correlation between the size of language-specific data in MADLAD-400 and the effectiveness of VE in task performance. On the basis of these results, we hypothesize that the two source LLMs used in our experiments might have been trained on very limited language-specifec data, or possibly not at all, for Burmese, Amharic, and Sinhala. In contrast, Tamil, Telugu, Bengali, and Gujarati each have over million articles, making them at least 7.4 times larger than the Burmese dataset. This further suggests a higher likelihood of their inclusion in the pre-training data of the source L.Mss.
Table 20: Data size of MADLAD-400
Kudugunta et al. (0223) for each language.
Language
Number of articles
Number of characters
Tamil
5.6M
10.6B
Telugu
2.5M
5.1B
Bengali
4.3M
4.3B
Gujarati
1.3M
2.1B
Sinhala
788K
1.9B
Amharic
245.2K
509M
Burmese
176.5K
1.3B
C.3 Additional Analysis with Newer and Larger Model
We examine the efficacy of ElChat against a newer and larger model compared to Qwen2.5 7B and Llama 3.1 8B. To this end, we employ a state-of-the-art Qwen3 14B model, applying ElChat for adaptation. Due to computational limitations, our evaluation focuses on Amharic, Bengali, and Telugu. Amharic is selected because it shows the most significant speedup gains in target language generative tasks with Qwen2.5 and Llama 3.1 (Tables 17 and 18). Bengali and Telugu are chosen as they the only languages covered by MGSM. Our analysis will focus on two key questions: (i) Does Qwen3 exhibit a similar trend in performance with ElChat? (ii) nol, what are the polential reasons for the divergence?
Safety. Table g shows that the results of Qwen3 align with those of Qwen2.5 in Figure 3 Specifically, CV outperforms ElChat on TOxIGEN and IMPLICIHATE by up to 5 points, while ElChat performs slightly better on TRUTHPUL.QA (except for Telugu). Notably, both ElChat and CV generally surpass the Chat baseline across tasks, indicating that they improve safety performance even in this newer, larger model.
Chat and Instructtion-folloiingg Table g demonstrates that ElChat significantly better recovers chat and instruction fooloooi-fooowiinggoooooo abilities compared to CV across all tasks and languages. For example, ElChat shows drops of up to 6, 6, and 0.41 points from the Chat baseline in IFEVAL, GSMSK, and MT-BENCHI, respectively. In contrast, CV experiences considerably larger drops of up to 41, 36, and 0.66 points for the same tasks. A similar trend is observed and CV outperform the Chat baseline; they are competitive in Bengali with 1-point difference, though CV outperforms ElChat in Telugu by 6 points. Overall, these trends are consistent with our previous observation in 5.1 that ElChat is more effective than CV in enhancing both chat and instruction-followie capabilities.
ALPACAEVAL. (Table 10 On MGSM (Table g both ElChat
Target: Language. Contrary to our observations with Qwen2.5 5 and Llama 3.1 ($5.2 Figure 4 left), ElChat does not consistently outperform Chat in Qwen3 for target language tasks, as shown in Table [ While we see consistent gains for Amharic, Bengali and Telugu show performance drops MT, MC, and GMMLU with docreases up to 18 points (c.g., Bengali GMMLU). Similar drops are also observed in Chat+ and CV, w whereas CPP generally maintains performance comparable to Chat.
32

---

Published in Transactions on Machine Learning Research (09/2025)
We hypothesize that Bengali and Telugu may already be well-represented within Qwen3 due to its signif- icantly enhanced multilingual capabilities. Indeed, Qwen3 was pre-trained on 36T tokens, covering up to 119 languages and dialects. This represents a substantial increase from Qwen2.5, specifically, 29 supported languages and 18T tokens, suggesting much broader and deeper understanding various languages. This hypothesis is further supported by Table 36 in Yang et al. 2025 which lists both Bengali and Telugu as officially supported by Qwen3, while Amharic not.
Finally, when comparing ElChat and CV, observe they perform competitively across tasks and languages, with maximum difference of 5.3 points in Amharic MT. This generally aligns with our observations in Qwen2.5 and Llama 3.1.
Source Language (English). Consistent with observations in $5.3 for Qwen2.5 and Llama 3.1, we observe from Table 13 that ElChat generally improves source language (English) performance compared to Chat+VV across tasks and languages. However, these improvements are marginal or modest, with a maximum gain of 6.3 points (Telugu MT). This smaller improvement in English indicates less performance degradation during the VE-process (i.c., less catastrophic forgetting of general English capabilities), suggesting enhanced robustness in Qwen3.
Comparing ElChat with CV, ElChat consistently demonstrates superior or equivalent performance in source language tasks across all languages. This aligns with our observation in $5.3 Notably, the gains tend to be more pronounced in the two generative tasks, SUM and MT, with improvements up to 19.3 points (Bengali MT).
Inference Efficiency. Finally, Table 19 shows that both ElChat and CV consistently provide speedup gains of up to 3.8x (Amharic SUM with FlChat) in generative tasks within Qwen3. While CV exhibits reductions in speedup ratios compared to Base+ VE (e.g., in Amharic and Bengali), ElChat maintains inference specdups similar to Ch across all tasks and languages. These findings are generally consistent with the results presented in $ for Qwen2.5 and Llama 3.1.
D License
This study uses various publicly available models and datasets with different licenses, as detailed below, all of which permit their use for academic research.
D.1 Models
Qwen2.5 and Qwen3 are distributed under Apache License 2.0. Llama 3.1 is licensed under the Llama 3 Community License Agreement 13
D.2 Datasets
XT-Sum is licensed under CC BY-NC-SA 4.0. Belebele and FL.ORES-200 are licensed under CC BY-SA 4.0. BBH, MMLU, GSMSK, and ImplicitHate are distributed under the MIT License. MGSM is distributed under CC BY 4.0. ToxiGen is licensed under Communtyy Data License Agrecment - Permissive Version 2.0. AlpacaEval, 1FEvll, MT-Bench and TruthfulQA are distributed under Apache License 2.0.
htttps://laaaa xata..com/llmms331ccnee//
33

---

Published in Transactions on Machine Learning Research (09/2025)
Table 3: Prompt template for each task and language.
Task
Language
Template
English
Translate English (X: target language): (sentence)
Amharic
177A,187 mg A +CM4": (sentence)
Bengali
QTEF कारााा लाताा कहमत. {sentence) =
Burmese
3a60800 @ifnne emargfflll (sentence)
En-Target MT
Gujarati
aio अobdnнei anjalE sel: {sentence)
Sinhala
@oG Booaaad a808maa woede:: (sentence]
Tamil
2y B Gri 5uو Glomm@@luuujzzssssp..
{sentence)
Telugu
eoffo Good Bewrió e6o50ooood:
(sentence)
English
Translate (X: a target language) to English: (sentence)
Amharie
K Øg X7M, nF +C740: (sentence)
Bengali
रासा atv किकनलक कमरान कगअन. (sentence) =
Burmese
Geusy zode35a omangiull (sentence) =
Target-En MT
Gujarati
IodddhHie лÍi RAgqlE अर:: (sentence)
Sinhala
Booessae gog a8dbmaD wodee: {sentence)
Tamil
5 (اا $ Glomm@@@uuujjeeeaay
(sentence)
Telugu
Beur's Kood esoffss e1ou50ozzo:: (sentence)
English
Write short summary of the following text in (langugge) Article: (text) Summary:
Amharic
Pa xV-3 OIFECT kPC 077249 k*tZ?= [text) hoC 0
Bengali
मिमनिर लायाी शारनास अरलारी लियमा ftext) नररककगा:
Burmese
eaoonoloooooo @éunomanle smpeiapeeao 6acs6:0l:
[text) zamia
SUM
Gujarati
+el attal quiRi ioolidllii silend qui.: (text] ziely:
Sinhala
BaD adde aooro Boweeed @csedes. 88c: (text) enchs
Tamil
&Cl 6 e sorfenw ااا @T TT5I
a (text) aा
Telugu
fold 33330 dwse aooodo Beuriso aossod.
angão: (text)
aooodo::
English
[passage} \n Question: (question) in A. fanswer 1) B. (an- swer 2) C. (answer 3 D. {answer 4)'n Answer:
Amharic
(passage) \n rft: [question] 'n A. (answer 1)/n B. (answer 2) |n C. (answer 3) D. (auswer 4) amAh:
Bengali
ppssagee) \n ghst: {question) \n A. fanswer 1Ann B. fanswer
2) yu C. (answer 3) D. [answer 4)',u Gेअस:
Burmese
ppassge \n surgá:: {question} \n A. (answer 1) B. {answer
2) Yu C. (answer 3)'n D. (answer 4) soa
Gujarati
{ {passage) \uu MBt: {question) \n A. (answer 1) B. (answer 2) yu C. (answer 3) D. (answer 4) 4 कqqाo:
Sinhala
{passage} in gafesos: {question] } \n A. fanswer 1 B. (an- swer 2) in C. (answer 3) D. fanswer 4)'n 88go (passage) In Gssire)): (question). 'nn A. (answer 1) B. (an-
Tamil
swer 2) (n C. (answwr 3) D. [answer 4 L [passage] \n Eoh (question) } \n A. (answer 1) B. fanswer 2) I C. (answer 3)'n D. (answer 43'n aasee:::
Telugu
34

---

Published in Transactions on Machine Learning Research (09/2025)
Table 6: Qwen2.5 2 7B chat, instruction-following and safety performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, re- spectively. Experiments are limited to models that use the chat template, except for EVELLL to verify the performance gain of CV over its adapted base model, Base+VE.
IF Eval
GSMSK
MT Bench MT-Bench
am
bn
my
ta
te
bn
Am
hn
to:
Base+ VE
.19
16
.14
.15
166
.17
.14
CV
.41
.36
.45
.36
.38
.38
-37
.65
.66
.73
.57
.56
11
.25
2.92
3.35
3.54
3.41
3.39
3.22
3.33
Chat
.70
.70
.70
.70
.70
.70
.70
.70
.70
.70
.70
.70
.70
.70
1.19
4.19
4.19
4.19
4.19
4.19
4.19
Chat++PT
.40
.43
.41
-31
-34
-35
-36
.63
.70
.66
.68
.70
.60
.71
3.55
3.51
3.52
2.93
3.19
3.38
3.29
Chat++VE
-39
.38
.45
.39
.40
.39
.40
.43
.63
.74
.47
.57
.29
.44
3.12
3.43
3.67
3:31
3.50
3.34
3.21
ElChat
.22
.54
.59
.52
.53
.52
.55
.66
.72
.74
.68
.72
.71
.70
3.30
3.59
3.81
3.58
3.75
3.69
3.58
TruthfulQA
ToxiGen
Implicit Hate
am
bn
my
gu
si
ta
te
am
bn
my
gu
ta
te
am
bn
may
gu
si
to
CV
.55
.57
.52
.58
.50
662
80
.21
.23
.22
.24
.23
.22
22
.16
.20
.19
.22
.20
.21
.19
Chat
.31
.50
.33
.41
-32
.33
-32
.10
.12
.07
.10
.09
.09
.09
.09
.12
.09
.10
.07
.09
.09
Clat+CPT
.55
.59
.38
.56
.47
.59
.55
.17
.17
.13
.16
.11
.13
.13
.14
.12
Chat VE
.59
.56
.51
.19
.59
.21
.19
.18
.19
.18
.18
ElChat
.44
.17
.15
.15
Table 7: Llama 3.1 8B chat, instruction-followiin and safety performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respec- tively. (L) stands for linear merging. Experiments are limited to models that use the chat template, except for TFEVAL to verify the performance gain of CV over its adapted base model, Base+
Iv'll
GSMSK
MT--Beuch
am
bn
my
si
ta
ba
13
DC
am
bn
te
Base VE
.17
.14
.13
.11
.15
.12
.12
CV
.38
.39
.42
.35
.41
.35
.35
.10
.27
.34
.30
.43
.40
.48
2.93
3.10
3.49
2.84
3.03
3.04
2.88
Chat
.73
.73
.73
.73
.84
3.93
Chat+CPT
.37
23
.32
36
.36
.46
.51
.51
.45
2.
2.76
2.15
2.72
Vhat+VE
.33
.37
.37
.32
.37
.36
.28
.26
.55
.38
.29
.44
.35
.07
2.28
2.85
2.64
2.29
2.54
2.71
2.34
ElChat
.45
.44
.54
.44
.47
.51
.47
.57
.57
.72
.56
.60
.56
.52
2.86
2.72
3.56
2.81
2.79
2.96
2.76
EIChat MMerg
27
338
.41
.30
.38
33
.31
.17
.54
.38
42
.35
.47
.00
2.29
2.79
2,86
2.42
2. 7.70
2.28
2.40
ElChat Copy
.40
.77
.3
.35
.40
.41
.39
.15
.44
.66
14
.42
.41
.36
2.61
2.82
3.09
2.45
2.41
2.70
2.48
ElChat (1.)
.48
.44
.55
.42
.50
.50
.48
.56
.55
.74
.55
.60
.55
.51
2.91
2.97
3.58
2.69
2.76
3.23
2.75
TruthfulQ A
Toxi
ImplicitHate
smn
bm
myy
gna
sd
ta
te
am
gu
st
ta
bc
am
my
guu
ta.
te
CV
.11
.30
.25
.10
.15
.42
.39
.17
.19
.21
.17
.18
.20
.11
.17
.17
.20
.15
.16
.17
Chat
.46
.30
.45
.08
.09
.08
.07
.10
.08
.09
.08
CPT
.80
.74
.94
000
.16
.13
.12
.13
.18
.17
.12
.15
.14
.13
.13
VIa
.30
.29
.09
.06
.30
.27
.21
.18
.22
.17
.22
.19
.21
.23
.14
.20
.17
.11
.17
.20
.21
EIChat
.33
41
.25
.15
.88
.22
.65
.10
18
14
.18
.14
.17
.15
.06
.16
.14
.17
.12
.16
.13
ElChat MMrrge
89
.33
.09
33
.28
.77
.53
15
21
18
.21
.19
.21
.22
.06
.17
.15
.18
.14
.16
.14
ElChal Copy
.42
.45
.7
.16
.22
.26
.60
.10
.99
.15
.19
.16
.18
.16
.06
.17
.15
.18
.14
.16
.14
ElChat (1.)
.43
.44
.23
.14
.46
.24
.63
.10
.18
.18
.14
.17
.15
.06
.16
.14
.17
.12
.16
.18
35

---

Published in Transactions on Machine Learning Research (09/2025)
Table 8: Qwen3 14B chat, Instruction-follovinng and safety performance. Darker blue and red shades indicate higher positive and negalive relative performance change over Chat per language and task, respectively. Experiments are limited to models that use the chat template, except for TFEVAL to verify the performance gain of CV over its adapted base model, Base+ VE.
IvEval
GSMsK
MT-Bench bn
am
bn
tc
am
bn
te
am
tc
Base+ VE
.27
.27
.26
CV
.24
.33
.33
.27
.69
.40
3.58
3.78
3.77
Chat
.65
.65
.65
.83
.83
.83
4.24
4.24
4.24
Chat+CPT
.57
.62
.63
.80
.84
.86
4.07
4.18
4.29
Chat++VB
-66
.58
.22
.79
.51
3.51
4.18
4.00
ElChat
.59
.61
.59
.77
.82
.84
3.83
4.13
4.11
TruthfulQA
ToxiGen
Implicit
am
bn
te
am
bn
te
am
bn
te
CV
.61
.81
.69
.32
.29
.26
.27
.26
.25
Chat
.45
.75
.66
.11
.13
.08
.11
.12
.08
Chat CPT
.61
.81
.67
.16
.10
.07
.15
.10
.06
Chat+ VE
.65
.88
.64
.29
.26
.23
.24
.24
.22
ElChat
.63
.84
.61
.27
.25
.21
.22
.23
.21
Table 9: MGSM performance Bengali (bn) and Telugu (te) by model. The experimental settings are the same as GSMSK. (L) stands for linear merging.
(a) Qwen2.5
(b) Llama 3.1 8B
(c) Qwen3 14B
Model
EM
Model
EM
Model
EM
bn
te
bn
te
bn
te
Chat
.3
.06
Chat
.30
.12
Chat
.31
.17
CV
.60
CV
CV
.28
Chat++CPT
.24
.14
Chat+CPT
.12
.02
Chat+CPT
.30
.16
Chat VE
.39
Chat+VE
.28
Chat
.20
ElChat (Ours)
.46
.35
ElChat (Ours)
.51
.41
ElChat (Ours)
.48
.22
ElChat Coppy
.28
.36
ElChat Merge
.43
.30
RlChat (T.)
.53
.42
Table 10: Win rates GPT-4 (1106 Preview) ALPACAEVAL v2.0. Darker red shades indicate larger negative relative performance change over Chat per language and base model, respectively. Experiments are limited Amharic (am), Bengali (bn), and Telugu (te) due computational constraints.
Qwen2.5 7B
Llama 3.1
8B
Qwnn3 14B bn
bu
tc
bn
CV
2.60
1.27
2.50
2.38
2.91
2.43
6.44
4.90
6.95
Chat
30.80
30.80
30.80
34.43
34.43
34.43
30.66
30.66
30.66
Chat+ CPT
4.67
3.07
3.75
2.16
1.68
2.53
10.08
24.22
27.09
Ch
2.09
3.65
2.08
2.33
2.10
1.62
8.42
9.12
6.74
ElChat
16.60
17.16
15.92
2.72
1.96
1.82
9.33
10.46
7.39
36

---

Published in Transactions on Machine Learning Research (09/202)
Table 11: Qwen2.5 7B task performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively. Note that GMMLU does not cover Burmese (my), Gujarati (gu), and Tamil (ta).
Target sum
English
Target mt
Target me
my
21
Rase
15.8
29.0
21.7..
21.5
23. 23.3,;
25.5
Pm.
4.6u,
32.2uu
15.7
22.A..
10/
24.8.s
18.3..
.57
Bas++CTT
17.3.
29.2.
21.7
21.4.
23. 9..
26.6
20. 20.
$1.2..
12.5...
30.9..
37
37.8..
35.5k.
13.8.
38.0..
74
.76
.67
.70
74
89
86
Buse+ VE
.71
19.0.
35.6,
38.5
29.7.
6..
38.5
22.5.a
36.9.
45.6.
39.2.
21.2.n
40.0.
32.0.,
73
Chat
14..
30.4.
10.
22.1.x
22.5 22.8
27.5.x
19. 1m
3,4.>
50 30.2.m
17.2..
21.2.
14.3.
25.7...
20. 30.1.0
-55
86
-30
58
47
54
49
Chut CPP
18.4.
32.2.
22.7
24.1.
26
20.
22
$0.9
42. 1..
30.3.,
96.5..
33.5..
42 42.6..
36.....
64
68
73
67
Chat+VE
29.7..
38.6,
22.9..
A..
30.0.
E3Chas
28.7.
35.5.
36.0
29.6,
7.G.
37.8%
30.7
28.1.
20.8%
42.4.
38.6,
37
25.8.
28.7..
.710
.72
.07
.00
English sum
Target
English mnt
Engliah me
ge
mry
Rase
24.4.
24.4,
24-4.
24.4,
24-4..
24.4.
24-4.
22.7.,
52.5.
26.6%.
48.8.
27.0..
36.7.
41.4o
CTP
22.6,
20.6.
23.4e
14.
23.3,
23.6..
22.0g
05.8u.
64.2..
60.8..
36.1.s
63.8...
42.4...
48. 18.5
.90
.90
.90
.90
.00
91
.99
Basc+
23.4.
22.5.
20.2.
22.9.
23.0,
31.8..
33.7.
28.
31.7..
36.1..
36.0.
Chat
242.
242..
242.
24/2.
2122.
24.2.
21.2
25.2.
f22.9k.
23.8...
29.A,.
31.1..
38.6.o
42.22..
592
192
522
512
52
912
Chut CPT
23.4.
2233
23.0..
20.6,
23.3.
23.0
22
47/A..
43.5 43.5
20.7..
47 47.0..
44.8
40.1..
47.0..
.91
.11
.02
91
.11
22.
23.5.
23.3.
23.0
25.2..;
35.5..
EBCLar
24.4.
24.1.
24.0a
24.1.
24.3
24.2.
23.88
44.7ua
44.A...
45.4
36.4..
40.7..
44.N...
44.4..
.91
.91
192
.91
.22
.91
Target gnamlu
English munlu
Ennglish bbh
my
51.
myy
Rase
47
.74
.74
.74
.74
.33
Rese+UCPT CPT
53
.71
.60
.71
.71
.70
.70
.70
.00
.43
A11
A3
A3
A3
Buse+VE
52
70
.72
.71
.71
.71
.71
.11
44
A22
42
Chat
.77
.78
.738
.78
.738
.728
.56
.56
256
76
76
%6
Chat+sCPT
52
70
.71
.71
.70
.70
.70
469
.51
.19
49
51
50
Chat
.70
ElCEas
.71
.71
.72
.71
.72
.71
.51
.99
51
51
37

---

Published in Transactions on Machine Learning Research (09/202))
Table 12: Llama 3.1 8B task performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively. (T.) stands for linear merging. Note that GMMLU does not cover Burmese (my), Gujarati (gu), and Tamil (ta).
Target
English
Target
Target
Base
11.0.:
27. .v.
18.5...
19.9.
20 2..
23.3..
18 4c.
9.8,
30.5 S...
21.4.:
30.0,
24.2..
36.2.s
31.0..
.35
/62
.51
52
.51
10.0
27. 5
2un.
19.4,
20.8.
22.8.,
18.11 18.1ex
24.3,
d1.0g.
26. 5..
55.9.1
331.5.:
$9.6,;
34.5ca
.65
.66
.20
-31
34
.37
.25
Base+VD VI
17.4w>
34.1 14.1.3
29.5va
38.7..
37.4a:
31.
30.7.a
39.7cx
40.3..
42.5..
.06
.55
10.5w>
a......
38.3..
38.7co
45.
39.2,
Chas
12.2
23.82
18.8...
22.
22.0..
26.5.a
20.4c.
9.5
39.7..
21.3.,
31.3a.
21.8.,
36.8x
12.2..
.36
J65
.42
.56
.02
.61
.59
Chat CPP
11.2
26. 8,
18.8
20.9.;
20.7..
20.4.:
19.6 6.
24.1.
40.0
28.0.;
31%
31.3.
39.3:
36.7.:
.34
.22
.28
.26
.29
.31
.27
Chass VD
17.0..
55.
29.9,
38
37.0.,
11.2..
41.3
41.3.,
15.5...
.61
58.
.50
.50
.52
1.a
20.6,s
38.1..
4..
21.1a
38.3..
16.
38.9
34.8.,
42.5aa
87.4..
EXChut Mergo
16.2..
53.0 9
Tax
20 4,
37 S..
36.7..
31 Ac.
23.1.
7..
45
45.
41
36.9..
47.5.
45.0..
.50
.52
.53
.50
.60
EXChat Coopy
19.3
53, 6.
26.5..
20.6 6
38.6.
38.0..
31. A..
19.0
17.9..
35,
38 16.9
25.1.1
42.9
38.1..
.07
.09
.57
.57
.72
.50
.61
EXChot (l)
10.3
55.
29.3
38
38.4.
20.8,
35 2..
3538
36.6...
.66
.55
/66
.56
English sum
Target
English
English mo
my
ge
ge
em
1x
gi
si
Base
23.5..
23.
25 5uo
20.5
23.5..
23.5.
25 S...
$3.3,
158 &
44.0.
57.9L.
10.00 ...
53, 1..
56.5.
.88
.88
.88
.88
.88
.88
.88
CPP
19.0..
17.2.
21.7.a
17.0.
10.4..
22.3.
21.8.
14.7.
47
42.9.
50.3.
46.4,s
41..6..
47.7.
.78
.82
.83
.00
.80
.81
Base
22.6.
22.9..
22.9,
O,.
42.6,
41.0...
23 28.4..
23.3.
25.6..
20. l..
23.2..
23.5. 5..
25 0u
$3.1.
35.6
30.7..
97.8...
30.4..
$4.2.
35.8.
.85
.83
.82
84
.83
Closs
26. S...
26.8,
2.....
35.8.
26.8.
26 26.8..
20.3,
67.5
36.2.
67.3
47.9.u
53.1.
66-8.
.91
.91
.91
.91
.91
.91
Chas CPT
22.6..
22.7.
21.4az
22.0.
24....
23.5..
25
$1.2.
36.35
37,
60.6
42.1.
39.
45.9.
.28
.83
83
.82
VB
A..
2..
2..
23.0,
23.7..
23.5..
26.5.
8..
03.5..
96.
36.0...
EXChot
24.1..
23.5a
20.
20.6,;
23.8.
24.2..
23 29.7cm
27.5u:
27.6
31.7
35.6,
25.1
31.5.a
36.0
.90
.89
.90
.50
.90
FXChmt Mergo
23.3..
23. aa
23.8..
22.8,
23.5
20.0.:
25-1ca
14:4,
34.0,;
37.2,
30.3..
35.9,
35.8..
.80
.52
.83
.70
[.5
.78
E3Chol (Cugg
24.2
24.0
26.1.
24.0.
24.0
24.3.
24.3.
27.7.x
27
38.3,
27.5 88.
3006,
31.5...
.90
.90
300
.50
.89)
.501
FlChnt
24.0.>
26.0..
24.2.
27.9.
.01
Targot gummuu
English mumtu
Eugllihh bbb
2m
hn
my
16
.in
1n
am
tn
Base
.31
.41
.16
.39
.53
.65
.65
.65
.65
.65
.65
.48
.46
.4f
.26
.46
.41
.45
Rase-CPPT
.00
.54
.37
Base+V
200
.88
.56i
.57
.47
.20
.47
.38
CV
.32
.31
-34
-35
.88
.50
.53
.47
.51
.48
.40
.61
.40
.40
.40
.40
.38
.41
Clusi
220
300
-30
.17
.67
477
.07
.67
.07
.67
.06
.56
.56
.66
.76
.56
.56
Chas CCP
128
.5i4
.512
.47
Chas VI
366
-96
.52
.52
.40
.30
.40
.A0
.40
E3Chml
.57
.36
.38
.96
.07
.00
.58
.61
.90
.60
.47
.47
.43
.44
.42
.45
E3Chol Milge
.55
.22
.38
.35
.44
.88
.53
.22
500
194
.19
.37
.37
.00
.37
38
36
.30
E3Chot Copp
.57
.57
.61
.45
F3Chot (1.)
.7
.96
.67
.47
A3
.48
A4
.48
38

---

Published in Transactions on Machine Learning Research (09/202)
Table 13: Qwen3 14B task performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively.
Target sum
English - Target mt
Target mc
am
bn
te
am
bn
te
am
bn
te
Base
17.0,.a
28.7p.
20.7..1
13.5,a
42.2..
36.4p.
.56
.84
.81
Base CPT
17.0,.a
31.1g.c
20.9..o
28.4o.1
43.5g.0
38.7
.77
.85
.79
Base VE
17.0,
34.0..1
30.5..1
19.7..:
32.9,
33.7p.
.78
.80
.72
CV
19.5..1
35.7G.1
31.2o.o
22.4o.1
33.4o.1
33.5,.1
.75
.72
.63
Chat
17.0g.a
32.0p.o
22.7a.o
13.6,..
41.3g.
33.9p.o
.53
.84
.76
Chat++CTT
19.1p.a
32.1o.o
21.8..0
26.2..
41.8p.1
36.8p.1
.76
.83
.76
Chat+VE
19.5..o
35.3p.
30.4g.o
23.7,.:
33.9gma
32.1p.
.78
.74
.65
ElChat
19.6..
35.5c.c
30.4..o
27.7..a
34.9,.a
31.7p.o
.78
.75
.64
English sum
Target +
English mt
English mc
am
bn
te
am
bn te
am
bn
te
Base
24.5p.a
24.5c.o
24.5..o
39.2..1
60.5..o
60.7
.94
.94
.94
Base CPT
23.1
23.5p.o
23.3..o
47.9...
49.7a.
51.4p.
.94
.94
.94
Base VE
24.2p.a
23.1p.0
23.8o.1
39.6,.a
38.5,.
35.3,
.95
.95
.94
CV
22.9g.a
22.6p.o
23.0n.o
42.7.a
31.8.
35.9,.
.94
.93
.94
Chat
24.2p.a
24.2c.c
24.20.0
38.6..o
59.6,.o
59.6g.e
.94
.94
.94
CPP
23.8..a
24.2p.o
24.10.0
54.8..1
59.0,.a
59.1p..
Vhat+VE
24.2..o
23.6..o
24.1
51.0p
50.6,.o
41.6p.a
.93
.94
.94
ElChat
24.1.. g.0
23.9.
24.4..o
53.0,
51.1g.
47 47.3p
.94
.94
.94
Target gmmlu
English mmlu
English bbh
am
bn
te
am
bn
te
am
bn
te
Base
.42
.64
.61
.80
.80
.80
.63
.63
.63
Base CPT
.54
.64
.60
.80
.80
.80
.59
.58
.58
Basc- VE
.57
.49
.45
.79
.80
.80
.55
.56
.56
CV
.37
.77
.77
.76
Chat
.37
.55
.77
.77
.77
Chat+GPT
.49
.57
.56
.76
.54
.71
.65
.65
.65
Chat+VE
.52
.39
.38
.76
.76
.76
.64
.63
.61
FlChat
.52
.40
.38
.77
.77
.77
.64
.64
.62
39

---

Published in Transactions on Machine Learning Research (09/2025)
Table 14: Qwen2.5 7B task performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively.
Target sum [ROUGE-L]
English + Target mt
cchrF+ F++++++]
am
bn
myy
gga
ta
to
am
bn
my
te
Base
1.8ka
1.7a.s
7.2..
2.8
42.4ps
2.4g.
18.7a.
3.8p
28.3o
12.3.
19.66.
9.9pa
20.7aa
15.3.1
Base+ CPT
2.1
1.8a.
5.5
3.0
42.5aa
2.2..
19.0ga
28.4
38.0..
24.5%
34.0%a
32.8.a
38. lo.2
33.7
Base+ VE
3.0aa
1.7
9.2.
4.2.
55.5.
2.8..
24.9.
20.5 20.5..
34.6,
35 ...
30.8..
35.7..
40. 1.
37.8, 8,.8..
CV
4.0g.
1.80.
9.7,a
5.2
55.1.s
4.1..
23.7..
20.1.a
32.8..
36.3,
35.8,
20.2..
34.6,
2,
Chat
1.5m.
1.3,
3.9..
3.0,..
35.9..
2.2,.
15.0.
2.9..
26.6,,
13.4..
18.8..
13.9..
21.5...
16.9...
CPP
2.5pa
1.4..
4.6.
5.20.3
49.4,.
2.4g.0
20.7pa
28. 2
37. 80.8
24.2
33.0g,
31.0 31.0..
36. 36.9g.o
32.3p
Chat++VE
3.704
1.6
9.8
4.8
56.0,.
3.6
25.1
20. 6
34.2..
35.9k.
36.6,.
29.9as
34. 94.
25.4p
ElChat
4.0Gz
2.0g
8.5
4.
56.1.
3.9,
24.1oz
25.5.
36.3,
33.5,,
To
34.8
39. Go.
34.3.,
English sum
Target + English mt
[ROUGE-L]
[chrF +
am
bn
myy
gpa
ta
to
bm
my
ta
te
Base
12.0g.
12.0.a
12.0ga
12.0g
12.0%a
12.0.
12.0.o
20.6
50.2
24.7
46..6..
25.1
84.8o.
39.2,
Base+ Base+CPPT CPT
11.0,.
10.1aa
11.4g0
7.2.a
10.9,a
11.1a.
10.2.t
13.0
41.4ga
38.6,.
43. 43.4,,
40.6,.
40.
43.8,
Base+VE VE
12.0.
1
11.7at
11.0,
11.0,
10.9,
11.4.
33.4.
35.4.
36.1..
36.5..
35.0..
34.9,
35.7.1
CV
11.5..
10.8,a
10.9ga
11.2..
11.0.a
10.8,
10.9
29.7
31.9,a
30.1.z
30.0,.
32.2..
32.3,.
34.1.;
Chat
11.8..
11.8,a
11.8m
11.8..
11.8.a
11.8.a
11.8
22.8.m
50.3.,
27. 5...
16.9..
28.6...
36. 36.2...
39.6...
Chat+CPP
11.5ga
10.7.a
10.9pa
9.9ba
11.1aa
11.1a.
10.8aa
44.5pa
40.7,.
28.2ga
45.4%,2
41.8.o
38. 18.
45.5.1
Chat+VE
11.9aa
11.0ga
11.0g1
11.2,.
11.2pa
11.1a.
10.9pa
33.0
34.4u,
31.4a.
36.3,,
36.3,.
35. 35.4g,
36.0pa
ElChat
12.2.
11.5,a
11.3.o
11.6,o
11.7,a
11.6,a
11.4,o
42.9.
42.3.
42.8.
44.4,,
43.4.
42.5.
42.2..
Table 15: Llama 3.1 8B task performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chal per language and task, respectively. (L) stands for linear merging.
Target sum ROUGE-L ga
English + Target mt [chrF+++]
am
my
to
AmD
ma
8u
Hase
1.6,..
1.9,.
6.2,.1
3.2,,
36.3..
2.0
18.3.
8.4,
35.6g.
16.9..
27.0ga
50.1
31 15..
PPse+tCPT
1.0..
199.
6.3...
2.9.
38.2,
1.6.
17.3..
22.0 22.0..
36.8.
21.1.
32.5..
29.0.
34.5.
30.5,
Dase+VVE VE
3.2.:
1.6..
8.8.
4.6...
55.0..
3.3..
25.2..
17.5..
36. A...
34.3..
36.4..
37 37.
41.9,.
37. 7..
CV
3.1ua
1.7..
8.2..
5.1.x
55.5..
1.1mo
25.8.
21.4.
3100..
35.6..
32.5..
35.6..
40.3..
38. 18.
Chat
1.70a
1.14,
4.20a
3.1g.
46.5..
3.0,a
21.3.
8.0,
35.7,.
16.8
28.1
20.9..
32.1.o
28. 7..
Clat+CPT
1.6.o
1.7..
3.5.
2.8.
36.5..
2.1
18.2.
21.9..
35.7a.
20.6,
31.5.a
28. 9..
34.3..
30.7..
Vh
2.7..
14,,;
8.S..
1.8...
51.7ux
3.4.a
25.9 25.9..
20. 1...
37.0,.
36.5,
38.0.z
38. 18.6...
42.8.,
38. S..
ElChat
3.4,a
1.3g,0
7.5.
4.8..
53.9,a
4.3,a
25.4,
19.1..
33.8,,
28.4,
35.0,
31.9,,
36.4,a
33.0,,
ElChat |Merge
2.5wa
1.6,,
8.9.a
5.0...
54.2ua
2.9
25.8.
20.4..
36.5,.
36 36.0,.
35 6..6
36 2..
41.6,.
38. 6,..
ElChat (Copy
3.2..
1.4.
7.6..
5.0..
53.7..
1.4.
2..
1..
33.0 G..
28 O,.
35.8..
32. 6..
37.1us
33.00 23.6,,
ElChat (1.)
8.7
1.3.
7.9.a
5.0g
54.1as
4.5
25.8.
18.9..
33.4g,
27.5 27.8.
35.0ga
11.1..
35.6,,
32. 2,.
English sum L ga
Target
English mt [chrF+++]
my
am
my
gu
Base
10.9..
10.9,a
10.9..
10.9,
10.9.
10.9..
10.9.
31.4,.
56.3..
42.2,
55.8
47.8..
50.9
14.1
BPse--CPT
8.7..
7.8.
10.0..
7.8.,
9.0.a
10.2.
10.0..
42.0..
41...
40.4.:
47.2.
43. Ln
39.3..
4199,
Base+VE VEE
10.7..
10.4..
10.5..
10-4a
10.5..
10.7..
10.6.
33.8..
37.7
36.4..
40.1a.
36.3..
36.2..
38.9,a
CV
11.6..
11....
11.40
10.9.a
11.0...
11.2..
10.8.
32.3..
33.8..
30.2..
34.1..
30.4..
32.5..
31. 31.6..
Chat
14.0.o
14.0aa
14.0.a
14.0aa
14.0..
14.0
14.0
27.4.
54.9t.
34.1.:
55.2,
45.
50.8..
54. 54.7..
CPa
10.6...
10.9..
10.4..
10.6,a
12.0,.
11.2.
11.1.
29.8..
37. 37.4.
35.1,
38.4.
39.4..
37.2..
41.3,
Vhat+VE
11.5,0
11.4a
11.6..
10.9,a
11.5gm
11.3,.
11.7..
30. 30.
34.7...
31.9..
35.7
31. 6..
34.8...
33. D,.
ElChat
12.1..
12.0..
11.9...
12.
12.6...
12.5..
12.3...
26.0...
26.3..
31.0,,
33.8...
23.8..
30.8..
6..
ElChat |Merge
11.3..
11.2,a
11.40.
10.4,a
11.1aa
9.3ka
11.1a1
26.7.a
36.8..
33.5g.
36.2,
34.4..
35.4aa
34.0,,
Elthat CCopy
12.2..
12.0,,
11.8
12.1aa
12.1..
12.0..
12.
26.2.aa
25.9..
31.
34.6,,
26. 26.5...
29 6...
30.3,,
RlChat. (L)
12.1.1
12.0g.
11.8p1
12.1ga
12.5ga
12.5g
12.3a
26.3..
25.3.
30.8p
34.3,
22.8,,a
30.7.a
32.2,
40

---

Published in Transactions on Machine Learning Research (09/2025)
Qwen3 14B task performance. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively.
Table 16:
English + Target mt Target sum [ROUGE-L
(chrF++ bn
am
bn
tc
am
te
Base
2.5g.0
0.0g.a
2.40.0
11.70.a
38.3.
32.6..1
Base- CPT
2.2o.0
0.0g.n
2.40.1
25.8..1
39.3p.u
34.6,.2
Base+ VE
2.6..2
0.0g.a
3.4p.
17.9..:
29.3a
29.6.
CV
4.1
0.0g.a
3.8p.
21.7..;
30.3p.
30.3p.1
Chat
2.3g.0
0.0g.a
3.0p.o
11.8..
37.2.
30.1p.
Chat+CPPT
2.90.1
0.0g.a
2.6..0
23.6,.1
37.7,.1
32.90.1
Chat VE
3.7..2
0.0g.o
3.1..0
22.8..2
30.2
28.2..2
EIChat
3.8
0.0g.o
3.0g.o
25.3
31.1a,2
28.0..o
English sum ROUGE-L
Target
+ English mt.
[chrF++]
am
am
tc
Base
12.0
12.0p.o
12.0..a
37.6,.
58.5g
58.8.
Base+ CPT
11.4g.0
11.3o.1
11.1o.0
45.50.2
47.2p.o
48.9o.a
Basc+ VEE
11.8,.0
11.0p.
11.6,.0
37.40.2
36.7a.
34.0p.
CV
10.8p.o
10.40.0
10.7
43.40.2
30.5p.1
35.5o.1
Chat
11.5p
11.5..0
11.5..0
36.6,.
57.3g.
57.4p.p
Chat+CPT
11.4p.
11.50.0
11.40.0
53.0p.o
56.4g.
56.6p.o
Chat+VE
11.5p.o
11.20.0
11.5..0
49.8..1
48.5g.
39.9o.a
EIChat
11.5..0
11.40.
11.70.0
51.1
49.0,
45.40.1
Table 17: Qwen2.5 7B inference speedup measured by the number of tokens generated per second. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and lask, respectively.
Turget
English
syr
20.0..
30.7ga
27.2u
82.40a
21.6.a
27.6Ga
83.2.,
30.9ga
3644.
30.0g
34.0.
17.5
PPT
307.
36.6.
98.8,
28.1,
28.0.
33.
35.5.
35.4.
St...
35/0.
363..
35.5
35.5.
28.5
93.1
15.2
17.1
Dase VE
100.4.
08.8
100....
1053.,
1023.
703.
88.5,
72:4.
00.
98.0.
102.7.
70.6
91.3
38.1
37.4
40.2
42.6
36.7
36.1
37.3
CV
121.8
76.2.
98.9..
08.4..
114.7
80.8..
66.8..
100.6,
73.6..
97.5
94.5.
70.4.
67.2.
82.3.
40.5
42.2
57.1
57.8
4..5
39.4
$5.4
Chas
20.4.
30.7..
20.6.1
27.8
20.8a
83.3,;
20.6.,
32.6.a
84.7
84.7aa
36.5v
47.9..
36.2.
86.5.
27.9
22.9
15.3
16.0
15.0
20.2
17.0
ChoteCPT
337.
32.2.
29.1.
20.3.
S3..
33.3.
26
$1.8.
31....
S3.
31.2.
353.
$1..
31.9.
98.8
23.5
15.5
17.2
16.7
17.5
Chats VE
1653.
1080.
RlChat
U7A..
708.
1134.
74..
91.6
37.4
Table 18: Llama 3.1 8B inference speedup measured by the number of tokens generated per second. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively (L.) stands for linear merging.
Engllihh
Tanget
Base
10
18..
15.5
31.0
31.5v.
13.5
16.0
12.7
CPP
31
331.1
31.6.
15.3
17.8
13.6
15.1
14.7
19.7
vg
114.7.
122.0
131.6.
122
37.01
37.8
UV
224.3.
1
1027.
1295.
135.6.
120.2
31.4
22.0
23
70.5.
1162.
105.6.
125..
003.
1.
212.
S1.b.
1380.
107.7.
136
89.8
107.2
44.5
34.1
36.9
35.00
37.6
67.8
(Capy
(1.)
75.T.
1
196..
145.
10.6.
1099.
769.
111
106
128
6..
105.8
40.8
37.7
47.5
41

---

Published in Transactions on Machine Learning Research (09/202))
Table 19: Qwen3 14B inference speedup measured by the number of tokens generated per second. Darker blue and red shades indicate higher positive and negative relative performance change over Chat per language and task, respectively.
Target sum
English
- Target mt
Target mc
am
bn
te
am
bn
le
am
bn
te
Base
14.6..1
13.5..1
13.2
15.4..1
15.6
15.6..1
17.7
16.3
12.6
Basc- CPT
14.6p.o
14.6G..
13.8
15.5.
16.1p.2
15.40.a
17.5
16.6
12.5
Base VE
52.3p
33.7
43.5a
36.3,.
29.6p.a
38.6o.a
17.8
17.5
17.8
CV
56.8
34.3o.1
44.0,.4
27.9..1
26.3o.2
35.2o.4
17.0
17.6
17.5
Chat
15.2
14.6..1
14.1p.1
15.7,.1
15.9p.
15.80.2
17.9
16.3
12.3
Chat CPT
CPPT
14.9p.1
14.5o.1
14.1g.a
16.0,.o
15.90.2
15.8..
17.4
15.9
12.3
Chat +VE
55.8p
34.3o.s
44.9,
28.1a.3
32.3pb
42.8o.7
17.4
17.6
17.6
ElChat
58.0g.a
34.5o.1
44.1n
43.5
30.9..
40.3o.a
17.4
17.6
17.8
42


Reviews:
[{"review_id": "UwkM6cjiLr", "reviewer": "Reviewer_BFxi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Addresses a realistic low-resource setting where base models or translated chat data are not available.\n- Leverages established techniques (VE, SLERP, token copying) in a well-integrated pipeline.\n- Includes instruction-following, safety, source/target language tasks, and inference speed.\n- Covers seven non-Latin-script, underrepresented languages using curated adaptation data.\n- Shows that merging and token copying each contribute meaningfully to final performance.\n\nWeaknesses:\n- ElChat is a combination of existing techniques rather than a fundamentally new algorithm or method. SLERP and token-copying are reused without innovation; the novelty lies in their integration only.\n- ElChat’s improvements over the Chat Vector baseline are modest and task-dependent. In some safety tasks (e.g., ToxiGen), and discriminative classification tasks (e.g., gmmlu), CV actually outperforms ElChat.\n- Only MGSM is used for multilingual reasoning, which is narrow in scope and covers just two languages.\n- No use of generalization/stress-test benchmarks like BigBench Hard (BBH).\n- Linear merging (ElChat-L) performs comparably in some tasks, yet SLERP is used by default without deeper analysis.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces ElChat, a method for adapting chat-based large language models (LLMs) to low-resource target languages without requiring access to a base model or labeled chat data. ElChat builds on vocabulary expansion (VE) by applying it directly to the source chat model and then recovering lost instruction-following capabilities through model merging (via SLERP) and special token weight copying. The approach is evaluated across two chat models (Qwen2.5 and Llama 3.1), seven typologically diverse languages, and a wide range of tasks including instruction following, safety, and multilingual understanding. The paper compares ElChat against strong baselines including Chat Vector (CV) and Chat+VE, and includes ablations and inference efficiency analyses.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Explicitly state in the paper that the novelty lies in the combination of known techniques under practical constraints, rather than in individual components.\n- Include more robust instruction-following or multi-turn dialogue evaluation in target languages.\n- Include results on AlpacaEval, Chatbot Arena, or similar frameworks to validate instruction-following behavior more holistically.\n- Add discussion or per-task breakdowns on where ElChat meaningfully outperforms CV, and where it does not.\n- Provide a rationale or deeper empirical comparison between SLERP and linear interpolation, especially since the latter sometimes performs as well or better.", "broader_impact_concerns": "While the paper does not raise immediate or serious broader impact concerns, it would benefit from a more explicit discussion on bias transfer, safety degradation, and potential misuse in low-resource language settings. These risks are especially relevant given the growing accessibility of adapted chat LLMs in under-monitored regions.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "venXC5BH0Z", "reviewer": "Reviewer_f5vZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, the motivation of this paper is very clear, and the work holds significant potential value for improving the fairness of LLM applications. The methodology employed is generally direct, and its effectiveness appears to be quite significant, as it demonstrably improves the performance of various LLMs on tasks in low-resource languages.\n\nHowever, the paper has several weaknesses, primarily in the following three areas:\n\n(1) Insufficient Experimentation: The experiments are limited, as the paper only compares models at the ~8B parameter scale. While I understand that conducting Continued Pre-training (CPT) on larger LLMs may pose practical challenges for academic institutions, this remains a weakness. Furthermore, the study does not report the performance of any recent state-of-the-art (SOTA) models (such as Llama 4 or Qwen 3) on the same tasks. This omission makes it difficult to contextualize the current progress in solving this problem.\n\n(2) Overly Brief Methodological Description: The description of the method is too simplistic, preventing readers from fully understanding the implementation details. The methodology section occupies less than a page, and many crucial details that should have been explained in the paper are instead briefly mentioned via references. Compounded by the lack of open-sourced code, the paper feels more like a high-level overview of a technical approach rather than a detailed study, which hinders reader comprehension.\n\n(3) Limited Novelty of the Method: Frankly, the method itself seems overly simple and appears to be merely a straightforward combination of existing techniques rather than a novel contribution.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel method for adapting Large Language Models (LLMs). The proposed approach involves conducting Continued Pre-training (CPT) directly on a chat model, followed by merging the weights of the original chat model with the newly trained one. A key contribution of this work is that it eliminates the reliance on base models, a common requirement for previous adaption techniques. The experimental results indicate that the adapted model achieves significant improvements in both performance on downstream tasks in less-resourced languages and in inference speed.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In addition to the weaknesses mentioned above, I believe the authors should provide an explanation for a somewhat counter-intuitive result observed in Figure 2(a). Specifically, it shows that the fine-tuned model achieves significantly better performance than the original chat model on the English-language versions of the TruthfulQA, ToxiGen, and ImplicitHate datasets. This outcome is unexpected and warrants further clarification.\"", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6r9NIxAdSg", "reviewer": "Reviewer_nw3R", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\n1.It is interesting to explore a new perspective of adapting the base LLM with VE (vocabulary expansion) on target unlabeled data.\n\n2.Vocabulary expansion is an effective method for adapting to training on unlabeled data. Vocabulary expansion is a good point for doing well on target unlabeled data.\n\n3.The proposed method makes sense in this setting and also perform well on several benchmarks.\n\n\nCons:\n\n1.About the description, “chat model” seems not so distinct with the so called LLM. What the core difference between the two kinds of models? Which point in the differences between chat model and LLMs make that we need to customize a new kinds of method. This paper mainly talks about chat model, so this paper should make them clear and also make it clear that the key contributions of the proposed method are indeed related to the chat model.\n\n2.As LLM is known trained on a very large scaled data, the vocabulary may be already large enough. In this way, do is still necessary to expand the vocabulary? \n\n3.As for the core operation, “VE”, it is not so clear about how to process VE. The authors mentioned that expanding its input and output head matrices with new tokens. But which part is the input head matrices? Which part is the output head matrices? How to expand with new tokens? The s specific roles (e.g. user, system, or assistant) is also not so clear. Why do we need a user/system/assistant in VE. VE is just to expand the vocabulary. How to expand the vocabulary with the user/system/assistant? The authors may provide some details in the appendix, but in the paper body, it is necessary to make the method clear to understand.\n\n4.The technical novelty reflected in the section 3 is also limited. VE is something about expanding matrices and prompt engineering (also cannot get any insights from the prompt engineering). Source and Target Chat Model Merging comes from existing papers.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "To adding new tokens and continuing pre-training on LLMs for target data, in this paper, the authors try to first adapt the base LLM with VE (vocabulary expansion) on target unlabeled data. The proposed method then converts the base LLM with VE to a chat LLM by adding a Chat Vector (CV) according to the weight difference between the source base LLM and chat LLMs. The proposed method, ElChat, performs more robust and competitive on target language and excel strong baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the weakness", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)