Long Chain-of-Thought Reasoning Across Languages
Authors: Josh Barua
Seun Eisape
Kayo Yin
Alane Suhr
Venue: ICLR 2026 Poster (2026)
Keywords: Multilingual
Reasoning
Long Chain-of-Thought
Primary area: foundation or frontier models, including LLMs
TL;DR: We analyze scaling trends, pretraining, and inference to understand long CoT reasoning transfer to non-English languages, demonstrating that post-training techniques overcome key limitations.

Abstract: While large reasoning models have shown remarkable ability to generate long chains-of-thought (CoTs) in English, we still lack understanding of how these long-form reasoning abilities transfer to the vast majority of the world’s languages. In this work, we systematically investigate four key stages of model development–scaling, pretraining, post-training, and inference–to understand how long CoT capabilities extend beyond English. We compare two reasoning settings across nine non-English target languages: En-CoT, where models process target-language inputs, but reason in English; and Target-CoT, where models both process inputs and generate long CoTs in the target language. We find that scaling reasoning model size improves multilingual task performance in En-CoT, but Target-CoT performance lags behind. This gap widens for tasks requiring long, multi-step CoTs such as mathematical reasoning. Shifting to pretraining, we find that adding a specialized reasoning stage enhances En-CoT performance but degrades Target-CoT, whereas broad multilingual pretraining improves both modes simultaneously. Given the scarcity of high-quality reasoning traces in languages other than English, we explore synthetic data curation approaches for post-training. We demonstrate that fine-tuning on reasoning traces
automatically translated from gold English traces outperforms fine-tuning on target-language traces distilled from large reasoning models. Finally, we report disparities in inference efficiency between languages and uncover language-specific failure modes
in CoTs. We release models, datasets, and code to foster further research.

Published as a conference paper at ICLR 2026
LONG CHAIN-OF-THOUGHH REASONING ACROSS LANGUAGES
Josh Barua Seun Eisape Kayo Yin
Alane Suhr
University of California, Berkeley (joshbarua, suhr)@berkeley.edn
ABSTRACT
While large reasoning models have shown remarkable ability to generate long chains-of-thought (CoTs) in English, still lack understanding of how these long-form reasoning abilities transfer to the vast majority of the world's languages. In this work, we systematically investigate four key stages of model development-scaling, pretraining, post-training, and inference-to to understand how long CoT capabilities extend beyond English. We compare two reasoning settings across nine non-English target languages: En-CoT, where models process target-language inputs, but reason in English; and Target-CoT, where models both process inputs and generate long CoTs in the target language. We find that scaling reasoning model size improves multilingual task performance in En-CoT, but Target-CoT performance lags behind. This gap widens for tasks requiring long, multi-step Co'Is such as mathematical reasoning. Shifting to pretraining, we find that adding specialized reasoning stage enhances En-CoT performance but degrades Torget-CoT, whereas broad multilingual pretraining improves both modes simultancously. Given the scarcity of high-quality reasoning traces in languages other than English, we explore synthetic data curation approaches for post-training We demonstrate that fine-tuning on reasoning traces automatically translated from gold English traces outperforms fine-tuning target-language traces distilled from large reasoning models. Finally, we report disparities in inference efficiency between languages and uncover language-specifie failure modes in CoTS. We release models, datasets, and code to foster further research. [
INTRODUCTION
Reasoning has emerged as central focus of large language model (LLM) research in recent years. By scal- ing inference-time compute to generate chains-of-thought before answering queries, LLMs have achieved expert-level performance in coding, mathematics, and scientific domains (OpenAl 2024 DeepSeek-Al (et al.] 2025). These long chains-of-thun tft-uhutttthtt demonstrate sophisticated reasoning processes processe-ccxplring different solution pathways, identifying and correcting mistakes, and setting intermediate subgoals- result in substantially longer reasoning traces. They not only improve accuracy but also expose intermediate steps, offering transparency, error-recovery, and interpretability (Korbak et al. 2025).
Despite this progress, reasoning research has focused almost exclusively on English. Even when models evaluated on multilingual tasks, they often generate the intermediate reasoning in English rather than matching the input language (Qi et al. (025). This creates both a scientific and practical blind spot. From a scientific perspective, it is unknown whether the mechanisms of long-fome reasoning generalize and transfer to other languages, or whether they rely on English-centre celis---entric pretraining signals. From practical perspective, producing reasoning steps in English reduces usability for billions of non-English speakers: it becomes harder to audit steps, reproduce solution, or diagnose errors when the explanation is not in the user's language. It also potentially limits reasoning quality tasks that are culturally or linguistically grounded.
Therefore, evaluating reasoning models requires assessing two distinct capabilities: their ability to understand inputs in languages other than English, and their capacity to accurately reason user's native language. In this work, we present systematic study of long chain-of-thought reasoning across nine languages beyond English: three high, three mid and three low-resuurce. We systematically compare
httpp: /rrpaa:/giiae whwb. Heekel -il. nggull -onguulll-nnCCooo Long-Co

---

Published as a conference paper at ICLR 2026
En-Only
En-CoT
Target-CoT
Model iuput
Model input
Model ingut
sum intearr beses
Pats justa ya misingi zose nambari kamili 6 9 anisaoo 17,, mi
Plex jamis ya misingi zote za namiteri kamili 6 > umbuoo 17, mi
b. 9 for which 17, aivisor 9797,
migsomyaji 976-
mgswarygji 97,
Long cfuint-o-lloughtt
Long chainn- ohiinghf
Long thiin-f-lliuughh
The prohlem the sum
Six the problem is: Find
Tamepews jikanu kuttifiro jumla besi zote kamili b 9 kwa antaaoo 17 ni kaguwanyo cha 97,, Kwanza, ninahiaji kudlesn kiracheulizwn hapa. Najua kwamio nunbani zilzcondikwa kwa tomu ya mi msingi wa Kwa rifano. 17, ni jumia yo 1-6 + 7---8 ambayo ni 1-8+7
inzeger boses > 9 for which 17, is
integer beses b for whichh 17, is divisor of 97g- We want all bases b >- ofter comeeting 17,
divisor of STy, We want all bases
>- flor,
converingg 17,
and 177,
the sumber 17,
10, number 17, divides 977 Piast, recall how to convent from base to hase 10.
divides 57y. Hist, recal how to convert from howe to hase 10.
Wai,, another thought less than 16, sn 7- would give b 7. which not grenter than 9, ilut's invilid Similorly, lower would give even smiller Hence, only valid boses are 21 and 49, So, thir sum is 70. The answer should be 70
divisar?
For digit 1721 are and both less than 21. In 9771, dight we 7. and U2 all less than 21 valid. For 1749
Vilevile, 9- 47-67
Hodo mabali 202, Kwa, hatfanyi kasi Kun hivyn zinzzofanya 12, 14, 50. Junll yau ai 12 14- 26
than 49, sp digits are 9, and o all less than 49 valid. So, both bases are valid, and their sum 70.
50 = 76. Kwwa hivyo, jumaa ni 76.
Model answer
Model answer
Model
... Bodh bases 21 and 49 satisfy the conditios. Thersfore, the all such
... Booth bases 21 and 49 sacady the digit conditions for respective bases.
Jumll besi bizi ni: 12 14
50- 76. Kwa hivyo, jusia besi
is is 21 49:0 70. /
of these boses is 21 40 70 /
kamili 6 2- mi: 76 X
Figure 1: Example inputs, long chains-of-thought, and answers for En-Only, En-CoT, and Target-CoT settings, drawn from DeepSeek-R Bicc--R--D RI-Distilll-L Disttil--llama-00B on AIME 2025, Orange boxes denote English text and blue boxes denote Swahili text. En-Only (input & reasoning in English) and En-CoT (input in Swahili, reasoning in English) lead to correct answers, while Target-CoT (input & reasoning in Swahili) contains a reasoning error (highlighted in red) and leads to an incorrect answer.
three reasoning setups (illustrated in Figure En-Only, where models process input and reason in English: En-CoT, where models process target-language inputs but reason in English; and Target-CoT, where models perform both processing and reasoning in the target language. We identify weaknesses and opportunities for improving multilingual LLMS across four key dimensions of model development: scaling, pretraining, post-training, and inference. We provide empirical findings on mathematical and general-domain reasoning, highlighting where advances in English reasoning fail to generalize, and where targeted multilingual interventions close the gap.
In particular, our study reveals that while model scaling and multilingual pretraining substantially improve cross-lingual comprehension, performance in sarget-lanuuage chain-of of-thhuggtttt remains far behind English even at large parameter scales. Moreover, lightweight post- training using translated reasoning traces lifts multilingual performance far more effectively than large-scale English post-training, enabling competitive gains for mid- and low-resource languages with modest data. Finally, inference-time analysis highlights distinct failure modes when reasoning non-English languages, underscoring the challenges unique to multilingual reasoning and paving the way for further improvements.
2 RELATED WORK
Long CoT Reasoning, Early studies demonstrated that inference- time techniques enhance LLM reasoning capabilities by enabling models to perform intermediate reasoning steps before generating final answers. These techniques include chain-of-thoughh prompting (Wei et al. (022), which guides models to explicitly show their reasoning process, and scratchpads (Nye et al. 2021), which provide dedicated spaces for intermediate computations. By scaling inference-time compute through longer CoTs, frontier LLMs have achieved expert-level performance on challenging benchmarks across programming and STEM disciplines (OpenAl 2024 Deeppeek-AA et al. 2025). Recent work has sought to understand inference- scaling (Wu et al. 2024 [Snell et al. 2025). ) with special attention given to long CoT reasoning as a scaling approach Yeo et al. 2025 Muennighoff et al. (025)..
Multilingual Reasoning, Early research on multilingual reasoning in LLMs found that English CoT
outperforms target language reasoning on grade-school math problems (Cobbe et al. 2021 Shi et al. 2023
2

---

Published as a conference paper at ICLR 2026
This finding motivated series of works that sought to bridge the multilingual gap by using English as a pivot language when reasoning. Researchers have explored this approach both externally, by translating
non-English queries into English before processing (Huang et al. [023 Qin et al. 2023 Ko et al. 2025).
and internally, by aligning the model's latent representations across languages to an English-centre space
(She et al. 2024 Huang et al. 2024 Yoon et al. 2024 Ruan al. 2025).
These works were conducted in the short CoT setting, where reasoning chains are typically a few steps. Recent studies have explored whether more advanced inference-time scaling techniques, including long COT, transfer across languages. Yong et al. (2025) demonstrate that scaling inference compute for models fine-tuned on English reasoning traces improves mathematical reasoning in high-resoure, languages but not low-resource languages. [Son et al. (2025) find that inference- time scaling methods such as budget forcing are comparable to traditional scaling methods like best-of-N once constrained to similar levels of inference FLOPs. These studies, however, share key limitation: they focus primarily on models that perform reasoning in English. Only recently has research begun examining scenarios where models reason directly in non- non-Englissh languages. Qi et al. (2025) identify trade-off between language compliance and accuracy, while Wang et al. (2025) explore the phenomenon of language mixing during reasoning. Our work adopts a complementary perspective by exploring the cross- poss-linguul transfer of long CoT reasoning through all stages of the model development pipeline.
EXPERIMENTAL FRAMEWORK
Preliminaries on Long CoT reasoning. We use the term long chain--ofthought (long CoT) to refer to sequences of thinking tokens that extend far beyond typical model responses- tens of thousands of tokens and demonstrate complex behaviors such as branching, backtracking, and verifiiciiin (Gandhi et al. 2025). Training models for long CoT typically involves two stages: supervised fine-tuning (SFT) on datasets of questions paired with reasoning traces followed by reinforcement learning (RL) that rewards correct final answers using techniques like PPO (Schulman al. 2017). However, developing long CoT capabilities in non-English languages presents significant challenges. High-quality reasoning traces for SFT are scarce outside of English, and RL-based reward shaping is highly sensitive in non-English contexts, leading to poor language compliance PPark et al. 2025).
Reasoning setup. Following Ko al. (2025), we decompose a model's performance on multilingual reasoning tasks into two distinct factors: understanding the input language (Lipa) and reasoning in particular language (Ireason). Throughout this paper, we refer to the language of the input and by extension, the user's desired language for the interaction as the target language. To isolate the influence of target-language comprehension from reasoning capabilities in task success, we consider three experimental settings (see Figure []. En-Only refers to having English as both input and reasoning languages, and showcases the English baseline performance. In En-CoT, the model receives inputs in the target language but reasons in English. In Target-CoT, both input and reasoning are in the target language. Although target- language solutions would be ideal, we instruct models in the En-CoT setting to output solutions in English due to validation challenges- short non- -nnglih texts with mathematical notation often cause inconsistent language detection. The En-CoT setting evaluates comprehension: weak performance compared to En- Only indicates difficulty understanding target-language inputs. The Target-CoT setting evaluates reasoning: comparing its accuracy to En-CoT reveals the model's 's ability to reason in the target language.
Languages. On top of English [EN], we study three high-resource languages (Chinese [ZH], French Japanese [JA]), three mid-resource languages (Afrikaans [AF], Thai [TH], Latvian [LVJ), and three low-resource languages (Marathi [MR], Telugu [TE), Swahili [SWJ). We assign resource levels based on each language's 's representation in the mC4 pretraining dataset (Xue et al. (021). These languages cover broad mix of scripts, geographic regions, and language families, allowing us to study cross-lingun COT reasoning at scale (see Appendix A. for more details on each language).
Multilingual benchmarks. We perform evaluation on two types of tasks: mathematical problem solving and multi-task language understanding, For math, we use MATH-500 (Hendrycks et al. 2021) and AIME-Combined (concatenating AIME 2024 and 2025 (Art of Problem Solving (025)). We automatically translate MATH-500 and AIME 2025 problems using Gemini-2..-Faashh and use the MCLM benchmark (Son et al. 2025) to source translated versions of AIME 2024 problems. For general reasoning, we use MMLU-ProX (Xuan et al. 2025). a multilingual, multi multi-disciplinary benchmark that tests knwwledge-groundee reasoning beyond math. It covers all our languages of study except Latvian. Additional details on inference and evaluation can be found in Appendix A.5
3

---

Published as a conference paper at ICLR 2026
Experiments. In Section f report the effects of scaling model size, separating gains in target language comprehension from target language reasoning. In Section holding scale and post-training fixed, we compare pretrained backbones to disentangle the effects of a ceasoning-speciallinn
stage versus broad multilingual coverage. In Section 6 we examine post-training. fine-tuning with small language-specifit datasets created either by translating English traces or by distilling target language traces. Finally, analyze inference behavior in Section language-specific failure modes.
7 quantifying efficiency-accuracy trade-off's and cataloging
SCALING MODEL PARAMETERS
Prior work has consistently shown that scaling model parameters improves multilingual performance across various tasks (Conneau et al. 2020 Li et al. 2021 Chowdhery et al. 2023 He et al. 2025). We revisit this question in the context of long CoT reasoning. An ideal scaling expenment should (1) keep the post-training recipe and dataset fixed, (ii) use common pretrained backbone to reduce corpus/tokenize differences, and (iii) span multiple sizes to reveal scaling trends. We meet these criteria with the DeepSeek-RI-Diste series: the 1.5B-32B variants share the same post-training procedure on the same 800k reasoning traces and derive from the same base family (Qwen 2.5), allowing us to attribute changes in performance primarily to parameter count. Increasing model capacity through scaling may not uniformly improve input comprehen- sion and reasoning in the target language. Thus, we report performance in both En-CoT (comprehension) and Target-CoT (comprehension + reasoning) to pinpoint where scaling provides the most benefit.
En-CoT performance. We first examine how increasing model size affects input comprehension in the target language. F En-CoT results, where models receive inputs in the target language but perform reasoning in English. The scaling trends are aagguag--eeenddeen such as Chinese and French, all model sizes (1.5B, 7B, 14B, and 32B) perform on par with the En-Only baselines (dashed horizontal lines), suggesting that target-language comprehension is already strong even in smaller models. For Japanese, Afrikaans, Thai, and Latvian, the 7B models are closest the En-On En-Only baselines, and larger models remain within roughly 10% of En-Only performance. This relatively small gap indicates that, once pretraining exposure is sufficient, scaling model capacity largely removes comprehension as barrier to cross-lingual reasoning. In contrast, lower-resource languages (Marathi, Telugu, Swahili) remain substantially below the En-Only baselines at every scale, including 32B. This persistent deficit highlights that English reasoning does not universally solve cross-lingual long CoT, because comprehension bottlenecks can still dominate for languages with limited pretraining coverage.
For high-resource languages
Target-CoTl performance. As shown in Figure 2 reasoning in the target language (Target-CoT) never approaches English reasoning capabilities, even for high-resource languages like Chinese and French. Remarkably, at 32B parameters, all tested languages perform at or below the 7B En-Only baseline- model with 4x fewer parameters. Lower- resorrres languages (Marathi, Telugu, Swahili) show virtually no sensitivity to scale, with performance remaining near zero. Overall, switching from English target language reasoning at 32B reduces accuracy by 28.8% on average. These results reveal that while scaling effectively addresses input comprehension challenges, it fails to enable sustained, structured reasoning in non-English languages. This contrasts with findings on short CoT, where performance gaps stem primarily from input comprehension rather than the language used for reasoning (Ko al. 2025). In Appendix B.2, we provide evidence that the length of the reasoning chain explains this discrepancy and observe cases where Target-CoT oupprrorrs En-CoT.
5 MULTILINGUAL PRETRAINING
Recent technical reports indicate that companies are revisiting pretraining for reasoning, adding dedicated "reasoning stages" (Yang et al. 2025a GLM 4.5 Team et al. 2025) to improve foundational capabilities. Yet the relative contributions broad multilingual coverage versus domain-specialized reasoning data to downstream multilingual reasoning remain underexplored. We take first step toward disentangling these factors through a controlled study that fixes model scale and post-training while varying only the pretrained backbone.
Models. Qwen 2.5-7B provides a strong multilingual base trained on 18T tokens across 29 languages, including coverage of Chinese, French, Japanese, and Thai (Yang et al. 2025b). Building on this base, Qwen2.5-Mar ---55--Mt------ 7B (Yang et al. (024) continues pretraining with over IT totenns of math-focused data in English and Chinese, enabling controlled test of domain-specialized pretraining when multilingual
4

---

Published as conference paper at ICLR 2026
Chinese
Japanese
0.8
0.5
0.0
0.7
0.7
0.6
0.5
0.6
0.5
0.5
0.5
0.4
0.3
0.3
0.3-
0.2
0.2
0.2
0.1
0.0
En.CoT
CorgeeCoT
En-CoT
Cor
En-CoT
Target
Afrikaans
Thai
Latvian
Accuracy
Et-COT
Terel-CSS
Ee-CUT
Terge-Cot
Tegel-Cri
Marathi
Telugu
0.8
0.8
0.8
0.7
0.7
0.7
0.6
0.6
0.6
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.7
0.2
0.7
0.1
0.1
0.1
0.0
0.0
En-CoT
CoT
0.0
DoggttCoo
SeekSeek-R1- Rii-----iiiil--ee--....
DeepSeek-R1-De Dii.RRI---Diii--------
En-Only (1.58)
En-Only (148)
DeepSork R1 Distill Distill-Qwen-78 Qwen-7B
DeepSock R1 Sastill--wan-3ar 328
En-Only (78)
En-Only (328)
Figure 2: Evaluation of scaling trends in DeepSeck-R1-Dee RI-Disill models on AlME-Combined. For high- and mid-resource languages, En-CoT performance increases and approaches En-Only performance with scale, while Target-CoT performance is consistently lower than En-CoT, highlighting target-language reasoning as the bottleneck.
coverage is held constant. To examine how broad multilingual coverage influences multilingual reasoning. we select two additional models. (Wwe3-8B-Basse is pretrained on 36T tokens across 119 languages and dialects, with coverage for all tested languages Yang et al. (025a). Gemma3-12B-PP (Gemma Team
al. 2025) pretrained on 12T tokens covering over 140 languages.
Experimental setup. We perform supervised fine-tuning of all models 20k English reasoning traces
which encompasses math, code, and science questions Guua et al. (025) (see Appendix A.3 for complete details). We only evaluate mathematical reasoning
randomly sampled from OpenThouuhhs3-11222
500, (IME-Combined) tasks to align with the mathematical pretraining of Qwen2.5-Math.
Impact of specialized reasoning pretraining. We compare Qwen2.5-7B with Qwen2.5 1rem2.5-MMthh-7B (Table [ to measure the impact of specialized reasoning pretraining. In En-CoT on MATH-500, most languages small lifts, though Swahili declines by 5.4%. By contrast, Target-CoT drops sharply for many languages (French -46.0%. Afrikaans -39. -3,44%), suggesting weakened target language reasoning despite in-domain content. Surprisingly, this degradation occurs even in Chinese (-36.4%), despite Chinese data being included in the reasoning pretraining stage. AlME-Combined shows the same asymmetry: En-CoT improves by 5.6% average while Target-CoT falls by 4.3%. Hence, specialized pretraining can improve English reasoning within its domain but undermines target language performance, even for languages explicitly included in the specialized training stage.
Impact of multilingual pretraining, Comparing scores between the Qwen 2.5 models and Qwen and Gemma 3 is challenging due differences pretraining, architecture, and scale. We therefore
Qwen naming convention for base models in the Qwen 3 series, hence the discrepancy.

---

Published as conference paper at ICLR 2026
Table 1: Evaluation of each base model after identical English post-training. AVG excludes English. and EPR (English Performance Recovered) is computed as AVG EN (%), measuring cross-lingual transfer efficiency. Pretraining on math data improves En-CoT performance but degrades Target-Col Broad multilingual pretraining in Qwen3-8B-Ban and Gemma3-12B-PP recover 85-97% of English performance for En-CoT while also yielding substantial EPR gains for Target-CoT.
Base Model
EN
ZH
FR
JA
AF
TH IV
MR
TE
SW
AVG
EPR
MATH-500
Qwen2.5-7B
90.2
89.6
89.6
87.0
88.6
87.0
82.2
74.8
74.2
50.4
80.4
89.1
En-Coft
Qwen2.5-Math
91.4
81.8
80.8
81.4
88.3
Qwen3-88 8B-Base
94.6
96.0
94.4
91.6
93.6
89.8
90.2
71.4
90.4
95.6
Gemma3-12B-B
76.6
75.8
77.8
76.4
75.2
73.8
74.6
75.6
72.8
67.0
74.4
97.1
Torget-CoT
Qwen2.5-7B
90.2
76.2
67.2
33.4
45.0
28.6
19.4
11.2
9.4
0.2
32.3
35.8
Qwen2.5-Muth-7B
92.2
39.8
21.2
21.6
5.6
11.6
6.4
3.2
3.6
0.2
12.6
13.7
Qwen3-8B-Bass
94.6
70.6
89.2
77.6
76.8
77.4
70.0
45.6
30.0
21.2
62.0
65.6
Gemma3- 12B-PT
76.6
41.6
55.4
32.8
60.4
34.4
52.8
40.4
39.7
51.8
AIME-Combined
Qwen2.5-7B
29.7
28.3
26.0
29.0
27.3
25.7
21.0
16.7
23.7
79.0
En-CoT
Qwen2.5-Math-77
36.7
31.3
38.0
33.7
36.3
30.7
33.3
25.3
27.0
8.3
29.3
79.8
Qwen3-8B-Bal
42.3
43.0
45.7
40.0
46.0
44.0
41.7
42.0
34.0
24.7
40.1
94.8
Gemma3- 12B-PT
14.3
12.0
11.7
10.3
13.7
11.0
12.3
13.3
14.0
11.3
12.2
85.2
Target-CoT
Qwen2.5-7B
30.0
21.3
15.7
8.0
6.0
2.3
0.7
0.0
0.0
0.0
6.0
20.0
Owen2.5-Math-
36.7
1.7
Qwen3 -B-BeBs
42.3
24.3
31.0
21.3
23.3
22.3
5.0
37.6
Gemma3- 12B-PP
14.3
4.0
9.0
3.0
7.7
4.7
6.3
3.3
2.0
5.7
5.1
35.5
focus the percentage of English performance recovered (EPR) for each model. Broad multilingual pretraining in Qwen3-8B-Base and Gemma3-12B-PT nearly closes the target language comprehension gap. In En-CoT, both models recover least 95% of En-Only performance MATH-500 and 85-95% on
(Table [ with shortfalls concentrated in the lowest-resource cases. However, when reasoning the target language, large gaps remain with EPR ranging from 35% 65%. This pattern indicates that multilingual pretraining establishes the foundation for understanding non-English inputs, but further work needed to teach models to produce long, structured CoT' in those languages. Importantly, the EPR of Qwen 3 and Gemma 3 are substantially higher than the Qwen 2.5 models, demonstrating the advantage of broad language coverage pretraining for more balanced reasoning.
6 SYNTHETIC DATA FOR POST-TRAINING
In Section 5 we found that multilingual pretraining helps bridge comprehension gaps between languages, yet the ability generate long reasoming chains beyond English remains brittle. This section explores whether post-training can meaningfully improve langu-ga reasoning. A fundamental challenge immediately emerges: high-quality reasoning traces are rarely documented in languages beyond English and Chinese. This scarcity raises critical question-hove should we optimally curate synthetic data for multilingual reasoning? We identify two primary approaches. Translation- based approach: translate existing English reasoning traces into target languages, leveraging the extensive availability of English data. Direct distillation approach: employ language forcing to directly distill larget-language traces from advanced reasoning models like DeepSeek-RI R1, potentially preserving aaguaae--peciffe reasoning patterns. In this section, we investigate both the effectiveness of post-training for target- langu-la reasoning and compare these two synthetic data curation strategies.

---

Published as a conference paper at ICLR 2026
6.1 EXPERIMENTAL SETUP
We curate two synthetic datasets from s1k (Muennighoff et al. 2025). a dataset of 1,000 high-quality English reasoning traces distilled from DeepSeek-RI. Translated-slk: translated version using Gemini-2.0-Flas We selected Gemini-2.0-Flash after verifying that it outperforms strong machine translation models on the FLORES-20 200 benchmark and achieves higher quality estimation scores when translating reasoning traces (..oiixxB//o// [B6 for details). Distilled-sIk, directly distilled traces from DeepSeek-Re -L,, using language forcing to elicit reasoning in target languages. We continue re-sampling until all 1,000 distilled traces for each language pass our language compliance check (Appendix [A.7]. For each dataset and language combination, we fine-tune separate Qwen3-8B-Bas models, resulting in 18 models trained exclusively on target-language supervision. Appendix A.3 provides detailed hyperparameter configurations for training. We evaluate on both mathematical (MATH-500, ( and general (MMLU U-PrrX) reasoning tasks. As a baseline, we include Qwen3 3-seBBas fine-tuned on 20k English traces from 3penThoughts3 -eeff-rrro model in Section 5) Although this introduces a difference in compute, this setup represents realistic tradeoff practitioners face: choosing between abundant English data sources and smaller larget-languge datasets. In Appendix [8.4, we present results under -euuivleeuuvvlnntt conditions. For reference, we also include results from Qwen3-8B, a state-of-the-ar reasoning model at this scale.
(see Appendix
6.2 RESULTS
Translation vs Distillation. In Table translation emerges as the generally stronger and more stable approach over distillation, achieving particularly substantial gains in Chinese (+24.2%) and French (+9.2% when averaged across the three benchmarks. Most other languages show more modest improvements of 1-4% on average, with Marathi being the sole exception where Distilled-slI outperforms counterpart. For mid- and low-resource languages, the performance gap between the two approaches remains relatively narrow, suggesting that practitioners could reasonably choose either method based on available resources. However, several open questions remain. First, how do these approaches scale with increased data? Additionally, as next-generation reasoning models develop stronger multilingual capabilities, the relative effectiveness of distillation methods may evolve, warranting continued evaluation of both strategies.
When to Use English vs Target-language Data. Given the scarcity of target-language reasoning data, practitioners must choose between leveraging abundant English resources or investing in smaller target-language datasets. Our experiments reveal that this choice depends on the target language's 's resource level. For high-resource languages like French and Japanese, fine-tuning on Open OpenThoughts332222 --0k (English-only) achieves modest improvements over both Translated-s and Distilled-s/ suggesting these languages can leverage cross-lingual transfer from English reasoning data. In contrast, mid- and low-resource languages show substantial gains from target-language fine-tuning, achieving competitive performance with just 1,000 traces-200 less data than the English baseline. Remarkably, this minimal fine-tuning even enables our models to surpass Qwen3-8B across Marathi, Telugu, and Swahili. These findings offer clear practical guidance: while target-language synthetic data benefits all languages, it becomes essential for mid- and low-resource languages.
variants,
We provide additional validation in Appendix B.4 under compute conditions and in Appendix B.5 using Olmo3-7B as the base model.
7 ANALYSIS OF LONG COT INFERENCE
In this section, we examine the chains-of-though themselves to identify weaknesses that benchmark scores alone cannot reveal. Our analysis focuses on two key areas: inference efficiency trends and error patterns across languages.
Inference Efficiency. Section 6 showed that fine-tuning on target- languae data improves Target-CoT performance over English data, but the inference efficiency of these two approaches remains unclear. When generating long CoT's spanning tens of thousands of tokens, cost can be as important as accuracy in deployment. Drawing inspiration from a similar analysis done by Yong et al. (2025), we plot accuracy and Translated-s1k (Figure 3). We observe strong negative correlations between cost and performance in both settings -0=40.824 and r= 009011.. We also find interesting trends within each setting. The model trained on English-only data is significantly more efficient in English, yet these gains do not transfer to other languages. In the left plot, English lies on the efficiency frontier and is far from the best-fit line that characterizes the other
against average response tokens for models fine-tuned on
7

---

Published as a conference paper at ICLR 2026
Table 2: Evaluation after fine-tuning Qwen3-8B- on different synthetic datasets. All evaluations are done in the Target-CoT setting. Subscripts are calculated using OpenThoughts3-2222 as the baseline for each benchmark. Qwen3-8B a state-of-the-ait reasoning model, evaluated with thinking mode enabled for long COT. Gray indicates the absolute difference is less than or equal to 2 points. The AVG excludes English and ignores missing entries ('-"). Fine-tuning with translated data leads to higher performance than distilled data, and target-language data outperforms the English baseline, despite using 20x less data.
SFT Dataset
EN
FR
MATH-500
Open Thoughts3 20k
94.6
75.8
89.2
77.6
76.8
77.4
70.0
45.6
30,0
21.2
62.6
Translated-slk
92.6
2.0
87.2
87.0
2.2
70.0
1.8
87.8,
11.0
81.6
83.2 18.2
70.8
7
22.2
724,42,
64.4 (45.2
78.3
Distilld-sIk
92.6
60.0
15.8
78.8
64.4
8.4
83.8
82.0+12.0
36.2-30.0
67.2
37.8
72.8
Qwen3-8B (Thinking)
85.2
47.4
8.4
71.9
AIME-Combined
Open Thoughts3 20k
42.3
27.0
31.0
21.3
23.3
22.3
12.7
5.0
3.3
0.3
16.2
Translated-sIk
33.0
29.3
12.
29.0
2.0
160
5.3
27.7
4..4
19.3
24.7
422.0
14.7 14.7.9..
12.0
11.3 (11.0
20.8
Distiled slk
33.0
10.7
16.3
21.0
13.0
28.3
28.3
19.7-
19.7
22.
16.3
13.7
12.7
17.5
Qwen3-8B (Thinking)
26.0
4.3
1.0
27.5
MMLU-ProX
Open Thoughts3 20k
71.1
59.4
66.2
53.9
57.1
52.9
31.5
15.5
13.9
418
Translated-sIk
69.0
0..
59.5
6.7
41.2
12.7
55.1
2.0
49.1
40.0...5
18.0
2655112 112.6
45.5
Disilled slk
69.0
32.0
48.0
38.8
53.2
48.5
40.7
35.9
20.
39.8
Qwen3-8B (Thinking)
78.1
30.8
1.5
46.2
ThonThought
Translated-sik
0.824
0.915
FC
En
Accuracy (%)
30
Zh
30
ZR
Af
T
20
20
Th
Mc Te
Sw
10
17.5
20.0
22.5
25.0
27.5
30.0
17.5
20.0
22.5
25.0
27.5
30.0
Average Response Tokens (thousands)
Figure 3: Inference efficiency of fine-tuned models on ALME-Combined in terms of tokens. Accuracy negatively correlated with cost. Fine-tuning on translated data (right) narrows the efficiency gap across languages.
languages. In contrast, fine-tuning models on target-language data narrows the efficiency gap across languages (right plot). This approach benefits low-resource languages, though may reduce efficiency in high-resource languages that could otherwise leverage rioss-lnggaal transfer from English. While token counts directly relate to computational costs, they may not fully capture information content due to tokenization disparities across languages (Petrov al. 2023). In Appendix B.7 we bytes instead of tokens and observe weaker negative correlations (r 0.155 and 7 0.355), suggesting that raw information content and performance less related.
Error Analysis. Complex problems requiring long CoTs manifest distinct error patterns across languages. Using taxonomy of five errr categories (described in Appendix A.8. analyzed incorrect responses
8

---

Published as a conference paper at ICLR 2026
Japanese
70
70
70
50-
40-
40
40
20
20-
70-
10-
10-
10
Afrikaans
Thai
Latvian
Percent of Total Errors
70
70
70
50
50
50
40-
20
20
20
Marathi
Telugu
Swahlll
60-
60-
60-
50-
50-
50-
30-
30-
30-
20
20
20
Problem Comprehension
Reasoning
Output Generation
///A Target-CoT
Conceptual
Figure 4: Distribution of error types found in incorrect responses from DeepSeek-R1-Distiee 70B AIME-Combined. The majority of emros in En-CoT stem from reasoning mistakes, while Target- CoT exhibits higher proportion of output generation errors and conceptual errors compared to En-CoT.
Llam on AIME-Combing We leverage Gemini-2.5 2.5-Prro for emor classification, with native speaker validation confirming accuracy (see Appendix(A.9) A. for details). Figure reveals fundamental asymmetry in how emors manifest across reasoning settings. When models reason English (En-Co'T), nearly half of all emrrr (47.6%) stem from faulty reasoning steps, while conceptual misunderstandings account for 21.4% and output generation errors are negligible at 0.7%. In contrast, when reasoning occurs in the target language (Target-CoT) the error profile shifts markedly: reasoning errors decrease to 34.4%, while language specific challenges become more prominent-oue output generation errors increase significantly to 11,%, and conceptual emrors rise slightly to 24.9%. Interestingly, problem comprehension and calculation errors remain relatively stable across both settings, indicating these error types are largely independent of the reasoning language choice.
from DeepSeck Biieeck--R--istiiii I--iitiill-Laaee -laa-LLaaa----- 70%
I -anguage-specffic failure modes also emerge. Telugu's conceptual error rate doubles when reasoning in-language (28% of emors Target-CoT 14% of errors En-CoT), while Latvian shows persistently high comprehension errors regardless of reasoning language (about 34% of errors). These patterns suggest En-CoT and Target-CoT fail through fundamentally different mechanisms. En-CoT emros primarily reflect the inherent difficulty of reasoning, manifesting logical failures once models engage with problems. Target-CoT, however, introduces additional barriers unstable generation (e.g., endless repetition) in low-resource languages and difficulties applying concepts-thal that often prevent models from reaching the reasoning stage entirely.
CONCLUSION
This paper systematically investigates long CoT reasoning across nine non-English languages. We show that larger models do not necessarily lead to stronger multilingual reasoning, 7B model reasoning in
We chose this model it had the well-rounded performance all tested languages.

---

Published as a conference paper at ICLR 2026
English outperforning 32B model reasoning in another language. We find that broad multilingual pretraining substantially improves both comprehension and reasoning; then, post-training on just 1,000 non-English language traces matches the performance of 20x more English traces for mid- and low-resource languages. Error analysis reveals fundamentally different failure modes: En-CoT primarily exhibits reasoning failures while Target-CoT suffers from language-specific generation errors and conceptual misunderstandinng, particularly in low-resource languages. These findings show that developing models capable of seamless multilingual reasoning requires targeted interventions each stage cross-lingg mooss-liggual transfer from English alone is insufficient.
We would like to thank Syrielle Montariol and the NeuLab at CMU for valuable discussions.
REFERENCES
Art of Problem Solving, AIME Problems and Solutions, http::://Attoffrroooemeollllos.p
wiki/index php/AIME Pwoblessrrbees..eee and_ Solutions 2025.
Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran EI- Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp, Ehud Karpas, Ran Zilberstein, Jiaqi Zeng, Soumye Singhal, Alexander Bukharin, Yian Zhang. TTugrul Konuk, Gerald Shen, Ameya Sunil Mahabaleshwarkar, Bital Kartal, Yoshi Suhara, Olivier Delalleau, Zijia Chen, Zhilin Wang. David Mosallanezhad, Adi Renduchintala, Haifeng Qian, Dima Rekesh, Fei Jia, Somshubra Majumdar, Vahid Noroozi, Wasi Uddin Ahmad, Sean Narenthiran, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Siddhartha Jain, Igor Gitman, Ivan Moshkov, Wei Du. Shubham Toshniwal. George Armstrong, Branislav Kisacanin, Matvei Novikov, Daria Gitman, Evelina Bakhturina, Jane Polak Scowcroft, John Kamalu, Dan Su, Kezhi Kong, Markus Kliegl. Rabeeh Karimi, Ying Lin, Sanjeev Satheesh, Jupinder Parmar, Pritam Gundecha, Brandon Norick, Joseph Jennings, Shrimai Prabhumoye, Syeda Nahida Akter, Mostofa Patwary, Abhinav Khattar, Deepak Narayanan, Roger Waleffe, Jimmy Zhang. Bor- BoinYiing Su. Guyue Huang, Terry Kong, Parth Chadha, Sahil Jain, Christine Harvey, Elad Segal, Jining Huang, Sergey Kashirsky, Robert McQueen, Izzy Putterman, George Lam, Arun Venkatesan, Sherry Wu, Vinh Nguyen, Manoj Kilaru, Andrew Wang. Anna Warno, Abhilash Somasamudramath, Sandip Bhaskar, Maka Dong, Nave Assaf, Shahar Mor, Omer Ullman Argov, Scot Junkin, Oleksandr Romanenko, Pedro Larroy, Monika Katariya, Marco Rovinelli, Viji Balas, Nicholas Edelman, Anahita Bhiwandiwalla, Muthu Subramaniam, Smita Ithape, Karthik Ramamoorthy, Yuting Wu, Suguna Varshini Velury, Omri Almog, Joyjit Daw, Denys Fridman, Erick Galinkin, Michael Evans, Katherine Luna, Leon Derczynski, Nikki Pope. Eileen Long, Seth Schneider, Guillermo Siman, Tomasz Grzegorzek, Pablo Ribalta, Monika Katariya, Jocy Conway, Trisha Saar, Ann Guan, Krzysztof Pawelec. Shyamala Prayaga, Oleksü Kuchaiev, Boris Ginsburg, Oluwatobi Olabiyi, Kari Briski, Jonathan Cohen, Bryan Catanzaro, Jonah Alben, Yonatan Geifman, Eric Chung, and Chris Alexiuk. Llama-Nemotron: Efficient Reasoning Models, 2025.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sashank Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Bames, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayam Pillai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang. Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstert Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. PaLM: scaling language modeling with pathways. JMLR, 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jemry Tworek, Jacob Hilton, Reñcchrr Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems, 2021.
Alexis Conneau, Kartikay Khandelwal. Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised Cross-lingual Representation Learinng at Scale. In ACI, 2020.
10

---

Published as a conference paper at ICLR 2026
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, YU Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li. Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang. Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang. Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang. Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. I.. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun. Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang. Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L.. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang. Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song. Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu. Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang. Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan. Yiyang Ma. Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yubeng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang. Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang. Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang. Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-Ri R1:: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, 2025.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars, 2025.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, lain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju. Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Chooo CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov,, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimezak-Plucifska. Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Abadie, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Femaaddez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho. Marvin Ritter, Matt Hoffmann, Matthew Watson, Mayank Chaturvedi. Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq lqbal, Shashir Reddy, Shruti Sheth, Siim Pöder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger. Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberr. Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustayo Martins,
11

---

Published as a conference paper at ICLR 2026
Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias. D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Flena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sehastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma 3 technical report, 2025.
GLM 4.5 Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng. Jiajie Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang. Xuancheng Huang. Yao Wei, Yean Cheng, Yifan An, Yilin Niu, Yuanhao Wen, Yushi Bai, Zhengxiao Du, Zihan Wang. Zilin Zhu, Bohan Zhang, Bosi Wen, Bowen Wu, Bowen Xu, Can Huang, Casey Zhao, Changpeng Cai, Chao Yu, Chen Li, Chendi Ge, Chenghua Huang, Chenhui Zhang. Chenxi Xu, Chenzheng Zhu, Chuang Li, Congfeng Yin, Daoyan Lin, Dayong Yang. Dazhi Jiang, Ding Ai, Erle Zhu, Fei Wang. Gengzheng Pan, Guo Wang. Hailong Sun, Haitao Li, Haiyang Li, Haiyi Hu, Hanyu Zhang. Hao Peng, Hao Tai, Haoke Zhang, Haoran Wang. Haoyu Yang, He Liu, He Zhao, Hongwei Liu, Hongxi Yan, Huan Liu, Huilong Chen, Ji Li, Jiajing Zhao, Jiamin Ren, Jian Jiao, Jiani Zhao, Jianyang Yan, Jiaqi Wang. Jiayi Gui, Jiayue Zhao, Jie Liu, Jijie Li, Jing Li, Jing Lu, Jingsen Wang, Jingwei Yuan, Jingxuan Li, Jingzhao Du, Jinhua Du, Jinxin Liu, Junkai Zhi, Junli Gao, Ke Wang, Lekang Yang. Liang Xu, Lin Fan, I Lindong Wu, Lintao Ding, Lu Wang. Man Zhang, Minghao Li, Minghuan Xu, Mingming Zhao, Mingshu Zhai, Pengfan Du Qian Dong, Shangde Lei, Shangqing Tu, Shangtong Yang. Shaoyou Lu, Shijie Li, Shuang Li, Shuang-Li, Shuxun Yang. Sibo Yi, Tianshu Yu, Wei Tian, Weihan Wang, Wenbo Yu, Weng Lam Tam, Wenjie Liang, Wentao Liu, Xiao Wang, Xiaohan Jia, Xiaotao Gu, Xiaoying Ling, Xin Wang, Xing Fan, Xingru Pan, Xinyuan Zhang, Xinze Zhang, Xiuqing Fu, Xunkai Zhang. Yabo Xu, Yandong Wu, Yida Lu, Yidong Wang, Yilin Zhou, Yiming Pan. Ying Zhang, Yingli Wang, Yingru Li, Yinpei Su, Yipeng Geng, Yitong Zhu, Yongkun Yang. Yuhang Li, Yuhao Wu, Yujiang Li, Yunan Liu, Yunqing Wang, Yuntao Li, Yuxuan Zhang, Zezhen Liu, Zhen Yang. Zhengda Zhou. Zhongpei Qiao, Zhuoer Feng, Zhuorui Liu, Zichen Zhang, Zihan Wang, Zijun Yao, Zikang Wang. Ziqiang Liu, Ziwei Chai, Zixuan Li, Zuodong Zhao, Wenguang Chen, Jidong Zhai, Bin Xu, Minlie Huang. Hongning Wang, Juanzi Li, Yuxiao Dong, and Jie Tang. GLM-4.55: Agentic, Reasoning, and Coding (ARC) Foundation Models, 2025.
Google DeepMind. Gemini 2.0 Flash Model Card, 2025. URL httph: http:://etoraasegooleeeaa Com/model-Cal wwwp//ntuuuuucccoo/ooomoo/ gem f2l psh.pp..
fl
Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smymis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar. Kartik Sharma, Charlic Che Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li. Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, and Ludwig Schmidt. OpenThoughts: Data Recipes for Reasoning Models, 2025.
Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishray Chaudhary, Han Zhao, and Xia Song. Scaling Laws for Multilingual Language Models. In ACL, 2025.
Dan Hendrycks, Collin Burn,, Saurav Kadavath, Akul Arora, Steven Basart, Fric Tang, Dawn Song, and Jacob Steinhardt. Mcasuring Mathematical Problem Solving With the MATH Datasct. In NeurlPS, 2021.
Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022.
Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual Thouuht Prompting. In EMNLP, 2023.
Zixian Huang, Wenhao Zhu, Gong Cheng, Lei Li, and Fei Yuan. MindMerger: Efficiently Boosting LLM Reasoning in non-English Languages. In NeurlPS, 2024.
Hyumwoo Ko, Guijin Son, and Dasol Choi. Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap, 2025.
12

---

Published as a conference paper at ICLR 2026
Tomek Korbak, Mikita Balesni, Elizabeth Bares, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, Daniel Kokotajlo, Victoria Krakovna, Shane Legg. David Lindner, David Luan, Aleksander Madry, Julian Michael, Neel Nanda, Dave Orr, Jakub Pachocki, Ethan Perez, Mary Phuong, Fabien Roger, Joshua Saxe, Buck Shlegeris, Martín Soto, Eric Steinberger, Jasmine Wang, Wojeiech Zaremba, Bowen Baker, Rohin Shah, and Vlad Mikulik. Chain of thought monitorability: A new and fragile opportunity for ai safety, 2025,
Sncha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapma, and Orhan Firat. MADLAD-400: A Multilingual And Document- Large Audited Dataset. In NeurlPS, 2023.
Bo Li, Ruoming Pang, Tara N. Sainath, Anmol Gulati, Yu Zhang, James Qin, Parisa Haghani, W. Ronny Huang, Min Ma, and Junwen Bai. Scaling End-to-End Models for Large-Scale Multilingual ASR. In ASRU, 2021.
Meta LLaMA Team.
Llama 3.3 model card.
kttpps://githb. www.coo///oooooe /wt./ 1
11 alama model 1 htt//blbb/nte ob/ma intainn/mmn n/model 1 s/ / ama aamaa_33MMaa 3_33MMDEI, CARD .m 2025.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettle- moyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025.
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. Show Your Work: Scratchpads for Intermediate Computation with Language Models, 2021.
Olmo Team, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, Pradeep Dasigi, Robert Berry, Saumya Malik, Saurabh Shah, Scott Geng, Shane Arora, Shashank Gupta, Taira Anderson, Teng Xiao, Tyler Murray, Tyler Romero, Victoria Graf, Akari Asai, Akshita Bhagia, Alex Wettig, Alisa Liu, Aman Rangapur, Chloe Anastasiades, Costa Huang. Dustin Schwenk, Harsh Trivedi, Ian Magnusson, Jaron Lochner, Jiacheng Liu, Lj Miranda, Maarten Sap, Malia Morgan, Michael Schmitz, Michal Guerquin, Michael Wilson, Regan Huff, Ronan Le Bras, Rui Xin, Rulin Shao, Sam Skjonsberg, Shannon Zejiang Shen, Shuyue Stella Li, Tucker Wilde, Valentina Pyatkin, Will Merrill, Yapei Chang, Yuling Gu, Zhiyuan Zeng, Ashish Sabbarwal, Luke Zettlemoyer, Pang Wei Koh, Ali Farhadi, Noah A. Smith, and Hannanch Hajishirzi. Olmo 3. Technical report, Allen Institute for Artificial Intelligence (Ai2), November 2025. URL www. Sattomms-ssssee com/
6483771763646865-111 _emoo_______eee S_technicallll. caa.aaiccarrrpo--..p.....
OpenAL. Learning to Reason with LLMs, 2024. URL httpp:://peeai... index
www.on-aasnnneeeeeee.eee 11ms/
Cheonbok Park, Jeonghoon Kim, Joosung Lee, Sanghwan Bae, Jacgul Choo, and Kang Min Yoo. Cross-lingual collapse: How language-centric foundation models shape reasoning in large language models, 2025.
Aleksandar Petrov, Emanuele La Malfa, Philip H.S. Tom, and Adel Bibi. Language model tokenizers introduce unfaimess between languages. In NewrIPS, 2023.
Jirui Qi, Shan Chen, Zidi Xiong, Raquel Feránndez, Danielle Bitterman, and Arianna Bisazza. When models reason in your language: Controlling thinking language comes at the cost of accuracy. In EMNLP, 2025.
Libo Qin, Qiguang Chen, Fuxuan Wei, Shijue Huang, and Wanxiang Che. Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages. In EMNLP, 2023.
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 7eRO: Memory Optimizations Toward Training Trillion Parameter Models. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '20). IEEE Press, 2020. ISBN 9781728199988 doi: (0.5555/3433701.3333777
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. DeepSpeed: System Optimizations Enable Training Deep Leaming Models with Over 100 Billion Parameters. In KDD, 2020.
13

---

Published as a conference paper at ICLR 2026
Ricardo Rei, Nuno M. Guerreiro, JosÄ@ Pombal, Daan Stigt, Marcos Treviso, Luisa Coheur, José G. C. de Souza, and André Martins. Scaling up CometKiwi: Unbabel-IST 2023 submission for the quality estimation shared task. In WMT, 2023.
Zhiwen Ruan, Yixia Li, He Zhu, Longyue Wang, Weihua Luo, Kaifu Zhang, Yun Chen, and Guanhua Chen. LlyAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wisr Wise Adaptive Fusion and Alignment Strategy. In NAACL, 2025.
John Schulman and Thinking Machines Lab. Lora without regret. Thinking Machines Lab: Connectionism, 2025. doi: 144434/ml.20250020 https//tiinkngmaacinssss ho/lo////o.///.//rr......
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017.
Shuaijie She, Shujian Huang, Wei Zou, Wenhao Zhu, Xiang Liu, Xiang Geng, and Jiajun Chen. MAPO: Advancing Multilingual Reasoning through Multilingual Aligmment-ass -s-ef---receeeeeeeeee Optimization. In ACI., 2024.
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang. Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Language models are multilingual chain-of-thought reasoners. In ICLR, 2023.
Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute
Optimally Can be More Effective than Scaling Parameters for Reasoning. In ICLR, 2025.
Guijin Son, Jiwoo Hong, Hyunwoo Ko, and James Thome. Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning, 2025.
Peter M. Stahl. Lingua-py: Natural language detection for python. https://github com/
[eemistahl/linguar py 2025.
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffeman, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang. Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Saffyyah Saleem, Holger Schwenk, and Jeff Wang. No Language Left Behind: Scaling Human-Centered Machine Translation, 2022.
Mingyang Wang, Lukas Lange, Heike Adel, Yunpu Ma, Jannik Strötgen, and Hinrich Schütze. Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causes, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoe V Le, and Denny Zhou. Chain-of-Thhoulee Prompting Elicits Reasoning in Large Language Models. In NeurlPS, 2022.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference Scaling Laws: An Empirical Analysis of Compute-Optiman Inference for LIM Problem-Solving In ICLR, 2024.
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese- Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, and Irene Li. MMLU-ProX: Multilingual Benchmark for Advanced Large Language Model Evaluation, 2025.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mTS: A massively multilingual pre-trained text-to-text transformer. In NAACL, 2021.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin. Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Seff-Improvement, 2024.
14

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang. Le Yu, Lianghao Deng. Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang. Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 Technical Report, 2025a.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 Technical Report, 2025b.
Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying Long Chain-of-Thought Reasoning LLMs, 2025.
Zheng-Xin Yong, M. Farid Adilazuarda, Jonibek Mansurov, Ruochen Zhang, Niklas Muennighoff, Carsten Eickhoff, Genta Indra Winata, Julia Kreutzer, Stephen H. Bach, and Alham Fikri Aji. Crosslingual Reasoning through Test-Time Scaling, 2025.
Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim, Sheikh Shafayar, and Minjoon Seo. LangBridge: Multilingual Reasoning Without Multilingual Supervision. ACI. 2024.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yonggiang Ma. LlamaFactory: Unified Efficient Fine- Ene-Tuning of 100+ Language Models. In ACL, 2024.
15

---

Published as a conference paper at ICLR 2026
A
EXPERIMENTAL DETAILS
A.1
LANGUAGES
Table lists the script, family, region, and approximate resource level of all the languages studied.
Table 3: Tested languages with their script, family, region, and approximate resource level.
Language
Script
Language Family
Region
Resource Level
English
Latin
Indo-European (Germanic)
Global / Europe
High
Chinese
Han
Sino-Tibetan (Sinitic)
East Asia
High
French
Latin
Mudo-European (Romance)
Europe Africa
High
Japanese
Kanji + Kana
Japonic
East Asia
High
Afrikaans
Latin
Indo-Europent (Germanic)
Southern Africa
Mid
Thai
Thai
Kra-Dai
Southeast Asia
Mid
Latvian
Latin
Indo-European (Baltic)
Northern Europe
Mid
Marathi
Devanagari
Indo-European (Indo-Aryan)
South Asia
Low
Telugu
Telugu
Dravidian
South Asia
Low
Swahili
Latin
(Bantu)
East Africa
Low
A.2 MODELS
We evaluate models across multiple scales and architectures to study cross-lingual long CoT reasoning. For scaling experiments (Section[4 4) we use the DeepSeek-R1-Distte family (1.5B, 7B, 14B, and 32B parameters), which share identical post-training on 800k English reasoning traces and derive from the Qwen 2.5 base family, allowing us isolate parameter scaling effects. For pretraining experiments (Section 5). we compare four models: Qwen2.5-7B (18T tokens, 29 languages), Qwen2.5-Math- (Qwen2.5 7B with additional IT+ math-focused tokens), Qwen3-8B -B--BBBBBe (36T tokens, 1 119 languages), and Gemma3-12B-B (140 languages) enabling us to disentangle specialized reasoning pretraining from broad multilingual coverage. For post-training experiments (Section (6] Appendices [B.4 and [B55), fine-tune Qwen3-8B-Bas and Dlmo3--B-Instmunn computational efficiency with reasoning capabilities for most experiments. To validate findings larger scales (Appendix [B3). we evaluate DeepSeek-FR BeepSeck-Rl-Ding RI-Distill-LLams 20B and L. B--Nemotron-Super--eeeee both based on Llama-3.3 3.3-77BB-Instenn
We choose the 7B/8B scale because it balances
A.3 TRAINING SETUP
Tables and
provide the hyperparameter configurations for supervised fine-tuning (SFT) on OpenThoughts 20k and Translated/Distilleedss.k respectively. All models undergo full fine-tuning except Gemma3-1222 12B- For this model, we apply LoRA fine-tuning (Hu al. (2022) with parameters detailed in Table 6 based on guidelines from Schulman & Thinking Machines Lab (2025). To enhance training efficiency, we employ the ZeRO optimizer (Rajbhandari et al. (020) as implemented in the SeepSpeed library (Rasley et al. 2020). For SFT, we use the a.L.A-Facctory (Zheng et al. 2024) framework and four NVIDIA L40S GPUs.
A.4 LANGUAGE FORCING.
Controlling Lyeason during inference is difficult for reasoning models that have been heavily post-trained on English traces, particularly for low-resource languages. To address this, we employ language forcing (Yong) et al. 2025), prompting technique that encourages LLMs to reason in specific target language. Language forcing works by injecting translated phrases and system prompts at the beginning of the generation process to guide the model's reasoning language. For example, to elicit French reasoning, we would seed the model's generation with D accord, je vais essayer de trouver une solution (English: Okay, / will try to find solution).
16

---

Published as a conference paper at ICLR 2026
Table 4: SFT hyperparameters for OpenThoughss3-20k
Global Batch Size
Context Length
Learning Rate
Epochs
64
32,768
le-5
Table 5: SFT hyperparameters for Translated-s1k and Distilled-sIk.
Global Batch Size
Context Length
Learning Rate
Epochs
32
32,768
le-5
Table 6: LoRA hyperparameters for Gemma3-12B-PT.
Target Modules
Rank
Alpha
Dropout
Learning Rate
All
64
128
0.05
le-4
A.5 EVALUATION SETUP
We evaluate on MATH-500, AIME-Combined, and MMLU-ProX using identical decoding settings (temperature 0.6, top-p 0.95, max output 32,768 tokens) and language forcing (defined in Appendix A.44 to control the reasoning language. We count a generation as correct only if the extracted final answer (e.g., the contents of \boxed :) matches the gold label and the response is Appendix A.7 ( (at least 90% target-language chunks). We mark non-compliant outputs as incorrect. For MATH-500 and MMI.U-roXX, we decode one sample per item and report pass @ For AMM--Cobined, we decode five independent samples per problem (60x5 = 300) and report the mean correctness over all 300 generations (not best-of-k) to get more robust performance estimates.
per
A.6 PROMPTS
The prompts used for translation and emrrr analysis are presented in Figures and respectively.
A.7
LANGUAGE COMPLIANCE ALGORITHM
Our language compliance algorithm validates whether a text response is written in specified target language through multi-stage processing pipeline. First, we remove LaTeX mathematical expressions, code blocks, symbols, and formatting while preserving natural language content across multiple scripts (Latin, CJK, Thai, Devanagari, Telugu). The cleaned text is then divided into fixed-size character chunks (128 characters), with duplicates removed. Each chunk undergoes language classification using either a script-ratio heuristic for non-Latin languages (checking if at least 75% letters belong to the target script) the Lingua statistical language detector (Stahl 2025). We discard chunks where the language classification confidence below 90%. The algorithm computes a compliance score as the ratio of target-language chunks total valid chunks, requiring at least 90% of chunks to be in the target language for the response to be considered compliant.
A.8 ERROR TAXONOMY
We develop taxonomy comprising five distinct error categories that capture the primary failure modes observed in reasoning tasks. Table presensss these categories along with their definitions and representative examples.
A.9 ERROR LABELING
To identify patterns across languages, randomly sample 100 incorrect, saaguagg-complinng responses for each language and setting (En-CoT, Target-CoT) from DeepSeek-RR R1 -isttill--Lamaa 70B's 'ss outputs AlMEE-ommbied. We focus specifically language-compliant responses to isolate reasoning
17

---

Published as a conference paper at ICLR 2026
Table 7: Taxonomy error categories reasoning. Each category represents distinct failure mode with specific characteristics and manifestations. Category
Definition
Example
Problem Comprehension
Misinterprets the objective, constraints, variables, or key terms.
Asked for perimeter, computes area.
Conceptual
Lacks/misuses required fact, formula, theorem, or principle.
Uses circle formula find circum- ference.
Reasoning
Flawed logical step invalid inference; incorrect plan built from correct concepts.
From "If then B," concludes "If B then A."
Calculation
Ariihnetic/ymholic mistake despite correct sctup and plan.
Sets ==3x7 but computes 22.
Output Generation
Output violates constraints (format) Missing degenerates (stability).
boxed output: repetitively generates "The next step is."
eirors from language compliance issues (see Appendix for compliance rates). For error classification, prompt Gemini-2.5-Pro to categorize each response according a single error label based on the most critical mistake that leads to the incorrect answer (prompt shown in Figure [). To validate this approach, native speakers manually review 20 samples each for Chinese, Telugu, French, and Japanese, confirming high agreement with the automated classifications.
five-category taxonomy, assigning
B ADDITIONAL RESULTS
B.1 LANGUAGE COMPLIANCE RATES ACROSS MODELS
Because our evaluation criteria require models to generate responses that are both correct and language- compliant (Appendix [A.7). results can conflate genuine target-language reasoning capabilities with model's ability to follow instructions to reason in the target language. Table [ presents language compliance rates for all models studied. We observe perfect language compliance when models reason in English (En-Only and En-CoT settings), are consistently high across models and languages, with a few exceptions. Qwen2.5-Math-1 has undergone specialized continual pretraining on mathematical reasoning data, shows poor compliance in French, Afrikaans, and Swahili. Manual review reveals that model outputs typically begin with few coherent sentences in the target language before switching to English. Notably, even after switching to English reasoning. the model's 's accuracy remains extremely low-15.6% for French, 13.6% for Afrikaans, and 5% for Swahili far below its En-CoT performance. We also observe low compliance rates for DeepSeek-Re -1e R-------DD--------- --istiill-Qwnn------ -weill--een-1.BBBB and DeepSeek- RI Distiill-Llama--000 when reasoning in Japanese. In both cases, manual review reveals frequent oode-switching to Chinese. As with Owen2.55Math-77 these models predominantly produce incorrect answers after switching languages, suggesting that the language switch does not improve their reasoning performance.
omit those results. Overall, language compliance rates
which
B.2 THE EFFECT REASONING CIAANN LENGTH PERFORMANCE GAPS
In Section 4 observe that performance gaps in long CoT are influenced more by the language used for reasoning than by the language of the input. This result contrasts with prior findings on short COT, which report input comprehension the primary obstacle for models (Ko et al. 2025). To explore this difference further, we use MMLU-ProX as a proxy for short CoT, as its questions elicit 2-4x fewer reasoning tokens than AIME-Combnned (Table 9). Figure shows that larget-language reasoning performs competitively with English reasoning for high- and mid-resource languages on MMLU-ProX. Notably, target-language reasoning even outperforms English reasoning in Chinese and French at 32B parameters. This finding contrasts with the clear performance gap observe on AIME-Combined (Figure 2 The most natural interpretation is chain length: when the chain is short, scaling mainly needs to recover imput comprehension and the gap narows; when the chain is long, sustained reasoning in the target language remains brittle and the gap widens. This pattern supports the result of Ko et al. (2025), where performance in short-Col) settings limited more by comprehension than by the language used for reasoning.
18

---

Published as a conference paper at ICLR 2026
Table 8: Language compliance rates (%) across all models. Percentage of responses that comply with language forcing instructions in the Tal setting on AIME- ALME-Combined, Most models achieve high compliance rates across languages, with few exceptions in certain model- model-language combinations.
Models
EN
ZH
FR
JA
AF
TH
LV
MR
TE
SW
Off-the-shelf Models
DeepSeek-R --Dissill-----n--
100
93.3
83.7
36.7
91.3
66.0
92.7
98.3
94.3
82.0
DeepSeek-Rl-Disiil--weeess
100
100
93.0
63.3
89.7
53.7
90.7
96.0
89.0
87.7
DeepSock-Ri Daittii--Qwennn Qwen-14B
100
100
94.3
98.3
65.7
89.7
94.3
96.3
92.0
99.3
DeepSeck- --ittes-----iiiill------ Distill-Qwen-32B
100
100
95.0
97.3
98.7
97.7
98.0
99.3
99.7
98.3
DeepSeck-RI 11 D-sttilll- Prsstill--lama-777
J.ama-3.--Nemmn Llama-70B
100
94.0
87.7
25.3
84.0
81.3
94.0
98.0
100
99.3
100
92.0
86.0
97.3
73.7
98.7
95.7
94.0
98.3
94.7
Qwen3-8B (Thinking)
100
85.3
78.7
97.0
86.3
100
98.7
99.7
100
98.0
Fine-tumed on OpenThoouhhs33200
Qwen2.5-7B
100
100
44.7
93.7
58.0
91.3
99.0
90.3
95.0
98.3
Matthh55Math--B
100
71.7
34.0
67.7
29.3
94.0
90.7
75.3
94.3
34.0
Qwen3 8B 1aa3----Baaee
100
91.0
97.7
100
93.3
100
94.3
83.7
69.0
88.7
G 12B-PP
100
84.7
97.7
95.0
95.3
90.7
97.0
85.7
88.0
97.7
Fine-tuned on slk Variants
English-slk Translated-sIk Distilled-sIk
100
70.0
70.7
91.3
72.0
78.7
72.0
78.3
90.3
76.3
100
92.7
100
100
99.7
100
100
100
100
100
100
94.7
100
100
100
99.7
100
99.7
100
100
Table 9: Average response length (in thousands of tokens) by language and benchmark for DeepSeek-
RI-Distill- RI-Distil-QQwen-322. Responses on AIME-Combined are consistently 2-4x longer than on MMLU-ProX.
Benchmark
EN
ZH
FR
JA
AF
TH
LV
MR
TE
SW
AVG
En-CoT
ATMF-Combined
12.1
11.0
11.0
11.2
11.6
11.1
11.1
11.1
10.7
11.7
11.3
MMI.U-roxX
3.1
2.8
3.1
2.8
2.9
2.4
2.9
3.0
3.4
2.9
Target-CoT
AlME-CCobined 12.1
15.7
7.7
19.0
16.0
18.9
20.7
28.3
31.1
23.0
19.3
MML.U-ProX
3.1
4.1
1.7
5.2
3.6
5.6
17.3
26.7
8.5
8.4
B.3 VALIDATION OF SCALING TRENDS ON LARGER MODELS AND DIFFERENT ARCHITECTURES
In Section 4 we examine the DeepSeek-PP -1--sstt family of models ranging from 1.5B 32B parameters. This family provides an ideal testbed for isolating scaling trends, as all models share the same post-training procedure and common pretrained backbone. To validate these findings at larger scales and across different architectures, we extend our evaluation to two additional models: DeepSeek-R1-Diss -ist------itlll--aaaaaa -lamml----a--- and 3.3-70B-Instruct-
(Bercovich et al.. 2025). Both
model "optimized for multilingual dialogue use cases" (Meta LLaMA Team 2025)-as as their reference model. As shown in Table IO we observe the same substantial performance gap between larget-language reasoning and English reasoning AIME-Combined that we identify in Section 4 On average, the gap reaches 28.4% for DeepSeek RI RI-Distill-LLaes 70B and 40.8% for Llama-3.3-Nemm 3.3-Nemotron Super-49B. These results confirm that the performance disparity persists across both larger model scales and different base architectures.
models use Llama-3.. 1.3ma-3.3-70B---
B.4 COMPARING ENGLISH AND TARGET-LANGUAGE DATA UNDER FIXED COMPUTE
In Section G we compared abundant English reasoning traces (e.g., Open 3 with much smaller larget-language datasets to examine the tradeoff' between cross-lingual transfer and targeted supervision. However, this comparison conflates two factors: differences in data distributions (OpenThoughts3 vs. slk) and, more critically, differences in fine-tuning compute. To isolate the effect of target-lan langg-la training, we conducted controlled experiment where all models were fine-tuned synthetic datasets from the same English reasoning traces to ensure consistent data distribution. Table compares English-slk (baseline) against Translated-s k and Distilled-s1& variants. This comparison reveals
exactly 1,000 traces. We generated
19

---

Published as a conference paper at ICLR 2026
Chinese
Japanese
0.8
0.5
0.0
0.7
0.7
0.6
0.5
0.6
0.5
0.5
0.5
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.0
En.CoT
Cor
En-CoT
Tarqe:Col
En-CoT
Target
Afrikaans
Thai
Marathi
Accuracy
ETHOOT
Tegee-CsT
EX-CUT
Terge-Cot
Segel-GrT
Telugu
Swahili
0.8
0.8
0.7
0.7
0.6
0.8
0.5
0.5
0.4
0.4
0.3
0.3
0.7
0.2
0.1
0.1
0.0
0.0
En-CoT
SeepSeek-R1- Dist----isiill--e--....
DeepSeek-R1-Dn Dii.RR---Dsiill-e----l
En-Only (1.58)
En-Only (148)
DeepSork R1 Distill Distill-wenn778
DeepSock R1 Distill Saatil--wwn--aee 328
En-Only (78)
En-Only (328)
Figure 5: Evaluation of scaling trends in DeepSeck-R1 R1--istll models on MMLU-ProX DeepSeek- RI-Distill models demonstrate narrower gaps between Target-CoT and En-CoT on short CoT tasks, with langet-language reasoning in Chinese and French even outperforming English multiple scales.
Table Large reasoning models exhibit persistent performance gaps between En-CoT and Tor on AIME-Combing AVG excludes English, and EPR (English Performance Recovered)
computed as AVG EN (%).
Model
EN
ZH
FR
JA
AF
TH
IN
MR
TE
SW
AVG
EPR
Lama 3.3 J.2-NemotroeSiee Super 49B
ba-CoT
69.7
70.7
69.7
68.7
67.7
66.0
61.0
63.3
56.7
65.7
65.5
94.0
Target CoT
DeepSeck- Ri-DistillI D-Distil---ana--00
CoT
ToT
$6.3
39.7
41.7
6.0
30.3
21.3
21.3
14.7
17.3
25.3
24.2
43.0
that target-language training yields dramatic improvements across all benchmarks. Both Translated-s1k and Distilled-s1Ik substantially outperform English-slk on non- non-Englishh languages, with Translated-s achiev- ing average gains of 44.9% on MATH-500, 12.8% on AIME-Combined and 21.2% on MMLU-ProX. These results demonstrate that the apparent advantage of English-only training for high-resource languages in Section was driven primarily by its 20x data advantage rather than effective cross-ling cross-lingual transfer. When compute is held constant, target- langual synthetic data is optimal across all resource levels.
B.5 VALIDATION POST-TRAINING RESULTS ON OLMO MODELS
synthetic data experiments Qwen3-8B BB-Base as the base model for post-training. To test whether these findings generalize beyond Owen, we replicate the compute-equivallee setup with the open-source Olmo3-7BB-Instmu Olmo Team et al. (2025) the base model. Table T2 shows
In Section 6 and Appendix B.4
20

---

Published as a conference paper at ICLR 2026
Table 11: Evaluation after fine-tuning Qwen3-8B- across compute-equivalent synthetic datasets (Target-CoT setting). Subscripts are computed relative to English-sIk (compute-equiig conpute-cquivalent baseline). The AVG excludes English and ignores missing entries (-). Both Translated-slI and Distilled-sIk substantially
outperform English-slk across non-English languages, with Translated-s achieving the largest gains.
SFT Dataset
EN
7H
FR
JA
AF
TH
IN
MR
TE
SW
AVG
MATH-500
English-slk 92.6
42.6
324
66.0
34.8
33.8
33.2
24.2
21.0
13.0
33,4
Trandated-sik
92 44.2
55.0
40.0
87.8.60.0
81.6.-77
83.2 43.2000.0
70.8.440.6
72.4.6..
64.4-5144
78.3
DistiledssIk
92.6.0.0
60.0
788844664
6..4-1.6
85.4450.8
83.8
82.0.48.8
76.2.22.
67.24466.
67.8
72.8
AIME-Combined
English-stk
12.7
10.3
10.3
8.3
2.7
1.3
Tramlated-slk 33.0,0.0
29.3
15.7
46.
27.7417.4
19.3 24.7-16.0
14.4412.
1..0
15.3411
11.3--100
20,8
Distilled-slk
33.0.0.0 10.7 22.0
21.0,7.7
13.022.
28.0
19.7-1114 22.0.
163,
13.7 49.7
12.7-1114
17.5
MMX
English-slk
69.0
28.9
31.0
45.6
23.6
29.3
14.6
12.8
8.2
24.3
Translatd-silk 69.0....
28.7420.8
29.8
29.5.22..
41.2
4.4
45.1431.6
49.1-19.8
40.0.22.4
44.022.2
26.5-19.3
45.5
Distilld-sllk 69.0 10.0
32.0
3.1
48.0
11.0
38.8
53.2
12.0
48.5 10.2
41.7 (27.1 35.9 (23.1 20.4 12.2
39.8
Table 12: Evaluation after fine-tuning Olmo3-7B-Instruet across compute-equiivaent synthetic datasets (Target-CoTT setting). Subscripts computed relative to English-slk (compute-equivalen baseline). The AVG excludes English and ignores missing entries (-). Results show trends consistent with
those observed using Qwen3-8B-Bas as the base model.
SFT Dataset
ZH
FR
AF
TH
LV
MR
TE
AVG
MATH-500
English-slk
91.8
64.6
76.8
44.2
40.8
17.0
15.4
25.2
16.8
14.4
35.0
Thanslaed-sIkk
9.. .1.0
79.843.0
13.4
69.64228.
41.6.21 6
48.8--344
30.645.4
16.0 o..
43.8
43.8
51.5
Distilled-stk
91.8
50.4
14.2
72.8
,8.0,
69
46.2. 29.2
45.8.
30.4
37.
2-.4
39.2
49.0
ALME-Comblned
Fnglish-slk
33.7
11.7
16.7
3.3
3.0
0.0
1.0
0.0
0.0
1.3
4.1
Translated-sIk
33.7-a.0 16.7...0
20.0.s.s
4.3
13.0
63....
6.066.
2.422.7
1.3.
9.0-7.7
9.3
Distiledaslk
33.7
5.0 6.7
18.0
87.6.4
16.7
18.7
8.3 18.3
9.0 8.00
27 27.27
4.0
7.7
8.9
MMLU-ProX
English-slk
53.6
22.8
34.4
9.7
17.5
11.1
5.6
14.0
Translate-ssIk 53.6
4..
33.0-1.4
11641.9
21.443.9
6.143.0
68
2.7-
128.
15.2
Disilledslk 53.6 1..0-6.
6.0
13,6
25.9.a.4
7.5
7.8 3.3
2.7-2.0
8.3-0.5
13.8
similar trends: larget-language training substantially outperforms English-only training under equivalent compute budgets.
Translated consistently outperforms English-sIk across most benchmarks and languages, with average gains of 16.5% MATH--00,, 5.2% AlME-Combined and 1.2% MMLU-ProX. Although these gains are smaller than those observed with Qwen, the effect is directionally consistent: target-language synthetic data improves multilingual performance more than cross- linsull transfer from English at fixed compute. Distilled-sIk exhibits more variable behavior, particularly underperforming on high-resource languages such as Chinese and French MATH-500 and MMLU-Pro suggesting that the benefits of distillation depend on the base model's 'ss multilingual strengths.
B.6 EVALUATION OF TRANSLATION CAPABILITIES
We first evaluate En + translation quality on FLORES-200 devtest using spBLEU and chrF++ (Table We compare Gemini-2.00 2.0-Flash (Google DiepMind] (025), general-purpe LLM, with NLLB-200-3.3 (Team et al. 2022) and MADLAID-400- (oaaooTTBBKKKuuggg (Kudugunta al. 2023), which are dedicated MT systems
21

---

Published as a conference paper at ICLR 2026
Table 13: English-to-targe translation performance on FLORES-200 devtest. Comparison of Gemini-2.0-Flash (general-purpos LLM) against NLLB-200-33.0 and MADLAD-400-10B (dedicated MT systems). Scores reported spBLEU chrf++ with best scores in bold. Gemini-2.04 2.0--lash achieves the highest performance across all languages, supporting its selection for generating Translated-s
En -X
Gemini-2.0-Fhan NLLB-200-3.3B
MADLAD-400-10B
ZH
39.5/ 31.7
22.3 19.6
34.8 / 27.2
FR
59.2 72.1
53.5 68.0
55.7 69,9
JA
35,8 37.2
16,0//3.6
24./
AF
45,9 64.9
/
44.0/
TH
45,4 49.7
28.8/37.9
30.7 39.8
LV
43.5 61.1
27.1/48.9
38.3/57.2
MR
28.1 / 45.1
26.8/45.0
9.5/26.2
TE
40.4 55.2
36.5/51.9
28.3/
SW
43.3 / 64.0
36.2 58.44
30.1 /
Table 14: Reference-free quality estimation for paragraph-Ievel translation of English reasoning traces. We compare Semmini-2.0-Flashh and NLLB-200-3.3B by translating English reasoning traces paragraph by paragraph and scoring each English-targen pair with WMT23-COMETK Kiwi-D--L. Scores range from (random translation) sampled traces for each language, with the best score in bold. MADLAD-400- 10B is omitted because could not reliably translate long paragraphs.
(perfect translation). We report the mean score over 100 randomly
En +X
Gemini-2.0-Flaar
NLLBB-00-3.3B
ZH
0.619
0.473
FR
0.602
0,442
JA
0.632
0.404
AF
0.617
0,459
TH
0.617
0.490
LV
0.609
0,418
MR
0.526
0,384
TE
0.532
0,434
SW
0.576
0.390
covering roughly 200 and 400 languages, respectively. Gemini-2.0-Flash achieves the best scores on all evaluated languages, motivating use generate Translated-sIk.
However, FLORES-200 primarily contains general-domain prose and may not fully reflect the kinds of text we translate Translated-sIk, such as mathematical notation and symbols. We therefore directly assess the quality of translations in Translated-slk Because we do not have human reference translations of the target-language reasoning traces, we use reference-free quality estimation (QE) model, SM112--COMETKKiii---X.
(Rei et al. 2023). We translate English reasoning traces paragraph by paragraph and score each English- target pair, reporting mean scores over 100 randomly sampled traces per language in Table 14] Across all languages, Semini-2.00-Flase again outperforms the dedicated MT models, providing further support for our choice of translator.
B.7 INFERENCE RFFICIENCY IN TERMS OF BYTES
We re-evaluate efficiency using UTF-8 bytes emitted rather than tokens (Figure 6). Across languages, the correlation between cost and accuracy becomes weakly negative indicating that cost is only loosely tied performance when using less biased measure of sequence length. These results suggest that script/to- kenizer effects are primarily responsible for disparities in inference efficiency opposed to overthinking.
22

---

Published as a conference paper at ICLR 2026
OpenThoughhs3-20k
Translated-s3k
En
-0.155
00.355
40
En
Accuracy (%)
30
30
Ah
Af
Th
20
20
TH
Mr
SW
10
10
Sw
40
50
60
70
80
90
100
40
50
60
70
80
90
100
Average Response Bytes (thousands)
Figure 6: Byte-based inference efficiency AIME-Combined for fine-tuned Qwen3-8B-Ban models. Cost-accuracy correlations are weakly negative when measured UTF-8 bytes rather than tokens.
Your job is to return a translated version of the English text. Translate to (target_lang).
The translation must be fluent, sasy to read by native speakers. not solve the prompt translate it.
You must preserve all details including math notations (latex) code.
The math notations and code must be translated, keep it as is. Return your translation in the following format.
<translation>
..trrnssatedd text...
</translati on>
The following is the source text for you task:
<English>
lsource_tex::
</English>
Figure 7: Prompt for translation.
23

---

Published as a conference paper at ICLR 2026
expect identifying criticel ezrors model- sedee-ggeeztted responses.
case, you will receive:
A set
dafining distinct errors.
2. A problem atatement and model-generated responae that La Incorrect.
**Your task:**
Identify which error, as defined by the rubrics, is the moat critical in the model zesponse. should only select single ercor that the model ansver being ancorrect. Your finel answer must ba valid USON abject in the format below, only one category should present: 'Jjoon
influential leading
"Problem Comprehenaicn": ("present:: 110, "explansticn": "..."),
"Conoeptual""
"Reasoning"; ...
"Ca "Calcslatior":
"Output Generation":
Rubrics:
1. s>Problem Comprehenssin BZzorAs
This error when the model =misunderstands what it's being asked to doss. The ressoning can't be corract if it's aimed at the wrong goal.
veDefinitionee: The model Incorrectly interprets the problen's objective, constraints, variables, or key terms. v*Exampless: Asked for the eeperinsterts shape, the mocel caloulates its **rzaat,
ccCoopptuall
This eitor occure when the aodel understands the goal but wlaacks or misuses the necessery knowledgevr devise correct plan.
DDeefinition::: The model fails to apply or incorrectly applies a required fect, formula, theorem, scientific principle. **xxample*s Using the formula for a civole's area to find its rounferrnce.
#44 3, <>esssonngg Berorva This is an error in the mlogiiaa flows of the solution. the model understands the problem and may know the right cancepta, but it connects them incorractly.
model employs oz coastructs an incorrect solution plan from correct concepts. wexemplexxs
i:Definitione::
logical step,
invalid inference,
B, incorrectly concluding *If
#44 4, e*Caloulation Erroz** This mechanical errer execution. plen logie sound, is wrong.
DDefinition::: All pricr arithmetic or symbolic computation. vexxamplesss Correctly setting
but the model. makes
Vii is caloul result
5. exOutpus Generation on Error<
This esror any where the model fails deliver the output correctly.
explicit oz implicit constraints. This includes isaues with formatting (missing "Vboxed(( Y, stability (sepetitive locps), vxexcmplexx: endleasly repaating "The next step is...".
veDefinitionss; model fails produce output according
instructions (wrong language).
Response analyza: (esppose) reaponse, Output your object:
Figure 8: Prompt for enror analysis.
24


Reviews:
[{"review_id": "xefaX64VbY", "reviewer": "Reviewer_ZAqq", "summary": "The paper investigates whether the \"long chain-of-thought\" abilities of large reasoning models actually transfer beyond English. It disentangles two evaluation modes across nine non-English languages. The first is En-CoT, where inputs are in the target language but the reasoning chain is in English, and the other is Target-CoT, where both input and reasoning are in the target language. Scaling improves multilingual performance primarily in En-CoT, while Target-CoT is left behind, especially on math tasks that require long multi-step chains. A controlled comparison reveals that math-specialized pretraining enhances En-CoT yet often harms Target-CoT, whereas broad multilingual pretraining improves both modes. For post-training, modest target-language supervision built by translating gold English traces generally outperforms distilling target-language traces from a strong teacher model. The paper also analyzes inference efficiency and error profiles, finding language-specific failure modes when reasoning in the target language.", "questions": "Based on the weaknesses, I may post the following questions and suggestions:\n\n(1) Adopt existing translation quality estimation metrics, like xCOMET, MetricX, to directly measure the translation quality.\n\n(2) Evaluate with at least one DeepSeek series reasoning model, say, DeepSeek-R1-Distill-Qwen-7B or other preferences.\n\n(3) Adopting RL post-training to see if it benefits Target-CoT, like using GRPO with a format reward of 0.2 if the language of the thinking traces matches the (low-resource) question language.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well-written and framed clearly with three setups and a comprehensive evaluation of nine languages, covering high/middle/low resource languages.\n\nAlso, the scaling study is carefully controlled and highlights that Target-CoT never reaches English-reasoning levels, even at 32B; switching to target-language reasoning at 32B still performs lower than a 7B English baseline.\n\nBesides, the post-training section is practical. It shows that with only ~1k target-language traces, translated from high-quality English rationales, models substantially outperform target-language distillation in aggregate and become comparable with much larger English-only SFT. It especially benefits mid/low-resource languages. \n\nIn addition, the analysis of inference efficiency shows that accuracy is negatively correlated with response length in tokens and that target-language SFT mitigates cross-lingual efficiency gaps. The byte-based view further investigates tokenizer effects. \n\nFinally, the error analysis is quite insightful. Most of the errors in En-CoT are inference flaws, while the errors in Target-CoT are more reflected in output generation and conceptual levels. It also provides a qualitative example of a situation where Target-CoT fails while En-CoT succeeds.", "weaknesses": "Although the authors benchmark translators and justified the usage of  Gemini-2.0-Flash in Appendix B.5, it may still be promising to further measure the quality of the translated datasets with existing translation quality estimation metrics (e.g., xCOMET, MetricX). These scores will directly show that the translated datasets are reliable and trustworthy.\n\nThe evaluated model only covers one language family, that is, the Qwen series. Although Deepseek-Distilled-R1 is trained mainly on English and Chinese data, it still shows capability in multilingual reasoning. So I may suggest testing at least one Deepseek model and seeing if a similar phenomenon also happens there.\n\nFor the improvement part, the work only adopts supervised fine-tuning but does not try reinforcement learning, which has already become a popular strategy nowadays. Specifically, it may teach structured reasoning in non-English languages via verifiable rewards.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pNi7vdUKRC", "reviewer": "Reviewer_pui2", "summary": "This paper investigate the long-chain reasoning across languages, examining four stages of model development: scaling, pretraining, post-training, and inference. To achieve this, this paper compare three reasoning settings across nine non-English target languages: En-Only, En-Cot, and Target-Cot. The key findings are: (1) scaling reasoning model size improves multilingual task performance in En-CoT, but Target-CoT performance lags behind. (2) adding a specialized reasoning stage enhances En-CoT performance but degrades Target-CoT, whereas broad multilingual pretraining improves both modes simultaneously. (3) fine-tuning on reasoning traces automatically translated from gold English traces outperforms fine-tuning on target-language traces distilled from large reasoning models.", "questions": "1. In section 6, is it reasonable to translate English to low-resource language? How to ensure the translation quality?\n2. Why conducting SFT in section 5? In section 5, the point is ``multi-lingual pre-training''.\n3. In section 5, comparing Qwen-3 to Qwen-2.5 is not fair. Except the languages of pre-training data, the size, pre-training task, pre-training strategies are all different. Controlling variables for experiments is the optimal method.\n4. In lines 181-182, I do not observe a ``consistently approach'' trand as model size scales. 7B LLMs with En-Cot are closest to corresponding En-Only baselines.\n5. In section 6, comparing Translated-s1k and Distilled-s1k to OpenThoughts3-20k is unfair and unreasonable.\n6. There are some issues of the format of Table 1.\n7. This paper states many viewpoints in Introduction with only one citation. Although some of them are accepted by most researchers, citing their sources are very important.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. This paper first investigates the long cot in LLMs across languages.\n2. The expriments offer  some insights for furture improvement.", "weaknesses": "1. The quality of the experiments remain to be improved.\n2. Some experiment settings are strange.\n3. The analyses could be more thorough.\n\nAll can refer to the questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5tlAhJWyfi", "reviewer": "Reviewer_SXHs", "summary": "This paper presents a systematic study of long chain-of-thought (CoT) reasoning across nine non-English languages, examining how reasoning ability in large language models generalizes beyond English. The authors analyze four development stages—scaling, pretraining, post-training, and inference—and show that while model scaling and multilingual pretraining enhance comprehension of non-English inputs, reasoning within target languages lags far behind. Specialized reasoning pretraining (e.g., math-focused) improves English reasoning but often harms target-language reasoning. Fine-tuning with translated reasoning traces proves more effective than direct distillation, particularly for mid- and low-resource languages, and even small amounts of target-language data significantly improve performance. Error analysis reveals that English reasoning failures are mostly logical, whereas non-English reasoning is hindered by language-specific generation and conceptual errors.", "questions": "No.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper tackles a novel and meaningful question—how long CoT reasoning transfers across languages—addressing a major gap in multilingual LLM research.\n\n- Comprehensive evaluation across nine languages and multiple resource levels offers strong empirical grounding.\n\n- The finding that translated synthetic data can substitute for large English datasets is practical and impactful for multilingual model training.", "weaknesses": "- The study relies solely on Qwen-family models (Qwen2.5, Qwen2.5-Math, Qwen3), limiting generalizability; results might be model-specific rather than universal.\n\n- Although reports describe these Qwen models’ data composition, the exact pretraining and fine-tuning details remain opaque; using them as representative backbones may reduce methodological rigor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KScrJIUMao", "reviewer": "Reviewer_yQ3S", "summary": "This paper systematically investigates long CoT reasoning in large language models across nine non-English languages. It distinguishes three reasoning settings—En-Only, En-CoT, and Target-CoT—and examines how multilingual reasoning evolves through model scaling, pretraining, post-training, and inference. The study finds that scaling enhances cross-lingual comprehension but fails to improve target-language reasoning; specialized reasoning pretraining can even degrade non-English performance, whereas broad multilingual pretraining benefits both. Post-training on translated reasoning traces outperforms distillation in target languages, especially for mid- and low-resource cases. Error analysis reveals distinct failure modes between English and non-English reasoning, underscoring that effective multilingual reasoning requires targeted interventions beyond simple cross-lingual transfer.", "questions": "1. **Line 112–116:** Could the authors clarify whether the trends observed for long chain-of-thought (Long CoT) reasoning are consistent with those for shorter CoT reasoning across languages? In particular, do the same cross-lingual gaps persist for shorter reasoning chains?\n\n2. **Line 155:** The paper mentions using *Gemini 2.0 Flash* for translation. How do the authors ensure translation quality and linguistic diversity, especially across mid- and low-resource languages? Are there quantitative or human evaluations to validate translation accuracy?\n\n3. **Section 4:** The scaling experiments primarily rely on long reasoning models such as DeepSeek-R1. Have the authors observed similar multilingual patterns in other model families (e.g., open-weight LLaMA or Qwen models) that are not explicitly optimized for long CoT reasoning?\n\n4. **Line 258:** For high-resource non-English languages such as Chinese, Target-CoT performance drops even after scaling. Could this be partly due to bias introduced by the 20k English SFT data used during fine-tuning?\n\n5. **Suggestion:** Including results from LLaMA-based models in the main paper (rather than the appendix) would strengthen cross-model comparisons and make the findings more broadly convincing.\n\n---", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper conducts an extensive assessment of long chain-of-thought reasoning across nine non-English languages spanning high-, mid-, and low-resource settings, providing a solid empirical foundation for future multilingual reasoning research.\n\n2. The study disentangles the effects of scaling, pretraining, post-training, and inference, offering clear and interpretable insights into how multilingual reasoning capabilities emerge and evolve.\n\n3. The results reveal several unexpected yet important trends—such as specialized reasoning pretraining degrading target-language reasoning, and translated reasoning traces outperforming distilled ones—offering actionable implications for improving multilingual LLMs.", "weaknesses": "1. While the empirical results are comprehensive, the paper provides limited theoretical insight into why Target-CoT performance remains low even as models scale. A deeper analysis—e.g., examining the role of tokenizer design, cross-lingual alignment, or representational interference—would strengthen the interpretation of the observed trends.\n\n2. The experiments primarily focus on mathematical reasoning tasks, which may understate the linguistic challenges inherent to multilingual CoT. Including broader reasoning or commonsense understanding benchmarks would make the conclusions more general and impactful.\n\n3. The amount of English reasoning data greatly exceeds that of the target languages, which could introduce residual bias despite the normalization analysis in Appendix B.4. More controlled data scaling or balanced sampling experiments would help validate the robustness of the conclusions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)