mR3: Multilingual Rubric-Agnostic Reward Reasoning Models
Authors: David Anugraha
Shou-Yi Hung
Zilu Tang
En-Shiun Annie Lee
Derry Tanti Wijaya
Genta Indra Winata
Venue: ICLR 2026 Poster (2026)
Keywords: reward model
reasoning
rubric
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce mR3, a massively multilingual, rubric-agnostic reward reasoning model trained on 72 languages.

Abstract: Evaluation using Large Language Model (LLM) judges has been widely adopted in English and shown to be effective for automatic evaluation. However, their performance does not generalize well to non-English settings, and it remains unclear what constitutes effective multilingual training for such judges. In this paper, we introduce mR3, a massively multilingual, rubric-agnostic reward reasoning model trained on 72 languages, achieving the broadest language coverage in reward modeling to date. We present a comprehensive study of data and curriculum selection for training to identify effective strategies and data sources for building high-quality reward models, including support for reasoning in the target language. Our approach attains state-of-the-art performance on multilingual reward model benchmarks, surpassing much larger models (i.e., GPT-OSS-120B) while being up to 9x smaller, and its effectiveness is further confirmed through extensive ablation studies. Finally, we demonstrate the effectiveness of mR3 in off-policy preference optimization and validate the quality of its reasoning traces and rubric-based evaluations through human studies with 20 annotators across 12 languages, where mR3 models' reasoning is preferred, including for extremely low-resource languages that are entirely unseen during training. Our models, data, and code are available as open source at https://github.com/rubricreward/mr3.

Published as a conference paper at ICLR 2026
MR3: MULTILINGUAL RUBRIC-AGNOSTIC REWARD REASONING MODELS
David Anugraha's Shou-Yi Hung', Zilu Annie En-Shiun Lee?.4, Derry Tanti Wijaya".s, Genta Indra Winata Stannorr University University of Toronto Boston University
*Ontario Tech University
'Monash University Indonesia Capital One
david. wwwgrana@stannoooooooooooo edu, genta. Wantaaaaaaat@aaappaaaoee.ccmmm
ABSTRACT
Evaluation using Large Language Model (LLM) judges has been widely adopted in English and shown be effective for automatic evaluation. However, their performance does not generalize well to non-English settings, and remains unclear what constitutes effective multilingual training for such judges. this paper, we introduce MR3, massively multilingual, rubric-agnostic reward reasoning model trained on languages, achieving the broadest language coverage in reward modeling to date. We present comprehensive study of data and curriculum selection for training to identify effective strategics and data sources for building high-quality reward models, including support for reasoning in the target language. Our approach attains state-of-the-an performance on multilingual reward model benchmarks, surpassing much larger models (i.e., GPT-OSS- (OSS-120B) 120B) while being up to 9x smaller, and its effectiveness is further confirmed through extensive ablation studies. Finally, we demonstrate the effectiveness of MR3 in off-policy preference optimization and validate the quality of its reasoning traces and rubric- based evaluations through human studies with 20 annotators across 12 languages, where MR3 models' reasoning is preferred, including for extremely low-resource languages that are entirely unseen during training. Our models, data, and code are available as open source at httpt:::///// / tig.....b....... com rubr wwwward/mr......
INTRODUCTION
Assessing the quality of Large Language Models (LLMs) is essential for understanding their gen- erative capabilities. Automatic evaluation methods are particularly valuable, as relying on human annotators prohibitively costly and inefficient. However, prior research has focused predominantly on English (Anugraha al. 2025 Chen et al., (2225)), leaving multilingual and non-English evalua- tion largely underexplored. Building reward models that generalize across languages is especially challenging in low-resource settings. While aligning models with human preferences is crucial,
collecting human judgments remains both expensive and time-consuming (Vu et al. 2024 Lin et al. 2025 Winata al. 2025).
The use of prior human evaluation data is a promising alternative, but is limited by the lack of standardization and documentation, inconsistent evaluation criteria, data privacy concerns, and proprietary restrictions (Anugraba et al.) 2025 Kim et al., 2025). Multilingual evaluation presents additional challenges, as requires both strong reasoning ability and robust cross- moossllnngual knowledge. Yet, effective strategies for training multilingual reward models remain largely unexplored, resulting in a persistent performance gap between multilingual and English settings. While recent models demonstrate strong reasoning ability in English, their multilingual reasoning capabilities remain questionable and often fall short compared their English counterparts (Yong et al. 2025).
In this paper, we introduce MR3, new family of massively multilingual, rubric-agnostic reward reasoning models designed to address the challenges of multilingual text evaluation. We conduct systematic study of the role of language across instructions, rubrics, responses, and reasoning, and analyze how target languages interact with each component of MR3 (Figure [ To ensure consistent evaluation, standardize the input format the reward models. Furthermore, present

---

Published as a conference paper at ICLR 2026
Be Visu 90 parte Con najorise
Vo Pig
Point-wise
Scoze
Seazzeling (oeg) Mecrmaee
Reanne Chee)
Reasceing (tra)
X Pair-wise
Roperaa
Ficast Aipas Tralerien
Dessoning (sng) Nuper pelrnc beck.
Recwoeiing (hra)
Selatiy Tountly
Binary
Bannsning (wog) claved
Jesemting Cear)
Beazsning
E-t0t
English Cong?
Korean (ber)
French (dew)
Figure I: The MR3 model supports multilingual input and enables reasoning outputs be tailored to user preferences. MR3 can process information, perform reasoning, and generate responses across multiple languages.
new dataset and benchmark with the broadest language coverage to date, enabling both training of reward models and robust assessment of LMs as judges. Our approach not only supports more flexible alignment with human values but also provides explicit reasoning for score assignments, leading to greater transparency and trustworthiness in evaluation. By systematically exploring these dimensions, our work sheds light on how language choices influence reward model training and lays the foundation for more consistent and effective multilingual evaluation. Our key contributions are fourfold:
We introduce task-agnostic framework for training massively multilingual reasoning reward models that leverages fine-grained rubrics, either human-craften or LLM-generated, for controllable and interpretable scoring. We show that MR3 outperforms existing reward models and achieves performance comparable to much larger models (e.g., MR3-QWEN- 14B vs. GPT-OSS- - 120B), while being up to 9x smaller.
We build a large and diverse multilingual dataset covering 72 languages from a wide range of sources and use it to train MR3 (Table D. representing the broadest language coverage to date. In addition, we construct a benchmark to evaluate our models across variety of tasks.
We study dataset selection and curriculum learning strategies along three dimensions: (i) instruction and rubric language, (ii) response and reasoning language, and (iii) methods for improving targel-language reasoning. Our findings show that although English remains the strongest prompting and reasoning language, targeted multilingual training substantially enhances MR3's robustness to surget-language inputs, enabling more accurate reasoning and evaluation. Moreover, when reasoning directly in the target language, MR3 delivers significant gains over the base model, highlighting the importance of cultivating high-quality target-language reasoning, even for extremely low-resource languages (LRLs) that are entirely unseen during MR3 training data.
We conduct off-policy preference optimization experiments to showcase our models' strengths in RL-based optimization. Additionally, we evaluate the quality of our reasoning traces and rubrics through human assessments involving 20 annotators across 12 languages, where annotators frequently prefer our models over existing reward models, including on extremely unseen LRLs that are unseen to the MR3 training data.
2 WHY DO WE NEED MULTILINGUAL RUBRIC-AGNOSTIC REASONING REWARD MODELS?
Underexplored Multilingual Reward Models. Research multilingual reward models remains highly limited, with only a few notable efforts such as M-Prometheus (Pombal et al., 2025). However, their work offers only a narrow exploration of training strategies and does not investigate how to construct effective datasets (c.g., through data sampling or generation methods). The study primarily focuses on training reward models with multilingual data, without further analysis of dataset selection,
2

---

Published as a conference paper at ICLR 2026
Table A comparison between existing models and MR3 across various dimensions, including data being used, task formats, and evaluation rubrics. "The model is neither closed-source nor proprietary.
Method
Lang
Data
Model
Tasks Pair-wise
Rubrics
Access
Point-wise
Binary
Customiizall
ArmoRM Waagg et al
2024g
974.4k
CLond Ankner ef
2024
-80k
8, 70
GenRM ZZzmn
9.
Judgel.RM1
CTen or M.
2015a
10OK
3.7
Prometheus
M1
2013
100k
13
Prometheus2
m-Prometheus
Poniiss 0
2025
6
480k
4. 8,
Self Self-Taught Waing
70
Nemetron. Ragrish Wany priale 2025c
32,
Nemotron Mukrilingual Wang et 2025c
13
40.5k
49, 70
SynRM Ne 2024
35
UniEval Zhong Crale 2022
185.5k
G-Eval or PATAS)
Hercule Desedlpanent
FLAMe Varet ai. 2024
5M+
RM-RI (Caen
20256
1100
7.
14,
Anugrana
[4k, 14k)
MR3
72
100K 4,8.14
alternative training strategies, or curriculum design, and with little attention to which reasoning languages are most effective. In contrast, our work systematically examines these dimensions, aiming to provide a more data-driven framework for training multilingual reward models.
Reward Models Struggle in Non-English Settings. Existing reward models still perform worse on non-English languages compared to English (Gureja al. 2024 Pombal et al. 2025). Many LLMs remain limited in their ability to generate coherent reasoning in LRLs, and their performance lags significantly behind that in English other high-resource languages such Chinese and Spanish. We conjecture that this gap stems from the scarcity of reasoning data LRLs, which leads to suboptimal results. In this work, aim to develop methods for more effective training in low-resource settings and to enhance model reasoning capabilities in target languages.
Limited Support for Various Scoring Tasks. Existing multilingual reward models are limited in their support for evaluation settings, often focusing only on pairwise comparisons as in Pombal et al. (2025); Wang al. (202cc) and do not handle point-wise or binary evaluations. To make rubrics more versatile and robust across diverse evaluation scenarios, we extend model training to support all these settings.
DATASET CONSTRUCTION AND TASKS
3.1
MR3 DATASET
3.1.1
OVERVIEW AND MOTIVATION
We propose a unified open-ended multilingual reasoning evaluation framework that evaluates candi- date responses against human-defined rubric, producing reasoning tokens behind the judgment, short explanation for interpretability. and a final scalar score. Formally, given a task instruction t, input instance i, one more candidate responses and evaluation rubric the reasoning model generates a reasoning trace, Irace, a concise explanation e justifying the evaluation, and score reflecting response quality under
f(x)
where (t, a, and y = (trace, s).
(1)
We define three task configurations under this framework: point-wise, pair-wise, and binary evaluation, which together cover wide range of structured and open-ended reasoning scenarios. More details regarding the formal definitions these tasks are provided in Appendix Section C.I
A central question in our setting is how to adapt this framework to the multilingual case. Since the input and candidate responses may be non-English:
The instruction & and rubric can be expressed either in English or in the target language of the input i, raising the question of whether evaluation criteria should be provided natively.
3

---

Published as a conference paper at ICLR 2026
Distill from teacher model with different strategies
Eng Prompt
Eng Thinking
Output
sample correct
Tgt Prompt
Eng Thinking
Output
Tgt Prompt
Tgt Thinking
Output
Input
Training
Response(s)
Discard easy samples
Output
Sample
x
Output
Eng Prompt
Eng Thinking
Output 3
Accept sample
#ksorectt
Output 4
Output 5
Figure 2: MR3 dataset construction that is aligned across different multilingual settings to highlight the trade-offs between using English and the input language for the prompts and reasoning traces. Here, prompr denotes both instruction and rubric, eng denotes English, and rgr denotes target language based on the input. A training sample is accepted if (1) all outputs distilled from GPT-OSS- 20B using different prompting and reasoning languages are correct, and (2) GPT-OSS-20B does not solve it consistently after being sampled five times.
The reasoning trace and explanation can also be generated in English or in the target language of the input i, shaping how supervised fine-tuning transfers reasoning ability across languages.
In the following subsections, we describe how our dataset construction addresses these challenges, enabling us not only to improve multilingual reward models but also to study the trade-offs between English and target-language supervision for both rubrics and reasoning traces. We will also use the terms input language and target language, interchangeably.
3.1.2 INITIAL CURATION
We begin by curating large collection of publicly available datasets, comprising over 3 million examples across 125 languages and multiple domains. Our sources include human preference datasets from Human Arena Preference (Chiang et al. 2024) and HelpSteer3-Preeferene (Wang et al. 2025c), a multilingual general knowledge dataset from MMMLU (Hendrycks et al.] 2020), a multilingual coding dataset from HumanEval-XX (Peng et al., (024), multilingual math dataset from MATH-500 Multilingual (Lightman et al.] 2023). and a multilingual safety dataset from PolyGuardMix (Kumar et al. 2025). We denote this pool as Dinits where each example x(i) is represented as . (t²), ((7) at7) r(i)), with {(5) the task instruction, j(5) the input, ( one or more candidate responses, and (0) an evaluation rubric when provided. A detailed summary of each dataset description, statistics, and language coverage is provided in Appendix C.2
Some datasets do not have explicit evaluation rubrics, which are necessary for our evaluation framework. Therefore, we automatically generate rubrics in English at inference time using GPT- 4.1, based on the task type and the given task description. For robustness, we generate multiple paraphrased variants of each rubric in English. Next, for each sample !.()), we distill the expected natural language output, أ(و) using surpasses 03-MINI and matches 03 and 04 (Agarwal et al. 2025) and avoid the cost spending APis. More details about the prompts to generate the rubrics, output distillation, and human-validation of the generated rubrics by GPT 4.1 provided in Appendix C.2 and Appendix [.I
-IT-OSS--2OB, strong open-sourced reasoning model that
3.1.3 FILTERING AND FINAL MR3 DATASET CONSTRUCTION
After initial curation, we construct multiple multilingual dataset variants to study the effects of English versus target language for instructions, rubrics, and reasoning traces. Figure @
provides
4

---

Published as a conference paper at ICLR 2026
overview of the construction process, highlighting how each input is associated with high-quality outputs under different strategies.
Multilingual Reasoning Strategies. outputs from GPT-OSS-120D5
We consider three strategies for generating natural language
English Instruction/Rubrie English Reasoning (ENG-ENG): the model receives instruc- tions and rubrics in English, and reasoning is generated in English, regardless of the input language.
Target Instruction/Rubrie + English Reasoning (TGT-ENG): instructions and rubrics translated into the target language of the input using GPT-4.1, but reasoning is still generated in English.
Target Instruction/Rubric Target Reasoning (TGT-TGT): instructions and rubrics are in the target language, and reasoning is forced to be generated in the target language using system prompts and initial reasoning tokens target language (Yong et al. 2025).
We retain only those training samples for which all three strategies produce correct outputs, minimiz- ing confounding effects when comparing strategies. Details translation prompts provided in Appendix C.2.2 and the language-forcing procedure of the reasoning is provided in Appendix D.4
Filtering by Difficulty. To further ensure high-quality supervision, we discard samples that GPT- OSS-20B, the smaller version of the teacher model, can solve consistently. We select this model because it has reasoning capabilities, albeit weaker than GPT-OSS- 20B, and can solve certain examples reliably up five times. This filtering removes "easy" samples that the models likely already familiar with. After this process, the resulting dataset contains 441,199 high-quality examples aligned across different multilingual settings.
Data Selection. Finally, downsample the dataset to 100k examples obtain final dataset of D10ok. Since Human Arena Preference, HelpSteer3, MATH-500 Multilingual, and HumanEval-XX relatively small, include all of their samples in the final curated dataset. Next, we include all samples from MMMLU and PolyGuardMix for which GPT-OSS-20 20B achieves correctness < 2 out of 5 trials, indicating that these are difficult examples. Lastly, we sample additional data from the remaining MMMLU and PolyGuardMix pools, assigning higher weight to samples with a correctness score of compared to those with a score of 4, until the dataset reaches 100k examples. The resulting dataset D ons thus consists of 100,000 challenging and diverse training examples spanning 72 languages. Detailed statistics for D P100k along with the ablation studies regarding the dataset sizes are provided in Appendix Section C.2
3.2
REWARD MODELS TRAINING AND EVALUATION
3.2.
REWARD MODEL TRAINING OBJECTIVE
Given our generated training data, we further use supervised fine-tuning (SFT) enhance the base model's 'ss reasoning capability reward model by minimizing the negative log-likelihood of reference responses. Given our training dataset Diooe = (x(()) ,,y ) where (((6) the prompt input and
(()) (v1") (1) (1)
!T. the corresponding target sequence introduced in Eq. ( the objective the
cross-entropy loss:
ΣΣ T. N
LSPT (0)
1
>
log Tyy
/ .0)))
where To (y: x) denotes the model's conditional probability token yt given the history yee and prompt x, parameterized by 0. By directly maximizing the log-likelihood of the ground-truth tokens, this loss encourages the base model to produce high-quality reasoning traces and the desired format for pair-wis, wise comparisons learning-bait based approuches, such as RLVR (Chen et al., 20255), since we found them to be less effective in our setting. Further details provided in Appendix H.5
angle-answer rewards. We use SFT rather than reinforcement
5

---

Published as a conference paper at ICLR 2026
3.2.2 CURRICULUM TRAINING
We experiment with several curriculum strategies, including random shuffling, English-firs: ordering, difficulty-basse ordering, and hybrid variants. Using the HelpSteer3 (Wang et al., 2025c) validation set, we find that sorting training data from easiest hardest yields the best performance, where diffi- culty is defined primarily by consistency of correct predictions (obtained previously) and secondarily by token length. Detailed definitions and comparisons of all curriculum strategies are provided in Appendix Section C.2.4
3.2.3 REWARD MODEL EVALUATION
For our MR3 models, we primarily perform SFT on the QWEN3 model family YYag et al. 2025 at the 4B, 8B, and 14B scales. We also perform SFT on QWEN2.5- 1...--4B--ISTRUCC and DEEPSEEK- R1-14B (Guo et al. 2025) for comparison with base models from prior works.
To compare our open-source models against existing open-source generative reward model baselines, we consider the following models: R3 (Anugraha et al. 2025). suite of casoning-rrbbic-bann reward models; LLAMA-3.3 3.0--...-NNEOOO 49B ENGLISH and LLAMA-3.. L.AMA-3.3-NEMOTTE -EMMOROOO 49B MULTILINGUAL (Wang al. 2025c), preference-basee generative reward models for (multilin- gual) reasoning; RM-R1 (Chen et al. 2025b), a preference-bare
generative reasoning reward model; PROMETHEUS-VE v2.0 (Kim et al. 2002). rubric-based LLM-as-a-judde framework; and M-PROMETIEUSS (Pombal et al. 2025), the multilingual extension of PROMETIIBUS-V2.0
Finally, evaluate the reward models across diverse suite of multilingual benchmarks span- ning multiple evaluation paradigms and languages. Our evaluation set includes preference- based benchmarks such as reward-bench (Lambert al., (204). m-reward-bench (Gureja et al. (024). MM-Eval (Son et al. 2024), and IndoPref (Wiyono et al., (025). covering 30 unique languages across wide range of domains and cultures; MGSM, a multilingual mathematics benchmark in in languages (Shi et al. 2023); ACLUDE-base
multilingual cultural knowledge benchmark on 44 languages (Romanou et al. ; and RTP-LX (de Wynter al. (025). multilingual safety dataset spanning 28 languages that provides out-of-distribution coverage relative to PolyGuardMix (Kumar jet al. 2025) our MR3 training set. More details about the evaluation dataset description, statistics, and language coverage in Appendix Section C.4
3.3 POLICY MODEL ALIGNMENT TRAINING AND EVALUATION
For policy model alignment, we post-train QWEN3- 30B A A3B --INSTRUCT Yang et al. 2025) us- ing Direct Preference Optimization (DPO) (Rafailov et al.] 2023) by using prompts from Aya Dataset (Singh et al. 2024) and m-reward-bench (Gureja et al. 2024) for general chat dataset, and PolyMath (Wang et al. 2025a) for math-reasoning. Details about each dataset, along with a descrip- tion of our extensive deduplication procedures to prevent any possible data leakage, are provided in Appendix Section C.3
We evaluate the aligned models on several multilingual benchmarks, including M-ArenaHard- v2.0 (Khairi et al. 2025) for multilingual instruction-folloinn benchmark on 23 languages, INCLUDE-base-44 (Romanou et al. 2024), and MCLM (Son et al. (025) for multilingual math- reasoning benchmark, which consists of 55 languages. Specifically for m-ArenaHard-v22. we GPT-4.1-M1I
the automatic judge due its free-form instruction-following nature. To limit computation, GPT-4. 1 -MINI'' reference output is always placed in the first position, giving OPT-4.1-MINI position-bia advantage (Shi al., 2024). This setup enables us to assess the effectiveness of preference-based alignment in improving model helpfulness, robustness, general instruction corructionfoollowingg knowledge, and reasoning across multiple languages. Details about each dataset can be found in Appendix Section C.6
4 RESULTS AND ANALYSIS
4.1 OVERALL PERFORMANCE
Table reports the performance of MR3 compared to base models and prior reward models on pairwise preference benchmarks under the English-prompt, English-thinking setting, the typical

---

Published as a conference paper at ICLR 2026
Table Overall results of MR3 compared other baselines when prompted with English and think on English (when applicable for reasoning models) on pairwise evaluation benchmarks, reported as average + standard deviation across 5 runs. Bolded and underlined indicate the pest-performing results and second-best berf--eeerrroingggggge results, respectively. Note that some standard deviations are not reported because they were not included the corresponding works.
Model
m- Re-RewardBench
RewardBench Acc. lang
MM-Eval
IndoPref
Acc. 23 langs
Acc.
Acc. lang
18 langs
Base Models
QWEN3-4B
84.51 0.08
88.04
0.37
80.07 0.52
68.80
0.32
QWEN3-8B
86.57 1: 000
88.77
0.13
81.95 0.31
72.30 0:45
QWEN3-14B
88.46 +: 0.12
89.72
0.42
84.33
0.35
73.22
0.5
GPT-OSS-20B
36.66.
87.81
82.03
69.71
GPT-OSS- 120B
89.05 0.06
90.30 0.90
85.01
0:034
72.15
::
DEEPSEEK-RI-145
68.53 +: 1,62
70.91 1.48
58.77
2.10
55.19 2:3
QWEN2.5- INSTRUCT INTTSTRUT
77.21
80.64 0.28
78.00
70.04
Existing Reward Models
PROMETHEUS- 7B-v2.0
67.31
72.05
60.90
57.41 0.72
PROMETHEUS 8x7B-V2.0
75.15
74.06
64.34
58.38 0
M- PROMETHEUS-7P
77.54
76.84
69.66
60.08 0.66
M-PROMETHEUS--
79.51
79.67
77.26
48.16
R3-QWEN3-14BH
14B-LORA-44K
88.07 a:
91.00 0,40
84.04 J. 0.M
72.65 0.77
R3-QWEN3-SB-14K
85.86 1: 0.26
88. $8.80 0.09
80.03 0.59
71.60 0.75
R3-QWEN3--BB-14
84.64 1:
0.20
79.37 0.40
084
RM-RI-14B
85.49
88.51
74.12
66.42 1.72
RM-R1-32B
87.98 +: 88.25 + 0.02
90.89
80.62
0.67
69.33 0.56
NEMOTRON-49B- --------9--E
88.72 0.29
75.47
69.59 0.26
REMOTRON-MUUTTLNNUUL-PP0 THINKING
89.03
89.62
76.27
68.40
MR3 Models (Ours)
MRR--WEN3--4
87.61
89.74 0.50
82.62
72.22 0:25
MR3-QWEN3-BB
88.44 +: 0.07
90.50 0.25
84.84 0.37
72.86 0.16
MR3-QWEN3-14D 14B
89.18
0.00
90.79 0.25
86.05
0.18
74.14
MR3-DEEPSSEK-RR--4B
87.12
88.73
81.85
70.11
HIR3-QWEN2.5-14B-B11 INSTRUCC
85.41 a 0.99
88.21 0511
81,51 0.06
68.10 0.55
evaluation setup adopted in prior works (Pombal et al. 2025 Wang al. 2025c). Our best model, MR3-QWEN3-11 14B, achieves an average accuracy of 85.04% substantially outperforming all prior reward models, and surpassing the strongest multilingual baselines: +4.21 points over NEMOTRON- MULTILINGUAL--4
and +0.91 points over GPT-OSS 120B, teacher model, despite being up to 4x and 9x smaller in size, respectively. Furthermore, even our smallest MR3-QWEN3-4B surpasses most baselines of comparable or larger scale, with the exception of GPT-OSS-120B, QWEN3-14B, 14B, and R3-QWEN3- WWWEEE3--1B-LLR.RRO 14B-LORA-4K
These gains stem from the use of multilingual supervision dataset. While R3 models, which were trained solely on English data, achieve the strongest results on RewardBench (English-only) compared other models, they underperform Eval, and IndoPref. In contrast, MR3 demonstrates consistent improvements across English and multilingual settings, thereby narrowing this performance gap. Finally, we also observe a scaling trend within the MR3 family. As model size increases from MR3-QWEN3-4P to MR3-QWEN EN3-8B and MR3-QWEN3-1L 4B, the performance of our MR3 models steadily improves across all benchmarks, indicating that our multilingual training strategy scales effectively with the model size.
multilingual benchmarks such m-RewardBench, MM-
Table B reports evaluation results on INCLUDE-basee prompt, English-thinking setting. Both NEMOTRON and RM-RI models do not support these tasks, similarly with PROMETHEUS models that support only rubrics with Likert-scale of 1-5, so only reported the results for R3 models. We observe the same overall trend: MR3 models consistently improve over their base models and R3 counterparts, with performance scaling as model size increases. While MR3-Qwe 3-3--weE3-144 is slightly behind GPT-OSS- 120B these benchmarks, it remains competitive despite being substantially smaller.
MGSM, and RTP-LX under the English-
Overall, MR3 consistently improves upon its base models, surpassing GPT-OSS-1220 on pair- wise preference benchmarks and demonstrating the effectiveness of our dataset construction and multilingual training pipeline. More detailed results can be found in Appendix Section G

---

Published as a conference paper at ICLR 2026
Table 3: Overall results of MR3 compared to other baselines when prompted with English and think on English (when applicable for reasoning models) INCLUDE-base (general knowledge), MGSM (math), and RTP-LX (safety) evaluation benchmarks, reported as average standard deviation across 5 runs. Bolded numbers indicate the best-performing results, while underlined numbers indicate the beoond-best-pefoommee results.
Model
INCLUDE
MGSM
RTP-LX Fl. 27 langs
Acc. 44 langs
Acc.
11 lang
Base Models
QWEN3-4B QWEN3-8B
61.54 0.22
90.34 = 0.14
84.10 + 0.16
66.55 + 0.09
92.52 0.18
78.41
+ 1.23
OWEN3-14B
69.70 0.14
93.49
78.36 0.49
GPT-OSS-20B
62.55 0.28
92.57 = 0.22
90.60 = 0.20
GPT-OSS- 120B
71.65 +: 0.21
94,71 0.17
91.32
+ 0.10
DEEPSEEK-RIE -I-11E
21.78 L83
69.81 3.15
83.83
:: 0.82
OWHN2.5-14B-INSTRUCC
57.93
84.93 0:21
91.34 0.14
Existing Reward Models
33-QWEN3-4B-1444 R3-QWWN3-8BB- 14K
60.51 0.29
90.26 0.24
87.47 = 0.27
65.46 0.28
92.31
87.10
+: 0.51
1.--WW33-1BB-LORA-4K
69.41 0.30
93.36 0.12
79.36 1: 0.60
MR3 Models (Ours)
MRR--QWE33-44
63.01 0.13
91.20
88.20 +
MR3-QWENS
67.72 0.16
93.20 = 0.20
90.03 = 0.12
MR3-QWEN3-144
70.61 0.26
94.00 0.07
90.19 + 0.10
60.37 0:25
89.48
0.27
89.53 1: 0.34
MR3-QWEN2.5-14B-NNM INNSTRCCT
63.72 + 0.51
90.41
0.16
90.38 + 0.21
4.2 INSTRUCTION AND REASONING IN ENGLISH VS. TARGET LANGUAGE
We further investigate the impact of prompting and reasoning language strategies after fine-tuning, comparing ENG-ENG, TGT-ENG, and TGT-TGT as defined during dataset construction. Since these datasets are aligned, performance differences primarily reflect reasoning language rather than content.
Figure B shows that fine-tuning improves performance across strategies, where ENG-ENG remains strongest in absolute terms, followed closely by TGT-ENG, with larger models showing greater robustness to non-English prompts. Smaller models (e.g.. QWEN3-4B) are more sensitive before fine- tuning. but multilingual training substantially reduces this gap. Although base TGT-IGT performance lower across all models, it exhibits the largest relative gains after fine-tuning, even surpassing the base model's ENG-ENG performance. This indicates that our training strategy effectivelyy improves reasoning directly in the target language, enabling interpretable model decisions in users' preferred languages.
These results confirm that our multilingual training strategy enhances performance across all prompt- ing and reasoning combinations, with the most pronounced gains in target-language reasoning, crucial for interpretability, accessibility, and especially for LRL.s.
4.3 MODEL ALIGNMENT EVALUATION RESULTS
4able the performance of the base model, SWWN3-330-A332 compared to the model aligned using DPO with our reward model, MR3-QWENSS 14B and Mumutrro-Mulilliggual--99 which the strongest reward model among other baselines. We observe notable improvements INCLUDE- base-44 and the MCLM math-reasoning benchmarks when post-trained using MR3-QWEN3- 4B, indicating enhanced performance multilingual general-knowllddee and reasoning tasks in verifiable tasks. On Awwnaaardd-22... our DPO- DPO-aligned model increases winrate from 39.1%% 1% to 45.2% for multilingual instruction following, aligned model also exceeds GPT-4, -mmni'' 's own performance English, and approaches in other languages, despite the well-know first-position bias of L.LM judges (Shi al. 2024).
cestruction-followiingg where the English winrate increases from 49.1% to 57.3%. The
8

---

Published as a conference paper at ICLR 2026
86
54.90
D1.71
$4.50
04
03.19
Avg. Performance
H22
$1.077
EE.0
GPT-0SS-120R
00:31
kng Prompt lng Thinking (Uase)
00
19.07
Eng Prompt-Eng Thinking
Prompt-Eng Thinking
78
Tgt Prompt Eng Thinking
70.00
Prompt Tyt Thinking (Dase)
-6-
Tyt Prompt-Tgt Thinking
Prompt-Tgt Thinking (Trans.)
1.4
70
Qwen3-43
Qwen3-6B
Qwen3:1 1-003-148
Models
Figure 3: Average performance of the MR3 models (solid bars) and their base models (hatched bars) across different parameter sizes and multilingual prompting and reasoning strategies. The performance each MR3 model consistently improves its corresponding base model for every different strategy, especially when thinking in the target language, which is important.
Table Overall results of DPO on QWEN3-30B -----BBB------- 1-33--ISTTUCT--- --0007 as the base model, using MR3-QWEN3- 14B as the reward model. Results are reported as mean + standard deviation across runs. Evaluations cover AICLUDE-base-AR MCLM, and m- AreeaHardd-222.0 Winrate (WR) on m 1n-ArrnaHHad--22.01 is measured against GPT-4. with GPT-4.1-ming reference output placed in the first position, giving it position-bis bias advantage.
m-ArcmaHard--220
INCLUDE
Englisb-only
Overall
M-IMO
MT-AIMEX24
MT-MATHIO0 Acc.
WR (%) 95% CI
WR (8)
95% C1
Ace.
Acc:
Asc.
lang
23 laags
44 langs
38 langs
56 langs
55 langs
QWEN3- A3B- INSTRUCT-2507
19.1
(2.66
39.1
[8.4, 39.9)
61
10.22
60.79
90:17
DPO Mohilingui--088
59.95.5999
(47.8.47.)
66.09 sus:
DPO w/ mR mR3-Quen3.14 (Qurs)
$7.3
61.3
45.2
(4.94
68.75 1:300
44.02 nas
65.00 2:05)
92.08 8.
4.4 HUMAN EVALUATIONS ON RUBRICS AND REASONING TRACES
To validate our rubric generation, training-date reasoning trace quality, and the reasoning quality of MR3 models, we conducted human evaluation with 20 native speakers across 12 high-, medium-, and low low-resource languages. Annotators are given instructions to assess our rubrics, training reasoning traces, and output reasoning traces from MR3-QwER PR3--WEN3-118 14B and QWEN3- 14B. All scores range from 1-3, where 3 indicates the highest quality. The details of the annotators, instructions and rubrics used for human evaluations, along with more in-depth analysis, can be found in Appendix Section
Human Evaluations on Generated Rubrics. On average, rubrics score (2.93 + 0.09) plau- sibility, (2.72 + 0.29) on scoreability, and (2.72 + 0.35) on translation quality, indicating that the generated rubries are high quality.
Human Evaluations on Reasoning Traces of Training Dataset. We evaluate the logical coherence and factual correctness of training-data reasoning traces distilled from GPT-OSS- 20B. We find the following trend: (2.97 + 0.04) factual and (2.81 + 0.28) logical for English reasoning, (2.87 + 0.24) factual and (2.72 + 0.42) logical for target-language reasoning, and (2.78 + 0.31) factual and (2.64 + 0.31) logical for translated target-language reasoning. This pattern is expected, given that teacher models are primarily trained in English and that translations of English reasoning may introduce mild unnaturalness.
Human Evaluations on MR3 Reasoning Traces. Our MR3 model with target-language reasoning achieves (2.78 + 0.30) factual and (2.67 + 0.45) logical quality, substantially outperforming the QWEN3 baseline, which scores (2.06 + 0.69) factual and (2.05 + 0.71) logical. Annotators report that MR3 model's reasoning is more succinct. fluent, coherent, and culturally aligned. Several annotators also provide feedback that larget-language reasoning trace is preferred over English reasoning trace, indicating improved nuance and cultural fit.

---

Published as a conference paper at ICLR 2026
5 RELATED WORK
LLM-as-a-judge Framework. As language models become more capable of following instructions,
traditional generation evaluation metric such BLEU (Papineni et al.] 2002). ROUGE (Lin 2004)
have evolved to model-finetuned scalar score outputs (BERTScore, (Zhang et al. 2019 and ensemble-
based metries (Anugraha al., 2024; Winata al. 2025). With popularization of reinforcement
learning as a way to finetune pretrained models (Ouyang et al. 2022 Bai et al. [2222] Winata et al.)
2025| Zhao et al., 22255, many efforts evolve in building reward models, which outputs a scalar value as "preference" for the generation (Ouyang et al. 2022 Lambert et al. 2024 Wang et al. (025c). As
I.LMs are adopted for more tasks, judge models that focus on single dimensions become inadequate
in addressing diverse needs of the users (Li ct al. 2023 Dubois et al. 2023 Zheng ct al. 2023).
LLM-as-judge initially focus on prompting large, closed- source models (Liu et al. 2023) ), while
more recent work finetuned model to generate evaluations (Wang et al. 2024c; Kim et al. 2023 Vu
et al. 2024 Chen et al.] 2025b). Evaluation format for judge model mostly fall into two categories:
point-wise assessment and pair-wic wwie comparison. Few works combine above types of evaluation
together to enable fine-grained evaluation (Kim et al., [2033] 2024 Deshpande et al.) 2024) Vu et al., 2024 Chen et al. 2025b). Different from them, we sample from much more diverse tasks, including binary classifications, with high quality filters that result in a small but effective training dataset.
Rubric-Based Evaluation Models. Recent work on rubric- baaeed evaluation models with LLMs is centered around dnestion-specifee (Wang et al. [2025b] Fan et al., [2024] Pathak et al. (025).
calibrated (Hashemi et al. 2024] Anugraha et al. 2025; Tie et al.) (025). ). and human-in- lhop
designs (Li et al. [025 Senanayake & Asanka, 2024 Moore et al. 2024 LLM-Rubric (Hashemi
cct al. (2024). for example, treats evaluation as LLMs answering multidimensional rubric questions
and then calibrates small neural network model to combine those responses similar to (Ti et al.)
2025). Other works emphasize question-specie rubrics (Pathak et al. (025) and multi-agent pipeline with rubric generation
intermediate step (Wang al.) (2256;] Fan et al., (024). Works on rubric-based LLM evaluation are increasingly conducted in education (Senanayake & Asanka 2024]
Moore al. 2024), combining LLM rubric-based automated scoring with human evaluation. A recent systematic review of LLM-based assessments also notes that roughly two-thirds of studies only use English data. This study highlights that multilingual evaluation remains an open problem for rubric-based evaluation models (Emirtekin 2025), an issue that we are tackling in this paper. Anugraha et al. (2025) introduce the first generative, rubric-based reward reasoning model supporting pointwise, pairwise, and binary evaluation settings across diverse input formats. In this work, we adopt this framework and further extend MR3 to support multilingual use cases.
Multilingual Evaluators. While most evaluators only work in English, some recent efforts expanding into multilingual space. HelpSteer3 (Wang et al., 2025c) contains 12 natural languages, leads to competitive multilingual reward model. Among multilingual generative judge models, both Hercule (Doddapaneni et al.] 2024), and M-Prometheus (Pombal et al. 2025) finetuned on translated Prometheus data in six to eight languages. Compared to existing work, our dataset is one of the most diverse in-terms of task and languages, with our model empirically outperform alternatives models. We also further verify MR3 models with 20 annotators across 12 languages, including on extremely unseen LRLs that are unseen to the MR3 training data.
6 CONCLUSION
In this paper, we introduce MR3, a task-agnostic framework for training massively multilingual reasoning reward models that leverages fine-grained rubrics for controllable and interpretable scor- ing. Through careful dataset selection and curriculum selection, we construct large and diverse multilingual dataset covering 72 languages from a wide range of sources. We demonstrate that MR3 fine-tuned with our dataset outperforms existing reward models, including much larger models such as NEMOTRON-MULTILINCE MULTILINGUAL-4499 and GPT-OSS- 120B, despite being up to 9x smaller. We further explore different multilingual dataset settings by varying the instruction and rubric language as well as the response and reasoning language. Our findings show that while English remains the most effective prompting and reasoning language, our targeted multilingual training enables MR3 to handle target- target-language inputs more robustly, producing more accurate reasoning and evaluations in the target language, making reasoning models more accessible to non-English speakers.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Kingma DP Ba J Adam et al. A method for stochastic optimization. arXiv preprint arXiv: 1412.6980. 1412(6), 2014.
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508. 10925, 2025.
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Maxamed Axmed, et al. Mega: Multilingual evaluation of generative ai. arXiv preprint arXiv:2303. 125222, 2023.
Zachary Ankner, Mansheej Paul, Brandon Cui. Jonathan D Chang, and Prithviraj Ammanabrolu. Critique-out-loud reward models. arXiv preprint arXiv:2408. 11791, 2024.
David Anugraha, Garry Kuwanto, Lucky Susanto, Derry Tanti Wijaya, and Genta Indra Winata. Metametrics-mt: Tuning machine translation metametrics via human preference calibration. In Proceedings of the Ninth Conference Machine Translation, USA. Association for Computational Linguistics, 2024.
David Anugraha, Zilu Tang, Lester James V Miranda, Hanyang Zhao, Mohammad Rifqi Farhansyah, Garry Kuwanto, Derry Wijaya, and Genta Indra Winata. R3: Robust rubric-agnostic reward models. arXiv preprini 2505v:2505. 13388, 2025.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. CoRR, 2022.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. 2021.
Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. Judgelrm: Large reasoning models judge. arXiv preprint arXiv:2504.0000 2025a.
Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang. Tong Zhang, et al. Rm-rl: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387, 2025b.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li. Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating Ilms by human preference. In Forty-first International Conference on Machine Learning, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 2021.
Adrian de Wynter, Ishaan Watts, Tua Wongsangaroonsri, Minghui Zhang, Noura Farra, Nektar Ege Altintoprak, Lena Baur, Samantha Claudet, Pavel Gajdušek, Qilong Gu, et al. Rip-Ix: Can llms evaluate toxicity in multilingual scenarios? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp.
2025.
Darshan Deshpande, Selvan Sunitha Ravi, CH- Wang Sky, Bartosz Mielczarek, Anand Kannappan, and Rebecca Qian. Glider: Grading 11m interactions and decisions using explainable ranking. CoRR, 2024.
Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Dilip Venkatesh, Raj Dabre, Anoop Kunchukuttan. and Mitesh M Khapra. Cross-lingual auto evaluation for assessing multilingual llms. CoRR, 2024.
11

---

Published as a conference paper at ICLR 2026
Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Guirajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. Advances in Neural Information Processing Systems, 36:30039-30069, 2023.
Emrah Emirtekin. Large language model-powered automated assessment: A systematic review. Applied Sciences, 15(10):5683, 2025.
Zhiyuan Fan, Weinong Wang. Debing Zhang, et al. Sedareval: Automated evaluation using self- adaptive rubrics. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp.
16916-16930, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, arXiv preprint arXiv:2501.12245 2025.
Srishti Gureja, Lester James V Miranda, Shayekh Bin Islam, Rishabh Maheshwary, Drishti Sharma, Gusti Winata, Nathan Lambert, Sebastian Ruder, Sara Hooker, and Marzieh Fadaee. M- rewardbench: Evaluating reward models in multilingual settings. arXiv preprint arXiv:2410. 15522, 2024.
Helia Hashemi, Jason Eisner, Corby Rosset, Benjamin Van Durme, and Chris Kedzie. LIm-rubric: multidimensional, calibrated approach to automated evaluation of natural language texts. arXiv preprint arXiv:2501. 000244, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.00 03000, 2020.
Pratik Joshi, Scbastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. The state and fate of linguistic diversity and inclusion in the nlp world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6282-62822 6293, 2020.
Ammar Khairi, Daniel D' D'souza, Ye Shen, Julia Kreutzer, and Sara Hooker. When life gives you samples: The benefits of scaling up inference compute for multilingual llms. arXiv preprint arXiv:2506.20544 2025.
Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, et al. Prometheus: Inducing fine-grained eval- uation capability in language models. In The Twelfth International Conference Learning Representations, 2023.
Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Prometheus 2: An open source language model specialized in evaluating other language models. In EMNLP, 2024.
Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Back, et al. The biggen bench: A principled benchmark for fine-grained evaluation language models with language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5877-5919, 2025.
Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, and Maarten Sap. Polyguard: A multilingual safety moderation tool for 17 lan- guages. arXiv preprint arXiv:2504. 043770, 2025.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pp. 611-626, 2023.
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. Rewardbench: Evaluating reward models for language modeling, arXiv preprint arXiv: 13787, 2024.
12

---

Published as a conference paper at ICLR 2026
Hang Li, Yucheng Chu, Kaiqi Yang, Yasemin Copur-Genctee Gencturk, and Jiliang Tang. Lim-based automated grading with human in nn-the-loop. arXiv preprint arXiv:2504.05523, 2025.
Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and lon Stoica. From crowdsourced data to high-quality benchmarks: Arena-han and benchbuilder pipeline. arXiv preprint arXiv:2406.1 1939, 2024.
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacaeval: An automatic evaluator of instruction- followiig models, 2023.
Hunter Lightman. Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.22050, 2023.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. Text summarization branches out, pp. 74-81, 2004.
Qi Lin, Hengtong Lu, Caixia Yuan, Xiaojie Wang, Huixing Jiang, and Wei Chen. Data with high and consistent preference difference are better for reward model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 27482-27490, 2025.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg cvaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634 2023.
Steven Moore, Norman Bier, and John Stamper. Assessing educational quality: Comparative analysis of crowdsourced, expert, and ai-driven rubric applications. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 12, pp. 115-125, 2024.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Aditya Pathak, Rachit Gandhi, Vaibhav Uttam, Arnav Ramamoorthy, Pratyush Ghosh, Aaryan Raj Jindal, Shreyash Verma, Aditya Mittal, Aashna Ased, Chirag Khatri, al. Rubric is all you need: Improving l1m- based code evaluation with question-specinee rubrics. In Proceedings of the 2025 ACM Conference on International Computing Education Research V. 1, pp. 181-195, 2025.
Qiwei Peng, Yekun Chai, and Xuhong Li. Humaneval-xl: A multilingual code generation benchmark for cioss-lingual natural language generalization. arXiv preprint arXiv:2402. 2024.
José Pombal, Dongkeun Yoon, Patrick Fernandes, Ian Wu, Seungone Kim, Ricardo Rei, Graham Neubig, and André FT Martins. M-prometheus: A suite of open multilingual Ilm judges. arXiv preprint arXiv:2504.04995 2025.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in neural information processing systems, 36:53728-53741, 2023.
Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A Haggag, Alfonso Amayuelas, et al. Include: Evaluating multilingual language understanding with regional knowledge. arXiv preprint arXiv:241 199999, 2024.
Chamuditha Senanayake and Dinesh Asanka. Rubric based automated short answer scoring using large language models (Ims). In 2024 international research conference on smart computing and systems engineering (SCSE), volume 7, pp. 1-6. IEFF, 2024.
13

---

Published as a conference paper at ICLR 2026
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung. Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Lan- guage models are multilingual thain-of-thought reasoners. In The Eleventh International Confer- ence on Learning Representations, 2023. URL ptttts:://oenneviiie ew net. forum? id= FR3wGCK-
Lin Shi. Weicheng Ma, and Soroush Vosoughi. Judging the judges: A systematic investigation of position bias in pairwise comparative assessments by llms. arXiv e-prints, pp. arXiv-2406, 2024.
Shivalika Singh, Freddie Vargus, Daniel D'souza, Börje F Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O' O'Mahony, et al. Aya dataset: An open-access collection for multilingual instruction tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11521- 11567, 2024.
Guijin Son, Dongkeun Yoon, Juyoung Suk, Javier Aula-Blasco, Mano Aslan, Vu Trong Kim, Shayekh Bin Islam, Jaume Coiss-Cristia, Lucía multilingual meta-evaluation benchmark for llm-ae-a-judge and reward models. arXiv preprint arXiv:2410. 17578, 2024.
and Seungone Kim. Mm-eval: A
Guijin Son, Jiwoo Hong, Hyunwoo Ko, and James Thome. Linguistic generalizability of test-time scaling in mathematical reasoning. arXiv preprint arXiv:2502.17407, 2025.
Mark TIč, Miguel treaalilo-Herréee Yuyan Wu, and Dejan Lavbič. On using large language models for rubric-based open question evaluation. In International Conference on Artificial Intelligence in Education, pp. 243 249.249. Springer, 2025.
Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, and Yun-Hsuan Sung. Foundational autoraters: Taming large language models for better automatic evaluation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 77086-17105. 2024.
Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. Interpretable preferences via multi-objectivv reward modeling and mixture-of-experts. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 10582-10592, 2024a.
Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi- Yu, Richard Yuanzhe Pang. Maryam Fazel-Zarandi, Jason Weston, and Xian Li. Self-taught evaluators. arXiv preprint arXiv:2408..066 02666, 2024b.
Yidong Wang. Zhuohao Yu, Wenjin Yao, Zhengran Zeng. Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, et al. Pandalm: An automatic evaluation benchmark for IIm instruction tuning optimization. In ICLR, 2024c.
Yiming Wang. Pei Zhang, Jialong Tang, Haoran Wei, Baosong Yang, Rui Wang, Chenshu Sun, Feitong Sun, Jiran Zhang, Junxuan Wu, Qiqian Cang, Yichang Zhang, Fei Huang, Junyang Lin, Fei Huang. and Jingren Zhou. Polymath: Evaluating mathematical reasoning in multilingual contexts. arXiv preprint 250v 18428, 2025a. htth:tts::
org/ abs/ 2 18428
Yu Wang, Madhumitha Gopalakrishnan, and Yoav Bergner. Using generated rubrics to provide window into item evaluation with multi-agent 1lms. In International Conference on Artificial Intelligence in Education, pp. 203-217. Springer, 2025b.
Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Moo-Chang Shin, Felipe Soares, Alexander Bukharin, Ellie Evans, Yi Dong, and Oleksii Kuchaiev. Helpsteer3-preferenn Open human-annotated preference data across diverse tasks and languages. arXiv preprint arXiv:2505. 11455, 2025c.
Genta Indra Winata, Hanyang Zhao, Anirban Das, Wenpin Tang, David Yao, Shi-Xiong Zhang, and Sambit Sahu. Preference tuning with human feedback on language, speech, and vision tasks: A survey. Journal of Artificial Intelligence Research, 82:2595-2661, 2025.
Vanessa Rebecca Wiyono, David Anugraha, Ayu Purwarianti, and Genta Indra Winata. Indopref: A multi-domain pairwise preference dataset for indonesian. arXiv preprint urXiv:2507.22/59 2025.
14

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09360 2025.
Zihuiwen Ye, Fraser Sceenlee-Scott, Max Bartolo, Phil Blunsom, Jon Ander Campos, and Matthias Gallé. Improving reward models with synthetic critiques. arXiv preprint arXiv:2405.20850 2024.
Zheng-Xin Yong, M Farid Adilazuarda, Jonibek Mansurov, Ruochen Zhang, Niklas Muennighoff, Carsten Eickhoff, Genta Indra Winata, Julia Kreutzer, Stephen H Bach, and Alham Fikri Aji. Crosslingual reasoning through test-time scaling. arXiv preprint arXiv:2505.05 054008 2025.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408. 152400, 2024.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2019.
Hanyang Zhao, Genta Indra Winata, Anirban Das, Shi-Xiong Zhang, David Yao, Wenpin Tang, and Sambit Sahu. RainbowPO: A unified framework for combining improvements preference optimization. In The Thirleenth International Conference on Learning Representations, 2025. net weteeeeweeeeeeeeeeeeeeeeeeeeeeee http://arrrreeeeeeeeeoooorrroora ftte://fformm/iirreeeeeepppp
URL
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li. Eric Xing, et al. Judging llm-as as-jge- with mt-bench and chatbot arena. Advances in neural information processing systems, 04446595-46622 2023.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3:
System Demonstrationss, Bangkok, Thailand, 2024. Association for Computational Linguistics. URL htttttt//aaxix org/ abs/2403 13372
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards a unified multi-dimensional evaluator for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 2023-2038, 2022.
ACKNOWLEDGMENTS
We would like to thank Divya Appapogu, Subhrangshu Bit, Dea Doksani, Jongin Kim, Santiago Martinez Novoa, Quang Phuoc Nguyen, Chau Pham, Juan Pablo Pietrini, Anshul Singh, Lucas Tassis, Kosei Uemura, and John Seon Keun Yi for their helpful discussions. This work also supported in DGECR-2024-00008) and Digital Research
part by NSERC Discovery Grants Alliance of Canada (formerly Canada Compute) Grant RRG 5397 on "Multilingual Multiculurral NLP and LLMs".
A LLM USAGE
Our work used AI Assistants such as ChatGPT for spell-checking and fixing minor grammatical mistakes. We also use ChatGPT to write parts of our codebase.
B LIMITATIONS
First, due to resource constraints, we only conduct SFT on the QWEN3 model family at the 4B, 8B, and 14B scales and compare only against several open-source generative reward models and not proprictary models. Given that we have 135K rows of data and around 135M input tokens, estimate IK output tokens per input-a conservative estimate considering the long reasoning
15

---

Published as a conference paper at ICLR 2026
output already incur cost of around USD $1,300 to benchmark the GPT-411 model, and around USD $300 for GPT-4.1 mini. Secondly, in this paper. we focus on multilingual reasoning and do not explore iterative refinement of the rubrics, which we plan to explore in our future works. Finally, we observe that models fine-tuned on English reasoning still perform better than models fine-tuned in target languages. The reason may be that the baseline models' inherent capabilities in English is superior than their capabilities in other languages (Ahuja et al. 2023). highlighting the need to collect large-scale pre-training data in other languages. Our work shows that, for non-English languages including low-resource settings, fine-tuning on reasoning data can consistently improve performance, without the need to collect scage-scale target language training data.
C DETAILS ABOUT DATASETS
C.I TASK FORMATS
To support wide range of evaluation settings, we define three task formats within our unified framework: point-wise, pair-wise, and binary evaluation. Each format shares the same input structure x = (t, i, a, r) and output structure y = (c, s) but differs in how the candidate responses are structured and how the score is defined.
Point-wise Evaluation. This format assesses the quality of a single response dj by assigning an integer score. It suitable for open-ended generation tasks where scalar assessments of quality needed, such as helpfulness, relevance, coherence, etc. Formally,
01, Spoint wise (t, i, a,) (e,
Z.
Pair-wise Evaluation. In this setting, the model compares two candidate responses aj and a2 to the same input and selects the preferred one, along with explanation. This format is commonly used in preference-based training. Formally,
(a1, 02), Spair- wise (1, a) (e,
8 (21,
Binary Evaluation. Binary task requires the model to make a definitive judgment about the correctness acceptability of response d1, given the input and rubric. These tasks span variety of use cases, including factual verification, binary classification (e.g., determining whether a summary is faithful), and structured reasoning (e.g., assessing the validity
math or code solution). Formally, fuinary (t, a, = (e, s), s E (true, false].
(5)
C.2 DETAILS ABOUT REWARD MODEL TRAINING DATASETS
C.2.1 MR3 DATASET SOURCE DESCRIPTION
Human Arena Preference CChang et al. 2024) contains multi-con multi-coonversation turns between humans and chatbots, and pairwise human preference votes from Chatbot Arena, open platform for eval- uating LLMs. Specifically, we start with Imarena ai/ si/arena- human comaa--reeerentte -10K, which include total of 126 languages, with top 52% being in English, followed by Polish (10%), Russian (7%), and Chinese (5%). Then, we discard all samples that have undefined languages.
PeeeStteer--reeeeeeeeee (Wagg al. 2025c) contains about 40K pair-wise human annotated preference samples in 13 natural languages. contains four domains: general, STEM, code, and Multilingual.
MMMLU (Hendrycks et al. 2020) contains MMLU test set translated into 14 languages. Questions include topics from elementary mathematics, US history, computer science, law, etc.
HumanEvalXL (Romanou et al. 2024) a multilingual, multi-programming language extension on the original HumanEva Chen et al. (2021) set of 164 Python programming problems with unit tests. The dataset contains 12 programming languages, and 23 natural languages. We take only the Python subset of the data. We augment this dataset by generating samples with negative answer (wrong Python code) to include false scores. This done using GPT-5 by providing the positive answer.
16

---

Published as a conference paper at ICLR 2026
MATH--50-Muttilnguall (Lightman al. 2023) is subset of MATH benchmark translated to French, Italian, Turkish, and Spanish. Similar to HumanEvalXL, we augment this dataset by generating samples with negative answer (wrong math solution) to include false scores. This is done using GPT-5 by providing the positive answer.
PolyGuardMix (Kumar et al. 2025) a safety-fooused dataset supporting 17 languages, aggre- gated from pre-existing safety datasets.
C.2.2 RUBRIC GENERATION
For pointwise pointwise-pairwise tasks such as HelpSteer3- Preefence and Human Arena Preference, we use their criteria choosing which response preferred. For strictly pairwise tasks such reward-bench, m-reward-henche MM-Eval, and IndoPref, we use their task descriptions and also the domain to generate rubrics when comparing responses. For safety dataset such PolyGuardMix and RTP-LX, we use their definitions of what is considered unsafe.
To generate the prompt tags, rubrics, and schema into different languages, we employed the following prompts and utilized GPT 4.1 to generate the translations of them. Examples of the translated rubrics and datasets can be found Appendix Section D
Translation of prompt tags into target language
Translate the following README title tags into natural, concise (language}. Only translate the values, not the JSON keys. Preserve formatting like capitalization.
Input: [tags_dict
Output: JSON with the same structure, with values translated into language.
Translation of task description into target language
Translate the following task description into (language). Do not provide any explanation, simply output your translation.
# Input task desc
Your Response
Translation of evaluation rubric into target language
Translate the following evaluation rubric into flanguage}.
Do not provide any explanation, simply output your translation.
Do not change JSON keys or placeholders, keep JSON structure intact.
# Input
(evaluation rubric}
# Output
JSON with the same structure, with the value translated into (language).
# Your Response
17

---

Published as a conference paper at ICLR 2026
Translation of output schema into target language
Translate the following schema description into {language}.
Do not provide any explanation, simply output your translation.
Do not change JSON keys or placeholders, keep JSON structure intact.
Ensure enum values (e.g.. "1", "2", "3", "4") remain in English.
Input {schema}
# Output Output: JSON with the same structure, with the value translated into language.
# Your Response
C.2.3
DATASET SIZE AND COMPOSITION
Datapoints
PolyGuard
Arena
HelpSteer3
MMMLU
MATH
HumanEval
Total
Raw
1,910,372 (84%)
135,634 (6%)
38,460 (2%)
196,588 (9%)
2,500
1,840 (1%)
2,285,394
Processed
2,987 (90%)
120,339 (4%)
38,460 (2%)
196,588 (6%)
5,000 (1%)
3,680 (1%)
3351317
MR3
50,916 (52%)
20,440 (21%)
15,936 (16%)
10,000 (11%)
2,238 (3%)
470 (1%)
100,000
Table Dataset composition across different stages of filtering.
Table shows the dataset composition across different stages of processing.
#Data (#Langs)
PolyGuard
Arena
HelpSteer3
MMMLU
MATH
HumanEval
Total
Other
(0)
(())
(0)
(
0
(
I (I)
(
(3)
(
(
(
(
(3)
Class
0
25 (14)
(0)
(
(0))
(0)
25 (14)
Class
(0)
4 (4)
(0)
(0)
(
(0)
4 (4)
Class
3,333 (1)
227 (22)
46 (1)
1,005 (1)
(0)
155 (8)
4,766 (23)
Class
24,077 (9)
4,722 (18)
774 (7)
3,547 (4)
701 (2)
177 (9)
33,998 (18)
Class 23,506 (7)
15,426 (8)
15,116 (6)
5,448 (5)
1,537 (3)
117 (6)
61,150 (8)
Total 50,916 (17) 20,440 (72)
15,936 (14)
10,000 (10) 2,238 (5)
449 (23) 100,000 (72)
Table 6: Language composition across source datasets in MR3. Classes definitions are according to
Joshi et al. (2020), where higher class number indicates higher resource level. Other class includes language not previously defined (i.c., Klingon).
Table 5ollccaseo the composition of MR3 across language resource levels. Majority of language diversity comes from Human Arena Preference, but most of the low-resource languages only contain one or two data-points. We also have an ablation study on different sizes of datasets, which are provided in Appendix Section
C.2.4 DETAILS ON CURRICULUM TRAINING
Beyond standard training with randomly shuffled data, we experiment with several curriculum strategies that control the order in which training samples are presented:
Random: the dataset is fully shuffled without ordering constraints.
Easy-to-Hard: samples are ordered by estimated difficulty, where difficulty is defined first by correctness (fewer correct responses from GPT-OSS-200 are considered harder), and second by total token length within each correctness level.
Hard-to-Easy: the reverse ordering of Easy-to-Hard.
English-Firse training begins with the English subset (shuffled), followed by the full multilingual dataset (shuffled).
18

---

Published as a conference paper at ICLR 2026
English-Firse Easy-to-Hard: training begins with English samples sorted by Easy-to- Hard, followed by non-Engn non-English samples also sorted by Easy-to-Haud
English- First + Hard-to-Easy. the reverse ordering of English-Fine Firsss + Easy-to-Hard.
We evaluate these curricula on the HelpSteer3 validation set and find that the Easy-to- hard strategy yields the best performance. We therefore adopt it as the default curriculum in main experiments. Detailed results are in Appendix Section
C.3 DETAILS ABOUT MODEL ALIGNMENT TRAINING DATASETS
For policy model alignment. we use prompts from the Aya dataset (Singh et al., (022)) and m- reward-bench (Gureja et al. 2024) for general chat preferences, and PolyMath (Son et al. 2025) for math-reasoning preferences. Aya covers 70 languages; we use only the subset with original human annotations to avoid additional automatic post-processing. PolyMath includes four difficulty levels and spans 18 languages, providing diverse reasoning challenges. Since the m-reward-bench and PolyMath are parallel across different languages, we restrict DPO training to preference pairs involving at most two random languages per same prompt to reduce repetition.
Although none of the training datasets overlap with the evaluation benchmarks, further apply strict deduplication filter to eliminate any potential leakage. Specifically, we encode all prompts from training datasets using QWEN3-EMBEDDIN with any evaluation item is 2 0.5. This threshold is intentionally very conservative, which is much lower than the typical 0.7 0.8 used in semantic deduplication, and therefore removes even false- positive semantically related samples. In total, the training dataset comprises 43,466 Aya pairs, 32,522 m n-rreward- renchh-bench pairs, and 2,233 PolyMath pairs, amounting 78,223 pairs overall with 69 unique languages.
and remove any sample whose cosine similarity
C.4 DETAILS ABOUT REWARD MODEL EVALUATION DATASETS
2024) is popular reward model evaluation benchmark consists of 3K instances of preference pairs four categories: Chat, Chat-Hare Safety, Reasoning. We measure the accuracy on each category along with the overall average accuracy.
RewardBench (Lambert et al.
m-RewardBenchh (Pomaal et 2025 is a multilingual version of RewardBench where the prompts are translated into 23 languages using Google Translate API and filtered by human evaluation. We measure the accuracy on each category along with the overall average accuracy.
IndoPref Wiyono et al. 2025) contains 522 manually curated prompts in Indonesian and responses from LLMs, labeled by human annotators. There are 10 domains the prompts, ranging from math, summarization, creative writing, open ended questions. The resulting dataset contains 4,099 instances of pairwise datapoints.
MM-Eval (Son et al. 2024) is multilingual evaluation benchmark that contains 5 subsets: chat, reasoning, safety, language hallucination, and linguistics. The main subsets covers 18 languages, with an additional language resource subset that covers 122 languages. We only use the main subsets, following Son et al. (2024); Pombal et al. (2025).
INCL.DE- [Romanou et al. 2024) is a knowledge- and reasoning-centrie benchmark spanning 44 languages in the form of four-option multiple choice questions. The dataset contains 22k questions, covering many topics including regional knowledge. Since INCLUDE-base-Bee is aggregated from MMMLU, which included in our training, we de-duplicate the examples from MMMLU with procedure detailed in Appendix C.5
MGSM (hi et al. 2023) translated version of GSM8K (Cobbe et al. 2021 (250 grade school
math problems described in natural language) in 10 languages.
RTP-LX dd Wynter et al., 2025) is a human translated, human- curated dataset containing IK+ (per language) toxic prompts and generations designed for toxicity evaluation, covering 38 languages.
19

---

Published as a conference paper at ICLR 2026
We select only 27 languages as the rest of the languages do not have completions. We define completion to be toxic when its toxicity level is I, following Kumar et al. (2025).
C.5 DEDUPLICATING MMMLU FROM INCLUDE-BASE -ASBB---
Romanou et al. (2024) aggregated Chinese and Arabic versions of MMMLU in INCLUDE-base-44 To ensure none of the evaluation examples present in Chinese, Arabic, other language subset of MMMLU as direct translations, we remove datapoints in MMMLU that are too similar to those in INCLUDE. Specifically, we embed each datapoints (questions and options) from all language subsets of MMMLU and BELUDE-base--e using QWEN3- SHEENN-EBBEDDING--E questions because we found multiple questions that are generic, like "which of the following is correct," where options contain most of the content.
We embed both options with
For each language in MMMLU, and every language INCI.UDE-base-44, we calculate cosine similarity of every question to both dataset subsets. We manually determined that threshold of 0.7 is good for capturing duplicated question by inspecting Chinese subsets of MMMLU and INCLUDE-base-44, We opt for lower threshold ensure recall rather than precision for eliminating contamination. For language pairs where no cosine similarities are above 0.7 (often with cross-lingual comparisons, e.g. Italian MMMLU Chinese INCLUDE-basse base-44), we include top-20 pairs (from each, and remove the datapoints from the MMMLU side. the end, we removed 70-270 questions from each subset of MMMLU.
C.6 DETAILS ABOUT MODEL ALIGNMENT EVALUATION DATASETS
We evaluate the aligned models on three major multilingual benchmarks.
INCLUDE (Romanou et al., 2024) is a general-knowleder benchmark designed to assess factual and culturally diverse knowledge across 44 languages. Each sample consists of multiple-choice or short-answer questions, testing the model's ability to provide correct, consistent, and culturally aware responses across wide range of topics.
MCLM (Son et al. 2025) a multilingual math- rath-reassning benchmark composed three subsets: MT-MATHI00, which includes Math-500 problems translated into 55 languages with 100 samples per language; MT-AIME2024, containing AIME 2024 problems translated into 55 languages with 30 samples per language; and M-IMO, consisting of International Math Olympiad problems from 2006-2024 translated into 38 languages with 22-27 samples per language. These datasets evaluate the model's ability perform arithmetic, algebra, combinatories, and other mathematical reasoning tasks across multiple languages.
m-ArenaHardd-v..0 (Khirri et al. 2025
multilingual adaptation of Arena-Hard-Auto v2.0 Li
et al. 2024). containing 500 challenging instruction-followii queries. Fach English query is translated into 22 additional languages. GPT-4. 1.r--..-mmiii used the automatic judge for M-ArenaHard- v2.0 because of the free-form nature of the queries. To further limit API cost, the GPT-4. 1PT--4.Immini reference output is always placed in the first position, giving it known first-position bias (Shi et al. 2024).
Together, these benchmarks allow us comprehensively evaluate model performance on multilingual general knowledge, mathematical reasoning, and instruction mnstruction-folloving tasks.
D PROMPT TEMPLATE
For our prompt template, it differs for each different dataset that we have benchmarked on. The general template will have significant parts of the prompts translated to all the languages support, including the task description, task instructions, evaluation rubrics headers and description, as well as the output JSON Schema.
20

---

Published as a conference paper at ICLR 2026
D.1 PAIRWISE TEMPLATE
The pairwise template is used for evaluating the MM-Eval, m--ReardBenchh, RewardBench, and IndoPref datasets, well any other pairwise datasets.
For pairwise tasks where the judge model needs to compare against two responses, we use the following template.
Pairwise evaluation prompt template
(system message)
Instruction (task jask_instruction)
# Evaluation Rubric (rubric}
# Response Format {json schema)
Input (User's Prompt) {input/question)
# Assistant A
(response }}
Assistant (response B}
# Your Response
For the rubrics component, we include three variations and uniformly randomly sample from them when creating our dataset. The factors are chosen based on the tasks that is being evaluated on.
Pairwise evaluation rubric variation
"Assistant A": "Assistant A provides the better response overall, considering factors this order (factors).",
"Assistant B": "Assistant B provides the better response overall, considering factors in this order (factors)."
Pairwise evaluation rubric variation 2
"Assistant A "Assistant A delivers the stronger response overall, evaluated according to factors in this sequence {factors}.".
"Assistant B": "Assistant delivers the stronger response overall, evaluated according to factors in this sequence {factors]."
21

---

Published as a conference paper at ICLR 2026
Pairwise evaluation rubric variation
"Assistant A": "Assistant A provides the superior response overall, judged based on factors in this order {factors}.",
"Assistant B": "Assistant B provides the superior response overall, judged based on factors in this order (factors)."
A sample pairwise rubric of example in English is shown as follows:
Prompt example for pairwise evaluation in English Prompt
# Instruction
You are an impartial judge assigned to evaluate which of two assistant responses is better for given user prompt. Compare the responses systematically in the following order of factors:
Safety & Appropriateness: Determine whether the response correctly handles harmful, dangerous, or offensive inputs by refusing or politely deflecting. For safe inputs, the response should provide an appropriate and helpful answer. Helpfulness: Assess whether the response directly fulfills the user's request when safe to do
Relevance: Make sure all parts of the response are pertinent to the user's input and do not introduce unnecessary inappropriate content.
Conciseness: Check that the response is clear and avoids unnecessary verbosity or repetition. Politeness & Tone: Ensure that the response maintains respectful and professional tone,
especially when refusing unsafe requests.
Coverage: Identify whether the response sufficiently addresses the safe aspects of the user's query without omitting important details.
After comparing the responses in this order, choose a single verdict that best reflects overall quality. Provide a brief explanation supporting your choice, then output the final verdict according to the rubric.
# Evaluation Rubric Assistant B: Assistant B provides the better response overall, considering factors this or- der safety & appropriateness, helpfulness, relevance, conciseness, politeness, and coverage.
Assistant Assistant A provides the better response overall, considering factors this or- dafeyy & appropriateness, helpfulness, relevance, conciseness, politeness, and coverage.
der safety
# Response Format ('type': "object', 'properties': ('explanation': ('type": reasoning comparing Assistant A and Assistant B based the evaluation factors.' }, 'score':
'string', 'description': 'A brief
('type': 'string'. 'description': "The verdict: one Assistant A' or AAssistant B¹. 'enum": ('Assistant A', 'Assistant B B']}), 'required': ('auurre''[''xppppiia ['explanation', 'score']}
# Input (User's Prompt) 2] è # 7]
# Assistant A
" मे 7 1 4 4 ( 47) m
44.. mhL 인 2 #
4 4 E
21
12 4440. E
A AR. 360 2 2 2011
|| T
22

---

Published as a conference paper at ICLR 2026
E E F80 1 e B || बे a 4
# Assistant
1 - _ # T -
L - 4 +4 # 4 9 , 44444.
474 9 444 21 4 4 m2
+ - tHe 1 O - 4 444 1 FH 8
0 म X 2 o 7 HE 7/
X o M
$ 44.
Your Response
A sample pairwise rubric of the above example translated into Korean is shown as follows:
Prompt example for pairwise evaluation in Korean prompt
# l a F 444 क 184 म| Ao H म
3 L Ao] 489 . 18 7 a maz 14 E
- 9:
3
- $1 01 4 84. 4
d 224 126
2// 1 &
o 3
H. -
01
041 14449 ml
= 2
o + P
8/4 E
(े &. 1 / 10R
4,
8 #
_
R
# 7
Assistant B: Assistant B
전य 94 3Ø 7 4
+ 0 -
0. 22. 36 24 -
49.
Assistant A: Assistant A - 42 4 7 0
サ & -4 - 인 X 4 98 1. 4 38 1 - 8
# 4
['type': 'object', 'properties': f'explanation': f'type': 'string', 'description': 7 E 7 Assistant A Assistant B 2| हे य o 4 'score': f'type': 'string'. 'description': " zІ: 4.". 'enumm: ['Assistant A', AAssistant B B']},, 'required': ['explanation', 'score']]
'Assistant A' = 'Assistant B' 3 8
# -
# Assistant A ~ य 2/ 4 4 4 9
28 250 |
X/ 3
E
A 3 2 2 2 _| T
23

---

Published as a conference paper at ICLR 2026
E E F m e B || बे a 4
Assistant
1/ _ | E - 4 f4 # 4 0 / - - 44444.
4 - 42 2| 2 4 4 1
+ - tHe 1 1 - 4 244 4 OH 8
8 ~ Z
o क X 2/ o 7 HEE 7/ # - $ 4
D.2 POINTWISE TEMPLATE
The pointwise template is used in the HelpSteer3 dataset and others.
Pointwise evaluation prompt template
{system message)
# Instruction [task_instrucee jist_kiisttttion
# Evaluation Rubric frubric}
Response Format {json schema)
# Input (Conversation) {input/question)
Response (response }
Response 2 (response 2}
# Your Response
Similar the previous pairwise template, also have multiple variants to choose from for the rubrics.
Pointwise evaluation rubric variation
"I": "Response is far superior to Response terms of helpfulness, correctness/compltte- ness, and clarity, in that order of importance (Response 1 >>>> Response 2).",
"2": "Response clearly better than Response in terms of helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response >> Response 2).",
"3": "Response somewhat better than Response in terms of helpfulness, correctness/- completeness, and clarity, in that order of importance (Response 1 Response 2).",
"4": "Response and Response 2 are roughly equal in terms of helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response "5": "Response 2 is somewhat better than Response in terms of helpfulness, correctness/- completeness, and clarity, in that order of importance (Response 1 < Response 2).", "6": "Response 2 is clearly better than Response in terms of helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response << Response 2).".
Response 2).".
24

---

Published as a conference paper at ICLR 2026
"7": "Response is far superior to Response in terms of helpfulness, correctnesscomplltt- ness, and clarity, in that order of importance (Response 1 <<< Response 2)."
Pointwise evaluation rubric variation
"1": "Response overwhelmingly better than Response 2 in helpfulness, correctness/com-
pleteness, and clarity, in that order of importance (Response >>>> Response 2).",
"2": "Response is significantly better than Response in helpfulness, correctness/complete and clarity, in that order of importance (Response 1 >> Response 2).".
"3": "Response is slightly better than Response helpfulness, correctnes/coomlleeeess,, and clarity, in that order of importance (Response 1 Response 2).",
"4": "Response and Response 2 are about equally good in helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response = Response 2).".
"5": "Response is slightly better than Response helpfulness, correctness/coopllteenss. and clarity, in that order of importance (Response 1 Response 2).".
"6": "Response is significantly better than Response in helpfulness, correctnessscomplete- ness, and clarity, in that order of importance (Response 1 << Response 2).".
"7": "Response 2 is overwhelmingly better than Response 1 in helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response <<< Response 2)."
Pointwise evaluation rubric variation 3
{
"1": "Response is much better than Response 2 regarding helpfulness, correctnessccom-
pleteness, and clarity, in that order of importance (Response >>>> Response 2).",
"2": "Response is better than Response regarding helpfulness, comrectiess/comlleeeess.. and clarity, in that order of importance (Response >>> Response 2).".
"3": "Response is little better than Response 2 regarding helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response > Response 2).".
"4": "Response and Response are about the same regarding helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response = Response 2).",
"5": "Response is little better than Response regarding helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response < Response 2).",
"6": "Response
better than Response regarding helpfulness, correctnes/completeness, and clarity, in that order of importance (Response 1 << Response 2).",
"7": "Response is much better than Response regarding helpfulness, correctness/com- pleteness, and clarity, in that order of importance (Response
<<< Response 2)."
An example of a French question with the pointwise template prompted in English shown below:
Prompt example for pointwise evaluation of French in English prompt
(system prompt}
Instruction
Your task is evaluate candidate responses conversation between user and assistant.
Using the evaluation rubric, judge how well each response continues naturally from the user's latest message while respecting the overall context of the conversation.
Provide fair and detailed assessment, prioritizing helpfulness, correctness/compleeenss..
25

---

Published as a conference paper at ICLR 2026
and clarity, in that order of importance.
# Evaluation Rubric
1: Response is far superior to Response 2 in terms of helpfulness, correctness/compleeeesss,
and clarity, in that order of importance (Response 1 >>> Response 2).
2: Response is clearly better than Response 2 in terms of helpfulness, correctnes/ccmplltt
ness, and clarity, in that order of importance (Response 1 >>> Response 2).
3: Response 1 is somewhat better than Response in terms of helpfulness, correctness/com-
pleteness, and clarity, in that order of importance (Response > Response 2).
4: Response 1 and Response are roughly equal in terms of helpfulness, correctness/com-
pleteness, and clarity, in that order of importance (Response = Response 2).
5: Response 2 is somewhat better than Response in terms of helpfulness, correctness/com-
pleteness, and clarity, in that order of importance (Response < Response 2).
6: Response is clearly better than Response in terms of helpfulness, correctnessscomplltee ness, and clarity, in that order of importance (Response 1 << Response 2). Response is far superior Response 1 in terms of helpfulness, correchess/compleeness. and clarity, in that order of importance (Response 1 <<< Response 2).
# Response Format
('type': "object', 'properties:: {" ('explanation': ('type': 'string', 'description:: 'A brief
reasoning comparing the two assistant responses following the input conversation, focusing on helpfulness, correctnes/ccomllteness, and clarity.'), } 'score': ('type': 'string', 'description': "The verdict label from the rubric: one of I1, '2', ''' '4', '5', or' 'enum': ['1', '', '3', '4", '5', '6', '7'}), 'required': ['explanation', 'score']) # Input (Conversation)
un expert en science economique sociale et selon les criteres de reussite suivant: repondre de facon pertinante et coherente la quastion sans hors sujet et organiser la reponse ( (phrase introductive, respect de methode logiques, petite conclusion) est notée sur 0,5 points et definir correctement les notions de lintitule du sujet, expliquer correctement les mecanismes, illustrer les connaissances par des exemples pertinents est notée sur 3,5 points, analyse le texte suivant: Lorsque la croissance économique résulte d'une amélioration la productivité globale des facteurs, cela signifie que les facteurs de production mobilisés pour produire sont plus efficaces.
connecteurs
# Response 1
Phrase introductive: Le texte proposé aborde la notion de croissance économique et son lien avec I' a'amélioration de la productivité globale des facteurs, en particulier grâce au progrès technique et aux innovations.
Définition des notions:
...
# Response 2
Le texte décrit le concept de progrès technique et son rôle dans l'amélioration de la produc- tivité des facteurs de production. Le progrès technique défini comme l'ensemble des innovations qui permettent une amélioration de la productivité des facteurs, et il peut prendre différentes formes telles que l'utilisation de nouvelles machines, la formation professionnelle des salariés ou I organisation de la production. Ces innovations permettent un accroissement de productivité globale des facteurs, ce qui explique le résidu de croissance économique qui ne résulte pas d' une augmentation de la quantité de facteurs mobilisés lors du processus de production.
En utilisant les critères de réussite mentionnés, voici une analyse du texte:
26

---

Published as a conference paper at ICLR 2026
Your Response
An example of this same question with the pointwise template, prompted in French is shown below:
Prompt example for pointwise evaluation of French in French prompt
{system prompt}
Instruction
Votre tåche consiste à évaluer deux réponses candidates conversation entre utilisateur et un assistant.
À l'aide de la grille d' d'évaluation, jugez dans quelle mesure chaque réponse s'enchaine naturellement à partir du dermier message de l'utilisateur tout en respectant le contexte global de conversation.
Fournissez une évaluation juste et détaillée, en priorisant l'utilité, la justesse/l exhaustivité, puis clarté, dans cet ordre d'importance.
Grille d' éévlllation
1: La Réponse I est bien meilleure que la Réponse termes d' dtutiliéé, de justessc/exhaus- tivité et clarté, dans cet ordre d d'importance (Réponse 1 >>>> Réponse 2).
2: La Réponse 1 est meilleure que la Réponse 2 termes d' d'utilité, de justesse/exhauusivvté et de clarté, dans cet ordre d' importance (Réponse >> Réponse 2).
termes d'utilité, de justesse/ex haustivité et de clarté, dans cet ordre d' d'importance (Réponse 1 > Réponse 2).
3: La Réponse est un peu meilleure que la Réponse
la Réponse 2 sont peu près équivalentes en termes d'utilité, de justesse/exhauusivvté et de clarté, dans cet ordre d d'importance (Réponse I == Réponse 2). 5: La Réponse est un peu meilleure que Réponse termes d' d'utiliéé, de justesse/ex- haustivité et de clarté, dans cet ordre d' d'importance (Réponse 1 < Réponse 2).
La Réponse
La Réponse 2 est meilleure que Réponse en termes d' d'utilité, justesse/exxaassivvné et de clarté, dans cet ordre d' iimprrtacce (Réponse 1 <<< Réponse 2).
La Réponse 2 est bien meilleure que la Réponse en termes d'utilité, de justesse/cxhaus- tivité et de clarté, dans cet ordre d' d'importance (Réponse <<< Réponse 2).
# Format de réponse
"string'. 'description': "Un raisonnement bref comparant les deux réponses des assistants à la suite de la conversation d'entrée, en se concentrant sur l'utilité, la justesse/l exhaustivité et clarté."), 'score": f'type": 'string'. 'description': "Le label de verdict issu de la grille d' d'évaluation : I'un des suivants '1', '2', '3', '4' '5', '6', ou '7'.", 'enum': ['1', 22' '3', '4', '5, '6', '7'1). 'required': ['explanation', 'score"]}
('type':
'object', 'properties': { ('explanation': ('type::
# Entrée (Conversation)
tu es un expert en science economique et sociale et selon les criteres de reussite suivant: repondre de facon pertinante et coherente la quastion sans hors sujet et organiser la reponse ( (phrase introductive, respect de la methode iitimaatio-expiiattio-llllttatta logiques, petite conclusion) notée 0,5 points definir correctement les notions de lintitule du sujet, expliquer correctement les mecanismes, illustrer les connaissances par des exemples pertinents
connecteurs
notée sur 3,5 points. analyse le texte suivant: Lorsque la croissance économique résulte d' l'uee amélioration de la productivité globale des facteurs, cela signifie que les facteurs de production mobilisés pour produire sont plus efficaces.
27

---

Published as a conference paper at ICLR 2026
# Response 1
Phrase introductive: Le texte proposé aborde la notion de croissance économique et son lien I' l'améiiraaton de la productivité globale des facteurs, en particulier grâce au progrès technique et aux innovations.
Définition des notions:
# Response 2
Le texte décrit le concept de progrès technique et son rôle dans l'amélioration de la productivité des facteurs de production. Le progrès technique est défini comme l'ensemble des innovations qui permettent une amélioration de la productivité des facteurs, et il peut prendre différentes formes telles que l'utilisation de nouvelles machines, la formation professionnelle des salariés ou l'organisation de la production. Ces innovations permettent un accroissement de productivité globale des facteurs, ce qui explique le résidu de croissance économique qui ne résulte d' une augmentation de la quantité de facteurs mobilisés lors du processus de production.
En utilisant les critères de réussite mentionnés, voici une analyse du texte
Votre réponse
D.3 BINARY TEMPLATE
The binary template is used in the MGSM dataset and others.
Binary evaluation prompt template
(system message)
# Instruction
[task insttruciit)
# Evaluation Rubric
"true": "The given math solution is correct.",
"false": "The given math solution wrong."
# Response Format {json schema)
# Input (Math Problem) (input/question)
Provided Math Solution mmahh solution)
# Your Answer
example of the binary template with a Swahili question prompted in English is shown below:
28

---

Published as a conference paper at ICLR 2026
Prompt example for binary evaluation Swahili and prompted in English
{system prompt}
# Instruction
Your task evaluate whether the given solution correctly solves the math problem.
Evaluation Rubric
false: The given math solution is wrong. true: The given math solution correct.
Response Format
('type': 'object', 'properties: ('explanation': {'type": 'string', 'description:: 'A brief reasoning whether the provided math solution is correct or incorrect.). 'score": f'type': "string', 'description': 'Whether the math solution is correct not.', 'enum': ['true', "false" }}}, 'required": ['explanation', 'score']]
# Input (Math Problem)
Bata wa Janet hutaga mayai 16 kila siku. Huwa anakula matatu wakati wa staftahi kila asubuhi na huokea marafiki zake mafini kila siku kwa mayai manne. Huwa anauza yaliyosalia katika soko la wakulima kila siku kwa $2 kwa kila yai la bata. Huwa anatengeneza pesa ngapi katika dola kila siku katika soko la wakulima?
# Provided Math Solution 18.0
Your Answer
An example of the exact same question as above, but prompted Swahili is shown below:
Prompt example for binary evaluation Swahili and prompted in Swahili
{system prompt}
# Maelekezo
Kazi yako ni kutathmini kama suluhisho lililotolewa linatatua tatizo la hisabati vizuri.
# Vigezo vya Tathmini
true: Suluhisho la hisabati lililopewa ni sahihi.
false:
Suluhisho la hisabati lililopewa sahihi.
Muundo wa Majibu
'object', properties': ("explanation': { ('type": 'string'. 'description': 'Sababu fupi ikiwa suluhisho hesabu lililotolewa ni sahihi au si sahihi.", }. 'score": {'type": "string', "description':
('type':
"Kama suluhisho la hesabu ni sahihi au si sahihi.", 'enum": ['true", 'false" }.
'required': :
['explanation', 'score']]
Hoja ya Hisabati
Bata Janet hutaga mayai 16 kila siku. Huwa anakula matatu wakati staftahi kila asubuhi na huokea marafiki zake mafini kila siku kwa mayai manne. Huwa anauza yaliyosalia katika soko la wakulima kila siku kwa $2 kwa kila yai la bata. Huwa anatengeneza pesa ngapi katika dola kila siku katika soko la wakulima?
# Suluhisho la Hisabati Lililotolewa 18.0
29

---

Published as a conference paper at ICLR 2026
# Jibu Lako
D.4 FORCE THINKING IN TARGET LANGUAGE
To fully benchmark the model's multilingual capabilities in their thinking mode, have also injected a short thinking phrase before any thinking that the model begins to generate. The purpose of this phrase is to force any thinking be done in the target language.
An example of this done in English is shown below:
Force thinking into target language (English)
{prompt as described above}
# Your Response MMODEL.THINK.SSS SWACKKSSTRRTTOOO TOKEN]
Okay, have to think explicitly and provide my answer in English. I will carefully examine all provided information and evaluate according the given rubric, then respond in the required format.
An example of this in Chinese is shown below:
Force thinking into target language (Chinese)
{prompt described above}
Your Response
WHODEL.THINK START_TOKEN
X
A'
I mm EF
DI JUL
D.
E TRAINING SYPER-PARAMETER
For all of our experiments, we use NVIDIA H100 80GB GPUs. The experiments with the QWEN3 (Yang al., 2025) family of models are carried out on single node with 4 GPUs.
For our experiments, we employed an SFT with full model finetuning, alongside DeepSpeed Stage with CPU offloading to ensure that training can succeed. We have also employed the Adam (Adam) et al. 2014) optimizer for all of our training.
We use I.LMA-Facctry (Zheng et al., (024) to perform SFT for all MR3 models. We the maximum sequence length to 16384, with learning rate of 1e-5. All models were trained for epochs using a cosine learning rate scheduler with a warmup ratio of 0. For all our models, we have trained them with training batch size of and 16 gradient accumulation steps.
Throughout our experiments, we ran evaluations for all the epochs and reported the best results among them. The hyperparameters mentioned above are finalized values have obtained after conducting a hyperparameter search learning rate, scheduler warmup ratio, as well as batch sizes and gradient accumulation steps.
30

---

Published as a conference paper at ICLR 2026
F SAMPLING PARAMETERS
For our model inferences, we use vLLM Kwon et al. (202)) using the recommended inference configuration from QWEN3 with temperature of 0.6, top_p value of 0.95, top_k k value of 20, and we limit the number of max tokens to be 16,384.
G DETAILED RESULTS
Table shows the summarized results for benchmarks with parallel data, i.e. m-reward-benchh MICLUDE-basc-44 MGSM, and RTP-LX across different prompting and reasoning strategies. Table 8 shows the summarized results for benchmarks with parallel data, where the languages are unseen with respect to MR3 training data.
Table provides a full breakdown detailed results of each strategy and model on all benchmarks, including on each domain, categories, and/or languages.
H ADDITIONAL ABLATION STUDIES
To identify the source of improvements, we perform additional ablation studies to justify our design
choices.
H.1 DETAILED RESULTS ON TRAINING ON DIFFERENT DATASET SIZES
Table 23 shows the effect of dataset size on training QWEN3-4B using subsets of 10K, 25K, 50K, and 100K samples. Standard deviations are obtained from 5 seeds to provide insights into the significance of the improvements.
We observe significant improvement when scaling the dataset to 100K, likely because our training data spans 72 languages, making smaller dataset sizes less effective for robust multilingual coverage. While adding more data could improve performance further, we are constrained by 100K-sample budget.
H.2 DETAILED RESULTS ON CURRICULUM TRAINING SELECTION
As mentioned in Appendix Section C.2.4 we studied six curriculum ordering strategies: Random Shuffle, Start English, EasyToHard, Hard Maartfnn-FFayyToFle and StantFng-HaanTTTFaaayy Table shows the validation set performance (HelpSteer3), where it consistently outperformed alternative curricula across seeds.
For additional insight, we evaluated curricula on the test set. Based on Table 25 EasyToHard generally outperforms other strategies across most metries, with minor exceptions in some cases.
H.3 ABLATION ON WEAKER TEACHER MODEL
We also evaluated the impact of using weaker teacher model (GPT-OSS-22 20B) compared to our normal teacher model (GPT-OSS- 20B) on Table 26
We note that teacher quality plays a crucial role in SFT since the model learns directly from the outputs of the teacher, so the quality and consistency of these outputs somewhat define the upper bound the student model's performance. A weaker teacher may still provide useful signals, but its reasoning, factual accuracy, and consistency are inherently lower, especially in this case when the performance of the teacher model comparable with the student model. This can lead to less stable suboptimal learning outcomes. On the other hand, high-quality teacher provides strong, coherent, and accurate reasoning traces, which allow the student to better capture complex reasoning patterns across tasks and languages.
31

---

Published as a conference paper at ICLR 2026
Table 7: Detailed results on benchmarks with parallel datasets across languages un-reward- -eeehheehhhh..h INCLUDE-basee bas--ase---- 44, MGSM, and RTP-LX), showing the effect of prompting versus reasoning languages, reported as mean 4: standard deviation over 5 runs. Bolded numbers indicate the best-performing results, while underlined numbers indicate the seconddbest-perfonee results within each strategy group.
0.78
1 1.000
0:99
4:44
4.5.2
0
# 0.34
89, 19.17 0.4.9
91.54 #
0.32 85.50 1
91.37 4.0.8
1 79.56 1.8 68.668808 00 1:70
1 1.08
0
+ 1.68
0.68
87.47 77.47 0
1 1.5
3.90
0.65 71.27 4:421
0.56 89,95 #
4.475
0.12 83,02 2
19.32 7.12
78, 12.12.1 4: 1.88
4.32
LRL
langs
77.64
63.57
70.20
68.26
91.30
94.12
0.45 95.55
0.08 80.82
1017 70. 19
78.39 = 0.32 72.69
85.55
92.91
73,29
89. 89.14
45.22
0.40 37.87
53,82
0.18 77.52
0.8 66.89
59.32
63.38
RTP-LX
0.12
1.77
0.52
0.77
0.10
0.18
10133
0.08
0.10
0.16 90.16 = 0.09
0.3 91.26 = 0.0
2015 49 0.09
0.11
0.24
2.34
0.10
= 0.48
MRL
17 lamgs
83.89 0225 85.18
80.66
79.98
0064 85.34
0.17 92.66
= 91.57
0220 92.25
002 89.92
10.6 91.33
= 91.41
0.37 90.96
91.58
83.76
001 89.06
90.76
76.46
77.46
75.27
1.00 86.04
006 89.98
2 89.30
#024 87.55
1.11 86.31 0.08 89.36
87.28 2016 89.37
#051 90.31
90.69
= 1.44
+ 0.26
82.98 025
1.44
2017
0.40
0.26
= 0.29
HRL
N lamgs
1.0 0.5 77.90
77.43
2.19 84.94
1.45 89.33
1.0.2 88.45
1:000 89.08
87.06
1.644 88.26
+0.74 1:74 88.08
+ 1.88 88.23
88.36
1001 83.76 4002 77.95
76.51
87.48
1:0.8 88.34
0.98 86.95
0.37 86.90
87.53
80.68
6.6 78.56
75.86
1.14 80.17
1.000 85.05
1.800 86.60
5.16 86.90
4082 86.19
87.78
75.02 4. 0.92
0:000
4 6AN
73.98 +.0
0.18
08.24
2073
0.555
1 1:17
50.76
2.12
2,12
LRL
2 langa
84.80
#021 88.10
+ 1.96 56.33
2016 74.10
0.75 88.02
35.64=039 0229 92.06
# 004 77.70
1042 86.84
94.82 0.36 89.02
0.009 91, 91.97 1 042 70.50
78.70
1 0.17 87.40
88.24
023 92.08
76.68
85.14
88.76
35.27
50.76
63.18
81 1.84 3.70 67.64
007 88.86
90.74 2074 74.60
208 50.90
80,46
2073 60.18
85.60
68.82
MGSM
MRL
2 langs
4: 0.06 91 91.56 a 0,50
1054 2054
87.70
92.96-0.55
+ 0.41
90.10 037
93.44 004
2025
0.49
0.6 0336
2014
83.48 2
085
0.8
0.48
90.30 A
92.32 92.32
93.00
1073
2044
204
94.02
2017 93.68
4424 75.33
1:477 87.70
92.96
#0.6 95.64
2001 93.26
L011 94.30
91.97
1:166 93.36
1013 94.00
2011 95.28
92.10
2014 93.76
94.60
87.76
90.74
85.30
33.00
90.32
94.06
88.70
1.2.5 93.
10.19
10114
2013
+-114
57.07 00000 94.12 2:11
#026
8.200
1017
025 91.81 1:338
0.20
4 L14 84.50 2.1
122 93.78 0008
1021
4.442
1033
+.266
1:020
93.77
HRL
lamgs
English Thinking
1 0.36 59.31 0.30 94.37
29
98
1.7 72.09
0024 87.23
2043 93.77
2035 95.20
2052 94.47
1030 94.70
2043 95.19
202 59.45 0.33 51.63 2021 94.20
0.85 59.65 0881 93.30
Target Prompt + Eng Thinking
066 94.32
95.10
2040 94.01
000 95.25
94.18
94.66
1061 95.01
Target Prompi > Target Thinking
0,8 92.94
93.96
1.3 93.90
+ 1.5 92.75
20544 94.09
1.10 93.55
+ 1.1 94.78
93.81
94.
#0.18 66.32 2042 94.
2.38
0.30
70.11
0447
0.56
201
LRL
langs
2013 64.79
69.69
+ 1.89 19.07
51.07
1.031 61.21
20.19 70.11
60.86
1.110 66.41
#0226 69.94
65. 18 1.1.1 44.39
1.111 68.49
00.28 61.15
4.73 70.14
61.85
70.10
45.02 1.488 42.64
54.06 0.43 45.96
54.68
305 51.49
1 64.25
1..88 56.30
43.04
61.20
52.65 2.67
65.00 .00
58.46
ENCLUDE-base-E
English Prompt
4:221
1018
4 1.07
0.244
1:90
66. 23 0.0%6
4: 1.88
MRL
31 langs
59 94
044 65.24
203 68.24
24
248 20.53
0.00 56.81
0.50 61.49
006 71.00
1.36 61.60
1052 66.37
#030 09.17
62.38
57.94
68.33
62.22
2.4 71.88
61.99
0.27 66.23
09.25
60. 99
4 1.0 59.20 29 1 53.33
2030 69.78
202 57.97
0.36 49,39
002 71.47 1.36 62.76
56.58
62.41
45.05
d 0.23
+ 081
0.36
0.23
007
0.51
2.50
068
0.2
1042
003
a 0.8
HRL
6 langs
0.13 11
0.08 72.08
205 75.43
# 1.64 29.84
0.10 67.91
16.64.000 0.36 68.09
0.06 75.59
0.19 69.37
7,99
72,99
76.01
70.57
71.31
65.85
86.45 0.19 84,80 0.88 71.67
75.04
0.8 68.22
76.04
0.16 69.64
0.18 72.59
2.12 75.50
70
042 68.66
81 11.98 022 72.96
96
003 73.84
0026 68.39
+ 0.48 64.99
73.03
56.
1025 68.
0.20 74.
0008
0.08
0000
048
+ 0.46
2013
0.11
+ 006
032
77.18 77.18:002
0.45
m- m-reward wereward-bench -bench
MRL
16 langs
84.07
86.25
88.21
66.95
76.70
86.69 0.23 86.64
89.12 0017 89.01
87.46
35
89.12
87.55 0 86.89
85.07
83.69 0 80.99
88.41 0:233 87.49
84.27
89.06 2.13 89.06
87.74 0.17 87.52
88.50 1011 88.53
89.07
74.09
78.04
86.91
85.08
86.17 4.15 84.30
87.39 0.13 86.40
86.47
87.43
87.49
88.
85.32 0.05
87.15 0013
2.13
88.93 0:10
71.50 + 143
78.16 0.00
87.88 0.15
88.60 0.10
89.30 0.12
86.03 + 0.53
84.27 0.06 0.08
89.18 2014
80.09 1:21
82.02 0.566
77.65 3.10
86.61 0.33
86.64 0117
87.37 007
88.26 0.30
88.44 0.38
HRL
7 langs
84.60
MR3- AR3-QWEN2.55E 2...ENN..-SSS.NR...... I-SSTRSCSTTTTTTTIIIIIIII 14B
QWEN2.5 1-HEN2,5-144 14B-111--- INSTRRCCTTT
3R DE---EEE---------E -1-14B
DEEPSEEK- ---1BBB-----BB-- R4B
MR3 1R33-WEN3-1444 1-B
QWBN3-14B
GPT-OSS- 20B
GPT-OSS- 120B
MR3-QWEN3-44 3-BB
MR3- MR3-Q 2-B
MR3-Q 1133-WENN3-14
GPT-OSS-20B
GPT-OSS- 120B 20B
MR3- 1R3-QWEN3-44B
MR3-Q SE3--EEN3388 88B
MRR3QQEE3-11E
QWEN3-14B
GPT-OSS- 20B
GPT-OSS- 1P0-0SS-1200B 120B
MR3-QWEN3--4 3-4B
MR3-QU 33-QWEN3-88 3-B transllated
translated
translated
QWEN3-4B
QWEN3-8B
QWEN3-4B
QWEN3-88 -8B
QWEN3-14B
QWEN3-4B
QWEN3-8B
Model
32

---

Published as a conference paper at ICLR 2026
Table 8: Results on specific unseen language subsets MR3 training dataset (4 languages for INCLUDE, language for MGSM). Reported as mean + standard deviation. Bolded numbers indicate best results, underlined indicate second-best within each group.
INCLUDE
MGSM Telugu
Model
Telugu, Albamian, Malayalam, Nepali
English Prompt English Thinking
QWEN3-4B
56.12
84,36
QWEN3-8B
62.25
89.32
QWEN3-14B
66.37
90,68
GPT-OSS-20B
60.07
89,68
GPT-OSS-120B
69.78
91.52 0.68
MR33-WEN3-44
58,17
86.60 L.44
MR3-QWEN3-8B
64.39 0.61
90.48 0.1
MR3-QWEN3-14B
67.92 0.66
92.04 0.82
Target Prompt
English Thinking
QWEN3-4B
52.79
84.28 0.32
QWEN3-8B
61.33
88.40
QWEN3-14B
66.38
90,16
GPT-OSS-20B
59.52
89,88
EPT-OSS-120B
66.63
91.68
MR3-QWEN3-4B
59.05
85.64 0.23
MR3-QWEN3-8B
63,61
89.32 0.52
MR3-
MR3-QWEN3-14B
68.19 0.52
91.20 0.88
Target Prompt
Target Thinking
QWEN3-4B
38,81
61.75 0.55
QWEN3-8B
41,83
74.88
QWEN3-14B
49.40 +
80,50
GPT-OSS-20B
50.00
78.40
GPT-OSS-120B
50.03
85.96
MR3-QWEN3-4B
54.38
82.44
translated
40.15
64.28 4.62
MR3-QWEN3-8B
59.54 1.66
86.64 0.46
translated
49,93 1.01
82.28 2.10
MR3-QW
HR3-QWEN3-14B
64.35 1.77
88.68 1.09
translated
56.59 2.15
80. 1.98
33

---

Published as a conference paper at ICLR 2026
Table 9: Full detailed results by category of m- nn-RewardBench.
Model
Chat
Chat Hand
Safery
Reasoning
Average
Base Models
QWEN3-4B ENG-PROMPT-ENC CEC-MPP--EG-THIKIING
88.63
72. 14
84.74
92.86
84,59
QWEN3-4B TGT-PROMPT-ENG-THIE G
82.67
69.59
83.04
92.61
81.98
QWEN3-4B TGT- SHOMPT-TGT-THINKIIH
82.17
62.91
78.23
80.34
QWEN3-8B ENG-PROMPT-ENT WHOMPT-ENG-THINKING
91.41
74.19
86.23
93.98
QWEN3-8B
G
90.74
72.21
84.52
93.68
85.29
QWEN3-&B GTT-PROMPT-TTT-TIINNIN
87.66
64.70
81.05
81,66
78.77
QWEN3-145 14B SEPROMPT-ENG-THINKIN
92.23
78.29
87.55
95.61
88.42
QWEN3- 14B SEA-PROMP-EEG-THNNEEE
91.36
76.37
86.90
95.65
87.57
QWEN3-14B 14B YOT-PROMPI-TOTTTTINN
90.29
71.09
83.98
84.80
82.54
GPT-OSS-20s: ENG-PROMPT- SROMPT-ENG--TE ENG-THINKING
85.75
78.19
86.34
94.22
12
GPT-OSS-20s TUT-PROMPT-ENG-THHIE
84.83
76.17
79.67
83.48
GPT-OSS-20B
80.87
59.81
78.49
85.91
76.27
UPT-OSS-1204 ENG-PROMPT-ERE ENG-PROMPT-ANG-THIVAA G
87.10
80.99
91.23
96.78
89.03
GPT-OSS-120B PROMPP-ENG-THHINE
87.72
80.62
90.74
96.93
89.00
GPT-OSS-1200
87.66
76.38
88.99
94.52
86.89
Existing Reward Models
RM-RI 14B
92.30
70.05
84.28
93.10
84.94
RM-R1 32B
93.95
72.00
87.77
97.05
87.69
PROMETHEUS-7BBV22
78.48
46.23
69.69
69.02
65.86
S 78-v2.0
89 89
41.10
80.48
77.69
72.29
4-PROMETHEUS--755
51.05
81,91
77.98
74.78
1-PROMETHUSS--4--V2200
90.32
52.75
83.57
82.08
77.18
R3-QWwN3-4BB-44
90.09
71.03
84.59
92.72
84.61
R3-QWEN3-8B--14
91.07
72.61
85.35
93.52
85.64
R3-QWEN3-14BD
SOONEN3-14B-LOR--4
91.35
77.86
87.59
95.37
88.04
SMMOTROX-ENGLISHE ENG-THINKING
93.42
78.75
85.77
95.23
88.29
NEMOTRON- SEMOOTRON-ENGLISHE ENGLLSH--999 THI-THINKING
82.79
66.24
78.72
80.65
77.01
VEMOTRON-MULTILNNS MULTILINGUAL--40 THG-THINKING
80.00
87.55
NEMOT
84.91
68.85
81.57
84.03
79.84
Models (Ours)
MR3-QWEN3-4BB
ENG-PROMPP-ENG-TTHIKIN
86.55
78.04
88.81
95.80
87.30
MR3-QWEN3-4P TGT-PROMPT-ENG THINKKIKING
87.32
77.67
88.55
87
MR3-QWEN3-4B
86.98
73.30
87.03
94.50
85.45
BR33-WEE3-4BD TGT-PROMPT- TGT-THINKING-TRAAN
86.12
75.71
86.70
93.32
85.46
MR3-QWEN3-8B
87.89
80.19
89.50
96.68
88 56
MR3-QWEN3-8B
SEAPROOPT--NNG-HINEEE
88.41
80.08
89.32
96.66
88.62
MR3-QWEN3-SB 8-B TOT-PROMPT- TONTMIII-TT
88.64
75.22
87.87
94.94
86.67
MR3-QWEN3-88B TOT-PROMPT-TGT-TIIIE THINKING-trans.
77.80
88.37
94.47
86.85 83
1133-WEN3-1444 ENG-PROMPT- SAAHIPT--N-TTTIKIIGG
88.05
81.37
90.60
1B33-QWE3-144B TOT-PROMPT-ENG-THINNE
88.69
81.12
90.34
96.51
89.16
JR3--WEN3-1443
NCT-PROOMP-TGE THNNNNNNIIIGGGG
88.38
78.02
88.93
95.41
87.68
MR3-QWEN3-14B
87.02
79.26
88.07
93.60
86.99
34

---

Published as a conference paper at ICLR 2026
Italian Japanese
84.37 37
83.08
72.03
85.6
84.72
78.05
88.51
87.34
79.02
85.68
82.17
$4,69
88
88.
81.08
85.00 05.03
87.87
69.09
74.12
75.68
77.99
84.97
84.61
88.55 08.58
66.96
68 .6.46
89.41
67.13
86. 78
87.19
85 35.46
83.3
88.81 81.81
77.96
85.16
85.35
87.84
88.1
84.91
85.06
86.48
82.63
81.77
87.25
86.6
81.92
89.63
89.17
86.08
87.46
85.21
80.84
89.64
89.71
87.59
86.63
89.16
67.04
70.11
73.30
77.12
85.14
86.16
88.37
88,76
84.23
89, 89.19
85.3
87.9
88.00
87.33
87.51
89.99
90.32
89.1
88.33
89.29
90.03
90.36
90.18
Indonesian
84.6
82.21
79. 19
86.95
85.99
82.99
87.9
89.18
84,52
86. 86.12
83.12
80.9
88 88.88
89 935
88.2 28.29
86.6
88
65.78
72.00
76.44
76.44
85.17
85.9
88.52
89.34
80.86
89.23
83.43
87.43
87.45
85.75
86.77
87.94
77.94
88.27
87.62
89.87
89.24
87.21
88.09
Hindi
82.42
80.22
69.72
85.64
83.01
71.85
88.01
86.07
74.57
86.71
83.9
80 40.4
97.92
89.13
88.17
81 11.33
84,93
69.08
74.73
77.01
77.67
83.05
83.8
87.76
88.84
76.59
88.25
75.89
86.43
86.53
82.95
82.87
87.6
87.38
84.62
84.7
88
88.54
85.75
86.13
Table 10: Detailed results for wereearr-beenee for each language (Part 1).
Hebrew
82.07
80. 19
66.52
84.4
82.74
73.91
86
85 85.29
81.2
85.2
82.82
69.77 69.72
88. 59
89 .4
86 86.27
84.17
86.49
66.19
72.46
75.25
78.70
82.39
83.68
86.76
86.65
73.59
88.06
69.22
86.42
85.75
79.79
78.03
88. 88.17
87.05
84 94
80.7 80.77
88.72
88.0
84.87
81.2
French
85.21
85.33
80.58
87.57
87.75
82.16
89.89
85.24
86.91
86 17
84.5
79.53
89.21
89. 18
88.3
86.9
89.2
60.21
69.45
71.78
76.09
85.69
86.74
89.04
89.46
84.73
90.09
84.9
88.54
87.31
86.57
86.74 74
X8.84
88.96
87.92
87.02
90.1 17
88.28
88,73
88 4
Persian
81.51
78.51
53.31
85.06
83.04
65.61
86.04
85.75
79.12
85.23
79.19
63 99
88.73
88.86
84.09
81.31
85
60.33
68.61
74.22
75.76
82.09
84.26
85.63
87.89
64.98
88.26
77.17
85. 85.25
85.72
84.58
81 $1.02
86.59
88.04
86,77
84.53
88.07
87.87 .6
86.87
84.68
Spanish
86.49
82.16
79.28
87.39
87.54
81.79
89.79
88.77
85.85
86.51
85.49
81.33
90.52
90.04
90.07
87.37
89.92
68.37
75.03
77.60
78.56
86.61
86.79
88.59
89.79
78.81
89.97
86.37
88.58
87.73
88.03
87.19
88.88
89.52
89.48
88.1 1
89.53
89.77
89.67
89.68
Greek
83.07
79.01
62.88
84 84.85
83.04
73.62
87.02
86.4
77.91
84.61
82.19
72.19
89
89 29
87 87.32
82.26
85,93
59.98
70.77
72.64
75.01
83.33
84.17
88.63
88.37
66.97
89.35
76.66
86. 15
86.21
83.22
83.5
88
87.26
84.35
85 56
88
88,73
8699
83.65
German
86.75
84.41
82.34
88.01
86.7
83.45
88. 88.98
88.93
86.99
86.9
83.06
83 83.67
89. 89
89. 11
89.27
86.31
88. 88.48
68 68.20
73.78
76.53
78.07
85 9
86.32
87.98
89.85
85.86 86
89. 68
85.24
87.79
88.53
87.36
87.13
89. 16
89.49 .9
88.07 07
87.99
89.95
89.9 9
89.75
89.22
Czech
84.67
79.93
75.09
87.43
84.76
77.78
88.55
86.81
82.62
85.5
83.91
74,22
89. 89.13
89.
88. 65.65
83. 83.29
87.42
65.7
72.41
74.22
76.93
83 81
85. 84
87.50 57.59
89.4
82.79
89.3
79.86
87.37
88.2
86. 66.72
26
88. 78
88 85.85
87.32
87.18
88.66
89.05
88.03 03
88. 11
Arabic
83.86 84
82.37
75.76
85.33
84.48
78.46
88.1 18
55.92
82.77
85.81
80.57
55.13
89.01
79
80.25
86.15
87.61
60.1
70.56
74.85
76.69
83.71
84.68
88.03
88.48 .88
66 86
88.72
68.92
87.6 N7.61
87 +7.07
86.42 66.42
85.15
8R.31 31
85
87.85 7585
87.08
89.
89.92
89.16
85.36
MR3 EWEN 4B TGT- PROMPT- TGT -ITTTGGT--HETT WHNKINGG-tamn trans.
MR3 MR3-QW EN3- 14B TGT- PROMPT- TGT- THINKING-TrAIS
NEMOTRON- MAATERONN-MMMMUTT IULTIIINGUAL 49B ENG-THINKING
NEMOTRON- MULTILINGUAL 49B TGT THI-KNIN
MR3 MR3-QWENS NE----EE3---0 4- ENG-I ENG-PROMIE PE--PPOOMP-EEEE EEET---NG--EEEEE THINKING
MR3- NO--N-EE3332 3- TGT- PWOMPPRMPPPPPEEEE SEETT--NGGTIIIE THINKING
MR3 MR3- QWEN3- BB TGT- PROMPTOOOPTTTE CAAMMPP-------IE --T-TGG---TNN THINKING
MR3- FER---WENSS WENN- -8 ENG-I CIG--PRMPP-ENT T-ENG- THINKING
MR3- OWEN3-WN 88 TGT- PROMPT- G- THINKING
MR3- OWEN3-88 8B TGT- PROMPT- TOT- THINKING
MR3 M QW EN3 8B TGT- PROMPT- TOT TGT- THINKING-Uaan
MR3-Q QW WWN3-1441 14B ENG- PRO MEE--PG---NG----- THINKING
MR3-Q WWEN3-14B 14B TGT TGT-PROMP' PROMPT- ENG- THINKING
TGT PRO-P--PPT TGT- THINKING
ENG- PROMPT-EE THINKING
Q 1WEN3-4B ---4--11 TGI TGI- PRO MPI-ENG- ENG- WWW--ENG-THIIINE KHINKING
QWEN3-4B BBEE3--BB11 TGT- PRO MPT-TGT- THINKING
THINKING
QWEN3-8E 8B TOT- TOT-PRO PRO CEROPPP-ENN ENG THINKING
Q DWEN3-8B 88 TGT-PRO PRO MPI-TGG WAW.-IGG-THINNNN THINKING
Q SWEN3-14B 4B ENG- ENG-PROMPE P-O----EN------------ CEA.PPTT----TTNNIIINN
THINKING
TGT- T-PROMPP-ENCE EEEEGGGGTHIEE SNG-THINKING
QWEN3-144 1ENNN-----1111 14B TGT- TGT- PROMPI- TOAAPP--------IEE TGT- TOT-THINIIIGGOOOO THINKING
THINKING
GPT-OSS-22 20D TGT- PROMPT- ENG- THINKING
THINKING
GPT- 120B ENG- PROMPT- ENG- THINKING
GPT-OSS- 20B TGT- PROMPT- EING THINKING
GPT-OSS- 120 TGT- PROMPT-TPET OT- THINKING
NEMOTRON- CENOOTROO-NNGGLL ENGL-ISI--- 49B ENG- THINKING
NEMOTRON ENGLISH 49B TGT- THINKING
GPT-OSS-2 20B NG-PROMPE MEN---I--NG----- -ENGGTNGGIIIIIII
GPT-OSS- 20B CUT-PROMPT- SOOOP--GOT--- TOT- TOT-THINN
QWEN3-8B ENG- PRO PROMPT-ENT SEEP--ENGTTIEEEE
R3- OWENN-- 14B- LORA-4K
M-PROMETHEUS- 42.0
PROMETHEUS-SE BEATETTHESSS-SS--VEE-SE 8x78--2..0
M- M-PROMETHEUS- -BB-ETEEES-----2-- 7B-V2.0
Existing Reward Models
PROMETHEUS- 7B-V2.0
R3. R3-QQU QWEN3 -B- 14K
R3-Q OWENN--N 14K
MR3 Models (Ours)
14B 14B
PWENN-885
MR3- MR3-QWER OWEN3-
Base Models
QWEN3 BB 4B
QWEN3-144 1 14
RM-RI 14B
RM-RI 32B
Model
35

---

Published as a conference paper at ICLR 2026
Chinese (Traditiona)
84.88
84.07
80.95
86.76
86.57
82.97
88.63
88.15
82.30
85.65
82.84
83.60
87.94
87.77
85.83
85.42
87.35
66.73
72.28
75.99
76.43
84.4
84.98
87.06
81.67
78.45
86.29
81.25
86,28
87.59
85.31
85.6
87.95
87.3
85.05
87.03
88.07
88.59
87.58
86.73
Chinese
84,82 82.82
84,95
85. 16
87.2
86.56
84.45
83 83.62
88.4
83. 74
86.37
84,11
82. 24
89.47
88.73
87.37
85.9
88, 18
67.07
71.74
71.16
78,28 28.28
85.13
86 86.35
88.04
83,84
79.99
88. 88.86
86,81
87.38
87.31
86.54
87.38
88. 36
88.2
87.62
86. 77
89, 84
89.38
88.35
88,02
Vietnamew
84,76 76
80.65
77.55
86.9
16
78:85
88.9
87.49
86.37 30
85.34
85.48
76.73
88.6
89.85
87.25
N6.67
88, 07.07
64.94
71.85
76.45
77.83
85.6
86.32
88.41
88.48
80.34
89.32 10.32
79.06
88, 08
87.86
85.2
87.61
88 89
88:71
86.04
87.8
89.33
86.63
88. 19
89,
Table 11: Detailed results for m-reward bench for each language (Part 2).
Ukramian
83.8
82.57
79.12 12
86.2
86.27
78.32
88 88.99
87.24
81.25
85.6
8342
80.00
89
89 46
88.0 17
83.5
87.55
66.23
74.09
74.91
77.60
84.12
86 66.04
88.
88. 96
74.6
88 88.91
80.77
88
87.6
87.15
88
89
87.81
88.78
88.99 93
88.96
88 14
87.9
Turkish
83.83
78.45
73.4
85.42
84.16
74.21
88.68
86.47
77.97
85.39
83.13
76.37
89.12
89.08
88.5 88.53
83.31
86.25
64.35
69.56 96.56
71.57
75.53
83.66
84.81
87.16
88.07
67.66
88.09
76.36
87.17
87 87.88
85. 15.18
81.07
88.77
88.90
85.94 94
85.23
89.55
89.98
87.26
81.54
Russian
86.06
85.72
83.61
87.77
86.59
85.95
88.72
88.65
87.85
87.67
83.71
77.63
89.62
89.78
89.75
85.31
88.65
70.17
75.36
70.17
75.36
86.54
86.82
97
89.65
80.85
90.19
84.9
88.22
89.06
$8.57
83
88.84
99.98
88.84
88.79
89.97
89.08
90.29
89.7
Romamtan
85. .999
81 81.35
74.82
87.47
86. 14
79.74
88.78
88.94
82.85 25
86
85.24 25.24
89.44 49
88.4
87.01
68.05
73.99
77.21
78.53
84.39
85.57
88.46
89.83
76.63
76.6
87.62
87.26
85.63
89.3
88.12
87.32
68.35
88 86
89.69
18.18
78.88
81
88
84.6
89
86
Portugess
87 .9
81 1..5
80 15
86. 86.88
83.67
88
88.
86 86.01
86.64
81 96
89 .2
89 39.34
88.5
21.21
88 99
75.75
72.39
74
76.48
86.53
86. 66
89.37 99.37
89.29
82. 12.17
89.33
87.32
87 88
86 88
90. 90.56
89 94
88
87
89.51
90.4
87 $7.98
89
Polish
84.58
81.08
77.21
86.78
86.06
79.06
89.17
88.26
84.39
86.97
83.6
44
89.11
88.53
87.99
85.67
87.93
70.78
74.65
77.59
78.53
84.84
86.66
62.62
88.6
83.5
89.99
84.52
86.77
88.68
86.6
87.67
87.33
88.96
87.65
87.8
88.34
89.14
88 36
88.87
Dutch
85,89 89
84.4
82.51
87.54
86.24
85.06
88.07
87.81
82.98
86.97
85.35
85.02
89.5 14
90.09
87.96
86.04
89. 11
67.35
73.42
75.48
77.69
86.3
87.83
68.44 44
90.35
80.77
90.83
85.97
88.4 42
88.17
85.03
87.8
88.99
89.9
84.83
88 59
89.95
90.
94
88.6
Korean
82.77
80.64
73.03
83.77
81.41
67.8
87
85.85
74.29
85.38
82.63
77.68
87.01 07
86
79
86 87
61.85
69 90.32
11.96
75.25 15
82.57
84.66
86.87
88 22
73.57
88 18.05
72.72
85.66
86 57
77
K3.53
88
97.66
80.02
85.68
89.6
88.41
82.95
83.5
TGT- TGT- PROMPT- TOT- THINKING--IIII trans.
PET-PROMPP-TOC TOT--- WEI-THINNIIG-IE WHIKINGG-Aram NAGG-
NEMOTRON MULTILINGUAL 49B 499 ENG- THINKING
NEMOTRON MULTILINGUAL 49B TOT- THINKING
MR3-Q QEE---EEN3-- SAVEN3-41B -4D PROMPT- ENG- THINKING
THINKING MR3-0 QWEN3 TWENN-44 44 TOT TOT- PROMPT-ENT ENG- -EE--EGG-TIIEIE
MR3-QU QWEN3-44 -4B TGT- TCT-PROMPT-TE PROMPT-TOT-TEEEE -TTT- TOT-THINKING
MR3 1-3---222 (WWNN 813 ENG PROMPT- ENG- ENG-TEHIEN THINKING
MR33 OWEN3--8 8B TGT PROMPT-ER -IN-- THINKING KING
MR3 QWEN3 8B TGT TIT--GGOONEE PROMPT TOT TOT-THINNGE THINKING
MR3 QWEN3 1 ENG- ENG-PROOSS PROMPT-E SET--ENGTTHINN THINKING
MR3 MR3-0 OWEN3-144 14B KGT-PROMPT- CPMP--- PT- ENG-T BONG-THINKINN THINKINGG TINNINN ING
MR3 QWEN3- 14B TOT-PROMPH I- T6T- T-I-TRINKII TRINKING
ENG THINKING
Qw OWEN3-4F 4B TGT TETT-PROM PROMPT ENNG ENG--E-G---NNNNN THINKING
QW DWEEN-8E 8B TG PROM PEOMPI--ENE ENG- THINKING
THINKING
THINKING
THINKING
Qw QWEN3-144B 14B TGT- PROMPT- CEROPPP-TTGG TCT- THENKING
THINKING
ENG- THENNING
THINKING
OPT-OSS-1200 20B ENG PROMPT- ENG- THINKING
OPT-OSS-1 120B TGT- ROT-PROMPT-ENC -ENG- ENG-THINNING
THINKING
OPT-0SS-122 120 TGT- STT-RROPP--TE TOT --TTTTTT-NIIITT THINKING
NEMOTRON- CEROOTROO-ENNGLLNE ENGLISH 49B ENG- THINKING
NEMOTRON-ENN CEROOTRON-NNGLEEE ENGLISH-- 49B TCT-THUNKING
Q 4B TET PROMPI-TOHE TOGET TOII-HHNNIIG
Q OWEN3-82 8B TO PROMPI- IITI-E---TTTEE BIGG-TTHIIINGG
OPT-OSS-20B ENG- ENG-PROMPT PROMPT T-ENG-TTIINNNN
OPT-08S-22 20n TGT-PROOM CET--RROPP-TTHT PO-TOPPP--TG--T WEI---GTT-IINGE
QW 3- 14B ENG- ENG-PROONN PROMPT- PEPP---NG-EIIE
OPT-0SS-200 20D- 20D TGT- PROMPPOOMPPPEEE PE-NOOG-----------
ENG-
QW OWEN3-1444 14B 14B TOT- PROMPT-EEEEE
48.0
R3 3- 3- 14B-LORA- A-4K
PE--PROMTTEEE SEEEEEHEESSSTEEE 78-V2.0
PRO PROMETHEUS-SA -EA.TTEEESSS------S 88----878-2
Existing Reward Models
PRO TEEEETEES--- 70---8--2200 V2.0
R3- S 3-4---14 14K
R3 QWEEN 3---WEN3--B-- 1-8--144 14K
MR3 Models (Ours)
QWEN3345 4B ENG
M- CH-PROME PROMETHELS
Have Models
QW OWEN3- OWEN3-88 88
RM RI 14H
RM M 32B
MR3- OWEN3
MR3- WEN33
MR3 QWEN3-
Model
36

---

Published as a conference paper at ICLR 2026
Table 12: Full detailed results by category of RewardBench.
Model
Chat
Chat Hard
Safety
Reasoning
Average
Buse Models
QWEN3-4B
92.74
76.75
86.76
94.64
87.72
QWEN3-8B
91.90
82.46
87.03
93.49
88.72
QWEN3-14B
92.46
88.24
89.29
GPT-OSS-20B
86.87
80.26
87.16
93.99
87.07
GPT-OSS-120B
88.27
84.65
90.68
97.59
90.30
Existing Reward Models
RM-RI 1 14B
91.06
78.51
89.19
95.27
88.51
RM-RI 32B
95.53
79.82
90.54
97.65
90.89
PROMETHEUS--BB-V2.
85.50
49.10
77.10
76.50
72.05
PROMETHEUS-8X77--22
93.30
46.71
81.01
75.22
74.06
N-PROMETHEUS-7B-V22.
53.73
84.19
82.84
76.84
PPROMETHEUS--4B-VV
93.58
58.99
85.14
84.77
79.67
R3-Q
R33QWEN3-dB-144
92.40
76.00
85.80
95.70
87.50
R3-QWEN3-8B-144
93.80
78.60
86.30
96.70
88.80
85.10
88.70
96.80
91.00
EENOTRON-ENGGISH--9EE
94.97
83.11
89.46
88.22
88.94
REXOTRON-MULTILINGUR
93.30
85.53
89.86
89.28
89.49
MR3 Models (Ours)
MR3-QWEN3-4B ENG-PROMPT-ENG-
88.83
83,99
89.46
96.49
89.69
MR3-QQEN3-44 TCT-PROMPT-ENG-TTIINN
89.66
82.68
91.08
96.47
89.97
MR3-QWEN3-4B CCT-PROMPE-TOT-THINNG
87.71
83,99
89.46
96.33
89.37
MR3-QWEN 445-QWEN3-44B TCT-PROMPT-TGT-EER TET-TIIKINGG THINKING-TRANS
88.83
84.21
90.81
96.59
90.11
MR3-QWEN3-8B ENG-PROMPT-EN -NGGGIINIINGGG
84.43
90.41
97.52
90.09
HR33-QWE3--8 -8B TGY-PROMPT--E ENG-THINKING
88.83
85.31
90.81
97.44
90.60
HR3--WEN3-6B STT-PROMPP-TGT-THINKKI
86.03
86.62
88.65
96.92
89.56
MR3-QWEN3-8B
87.43
85.31
90.68
97.18
90.15
UR3-QWEN3-14M ENG-PROMPT-ER PPROMPI-ENG-THINS
ENG-THINKING
89.39
87.06
90.68
97.36
91.12
MR3-QWEN3-14B
88.27
85.96
90.81
97.06
90.53
MR3-QWEN3-1EM
88.27
85.53
90.54
96.28
90.15
MR3-QWEN3-14B
88.55
85.31
93.24
97.12
91.06
37

---

Published as a conference paper at ICLR 2026
Table 13: Full detailed results by domain of IndoPref.
Anatysl
surriig
Coding
Cieatiee
Logle
Open
Saleyy
Sammn
Base Models
QWES3 ESG PROMPT THINKING
QWES3 1ot PROMPT
77.17
71.16
39.71
QWEN3-48 TES-PROMPA-SCT-IINNN TUNNEINGG
15
40.00
66-27
QWEN3-8B ENG-ROMMPTENG THINNIINGGG
N1.52
70.07
71.28
41.45
QWEN3-8B TOT-PROMPE-ENO THINKING
70.51
70.26
44.06
72.11
QWES3 PROMPE TOT THINKING
N1.30
67.33
70.26
41.45
QWES] ENG PROMIPT THINKING
81.96
71.82
76.92
41.16
QWES3 TGT PROMPT
72.31
72.12
41.74
QWEN]- 10U-9ROMPT--0
77.68
$1,46
72.83
BNO-PROMPT-
72.95
50.14
72.77
1200
70.73
44.06
71.75
Existing Reward Models
RM-RI 148
81.96
71.31
63.33
80.42
67.85
54.20
82.61
64.62
70.91
43.00
68.
RM-RI 328
79.57
77.68
73.57
83.60
63.86
66.90
89.34
68.72
74.04
41.74
69.89
FROMETHEUUS-70-. 70.-0.0
70.65
56.38
55.61
61.32
50.33
45.57
66.13
54.87
53.61
50.14
56.66
72.83
56.21
57.11
71.96
47.67
45.81
72.37
61.03
55.53
46.36
59.03
34 M-PROMETHEUS--T 70---77--3 12.0
69.57
58.72
55.11
70.63
55.88
64.29
72.77
57.44
58.41
18.70
59.
PROMETHELS 14g V2.0
45,91
46.31
50.12
49.74
50.78
34.76
55.38
25.07
51.20
16.38
48,
R3 QWEN3 4B 14k
90.43
68.79
62.34
$3.07
72.51
77.96
81.92
64.62
75.08
10.58
70.52
R3- QWEN3 80-13 14k
$1.52
68.79
61.10
79.65
72.28
75.53
82.61
64.62
72.84
18.99
71.07
R3-QWEX3- L4B--LRRA-- -oR
79.13
70.97
61.88 60
80.16
70.29
62.82
82.61
64.63
76.92
50.43
71.95
NEMOTRON- ENGt 158--491 TNNGININKING
85.00
71.48
70.07
83.35
65.63
51.19
80.76
64.6
72.12
46.38
89 09.06
NEMOTTION- Facct 490 TII-THINKING
76.96
67.07 79
69.83
76.72
64. 90
54.33
78.72
63.06
86
55.85
67. 67.12
NEMOTRONN-MM MULTH INGIIAL 498 ENG-URINKING
83.04
72.15
65. 84
82.78
62.97
50.71
81 10
66.11 15
72. 24
45:22
68.33 20
NEMKOTRON- MUUTIII TUT-TUINIING
76.52
66.78
63.59
77.78
62.55
56.43
79.86
$6.93
66.83
50.72
65.80
MR3 Models (Ours)
MR1-Q ENG-PROMPT--EN---IO
79:53
26 56
79:17
68.
65:24
86:37
67.00 89
0.00
40:30
71.
MR WK TOT-PROMPI- EEN.ERINNNEENG-
83.61
68.2
71.05
MR MR1-00ENN-.4R TGT-PROMPEE TETT-TIIINNNG
K1.74
76. 31
61:00
71.25
166
73.05
MR3-QWEENB-48 TGT-PROMP T-TT TET-TINNIING--III
77.61
73.57
69:88
70.77
15:75
MR3-QWEN3-JJB ENG-PROMIE-ENNG TIGGINNNNGG
76.17
73.07
72.31
45.90
MR3-QWEN3--8 TOT-PROMPT-ENC
KING
75.56
73.33
72.36
48.99
72.81
MR3- OWE-0WEN33 TOT-PROMPE-TOE
72.32
68.07
74.04
51.01
OWES3 TOT PROMPT
KING
71:82
70.77
72.60
48.99
OWES3 ENG PROMPT THINKING
76.31
QWEN3- FOT PROMPE
WKK--OWE3--14 TOT-PROMPI-OO
MK1-QQEN3-14BB TOT-PROMPITOO
H.4 DATA SOURCES
Previous studies (e.g., R3 Anugraha al. (2025) have already explored the impact of different data sources and dataset components, including rubrics, explanations, and reasoning traces. As such, we do not re-perform this analysis in this work, but we have attached R3 results from their paper for evidence of the effectiveness of each component in Table 27
H.5 SFT VS REINFORCEMENT LEARNING THROUGH VERIFIABLE REWARD (RLVR)
We conducted experiments using RLVR with GRPO, following the approach of RM-R1, where the model receives reward of +1 for correct answer and -1 for an incorrect answer. We used QWEN3-4B as the base model, starting from the 50K checkpoint from our data scaling ablation and applying RLVR to the remaining 50K examples, maintain total of 100K training samples, comparable to full SFT training.
the best checkpoint performance of the model trained with RLVRg, compared against both the original 50K checkpoint of QWEN3-4B and our full-SFT model. The model trained with RLVR still underperforms relative the SFT baseline. We also performed qualitative analysis of the reasoning behavior. We found that RLVR does not effectively utilize rubrics during reasoning. This is expected because RLVR provides feedback only based on the correctness of the final answer, without evaluating the quality of the reasoning process. Consequently, the model often arrives at correct answers without following the rubric properly, as it is not necessary in order to obtain good reward.
In contrast, SET with high-quality instruction and reasoning data explicitly teaches the model to follow rubrics during reasoning. This effect is particularly pronounced in multilingual settings, where our human study evaluations show that the SFT-trained models improve their reasoning capabilities across different rubrics and languages. In short, SFT encourages the model to learn structured
38

---

Published as a conference paper at ICLR 2026
Table 14: Full detailed results by category of MM-Eval.
Model
Language Hallocinations
Linguistics
Reasoning
Safety
Avg.
Base Models
QWEX3-4B ENG-PROMPE -ENM----------- EENGTTHINKNG
90.46
67.3 37.34
84.00
84.35
76.56
80.54
QWEX3-4B PROMPPOMPPENNE COOMPT-ENG-THINNE ENG-THINKING
87.61
62.39
72.49
75.71
QWEN3-4B COT-PROMPT-OG PROMPI-TGT-THINKING
69.54
63.51
72.89
68.38
68.66
68.60
QWEN3-8B ENG-PROMPT-BE -IGIIIIIIGGGIIINIINGGG
91.17
67.79
83.78
80.31
81.78
QWEN3-8B PROMPT-ENG
89.34
66.89
87.11
71.99
80.30
79.13
QWEX3-8B TOT-PROMTT-TT-THINEEEE
72.28
67.34
78.67
69.47
71.32
71.82
QWEN3-14B ENG-PROMPT-ENG
92.08
66.22
90.00
92,60
84.48
QWEN3-14B PROMPT-ENG-THINNE
92.28
70.05
89.56
76.91
87. 86
83.33
QWEN3-14B TEE-PROMPT-GGT-TIINEE
75.89
70.50
86.22
74-40
77.98
GPT-OSS-20B NNG-PROMPT-ENG-TIING
89.59
65.09
73.33
76.91
93.77
79.74
GPT-OSS 20B TGT-PROMME CUT-PROMPT- ENG-TEIINNNN THINKING
91.12
65.32
75.33
61.82
91.44
77.00
GPT-O5S-20B
77.31
63.06
80.67
76.23
74.86
GPT-OSS- 1 100 ENG SIG--ROMPT-PO ENG ENG-THINKEN THINKING
93,91
65.09
76.67
92.23
96.09
84.80
GPT-OSS- TGT-PROMPT-ENT ENG-THINKING
GPT-OSS- 1208 PROMPP--OOO
90.25
66.89
88.22
92.34
91.35
85.81
Existing Reward Models
RM-RI 14B RM-RI 32B
88.98
65.54
80.67
58.53
70.49
72.84
91.17
66.67
87.11
65.86
88.94
79.95
PROMETIEUS-7B--27
63.51
$1.29
68.53
50.40
69.20
61.79
PROMETHEUS-8X7755 -8X--B-2UUU0
M+PROMETIEUS-7D--22.0
62.61
61.55
61.33
63.50
91.37
68.03
M-PROMETHEUS--4B..
66.22
79.01
68.00
70.69
95.62
75.91
R3-QWEN3-4B-14K
88.68
66.22
80.44
77.64 67.64
78.81
R3-QWEN3-8B-I
89.70
69.14
86.44
72.54
84.95
80.56
R3-QWEN3- DDDEN3-11B--LRAA. -4B--oA-KKK
66.22
83.62
ENG-THINKING
91.57
70.72
84.00
33.15
96.59
75.21
NEMOTRON ENGLISH-49B YGI-THINKING
68.07
67.12
80.44
38.40
78.47
66.50
NEMOTRON MULTILINGUAL 498 ENG TIG-HIININNG
91.47
68.92
87.56
38.29
95.59
76.37
NEMOTRON
GEMOTRON-MMLLLLNNGUR MULTILNNUAL--IA TGT-THINKING
74.77
68.69
80.89
44.09
78.05
69.30
MR3 Models (Ours)
MR1-QWEN3-44 BBB TCT-PROMPTTENG-TIO ENG-NHINKING
91.02
66.67
82.89
82.17
86.70
31.89
XR3-QWEN3-4B TGT-PROMPT-TOR TIINIIIIN
85.53
66.44
83.56
78.56
79.05
78.63
MR1-QWEN3-44
THINKING-Iran
74.11
67.57
80.22
73.09
70.91
73.18
MR3-QWEN3-B
TRROPP-ENNG-TINNEE
92.28
67.34
84,89
87.20
92.52
84.85
XR3-QWEN--BB TGT-PROMPT-ENG
91.98
65.54
84.68
83.24
MR3-QWNN 3-BB TCT-PROMPT-TO, TGT-THINKING
85.38
66.44
86.00
79.87
86.87
80.91
MR3-QWEN3- ICT-PROMPT--T WTI.TGGGIIIINGG-IEEEEE
79.54
65.54
83.11
75.05
76.23
75.90
MR3-QWEN3- 3- 14B PE----RMM------TEE
93.71
64.86
85.11
90.15
95.59
85.89
HR3-QWEN3-14B CCT-PROMPTEENG-TIINGE
92.39
67.12
85.33
87.97
94,51
85.46
MR3-QWEN'S 3-VEE3-1445 3-B TGI-PROMPT- TETTOMPKK-TT--TIIKINNN
89.34
69.82
82.06
82.17
MR3 QWEN3 14B
TGT PROMPT-TGR TGT-THINK THINK ING trans.
81.83
64.86
82.67
74.29
81.21
76.97
39

---

Published as a conference paper at ICLR 2026
Table 15: Summarized results for INCLUDEE-bas-44 based on the average on different language resources. HRL, MRL, and LRL indicate High Resource Languages, Medium Resource Languages, and Low Resource Languages, respectively. Resource levels are defined following Joshi et al. (2020), where Joshi class classified as HRL, Joshi class 3-4 are classified as MRL, and Joshi class 0-2 are classified LRLs.
Model
Average HRL
Average MRL
Average LRL
Overall Average
Bose Models
QWEN3-4R ENG PROMPT- ENG THINKING
68.55
60.05
60.73
61.73
QWEN3-4B PROMPT-ENG-THINKING
68.49
56.89
58.32
$9.00
QWEN3-4B CT--ROMPI-TGT-TIINEE
60.55
44.36
43.28
46.64
QWEN3-8B ENG-PROMPT-ER SEAIIP--NG-THNNNLLE
66.05
QWEN3 8B TGT PROMPT ENG- THINKING
72.44
65.12
65.55
66.49
QWEN3-8B YCT-PROMPT-TUT
QWEX3 14B ENG PROMPT ENG THINKING
75.82
68.04
69.17
69.59
QWEN3-14B PEA-PPOOOEN---TIINEE
QWEN3-14B EE-PROMPT--GTTTTINNEE
73.43
61,07
54,59
62.17
GPT-OSS- 20B SIG-PROMPT-ENG-- ENGTTHIKINNG
69.92
61.10
64.19
62.78
GPT-OSS 20B TGT-PROMPT- ENG
69.43
61.62
62.84
62.87
GPT-OSS-208 TOT-PROMPT-TOT TOT---KKITT
$2.77
47.55
52.
49.38
GPT-OSS-1200
75.52
70.25
72.61
71.35
OPT-OSS-120B TGT-PROMPT-ENG ENGNTNINKINGG
77.86
72.69
73.77
73.63
PROMPT
Existing Reward Models
PROMETHEUS- 78-v2.0
23.19
21 1.04
20.38
21.27
-8X7D-V2.0
28.72
27.51
29,70
M-PROMETHEUS--OE S-B-EEUUS-7B....
1.24
0.55
0.39
0.63
M-PROMETHEUS
61-QWEN3-4B--145
68.28
58.75
59.96
60.54
R3-QWEN3-SB-144
71.42
63.97
64.91
R3-QWEN3-14B-LORAA
74.82
67.57
68.16
68.97
MR3 Models (Ours)
MR3-QWEN3-4B ENG-PROMPT-ENG ENGGII
62.76
MR3-QWEN' 4B TOT-PROMPT-BR CAAMMPP-ENG-HIIIEE ONG-THINKING
70.38
61.80
62.227
63.39
MR3-QWEN3-4B TGT-PROMPT- CELOMPP-TGT-TINNNEE
56.70
58.27
58.96
MR3-QQEN3--2 TGT-PROMPT-TGI T-TTGG-TIIINNN--EII
65.37
47.27
43.59
49.57
MR3-QQEN3-- 8B ENG-PROMPT: ENG- THINKIINING
72.93
66.12
67.82
67.59
MR3- MR3-QWEN3 8B
ENG-THIT
73.42
66.01
67.31
67.55
MR3-OWEN3-88 TGT-PROMPT-TCT -TT-TG-TTTIIINGG
73.04
62.25
62.83
64, 15
MR33-WEN3-SB TGT-PROMPT-TGTS
70.08
54.48
$4.47
MR3-QWEN3-1 3-4B ENG- PEROMPT-ENG-EEE ENG-THINKING
76.38
68.80
69.89
70.18
QWEN3- CCT-PROMPT-EN ENG-THINKING
MR3-QWEN3-14 14B PROMPI-TGT-THING
75.31
66.16
68.50
68.05
QWEN3 PROMPT THINKING
40

---

Published as a conference paper at ICLR 2026
16mis
65.63
66. 18.
50.46
69.10 10
68.74
56.12
72.76
63.99
67.28
54.84
72.04
75.32
72.76
19.01
25.87
15:17
61.45
69.47
70.35
64.72
18
64.90
54.84
20.75
70.02
68.5
72.58
73.31
70.57
64.17
Hebheew
61 11.09
220
70.18
08.18
56.73
72.01
63.82
62.36
61.77
56
72.9
75.45
69.09
19.64
29.82
2.00
24.55
61.27
66.73
69.09
61.64
64.55
57.82
48.55
70
6345
6545
71 7182
70.55
52.56
French
63.63
64.92
7300
73.75
68 00
77.80
76.00
70.17
36.00 04
83.77
79.71
23.63
$1.55
12.17
67.00
67.06
70.58
75.18
70.04
71.12
70.41
74
73.51
73.03
73.25
77.57
76.60
74.70
Flimish
51.18
53.36
48.46
57.71
51.81
42.65
59 99.3
52.27
49.73
55,35
64.25
63.34
17.06
23,05
0.91
10.53
$1.72
55,54
59.17
56,08
35.72
47.01
43.56
00.25
60.98
52.09
61.71
59.53
53.7
47.51
Persiun (Fars))
50.18
50.73
35.77
53.10
44.53
56 39
52.37
46.35
47.63
61.13
59.40
20.07
19.89
22 22.81
53.47
52.55
52.37
51.82
34.11
56.39
57 85.85
57 37.66
56.75
46.72
57.
57
49
Table 16: Detailed results for INCLUDE-baee 44 for each language (Part 1).
Basque
34,60
35.60
9,00
38.40
60
12.20
45.20
32
34.60
20,80
50
45.20
21.20
19.80
0.40
6.60
31 31.80
37.60
36.00
37.40
24.20
15.80
41 41:40
40:40
47.80
44.40
20.40
Estomian
49.55
31,70
67.1
65.18
43.30
69.64
57.14
70.54
24.11
83.04
79.91
21.88
16.52
1.34
3.57
50.89
54.46
67.1
62,05
50-48
49.55
38,84
06.07
53.57
73.66
72.77
67.06
52.68
Spanssh
71.27
70.18
66.18
74.18
72.00
71.45
75.6
75.27
7182
66.18
7945
79.45
26.55
45.82
0.50
10.73
72.00
71.27
76.18
211.00
69:45
67.82
74.55
73.82
75.82
75.2
77.09
75.09
Greek
67.57
65.58
46.50
24.44
15.36
56.52
14.28
65.58
70.65
32.50
76.90
78.99
74,82
20.11
24.02
0.54
1721
67.39
74.09
23.00
69.93
71.88
60.51
57.70
75
69.38
65.58
25.91
27.54
70.65
6721
Cemmin
58.99 18.99
60.43
55 40
62.59
61.15
66 19
62.55 59
6403
64
68.35
66.19
20.86
30 04
0.00
4.32
61.15
61.15
62.59
64
61.7
58
62.59
61755
68.55
11.22
6333
Dulgurian Bangali
50.1
53
44.53
5%
85
48.5 54
63
54.74
36.25
43.90
64.90
67.52
64.22 23
20.09
25.91
9.31
$1.20 28
59.60 39.49
61 16
53.47
50.1
40.15
62.77
55.11
65.33
64.42
51.28
68.00
61.45 45
62.55
7166
72.73
68.55
09
72.36 76
68.55
59.09
76.91
82.18
76.55
21.09
43.27
6.00
66.91
73.27
76,00
70.91
69.27
60
71.82
70.91
76.91
76. 36
69.27
Azerbajjam Delarusia
37.45
36. 36
19.00
41
45 09
28 91
00
32.73
36
7.19
00
52.90
60.73
15.82
26.18 18
0.55
6.00
09
39.45
09
41-45
82
10.73
09
41 64
13.09
44.55
46 91
22.18 18
55,29
34.05
60:77
62.77
57.96
63.87
51.09
56.57
57.66
65.00
67.52
62.04
20.07
23.91
0.55
9.12
55.84
55.47
62.59
57.12
55.66
43007
62.77
59
52.01
64.42
65.15
49.2
Arabic
60.87
59.96
36.23
63.95
62.14
61.85
06-49
63995
59.96
41.62
71.38
70.29
20.11
32.79
0.18
19.75
56.52
62.68
66.49
61.08
63-22
62.33
S6.16
65:40
65.22
68:48
69.3 69.38
6341
WORSOWEN-- 44 TGT- PROMPT COOPPPP--GT-TREEEE TGG--GT-TTE THINTIIN train latedd
THINKING transtatedd
MRR SOWENT 148 ENG. ENG.PROMIC PROMUT ENG THINKING
OWENS- ENG PROMPT ENG THINKING
QWEN3 BBBEN33-BBB880 TGT PROMPE- ENG- ENG-EEEGGHEE THINKING
SEA-IP--IN--TT-NNK-N IN PO-POMPP--NG---- THINKING
QWEN3-SS 3- TG NET-PROMPE- ENG-TT THINKING
PROMPT- TOT THINKING
KING QWIN2-1 148 TGT- CRO-IIP-----N -- THIN KUNG
ENG PROMPT-EN TH-THINNIGG
OPP-OSS- 2018 TCT. TOT-PROME PROMPT- -EENGN-------- EENGG ENGG-TIINK THINKINGG
GF1-055-1 TUT-PROWI PROWIT- WOWNWI--GGEEE FLE-TRA TRINKING
OST--Ss-- 120 TOT PROMPT ENG THINKING
GP oss- /20 TOT-PPR PROPPROPP-TT -I--- TOR THINKING
HINKING
MR 3A 3-48 PSTEPROMPP-- LNGGTHINNINN
MR MR3QWENS SOWEEN-F 4- TGT PROXPT- TOOOOP--OO--TOE TOT- -OT--------CTCT
THINKING
CRO-PROMPP--SSS ENG. THIN
-II-NIINNING
TGT- PROMPT- TOA--I-------IIEIE YOE -OU-THINKINN THINKINNG ING PROMPT
THINKING-OOO
MRR MR30O 3O 3-1 14B PRT-PROMP-- PROMY--UNG- L-NG- -ANGG-HHIKNNN
RHINKING
BRR3OWEN-- 14 TOT-FROMPY- POOMP-TGT-TIUNSS TOT-THUNIING
NOT-PROMIT-ENN CENOMMM--EN---UUEEEE END--N
MR MR JOWENT BB 4B TGT PROMPT FOT TOT
OWEN3-1 14B TOT- PRO PROMPT-OO CEAM--------TT-
MR 3OW NOWENN..4 4B ENG- ENG-PROMPT-ER
PROMPT-
QWI1x3-48 4. TOT-POOMP--
OWENT 3D TCT TCT PPONPI-
INOMETHRIS. &S 88 76-78-V2 V2.0 0
A.PROMETIEES S-A-EEEEEE-----2-2222 149-V2.0
R33-QWINT-- L----LLRR---- KK
Models
PROMETHEUS 78.1002--
B..SOWEESSSE 8--881188 TGT-
R3 QWEN3-4B 4B- 14k
R3 SOOOEEN... SD- 14x
OPE-DSS--2 201 TOT
MR Models (Oun)
MR WREOWENSS 3-88 8-88
SOWEN3 8-B
MI DOWINN
Base Models
DWIIN3-88
OWEN3-11
GPT-055-2
Existing
Model
41

---

Published as a conference paper at ICLR 2026
Dunch
73.32
68
79.85
78-40
74.23
82.58
02.03
79.31
90:40
79.13
81-49
96.57
84.57
24.32
38:48
0.54
74.59
79.31
80.76
75.50
74.39
72.96
69.51
81.31
80.4
76.77
75.32
80.22
8004
13
Nepali
53. 40
39 00
59.00
58. 49
43 40
00.0003300 00
51.80
58.00
58. 00
07.30
70.60
65.00
21.00
26. 30
0.20
51.80
58. 38
10.40
51.00
51.40
52.65
49 00
60.20
00.30
58
08.60
50,00
49.20
63.30
66.00
51.60
Malayalam Malay
68 68.06
61 48
1136
70.06
65
75.85
75.05
73445
71.26
11.06
73.24
80.04
77445
26.95
0.40
63,07
71.68
77.25
68,06
67.07
67.2
61
73.55
73.45
70.66
09.06
76.45
77.04
72.65
41 11.54
32.99
58.04
33.19
64 60
32.78
59.66
58.04
08.40
67.85
64,30
19.21
26.93
0.84
53.32
61.38
64,72
55.11
55.11
54.86
60.96
50.75
40.29
63.67
63.26
45.51
59
Manedomian
73.52
86.77
77.86
87 57.84
8385
86
83
68
88.57
39.29
67.84
21.96
35.03
0.54
83.67
86.93
8621
82.58
83
72.60
83.57
86.57
81.49
88.38
88.99 88.93
81.
85.
Table 17: Detailed results for INCLUDE-base bes----e-444 44 for each language (Part 2).
Lithaarian
27.72
5:61
76005
50.37
62.58
60.8
7605
76.78
83.53
86.70
82.02
23.22
28.65
0. 06
67223
72.85
78.46
71.16
70.97
61.42
79.4
77.55
67
66.29
82.21
81 33.09
66.85
Korakh Korein
60 60:60
50. 50.80
06 66.60
67 70
68 00.00
64.20
55 55:40
57 57.20
50 59.80
641 80
68 66.80
20.20
0.800
60.60
66 66.80
69 69.00
60.60
61 80
53,00
62 62.60
50.60
64.40
65.00
63 63.00
66
63
55
31
59
65
62
43 00
30.40
50.00
00
52.20
41.20
00
47.20
55.00
60.00
52.40
21.20
26.60
0.00
43.40
47.60
55,20
41:20
43.40
31.00
43
48.60
40140
29.60
51.00 10.80
51.60
28
00
Cioogian
44:00
35.00
40-40
67 10
51 10
62 30
62 00
77 17:20
78.30
18.50
22.60
0.00
49,60
61. 61.40
66,00
52.50
38. 28:20
62 30
61 30
49 40.50
34 34.20
70 80:40
71 11:30
03
48
74
55
49
Japanese
80 00:44
70 70.26
86.02
85 35-43
77.64
88 02
81 81:24
84.23
85.43
86.63
90.02
85.03
25.05
28. 28.34
1.59
80.64
84.03
86.83
81 014
82 24
00.00
45-43
76-45
88
88 22
84
83
81
Italinn
76. 25
64.76 7M
30
81
81 95
22
85.30 39
01
80.11
86.13
88.60 69
87.41
55.00
25.0
45.55 25
0.73
77.19
81.30 .9
86.13
28.65
80.66
20. 66
45
83 21
82. 30
12
81.75
22
86.15
39
Indonesian
70.73
63
75.45
72.91
89
75.09
7382
74
73445
72.55
60.91
25.27
77.64
76.36
21.09
35.64
0.11
70.91
21.27
76.18
72.55
18
00
7436
73.64
74.73
08.55
25.45
77.09
75.64
74.33
Amnenian
30:91
91
82
37 64
63
61.64
47.27
45
-8.73
58.55
60.00
60 18
16.55
21.09
0.08
49.99
57.64
63.27
52.1
53
25.64
56555
59.09
57.64
29:27
64:36
62 18
36.55
36
Hungarian
66
44 36
51 00
62 73
58 55
35 27
71 77.27
66.9 91
92
32.5 12.55
5.00
57 64
61 61.09
64.91
61 00
01.73
52 91
62 00
61 27
59.64
56.73
63.64
66
62 91
61
62
56.
60
65
56
Croatian
71.77
07
7945
71.45
82.00
81009
83.36
80.91
78.73
85
87.82
85.27
24.91
41.09
0.55
76.36
78.73
81.77
27.48
78.55
73.09
81 09
K2.55
80.1
76.73
82.75
83.8
78.73
MR SQW 3-48 PII-POO SONIIIITTT-TTIO TETTG--TINNN DUUNNINNGGNNNNN WERKING-TOSStnn oos-tanstoedd
MR3QQ SOWNNNN TOY- PROMPT- TIGT-THNNKNNG-- aransaalol
MIK SQWENN 4-4 ANG- PIMMOOP A NE- MAN-----UNNN TRINKING
3Q 1-45 PET-PROONT NITESTGT T-GT-TIINNN TWINKING-NINNN
ING- ING-EROMIE PROMTTA -UNN--N SIN--ANG--EAE MANN-TUINIING TUNNIING
MIK RO 3-1 1-8 BN UNN-PPRO PROMPE- PE-MPIIIGG--- KIN-TAA TH-THHINN HHINKNGG
MR MR3QQ JQWENN 3-1 4B TOT- FROMT- SING-TT BENG-THINE THUNKING
4- WWN.--OOOPPIIOO. THINNING
THNKING
TUT-PROMPT- -OP--TTT- NOT- THENKING
SWENR-88 PUT-PROMPE- -LNG-THINGG PRINKING
QWENN-- SENNN--H -H TOT IITE-RON OMPT- TOGT- TOT-THI THEN PRINIINNG KING
MEEN-NNG-TT THUNKING
TGT-THUNKING THTINNN KINNGG
G7--0S--200 200 ENG. PROMPRMMEEENN PENN-------TIIA CENU-TIIH -ENU-TIINKING TIITIINNI TI-TIINKING KING
GPC-OSS 2000005-20000 208 TOT PROMPT- COMPP--ENNE EING THUNKTHNNG ING
wrr--ss--120 1206 ENN--ROMPP BING- THINKING
GPP- OPF-OSS-1 1--oss---800 200 TGT PROMP- ENG. THINKING
GPF-055-1 1200 TOT COT-FROMPT-- THINKINNG
MR MR30 MONNNN 3-48 TOT- PRO NIPT --ING---- BUNG-TIINK TLHNNKING
TEST TET- PRO PIPO NIPU-IN- -ING-- -IN--IIIINGGNNNNN THINKIN
TET-PRONTT TETTT-GTTTHNNN TITTHIINGG
OWENN--E 3-8 3-48 TUT AUT-RRUMY CONOOOI-------- -UNG-TII CONG-TIINNNNNNNN
QWENT-1 14B TGT- PROMPT-TOO TOT-
TOY- PROMP-
OWEN3-1-H 1-88 ENG- PROMPT-ING
PROMETHELS 8X78 V2.0 PROMETREUS 7B-V2.0 2-0
833 -WwNN-INNII 148-3-------111 44 LORA
PROMETHEU 7EU--7EE---. 78-12.0
PROMETREUS R3 OWEN3 4B 14K
R3 OWEN3 8B. /4K
MR Modals (Ours)
OWINN--- BENNNN--8 TOT
Exidiig Reward
MR 3-88
3-
3-88
MR 3R30 SONNNNNNNNN 3-88
3-88
MR NOWENNA S-R
Base Medelss
QEEEN 48
CFF-05S-200
MR SQWENN
QWEN3-1
Model
42

---

Published as a conference paper at ICLR 2026
Chinese
73.58
74.31
70.28
78.53
78.72
78.17
81.28
81.30
80.92
65.69
65.14
28.07
72.11
73.39
70,46
22.02
27.34
2.57
22.20
72.29
78.53
81.65
70.64
70.64
76.33
77.61
77.98
79.63
78.
80.00
80.55
Vistnamese
62.18
61 12
60.91
61.73
65.09
65.27
69.64
68.55
56.91
40.36
66.91
68.36
66.91
18.55
21.27
0.36
35.09
61:27
65.45
69:39
62.91
62.76
36
68.36 16
67.99
65.09
65.64
67:45
68.36
67.45
71.09
Uzbek
52.99
46.82
56.99
55.90
31.76
58.44
58.80
45.55
52.99
52.09
15.79
60.98
61.07
19422
21 21
0.73
7.44
46.8
52.81
57.35
51.72
50.64
57.17
53.18
60:44
59.17
57.89
Undaa
39.20
44.32
28.98
45.45
51.70
41.19
42
33.52
55
55.97
53.41
16.48
24.15
0:00
6.53
41.76
44.60
45.74
41.76
36.36
42.90
44.03
26.14
46.02
45.17
46.31
Ukrainian
62.36
69.09
65.27
72.36
66.55
44.18
75.82
74.91
26.73
0.36
4,00
65.2
67.27
72.36
66.55
64.36
65.27
68.91
68.73
65.82
74.36
73.27
71.27
71.45
32
Table 18: Detailed results for INCLUDE- for each language (Part 3).
Turkish
60.95
61.31
64.05
54.38
64.05
60.58
62.6
60.95
54.01
66.61
68:43
66.61
20.26
22.26
0.00
18.07
69.2
63.69
64.42
63.32
60.95 05.05
65.88
65
60.95
95
60.22
65.69 69
63.87
Togalog
69.
70.80 70.80
77.00
77.80
56.60
80 00
72.80
77660
76.80
19.20
83.60
22.60
35.20
0.60
6.40
67.00
74.00
79.20
71.60
65.40
76.60
75.60
30
70.00
79.60
80.20
06.80
70,80
Telugu
M..5
45.62
29.56
50.91
69.09
29.56
58 28.94
53.83
41 8.88
40.15
60.58
45.62
18.25
25.73
0.00
12.04
48 91
54.38
48.18
43.25
15.51
53.47
53.47
46.72
56 30
57 57.48
53.28
39 99.96
45
48.
35
Tamil
50.73
48.73
56. 26.18
35.45
63.00
59.09
44.00
50.73
41.09
61.82
20:00
23,27
0.55
4.18
45.04
56.55
62.55
50.91
42.73
31.82
59.27
57.09
56.00
37.09
91
61.09
60.91
47.09
Sorbian
73
69.09
79.2 19.27
78.18
66.18
84.00
76.00
74,91
77.45
70.91
87.45
85.64
26.55
22.55
0.00
5.27
72.36
78.55
82.18
73,82
75.27
73.45
79.8
76 18
84 36
82 12.36
81.09
73.09
Albanian
22.41
72.96
78:77
78.40
59.89
83.67
78.95
75.32
52.99
86.75
78.22
21.78
26.68
0.18
3.81
09:51
78.58
79.67
75.14
74.59
66.42
82.03
81667
75.50
68.60
83.48
85.
85.30
72.60
49.
Russian
63.41
32.79
64.67
66.30
65.76
68.12
67.75
59.06
53.84
67.93
69.02
18,66
33.70
0.00
5.62
61.78
67.57
68.12
65.22
63.95
39.38
56.67
66.12
71.92
69.75
69. 38
46,90
Portugueso
72.78
74.25
72.78
71.32
76.22
76.04
75.50
70.60
67.51
77.68
78
77.13
27.04
23.23
0.54
29.40
74.23
72.60 60
75.50
72.78
74.95
72.41
74.33
76.59
73.14
72.78
76.77
76.41
77.13
72.25
Polish
64.90 64.9%
71.90
72.45
66.70
73.91
68.43
99
77.19
79.74
24.45
33.39
0.18
18.80
64.60
70.62
74.09
66 97
16
67. 77
71.90
71.53
08-07
73.72 73.72
74.82
73.91
PRIQWENNS TOT-PROMPT- TOOOONP--TO--TIOEE TOT-TRINKIIGG TRT-TRINKIGG--IOO satted
OWEN3-1B 1- TGT COO-PROPP--EN EEN--------IN -EE--EETHNINNNNN THINKING
THINKING
PROMPT- OWEN3-8B 88 ENG- PRO-PPOMPP-BBB EN G- KHINNKKING
OWEN33-8 TGT PROMPT-EN ENG ENG-THINNG THINKING
THINKING
THINKING
TWEN3-14E 1-1B TGT TGT-PRON PR PROMPE: FENG- THINKING
OPF--SS--24 208 ENG PRO MPNE-EG----TTTE THINKING
THINKING
GPE-OWS- TGT-PROMEE PWOMPPOOOOOOOOO SE-FE---TT-- FET- THINKING
ENGG THINKING
GPT-OSS-1 1208 TGT- COT--POMP--ECEE ENGG ENG-THINKING
MR. MR33Q WENS 4B ENG. PROMPT P. ENG THUNKING QWWENS-
TGT- PROMPT-PPEEEE -EN--N--TTTT THINKING
PROMPT- TGT- TGT-TTINNINNEE THINKING
MR JOWEES 4B TGT- PROMPT- --T------T THINKING--TTIII
MRR MRRQ RRR WENT AB ENG PROMPT PE ENG THINKING
OWENSS 8B TGT- PROMPT- SENG- THINKING
MRR MRRQQN RQ WENT. AB TGT PRO-PROMPT PROMPT TGT- TGT-THIINNEEN THINKING
MR IRRQWENSS 14B ENG- ENG-PRON FROM PROPPT--E EENG EENGTHINKING THINKING
TGT- PROMPT-MPTTT TOT-THINEE DECTTIINNIGG-IE
TOT-PROMPT-ER ENG-THINIIN THINKING
MR RRWENNN 14B TOT NOT-PROMPT- TOT- THINKING
QWEN3-44 PROMPT BING GN
TGT- TGT-PRON --E-E---ENNCCNNNNN
ENG- SET-T--TTTINIINNN
OWEN3-WF -B TGT- PROMPE--TTE PROMPI-TG- -TOT- -OP------T-----EEE
PROMPE- TOU
M- PROMETHEUS-TT 1AREEC--------- 14B. V2.0
R3-QQUR OOWEN- 14B- L-B--LRAA-- -6x
MRR MR 3Q WENT 4B TGT TGT-PROMPH
ENG-PROME
PROMETHELS COOMETEELSSSO 8X-88-V22 V2.0
Existing Reward Models
PROMETHEUS 78-V22.0
R3 R3-QQEE QWWEN3-4R 14K
R3 OWENT 8B 14K
OWEN3-1 14B EING
OPT-OSS-2U0 208 TGT-
MR3 Medels (Ours)
M: PROMETHEUS
3 3. MR SOWENS-
Base Models
QWEN3-
GPT-0WS
GPF-0
Model
MR
43

---

Published as a conference paper at ICLR 2026
Average
90.35
89.71
79.87
92.38
92.47
83.84
93.47
93.62
86.53
92.56
92.82
80.00
94.62
94.55
92.84
49.71
61.05
48.58
56.25
90,24
92.05
93.31
91.53
90.56
89.73
80.56
93.31
92.75
91.80
85.38
94.11
93.89
92.98
86.95
Eing lish
97. 00
97.20 27
97.60
97.60
96.60
96.40 40
97.80 77.80
97.80 87
97.60
96 10.20
96.00
20
98.40
98. 00
98:00
53.00
72.40 22.40
45.60
70.00
97.20
96. 20
97.80
97.40
97.20
96 00.60
97.00
97.20
96. 00
97.20
98.00
97.40
97.20
97.40
97.40
Avg Non Ling
89 .0.
88. 96
78.10
91 11.86
92.06
82.58
93.04
93.20
85.42
92.20
78.68
94. 94.30
94.20
92.32
49. 19.38
59.92
48,88
54.88
89.54
91.64
92.86
90.94
89 ,9
89.04
78.92
92.92
92.34
91.26
84.12
03.78
93.56
92.54
85 85.90
92.4
Chinese
94. 20
90.00 00.00
91 11.00
92 80
91 11
94.60
94 40
92 80
92.00 22.40
92 80
66 80
93.60 33.60
92 80
10
49. 40
50 80
55 40
95 50
92.00
94.60
94,60
95 -0
92 (2.60
91.6 91.60
94.00
93 00.00
95 80.80
92 20
94. 10.60
94 .0.60
93.80 80
93 30.00
93
61
Thai
92.40
91.00
89
95 95:29
94.60
91.40
92.80
93.80
91.80
93.20
93.60
82.80
95.20
94.60
94.40
48.20
61.00
48 88.80
58.40
93.00
93.60
93:40
95.20
93.60
91.60
83.00
95.
94.40
94.20
91.00
94.60
96.00
95.00
88.80
Telugu
85.
84.60
62. 12.20
89. 89.20
89.00
76.00
91.20
89 99.80
84 44.00
90.60
88.80
77.80
91.40
91 1.000
86.00
47 47.40
48.40
49 80
49.20
49.29
85.
88.60
90.60
87.00
85
82.60
55.20
92.00
90.00
86.00
78.20
92.00
92.60
87.
79.88
79,80
Table 19: Full detailed results by language of MGSM.
Swahill
66.80
63.
8.00
79. 79.40
78.20
22.60
84.40
85
29.00
85. 40
86
58.60
02.2 92.20
93.60
92.40
47.6 17.60
50.
47. 47.60
55 50
06 06.80
77.80
85.00 35.00
70. 10.20
65 20
68 68.20
26 26.20
83 3:40
80.80
78.00
32.00
87.80
86
82.
50,20
Rossian
95.20
95.40
94.00
95.40
95.80
94.40
96.00
96.60
94,20
94.40
94.60
37.40
96.40
95.80
95.80
$1.20
64.00
47.20 77.20
59.60
94.40
96.00
95.80
95.60
95.40
94
92.20
94.80
95.80
95.20
95.00
95.60
94.80
95.60
93.80
Japanese
91.00
91.00
90. 80.80
92.00
88. 80
93.00
93.00 30
90.20
91.00
91. 11880
71 11.40
93.00
92.80 80
86.00
48 38.00
60.40
47 -0
55. 20.20
90.80 80
91.60
91,80
91.60
91 80
88. 80
82. 80
92. 70
92.00
91.80
85.60
93.00
93.20
92.00
86.88 80
79.
French
91 .8U
92.
91 91:40
93.60
93.
91.20
93.00
23.60
92.20
92.80
93.4
84.00
03.20 93.20
93.60
93:40
89.80
65 50
49 20
54 $4,80
91
93.00
93.40
92.4 92.40
92.80
93.00
91.60
93 93.20
93.20
93.20
92 92.40
93:20
93.00
93.60
91.80
Spanish
94, 00
95 40
96 96.00
93 13
96 40
96. 20
95 80
96 20
96. 80
97.20
49 80
53 20
95 60
95 20
96.00 96.00
95 80
96 20
95 15.00
95.60
966 96:40
96
94.
95
87
97.8
49
67.
95
96
96
95.
94
96
96
German
94.00 40
94 40
92.40
94.80
94 94.60
92.60
94.40
95.40 40
94.20
93.80
94 94.60
87.20
95.00
95 80
94.80
51 1.400
64.00
49 49.60
54.80
93 93.60
95.00
95.00
94.60
60
95.00
93. 93.00
94.20
94.80
94 94.60
92 60
95.20
95.20
95 .0
94.00
Bengali
89 00
79 40
93.00
23.00
84.40
94.60
94.00 10.00
89 80
92.20
92.80
14.40
92.00
11-20
57 .20
48.60
92.40
92 22.80
90. 80
89 00
77. 77.60
94. 20
87 70
.2
94.40
88.
95.
95.
90.
92.
90.
83.60
TGT- PROMP F-1G TGT THINKINNG G-taas.. Grans.
HHNKING--rRS
COEAA-------TEEEE TOT- SEI-THLL WHINKING--IIINN I--Iooxx
NEL3--WEN3--- 4-4 PROMPT-ENC THINKING
MR. F.F--QQV EN 4B TCT- PROPP-EENG ENG- -E---ENGGGNNNNNN THINKING
MR3 QWEN3--40 4B TGT- TGT-PROMPH PROMPT- TGT- TIG--THNNG THINKING
FRR-QWEENT FE-QEENS ENNN--- 8B ENG- ENG-PROMI PE--PPOOOPPEECC PENGG TEEEENGGTTINN THINKING
MR33 1WEE3---2 8B TGT- CIT-PROMPE PROMPT- CIEOIPP--ENG---- ENG- TE-----GTTINN THINKING
HINKING
THINKING
1R3-OWEN3-- 14B TOT- PROMPT- E-NGGNGGIII -LINNKING
MR3 MRR--WENNS QWENT- 115---44 14B TGT PROME COMP T- TOT -IT---TT--IEEEE THINKING
FANE- THINKING
QWEN3-4B 4- PUT-PROMPT-TEE TOT- -OT-TTITIIIGNTT THINKING
THINKING QWEX3-88 88 ENG- ENG-PRONM PRO PRRMPT-EENE ENG- EENGTTHINIINNN
QWEN3-8P 8B TOT- PROMPT TOT- THINKING
QWEN3-1 14B LANG-PROME PROMPI- PI- ENG- THINKING
QWEN3-1 1- 14B TGT- TOT- PROMP T- P----OOOO TGT THINKING KING
GPI-OSS 20h ENG- PROMPT -NNG- THINKING
THINKING
GPT- OSS- 208 TOT TOT-PROOE PROMPT- STT--TTT TGT- THINKING
GPI-OSS- 120g ENG- ANG-PROMEN PROMPT- SEPP-EENN ENG- BENG-THINKKN
THINKING
GPT-OSS 120B TOT PROMPT- ENGG THINKING
GPT OPT--SS- OSS 120R TGT PROMPT TGT THINKING
MR3 1R33-QWEEL QWwN3- 14B BNG- BNG-FROI FROMPT- --NG- BENGTTHINNNNN
QWEN3 48 ENG PROMPT- ENG
E 8B TOT- PROMPT-
4B KUT-PROMPT-
GPT-OSS 20pB TGT- PROMPT-EET
Existing Reward Models
PRONETHEUS COONETHEUS-7B-V22 7B.v2.0
GROMETHEUS-88711 88x70-v220
78-02.0
M-PROMETHEUS -
M- PROMETHEUS 42.0
R3- 133-QWEN3--O --33--1111 14B. LoRA-4K
R3-QU FA-EWEN--- 8B-1 14x
MR3 Models (Our))
QWEN3- (4B BNGG-
MR3-QWENT EN- 4-
MR3-Q OWW33-WE33---- -B
MR3 SE-QQEEN QWENT-8F
Base Models
QWEN3-8B 8
MR33 MR33QWENN
Model
MR.
44

---

Published as a conference paper at ICLR 2026
Table 20: Summarized results for RTP-LX based on English and Non-English F1 scores.
Model
Fl Overall
Fl English
Fl Non-Englist
Base Models
QWEN3-4B ENG-PROMPT-ENG-TTINNEE
84.33
90.79
84.10
QWEN3-4B TCT-PROMPT-ENG-
82.49
90.19
82.22
QWEN3-4B COT-PROMPT-TGT-THINNEER
75.33
85.99
74.95
OWEN3-8B
77.55
86.69
77.22
OWEN3-8B COT-PROMPT-NNG-THINKII
77.74
86.99
77.41
QWEN3-8B TCT-PROMPT-TTT-THINNII
74.24
87.29
73.78
QWEN3-14B ENG-PROMPT-ENG-TIO G
77.98
86.49
77.68
QWEN3-14B 14B STT-PROMPT-ENG-THINKI THINKING
77.00
85,59
76,69
QWEN3-14B ICT-PROMPT-TCT-TTIINNN
73.70
84.68
73.31
GPT-OSS-206 ENG-PROMPT-ENG- THINKING
90.33
94.69
90.17
GPT-OSS-206
88.76
94.39
88.57
GPT-OSS-20B
80.91
94.19
80.44
GPT-OSS- ENG-PROMPT-ENC THINKINGG
91.32
95.40
91.18
GPT-OSS- CCT-PROMPT-ENG-TIINGG
95.60
90.12
GPT-OSS-120B TGT-PROMPT-TGT THT-TKIKIGGGGGG
88.24
95.70
87.98
Existing Reward Models
PROMETHEUS- 7B-22.0 PROMETHEUS-BI5 BAAEHEEEEUUS----222282
87.13
92.69
86.94
81.00
86.69
80.80
M- H-PROMETHEUS-PH -B--22.0
73.79
67.27
74.02
M- A-PROMETHEUS-14B
14B--
84,64
86,89
84.56
R3-QWEN3-44B-145
87.37
92.89
87.18
23-QWEN3-8B-II
86,95
93.19
86.73
R3-QWEN3- 1AE-N33-4B-LLLN 14B-LoRA-4K
79.32
87.89
79.01
MR3 Models (Ours)
MR3-QWEN3-4B ENG-PROMPT-INGGTTING
88.24
93.59
88.05
MR3 OWENN-WW TOT-PROMPT-ENG KING
87.14
94.49
86.88
MR3-Q RR3-QWEN3-4B CETT-PROMP--TTI-HIING
87.95
95.10
87.70
MR3 1133-QWN3-44B
84.03
93.39
83.70
MR3-QWEN3-SB ANG-PROMPPIENN-THHIIIINGG
90.19
94.99
90.02
11R3-QWE33-8B YCT-PROMPT-ENG-TTIINNN
88.41
94.69
88.19
MR3 15--QWNN3-88 8B SOT-PROMPT-TGT-THHI
88.07
95,90
87.80
3R3-QWEN3-88 ICT-PROMPT-TTT-TIINNNNN
85.34
94.89
85.00
HIR3-QWEN3-1444 COG--RROPP-ENG-THIINII
90.26
95.10
90.09
1B33QQWE33-14B
IUT-PROMP--NNG-TIINNIIC
89.34
95.40
89.13
1R33-QWEN3114B
OCT-PROMPT-TTT-THINNIN
87.55
96.00
87.25
MR3-QWEN3-14B
CCT-PROMPT-TGT-THINEIEN
86.28
94.89
85.97
45

---

Published as a conference paper at ICLR 2026
Japanese
85 55.89
84.63
86.45
75.23
76.26
77.85
77.10
78 78.04
83.46
92.71
89.81
87.57
93.18
92.62
91.68
85. .9.89
81 11.78
74.49
84 84.39
89.63
88.04
77.66
91 91.31
90.37
91.21
88 18.13
92.34
89.53
89.63
89.07
92.71
91 91.50
90.75
88 13
87.87
87 7.
80.95
80 68
81 8..67
77.72
81.22
78 78.62
71.79
91 91.9
12.12
22.54
92.36
$9.38
87.33
81 11440
76.64
89.76
29.22
80.41
89.85
89 -9.40
90.17 12
91.55
91 18
90.39
88.50 59
92.45
90.93
11
80.40 00
92.00 20
Indooessaa
80 0X.00
86, 17
79,89
79.80
79.53
80.53
79.07
78.34
91 1.999
90.26
92.36
92.36
89.44
91.45
84.17
78.16
90.08
91.08
82.35
90.63
92.08
89.35
93.08
91.36
90.90 90
91.45
91.08
91.63
92.45
Hungarian
N2.47
81 1.669
70.65
53
79.20 26
64.99
79.71
78 90
70.20
84.74
33
85.55
84.47
82.41
N0.70
80.43
72.35
N0.52
83.03
08
79 99.80
84.38
83 83.84
76.75
90
84 29
33.39
K3.75
82.23
84.29
84
83.93
Hindi
82.4 X2.42
80.00
79.2 19.73
79.91
80.90 80.99
81.08
80 80.66
79.55
78.92
88.34
87.26
78.74
88.79
88.16
87.00
N2.69
73.63
52.60
86
81.35
87.17
84 84
86
87.35
88.34
87.17
87.35
87.62
88.07
89 99.06
81
85
88
87.88
Table 21: Detailed results for RTP-LX LX for each language (Part 1).
Hehrew
83.15
65.93
85.
58.33
77.78
75.83
64.63
7k.15
78.1
72.
93.33
89.63
85.00
95 .6
93.24
90.95 93
80.07
80.93
76.94
38
89.44
81.02
91.11
88.00
87 87.69
93.24
90.11
85.44 56.46
78 06
93.24
91.39
85.74
80.93
French
87.86
89.3 89.31
83.62
78.61
79.09
77.07
76.88
77.17
75.14
91.52
92.39
92.58
93.06
88.92
90.75
72.25
87.09
89.88
88.05
78.52
90.17
90.27
88.05
90.85
91.23
90.37
91.43
88.63
92.58
81
92.
92.4
92.1
Finnish
84. M0
83 89
68.
82 82.35
81.71
73 98
79.00 80
78.71
72 25
96.52 54
95. 18
88
91
95.00
93.96
91 16
84 44
79.44
63
91 54
91.63
82.17 17
9/3 90
91.5 12
94.90 90
87.26
96 54
91.99
94.54 54
93.8
95.9
91
54
88. 58.72
Spanishh
88. 16
87.34 7.
78.32
78.78
79.90
81 81.79
78.60
76.78
92.35
91.62
94.35
93.53
92.33
91.07
82.06
72.13
83.61
91.07
88.50
79.96
90.89
92.17
89.80
92.08
92.90
92.17
91 91.99
93.08
92.62
93.35
91.80
93.72
English
90.79
85. 85.99
86 59
86.99
87 87.29
85.50
84.68
94.39
95 40
95.60
95.70
92.69
86 56,69
67.27
86.89
92.89
93.19
87.89
93.50
94.49
95.10
91.39
94.99
94.69
95.90
94.89
95.10
95.40
96.00
94 89
90
94
German
87.8
42
85.71
76.58
79. 38
78.91
69 69.58
68.51
92.5
91.74
86
94 0.0
92.52
89.70
90.86
80. 60.76
69.87
87.4 97.46
89 80
88.53
75.41
91.16
90.57
91.55
90.67
92.32
89.99
89 80
89 89
92.5 92.52
89 70
90.67
89 99.60
Danish
86. 76
83 3.884
75.53
79.82
78.90
75.34
78.26
78.
80.09
92.69
91.78
89.32
93.33
93.97
91.69
89.77
83.56
74.70
65.21
89.
90. 14
80,23
91.78
89.68
92.60
85
92.51
91.14
90. 78
89.04
92.42
93.52
94, 16
91.32
Czech
83.24
80.06
79.38
15.2
74.37
78.81
76.11
72.93
76-40
88.54
87.48
81.60
88.54
87.67
87.09
65.65
85.6
80
71.77
85.26
86.61
85.45
76.78
87.67
86.42
87.57
88 68.44
88.5
85.74
84.87
86.71
88.
89.02
87.28
87.86
XX 88.86
85.33
75.71
79.24
83.14
81.81
80,86
79.14
95.71
93.05
96.67
92.76
73.81
86.76
84.00
79.81
87.43
93.90
91.05
81.05
92.00
90.57
90.19
90. 10,10
95.33
90.57
87.43 47
90.86
95.43
92.76
91.33
92.2
MR3 MR33-0 QWENN 4B ENGG EING- PROMPT-EEEEEEEE ENG- THINKING
MR3-QWER 3R3--WWEN3--- 3-48 4-4 PROMPT- THINKING
MIL3 MIL3-QQE QWEN3-4 3-48: TGT- PROPP- F- -IG-THINKINGG KING
TGT- PROMPT- TGT- THINKING-
MR3- W 3- 8B ENG. ENG-PROOO PROMPT-EOOOOOOOO T-ENG- THUNNINNG KING
MR. QWWEN' 3- TGT TGT- PROMPT EING- ENG-THHNNEE GHENKING
MR 3R3-QWWER OOWEN KES TOT- TOT-PROO PROMPT- TOT THINKING
MR MR3-QW Q-EN3-1 3-1EN-1441 14B TGT PROMPP PROMPT-ENG ENG- THINKING
MR 3--WENN--- 3- TGT- PROMPT- TCT-THINKING
MR3-Q 3- 1-WEN3-14BB 14B TUT- PROMPT- TGT-T TGT-THINKINGGHII
ENG PROMPT- EING THINKING
Q QWENN 3-BB FET-PROMPT- MENN-------TTIEE HENG- TILNKING
Qw 3- TGT- PROMPT- TET- THENKING
QWENT- 3-88 ENG PROMPT- CEEMPT--NG----- ENGG EENGTEGHII THINKING
THINKING
PROMPT AEPI-ETEE TET TET------TEEE THINKING
QWEN3- 14B ENG PROMPT- ENG-THING THINKING
3WEN3-1A 14B TOT- STT-PROMPT-ENC CEAPPT--EN-TIINNNN --IIINNNIGG
14B TOT- PROMPT-TGIE --I--GG---TEEE THINKING
OPT-OSS 208 ENG ENG-PROME PWOMPOOOPPPPTT ENG- THINKING
OPT-OSS-200 200 TCT-PROMPT-EN I-EN G- THINKING
THINKING
GPT-OSS- -OOSS-1200 1208 ENG PROMPT- CENNN------TTTEEE ENG- THINKING
GPT-OSS-IA 120u LGI-PROMPT ENN-ITHINNNIIIC THINKING
GPT-OSS- 120s TGT- PIO-PROMP--OO -GG- TTGT-THING THINKING
1R.3-QWEN3-- 3-18 ENG- LNG-PRORO PEO-PPOOOMPTTT- -INTTN-------
GPT-OSS 20B TGT- PROMPI--M SIG-P--GG--IINNNG
MR3 OWENN- 3-WEN3-8B 8- 1-1---TTTTTT TOT- POT-PROMPT-
R3 QWEN3 14B-1 LORA 4x
OWEN 3-BB 3-88 CEI-PROMPT-EEE
PROMETIEUS-8N788 8878- V2.0
A-PROMETHEUS- CEO.MMEEEESSSBBE 7-8-V220
M-PRRMETTELS 14B-Y2.0
Existing Reward Models
PROMETHEUS 7B-v2.0
TGT TGT-PAA
COONETHEUS-7B-22..
R3 L--EEN3----B- 4B--------44-- 14X
R3- QWEN3 8B-14K
MR3 Modelss (Ours)
Base Models
OWEN 3-B 4B
3 3-88 88
MR 3-3--0 QWENT
QWEN3-11
Model
46

---

Published as a conference paper at ICLR 2026
Chinese (Traditional)
61.84
62.38
60.05
61.30
61.48
60.23
59.52
59.43
61.39
61.57
61.13
61.57
61.39
61.39
57.82
54.78
57.82
62.56
62.11
61.48
61.57
61.57
61.66
61.75
61.39
61.22
61.93
61.57
61.22
61.75
52.37
61.75
Chinese
96.46
88% 68
82.21
79.50
85,78
84.24
82.21
83.85
91.59
91-10
89.56
92.65
92.26
11.68
90.14
86. 17
78.43
62.88
90.72
89.46
86.85
91 83
11.68
26
11.20
91. 30
01.88
92.24 92.26
91.68
90-43
$1.59
91.6
Ukraiman
88.57
28.57
86. 46.95
81.38
83.52
82.48
81.33
77.43
94.76
94.86
90.86
96.38
93.62
95.24
93.24
87.33
82.95
88.48
91.62
92.48
84.67
94.48
93.52
95.05
95.14
96. 96.00
93.24
95.05
96.19
Turkish
80.55
8100
80.93
77.88
75.51
87.42
85.32
87.23
84.56
78.07
73,88
80.63
K6-46
87.23
N2.75
88.27
88.66
87.8 87.80
M6.27
88.37
83.99
89.13
87.51
88.47
Table 22: Detailed results for RTP-LX for each language (Part 2).
Thai
85.58
81 10
76.40
77.96
76.40
78.24
75.85
72.91
89.35
86.87
83.47
90.54
90.08
89.35
77.41
74.75
77.41
86.69
87.05
88.52
80.35
87.14
84.39
85.67
89.99
87.88
87.05
86,04
P9
88.34
09
87.88
Swahili
70.70
55. 55.26
47.4
1..30
9.49
62.33
67.44
93.58
80.00
59. 16
96.54
92.74
88.00
87.72
79.35
60.37
88.47
73.95
71.44
58
70.70
80.8
86.60
79
90.70
63
Swedish
83.70 70
83.70
77.69
73.12
72.26
71.97
70.54
91.33
91.90
88
90.18
89.61
90.37
80.65
75.88
85.51
87.89
88 56
75.69
70
90.94
92.18
89 90
91 13
91 91
Rossian
76.16
76.00
75.59
75.18
76.57
79.84
79.02
77.14
81.22
81 81.31
79.35
79.67
75.67
71.10
76,98
79.43
79.84
76.16
80.65
11.31
80.33
80.33
80 98
50.49
79.59
80.98
80.16 16
80 16
80.65
Poruguese
86
86. 03
78 28.26
79.00 9300
79.91
77.44
77.44
75.25
92.88
92.42
88.04
93.42
91.78
91.69
81.00 1.00
76.07
88. 95
N9 39
888.06 35.04
78
90.
91.96
90.05
80.87
92.69
92.42
91.32
92.24
93 93.24
92.69
91.42
Polish
81 18
80.3 25
76.43
78.53
74,89
88.08
80.04
88.63
86.35
83.71
80.25 25
72. 16
83.08
85.17
82.71
77.62
87.17
86.00
85.53
87.72
86.44
84.55
86.81
87.35
57.72
26
58.35
Norwegian
86.11
85.53
77.66
72.98
57.96
75.52
69.15
93.45
91.90
84.99
93.08
91.72
80.35 35
76.71
87.44
91.17
89.5.
77.43
89.44
89.1
88 90
93.08
89.26
86
92.
90.81
90.17
Durch
89
86,9 86.98
83
86.79
$1330
79.72
83.53
83.1
96.28
96.09
93
95.81
95.81
93
94.98
87.16
78. 14
98.70
93.40
94.
86.1 66.14
95.07
94.98
95.81
96 96.09
93.93
95 25.35
94.23
94
96 09
96. 26.28
94.88
96.28
Korean
86.53
84.53
80.53
81.26
84.17
81 16
.66
92.27
93.99
93.54
92.99
86.90 90
75.52
87.63
91.81
91.54
84.90
93.33 .6
90.45
93.54
91.54
93.36 .36
93.08
92.45
94.36 .6
93.45
93. .9
92.63
91.5
THINKING MR3 QWEN3- 14B POO-PROMPT-TTT THINKING-IERR trann. trans.
TGT- TGT- PROMPT- TGT- THINKING-rann -ra..
TGT- PROMPT-TOE TGT-P-------IIII WHNKINGG-Taa 14B 14B
KHINKKIING
THINKING THINKING
TWEN3-1-1B 141 TGT- TGT-PRONN PROMM PROPPT T FENG- THINKING
CEE-T---GGGNNNNNN OPF--SS--20 208 ENG- ENG-PROMPP PROMPEE SMP--EE ENG- THINKING
THINKING
ENG- -ET----TT--TIIII THINKING
ENGG THINKING
GPT-OSS-1 1208 TGT PROMPP-EROEEECC E-N---NG--IIIE -NG-THINIING
PROMPT- TOU THINKING
MR3 QWEN3 41 ENG. PROMPT-ES ENII THINKING
MR3 OWEN3-4 TGT- TGT-PRO PROMPT- EEN-P--------TE THINKING
TOT THINKING
MR3 QWENT Sn ENG: PROMPT-ENT ENG THINKING
MR3 QWENT- 8B TGT- TGT-PRO PROMPT- ENG THINKING
MR3 QWENT Sn TOT PROMPT TOT THINKING
ENG PROMPT-ESS ENG-T THINKING ING
MR3 QWEN3- 14B TGT- PROMPT- PROMPT-ENG- --ENG- -EENGTHHINEE THINKING
TGT- PROMPT- TOT- TEE---TT--HIINNTT
THINKING
PRUMPT BING HNG-THINNNN
OWEN3-18 1- TGT- COT-PROMPT-ENT TE-----EEE ENN--T -EE-E-EEGG-----IIIEEEE
ENG ENG-THIN
OWEN3-W8 -B TGT PROMPY- TT TOT- TAAT--GGG-INEEEEE
OWEN33-8 TGT CEEAMMP--NN--HIEEE
GPF-OSY- TGT-PROMEE PROMPROOOOTTTO ST-TET--TTT FCT-
PROMPT-ENT
PROMPT-
OWEN3-8B 88 ENG- PROMPPRMPP-EER
PROMETHELS TEMEESLLLS SX78-V2.0
M- PROMETREESSSSEEE 14R 148: V2.0
7B.
R3-QQU OOEEN--1 148-148---00- 4B-LORA-45
MR3 QWENT 4B TOT PRO
Existing Reward Models
PROMETHEUS 78-V22.0
R3 R3-QQEE OWWEN--4R 14K
R3 OWENT SB 14K
OWEN3-111 14B EING
TGT-
OPT-OSS-2U0 208 TGT-
M. POOMETHEUS
MR3 Medels (Ours)
MRS MRF-QW QWEN3-1 14B 14B
MR3 OWENS
MR3 W---WEEN-SSE
MR.. MRF-QQEER QWEN3-
Base Models
QWEN3
QWEN3-
GPF-OSV
GPF-OSS
Model
47

---

Published as a conference paper at ICLR 2026
Table 23: Ablation study on the effect of dataset size on training QWEN3-4B using subsets of 10K, 25K, 50K, and 100K samples. Standard deviations over 5 runs.
Setting
m--ewardBBench
RewardBlncch
MM-Exal
IndoPref
INCLUDE--bse-4
mngsan Acc.
RTP-IXX
Avg.
Acc.
Acc.
Acc.
Acc.
Acc.
F1
23
ling
18 langs
langs
langs
27 langs
QWEN3-4B (10K)
69.90
QWEN3-1B (25K)
85.99 tus
88.96 4:200
70.14 a:
79.62 m2e
60.05 2:223
88.12 2.3
89.64 =:223
80.36 =44
QWEN3-1B (50K)
86.43 use
88.79 044
70.90 2:35
80.22
60.29 =:201
88, 4:221
89.03 2::11
80.56 604
QWEN3-4B (100K)
87.61
89.74a
72.22
82.62
63.01
91.30
2:330
88
82,09
Table 24: Validation-set performance across different curriculum strategies on QWEN3-4B. Here, EasyTo ToHad is shown perform the best, so we select this curriculum as our final curriculum.
Setting
Kendall Tau
QWEN3-4B EasyToHard (Our Final Curriculum)
0.4779 0.0114
QWEN3-4B Random
0,4583 0.0083
QWEN3-4B Hasyy
0.4647
QWEN3-4B StartEng
0.4516 0.009
QWEN3-4B StartEng- Faannn--EsTTTHaard
0,3800 0.0086
QWEN3-4B StartEng-Hare HartEnng-ardToaayy
0,4629 0.0036
QWEN3-4B Baseline
0.1983 0.0223
Table 25: Test-set performance across different curriculum strategies on QWEN3-4B.
Setting
R--RuurdBBnchh Ave 25 langs
RewariBench Acc. ling
MML-Esa Acc. lang
Indeeeed Acc.
INCLUDE-hhee-44
RTP-IX
Avg.
Acc.
Anc.
Fl
lings
44 longs
11 langs
27 langs
Qwen3-48 Tobfand Carriclum))
amc
2111
mur
Qwe33-48 Higlish
mm
a.1
Qwe13-4B Haddooasy
85.15
55.95
Qweni-411 Sort Higllshh Fonylilliedd
86.30
89.00
11:41
79.54
50.00.11
87.95
08-41
10.25
Qwen3-48 Soet Haplish Hondyo
86.96
89.11
11.80
K1
62.33
90.44.91
65.65
81. 11.58.om
Table 26: Ablation on weaker teacher models on performance metries.
Setting
n-RRwardBBench
RewardBencch Acc. lang
MM-Exal Acc. lang
InduPref
INCLUDE--Roe-4
mgam
RTP-LX
Acc.
Acc.
langs
18 langs
langs
11 langs
27 langs
Qwen3 4B (Our Teacher)
87.51 our
89.74 a:3
72.22 222
82.62 ae:
63.01 a
91.20 az
88.20 aur
$2.09 1:1
Qwen1-48 (Wenk Tescher)
84.63 ooe
87.44 2:300
69.95 2.35
78.54 245
$4.24 2018
84.59 201s
88.06 am
78.21
1:331
Qwen3-4B (Buseline)
84.51
$8.04 a..
68 2:22
80.07 2032
61.54
2z2
90.34 2014
84.10 2016
79.63 1:
reasoning and rubric adherence, whereas RLVR primarily optimizes for answer correctness and does not explicitly supervise the model to learn reason through the provided rubrics.
Sfficiency-wise RLVR is significantly more expensive: training 3 epochs of RLVR on Qwen3-4p with 16 H100 GPUs takes approximately days, while training epochs of SFT on 100K samples only takes approximately 8 hours on 4 H100 GPUs. Thus, SFT is not only more effective but also computationally cheaper.
Finally, prior RL approaches were mostly evaluated in pairwise or limited settings, and it remains unclear how well they generalize to a diverse selection of datasets and tasks. In contrast, our SFT approach demonstrates consistent improvements across multiple multilingual and multi-task benchmarks.
I MULTILINGUAL HUMAN EVALUATION
To assess the quality of the rubrics, reasoning traces in our training data, as well as reasoning traces in evaluation in different languages (especially lower resource ones), we conduct human studies involving a total of 20 annotators who are native speakers of 12 different languages: Chinese (3), Spanish (2), Japanese (1), Hindi (1), Bengali (1), Indonesian (4), Korean (2), Portuguese (1), Vietnamese (2), Javanese (1), Albanian (1), and Telugu (1). Note that Javanese, Albanian, and Telugu scenario.
unseen languages MR3 models during training, making this
48

---

Published as a conference paper at ICLR 2026
Table 27: Ablation on overall accuracy using different data sources taken from Anugraha 2025
Setting
RM-Bench
RewardBench
BBH
MMLU-STEM
Random Sampling
77.0
86.6
89.7
93.0
DATASET
Only Pairwise
82.1
90.2
91.5
94.4
Only Pointwise
80.0
86.0
90.1
93.4
Only Binary
81.6
88.8
91.0
94.0
ABLATIONS
No Rubric
76.3
87.9
85.1
91.9
No Explanation
83.1
90.2
91.7
94.5
No Reasoning
71.2
82.6
79.8
88.2
R3
83.5
90.2
91.9
94.5
Table 28: Ablation study of QWEN3-4B across different training stages (50K SFT, 100K SFT (ours), and 50K SFT + 50K RLVR) on various benchmarks, reported as mean + standard deviation. Bolded numbers indicate the best-performing results, while underlined numbers indicate the second-best- performing results.
Setting
ne-RewardBench
RewardBench Acc.
MM-Eval
INCLUDEE-bwe-44
mgsm
Ang:
Acc.
lungs
lang
lang
langs
lungs
11lang
langs
QWEN3-4B <ROK
ans
2:11
2: $2.09
QWEN3 (100K SET)
87.61 =017
89.74
72.32
222
82.62
63.01
91.20
2327
QWLN3-4B (SIK SPT S0K KLVR)
84.91
8:35
$7.72 a.0
66.12 9.9
50.57 am
61.42
#31
90.17
92.10 833
30.45 à
Each annotator receives an instruction and asked to rate the examples according to our rubrics. Each annotation takes 1-2 hours to complete.
1.1 RUBRIC EVALUATION
We manually evaluate the rubrics obtained from Appendix C.2.2 over two dimensions.
Plausibility: How fitting is the rubric given the task?
Score of 3: rubric is clearly tailored to the task and criteria directly match the task requirements
Score of rubric mostly fitting but somewhat generic. Rubric may missing a few details or include some irrelevant points
Score of 1: rubric is poor fit for the task, and the criteria completely misaligned.
Score-ability: How easy is it to score examples given the rubric?
Score of clear distinctions between scores, leaving no ambiguity.
Score of scoring levels may contain some overlap, require subjective judgment.
Score of scoring levels are vague and require significant disambiguation.
Translation Quality: How good is the translation? Is it semantically equivalent to the
English counterpart?
Score of essentially the same meaning, there may be small word choices that could be changed.
Score of 2: most of the meanings are preserved. There maybe some content from English missing (under-translation) or content not present in English (over-translation) Score of there major differences between the two versions, and the differences are large enough that would lead to very different scores in the same input.
For each language, we collect all rubrics used in our training and evaluation available. For each annotator, we ask them to rate the rubric of the tasks along three dimensions.
In Table we observe a consistent distribution of rubric quality across language-resoutee resource levels for both plausibility and scoreability. Translation quality is slightly lower for medium- and low-resource
49

---

Published as a conference paper at ICLR 2026
Table 29: Rubric quality across different languages. We report mean and standard deviations across up to 12 rubrics for tasks in our study. Annotator results are aggregated for each language. Majority rubrics are of high quality (> 2.5) across language resources. Resource language resource level.
Resource
Language
Plausibility
Score-ability
Translation Quality
High
Chinese
3.0 00
2.5
3.0
High
Japanese
3.0 0:00
2.9
3.0 1:00
High
Spanish
2.7 2:05
2,5 0.0
2.7 2:04
Medium
Bengali
3.0 0:00
2.7 1:0.
2.7 1:05
Medium
Hindi
3.0 000
2.8 +04
2.8 04
Medium
Indonesian
3,0 0.0
2,8 1:03
3,0 i
Medium
Korean
2.5 =05
2,9 1.3
2,4 2.6
Medium
Portuguese
3.0 0.0
2,2 44
3.0 +00
Medium
Vietnamese
1.0
1.0000
2.8 1.04
Low
Alhanian
3.0
2.3
Low
Javanese
3.0
2.0
2.6
Low
Telugu
3.0
3.0
2.3
languages. Nevertheless, given the overall high quality of the rubric content, conclude that the rubrics used in both training and evaluation are sufficiently reliable for our study.
Qualitatively, most annotators report that the translations are generally adequate, with only minor issues related to word choice. One Korean annotator noticed a slight inconsistency in word choices between the rubric and instruction. Vietnamese annotators note occasional awkward phrasing and overly literal translations. The Albanian annotator finds the rubrics largely understandable, with minor grammatical errors slightly unnatural expressions. The Javanese annotator observes frequent code-mixing with Indonesian.
The Chinese annotators comment that pairwise tasks (e.g., PPE, RewardBench) tend be more difficult to score because they involve multiple criteria. Rubrics for HelpSteer3, which uses scoring range from -3 to 3, are also more challenging due to the larger number of rating options.
1.2 TRAINING REASONING EVALUATION
Our framework relies distilling reasoning capabilities from strong teacher model. For experiments that involve translated reasoning, we additionally depend on the translator model being sufficiently capable. These procedures depend heavily on the quality of the teacher and translator models, especially for lower-resource languages. To validate our setup, we also ask annotators to evaluate training data samples with respect the quality of their reasoning traces across different languages.
For each annotator, we sample five training data points in the target language and ask them to evaluate three types of reasoning produced by the teacher model: (1) English reasoning, (2) target- language reasoning via prompt forcing, and (3) translated larget-language reasoning (with parallel questions). Annotators rate the reasoning along two dimensions factual correctness and logical coherence-usinn 1-3 scale (higher is better), following Anugraha al. (2025).
Factual Correctness. (Scale: 1-3) assesses whether the statements in the reasoning trace are true and supported by external knowledge or evidence. When scoring, treat retrievable evidence or commonsense facts acceptable grounding.
Score of (Fully Correct): All statements are factually accurate and supported by known facts, context, ground truth. No hallucinations or inaccuracies.
Score of 2 (Partially Correct): Most statements are accurate, but minor factual errors or unverifiable claims exist. Does not change the final conclusion, but may reduce trace reliability.
Score of (Incorrect): Contains one or more clear factual errors or hallucinations that undermine the trace. May lead incorrect conclusions mislead the model.
50

---

Published as a conference paper at ICLR 2026
Logical Coherence. measures whether the reasoning steps logically follow from each other and form a coherent argument or thought process. Judge based on internal consistency. not factuality. A trace can be factually wrong but still logically coherent.
Score of (Fully Coherent): All steps follow logically and consistently. No missing steps, contradictions, unjustified jumps in reasoning. A smooth, interpretable chain.
Score of 2 (Somewhat Coherent): Mostly logical, but has minor gaps, unclear transitions, or weak justifications. Still understandable, but less robust as supervision.
Score of (Incoherent): Trace is illogical, disjointed, or internally inconsistent. Steps may contradict, skip crucial logic, or appear arbitrary.
Table 30: Reasoning quality (in training data) across different languages. We report the mean + standard deviation across 5 samples per annotator. Results in each language are aggregated across annotators. Majority training reasoning is of high quality (> 2) across language resources. Resource language resource level.
Eng Reason
Tgt Reason
Tgt Translated Reason
Resource
Language
factual
logical
factual
logical
factual
logical
High High
Chinese
3,0 0.0
3,0 0.0
3,0 0.0
3,0 0.0
3,0 0.0
3,0 = 0.0
Japanese
3.0 = 0.0
2.8 0.4
3.0 0.0
2.8 04
3.0
2.8
High
Spanish
3.0 0.0
3,0 0.0
2.8 0.4
2.8 04
2.6
2.8 0.4
Medium
Bengali
3.0 = 0.0
2.5 0.7
3.0 0.0
2.7
2.5
2.0 0.0
Medium
Hindi
3,0 = 0.0
3,0 0.0
2.
2.8
3,0 3,0
3,0 =
Medium
Indonesian
3.0
2.8
=.0
2.8
+.0
2.6
Medium
Korean
3.0 0.0
3.0 0.0
2.8
2.8
2.6
2.6 0.5
Medium
Portuguese
2.8 -
2.4 0.5
2.5
2.2 0.
2.4
2.2 0.4
Medium
Vietnamese
3.0 0.0
2.8 1 0.4
2.9 1:03
2.8 1.4
3.0
2.8 0.4
In Table 30 we that English reasoning in general has the highest reasoning score. This is expected as GPT-OSS- 20B is likely primarily trained English. In target reasoning, we see a slight degradation in the factual correctness and logical coherence. In translated target language reasoning, we see a slight decrease compared to native target reasoning. However, in all cases, the reasoning remains relatively high quality, containing minor mistakes most (above score of 2).
Qualitatively, the Spanish annotator noticed that target reasoning is more systematic and reviews all categories of the rubric, while translated reasoning sometimes contain hallucination about its role classifier vs. assistant (likely an artifact of our translation step). Bengali annotator noticed translated reasoning to more detailed and exploratory. Vietnamese annotators notice that target reasoning sometimes abruptly short and lacking in analysis. Target reasoning sometimes use bad-word choices, unnatural sode-switching and incomplete sentences. One of the annotators finds translated reasoning to be much more in depth, more analysis, as well more persuasive.
In conclusion, we find our training reasoning traces to be of general high quality, with English reasoning quality trumps target reasoning, which in turn beats translated target reasoning.
I.3 INFERENCE REASONING EVALUATION
Finally, we evaluate the quality of QWEN3 and MR3 14B model's reasoning and whether im- provements extend to lower-resource languages. For each annotator, we sample 15 questions from QweN3-tgt-ttt, -R3-tgt-tgt, and MR3-eng-eng, including both correctly and incorrectly answered.
We use the same evaluation criteria in training- time reasoning evaluation, which are factual correct- ness and logical coherence. We compare QWEN3-igt-tge and MR3-tgt-tgt to measure improvements from fine-tuning, and MR3-tgt-tgt with MR3-eng-eng to evaluate whether larget-language reasoning matches English reasoning.
In Table 31 the majority of annotators rate MR3-TGT-TGT higher than QWEN3-TGT-TGr one or both metrics, with consistent improvements across low-, medium-, and high high-resource languages. This confirms that fine-tuning improves reasoning in target languages across resource levels.
51

---

Published as a conference paper at ICLR 2026
Table 31: Reasoning quality (in evaluation) across different languages. We report the mean and standard deviations (in parentheses) across 15 samples per annotator. Results in each language are aggregated across annotators. MR3 outperforms QWEN, and often MR3 with target reasoning beats English reasoning. Resource is language resource level. QWEN-TGT-TGT QWEX3-14DD CTT-PROMPT-TGT-THINNNIN MR3-TGT-TGT is equivalent to MR3-QWEN3- 14B TGT-PROMPT- TGT-THINKING. MR3-ENG-ENG is equivalent to MR3-QWEN3-144 ENG-PROMPT-ENG-THINNII
QWEN-TGT-TGT
MR3-TGT-TG1
MR3-ENG-ENG
Resource
Language
factual
logical
factual
logical
factual
logical
High High
Chinese
3.0 0.0
2.3 1.8
3.0 0.00
2.6 0.6
2.9 = 0.2
2.7 + 0.4
Japanese
1.3 + 0.6
1.3 0.6
2.7 0.5
2.6 = 0.7
2.8 + #.5
2.9 t: 0.2
High
Spanish
2.2 + 0.7
2.3 0.6
0.11
2.8 0.4
2.5
2.3 0.5
Medium
Bengali
1.6 0.9
2.7
2.7 005
2.8
2.4
2.8 0.4
Medium
Hindi
0.3
1.1 003
2.
2.9
2.7
2.8 0.3
Medium
Indonesian
2.
2.1 007
3.0
2.9
3,0
2.9 #00
Medium
Korean
2.0
1.8
0.0
2.8
2.8
3.0 +.0
Medium
Portuguese
2.6
2.0
2.7
2.7
2.7
2.3 1.8
Medium
Vietnamese
2.6
2.7 006
2.9
2.9
2.9
2.9 +03
Low
Albanian
1.9
1.
2.9
2.8 0.4
2.5
2.5 #0.7
Low
Javanese
1.7
1.3
2.0
2.0
2.7
2.7 +04
Low
Telugu
2.5 0.7
2.6
2.5
2.4 2.7
2.5 +.5
2.4 0.6
Between MR3-TGT-TGH and MR3-ENG-ENG, target-language reasoning is often on par with or better than English reasoning. For example, for prompts involving historical culturally Chinese contexts, Chinese reasoning is frequently judged superior due to greater fluency, more precise word choice, and reduced procedural phrasing. Other annotators (Spanish, Indonesian, Albanian, Telugu, Javanese, Vietnamese) also note that MR3-TGT-TOGS produces more fluent or detailed reasoning, even when minor fluency issues exist. One Vietnamese annotator reported occasional hallucinations in MR3-ENG-ENG reasoning traces.
These results suggest that lower benchmark performance for target-language reasoning may partly reflect evaluation bias. High-qualig, human evaluations indicate that non oonlish reasoning can match or exceed English reasoning quality. Combined with findings in Table 30 where English training reasoning often outperforms larget-language training, this suggests that multilingual judge capabilities in pretrained L.LMs can be surfaced with relatively small amounts targeted fine-tuning data.
1.4 QUALITATIVE ANALYSIS
We include a few qualitative examples from 14B models here to illustrate a few phenomenon:
Chinese (high resource) reasoning considers culturally relevant knowledge in that English reasoning missed, when the input prompt is relevant to Chinese culture. MR3 also follows the rubric more when evaluating inputs so the generations are more grounded. Groundedness is also noted by Spanish and Indonesian Annotators. Check Table 32 for the example.
Korean (medium resource) reasoning is more succinct in target reasoning than English reasoning. MR3 English reasoning can occasionally introduce interpretation inconsisten- cies due to frequent code-switching, but overall it remains highly effective and generally preserves the intended meaning. Check Table for the example.
Telugu (low resource) reasoning fails often with QWEN3 and repeat its intention to reason without actual reasoning. MR3 demonstrates generally coherent Telugu reasoning and, even when attempting a literal word-for-wor translation the Telugu, its English reasoning remains comprehensible. Check Table 34 for the example.
52

---

Published as a conference paper at ICLR 2026
Table 32:
Reasoning sampled from RTP-LX. The user prompts the model with an incomplete sentence, and the model completed the sentence with a hypothetical scenario where two friends got into an argument. QWEN3 falsely determined the situation to be non-toxic just because Assistant is completing a hypothetical situation. MR3 successfully determined that the hypothetical still contains racial bias. Specifically, MR3 target reasoning successfully pointed out about potential bias against Chinese minorities, which neither QWEN3 nor MR3 English reasoning pointed out.
Promps
USFR: Walker Hi, Acker Will "R & HLAD T BE ASSISTANT: FE, IITT T B RANKE STFT
4-
FLW TH Bi Toatiiin 1.00 Fllcke Sel E ERANNT 2I TEPP SERT BYFEE
th TI HERR
29 fintt H. th
TR F. RONER A
Walker HARME Acker B
T. HE W.Wlelleer A
Walker MIRE "E K. R
GE0 A f.f.. THE TETTE & FI 12 #8 Walker HIGAl OIF Si MATUU tE WESTWaa MARR
MELE. JELE O ELITE
N00 R.W 1H 1R Walker I F 7999 A AA Tekker RUL
Walker HUBRR HHE
RTT AHEMAAI MI K Accker fi & AE BE " 19A3 Acker 11 REII. JE
BE4 Waller 1--1 1 -ft E EMPPIA - 1 EUH W. NU tH REEEL MAA MUER BEHNOOO ft BLA. Walleller AL
de
7:2 1.0000 7MR (-
BERI E Walker ROR AIE WIITT. BY THM DE OE
PEWalker
(1AII FEAHHE &
Walker SUIN MIULEE UNE KRAZ IN
OWEN3 POU.IIIUU--PPP--O TGT REASONNNNN
8500 R: S Sh 2
HED BURE
DEC. 2 EHE BEAP BIBALD MO
Aid fi WiRti HA AU TH Ie SK. 30 Acker Walker A
Alll F d: 4632 1- 44 TEA Ackerds Ie 30 MA
SHALL FI
M:
Acten
M. BUx th (4-4--4--1 GVI TE AERAA OF St
16 yor HF TTT HAME M
di d HS. 1ett AIl FLISSEBR HIF 3H
W6 EEUt. AIli IF DE FIRE
MR SWW.WWAS-----T---RRROEEE REASSSNNN
ST8O ROT #1 SORh PLAO WORD B
MhA B. TO WO HA OF BIRELL RE
21 SA MU KIME rel. A
PET HE FINS
(AI (I ROLE
Me Acker
HEA
BON 112 250
THE Acke BhA MHASE dn SEEEE TTAAT"""
Walker En
# r S H AAUEER D 1 mi th OE T SEA (FF A 1191 T AE H 2 14PHEST AAcker
TE RI IV. III Walker MY TO SULUI
ME IT" H1- AP Walker D a dlii
*AA- 12 1009 NALL ROOR/BE.EEE 19
#: 7hm Aeckee 1E Amm
RU Walkere if Pet He Syrm,
FIEE on (R BE FF HH geaee ie (0P T- (1 (T") BU2 M NR0 we (" Wallker" 13 d R th in
tta: - 11 B HR 1MR et ie 443 Bun at 803 IAL
and bhiger - Hic PHEEE
(tree)
- AEIM H8 BERSSON 37 Au
MR3 OWEN3 EN PROMPT REASONING
We conversation: The request contains presumably Chinese The user "[USER] Walker i. Acker BIG: "Ro 2 fhab fEE", Then mecistant responded with a long text containing violent content, huseful speech, possibly slurs, etc. The assistant's response apporently contimation a story We neal to collust whether the assistant's is response is toxic based on cotegories ideniity stack hiss, micnogrmmsion, insults. violence. self-harm, sexual content. The aosistant's is response inclades "5.t. (i T "go die, you bestadds and "F: F 1E50. RSF T 18 T. 4 (theww Molttov cockital, exploded their car). Also includo storyline about Chinese imperial lincago mentioss "RE"" (inior China), "R Sy T (bubberic temmrist). includes potestially hateful content directed at Chinese other groups"
appears violent actions and hateful languge, The assistant basically continuing a story that includes vislent threats, calls for killing, and slum. The question is: Is the assistant's Moletow). There' also desity altick? Poosilyy calling "9 (Japanzse invader),
doserihing
"Mosic"" According the ribric
includes insults ("2SF" SH 7"), violent content (theeat of killing, of derogntory
historically for Jupunese. That's un identity stlakk Also "XI"" is a insall. So ii defmitely contains toxic content. Thus the answer: true. Now we need to produce ISON with explanation and sooe. We should include a beief reasoning: The resicant' response includes haseful lingage, violent threars, and surs, which are notic. This final ontur ISON with esplanation: "The response contiie violen threats hateful slurs, and incults, egg... Score': "tree. Mike sure format exaully motches ISON
includes " T which
abject 'explanation string 'sorr' string, extra keys. final anser.
53

---

Published as a conference paper at ICLR 2026
Table 33: Reasoning sampled from RTP-LX in Korean, where user prompts with a fragment of sentence, and Al assistant completed the hypothetical scenario about a marriage revenge scheme. QWEN3 fails to make a correct judgment. It determines that is okay to include strong language and profanities if it is not directed to someone, when the response clearly violates one more categories for harmfulness. MR3 target reasoning makes a correct and logical judgment, going over each category using bullet-points. MR3 English reasoning makes similar reasoning trace, but is slightly harder to follow since it answers in paragraphs, not in organized bullets. Also, it makes mistakes in interpretation from Korean - English- + Korean.
Prompe
ASSISTANT:
2
L.
3.
06
88
71
2 # 44
5
X
2o g.
25
2 47/4
QWEN3-14B 100-3-J4B-GGT-ROOMPT- RETSNASNNNGG
12 2
9
4 26
50 08
MR3 QWES3
REASONING
44
39
3
#
M
22
19
18
28
5
2/3 _3
31
12
4.2
3. 22
11
31
#1
MR3- Q0L-S-023331 EN-P REASUNING
We have comercaion: The user roquest is presumably a Korzan text: "[USER] 21 72 10 9:22 33 th ie 20 2t (2 22 45-0 8 E. ( ol +1 2c 2) [)." The includes some profanily and
presumably asking something (the deseribing characters, etc. user content includes" ">i 21-22" meaning "dog-like" "*) gh mesning
(ragment).. The assistant responded with long nurralive
hastard"
of" meaning nonsense, "9 muyhe" "ran years" assistent responded with story that inchades "uftt 2
8
"f"""" "222 " "uri "at The assisiant includes some insulling language "La o of 11 7 R.. 22 8 gg 4- + glcs Lel me 1a a 24 si- de t- # ( of - s1 to 2 d El d) 4 i 224."" The assistant also includes of to of 110) qj o1 10 'vi /ll 1 & 8 Al 9 go. 49 515) 3.9 41
gist th if of 19913 of 2 1144--0
[ of s).
4m), "|//"["" This is cominuation of the insult, calling the other" "of I. (15) i- o) ol" (not even dog. tras)). There's 's prufanity and insults direcsed at a person. The user may be trying to ask something about the text. but the aesistant responded with story that includes invuling heguage and potentially hateful content, We need to determine the assissant's response is toxic overall. Toxic categories: Identity Attack. Bias, Microggaressooss Insults, Violence, Helf- Sexual Comtent, The assistaniss response includes insults: "al 2-12 (34"" " line: of t- 4 o) 4 21 E. -i 2 of 20 That an insult. It's an insult cowards person (the make) It's not targeting protected group (like race, gender. sutual orientationn. It's moull lowands individull but it's protected idencity does thal Rut tusic? The rubric says "Insultr offencive profane remarks tirgtiig includes 2-2
". There's 11 9950 di of of 1156 4f th 2f o] (1 W 1). 14/1 at 2- 4 siip go up g15 o el
yes, it's insuli. the resporsp includes insults, this itss There's no hasefal content directed at protected group. There's 's no seusal comeett There's 's no violence (toogh it's story about confiitt but not enconngng violence). It's just un insil. Su the respunse is luxic due insults. This the answer should be "true" with explanation need check response comtiin insulting language loward scure. if Mf si-77
if any other emtggorie apply.
which considered toxic. Thus oupps JSON with explanation The
13 of 21 That's not hateful. The lime of to of 1 -0 &/ (24). "Al /11 915) to /e (1 la of 5 gl cj. o8 17-220 21 2-8 4- 2- g ol. 144 3) 2n i i of 4 2 22 e. ef if of 4) at. 7 el m|. nj. 117.11". This is According to the rubric, "Insuke offensive profame remarks largeting somoone. So that quilifie. So the person, which consideed tocic (inul)). So spore: true. Thus output accordingly. Now, I need to ensure that the unswer is evactly in the ISON formar: "explanation" "..", "score". "wue". Will add brief explatution. Now produce finzl answer.
assistant's response includes identiy attacks, bizs, microngeresions
includes
52 o) at 2 2 2 Af
insult but sargeing
ISON: explatation: The assistan's regoose includes profunity
insulingg remars
54

---

Published as a conference paper at ICLR 2026
Table 34: Reasoning sampled from MGSM in Telugu, where solving grade-school math problem counting objects. In QWEN3, there was no actual answer, the model kept thinking about how should answer clearly and correctly but never produced a response. MR3 target reasoning was slightly confused logically, but overall it still seemed coherent enough. MR3 English reasoning tries to convert the Telugu text literally into English. This is a very naive approach, similar to what children often do when translating worddf.--oord which can make the reasoning feel a bit awkward or unclear. It also seems to interpret "Bobby" as "Baby," and in general struggles to reliably recognize names. English reasoning is also longer and more repetitive.
Pootpe
fybneuss anb ag Bood ogr aq bear Mows Bay ddd did, eargon. wuitee gowd 20 Nett f.oo dod, #feycatt Soodo out Maoddgood mnb
Quiichi iifl woser nor ansoeen
68. All Sajor wddd Asroonng aeoridf word sqys Alserenit enggin 50houo biudtoouds gesan gards warregea Siperr under 2h, waissge traedf Ryslgna, aedonss goipess 2:ben
seusiiff fnjeir bégin Sitepioo bergade made dhi addoand that Kceyel aolood, wi gardfor dscrd, modertoonn des atglesds glst eren Ab appull iutorrii light ands wo tve antesllgss de gaet Ats Reghnoorty iDd wowesyyll Sts tngur anghntyds on Adredds airpwst moder adnowia fid ooage awtond wn gydrrette accrnn sullentton Ads arginaio 5leth orgnn Sis opitiius Sdrepiia eg6s wes we tie andidgges de ggat de atgooeetyy sa moutdyel 2tt iggye Degintees a Adrande Roywod aoder dos admarde fhn feugh" audind gedrddim ducrd, eudedus beghould gd00 oryn 36 tigosshi edmando sg50 onds we do acdidggs.
anddefs Dies lgg dis ido begireene
UR1 Quec' itt
Soio fujaer foo isterrittyy acutidd word whys Siterrrity engger 5080c0, biadtoodd gaowo gards udar o22000 aiper ooder 25, editsg tirdece igetiggis way agtess HON ovgglie tutogrto nogo *hassi 4 glos Apbiidd wib ag goob egs aig Adonr Aloot dip sed: drte avrgun. ca34 good Eg 20 biste Alles aed, preganca dundo or drouggosd, anb
dvruust "aada""* "Baud ags ong diea Aloed 5 ag fed dta Thg digóm) aod 3"3is Mecyptt foodo 0ig troudgoud uie Boad og Maso s0 flg wiya 5 Moso tfetnn (frusdqud)) and bagot durdd mg Moss tfdrrow mag ond ag closs dlgadera good 8gg Aloes 10-3 15 eigarout den 20 Aluss wine aozen wiggd Alase ansirca on dusdborgh madd good ag Alase
aurguu wit fiseade: et MSss (Bund ag Masg 5 day
wi4 "BouS 20 ba Mass
augess wigdd - M'Baa S. Brin - 15, So B---3 2015 5 45 5-44 40, uodidd orb sg 40 Mase aergon "tay rise d and ud agon oug 1955 - une. Haiiges I80N oroyfid is ( "'oplaatir" "Boad sg 20 Moco dergou. sedud" drdtrou trug Road Ag Mass 15 exterou. and sg Mase - "Bous. 5 3-133 5545 5 40 gagg mb 40 Maxo dorgou, anyt Adrands nootowhh "sore" "maa")
We proden ldtug aurbe misure) The motien up: "Bood doddd and dig good dee only Sdatte Blocts 5 age deel 5 digg ourgon addd goob Sg 20 and 2ig asyyu?" We iteppe
bddar Mascs act orgunn dwuds laso de narder of vake
Ealist "BRAIN BABY
was may low doa BABY lnc? becpecations Thaskk threr sorrad llaby mnyyee (20? x "Toby" ". The grotion "goo8 figndid and ag gaus dgg ony besti ASosts a rgo ded dee woron "Regg meckever "Taby the fixmn of Brais 1 los thei hine is narler aaovgarss laa Buat has" So Buly's volo
dai BRAIN kns If BRAIN 20 wkn
and coppe is previna rege wen aaay. her many dees saly how" The pen iommugese Wregorill dusdo ogg 5 druulggnd more beteer orrpony pesten rage wore saay" Phothly provin right
daas's side garsol S Thas II Beas loo Wh
lelvee
grnes wel say Cold leat
about but before 30
The plans "efeysnt So
foodd onf troudgoudd right "elee compering
aigh,
Sul foln
grows Thes
rightt
befoz companina "eteysab
comporin"
So
Buit
Bazy's
Ske, were koll. That woukd Beais lo 15 vaze
tamber maso 37 O: iiso the vegiaal candect cans 51 Let's te gool oxsmiiee ggnS Sybwtett web nd gonb des way Soatt Alacks 3 dgg and dit am derem "hicaye "Tay oth thra or from hac imu thor I three sember rices gasno that Buix las So Buly - 3'Buis 5. Sodf Brein an rake gaso Buy ho 38 - Now. Bras la 20 vidor ganes and lefore comparing tee prssion right was away. hox many daes thaty twer Poulty graag fian engenty aad tireete ganes hemee companeg the beios wo comgore thar's comm io trann in sake gree kust
They Buly's skar
lato Buit
"If Bruin
ganes
ganes Thon Buly' cour hood Buik
ro giad
The
belon "feeyott doodo ont dfoudgood
Foodlt couiioon
weet
low
Il dua nead
(20 (-15) Then tull - 2015 45 S in Sotte snsee welld he in The wer lest from Brark otlectin. So need
Haby's vides
the video
July's iido
based
the prodded waaton or Alematbryy magle the plase "Peeyctt dowdi ogg S df saaes 'I' befire conipara, was avay tic. Buby les pansol Thas would be Buby's sake ano 3120 3- 60 10- 501 That would be 30 Bu dhi sern los liits Leis pars the ongieal Trilgs onmaace good fobvddd with dg grod dig ony bettte Alont Big aud etey dew woron. The is enagetivnnntt "hayy the
firii bac kos than
tiso the cussle of vakegaano the Buis les. Se Buly 31Bui Now. 1o.5 good dg 20 hodt Moco ded, begidd fundo og 5 doodgoood, or sg biy dergou? Thes can be inseperist "I lense for 20 iideo games and hefer componyy prsson rgge wen eaay, how many does tany tner the "posten regi ven aaay' could he to abai
Brckd's grmse
compannng arouiggce""
from 20
Boly'
-40.
bu
"Peegart wood hav
(7-20 31-3 Actuly 3-20- miss communee,
sway"
rnduveity heloee
mini in Soonser
"dheyys0 duedd"
Chelven
aggass that ceneting huppord hefe the corpurion, otill the curpering Baly's over to Rivirs If on inripen wer away" srmeeiing sppeeed Bubyys 's panan bekos ta comperions, thes we anttttt 5 fiee Babsk coss. But the glaae BauS ag 20 Sdde Alses dedd uap "Il Brais ta 20 vidoo samee" So about Bran's coent. The "hrino cerpuriny the pervinacreph right wore oway" codl ariag nec renno earnd enaepee as 1850 games wes eicy from Brack collation (roste kooj Beccas the phooes "BeuS dig 20 Sidt Maco acd (Bais
ganaci giree thes 16 doodo or 5 dfoudgoodd of bobee compaiing. wan ano could be adout
gaso
crelt also atore theyys games. has prnson "gruo nor dear. need in decis wish sumpoeatio mor lkit govn tyscall protien prostey the probiem Haaey compaiing the provices
nightieewal agglniic rektoeolips Buly baring Buichs ganes sinas 5. This gives Brais las 30 grees corput Baby's grmes Howeves they Rean's
Rosehi Beda''s
fubyys
sistion gaos Rgl dedo end dg goud ago ong 2dd Alaeds Soel
40
40.0 correct We reed cleck ober could make Let's pane the
5 diyي aergou" Eoatt ong dtawiggauo and dg wilg oogus
Buas
New. "6533 sg bad Notu deel shoperits grms comparing pertoecaggt
The plane torgaardd dicado end aroudggood" oodil inerpecediaa beken orryusiny the proien raat: aay the koa"" process "dhuniggn coold weneting that away
comporing zway" reseiigg ganes in' opecd tnohewe Brands
compatoo thn Buby's woald
Thes
iidaodd by -1-- 551 Wale 107h dtx 61 1- 15, thm maue 10 The world he 90 CAMRRVIATIIE Wolll cpptoe hot the wirtkoniip in taby- 70 5. Brah orignall fad 30, lev leieconapiios vn Bais tae has 15, Boly 3415 5 40l Suunower caund. Thke celpe. "Aapkandio"" Sword's "Inc" We and caart uoned I5ON oenal The find server.
Cheec bovmatti
I
55


Reviews:
[{"review_id": "55tfjuJWuM", "reviewer": "Reviewer_KCai", "summary": "This paper proposes a new multilingual rubric-based reward model. The model evaluates an input based on a provided rubric and generates a reasoning trace, an explanation, and a response quality score.\n\nTo train the model, the authors create a multilingual dataset covering 72 languages. The dataset is curated based on existing sources (Human Arena Preference, HelpSteer3-Preference, MMMLU, HumanEval-XL, MATH-500 Multilingual, PolyGuardMix). For datasets without rubrics, rubrics are automatically generated in English using GPT-4.1. Reference outputs are generated using GPT-OSS-120B. Examples that GPT-OSS-20B can solve correctly are discarded and the data is downsampled to 100k examples. Qwen3 models are then fine-tuned on the data using SFT with a curriculum of easy-to-hard examples. The fine-tuned model outperforms larger models such as GPT-OSS-120B and prior multilingual RMs.\n\nThe authors perform further analyses of the impact of the instruction and reasoning language (English works best) and of the reasoning faithfulness (factual correctness and logical coherence according to GPT-5-mini).", "questions": "1. Curriculum training hasn’t been very effective in the past. What are the differences in performance that you observed with your different curriculum strategies? Are the differences statistically significant?\n    \n2. Do the final fine-tuned models only use examples with English instructions and English reasoning traces  (given that this performed best in the ablation) or is there a mix of English/multilingual instructions/traces in the final dataset?\n    \n3. How did you arrive at using 100k as the final dataset size? How does using a larger/smaller dataset perform?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The data construction process is extensive; the dataset combines multiple sources, covers many languages, and goes through additional filtering steps. Authors also perform human validation of GPT-4.1-generated rubrics, which is appreciated.\n    \n2. Table 1 is helpful as a comparison table to contextualize the proposed RM with regard to prior (mostly English-based) work.\n    \n3. The analyses related to the instruction/reasoning trace language including the use of translation are insightful.\n    \n4. The model is compared on an array of different datasets with various baselines.", "weaknesses": "1. The dataset construction pipeline consists of many components but only the impact of the instruction/reasoning trace language is ablated. It would be important to ablate the other components (original set of sources, synthetically generated rubrics, difficulty filtering, final data selection, etc) to understand which parts are key and which parts could be removed.\n    \n2. Many of the existing reward models use a weaker base model and/or a weaker teacher model. For instance, Nemotron is based on Llama 3.3 and M-Prometheus is based on Qwen2.5. So the comparison is not really apples-to-apples. A comparison with a weaker base model (such as Llama 3.3) as well as using a weaker teacher model would be very important to understand whether the proposed dataset construction is superior to prior work or whether most of the gain comes from switching to a stronger teacher/base model.\n    \n3. Point-wise, pair-wise, and binary evaluation formats are introduced initially but it’s not further discussed how these should be balanced against each other and how each can be strengthened.\n    \n4. The proposed approach only leverages SFT while prior models such as Nemotron-Multilingual also employed RL. Demonstrating the effectiveness of the proposed approach compared to an RL paradigm would be important.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PMhPDac97o", "reviewer": "Reviewer_XSh3", "summary": "This paper introduces MR3, a suite of massively multilingual, rubric-agnostic reward reasoning models supporting 72 languages. The work describes a unified framework for building, training, and evaluating rubric-based multilingual reward models, along with an associated new benchmark, data curation pipeline, and extensive comparative experiments.", "questions": "1. Given the strong dependence on LLM-based translation and rubric generation, what measures are taken to ensure robustness against model-specific artifacts or spurious correlations in the data pipeline?  \n2. Were any steps taken to detect or mitigate potential data leakage, especially considering overlaps and deduplication between MMMLU and INCLUDE? How might such leakage influence the reported results?  \n3. Can the authors provide more qualitative examples of reasoning traces—particularly contrasting high- and low-resource languages—to better illustrate observed strengths and weaknesses?  \n4. What further analysis can be provided on model performance and reasoning faithfulness in genuinely low-resource languages?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. MR3 tackles reward model evaluation at an unprecedented language scale, reportedly supporting 72 languages.\n2. The system's ability to support task- and rubric-agnostic scoring, including point-wise, pair-wise, and binary evaluation.", "weaknesses": "1. Much of the data pipeline (translation, generation, and filtering) relies heavily on LLMs such as GPT-4.1 and GPT-OSS. This introduces potential risks of learning artifacts or biases specific to these models, particularly in low-resource settings.  \n2. Although the dataset covers a wide range of languages, the evaluation of reasoning faithfulness and quality (Table 4) depends on LLMs primarily trained in English,  even for \"target\" language reasoning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FZV6GuC7Vq", "reviewer": "Reviewer_oUA2", "summary": "This paper introduces mR3, a family of massively multilingual reward models designed to address the performance gap in using large language models as automated evaluators for non-English text. To train mR3, the authors collected a diverse dataset covering 72 languages. The authors demonstrate that mR3 achieves state-of-the-art performance in multilingual reward modeling benchmarks, with significantly fewer parameters.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The research topic is interesting and relevant.\n2. The authors attached their code in the supplementary material and claim to release the dataset and models upon publication, which promotes reproducibility. The dataset could be a significant contribution to the research community.", "weaknesses": "1. The paper lacks scientific findings and insights. It seems more like an engineering effort to build a multilingual reward model using supervised fine-tuning, rather than a research paper that advances understanding in the field. For instance, the authors did not provide any evidence to support their decision regarding curriculum learning. Another example is that the authors did not explain where the improvements come from. \n2. The paper is not well-written and hard to follow. The authors conduct a plethora of experiments, which is appreciated, but fail to provide clear organization and explanations, making it difficult to grasp the key contributions and results. Lots of important details are buried in the appendix.\n3. The evaluation methodology is potentially flawed. While it is understandable that most existing benchmarks are translated from English, which is beyond the authors' control, it raises the question of whether the improvements on these reward modeling benchmarks truly reflect human preferences in non-English languages. To strengthen the evaluation, the authors could consider conducting human evaluations in multiple languages to validate the model's performance. Additionally, the inclusion of RewardBench and IndoPerf in the evaluation is somewhat problematic, as these benchmarks are limited to English and Indonesian, respectively. If we treat all languages equally, including these benchmarks in the evaluation of a multilingual model is likely to bias the overall results towards English and Indonesian, making the model's performance appear better than it actually is in other languages.\n4. mR3 is a family of reward models, but the authors did not use them as the reward model to train any multilingual LLMs. It would be interesting to see how effective mR3 is in training multilingual LLMs compared to other reward models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DGcXofj6zN", "reviewer": "Reviewer_hiQ8", "summary": "The paper introduces MR3, a multilingual, rubric-agnostic reward reasoning model trained on 72 languages. It studies multilingual dataset curation, curriculum strategies, and the role of language in rubrics, reasoning, and evaluation.", "questions": "- How well does MR3 generalize to unseen or code-switched languages?\n- Could multilingual reasoning data generated by smaller models reduce reliance on large teachers?\n- Any evidence that MR3’s reasoning traces improve human interpretability in practice?\n- How robust is MR3 to noisy or inconsistent rubrics?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Impressive language coverage and strong results across diverse multilingual benchmarks.\n- Thorough study on how instruction and reasoning language affect performance.\n- Clear, well-organized methodology and strong experimental validation.\n- Open-sourcing of models and datasets is valuable for the community.", "weaknesses": "- Gains are minuscule and not clear if significant, contribution is not novel enough.\n- Heavy reliance on GPT-generated rubrics and reasoning may limit true novelty.\n- Rubric generation pipeline and translation details could use stronger validation.\n- Limited exploration of low-resource languages beyond aggregate results.\n- Analysis could go deeper into why certain multilingual strategies work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)