RADAR: Reasoning-Ability and Difficulty-Aware Routing for Reasoning LLMs
Authors: Nigel Fernandez
Branislav Kveton
Ryan A. Rossi
Andrew Lan
Zichao Wang
Venue: ICLR 2026 Poster (2026)
Keywords: routing
adaptive reasoning
item response theory
reasoning models
large language models
Primary area: foundation or frontier models, including LLMs

Abstract: Reasoning language models have demonstrated remarkable performance on many challenging tasks in math, science, and coding. Choosing the right reasoning model for practical deployment involves a performance-cost trade-off at two key levels: model size and reasoning budget, where larger models and higher reasoning budgets lead to better performance but incur greater cost and latency. In this work, we tackle this tradeoff from the angle of model configuration routing for different queries, and present RADAR (Reasoning–Ability and Difficulty-Aware Routing), a lightweight, interpretable, and scalable routing framework. Inspired by psychometrics, RADAR learns an item response model from model responses with different budgets to different queries, with interpretable parameters including query difficulties and model-budget abilities. RADAR then routes queries with higher difficulty to model-budget pairs with higher ability, and vice versa. We conduct extensive experiments on 8 widely used challenging reasoning benchmarks, demonstrating the superior performance of RADAR compared to state-of-the-art model routing methods. RADAR also exhibits query generalization capabilities, achieving strong performance on out-of-distribution queries on all benchmarks. RADAR is also scalable and can efficiently integrate additional models by dynamically selecting a small set of evaluation queries to estimate their abilities.

Published as a conference paper at ICLR 2026
RADAR: REASONING-ABILITY AND DIFFICULTY- AWARE ROUTING FOR REASONING LLMS
Nigel Fernandez"), Branislav Kveton'. Ryan A. Rossi Andrew S. Lan', Zichao Wang?
University of Massachusetts Amherst, 2 AAoob Research
{ni onigel, andrewlan) @ccs umass. edu, ryrossi, a [ackwwa@@ddobe.com
ABSTRACT
Reasoning language models have demonstrated remarkable performance on many challenging tasks in math, science, and coding. Choosing the right reasoning model for practical deployment involves a performance-cost trade-off two key levels: model size and reasoning budget, where larger models and higher reason- ing budgets lead to better performance but incur greater cost and latency. In this work, tackle this tradeoff from the angle of model configuration routing for different queries, and present RADAR (Reasoning-Alil Ability and Difficulty-Aarc Routing), a lightweight, interpretable, and scalable routing framework. Inspired by psychometrics, RADAR R learns an item response model from model responses with different budgets to different queries, with interpretable parameters includ- ing query difficulties and budel-budget abilities. RADAR then routes queries with higher difficulty model-budget pairs with higher ability, and vice versa. We con- duct extensive experiments on 8 widely used challenging reasoning benchmarks, demonstrating the superior performance of RADAR compared to state-of-the-ar model routing methods. RADAR also exhibits query generalization capabilities, achieving strong performance on out-of-distribution queries on all benchmarks. RADAR is also scalable and can efficiently integrate additional models by dynam- ically selecting a small set of evaluation queries to estimate their abilities.
INTRODUCTION
Recent advances in large language models (LLMM)) have leveraged reinforcement learning (RL.) (Shao et al., 2024) to train models to reason using chain-of-thought before generating an output. These reasoning language models (RLMs) (Yang al.) 2025] Guo al.] 2025 OpenAl & et al. 2024) have demonstrated impressive performance across a diverse range of challenging tasks, including math (MAA 2024), science (Rein al. 2024), coding (Jimenez al. 2024 vi-
sual perception (Lu et al. 2024). and tool use Yao et al. 2025 The excitement has led to flurry of new open-soures and proprietary RLMs; for example, Hugging Face already lists 4, 492 RLMs as of March 8th, 2026. These models have varying sizes, specialize in different domains, and offer various configurations, including reasoning budgets to balance performance and cost. For example, OpenAl's reasoning models (OpenAl& & et al. 2024) have "low", "medium" and "high" reasoning budgets for developers to choose from depending on their application.
Always choosing the "best" and most expensive RLM configuration with the highest level of reason- ing budget is not always the "right" choice for every query: for some simpler queries, there might exist a "worse" and cheaper RLM configuration with low or reasoning budget that correctly an- swers the query, resulting in significant cost savings without sacrificing performance. Indeed, we empirically observe the same phenomenon in Figure 1 where we show that over 50% of the queries from MATH-500 (Hendrycks et al.) 2021c) can be solved using an RLM as small as Qwen3-0.6B with minimal reasoning budget (measured by the number of reasoning tokens). On the contrary, some challenging queries require a much more capable RLM with a high reasoning budget. Strong RLMs can also "over-think" which could hurt performance even for simple queries (Su al. 2025) Hassid et al. 2025 Hong et al.) [025] [Shojaee et al.) [025)] Ghosal et al. 2025). This performance- cost tradeoff presents challenge for practitioners: how to choose the "right" RLM and its configu-
Work done during an intersship Adobe Research.

---

Published as a conference paper at ICLR 2026
100
Average Performance (%)
Alkiange
80
Pass@1
Grade
AllSmal
Random-Al
40
RADAR
Owan3o.00
Owinn-1170
Allmmall
BouterBench
5000
7500
20000
12500
15000
0.0
Reasoning Budget (Num Takens)
Total Cost (S)
Figure I: Left: Our pilot study on MATH-500 (Hendrycks et al. 2021c) shows performance dif- ference over (RLM, reasoning budget) configurations with the smallest RLM already solving over 50% of the queries with minimal reasoning. Right: RADAR exploits this performance differential by jointly estimating query difficulties and configuration abilities, and routing queries to sufficiently able configurations, thereby optimizing performance-cos out-of-domain queries from FRAMES (Krishna et al. 2024), RADAR can match 90% of the perfor- mance of OpenAl o4-mini with high reasoning budget at just 10% of its cost, with the next best method (Song al. 2225)) requiring 30% of the cost. Solid lines in the figure denote the convex hull corresponding to each curve.
tradeoffs towards the Pareto front. On
ration (e.g., the reasoning budget) that is sufficiently capable of correctly answering a query, thereby maximizing performance while minimizing cost?
In this work, we propose framework entitled RADAR (Reasoning-Abilit and Difficulty-A Routing) to address the above challenge. Given pool of (RLM, reasoning budget) configurations, a user-desired performance-cost tradcoff profile, and a new query, RADAR chooses the optimal configuration for the query according to the tradcoff profile. RADAR is lightweight and efficient: decides the configuration in real time (~7 milliseconds latency overhead) at the query level (it performs the assignment before the RLM ingests the query) and does not require model switching during generation, thus avoiding the need to re-query the RLM multiple times or recompute the KV- cache that could occur for cascading-based routers (Chen al. 2023; Zhang et al., 2024). RADAR is also designed to be blug-and-play it treats RLMs as black boxes and uses them as-is without the need fine-tune them, which is convenient for practitioners who can use RLMs and their configu- rations in a standard API call. When new RLM becomes available, RADAR can rapidly include it into its pool of (RLM, reasoning budget) configurations available for future queries.
The key enabling ingredient in RADAR is a custom item response theory (IRT) model, a classic technique inspired by psychometrics and educational assessment (Rasch) Lord] [012] van der Linden & Hambleton] query difficulties and RLM abilities at different reasoning budgets. Specifically, we first perform a calibration step, where we collect evaluation responses of each (RLM, reasoning budget) configura- tion to a collection of queries. We then model this evaluation response matrix via IRT to estimate the latent ability and difficulty parameters. To make this approach generalizable (OOD) queries, we parametrize the query difficulty using a learnable vector that, when multiplied by the query embedding obtained from an off-the- thel-shelf embedding model, yields the query difficulty. We also parametrize each RLM configuration with learnable, scalar-valued ability. To include a new RLM configuration in RADAR, we estimate its ability by evaluating it small of dynam- ically selected queries, employing a classic technique inspired by adaptive testing in educational assessment (Wainer et al. 2000; Hofmann et al.) 2025). These design choices enable RADAR (1) handle new queries in real time and (2) generalize well to new RL.M configurations.
DeMars 2010). We use IRT model to jointly estimate interpretable
out-of-distribution
We formulate model configuration selection as multi-objective optimization (MOO) that searches for the configuration at the Pareto front of the performance-cost tradeoff curve using scalarization techniques (Micttinen) 1999). MOO (Keeney & Raiffa 1993) [Emmerich & Deutz 2018) a well- established framework for optimizing multiple objectives, with major applications in engineering
(Maleer & Arora 2004). product design and manufacturing (Wang et al., 2011). and economics Ponsich et al. 2013). This work is the first application of MOO, beyond linear scalarization, to
LLM routing. We conduct extensive experiments on widely recognized challenging reasoning benchmarks. RADAR demonstrates superior performance compared existing state-of-the-ane rout-
2

---

Published as a conference paper at ICLR 2026
Query Shev Difficilty
Calibation Queries
Query Diflinally Extimaiion by Kg
Model Budget
Purcto Frout reesoring budgett Choiees
Auw Mhr
IRT
(RLM, reasoning budget)) Abilay Estimation 9,
3-9148 (UP
Pofomuacc
(RLM, reasoning budgec) Ability Estimation 6
0.68
Oprimal Allecation
Regid New (RLM, ressoning bradpet) Calibcation
with a 0.9
$7.0.8
Cost
Figure 2: Illustration of our RADAR framework. Left: RADAR jointly estimates interpretable query difficulties and RLM configuration abilities using IRT (simplified for illustration purposes; full de- be rapidly added by estimating their ability on
tails in Section 3.3). New RLM configurations a small subset of dynamically selected queries using adaptive testing (Section 3.5). Right: RADAR formulates routing as multi-objective optimization and routes queries sufficiently capable config-
urations, optimizing performance-cos tradeoffs towards the Pareto front (Section 3.2)
2021c). RADAR can match 90% of the performance of OpenAl o4-mini with a high reasoning budget at 1.31% % of its cost. RADAR exhibits strong generalization to OOD queries, including long-context multi-document QA (Krishna al.) 2024), despite being primarily trained on shorter queries. Further, RADAR scales and generalizes well to new RLM configurations, showing an improvement in routing performance. summarize our key contributions below.
ing methods. For example, on MATH-500 (Hendrycks
[C1] We cast adaptive reasoning as routing over discretized model-budget configurations and select configurations via a Pareto-optimal performance- cost objective, all in black-box setting.
[C2] RADAR adapts item response theory to learn interpretable query difficulties and configuration abilities from data, enabling low-latency routing and generalization to unseen queries.
[C3] RADAR supports plug-and-play integration of new reasoning models via adaptive calibration that estimates abilities from small, informative subset of queries.
[C4] Across 8 challenging reasoning benchmarks, RADAR achieves superior performance-com tradeoffs and strong out-of-distribution generalization, including long-context document QA tasks.
2 RELATED WORK
rapidly growing literature seeks to make reasoning models more effi cient: see Yue al. (2025) for overview. Methods such L1 (Aggarwal & Welleck (025) and S1 (Mucnnighoff et al. 2025) provide length control, encouraging shorter reasoning chain-of- thoughts that lead to correct answers. Others prune adapt the reasoning process by dynamically shortening or extending reasoning (Hou et al. 2025) Xu et al., [025] Wang et al.) 2025); adaptively controlling inference steps (Huang et al. 2025) and analyzing when additional reasoning is ben- eficial or wasteful (Su & Cardie] 2025] Su el al. 2025; Yu et al. 2025 Ghosal et al.) (025). Our approach is complementary to single-model efficiency: RADAR can include these efficient RLMs as an additional model configuration routing candidates. In contrast to static single-model tuning, which requires access to RLM weights, RADAR works in a black-bor setting, leveraging the com- plementary performance of multiple RLMS in a rapidly growing heterogeneous RLM landscape, and dynamically shifts performance-cost tradeoffs depending on user applications.
Efficient Reasoning.
Routing for Foundation Models. Recent work in model-routing (Ding et al. 2024 Ong et al. 2024 Chen et al. 2025 Hu et al. 2024 Zhang et al. 2025) focuses on model selection with black- box predictors or model-cascades (Chen et al., 2023). In contrast, we explore adaptive reasoning through the lens of routing over model-budget configurations of RLMs and provide a novel for- mulation of routing an MOO. Unlike opaque routing regressors (Chen et al.) 2023 Ding et al.) 2024 Ong et al. 2024). we employ an IRT parameterization to model latent query difficulties and model configuration abilities interpretable parameters. Compared to concurrent IRT IRT-based rout- ing method (Song et al.) (2025), our work (1) provides a novel problem formulation of routing, on previously unexplored reasoning models, MOO opening powerful toolkit of MOO solu-
3

---

Published as a conference paper at ICLR 2026
tion techniques like Chebyshev scalarization, (2) uses different IRT parameterization with fewer parameters providing an interpretable scalar-valued ability ordering among models and potentially requiring less training data, (3) provides new MOO-based routing performance metrics evaluating the coverage of the Pareto front, and (4) presents an adaptive-testing based method to quickly gen- RLMs for improved performance. We provide expanded
eralize the routing framework related work section and detailed comparison in Appendix A
3 METHODOLOGY
In this section, we introduce our RADAR framework for adaptive reasoning through routing RLM configurations. We begin by formulating adaptive reasoning as MOO, where the router selects the optimal (RLM, reasoning budget) configuration for chosen performance-cost tradeoff. Under this formulation, then detail (1) how to estimate particular RLM's performance for given query using IRT and (2) how to solve this optimization problem.
3.1 ROUTING-BASED ADAPTIVE REASONING IN RLMS THROUGH DISCRETIZATION TRICK
Unlike classic model routing Ong et al., 2024 Ding al. 2024p, which chooses from a set of dif- ferent base models, choosing the right RLM for practical deployment involves a performance and cost trade-off at two key levels: base models and reasoning budgets. We unify these decisions by discretizing each RLM m E .M by its available set of reasoning budgets E Um. For example, the available reasoning budgets can (low, medium, high} for proprietary RLMs (e.g.. OpenAl o4-mini), or a user-defined discrete set of values such as {0. 1 2k, 4k, 8k, 16k), for open-source RLMs. To enforce a reasoning budget on an open-source RLM (c.g., Qwen3). we count the number of thinking tokens during generation. If this number exceeds the specified budget, we append an interruption message (e.g. "output answer based on current thinking") along with the RLM's end of thinking token (e.g., </think>> to complete the thinking chain-of-thought and preemptively start generating answer completion tokens. Each discretization is referred to as a model configuration
g = (m, u) E G with g C UmEE Uwez... {(m,, 3)} being the set of all model configurations. Discretization helps enable routing in RLMs at the configuration level. We use the general term "configuration" here since, apart from the reasoning budget, RADAR can be used to select among other model settings, such as parameterizations of the RAG pipeline attached to the RLM or decod- ing methods employed.
3.2
FORMULATING RLM ROUTING AS A MULTI-OBIECTIVE OPTIMIZATION PROBLEM
We present a novel view of model routing through the lens of MOO, allowing to leverage effective solution techniques from MOO literature (Miettinen 1999 Branke et al. 2008 Murata & Ishibuchi]
(q),-... - 9s and a of candidate .9n }. our goal is to assign each query 1 E Q to the optimal configuration gi e G which maximizes performance and minimizes cost. For each query q (index i dropped for brevity), we define an MOO with two objective functions: performance and cost. The performance prediction function Py :g G- [0, 1] predicts the probability of correct response by running configuration g query q. Similarly, the cost prediction function Cg g + [0, 1] predicts the cost of running configuration g on query 9. normalized
1995 Zhang & Golovin 2020). Given set of queries
model configurations g
[0. 1). We formulate the optimization problem two-dimensional a-priori MOO (Branke et al. 2008) and solve it using scalarization
techniques (Murata Ishibuchi 1995 Zhang Golovin| 2020), written as:
g" arg max c() 9Eg
where is scalarization function. Scalarization aggregates the objective functions of an MOO to solve a single-objective problem (SOP), such that the optimal solutions to the SOP are Pareto optimal solutions to the MOO. We explore two scalarization techniques: linear scalarization (Murata & Ishibuchi 1995) and Chebyshev scalarization (Zhang & Golovin 2020).
Linear Scalarization. Linear scalarization Murata & Ishibuchi 1995) uses non-negative weights for each objective function of the MOO, and maximizes the weighted sum of rewards (or equivalently minimizes the weighted sum of costs). By using different weights in the aggregation, we can obtain different points the performance-cost Pareto front. The linear scalarization problem (LSP) of our
4

---

Published as a conference paper at ICLR 2026
MOO with weight vector w R2 is given by:
arg max W1Pg(9) + w2( w2(-c,(9)). 9Eg
Since we can factor multiplicative constant out of the weights, we use weight vector that sums to 1. Further, since we have a two-dimensional MOO, we can simply set w2 = 1 - wi Our LSP becomes:
LSP = arg max W1Pg(0) - (1 w1) w1)c((())
(3)
gEg
We note that Equation recovers the routing formulation presented in existing routing methods (Hu
et al. 2024
Song el al. 2025 Zhang et al. 2025) but arrived at through the lens of an MOO.
Chebyshev Scalarization. In general, if the Pareto front is non-convex, there could be points on the Pareto front that cannot be obtained as the solutions of any weight-paramcterized LSP. We therefore also explore Chebyshev scalarization (Zhang & Golovin 2020 which uncovers points in the concave parts of the Pareto front by formulating the SOP as a weighted Chebyshev distance to an ideal reference point. In our case, the ideal reference point has a performance of and cost of O. Chebyshev scalarization aims to minimize the maximum weight-scaled penalty over all dimensions of the MOO from an ideal reference point. The Chebyshev scalarization problem (CSP) of our MOO with weight vector w E IR2 is given by:
20
CSP q
arg min max wi P,()),, (1 un ())))(())
gEG
The weight parameter W1 controls the trade-off between performance and cost: a larger value wi means a preference for performance over cost by favoring stronger model configurations more often, while a smaller value of wi prefers weaker but more cost-effective model configurations. Given user-specified tradeoff profile with weight wi and query q. RADAR assigns configuration LSP 1 (or configuration g = CSP." depending on the chosen scalarization scheme) to maximize performance and minimize cost.
3.3 IRT-BASED CALIBRATION OF RLM REASONING ABILITY AND QUERY DIFFICULTY
A key component in solving the MOO in Equation D is accurate parameterization of the perfor- mance prediction function Py (g). which predicts the probability of a correct response by configura- tion y query q. We leverage IRT (Rasch 1960 Lord 2012 van der Linden & Hambleton 1997 DeMars 2010) that is often used to model student responses to test items, specifically the two- parameter logistic (2PL) model (Lord. 1951 Birnbaum 19688), to parameterize our performance prediction function. IRT assumes monotonicity, i.e., as a model configuration's ability increases, its probability of correctly answering a query also increases. The 2PL model takes query char- acteristics into account, difficulty and discrimination. Intuitively, a configuration' 's ability estimate impacted differently after it answers query correctly, depending on the difficulty of the query. Query discrimination encodes the varying rate at which the likelihood of a correct response increases with the model configuration's 's ability.
In RADAR, we embed query zj into a dy- dimensiiia vector e, using frozen embedding model. Leveraging the content of the queries through embeddings helps RADAR generalize to OOD queries, including ones from long-conter multi-documedt QA, despite being trained on shorter queries. We obtain the vellur difficulty by E R and discrimination aj E by linear transformations the query embeddings. i.e., a, = w. ej. by w.e ej, where wa w, E Rd are learnable d, -dmmensionnl transformation vectors. Our design choice of simple linear transformations and frozen embedding model helps ensure minimal router latency, which we analyze in Section 4.3 and Appendix [.. We scalar-valued ability parameter 0, E for model configuration gi. In the 2PL model, the prob- ability that a model configuration gi correctly answers a query qj is given by Pij = (a, b,)) where (-) is the sigmoid function.
We note that a concurrent work in IRT-b model-routing (Song et al. 2025) uses a multi- dimensional IRT model (MIRT) (Reckase 2009) to parameterize the performance function. In con- trast, we use scalar-valued model configuration abilities, enabling learned ability values to capture ordering information among model configurations and thus be interpretable. The fewer number of
5

---

Published as a conference paper at ICLR 2026
parameters in the 2PL model means that it requires less data to train than MIRT model. Fur- ther, instead of the qualitative model profile embedding approach to model generalization adopted adaptive testing- based approach to quickly estimate the precise scalar ability of any new RLM configuration, enabling RADAR to rapidly include it in the pool of RLM configurations for routing for future queries (see Section 3.5)
by [Song et al.] (2025),
To train the IRT model, we construct a binary-valued evaluation matrix Y E (0. 1)mx* of responses by n RLM configurations, i.e., different models with different reasoning budgets, on of training queries. We minimize the average negative log-likelihood (binary cross-entropy) over all nk observed responses, given by:
L2
Jiy log Pij Viy) log(1 -Pts)]
nk
where Pıy (a; b,)) denotes the probability under the 2PL IRT model that configuration 9: correctly answers query 4j.
3.4
COST PREDICTION
Apart from the performance prediction function, the other component in solving our routing MOO (see Equation 1 is the cost prediction function (g). Following prior routing work (Hu et al, 2024 [Song et al. 2025). we adopt a heuristic-based approach. We calculate the output cost of using configuration y to answer query by multiplying the cost per token ly of the base RLM of configuration 9. with the total number of reasoning tokens "9(g) and completion tokens a(()) generated, which is then averaged over all training queries. This heuristic is given by:
cmp
(g(g) = 1/12) (/
cmp " a(() lgg
(6)
g(g)
where we obtain the cost per token lg in US dollars of the base RLM from the official website for proprietary models (e.g., OpenAl for o4-mini) or from cloud provider for open source models (e g.. Qwen3). We rescale (using min-max normalization) each configuratio cost c. (g(g) to the range [0, 1], to ensure that both predicted cost and performance are on the same scale. We leave the development of more elaborate cost prediction methods to future work.
3.5 EXPANDING THE POOL OF R.MM CONFIGURATIONS THROUGH ADAPTIVE TESTING
To add a new model configuration 9: to RADAR, we need an accurate estimate of its ability 0, E IR Given a fitted 2PL model with learned query parameters {az. bز we could estimate the ability
9 of 9i by observing its responses to a query subset S C Q. We define the corresponding negative log- likelihood as:
LXPL (0; = Σ [us log (a, - b,)) + (1 - Miy) log(1 -oa, -0-
JES
where Viy E {0, 1} denotes the response of configuration gi to query G. The ability estimate is obtained by minimizing the negative log-likelihood, given by 0, arg ming Can (0; S,9:).
A complete evaluation corresponds to S Q. yeilding 0, arg ming Za (0: 9.).) However, evaluating on all queries is computationally expensive. To reduce evaluation cost, we adopt an adaptive query selection strategy inspired by computerized adaptive testing (Wainer et al. 2000). We iteratively construct an evaluation subset S; for configuration 9:. For clarity, we suppress the configuration index i during the iterative procedure. We initialize So Ø and 0g = 0. For iteration t 2 1, inspired by |Hofmann al.) (2025), we select the query with the highest Fisher information on the current ability estimate, given by:
je arg max 1(00-1,45,b)) by), jEQUS
(8)
where the Fisher information under the 2PL IRT model given by:
T(0. by) = (a, - b,))(1 - a (az (((000-)))))) b,))]
(9)
https://ww..tggetterr.a///picccp

---

Published as a conference paper at ICLR 2026
Table 1: Routing performance on ID queries across benchmarks reported on the hypervolume met- ric (higher is better). RADAR outperforms baselines, denoting better performance-con tradeoffs towards the Pareto front. See Table 13 for performance on OOD queries. Best performance is in bold and second best is underlined.
Benchmark
Random-Pair
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamond
0.5545
0,6866
0.6942
0.7513
MMLU
0.8592
0.8604
0.8720
MMLU-Rede Reduux
0.7281
0.9053
0.9117
0.9230
MMLU-Pro
0,5589
0,7819
0.7812
0.7995
LSAT
0.6913
0.9125
0.9163
0.9188
AIME
0.5159
0,7680
0.7766
0.7760
MATH-500
0.7433
0.9528
0.9420
0.94499
FRAMES
0.6589
0.8325
0.8501
0.8762
We update our evaluation subset with the selected query as S. = S.-1 U {gr}. After observing the response of configuration 9i on the selected query jt. we update its ability estimate as 0 = arg ming L2P1 (0; 9t). We repeat this procedure until the size of our evaluation subset [Sp] reaches a budgeted size returning Ör as the estimated ability of the new configuration gr.
EXPERIMENTAL EVALUATION
In this section, detail our evaluation setup, including benchmarks, metries, and baselines, used for a comprehensive evaluation of RADAR.
Benchmarks. We evaluate eight reasoning benchmarks, including AIME (MAA
2024 MATH (Hendrycks et al. 2021c). GPQA (Rein et al. 2024), LSAT (Wang et al. 2022; Zhong et al., MMLU (Hendrycks et al. 2021b a). MMLU Redux (Gema et al. (2024) MMLU Pro (Wang et al. 2024), and FRAMES (Kishhaa et al., 2024), spanning competition math, PhD-level sci- ence, law, and general knowledge with both multiple-choice and open-ended formats. In particular, FRAMES probes RADAR'S generalization to long-con contect queries in a multi-doc QA setting; long- context evaluation is largely absent prior routing exception, but on a private in-distribution RAG set. See Appendix C for details.
2021
(Hu et al. 2024 is notable
Metries. We report hypervolume (Emmerich & Deutz 2018), which, in our setting, corresponds to the area under the performance-cost trade-off curve across weights wi (higher is better). We also use the cost perss-eerforance threshold (CPT) metric, akin to call-performance thresholds (Ong et al.) 2024]: relative to the best- rest-performing configuration, o4-mini- high, CPT(x% is the minimum cost required to reach æ% of that performance, normalized by the cost of 04-mini-high. For example, CPT((9%) 0.1 means achieving 90% performance at 10% of the cost.
Baselines. We compare RADAR to several recent, state-of- of-the-art model routing methods, includ- ing RouterBench (Hu et al., 2024, Chen et al., 2025) and IRT-Router (Song et al. 2025). These methods do not natively generalize for adaptive reasoning or RLM configuration routing. We there- fore adapt these methods for our experimental setting; details are available in Appendix [D.3 We also compare several heuristic-based baselines, including All-Large (o4-mini, high budget), All- Small (Qwen3 0.6B, tokens), Oracle (chooses the cheapest rest-performing configuration given test-set performance), and Random-All (uniform over configurations). Random- Paiir seleets the largest configuration with probability wh, the user-defined performance-cos weight.
Evaluation Setup and Implementation Details. We employ Chebyshev scalarization in RADAR. We conduct both in-distribution (ID) and out-of-distribution (OOD) evaluations. For ID experi- ments, we aggregate the training splits of all benchmarks into a single training set for training the 2PL IRT model (see Section 3.3) and report performance on the test split of each benchmark separately. We use an 80% 20% train-test split for benchmarks without a predefined test set. For OOD, for each benchmark, we aggregate the training splits of the other remaining non-overlapping benchmarks into a single training set and report performance on the test split of this benchmark. For example, for the OOD experiment AIME, the training split of AIME and of overlapping benchmarks (MATH since MATH includes questions from AIME), are held out from the training set.
7

---

Published as a conference paper at ICLR 2026
Table Routing performance on ID queries across benchmarks reported on the CPT (90%) metric (lower is better). CPT (90%) denotes the fraction of the cost of running OpenAl 04-mini with high reasoning budget match 90% of its performance. See Table 14 for performance on OOD queries.
Best performance is in bold and second best is underlined.
Benchmark
Random- Pair
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamond
80.36%
57.13%
53 90%9%%
13. 21%
MMLU
76.30%
2.66%
2.69%
MMLU-Rede Reddux
75.06%
2.59%
2.80%
2.42%
MMLU-Pro
83.57%
6.19%
83%
3.89%
LSAT
80.14%
2.02%
1.93%
1.82%
AIME
87.22%
65.65%
63%
60. 60.69%
MATH-SUP
76.15%
1.34%
1.44%
1.31%
FRAMES
77.90%
43.50%
31.53%
13.11%
Table 3: Routing performance across benchmarks reported on the hypervolume metric (higher is better), before (RAdAr) and after (RADAR++) adding new RLM configurations from Qwen3- 14B, to test RADAR'S model generalization capability. RADAR+ quickly estimates the abilities of new configurations through adaptive testing for improved routing performance. Best performance is in bold and second best is underlined.
In-Distribation (ID)
(OOD)
Benchmark
RADAR
RADAR++
RADAR
RADAR++
GPQA-Diamon
0.7513
0.7535
0.7466
0.7463
MMLU
0.8720
0.8731
0, 0,8609
0,8698
MMLU-Redux
0.9230
0.9238
0.9072
0,9091
MMLU-Pro
0.7995
0.8021
0.7858
0.7951
LSAT
0.9188
0.9233
0.9146
0,9255
AIME
0.7760
0.7828
0.7566
0.7566
MATH-500
0.9449
0.9461
0.9368
0.9368
FRAMES
0.8762
0.8830
0.8865
0.8931
We route over 35 configurations comprising OpenAl 04-mini (budgets: low, medium, high) and Qwen3 models (3B/1.7B/4B/8B) with budgets 0, 256, 512, 1k, 2k, 4k, 8k, 16k (Yang et al. 2025 OpenAT] 2025). Each configuration is evaluated
per training query with standard prompts (Ap- pendix D.4): for AIME's small test set, we average over eight runs. Results closely match reported RLM performance (Yang et al., 2025 OpenAl 2025). In total, we collected 1.75 million binary re- sponses over 50, 50,139 unique questions across train/test splits of all eight benchmarks. Further details are in Appendix D
4.1
MAIN QUANTITATIVE. RESULTS
RADAR outperforms state-of-the-art model routing methods. Table Table 2 report rout- ing performance of all methods across 8 reasoning benchmarks evaluated in the ID setting on the hy- pervolume and CPT(90%) metrics, respectively. We include results on the CPT metric at additional thresholds in Appendix B.2 We see that RADAR outperforms all baselines and performs comparably to the best existing baseline on the remaining benchmarks. RADAR out- performs IRT-Router (Song et al.) 2025), a concurrent IRT-based routing work, suggesting that our novel formulation of RLM routing as an MOO, as well as the use of solution techniques like Cheby- shev scalarization, enable better recovery of the Pareto performance-cost front. For example, on the challenging APQA-Diamonn benchmark, RADAR demonstrates an 8% performance boost over the second-Be best baseline on the hypervolume metric. On the CPT metric, on MATH-500, RADAR is able to match 90% of the performance of o4-mini with a high reasoning budget at 1.31% of its cost. Similar gains are seen across all benchmark tasks.
most benchmarks
RADAR exhibits strong query generalization capabilities. We report routing performance of all methods across reasoning benchmarks evaluated in the OOD setting on the hypervolume and CPT (90%) metrics, respectively, in Appendix B.3 We show the Pareto performance-cost tradeoff curves for all methods on OOD queries from FRAMES (Krishna et al. 2024) in Figure [[ We that RADAR exhibits strong generalization OOD queries, outperforming existing state-of- the-art methods on most benchmarks. In particular, we highlight its dominant performance on the challenging long-context, multi-document reasoning- QA task from FRAMES (Krishna al.
8

---

Published as a conference paper at ICLR 2026
00.3
RLM Configuratio (AbllRy 0N
00:4.33 (250026000 20.4
0243 (250 re.dd
00-48 tbo
20-30 16.46
Ge extua tuuk
04-mit Pight (622)
MAT H 500 Ground Truhh Question Level (Dificuit)
Performance Cost Trao coff Waight [M1]
Figure 3: RADAR estimates interpretable query difficulties and RLM configuration abilities. Left: Mean predicted correctness probability of configurations on questions with different ground-truth difficulty levels in MATH-500. As difficulties increase, configurations with higher abilities are predicted to perform better. Right: Fraction of routing calls on MATH-500 queries spread across RLM configurations when varying the performance-cost tradeoff weight. A lower (higher) weight leverages a higher fraction of Qwen3 (o4-mini) configurations, prioritizing cost (performance).
2024). despite primarily being trained on much shorter queries. When generalizing to OOD queries with significantly higher difficulty (e.g.. AIME) than those seen during training, RADAR tends to assign a model configuration with a slightly lower ability than optimal, resulting in a slight decrease in performance. This weakness can be addressed by including small number of representative queries during training, as shown in Appendix B.4.
we show that Chebyshev scalarization outperforms linear scalarization, which is adopted in prior routing work (Song et al. 2025] Hu et al.) 2024). in the OOD experimental setting due to its ability to explore both convex and concave points the Pareto front. the II) experimental setting, both scalarization techniques perform similarly, with linear being marginally better. Our novel formulation of routing other MOO solution techniques in future work. We also conduct an ablation on the size of the train- ing matrix. Using just 20% of subsampled training queries, RADAR achieves a similar performance to using the entire training set as shown in Appendix B.6
Ablation study. In Appendix B.5
MOŌ opens the door leveraging
4.2 MODEL SCALABILITY AND GENERALIZATION EVALUATION
We evaluate the scalability of RADAR to new RLMs by adding new model configurations from the Qwen3 14B RLM. Table the result; using adaptive testing, RADAR accurately estimates the abilities of these new configurations by dynamically selecting just 5k training queries ( 12%
of the training set) for evaluation, resulting in improved routing performance. We compare against uniform random sampling in Appendix B.7 We show how routing shifts to new RLM configurations in Appendix B.8 In contrast to a concurrent IRT-based routing method (Song et al. 2025), which embeds qualitative profile of the model by prompting ChatGPT, based routing methods (Ding et al.] 2024), which assume the new model-pair has similar ability difference, RADAR can accurately estimate the ability of any new model configuration.
existing model-pair-
4.3 INTERPRETABILITT AND LATENCY ANALYSIS
RADAR estimates interpretable query difficulties and RLM configuration abilities. We highlight the interpretable nature of RADAR through case study on ID queries from MATH- 500 (Hendrycks et al.] 2021c) where queries are annotated with one of five levels of increasing difficulty. On the left panel of Figure we find a moderate Pearson correlation coefficient of 0.509 between RADAR-eSte -estimeed query difficulties and the five ground-truth levels. With an increase in query level, we see that configurations with higher abilities are predicted to have higher correctness probabilities, leading to a mean answer correctness prediction accuracy of 84.42% over all config- urations and queries. On the right panel of Figure 3] show the fraction of routing calls across configurations as the performance-cn preferred at lower weights and performant o4-mini models preferred higher weights.
tradeoff weight is varied, with cost-effective Qwen3 models
RADAR works in real time with minimal latency overhead. We measure the latency of RADAR and compare it the latency of the smallest RLM configuration (Qwen3-0.66 with 0 reasoning

---

Published as a conference paper at ICLR 2026
budget) used to generate answers to queries. The average per query routing latency overhead of RADAR over three runs of 500 queries from MATH-500 (Hendrycks et al. (021c) 6.89 = 0.53 milliseconds. Compared to the time taken for the smallest RLM configuration answer the query, which is 869.56 1.1 milliseconds, RADAR adds negligible overhead. We analyze throughput in Appendix B..9 B.9
5
CONCLUSIONS AND FUTURE WORK
We introduced RADAR, reasoning ability and difficult-aware routing framework that (1) for- malizes adaptive reasoning as an MOO and (2) leverages item response theory to adaptively assign queries RLM model-budget configurations. RADAR achieves strong cost-performance tradeoffs, consistently outperforming prior routing methods across eight challenging reasoning benchmarks, and generalizes well to cut-of-diitribution queries. Beyond efficiency. RADAR offers interpretabil- ity by exposing query difficulty and model abilities, and supports plug-and-play integration of new RLM configurations through adaptive calibration. Several promising avenues for future work ex- ist. First, we would like to extend RADAR beyond text to multi-modal reasoning settings. Second, incorporating additional configurations beyond the reasoning budget, such as retrieval, tool usage, and decoding algorithms, may yield fine-grained routing decisions for wider range of applica- tions, such as ultra-long context QA and deep research. Third, exploring RADAR in other constraint scenarios, such when there is total budget constraint batch of queries. Together, these direc- tions highlight the broader potential of RADAR as principled, interpretable foundation for adaptive reasoning in ever-evolving RLM ecosystem.
ACKNOWLEDGEMENTS
The authors would like thank Tong Sun and Jaewook Lee for helpful discussions. We also thank the anonymous reviewers for their helpful comments.
REPRODUCIBILITY STATEMENT
We structured the paper and the Appendix so the results can be independently re-implemented and reproduced. The problem setup, routing objective, and IRT modeling details are specified in Sec-
tion 3 Dataset sources, preprocessing, and ID/OOD split procedures are detailed Appendix C
Experimental details such baselines, evaluation protocol, metrics, and other implementation de- tails are available in Section and Appendix D
ETHICS STATEMENT
We affirm adherence to the ICLR Code of Ethics. Our study evaluates routing public reasoning
benchmarks and does not involve human subjects or personally identifiable data; we follow dataset
licensing and attribution guidance and document detailed preprocessing steps in Appendix [ We note that any routed answer inherits the properties, including potential social biases or safety issues, of the underlying RLM configurations; our method does not itself mitigate these risks, and we caution against deployment without domain ampainaappropriate safeguards and bias auditing.
REFERENCES
Pranjal Aggarwal and Sean Welleck. LI: Controlling how long reasoning model thinks with reinforcement learning. arXiv preprint (xXi:25003.4667 2025.
Maria-Florina Balcan, Steve Hanneke, and Jennifer Wortman Vaughan. The true sample complexity of active learning. Machine learning, 80(2):111-8 11-119,, 2010.
Allan Birnbaum. Some latent trait models and their use in inferring an examinee's 's ability. Statistical theories of mental test scores, 1968.
Jorgen Branke, Kalyanmoy Deb, Kaisa Miettinen, and Roman Slowinski. Multiobjectivee Optimiza- fion: Inferactive and Evolutionary Approuches. Springer Science & Business Media, 2008
10

---

Published as a conference paper at ICLR 2026
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint Prxxiv23055001760 2023.
Zhou Chen, Zhiqiang Wei, Yuqi Bai, Xue Xiong, and Jianmin Wu. Tagrouter: Learning route to
Ilms through tags open-domain text generation tasks. arXiv preprint arXiv:2506. /2473, 2025.
Christine DeMars. Item response theory. Oxford University Press, 2010.
Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. Hybrid llm: Cost-efficient and quality-aware query routing. arXiv preprint arXiv:2404.14000 /4618, 2024.
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. arXiv preprint SEE3:v1900000000 2019.
Michael TM Emmerich and André H Deutz. A tutorial on multiobjective optimization: fundamentals and evolutionary methods. Natural computing, 17(3):885-609) 2018.
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du Mohammad Reza Ghasemi Madani, et al. Are we done with mmlu? arXiv preprint arXiv:2406.04111 /27, 2024.
Soumya Suvra Ghosal, Souradip Chakraborty, Avinash Reddy, Yifu Lu, Mengdi Wang, Dinesh Manocha, Furong Huang, Mohammad Ghavamzadeh, and Amrit Singh Bedi. Does think- ing more always help? understanding test-time scaling in reasoning models. arXiv preprint arXiv:2506.0m 042200 2025.
Maharshi Gor, Hal Daumé III, Tianyi Zhou, and Jordan Boyd-Graher. Do great minds think alike? investigating human-ai complementarity in question answering with caimira. arXiv preprint arXiv:2410.06524. 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Steve Hanneke. Theory of active learning. Foundations and Trends in Machine Learning, 7(2-3), 2014.
Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz. Don't overthink it. preferring shorter thinking chains for improved I1m reasoning, 2025. URL httpps://arxiv.orr//aat
2505.17813
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning ai with shared human values. Proceedings of the International Conference on Learning Representations (ICLR), 2021a.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song. and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- tional Conference on Learning Representations (ICIR), 2021b.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv 2021c.
preprint
Valentin Hofmann, David Heineman, Ian Magnusson, Kyle Lo, Jesse Dodge, Maarten Sap. Pang Wei Koh, Chun Wang, Hannanch Hajishirzi, and Noah A Smith. Fluid language model benchmarking. In Second Conference on Language Modeling, 2025.
Jialiang Hong, Taihang Zhen, Kai Chen, Jiaheng Liu, Wenpeng Zhu, Jing Huo, Yang Gao, De- peng Wang. Haitao Wan, Xi Yang, Boyan Wang, and Fanyu Meng. Reconsidering over- thinking: Penalizing internal and external redundancy in cot reasoning, 2025. URL https: //arxiv..o org. abs/ htabs///28...m 02178
11

---

Published as a conference paper at ICLR 2026
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang- Thinkprune: Pruning long chain-of of-thought of Ilms via reinforcement learning. arXiv preprint arXiv:2504.01296. 2025.
Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system. arXiv preprint arXiv:2403./2031, 2024.
Shijue Huang. Hongru Wang. Wanjun Zhong, Zhaochen Su, Jiazhan Feng, Bowen Cao, and Yi R Fung. Adactrl: Towards adaptive and controllable reasoning via difficulty-wware budgeting. arXiv preprint arXiv:2505. 18822, 2025.
Carlos E Jimenez, John Yang. Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL
het.foorumidd/TTSSWWW66..
Ralph Keeney and Howard Raiffa. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Cambridge University Press, 1993.
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: unified evaluation of retrieval- augmented generation. arXiv preprint arXiv:2409. /2941, 2024.
Merve Şahin Kürşad. The effects of different item selection methods on test information and test efficiency in computer adaptive testing. Journal of Measurement and Evaluation in Education and Psychology, 14(1):33 46, 2023.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and lon Stoica. Efficient memory management for large language model serving with pagedattention, 2023. https:/:///axi htattp:://ariiv.orgg/abb//// http:/abb/2309..00
Celine Lee, Alexander M Rush, and Keyon Vafa. Critical thinking: Which kinds of complexity govern optimal reasoning length? arXiv preprint wrXivv2504.0195: 2025.
Frederic M Lord. A theory of test scores and their relation to the trait measured. ETS Research Bulletin Series, 1951(1):i-16,, 1951.
Frederic M Lord. Applications of item response theory to practical testing problems. Routledge, 2012.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), 2024.
MAA. MAA Invitational Competitions: Mathematical Association of America maa.org. https:
/maa- / cow.iiiiinccocmtttiiiiioooss///s 2024. [Accessed 03-09-2025].
Timothy Marler and Jasbir Arora. Survey of multi-objectivve optimization methods for engineering. Structural and Multidisciplinary Optimization, 26(6):369-339) -900 2004.
Guangyu Meng, Qingkai Zeng, John P. Lalor, and Hong Yu. A psychology-based unified dynamic
framework for curriculum learning, 2024. Utthttps::/ /ttp://///xxvvv/// wwtp:///g//bb//aeeeee 0074660.66366
Kaisa Miettinen. Nonlinear multiobjectiive optimization, volume 12. Springer Science & Business Media, 1999.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint DrXiv:2501.19393 2025.
Tadahiko Murata and Hisao Ishibuchi. MOGA: Multi-objective genetic algorithms. In Proceedings of 1995 IEEE International Conference on Evolutionary Computation, pp. 289-294, 1995.
12

---

Published as a conference paper at ICLR 2026
Isaac Ong. Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route Ilms with preference data. arXiv preprint wwXh::240611866. 2024.
OpenAl.
Introducing openai 03 and 04-mini. https://openal.ccmm/in
oatrodduiing-o3-and------- o4-iin- April 2025. Accessed: 2025-09-24.
OpenAl and et al. Openai ol system card, 2024. URL httttsss/aarrvv.orr////1221.
Felipe Maia Polo, Lucas Weher, Leshem Choshen, Yuekai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating lims with fewer examples. In Proceedings of the s International Conference on Machine Learning, ICML'24. JMLR.org, 2024.
Antonin Ponsich, Antonio Jaimes, and Carlos Coello. A survey on multiohjective evolutionary algorithms for the solution of the portfolio optimization problem and other finance and economics applications. IEEE Transactions on Evolutionary Computation, 17(3)321-344, 2013.
Georg Rasch. Studies in mathematical psychology: I. and attainment tests. 1960.
probabilistic models for some intelligence
M.D. Reckase. Multidimensional Item Response Theory. Springer New York, 2009. ISBN 9780387899763. doi: 1.1107/978-003387-9997666 978-0-387-89976-
URL. http://dx.do1 doi org/10gg//... 1007/
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-prov q&a bench- mark. In First Conference on Language Modeling, 2024.
Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, and Jordan Boyd-Grabed Graber. Evaluation examples are not equally informative: How should that change NLP leaderboards? In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Pro- ceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume 1: Long Pa- pers), pp. 4486-4503, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021 hww.:o/o1....loooo.4636.66
www.orrggg......e.o 2021. aac-lllngg 346/
org.
Marija Šakota, Maxime Peyrard, and Robert West. Fly-swat or canmon? cost-effective language model choice via meta-modeling. In Proceedings of the 17th ACM International Conference Web Search and Data Mining, pp. 606-615, 2024.
Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, and Andrew Lan. Smart: Simulated students aligned with item response theory for question difficulty prediction.
arXiv preprint arXiv:2507.055/22 2025.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint wxxiv:2402.033000 2024.
Parshin Shojace, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. The illusion of thinking: Understanding the strengths and limitations of reason- ing models via the lens of problem complexity, 2025. https::tt::///at /ttp///awwv.r/////55////// (06941]
Wei Song, Zhenya Huang, Cheng Cheng, Weibo Gao, Bihan Xu, GuanHao Zhao, Fei Wang, and Runze Wu. Irt-router: Effective and interpretable multi-llm routing via item response theory. arXiv preprint arXiv:2506.0104 2025.
Jinyan Su and Claire Cardie. Thinking fast and right: Balaneing accuracy and reasoning length with adaptive rewards. arXiv preprint a50iv:2505. 18298, 2025.
Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and over- thinking: An empirical study of reasoning length and correctness in llms. arXiv preprint arXiv:2505.00127, 2025.
13

---

Published as a conference paper at ICLR 2026
Wim J. van der Linden and Ronald K. Hambleton (eds.). Handbook Modern Item Response Theory. Springer, New York, NY, 1997.
Howard Wainer, Neil Dorans, Ronald Flaugher, Bert F Green, and Robert J Mislevy. Computerized adaptive testing: A primer. Routledge, 2000.
Lihui Wang. Amos Ng, and Kalyanmoy Deb. Multi-Objective Evolutionary Optimisation for Prod- uct Design and Manufacturing. Springer, 2011.
Siyuan Wang, Zhongkun Liu, Wanjun Zhong, Ming Zhou, Zhongyu Wei, Zhumin Chen, and Nan Duan. From Isat: The progress and challenges of complex reasoning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022.
Xiangqi Wang, Yue Huang, Yanbo Wang, Xiaonan Luo, Kehan Guo, Yujun Zhou, and Xian- gliang Zhang. Adareasoner: Adaptive reasoning enables more flexible thinking. arXiv preprint 25d.::2505.731122 2025.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro. task language understanding benchmark. Advances in Neural Information Processing Systems, 9222299699920, 2024.
canlu-pro:Aa : A more robust and challenging multi-
Yuhui Xu, Hanze Dong, Lei Wang, Doyen Sahoo, Junnan Li, and Caiming Xiong. Scalable chain of thoughts via elastic reasoning. arXiv preprint arXiv:2505.05315. 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint wrXiv:2505.09388 2025.
Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik R Narasimhan. [SttauS]-benchh A benchmark for lunderline ( Jol--uunderl [eneen)t-udderiinn [U]ser interaction real-world domains. In The Thirteenth International Conference Learning Representations, 2025. URL
https://openreviees http://eneeeieeeeeteeoormmnniiia
Zishun Yu, Tengyu Xu, Di Jin, Karthik Abinav Sankararaman, Yun He, Wenxuan Zhou, Zhouhao Zeng, Eryk Helenowski, Chen Zhu, Sinong Wang, al. Think smarter not harder: Adaptive reasoning with inference aware optimization. arXiv preprint arXiv:2501.17974 2025.
Linan Yue, Yichao Du, Yizhi Wang, Weibo Gao, Fangzhou Yao, Li Wang, Ye Liu, Ziyu Xu, Qi Liu, Shimin Di, et al. Don't 't overthink it: survey of efficient rl style large reasoning models, arXiv preprint arXiv:2508.022 20, 2025.
Richard Zhang and Daniel Golovin. Random hypervolume scalarizations for provable multi- objective black box optimization. Proceedings of the 37th International Conference on Ma- chine Learning, 2020.
Xuechen Zhang, Zijian Huang, Ege Onur Taga, Carlee Joe-Wong, Samet Oymak, and Jiasi Chen. Efficient contextual IIm cascades through budget-constrained policy learning. Advances Neural Information Processing Systems, 17:9169112 (1699-997722,2 2024.
Yiqun Zhang, Hao Li. Jianhao Chen, Hangfan Zhang, Peng Ye, Lei Bai, and Shuyue Hu. Beyond gpt-5: Making iims cheaper and better via performance-effficeenn optimized routing, 2025. URL https://arxiv.orgt org. / 12631
Wanjun Zhong, Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Jiahai Wang, Jian Yin, Ming Zhou, and Nan Duan. Ar-lsat: Investigating analytical reasoning of text, 2021.
Vilém Zouhar, Peng Cui, and Mrinmaya Sachan. How to select datapoints for efficient human evaluation of nig models? arXiv preprint urXiv:2501.18251, 2025.
14

---

Published as a conference paper at ICLR 2026
A EXTENDED RELATED WORK
Efficient Reasoning. A rapidly growing literature seeks to make reasoning models themselves more efficient; see Yue et al., 2025) for broader overview of this direction. Methods such as L1 (Aggarwal & Welleck 2025) and S1 (Muennighoff et al. (025) provide length control, enabling reasoning models to trade off aceuracy and cost by constraining chain-of-thought length. Others prune or adapt the reasoning process by dynamically shortening or extending reasoning (Hou et al.)
2025 Xu et al.) 2025] Wang et al., (025); adaptively controlling inference steps (Huang et al. and analyzing when additional reasoning is beneficial or wasteful (Su & Cardie 2025 'uu
2025
et al. 2025; Yu et al. 2025 Ghosal et al., 2025). Theoretical perspectives further study optimal reasoning length (Lee et al.) 2025). These works aim to make single model more efficient. They also require access to model weights, which usually do not apply for closed-source or black-box settings. Our approach is complementary: RADAR treats any such efficient reasoning model as an additional candidate in its pool of (model, reasoning budget) configurations. This means advances in adaptive or efficient reasoning can be seamlessly integrated into our framework, while RADAR contributes orthogonally by providing per-query routing, interpretability through IRT, and Pareto- optimal pest-performance control. contrast to static single-model tuning, which requires weight access, R RADAR operates in a black-box setting and leverages the complementary strengths of diverse RLMs. This enables RADAR to dynamically shift along performance-cos tradcoff's depending on application needs in heterogeneous RLM landscape.
Routing for Foundation Models. Recent work studies cost-quality routing across multiple
LLMs (Chen et al.
2023; Zhang et al.
2024; Ding et al. 2024 Ong et al.] 2024| Hu et al. 2024]
Sakota et al., 2024, Chen al. 2025 Song et al.
2025
Most methods focus on model selection
with black-box predictors cascades CCen al. 2023 Ding et al. 2024 Ong et al. 2024 Sakota
itt al. 2024: Chen et a 2025), though TREACLE additionally co-selects prompt types under bud- get constrains (Zhang et al., 2024). We instead study adaptive reasoning and cast this problem as routing over model-buuger configurations, where the budget controls the number of thinking to- kens, making reasoning cost an explicit decision dimension in addition to the model itself. Routers also differ in when they commit: cascaded approaches may re-query model (Chen et al. [023]] Zhang et al.) 2024), while others choose once per query (Ding et al., 2024). Our router makes a single assignment before generation, avoiding mid-turn switching (and KV-cache recomputation) or multiple re- re-querying while still retaining favorable cost costtquality trade-offs. Finally, we emphasize
interpretability and control. Unlike opaque regressors (Chen et al., 2023 Ding et al., 2024 Ong et al. 2024), we use an IRT parameterization to expose query difficulty and configuration ability.
Comparison with IRT-Router Compared to IRT-Router (Song et al. 2025). a concurrent and recently released work, RADAR makes the following contributions:
1. Novel MOO Formulation: We're the first to formulate model routing in a mathematically principled way as multi-objective optimization (MOO) that searches for the model at the Pareto front of the performance-cost tradeoff curve using scalarization techniques. The objective used in IRT-Router is introduced ad hoc and is a special case of our MOO for- mulation with linear scalarization. Beyond simple linear scalarization, which cannot cover non-convex Pareto fronts, our MOO formulation allows the LLM routing community to leverage powerful solution techniques from well-established MOO literature, including Chebyshev scalarization. In RADAR, we find that Chebyshev scalarization outperforms linear scalarization on the stable hypervolume metric, in the challenging OOD experimen- tal setting, due to its ability to explore both convex and concave points on the Pareto front (see Table 18) In the easier ID experimental setting, both scalarization techniques per- form similarly, with linear being marginally better. We leave the exploration of other MOO solution techniques, such as lexicographic methods, for future work.
2. Model Generalizability: Another significant contribution of RADAR is its effective model generalization capability. IRT-Router uses an ad hoc approach: it queries ChatGPT (web search mode) for a description of the new LLM, which is then corrected by manual inter- vention and embedded. IRT-Router notes the limited generalizability of their approach and highlights model generalizability as an important direction for future work. In contrast, to add new RLM configuration in RADAR, we simply need its ability. To precisely estimate
15

---

Published as a conference paper at ICLR 2026
Table 4: Dataset statisties of prompt tokens across reasoning benchmarks used.
Dataset
Samples
Mean Tokens
Min Tokens
Max Tokens
AIME
1,035
143.00
30
3,312
MATH
8,000
86.16
24
806
GPQA
448
250.27
82
2,812
LSAT
2,025
263.63
174
570
MMLU
13,937
150.50
66
1,040
MMLU Redux
5,298
135.50
68
1,000
MMLU Pro
12,032
237.51
70
1,700
FRAMES
561
16,272.19
690
31,954
the ability of a new RLM configuration, RADAR follows a principled and fully automated approach by evaluating it on small set (12% of training queries) of dynamically selected queries, employing a classic technique inspired by adaptive testing in educational assess- ment.
3. Informative Metrics: IRT-Router reports performance at three arbitrary performance-cost tradeoff weights (0.2, 0.5, and 0.8), which fail to provide an accurate evaluation of rout- ing methods. In contrast, metric from MOO literature, which corresponds to the area under the performance-cost trade-off curve across the entire domain (0 to 1) of the trade-off weight, yielding more stable and informative metric.
MOO-based routing formulation leverages the hypervolume
4. Custom Interpretable IRT Model: IRT-Router employs a standard Multidimensional-Ie (MIRT) model, which uses non-interpretable vectorized abilities. In contrast, RADAR uses a custom, interpretable IRT model with multidimensional embeddings for queries to enable OOD generalizability and scalar model abilities to support interpretable ability ordering across models. This ability ordering, as seen in the y-axis of Figure 3 is helpful for RLM benchmarking and evaluation.
5. Focus on Reasoning LLMs: marily focused on LLMs, we present routing formulation for RLMs incorporating reason- ing budgets. Choosing the right RLM for practical deployment involves performance-cost trade-off at two key levels: base models and reasoning budgets. We unify these decisions by discretizing each RLM by its available set of reasoning budgets. Although simple, dis- cretization trick easily extends to selecting other model settings, such as parameterizations of the RAG pipeline attached to the RLM decoding methods employed.
In contrast to past work, including IRT-Router, which are pri-
Item Response Theory in Machine Learning. Originally designed for assessment and other ed- ucational applications, IRT has emerged as a versatile tool for understanding and improving foun- dation models. It has been applied to evaluation and benchmarking such as jointly estimating model ability and item difficulty to build adaptive or efficient test suites (Rodriguzz al. 2021 Zouhar et al. 2025 Hofmann et al. 2025 Polo et al. 2024); to training and curriculum design, where IRT- based difficulty estimates guide data selection for faster and more effective learning (Meng et al., 2024 Scarlatos et al.) 2025); and to the diagnostics and bias analysis, exposing strengths and weak- nesses of models relative to humans or idcological leanings (Gor et al.) 2024). Most relevant to our setting, IRT has recently been explored for multi-model routing, where parameterizes query difficulty and model ability to guide cost pess---rrormaceee trade-offs with interpretability (Song et al., 2025). Our work extends this line by applying IRT not only model selection but also to adap- tive reasoning configurations (model-reasonins budget pairs), thereby contributing to the ongoing exploration of IRT for foundation models.
B ADDITIONAL RESULTS
B. ABLATION STUDY ON LINEAR TRANSFORM FOR QUERY DIFFICULTY
To test the sufficiency of our linear transformation for query difficulty and discrimination, we per- form ablation study with classic two-layer multilayer perceptron (MIP) model using ReLU
16

---

Published as a conference paper at ICLR 2026
Table Ablation study on the linear transformation for query difficulty in RADAR. Linear transfor- mation performs similarly to a classic two-layer multilayer perceptron (MLP) model using a ReLU non-linearity on ID queries.
Hypervolume (higher is better)
CPT(90%) (lower is better)
Benchmark
RADAR
RADAR (MLP)
RADAR
RADAR (MLP)
GPQA-Diamond
0.7513
0.7308
13.21%
14.11%
MMLU
0.8720
0.8707
2.69%
2.8%
MMLU-Redus
0.9230
0.9239
2.42%
2.51%
MMLU-Pro
0.7995
0.7955
3.89%
3.91%
LSAT
0.9188
0.9132
1.82%
2.32%
AIME
0.7760
0.7687
60.69%
53.28%
MATH-500
0.9449
0.9365
1.31%
1.33%
FRAMES
0.8762
0.8777
13.11%
11.37%
Table Ablation study on the linear transformation for query difficulty in RADAR. Linear transfor- mation performs similarly to classic two-layer multilayer perceptron (MLP) model using ReLU non-linearity on OOD queries.
Hypervolume (higher better)
CPT(90%) (lower is hetter)
Benchmark
RADAR
RADAR (MLP)
RADAR
RADAR (MLP)
GPQA-Diame
0.7466
0.7245
17.6%
23.49%
MMLU
0.8609
0.8573
2.63%
2.82%
MMLU-Redux
0.9072
0.9067
2.54%
2.63%
MMLU-Pro
0.7858
0.7844
3.54%
4.82%
LSAT
0.9146
0.9101
2.15%
2.12%
AIME
0.7566
0.7726
55.30%
59%
MATH-500
0.9368
0.9424
1.55%
1.46%
FRAMES
0.8865
0.8771
9.99%
11.17%
non-linearity to obtain the difficulty and discrimination of queries. Our results reported in Table and Table Shsow similar performance in both the ID and OOD experimental settings across beneh- marks. These none to marginal gains, obtained at the cost of diminished interpretability and in- creased latency, further justify the simplicity of our design choices in RADAR.
B.2 RESULTS ON THE CPT METRIC AT VARIOUS TURESHIOLDS
In addition to 90%. we report results on the CPT metric at various thresholds, including 80% (Table and Table 8). 85% (Table 9 and Table 10). and 95% (Table 11 and Table 12). on both ID and OOD experimental settings across benchmarks. We observe similar patterns: RADAR outperforms baselines majority of datasets in both ID and OOD experimental settings.
Table 7: Routing performance on ID queries across benchmarks reported on the CPT (80%) metric (lower better). CPT (80%) denotes the fraction of the cost of running OpenAl 04-mini with high reasoning budget to match 80% of its performance. Best performance is in bold and second best is underlined.
Benchmark
Random-Padd
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamond
62.69%
17.69%
8.4%
5.05%
MMLU
52.61%
1.22%
1.31% %
1.41%
MMLU-Redd
52.93%
1.32%
1.32%
1.4%
MMLU-PI
68.245 68.24%
1.7%
1.74%
1.37%
LSAT
60.9%
1.47%
1.36%
1.08%
AIME
74.45%
24.73%
24.34%
21.38%
MATH-500
53.79%
0.7%
0.7%
0.79%
FRAMES
60.92%
1.1%
1.08%
0.78%
17

---

Published as a conference paper at ICLR 2026
Table 8: Routing performance on OOD queries across benchmarks reported on the CPT (80%)
metric (lower is better). CPT (80%) denotes the fraction of the cost of running OpenAl 04-mini with a high reasoning budget to match 80% its performance. Best performance is in bold and second best is underlined.
Benchmark
Random-Pair
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamond
66.05%
5.2%
8.49%
5.55%
MMLU
52.58%
1.45%
1.23%
1.42%
MeMUU-Redux
52.5%
1.47%
1.29%
1.53%
MMLU-Pro
66.14%
1.73%
1.82%
1.4%
LSAT
61.0%
1.44%
1.36%
1.13%
AIME
71.82%
13.37%
30.5%
MATH-500
53.58%
0.68%
0.71%
0.63%
FRAMES
60.73%
1.33%
1.05%
1.4%
Table Routing performance on ID queries across benchmarks reported on the CPT (85%) metric (lower is better). CPT (85%) denotes the fraction of the cost of running OpenAl o4-mini with high reasoning budget to match 85% its performance. Best performance in bold and second best is underlined.
Benchmark
Random-Pard Pair
RouterBench
IRT RRT-Router
RADAR (ours)
71.53%
37.41% 4
31.19%
8.93%
MMLU
64.46%
1.61%
1.85%
1.68%
MMLU-Reduu
63.77%
1.85%
1.97%
1.67%
MMLU-Pro
75.91%
2.18%
2.14%
2.23%
LSAT
70.35%
1.73%
1.64%
1.33%
AIME
80.84%
45.19%
42.39%
41.04%
MATH-500
64.97%
0.94%
0.95%
0.93%
FRAMES
69.41%
10.17%
1.22%
1.21%
Table 10: Routing performance on OOD queries across benchmarks reported on the CPT (85%)
metric (lower is better). CPT (85%) denotes the fraction of the cost of running OpenAI o4-mini with a high reasoning budget to match 85% its performance. Best performance is in bold and second best is underlined.
Benchmark
Random-Pair
RouterBench
IRT-Router
RADAR (ours)
-PQ.A-Dimood
74.3%
24.53%
31.34%
10.86%
MMLU
63.55%
1.96%
1.88%
1.73%
MMLU-
63.09%
1.99%
1.92%
1.85%
MMLU-Pro
74.4%
2.44%
2.2%
2.13%
LSAT
70.54%
1.73%
1.65%
1.5%
AIME
77.92%
32.78%
42.9%
MATH-500
63.55%
0.9%
0.97%
0.73%
FRAMES
68.74%
16.7%
1.21%
1.55%
Table 11: Routing performance on ID queries across benchmarks reported on the CPT (95%) metric (lower is better). CPT (95%) denotes the fraction of the cost of running OpenAl o4-mini with high reasoning budget to match 95% its performance. Best performance in bold and second best is underlined.
Benchmark
Random- Pair
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamont
89.19%
76.86%
76.78%
21.1%
MMLU
88.15%
15.27%
6.73%
7.52%
MMLU-E MeMUU-Redux
86.79%
8.69%
5.45%
5.16%
MMLU-Pro
91.42%
38.78%
43.64%
12.08%
LSAT
90.01%
3.14%
3.48%
3.62%
AIME
93.61%
86.11%
80.61%
80.34%
MATH-500
87.63%
2.03%
2.26%
2.58%
FRAMES
88.84%
77.6%
65.76%
31.81%
18

---

Published as a conference paper at ICLR 2026
Table 12:
Routing performance on OOD queries across benchmarks reported on the CPT (95%)
metric (lower is better). CPT (95%) denotes the fraction of the cost of running OpenAl o4-mini with a high reasoning budget to match 95% of its performance. Best performance is in bold and second best is underlined.
Benchmark
Random-Pait
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamond
90.78%
63.83%
77.05%
36.66%
MMLU
86.42%
29.76%
8.37%
7.14%
MMLU -MeUU-Redux
86.14%
10.57%
5.81%
5.62%
MMLU-Pro
91.32%
51.8%
45.25%
36.2% %
LSAT
90.0%
8.39%
3.52%
5.63%
AIME
92.44%
77.59%
MATH-500
86.34%
2.58%
2.13%
2.79%
FRAMES
89.3%
61.44%
22.13%
Table 13: Routing performance on OOD queries across benchmarks reported on the hypervolume metric (higher better). RADAR outperforms baselines, denoting better performance- tradeoffs towards the Pareto front. Best performance is in bold and second best is underlined.
Benchmark
Random-Pait
RouterBench
IRT-Router
RADAR (ours)
GPOA-Diamond
0.5369
0.7047
0.6938
0.7466
MMLU
0.6934
0.8398
0.8550
0.8609
MMLU- -Reduxx
0.7298
0.8948
0.9050
0.9072
MMLU-Pro
0.5686
0.7703
0,7800
0.7858
LSAT
0.6887
0.9046
0.9175
0.9146
AIME
0.5283
0.6890
0.7915
0.7566
MATH-500
0.7493
0.9326
0.9385
0.9368
FRAMES
0.6624
0.8230
0.8548
0.8865
B.3 RESULTS ON OOD QUERIES
Table 13 and Table 14 report routing performance of all methods across reasoning benchmarks evaluated in the OOD setting on the hypervolume and CPT (90%) metries, respectively. RADAR exhibits strong query generalization capabilities.
B.4 IMPROVING OOD PERFORMANCE ON AIME
Our analysis reveals two special characteristics of AIME. First, AIME contains queries with the greatest average difficulty, significantly higher than other benchmarks as seen in Table [5] Second, performance on AIME consistently improves with increasing reasoning length and does not plateau, unlike other benchmarks. the OOD setting, RADAR, which not exposed to difficult math
Table 14: Routing performance on OOD queries across benchmarks reported on the CPT (90%) metric (lower is better). CPT (90%) denotes the fraction of the cost of running OpenAl 04-mini with a high reasoning budget to match 90% of its performance. Best performance is in bold and second best is underlined.
Benchmark
Random-Pad
RouterBench
IRT-Router
RADAR (ours)
GPQA-Diamond 82.5.1%
44.18%
54.19%
17.6%
MMLU
74.53%
2.94%
2.61%
2.63%
MMLU-Redd
74.61%
2.90%
2.71%
2.54%
MMLU-PI
82.65%
7.67%
4.02%
3.54%
LSAT
80.07%
2.27%
1.96%
2.15%
AIME
84.88%
55.19%
55.30
MATH-500
74.94%
1.4%
1.29%
1.55%
FRAMES
78.61%
48.52%
29.49%
9.99%
19

---

Published as a conference paper at ICLR 2026
Table 15: Predicted query difficulty on ID test queries comparing AIME against the remaining benchmarks. IRT difficulty values usually lie in [-3, 3], with higher values indicating greater diffi- culty.
Benchmark
Difficulty Avg
Difficulty Std
AIME
1.038
1.275
All Benchmarks
-0.55
1.018
Table 16: Predicted query difficulty on test queries comparing ID AIME queries to OOD AIME queries. IRT difficulty values usually lie in the range [-3, 3), with higher values indicating greater difficulty.
Benchmark
Difficulty Avg
Difficulty Std
AIME (ID)
1.038
1.275
AIME (OOD)
-0.461
1.043
problems, underestimates query difficulty, shown in Table 16 and underperforms by allocating less capable models.
In real-world use, it is practical assume exposure math problems during model training. We perform partial OOD experiment by exposing RADAR to a fraction of the AIME and MATH queries in the training data. In Table [7 we see a significant increase in performance with just 5% of exposure, and a similar performance to the ID setting with 30% of exposure.
B.5 ABLATION STUDY SCALARIZATION
See Table 18 for an ablation study which shows Chebyshev scalarization outperforms linear scalar- ization on OOD queries due to its ability to explore both convex and concave points on the Pareto front.
B.6 ABLATION STUDY ON MATRIX SIZE
See Table 19 for an ablation study on the size of the training matrix RADAR. Using just 20% of subsampled training queries, RADAR achieves similar performance to using the entire training set.
B.7 ABLATION STUDY ADAPTIVE TESTING
We conduct ablation study comparing Fisher informationnbasse adaptive testing with uni- form random sampling on routing performance. Similar to well-established findings in adaptive testing (Kürşad, 2023) and online learning (Hanneke 2014: Balcan et al. 2010) literature, find that our method performs better and with lower variance than uniform sampling. especially when the sample size small, and both methods converge perform similarly the sample size increases. We report results on ID qeuries in Table 20 (hypervolume) and Table (CPT), and on OOD queries Table 22 (hypervolume) and Table 23blle33(C1)) (CPT).
For example, with small sample size of just 100 items, our method consistently outperforms uni- form sampling by wide margin on ID benchmarks and AIME and MATH benchmarks for OOD. Further, uniform sampling exhibits high variance seen in FRAMES in the ID) setting, achieving 2.6% CPT(90%) with 100 items, which counterintuitively jumps 22.2% with larger set of 500
Table 17: Partial OOD experimental setting showing that small of training queries can recover ID performance.
Benchmark
RADAR (ID)
RADAR (OOD)
RADAR (OOD + 5%)
RADAR (OOD 30%)
AIME
0.776
0.7566
0.7665
0.7787
20

---

Published as a conference paper at ICLR 2026
Table 18: Ablation study showing Chebyshev scalarization outperforms linear scalarization on OOD queries due to its ability to explore both convex and concave points on the Pareto front.
Hypervolume (higher is better)
CPT(90%) (lower is better)
Benchmark
RADAR (LS)
RADAR (CS)
RADAR (LS)
RADAR (CS)
GPQA-Diamond
0.7280
0.7466
29.94%
17.60%
MMLU
0.8580
0.8609
2.50%
2.63%
MMLU-Redux
0.9049
0.9072
2.25%
2.54%
MMLU-PP
0.7812
0.7858
3.81%
3.54%
LSAT
0.9165
0.9146
2.00%
2.15%
AIME
0.7464
0.7566
56.23%
55.30%
MATH-500
0.9331
0.9368
1.41%
1.55%
FRAMES
0.8656
0.8865
21.56%
9.99%
Table 19: Ablation study the size of the training matrix of RADAR. Using just 20% of subsampled training queries, RADAR achieves a similar performance to using the entire training set.
Hypervolume (higher better)
CPT(90%) (lower better)
Benchmark
RADAR (20%)
RADAR
RADAR (20%)
RADAR
GPQA-Diamond
0.7526
0.7513
16.29
13.21
MMLU
0.8726
0.8720
2.67
2.69
MMLU-Redux
0.9207
0.9230
2.69
2.42
MMLU-Pro
0.7990
0.7995
3.59
3.89
LSAT
0.9175
0.9188
1.95
1.82
AIME
0.7832
0.7760
57.85
60.69
MATH-500
0.9450
0.9449
1.15
1.31
FRAMES
0.8940
0.8762
10.70
13.11
items. When uniform sampling marginally performs better, its high variance might indicate fortu- itous estimate of ability rather than an accurate one. In contrast, our method provides reliable and precise estimate of ability, which results in stable routing performance across sample sizes.
B.8 MODEL SCALABILITY AND GENERALIZATION EVALUATION OF RADAR
We evaluate the scalability of RADAR new RLMs by adding new model configurations from the Qwen3 14B RLM. Using adaptive testing, RADAR accurately estimates the abilities of these new configurations by dynamically selecting just 5k training queries (12 of the training set) for
Table 20: Routing performance on ID queries across benchmarks reported on the hypervolume metric (higher is better), before (RADAR) and after (RADAR++) adding new RLM configurations from Qwen3-14B. Rnd X denotes set of items selected with uniform random sampling, while Fshr X denotes set of X items selected with our method of maximum Fisher information-basee adaptive testing.
RADAR+
Benchmark (ID)
RADAR
Rnd 100
Fshr 100
Rnd 500
Fshr 500
Rnd 5000
Fshr 5000
GPQA-Diamond
0.7513
0.6171
0.7511
0.7017
0.7513
0.7535
0.7535
MMLU
0.8609
0.8591
0.8731
0.8682
0.8720
0.8745
0.8731
MMLU-Redux
0,9072
0.8929
0,9232
0.9168
0.9230
0.9228
0,9238
MMLU-PE
0.7858
0.7764
0,8009
0.7941
0.7995
0.8038
0.8021
LSAT
0.9146
0,9270
0,9205
0,9283
0.9188
0.9259
0.9233
AIME
0.7760
0.6883
0.7636
0.7311
0.7760
0.7840
0.7828
MATH-500
0.9449
0.9426
0.9456
0,9457
0.9449
0.9478
0.9461
FRAMES
0.8865
0.8629
0.8763
0.8587
0.8762
0.8897
0.8830
21

---

Published as a conference paper at ICLR 2026
Table 21: Routing performance IDD queries across benchmarks reported the CPT (90%) metric (lower is better), before (RADAR) and after (RADAR++) adding new RLM configurations from Qwen3-14B. Rnd X denotes denotes a set of X items selected with our method of maximum Fisher information-be -aasedd adaptive testing. NR denotes not reachable.
of X items selected with uniform random sampling, while Fshr X
RADAR+
Benchmark (ID)
RADAR
Rnd 100
Fshr 100
Rnd 500
Fshr 500
Rnd 5000
Fshr 5000
GPQA-Diamond
13.21 %
NR
13.61%
45.24%
13.21%
12.28%
13.56%
MMLU
2.63%
2.67%
2.69%
2.75%
2.69%
2.69%
2.69%
MMLU-Redus
2.54%
2.38%
2.42%
2.42%
2.42%
2.42%
2.42%
MMLU-Pro
3.54%
3.73%
3.9%
3.93%
3.89%
3.9%
3.9%
LSAT
2.15%
1.82%
1.82%
1.82%
1.82%
1.82%
1.8%%
AIME
60.69%
NR
65.12%
72.43%
60.69%
57.7%
58.45%
MATH-500
1.31%
1.34%
1.31%
1.32%
1.31%
1.32%
1.31%
FRAMES
9.99%
2.62%
13.06%
22.20%
13.11%
3.25%
5.97%
Table 22: Routing performance on OOD queries across benchmarks reported on the hypervolume metric (higher is better), before (RADAR) and after (RADAR++) adding new RLM configurations from Qwen3-14B, Rnd X denotes set of X items selected with uniform random sampling, while Fshr denotes a set of items selected with our method maximum Fisher information-base adaptive testing.
RADAR++
Benchmark (OOD) RADAR
Rnd 100
Fshr 100
Rnd 500
Fshr 500
Rnd 5000
Fshr 5000
GPQA-Diamon
0.7466
0.7402
0.6362
0.7466
0.7177
0.7467
0.7463
MMLU
0.8609
0.87
0,8679
0.8697
0,8684
0.8697
0.8698
AMLU-Reduu
0,9072
0.9086
0.9093
0.9090
0.9098
0.9094
0.9091
MML.U-Pr
0.7858
0.7966
0.7927
0.7958
0.7928
0.7961
0.7951
LSAT
0.9146
0.9287
0.9273
0.9264
0.9267
0.9278
0.9255
AIME
0.7566
0.6543
0.7566
0.7385
0.7566
0.7694
0.7566
MATH-500
0.9368
0.9186
0,9368
0,9367
0.9368
0,9389
0.9368
FRAMES
0.8865
0.8658
0.8699
0.8864
0.8872
0.8938
0.8931
Table 23: Routing performance on OOD queries across benchmarks reported on the CPT (90%) metric (lower is better), before (RADAR) and after (RADAR++) adding new RLM configurations from Qwen3-14B. Rnd X denotes a set of items selected with uniform random sampling, while Fshr X denotes set of X items selected with our method of maximum Fisher information-based adaptive testing. NR denotes not reachable.
RADAR+
Benchmark (OOD)
RADAR
Rnd 100
Fshr 100
Rnd 500
Fshr 500
Rnd 5000
Fshr 5000
GPQA-Diamond
17.6%
22.6%
NR
15.8%
37%
17 17.48%
16.63%
MMLU
2.63%
2.63%
2.63%
2.63%
2.63%
2.63%
2.63%
MMLU-Redux
2.54%
2.54%
2.54%
2.54%
2.54%
2.54%
2.54%
MMLU-Pro
3.54%
3.69%
3.54%
3.69%
3.54%
3.7%
3.56%
LSAT
2.15%
2.19%
2.15%
2.53%
2.15%
2,48%
2.15%
AIME
55.3%
59.2%
55.3%
56.5%
55.3%
52.5%
55.3%
MATH-500
1.55%
1.98%
1.55%
1.53%
1.55%
1.57%
1.55%
FRAMES
9.99%
8.72%
2.58%
10%
5.16%
2.57%
5.25%
22

---

Published as a conference paper at ICLR 2026
SUN Confpuurattin (bbiity 01
03-43 Daw 10.32
3. Conn Seieee en Aeelity
0243 ALLS
1.1 ExE Gue coe Gil 330 Las
Performance- Tradeaff
Figure 4: Fraction routing calls on OOD queries from FRAMES spread across RLM configura- tions when varying the performance-cost tradeoff weight before (left) and after (right) adding new RLM configurations from Qwen3- 14B. RADAR rapidly estimates the ability of Qwen3-14B at 16K reasoning budget to leverage it for improved performance.
0.6
0.5
Pass
Pass@1
0.4
0.7
0.3
Gwen2-3
Owe13-0
0.5
2500
55000
Reasoning Budget (Num Tokens)
Reasoning Budget (Num Tokens)
Figure 5: Left: Our pilot study on MATH-500 (Hendrycks et al. 2021c) shows a performance dif- ferential over (RLM, reasoning budget) configurations with the smallest RLM already solving over 50% of the queries with minimal reasoning. Right: Performance on AIME consistently increases with an increase model size and reasoning budget.
evaluation, resulting in improved routing performance. Figure shows how routing shifts to new RLM configurations.
B.9 THROUGHPUT ANALYSIS
For throughput analysis, compute the queries/second for the smallest (0.6B) and largest (8B) open-source Qwen3 models on the MATH-500 benchmark. averaged across three runs on a single Nvidia A100 80GB GPU, using vLL.M for batched inference and embedding. We exclude Ope- nAI o4-mini models due to their variable API-based throughput. Qwen3-0.66 with zero reasoning budget processes queries at 1.1529 queries/sec, while Qwen3-8B with 16K reasoning budget pro- cesses queries at 0.2337 queries/sec. Adding RADAR's routing overhead decreases throughput by 0.78% for Qwen3-0.6B with zero reasoning budget, 1.1438 queries/sec, and by 0.15% for Qwen3-8B 8B with a 16K reasoning budget, to 0.2237 queries/sec. Depending on the user-specified performance-com tradeoff weight, RADAR routes queries to convex combination of model con- figurations, with the throughput therefore bounded between 1.3488 queries/sec as the upper bound and 0.2237 queries/sec as the lower bound. Multi-GPU implementation and inference optimization methods can further improve RADAR's throughput.
B. PERFORMANCE VS REASONING BUDGET CURVES
We include performance reasoning budget curves in Figure
23

---

Published as a conference paper at ICLR 2026
B.11 PERFORMANCE COST PARETO CURVES
We show Pareto performance-cost tradeoff curves for all methods on ID queries across all bench- marks in Figure and on OOD queries across all benchmarks in Figure
C DATASET DESCRIPTION
The 9 benchmarks are: 1) AIME (MAA| 2024): A benchmark of competition math problems from American Invitational Mathematics Examination (AIME), which determines qualification for the United States Mathematical Olympiad, 2) MATH (Hendrycks et al. 2021c A benchmark of math problems drawn from various math competitions, 3) GPQA (Rein et al. 2024 A benchmark of PhD-level science multiple-choie questions (MCQs) written by domain experts, 4) LSAT (Wang et al. 2022 Zhong al. 2021): : A benchmark of MCQs from the three tasks of the Law School Ad- mission Test (LSAT), including analytical reasoning. logical reasoning and reading comprehension, 5) MMLU (Hendrycks et al. 202Tbla): A benchmark of MCQs from various branches of knowledge covering diverse domains, 6) MML.Reddux (Gema et al. [2024]: A subset of MMLU with manually corrected MCQs to remove errors from the original benchmark, 7) MMLU Pro (Wang al.) 2024): An enhanced MMLU benchmark with a focus on reasoning questions with increased answer options from 4 to 10, 8) DROP (Dua et al. 2019)): A benchmark of reading comprehension questions requir- ing discrete reasoning over the question's 's associated paragraph, and 9) FRAMES (Krishna et al.) 2024]: A benchmark of long-contex reasoning-base questions associated with multiple wikipedia articles. Table shows the statistics of each dataset.
C.1 PREPROCESSING DETAILS
Across datasets, we standardize formatting; compute prompt token counts with the Qwen/Q Owen/Qwen3-O-.. 6B tokenizer (no padding, truncation, or added special tokens); and discard items exceeding configured token budget. To prevent leakage, we compute content-based item key and apply deduplication when specified for a given dataset, with some datasets deferring duplicate handling to later analysis. Where applicable, numeric answers. All datasets are mapped into a unified compt-response format; detailed prompt templates are provided in Appendix D.4
normalize available metadata and extract missing
We preprocess AIME by standardizing sources and prompts, then filtering and deduplicat- ing. Training data span years 1983-20 1983-2023 while test data consist of the union of unique items from AIME 2024 and 2025 to reduce evaluation variance. Evaluating on AIME 2024 and AIME 2025 separately resulted in high evaluation variance even after averaging over multiple runs. Examples with prompt length exceeding the maximum token budget are discarded. For AIME 2025, only the problem text and numeric answer are retained; missing fields such as solution or difficulty are set to "NA."
AIME.
MATH. We construct the training split by combining the seven subject configurations of the MATH (7,500 problems total) and use the fixed 500-problem test set % Examples exceed- ing the maximum prompt length are removed. When an explicit numeric answer is missing, it is extracted from the provided solution. Metadata such as subject/type and level are normalized, and any available unique_id id is preserved. We shuffle the data with a fixed seed.
GPQA. We preprocess GPQA by combining the main and diamond subsets and verifying that diamond IDs are contained within the main set. Each example reformatted into multiple-choice format with options A-D, and the correct answer is recorded as a letter. Answer options are ran- domly permuted with fixed seed. Items exceeding the maximum prompt length are filtered out.
json json
24

---

Published as a conference paper at ICLR 2026
100
Average Performance (%)
Average Performance
AlHLarge
Grade
AlHLarge
AiHLarge
SlllSmall
Small
Randiamslll
Random-All
RADAR
All-Small
INT-Rlouter
All-Smal
Ro RtterBeench
RouterBench
Randiom-Par
Random-Par
Total Cost ($)
Total Cost (S)
(a) GPQA-Diamond
(b) MMLU
100
100
Average Performance (96
Al-Large
Average Performance
All-Lang
Dradle
Gracie
Large
Random-All
Random
RADAR
40-
NADAR
All-mmal
RouterfBench
Rauterflench
Random Pair
20-
Random Par
20
Total Cost
Total
(c) MML.U-Redus
(d) MML.U-Pr
100
Average Performance (%)
Alslarge
Average Performance (%))
80
Drade
Oracie
All-Large
All-
Aeeradaii 5mg)
Ali-Small
Ali-Small
20-
Random-All
RADAR
All-Smaj]
RAIDAR
IRT
RouterBeench
Randiom Pair
Random Pair
10
20
30
Total Cost ($)
Total Cost (S)
LSAT
AIME
100
Average Performance (%)
Al-Lange
Average Performance (96)
80-
Allgange
All-Large
All-Small
All-Small
RADAR
RADAR
All-mmal
IRT-Router
All Small
IRT-Router
RouterBench
RouterBench
Randism-Pair
Random-Par
0.0
0.5
1.0
Total Cost ($)
Total Cost (S)
(g) MATH-500
(h) FRAMES
Figure 6: We show the Pareto performance-cost tradeoff curves for all methods on ID queries across benchmarks. RADAR outperforms basclines, denoting better performance-cos! tradeoffs towards the Pareto front. Solid lines in the figure denote the convex hull corresponding to each curve.
25

---

Published as a conference paper at ICLR 2026
100
Average Performance (%)
Average Performance
AlHLarge
Crade
AlHLarge
AiHLarge
All-Small
Small
Randiamslll
Random-All
RADAR
All-Small
INT-Rlouter
All-Smal
Ro Rtttreeench
RouterBench
Randiom-Par
Random-Po
Total Cost ($)
Total Cost (S)
(a) GPQA-Diamond
(b) MMLU
100
100
Average Performance (96
Al-Large
Average Performance
All-Lang
Dradle
Gracle
Large
Random-All
Random
RADAR
40-
NADAR
All-mmal
RouterfBench
Rauterflench
Random Pair
20-
Random Par
20
Total Cost
Total
(c) MML.URR MMl.U-Redux
(d) MML.U-Pr
100
Average Performance (%)
AlHlarge
Average Performance (%))
80
Drade
Oracie
All-Large
All-
Aeeradaii 5mg)
Ali-Small
Ali-Small
20-
Random-All
RADAR
All-Smaj]
RAIDAR
IRT
RouterBeench
Randiom Pair
Random Pair
10
20
30
Total Cost ($)
Total Cost (S)
LSAT
AIME
100
Average Performance (%)
Al-Lange
Average Performance (96)
80-
Allg
60
All-Larg
All-Small
All-Small
RADAR
RADAR
All-mmal
IRT-Router
All 5mall
IRT-Router
RouterBench
RouterBench
Randism-Pair
Random-Par
0.0
0.5
1.0
Total Cost ($)
Total Cost (S)
(g) MATH-500
(h) FRAMES
Figure 7: We show the Pareto performance-cost tradeoff curves for all methods on OOD queries benchmarks. RADAR outperforms baselines, denoting better performance-cost tradeoff's wards the Pareto front. Solid lines in the figure denote the convex hull corresponding to each curve.
26

---

Published as a conference paper at ICLR 2026
Deduplication is performed using content- content-based key, and evaluation is conducted with the diamond subset as the test set.
LSAT. We preprocess LSAT by standardizing items from the official AR-LSAT release Sasaasse@@ppnnnnnn spanning reading comprehension, logical reasoning, and analytical reasoning. Each example is reformatted into multiple-choice prompt with options A-E. Items exceeding the token budget are discarded. We preserve the original section and split labels, and record the gold answer both as an index and as letter. Data are shuffled deterministically with fixed seed.
MMLU. We use only the official test split of MMLU Each example converted into a standard- ized multiple-choice prompt (options A-D), retaining both the textual correct answer and its letter index. Items exceeding the token threshold are discarded. Subject metadata are preserved, and the dataset is shuffled with fixed seed.
MMLU Pro. We use the public test split MMLU Pro Each example is constructed into a multiple-choice prompt with options A-J, and the gold answer is stored as a letter. Prompts exceeding a 32k token budget are discarded. The dataset is shufiled deterministically with fixed seed.
MMLU Redux. We preprocess MMLU Redux by aggregating all subject configurations and dis- carding items flagged with metadata error_ soree_tp_____ & ok. Each example is normalized and converted into a multiple-choice prompt (options A-D), with both the correct answer letter and text recorded. Items longer than the token budget are removed, and the dataset is shuffled deterministically with fixed seed.
FRAMES. We preprocess FRAMES by retrieving and cleaning the corresponding Wikipedia pages for each example. Cleaning removes site chrome, images, hyperlinks, citation markers, and irrelevant sections, while preserving tables and converting text to Markdown. Examples are then converted into document QA style format. Items exceeding the token budget are filtered out, and unique article texts are cached to avoid re-downloading.
D ADDITIONAL EXPERIMENTAL DETAILS
D. EVALUATION SETUP AND IMPLEMENTATION DETAILS.
We conduct both in-distribution (ID) and -iddttrbbbtitnoo (OOD) evaluations. For ID experi- ments, we aggregate the training splits of all 8 benchmarks into single training set for training the 2PI. IRT model (see Section 3.3p and report performance on the test split of each benchmark separately. We use an 80% 20% train-test split for benchmarks without a predefined test set. For OOD, for each benchmark, we aggregate the training splits of the other remaining non-overlapping benchmarks into single training set and report performance on the test split of this benchmark. For example, for the OOD experiment on AIME, the training split of AIME and of overlapping bench- marks (MATH since MATH includes questions from AIME), are held out from the training set. We route over 35 configurations comprising Open 04-mini (budgets: low, medium, high) and Qwen3 models (0.6B/1.7B/4B/BBB with budgets 0, 256, 512, 1k, 2k, 4k, 8k, 16k (Yang et al. 2025 OpenAT, 2025). Each configuration is evaluated once per training query with standard prompts (Appx. D.4); for AIME's s small test set, we average over eight runs. In total, we collected 1.75 million binary responses over 50,139 unique questions across train/test splits of all eight benchmarks.
27

---

Published as a conference paper at ICLR 2026
D.2 HARDWARE
For all open-source models, we use vLLM (Kwon et al. 2023) to host the model. All experiments involving open-source models are NVIDIA A100 80GB GPUs. Each model is hosted using a single such GPU.
D.3 BASELINES
2024), we adopt its k-nearest neighbors (kNN) parameterization, which model routing work, IRT- Router (Song et al. 2025). : by using the same 2PI. IRT model parameterization and query embed- der as RADAR, thereby improving its embedder handle long-context queries for fairness. For both model routing methods, RouterBench and IRT-Router, we adapt them to RLMs by using all RLMs their respective fixed maximum budgets, and use their performance-cos formulation simi- lar to linear scalarization (see Equation 3). In addition, we include simple heuristic-based baselines: All-Large (o4-mini high budget) and All-Small (Qwen3 0.6B zero budget) as approximate upper and lower bounds on performance and cost, respectively. The Oracle router, provided with model configuration performance on test queries, serves as idealized approximate upper bound of the performance-coss tradeoff by picking the cheapest best- best-performing configuration Random-All serves as a diversity baseline, selecting configuration at random to answer cach query. Random- Pair selects the largest configuration (o4-mini at high budget) with probability W1, and the smallest configuration (Qwen3 0.6B at zero budget) with probability Wi, where w1 the user-defined performance-cost tradeoff weight.
For RouterBench (Hu et al.
performs best (Chen et al. 2025 We adapt concurrent
D.4 QA PROMPTS
The prompts below are applied to all RLM configurations. For AIME and MATH, we use the following prompt:
{question]
Please reason step by step, and put your final answer within \boxed().
For GPQA, LSAT, MMLU, and MMLU Redux, we use the following prompt:
Answer the following multiple choice question.
fquestion}
{option_A}
B) (option_Bi B}
C {option_C)
{option_Ds
Please reason step by step, and put your final answer option within \boxed(). Only put the letter in the box, e.g. \boxed(A). There is only correct answer.
For MMLU Pro, we use the following prompt:
Answer the following multiple choice question.
[question)
[options]
Please reason step by step, and put your final answer option rithin \boxed(). Only put the letter in the box, e.g. \boxed(A). There is only one correct answer.
For FRAMES, we assemble the prompt programmatically that includes the context all documents relevant the question:
28

---

Published as a conference paper at ICLR 2026
prompt = f"""You are asked to read {len(docs) Wikipedia article extracts
2 and answer a question. Please reason step by step, and put your final
3 answer within \/boxed{}." "m.
4 for i, doc in enmmerateddocs):
prompt
f" Wikipedia article [i+1I:nfdooc)"
prompt += f" Question: (question)"
7 prompt += """ [n\neeeee reason step by step, and put your final answer within ||boxed[].
D.5 IRT IMPLEMENTATION DETAILS RADAR
We employ a two-parameter logistic (2PL) IRT model implemented as a custom PyTorch model class. Input queries first processed into fixed embeddings by frozen-weight Bttn//wen3-Embeddeeee
the dimension dg = 4096 for both the query embedding and the learnable weights Wa wh. Training runs for 100 epochs with learning rate x 10 batch size 32 for both training and evaluation, gradient clipping norm 1.0, and gradient accumulation of 1 step.
D.6 METRICS
Our formulation of adaptive reasoning as an MOO naturally lends the use of the hypervolume indicator metric (Emmerich & Deutz 2018), which measures the size of the dominated space re- covered by the MOO solution method, with higher value indicating performance close to the Pareto front. In our two-dimensional routing MOO, hypervolume intuitively measures the area under the performance-cost tradeoff curve recovered by the routing method for various values of tradcoff weights w1. An advantage of hypervolume over similar area-based metrics defined in exist- ing routing work (e.g.. AIQ in Hu et al. (2024) is its generalizability to measuring the performance of a multi-dimensional routing MOO. In future work, additional dimensions such as latency, bias, and carbon emissions can be added the routing MOO. We also formulate cost-performance threshold (CPT) metric, similar to the call-performance threshold metrics Ong et al. (2024), useful metric for real-world applications quantifying the cost required reach specified perfor- mance level. Given a performance threshold 2%, CPT(x%) measures the minimum cost required to achieve æ% of the performance of the largest configuration (OpenAI 04-mini with high reasoning budget). We normalize this cost to [0, 1] by dividing by the cost of running the largest configuration. Therefore, CPT(00%) of 0.1 implies that the routing method can match 90% of the performance of o4-mini high at 10% of its cost.
E THE USE OF LLMS FOR THIS PAPER
LLM usage is limited to writing and editing suggestions, such as word and phrase choices, high- lighting grammar issues the authors' draft, and polishing the writing. Other LLM usage includes LLM-augmented search engines to assist in identifying prior and concurrent related work.
29


Reviews:
[{"review_id": "AeLC845er9", "reviewer": "Reviewer_St66", "summary": "This paper presents a method for the reasoning language model (RLM) routing problem, which aims to select the best RLM configuration for an input query, given a pool of RLMs,  the reasoning budget, and a user performance-cost trade-off profile. It leverages an item response theory model to estimate the query difficulty and RLM ability at different reasoning budget, by parameterizing the query difficulty and training on collected evaluation responses. Under the user performance-cost trade-off profile, it uses multi-objective optimization to find the Pareto optimal of model configurations, which is also empirically shown to be effective.", "questions": "See weaknesses  above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The problem setting of user-desired performance-cost trade-off profile is interesting and practical.\n- The proposed approach of modeling query difficulty and solving the multi-objetive optimization problem is reasonable.\n- RADAR can generalize to OOD queries.\n- Comprehensive empirical evaluation on different benchmarks.", "weaknesses": "- Limited technical novelty against the baseline IRT-Router with conceptually similar approach of leveraging IRT.\n- It is not convincing that using a linear transformation on the embedding of questions could successfully model the underlying difficulty of the question. Using a more expressive parameterization would  intuitively give a better results given the high dimensionality of the embedding. However, no ablation has been shown to justify the linear transformation is sufficient.\n- The empirical gains are limited based on the results from Table 1, 2. The evaluation method CPT with only one threshold (i.e., CPT(90%)) is not convincing. More evaluation threshold should be shown to justify its effectiveness under different desirability.\n- Typos. E.g., there is a missing “running” before “configuration” in line 196.\n- Sec. 3.5 depends on a small set of evaluation queries, which can be adaptively updated. However, whether this set (queries with high Fisher information) approximates the global data distribution Q is unclear. Ablations on how this approach differs from uniform sampling or simply estimation from historical observations would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eFOaZ5Ce1a", "reviewer": "Reviewer_aypQ", "summary": "The authors proposed RADAR (Reasoning–Ability and Difficulty-Aware Routing), a lightweight, interpretable, and scalable routing framework. They firstly show with statistics that reasoning models today struggle to balance between performances and budgets. The authors then proposed to model this problem as a multi-objective optimization problem. With extensive evaluations on multiple benchmarks, the authors showed that their methods are superior to other methods, and the advantage could generalize to new RLM settings.", "questions": "1.As I said above, could you give an explanation of your results?\n2. The cost function looks simple. For example, how can RADAR incorporate real-time latency, KV-cache reuse, or batching discounts into its cost function?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well written and nicely presented.\n2. The method the authors proposed is simple, yet more reasonable to generalize.", "weaknesses": "1. The authors did not well explain their experimental results. For example, the gain looks marginal on some benchmarks while significant on others. The authors could help readers better understand why the biases exist.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cC7FC9cuXW", "reviewer": "Reviewer_EkVW", "summary": "This paper introduces RADAR (Reasoning–Ability and Difficulty-Aware Routing), a lightweight and interpretable routing framework that optimizes the trade-off between performance and cost when deploying reasoning language models (RLMs) with varying model sizes and reasoning budgets.\nRADAR draws inspiration from Item Response Theory (IRT) in psychometrics to estimate query difficulty and model-budget ability. Using these interpretable parameters, it routes easier queries to cheaper, smaller models and harder ones to more capable configurations. The authors formalize routing as a multi-objective optimization (MOO) problem balancing performance and cost via linear and Chebyshev scalarization.\nExperiments across 8 reasoning benchmarks (AIME, MATH-500, GPQA, LSAT, MMLU, MMLU-Redux, MMLU-Pro, FRAMES) demonstrate that RADAR achieves superior performance-cost Pareto efficiency, generalizes well to out-of-distribution queries, and scales effectively when adding new models. It operates in real-time with ~7 ms routing latency.", "questions": "Check weakness above.\n1. A comparison/discussion with previous works on selecting the best answer from different LLMs would have been useful for latency. \ni. Uncertainty-Aware Answer Selection for Improved Reasoning in Multi-LLM Systems.\nii. Scalable best-of-n selection for large language models via self-certainty. \n2. Which embedding model was used to represent queries?\n3. Could RADAR dynamically adjust the user weight w₁ during a session (based on feedback or budget exhaustion) to continuously optimize global cost-performance?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "First to frame reasoning model routing as a Multi-Objective Optimization (MOO) problem. Innovative use of Item Response Theory for modeling reasoning ability and query difficulty, an elegant and interpretable alternative to opaque routing regressors.\n\nThe methodology is rigorous, combining psychometric modeling with MOO and cost modeling in a consistent mathematical framework.\nExperiments are extensive, covering 8 benchmarks and both in-distribution (ID) and out-of-distribution (OOD) settings, with clear comparisons to baselines such as RouterBench and IRT-Router.\n\nThe paper is very well written and structured, with clear motivation.\n\nThe work offers a generalizable and practical approach for efficient LLM deployment under cost constraints.", "weaknesses": "1. While RADAR clearly outperforms IRT-Router and RouterBench, the paper does not compare against mixture-of-experts or test-time adaptation methods that could serve as strong baselines for dynamic routing under cost constraints.\n2. More discussion on relation to cascading methods (e.g., FrugalGPT) would strengthen the positioning.\n3. Although latency is reported (~7 ms), no end-to-end system-level throughput analysis is provided. \n4. Some OOD performance dips (e.g., AIME benchmark) suggest the need for improved handling of unseen high-difficulty queries.\n5. The model for estimating difficulty is too simple and may not capture the real question difficulty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fB6sJaDdf6", "reviewer": "Reviewer_LEBC", "summary": "This work considers the problem of query-based routing reasoning language models (RLM) with variable budget. \nThe authors here formulate such problems into multi-objective optimization (MOO) paradigm with certain scalarization (linear/Chebyshev) for both performance and cost. \nThe MOO's variables include 1) model-budget configuration and 2) query difficulty.\nModel-budget configurations are (model, token) pairs with reasoning token count discretized and the authors propose to leverage IRT (item response theory) to learn the difficulty/discrimination projector with a frozen embedding model and a model-configuration _ability_ parameter to maintain both interpretability (low parameter) and low latency during inference.\n\nThis paper is well written and the idea is well presented with good motivation and strong results.", "questions": "Same as weaknesses, would love to see some experiments on some more capable difficulty/discrimination projectors or a bit more complex model than IRT. While it is very reasonable and probable to end up with the conclusion that these more complicated model could obfuscate the interpretability and spike up latency while yielding only marginal performance, it will further strengthen the argument for RADAR.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Well-motivated: authors propose a method to achieve pareto frontier in terms of performance vs. cost in the case of multiple RLMs with varying size and reasoning budgets. This is a very realistic scenario and could be rather impactful. In addition, the proposed method is easy to adapt to any sort of models as they are treated as black boxes and readily extensible to newer and hopefully stronger or faster models.\n2. Good performance: the authors compare RADAR against IRT Router and RouterBench and a couple of heuristics and show that RADAR achieves better hypervolume metric.\n3. Fast: for each inference (unseen) query, RADAR performs an embedding step through an embedding model and a proposed linear projector to get the difficulty/discrimination scalars and feed these 2 scalars into the IRT model. This is rather fast in comparison to the later pipeline considering the number of parameters and reasoning budget of the following models.\n4. Interpretability: IRT model only has two parameters (difficulty & discrimination in terms of input query) which affords natural interpretability.", "weaknesses": "The IRT model used in the paper assumes monotonicity, and the projectors are linear with the input embedding model frozen. I know it is probable that using a more complex model could result in a mere marginal improvement in terms of hypervolume, but would like to see some results in this regard. So is the case with the IRT model", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)