SLM-MUX: Orchestrating Small Language Models for Reasoning
Authors: Chenyu Wang
Zishen Wan
Hao Kang
Emma Chen
Zhiqiang Xie
Tushar Krishna
Vijay Janapa Reddi
Yilun Du
Venue: ICLR 2026 Poster (2026)
Keywords: large language model
small language models
Primary area: foundation or frontier models, including LLMs

Abstract: With the rapid development of language models, the number of small language models (SLMs) has grown significantly. Although they do not achieve state-of-the-art accuracy, they are more efficient and often excel at specific tasks. This raises a natural question: can multiple SLMs be orchestrated into a system where each contributes effectively, achieving higher accuracy than any individual model? Existing orchestration methods have primarily targeted frontier models (e.g., GPT-4) and perform suboptimally when applied to SLMs. To address this gap, we propose a three-stage approach for orchestrating SLMs. First, we introduce SLM-MUX, a multi-model architecture that effectively coordinates multiple SLMs. Building on this, we develop two optimization strategies: (i) a model selection search that identifies the most complementary SLMs from a given pool, and (ii) test-time scaling tailored to SLM-MUX. Our approach delivers strong results: Compared to existing orchestration methods, our approach achieves up to 13.4% improvement on MATH, 8.8% on GPQA, and 7.0% on GSM8K. With just two SLMs, SLM-MUX outperforms Qwen 2.5 72B on GPQA and GSM8K, and matches its performance on MATH. We further provide theoretical analyses to substantiate the advantages of our method. Additional experiments show that the core principle of SLM-MUX extends to open-ended generation tasks (e.g., HumanEval) and benefits other model classes, including frontier LLMs and domain-specific fine-tuned SLMs. In summary, we demonstrate that SLMs can be effectively orchestrated into more accurate and efficient systems through the proposed approach.

Published as a conference paper at ICLR 2026
SLM-MUX, ORCHESTRATING SMALL LANGUAGE MODELS FOR REASONING
Chenyu Wang 197
Zishen Wan
Hao Kang²
Emma Chen'
Zhiqiang Xie
Tushar Krishna
Vijay Janapa Reddi'
Yilun Du'
Harvard University
Georgia Institute of Technology
Stanford University
ABSTRACT
With the rapid development of language models, the number of small language models (SLMs) has grown significantly. Although they do not achieve sfate-of-the- art accuracy, they are more efficient and often excel at specific tasks. This raises a natural question: can multiple SLMs be orchestrated into system where each contributes effectively, achieving higher accuracy than any individual model? Ex- isting orchestration methods have primarily targeted frontier models (e.g.. GPT-4) and perform suboptimally when applied to SLMs. To address this gap. we propose a three-stage approach for orchestrating SLMs. First, we introduce SLM-MUX, multi-model architecture that effectively coordinates multiple SLMS. Building on this, we develop two optimization strategies: (i) a model selection search that identifies the most complementary SLMs from given pool, and (ii) test-time scaling tailored to SLM-MUX. Our approach delivers strong results: Compared to existing orchestration methods, our approach achieves up 13.4% improvement on MATH, 8.8% on GPQA, and 7.0% on GSM8K. With just two SLMs. SLM-MUS outperforms Qwen 2.5 72B on GPQA and GSM8K, and matches its performance on MATH. We further provide theoretical analyses to substantiate the advantages of our method. Additional experiments show that the core principle of SI.M MUX extends open-ended generation tasks (e.g.. HumanEval) and benefits other model classes, including frontier LLMs and comain--peciffe fine-tuned SLMs. In sum- mary, we demonstrate that SLMs can be effectively orchestrated into more accurate and efficient systems through the proposed approach.
INTRODUCTION
Recent years have witnessed a surge of small-sized language models (SLMs) containing billions to tens of billions of parameters (Wang et al., 2024a; Javaheripi & Bubeck, 2023; Guo et al., 2025; Allal et al., 2025). While these models may underperform state-of-the-art frontier language models, which usually contain hundreds of billions offer substantially lower inference costs, are more affordable to train and fine-tune, and allow edge deployment due their small size (Belcak et al., 2025). Meanwhile, frontier models have reached villion-paramete scales where further increases in size and training data yield diminishing returns. This mirrors well-known challenge in computer architecture two decades ago: when enlarging single CPU cores no longer delivered proportional performance gains, computer architects turned to designing multi-core processors, where multiple smaller cores working together enabled sustained improvements. This parallel suggests that combining multiple SLMs could offer a promising alternative to scaling ever-larger frontier models.
trillions of parameters,
any given query, they
Recent works have explored orchestrating multiple LLMs (e.g.. GPT-3.5 and GPT-4o), combining them into one system process an input collaboratively. Representative approaches include Mixture- of-Agent (Wang et al., 2024b), LLM-Debate (Du et al., 2023), and Multi-Agent Verification (Lifshitz et al., 2025). These approaches share key assumption: that models possess strong reasoning and deliberation abilitics, so that interaction through natural language can reliably correct mistakes. However, when applied to SLMs, this assumption longer holds. Our study finds that such discussion-based orchestration often fails to improve performance for SLMs, and some cases even
"Equal contribution.
Cooresponnence chenyu_wang@seed ha saavard.ccom edu, z shenwan seas. www.haa.vaa..... edu.

---

Published as a conference paper at ICLR 2026
reduces accuracy by over 5%. Instead of correcting mistakes, SLMs tend to fall into groupthink during interaction, amplifying errors rather than mitigating them. The assumptions that language models can correct each other's 's answers behind existing orchestration methods do not hold for SLMS (Taubenfeld et al., 2024; Huang et al., 2024; Liu et al., 2023; Fu et al., 2025).
To address this issue, we propose SLM-MUS a multi-model architecture for effectively orchestrating SLMs while avoiding explicit text exchanges between models. Our key insight is that SI.M-MUX leverages complementary abilities from different models by selecting outputs based on confidence scores without any model training.
After introducing SLM-MUX, another question arises: which models should be orchestrated to- gether? Not all combinations are effective if one model is weaker across all dimensions, it provides no benefit when paired with a stronger one. In contrast, combining models with com- plementary strengths (c.g., one stronger in alge- bra, another in geometry) allows the system to succeed where a single model would fail.
62.5
Better
Agent Forest
60.0
Accuracy (%)
SLM-Mux (Ours)
57.5
55.0
Lluma-SC
52.5
MoA
Debate
50.0
Llama
MAV
10s
10²
To address this, we develop a model selection search strategy for SLM-MUX, which system- atically evaluates and identifies model subsets with complementary strengths. By maximizing union accuracy while penalizing overconfident contradictions, the search procedure finds the most suitable models for a given model budget.
Token Usage
Figure 1: Head-to-Heade Comparison of SLM-MUX with Other Methods. SLM-MUX outperforms exist- ing methods such as Self- Coniisteen (SC) (Wang et al., 2023), Mixture-of-Agent (MoA) (Wang et al., 2024b), LLM-Debate (Du al.. 2023). Multi-Agent Verification (MAV) (Lifshitz al., 2025), and Agent Forest (Li al., 2024). Results reported on MATH dataset with SLMs.
In addition, we explore compute scaling strate- gies for the selected model ensembles further enhance performance. By adjusting the number of models and samples at inference time, we further boost performance and identify practical sweet spots in the accuracy-computt tradeoff.
Our experiments demonstrate significant improvements across multiple benchmarks. By combining only two SLMs. we achieve accuracy improvements of up to 6.7% on MATH, 5.7% GPQA, and 4.8% on GSMSK, compared to the best-performing single SLMs in the system. Our method consistently outperforms existing discussion-based approaches for SLMs, with gains of up 13.4% on MATH, 8.8% on GPQA, and 7.0% on GSM8K. Most importantly, with just two SLMs, SLM- MUX outperforms Qwen 2.5 72B on GPQA and GSM, and matches its performance MATH.
Finally, we complement these empirical findings with theoretical and experimental analyses. Our approach shows superiority in multiple scenarios compared with previous methods (Figure 1).
Our main contributions are as follows: (i) We identify a fundamental limitation of existing orchestration methods: Through systematic evaluation, we demonstrate that existing discussion- based methods, which show consistent improvements for frontier LLMs, actually harm performance when applied to SLMs. This counterintuitive finding challenges the assumption that orchestration methods transfer across model scales and reveals the need for SLM-specific method. (ii) We propose SLM-MUX, novel multi-modee architecture designed specifically for SLMs that avoids the error amplification problems of daccussion-based methods. SLM-MUX achieves consistent gains across multiple benchmarks (MATH, GPQA, GSMSK) and significantly outperforms existing discussion- based methods by large margins (up to 11 1.6% on MATH). (iii) We develop principled optimization strategies for the LUX, including model selection search that identifies complementary model selections and compute scaling strategies, further boosting performance while maintaining efficiency.
2 RELATED WORK
Discussion-based Orchestration Methods. We use discussion-basse orchestration to refer to orchestration schemes where multiple LM instances exchange evaluate natural-language mes- sages (Fu et al., 2025)-225500 -ucch as proposing answers, critiquing or debating, verifying from different aspects, and finally aggregating into one output. Representative approaches include Mixture-of Agents (Wang et al., 2024b), which uses a dedicated LLM to aggregate outputs from several models; L.LM-Debate (Du al., 2023), where models critique and refine each other's reasoning; and Multi-
2

---

Published as a conference paper at ICLR 2026
Question
Question
Question
Question
Proposer
Propaser
Proposer
Proposer
Proposer
Proposer
Proposer
Proposer
Aggregstor
Aspezt Venifer
Aspazi Veriber:
Aspezi Veniierr
Aupazt Venifier
Non-I.1.M Protocol Aggiogator
Rost
Best
(a) Mixture-of-Agent Paradigm
LLM-Debate Paradigm
Multi-Agent Verification Paradign
SLM-Mux (Ours)
Figure 2: Comparing SLM-MUX (Ours) with Existing LLM Orchestration Methods. (a) Mixture-of-
Agents, (b) LLM-Debate, (c) Multi-Agem Verification, (d) SLM-MUX (Ours).
Agent Verification (Lifshitz et al., 2025), which assigns models independently evaluate candidate solutions before selecting the final answer. These methods assume that participating models have sufficient reasoning ability to self-correct through interaction. Prior evaluations have been conducted on frontier LLMs, while their effectiveness for SLMs remains unstudied.
Optimization for Multi-LM Orchestration. Given these orchestration methods, some works study how to further improve their performance e.g.. how to select models to include, how to optimize prompts, how to adapt the architecture for specific tasks (Chen et al., 2023a; Ong al., 2025; Chen et al., 2024). Prompt and workflow optimization methods (Khattab et al., 2023; Opsahl- et al., 2024; Saad-Falcon et al., 2025; Zhang et al., 2025a) generally assume strong catruction-following ability, which makes them less effective for smaller models with limited such capabilities..
Another line work is model selection for orchestration (Chen et al., 2025; Poon et al., 2025). These methods often select models based on accuracy, assuming that combining models with higher standalone accuracy will yield stronger orchestrations. However, most selection criteria are not end- to-end: they evaluate models independently without directly assessing the performance of the overall orchestration. This overlooks how models interact with each other- overconfiden but incorrect predictions from one model can dominate and suppress correct predictions from others, meaning that the best standalone models may not yield the best orchestration.
Test-time Scaling Strategies. Test-time scaling methods improve performance by using additional computation during inference without retraining (Snell et al., 2024; Muennighoff al., 2025; Zhang et al., 2025b). A common single-model approach is self-consistency (Trad & Chehab, 2025; Thirukovalluru et al., 2024; Chow et al., 2024), which draws multiple samples from one model and selects the majority answer; accuracy typically improves as the number of samples increases. Agent Forest (Li et al., 2024) extends this idea to multiple models by collecting one output from each model and applying majority voting across all answers.
METHODS
In this work, we set out ask two critical questions: given a pool of available SLMs, how can we (i) orchestrate their outputs to achieve the best overall performance, and (ii) select an effective subset of models that maximizes accuracy?
To answer question (i), we present the SLM-MUX (Section 3.1), a simple yet effective orchestration method. To answer question (ii), we propose model selection search (Section 3.2) that identifies complementary subsets from dozens of available SLMs. Finally, we explore compute scaling strategies (Section 3.3) to further enhance the reasoning accuracy during inference.
3.1 SL.M-MUX FOR ORCHESTRATING MULTIPLE SMALL LANGUAGE MODELS
At a high level, our intuition is that we do not need to let SLMs discuss with each other. Instead, we can develop simple rule-based method that estimates the confidence of each model's answer and then selects the final output from the model with the highest confidence. We term our method SLM-MUX, which operates in two phases.
Independent Generation Phase. For given question, we first let each SLM independently generate multiple candidate responses to the same query prompt with temperature > 0, producing pool of sampled answers per model.
3

---

Published as a conference paper at ICLR 2026
Question: Which the following physical theories never requires regularization at high energies?
(A) Superstring Theory
(B) Classical Electrodynamics
(C) Quanturn mleutrodymummis (QED)
(D) Quantum Chromodynamies (QCD)
Correct Answer: (A)
Independent Generation Phase
SLM 1:
SLM 2:
SLM 3:
Output 1: the final answer (A)
Output I: ... the final
Output I: the final answer is (A)
Output 2: the final answer is (A)
Output 2: the correct choice is (B) X
Output 2: the correct choice is (B)
Output 3: therefor the correct
Output 3: I think the best answer is (B)
Output 3: think the best answer
answer is (B)
(C)
Reliability Estimation Phase
Confidence: 67% Historical Accuracy: High
Confidence: 67%
Confidence: 33%
X
Historical Acuuaay: Low
Historical Accuracyy Low
Illustration of SLM-MUX Workflow. (1) Each SLM first independently generates multiple outputs for the same question. (2) The most frequent answer from each SLM is selected, and its frequency in the answer pool used as the confidence score. (3) The answers with the highest confidence score are selected. (4) multiple answers share the same confidence score, the tic is broken by selecting the answer from the SLM with the highest accuracy on the validation set.
Figure 3:
Algorithm SLM-MUX Working Flow
Input: Models Mi...., , Mn, query z, samples per model k, validation accuracies d Un Output: Final answer
Independent Generation: each model produces multiple candidate answers independently
for = .,...,n do
2: Sample k answers Y; = {v; :i
(1)
(k) from Mj
3: Compute f((y) = Σ) 1/ =
4: Let vi arg max, ft) and s; f((
Confidence Estimation: measure confidence and break ties by validation accuracy
5: Smax max; 84, 1 {il 8t Smax
6: if [|] then
7: i" <- the unique index in I"
8: else
9:
+- arg maxig7 7 Og
10: return ==: -y.
Confidence Estimation Phase. We evaluate the confidence of cach SLM's outputs by measuring their consistency across their own outputs. Intuitively, a model that places higher probability mass on the correct answer will reproduce equivalent answer across samples, whereas uncertain model will produce varied outputs. For instance, if SLM produces three equivalent answers while model B produces three different ones, the answers from model A are more consistent and should be selected. This correlation between consistency and correctness is observed by previous papers. (Wang et al., 2023; Xie al., 2024; Taubenfeld al., 2025; Chen al., 2023b), and we empirically revalidate this observation in Appendix D.I.
In cases where two SLMs are equally consistent but disagree, we use their validation accuracy as tie-breaker. Prior work has shown that consistency is strongly correlated with correctness, which provides a rationale for this design.
For more details, Algorithm summarizes the workflow step by step. Figure 3 provides a visual example of the workflow. The evaluation SLM-MUX is presented in Section 4.2.
3.2 MODEL. SELECTION SEARCH FOR MUX OPTIMIZATION
At a high level, the idea of model selection search to combine models with complementary skills. The goal is not simply to add more models, but bring capabilities add models. Figure 4 illustrates this intuition: Qwen2.5-78 consistently outperforms Llama3.2-33 3.23B across all subjects, so combining them offers capability beyond what Qwen2.5-7B already provides. In contrast, Mistral
4

---

Published as a conference paper at ICLR 2026
Small 24B and Qwen2.5-7B Mistral Small 24B and Qwen2.5-7B excels in different subjects, making their combination more effective than either model individually.
We frame model selection as a search problem on the validation set with two competing objectives. Our first objective is Union Accuracy, which reflects the overall accuracy of the system. The higher the union accuracy is, the more questions a system can potentially answer. Formally, let denote the of candidate models and D the validation set. For each model my E M, we record the subset of validation instances it solves correctly. Given a candidate subset S C M, the union accuracy defined as
M {m......;
UnionAce(S))
1 ES::::: m(x)) is correct) D
RED
The second objective is the Contradiction Penalty. It captures problematic cases where overconfident wrong answers suppress correct predictions from other models. Consider two SLMs answering the same multiple-chore choice question three times: the first model consistently outputs "A" (correct), while the second consistently outputs "B" (incorrect but confident). Since SLM-MUX selects based on consistency, both models would appear equally confident, making it impossible to distinguish the correct answer from the confident but wrong one. We define this penalty as the percentage of questions where at least one model consistently gives the wrong answer while another provides the correct answer:
3m1 E S:: :::mmaa m1(x) consistently wrong, [D] Σn
1
Contradiction(S)-
3m2 SS: m2(x) correct
The final objective balances these competing factors:
O(S) UnionAcc(SS) A Contradiction
Where À is hyperparameter. Since the number of candidate models is not very large, we perform an exhaustive search. We present visualization the two search objectives and evaluation of the searched model selection in Section 4.3.
The rationale behind this search objective is follows: UnionAce represents optimistic upper bound for -UX performance. assumes an ideal selection mechanism capable identifying the correct answer whenever at least model provides it, which is unrealistic in practice. Conversely, when X = 1, the search objective represents a pessimistic lower bound of SLM-MUX accuracy. This setting assumes that in cases involving confidently wrong answers, the system will invariably select the incorrect one. In practice, due to factors such as tie-breaking rules and the presence of confidently correct answers, such a worst-case scenario will not always happen. Consequently, by employing the objective (SS) (Sc(S) - X Contradiction(S) we estimate an approximate accuracy between the theoretical upper and lower bounds of the SLM-MUX accuracy.
3.3 COMPUTE SCALING STRATEGIES
Next, we empirically investigate two dimensions of test-time scaling to fur- ther enhance the performance of our SLM-MUX with selected models.
G
D
9.20 4.600..
0.20 60..0.....
Adding More Participating Model Types: As we scale the model participating model types used in the system by adding more SLMs with complementary strengths, we expect the overall accuracy to improve. For each budgeted number of models, we use the search method proposed in Section 3.2 to identify the best selection from the pool.
F
F
Qeeen Small
Figure 4: Comparison of Model Choices. Accuracy on subjects for two model selection settings on MATH dataset. Subjects are denoted as: A = Prealgebra, B = Algebra, C= Intermediate Algebra, D = Number Theory, E = Counting & Probability, F Geometry, G Precaleulus.
Drawing More Samples per Model:
For a fixed model selection, we can
increase the compute budget by scaling the number of samples drawn by each model. Since confidence evaluated by counting the frequency of majority answers, adding more samples per model is expected to provide more accurate confidence estimate.
5

---

Published as a conference paper at ICLR 2026
These two compute scaling dimensions evaluated in Section 4.4.
4 EXPERIMENTS
In our experiments, we first demonstrate the fundamental limitations of existing discussion- based orchestration methods when applied to SLMs (Section 4.1). We then evaluate the proposed SLM- MUX in Section 4.2. In Section 4.3, we assess our proposed search strategy. Finally, in Section 4.4, we examine the compute scaling strategies.
4.1 EXISTING DISCUSSION-BASS ORCHESTRATION METHODS HARM SLM PERFORMANCE
To understand whether orchestration methods developed for frontier LLMs are suitable for SLMs, we conduct a systematic comparison across model scales. We evaluate three prominent discussion- based methods LLM-Debate (Du et al., 2023), cexture-of-Agemt (Wang et al., 2024b), and Multi- Ageet Verification (Lifshitz et al., 2025) using identical experimental settings both SLMs (Llama 3.1 8B (Jiang et al., 2024), Mistral 8x7B (Grattafiori et al., 2024), Gemma 2 27B) and frontier L.MMs (DeepSeck V3 (DeepSeck-A)) al., 2025), Gemini 2.0 Flash (Google Cloud, 2025). GPT-4o (OpenAI et al., 2024)). Evaluation is conducted on MATH and GPQA datasets using original code and prompts.
Results, As shown in Figure 5, discussion-basee methods generally outperform the single best- performing models in the frontier LLM group, achieving up 2% increase accuracy. However, when applied to SLMs, these discussion- baased methods fail to outperform the best single model in the orchestration, and even incur accuracy drops of up 5.5%. This performance gap observed across all three methods and both benchmarks.
To understand this counterintuitive result, analyze SLM behavior in discussion settings. We find that discussion-based methods amplify rather than correct errors in SLMs due to a key limitation: SLMs tend exhibit groupthink, reinforcing incorrect reasoning during discussions rather than correcting mistakes. In Appendix C, we provide detailed analysis showing that 59.5% of failures attributed to groupthink, and that the performance gap persists even after extensive prompt optimization.
Single Model Max
Mixture-of-Agents
LLM-Debate
Multi-Agent Verification
90
90
90.1
90.8
80
80
Accumey (%)
Accurccy (%)
-0
70
63.6
65.6
64.2
60
56.8
60
$8.6
50
46.2
51.6
48.4
50
38.8
35.1
40
30
30
MATHI
GPQA
MATH
GPQA
(a) SLM Orchestration
(b) Large-Scale Scale Model Composition
Figure 5: Comparison of discussion-barn orchestration when invoking SLMs and LLMss We compare three orchestration methods Mfituure-ffAAeente LLM-Debate, and Verification) using SLMs (Llama 3.1 8B, Mistral 7B, Gemma 27B) and (b) frontier LLMs (DeepSeek V3, Gemini 2.0 Flash, GPT-4o) on the MATH and GPQA datasets. The baseline (Single-Modes Max) reflects the best performance individual models. An orchestration is considered successful if it surpasses Single-Mode, Max. All discussion- methods are evaluated with temperature=0. The standard deviations of the accuracies are presented in Appendix B.3.
4.2 SLM--MUX ACHIEVES SI.M ORCHESTRATION WHERE EXISTING METHODS FAIL.
To evaluate whether our proposed SLM-MUX can suc- cessfully orchestrate SLMs, we test it against the same basclines from Section 4.1. We use Mistral 8x7B, LLaMA 3.1 8B, and Gemma 27B (Team et al., 2024) as base mod- els. We implement the SLM-MUX as follows. First, we generate three rounds answers with a temperature of 0.3. Next, we compute confidence score by counting how often the most common answer appears across these rounds. The final answer for each model is chosen as the most frequent one; in the case of tie, we select the answer from the model with the highest validation accuracy.
21.2%
30.0%
No Ehsogneeeent
Lama 3.1
Figure 6: Final Output Attribution. We report the percentage of outputs contributed by each model the MATH dataset for SLM- MUX. These results are from the same run as in Table I.

---

Published as a conference paper at ICLR 2026
We evaluate three types of baselines. First, we measure
the accuracies of individual models and report the best-
performing ones. Second, we apply self-consistency each of the three base models independently. reporting the best-performing result as the Single-Best-Se baseline. Next, for comparison with exist- ing discussion-based methods, we include I.LM-Debate (Du al., 2023), Mixture-of-Agent (Wang et al., 2024b), and Multi-Agent Verification (Lifshitz et al., 2025). We follow the original code and prompts described in their papers. Experiments are conducted on three benchmark datasets: MATH (Hendrycks et al., 2021), GPQA (Rein et al., 2023), and GSM8K (Cobbe et al., 2021).
Results. Table 1 summarizes the results. In our experiments, we find that for SLMs, existing orchestration methods do not consistently outperform the strongest individual base models or self-consistency approaches. In contrast, our SLM-MUX generally achieves an accuracy gain. Compared with other approaches, our method yields up to a 13.4% improvement on MATH, up to 8.8% on GPQA, and up to 7.0% on GSM8K. These results demonstrate that the SLM-MUX itself provides a clear advantage over alternative orchestration approaches at the architectural level.
To better illustrate our proposed SLM-MUX we plot the output attribution for the MATH experiment (Table 1) in Figure 6. By selecting diverse outputs from the generation, SLM-MUX leverages the complementary strengths of different SLMs.
Method
MATH Ace (%)
GPQA Acc (%)
GSM8K Acc (%)
Mixture-of of-Agetts
51.4 + 2.2
33.3 3,4
81.6 + 1.7
L.LM-Debate
51.6 2.2
36.8 3,4
80.8 + 1.8
Multi-Agent Verification
48.4 2.2
35.3 3.4
86.4 + 1.5
SLM-MUX (Ours)
61.8 1.2
42.1 0.3
87.8 + 0.6
Single-Best Single-Best-SC
56.8 + 2.2
38.9 + 3.5
84.2 + 1.6
58.0 + 2.2
42.4 + 3.5
86.8 + 1.5
Table Quantitative Results. Accuracy and standard emror across MATH, GPQA, and GSM8K. "SC" denotes self-consisten consistency decoding (majority vote over samples from a single model), and "Single-Beste reports the highest accuracy among the three base models when each applies self elffcconistency individually.
self
4.3
MODEL SELECTION SEARCH BOOSTS SLM-MUX PERFORMANCE
Search Objectives for GPQA
Search Objectives for GSMBK
Search Objectives for MATH
0.5
0.70
Obgedion Valun
0.3
2.58
Objective Value
0.50
0.2
0.96
0.66
0.64
0.2
Unica Accuracy
3.53
Unist Acturayy
0.82
Unios Avcuracy
Contradition Feralty
Contradaction Penalty
Contradiatiun Penally
-0.3
3.50
Figure 7: Union Accuracy and Contradiction Penalty both Increases as more models are added. We plot the search objectives the number of models (K) increases from 2 5 across three benchmarks. The green line denotes the union accuracy across models, the grey area indicates the contradiction penalty, and the blue line represents the overall search objective score. For each value of K. the plotted quantities are computed for the single model combination that maximizes our model selection objective defined in Section 3.2.
To examine whether model selection search benefits SLM-MUX, we construct a validation set of 500 questions sampled from the training splits of MATH GPQA, and GSMSK. The candidate pool consists of five SLMs: Gemma 2 27B. Llama 3.1 8B, Mistral Small 24B (Mistral AI, 2025), Mixtral 7B, and Qwen2.5 7B (Qwen et al., 2025). For each question, we collect three independent generations per model with temperature 0.5, repeating this process three times to obtain stable accuracy estimates. The search procedure considers orchestrations with K = 2 to 5 models and is guided by an objective function mentioned in Section 3, with hyperparameter À = 1. The behavior of this objective is illustrated in Figure 7, showing the trade-off as K increases. For simplicity, we select two representative two-model combinations from the search results for evaluation on the test set.
Results. Table summarizes the outcome of the search. The table lists the top perfforrnnnng two-model combinations identified on the validation set, along with their evaluation the held-out test set. Across benchmarks, these optimized orchestrations yield consistent improvements over the strongest individual models: accuracy increases by 4.5% MATH, 4.4% on GPQA, and 4.3% GSM8K.
7

---

Published as a conference paper at ICLR 2026
Best Single (Acc. %)
Composed
Benchmark
Group
Model Selection
(Acc. %)
(Gain)
Mistrul Small 24B
MATH
1
5wen2.5 7B
75.5 # 1.5
80.0 4: 0.7
4.5
Qwen2.5 7B
2
75.5 + 1.5
77.7 0.7
22.2
Ilama 3.1 88
Gemma 27B
GPQA
Mistral Small 24B
45.1 + 2.8
49.5
Llama 3.1 8B
2
Mistral Small
45.1 2.8
48.8
3.6
Mistrul Small 24B Qwen2.5
GSM8K
1
88.5 +
92.8 + 0.6
44.
Llama Mixtrul 8
2
80.8 + 2.1
85.2 4 0.7
4.4
Table 2: Model Selection Search and Evaluation Results. We show the top two model groups identified by our scarch for each benchmark. For each group, report the accuracy of the best-performing single model within the orchestration, the accuracy achieved by our SLM-MUX, and the resulting performance gain.
This contrasts with Section 4.2, where naive three-model combinations provide little to no benefit GPQA. Figure 7 further illustrates the underlying trade-off: while union accuracy rises with additional models, the contradiction penalty also grows, emphasizing that effective orchestration requires balancing these competing factors rather than simply enlarging the orchestration size. In Appendix D.3, we show that the SLM-MUX architecture itself yields consistent gains even with randomly selected model combinations; the search procedure provides an effective and data-efficient way to further boost accuracy.
4.4 COMPUTE SCALING STRATEGIES REVEAL OPTIMAL RESOURCE ALLOCATION
To evaluate the "Adding More Participating Model Types" dimension of compute scaling, we assess how performance changes as the number of models in the orchestration increases, For each number of models from 5, first apply the search method from Section 3.2 to identify the optimal model selection from our pool. We then evaluate SLM-MUX with selected models on the validation set. Figure plots the resulting mean accuracy (blue line, left y-axis) for each value of K. To illustrate the theoretical performance ceiling of each ensemble, we also plot the union accuracy (grey line, right y-axis), defined as the percentage of questions solved by at least one model the group. For each value of K in Figure 9, show the single model combination that achieves the highest value of our model selection objective from Section 3.2; the search procedure used to find the best combination under fixed K, rather than choose K itself.
Benchmark
Samples
SLM-MUX
Agent Forest
(Gain)
76.8 = 0,7 79.5 = 0.4
72.3 + 1.5 79.2 + 0.4
+4.5
MATH
Best
+0.3
2 Best
46.3 = 2.3
40.4 4: 2.3
+5.9
GPQA
48.8 = 1.2
17.6 + 1.4
+1.2
82.1 = 0.7
77.7 1: 0.2
+4.4
GSM8K
Best
86.5 = 0.8
84.3 + 0.8
+2.2
Table 3: Comparison of SLM-MUX and Agent Forest. We compare SLM-MUX and Agent Forest in two settings: (1) with 2 samples model (Samples=2), and (2) using the best accuracy found during scaling for each method (Samples=best) In the second setting. the number of samples per model may vary.
For the "Drawing More Samples per Model" dimension, we reuse the two groups of models listed in Table 2. We vary the number of samples per model from 2 to 9 and report the mean accuracy of SLM-MUX over three runs for each sample budget. The results are presented in Figure 8, along with a baseline, Agent Forest (Li et al., 2024), for comparison. To ensure fairness, Agent Forest reproduced using the same models from Group 2. We report the best accuracy achieved by the SLM-MUX when scaling with Samples per Model and compare it to the accuracy of the single best model in the orchestration. as shown in Table 2.
Results. The effect of "Adding More Participating Model Types" varies substantially across bench- marks. On GPQA, accuracy peaks when combining two models and declines thereafter. On GSM8K, accuracy quickly saturates two models without further gains. In contrast, MATH, accuracy
8

---

Published as a conference paper at ICLR 2026
Drawing More Samples per Model GPQA Drawing More Samples Model for GSM8K Drawing More Samples Model for MATH
40
447
88-
Accuracy
86
88
82-
41
42
Group
Group 1
80
Covap
Covup 2
73
Group
Croup
Ageni Porrot
75
Agest Porest
72
Agunt Porest
Samples per Model Figure 8: Drawing More Samples per Model Improves Accuracy. We report mean accuracy of SLM-MUX the number of samples per model increases from 2 across three benchmarks. Group 1 and Group from Table 2. We also plot the mean accuracy Agent Forest (Li al., 2024) in grey line.
Samplnsgee Model
Samples Model
Adding More Porticipating Models for GPQA Adding More Perticipating Models for GSM8K Adding More Participating Models for MATH
51.
(N3
Unioo Accuracy (6)
50-
Union Accursic
Accuracy (90)
Ancureyy
Accurary
00
Union
47-
Accura-y
Accurscy
Jucuracy
2
Number Participating Models
Number of Partiippting Models
Numbor of Partscipating Models
Figure 9: Adding More Participating Models Affeets Accuracy Differently. We report the mean accuracy (blue line) of the optimal ST-M-MUX obtained when using to models across three benchmarks. We also report the union accuracy (grey line), defined in Section 3.2. The blue line (Mean Accuracy) plotted against the left-hand Y-axis. The grey line (Union Accuracy) is plotted against the right-hand Y-axis. For each K, both curves correspond to the single model combination that maximizes our model selection objective (Section 3.2) under that fixed K.
continues improve as additional models are included. Despite these differences, the union accuracy of model orchestration consistently increases with more models, emphasizing the role of output contradictions among models, as elaborated in Section 3.2. We also validate this scaling behavior on the test set; see Appendix D.4 for details.
"Drawing More Samples per Model" yields more consistent improvements across benchmarks. Moreover, under this setting, our SLM-MUX systematically outperforms Agent Forest, with the largest margin observed on GPQA, where single-model accuracy is lowest.
Group 1
Group 2
Benchmark
Qwen-2.5 72B Acc.
Acc.
A (Gain)
Acc. %
(Gain)
MATH
81.9 + 0.2
+6.4
79.5 + 0.4
+4.00
82.3 + 0.5
GPQA
49.9 + 1.8
+4.8
48.7 + 1.2
+3.6
44.9 + 0.5
GSM8K
93.7 + 0.2
+5.2
86.5 + 0.8
+5.77
90.4 + 0.3
Table 4: Best Accuracy after Sample Scaling beats Larger Model. Acc indicates the highest accuracy achieved through scaling. Groups 1 & 2 are defined in Table 2. Gain represents the improvement over the hest single-model accuracy reported in Table 2. For reference, also include the performance of the large model Qwen-2.5 72B, showing that our composed small models can outperform it on GPQA and GSM8K.
DISCUSSION
Mathematical Intuition behind SLM-MUX. Different SLMS have complementary strengths: for any given question, some models are more likely to answer correctly than others. SLM-MUX exploits this by selecting the most self-consistent model's output through a simple rule-based mechanism that requires no inter-model communication.
The key insight is that the confidence score can identify the strongest model. We assume that for each question, there is a unique correct answer, while incorrect answers are scattered rather than clustered. Under this assumption. model with higher accuracy P: produces the correct answer more frequently across N samples, leading to a higher confidence score. Therefore, selecting the model with the highest confidence score effectively identifies the model most likely to be correct.

---

Published as a conference paper at ICLR 2026
More formally, consider K models where model i has probability P. of being correct. Let i" arg max; P: denote the strongest model with margin y = P:. f Pj > 0. Under our assumption, the confidence score s; (the frequency of the most common answer over N samples) concentrates around p:- Applying Hoeffding's 's inequality and a union bound, the probability of correctly selecting the strongest model satisfies:
Nº 2
Pr(i = i²) 21 - 2(K - 1)
This bound shows that the probability of misidentifying the strongest model decays exponentially with sample size N.
This selection mechanism explains why SLM-MUX outperforms alternatives. Unlike a single fixed model, SLM-MUX performs per-question routing, effectively achieving accuracy Pmax by always selecting the strongest available expert. Unlike pooling methods such as Agent Forest that aggregate outputs from all models, SLM-MUX avoids interference from weaker models. For instance, if the strongest model has p1 = 0.8 and a weaker one has p2 = 0.3, pooling their outputs merely dilutes the correct answer's frequency. By isolating the strongest model and selecting its most frequent answer, SLM-MUX preserves the full predictive power of the most reliable source. We provide a more detailed comparative analysis with self-consistency and Agent Forest in Appendix D.2.
Extending SLM-MUX to Open-Ended Generation. Although the current implementation of SLM-MUX relies on majority voting and is therefore restricted to tasks with discrete answer spaces, the underlying idea of selecting the most self-consistent model is more general. For open-ended generation, one can replace majority voting with alternative consistency estimators, such as LLM- as-a-judge scoring or embedding-baste based similarity measures. Appendix E. we show a simple extension of SLM-MUX to HumanEval (Chen et al., 2021) using this idea and observe strong empirical gains.
Extending SLM-MUX Beyond Generalist SLMs.. The experiments above focus on general- purpose SLMs. We further evaluate whether the consistency-based selection principle extends to other settings: (1) frontier LLMs such as GPT-4o and Gemini-2.5-Flas and (2) domain-specific fine-tuned models such as code and math specialists. In both cases, SLM-MUX achieves consistent improvements over the best single model. Full experimental details are provided in Appendix E.
Limitation and Future Work. The SLM-MUX framework has two main limitations. First, its design is static and does not adapt to specific questions. For every query, it uses a fixed group of models that are pre-selected through exhaustive search a method that is slow and costly when there many models choose from. When models are tied, the framework uses their past accuracy on a validation set to decide, which is also a fixed, non-adaptive rule. Second, the way the framework measures model confidence is simple. It relies only on self-consistency- how often a model produces the same answer. This can be a problem because a model can be very consistent while still being incorrect.
Conclusion. This work demonstrates that orchestration methods designed for frontier models paradoxically degrade the performance of SLMs by amplifying errors. To address this, we propose SLM-MUX, framework that avoids inter-model discussion, instead selecting the most reliable output based on each model's self-consistency. We further introduce a model selection search algorithm to find complementary model combinations. Experiments show our method not only substantially outperforms existing strategies but also enables ensemble of just two SLMs to surpass the much larger Qwen-2.5 72B model on key reasoning benchmarks. In summary, our work validates that intelligently orchestrating multiple efficient models-2 "multi-core" approach- is promising alternative to scaling monolithic models on the path toward more capable Al systems.
LEDGONSSSS
We thank Tom Griffiths from Princeton University, whose insights and feedhack instrumental in initiating this project.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarin, Vaibhav Srivastav, et al. Smollm2: When smol goes big-data-centric training of a small language model. arXiv preprint arXiv:2502.02737, 2025.
Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Ce- line Lin, and Pavlo Molchanov. Small language models are the future of agentic ai. arXiv preprint wwXiv:2506002153. 2025. 1
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance, 2023a. URL https: ptttt::::/aaxiv.orrgg.oo 2330. 05176. 3
Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Matei Zaharia, James Zou, and Ion Stoica. Optimizing model selection for compound ai systems, 2025. URL pnttts://arxx: org/abs/2502 2502. 14815.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang. Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code, 2021. URL https:: /ttps:///arxiviy
abs 2217 03374. 10
Shuhao Chen, Weisen Jiang, Baijiong Lin, James T. Kwok, and Yu Zhang. Routerde: Query- based router by dual contrastive learning for assembling large language models, 2024. URL /ettps:/aarxte org/ 20tps//409..... 19886. 3 3
Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou, Universal self-consistency for large language model generation, 2023b. URL /ettps://arivvv wotg/bbs////eooo.oo.o.ggoo//bbrr 2 4
Yinlam Chow, Guy Tennenholtz, Izzeddin Gur, Vincent Zhuang, Bo Dai, Sridhar Thiagarajan, Craig Boutilier, Rishabh Agarwal, Aviral Kumar, and Aleksandra Faust. Inference-aware fine-tuning for best-of-n sampling in large language models, 2024. URL https: /ttp//aaivvv../// www./ba//////ooooooo///o../eaaer /2412.
15287. 3
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https: //tpxiv.wrgg///////......... abs/ 14168,
DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu. Haocheng Wang. Haowei Zhang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang. Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jiawei Wang. Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang. Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang. Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shuting Pan, T. Wang,
11

---

Published as a conference paper at ICLR 2026
Tao Yun, Tian Pei, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wanjia Zhao, Wei An, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang. Xiao Bi, Xiaodong Liu, Xiaohan Wang. Xiaojin Shen, Xiaokang Chen, Xiaokang Zhang, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xingkai Yu, Xinnan Song, Xinxia Shan, Xinyi Zhou, Xinyu Yang. Xinyuan Li, Xuecheng Su, Xuheng Lin, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. X. Zhu, Yang Zhang, Yanhong Xu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang. Yi Yu, Yi Zheng, Yichao Zhang. Yifan Shi, Yiliang Xiong, Ying He, Ying Tang, Yishi Piao, Yisong Wang. Yixuan Tan, Yiyang Ma. Yiyuan Liu, Yongqiang Guo, Yu Wu, Yuan Ou, Yuchen Zhu, Yuduan Wang, Yue Gong. Yuheng Zou, Yujia He, Yukun Zha, Yunfan Xiong, Yunxian Ma, Yuting Yan, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Z. F. Wu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhibin Gou, Zhicheng Ma, Zhigang Yan, Zhihong Shao, Zhipeng Xu, Zhiyu Wu, Zhongyu Zhang. Zhuoshu Li. Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Ziyi Gao, and Zizheng Pan. Deepseek-v3 technical report, 2025. URL htt /ttp//aaxvvv//..///vv///.../..av http//aawivv......a/./// owggabsg////oooooooro....os..aor 2212 1943.. 6
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate, 2023. URL https:
//arxiv. org/ abs 2200. 14325. 2,
Tianyu Fu, Zihan Min, Hanling Zhang. Jichao Yan, Guohao Dai, Wanli Ouyang, and Yu Wang. Cache-to-cache: Direct semantic communication between large language models. arXiv preprint arXiv:2510.03215, 2025. 2
Google Cloud. Gemini 2.0 flash generative ai on vertex ai. https: / /cl.ooooooooolloe..ooooooooooo com/ vertex- ai ai/generative ai/ htt//docs/modell/////eee httm//mmmmlmmmmiaaamm 0- flash, 2025. Accessed: 2025-09-24. 6
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Amthony Hartshorn, Aobo Yang. Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatr, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino. Dicuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiccao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe. Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Ke Li, Kenneth Heafield, Kevin Stone, Khalid EI-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala- Yeary, Laurens der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si. Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng. Prajjwal Bhargava, Pratik Dubal, Praveen Krishman, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong. Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan,
Plawiak,
12

---

Published as a conference paper at ICLR 2026
Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang. Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape. Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine. Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery. Elconora Presani. Emily Hahn. Emily Wood, Eric-Tuar Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filipposs Kokkinos, Firat Ozgenel, Francesco Caggioni. Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan. Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina- Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher,
Gaya, Jeff Marcus, Jeff Tang, Jenmifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang. Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer. Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari. Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal. Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey. Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov. Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng. Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang. Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanic Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad lonescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang. Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Vito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The
13

---

Published as a conference paper at ICLR 2026
llama 3 herd of models, 2024. URL htt.tp/://aa.xt.rro/o//// htt.b//220.222.....
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprins arXi::2501.12945 2025.
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https: /aw.iviivvv..////////aa///aaaav. otg/ abs 2103. 03874.
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet, 2024. URL https: wrg. abs /210 01798.
Mojan Javaheripi and Sébastien Bubeck. Phi-2: The surprising power of small language models. Mi- crosoft Rescarch Blog. 2023. URL http:://www. http:://wwwwwwwwww/teeeettt www/ccoo//////e cen-us/researe blog/phi- the
small
models/.
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang. Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of experts, 2024. URL https:
/tsttss://axiiii http///axvvv..ee//ee wtg. abs /2001.40000000 04088.
Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vard- hamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into self-improving pipelines, 2023. URL https: arxiv. org/
wwtpabs//////eoo..eo....t...aere at//abb//23/1 03714.
Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. More agents is all you need, 2024.
URL https:: /arxivvvvvvv org/ abw/a/b///00 05120. 2, 8,
Shalev Lifshitz, Sheila A. Mcllraith, and Yilun Du Multi-agent verification: Scaling test compute with multiple verifiers, 2025. URL htwph:tt////eee //tp:ss://///vv////// a2t////.s///2222///////// 22302222222222.. 20379. 1, 2,3,6, 6.7
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts, 2023. URL https: //rxxiv. org/ wtgg/bb//// 2
2
Mistral AI. Mistral small 24b instruct. https: http://hhggggggggeg wocacce.coo/ee wtt.c///m/aaaaa httm/missrraaal// Mistral- maatrrallma---- 24B Instruct 2501, 2025, Accessed: 2025-09-23. 7
Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL U abs /2501.2511... .tt.://.............
Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning route llms with preference data, 2025. URL https:://///......t. /tappp::///xeee http//xxvvvorr/ www.ooog/////ooooooooooooooss//e
abs /tt.////45550 2406. 18665.
OpenAl, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker- Whitcomb, Alex Beutel, Alex Borzunov, Alex Cameey, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Baek, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky. Ben Wang. Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon
14

---

Published as a conference paper at ICLR 2026
Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shem, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding. Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine Mcl.aavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czarnecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin, Daniel Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong. Ehsan Asdar, Elizabeth Prochl, Elizabeth Yang, Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Eugene Brevdo, Evan Mays, Farzad Khorasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Hannah Wong, Haoyu Wang. Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner, Henrique Ponde de Oliveira Pinto, Hongyu Ren, Huiwen Chang, Hyung Won Chung, Ian Kivlichan, Ian O' Connell, Ian O' D'Connell, Ian Osband, Ian Silber, Ian Sohl, Ibrahim Okuyucu, Ikai Lan, Iya Kostrikov, Ilya Sutskever, Ingmar Kanitscheider, Ishaan Gulrajani, Jacob Coxon, Jacob Menick, Jakub pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shich, Ji Lin, Jiahui Yu, Jiayi Weng. Jie Tang, Jieqi Yu. Joanne Jang, Joaquin Quinonero Candela. Joe Beutler, Joe Landers. Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman. Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Workman, Leher Pathak. Leo Chen, Li Jing, Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka. Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang. Louis Feuvrier, Lu Zhang. Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang, Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang, Mengehao Zhong, Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang. Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang. Minal Khan, Mira Murati, Mo Bavarian, Molly Lin, Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige, Nitish Keskar, Noah Deutsch. Noel Bundick, Nora Puckett. Ofir Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moore, Patrick Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang, Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman. Sam Schoenholz. Sam Toizer, Samuel Miserendino, Sandhini Agarwal. Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balaji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman, Thomas Degry, Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng, Todd Underwood, Todor Markov, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Eloundou, Valerie Qi, Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojeiech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Gpt-40 system card 2024. URL https: / /awxivvvvvvv.///iv//.aa///aaavvv org/ abs0 21276.
15

---

Published as a conference paper at ICLR 2026
Krista Opsahl-Ong, Michael Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. Optimizing instructions and demonstrations for multi-stage language model programs, 2024. URL https: http://awawworaa////eeeee// 116955. 3
Manhin Poon, XiangXiang Dai, Xutong Liu, Fang Kong, John Lui, and Jinhang Zuo. Online multi-llm selection via contextual bandits under unstructured context evolution. arXiv preprint arXiv:2506. 17670, 2025. 3
Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang. Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https:// /atp:///aavvv..// org/ abs 2412.15115.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level poogle-proof q&ea benchmark, 2023. URL https:: : / arxii. org/ abs 00los/23110 22122111111 12022. 7
Jon Saad-Falcon, Adrian Gamarra Lafuente, Shlok Natarajan, Nahum Maru, Hristo Todorov, Etash Guha, E. Kelly Buchanan, Mayee Chen, Neel Guha, Christopher Ré, and Azalia Mirhoseini. Archon: An architecture search framework for inference-time techniques, 2025. URL https:
//arxiv. wwa.xi..crg/ab////... 22409 2409. 15254.3
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally org/ /bbs/
can be more effective than scaling model parameters, 2024. URL 2408. 03314. 3
Amir Taubenfeld, Yaniv Dover, Roi Reichart, and Ariel Goldstein. Systematic biases in 11m simula- tions of debates. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 251-2677. Association for Computational Linguistics, 2024. doi: 10. 10.18653/v1/2024, emnlp-main. 16. URL http: / /x. doi
10. 1111///1/.113///222224444 emnlp-main 16.
Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves self-consistency in 1lms. In Findings of the Association for Compu- tational Linguistics: ACL 2025, pp. 20090-20111 11. Association for Computational Linguistics, 2025. doi: 10.18653/w1/2025. findiigs--nnnnii 1030. URL http: dxx doi v1/2025. www.ffinnnndann acl. 1030. 4
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor. Jean-Bastien Grill, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Weinberger, Dimple Vijaykumar, Dominika Rogozińska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei. Glenn Cameron, Gus Martins. Hadi Hashemi, Hanna Slimezak-Plucinsse Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder Singh, Mark Iverson, Martin Görner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal, Mehran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalie Dao, Nenshad
Harleen Batra,
16

---

Published as a conference paper at ICLR 2026
Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej. Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat, Sara Me Carthy, Sarah Cogan, Sarah Perrin, Sébastien M. R. Amold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Kocisky, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk Kwon, Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Sehastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alck Andreev. Gemma 2: Improving open language models at a practical size, 2024. URL https: /tta:://aaxvvva www.oogggg//.oooooooooooo/t..eet 2408 00118. 6
Raghuveer Thirukovalluru, Yukun Huang, and Bhuwan Dhingra, Atomic self-consistency for better long form generations, 2024. URL https: /tt///aaivvv..////e /ttp////ww...r///////........
Fouad Trad and Ali Chehab. To Ensemble or Not: Assessing Majority Voring Strategies for Phishing Detection with Large Language Models, pp. 158-173. Springer Nature Switzerland, 2025. ISBN 9783031821509. doi: 10. (0.1000/978-3-03) -811100--1-1-1111 13. URL http: http:/xxxxxddoooooooooeooo wwt.:oo/o..oo.o./o
031 ----8--111--11 93.
Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tzuhao Mo, Qiuhao Lu, Wanjing Wang. Rui Li, Junjie Xu, Xianfeng Tang, Qi He, Yao Ma, Ming Huang, and Suhang Wang. A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with Ilms, and trustworthiness, 2024a. URL https:://ta.i...t..//// abs/2411 03350.
Junlin Wang. Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances onhancess large language model capabilities, 2024b. URL 1, 2, 6,7
0492.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models, 2023. URL https: http://axvvvvvo
abs 2203. 11 11171. 2
Zhihui Xie, Jizhou Guo, Tong Yu, and Shuai Li. Calibrating reasoning in language models with http://..rrrab///220000 18711....11.... 4
internal consistency, 2024. URL https:
Jiyy Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong. Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. Aflow: Automating agentic workflow generation, 2025a. URL https: //tpx//..gg/////////
abs/ 0bss22110 10762.0
Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. A survey test-time scaling in large language models: What, how, where, and how well?, 2025b. URL https:// //arxiva....////////a wttp:/o//////0///// 242355. 3
17

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
The appendix is organized into five main sections. Section A states the usage of LLMs in preparing this paper. Section B provides experimental details, including visual illustrations, single-model accuracies, and standard deviation calculations. Section C analyzes why discussion-based methods fail on SLMs, presenting groupthink analysis and prompt sensitivity studies. Section validates the SLM-MUX design through aeaaistene--cccuraee correlation analysis, comparative analysis with voting- based methods, model selection search analysis, and test-set scaling validation. Section demonstrates the generalization of SI.M-MUX to open-ended generation, frontier LLMs, and domain-specific models. Finally, Section F provides dataset licenses.
A LLM USAGE STATEMENT
We used Cursor for coding, Large language models (LLMs) were employed to help polish drafts written by humans, and to assist in searching for related papers. The final choice of related work included in this paper was made entirely by the human authors after careful screening. LLMs were also used for proofreading and for providing suggestions.
B EXPERIMENTAL DETAILS
B.1 VISUAL ILLUSTRATIONS OF SLM-MUX
To more effectively illustrate the workflow of our proposed composition method, we select several representative examples from the logs. We demonstrate them in Figure 10, Figure 11 and Figure 12.
SLM-MUX surpasses majority voting in scenarios with initial disagreement among models.. As illustrated by Figure 10, during the independent generation phase, Gemma-2-277 is the sole model provide the correct answer. Hence, majority voting applied directly would fail to select the correct author.
Question: Express in
Correct Answer:: 4210
Independent G eneration Phase
Llama:
Gemma:
Mixtral:
Output I: To convert the docimal number, ..., 4220
Output I: Hene's how
Output I: First, need to
X
convert 555, 4210
perform repeated, ... Output To express
X
Output express base, 4210
Output 2: Here's
convert 555, 4210
decimal number. 4121
Output 3: To express 555 In .... 100
Output 3: Here's how to
Output First, we need to perform repeated, ....
convett 355 from, 4210
Reliability Estimation Phase
Confidence: 33%
X
Confidence: 100%
Confidence:
Historical ccuracy: 49%
X
Historical ccuracy: 57%
Historical A Accuracy: 32%
X
independent generation phase, three models are used: ILLMA-3.1 1-88 (denoted as Llama), Gemma-2-278 (denoted as Gemma), and Mixtral-8x (denoted as Mixtral). Because the three models provide different answers at first, each model is invoked two more times. Gemma obtains the highest confidence score and is therefore selected the final output.
Figure 10: illustration
SLM-MUX method applied to the MATH dataset.
B.2 ACCURACY OF SINGLE LLMS
We evaluated the accuracy of single model accuracy under the condition of temperature equal to zero. The results are shown in Table 5 and Table 6.
Model
MATH Ace (%)
GPQA Ace (%)
GSM Acc (%)
Llama-3.1-88 1-BB
48.6
23.7
84.2
Mistral-8x77
31.6
31.9
63.4
Gemma-2-229
56.8
38.8
81.6
Table Small Model Base Performance. Base model accuracy MATH, GPQA, and GSMSK.
18

---

Published as a conference paper at ICLR 2026
Question: Elvis has monthly saving larget $1125. In pri, wants to save twvice as much daily in the second half he savees in the first half in order hit his target. How much does he hove for each day in the second half of the mooth? Correct Anwerr: 50
Independent G Generation Phase
Iaamm:
Gemma:
Mixtral:
Output 1: To solve this
O Output Here's how to
Output I: First, let's
problem, .500
X
solve the problem, .50
determinc how, .... 150
X
Output 2: solve this
utputt Here's solve the problem, ,550
Output First, let's
problem, 50
determine how; 25
X
Output
Output Here's
Output let's
the problem step.
solve the problem, .50
determine how, 50
Reliability Estimation Phase
Confidence: 33%
Confidence: 100%
Confidence: 33%
Historical Á ocuracy: 84%
Historical Accuracy: 82%
Historical Accuracy: 64%
Figure 11: An illustration of the SLM-MUX method applied to the GSM8K dataset. In the independent generation phase, different models produce different answers. However, when we invoke each model multiple times, we observe that Llama and Mixtral only yield correct answers approximately one-third of the time. In contrast, Gemma demonstrates stable performance.
Question: Question: A student regrets that fell asleep during a lecture electrochemistrr. facing the following incomplete statement in test: "Themodymmically. oxygen oxidant in basic solutions. Kinetically, exygen reacts ...... acidic solutions." Which combination of weuker'stronger fistenslowerr is correc??
(A) weaker slower
stronger
(C) weaker faster
stroager faster
Correct Answer: (A)
Independent Generation Phase
L laaa:
Gemma:
Mixtral:
Output newer: C,
Output nswen.
Output To answer this
Explanation:
question, ....A
Output 2: nswer. A, In besic
O Output 2: Answer: D,
Output 2: To answer this
solutions,
question,
Output A Anser:: D. In basic solutions,
0 uttput 3: Answer: D,
Output To answer this question, A
Reliability Estimation Phase
Confidence: 33%
X
Confidence: 100%
Confidence 100%
Historical A ccuracy: 24%
Historical Acturay: 32%
Historical Accuracy: 39%
Figure An illustration of the SLM-MUX method applied to GPQA dataset. During the independent generation phase, Gemma and Mixtral obtain the same confidence score. However, considering historical accuracy, Mixtral ranks higher. Therefore, Mixtral's answer is selected as the final output.
B.3 STANDARD DEVIATION OF THE DATA POINTS IN FIGURE 5
Although all experiments are run in a deterministic setting with temperature set to zero, we can still compute the standard deviation of each datapoint by treating the outcome Bernoulli variable. Specifically, if there are Nocoeeet correct answers and Yownng incorrect answers, the standard deviation
MATH
GPQA
Model
Accuracy (%)
Token Usage
Accuracy (%)
Token Usage
DeepSeek V3
87.0
419,513
55.1
173,885
Gemini 2.0 Flash
90.4
361.737
63.6
195,576
GPT-4o
79.8
408,410
51.0
212,037
Table 6: Large Model Base Performance. Base model performance and token usage on MATH and GPQA datascts. Accuracy is the percentage of correct answers, and token usage reflects total tokens consumed (prompt + response) over the entire datasct for each model.
19

---

Published as a conference paper at ICLR 2026
Nearrect
Tonmeet
VVa((X)
v pp(1 1p1-p)
Thotal
Nhotta
Rrotal
/Pootal
Thotal
where Thtotal Vicorrett
Wornng
The results are summarized in Table 7.
Table 7: Accuracy and estimated standard deviation MATH (n 500) and GPQA 196) using datapoints from Figure 5.
MATH (n 500)
GPQA (n 196)
Method
SLM orchestration
LLM composition
SLM orchestration
LLM composition
Single Model Max Mixture-of-Agents
56.8 2.22
90.4 1.32
38.8 3.48
63.6 1 3.44
46.2 1 2.23
88.8 1 1.41
33.3 1 3.37
58.6 1 3.52
LLM-Debate
51.6 1 2.23
90.8 + 1.29
36.9 1 3.45
65.6 + 3.39
Multi-Agent Verification
48.4 2.23
91.6 1.24
35.4 + 3.42
64.2 + 3.42
C WHY DISCUSSION-BASHD METHODS FAIL ON SLMs
C.1 GROUPTHINK ANALYSIS
We analyze the experiment logs of LLM--ebbte using small language models (SLMs) in Section 4.1. Among 500 debate problems, 242 resulted failure (48.4%) For each of the 242 failed debates, first used an analyzer LLM to produce a process-focuse focused failure analysis. We then used a separate labeling L.L classify whether each failed debate was due groupthink.
The labeling results shown in Table 8:
These results reinforce claim that groupthink is major failure mode in SLM--bseed L.MM-dehate.
We provide the exact prompts used by (i) the analyzer LLM generate the 242 failure analyses (Figure 13) and (ii) the groupthink labeler LLM to classify groupthink (Figure 14). Placeholders such (problem) indicate runtime substitutions by our code.
C.2 PROMPT SENSITIVITY ANALYSIS
natural concern is whether the performance gap between discussion-basee methods and SLM- MUX is due suboptimal prompt selection rather than inherent limitations. To address this, conduct prompt sensitivity analysis on the Mixture-of-Agents (MoA) baseline using the MATH benchmark.
We Gemini-2.5-Flash generate 10 diverse aggregator prompts for MoA. Table 9 summarizes the results. Our baseline prompt (46.2%) outperforms the average of the tuned prompts (41.8%) and surpasses 7 out of 10 generated prompts. Even the best tuned prompt (48.4%) remains substantially below the best single model (56.8%) and FLM-MUUS (61.8%).
To further stress-test this result, we conducted iterative prompt optimization directly on the test set for rounds, selecting the pest-perfonming prompt at each iteration. Even under this extremely favorable setting for MoA, the accuracy peaked at 55.6%, still below the best single model (56.8%) and far below LLMM-MXX (61.8%). This confirms that the performance gap reflects inherent limitations of discussion-baas aggregation when applied to SLMs, rather than an artifact of prompt selection.
D VALIDATION OF SLM-MUX DESIGN
D.1 CONSISTENCY VS ACCURACY CORRELATION
We empirically study how per-question self-consistency correlates with accuracy on four datasets: GSM8K, MATH, GPQA, and HUMANEVAL. For each model-datased paír, we compute a self- consistency for every question (as defined in the main text) and group questions into three bins according to this score: Low [0.0, 0.5). Medium [0.5, 0.8), and High [0.8, 1.0]. We then measure the empirical accuracy (fraction of correct answers) within each bin.
20

---

Correction:** How did initial errors influence later rounds? Were there moments where a correct idea was introduced but ignored overruled? why did the debate fail to self-correct? Published as a conference paper at ICLR 2026
As an expert in analyzing multi-agent AI systems, your task is to analyze why an 'LLM Debate' process failed to find the correct answer. Your focus should be on the *debate dynamics and process*, not just the mathematical details. The goal is to understand the failure of the debate methodology itself.
Ground Truth:**
PProblem Statement:** (problem)
*CCrrect Answer:* (ref answer)
+Debate Information:**
**Final Incorrect Answer from System:** (systen_answer)
**Analysis of Round 1:**
**Model 'mmoee_namme)' proposed:**
Answer: "[extracted_answer
Reasoning:
[full_ full_text]
(repeats per round and per model)
++Your Analysis Task:**
Based on the debate history, provide a "Debate Failure Analysis".
Do not focus on simple calculation mistakes. Instead, analyze the interaction between the models and the structure of the debate. Pinpoint the core reasons the *debate process* failed. Consider these questiens:
1. *#Error Propagation
2. *+Groupthink and Influence Dynamics:** Did the models converge on flawed consensus? Did one or more influential but incorrect models lead the group astray? Was there evidence of independent reasoning that was shut down?
3. *rrgumentation Quality:** Did the models provide convincing but uitimately flawed arguments? Did they effectively
challenge each other's reasoning, or was the debate superficial?
4. ++Critical Failure Point the Debate:** Identify the single most critical turn moment in the debate that sealed its failure. What happened, and why was so impactful?
5. +Improving the Debate:** What is the single most important change to the debate protocol or dynamics that could have prevented this failure? (e.g., different communication rules, promoting dissident opinions, etc.)
Provide a concise, expert analysis focusing on the *process* failure.
Figure 13: Prompt Template for Failure Analysis.
21

---

Published as a conference paper at ICLR 2026
Metric
Count
Rate
Total Debates Analyzed
500
100% total
Failed Dehates (System Error)
242
48.4% of total
Breakdown of Failed Debates:
Attributed to Groupthink
144
59.5% of failures
Attributed Other Causes
79
32.6% of failures
Classification Unsuccessful
19
7.9% of failures
Table Failure Cause Attribution This table shows cause attribution for LLM-Debate when involving SLMs.
You are expert analyst of multi-agent LLM debates. Your goal is to determine whether the failure primarily involved groupthink/ dynamics, Groupthink indicators include: early flawed consensus, explicit capitulation to majority, social proofing, adopting peers' answers without critique, abandoning independent reasoning to match others, or reinforcing an incorrect majority despite available dissent. Not-groupthink includes failures due to independent arithmete c /logic errors, argument complexity/veneer effects without convergence, chaotic divergence with no consensus influence Return STRICT JSON only, with keys: groupthink (bool), confidence (float 0-1), reasons (string), , cues (array of strings).
Figure 14: Prompt for Groupthink Classification.
Figure 15 reports the resulting accuracies for two representative SLMs on each dataset. Across GSM8K. MATH, and HUMANEVAL we observe strong positive relationship between self- consistency and accuracy: questions the high-consistency bin are substantially more likely to be answered correctly than those in the low-consistency bin. GPQA exhibits weaker but still positive correlation. Overall, these results provide empirical support for the link between self-consistency and correctness assumed in our method.
D.2 COMPARATIVE ANALYSIS WITH VOTING- BASED METHODS
Since SLM-MUX also involves voting on model outputs, we examine its differences from standard self-consistency and Agent Forest to better explain the source of our improvements.
To explain stronger performance,
note limitation of self-consistency methods. Suppose a model has probability p of answering a question correctly. When self-consistency samples N responses, the probability of obtaining the correct answer after majority voting follows binomial distribution:
A(N, Pr 1
Σ
NN xx) (x)
1 ~ー X
(Binomial(N1)
k=[N/2]
Table 9: Prompt sensitivity analysis for MoA on MATH. The baseline prompt used in our experiments is alrcady ncar-optimal.
Setting
Accuracy
SLM-MUX (Ours)
61.8% %
Best Single Model
56.8%
Tuned Prompt (Best)
48.4%
Baseline Prompt (Used in Paper)
46.2%
Tuned Prompt (Average)
41.8%
Tuned Prompt (Worst)
25.8%
22

---

Published as a conference paper at ICLR 2026
(a) GSMBK
(b) MATH
Accuracy (fraction correct)
0.-
0.6
0.
Limoo 2-188 (-0.44)
0.0
0.0
(0.2.05)
(9..00.)
(0.8110)
(..0.0.5)
(0.3.00
(0.4.1
(c) GPQA
(d) HumanEval
0.3
0.9
0.3
0.5
Uiiie- (26.11)
4thh
Hich 1..
19.0 0.5)
19.5
[0..100)
[0.0, 0.5)
(6.5 0.80
Self-Consistency Score (per question)
Figure 15: Accuracy function of per per-question self-consistency score on GSM8K, MATH, GPQA, and HUMANEVAL. Questions grouped into three bins self self-consistency: Low |0.0, 0.5), Medium [0.5, 0.8), and High [0.8, 1.0]. Each line corresponds to different SIM; legends report the Pearson correlation between self-consistency and correctness.
We observe that A(N, p) exceeds only when > 0.5, meaning self-cconsitency is effective only in this regime. When p < 0.5, self-consistency can actually lower overall accuracy.
For any dataset, we can conceptually divide examples into three types of questions. Type 1: p 100%, the model always answers correctly. Type 2: > 50%, the model is more likely than not to be correct. Type 3: p 50%, the model is more likely to wrong. The overall effect of celf-consistency is then the improvement from Type 2, offset by the degradation from Type 3. Improvement occurs only when the dataset contains sufficiently large proportion of Type 2 questions.
For SLM-MUX, we select the output from the most confident model, so the accuracy can be approximated as A(N,Pmax) Pmax), where Pmax is the highest probability among the participating models. By routing to the model with the highest Pmax on each question, we effectively enlarge the proportion of Type 2 questions, leading to higher overall accuracy.
For the Agent Forest approach, answers drawn evenly from all models, its accuracy be approximated as A(N, p). where ğ is the average probability across models. This generally results in lower accuracy than SLM-MUX, weaker models dilute the signal from stronger ones.
D.3 MODEL SELECTION SEARCH ANALYSIS
D.3.1 SEARCH-SET SIZE STABILITY
We analyze how the size of the search set (number of problems used in the search phase) affects the resulting model ranking. For each of the three benchmarks MATH, GSM8K, and GPQA, we treat the full benchmark (approximately 500 problems) as the search pool and first run our search procedure on the full set to obtain a ranking of all candidate models. We then record the models occupying Rank I, Rank 2, and Rank 3 under this full-set ranking.
Next, we subsample the search set to smaller sizes and re-run the search. Specifically, for each dataset we evaluate search-set sizes of 100, 200, 300, 400, and the full set. At each size, we recompute the ranking over all models and track the ranks of the three models that were Rank 1-3 under the full-set setting.
Figure 16 shows the results. Each curve corresponds to one of the Rank 1/2/3 models under the full-set ranking, and the y-axis reports its rank when the search-set size is changed. Across all three
23

---

Published as a conference paper at ICLR 2026
datasets, these models consistently remain within the top three positions when the search set reduced to as few as 100 problems, with only minor swaps in their relative order on MATH and GSM8K. This suggests that search set of a few hundred problems is sufficient to stably identify the top-perforning models.
GSMBK
GPOA
Fare
kand Likan
Rank
200 Searcheaat problems)
320
400
302
120
202
320
4333
300
130
Searcheat probleme)
Search-tat sioe problems))
Figure 16: Stability of model rankings with respect to search-set size MATH, GSM8K, and GPQA, For each dataset, we first determine the top three models using the full search and then track their ranks the search-set size is reduced.
D.3.2 RANDOM MODEL SELECTION BASELINE
natural question is whether the performance gains of SLM-MUX stem from the model selection search from the orchestration architecture itself. To isolate the contribution of the architecture, we conduct an experiment where model combinations are selected randomly rather than through our search procedure.
For each dataset, randomly sample model combinations of size K 2,3,4 from our pool of five SLMs and apply SLM-MUX. We compare the resulting accuracy against the best single model within each random pool. Table 10 reports the average performance across all random combinations.
Table 10:
Performance of SLM-MUS MUX with randomly selected model combinations. Even without optimized model selection, L.M consistently outperforms the best single model in the pool.
Dataset
K
SLM-MUX (Random)
Best Single Model
A (Gain)
MATH-500
2
67.2%
66. %
+1.1
3
71.5%
69.8%
+1.7
4
73.7%
71.8%
+1.9
GSM8K
2
84.1%
81.7%
+2,4
3
87.3%
83.6%
+3.7
4
88.8%
84.0%
+4.8
GPQA
43.0%
41.3%
+1.7
3
44.5%
44.4%
+0.1
46.3%
46.0%
+0.3
As shown in Table 10, SLM-MUXX consistently outperforms the best single model even when the model combination is selected randomly. On MATH-500 and GSM8K, the gains are substantial (up to +4.8 on GSM8K with K = 4). On GPQA, where consistency is weaker signal for correctness, the gains are smaller but still positive.
To further quantify how often SLM-MUX improves over single models, we compute the Effective Combination Rate: the percentage of all possible K K-model subsets where SLM-MUX outperforms the best single model in that subset. Table 11 reports the results.
On GSM8K, 100% of combinations are effective across all values of K. On MATH-500 and GPQA, the effective rate increases with K, reaching 100% at K 4. Even K 2, the majority of combinations (60-100%) are effective. These results demonstrate that the space of "workable" model combinations is dense, and does not need to search extensively find an effective subset. The
24

---

Published as a conference paper at ICLR 2026
Table 11: Effective Combination Rate: percentage of model combinations where SLM-MUX outperforms the best single model the subset.
Dataset
K 2
K
K 4
MATH-500
70%
100%
100%
GSM8K
100%
100%
100%
GPQA 60%
80%
100%
model selection search provides additional gains by identifying the optimal combination, but the architecture is robust even without it.
D.4 TEST-SET VALIDATION OF SCALING BEHAVIOR
In Section 3.3, evaluated the "Adding More Participating Model Types" scaling dimension the validation set. Here we report the corresponding results on the test set to verify that the observed trends generalize.
Table 12 summarizes the test-set accuracy as the number of participating models K' increases from 1 to 5. For each value of K, we use the model combination selected by the search procedure described
in Section 3.2. Figure 17 visualizes these results.
Table 12: Test-set accuracy (%) the number of participating models K increases. For K 1, the best single model reported.
K
GPQA
GSM8K
MATH
I (best single)
47.98
90.20
78.20
2
50.21
91.51
80.27
3
51.81
91.29
81.09
4
49.16
91.44
81.00
5
45.18
90.80
80.64
Adding More Participating Models for GPOA
Adding More Participating Models for GSMBK Adding More Particlpating Models for MATH
83-
02-
47
45
Accunccy
Number Partocipating Miodete
Nomber of Participating Models
Number Participating Models
Figure 17: Test-set accuracy as a function of the number of participating models on GPQA, GSM8K, and MATH. The trends mirror those observed on the validation set: GPQA peaks K = 3, GSMSK saturates quickly, and MATH shows continued improvement up
E GENERALIZATION OF SLM-MUX
E. OPEN-ENDED GENERATION (HUMANEVAL)
In the main text, SLM-MUX instantiated on tasks with discrete answer spaces, where self- consistency can be measured via majority voting over sampled outputs. To check whether the same principle extends to open-ended generation, we apply SLM-MUX to the HUMANEVAL code- generation benchmark.
Consistency estimator for open-ended generation. On HUMANEVAL, exact-string majority voting not appropriate, so we replace with a semantic consistency estimator. For each model and each problem, sample N 5 code generations with temperature 0.3. We then encode the generations using the pretrained embedding model Salesforce and compute pairwise cosine similarities, yielding Sefooc///odee.55ie ap-det.5-1111 55p-110m-embecdiine embeddiris similarity matrix. From this matrix,
25

---

Published as a conference paper at ICLR 2026
Table 13: Pass@ I on HUMANEVAL for individual SLMs.
Model
Pass @
Llama-3.14 1-------nsntuue Instruct
0.178
Qwen2.5-7B- -B------ntttttttt
0.485
Mistral-Small
0.870
Qwen2.5-Coder-78 0.893
Table 14: SLM-MUX on HUMANEVAL (Pass@) 1). Each row corresponds to pair of SLMs; SLM-MUX selects the output from the model with higher ambedding-basse consisteney.
Setup
Models combined
SLM-MUX (Pass@) I)
Exp 1 Exp 2
Llama-3. 1-8--Insttttrtt + Bwven2.5-BB-Insteen
0.506
Mistral-Small--4B + Qwen2.5-Coder-7B
0.939
we identify the most coherent cluster of generations (of size .)) and use x/5 as the model's self- consistency score for that problem, analogous to the confidence score derived from majority voting in the discrete-answe setting.
We also experimented with an LLM-aa-a-judge-tt Instruct the judge) and found that the embedding- based estimator exhibits stronger correlation with ground -ouund-truth correctness (Pass@) I). All results below therefore use the embedding-haid based consistency score.
consistency estimator (using Qwen2.5-77
Results.. Table 13 reports the Pass@1 of each individual SLM on HUMANEVAL. Table 14 reports the Pass @ I of SLM-MUX when combining two models time; for each problem, SLM-MUX selects the solution from the model with the larger embedding consistency score.
E.2 FRONTIER LLMS
We evaluate whether SLM-MUX can exploit complementary strengths between state-of-the-ar frontier models. We pair GPT-4o with Gemini-2.5-Flash and apply SLM-MUX on MATH- GPQA, and GSM8K-500. For each problem, we sample N responses per model at temperature 0.3 and apply self-consistency routing: for each problem, perform majority voting within each model's s samples, then route to the model showing higher agreement.
Table 15 summarizes the results. As a reference, "Perfect Routing" indicates the theoretical upper bound achievable if the system always selects the correct model when least one succeeds.
Table 15: SLM- MUX performance when applied to frontier LLMs (GPT-4o and Gemini-2.5 2.5-ni--.55-Fashi
Benchmark
GPT-4o
Gemini-2.5-Flash
SLM-MUX
Perfect Routing
MATH-500
73.0%
92.1% 1%
92.8%
94.2%
GPQA
50.7%
51.1% %
60.1 %
73.7%
GSM8K-500
89.1 %
85.7%
89.4%
91.4%
The results reveal two distinct regimes. On GPQA, the two models exhibit complementary error patterns, and SLM-MUX achieves 60.1% % accuracy, surpassing the best single model by nearly 10 percentage points. This demonstrates that SLM-MUX effectively exploits complementary strengths even the frontier scale. On MATH and GSM8K, the Perfect Routing bounds (94.2% and 91 91.4%) only marginally higher than the single-model baselines, indicating high overlap in the models' correct predictions. The limited gains on these benchmarks reflect this ceiling rather than limitation of the routing mechanism.
E.3 DOMAIN-SPECIFIC FINE-TUNED MODELS
Somain-specift fine-tuned models are widely deployed in practice. We evaluate whether SLM- MUX can effectively orchestrate such specialized models by testing on two domains: code generation and mathematical reasoning.
26

---

Published as a conference paper at ICLR 2026
Table 16: SLM-MUX performance when orchestrating domain-specifie fine-tuned models.
Domain
Benchmark
Best Single Model
SLM-MUX
A (Gain)
Code Generation
HumanEval 89.3%
93.9%
+4.6
Math Reasoning
MATH-500
58.8%
62.2%
+3.4
For code generation, we pair Qwen2.5-Coders (a code-specialized model) with Mistral-Small-24B (a general-purposs, model) on HumanEval. We use the same embedding- bastn consistency estimator described in Section E. For mathematical reasoning, we pair PeepSeek-Math-7BB-. on MATH-500, using standard majority voting for
(a math-
specialized model) with --aaa-3.1-8B-Insse consistency estimation. Both experiments sample N 5 responses per model at temperature 0.3.
As shown in Table 16, SLM-MUX achieves consistent improvements in both domains. On Hu- manEval, the orchestrated system reaches 93.9% Pass 1, outperforming the code specialist (89.3%) by 4.6 percentage points. On MATH-500, combining the math specialist with general-purpose model yields 62.2% accuracy, 3.4pp improvement. These results demonstrate that SI.M-MUX generalizes effectively to domain-specifie fine-tuned models, successfully capturing complementary strengths between specialists and generalists.
F LICENSES FOR DATASETS
The MATH dataset is licensed under the MIT License. The GPQA dataset is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License.
The GSM8K dataset is licensed under the MIT License.
27


Reviews:
[{"review_id": "m0m35xiksy", "reviewer": "Reviewer_XgnD", "summary": "This paper addresses the challenge of orchestrating multiple small language models (SLMs) to achieve higher reasoning accuracy and efficiency than any individual SLM. The authors propose SLM-MUX, a multi-model architecture that coordinates SLMs without explicit inter-model discussion. The approach consists of three stages:\n\n1. SLM-MUX orchestration: Models independently generate answers; the most consistent answer (by self-consistency) is selected.\n2. Model selection search: Identifies complementary SLMs to maximize union accuracy and penalize overconfident contradictions.\n3. Test-time scaling: Adjusts the number of models and samples to optimize accuracy-compute tradeoff.\n\nEmpirical results show SLM-MUX outperforms existing orchestration methods (Mixture-of-Agents, LLM-Debate, Multi-Agent Verification) on MATH, GPQA, and GSM8K benchmarks, sometimes surpassing much larger models (Qwen 2.5 72B). Theoretical analysis and ablation studies support the design choices.", "questions": "1. There are multiple combinations of models possible for each value of K. Are the plots in Fig 7 only considering a single combination for each K? If yes, which combination is it?\n\n2. How/why was $\\lambda = 1$ chosen?\n\n3. Why is the test set same as the validation set for the plots in Fig. 9? Shouldn't they be different?\n\n4. In line 467 where you say, \"By increasing $p_\\max$...$, what part of your approach are you referring to? Does model-selection increase $p_\\max$? Can it be proved?\n\n5. From Fig. 5 (b) it seems that the improvements with discussion-based methods are fairly limited even for frontier LLMs. I would recommend trying two ablation studies - a) Applying SLM-MUX to frontier LLMs and b) Applying model-selection + discussion-based methods to SLMs. This can better illustrate the effectiveness of the individual components of your approach and also show if SLM-MUX is a viable solution even for frontier LLMs.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Originality:** The approach is based on a good intuition (that of multiple smaller processors being more effective at computing than a single large processor) and on a clearly identified problem - the failure of discussion-based methods in orchestrating SLM responses due to their tendency to fall into groupthink.\n\n**Quality:** The paper puts together three simple ideas into a solution that is evaluated using rigorous experimental methodolgy, with comprehensive benchmarking across MATH, GPQA, and GSM8K, and shows clear improvements over baselines.\n\n**Clarity:** The writing is clear and well-structured, making the technical content accessible. The motivation for SLM-MUX is articulated up front, and the different components of the solution (SLM-MUX orchestration, model selection, and test-time scaling) are explained clearly, step-by-step.\n\n**Significance:** Owing to the ever-growing cost of LLMs and the enormous demand for AI agents, it is critical to build agentic frameworks around smaller, lower-cost models, if we want them to scale across users and platforms and be cost-effective. This paper is a step in that direction as it shows that we need to rethink approaches to model orchestration if we wish to effectively use SLMs in agentic scenarios and then proposes a solution that improves over baselines and is also clear and easy to implement.", "weaknesses": "1. The consistency-based confidence-estimation will not work for open-ended question-answering which forms a major part of the use-cases of LLM-based agents and chatbots.\n\n2. It seems to me that the contradiction penalty in Section 3.2 will not differentiate between sets with different proportions of correct and wrong answers and so it may not fully overcome the problem of including over-confident but wrong models.\n\n3. The brute-force solution of O(S) in Section 3.2 will not scale to a large number of candidate models and so test-time scaling via adding participating model types is going to be fairly limited.\n\n4. It is not really a fair comparison if you follow the workflow and prompts designed for LLMs in the baselines (lines 313-314) when it is known that prompts are so sensitive to the underlying model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "roQE8EovV2", "reviewer": "Reviewer_8HgN", "summary": "The paper addresses an important problem of small language model (SLM) orchestration. Due to the limited capabilities of SLMs, existing orchestration methods which typically targeted frontier models often performs suboptimally when applied to SLMs. This paper introduces SLM-MUX which is an effective SLM orchestration architecture which optimizes model selection search and test-time scaling. The paper shows strong results of SLM orchestration with significant performance improvement across various reasoning datasets.", "questions": "In practice, small language models are often fine-tuned to become domain-specific, and these expert SLMs typically exhibit strong reasoning capabilities. While the paper demonstrates notable gains from orchestrating general-purpose instruction-tuned SLMs, it remains unclear how SLM-MUX would perform when combining domain-specific fine-tuned SLMs. Could the authors discuss how the method helps with these more domain-specific fine-tuned SLMs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper tackles a crucial problem of SLM orchestration which is often overlooked by existing literature of multi-agent orchestration\n- The method is training-free and light-weight\n- The experiments show strong performance across multiple reasoning benchmarks", "weaknesses": "In practise, often SLMs are fine-tuned to be domain experts which forms strong reasoning models. Empirical evidence show strong performance gain through orchestration, however, it is unclear how the framework performs with fine-tuned domain-specific SLMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T9kAGVt6LP", "reviewer": "Reviewer_2yhV", "summary": "The authors propose an SLM orchestration method called SLM-MUX. The authors show that orchestration methods that work for LLMs interestingly do not work when orchestrating SLMs. The authors propose a simple orchestration framework for SLMs based on picking the answers which are the most self-consistent after multiple generations. The authors provide an SLM model selection protocol for identifying sets of models which are able to orchestrate well. In mathematical reasoning datasets the authors demonstrate their approach outperforms existing orchestration methods.", "questions": "* I’m confused by the definition of the Contradiction metric. How does it generalize to more than two models. Surely encouraging contradiction enables a more diverse set S for SLM-MUX, in lines 322-323 you specifically mention that diverse outputs enables SLM-MUX’s abilities. Did you try varying the model selection objective? A good comparison for the model selection criterion would be simply a random selection of SLMs.\n\n* What would happen if you have no training set and you want to evaluate SLM-MUX 0-shot on a new test set? In algorithm 1 you do not have any validation accuracies or ground truth labels $y$ to select the most consistent model?\n\n* What does SC signify in the row Single-Best-SC in Table 1?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* The paper is well written and easy to follow.\n\n* The authors provide a detailed empirical analysis to demonstrate the effectiveness of SLM-MUX versus baselines. Also they additionally provide ablations of important components such as the trade-offs in the model selection objectives and the performance against the number of predictions made by the SLMs.", "weaknesses": "* The contribution of the paper is limited. SLM-MUX is doing a maximum consistency selection over SLMs which is open to abuse from SLMs that are consistently wrong on a test set (see question below)? Additionally SLM-MUX requires a model search prior to being used to remove SLMs which are highly confident in their wrong answers, so do we require a training set?\n* The mathematical analysis needs development: I think I understand that using $p_{max}$ versus $\\bar{p}$ like in AgentForest results in a higher number of correct answers? I think this deserves a bit more development. Like how is it related to multiple models? You mention $p_{max}$ is the max of three models (in line 466) which is rather ad-hoc, this could be any number of models. However you cannot evaluate $p$ without a training set? If you only have a test set you will consistently pick wrong answers potentially and you cannot assess $p_{max}$?\n* The results in Figure 7 are rather disappointing showing that 2 SLMs maximize the score which balances union accuracy while minimizing the contradiction penalty. I feel that this is a fundamental limitation of this LM orchestration technique that if you do not collaborate to come up with an answer, there will inevitably be some contradictions. So I can see that having 2 SLMs will reduce this contradiction in the responses, however this also feels sub-optimal. We should be in a state that adding more SLMs strictly improves performance.\n* Inconsistencies in the evaluations. Some results are reported over multiple runs with means and standard deviations e.g. Table 1. While others which are important such as Figure 5 which demonstrate how existing orchestration methods work for LLMs but not for SLMs only report a single seed. For instance it looks like the existing orchestration methods don’t work either for LLMs, but with no error bars to assess the noise in the experimental process this is difficult to conclude.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UCzd19GJTa", "reviewer": "Reviewer_7wuE", "summary": "This paper proposes an orchestration framework for small language models (SLMs), called SLM-MUX, and further explores test-time scaling strategies specifically for this method. The authors show that SLM-MUX can effectively coordinate multiple SLMs and achieves strong performance across various mathematical reasoning tasks.\n\nTheir main contributions are as follows:\n1. They identify the limitations of applying traditional orchestration methods, originally developed for large language models (LLMs), to SLMs.\n2. They introduce SLM-MUX, an orchestration approach that works particularly well for SLMs compared to discussion-based methods.\n3. They investigate strategies for model selection and search–compute scaling, tailored to the SLM-MUX framework.", "questions": "I understand that this method requires a validation set for model selection and related adjustments. Do the baseline methods (such as the discussion-based approaches) also rely on a validation set? How do you ensure that the demonstrated advantage is not due to access to additional information, or better overfitting capability?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The authors identify an interesting problem: existing orchestration methods developed for large language models (LLMs), such as discussion-based approaches, do not directly transfer well to small language models (SLMs). \n\nThe paper provides extensive empirical evaluations and complementary mathematical analyses to study the proposed SLM-MUX framework.", "weaknesses": "The proposed method, including its model selection objectives, appears somewhat heuristic. While the consistency of sampled answers may indeed correlate with correctness, relying on it as the primary criterion for selecting the best responses seems somewhat ad hoc. A stronger theoretical justification would help strengthen confidence in this approach. The analysis presented in the discussion section is a good starting point, but it would benefit from being expanded and made more rigorous to support the claims more convincingly.\n\nRegarding the contradiction penalty, it also comes across as rather ad hoc, as it penalizes models that consistently produce incorrect answers. Overall, these components feel more like empirical adjustments designed to boost performance metrics rather than principled elements of the framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)