Trinity: An Evolved LLM Coordinator
Authors: Jinglue Xu
Qi Sun
Peter Schwendeman
Stefan Nielsen
Edoardo Cetin
Yujin Tang
Venue: ICLR 2026 Poster (2026)
Keywords: evolutionary strategies
multi-agent LLM systems
role-based delegation
logits-to-agent mapping
Primary area: foundation or frontier models, including LLMs
TL;DR: A 0.6B coordinator whose ES-evolved 10k-param head reads the penultimate token to pick agents and assign worker/thinker/verifier roles—beating single models/routers without SFT or RL.

Abstract: Combining diverse foundation models is promising, but weight-merging is limited by mismatched architectures and closed APIs. **Trinity** addresses this with a lightweight coordinator that orchestrates collaboration among large language models (LLMs). The coordinator, comprising a compact language model ($\approx 0.6$B parameters) and a lightweight head ($\approx 10$K parameters), is optimized with an evolutionary strategy for efficient and adaptive delegation. **Trinity** processes queries over multiple turns, where at each turn the coordinator assigns one of three roles (*Thinker*, *Worker*, or *Verifier*) to a selected LLM, effectively offloading complex skill acquisition from the coordinator itself. Extensive experiments demonstrate that **Trinity** consistently outperforms individual models and existing methods in various tasks, including coding, math, reasoning, and domain knowledge, while robustly generalizing to out-of-distribution tasks. On established benchmarks, **Trinity** achieves state-of-the-art performance, including a new record of $86.2\%$ on LiveCodeBench. Theoretical and empirical analyses highlight two key factors driving this success: (1) the coordinator’s hidden-state representations provide rich contextualization of inputs, and (2) under high dimensionality and strict budget constraints, the separable Covariance Matrix Adaptation Evolution Strategy algorithm provides substantial advantages over RL, imitation learning, and random search, leveraging potential block-$\varepsilon$-separability.

Published as a conference paper at ICLR 2026
TRINITY: AN EVOLVED LLM COORDINATOR
Jinglue Xu Qi Sun 1.3, Peter Schwendeman 2t, Stefan Nielsen'. Edoardo Cetin', Yujin Tang' Sakana AI, Japan University of Michigan, USA
Institute of Science Tokyo, Japan
ABSTRACT
Combining diverse foundation models is promising, but weight-merging is lim- ited by mismatched architectures and closed APIs. TRINITY addresses this with a lightweight coordinator that orchestrates collaboration among large language models (LLMs). The coordinator, comprising a compact language model (= 0.6B parameters) and a lightweight head (Rs 10K parameters), optimized with an evolutionary strategy for efficient and adaptive delegation. TRINITY processes queries over multiple turns, where at each turn the coordinator assigns one of three roles (Thinker, Worker, Verifier) to selected LLM, effectively offlood- ing complex skill acquisition from the coordinator itself. Extensive experiments demonstrate that TRINITY consistently outperforms individual models and ex- isting methods in various tasks, including coding, math, reasoning, and domain knowledge, while robustly generalizing to out-of-distribution tasks, On estab- lished benchmarks, TRINITY achieves state-of-the-arr performance, including a new record of 86.2% LiveCodeBench. Theoretical and empirical analyses highlight two key factors driving this success: (1) the coordinator's s hidden-sta state representations provide rich contextualization of inputs, and (2) under high dimen- sionality and strict budget constraints, the separable Covariance Matrix Adapta- tion Evolution Strategy algorithm provides substantial advantages over RI., imita- tion learning, and random search, leveraging potential Block---separabliig
INTRODUCTION
A prominent line of work involving large language models (LLMs) aspires scale in line with em- pirical scaling laws, targeting gains by enlarging model size, training tokens, and compute (Kaplan et al., 2020; Hoffmann et al., 2022). Yet the extent which such scaling remains efficient and yields sustained returns is uncertain and often resource intensive. An alternative at the micro level is model merging (Akiba et al., 2025; Wortsman al., 2022: Yang et al., 2024; Kuroki et al., 2024), which seeks parameter-level integration. However, this approach is frequently impractical due to architec- tural incompatibilities and the closed-source nature of many high-performing models. In light of these limitations, adopt macro-level approach: test-time model composition via coordination, which fuses the complementary strengths of multiple statc-of-the-an models from diverse providers without modifying their weights. Leveraging prior data and training investments, this coordination can deliver performance improvements without retraining individual models.
The central challenge for such coordinator is to acquire rich contextual understanding of given query to make an effective decision. We posit that this signal can be efficiently extracted from the internal representation of compact language model, specifically, its hidden states (Allen-Zhu & Li, 2023). In a self-attention-bart -based transformer model, hidden states encode contextual representations of the input (and. after generation, the output) sequence. Hidden states extracted from inputs alone reflect input context, and those taken post-generation additionally capture the model's produced output and latent reasoning. For output sequences, the penultimate token's hidden state carries rich context. It attends over the entire sequence and guides the prediction of special token (such as <htni>>>> or the EOS token), ensuring stable output distribution. This leads to our central hypothesis that contextual representations from small language model (SLM) contain sufficient
"Equal contribution
Work done during internship Sakana AI Correspondence j ng uexu@sakana .ai. ai, gisun@sakana.ai) ai, yujintang@sakana.da

---

Published as a conference paper at ICLR 2026
Bina purchased vehicle January eghtt years with
estimates
User Query
Using coub docining balance depreciation method. your depoeciiiion on expenes
computt
Concat
The user requires the calcultioon the secord--eet ceprecatioo experse for vehicie using the double declining: balance mothod. initaal response empty. nexd apent should act solver to perform calculation.
SLM
Msg Procossing
Here step-
problem:
Deterrine straight- deprecition rate
2. Carculato the double deeclining ballndo rate...
Lightweight Head
Calcu the git-t depreciation rate. gitt-line div
LLMS
Rolees Trinity
Calcuate the double deciiiing- balance rate.
This bolce the straight- rall...
The calculation provided the current response apocars to be correct and follows the double declining bollance depresiation method accuratety The steps are cieatty outlined, and the final second-yyaal dappes lation exparise is conecclly calculaled $2.812.50.
one potertial vertly whather switch
deccining-brnaaee
results
Resporew
Figure I: Overview and an example of our coordination method. Left: architecture. In each turn, the full conversation transcript is passed to compact coordinator model. A lightweight head selects an LLM and assigns one of three roles: Thinker (T), Worker (W), or Verifier (V). A message processing module injects role-specific prompt before the request is sent to the chosen LLM. Right: An example of multi-turn coordination. To solve a complex depreciation problem, TRINITY invokes a Thinker (Tumn 1) to decompose the task, Worker (Turn to perform the calculation, and a Verifier (Turn 3) to validate the answer and identify edge cases.
The cyclical coordination
semantic signal for a lightweight head to coordinate multiple LLMs effectively, a possibility that remains underexplored in existing works (see Section 5).
Given these contextual representations, our method, TRINITY, employs an SLM (0.6B parameters) with lightweight head orchestrate multiple LLMs (both open- and closed-source models) in multi-turn protocol, with the total number of learnable parameters under 20K. At each turn, TRINITY selects an LLM and constructs its input by concatenating the original query with the full transcript of prior turns. To ensure the coordinator remains lightweight and offloads complex skill acquisition, TRINITY assigns the selected agent one of three distinct roles: (I) thinker to devise high-level strategies and decompositions: (2) a worker to perform concrete problem-solving steps; and (3) verifier to evaluate the current solution's 's soundness and completeness. The process halts when the verifier is selected and accepts current response as the final answer, or when fixed-turn budget is exhausted. Figure gives an overview of our method, together with an example of our coordination.
Optimizing this caapesenttatiio-tooooodeeeeeeeee among parameters - each has only a tiny influence on the sealar reward, making traditional methods like REINFORCE's ' perramaeetter gradients IoWRSNR and therefore ineffective. Training is further constrained by cost, since each step requires running the coordinated agents for inference. We find that derivative-free Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen et al., 2003) with diagonal covariance, separable CMA-ES (sep-CMA-ES) (Ros & Hansen, 2008), is effective in this particular regime: high dimensionality, weak parameter correlations, and high per- step cost. We provide theoretical and empirical evidence that, in this extremely budget-tight scenario (1.5k-40k evaluations for a 10k-dimensional problem), sep-CMA-ES significantly outperforms RL and the random search baseline, suggesting strong block-e-separabilit (see Definition 1) in the optimization objective.
mapping is challenging. We observe weak coupling
Across four in-distribution benchmarks including Math500 (Lightman et al., 2023), MMLU (Hendrycks et al., 2020), RLPR (Yu al., 2025), and LiveCodeBench (Jain et al., 2024), TRINITY consistently outperforms prior methods, achieving a mean relative error reduction of 21.9% over the second-best approach. It also outperforms all single-model baselines with fair, adjusted output-token budgets. Remarkably, TRINITY sets new state-of-the-art on LiveCodeBench (Jan - Aprl 2025), achieving a pass@1 of 86.2 = 0.5%. Furthermore, TRINITY is able to zero-shot transfer to four unseen tasks consisting AIME (Veeraboina, 2023), BigCodeBench (Zhuo et al., 2024), MT-Bench (Bai et al., 2024), and GPQA-D (Rein et al., 2024), with performance surpassing each of the single models it orchestrates.
2

---

Published as a conference paper at ICLR 2026
Our main contributions are summarized as follows:
A lightweight and effective coordination mechanism. We show that rich contextual sig- nals from the hidden states of an SLM sufficient for tiny head to coordinate multiple diverse LLMs (with the total number of learnable parameters under 20K), a previously underexplored approach model composition.
A highly efficient training methodology. We demonstrate theoretically and empirically that under the challenging. budget-constrained conditions of our problem, sep-CMA-ES superior optimization choice over RI., imitation learning, and random search.
Statc-of-the-art performance and generalization. TRINITY sets a new record on Live- CodeBench and outperforms existing methods wide range of benchmarks. It also gen- eralizes robustly to unseen tasks and develops emergent, task-aware coordination strategies.
2 PROBLEM FORMULATION
Let S be the set of interaction states (the original query together with the full multi-turn conver- sation far). An SLM maps each to representation state h(s) H. 12 (c.g., penultimate- token hidden vector). A lightweight coordination head with parameters E C IR" takes h(s) as input and outputs logits over finite action set A of agent-role pairs:
fo 7.. + RlA,
Ta(a s) & exp(fo(b(s)))) a
The policy ne induces distribution over all multi-turn trajectories T, where trajectory is (so, ag.... .............................. ST) with horizon T S Brarn where Brumn denotes a fixed turn budget. terminal reward R(T) E {0, 1) is revealed at the end. The optimization objective
((0) = (er=n([((T))
the expected terminal reward of the coordinator contextual features, while the coordination space P parametrizes policies over all trajectories in T. We regard each single, complete, end-to-end run (i.e., sampling of a trajectory т) as an atomic evaluation, Bernoulli call since the rewards follow the Bernoulli distribution. And since each run involves multiple LLM calls, which a cost we wish to constrain, we seek 0' E arg maxøep (()) under tight atomic evaluation budget Beuv that counts individual Bernoulli calls of the terminal reward used when estimating J (0) (e.g., via replication//averainng))
short, the representation space H provides
3 TRINITY
To address the problem outlined in Section 2, we propose TRINITY, a lightweight and adaptive framework for coordinating multiple diverse LLMs (Figure 1, left). At its core, approach intro- duces a coordinator, optimized via Sep-CMA-ES, that learns to orchestrate a pool of external LLMs and assign them distinct roles throughout a multi-turn reasoning process.
3.1 EFFICIENT PARAMETRIZATION
To efficiently derive the representation and coordination space, the coordinator employs a highly efficient parametrization scheme, as illustrated in Figure 2. We a pre-trained SLM as backbone and introduce two distinct sets of trainable parameters.
First, we append lightweight head directly after the coordinator SLM's final hidden layer. To coordinate L agents, this head projects hidden state h E Idd to an output of size L + 3, which provides two sets of logits: I. logits for selecting an L.LM and three logits for assigning its role. This head defines the fundamental structure of the coordination space. Second, inspired by recent work in efficient fine-tuning (Sun et al., 2025), we adapt a small set the backbone's layers using singular value fine-tuning approach. For selected subset of the coordinator SLM's weight ma- trices, we perform singular value decomposition and only learn the singular value scales, keeping the orthogonal matrices fixed. This parameterization scheme is highly efficient, keeping the total number of learnable parameters below 20K, orders of magnitude smaller than typical finningning, while still yielding representational benefits (Figure
3

---

Published as a conference paper at ICLR 2026
Crucially, our method only relies the head's logit outputs, and the coordinator's generated text is discarded because the job of prompting is delegated to the LLMs in the pool (Section 3.2). Rather than waiting for full generation, this allows the coordinator to take hidden states corresponding to an earlier token instead of the penultimate to make a quick decision. This combination of extreme parameter efficiency and the potential make rapid inference makes training the entire TRINITY system with evolutionary strategies uniquely feasible (Section 3.3), avoiding the significant data and computational overhead of imitation learning or RL.
SLM
"Write code for
binary search"
Input Text
LM Head
Output Text
LLM logits
R
Penultimate Output Token
Lightweight Head
R
logits
i E R24
Hidden
<OS>>
"python'vn def bsearch ...
Linear Low-rank
<Hea Input><EOS>
Sparse Block-dlagonel
Figure 2: Parametrization of the TRINITY coordinator. lightweight head (see Appendix ??) operates in parallel the base model's s LM head. It takes the hidden state h corresponding to the penultimate output token as its sole input. This head ʃe is responsible for coordination decisions, producing two sets of logits, one select LLM from the pool of L models, and another assign one of three roles. We also fine- tune the singular value scales of the parameter matrices in the SLM's layers, indicated by the red diagonal lines. In the figure, the hidden state the position marked by "<Heed Input>" is the input to lightweight head. Note that the semantic correspondence of the decoded message "<BOS>.... to the hidden state is only for illustrative purpose, as the lightweight head operates on the internal hidden state from that position, not the final decoded text.
3.2 TRI-ROLE COORDINATION
Next, we discuss the set of multi-agent interaction patterns available to the coordinator, which are the remaining constructs that define the coordination space.
key principle of our approach is that the coordinator itself need not be as capable as the underlying agents, its primary function is leverage and orchestrate diverse LLMs. Coordination proceeds over at most K turns for a given user query Q. Let the transcript after k-1 turns be Ck-1 = At turn k; the coordinator selects agent (i.e., LLM) A& from the pool
(Q, O,,.,,.., Ok-1). M and role R& (Thinker (T), Worker (W), Verifier (V)). The coordinator then prepares brief, role-specific prompt based on Cx-1, queries A₁ to obtain message Mg. and lightly post-processes Mj into Oz, which is appended the transcript for the next
In TRINITY, we define three roles, namely Thinker, Worker, and Verifier, each of which enforces a distinct contract between the coordinator and the selected LLM:
Thinker strategizes. The thinker analyzes the current state and returns meta-level guid- ance, including high-level plans, decompositions, mally, it may propose plan over subgoals, which the coordinator condenses into O₃ to steer subsequent turns, it can also specify the role of the agent along with the plan.
critiques of partial solutions. For-
Worker executes. The worker acts directly on the task to make concrete progress toward final solution. Given C produces actionable content (e.g.. derivation, code snippet, numerical result). The coordinator extracts the key information and stores it as Oş-
Verifier evaluates. The verifier checks whether the accumulated solution in Ck-1 is cor- rect, complete, and responsive to Q. outputs judgment 44 RECCEET,,EVISSS and an optional diagnosis ōg. The coordinator records (ux, 6x) as Og and, if ug = ACCEPT, signals termination.

---

Published as a conference paper at ICLR 2026
The termination time is minf k I K R₂ V and uk ACCEPT }, with K if no acceptance occurs. The final answer returned to the user is O,. This rule provides a simple, verifiable stopping condition while allowing the coordinator to allocate the compute budget adaptively across planning, execution, and quality control. See Figure I (right) for an example.
3.3 LEARNING WITH AN EVOLUTIONARY STRATEGY
To determine suitable training algorithm, we examine the structure of our problem objective. By varying the head architecture (see Appendix A.4), we observe that the head block-diagonal-10 re- tains large fraction of the performance despite its tiny parameter count (see Section 4.7). These observations reveal that the optimization problem defined in Section 2, when embodied in our repre- sentation and coordination space, exhibits strong block-s separability (Definition 1). This geometry strongly favors diagonal methods: most of the informative signal is concentrated within blocks, while inter-block interference remains negligible. Conversely, this geometry undermines the REIN- FORCE baseline (as shown in Section 4.8); noisy global returns swamp weak inter-block signals, yielding ill-conditioned gradients, poor credit assignment, and unstable learning.
We therefore adopt sep-CMA-FSS, black-box evolutionary strategy that iteratively improves cen- tral "parent" policy by sampling a population of perturbed parameter vectors, evaluating each can- didate obtain fitness score, and recombining candidates via fitness-weighted averaging to form the next parent. Unlike full CMA-ES, sep-CMA-ES maintains only a diagonal covariance matrix, making the algorithm especially well suited to block-diagonal landscapes.
In Appendix 1. we provide a theoretical analysis tailored to our specific problem regime: a coordi- nation head with about 10K parameters, tight evaluation budgets, binary terminal rewards, and weak but nonzero cross-block couplings. In the following, we present a short comprehensive summary.
Let be the head dimension, À [4 3n be the CMA-ES population size, and MCMA/MES be the replication counts (number of evaluations per candidate). Denote as the optimization iter- ation count. Then, for the small-7' regime. Proposition shows that sep-CMA-ES's improvement grows roughly linearly with the number of iterations, while random search (RS) grows only with the logarithm of how many candidates it can test. Thus, for modest T, sep- -M--E outperforms RS. In the specific regime of our study (n z 10000, A 32, MCMA = 16, mas = 32), budget matching yields about 16T RS candidates; the gain ratio behaves like In([6T) 17² where 1) is a reliability factor between 0 and 1, usually close to one. This ratio is greater than one even for small T.
Proposition 2 states that after about iterations of calibration, sep- -M--CMA-ES enters a steady regime where each step reduces the remaining error by a fraction of order 1/n, with a rate constant close to Ry,,A where the constant R, = (1) denotes the CMA recombination efficiency. By contrast, RS continues to gain only logarithmically even with repeated rounds. Hence, as T increases, sep- CMA-ES becomes better and the gap compared to RS grows wider.
4 EXPERIMENTS
We demonstrate the effectiveness of TRINITY through three key dimensions. First, we directly compare against both multi- and single- agent baselines in controlled settings. We also show that TRINTYY establishes a state-of-the-art performance on the LiveCodeBench task. We then evaluate its generalization capabilities across a diverse set of unseen tasks. Finally, we present analytical results, including ablations and the contextual information encoded in the extracted hidden states, and compare our evolution-based approach against RL, imitation learning, and RS.
4.1 EXPERIMENTAL SETUP
Coordinator and agents. We use Qwen3-0.6B (Yang et al., 2025) as the coordinator's SLM, paired with a single linear layer of 10K parameters as the simple but effective head, and select the second-to-last layer of the 0.6B model for singular value fine-tuning, Table 6 reports the param- eter counts of the various head architectures and the number of parameters updated during singular value fine-tuning. Our model pool contains seven models from both open- communities and closed-source API providers. These are, three top-tier closed-source models currently available (GPT-5 (OpenAI, 2025), Gemini-2.5-pro (Comanici al., 2025), and Claude-4-Sonne (Anthropic,
5

---

Published as a conference paper at ICLR 2026
2025)), and four well- kollnn open-source models (Gemma-3-21) 3-27BB (Team et al., 2025). DeepSeek- DDistill-Qwen-3222 (Guo et al., 2025). Qwen-3-32B (direct), and Qwen-3-322 (reasoning)). Our LLM and training task selection principle is detailed in Appendix A.6.
Tasks and protocols. We train and evaluate TRINITY across four diverse tasks, including MATH500, MMLU, RLPR, and LiveCodeBench. For each task, train the designated training set and assess performance on the corresponding test set, utilizing official splits where available. For LiveCodeBench specifically, we use the VI release (400 samples) for training and conduct evalua- tion on the newly introduced questions in the V6 release (175 samples). To ensure consistency be- tween open and closed models and facilitate training, we set the default maximum generated tokens to 4096 for each LLM, with minimal reasoning effort. We also set the maximum number of coordi- nation turns to five. For assessing generalization capabilities, we further evaluate our approaches on four challenging held-out tasks (AIME2025, BigCodeBench, MT-Bench, and GPQA-D), spanning diverse domains and problem types.
Baselines. We compare TRINITY against several categories of baselines. For multi-agent routing methods, we compare against state-of-the-art approaches including MasRouter (Yue et al., 2025), RouterDC (Chen et al., 2024), Smoothie (Guha et al., 2024), MoA (Wang et al., 2024) and random agent selection. We also evaluate individual LLMs in our pool (GPT-5, Gemini-2.5 2.5-i--.--pr-- Claude- 4-Sonnet) both 4K and 20K (marked as 5x CTX) inference tokens to assess performance under accumulated inference budget, and single agent self-reflection over five turns (5x SR). In addition, include majority-voting baseline samples and baseline with an LLM the coordinator (see Appendix A.7.3). Detailed experimental settings are provided in Appendix A.7.1.
0.492.22 2R.
3.3g 3308-700-72 6-72
0.53 0.773.17
190
0.22
Math300
0.8
04
3.1
3.64
LiveCceeeleeg
6.c
2-2 Se
8.47
1356220 32
0
64
0.48
Genual Foe Be Crd)
Denen Pre
fonwe cro
Sormet
OPr3
CTS)
do)
Seniet both
deevorsie (Drgs)
Cueed bart destt
Cominn Rrmmii-ee22 Gerond fyo Gemai
CTe)
Simven d's CYS)
Swene te del
oPto cer suac Kandim
Misk
broggtia Bestoo!
Resterttc
droennii (D)))
Whotty (Ours)
GPrs
Reioby
Figure 3: TRINITY outperforms single- and multi-model baselines across four benchmarks. Our approach (boldface on the x-axis) achieves the highest performance across four tasks, surpassing the baseline methods. In Math500, MMLU and LiveCodeBench, our performance is close "Per- Question-Bess BBett, representing upper bound achieved by taking the union of all correct answers from the single LLMs.
4.2 IN-DISTRIBUTION EVALUATION
As shown Figure 3, TRINITY consistently outperforms existing multi-agents methods across all four benchmarks, demonstrating its superior ability to harness the strengths of a diverse LLM pool. While some baseline methods achieve moderate performance on individual tasks, such MoA's strong results on Math500 (0.83) and RLPR (0.38), they fail to maintain consistency across tasks, as evidenced by its relatively weaker performance on LiveCodeBench (0.39). This inconsistency highlights the difficulty of effectively coordinating diverse agents. Notably, some collaboration approaches even degrade performance below random baselines, as seen with Router DC's RLPR score of 0.28 compared to random selection's 's 0.32, further emphasizing the challenge.
In contrast, TRINITY achieves robustly high performance across the board, including a remarkable 0.61 pass @ 1 score on LiveCodeBench v6, substantially surpassing all competing methods. Also, we achieve a 11.76% relative error reduction on MATH500 compared to the 2nd best method (Gemini Pro 2.5 with 5x CTX). These results suggest that, while diverse agent capabilities offer significant potential, effective collaboration requires sophisticated mechanisms for optimal organizational de- cisions, which simple heuristic-hased routing approaches cannot easily achieve.

---

Published as a conference paper at ICLR 2026
Table I: Performance Across Hold-Out Tasks
Model
AIME
BigCodeBench
MT-Bench
GPQA-D
Average
Gemini Pro 2.5
46,67
35.10
9.37
75.25
52.34
GPT-5
46.67
33.80
9.35
72.73
51.07
Claude-4-Sonnet
35.33
35,80
9.28
67.30
46.14
Qwen3-32B (reasoning)
23.33
20.90
8.99
59.09
34.44
DeepSeck-RR BAck-RR-QQeR RI--nn-33
30.00
24.30
8.43
51.01
35.10
Qwen3-32B (direct)
20.00
23.00
9.03
54.05
33,46
Gemma-3-27B-IT
20.00
20.30
8.76
33.33
21.38
TRINITY (Ours)
50.00
35.80
9.60
76.82
54.21
Compared with single-model baselines, TRINITY outperforms every individual model in the pool, even when they are enhanced with either extended inference budget (5x CTX) or self-reflection set- tings (5x SR). The 5x inference budget matches our maximum turn setting of five, ensuring that comparisons are fair, and some cases even favorable to the baselines. A closer look at Figure reveals distinct strengths and limitations for each model. For example. Gemini excels RLPR and MATH500 but shows moderate performance on LiveCodeBench while GPT-5 dominates Re- markably, TRINITY achieves optimal performance across all tasks, demonstrating its ability to dy- namically leverage each model's 's strengths and compose them effectively for different challenges. To further contextualize TRINITY'S capability, we also include an upper bound ("Per-Question-Beet representing the performance achieved by taking the union of all correct answers from the seven LLMs in the pool. Our method approaches this limit closely on three of four tasks, demonstrating its ability to harness the collective capabilities of the model ensemble. TRINITY also exhibits upper-tier token efficiency compared other methods, especially coordination methods. (see Appendix A.7.4 for detailed comparison).
Model Performance Comparison
4.3 ZERO-SHOT TRANSFER TO UNSEEN TASKS
This suggests that TRINITY does more than simply select the best agent for task. To assess TRINITY'S general- ization capability, we tested its zero-shot performance four held-out benchmarks. As summarized in Table 1, TRINITY achieves the highest average score (54.21) and outperforms every individual baseline on each of the four tasks. It secures top performance AIME (50.00), MT- Bench (9.60) and GPQA-D (76.82), and ties for first on BigCodeBench (35.80). This result highlights key ad- vantage of our approach. While individual models exhibit specialization strengths and weaknesses (e.g., Gemini Pro 2.5 and GPT-5 perform better on reasoning tasks com- pared coding benchmarks, and Claude-4-Sonnet shows relatively balanced performance). TRINITY delivers con- sistent results across all domains. It effectively synthe- sizes the capabilities of the entire pool to achieve emer- gent performance that surpasses any single constituent find that Qwen3-32B reasoning mode underperforms direct mode on BigCodeBench, this mostly due its verbose reasoning, causing formatting failures in certain test cases.
64-
Performance Maximum Turna
0.090
6.55
Maximum
model. Surprisingly,
Figure 4: LiveCodeBench Results. Top: TRINITY achieves state-of-the- art. Bottom: TRINITY benefits from in- creasing maximum turns budgets.
4.4 UNLEASHING FULL POWER
Due to hardware constraints
serving open-source models, we limited the maximum output length for all LLMs in the pool for fair comparisons in the previous experiments. For the LiveCodeBench task, the coordinator's 's LLM selection narrows down to the three closed-models after training. This allows to remove the output length constraint and observe the full power of TRINITY Live- CodeBench. Notice that we simply remove the constraints and do not retrain TRINITY.
In Figure 4 (top), TRINITY demonstrates substantial improvements over the constituent models, and achieves state-of-the-an performance with a pass @ I score of 0.862 on LiveCodeBench V6, newly-released questions spanning January April 2025. This represents significant improvement

---

Published as a conference paper at ICLR 2026
Table Ablation study results. We compare the performance on in-distribution tasks when we (1) remove the singular value fine-tuning in SLM; (2) remove the thinker-role selection (3) remove the
tri-role selection; and (4) use the last instead of the penultimate token. TRINITY achieves the best
overall performance. (5) remove agent selection but keep role selection
Method
LiveCodeBench
MATH500
MMLU
RLPR
Average
TRINITY
61.46
88.00
91.56
40.72
70.44
w/o Singular value fine- tuning
55.68
85.85
90.10
39.77
67.85
w/o Thinker-role selection
57.80
86.20
92.75
38.00
68.69
w/o Tri-role selection
58.28
82.00
91.64
36.15
67.02
w/ Last token
50.85
87.00
82.19
38.60
64.66
4laune-4-Sonne only
39.09
82.25
88.23
34.90
61.12
Gemini Pro 2.5 only GPT-5 only
46.51
83.05
79.41
43.00
62.99
59.54
75.66
90.74
37.87
65.95
over leading baselines: GPT-5 (0.838). Gemini 2.5-Pro (0.672), and Claude-4-Sonne (0.465). In addition, Figure 4 (bottom) also shows that TRINITY benefits from increasing max collaboration turns, improving from 0.823 to 0.863 turns increase from 2 to 6. This pattern makes intuitive sense and implies that TRINITY's 's capability stems from complex coordination and goes beyond naive routing.
4.5 ABLATION STUDIES
We conduct ablation studies to verify the effectiveness of our design choices, with results summa- rized in Table 2. First, removing the singular value fine-tuning consistently lowers scores, confirm- ing the benefit of adapting the coordinator model's internal representation directly, which allows it generate more effective signals for the head. Next, gradually removing the tri-role selection -first the thinker role, then the entire tri-role selection- proves detrimental to complex reasoning, causing substantial degradation switching to the final token, which often corresponds to a semantically sparse EOS token, causes severe performance collapse, particularly LiveCodeBench (more than 10 points drop). Finally, when we remove agent selection and instead send all queries to a single fixed agent while retaining only role selection, performance is significantly undermined. Together, these findings underscore the necessity of the full TRINITY design.
MATH500 (-6.0 points) and RLPR (-4.57 points). Additionally,
4.6 SEPARABILITY IN REPRESENTATION SPACE
The success of our lightweight coordinator depends on a well-structured representation space where hidden states are separable by task. We verify this by extracting hidden states from the coordinator during in-distribution runs and analyzing their linear separability using suite of methods, including linear classifier (SVM) and dimensionality reduction (t-SNE).
LCB
102
meru
mach000
Prispp Compunotz
50
0.72
D4T2
-10
Kandom:
0.250
-100
-20
-20
-10
10
20
30
-200 -50
100
Figure 5: Task type separability in extracted hidden states. Both are based on penultimate-tokke hidden states processed by the SLM on the input sequence, and the labels are from the task metadata.
Appendix A.3 reports the full results, and Figure presents two key analyses of the representation space. A linear SVM achieves perfect classification, far above chance level (0.25 for four classes), indicating near-perfect linear separability. The t-SNE visualization likewise exhibits clear, well-
8

---

Published as a conference paper at ICLR 2026
separated clusters, corroborating strong non-linear separability. This high degree of separability key factor that enables our lightweight, linear head to make effective coordination decisions with extreme parameter efficiency. Additional experiments in Appendix A.3 also indicate positive correlation between the separability in the representation space and the coordinator's performance.
4.7 SEPARABILITY IN PROBLEM OBJECTIVE
Changing the head architecture (see Appendix A.4) not only alters coordinator performance, but also reveals structural properties of the problem objective, namely the mapping from hidden states to agent/role choices that maximizes downstream task reward. Table shows that linear is the most re- liable choice overall across I. LiveCodeBench, RLPR, Math500, and MMLJU with sparse edging it out by a negligible margin on MMLU only. The block-diagonal-I head paired with an a rgmax output conversion intentionally designed to maximize independence among the ten logits-one block per agent/role-thereby suppressing inter-logit correlations. Parameter-count wise, this head uses only d, weights (about 10x fewer than the linear's d, na: e.g., 1,024 vs. 10,240 when 1,-1024, n.=10) and still retains competitive mid-tier performance. Importantly, argmax further increases independence by removing the softmax simplex constraint. With argmax, decisions depend only on the largest logit, so perturbations to non-maximal blocks neither reduce nor redistribute proba- bility mass, which reduces cross-block interference in both inference and fitness attribution. This result suggests strong block-5 separability (Definition 1) as property of the coordination objective, in addition to the geometric separability of hidden states studied in Section 4.6.
Table 3: Results by varying heads and output conversion. By default, the output conversion is softmax normalization. For block-diagonal 10, the output conversion is argmax.
Head
LiveCodeBench
MATH500
MMLU
RLPR
linear
0.615
0.880
0.916
0.401
low-rank
0.597
0.770
0.914
0.344
sparse
0.400
0.811
0.917
0.372
black-diagonal-2
0.336
0.776
0,897
0,378
Biock-diagonal-1 -0 argmax
0.551
0.812
0.802
0.376
4.8 SEP-CMA-ES vs RANDOM SEARCH Vs REINFORCE vs SUPERVISED FINE-TUNING
To empirically demonstrate the advantages of sep-CMA-ES for our setting (Section 2). we compare it against REINFORCE (Williams, 1992), SFT, and RS with fitness averaging, which is appropri- ate for binary rewards (see Appendix A.5). Table 4 shows that sep-CMA-ES outperforms other algorithms for training the coordinator, consistent with our theory (Section 3.3, Appendix A.1). REINFORCE exhibits jagged, high-variance learning curves with weak overall progress, which is expected under terminal (binary) rewards and weak parameter correlation.
Table 4: Comparison of sep-CMA-ES with REINFORCE, SFT, and RS. We compare the per- formance on in-distribution tasks for four learning algorithms under comparable budgets
Method
BeveChdeBench
MATH500
MMLJ
RLPR
REINFORCE
0.253
0.459
0.500
0.266
RS
0.374
0.794
0.897
0.345
SFT
0.592
0.786
0.906
0.360
sep- CS
0.615
0.880
0.916
0.401
As shown in Figure 6, Sep-CMA-ES adapts to a meaningful agent selection distribution that favors LLMs. By contrast, REINFORCE maintains an almost uniform selection pattern, indicating ineffective policy improvement. Although not shown in the figure, RS often collapses to unipolar choices, over-selecting a single agent or role and thereby significantly limiting diversity of agents and roles, which degrades performance. While SFT achieves competitive gains, does not scale to multi-turn coordination due to the prohibitive cost of label generation (see Appendix A.2).
5 RELATED WORKS
We use Model fusion to refer to methods that combine multiple models into a more capable system. Prior work divides into complementary levels: micro-level fusion in parameter space, where

---

Published as a conference paper at ICLR 2026
Age Swarosden toweoe
Agen Meribaaan Vaaiing MAA
Sward 2-3
Cwarc
2-2234214
DeepSenc3328
pari Garreu277
Figure 6: LLM selection distribution evolves as the coordinator learning progresses. Left:
Distribution evolution from -ep-CM Right: Distribution evolution from REINFORCE.
weights of parent models merged into child model, and macro-level fusion in data-flow space, where activations or outputs are passed across fixed models or model components.
Micro-level. Early approaches in micro-level utilize static recipes such as weight averaging task-balanced interpolation to integrate multiple model capabilities across domains with minimal computation (Goddard et al., 2024). More recent work has introduced optimization-basee methods to model-merging: for example, an evolutionary framework that searches over "merging recipes", demonstrating that learned strategies can outperform hand-designed ones and yield stronger gen- eralization (Akiba et al., 2025). However, because micro-level model-fusion is performed in the parameter-space these methods face the core limitation that they require access to model weights with compatibility requirements (Yadav et al., 2023; Yu et al., 2024). This confines their applicabil- ity to open-source checkpoints, while excluding the closed-source models that currently define the frontier of performance. Consequently, micro-level fusion cannot incorporate the strongest available models, motivating the exploration data-space approaches that treat models as black boxes.
Macro-level. Model fusion in the data-flow space can itself be performed at multiple degrees. Ear- lier works have allowed propagation of tensors through layers taken from different models (Bansal et al., 2021) sequentially processing individual tokens by different models (Mugeeth al., 2024). Our work most relates to a broader view of macro-level model fusion in which methods create stronger singular models by scaffolding or routing between multiple agents. In particular, multi- agent scaffolding techniques like Mixture of Agents (MoA) (Wang et al., 2024) and Multi-Agent Debate (MAD) (Liang al., 2023) form networks of agents which can extract capabilities from each individual model. Routing methods, such as Smoothie (Guha et al., 2024) or RouterDC (Chen al., 2024) aim to choose the best model or model response for given question. Similarly, Mas- Router (Yue et al., 2025) combines both by routing agents and human-designed scaffolds to form an adaptive multi-agent model per question. These methods rely on expensive multi-model inference or static, human-designed collaboration patterns. In contrast, TRINITY introduces a lightweight, learned coordinator that adaptively assigns dynamic roles to LLMs, utilizing the contextual repre- sentation generated from a SLM.
CONCLUSIONS
In this work, we introduce TRINITY, framework demonstrating that lightweight coordinator can orchestrate diverse LLMs to achieve state-of-thh-art performance. Leveraging tri-role protocol and trained with a highly efficient evolutionary strategy, our results suggest a promising path forward lies in engineering collaborative AI ecosystems rather than scaling monolithic models. A key limitation, however, is the gap between abstract reasoning and grounded execution, as the system can devise plans involving tools but cannot yet act them. Future work will therefore focus integrating a more heterogeneous pool of agents, including code interpreters and APIs, to bridge this gap and create general and capable problem-solving system.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We thank Koshi Eguchi and Kou Misaki for the infrastructure support, and the entire Sakana AI R&D team for their valuable comments and suggestions.
AUTHORS CONTRIBUTIONS
Jinglue Xu designed and curated the training datasets, conducted both theoretical and empirical analyses, and led the subsequent algorithmic and implementation development. Qi Sun proposed, named, and implemented the role selection algorithm, and led the training and evaluation experi- ments. Stefan Nielsen designed training and evaluation configurations, implemented baselines, and contributed shaping the trinity roles. Edoardo Cetin contributed to shaping early algorithmic designs and advised the project. Yujin Tang initiated and led the project, implemented the initial algorithm, and conducted the first experiments. All authors contributed the experimental design and paper writing.

---

Published as a conference paper at ICLR 2026
Ethics statement. Our approach focuses on collaboration between agents achieve better per- formance on existing benchmarks. As this work involves only computational improvements to established evaluation tasks without involving human subjects, sensitive data, potential misuse applications, we identify no ethical concerns.
Reproducibility statement. To ensure full reproducibility of our results, we provide comprehensive resources in the supplementary material, including source code and trained model weights. We also detail all model and task selection decisions within the paper. All base models and datasets used in this work are publicly available.
REFERENCES
Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. Evolutionary optimization of model merging recipes. Nature Machine Intelligence, 7(2):195-244, 2025.
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. arXiv preprint WxXiv2239.143166 2023.
Anthropic. Claude sonnet 4. https: www. http://wwwwtrroo.cccoo//e/////et
2025. Ac-
cessed: 2025-08-29.
Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, et al. Mt-bench-101: A fine-grained benchmark for evaluating large language models multi-turn dialogues. arXiv preprint arXiv:2402. 147662 2024.
Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revisiting model stitching compare neural representations. Advances in neural information processing systems, 14:225-236, 2021.
Shuhao Chen, Weisen Jiang, Baijiong Lin, James Kwok, and Yu Zhang. Routerde: Query-based router by dual contrastive learning for assembling large language models. Advances in Neural Information Processing Systems, 37:66305-6633 2024.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, 249-256. JMLR Workshop and Conference Proceedings, 2010.
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's 's MergeKit: A toolkit for merg- ing large language models. In Franck Dernoncourt, Daniel Preopiuc-Pietro, and Anastasia Shimorina (eds.), Proceedings of the 2024 Conference Empirical Methods in Natural Lan- guage Processing: Industry Track, pp. 477-485, Miami, Florida, US, November 2024. Associ- ation for Computational Linguistics. doi: 10. 10.18653/w1/2024.1 http:////......miinmieieem mnnlp-industry.36. //ac //aclanthology www/o/////22 2024. emnlp indust r3366/
URL https:
Neel Guha, Mayee Chen, Trevor Chow, Ishan Khare, and Christopher Re. Smoothie: Label free lan- guage model routing. Advances in Neural Information Processing Systems, 37:127645-127672, 2024.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 12948. 2025.
Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos, Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary computation, 11(1):1-18, 2003.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.080 033000 2020.
12

---

Published as a conference paper at ICLR 2026
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Train- ing compute-optimal large language models. arXiv preprint arXiv:2203.15556 2022.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang. Armando Solar-Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.0799 2024.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836/ 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL https:
httpg//bs/1111 /ttbs///11.... 6980.
So Kuroki, Taishi Nakamura, Takuya Akiba, and Yujin Tang. Agent skill acquisition for large language models via cycleqd. arXiv preprint arXiv:2410. 147.5555...473,, 2024.
Tian Liang Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking large language models through multi- agent debate. arXiv preprint arXiv:2305. 19118, 2023.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman. Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint 0rxiv:2305..00500 2023.
Mohammed Mugeeth, Haokun Liu, Yufan Liu, and Colin Raffel. Learning to route among special- ized experts for zero-shot generalization. arXiv preprini arXiv:2402.05885 2024.
OpenAI. Introducing gpt-5. https: /ttp////paaaaa///////e//......aa aatpraai.cccn/mmeeee/tttoodddde gust 2025. Accessed: 2025-08-29.
Au-
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a bench- mark. First Conference on Language Modeling. 2024.
Raymond Ros and Nikolaus Hansen. A simple modification ema-es achieving linear time and space complexity. In International conference parallel problem solving from nature, pp. 296- 305. Springer, 2008
Qi Sun, Edoardo Cetin, and Yujin Tang. Transformer-squared: Self-adaptive LLMs. In The Thirleenth International Conference on Learning Representations, 2025. URL https://
openreview festt/forum?e 4 9qmcvk.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
Hemish Veeraboina. Aime problem set 1983-2024, 2023. URL httpt:ht/////t kaggle. com/ datasets /ites//emms wwwemravveeebbnnoooo
me -et--eemmtt-----220000
Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances
large language model capabilities. arXiv preprint arXiv:2406.04400 04692, 2024.
Ronald J Williams. Simple statistical fralientt algorithms for connectionist reinforcement learning. Machine learning, 8(3):229-256 1992.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lppes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference on machine learning, pp.
PMLR, 2022.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging Re- solving interference when merging models. Advances in Neural Information Processing Systems, 36:7093-7115, 2023.
13

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 00Xiv:2500509388, 2025.
Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in Ilms, milmss and beyond: Methods, theories, applications and opportunities. arXiv preprint Ar0:::240 24l.:2060..6666, 2024.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Ab- sorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning, 2024.
Tianyu Yu, Bo Ji, Shouli Wang. Shu Yao, Zefan Wang, Ganqu Cui, Lifan Yuan, Ning Ding, Yuan Yao, Zhiyuan Liu, et al. Ripr: Extrapolating rlvr to general domains without verifiers. arXiv preprint arXiv:2506. 18254, 2025.
Yanwei Yue, Guibin Zhang, Boyang Liu, Guancheng Wan, Kun Wang, Dawei Cheng, and Yiyan Qi. Masrouter: Learning route lims for multi-agent systems. arXiv preprint arXiv:2502. 11133,
2025.
Tery Yue Zhuo Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indrancil Paul, et al. Bigcodebench: Bench- marking code generation with diverse function calls and complex instructions. arXiv preprint arXiv:2406.18877 15877, 2024.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 THEORETICAL ANALYSIS OF SEP-CMA-ES
In this section, we compare sep-CMA-ES with random search (RS) for maximizing J over P under binary rewards and strict budgets. All analyses are carried out in a covariance-normalizza chart and mapped back through the current diagonal D., fixing the metric mismatch between selection (in whitened coordinates tied to P) and stepping (in the original coordinates of P). A Hessian-based block-s separability condition on g := J controls inter- inter-block couplings after a positive diagonal scaling and links to the algorithm'ss ' dynamic scaling via a diagonal-cone comparability assumption. Con- centration bounds translate replication m into rank-quality attenuation without moment swapping. We also instantiate the specific case in our study (n s 10000, mRs = 32, MCMA = 16), directly tying the representation iss)) E H and the coordinator parameters 0 E P the observed efficiency of CMp-CMA-ES.
A.1.1 DEFINITIONS AND ASSUMPTIONS
Notations. We optimize J(Ø)- = E R(r) over the coordination space P C IK" induced by the head So H + |I acting on representation states h(s) E H C Bd. Set g(0) = -0) and (0)) g0)). We analyze contraction toward the origin (w.loo.g by re-centering P) in compact domain D C P. sep-CMA-ES maintains mean (iterate) m2 E P with radius re == ||mell. step-size og 0, and diagonal scaling
Dj diag( $1,1. ................................ 5n.1) > 0, y m. 022 2 In).
Whitened chart: x = D. (y m²) 0₂ (isotropic sampling). Direction: Up == m. m mm
projection Z 2) NO 1). Population size À [4+ 3 n] (z 2), l parents with weights (w,) 2y:A are order statisties. RS uses fixed replication count per candidate; in this appendix we set mRs- == 32. The atomic budget Benv counts Bernoulli calls.
Blocks, scaling, and operators. Let (Bi..... Bu} } partition f1.....m) n) (coordinate blocks in P). For any matrix M, (M) zeroes its diagonal; (M) zeroes diagonal and within-block entries. For diagonal D, let mD), 8min (D) be its largest/smallest diagonal square-roots and define
Smax (D)²
= Smax (D.)²
KD
Smin (D)2
(D(()
Smin (D,)²*
Definition 1 (Hessian-based block-s separability in P) There exists structural diagonal scaling
diag(s) .t..... 8n) > 0 such that the scaled Hessian Hs(0) == (0) is uniformly
nearly block-diagonee on D. With D(0)= diag( (f)),, one of the following dimensionless bounds
holds with a common E
E [0, 1):
sup
1/( (Hs(O)) D(#) 1/2
S SEH.
(B1)
BED
[1ssa)) $ EH,
sup
max
(B2)
OED iEB JEB, V (0)): Hss)))
pfg Ejen, gtp ||Is (0)aal
sup max BED KE
I SH (< 1).
(B3)
[0s
Within-bloch structure is unrestricted; 0 < pa S {0)} S L, 00 on D.
Assumption (Diagonal comparability) There exist constants Camp, Cemp > O such that for all
4,
Cemp
Cemps
equivalently Cemp/ Of (D)p
Si,t
This links the structural scaling S in Definition I the algorithum's dynamic scaling Dr.
Definition 2 (Metric-alignment factor) For any unit u and diagonal D 0.
(uT Du)2
LND 1] [1.1]
x(u, D)
TT D2u
15

---

Published as a conference paper at ICLR 2026
the squared correlation between the ranking score (u, 2) (whitened) and the progress score (Du, 2) (original metric on P).
Assumption (Local linear score with curvature remainder) There exist 0, Leurr 2 0, and a step-size window such that along the trajectory (in whitened coordinates)
Jmn + a D+2)
= + 7o (uz, z) Ex(z),
[sx()) < (heurv + CHEE) + o2|1=11?:
with constant Cи
>
Definition 3 (Rank attenuation under replication) Ler N = Beav/ /mRs] be the total number of RS candidates evaluated under the budget and Z == minjekeN (ux, 2 (k)).. With x- == min [z, 0},
(mme) )
E (Z )
E
Ues Eラ=1 Wg33:
PRS
2 [0, 1],
PCMA
E [0,1].
E
(2))
Ui E 2
(-2)) 27
Assumption 3 (Metric-alignment comparability) There exists Cx N 1 such that for relevant t,
1 x(us, (t)/
S
S Cx-
Cx
X(un, Do) (0)
Thus the alignment efficiency x(u,, Dt)/RD(t) stays within a bounded factor of its initial value.
A.1.2 SEP-CMA-ES VS RANDOM SEARCH WITH FITNESS AVERAGING
Rank noise and attenuation. Consider two candidates 21, 22 in the same batch with linear score gap A = yal 21 - 22)].. Averaging m Bernoulli draws per candidate yields the misorder bound
Pr Pim + D(zi)) & Am, + Di22) but (m, +oDizi) > J (m, + +
< Ce -em + Pr(curv +
where the curvature event {curv > /2 is due to & and admits the tail
Pr(curv
> < C' e'exp ( d
exeexp-d
+ O(en).
Hence, for a in a local monotonicity window (Assumption 2) the bggnal-o-curvaturre ratio order 1/EH, giving exponential suppression of curvature-induced flips. To scale this pairwise guarantee to batch selection, restrict attention to the Ollog N) (RS) or (log A) (CMA) most competitive order statistics: by extreme-value theory, the typical spacing between the winner and the next competitors is OLI/VInNN) and union- bounding only within this top cluster yields
P 2 - C,N log N. .Papp "Paip (mgs) - C2EH, PCMA 2 - CiAlog X Pap (mCMM)) - C2EH,
with paip (m) S aomppo2 + -dd/(s))) + O(en). In particular, choosing
1
In In
or m c220² (In (nn*+
m c2202
ensures 72 2 O(EH) for RS or CMA respectively. This gives a direct budget-replication trade-off inside
Budget-normalized single-round RS gain. Let Zi min (uo, 2(k) and ox E|( [](-21))] 2ln + O(InIn N). Define the high-probability event controlling batch norms
EN := {
max I 11=(2) < 2 21}.
t= In(N/co).
16

---

Published as a conference paper at ICLR 2026
so that Pr(EN) 1 - cu by Lausent-Massart tail plus union bound. On EN, the oracle step along 2o (with 2| the noisy-rank-selee selectel candidate) is
o' ='=(- (mo; )ozsel) || VV. r mmmo + o² 1022 ra
((uo, Do
D
Because selection is driven by (uo, 2) in the whitened chart and geometric progress depends
on (Douo,,), 2), the squared correlation factor x(ua, Do) = (ag Do "6 Dg E [1/kD, appears multiplicatively in the numerator's expectation, while the denominator is controlled by KD = mmax Do)2/ / /mmi (Do)² on Ex. After integrating out the event complement (which contributes O(v co (
NN we obtain
ró E/min 20 |Img- + DD sel 1|3
5N) Pas u.
2 x(uo: Do)
Ce
r0
KD KD 2//In((/co) 2ln((//oo)
(1)
for a universal C and ōN ((In N) fixed within the local monotonicity window loses only a universal constant factor.
Per-iteration CMA gain and geometrie regime.
Let
Op,A = utt Σ Wy (1))) j=1
Σ J=1
||
Bn 0(1/n),
1y 23:A
Kyg,
BAd
and Rn,A = 71 Kin = (1).
The oracle scalar step along Dr Σド= UyZ yields
Elr?
r 2 x(ue, De) KD(1) KA, PÊMA CêH.
??
The factor PCMA absorbs all rank noise effects (including sign inversions of the recombination direction); x(u,, (t) quantifies directional metric mismatch; and the O( term accounts for inter-block perturbations. Under a standard diagonal learning rate Ccov @(1/n), block-E separability and diagonal comparability imply that after T = (n) iterations D, enters an O(sH)- neighborhood stationary point, with
Eir rel <
Rin t--de-il (1-O)))) r
(3)
so the method achieves geometric decay at rate 2(1/n) per iteration once stabilized.
Head-to-head ratio and multi-round RS. Under a common atomic budget Beny, CMA uses mCMA À evaluations per iteration so T = |Beuv (eur/(mmMAA]]] while RS evaluates N = Beny/ /mss. candidates. Combining equation and equation 2, and invoking Assumption 3 to cancel X K up constant, gives
CMA gain
R 2
Benv
n 2 InN + 2
PêMA -2
CSH,
oy ~ 2 In N. (4)
RS gain
ICMMAÀ
UN
Phs
If RS expends its budget across T rounds with fresh batches N, and fixed (or monotone) a within the window, then gains add roughly E. O(In N,)/n), which is at most O(In Beuv)/n) for bal- anced Ntsstill logarithmic in bherea CMA accumulates linearly across iterations (until stabilization), explaining the systematic advantage in budget-tight regimes.
Trinity-scale instantiation. For n R 10000, À = [4 + 3 n] [4 + 3 31n 10000] = 32. With mcMA 16 and mRs = 32, budget matching across T CMA iterations yields N = (mcMAA/mRS)) T| |16 32/32) & [16T]. This gives oy R N.. Replication en- sures POMA (up to ((=H)). Plugging these into equation 4 shows that with the same Beny CMA's gain dominates for modest T (a few to a few dozen iterations), consistent with empirical results where the head acts h(s) H and updates 0 P under strict budgets.
17

---

Published as a conference paper at ICLR 2026
Proposition 1 Fix T
60] and let the CMA budget be Beuv McMAXT. If the replication
schedule ensures PCMA/ PRS 2 E (0, 1] and the metric-alignment efficiency stays comparable
across iterations (Assumption 3), then, up to O term,
CMA gain
T
2 In( maxfe, [ImcMA\/mRs TJ}) 7T
RS gain in J
C
In maxfe, L(momA (OROOMAA//Rss]]]1
The inequality holds for oracle step-sizes and, up to a universal constant factor, for fixed step-sizes within the local monotonicity window (Assumption 2).
Proposition 1. Proof. Set N [ImcMAA/MRS) so both methods consume the same bud- get. Use equation with v 2ln N + (n and ōy O(In (( to bound RS improvement. Sum equation 2 over
... to get CMA improvement at least CT&и- Apply Assumption to replace iteration-wise factors by constant multiple; the metric terms cancel in the ratio. Substitute Kp,2 Rp,2/ and compare to v ~ 2 N to obtain the bound with (PCMA/PRa) /ōRs)².
Ex(x(u, (t)
Proposition 2 Under Definition 1, Assumptions 1, 2, and 3, and replication schedule with POMA= ((H)), sep-CMA-ES achieves, after ((n) transient, the per-iteration contraction
Rv.A O(E)),
i.e. Elr?] exp(-e'T/n) for some d' > 0 depending on &,.A and the residual O Restrict- ing to diagonal covariances incurs only an O(EH) multiplicative loss relative to the dickonaa optimum.
Proposition 2. Proof. (i) Scale stabilization: With Czov (1/nn) and block-Eи separability
plus diagonal comparability, standard CMA drift shows D; reaches O(e --ciigbborhood of a stationary point To (nn) steps; then kt(() @1)) and typical x(u, D.) = ((1).
(ii Uniform per-iteration gain: Insert these bounds into equation 2 to get E(r rl] <
(1 k Kp.APCMA/me
))r+) iterate to obtain geometrie decay with rate 22(1nn). (iii) Close-
ness to the independent-blod ideal: Since H() is ((E J-close (operator norm) to block-diagonal on D, the population-optime full-covariance CMA differs from its block-diagonal part by O(en),
so using only diagonals loses (e) in the contraction constant. (iv) Rank reliability: Replication with mcMA Z (7"02)-1 log À keeps PCMA- = 1 O(en).
A.2 SUPERVISED FINE-TUNING
A.2.1 EXPERIMENT DETAILS
In this section, describe our setup and results for experiments with widely used imitation learning method, SFT. Concretely, we use a direct single-step state-action formulation where each training example consists of state and discrete action corresponding the choice of single LLM from the pool. SFT trains on these observed state-action pairs to imitate an oracle policy. Given state, the model is optimized our setting, the state is the coordinator's hidden-star state representation of the input, and the action is the index of the selected LLM.
predict the oracle's action via maximum-likelihod estimation. In
We first extract the labels from our per-question-ber oracle results. Specifically, each label generated by first identifying, for each seed independently, which LLM achieved the highest reward on that question. When multiple LLMs tie at the maximum reward, we uniformly sample one from the tied set. voting, The LLM selected most frequently across seeds becomes the final label for that question. In cases where multiple LLMs receive equal votes, we uniformly sample from the tied candidates to ensure unbiased label assignment. This approach yields realistic per-trial performance estimate while maintaining label diversity across the model pool. Table shows the resulting agent label distribution over different tasks.
Dataset.
then aggregate these per-seed selections across all seeds via majority
18

---

Published as a conference paper at ICLR 2026
Table 5: Agent label distribution by task. Percentage and count of datapoints where each agent was selected as best via majority vote across seeds.
Agent
LiveCodeBench
MATH500
MMLU
RLPR
Overall
Gemini Pro 2.5
17.7% (31)
18.0% (18)
16.1% (247)
17.3% (898)
17.1 % (1194)
GPT-5
39.4% (69)
13.0% (13)
16.4% (251)
14.7% (762)
15.7% (1095)
Claude-4-Sonne
17.7% (31)
21.0% (21)
14.4% (221)
14.4% (748)
14.6% (1021)
Qwen3-325 (reasoning)
7.4% (13)
12.0% (12)
12.9% (198)
15.1% (781)
14.4% (1004)
DeepScek-RI Baooseck-R--Qwe-3222
3.4% (6)
15.0% (15)
14.6% (224)
14.5% (750)
14.2% (995)
Qwen3-32B (direct) Gemma-3-27B-11
10.9% (19)
17.0% (17)
16.3% (249)
14.3% (739)
14.6% (1024)
3.4% (6)
4.0% (4)
9.2% (141)
9.8% (506)
9.4% (657)
Training. We optimize the coordinator using Adam (Kingma & Ba, 2017) with the frozen SLM, training only the linear head. After experimenting with various learning rates and batch sizes, we found that learning rate of 1 x 10-6 and batch size of 64 yield the best coordinator performance. The trained coordinator achieves scores of 0.592, 0.786, 0.906, and 0.360 on LiveCode MATH500, MMLU, and RLPR respectively. Figure shows the learned agent selection distribution, illustrating which agents the coordinator preferentially select for each task type.
100
Qwen-D
Selection Percentage (%)
Qwen-R
80
Claude
DeepSeek
Gemini
60
Gemma
GPT
40
20
livecadebench
math500
mmlu
rlpr
Benchmark
Figure 7: Agent selection distribution by task, Percentage of datapoints where each agent was selected by the trained coordinator.
A.2.2 COST IN LABEL CENERATION
The cost profiles of SFT and label-free training methods, such as sep-CMA-ES, REINFORCE, and RS differ substantially. For SFT, the dominant cost lies in label generation. Labels can be produced reasonable cost for direct mapping from representation space single-step agent selection, but become quickly intractable in multi-turn settings. In our direct-mapping setting, generating labels requires running seeds on datapoints across agents, resulting x queries.
147k L.M
For multi-turn coordination, the label complexity grows exponentially. Under our experimental configuration with up to 5 turns and 7 candidate agents per turn, the number of required LLM queries for agent selection alone scales by a factor of 74 x 2.4 x 103 relative to the single-step setting. Moreover, in multi-tur turn settings the role selection (among 3 roles) is also relevant at each of the turns, introducing an additional factor of 243 2.4 10?. In total, this yields a multiplicative factor of 71, 35 583, 443 5.8 105, inflating the cost to enormous 1.5 x 10 5.8 10 s
19

---

Published as a conference paper at ICLR 2026
8.7 10 LLM queries. By contrast, label-free training methods such Ssp-CMA-ES require explicit label generation and instead optimize the coordinator directly based on task rewards.
In summary, while SFT provide performance gains for direct representatioontto-gee mapping, its prohibitive bebl-eeenerttoon cost makes it unsuitable for training multi-turn coordinators, limiting its scalability.
A.3 FULL ANALYSIS OF SEPARABILITY IN REPRESENTATION SPACE
This section examines how well the extracted hidden states and the coordinator's output logits sep- arate relevant classes. For hidden states, greater separability implies that the SLM's representations encode richer context, providing a stronger signal for the lightweight head to make task-aware deci- sions.
First, we examine separability along three complementary axes: (i) Notion of separability; linear vs. non-linear; (ii) Label source: task-type labels (from metadata; input-side) vs. agent/role selection labels (from the head's logits; decision-side); (iii) Feature space: raw SLM hidden states (represen- tation space) vs. the coordinator head's 's output logits (coordination space).
For each cross consiniombin,,on, we use standard cimeesiionllity-rcduttiioo visualizations (PCA/LDA for linear structure; t-SNE/UMAP for non-linear structure) and report classification accuracy using linear and RBF SVMs as quantitative proxies for linear and non- non-linear separability, respectively. Features are standardized; visualizations are used qualitatively, and SVM accuracies provide the quantitative assessment. Figures 8-13 summarize the results.
Agent Selection from
Agent Head
10
PC2 (0:200)
-15
20
10
20
10
PCI (0.299)
PCL (0.419)
Type
Type
10
PC2 (0.203)
(0222))
Taik Tpp
-15
20
10
20
10
PCI (0.299)
PCL (0.419)
Figure 8: PCA analysis. All four plots demonstrate clear clustering patterns.
20

---

Published as a conference paper at ICLR 2026
Agent Selection from SLM
Agent Selection from Head Rakers 140
Sener 8:300: 2.65
Unowe Dicrminant
-2
Disc
Osscmmaanc
Teak Type from SLM
Task Type from Head
Thab Type
Uner
Unoer
Useer Diorr mirant
Uneer O Dectminnanc
Figure 9: LDA analysis. The Fisher's ratios indicate that the between-class scatter is approximately two to three times greater than the within-class scatter.
Agent Selection from SLM
Agent Selection from Head
15
UNMP2
UNAP2
-10
-10
10
15
20
10
20
UMAP
UMAP
Type
Typa Head
'
UMAP
Tei Tpp
-10
.5
15
URAP
UNAP1
Figure 10: UMAP analysis. The clustering patterns indicate strong non-linear separability.
21

---

Published as a conference paper at ICLR 2026
Agent
Agent
Appnt
1-SNE
JSNNE
Task Type from SLM
Task Type from Head
Type
1-SNE
1-SNE
man1500
Figure 11: t-SNE analysis. The analysis demonstrates particularly strong separability of task types in the hidden states extracted from the SLM.
Unee DoM Yark Tyae
BBP CVR Teak Type
Sereciid Aygir
Figure 12: SVM analysis hidden states extracted from the SLM. Classification accuracies: Linear SVM (task type)= = 1.000, RBF SVM (task type) 1.000, Linear SVM (agent selection) 0.713, RBF SVM (agent selection) = 0.776.
22

---

Published as a conference paper at ICLR 2026
Linaal Teak Type
EXP Sype
Toun Tyre
SaRbon 122
Selecedd Jqgert
Figure 13: SVM analysis on output logits. Classification accuracies: Linear SVM (task type)= 0.945, RBF SVM (task type) = 0.955, Linear SVM (agent selection) = 0.786, RBF SVM (agent selection) 0.783.
From Figures 8-11, both linear (PCA/LDA) and non-linear (UMAP)t-SNE) views reveal clear struc- ture. LDA's reported Fisher ratios (between/within scatter =2-3x) corroborate that much of the variance aligns with ais-discriminativee directions, while PCA shows separation already in the top components, suggesting a substantial linearly aligned subspace.
The SVM results (Figures 12-13) are especially revealing: in the representation space, task-type classification is near-perfect for both linear and RBF kernels, implying that penultimate-token hid- den states encode task semanties in a nearly linearly separable manifold even after standardization and class-balance controls. In the coordination space (head logits), task-type accuracy decreases while agent/role-selection accuracy increases (notably for the linear SVM, which aligns with head linear (see Appendix A.4)), indicating that the head compresses and reorients input seman- tics toward low-dimensional, decision- -aligied axes. This redistribution is consistent with a policy that projects context onto agent-specific logit directions, yielding simpler, more linearly separable boundaries for agent selection.
Next, investigate how representation space separability relates to coordinator performance. We train linear SVMs to predict agent selections from the hidden states extracted from the SL.M using the coordinator's agent selection as labels. Across the four datasets, LiveCodeBBench, MATH500, MMLU, and RLPR, the classification accuracies are 0,844, 0.764, 0.679, and 0.544, respectively.
This ranking aligns with our experimental findings: Sections 4.2 and 4.4 demonstrate that TRINITY shows stronger performance advantages over baseline methods on LiveCodeBench and MATH500 compared MMLU and RLPR. While these agent selection labels reflect the coordinator's learned behavior rather than ground truth assignments, the correlation between classification accuracy and relative performance gains suggests that tasks exhibiting greater separability in the representation space may be more amenable to effective coordination.
To directly examine the relationship between the intrinsic separability among the datapoints one task in the representation space and the coordinator's 's performance, we conduct a controlled experi- ment using synthetic datasets. Directly controlling separability in real task distributions is imprac- tical, as interventions such as injecting noise into hidden states may introduce confounding factors beyond separability changes (e.g., distributing samples ture). Therefore, generate synthetic datasets that replicate the exact structure of the coordinator's 's representation space (1024 dimensions, 7 agent classes, 4 task type clusters) while systematically varying separability levels. We control separability by systematically scaling the distances between
didstribbuiion or altering semantic struc-
23

---

Published as a conference paper at ICLR 2026
class centers while maintaining consistent within-class covariance, generating datasets whose mea- sured separability index (between-clase variance / total variance) vary.
Test Classification Accuracy
0.9
0.8
0.7
0.6
0.95
0.96
0.97
0.98
0.99
Separability Index
Figure 14: Separability index head classification accuracy. Trained synthetic datasets with systematically varied separability, the head linear exhibits a strong positive correlation between separabilityy index and test classification accuracy.
We train the exact same head used in our experiments, linear, on these synthetic datasets. Figure 14 reveals strong positive correlation between the separability index and the head's classification accuracy, with test classification accuracy increasing steadily as separability index rises. This con- trolled experiment indicates that higher intrinsic separability for a task in the representation space enables better head's performance on the task, independent of task-specific confounds.
A.4 HEAD ARCHITECTURE DESIGN
We describe four heads that maps the SLM's hidden state h d& to agent and role selection logits z E IR subsequently turned into probabilities with a softmax or argmax.
Linear head refers the most direct affine mapping (without bias) from hidden states logits. computes
Wh, W W
(5)
This head has exactly d, trainable parameters and serves as strong baseline. allows un- restricted linear combinations of hidden dimensions to express agent and role preferences while remaining simple and fast to train.
Low-rank head refers to factorized bottleneck with nonlinearity that replaces single dense map by two smaller projections. We use
> 0 ELU(x) (a(c" -1), x x< 0.1,
ELU(Uh),
z Vu-o.
with U d V Xr and fixed non-trainable scale R. In this work fix the bottleneck to r = 14. This choice can result in more parameters than a strictly compressed low-rank setting, but it intentionally adds depth and nonlinearity the head can capture non-linear patterns
24

---

Published as a conference paper at ICLR 2026
reduced per-projection cost versus single wide mapping. We initialize with Xavier-uniform (Glorot & Bengio, 2010) using adaptive gains:
U ~ u
6
u
18
18
tn
Sparse head refers to learnable dimension-selection mechanism that gates hidden features before a linear projection. The logits are
z @hōa), WER
(9)
where a E Rd a data-agnostic, learnable selection vector. The target number active dimen- sions is hc max(1, (1, [d, s)) with a learnable sparsity logit p. During training we form a differentiable top-k mask by sampling Gumbel noise and sharpening with temperature
[1.0, 20.0]:
$ (s + e)/T, e Gumbel(0, 1),
(10)
Cxson &
Sksoft Top (s,
C
Σdh Osol.
(11)
At inference, we use a hard top-k binary mask a TopK (s, k). This head has dua, + dA + parameters (projection weights, importance scores, temperature, and sparsity logit) and offers both regularization and interpretability by exposing which hidden dimensions drive agent and role selections.
Block-diagonal head refers to structuring the projection matrix with disjoint blocks that couple only subsets of hidden dimensions subsets of agents or roles:
W1
0
0
W,h h1 W2h
W 2
0
W
Z =
(12)
W
BJ
W BBb
with h [hi:...: h W; E Raixk, We use two concrete variants. Block-diagonal-2 sets B and partitions both hidden and agent/role dimensions proportionally. e.g.,
0: min
Pla
na
Σ
ajj
h4
a:da
i <<3
dp
Ej<2hy h3,
i=
Block Brock-diagonal-111 denotes the high-independence case corresponding to our setting with n, = 10 logits. It creates one block per agent/role (B 10, 1) and distributes hidden dimensions as evenly as possible, yielding
(d, mod 10)
W hy,
h
=
10
otherwise
Block diagonaia blocks moderate amount of parameter correlations, whereas dlock-diagonal-11 maximizes independence across the ten logits.
Table 6: Parameter size distribution in training. The size calculated based the SLM Qwen3-
0.6B. SVF refers to singular value fine-tuning.
SVF
linear
low-rank
sparse
block Block-diagonale
block-diagonal-T
Parameter Size
9216
10240
20680
11266
5120
1024
Table 6 compares the parameter counts of the different head architectures alongside the parameters trained singular value fine-tuning. Block-diagonal- achieves an exact 10x reduction in head parameters relative to linear (1,024 vs. 10,240 parameters for d, 1024. low-rank replaces the single d, x projection with two matrices E x and V E R x (with 14) and ELU nonlinearity, increasing the head size to 20,680 parameters. This is roughly a 2 increase over linear, trading parameter efficiency for additional depth and non-linearity in the mapping from hidden states logits.
10). In contrast,
25

---

Published as a conference paper at ICLR 2026
A.5 EXPERIMENTATION WITH LEARNING ALGORITHMS
We also compare our learning strategy with the REINFORCE algorithm and RS with fitness averag- ing. To ensure the total evaluation budgets were equivalent, we configured the baselines follows. For REINFORCE, we used batch size equal to the per-iteration evaluation size of sep-CMA-EES and ran for 60 iterations. For RS, we performed 32 trials for each sampled parameter vector, continuing until the total number of trials matched the evaluation count of sep-CMA-ES
For RS, warmstart it by calibrating the sampling range using the high-performing weights ob- tained via sep-CMA-EE Specifically, we sample uniformly from |-0.5, 0.5), band that slightly exceeds the observed extrema of those weights. For each sampled parameter vector, we run 32 independent trials and compare the average reward.
A.6 DATASET-AGENT SUBSET SELECTION
To construct pool of complementary agents and a curriculum of datasets that together amplify coordination gains, we cast selection joint subset selection over datasets and agents. Our for- mulation and procedure adhere to two principles: (i) evaluate gains in the error space to capture practical improvements across varying accuracy regimes; (ii) enforce complementarity, not merely strength, so the coordinator can exploit heterogeneous capabilities.
OBJECTIVE: MAXIMIZE RELATIVE ERROR REDUCTION UNDER JOINT CONSTRAINTS
Let M (M....., Mx} be candidate agents and DD (Di....: Dy) be candidate datasets. Let E(Dy. Ma) E [0, denote the observed accuracy of agent M, on dataset Dy under fixed inference protocol (without coordination, identical output-token budget, and prompting). For any dataset subset C' C DD and agent subset M' C EM, define
ZC
1 |C1
max D2EC' M.EM
Σ
Sc. = max
Σ E(D,, Mz). (13)
M.EM'
[C|
DjEC
Here, Ze.Me denotes the combination performance- ie.. the best-per-dataset accuracy obtained by coordinating each D, C to its synergistic interactions among agents within a dataset. While Zc.M² may not fully reflect end-to- end coordinated performance, it serves with it. In contrast, SC is the best single-agent baseline on the same C, obtained by fixing one agent in M' for all datasets. We then optimize the relative error reduction (RER):
agent in MM'and thus ignores potential
tractable proxy that is typically positively correlated
RER(C.M)) ZC.M' -SC.M
(14)
1 -....
This criterion rewards settings where no single agent dominates across all datasets and where spe- cialization materially lowers error. Our joint selection problem is
(C" MC" arg
max CCD,
RER(C,
(15)
JOINT DATASET-AGENT SUBSET SELECTION
For each dataset D,, and chosen model subset M C M, the best model for the individual dataset (doubly constrained) is
MMMM arg max M.EM'
E(Dy: Ma).
(20)
Given subsets D with |C| S Y and M' M with |M S X, the joint combination strategy performance averages the pat-dataset best-in-subset performance:
ZC.
Σ E(Dy)M
(21)
[C|
D,EC
In contrast, the single-model performance on the dataset combination fixes one model M, E M' for all datasets in C: Sr.c
Σ
E(D M.).
(22)
C
DeEC
26

---

Published as a conference paper at ICLR 2026
The best single model for the joint combination is therefore
MC.M arg max Sa,O, MrEM
(23)
with corresponding performance
C.M SMa
max S2, M,EM
(24)
Problem. Find the optimal subsets (M) that maximize the relative error reduction:
ZC - SC.Me
arg
max CCD. ICISY M'
1- -CMMM
(25)
Equivalently:
(1- - (1 Zc.M)
MC", = arg
max CCD, IM'Sx
1-Sc.M
(26)
Expanded form.
ED,EC M.xM,EM' E(D,, Mo) MMxM,EM [Ò ED,ec E(D Mx)
max CCD M M
MMN TcI EDJeC E(Dy,
(27)
CANDIDATE FILTERING VIA A TOP-5% PERFORMANCE FRONTIER
The joint search space for the problem is combinatorial. We begin by computing the performance matrix, estimating E EDD,,, My) for all pairs (Dy, Ma)) under the standardized protocol. Next, perform quantile filter at the top 5 5%: let denote the 95th percentile of {E(Dy. M,)} across all pairs and define
Kgs { y: M,) E(D,, My) 2 r}.
(16)
This top-5% filtering concentrates the subsequent selection on strong, demonstrably effective pair- ings while preserving diversity across tasks.
JOINT SELECTION VIA EXHAUSTIVE ENUMER ATTON (AND WHEN HEURISTICS ARE NEEDED)
Although joint subset selection over datasets and agents is exponential in general, our experimental sufficiently small permit exhaustive enumeration under our evaluation budget. Concretely, we enumerate all pairs (C, satisfying the coverage constraint, compute Zc.M', S², and RER(C, MM') for each, and select (C*, .M*) maximizing RER. When two candidates exhibit statistically indistinguishable RER, we break ties by prioritizing diversity in task and agent types. For example, we favor balanced mixture of reasoning agents and agents with direct inference capabilities.
regime admitted an exact solution. The candidate
Exhaustive enumeration scales poorly as |D|+ [M| grows; beyond moderate sizes, even after frontier pruning. the search can become prohibitive. In such regimes, the same objective can be pursued with budget-aware heuristies (e.g., greedy seeding followed by annealed or beam-style refinement) while retaining the coverage constraint and the complementarity- tie- tie-breaking.
A.7 EXPERIMENTAL DETAILS.
A.7.1 BASELINE SETUP
Individual Agent: We compare against the strongest individual models in our agent pool-GPT-5, Gemini-2.5-pro, and Claude-4-Sonnet evaluated maximum token limits to account for the accumulated context at each hop in our multi-turn framework.
eoaeateeeeavl at both 4K and 20k(5x)
Random Agent Selection: A simple baseline where agent is selected randomly each turn during the multi-turn collaboration process, providing a lower bound for structured agent coordination. And the max turn number is 5, same TRINITY setting.
27

---

Published as a conference paper at ICLR 2026
Self-Reflection:
An extended version of standard reflection where single agent produces an initial answer and then reflects on own output over five turns, representing iterative self ceff-imprveement without collaboration with others.
MasRouter: A recently method trained using the same dataset as our approach, with model selection based best validation loss. The training follows recommend settings and em- ploys cost-regularization and as detailed in the original paper, using the MMRL dataset with 256 samples, validating every epochs, and selecting the best checkpoint after observing sufficient evidence of overfitting.
RouterDC: routing method trained on 500 samples from the MMRL dataset match the conditions specified in the original paper. Each sample is repeated times to collect average performance across all workers for given question, with this average performance incorporated part of the training label.
Smoothie: Applied as test-time method to questions and outputs from each agents, eval- uated under both dependent strategies (sclecting agent per individual question) and independent strategies (selecting one single agent for the entire test set).
Mixture of Agents (MoA): Implemented test-time scaffold with single MoA layer and single aggregator layer, totaling model calls per question, where a random model is chosen to as the final aggregator.
Per Question Best: A theoretical upper bound representing the optimal performance achievable by correctly selecting the best-performing worker model for each individual question, providing the argmax baseline for comparison.
A.7.2 AGENT DISTRIBUTION ACROSS TASKS.
LCB
87.9
9.3
2.3
0.0
0.0
0.0
0.5
70
60
math500
9.2
5.3
82.2
0.0
0.0
0.0
3.4
Task Type
Selection %
mmlu
24.7
74.3
1.0
0.0
0.0
0.0
0.0
30
20
ripr
22.2
16.3
59.4
0.0
0.0
2.1
-0
A0
A1
A2
A3
A4
A5
A6
Agent ID
A2: Gemini-2.5-pro, A3: DerpSSck-RR---isiil---ee... A4: Gemma-3-27b-II. A5: Qwen3-32B
Figure 15: Agent distribution tasks. A0: GPT-5, A1:
(reasoning), A6: Qwen/Qwen3-32B (direct). TRINITY demonstrates strong task-aware agent lection strategy.
A.7.3 ADDITIONAL. BASELINE RESULTS.
Parallel Sampling. We report additional baselines using majority voting over samples per ques- tion. While TRINITY is designed to handle broad spectrum of tasks, majority voting with samples
28

---

Published as a conference paper at ICLR 2026
is only applicable to settings with small, discrete set of candidate outputs, such as multiple-choies benchmarks. Table summarizes the resulting performance on MMLU.
Table 7: Majority @5 baseline results for MMLU. For each question, the answer is chosen based on majority voting over parallel inquiries.
Model
Avg Score
Gemini Pro 2.5
91.57 + 0.70
GPT-5 Claude-4-Souee
91.31 0.23
90.99 0.39
LLM as Coordinator: We also evaluated an approach where an LLM is directly prompted to select the model and role at each turn. Gemini Pro 2.5 was chosen the coordinator given its superior per- formance among agents. However, this prompting-basee coordinator (64.14 4 70.44 average score). We observe that the LLM struggles to comprehend and manage the properties of 7 agents, resulting in inconsistent and suboptimal selections. This demonstrates that prompting with closed sourced LLMs is insufficient for capturing agents' inherent characteristics, which a coordinator acquires through training. See Table for detailed results.
method underperforms TRINITY'S trained
Table 8: Comparison between TRINITY and LLM as Coordinator
Method
Math500
MMLU
RLPR
LiveCodeBench
Avg
TRINITY
88.00
91.56
40.72 26.83
61.49 26.28
70.44
Gemini 2.5 pro as Coordinator
78.67
83.26
53.76
A.7.4 TOKEN USAGE TABLES ON IN-DISTRIBUTION TASKS.
Table 9: Average output token number of coordination methods
Model
Math500
MMLU
RLPR
LiveCodeBench
TRINITY
2,853
1,200
2,141
1,999
MOA
6,871
5,218
11,086
21,634
RouterDC
624
374
811
1,552
Smoothie
6,472
4,718
10,580
17,864
MASRouter
4,260
1,847
5,370
8,401
Table 10: Average output token number of each model in 5x Reff-Reflection
Model
Math500
MMLU
RLPR
LiveCodeBench
Qwen3-32B (direct)
2,075
1.746
1,949
4,207
Qwen3-32B (reasoning)
2,692
2,213
3,349
7,575
DeepSeck-R11 ---eek-----iissil------
3,988
3,811
4,609
12,228
Gemma-3-27B
1,704
820
1,750
3,714
Claude Sonnet
1,834
1,293
1,580
3,210
GPT-5
577
428
895
1,971
Gemini-2.5-Pro
5,142
5,460
6,710
11,046
Table 11: Average output token number of each model in 5x Context
Model
Math500
MMLU
RLPR
LiveCodeBench
Qwen3-32B (direct)
447
152
392
192
Qwen3-32B (reasoning)
1,019
382
1,047
1,784
DeepSeek-RI-Distil--e
1.343
538
1,369
4,066
Gemma-3-27B
342
146
159
Claude Sonnet
367
218
300
518
GPT-5
221
66
219
1.207
Gemini-2.5 2.5-Pro
1.153
579
787
5,753
29

---

Published as a conference paper at ICLR 2026
Table 12: Average output token number of each model in Default Context (4096)
Model
Math500
MMLU
RLPR
LiveCodeBench
Qwen3-32B (direct)
521
154
406
419
Qwen3-32B (reasoning)
995
397
1,191
1,789
DeepSeek-RI BaaeeekkRR--DA -i----tiiiilll2
1,175
485
1,181
3,443
Gemma-3-27B
437
147
330
483
Claude Sonnet
382
217
304
530
GPT-5
218
66
220
1,113
Gemini-2.5-Pro
819
578
2,396
30


Reviews:
[{"review_id": "uT9cMjt2m2", "reviewer": "Reviewer_vdHP", "summary": "The paper proposes TRINITY, which uses a lightweight coordinator to orchestrate multiple diverse LLMs at test time without modifying their weights. The coordinator consists of a compact SLM (≈0.6B) and a small decision head, and trained via sep-CMA-ES (diagonal covariance CMA-ES). Experiments on in-domain and out-domain benchmarks show consistent gains over single-model and routing/scaffolding baselines. And the ablation studies support the design choices of TRINITY's modules.", "questions": "Please see the weakness above.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Motivation and Practicality: If substantiated, TRINITY offers a practical path to leverage strong closed and open models without retraining, with promising performance-to-cost potential. Additionally, the tri-role design and hidden-state routing signal could influence future multi-model systems.\n\n2. Originality: TRINITY leverages the hidden states of small LMs to capture the optimal models and roles for current inference states. Specifically, the explicit tri-role protocol is a clean and practical abstraction. Using penultimate-token hidden states (instead of generated text) to drive coordination decisions is also a thoughtful and effective (revealed by the ablations) design. And the choice of sep-CMA-ES and its theoretical explanations about block-ε-separability perspective are interesting optimization angles for noisy binary rewards and high evaluation cost.\n\n3. Clarity: The paper is well-written and easy to follow. Specifically, the method is well-defined, with clear problem formulation and a compact parametrization. The multi-turn protocol and role-specific prompting are coherent. Ablations and representation separability analyses support the claim that a linear head over SLM hidden states can be effective.", "weaknesses": "The main weaknesses are in baseline settings. Addressing these would substantially strengthen the paper.\n\n1. The paper sets per-call max generation (4096 tokens) and turn limit (K=5) but does not report actual token usage of TRINITY and other baselines.\n2. See the left three baselines (Gemini2.5 / GPT-5 / Claude) in Figure 3. They are set to generate 4k or 20k within a SINGLE turn while TRNITY adopts multi-turn designs (thinker, worker, verifier). And any parallel TTS techniques are not considered as well. I think it is necessary to evaluating simple parallel strategies (like major@5) or more complicated ones towards a more fair comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "24xEWHmiBe", "reviewer": "Reviewer_3SKS", "summary": "This paper introduces TRINITY, a novel framework for coordinating multiple, diverse foundation models (LLMs) at test-time. Instead of merging models at the weight level or using a large model for orchestration, TRINITY employs a highly lightweight coordinator. This coordinator consists of a small language model (SLM) (approx. 0.6B parameters) and an extremely small trainable head. The core mechanism involves a multi-turn protocol where the coordinator reads the full conversation history, extracts a contextual representation from the SLM's hidden state (specifically, the penultimate token's) , and then makes two decisions: (1) which LLM from the pool to select, and (2) which of three predefined roles—Thinker, Worker, or Verifier—that LLM should perform. A key technical contribution is the training methodology. The authors posit that standard reinforcement learning (RL) is ill-suited for this high-dimensional, budget-constrained optimization task. Instead, they successfully optimize the coordinator's lightweight head using a derivative-free evolutionary strategy, separable CMA-ES (sep-CMA-ES). The authors demonstrate that this approach is highly effective, achieving state-of-the-art (SOTA) performance on multiple benchmarks and setting a new record of 86.2% pass@1 on LiveCodeBench.", "questions": "see my comments on Weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper's empirical results are its strongest asset. TRINITY consistently outperforms all individual baseline models (including powerful ones like GPT-5 and Gemini-2.5-Pro) and existing multi-agent routing methods across a wide range of tasks.\n2. The paper's central claim—that a tiny coordinator with fewer than 20K learnable parameters  can effectively orchestrate a pool of powerful LLMs—is a significant and non-obvious finding.", "weaknesses": "1. The paper’s central claim of the \"lightweight\" coordinator's efficacy is insufficiently supported because it lacks comparisons to two of the most critical alternative coordinator designs. The authors never compare TRINITY to a system where a powerful LLM (e.g., GPT-4 or Gemini-2.5-Pro) is prompted to act as the coordinator. Such a baseline would use prompting to select the next agent and role. Its omission leaves a key question unanswered: is the 0.6B SLM's lightweight nature a genuine advantage (proving coordination is a simple task), or is it a limitation (acting as an understanding bottleneck that a stronger coordinator could overcome)? The paper’s comparison of training algorithms (ES vs. RL vs. RS)  is incomplete. A more standard and powerful baseline for training a router is SFT via behavioral cloning. The authors could have used their \"Per-Question-Best\" oracle data  to generate a dataset of expert decisions and then trained the 10K head. By omitting SFT, the claim that sep-CMA-ES is the superior optimization choice for this problem is not fully substantiated\n\n2. A significant weakness lies in the rigid design of the \"tri-role\" protocol (Thinker, Worker, Verifier). These three roles are not dynamic but are hard-coded into the coordinator's head architecture, which has a fixed output dimension. This design severely limits the system's flexibility. As the authors note in their conclusion, the system cannot yet act on plans involving tools. To add a new, necessary role like a \"Tool-User\" or \"Code-Executor,\" one cannot simply use a prompt; one would have to modify the model architecture and retrain the entire system from scratch. This makes the framework difficult to adapt to new capabilities or complex problems that do not fit neatly into the predefined tri-role schema.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z5vpVCzUHE", "reviewer": "Reviewer_Ev8j", "summary": "This paper introduces Trinity, a lightweight coordinator that selects appropriate LLMs and assigns specific roles for the selected LLM to solve the given query in multiple turns. The coordinator, which comprises a 0.6B SLM and a lightweight head, is optimized with sep-CMA-ES, a high efficient strategy for this problem. Experiments indicate that Trinity improves the overall performance on both in-distribution and out-of-distribution benchmarks in several tasks, and the sep-CMA-ES strategy significantly outperforms RL and RS for the optimization of the coordinator.", "questions": "1. In Section 4.6, how does the separability of different tasks in representation space affect the performance of the coordinator?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The Trinity framework is lightweight and concise, which facilitates the cooperation of multiple state-of-the-art LLMs to solve user queries. It also has a good generalization on several tasks including math, coding, reasoning and domain knowledge, indicating the potential of application on different tasks.\n2. The sep-CMA-ES strategy significantly outperforms the RL and RS methods for optimizing the coordinator, demonstrating its high efficiency.", "weaknesses": "1. The effectiveness of the Trinity framework is limited compared to single-model baselines. As shown in Figure 3 and Table 1, compared with Gemini Pro 2.5, Trinity only brings an improvement about 1%-3% on all the in-distribution and out-of-distribution benchmarks except LiveCodeBench (where it outperforms GPT-5 for merely 3%). Considering the cost of training a coordinator and multi-turn interaction for selecting models, it shows limited effectiveness compared with simply selecting LLMs according to their adept tasks (e.g. select GPT-5 for coding tasks while Gemini Pro 2.5 for others).\n2.  The necessity of LLM selection is not demonstrated in the ablation study. It is unclear whether the performance will degrade when the Tri-role selection process in the Trinity framework is applied on a fixed single-model baseline instead of a selected LLM.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1HgKoxwggz", "reviewer": "Reviewer_bpif", "summary": "This paper proposes a three-stage framework by calling LLMs to collaboratively solve a problem. Their method, Trinity, interacts with several LLMs by assigning one of three roles (thinker, worker, and verifier). They demonstrate consistent improvements over individual frontier LLMs on a variety of math, code, and reasoning benchmarks.", "questions": "* I would like to have further discussion on the ablation study for removing the tri-role selection. I assume that you mean instead of assigning three roles, you directly route the query to one LLM and have it solve the problem (so essentially only 1 role). My question is what happens if you have only two (e.g. merge the thinker and worker together, and one verifier)? In Figure 1 it looks like the worker is simply following the instructions of the thinker, so couldn't we merge the two and get similar results (and also reduce the number of calls you need to make)? \n\n* I am still unsure on why the SLM needs to be trained via RL (again, why I think section 2 is too short). I am sure it works, but why wouldn't it be sufficient to train on simple (state, action) pairs, where the action is just the choice of picking one LLM over another?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The paper achieves good performance on benchmarks (to the best of my knowledge, 86.2% is indeed state of the art on LiveCodeBench). \n* To the best of my knowledge, this is the first work I have seen that applies LLM routing to a multi-turn setting, so I think this approach is novel. \n* The study is comprehensive, with ablation studies for different design choices, and interpretability study to demonstrate the effectiveness of delgating tasks to different LLMs.", "weaknesses": "I am concerned about the organization of the paper. Some notes:\n* Section 2 (problem formulation) feels very short. Some notation is not properly introduced: what is the definition of $B_{turn}$? how is it related to turns $T$? what is $B_{env}$, and what does it mean for the budget to be atomic?\n* Figure 2 could have better presentation - it is not immediately obvious that the penultimate output token is a part of the input text (which is my understanding) \n* Section 3.2: too many important details are moved to the appendix. For example,when you move the empirical analysis to the appendix, it seems like it is more of an afterthought rather than properly trying to motivate the use of evolutionary strategies. \n* Figure 3 is hard to follow, not enough visual prominence to show your method over others\n\nThe choice of singular-value finetuning for the coordinator head seems a bit strange. To my understanding, the method is using so as to make the number of learnable parameters extremely small. Yet in appendix A.3.2 you show that linear fine-tuning (e.g. just applying a full-rank matrix head) has the best performance. I am a little bit confused by the story here; in my mind, given the coordinator model is already quite small (0.6B), I would rather have full-power finetuning on the head to maximize performance than to save a few learnable parameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)