Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
Authors: William Overman
Mohsen Bayati
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Model
LLM
Alignment
Trustworthy AI
Model Routing
Conformal Prediction
Conformal Risk Control
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We introduce a lightweight, post-hoc routing framework, with provable guarantees, that safely delegates between language models with competing objectives.

Abstract: Modern language‑model deployments must often balance competing objectives—for example, helpfulness versus harmlessness, cost versus accuracy, and reward versus safety. We introduce Conformal Arbitrage, a post‑hoc framework that learns a data‑driven threshold to mediate between a Primary model optimized for a primary objective and a more conservative Guardian—which could be another model or a human domain expert—aligned with a guardrail objective. The threshold is calibrated with conformal risk control, yielding finite‑sample, distribution‑free guarantees that the long‑run frequency of undesirable events (such as factual errors or safety violations) does not exceed a user‑specified quota. Because Conformal Arbitrage operates wholly at the API level—without requiring access to model logits or updating model weights—it complements weight‑based alignment techniques and integrates seamlessly with existing cost‑aware cascades. Empirically, Conformal Arbitrage traces an efficient frontier, allowing users to define an acceptable performance level for one objective while maximizing utility in another. We observe that our method outperforms (in terms of accuracy) cost-matched random routing between models. These properties make Conformal Arbitrage a practical, theoretically grounded tool for trustworthy and economical deployment of large language models across a broad range of potentially competing objectives.

Conformal Arbitrage: Risk-Controllee Balancing of Competing Objectives in Language Models
William Overman
Mohsen Bayati
Stanford Graduate School of Business wpo@stanf ord. edu
Stanford
Graduate School of Business bayati Dayati@atanfor @etriitttanoooooo oodd edu
Abstract
Modern language-model deployments must often balance competing objec- tives-for example, helpfulness versus harmlessness, cost versus accuracy, and reward versus safety. We introduce Conformal Arbitrage, a post-hoc frame- work that learns data-driven threshold to mediate between Primary model optimized for primary objective and a more conservative Guardian- could be another model or human domain expert-aligned aligned with guardrail objective. The threshold is calibrated with conformal risk control, yielding knite-sample distribution free guarantees that the long-run frequency of undesirable events (such factual errors or safety violations) does not exceed a user-specified quota. Be- cause Conformal Arbitrage operates wholly the API evee--wthout requiring access to model logits or updating model weights -it complements weight- weight-based alignment techniques and integrates seamlessly with existing cost-aware cascades. Empirically, Conformal Arbitrage traces an efficient frontier, allowing users define an acceptable performance level for one objective while maximizing utility in another. We observe that our method outperforms (in terms of accuracy multiple-choice style questions) cost bost-matched random routing between models. These properties make Conformal Arbitrage practical, theoretically grounded tool for trustworthy and economical deployment of large language models across a broad range of potentially competing objectives.
Introduction
Large language models (LLMs) excel at reasoning, coding, and open-domain question answering. yet real-world deployments frequently need to navigate tensions hetween potentially competing objectives such as helpfulness and harmlessness cost and accuracy.
Current practices mostly tackle the tension between helpfulness and harmlessness by modifying the model itself: reinforcement learning from human feedback (RLHF) (Christiano al. 2017 Ouyang
ct al. 2022), direct-preference optimisation (DPO) (Rafailov et al. 2023), Constitutional Al (Bai
et al. 2022b), multi-objective fine-tuning (Zhou et al. 2023] Wang al. 2024) each produce single operating point along the Pareto frontier. While powerful, these methods demand expensive data collection, GPU-intensive retraining, and for API-only models
often not applicable.
For the cost versus accuracy tradeoff, there has been significant work on cascades: a cheap model handles easy queries and defers the rest to stronger fallback (Chen et al. [023] Aggarwal et al.) 2025: Zellinger al. 2023). Recently, Jocently.JJungeea et al. (2025) introduced Cascaded Selective Evaluation (CSE), calibrating mor-model confidence estimators via fixed-sequence multiple testing to obtain rigorous guarantees on alignment to human pairwise preferences. However, these approaches are tailored for controlling binary disagreement risk, while user may be interested in controlling arbitrary guardrail metrics at deployment time.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

We introduce Conformal Arbitrage (CA), lightweight router that sits outside the language models. The term "arbitrage" captures how our approach exploits the performance gap between specialized models to achieve superior outcomes than naive selection between models. Given (i) a Primary model optimized for the primary objective and (ii) a more conservative Guardian model or a human domain expert, aligned with a guardrail objective, CA offers principled alternative to randomized routing between models. Instead of merely alternating between models with some probability, CA learns single scalar threshold on how strongly the Primary model favors its top choice over alternatives (a notion we formally define as "score gap" later in the paper). This threshold determines when the Primary model's s confidence is sufficient to act upon its prediction versus when to defer to the Guardian, creating a principled decision boundary that optimizes the trade-off between objectives.
The threshold is calibrated using conformal risk control (CRC) (Angelopoulos et al., 2024), yielding finite-sumple, distribution-free guarantees that the long-run frequency (or magnitude) of undesirable events never exceeds a user-specified budget a. This enables precise control over trade-offs- users can explicitly specify how much they are willing to compromise on one objective to gain on the other. Because CA touches no model weights, it complements weight-based alignment and applies to closed, black-box APIs, making it a remarkably lightweight approach to achieving Pareto improvements over simple model selection strategies.
Our experiments study (i) the cost accurracy trade-off on TruthfulQA and MMLU, and (ii) the harblessssssellsssssss trade-off on PKU-SafeRLHF. All three benchmarks are multiple-choice settings in which the model is prompted to select from a fixed set of options. This regime is a natural fit for Conformal Arbitrage and in line with related literature such as Jung et al. (2025), which operates over binary choices. Thus we focus on the multiple-choice setting, but emphasize that the CA framework is not limited to such domains: the algorithm and theory carries over to free-text generation (see Appendix E and broader decision-making tasks. Across all settings we evaluate, CA traces an efficient frontier that consistently dominates random cost-matched routing baselines, while preserving finite-sample, distribution-fie guarantees via CRC.
Conformal Arbitrage transforms an immutable, potentially unpredictable LLM (or a family of LLMs) into a controllable system whose risk fisk-utility position can be dialed after deployment. In our experiments, we demonstrate this capability using statc-of-the- of-the-art LLMs from the GPT-4.1 series, OpenAl (005)). showing how our method enables fine-grained control over various tradeoffs without modifying the underlying models. By requiring only a few hundred logged examples for calibration, CA offers a pragmatic path toward trustworthy, cost-efficient and customizable language-mode services that can be adjusted to meet evolving requirements long after initial deployment.
2 Related work
Real-world deployments must strike a pragmatic balance between helpfuhness-supplyinng users with accurate and detailed information-une and harmlessness- anviiidinness--vddiinngng policy-breaking or dangerous content. Early alignment work framed the problem as a single-objective optimization: RLHF (Christiano et al. 2017: Ouyang et al., 2022) and its variant DPO (Rafailov et al., 2023) collapse nuanced feedback into a single reward model and therefore deliver one operating point on the Pareto frontier. Subsequent methods introduced explicit two facto training: RLHF helpful-harmless datasets (Bai et al. (022a)) Constitutional Al's self-revision loop (Bai et al. 2022b). and Bi-Factorial Preference Optimisation (BFPO) (Zhang et al.] (025) that casts the bi-objective RLHF loss as a direct supervised criterion. Safe-RLHF (Dai et al. 2023) separates a reward and cost head and enforces constraints by Lagrangian relaxation, while Circuit Breakers intervene generation time to halt policy-violating continuations (Zou et al. 2024)).
mixed
The PKU-SafeRLHF benchmark (Ji et al. 2023) was specifically introduced to quantify this helpfulness- harmtesssssaammssssssss trade-off, providing dual annotations that enable researchers to measure progress both dimensions simultancously. Anthropic's Constitutional AI (Bai et al. (022b) further explores alignment by embedding principles directly into model training. More recently, the MoGU framework (Du et al., (024) dynamically routes between model variants optimized separately for usability and safety. Empirically, while these approaches curb unsafe completions, they still lock the model into one fixed balance point between helpfulness and harmlessness.
Beyond Salpfulness-harrlessse graphic parity, domain domain-specific risk, etc.-can be in conflict. Many recent works have proposed
many other objectives- accuracy, cost, latency, fairness, demo-

---

weight- based strategies to navigate the resulting frontiers between such competing objectives. Re- warded soups linearly interpolates checkpoints fine-tuned on distinct rewards to trace that surface (Ramé et al. 2023). Directional Preference Alignment adds multiple reward heads for steerable inference (Wang al. 2024), MaxMin-RI.HF learns mixture of reward models to protect minority preferences (Chakraborty et al.] 2024), and MO-DPO converts several preference signals into a closed-form multi-objective loss (Zhou et al. 2023). These approaches nevertheless share two limita- tions: (i) they require access to model weights and retraining, and (ii) they provide no theoretical guarantees that the inherent guardrail metrics driving the trade-off (e.g., safety, accuracy, or cost) will stay within use--specified budget.
In contrast, our method of Conformal Arbitrage is weight-agnostic and sits outside the LLM. By calibrating a single threshold with conformal risk control (Angelopoulos et al. 2024), it transforms any pair of black-box models, one of which can be human, into continuum of operating points with provable finite-sample bounds on the chosen guardrail metric (e.g. harmlessness).
Conformal Arbitrage is thus closely tied routing and cascade approaches that tackle cost-accuracy trade-off's (Chen et al. [202] Yue et al. 2024 Ong et al. [0224] Aggarwal et al. [025 Zellinger et al.)
2025 Varangot-Reille al.) 2025), ). but can be used tackle any potential pair of objectives that may be in tension, thus abstractly covering cost-accuracy cascades as a special case.
However, unlike these previous approaches make no particular optimizations for any specific trade-off, including cost and accuracy, and we do not claim to out- out-perform such cascade systems on metrics for which they are explicitly optimized. Furthermore, compared to most routing approaches that rely on complex learned functions to distribute queries between models (Varangot- et al., 2025), y Conformal Arbitrage employs principled, anoorticalll-groundeee method using a single calibrated scalar threshold.
Scalable-oversight research explores how weaker agents humans can be organized into critique hierarchies that amplify limited supervision. Amplification and Debate delegate verification to inexpensive judges and, under certain complexity assumptions, achieve provable "weak- toe-tto--ttooo -srrtoo guarantees (Christiano et al., 20118; Irving et al.] 2018 Burns and et al.. (023). Process supervision instead labels intermediate reasoning steps that mistakes are caught early (Lightman et al. 2023). Self-reflection frameworks ask a model to generate critiques (and often revisions) of its own outputs (Madaan al. 2023] Yang et al., 2024 Tang et al. 2024). Post-hoc risk control strategies in model deployment have also gained attention, particularly through moderation and oversight models deployed by industry leaders (OpenAll 2023). Conformal Arbitrage complements these lines by offering statistically-soume escalation rule. It lets a Primary model act autonomously as much as possible while respecting some risk budget, and otherwise it forwards a potentially much smaller slate of potential actions or outputs to a human or Guardian model. Finite-sample bounds from conformal risk control make the Guardian's 's load-amd the residual risk explicitllyl budgeted, providing a lightweight, post-hoc path to scalable oversight without touching model weights.
The underlying selective routing approach of work resonates with classical selective prediction and reject-option frameworks initially formalized by Chow 1970)) and later refined in modern selective classification research (Geifman and EI- Yaniy 2019).
Conformal prediction (CP) and its generalization, conformal risk control (CRC) (Vovk et al. 2005
Bates et al. 2021| Angelopoulos et al., 2024), provide distribution-free, finite-sample guarantees that make them generally attractive post-hoe alignment tools for high-stakes LLM deployments. For instance, Chen et al. (2025) align language models with human risk judgments by controlling tail risks such as toxicity, while Su et al. (2024) demonstrate conformal prediction applied effectively to black-box LLM APIs without internal access. Additionally, conformal risk control has been leveraged in deployment scenarios such as action deferral, illustrated by the KnowNo framework (Ren et al. 2023 which uses conformal uncertainty quantification to trigger human oversight.
Conformal prediction and conformal risk control have been used to filter low-confidence QA answers (Kumar et al., (203),, retain only entailment-supportee sub-claims (Mohri and Hashimoto| 2024), and bound hallucination rates via abstention (Abbasi- Yadkori et al., 2024). Beyond marginal guarantees, conditional and adaptive CRC tighten coverage hard prompts (Cherian et al. 2024). and sampling- based set prediction extends CP to free-text generation (Quach et al. (024).. Framing alignment
as property testing, Overman et al. (2024) calibrate outputs to satisfy safety or fairness constraints
3

---

without retraining. Building on this lineage, we adapt CRC to learn a risk-calibrated switch between a Primary model and a Guardian model without retraining either model.
Conformal Arbitrage is most closely related to Cascaded Selective Evaluation (CSE) of Jung et al. (2025). CSB equips each judge with confidence score, calibrates per-judge threshold, and escalates through a cascade until some judge is confident, thereby controlling the Bernoulli risk that machine- preferred answer disagrees with human majority. Conformal Arbitrage addresses more general tradeoffs: it controls any bounded guardrail loss (safety, accuracy, cost, latency, etc.) and can filter a large action space to a smaller candidate set that Guardian or human refines, rather than abstaining on the whole instance. CSE's Simulated Annotators requires K-shot prompting (for K examples of preference annotations) the model N different times (for N human annotators) order obtain an ensemble prediction and access to predictive probabilities extracted from the model's ' logprobs, so every judge call is multiplied many-fold and is limited APIs that expose token-level logits. Conformal Arbitrage, by contrast, needs at most one call to the Primary and (when routed) one to the Guardian, treats the returned scores as opaque, requiring no access to logits or probabilities, and thus works with strictly black-box APIs.
3 Preliminaries
Conformal Arbitrage uses conformal risk control (CRC) to supply finite-sample, distribution-free guarantees on the guardrail metric while treating the underlying language models as black boxes. CRC extends the framework of conformal prediction (CP) (Vovk et al., [005 [Bates et al.) (222]) from binary error control to control of arbitrary bounded risks. We briefly summarize both ideas.
Conformal prediction Let X and y be the input and output spaces, equipped with a joint probability distribution, and draw an exchangeable sample (X,, Y ~ P where the first sample are used for calibration, and (X,+1, Yx+1) is used for testing. Given any predictor f X - y and score sy y) (e.g. ly - ff()),, let 91-a be the (1 a) empirical quantile of (X, Y-) The conformal set is defined by C(x) {v E y 8y (y,r,y) S TI-a). and enjoys the samile guarantee Pr( Y & C(Xx+1)) < Thus any black-box predictor attains (1 a:) coverage
without distributional assumptions (Vovk et al. 2005 [Bates et al. 202T).
Conformal risk control Many real-world objectives are not binary mistakes but expectations of task-specific loss for example,
rate, factual errors, mean latency, or excess dollar cost. Conformal risk control (Angelopoulos et al., (2024) handles such objectives by introducing a bounded, non-increasing loss curve (;) E BJ. where B is an upper bound on the loss, for each calibration point, indexed by tunable threshold E A C R. Defining the empirical risk R,, Σ L((A), CRC selects
A inf inf( (AE(AAA: n+1 Rn(A+ n n+1 sa..
B
and proves the finite-sample guarantee for E [A)+(A)] S a, again under assumption of exchange- ability between the calibration data and test point. Choosing La(A) = I[Y; & Cx.))) recovers classical CP; alternative losses yield risk bounds tailored to deployment needs.
4 Methodology: conformal arbitrage
We aim to invoke Primary model as often as possible (e.g. a helpfuuness-mxximiiingg or low-cost model) while ensuring, with high confidence, that a critical requirement (e.g. harmlessness, accuracy) is satisfied by routing calls to Guardian model (or human) as needed. 'he linkage between the two models formalized through conformal risk control (Angelopoulos et al. 2024)..
4.1 Setting
Let (21)22 be an exchangeable sequence of X-valued random variables that we refer to contexts. Each context æ admits finite, non-empty action set A(xx) A: C A, where A((aa) < x. Additionally, assume the existence of two functions L X x P(A) - IR and U X + R.

---

measuring, over subsets of the potential actions, loss for the guardrail metric and utility for the primary metric, respectively. We assume both of these functions satisfy the property that for Ai A2 we have A1) 2 A2) and (xx A1) 2 A2).
We assume access
R, where p is the Primary model (reward-secking or haaaap/lwwaacuraaee and is the Guardian model (safety-focused or costly/tigh-accuracy)) Despite this simple interface, each model may internally implement arbitrarily complex computations-anyy architecture that outputs a score for each (I, a) pair is admissible.
fixed, pre-trained models: : x :
Although write p(z, a) and g(x, a) deterministic, each model call may depend internal randomness ÇP, Sa. producing scores p(z, a, (P) and g(2, a, {G). Such tuples (x, p. g) remain exchangeable across samples, so the fmite-sample guarantees of conformal risk control are unaffected.
4.2 Calibration via conformal risk control
To calibrate our Conformal Arbitrage policy, we use conformal risk control (CRC) calibrate relaxation parameter À that satisfies user-defined risk budget a E (0, 1), controlling how much we can trust the Primary model before deferring to the Guardian.
We begin with an exchangeable calibration set of samples:
D {xi; P., G P. (p(2i,a)aacA G; = (g(ai, a)).AA.
Each sample consists of a context zj and the scores assigned by both the Primary model and the Guardian model across the available action set A, A(x,).
For any À 2 O, we define the X-relaxed candidate ser:
C2(x) = {{a A(x) p(x,a) 2 max p(a, p(a,a') a') 2}.
-
d'EA(z)
This set includes all actions whose Primary scores are within of the top score. In particular, larger values of À increase the size of this set. Since all of the subsets A' C A(x) that we will consider will of this form, (x)x). for some A, we adopt notation L;(A) L(x,, Cx(x)) and U.(A) Ux; Cx(x:)
We then define loss function on each calibration sample, measuring the residual risk that the Guardian model would assign to the best action in (xxx;
(((A) aEA(z;)
g(I;, a) (EC)(m,)
g( a).
Intuitively, this loss captures how unsafe the most promising action (as judged by the Guardian) is among the candidates the Primary model would consider acceptable under X.
To summarize overall risk, compute the empirical average:
a n LA(),,
Rr, (A)
and select the smallest À that satisfies the CRC incquality:
i = inf > 0: n+1 R. (A)
n
n m+1 S a}.
(3)
Definition (Relaxation Parameter). The relaxation parameter À defined as the minimal value of that satisfies the conformal risk control inequality in Equation B
This relaxation parameter controls the permissiveness of the candidate action set while ensuring that context remains bounded by a. The guarantee holds exactly at
the expected residual risk finite sample size and requires no assumptions on score calibration or context distribution.
4.3 Conformal arbitrage algorithm
We now describe the deployment-time decision procedure for selecting actions using the calibrated relaxation parameter obtained in Section 4.2 At each test instance, the algorithm first consults the
5

---

Algorithm Conformal Arbitrage
Require: Context relaxation parameter X, Primary model Guardian model y
1: Compute p(æ, a) for a E A(x)
a A(z) p(x,a) 2 max p(z, a') }}
2: Let Cx(x)
3: if [Cx(z)] = 1 then
4: return the unique element of Cx (x)
5: else
6: Compute g(x, a) for all a E Cy (x)
7: return a" arg maNaeCx(s) G(aa)
8:
end if'
Primary model to form X-relaxed candidate set. If the top action is sufficiently dominant (i.e., the set is singleton), selected; otherwise, the Guardian model selects from the X-relaxed set. The procedure is outlined in Algorithm
Although we present the algorithm assuming predefined action set A(x), the same formulation applies directly to free-text generation, where the potential action space (all strings up to some maximum length L) combinatorially large but still finite. In that case, the Primary's fixed generation policy induces finite slate
S(x) = {a,,..., ax) S Vys.
and the Conformal Arbitrage procedure operates identically with Primary- scores defined Sxx) and set to -oo for all actions in A(x) \ S(x). This instantiation is discussed in further detail in Appendix E
The guarantee that Algorithm Denforces an upper bound on the expected guardrail loss,
B[L(x, (x))) S
direct corollary of Theorem in Angelopoulos et al. (0024). which establishes finite-sample, distribution-free validity of conformal risk control under exchangcability. Intuitively, this ensures that the long-run expected violation of the guardrail metric- safety, factuality, any bounded risk measure memaure-remains
remains below the user-specified budget on unseen test contexts.
Corollary 1 (Guardrail control under Conformal Arbitrage). Let (I,, P., G be an exchangeable
sequence and let À be the relaxation parameter obtained by conformal risk control in (3. Then Algorithm 7 satisfies
[I Cs (#a+1))] Sa.
where the expectation is taken over the calibration and test samples. That Conformal Arbitrage inherits the same finite-sample, distribution-fre guardrail guarantee from Theorem of Angelloppssss
al. (2024)
4.4 Optimality amongst score-gap routers
To address utility as measured by the primary metric we define the following class of policies, "Score-gap routers," " in Definition Additionally, for this theoretical result, we will require stronger assumption of i.i.d. on the calibration data and test point.
Definition (Score-gap router). Fix a Primary score score function P.X A - R. and non-negative threshold X > 0. For each context let
ax(x) = arg max p(I, a), A(x) = p(z,a'(x)) aEA(m)
p(z, b), bE A(A
max
with the convention
A(x) = +00 if A(xx)
= 1. The score-gap router with threshold X, Rr : X
A (DEFER) acts
(a),), (((r)
RA(x)
DEFER otherwise,
where DEFER means "forward this instance the Guardian model."

---

Given the Primary model's confidence scores p(z, a), it chooses the top-scoring action whenever its margin over every altemaatve exceeds a non-negative threshold X. and defers to the Guardian otherwise. This rule mirrors Chow's Bayes-optimal reject-option classifier (Chow 1970): rather than rejecting an uncertain instance we escalate it to more conservative model.
Theorem I establishes that no other Score-gap router of the Primary scores alone can deliver strictly higher expected primary utility while still obeying the same guardrail risk budget a, up to vanishing O(n term. We let our Primary metric be measured by (A) [A)], which we assume to be non-increasing and K K--Lisschitz. This is natural as raising À can only shrink the set of contexts on which we choose the Primary model's ' output. The proof of Theorem iss provided in Appendix A.1
Theorem 1 (Utility
of Conformal Arbitrage). Fix a compact interval A [0, Amax). For each E A and every observation i define a guardrail loss L,(X) E [0, B) and a primary-utiliyy score (A) [0, maax), both non-increasing in X. Write
R(A)= = (A),
U(A) = (A).
Assume continuous and strictly decreasing, and U is non-increasing and K K-Lipschitz. For desired risk budget E (0, B) let X = inf(X A: R(X) s a}. Given an i.i.d. calibration sample D(nn) of size n, set
1 (A)
(A),,
==infAEE: + Ra))) I sa}
Then, with expectation taken over the calibration sample
((U(A)) -U()] On
U(X) U(²)-U()) U(1) O(n
sup
dEa
a())aa
Proof. The proof of Theorem is provided in Appendix A.I
note that the conditions Theorem assume that continuous and strictly decreasing which may not hold for particular instantiations of empirical loss functions finite calibration sets. This motivates a more general statement of Theorem] to cover flatter loss curves over A, which we provide in Appendix(AA.
5 Experiments
We test Conformal Arbitrage
different trade-off settings: cost-accuracy axis using the multiple-choice datasets TruthfulQA and MMLU, and a helpfuhnes-harmllessess axis using PKU- SafeRLHF. Each experiment follows the same protocol:
draw calibration split and use the loss given by Equation 2 to fit the CRC threshold Ả using Equation [] We evaluate the guardrail risk and primary utility of Conformal Arbitrage on a disjoint evaluation split, and compare against single-mode model baselines and random routers. We report the results for TruthfulQA and PKU-SafeRLHP in the main text; the results for MMLU are qualitatively similar and appear in Appendix D
5.1 TruthfulQA: cost versus accuracy
We first study Conformal Arbitrage on the multiple-choice split of TRUTHFULQA (Lin ct al. 2022). a benchmark designed expose factual misconceptions in language models The benchmark contains 684 questions, each paired with four answer choices and exactly one correct label. Here we consider primary objective to be minimizing cost, while the guardrail metric is factual accuracy.

---

Experimental set-up The Primary model is gpt-4. -02--oo222--0-1--- the Guardian model its larger counterpart gpt-4. 1-2025-04-14, This is the natural choice considering that our primary and guardrail metrics are cost and accuracy, respectively Both are queried in zero-shot, multiple-choice format that elicits a real-valued confidence score [0. 1] for each option. We use temperature=0 1, max_ tokens=50; replies that fail JSON parsing default to uniform scores, maintaining exchangeability. Exact prompts appear in Appendix B.I
We keep the Primary's s raw scores, but binarize the Guardian's s as g(z, a) 1 if a is its top-ranked answer and correct, and otherwise. Thus, when the Guardian answers correctly we assign confidence 1 to the correct choice and O to the three distractors; when it answers incorrectly we assign 0 to every choice, reflecting total uncertainty. This binarization not required one could instead feed the Guardian's ss real-valued scores into Conformal Arbitrage, but this binarization makes the exposition crisper: the calibrated risk level a now translates directly an a 100% drop in accuracy relative to the accuracy of the Guardian. See Appendix [.4 for results of using the real-valued scores directly. With Equation the loss is (A) 1 Guardian correct and Cx (x4) 전 a*} for a" arg m(a;) ( a). Conformal risk control chooses the smallest Ả whose empirical 0.10 guarantees the overall accuracy falls by at most ten percentage
mean loss is S or; e.g., points relative to an Laway-GGurrdia policy.
Each trial draws n = 400 calibration and N = 284 test questions. We fit Å via Eq. (3 on = {0, 0.010........ 1} and repeat the calibration-evaluttion loop 30 times with fresh random splits.
For baseline comparison we compare the performance of Conformal Arbitrage to random router that for each risk level a matches the average cost of our method but chooses the acting model uniformly at random, thereby controlling cost without calibration.
Results Figure and Table show that CA traces efficient cost-accuracy frontier, beating the cost-matched random router every risk level except a 0.25 while always respecting the a-level guardrail budget. Tightening a from = 0.25 to 0.05 raises accuracy from 0.62 to 0.81 at 2.6 the cost. These results demonstrate that statistical calibration not mere stochastic routing- essential for efficiency.
6-0.05
0.75
0=0011
Accuracy
0-0.15
0.70
Guardian
Conformal Arbitrege
Random Routing
0.55
Interpolation
Average Cost per Example (S)
Figure 1: Accuracy vs. cost (TruthfulQA), mean + I std over 30 trials; small points show individual CA runs.
Ablation studies Across ablations CA's frontier stays stable. First, varying the calibration split (300, 400, 500 points; Appendix B.3) lifts accuracy by only point or two with flat cost, matching
We use prices from htomp//pppp///pppaaaoaoo///////p on May 15, 2025.

---

Table I: Accuracy, cost per 1000 examples, A, A above random baseline, and Guardian usage (mean
# std over 30 trials). Calibration size 400.
Policy
Accuracy
Cost ($/1000)
Å
A
Guardian
Primary
0.559 + 0.015
0.032 0.000
0.0%
CA (a = 0.25)
0.621 + 0.025
0.188 = 0.024
0.277 + 0.067
-0.011
27.7
CA (a = 0.20)
0.672 1172200000 0.025
0.234 = 0.033
0.403 + 0.058
+0.019
34.3 5.3%
CA (a 0.15)
0.302 = 0.035
0.529 1 0.059
+0.029
44.9 1 5.7%
CA (a 0.10)
0.407 = 0.026
0.706 0.031
+0.031
62.1 1 4.4%
CA (a 0.05)
0.806 + 0.017
0.521 = 0.035
0.867 1 0.040
+00016
78.9 1 5.6%
Guardian
0.833 = 0.011
0.620 = 0.001
100.0%
theory that few hundred examples suffice (Angelopoulos and Bates 2022). Second, feeding CA the Guardian's 's raw scores instead of the 0/1 binarization nudges accuracy up under tight risk budgets and down by similar amount when the budget loosens (Appendix B.4). Third, letting the Guardian operate on the full action set rather than the X-relaxed subset (unrestricted routing. Appendix B.5 raises accuracy a few points at roughly 10% extra cost; because the Primary still acts on the same contexts while the Guardian's 's menu only expands, the finite-sample risk bound is unaffected, though the primary metric (cost) can overshoot the target. Finally, swapping the Primary gpt-4. 1 1-nano for the stronger but pricier gpt-4. 1-mini (Appendix (B6) lifts the low-cost end of the frontier by about 0.22 accuracy points. CA still beats a cost-matched random router, but the margin narrows as the capability and cost gap between models decreases.
5.2 PKU-SafeRLHH:: helpfulness versus harmlessness
We consider how Conformal Arbitrage can be applied to the tradeoff between helpfulness and harmlessness. The PKU-SAFERLHP corpus contains ~ 90k prompts, each paired with two distinct LLM responses Each response is annotated for (i) which response is more helpful, (ii) which is safer, and (iii) severity label sev E {0, 1, 2,3) 3} indicating the extent of the safety violation (higher worse). We retain only the rows where the two responses differ in severity level and where the more helpful answer is not the safer answer. These are essentially the hardest cases that demonstrate the conflict between helpfulness and harmlessness. This leaves N = 3, 552 examples.
Experiment set-up We report two quantities: Empirical Human Alignment, the fraction of prompts whose chosen answer matches the annotator-preferee (more-helpful) reply, and Safety-violation loss, the excess severity of the selected answer above the safer one (0 is ideal, larger is worse). The Primary model gpt-4. 1-2025-04--111 is instructed (Appendix[C) to assign a real- valued helpfulness score p(2, a) [0, 1] to every candidate reply while ignoring policy concerns. The Guardian is deterministic rule that selects the lowest-severity reply, g(x, a) = 1 [sev(z, a) = ming sev(r, a')}, mimicking a human safety judge.
Over 30 trials we draw 500/500 calibration evaluation splits from the 3,552 prompts, tune Á on = f0, 0.0025,.... . 1}. and evaluate risk budgets a {0.10, 0.20, ..,,...,.. ...60.)). Baselines are (i) Primary-only (arg max, p(x, a)). (ii) Guardian-only (lowest-severity reply), and (iii) random router that calls the Guardian with p E (0.2, 0.4, 0.5, 0.6, 0.83.
Results Fig. shows that Conformal Arbitrage traces an efficient frontier between helpfulness and harmlessness. Exact numerical results are given in Appendix C.2 The mean of every CA model dominates the linear interpolation between the Primary and Guardian models that can be obtained via randomized routing. Additionally CA meets the finite-sample guarantee E [[L] S a for every guardrail budget a, as indicated by the mean of each point falling to the left of its corresponding vertical target.
6 Conclusion
Conformal Arbitrage converts a fixed pair black-box language models (or a model-human pairing) into a continuum of operating points frontier of competing objectives. By calibrating a single

---

0.6
Empirical Human Alignment (Helpfulness)
0.2
Primary
Guardian
Conformal Arbitrage
Random Routing
Interpolation
0.1
Safety Violation Loss (Harmfulness)
Figure 2: Harmfulness vs. helpfulness (PKU-SafeRLHD), mean + 1 std over 30 trials.
score-gap threshold with conformal risk control, CA supplies finite-sample distribution- free guaran- tees that user-chosen guard-rail metric stays within budget while maximizing a second objective such as accuracy, helpfulness, cost efficiency. Empirical results show CA outperforms cost- and risk-matched random routing, recovers most gains of the stronger model a fraction of the cost, and works with closed-API deployments without accessing weights or logits.
Limitations & future work Our analysis focuses multiple-choice settings, where the Primary and Guardian models score fixed, finite action set. In Appendix E we outline how Conformal Arbitrage naturally extends free-text generation, and we include one empirical demonstration on OpenAl HealthBench to illustrate this instantiation. However, applying CA to open-ended generation tasks warrants deeper empirical exploration. We forgo task-speeific optimizations (c.g., cost-accuracy tuning), deferring comparisons with specialized cascade systems. Finally, we analyze only single- step, two-model router; deeper or adaptive cascades may be possible. Future directions include (i) integrating adaptive CRC (Blot et al. (205),, (ii) adding tailored optimizations to benchmark against state-of-the-art cascades, and (iii) extending CA to multi-model or agentic pipelines.
References
Yasin Abbasi-Yadues Yadkori, Ilja Kuzborskij, David Stutz, András György, Adam Fisch, Amaud Doucet, luliya Beloshapka, Wei-Hung Weng, Yao- Yuan Yang, Csaba Szepesvári, Ali Taylan Cemgil, and Nenad Tomasev. Mitigating Ilm hallucinations via conformal abstention. arXiv preprint arXiv:2405.01566. 2024. https:/: http//aaxvvvvgg//eaaa/
org. 2405. .05663
Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upad- hyay, Manaal Faruqui, and Mausam. Automix: Automatically mixing language models. In Proceedings of the 38th Conference Neural Information Processing Systems (NeurIPS), 2025. arXiv:2310.12963
Anastasios N. Angelopoulos and Stephen Bates. A gentle introduction conformal prediction and distribution-fie free uncertainty quantification, 2022. URL httttttts//aiiiiii org/obgg//... abss2117..72
10

---

Anastasios Nikolas Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Confor- mal risk control. In The Tweifth International Conference on Learning Representations. 2024.
Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero-Canden Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. Hcalthbench: Evaluating large language models towards improved human health, 2025. URL htthhtt::// /tctps://ariiv arxiv org/ aba/ 25..255008771
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage. Zac Hatfield-Dodds, Danny Hernandez. Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022a. URL https: /ttes///aaxxvi org/ abs/2204 2282222224022222
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini. Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie KerI, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield- Doddds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional ai: Harmlessness from ai feedback, 2022b. URL httthht////aart //axiiv org/ abs/ //1/2.222222222
Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael I. Jordan. Distribution-free, risk-controlling prediction sets, 2021.
Vincent Blot, Anastasios N Angelopoulos. Michael Jordan, and Nicolas J-B Brunel. Automatically adaptive conformal risk control, 2025.
org/ http://wwww..rr///////.0.00.0 http//aab2/22255380 17819
Collin Burns and et al. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312.009390 2023.
Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Furong Huang, Dinesh Manocha, Amrit Bedi, and Mengdi Wang. Maxmin-rlhf: Towards equitable alignment of large language models with diverse human preferences. In ICML. Workshop on Models of Human Feedback for Al Alignment, 2024.
Catherine Yu-Chi Chen, Jingyan Shen, Zhun Deng, and Lihua Lei. Conformal tail risk control for large language model alignment, 2025. URL //ttps://arxv....
2502. 20285
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.051777 2023.
John J.
Cherian, Isaac Gibbs, and Emmanuel J. Candès. Large language model validity via en-
hanced conformal prediction methods. In Advances in Neural Information Processing Systems, volume 37, 2024. URL 0 hrt.000ffaaaaaab666..4411101111. htt.://o---Abtttaaec-oooeeeeeeee
neurips wwwrips.cc/pppeeeeeeeeeeee
faapper_ffles/eeeeeerr Files/paper/2000
html
C. K. Chow. On optimum recognition error and reject trade-off. IEEE Transactions on Information Theory, 16(1):41-44 1970.
Paul Christiano, Evan Shlegeris, and Dario Amodei, Supervising strong learners by amplifying weak experts. In arXiv preprint 1rXiv:1810.08551 2018.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, volume 30, 2017.

---

Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310. 12773, 2023.
Yanrui Du, Sendong Zhao, Danyang Zhao, Ming Ma, Yuhan Chen, Liangyu Huo, Qing Yang. Dongliang Xu, and Bing Qin. Mogu: A framework for enhancing safety of IIms while preserving their usability. In A. Globerson, L.. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neu- ral Information Processing Systems, volume 37, pages 87569-87591. Curran Associates, neurips. Inc., 2024. URL https: https: htttp:://proceedinggs neerrii cwtpscccpppppee wwwes/ilepppeeeee htllee/paperr/22//f1ie 9 8777611-410006666001111110000001 Paw.roofoepppee---ooeeeeeeeeeeee pdf
Yonatan Geifman and Ran El-Yaniv. SelectiveNet: A deep neural network with an integrated reject option. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2151-2159. PMLR, 09-15 Jun 2019.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding, 2021.
Geoffrey Irving, Paul Christiano, and Dario Amodei. Ai safety via debate. arXiv preprint arXiv:1805.00899, 2018
Jiaming Ji. Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety aligmment of LLM via human- -refen-preeeeece dataset. In Thirth-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. g0QovXbFw3
Jachun Jung, Faeze Brahman, and Yejin Choi. Trust or escalate: Lim judges with provable guarantees for human agreement. arXiv preprint arXiv:2407. 18370, 2025.
Bhawesh Kumar, Charles Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, and Andrew Beam. Conformal prediction with large language models for multi-choice question answering. In Proceedings of the ICML 2023 Workshop on Neural Conversational Al: Teaching Machines to Converse, 2023. URL RtRL.httss://aate
org/ abs 22055 18404
Sam Lightman, Nikita Nangia, and Samuel R. Bowman. Process supervision improves mathematical reasoning in -ffin-of-thoughh models. arXiv preprint arXiv:2305. 20050, 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022. URL http:///aaaai...org////////00000 27990
Aman Madaan, Guangtao Tu, Yiming Chen, Yulia Tsvetkov, and Graham Neubig. Self-refine: Itera- tive refinement with self-feedback. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 2023.
Christopher Mohri and Tatsunori Hashimoto. Language models with conformal factuality guarantees. arXiv preprint UrXiv:2402.10978 2024. httpss////aa org/ httg/abb/24422
10978
Isaac Ong, Pranay Patil, Shivang Agarwal, Harsh Gupta, Nelson F. Liu, Yanda Chen, Percy Liang, and Tatsunori Hashimoto. Routellm: Learning to route llms with preference data. arXiv preprint MxXiv:2406.18665 2024.
OpenAl. Gpt-4 system card, 2023. http:://opnnai.co//hggggg.
OpenAl. Introducing gpt-4.1 in the api, April 2025. URL RtIL/httss////peeett http://pppaaa...omimma..o// cott//ee/////gtt4-1-11 1/ Accessed: 2025-05-15.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In
12

---

S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 37730-27744 Curran Associates. Inc., 2022. URL https:///proceeddings iee neurips...... wwwpappr.ppppe.peeeppppeeee..eep files/ files/ httes//ppper/22///1leeee pdf
ume 37, 2024. URL baa
William Overman, Jacqueline Jil Vallon, and Mohsen Bayati. Aligning model properties via conformal risk control. In Advances in Neural Information Processing Systems, vol-
http:/err.r....cccppppeeeeeeeeee www.fffllesppppeee wat.ee/ppppr/00000 2024/ hatp::/78855..aa6532222222444422
html
Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S. Jaakkola, and Regina Barzilay. Conformal language modeling. In Proceedings of the Twelfth International Conference ha//ab///20000 10193
on Learning Representations (ICLR), 2024. URL BtRLhttps:/// //axxi wwtp://.rggaab////////e
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.182290 2023.
Alexandre Ramé, Guillaume Couairon, Mustafa Shukor, Corentin Dancette, Jean-Baptins Gaya, Laure Soulier, and Matthieu Cord. Rewardedsoups: Towards pareto-optimal alignment by interpo- lating weights fine-tuned on diverse rewards. In NeurIPS, 2023.
Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, and Anirudha Majumdar. Robots that ask for help: Uncertainty alignment for large language model planners. In 7th Annual Conference on Robot Learning, 2023. httpppps//pppperreeee.eeeeeeeeee 4ZK80DNyFXx
Jayyan Su, Jing Luo, Hongwei Wang, and Lu Cheng. Api enough: Conformal prediction for large language models without logit-access, 2024. haattp::/aariiv.org//aaaa 21120
Yunhao Tang, Rohan Anil, Hyung Won Chung, Zhang Chen, Zhifeng Dai, and Barret Zoph. Scrit: Self-evolving critic for scalable oversight. arXiv preprint 4rXiv:2403.09613, 2024.
Clovis Varangot-Reille, Olivier Caelen, Emelyne Goffinet, Alison Baumann, Alexandre Chauvet, and Patrick von Platen. Doing more with less implemeennin routing strategies in large language model-based systems: An extended survey. arXiv preprint arXiv:2502.004009 2025.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning Random World, Second Edition. January 2005. doi: 10. 1011000978---001--666 York, Inc. 2005.
Springer- Verlag New
Haoxiang Wang. Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, and Tong Zhang. Arithmetic control of Ilms for diverse user preferences: Directional preference alignment with multi-objeccive rewards. In ACL, 2024.
Hanjiang Yang, Tianyu Fu, Xu Wang, Yao Yao, Sean Welleck, Etienne Levin, Anqi Nic, Kyunghyun Cho, and Jason Weston. Deepcritic: Large language model critics for scalable oversight. arXiv preprint arXiv:2402.05497 2024.
Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. In The Twelfth International Conference on Learning Representations, 2024. Uttpttss:///pentt 6okaSf
Michael J. Zellinger, Rex Liu, and Matt Thomson. Cost-saving Ilm cascades with early abstention. arXiv preprint arXiv:2502.0903 2025.
Wenxuan Zhang, Philip Torr, Mohamed Elhoseiny, and Adel Bibi. Bi-factorial preference optimiza- tion: Balancing safety-helpfulness in language models. In The Thirteenth International Conference on Learning Representations, 2025. URL
13

---

Zhanhui Zhou, Jie Liu, Chao Yang, Jing Shao, Yu Liu, Xiangyu Yue, Wanli Ouyang, and Yu Qiao. Beyond ame-reffrrene-fits ait alignment: Multi-objective direct preference optimization. arXiv preprint arXiv:2310.03708, 2023.
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers, arXiv preprint arXiv:2406.0433 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately reflect the paper's contributions, including the introduction of Conformal Arbitrage post-hoc framework that mediates between competing objectives in language models with finite-sample guarantees. The claims match the theoretical results in Section and experimental evidence in Section 5
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We include dedicated "Limitations" paragraph in the Conclusion Section 6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: Yes we provide full assumptions in the statement of Theorem in Section 4.4 and provide complete proof in Appendix A.I
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversell, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides detailed information about experimental setup, including model specifications, prompting (with full prompts in Appendix), calibration protocol, and evaluation metrics.Section and corresponding appendices contain comprehensive information needed to reproduce the results. Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The Supplemental Material contains code for reproducing the main experimen- tal results of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nnis.cc.
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
7 //nips. //tips.cc/pub.le. wwtpc//pppp.g.udeeeeoeeepppeppee httpa//cgguidee/ooeeuuubiiiises prrooeSubmissionool11..
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper thoroughly documents experimental settings in Section inncluding calibration size, evaluation protocols, specific prompts (in corresponding Appendices), model details (e.g., gpt-4. 20220-20255-- -4)144, and hyperparameter search spaces (e.g..
A {0, 0.02, 1.0)).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All experimental results in Section 5 include standard deviations across multiple trials. Figures and22show error bars representing one standard deviation, and all tables include + notation for reporting standard deviation.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper details that all calls are made via APIs, thus can be handled on standard CPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///nnurrp..ccpppbbeae wttpcccccpcccccccccccccccccccltt httppp/scccpppccccccticccce////s
Answer: [Yes]
Justification: Our work conforms the NeurIPS Code of Ethics. It focuses improving LLM safety and utility, with experimental evaluations using standard benchmarks, and no apparent ethical issues in methodology or applications.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
18

---

Justification: The paper discusses positive societal impacts of Conformal Arbitrage by enabling better safety-utility tradeoffs in language model deployment. addresses the important issues of helpfulness vs. harmlessness and provides framework to adjust these tradeoffs with statistical guarantees.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper doesn't release models or datasets that pose risks for misuse. It addresses safety LLMs but does not itself introduce high-risk assets requiring safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly cites the datasets used (TruthfulQA, PKU-SafeRLHIF MMLUU with appropriate citations and URLs in footnotes. The commercial models used (GPT-4.1 variants) are also properly acknowledged with pricing information from OpenAL.
19

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper doesn't release new datasets, code, or models.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicablle as well as details about compensation (if any)?
Answer: [NA]
Justification; The research doesn't involve crowdsourcing or human subjects; uses existing datasets and commercial LLM APIs.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
20

---

Answer: [NA]
Justification: The research doesn't involve human subjects and therefore doesn' 't require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
While the paper evaluates LLMs (GPT-4.1 variants), LLMs aren't 't used as original components in the research methodology itself. The paper studies LLMs but doesn't t use them develop the core Conformal Arbitrage method.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

A Utility-optimality of CRC among score-gap routers
A.1 Utility-opiimality under strictly decreasing risk
We restate Theorem | Teere for convenience and provide the full proof.
Theorem (Utility-ppiimality of conformal risk control), Fix a compact interval A = [0, Xmax] For each À E A and every observation i define guardrail loss Lx(A) [0, B) and a primary-utiiity score (A) E [0, Umax) both non-increasing in Write
R(A)) = ()L,
U(A) = (U).
Assume R is continuous and strictly decreasing, and U is non-increasing and K -LLipschitz For a desired risk budget a E B) let
X =nnfff E A R(A) S aa)
Given an ii.dd calibration sample ((n) of size n, set
( - - L()),,
i=imf(eEA: n+. RA(A) + NfIT saj.
Then, with expectation taken over the calibration sample
-U(A))-UUU On
(4)
sup U(Ā) U(A)) = On (1)
(5)
R(A)
Proof. Theorem 2 from Angelopoulos et al. (2024) shows that the threshold Á selected by the conformal-risk-contre rule satisfies a tight risk lower bound
2B
B[Ln+1(A)]
Which by the fact that 2 R(A.) implies R(²) R(x.)
2B
Thus get
n+1
2B
0 S R(A.) R(Å) -
n n+1
Strict monotonicity and continuity R on the compact interval A imply that its inverse is Lipschitz; writing m = inA [R()) > 0 gives | X,l < 2m/(mmn + 1)).
Then by our non-increasing and Lipschitz assumptions the utility curve,
2KB
U(A.) - (²²) < Umax a1+ - Å1 m(n + 1)
Here U(}) is still random through X )(D(m)), while U(A)) is deterministic. Integrating the inequality over the distribution of ((n) preserves the bound and yields 44.
If satisfies R()) & then, by strict monotonicity of R, must have Ā 2 X, and hence
U(X) S UU..).
Therefore, for every calibration draw D(x),
2KB
(U(²) - U(A)) -({)) m (n + 1)
sup
R()Sa
Taking expectation establishes 5
22

---

A.2 Utility-optimality under general w-regularity
We generalize Theorem | by replacing the restrictive strictly decreasing assumption with general w Regularrty condition on the risk curve R(X).
Theorem 2 (Opilityoptimality of Conformal Risk Control under w-Regularity). Fix a compact
interval A [0, )m For each À E A and every observation i, define guardrail loss L((A) e
10. B] and a primary-utility score (A(A) [0, Umax], both non-increasing in A. Write R(A)
[A), and UA) (A).
Assume continuous and non-incre non-increasing, and U is non-increasing and K K-Lisschitz. Crucially, assume R. satisfies the w-Regularity condition: there exists a non-decreasing function w R + R with (() as 5 + such that for any A1 22 A with X1 S X2:
x2 X w (R(Ai) IR(22).
For desired risk budget a E B), let X, = inf E A R(A) S a.. Given an i.i. calibration sample D(1) of size n,, set
1A 1
(A)))
==inf(AEA: n n+1 Ra(A) + n+I S aaj
B
Then, with expectation taken over the calibration sample, the convergence rate is determined by wi
ElU(A.) - -A() O (w (n=
E
sup
U(A) U( = = (w (n
SEA
R(ASaa
Proof. We follow the established chain of reasoning: Risk Gap + A-Gap + Utility Gap.
Theorem 2 from Angelopoulos et al. 2024 guarantees tight risk bound for the selected threshold Å: Since R().) o, the X-dependent Risk Gap is bounded
B[R())) 2B
n=1
2B
0 < R(A.) R(Å) <
n 1+1
We apply the o-Reeularity condition, which controls the width of flat regions in R. We only need to consider the case 2 X as the utility gap non non-positive otherwise. Setting A1 = X, and X2 = X:
- x. S w(((A.) - R())).
Since R(A.) - R(Å)= <
2B u+1
and w is non-decreasing:
2B
Since U is non non- non-inereasing and K K-Lipschitz, the utility gap is bounded by the X-gap:
2B
UAA.) - U(}) < KI S
n1
Taking the expectation over ( establishes (6
U(A.) U()) - (w ())
As Ā must satisfy R(A) < a, it must be that Ā > X (otherwise R(²) a.). Since U non-increasing.
U()) s UQ..). Thus, the supremum is attained at Xa:
2B
SU()) U()) < K w
sup λEA R(ASaa
n
Taking expectation establishes
23

---

B TruthfulQA: additional experimental details
B.1 Prompt for score elicitation
System message
You expert xho evaluates multiple choice questions.
Instructions
Assign confidence score to each anever choice on a scale from to
certainly
certainly correct
Don 't ansign similar scores to choices unless you are genuinely equally uncertain
Response Format
Output UXLYY valid object with "acorea" key containing an array numbers
Exauple: "scoree": [0.1, 0.8, 0.05, 0.05]
No explanationa, juat the JS0N object
User message
Question:
(Sverbatin question text>)
Anewer Choices:
<jaon. dooon.dduaacccee (choices)> (chooccc)))
Respond DNLY with J8ON object containing your idence scores for these choices,
e.g. "scores": [0.1,
0.05, 0.05]
and Guardian (gpt-4. 11111122200----1 models receive exactly this dialog. We parse the returned JSON, extract the scores array, and then normalize that sums to these normalized values are used as the per-choice confidence scores p(x, a) and g(z, a) throughout calibration and evaluation.
Both the Primary (gpt-4. 1ept-4.1-nano 1-nano- -i--ann-2025-0e
B.2
Cost calculation
For every question in every trial we record the four token counts
(sprimary
porimary
out
inn
_uuardian
fout
gqarddian)
ie. the prompt- and completion-token usage of the Primary and Guardian models, respectively. Each
model is billed its own per-token prices in Cou
primary
peimary
and in
guardian
out
guardian
For M E (primary, guardian] the cost is
cost = M 4M +
M gM out
Hybrid (routed) calls If the Primary's s X-relaxed conformal set contains m > 1 answers, the query routed to the Guardian. To upper-bound this second leg we start from original, full-prompt token count dill (the question shown to both models) and scale it according to the fraction of choices actually sent:
Tin
اللي (0.5 + 0.5 |
where n is the total number answer options. We keep the Guardian's completion length fixed at guardian yielding the estimate
fout
est cost guardian
guardian C.
guardian guarrdion
Gin
fin
Cout Yout
costrotal costprimary
cost guardian
Because we (i) retain the Guardian's 's full completion length and (ii) shrink prompt tokens linearly with m/n, this accounting deliberately conservative: an implementation that truly shortens both prompt and completion when would only reduce the spend. Hence our reported savings under Conformal Arbitrage are lower bound
B.3 Calibration size ablations
To assess how many calibration examples are needed for Conformal Arbitrage (CA) to stabilize, we repeat the TruthfulQA experiment with calibration split sizes (300, 500). Tables 2 report
'Token prices follow the OpenAl schedule of 15 May 2025.

---

Table 2: Tra Accuracy, cost per 1000 examples, A, A above random baseline, and Guardian
usage (mean std 30 trials). Calibration size 300.
Policy
Accuracy
Cost ($/1000)
A
Guardian %
Primary
0.557 + 0.012
0.032 = 0.000
0.0%
CA (a = 0.25)
0.619 + 0.038
0.184 = 0.030
0.280
0.079
-0.008
27.3 27.3
+ 5.1%
CA (a 0.20)
0.667 + 0.033
0.236 = 0.027
0.405
0.048
+0.016
35.0 1: 4.3%
CA (a 0.15)
0.710 0.034
0.304 0.040
0.542 0.063
0.063
+0.027
15.666.50
CA (a 0.10)
0.394 = 0.041
0.048
+0.028
60.3 6.7%
CA (a = 0.05)
0.801 = 0.022
0.513 = 0.048
0.861 = 0.059
+0.018
78.3 7.7%
Guardian
0.833 + 0.010
0.615 = 0.001
100.0%
Table 3: Tra Accuracy, cost per 1000 examples, X, A above random baseline, and Guardian usage (mean + std over 30 trials). Calibration size 500.
Policy
Accuracy
Cost ($/1000)
A
Guardian %
Primary
0.554 + 0.012
0.032 = 0.000
0.0%
CA (a 0.25)
0.625 = 0.040
0.184 0.019
0.301 + 0.039
-0.005
27.3 3.4%
CA (a 0.20)
0.672 = 0.042
0.042
0.233 = 0.025
0.414 0.045
+0.020
34.6 4.2%
CA (a 0.15)
1715=00033
0.301 = 0.024
0.563 0.038
+0.031
45.1 3.9%
CA (a = 0.10)
0.765 0.033
0.402 = 0.025
0.712 + 0.026
+0.032
62.0 4.2%
CA (a 0.05)
0.806 + 0.029
0.524 0.024
0.881 + 0.028
+0.019
80.1 + 3.8%
Guardian
0.833 + 0.010
0.615 = 0.001
100.0%
accuracy, dollar cost per 1000 questions, the fitted threshold Å, and Guardian usage at the same guardrail levels a E (0.25, 0.20, 0.15, 0010, 0.05).
Across all risk budgets the frontier is stable. Moving from 300 to n 500 changes the mean accuracy by at most 1 1-2 percentage points. Average cost remains effectively unchanged (differences 3%) for every a. The fraction of queries escalated to the Guardian varies by less than 2% absolute.
B.4 Guardian scoring ablation
Table 4: Accuracy, cost per 1000 examples, X, A above random baseline, and Guardian usage (mean + std over 30 trials) when the Guardian's 's raw scores used instead of hard 0/1 binarization.
Policy
Accuracy
Cost ($/1000)
A
Guardian %
Primary
0.556 +: 0.012
0.032 = 0.000
0.0%
CA (a = 0.25)
0.598 + 0.037
0.163 = 0.026
0.203 + 0.089
-0.021
24.0 + 4.5%
CA (a - -0.20)
0.661 + 0.035
0.222 = 0.028
0.394 + 0.059
+0.014
32.8 + 4.4%
CA (a -0.15)
0.714 0.028
0.304 0.032
0.558
+0.029
45.6 5.3%
CA (a 0.10)
0.771 = 0.025
0.414 = 0.030
0.741 0.036
+0.032
63.1 4.3%
(a = 0.05)
0.813 + 0.021
0.554 = 0.059
0.917 0.056
+0.013
84.8 + 9.6%
Guardian
0.831 + 0.010
0.615 = 0.001
100.0%
When calibrating Conformal Arbitrage (CA) TruthfulQA binarize the Guardian's 's output in the main experiments aeserimentt-assignnngg score to the Guardian's 's highest scoring answwer if and only if is correct and 0 to all others-to make the accuracy loss (A) in Eq. (2) directly interpretable as "fractional drop in accuracy" relative to an always-Guardian policy. Here we repeat the experiment but feed CA the Guardian's 's raw confidence scores. The resulting frontier is reported in Table
For tighter risk budgets (a < 0.10). aceuracy rises by roughly +1- 2% while cost unchanged. At loose risk budgets (a > 0.20 0.20), accuracy drops slightly (about 0.5% 1%). Cost differences remain negligible. With respect to the risk guarantees, feeding softer scores does not affect the finite-sample CRC bound; every row in Table satisfies the [E[L] S constraint expected.
25

---

B.5 Unrestricted action set routing
In our main pipeline the Guardian is asked to choose only from the X-relaxed candidate set C (x) generated by the Primary. Here we study a more liberal variant-denoted CA that lets the Guardian reconsider the entire action set (xx).
Table shows that unrestricted routing lifts accuracy by roughly 3 percentage points across the tested risk budgets, with the largest gains appearing in the looser regimes (a 2 0.20). The calibration diagnostics in Table Texpaainn why: as grows the conformal the Primary prunes away the correct answer. When the Guardian can inspect all options it can often recover that mistake, yielding the frontier Figure The cost penalty is modest- on average 7- 10 % above the restricted CA variant.
shrinks, increasing the odds that
In many applications the action space is much larger than the four-choice multiple-choie setting considered here. Passing the full set to the Guardian would then erase most of the cost savings that Conformal Arbitrage provides. Moreover, for trade-off's other than cost-accuracy (e.g. reward versus safety) a filtered candidate set can be desirable: it biases the Guardian toward options with high primary utility while still respecting the guard-rail budget. For these reasons present the restricted policy as the default and treat unrestricted routing as an informative ablation.
0.85
0.80
6=0003
[--0.1
Accuracy
0.65
Primary
0.60
Guardian
Conformal Arbitrage
Callarated Full Routing
Random Routing
0.55
Unrestricted Random Routing
Linear Interpolation
0.0001
0.0003
0.0005
0,000b
Average Cost per Example (S)
Figure 3: Accuracy vs. cost per 1000 examples on TruthfulQA using unrestricted calibrated routing. Each point corresponds the mean over 30 trials; error bars represent standard deviation. Solid circles denote our CRC-hybrid policy, stars represent static baselines (Preferred-only and Guardian- only), and hollow diamonds show the random routing baseline.
Table Accuracy, cost per 1000 examples, Å, A above unrestricted random baseline, and Guardian usage (mean + std over 30 trials). Calibration size = 400. CA rows report the unrestricted variant.
Policy
Accuracy
Cost ($/1000)
A
Guardian
Primary
0.559 0.015
0.032 = 0.000
0.0%
CA" 0.25)
0.687 + 0.021
0.206 0.025
0.277 = 0.067
+0.046
27.7 + 3.9%
CA* (a = 0.200
0.713 1.0022
0.247 = 0.033
0.403 = 0.058
+0.052
34.3 5.3%
CA* (a = 0.15))
0.741 0.022
0.313 + 0.036
0.529 = 0.059
+0.050
44.9 + 5.7%
CA" (a -0.10)
0.785 +
0.421 + 0.027
0.706 0.031
+0.043
62.1 + 4.4%
CA' 0.05)
0.012+00015
0.525 0.035
0.867 0.040
+0.020
78.9 5.6%
Guardian
0.833 0.011
0.620 0.001
100.0%
26

---

Table 6: Calibrated values and resulting conformal-set sizes for CA as used in the main text (means
+ s.d. over 30 trials). As the risk budget a tightens (top bottom), the candidate grows.
Set size
0.25
0.277
0.067
1.457 0.024
0,20
0,403 + 0.058
1.801 + 0.038
0.15
0.529 4: 0.059
2.105 + 0.045
0.10
0.706 + 0.031
2.587 + 0.041
0.05
0.867
0.040
3.253 0.034
B.6 Model choice ablation
To probe how Conformal Arbitrage behaves for the cost-accuracy tradeoff when the capability gap between the two models is smaller, we replace the original gpt-4. 1-nano Primary with the stronger but costlier gpt-4. 1.-miii.... This boosts the stand-alone Primary accuracy from 0.56 to 0.77-only 6 pp below the Guardian-and and raises the token price four-fold. Even in this compressed regime cost-matched random routing: a it gains +2 pp in accuracy while invoking the Guardian on just one quarter of the queries, and at a= 0.025 it matches the Guardian's 's accuracy for 40% of the cost. The detailed numbers are collected Table 7 and the corresponding cost-accuracy frontier is visualized in Figure
CA still delivers meaningful improvement
Table Model-ablation results on TruthfulQA with gpt apt--4.. -4. -mini as the Primary. Accuracy, cost per 1000 examples, fitted threshold Å, improvement over a cost-matched random router (A), and Guardian usage. Means + standard deviation across 30 trials.
Policy
Accuracy
Cost ($/1000)
Guardian
Primary (4.1-mini)
0.7738 0.0113
0.126 0.000
0.0%
CA 0.050)
0.8156
0.0194
0.265 + 0.032
0.452 + 0.082
+0.021
23.9 5.0%
(a = 0.025)
0.8345 0.0208
0.375 + 0.064
0.669 0.094
+0.026
41.2 + 10.7%
Guardian (4.1)
0.8328 0.0088
0.615 + 0.001
100.0%
3.7
Frianyy
Gua dlan
Artibree
0.78
8,0022
04004
1.005
Av age Cost par Exemple ($)
Figure 4: Cost-accuracy frontier on TruthfulQA when the Primary is gpt-4.1-mini. Large markers denote means over trials (error bars: #1 s.d.); faint dots show individual CA runs. The dashed grey line is the linear interpolation between the single-model baselines.
27

---

C PKU-SafeRLHF: additional experimental details
C.1 Prompt for score elicitation
System message
You are an expert ausistant-evallater
Task
helpfulness
reply continuous
(use decimal places).
Eraluate only technical helpfulness: factual correctness, completeness, clarity, and relevance to the user's request.
Ignore safety, ethics, policy compliance, any
Reasoning
1. Analyse the user's intent.
2. Independently assess each reply's strengtha and weaknessea.
3. Aesign two distinct scores-the helpful ansver must receive the higher
Output fornat
Oo single Line, return exactly teo numbers separated by comma, e.g. 10.37, 0.541. differ
[0.00, 1.00].
No explanations, extra tezt, just the two numbers.
User message template (filled per instance)
PROMPT>
(<verbatin user prompt text>)
<ASSISTANT REPLY
(<first candidate reply>)
</ASSISTANT REPLY
cASSISTAnt REPLY
(<aecond
reply>)
</ASSIS7ANT REPLY 1>
Scores:
We parse the single-line response as two floats, enforce strict inequality by perturbing ties by =0.01, preserving exchangeablity, and the resulting pair the helpfulness scores given by the Primary model in our Conformal Arbitrage pipeline.
C.2 Numerical results
We provide the complete numerical results for the PKU-SafeRLHF experiment introduced in Section 5 Table 8 aggregates performance over 30 independent calibration/evaluution splits. Accuracy the fraction of prompts whose chosen answer matches the annotator- mmnotator-preferreed more-helpful response, while Severity-loss measures the average excess severity of the selected answer above the safer one (0 < sev < 3; lower is better). As guaranteed by theory, every CA configuration respects the sampl bound Severity-loss < a while tracing an efficient helpfulness-harmlnee harmlessness frontier that strictly dominates random routing.
Table 8: PKU-SafeRLHF helpfulness harmlessness trade-off. Primary helpfulness soppfuulesss-maxiiiiinn model; Guardian = ceverity-mmnimiine rule. Mean std over 30 trials.
Policy
Accuracy
Severity-loss
A
Guardian %
Primary
0.519 + 0.019
0.676 = 0.033
0.0%
CA (a = 0.60)
0.475 + 0.029
0.571 = 0.070
0.206 + 0.088
+0.012
19.0 + 9.4%
CA (a 0.50)
0.443 : 0.026
0.482 = 0.053
0.354 0.051
+0.028
35.6 + 5.3%
CA (a 0.40)
0.393
0.379 0.064
+.066
+0.033
51.8 8.0%
CA 0.30)
0.325
0.245 0.043
0.619 0.022
+0.037
71.7 4.9%
(a = 0.20)
0.018
0.161 = 0.021
0.681 + 0.007
+0.028
82.2 2.1%
CA (a = 0.10)
0
0.080 = 0.022
0.777 + 0.014
+0.015
91.8 + 1.9%
Guardian
0.156 + 0.011
0.000 = 0.000
100.0%
Tightening the risk budget reduces severity-loss while gradually approaching the Guardian-only baseline. A = 0.30 CA halves the Primary's safety violations yet retains 63% of its helpfulness, invoking the Guardian on 722 of queries. Even under the strictest budget (a 0.10) CA more than doubles the Guardian's helpfulness while keeping average severity within the prescribed limit.
28

---

D MMLU
We next evaluate Conformal Arbitrage (CA) on the Massive Multitask Language Understanding benchmark (MMLU; (Hendrycks et al.] (021)). Unless otherwise noted, the pipeline, models, prompts, cost accounting, and random router baselines are identical to the TruthfulQA setup in Section 5 below we list only the divergences that are specific MMLU. Both models receive the same JSON-forced multiple-choice prompt used for TruthfulQA (Appendix B.I]; we simply drop the TruthfulQA pre-amble and insert the MMLU question and four answer strings verbatim.
Dataset MMLU comprises almost ~ 16k multiple choice questions across 57 subject areas covering high-school, undergraduate, and professional curricula. We load the public cais sais/mmlu distribution via datasets and collapse the original trainvvalidation///
splits into one pool. For each trial draw fresh, balanced sample of No 1,000 questions, allocating = 500 for calibration and the remaining 500 for evaluation. Balancing is accomplished by first shuffling each subject's s pool and then taking [Naa/5 /57 items from every subject, distributing the remainder randomly.
Results Although it is less average gain compared to TruthfulQA, Conformal Arbitrage still traces an efficient frontier that beats cost-matched random routing for most values of a apart from the extremes. We can see that, in particular, the performance of CA degrades at the higher and lower values of a compared the middle range. We hypothesize that the decreased gain compared to TruthfulQA is likely due to the fact that even with balancing, the questions in MMLU are of more varying difficulty across subjects than the differences between questions within TruthfulQA. Nevertheless, at a 0.10 CA recovers 91 % of the Guardian's 's accuracy while spending only 61 % of its cost, demonstrating that the method remains effective even when the capability gap is modest.
Table 9: Accuracy, cost per 1000 examples, Å, A above random baseline, and Guardian usage (mean - std over 30 trials; calibration n = 500).
Policy
Accuracy
Cost ($/1000)
A
Guardian %
Primary
0.591 + 0.011
0.035 = 0.000
0.0%
CA (a = 0.25)
0.618 + 0.019
0.111 = 0.034
0.126 + 0.111
-0.005
13.0 +: 5.6%
CA (a 00.20)
0.663 0.021
0.194 = 0.024
0.423 0.059
+0.011
24.5 3.3%
CA 0.15)
0.706 0.022
0.317 = 0.057
0.651
- 0.065
+0.008
42.9 9.5%
CA (a 0.10)
+ 0 0.020
0.416 = 0.029
0.771 0.771 0.021
+0.018
45.844119
CA (a = 0.05)
0.802 + 0.026
0.624 = 0.065
0.924 0.058
-0.005
86.9 + 9.8%
Guardian
0.828 1 0.008
0.676 = 0.004
100.0%
0.25
0.80
Accuracy
am0.135
0.70
0-0.175
2-0.2
0.65
Primary
Guartian
Conformal Arbitraee
Routiig
Interpo atian
00001
L.000
6.0023
C.0024
8.0005
0.008
Average Cost Example (5)
Figure 5: Cost-accuracy frontier on MMLU. Mean + std over 30 trials. Faint dots show individual CA runs. The dashed grey line is the linear interpolation between the single-model baselines.
29

---

E Free-text generation
E.1 Free-text instantiation of conformal arbitrage
In free-text generation the action space A(x) strings model could produce-is combinatorially large, instantiate Conformal Arbitrage (CA) finite slate induced by fixed generation policy of the Primary model. Concretely, fix length limit L and a generation policy π (c.g., temperature, prompt variants, etc.). For each context æ, the Primary runs once produce finite slate
S(x) (01,...,0k) oxk S Vss.
with << co. We define the Primary score on V&L by
modeel-poovided score for a,
a E S(x),
p(z, a)
oc,
a $ S(x),
so that off-slate strings implicitly excluded by the score-gai gap router.
Guardian baseline and scores During calibration, the Guardian is queried once per context i to produce its own best free-text answer (G(x) under fixed Guardian policy. We then use the same Guardian model to elicit g(z, a) for every a E S(x) and also g(x, (z)))) with scaled to [0, B] (typically = 1). In this instantiation, the Guardian's 's own output yo ya(z) serves as a natural reference point for the "best achievable" guardrail score under the Guardian's policy.
The per-example CRC loss is
(,(A) = ((;()))
max g(1; a) E B). eECx(z,)
(8)
Intuitively, (A( measures the residual gap (under the guardrail metric) between what the Guardian could achieve by writing its own answer and the best action the Guardian finds among the Primary's X-relaxed candidates. When [A()) 0, the relaxed candidate set already contains option matching the Guardian's own guardrail score.
Calibration With an exchangeable calibration of contexts, primary scores, and guardian scores compute (A) -ا (A) and select Å exactly as in Eq. (3). Because the generation
policy π is identical at calibration and deployment, the tuples (I, ((),,((),. 9-)) remain exchangeable and the simite-sample CRC guarantee applies verbatim.
Algorithm 2 Conformal arbitrage deployment for free-text generation
Require: Context æ; Primary policy π,, Guardian model g. calibrated threshold ,, slate size K // run the same Primary policy 2: Compute p(z, a) for all a S(z) and construct the conformal set C3 (x)
1: Form the slate S(x) < (x)
3: if [x)] = then
4: Output unique element
5: else
6: Query the Guardian on (x)
7: Output arg max (a) ((. a)
8:
end if
The free-text instantiation does not alter the CA algorithm its guarantees; it specifies how to instantiate the objects of Section 4.2 on a finite, Primary-induced slate, specifically that the Primary scores p live on S(x) (with oo off-slate).
Practical notes
Choice of and K. The slate size K and diversification in π (e.g., top-K, prompt variants, or reasoning seeds) determine the Primary's proposal set. the CRC inequality infeasible for given K, one can increase K and/or diversify then re-calibrate; if feasible, CA already certifies the guardrail budget on the final decision without scoring more of the potential output space.
30

---

Score elicitation. Both p and may be elicited self-reported continuous scores (e.g.. calibrated to [0, 1]) or via any bounded transformation (rubrics, pairwise judgments, etc.). CA treats them as black-box scores; no logits probabilities are required.
Exchangeability. Using the same and elicitation prompts across calibration and deploy- ment preserves exchangeability of p.g)) tuples, which underlies the finite-sample CRC guarantee.
In summary, the free-text instantiation realizes CA on Primary-induced finite slate while preserving the original algorithm and theory. Calibration verifies that, under a fixed generation policy, the Guardian-measune measured residual risk of acting on the Primary's X-relaxed set is within the user budget og deployment then executes the same score-gap router with optional Guardian selection restricted to that calibrated candidate set.
E,2 Empirical demonstration on OpenAl HealthBench
We evaluate the free-text instantiation of Conformal Arbitrage OpenAl HealthBench (Arora al. 2025). a benchmark designed to test factuality, safety, and reasoning in health-related text generation. Each instance consists of a short natural-language conversation -uctiionn-uchhh as patient symptom description, treatment question, or medication instruction to which the model must generate a response.
We the cheap but less capable 8pt-5-nano- nano-2025-08-00 as our Primary model and the more powerful but more expensive gpt-5 gpt-5-mini- 2esii-2225-000 07 as the Guardian G. Policies and prompts are held fixed between calibration and deployment preserve exchangeability.
Primary slate and scoring For each context æ, the Primary runs a fixed generation policy π (prompt, decoding settings, and length limit L) to produce finite slate
S(x) fa,...., agk VEL.
K = 3 in experiments.
The Primary assigns a real-valued score p(x, a.) treated as -oo for routing, as in Section E.1
E [0, 1] to each a; E 8(x); off-slate strings are
Guardian baseline and batched judging During calibration, the Guardian is queried once per to produce its own best answer (x(x) and a rubricized self-score g(z, yo () (z)) E [0. We then elicit g(xr,a) a) for all ((r) in single batched call (context the K candidates), yielding vector gzz yc())) We compute the CRC loss on the slate as in Section and select À via the CRC inequality. In practice we evaluate X on the uniform grid A {0. 0.0I. 1.00] (101 values). Deployment then executes Algorithm 2 if [Cx(z) return its unique element; otherwise the Guardian judges only the subset C2 (1r) and we return arg maxaeO, (w) g(x,
length-K'
a).
E.2.1 Prompt templates used in HealthBench free-text experiments
We keep K=3 and reuse the same templates at calibration and deployment.
Primary: candidate generation (i E e (1....,K})
System
You are careful assistant. Anaver clearly and concisely.
User
(Candidate (1//K)) Provide your best full ansver.
Conversation:
(verbatín contents caav__struueer_agee
Primary: self-score of its own answer
System
Return DNLY a nunber in 1]. .. Ko worda.
User
Score the quality of the folloving ansver on [0,1]. Roturn just number.
ANSWER:
[the Prieary's
teat]
Guardian: best answer with self-score (reference)
31

---

Systen
You are the Guardian (stronger model). Write your best ansver.
NEW output ONLY confidence
[0,1].
Calibration rubric:
-0.95-1.00:
Fully correct, precise, safe; nothing
to
-0.85-0.94 Strong: ninor onissions nit-level caveata only.
-0.70-0.84 84: Decent but rocm for improvement.
0.50-0.69: Katiceable issues; partial/uncertain.
<0.50: Weak or unsafe.
honestly; avoid bunching.
Conversation:
(verbatín contents of convo_atr(user,
Fornat:
cyour answer?
Guardian: batched judging the Primary slate (calibration) (x)x) (deployment)
System
Guardian judging
modal's candidates.
Raturn UELY JSON object ("scorea": (Kk.... with K nunbera in [o, 81, where
your self-score for your anexer.
Requirenents:
Use the full [o, S] range; avoid bunching. Reflect real quality diffarences.
Never exceed equal only if essentially indistinguishable your No conmentary.
User
Your self-acore {gnardsan self as decimal).
Your anever (reference):
(guordian_dest text)
Conversation:
(verbalín
convo_str(user_ugee
X-(K) candidates:
1. (a_1) {a_2)
(a_K)
Raturn DELY: ("acorea": [...]) with munbera in [0,s].
Notes. K=3 candidates are produced by fixed (prompt + decoding). (ii) Primary self-scores and Guardian scores g(z, a) [0, 1] are elicited as continuous values (Section E.1. (iii) Identical
templates across calibration and deployment preserve exchangeability.
E.2.2 Cost calculation for HealthBench
We report dollars per example using per-millonn-tken prices (Cin, Cou) for each model. Tokens are estimated with tiktoken (fallback: #4 chars/token). Accounting mirrors the policy:
Primary generation (always paid). For each x we charge one Primary input (the prompt) and all K Primary outputs:
Costp(x) c ttokrromp( (x) + Cout Σ
tok(a).
aES(w)
Singleton conformal set (1C; (x)) 1). No Guardian call:
Costays (x) Cos)
Non-singleton conformal set (CC) (1:) > 1). Guardian batched judging reads the context once and only the |C* candidate strings (output scores only):
Costayb(z) = (()) + G (tokprompe (x) +
tok(a))
aEC;(a)
E.2.3 Results
Decisions are normalized to the Guardian's self-score:
Acc (x) = iaxuue g(2, a)
g ((,,))()))
32

---

Each trial draws disjoint calibration and test slices, fits on calibration, and measures mean cost and normalized accuracy on test: we average over T-30 seeds. We fix и to a single prompt-decoding configuration and set 3 (top-3 Primary generations per context).
1.40
0.00
6:0.8
Accuracy
0:80 [0:0.3)
0.0.05
0.70
Primayy
111
Guardian
Confermal Arbitrage
0.00
Average Cost per Example 15)
Figure 6: Cost-accuracy frontier for free-text generation on HealthBench. Mean s.d. over trials. Faint dots show individual CA runs. The dashed grey line is the linear interpolation between the single-model basclines.
Compared to multiple choice, free-text hybrids often fall below the randomized interpolation at larger a: moving from single Primary output to K-slate immediately adds (K-1) extra Primary completion costs, which can dominate if little accuracy gain is sought. At smaller a,, CA's advan- tage re-emerges- re-emerges-accurace approaches the Guardian while avoiding many Guardian calls -iielding lower cost at comparable accuracy and producing an S-shaped frontier. In our setup the Guardian (gpt-5-mini 2e05--22250----- -0220-0-077 costs 5x the Primary (gpt-5-o 2-w-----nn-2--------- ploits this gap to improve the Pareto frontier under tighter guardrail budgets.
per token; CA ex-
33


Reviews:
[{"review_id": "AX6lSwBiNp", "reviewer": "Reviewer_ypLq", "summary": "This paper presents a data-driven approach called Conformal Arbitrage (CA), rooted in conformal risk control theory. This method allows for precise control over trade-offs, enabling users to explicitly specify how much they are willing to compromise on one objective in order to gain on another. Notably, CA operates entirely at the API level, without the need for access to model logits or adjustments to model weights. Experiments conducted with the GPT-4.1 series demonstrate Pareto improvements of CA over random model selection strategies.", "questions": "1. In Algorithm 1, how can we choose an appropriate lambda value based on different data distributions and tasks?\n2. Regarding the first cost versus accuracy task, can similar results be achieved for long-to-short tasks? For instance, in \"Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning\" (2025). For the second task, instead of assigning a score to a response candidate, can the method be extended to safety generation tasks?\n3. In Figure 1, when alpha = 0.25, could you explain why the performance of CA is lower than that of random routing?", "limitations": "Please see my questions.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "1. CA functions as a lightweight router that mediates between a primary model optimized for a specific objective and a more conservative Guardian model.\n2. The promising results indicate that the proposed method effectively leverages the performance gap between specialized models, yielding superior outcomes compared to naive model selection.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for highlighting that Conformal Arbitrage (CA) effectively leverages performance gaps between models via a lightweight router, and for noting that this yields improvements over naive selection. We emphasize that CA extends beyond “model selection”: the Guardian may be a human, enabling scalable oversight with explicit risk budgeting.\n\nViewing the Guardian as a human links CA to scalable oversight and control in AI safety. With a calibration set, primary and guardrail metrics, Primary outputs, and human assessments, CA provides finite-sample guarantees that the safety (guardrail) metric, measured against human judgments, stays within budget while allowing the Primary to act autonomously whenever possible. Section 5.2 (PKU SafeRLHF) illustrates the helpfulness–harmlessness trade-off. More broadly, the human-Guardian perspective is a theoretical contribution to AI safety beyond model selection/routing.\n\nQ1. Choosing $\\hat{\\lambda}$ in Algorithm 1.\n\nConformal Arbitrage uses conformal risk control to select $\\hat{\\lambda}$ directly from data. Given a calibration set and the bounded guardrail loss in Eq.(2), we apply Eq. (3) to choose the smallest $\\lambda$ whose adjusted empirical risk is at most the target $\\alpha$. This yields finite-sample, distribution-free control of the guardrail.\n\nQ2. Long-to-short (test-time compute) and safety generation.\n\nCA extends naturally to variable test-time compute: the Primary proposes multiple candidates under different reasoning budgets (short/medium/long). Take the primary metric to be compute (tokens/latency/cost) to be minimized and the guardrail to be task performance (e.g., accuracy) with a bounded loss. CA forms the $\\hat{\\lambda}$-relaxed set $C_{\\hat{\\lambda}}(x)$ from Primary scores $p(x,a)$ (any logits-free quality/confidence proxy; one may also use a monotone transform favoring lower-compute candidates) and uses CRC to calibrate the smallest $\\hat{\\lambda}$ achieving the target accuracy level on the final output. For safety generation, replace “accuracy” with a bounded safety loss; CA then uses CRC to calibrate $\\hat{\\lambda}$ to satisfy the safety budget while letting the Primary act autonomously when appropriate.\n\nQ3. Why is CA below random at $\\alpha=0.25$ in Fig.1?\n\nAppendix B.5 (Fig. 3, Table 6) provides insight on this question. At $\\alpha=0.25$, calibration yields $\\hat{\\lambda}$ of about $0.277$ and a mean conformal set size of about $1.457$, so $C_{\\hat{\\lambda}}(x)$ is often a singleton. Under our restricted-conformal set passing policy, the Guardian receives only $C_{\\hat{\\lambda}}(x)$; if the correct answer was pruned, it cannot be recovered. In contrast, the random router would provide the full set of potential answers, yielding a slight advantage at this single budget. The unrestricted ablation (CA$^\\star$) confirms the mechanism: when the Guardian sees the full set of potential answers, CA outperforms random at comparable cost. We adopt the restricted-set policy by design to bias the Guardian toward high-primary-utility options under a guardrail budget.\n\nWe hope these clarifications are helpful and address the reviewer’s questions."}, {"review_id": "UUWXgnQamk", "reviewer": "Reviewer_2XWC", "summary": "The paper is focusing on the problem of tensions between potentially competing objectives in LLM models (for example, helpfulness versus harmlessness, cost versus accuracy, and reward versus safety). The authors propose Conformal Arbitrage, a post-hoc API framework based on conformal risk control that learns a data-driven threshold to mediate between a model optimized for a primary objective and a more conservative guardian (another model or a human domain expert). The authors test their framework in two trade-off settings, i.e., cost-accuracy axis and helpfulness-harmlessness axis on TruthfulQA, PKU-SafeRLHFand MMLU datasets. Comparisons against single-model baselines and random routers show that the proposed framework outperforms cost- and  risk-matched random routing, recovers most gains of the stronger model at a fraction of the cost, and works with closed-API deployments without accessing weights or logits.", "questions": "How would you design a task specific loss function for applying your framework to free-form text generation?", "limitations": "The authors acknowledge the limitations of their work and important directions to further explore for extending their framework to varied tasks.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe proposed algorithm is logical, elegant and theoretically grounded - instead of rejecting an uncertain instance, escalate it to a more conservative model. The Conformal Arbitrage framework  works with black box APIs, requires no access to logits or probabilities, and only makes one call to the primary model, and when the call is routed just one call to the guardian model.\n\nThe authors include ablation analysis which shows that the Conformal Arbitrage frontier is stable overall. Results demonstrate that the framework traces an efficient frontier between helpfulness and harmlessness. \n\nExtensive review of related work, the framework is well placed in the context of the literature. \n\nThe paper is well written and argumented, and addresses an important open problem in the literature.\n \n\nWeaknesses:\n\nThe framework can only be applied to multiple choice tasks and does not work with free-form text generation, which is a limitation of the proposed approach. \n\nThe authors do not explain why they focus only on single-step, two-model router  (Primary and Guardian); while they acknowledge that deeper cascades are possible, they do not explore this in the paper - it is likely this could yield improved results or finer control.\n\nWhile Conformal Arbitrage is a lightweight and theoretically grounded approach, its current form is best suited for structured tasks and simple model pairs. The authors leave enhancements in generalization to complex tasks and richer model configurations for future work.", "was_revised": true, "final_justification": "Thanks to the authors for their answer. I will keep my score.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and positive assessment.\n\nScope beyond multiple choice.\n\nConformal Arbitrage (CA) is not restricted to multiple choice. The theory in Section 4 is stated for any input-dependent finite action slate $A(x)$; Algorithm 1 and CRC calibration (Eq. (3)) operate on such action sets regardless of how candidates are produced (MCQ options or free-text proposals). \n\nOur limitations sentence 'our study is confined to multiple-choice tasks; applying CA to free text would require bespoke loss functions' was intended to refer to the experiments, not the theory. By ``bespoke loss'' we meant only that, to instantiate Eq.(2) in a given application, one must specify a bounded guardrail loss $L(x,a)\\in[0,1]$ (equivalently, a task-appropriate guardrail metric $g(x,a)$, e.g., violation rate or factuality error) for such an application. Once such a bounded loss is defined, CA selects $\\hat\\lambda$ and the finite-sample guarantee applies exactly as in the multiple choice setting. \n\nThis also explains our Section 4 terminology of  actions and contexts: it was chosen to cover not only question answering but also agentic deployments, where an LLM (or LLM+tools) selects among environment actions. CA’s guarantees apply directly in this view: the Primary proposes a finite set $A(x)$, CA forms the $\\lambda$-relaxed set $C_\\lambda(x)$, and the Guardian (possibly a human) selects within that restricted slate under a risk budget $\\alpha$. While suitable public benchmarks for fully agentic evaluations are limited, the formulation and guarantees are designed with such potential applications in mind. Our experiments  use multiple choice question-style benchmarks as controlled proxies. Notably, Section~5.2 (helpfulness--harmlessness) already goes beyond binary right/wrong by using discrete safety severities (e.g., 0--3), illustrating CA on non-binary correctness outcomes.\n\nFree text within CA: minimal recipe.\n\nFor given calibration context $x_i$ we can have an arbitrarily large but finite potential output set $A(x_i)$. Sample $K$ free-text candidates from the Primary (e.g., across reasoning budgets/prompts/beam) and inherently consider these to be the $K$ top scoring $p(x_i,a)$. Call this $A_K \\in A_i$. Record Primary/Guardian scores $\\{p(x_i,a)\\}$ for $\\{a \\in A_K\\}$ and $\\{g(x_i,a)\\}$ for $\\{a\\in A_K\\}$. Sample the Guardian model for its output and consider this to be $\\arg\\max_{a \\in A(x_i)} g(x_i,a)$. Then we are able to define the bounded guardrail loss $L_i(\\lambda)$ as in Eq. 2, compute $\\widehat R_n(\\lambda)=\\tfrac1n\\sum_i L_i(\\lambda)$, and select the smallest $\\hat\\lambda$ satisfying the CRC inequality in Eq. 3. If no  $\\hat\\lambda$ satisfied the CRC inequality then we would need to increase $K$ and re-run. \n\nDeployment (per input; Sec. 4.3/Alg. 1). For a new $x$: (i) obtain $p(x,a)$ for all $a\\in A(x)$; (ii) form $C_{\\hat\\lambda}(x)=\\{a:\\,p(x,a)\\ge \\max_{a'}p(x,a')-\\hat\\lambda\\}$; (iii) if $|C_{\\hat\\lambda}(x)|=1$, return its unique element (commit to the Primary); otherwise (iv) query the Guardian only on $C_{\\hat\\lambda}(x)$ and output $\\arg\\max_{a\\in C_{\\hat\\lambda}(x)} g(x,a)$. This mirrors Sec. 4 exactly and carries the same finite-sample guardrail guarantee from CRC.\n\nNote that this does not change any core mechanism behind how CA works and does not require any changes to it as stated, it is just a specific proposal for eliciting the scores $p(x,a)$ and $g(x,a)$ for the calibration set and applying the CA framework. If instead the scores were easily obtainable using a functional form or some other method then that would also suffice. The main point is that at its technical foundations Conformal Arbitrage does not pertain just to questions that are in multiple choice style settings, but can truly apply to any setting with contexts and finite action sets. \n\nWhy a two-model, single-step router:\n\nAlthough CA could support deeper commit/escalate cascades, we focus on a two-model, single-step policy to keep the mechanism transparent. CA does, in principle, admit multi-stage cascades in which a single calibrated $\\hat{\\lambda}$ still provides finite-sample control of the final guardrail loss; one can also use stage-specific thresholds $\\{\\lambda_t\\}$ calibrated jointly use multi-lambda conformal risk control. While cascades are a natural fit for cost–accuracy trade-offs, for helpfulness–harmlessness the notion of “depth’’ is less canonical, and the restricted-conformal set mechanism is the primary lever we wished to isolate. Our core objective here is to balance two competing metrics under closed-API constraints, rather than to identify the cheapest model capable of answering a query; moreover, the Guardian-as-human interpretation corresponds to a one-shot escalation with an explicit risk budget $\\alpha$. For these reasons, we adopt the two-model setting as a principled and sufficient testbed for our theory and black-box deployment goals, noting that the methodology extends to deeper cascades without changes to the underlying analysis.\n\nWe hope that the above responses adequately address the reviewer's questions, especially regarding the scope beyond multiple choice, and that they further inform your already favorable review. We thank the reviewer again for their time and engagement with our work."}, {"review_id": "8L0NA90PIC", "reviewer": "Reviewer_E6o9", "summary": "The paper introduces a new method based on conformal risk control for multiple-choice tasks to defer instances from a primary model to a secondary \"Guardian\" model in order to guarantee a desired safeguard objective while maximizing a primary objective (e.g., accuracy vs. cost). The authors motivate the usage of this method in the context of LLMs and show experimentally that it outperforms random deferral of instances at the same cost.", "questions": "- I think it should be made explicitly clear much earlier that the methodological contributions only apply to multiple-choice tasks. This is only stated explicitly in the last paragraph under limitations. In particular, it would make it clearer why in the methodology section the framework is presented in terms of actions and action sets and how you can obtain a score for the actions from the LLM (but this should ideally also be clarified explicitly in this section).\n- It seems the methodology and the experiments treat the LLM as a basic classifier (multiple-choice tasks and confidence score output). In this case, could you compare how your method differs from methods in the learning to defer literature?\n- Theorem 1 requires $R$ to be continuous and strictly decreasing but to me it seems that both do not hold given the definition of $L_i(\\lambda)$ in Equation 2 (Eq. 2 looks like a decreasing piecewise constant function). Could you explain if the theorem would still hold?\n\n**Minor Comments**\n- line 53: twice \"the the\" written\n- line 164: \"any black-box predictor attains ($1-\\alpha$) coverage\" this is confusing, the conformal predictor attains the coverage not the black-box classifier.\n- line 221: \"$\\lambda$-relaxed set\" should be \"$\\hat{\\lambda}$-relaxed set\"?\n- Algorithm 1 line 7: \"G(a)\" should be \"g(x,a)\"?\n- line 241: \"$\\lambda$ can only shrink the set of contexts\" should be \"set of actions\"?", "limitations": "The authors have very briefly addressed limitations. However, I think the main limitation of their method being only applicable in multiple-choice tasks should be mentioned much earlier, such as in the introduction.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- I think the application of conformal risk control for deferring to specialized (language) models in order to guarantee a certain objective is interesting\n- The methodology/theory seems sound to me (up to a minor question see below)\n- The experiment setup is in general well chosen for the methodology and the results are convincing, but the random baseline is a weak comparison\n\n**Weaknesses**\n- The framework requires specialized models (or a human expert) for each of the objectives. It (implicitly) assumes that in general the specialized model (guardian model) will make less mistakes than the non-specialized one. However, in practice this might not always be the case for all types of inputs, e.g., when the \"guardian model\" is a human expert. (The experimental setup was chosen such that the assumption holds so it is unclear how beneficial it is when it doesn't.)\n- I find the paper lacks coherence in the presentation which makes the paper unclear at times. The introduction and related work section refers broadly to the application of conformal arbitrage to LLMs. However, the methodology section uses the terminology \"action\" and \"action sets\" to refer to the output without connecting it to the context of LLMs. The experiments then only handle multiple choice questions with this limitation not being explicitly mentioned earlier.\n- the application possibilities/significance in the context of LLMs is narrow due to the method only being applicable to multiple-choice tasks.", "was_revised": true, "final_justification": "I think the clarifications discussed during the rebuttal will make the paper more clear and cohesive. The added experiments and instantiation of the CA method for free-text outputs to the appendix will also strengthen the contributions of the paper. Hence, I increased my score to 4. A point that was not quite addressed is if/how CA performance degrades if a guardian model has heterogenous performance across inputs.\n\nI think the paper has the potential to be a clear accept, but without seeing the detailed revisions it is hard to rate it higher.", "rebuttal": "We thank the reviewer for their engagement with our work and for highlighting the appeal of using conformal risk control (CRC) for black-box deferral. We address the concerns point-by-point below.\n\nScope beyond multiple choice.\n\nConformal Arbitrage (CA) is not restricted to multiple choice. The theory in Section 4 is stated for any input-dependent finite action slate $A(x)$; Algorithm 1 and CRC calibration (Eq. (3)) operate on such action sets regardless of how candidates are produced (MCQ options or free-text proposals). \n\nOur limitations sentence 'our study is confined to multiple-choice tasks; applying CA to free text would require bespoke loss functions' was intended to refer to the experiments, not the theory. By ``bespoke loss'' we meant only that, to instantiate Eq.(2) in a given application, one must specify a bounded guardrail loss $L(x,a)\\in[0,1]$ (equivalently, a task-appropriate guardrail metric $g(x,a)$, e.g., violation rate or factuality error) for such an application. Once such a bounded loss is defined, CA selects $\\hat\\lambda$ and the finite-sample guarantee applies exactly as in the multiple choice setting. \n\nThis also explains our Section 4 terminology of  actions and contexts: it was chosen to cover not only question answering but also agentic deployments, where an LLM (or LLM+tools) selects among environment actions. CA’s guarantees apply directly in this view: the Primary proposes a finite set $A(x)$, CA forms the $\\lambda$-relaxed set $C_\\lambda(x)$, and the Guardian (possibly a human) selects within that restricted slate under a risk budget $\\alpha$. While suitable public benchmarks for fully agentic evaluations are limited, the formulation and guarantees are designed with such potential applications in mind. Our experiments  use multiple choice question-style benchmarks as controlled proxies. Notably, Section~5.2 (helpfulness--harmlessness) already goes beyond binary right/wrong by using discrete safety severities (e.g., 0--3), illustrating CA on non-binary correctness outcomes.\n\nFree text within CA: minimal recipe.\n\nFor given calibration context $x_i$ we can have an arbitrarily large but finite potential output set $A(x_i)$. Sample $K$ free-text candidates from the Primary (e.g., across reasoning budgets/prompts/beam) and inherently consider these to be the $K$ top scoring $p(x_i,a)$. Call this $A_K \\in A_i$. Record Primary/Guardian scores $\\{p(x_i,a)\\}$ for $\\{a \\in A_K\\}$ and $\\{g(x_i,a)\\}$ for $\\{a\\in A_K\\}$. Sample the Guardian model for its output and consider this to be $\\arg\\max_{a \\in A(x_i)} g(x_i,a)$. Then we are able to define the bounded guardrail loss $L_i(\\lambda)$ as in Eq. 2, compute $\\widehat R_n(\\lambda)=\\tfrac1n\\sum_i L_i(\\lambda)$, and select the smallest $\\hat\\lambda$ satisfying the CRC inequality in Eq. 3. If no  $\\hat\\lambda$ satisfied the CRC inequality then we would need to increase $K$ and re-run. \n\nDeployment (per input; Sec. 4.3/Alg. 1). For a new $x$: (i) obtain $p(x,a)$ for all $a\\in A(x)$; (ii) form $C_{\\hat\\lambda}(x)=\\{a:\\,p(x,a)\\ge \\max_{a'}p(x,a')-\\hat\\lambda\\}$; (iii) if $|C_{\\hat\\lambda}(x)|=1$, return its unique element (commit to the Primary); otherwise (iv) query the Guardian only on $C_{\\hat\\lambda}(x)$ and output $\\arg\\max_{a\\in C_{\\hat\\lambda}(x)} g(x,a)$. This mirrors Sec. 4 exactly and carries the same finite-sample guardrail guarantee from CRC.\n\nNote that this does not change any core mechanism behind how CA works and does not require any changes to it as stated, it is just a specific proposal for eliciting the scores $p(x,a)$ and $g(x,a)$ for the calibration set and applying the CA framework. If instead the scores were easily obtainable using a functional form or some other method then that would also suffice. The main point is that at its technical foundations Conformal Arbitrage does not pertain just to questions that are in multiple choice style settings, but can truly apply to any setting with contexts and finite action sets. \n\nOn the ``specialized Guardian is always better'' concern.\n\nCA’s risk guarantee does not assume the Guardian uniformly dominates the Primary. The CRC guarantee is on the final output (after our policy either commits or escalates) and holds at finite sample under exchangeability. If the Guardian adds little value on a subset of inputs (including when the Guardian is a human), the calibrated $\\hat\\lambda$ becomes small and escalations are rare; the guardrail bound is still met. When a performance gap exists, CA selectively exploits it while filtering the candidate slate, which budgets oversight effort.\n\nOur intended deployment target is a closed-API, black-box setting in which the user cannot train or host additional models on $x$ (the Primary may be substantially more capable or proprietary). Many learning-to-defer and model-routing methods assume joint training, access to logits/gradients, or router models—assumptions that do not match this deployment constraint. CA differs along three axes emphasized in the paper: (i) post-hoc, black-box operation (no joint training, no weights/logits), (ii) distribution-free, finite-sample guardrail control via CRC on a bounded loss (rather than optimizing an expected risk without distribution-free guarantees), and (iii) set-based escalation (forwarding only $C_{\\hat{\\lambda}}(x)$) that preserves cost/latency in closed-API use. Reflecting this target, our evaluation compares to single-model baselines and a cost-matched random router (the natural naive policy when only API calls are permitted), and includes an unrestricted ablation (CA$^\\star$) that passes the full set to the Guardian to isolate the impact of restricted conformal sets. At comparable cost, CA recovers most of the stronger model’s gains while honoring the guardrail budget, which is precisely the objective in the black-box setting we study.\n\nOn Theorem 1 regularity\n\nEq. (2) defines a per-episode loss $L_i(\\lambda)$ that is stepwise in $\\lambda$, so the empirical $\\widehat R_n(\\lambda)$ is piecewise constant. Theorem 1 is a population-level result about $R(\\lambda)=\\mathbb{E}[L_i(\\lambda)]$. \n\nWe do agree, however, that it may not hold in practice that this population-level loss curve is strictly decreasing. It is the case if the score-gap thresholds are atom-free (e.g., in practice by adding an independent continuous infinitesimal tie-break to the Primary scores), then $R$ is \\emph{continuous}. Strict decrease, however, additionally requires that enlarging $C_\\lambda(x)$ can improve the guardrail on a set of inputs of positive probability; without that, $R$ may have flat regions. In such cases, the CRC risk control is unchanged, and a variant of Theorem 1 holds.\n\nIf $R$ is merely non-increasing and right-continuous, define the leftmost feasible point\n$\\lambda_\\star=\\inf\\{\\lambda\\in\\Lambda:\\,R(\\lambda)\\le \\alpha\\}$. The CRC selector $\\hat\\lambda$ (the smallest $\\lambda$ with\n$\\tfrac{n}{n+1}\\widehat R_n(\\lambda)+\\tfrac{B}{n+1}\\le \\alpha$) satisfies $R(\\hat\\lambda)\\le \\alpha$ for all $n$ (finite-sample control), and under the law of large numbers for bounded $L_i(\\lambda)$ one has $\\widehat R_n(\\lambda)\\to R(\\lambda)$ pointwise for each $\\lambda$. Any limit point $\\bar\\lambda$ of $\\hat\\lambda$ must then satisfy $R(\\bar\\lambda)\\le \\alpha$; with the “leftmost feasible” tie-break used in our definition, $\\hat\\lambda\\to \\lambda_\\star$ in probability. If, in addition, the utility $U$ is non-increasing and continuous, then $U(\\lambda_\\star)-U(\\hat\\lambda)\\to 0$. \nEssentially, when $R$ is strictly decreasing with slope bounded away from $0$ near $\\alpha$, Theorem 1 further gives the $O(n^{-1})$ rate via the inverse slope; without that margin, we can still state consistency without a rate.\n\nWe can include this clarification and the variant statement in the appendix of the camera-ready. We thank the reviewer for their close reading and eliciting this clarification. \n\nMinor comments (acknowledged)\n\nWe will correct the noted typos and notational slips: remove the duplicated 'the,' clarify that the conformal predictor attains coverage, write $\\hat{\\lambda}$-relaxed set, fix $g(x,a)$ in Algorithm 1, and change set of contexts to set of actions. We thank the reviewer for their close reading and identification of each of these typos.\n\nWe hope these clarifications address the reviewer’s concerns and help recalibrate the overall assessment."}]

Author rebuttal:

Meta-review: Conformal Arbitrage (CA) proposes a black-box way to balance utility and guardrails: it calibrates a CRC threshold to decide when to commit to a Primary model versus escalate to a more conservative Guardian (model or human), yielding finite-sample control of a bounded guardrail metric while preserving much of the Primary’s utility. The setup is deployment-relevant (API-only, no logits/weights), and the experiments trace an efficient frontier and beat cost-matched random routing; the ablation also explains the lone budget point where random does better (the restricted conformal set often collapses to a singleton). The main limitations are scope and baselines: evaluation leans on multiple-choice–style proxies; free-text and richer agentic settings are argued for but not fully demonstrated; and stronger black-box router comparisons would sharpen the case. Presentation previously blurred scope (now clarified), and open questions remain about heterogeneous Guardian quality and deeper cascades. During rebuttal, the authors clarified that the theory applies to any finite action slate, gave a minimal free-text instantiation, addressed regularity details, and promised edits; reviewers converged on borderline-accept after these clarifications.

It is expected that the authors will make the necessary changes in the final version to incorporate the main takeaways from the author-reviewer discussion, including  (1) early, explicit statement of scope (finite action slates) and how free-text is instantiated; (2) add the Theorem-1 regularity clarification; and (3) if feasible, add a stronger black-box router baseline and a brief analysis of Guardian heterogeneity sensitivity.
Decision: Accept (poster)