Routing, Cascades, and User Choice for LLMs
Authors: Rafid Mahmood
Venue: ICLR 2026 Poster (2026)
Keywords: LLM routing; human-AI interaction; game theory
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We develop a game theoretic model between LLM providers and users to study when model routing is beneficial or harmful for users

Abstract: To mitigate the trade-offs between performance and costs, LLM providers route user tasks to different models based on task difficulty and latency. We study the effect of LLM routing with respect to user behavior. We propose a game between an LLM provider with two models (standard and reasoning) and a user who can re-prompt or abandon tasks if the routed model cannot solve them. The user's goal is to maximize their utility minus the delay from using the model, while the provider minimizes the cost of servicing the user. We solve this Stackelberg game by fully characterizing the user best response and simplifying the provider problem. We observe that in nearly all cases, the optimal routing policy involves a static policy with no cascading that depends on the expected utility of the models to the user.
Furthermore, we reveal a misalignment gap between the provider-optimal and user-preferred routes when the user's and provider's rankings of the models with respect to utility and cost differ. Finally, we demonstrate conditions for extreme misalignment where providers are incentivized to throttle the latency of the models to minimize their costs, consequently depressing user utility. The results yield simple threshold rules for single-provider, single-user interactions and clarify when routing, cascading, and throttling help or harm.

Published as a conference paper at ICLR 2026
ROUTING, CASCADES, AND USER CHOICE FOR LLMs
Rafid Mahmood
University of Ottawa & NVIDIA
fahnood@telleer. www.awottt...................... ca
ABSTRACT
To mitigate the trade-offs between performance and costs, LLM providers route user tasks to different models based on task difficulty and lateney. We study the effect of LLM routing with respect to user behavior. We propose game between an LLM provider with two models (standard and reasoning) and a user who can re-prompt abandon tasks if the routed model cannot solve them. The user's s goal is to maximize their utility minus the delay from using the model, while the provider minimizes the cost of servicing the user. We solve this Stackelberg game by fully characterizing the user best response and simplifying the provider problem. We observe that in nearly all cases, the optimal routing policy involves static policy with no cascading that depends on the expected utility of the models the user. Furthermore, we reveal misalignment gap between the provider- optimal and user beer-preferred routes when the user's 's and provider's 's rankings of the models with respect to utility and cost differ. Finally, we demonstrate conditions for extreme misalignment where providers are incentivized to throttle the latency of the models to minimize their costs, consequently depressing user utility. The results yield simple threshold rules for single-provider, single-user interactions and clarify when routing, cascading, and throttling help or harm.
INTRODUCTION
Large language models (LLMs) have exploded into general-purpose technologies developed by small set of model providers that permit users to perform diverse applications via natural language interface (Maslej al. 2025). Often users subscribe to a provider, who affords users access to a family of models differentiated by quality and latency" [T] Providers face inference costs for each that vary depending the model and hardware (Leviathan al. 2023). This presents tiered ecosystem with a quality-latecy--ost trade-off.
Providers navigate these trade-offs via LLM routing and cascading (Chen al., 2023 Hu et al., 2024). Routing involves selecting an appropriate model per-use by reserving difficult tasks for larger, more expensive models and easier tasks for smaller, cheaper models. Cascading routes tasks over multiple rounds to smaller models first and escalating to larger ones if necessary. Routing has become industry practice, e.g., GPT-5 routes tasks between smart, efficient model that answers most questions, [and] deeper reasoning model for harder problems' (OpenAl 2025).
While routing literature includes algorithms estimate LLM performance and optimize the quality- latency-cor trade-off (Ding et al. 2024; Ong et al.] 2025 Dekoninck et al. (025). they often treat user response as exogenous. However, the prompt-based interface of L.MMs implies a recurring cost for model failures if users repeatedly interact with the system (Naor) 1969 Garnett et al., 2002). Depending on the task, a model fails to complete it, users may be patient and prompt the model again, give up the task (Castro et al. 2023 Wester et al. 2024 Krishna et al. 2024 Fu al.) 2025). In the worst case, user dissatisfaction may lead to un-subscription from the provider service. Consequently, optimizing for single-past pass costs may be penny-wise but welfare-foolish.
In this paper, we study the welfare gap between a single LLM provider with two models (standard and reasoning) and user under multi-round LLM routing game (Simaan & Cruz Jr 1973). Given
For list of LLM subscriptions, see:
ai
/
cooparri nininng111 11m- subscripsss... www.sriptioonpppipipipppppp ans

---

Published as a conference paper at ICLR 2026
Only the reasoning model has high accuracy for its latency
Both models have high accuracies for their latencies
Users will abandon tasies it given to the standard model and not cascaded to the ressoning model.
Users are always patient and continue using regardlass the modal choice.
Depending en the cost-of- oaa--assaaaa of each model and risk unsubscrtbing, providers either route to the standarc model, cascade at minimum rate, or route directly to the reasoning model.
Providers should route to the model lower cost of
Both models have low accuracy for their latencies
Only the standard model has high accuracy for its latency
Users are impatient & sbandon tasks ragardless model choice.
Users will sbandon tasks if given to the reasoning model but will stay il given the standard model.
Providers should route to the cheaper model unsubacribing is low, otherwise route to the reasoning model.
Providers should optimize over the user best resporse curve.
Key guidelines for optimal routing in the face of reactive users. User behavior depends on the state of the two models in terms of providing utility versus the delay incurred from inference compute. Provider policies must follow different thresholding rules in each region.
Figure I:
a task, the provider determines an initial model and a cascade rule on whether to escalate the task the standard model fails. For each pass, the provider pays compute cost and the user pays latency cost (i.c., delay from model response). If the model fails to complete the task, the user may either re-prompt churn the system, which imposes potential expected lost future revenue to the provider.
We prove the Stackelberg equilibrium, by first characterizing the user best response of when to churn versus re- re-prompt. We then show that the provider's 's problem reduces to single-variabe optimiza- tion problem that in many regimes yields simple threshold- -bsed routing policies. Importantly, we show that cascades rarely optimal outside specific regimes where the two models sufficiently differentiated in value minus latency. Finally, we study the operational implications of simple rout- ing heuristics, provider-user misalignment, and the effect of throttling latency. Figure F summarizes the key insights of expected user behavior and consequent routing policies. We observe:
User patience and sensitivity depend on the model net values. When both models deliver positive (respectively, negative) value, users stay (respectively, churn) regardless routing. When the models differ, users churn stay depending the cascade likelihood.
Optimal routing often collapses to simple threshold rules. When models deliver similar value, the optimal policy is to route with no cascade to one of the models. In the mixed cases, some regimes reveal routing to the standard model and then cascading optimal.
Users receive sub-optimal utility when the model quality and provider costs rank dif- ferently. Misalignment arises when the models deliver different net value, when churn penalties are low, when cost-per-success rankings conflict with user utility rankings.
Low churn penalties can incentivize providers to throttle latency. If the likelihood of users unsubscribing is low, then providers can lower their costs by discouraging repeated use and increasing latency, thereby deteriorating user utility.
2 RELATED LITERATURE
Model routing and cascading. LLM routing and cascades aim to allocate compute across heteroge- neous models to optimize accuracy, latency, and cost for each query (Chen et al. 2023 Ding et al.]
2024
Yue et al. 2023 Hu et al. 2024 [Deonninck al. 2025 Ding et al. (024) reduce calls to
2

---

Published as a conference paper at ICLR 2026
expensive models by predicting the difficulty of tasks and allocating easier tasks to cheaper mod- els. Cascades extend the routing approach to escalate tasks to stronger models only when cheaper models disagree (Chen et al. 2023 Yue et al., 2023). Dekoninck et al. (2025) propose unified op- timization algorithm for routing and eascading to show optimal routing strategies when the overall quality of model output with respect to task be estimated. This problem has grown recently to include benchmarks for tracking the performance of routers under cost and task completion (Hu ct al. 2024). Our paper builds on the behavioral aspect of routing to study when cost-optimal routing acts in user's 's best interest. Furthermore, we model the problem by treating the Al interaction via a mulli--ound prompting game (Castro al. [2033] Mahmood 2024). which naturally incorporates the single-round mechanism as a special case.
Operations and costs of LLM systems. The deployment of L.LM systems must factor for user engagement with respect to performance, costs, and latencies (Bergemann et al. 2025 Zhang et al.) 2025). For example, prompt-based interfaces permit users to interact with models via multiple rounds until task is completed (Bai al. 2024). Consequently, the cost of model, is evaluated via a cost-of-pass, which tracks the inference cost of a single pass of a model over the probability of the model completing the task in that pass (Mahmood) 2024 Erol et al. 2025). The number of passes in which a user interacts is also highly dependent on model quality with respect to different metrics (Castro al., 2023). Relatedly, [Shirali] (2025) study interactive alignment as Stackelberg game and show how explicit user-side costly signals can reduce the user burden of being understood. Finally, our work ties into the broader literature on game theory for LLM use (Sun et al. (2025).
MAIN PROBLEM
In this section, we formulate a Markov model of the Stackelberg game between an LLM provider routing tasks and user deciding whether to continue using the LLM. We derive closed-form ex- pressions for expected latencies, utilities, and costs. We motivate with the following example.
Motivating scenario. Consider a user of an LLM subscription who is using the LLM to help prove a theorem. The user submits an initial prompt describing the theorem statement and the proof approach they have in mind, and the provider routes the request either a standard model slower reasoning model. After seeing the output, the user either accepts it and stops, or continues the session with follow-up prompt (e.g.. "justify the inequality in line 3") if the proof is incomplete or incorrect. The provider routes the follow-up prompt again, potentially escalating to the reasoning model. If this repeats for multiple prompts, the user may decide to stop outsourcing this task the LLM; if this experience repeats for multiple theorems, they may eventually cancel the subscription.
3.1 MARKOV MODEL OF LLM ROUTING WITH USER RESPONSE
Consider provider with two LLMs (e.g., standard and reasoning models) denoted M, for í E (1,2)- A single use of LLM incurs a monetary cost c, [0. 1] (i.e., inference compute) to the provider and a latency cost I E [0, 1] (i.e., opportunity cost of delay) to the user; we assume the costs normalized to standard monetary unit. Given a task from the user, each LLM has a success probability p. E (0, 1) of successful completion. We define success as a Bernoulli trial (Mahmood,
2024
Erol et al. 2025). Finally, we assume t < t2, C1 < C2, and < P1 < P2 1.
If an LLM successfully completes a task, the user receives value V > O. Alternatively, if the user abandons the task, the provider incurs a penalty P > 0 (e.g., if user abandons the task, they may unsubscribe from the LLM service entirely with some probability, incurring an expected loss in future revenue for every abandoned task). Finally, we define the user net value per pass S := Vp,- 4 for each L.M as the expected value received from a single pass of the LLM. We describe M, as value-dominated if &, > and latency-dominated otherwise.
Figure 2 summarizes the Markov model. At time t, let X, [M1, M2, Success, Abandon] denote the state. Furthermore, let T" be the time of entering one of the two absorption states. Upon receiving a task at 0, the LLM provider commits to a routing and cascading policy (i,s), where i E {1, indicates the initial model to which the task is routed, and s E [0. 1] is the probability of cascading the task from Mj to M2 if Mj fails to complete the task. Given the provider's ''s policy, the user determines an abandonment policy q E [0, 1]. which is the probability that the user leaves the game without completing the task. We repeat the following sequence:
3

---

Published as a conference paper at ICLR 2026
(1 - (1-A))1 - 9)(1 - 5)
M₁ (Standard Model)
Abandon
11-pp p;)(1 -)s
Success
- P2)
M2 (Reasoning Model)
Pz
(1 Pa) - q)
Figure 2: Markov model of LLM provider-user interaction.
1. The provider routes to state M,, incurring user delay t, and provider cost e₁ The model successfully completes the task with probability Ps, which transitions the Success state and yields the user value V.
2. If the model fails, the user abandons the task with probability q, which transitions to the Abandon state and incurs provider penalty P. If the user remains and the provider initially tried M1, they can cascade to M2 with probability s or stay at M1 with probability 1 s.
Our key assumption is that the user observes the routing and cascading strategy before deciding stationary abandonment policy. In practice, users may abandon adaptively with less information. Furthermore, assume that the success probabilities p) are i.i.d. for any given task, whereas in practice, model output and user feedback may allow p: to change over time. Our assumption follows standard iterative user interaction (Mahmood et al. 2022 Erol et al. 2025). Finally, note that the provider does not need to observe ground-trute truth correctness of an output to cascade. Model failure is interpreted through user-side signals (e.g., the user continues the session with follow-up up prompts or provides explicit negative feedback), which the provider can act upon (Shirali, 2025)
3.2 USER PROBLEM
Let a := Pr(XTT a task and the user stays. Let (gg) = - 11x,= Xo = M2] 1/(p2 + (1- p2)g) be the expected number of times that M2 called before absorption, when initially routed to M2- Furthermore, we define S,(s, q) == Pr Success i} and L,(s, 9) := EI ł, i] as the success probability and total delay, respectively, for the user given a routing policy (i. and abandon policy q. That is, (s, q) is the likelihood that the total session eventually absorbs into Success, and L,(s, q) is the expected cumulative latency paid along the sequence of retries and possible escalation. We omit dependency on and q from these terms when it is obvious. Finally, depending on the initial model, the user success probability and total delay yield closed- form expressions:
{M,, M
be the probability that Mi fails
(1. = P1 + (4))22 1 - (1 S2(g) = 3(4)p, (1 g) = 1 alg) s) Ta(g) = 3(4)
L + (()))
Given provider and user policies, the user's expected utility is the expected value minus total delay
U.(s. = v1 Success} Σ
VSS(s.g) Ls(s.
Implicitly, V quantifies the trade-off in utility from successful completion of the task versus the delay from the use of the model. In practice for LLM subscriptions, users pay a fixed fee for use and there is explicit monetary cost to per query. As result, latency is the only marginal cost to the user at interaction time. The user's problem is, given a provider policy (i, s), to determine an abandonment policy that maximizes their utility q" (i, s) arg max, (s q).
4

---

Published as a conference paper at ICLR 2026
3.3 PROVIDER PROBLEM
Let (((8,)) :- + Cr | i] be the service cost incurred to the provider from trying to complete the task, which yields the closed-form expression
g) = Ci + (()))
3
1 -11--- a11)
The provider incurs cost equal to the expected service cost plus the penalty from user abandonment
J((8,)) =E Σ c PI
Abandon) i C,(s.)) + P(1 (x),
The provider's problem is to route to minimize their cost, assuming the user is maximizing utility, i... (i", arg aagrgming (i,)) (J. (i,,s,((iis))))) (i, (i, arg max, (s(s, {)).
CHARACTERIZING PROVIDER AND USER POLICIES
We characterize the Stackelberg cquilibrium by deriving the user best response and the optimal routing policy. When both models are either value- or latency-dominated, user hehavior is static and the provider prefers single, no-escalation route. If the models differ in state, user behavior exhibits threshold structures and the provider's policy almost always reduces to routing without cascading.
4.1 USER BEST RESPONSE
We characterize the user best response given provider policy (i, s). First, when routing to M2, the provider cannot cascade further, and user response depends on if M2 yields positive net value. This can be also interpreted as the user dynamics when faced with single model in isolation.
Theorem 1. If the provider sets = 2, then the user best response is q" (2, 1 {E2 0}.
Theorem shows that if the provider immediately routes a task to the better (i.e., reasoning) model, user behavior collapses to determining whether the model is value-dominated (i.e., {2 0) or latency-dan domennedd (i.e., E2 < 0)). If the user observes that the delay incurred from using the model greater than the expected value that the user receives from the model's output the task, then the user will abandon the task. For example in a coding or mathematics task, the user may find that they can manually solve the problem rather than query a model in the same time. This behavior is characterized by V. which is an internal variable qualitatively defined by the user.
We next characterize the user response policy if the provider routes to M1 and provides a cascading policy Here, the user response depends on the interplay between & and &z.
Theorem 2. If the provider sets = 1, then the user best response depends on &1 and Ez:
1. If 2 0, then the user best response is q"
for all
2. If s 0, then the user best response is q" (1, s) for all
3. f 0 < E2. then let so = -E1/(E//12 &x). The user best response is (1,
1 1 < 3o}.
4. fff, > 0 > Ez. then ler
F(s, q) = (1 - (p2 + (1 p2)g)² + Ezs (1 (1 p1)(1 - p2) =) g)²).
Let $ be the unique root in j0, 1] satisfying F(sL, 0) and let sy E./(E1 E2).
Then the user best response from routing to model 1 is
0
if S s1
r- (1,
ct()s)
if < s << $H 2
1
where q²(s) (8) is the unique root in [0, 1] the quadratic equation (s)) 0.
5

---

Published as a conference paper at ICLR 2026
User Optimal Policy
P1 0.1, p> 0.3, 1.0, t =.0, s= 0.25
1.0
0.5
00.6
3 0.0
1000
-0.4
-0.5
-0.
-1.0
-1.0
-0.5
0.0
0.5
1.0
81
Figure 3: Heatmap of the user response when i == 1 and 0.25. When both models are value- dominated (top right) or aatency-dominal models differ in their regime (top left and bottom right), the user response depends &2, Note that q² E (0, 1) is in the interior only in certain regimes for E 0 E2-
(bottom left), the user best response is static. When the
Figure visualizes the different regimes. From Theorem 2 and Theorem I when the two models undifferentiated in terms of being value- or latency-dominated, then the user response is static. both models are value-dominated, the user should stay in the system regardless of the routing and cascade policy. If both models are latency- dominatd,, the user should abandon the task.
Provider routing and cascading policies affect user behavior only when the models offered by the provider are differentiated in value. When & should only accept routing to Mj if the likelihood of cascading is high, i.e., 8 > 8p- Otherwise, users can abandon the task since the expected delay too large. replicating the single-model logic of Theorem [ Finally, if > 0 > &2. Mj is value-dominated and M2 is dieecc-ddoiinted. Here, routing directly M2 should encourage users abandon the task. If the provider first routes to M1 and assigns a low < then the user is incentivized to remain. Raising > 8и pushes users to abandon the task since may get cascaded interior regime (5) has a non-static q s1 = q! (s), where the user is incentivized to abandon the task with probability. This the only scenario where user behavior stochastic and changes with slight perturbation of Although the provider-optimal policy may find SH), it may be also strategic to avoid this regime for provider to ensure user behavior is predictable.
&2, Mi provides negative net value and users
latency-dominated M2. Most interestingly, the
8H)
Theorem 2 assumes that users know si. route i, but not s, Pt. or P2. Here, beliefs drive the same threshold logic with respect to belief summaries; for example, from past routing. Over repeated tasks, this leads to an exploration versus exploitation problem.
P1. P1.P2). In practice, the LLM interface may reveal the
can act using [s[s] where rTs) is posterior estimate of observed
4.2
SRWVIDER-PPTIMA. POLICY
Given the user best response, the provider optimization problem becomes taking the maximum of single-variable variable optimization problem and scalar:
min min Ji (s. (1,8)), (1,))), Jz (1 < 0))
The structure of (1 q" s)) depends on which models are value- or latency-dominatee We first prove the two cases where the models share the same sign with respect to &.. before considering the settings where the models are differentiated.
Theorem 3.
When both models share the same sign, the provider-optimal policy involves routing to one model without cascading:
1. 1f, E2 > 0, then
(1,0)
2 Pi P2 otherwise
if -
(i,s5))
(2,0)

---

Published as a conference paper at ICLR 2026
Provider Optimal Policy (51 <0 <0< 52)
Provider Optimal Policy 05 E2) P1 0.1, Pr 0.3, 4 =.5, 225.5 V=
P1 = 0.1, P2 6 2.0, 9=
20
20
Model 1,
Model 1, smo
Model 2, s=0
Nocel 2
Model 2, 5-3.043
Mocel 1, 5-1
15-
15-
Model 1,
10
10
5-
-2
cs/Pi-Calt
cs/P1 cc/P2
Figure 4: Left: Heatmap of the provider-optimal policy for & < Eg. Right: Heatmap of the
provider-optimal policy for Eι > 0 > &2. For both plots, we hold C₁ = 1 constant and sweep the difference cost-of-pass c1/P1 - C2/P2 as well as P. For most regimes, the optimal policy is either to route immediately to M1 or to M2 without any cascading. There exist only some regimes for č > E2 where the optimal policy involves probabilistic s E (0,
2. If &2 < 0, then, setting s [0, 1] as a free variable,
(i' (1,
UP C2
P2 p1
(2. 0)
otherwise
If the two models undifferentiated, i.c., both value- or Aatency-dominated. then cascading from one model to the other adds costs and variance without any benefits, making single routing optimal. When both models are value-dominated. the optimal policy is to route to the best model determined by the cost-of-pass (Mahmood 2024 Erol et al. 2025). Because users should not abandon tasks, the penalty for user abandonment is irrelevant and cascading cannot improve costs beyond routing users are likely to abandon tasks if the model cannot complete them. Here, the routing policy depends on P versus the incremental advantage in the cost-of-pase (e2 - -c1/ -11). If the penalty of user abandonment is low, the optimal provider policy is to route to the cheaper standard model. Moreover, the choice of is irrelevant since users will abandon regardless. On the other hand, if P is large, then the provider must route to the reasoning model reduce the risk of incurring the penalty.
to the appropriate model immediately. When both models are Satency-dominate
Theorem 4. Let P1 ==
P1) and let P3 == (o2loos)))tso/pppppoo If 0 < & then the movider-optimal policy breaks into four cases:
(1,0)
if
and P P
p1
P2
(2.0)
if
and P > P1
(i*s*)=
di
C2
(1,0)
if
and P < P2
(1,
if
and P> Pa
P1
P2
When & < 0 < ε2, users should remain if routed to M2. but leave if routed to Mj with limited chance of escalating to M2. However, if ci/p1 < 02/p2, then the provider should always route to M1. Alternatively if P < P1 for the threshold value PI. then the provider should still route to M1- Interestingly, even though the reasoning model is preferable to users, the provider-optimal policy is to route to Mr. unless the cost-of-pass of this model is sufficiently high. This leads to misalignment between the user and the provider when their respective rankings of the models disagree.
Finally, we consider the case when { defined by thresholds $ and sy. Given the monotonicity of the provider function, determining the optimal routing and cascade policy is equivalent optimizing over the following five settings:
&g. Here, the user best response has three regions
min { (0, 0), (1 0), min J1(s,g((s)), (1,, J2(0)
sE(32,8))
7

---

Published as a conference paper at ICLR 2026
Because the provider policy is not necessarily static in this regime, we focus on sufficient conditions under which the provider can employ static policies.
Theorem 5.
E > 0 > ε2, the following statements are true: (2/ the provider-optimal policy is route model only (1",0)==110
1. c/p1
2. If mmin(fi/r //p, (c2 - c1)//pp -p))), the provider-optimal policy route model
1 and cascade on failure (i" (0),5))= (1, where E 1s is free variable.
3. If (c2 - -c)/ pı) < P 02/p2 < c/p1, the provider-optimal policy is to route to model s") ().
Figure visualizes the two settings where & < < & and 0>> &2, respectively, For most settings, a static policy, i.., (i", E ((1,0), (1. 1), [). optimal. Moreover in both cases, route to M1 first. This underscores the value of routing
the optimal policy is almost always to a weaker model initially, when the models are differentiated. Furthermore, the role cascading differs depending on which of the two models is value-dominated. If Mj is the only value-dominated model, then cascading is effective specifically when the penalty is sufficiently low with respect to c c1/P1; otherwise, the provider is incentivized dominated, then aggressive cascading via = 1 is useful only when P is sufficiently small.
remain the standard model. If M2 is value-
Finally, we note that although we fix V, P1. and P2, the results naturally extend heterogenous setting with task-dependent parameters. Given a prompt the success probability (a() and user value V(x) [z)) to better estimate the per-task user value (x) = (x)) f and determine personalized optimal route.
routing algorithm may first estimate
WHEN ARE PROVIDER AND USER MISALIGNED?
In this section, we discuss the implications of the equilibrium between user and provider to un- derstand when the provider routing decisions act in the best interest of the user. We first define misalignment gap that characterizes when the provider's cost-minimizing routing strategy is sub- optimal for the user utility function. We then study the conditions when misalignment exists.
Consider the cost-optimal policy for a provider ("",))), The misalignment gap is the difference between user utility-optimal routing policy versus the provider's true policy
Au == max U.(s, (i, (f, s)) - U+ (8" (i", (i)))))
4,8
Ar O, then the provider-optimal policy is also optimal for the user, meaning that the provider's actions are in the user's best interests. However, if Au > 0, then there is utility gap for the user incurred by the provider minimizing service costs. Depending on the costs and penalties, misalignment gap exist for every possible configuration of &2,
Proposition The following statements are true:
1. If f1.1. E2 > 0, then AU = 0 if and only if sigu(ci/// - 10a00-2//ppp (P) - -
2. 1, < O, then Au 0 if and only if sign(E2 {1) sign P (02 - (0a--a)/60 11) -p)))-
3. If (igg)((E)) * sign(E2), then AU = 0 if and only if & > and 8
0.
When f2 > 0, provider routing collapses to static cost-of-pass rule. Here, alignment requires the provider and user rankings to match; otherwise the provider selects the cheaper model while the user prefers the more successful one. When E2 < 0, the user always abandons, meaning that alignment requires the provider to select the same "less-bad" model from the user's perspective. This only occurs if the provider's switch regime agrees with the user's 's ranking. Finally if the models are differentiated, any likelihood of cascading exposes the user to latency. Figure (Left, Middle) highlight this regime for the same parameters as in Figure showing the misalignment gap.
8

---

Published as a conference paper at ICLR 2026
P1 0:1.2==0..3.. m.2.0.0..0.............. 4.0.00 9.9
Miseligmment (6, 5
User Miselgnment (6 -005)))))) P1=01. 0: 0000==00000.6660 005.2 2=
Pt 0.1. 0.3, a =0555 0.5, 5 5-15515 V=8.0
Effect of Throtting Provider Costs
20
19
19
10
(ro toul)
Figure 5: Left: Heatmap of the user misalignment gap for & < ~ Eg. Middle: Heatmap of
the user misalignment gap for ς >> E2. Right: Heatmap of the effect of throttling on provider costs; the dashed line is the line P = min(ci/p,, 22 /P2)- For all plots, we hold e = constant and sweep the difference in cost-of-pass c1/P1 02/ P2 as well as P.
6 THE RISK OF THROTTLING LATENCY
Finally, we study an extreme case of misalignment where the provider may seek to inflate the latency of their models to encourage users abandon tasks more often, thereby reducing service costs from repeated prompts. Specifically, consider provider forced to a fixed subscription price due to factors such as market competition, meaning they cannot directly re-price in response to higher service costs. Instead, throttling latency functions and consequently, provider costs. Here, user abandonment causes a penalty P, but if is sufficiently small, this loss may offset the higher service costs.
hidden price increase that implicitly reduces usage,
Suppose the models have latencies t, < Vp, for E {1.2]. Let q" (i, s; t1, t2,)) be the user response for these models and let J = min g" (i,s;t1, t2))) be the provider-optimal cost. Given two throttled latencies 4 > 4 for E {1,2], 2}. let Jjout == min (Ji(s, q (1, s; b1, f2)] as the provider-optimal cost after throttling. Note that l are now variables that the provider can tune. If Joost S Jore: then the provider benefits from throttling, even though reduces the user's utility.
Proposition 2. If P min/c1/PL aaa..c/pp2
c2/P2). then setting f Vp, incurs Jpnst Jore
Figure (Right) visualizes the gain from throttling. For static policies, the gain min,fcs/pe min; (cs + P(1 - -p:)), which is linear in P. Moreover, throttling maximizes misalignment Au. since providers prefer users abandon tasks while users would prefer to successfully complete tasks. Most importantly, providers will reduce costs from throttling either (or both) models.
When P exceeds minfci/Pr c2/pp) the gain from throttling becomes negative and this misaligned policy loses its dominance. Therefore, to prevent throttling, the price of user abandonment must exceed the cost-per-pass, In practice, this can be realized either by users proactively unsubscribing from LLM providers, by providers voluntarily ensuring guarantee users, by permitting users to opt-out of routing and cascading by self-selecting models.
7 CONCLUSION
We study the problem of LLM routing as an interaction between a provider offering menu of models and user subscribing the LLM service seeking the best model complete their task. The user impact of routing depends on the patience of users facing the trade-off's between quality, latency, and cost. We formalize a Stackelberg game between single provider with two models, standard and reasoning, and a single user. The user maximizes their utility from L.M use by abandoning tasks with large inference delay, while the provider minimizes the cost of serving the user and the risk of users leaving the LLM subscription service when they abandon.
We find that user patience for the LLM provider governed by the sign of net value the user receives from each model. If both models offer similar value, then the user's action is unaffected by the routing policy. On the other hand, the two models differ in value, then users are incentivized to abandon their tasks with some probability if the likelihood of being routed cascaded to the worse model is high. On the other hand, the optimal provider routing policy is almost always static, showing that there is limited value in cascading except in narrow regime where the models

---

Published as a conference paper at ICLR 2026
differentiated in user net value. This leads provider-user misalignment when the value rankings from a user and the cost-of cost-of-pass rankings from provider disagree. Most importantly, misalignment grow extreme when users unlikely to unsubscribe. This motivates providers to artificially throttle latency, which saves their costs but yields poorer user value.
The extant routing literature often optimizes over the trade-off between solution quality and latency by estimated per-task success rates (Hu et al.] 2024). Our analysis suggests that user behavior can play core role in determining optimal routing policies. For instance, the development of this paper involved sustained use of an LLM service, where for example, deriving key theoretic results (i.e., high-value tasks) necessitated user patience from repeated prompts (see Appendix for details). Consequently, we advocate for LLM routing algorithmss to simultaneously estimate both model success rates and user value per task in order to better serve users.
We note several limitations and next steps. First, our analysis is of two models, but the framework can be extended more models. Second, our subscription framing abstracts away per-query mon- etary prices; pay-per-call API pricing can be incorporated by adding per-query charge to the user utility. Third, we assume that user can observe the provider cascade strategy and acts with a sta- tionary abandonment policy. In practice, routing strategies are hidden and users may need to adapt. Finally, we treat the per-pass success as i.i.d. and assume fixed user value and provider penalty that are known to both players. In practice, these terms may not always be immediately quantifiable and may require online learning.
ACKNOWLEDGMENTS
The author thanks the ICLR 2026 editorial chairs and anonymous referees for providing valuable feedback that improved the content of this paper. This work was funded partially by the Natural Sciences and Engincering Research Council of Canada.
REFERENCES
Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, et al. Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 7421-7454, 2024.
Dirk Bergemann, Alessandro Bonatti, and Alex Smolin. The economics of large language models: Token allocation, fine-tuning, and optimal pricing. In Proceedings of the 26th ACM Conference on Economics and Computation, pp. 786-786, 2025.
Francisco Castro, Jian Gao, and Sébastien Martin. Human-ai interactions and societal pitfalls. arXiv preprint arXiv:2309. 10448, 2023.
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How large language models while reducing cost and improving performance. arXiv preprint 2xXiv:2305.05/76, 2023.
Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach routing and cas- cading for 1lms. In Forty-second International Conference on Machine Learning, 2025.
Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. Hybrid Ilm: Cost-efficient and quality-aware query routing. In The Twelfth International Conference on Learning Representations, 2024.
Mehmet Hamza Erol, Batu EI, Mirac Suzgun, Mert Yuksekgonul, and James Zou. Cost-of-pass: An economic framework for evaluating language models. arXiv preprint arXiv:2504. 13359, 2025.
Yingjie Fu, Bozhou Li, Linyi Li, Wentao Zhang, and Tao Xie. The first prompt counts the most! an evaluation of large language models on iterative example-based code generation. Proceedings of the ACM Software Engineering, 2 2025.
Ofer Garnett, Avishai Mandelbaum, and Martin Reiman. Designing a call center with impatient customers. Manufacturing & Service Operations Management, 4(3):208-22) 220,,227,,0 2002.
10

---

Published as a conference paper at ICLR 2026
Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system. arXiv preprint 21Xi:22433.22031, 2024.
Satyapriya Krishna, Chirag Agarwal, and Himabindu Lakkaraju. Understanding the effects of iter- ative prompting on truthfulness. arXiv preprint arXiv:2402.06625, 2024.
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pp. 19274-19286 PMLR, 2023.
Rafid Mahmood. Pricing and competition for generative ai. Advances in Neural Information Pro-
cessing Systems, 37.75727-75748, 2024.
Rafid Mahmood, Sanja Fidler, and Marc T. Law. Low-budget active learning via wasserstein dis- tance: An integer programming approach. In International Conference on Learning Representa- tions, 2022.
Nestor Maslej, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Njenga Kariuki, Emily Capstick, Anka Reuel, Erik Brynjolfsson, John Etchemendy, et al. Artificial intelligence index report 2025. arXiv preprint 250.:25004,0139, 2025.
Pinhas Naor. The regulation of queue size by levying tolls. Econometrica: journal of the Econo- metric Sociery. pp. 15-24, 1969.
Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route Ilms from preference data. In The Thirteenth International Conference on Learning Representations, 2025.
OpenAl. Introducing gpt-5. https: http:///pppaaaaa/ee httpaaaa..co/nnnd////tttttoooee gust 2025, Accessed on September 8, 2025,
-pt- 5/ Au-
Ali Shirali. The burden of interactive alignment with inconsistent preferences. In The Thirty-ninth Anmual Conference on Neural Information Processing Systems, 2025.
Marwaan Simaan and Jose B Cruz Jr. On the stackelberg strategy in nonzero-sum games. Journal of Optimization Theory and Applications, 1(5):533-5555 1973.
Haoran Sun, Yusen Wu, Yukun Cheng, and Xu Chu. Game theory meets large language models: A systematic survey. 2025.
Joel Wester, Tim Schrills, Henning Pohl, and Niels Van Berkel. "as ai language model, cannot": Investigating Ilm denials of user requests. In Proceedings of the 2024 CHi Conference on Human Factors in Computing Systems, pp. 1-14, 2024.
Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thoughts representations for cost-efficient reasoning. arXiv preprint arXiv:2310.03094, 2023.
Zhoupeng Jack Zhang, Jiaqi Shi, and Shaojic Tang. Cloud or on-premise? a strategic view of large language model deployment. A Strategic View of Large Language Model Deployment (June 16, 2025), 2025.

---

Published as a conference paper at ICLR 2026
Task type
LLM role
Acceptance criterion
Typical outcome
Proof sketching
Suggest directions, arguments, and counterexamples
Human derivation repro- duces all steps
Often useful after 2
10 reprompts
Algehraic checking
Expand derivatives, mono- tonicity. bounding steps Prepare bullet form for text, condense paragraphs and en- force writing style Generate plotting scaffolds
Symbols match manual cal- culations
Often useful after 2-
4 reprompts
Draft writing
Preserves technical meaning
Useful with one pass
and improves clarity
Figure prototyping
Matches definitions and the- orem regimes
Useful with minor edits
Table 1: Summary of how LLM assistance was used and validated.
A PROTOCOL OF LLM USAGE
We used LLMs assistive tools throughout this project. The assistance covered: (i) deriving proof steps or searching for counterexamples; (ii) preparing first drafts for sections and editing with feed- back; and (iii) coding plots. All mathematical statements, derivations, and proofs in the final paper were human-verified end-to-end. All figures and text underwent human editing.
We interacted with the LLM in multi-round sessions. Each session targeted a single goal (e.g., proof sketch, draft writing and editing, or code). For many theoretical results, although we were routed to the complex reasoning model, found our behavior be patient in repeated interactions, suggesting the high user value placed on the work. For certain tasks and over time, we found our behavior be less patient, suggesting aare-narying nature on user value, which could be studied in future work. Table [ summarizes typical tasks, the intended role of the LLM, and acceptance criteria.
12

---

Published as a conference paper at ICLR 2026
B HELPER LEMMAS
In this section, we outline several lemmas required for the main proofs the paper.
Lemma 1. For any fixed the objective function for the provider is monotone in s.
Proof of Lemmal / It suffices prove that Ci(s) and (i)s are monotone in 8. Note that both (ss) and (i(s) can be written as fractions of two affine functions in s. As a result, the first derivative of these functions is ratio of a constant value over a non-negative quadratic function (1 - a(1-s)() =))?.
Lemma 2. For any fixed let (a) = (51 &2as/p2) a(1 s)) and let Uj (a) (&1 {2as)/(1 a(1 s)). Then for all q E [0.
UT (a) < U (8, - (a)
1 2 0
Uit (a) - (1 - Uj (a)
otherwise
Proof of Lemma 2
Note that (gg) is a decreasing function of q and is bounded [1, 1/p2]. Therefore,
for any fixed s, if &2 2 0, then & Ezas S &1 + E2a S E1 E2as/p2. Moreover, the inequalities are reversed when &2 <
Lemma 3. Let (s) = (1 - 8) + E28/p2 and K (8) = E1 - 8) + E22.
1. Ui (a)) monotone increasing (respectively, decreasing) a and therefore, decreasing (respectively, increasing) in if K (s)ss (respectively, if K (s) K*(ss)
0).
2. U1 (a) monotone increasing (respectively, decreasing) in a and therefore, decreasing (respectively, increasing) in if K (s) 0 (respectively, if K (s) 0).
Proof of Lemma 3
We take the first derivative of U,t as follows
dtitt 1a(1-a(1) (1- a(1
s))/pp a(()=(1-s))pp + (1 - s)(Ei + {2as/p2)
Kt(s)
da
s))2 s))
(1 -1-a(1 a(1 s)] s))2
Since the denominator is positive for all [0, 1], the first derivative is strictly positive if K (s) > 0 and strictly negative otherwise. Consequently, Ui* (s) is increasing or decreas- ing, respectively, in a. Finally, note that a(g) is decreasing function in g, meaning that U,t (a:g)) is decreasing or increasing, respectively q-
2. We take the first derivative dU; /da = K K (s) a(1 - =)) and use the same steps to show UT (a) is increasing in and therefore, decreasing in if K K-(s) >> 0 and the reverse otherwise.
Lemma 4. Let P1 == (c2/p2 - c1)((1 - (p). ffc//pi > c2/p2, then Pi > (C2 c1)/ PI).
Otherwise, P1 S (02 c1) P1).
Proof. Proof of Lemma
4 We subtract
ci
(p2 -P1) (c2 c1)(1 pi)
(1 -1-1 p2) (c C2f1
P1
C2
pz
pz
P2 P1
(1-11) (--P)(P2 - P1)
(1 P p)) Pt)
From the numerator, this fraction is positive when c1/p1 02/p2, and is non-positive otherwise.
Lemma 5. Let P2 = (c (1 sn) + c220//2)//110 + p1)))) If c/p/ < c2/p2. then Pa e (cr (2/2).
13

---

Published as a conference paper at ICLR 2026
We will show below that P2 is convex combination of c1/p1 and 02/ 22P2- Consider the general function
Proof. Proof of Lemma 5
c1(1 5)
2
dP:
C2pL
C1
P2(s)
P2
P1
Pi)s
ds
(p
+ ps)
Note that when c 22, this function P2(s) is monotone increasing. Furthermore, Pz(0) c1/P1 and P2(1) = C2/ C2/P2. Since So E (0, (so) =: P2 represents a convex combination.
Lemma 6. The following identities hold for all 8, q E [0, 1]:
1. (1 2 P1
2. minfci/PL. c2/p2} s (4001-9)4000000000001---0-000000 S max(ci/P1. 02/P2)
Proof of Lemma 6
1. We subtract as follows:
S1(s,g) 1 a(()(1 -3 Pi a(g) (8(g) + PI s]) 1-o(g)(1 8) 20
+ aa())((())p)
V8, [0,
(p:lss/(ppp11ppp/ 2. Let R(s) = (c]11s) 5) ((g))22))//(( (pi(1 - s) + 3(4))25). Note that
dR 3(4) (62p) p2(1)
ds P1 (((())p2 p1)
meaning that R(s) is monotonic increasing when c2 > c and is monotonic non- increasing otherwise. Furthermore, note that R(0) = c Pr and R(1) c2/p2- Therefore, R(s) is a convex combination of ci c1/p1 and c2/P2-
14

---

Published as a conference paper at ICLR 2026
C PROOFS OF MAIN RESULTS
C.I PROOFS FOR SECTION
Proof of Theorem 7 We take the first derivative of U2(1) to show
dU2 = dq
E2 p2)
(p2 + (1 D2)
that the derivative is positive for all e [0, 1] if & 0 and is non-positive otherwise. If the derivative is positive, then U2(g) is monotone increasing and the optimal response q² 1. the
derivative is non-positive, then the utility is non-increasing and optimal response is q" 0. 0
Proof of Theorem
1. If Ei, Lemma (*) 0 for all [0,1] and UT (a) is monotone non-decreasing in q- Consequently,
0: From Lemma 2 (1( < Uf (a). Furthermore because & 2 0, from
3 E28
1-p1 P2
(1( S (t) (U((((())) 1 p1)
(s,
showing that the envelope function is tight at the maximizing point q² 0.
2. If fi.
0: From Lemma 2 (s. q) s UT (a). Furthermore, because & S 0, from Lemma 3 K (s) < for all E [0, and UT(a) (a) is monotone non-increasing q. Consequently,
(1(5, S S (a) - UT((((()) = 1s
showing that the envelope function is tight at the maximizing point o" = 1.
3. If E1 < < E:: From Lemma (i I (.) Furthermore from Lemma the envelope function monotone decreasing in if K+()) > and monotone non-decreasing otherwise. Since K* (s) is convex combination of positive and negative value, have K (s) 0 for all > so and (+(s)) < 0 otherwise.
We first consider the case of > sp- Here, we leverage the monotone decreasing property of the envelope
E1 E28
E-s 1-p: (s
(1(8, S (T) s (a(()) = 1- P1) (1
P2
Similarly, when < 80. we leverage the monotone increasing property of the envelope
(s - (a) U (a(1))
1s,
Both cases show that the envelope function is tight at their respective maximizing points.
4. If & > > Es: We break the proof into three regimes for 2 sH, s S sp, and s
8n).
For 2 8H: We first characterize the user best response using the envelope arguments. From Lemma 2 and Lemma 3 we have (s, g) < UT(a), (a), which is monotone non-
increasing if and only if K, s) S 0. This occurs only in the regime for 2 sH. Here,
(s q) < (T) < UTT(a())
(s,
showing the envelope function is tight at the maximizing point.
For S s, For the remaining two regimes, the envelope will not be a tight upper bound, meaning this argument will not be viable and require an alternate approach.
15

---

Published as a conference paper at ICLR 2026
For notational simplicity, let (s) N(s, ()/.g)g))))))))) where N(s, q)
& + (2)( s and D(s, q) = 1 - alq) - s). We seek to identify a constant Sy such that for any < $1,,
Ui(3,9) q) (s ((s.0) - ((s. 00 D(s,9))(s,0)
Vq [0.1]
Since the denominator of this fraction is always positive, we consider the numerator
((,, 0) - [ss 0) (s,,g) 1)
(1)
= f1 + (1-p1) (1-pi)(1 - 9) {2
(p1 + (1-p1)s)
P2 + (1 - P2)q
{1 1- P1
E2
(pi + (1 - -pi) + (1 - pi) 8]9)
(2)
P2
= =22((1 - pPi)(r + (1 - -pi)s)
1-4 q
1
- (1 - (1-pi)(1 - -8)0 61
-
Pl
E28
P2 + (1 - (1-p2)q
P2
P2
(3)
P1 + (1 pı)s
1 Pı
= --22sq(1 p1)
- (1 --))(1111 - -s)9 (
E28
(4)
(P2 (1 - p2)9)p2
P2
Pi + (1 pı)s E2s
+ f1(1 - 5) + - -p1)(p(1 - E2s
= =-q(1 - P1)
(5)
P2 + (1 - P229 P2
p2
= q(1
--q(1 - pı)
[1(1 - 8) (G(1-))
828
Pr + (1 - p1)8 + (1 - 1p)pp((11 - 5)
(6)
P2
P2 + - (1-p2)q
-:((.)))
Above, 3 cancels & (p1 + (1 - p1)s) and groups the terms, 4) combines the fractions
in parentheses, and (5) factors q(1 p1). For notational simplicity, let T(s, q) be the equation within the parentheses of the final equality. Note that T 0) F(s, 0) for
F(s, q) as defined in the theorem statement. Rather than characterizing 8,, as the root of F(SL, 0) O, we will characterize it as the root of ((s O.
Given the definition T q). we observe that for (1 4) Ui (s, 0) S 0 for E [0, 1]. we must have
T(s, q) 2 min T(s, 9) 2 0. gE(0,1]
In order to find the minimizing q, we take the derivative below
ÖT( (P1 + (1 p1 (1-2)((2) 20 0
0g
(p2 + (1 p2)
where the inequality follows from &a < O. Because the derivative is always non-negative,
T(s, q) is monotone non- non-increasing and the function is minimized at q 0.
We now must find a value of st such that for all s < SL, we have T(s. g) 2 0, which implies further that (s, q) - (s, 0) > 0 for all E [0, Note that T(00, 0) = E 0 and 01, = E /p < 0. Furthermore, the derivative
dT = -
+ E2
P1 + ++1111 m) + 212 - P1) PP) p2) <0
111)
ds
P2
P2
meaning that for the interval j0, 1]. T(s, 0) begins positive, decreases monotonically. and ends negative. Therefore, must have unique root within the interval [0, 1). Let st be this root. Thus, for all s < sT. T's,, 0) is positive and the user best response 9" (1, s) = 0.
For E (SL, sH)): Here, we directly obtain the the optimal q" (1,, via the first-order opti- Because
mality condition ( = 00g = (N' s..)) D(s, q) N(s, 4) D'(s, the denominator is always positive, this condition equivalent to ((
0 N'(s,, q) D(s,g)
(p2 + - (1-P2)q) (1 (1 p1 p1)(1 () +))= -
(1 (1- p1 K28
(1 (1-Pi)( - g){28)
) (1 pi)(1 -s)
P2 + (1 -P2)q
16

---

Published as a conference paper at ICLR 2026
By factoring out -(1 p1) (1 p2)4)² simplify condition to
{2 (1 pı)(1 9)(1 5)) fi s)(p2 + (1 p2)9)²
+ (28 - s)(1 - p1)(1 -)2 + +1 p2)g)
(7)
= fi - sp = (1 - p2)4)²++
- (1 11-1P1)(11 p2) -) s)(1 9)
(8)
(1 s)p2 + (2s (1-p:)(11 - p2) -))
+ +1 (1 p2) + 1 (2-(1)))))
+ q² - s) - p2) (5i(1- - -1 -1)p
(9)
where 7 follow from expanding and collecting like terms. Let F(s, 9) == RHS 9 be the implicit function defining this equation. We must now prove that for any E 8H), there exists a unique q² (s) that satisfies Fss, 9 (s)) O.
To prove that unique root always exists, we first observe that for any fixed s. (s, q) is convex function q. since the slope of the first partial derivative is 2(1 =) - (5 P2) - (2=1- - pP1) 2 O. Furthermore, we show that that F(s, q) takes different signs at the boundaries, notably lim g-m-- a-+ 0--0 F(s, q) < and lim - F(s, 9) > O, where the
limits, the user response moves towards the optimal boundary solutions.
For the lower limit
0U1
lim F(s, x 8
lim
0q
8 a
U1 ④) UU,
lim
8- a-0
9
-(1 p1) (s, 9)q
lim
s-15 --55
D(s D(,
(1 (1- (1-p:)((ssd)
lim
x-t g)
for s SL.
where the inequality arises from the observation that T(s) > 0 in this regime. For the upper limit
aU1
lim 3
F(s,g)
x
lim
8-187-9-1 0q
N'(x,g)((( N(x,, (s,
lim
D(s,
= lim (1 - pı) (5, s6) E2s)
=> for < SH-
By the Intermediate Value Theorem, for any fixed E (sL, 8H), the function F(s. 9) passes for some q. Because F Fss. q) is a quadratic function q. can change direction at most once and therefore, the root y² (s) is unique.
C.2 PROOFS FOR SECTION 4.2
Proof of Theorem
1. If E1 > 0, then by Theorem 2 the user best response is always qi for all (1,2} and s E [0, 1]. We then compare the two separate cases of routing to model and routing to model 2.
17

---

Published as a conference paper at ICLR 2026
If the provider routes model 2, their objective function value is (2) c2 =
02/P2. P2. On the other hand, if the provider routes to model 1, their objective function value is
+ 1-P: C2
(s, 1 1 (1 P1) s)
From Lemma 1 this function is monotone in 8. To determine the direction, we take the first derivative
L- 02 - -p:)(1 8)) (p)) (cr + L-p1 028
0J1 (s,0)
p2
Ōs
(1 - (1 - pı)(1 s))2
1- P1
Pi 02
(1- (1-
P2
First consider the case of c1/P1 < c2/22 Here, we have 0J1 (8. 0)/Os > 0. meaning that (s, 0) monotone increasing and the optimal s* Furthermore, (1 0)
c1/P1 < C2/P2 < J2(s, meaning that the optimal policy is to route to model 1 without any escalation.
Conversely c/p/ c2/p2. we have 8J (s, 0)/0s < 0 meaning that Ji (s.
for all 8. Furthermore, (1, = 6 + (1 121-p1 P1 )02/ /22pp P2 = 01 + c2/p2 - P102/P2 > c2/P2 Jz(s, 0) where the inequality falls from the condition on the cost-of-pass. Therefore, in this regime, the optimal policy is to route to model 2.
2. If &2 < O, then by Theorem 2 the user best response is always (i, 8) = for all (1,2] and s E [0, 1]. We can compare the two separate cases of routing model versus routing to model 2.
If the provider routes model 2, their objective function value is J2 02 + P(1 P2). On the other hand, if the provider routes to model 1, their objective function value is 1s, 01 + P(1 P1). which is independent from s. Comparing these two objective function values yields
(2(s1) = P C2
(2(8,)
P2
This completes the proof.
Proof of Theorem From Theorem because & > 0, the provider's return from routing to model 2 is J2 )²) C2/ p2. From Theorem 2 the provider's return from routing to model is dependent on the escalation policy
+ P(1 P1)
if su
(s
C1 + 1-p1 02S
D2
otherwise
P1 + (1-P1)s
Furthermore from the proof to Theorem J₁ (s, q) is monotone increasing if e1 c1/p1 < o2/P2 and monotone decreasing otherwise. Thus, for sp, (s(",, (1(ss""") ) is minimized at s" so if the cost- of-pass of model is lower than the cost-of-pass of model 2, and is minimized at s" otherwise. Therefore, determining the provider-optimal policy simplifies comparing Jz(s, 0), (0, 1), and min (Su, J1,,
We first consider the case where c1/p1 02/P2. Here (1 J1(so, 0) due the monotone decreasing nature of the objective. Furthermore,
1
P2
C2
P102
C J2(s, 0)
(1,00) =
C2
P1
P2
P2
P2
18

---

Published as a conference paper at ICLR 2026
meaning that routing model 1 and escalating is strictly dominated by routing directly to model 2. Therefore, the optimal policy is to route to model 1 and set s = 0 if ci + P(1 p1) < c2/P2. Re- arranging this inequality yields the condition P < P1- Conversely, if the condition is not satisfied, the optimal policy is to route to model 2.
We next consider the case where c1/p1 < c2/p2. Here, Js 0) < J1(1,0) due the monotone increasing nature of the objective. Furthermore
(so, 0) - (2(s = C1 1-P1
C200
P2
C1 C2 <0
P1 + (1-
P2
P1 +
P1)80
meaning that routing to model 2 is strictly dominated by first routing model 1 and then escalating with probability 8o- Therefore, the optimal policy becomes to route to model and set s =
c1 + P(1 P1) < J1 [so 0). Re-arranging this inequality yields the condition P < P2-
Proof of Theorem 5 To determine the provider-optimal policy, we must check the objective function in four regimes: JI (s, 0) for E [0, sr., Ji (s, q' (s)) for (st, 8H), Ji (1, and (2)) Note that J1 (1, 1) subsumes J1 (s. 1) for E [H2 1], as the objective function is constant in this regime. For each of the three cases in the theorem statement, we prove optimality by checking against these regimes.
1. Note that (0, 0) = c1/p1. We first show that s = 0 is optimal within the regime [0, s1] by proving that the objective function is monotone increasing in this regime:
aJ1(s,0) 0)
1- P1
P1 12 c1 00
as
(1 - (1 p1)(1 - 3)}²
P2
where the inequality arises from c/p1 < c2/p2. Therefore, (0, S Os e [0, st..
To show that J. (0, <
(81,, sw), we take the difference
J (s)) (0,0)
(10)
(s))
(1 (s (s)))
(11)
Pı
= ((( - c
a (1 (1-p:) + P (1 (s,,
(12)
p
= -C((s - a
a g5(s, - -m1) + +1 (s)) p
a
pı
P1
(13)
(s)) -
(Si(s, - -P)) + (1- S1(s (1), P1 p1)
a)
p (P-
C1
P1
(14)
2
(15)
where
12 and 13) follow from adding and subtracting ci and (s)) respectively.
Then,
14 follows from re-arranging the terms, and (15) follows from Lemma 6
We then show that 0 yields lower cost than 1:
C1
C1 (1 p1) P(1 P1)
C1 <a + P(1 P1) (1,11)
P
P1
P1
Pi
Finally, to show that si (1, yields lower cost than routing directly to model 2, we first observe the identity
c1/11 -2
caP1 - G1P2 Off >
C1
P1
P2
P1
P2)
P2
p1
Then,
c1/P1 C2
C1
P>
>
S 02 + P(1 P2) = J2(1)
P1
1 P2
19

---

Published as a conference paper at ICLR 2026
2. Note that (1, c1 + P(1 p2) J1(1, [S 1]. We first show that routing and escalating offers a lower objective function value than routing immediately to model 2:
P<
C1 a + P(1 P1) < P(1 pp2) = J2(1)
P2 p1
To show that (1 S ( E (51, sH), we take the difference
= -(1,) Ci (s)) - ((S((s, - pP))
(16)
Gi(s,g( -1 - -P) (Si(s, - -pi)
(17)
(i(s, -M
2 c P) (s)(s) -p))
(18)
P
20
(19)
where the second inequality follows from Lemma
Finally, to show that (1, - (1ss,))ss E [L. we first note from Lemma that (s, 0) (c1 + (1
(ppppp/>)/++1111111 c1/p1 and is monotone decreasing when c2/12 < c//1. If this function is monotone decreasing, then
is monotone increasing when C2/ Pa >
(1 0) 2 (1(
9 >
a - pı) (1,, Pi
-
P1
On the other hand if (s(,,0) is increasing, then
(1 0) (1, > (1, 11,
(20)
1 1-P1
= c1 +
C2 - C1 - P(1 - p1)
(21)
P2
(1 PI)
C2 P
(22)
P2
C2 02 Ci
2 (1 - p1)
(23)
P2 P2 PL
1 - P1
(c1P2 C2P1)
(24)
p2 (p2 P1)
2 0
(25)
where 23
follows from P < (02 - a)/ - p1). 24 re-arranges the terms, and 25
follows from the assumption that ei /Pi > p2/p2-
3. Note that (1) C2 + P(1 P2). We first show that routing model immediately is more cost-effective than routing to model I and escalating on failure:
P
C1
C2 + P(1 p2) < +(1 pı) p=== (1,
P2 P1
To show that J2(1) S (1 g (s))Vs e SH), we take the difference
J (s (l (s)) J2(1)
(26)
= [()( C2 P (S)) -
(27)
-(((, -a (c2 a) ((S((s, -m) + P(pz -p1)
(28)
(())) - 9
(s)) - p) (Si(s,)((s)) - pi) + (p2 - Pi - (c2 - c1)) (29)
20
(30)
where the inequality follows from Lemma 6
20

---

0. Published as a conference paper at ICLR 2026
Finally, to show J2( - (s, 0)Vs [1], note
P< C where the final inequality follows from the monotone increasing nature of J1 (8. 0).
C (1, (s, Vs [0,
a P(1 p1)
P1
Pi
C.3 PROOFS FOR SECTION 5
Proof of Proposition 1
1. If E2 > 0: then q (i. for all (i, by Theorem Moreover, (i*,s*)
(arg min: c2/p2), 0) from Theorem 3 egy, we can compare
To determine the user optimal routing strat-
E1 1
E25
E2
Sap P2
&1
Ui U2(0)=
P2
P1 + (1 Pi)s
P2
P1
p1)s
If this difference is positive, then (L, is user-optimal, or otherwise (2, 0). Thus, arg m U.(s, q (i, 8)) = (2 0) when E2/D2 > EI/PI, and is equal to (1,0) otherwise. Thus, Av is equal to 0 if and only if 62/p² > E1/1i and ei hppp > ca/p2 or the signs of both inequalities are reversed.
2. If f1, &2 O: then q (1, s) = for all (i, s) by Theorem Moreover, (i*,s") (1, P < (c2 c12 P1) and is equal to (2, 0) otherwise. To determine the user optimal routing strategy, we compare
U2(1) (1 1) = E2 - E11
meaning that arg max;,, Us(s, (i, 8) = (2 when &2 > &ι and is equal to (1,8) other- wise. Thus, Au is equal to if and only if &2 > & and < (e2 - o1) - p1) or the signs of both inequalities are reversed.
3. If sign(E) # sign(E2): We break this into two cases.
(a) E < 0 f From Theorem 2 q' (2 0 and q (1,,s) 1(s < 5o). To deter-
mine the user optimal routing strategy, we can compare (0) versus (2 9"' (1. First, for any s Sp:
U2(0) - (s, S1 + IP E
51 PI) s) + 1- E2s
P2
E3
+ (1 Pi)s > 0.
+ (1 Pı)s
Next for any so:
Sapı 13
U2(0) - Us
P2
p1 (1 Pı)s
Thus, Ay is equal to 0 and only (i*,8") = (2,0).
(b) & > > E2: From Theorem g (2, = 1 and q" (1, s) is determined by three regimes. Starting from Lemma
&1 + $200
{ (0,0)
(1 S (o)
1 a
P1
where the second inequality arises from substituting q = 0, = 0 into UT (a). Therefore, the user prefers routing to model without cascading among all choices that involve routing to model 1. Finally, we compare against model 2 to show Un (1,0,) 0) U2(1) ///1 ε2 > 0, meaning that the user utility is maximized overall by routing to model and not escalating. Therefore, A = 0 if and only if
(i", ("*)")) = (1.
21

---

Published as a conference paper at ICLR 2026
C.4 PROOFS FOR SECTION 6
Proof of Proposition 2 Before proving, we note that when P < min(er/p1, 02 we have the
following two identities. First,
(1 J1 Ci(s P(Si(s,9) -p1) -p1)
Ci(s,g) - - P) (8) p1)
(s, p1
2 0
where the inequality rises from Lemma
6 showing (Ci(s,q) - ()/(5((s,g) Pı) 2
minfer/P1: 02/p2} 2 Second,
Jz(4) J2(1) C2(g) 02 - P(S2(q) - -p2)
C2(g) C2 p\ (S2(g) P2)
S2(g) - P2
C2 -P P (8(g) 1)p2
P2
where the inequality again rises from Lemma 6
We now break the proof into two cases, each with two sub-cases.
1. The effect of throttling model
(a) When & 0: We must compare the effect on the provider-optimal cost between the settings E1, &2 > O, where we let pre be the optimal cost; and E2 > 0 > & where we let Jpost be the optimal cost. Then,
Jpost I mi,(J)(0,))))) 1), (s,
(31)
mi, J2(0)
(32)
min
Ci + P(1 - P1).
C2
(33)
p2
C1 C2
S min
(34)
Pi P2
- pre
(35)
where (3T) follows from Theorem
32 follows from the identity stated at the start
of the proof, (33) follows from substituting the the terms, and 34 follows from P <
c1/P1. Finally, (35) follows from Theorem 3
(b) When & < 0: We must compare the effect on the provider-optimal cost between the
settings & > 0 & and 0 > &2, Let Jire and Joost be the provider-optimal cost in the first and second setting, respectively. Then,
J min min((2()),,)))) JI (0,
(s,,
(36)
2 min Ji 1))
(37)
min {c2 + P(1 - p2), ci P(1 - pi)}
(38)
post where (36 follows from applying Theorem start of the proof, and (39) follows from Theorem
(39)
follows from the identity at the
2. The effect of throttling model
(a) When E 0: We must compare the effect of the provider-optimal cost between the settings Ei > 0 where we let J pre be the optimal cost; and & > 0 > & where we let Joost the optimal cost. Then,
J < min (0. < min (1n)) = Jire
22

---

Published as a conference paper at ICLR 2026
where the first inequality follows from applying Theorem 2 and the second inequality follows from Theorem 3
(b) When č < 0: We must compare the effect of the provider-optimal cost between the
settings & 0 & where we let Spre be the optimal cost; and E E2 0 where let post be the optimal cost. Then,
pre min (J1(0, J1 (50, 0), J2(0)) 2 (in 1),
min (0, 1), J2(1)) J post
where both inequalities draw on the identities presented at the start of the proof.
23


Reviews:
[{"review_id": "b1utVGkUOW", "reviewer": "Reviewer_M77r", "summary": "The paper’s goal is to characterize operation regime of LLMs where providers offer models of different capability (and hence difference prices) and the users aim at minimizing their costs while getting the best utility out of the model. Some other variables of interest are the provider’s desire to minimize their inference costs and the user’s desire to minimize the latency. The paper studies multiple strategies, namely, routing to the best possible model and cascading through small models first and then sending to the larger model if the smaller model’s response is not good enough. The paper then proves different statements about optimal routing / cascading policies given different capability and latency ratios between the two models.", "questions": "Please see questions under weakness 1.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1. The high level problem of learning optimal routing strategies is quite well motivated.\n2. The formalism is thorough, though it could be quite dense at some points.", "weaknesses": "1. In general, I have quite a few concerns about how realistic the whole setup is. The tasks that the paper defines seem a bit different from what LLM users encounter in practice. The paper should describe (i) Why is the monetary cost to the user is not modeled? (ii) The user churning with some predefined probability seems plausible. But during deployment, users mostly cannot check the accuracy on each individual example. So is the framework here meant to be more suitable for scenarios when users are in the model testing phase? (iii) Why would the users keep trying the same model M1 if it doesn’t respond correctly? (iv) Users generally test on a set of independent queries. If the performance is bad on multiple successive queries, users could abandon the provider. How are the dynamics over multiple queries modeled? (v) Why is $s$ a number between 0 and 1. Shouldn’t it always be 1? If M1 fails, shouldn’t the provider always cascade to M2? (vi) The cascading step seems to assume that the model provider knows if the model answered the prompt correctly. Why is this a realistic assumption? In reality, the model providers are not even supposed to see the user inputs due to privacy and IP reasons. (vii) The notion of “value” that the user derives from the model is quite vague. Is this a real number or simply 0/1 accuracy. If it is the former, why?\n\n2. On a similar note, in Section 6, if the service costs are so high, why can the provider simply not raise the price rather than throttling users (disgruntled users might never return)? Not clear why this would be a rational policy.\n\n3. The value of the dense formalism is not clear and the insights that we draw looks to be derivable from simpler analyses. The main idea seems to be to compare the ratio between the accuracy and latency. The paper correctly points out that when the accuracies are the same, cascading doesn’t make sense since it add unnecessary latency by putting the smaller model in front of the bigger model. The results seem quite intuitive when considering this tradeoff. For instance, about the insight in Line 88, of course we expect users to stay if the net value is positive. The difficulty in practice is that it is very difficult to judge in advance for an unseen data point if the model will provide positive values, e.g., of the summary of an article will be good enough. Similarly, the insight in line 85 also seems straightforward. The main trouble is that we cannot predict when one model provides better value than the other (see weakness 1).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ld2I5ig6fU", "reviewer": "Reviewer_d6sm", "summary": "Advances in AI systems raise interesting questions for model deployers --- for any given user query, what model should be presented to the user, when models have different capabilities and costs to querying? And with a proliferation of model deployers, users too have a choice --- which deployer to engage with? The authors lay out these questions and formalize the interplay between model routing and user participation as a game, to which they develop game-theoretic insights into how actors may behave under different cost settings and how such a game could be \"gamed\" by model deployers in potentially harmful ways to users.", "questions": "See above re: questions on interpreting figure 4 (where is the Model 1 dashed line?) \n\nI would be interested in the authors' extension to not just other models -- but the case where multiple model deployers are simultaneously competing for the same user.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "I very much enjoyed reading this paper! The authors' formalization of the problem of user and model deployer interaction as it relates to what model is served is a highly (and increasingly) important to today's consumer AI dynamic. I found the connection to game theory quite creative, and I learned quite a bit from reading the paper. The authors' novel insights about possible gameification that model deployers could engage in, given such a game (re throttling latency) are likely quite a valuable contribution in their own right. I appreciated that the authors were open about their limitations of their set-up of the problem as well. I imagine there could be a nice blossoming literature around this and related works to better understand and model people and deployers' choices in the setting of multi-model choice.", "weaknesses": "I found quite few weaknesses in the work; however, I may have missed something in the mathematics. As someone a bit weaker on the theory-side, I did find some of the theoretical discourse quite dense and a little convoluted -- especially section 3 (but this may be my own naivete --- indicated in my lower confidence score). \n\nAs noted above, the authors seem quite upfront on their limitations (I would be interested in settings where the user may not be aware of the routing policy or s). \n\nMore minor but important -- I found some of the visuals a little confusing. There are quite a few colors in Fig 2 but it is not clear what they relate to. It would be helpful if the caption spelled out the assignment of colors to differences in the kind of computation. Figure 3 also indicates that Model 1 is shown with dashed lines, but I can't seem to see where this is?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PHAHZaLlsg", "reviewer": "Reviewer_pynF", "summary": "This paper studies the interaction between a large language model (LLM) provider that routes tasks across multiple models and a user who can choose to reprompt or abandon tasks depending on model performance. The authors model this as a Stackelberg game between the provider and a user, where the provider decides a routing and cascading policy, and the user optimizes their abandonment policy based on perceived utility. The setup assumes two LLMs — a standard and a reasoning model — that differ in accuracy, latency, and cost. The paper provides a closed-form characterization of the equilibrium by first solving for the user’s best response (Theorems 1–2) and then deriving provider-optimal routing policies (Theorems 3–5). The results show that: (1) static routing without cascading is optimal in most regimes; (2) misalignment gaps arise when the provider’s cost-based model ranking differs from the user’s utility-based ranking (Section 5); provider throttling (i.e., intentionally increasing latency) can emerge as an equilibrium when user churn penalties are low (Proposition 2, Section 6). Empirically, the authors visualize user responses, provider policies, and misalignment gaps across parameter regimes (Figures 3–5) and provide intuitive insights on user patience, value-dominated vs. latency-dominated models, and welfare trade-offs.", "questions": "Extension to multiple models: How would the equilibrium generalize to $n>2$ models? Would the threshold structure persist or collapse into pairwise comparisons?\n\nDynamic user learning: The analysis assumes fixed $p_i$. How might the equilibrium change if users learn about model success probabilities over repeated interactions?\n\nAlternative objectives: Have the authors considered a bi-level optimization where providers internalize user utility as part of a long-term revenue function?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "Originality:The paper introduces a novel behavioral–economic perspective on LLM routing. While prior routing work (e.g., Chen et al., 2023; Ding et al., 2024; Hu et al., 2024) focuses on minimizing cost–latency trade-offs, this paper uniquely models strategic user response via a multi-round prompting game (Section 3). This Stackelberg formulation, with users as rational agents, represents a conceptual advance that bridges operations research and AI system design.\n\nQuality: The analysis is mathematically sound. The closed-form results (Sections 4.1–4.2) are carefully derived and supported by additional lemmas in the appendix.\n\nClarity: The exposition is clear and well-structured. Visuals (Figures 1–5) effectively summarize equilibrium regions and threshold rules. The notation is consistent. The intuitive explanations following each theorem (especially Theorem 2 and 5) help readability.\n\nSignificance: This work is relevant for LLM system design and AI governance. The analysis gives explicit conditions for misalignment and welfare loss, which is useful for researchers and policymakers analyzing these AI platforms.", "weaknesses": "Empirical validation: The work is entirely theoretical. While this is appropriate for a conceptual contribution, the claims about user patience and latency manipulation (Figure 5 Right) would benefit from empirical support, e.g., simulations or user–provider experiments.\n\nLimited model diversity: The analysis considers only two models (standard vs. reasoning). While the authors acknowledge this in the conclusion (Section 7), the extension to $n$ models could meaningfully affect equilibrium behavior—especially when users can select between several public endpoints.\n\nSimplifying behavioral assumptions: The model assumes users observe provider cascade probabilities and adopt stationary abandonment policies (Section 3.1). In practice, users may have incomplete information or adaptive patience. A discussion of bounded rationality or stochastic user beliefs could improve this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jVnxViqyLI", "reviewer": "Reviewer_QacU", "summary": "The paper studies how LLM providers route requests to different models (in this case, non-reasoning vs. reasoning) to balance performance, cost, and latency. It models this as a Stackelberg game between a cost-minimizing provider and a utility-maximizing user, who may re-prompt or abandon a task if it fails. The theoretical findings show that optimal routing is often a simple, static policy, but there could be a misalignment if the provider is incentivized to slow down responses in order to save costs.", "questions": "- What do you think of your formulation in terms of models that automatically decide their effort based on each task?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Instead of studying LLM routing in a vacuum, just from the perspective of the provider, it models the interaction as a Stackelberg game, which formally includes the reactive behavior of a user (who can re-prompt or abandon a task) in response to the provider's strategy.\n- It shows a misalignment between provider and user when providers decide to slow down their models to nudge users to reduce their use.\n- It derives simple and practical insights, like the fact that a static policy (routing to one model always) is better than cascading.\n- The writing and motivation are clear", "weaknesses": "I don’t see major weaknesses, but perhaps my main concern is how relevant these findings and formulation will be long term. The paper deals with two models, one more capable and slower than the other. However, it seems like the direction from the main LLM providers is moving towards models with different levels of thinking capabilities depending on the effort. There, the model is routing in a way, and I’m not sure the findings here apply directly since the providers and users don’t have the same amount of control.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)