Rerouting LLM Routers
Authors: Avital Shafran
Roei Schuster
Tom Ristenpart
Vitaly Shmatikov
Venue: COLM 2025 (2025)
Keywords: LLMs
Routers
Adversarial Machine Learning
ML Security
TL;DR: Proposing a novel class of vulnerabilities, where adversaries can manipulate LLM routing decisions to their advantage.

Abstract: LLM routers balance quality and cost of responding to queries by routing them to a cheaper or more expensive LLM depending on the query's estimated complexity.  Routers are a type of what we call ``LLM control planes,'' i.e., systems that orchestrate multiple LLMs.

In this paper, we investigate adversarial robustness of LLM control planes using routers as a concrete example.  We formulate LLM control-plane integrity as a distinct problem in AI safety, where the adversary's goal is to control the order or selection of LLMs employed to process users' queries.  We then demonstrate that it is possible to generate query-independent ``gadget'' strings that, when added to any query, cause routers to send this query to a strong LLM.  In contrast to conventional adversarial inputs, gadgets change the control flow but preserve or even improve the quality of outputs generated in response to adversarially modified queries.

We show that this attack is successful both in white-box and black-box settings against several open-source and commercial routers.  We also show that perplexity-based defenses fail, and investigate alternatives.

Published as a conference paper at COLM 2025
Rerouting LLM Routers
Avital Shafran
Roei Schuster Wild Moose New York, USA. roei@wildmooe ai
The Hebrew University Jerusalem, Israel. avital. avitall.haafra@@m. www.saa@maillla..mmmmmmmmmmmmamm huji ac. i1
Thomas Ristenpart Cornell Tech New York, USA. ristenpartecoreell.ede
Vitaly Shmatikov Cornell Tech New York, USA. Shmat@cs.cornellee
Abstract
LLM routers balance the cost and quality of responding to queries by routing them to a cheaper or more expensive LLM depending on the query's estimated complexity. Routers are a type of what call "LLM control planes," i.e., systems that orchestrate multiple LLMs.
In this paper, investigate adversarial robustness of LLM control planes using routers as a concrete example. We formulate LLM control-plane integrity distinct problem in Al safety, where the adversary's goal is to control the order or selection of LLMs employed to process users' queries. We then demonstrate that is possible "gadget" strings that, when added to any query, cause routers to send this query to strong L.M. In contrast to conventional adversarial inputs, gadgets change the control flow but preserve or even improve the quality outputs generated in response to adversarially modified queries.
generate query-independent
We show that this attack is successful both in white-box and black-box settings against several open-source and commercial routers. We also show that perplexity-based defenses can be evaded, and investigate alternatives.
1 Introduction
Large language models (LLMs) exhibit remarkable capabilities many tasks. Today, hundreds of open-source and proprietary LLMs are available at different prices, ranging from expensive, ttate-of-the-art models to cheaper, smaller, less capable ones. Operators of LLMs (especially higher-quality models) typically charge per query, imposing non-trivial costs on LLM-based applications and systems.
Developers who want to integrate LLMs into their applications must therefore consider both utility and cost, i.e, how to maximize the quality of responses while minimizing the cost. The two objectives conflict with each other: larger models tend to generate higher-quality answers but charge more per query. For example, at the time of this writing, CPT-3.5-turbe costs $1.5/$1.5 per 1M input/ output tokens, GPT-40-mini $0.15 $0.6, GPT-4o ol-preview $15/ $60. The difference in quality between models not uniform across queries. For some queries, even cheap model can generate an acceptable response. More complex queries require an expensive model to obtain a quality answer.
A natural solution to balancing performance and economic considerations is to take advan- tage of the availability of multiple LLMs different price-performance points. Recently proposed LLM routing systems (Šakota et al., 2024; Ong et al., 2024; Ding et al., 2024; Mar- tian; Unify) orchestrate two more LLMs and adaptively route each query to the cheapest LLM they deem likely to generate a response sufficient quality. the two-LLM case, let M, be expensive, high-quality model and Mn weaker, lower-grade one. Given query

---

Published as a conference paper at COLM 2025
9, the routing algorithm R(.) applies classifier to q that outputs if My is sufficient for answering 4, or 1 if M3 is required. The system then routes 9 accordingly.
LLM routing is example of a general class of systems we call LLM control planes, which orchestrate the use of multiple LLMs to process inputs, as further described in Section 2.
Our contributions. First, we introduce LLM control plane integrity, i.e., adversarial ro- bustness of inference flow (rather than inference outputs), as a novel problem in Al safety. Robustness of control-plane algorithms to adversarial queries is distinct from adversarial robustness of the LLMs they orchestrate. Attacks on LLMs aim to degrade their outputs. In contrast, control-plane attacks introduced in this paper aim to change the order or selection of LLMs employed by the system while producing the same or even better outputs.
Second, we show that existing LLM routing algorithms can be manipulated by malicious users. We design, implement, and evaluate a method that generates guery-independent adversarial token sequences which we call "confounder gadgets." " If a gadget is added to any query, this query is routed to the strong model with high probability. Next, we show that this attack is effective even in the transfer setting where the adversary does not have full knowledge of the target LLM router but has access to another router (e-g,, an internally trained surrogate). We also evaluate the integrity of commercial LLM routers, showing that they can be confounded as well.
While our gadgets act like universal adversarial examples against query classifiers used by LLM routers, their objectives are different. Adversarial examples simply aim to change the output of the classifier. Our gadgets need to change the output of the target system's 's router and also ensure that the system's 's response to the confounded query is the same or better than the response to the original query.
Third, we investigate defenses. Our basic method generates gadgets that have anomalously high perplexity and can be easily filtered out. This defense can be evaded by incorporating low-perplexity objective into gadget generation. We also discuss higher-level defenses, such as identifying users whose queries are routed to the strong model with abnormal frequency.
Routing attacks can be deployed for various adversarial objectives, e-g., to ensure that the adversary always obtains the highest-quality answer regardless of the target application's internal routing policies and cost constraints, or to maliciously inflate the target's LLM costs. They can affect other users, e.g., by wasting the target's s strong-model quota and causing their queries to be routed to the weak model. As LLM control planes grow in importance and sophistication, we hope to motivate further research on their adversarial robustness.
2 LLM Control Planes and Routing
Inference using large language models (LLMs) is traditionally monolithic: a single model is applied to an input or sequence of inputs. Today, this methodology can be sub-optimal. State- of-the-art models are often expensive, charging as much as several dollars per API query. Different LLMs may excel at different tasks, and selectively using an appropriate LLM may improve overall quality on a diverse workload. Finally, combining multiple LLMs, even if trained for similar tasks, may become increasingly prevalent as performance improvements of individual LLMs plateaus (Reuters, 2024; The Information, 2024; Bloomberg, 2024).
Researchers and practitioners are developing inference architectures that use multiple LLMs to answer queries. These LLMs are orchestrated by what we call an LLM control plane (borrowing the terminology from networking (IBM, 2024)). The control plane may route queries or parts of queries to different LLMs, derive new strings to query to underlying LLMs, combine answers from underlying LLMs, and more.
LLM routers. A prominent example of this emerging class of LLM control planes are LLM routers (Ding et al., 2024; Ong et al., 2024; Stripelis et al., 2024; Šakota et al., 2024; Lee et al., 2024). LLM routers decide which of the two (or, sometimes, more) LLMs to use to answer a query. In prescriptive routing, the router applies some lightweight classifier to the input
2

---

Published as a conference paper at COLM 2025
query that determines which underlying LLM to utilize for a response. The classifier is itself a learned function that scores the complexity of the query. Deployments can then configure score threshold for when to route query to the more expensive LLM. This threshold can be tuned using representativg tive workloads to achieve a desired cost-performance trade-off. Figure 4 shows the basic workflow of binary LLM routers.
-on-prescriptive routing (Chen et al., 2023; Aggarwal et al., 2023; Yue et al., 2024) picks response from those produced by multiple LLMs. For example, FrugalGPT (Chen et al., 2023) submits the query to a cascade of models (ordered by price), stopping when obtains a response classified by the router as sufficient.
In contrast to routers motivated by controlling costs, several LLM router designs focus solely on improving quality of responses (Shnitzer et al., 2023; Narayanan Hari & Thomson, 2023; Feng et al., 2024; Srivatsa et al., 2024).
The LLM routers described thus far do not modify the queries or individual LLM responses. Other types of control planes do. Ensemble approaches such as mixture-of-expert (MoE) (Du et al., 2022; Fedus al., 2022; Riquelme et al., 2021; Shazeer et al., 2016) architectures select subset of underlying models to apply to each token of query and merge their responses. LLM synthesis (Jiang et al., 2023b) architectures operate similarly, but route the entire query to subset of underlying LLMs and merge their responses. These approaches reduce inference costs by using fewer and/or or less complex underlying models.
A key use case for LLM routers is to help LLM-based applications reduce cost. Several com- mercial routers, including Unify (Unify), Martian (Martian), NotDiamond (NotDiamond), and others, offer this service. These services select the optimal LLM and forward the queries, apparently resulting in significant cost savings: up to 98% in the case of Martian (Martian), and 10x in the case of NotDiamond (NotDiamond).
2.1 LLM Control Plane Integrity
In this section, we provide a high-level definition of LLM control plane integrity. A formal and detailed discussion is provided in Appendix A.
An LLM control plane R is a potentially randomized algorithm, parameterized by string W. It utilizes some number и of LLMs denoted by M. We will mostly focus on the case of n = 2, and, for reasons that will be clear in a moment, use Mg ("strong") and Mg ("weak")) to denote the underlying LLMs. Given an input x, inference flow is the sequence of LLM invocations that process x. This flow is dictated by an inference flow policy that represents the control plane designer's 's intention regarding the of the underlying LLMs. For example, in the context of binary LLM routers, the policy might specify that the strong and expensive model is used at most an fraction of inferences.
A control plane integrity adversary seeks to maliciously guide the inference flow by overriding the intended inference flow policy for obtaining some adversarial goal. Informally, control plane integrity means that decisions made by control plane algorithms cannot be subverted by adversarial queries. We focus on predictive LLM routing as a concrete example of control plane used in real-world L.M--based systems to manage inference cost.
Threat models. An adversary may seek to inflate the costs of victim application that utilizes an LLM control plane for cost management. Another adversarial goal is arbitrage. Consider an application that charges X dollars per query, whereas directly using Mg costs Y > X. This makes economic sense if the bulk of queries are routed internally to cheaper model My. An input adaptation attack in this setting can gain (indirect) access to Ma, obtaining an arbitrage advantage of Y X per query. To be effective, adaptations should not lower response quality (i.e., extract all value out of rerouting to Mg).
A lot of prior research on adversarial ML focused on attacks like jailbreaking and poisoning that impact the end users of LLMs. In our setting, the primary victims are applications that employ vulnerable LLM control planes. Furthermore, their users will be affected, too, if their queries are unfairly routed to the weak model. The adversary, on the other hand, gains unfair access to the high-quality LLM.
3

---

Published as a conference paper at COLM 2025
Apeliaaton
Apaliaation
$54
Queriies
Strong Model Provider
Queeries
Strons Model Provider
adverrsry
Router
Rowter
Wank Madel Provider
Weok
Resgortes
Provider
Figure 1: Overview of our attack on LLM control plane integrity. The attack adds each query a prefix (represented by the gear), called a "confounder gadget," that causes the router to send the query to the strong model.
We assume that the victim application's prompt includes a substring that can be controlled by the adversary. For example, chatbots, coding assistants, writing assistants, and other LLM-based applications include user inputs into their prompt. We consider several levels of knowledge that an adversary may have about the victim application's internal router.
In the white-box case, the adversary knows the control plane algorithm and its parameters w. In the black-box (transfer) case, the adversary does not know the control plane algorithm R and w for the target model, but knows instead another control plane algorithm R and its parameters. We refer to R the surrogate. For example, this could arise if an adversary trains their own router. In this setting attacks are also zero-shot in that they do not require any prior interaction with the target control plane.
3 Confounding Control Planes with Gadgets
Control plane attacks modify queries to mislead or "confound" the routing logic into routing these queries to an LLM of the adversary's s choosing. We focus on the binary router setting in which the router applies learned parameterized scoring function Sa(-), where 0 refers to the parameters, and routes any query whose score exceeds some threshold T to the strong LLM Mg. This setting has been the focus of (Ong et al., 2024; Ding et al., 2024; Lee et al., 2024) and is deployed in practice (see Section 6). We provide a more formal definition in Appendix B. See Figure 1 for depiction of our attack setting.
Confounder gadgets. Our approach works as follows. Given a query x,, we prepend confounder gadget c, which short sequence of adversarially chosen tokens. The modified query is 21 = qill;; where || denotes string concatenation. Intuitively, we will use optimization search for confounders that trick the scoring function into ranking & as sufficiently complex to require the strong model.
In the white-box, query-specific setting, we can choose c; function of x; and the known parameters w = (5,0, r). To avoid searching a confounder for every query, we focus instead on the query-inneependen setting, where single confounder can be prepended all queries.
We begin by fixing confounder length of и tokens and let I be a token dictionary (it should be a sufficiently large subset of the token dictionary used by 5). Then we set the gadget to initially be n tokens all fixed to the same value from I. The exact choice of the initialization token is not important; in our implementation, we used the first token in the dictionary (!!).
Denote this initial confounder ( (c(), ,
(0) [(.
Then, we perform a hill-climbing style approach to find a good confounder. For each iteration & [T), where T is the total number of iterations, do the following:
(1) Select a target index j E [1,m] uniformly.
(2) Generate a set B of + candidates. First set & = c(,), the current confounder. To generate B additional candidates, select replacement tokens from I uniformly, forming
4

---

Published as a conference paper at COLM 2025
the set {tp k- I) b- Replace the jth token in the current confounder èg with by, forming 6 (c ...., (8)
9-1 tou jt1 [t)] Let B {čo. (Co...., EB}
(3) Find the candidate that maximizes the score: c(++1) < arg cax Sa(c).
The final confounder c(T) can be used with any query x; by prepending to obtain f; = c| We early abort if, after 25 iterations, there is no update to the confounder gadget. Technically, we could abort early if we find a confounder whose score exceeds T. Running further can be useful when adversary does not know
The attack's runtime is dominated by T times the cost of executing S. In practice, S are designed to be fast (otherwise routers would significantly increase the latency of applications that use them). We report precise timings later; in summary, the attack is fast because we can set T to be relatively small and still find high-scoring confounders.
Due to the randomness in index and token selection, the method converges to different, yet similarly effective, confounder gadgets each run. Our evaluation will thus measure average performance over multiple gadgets.
The black-box setting: confounders that transfer. Finally, the attacks far are in the white-box setting, where the attacker can optimize directly against Sg. While in some cases routing control planes will be public knowledge, in others, including the proprietary control planes we explore in Section 6, they are hidden. This gives rise to the black-box setting. While an attacker might seek to perform model extraction attacks (Tramèr al., 2016; Lowd & Meek, 2005) to learn 0, we instead explore attacks that transfer from one router to another.
We assume the adversary has access to surrogale router Ryy, that is trained on data similar to that used for the target router. Then the attack is the same as above, except that we use the surrogate's ' scoring function S/, instead of the target's Sg. Again, we will see that query-independent confounders found for the surrogate transfer well to other routers.
We have experimented with variations of this approach that don't work quite as well, for example adding c as a suffix instead of a prefix. See Appendix I for details.
4 Open-Source Routers: Experimental Setup
This section explains our experimental setup (summarized in Figure 3). We provide more details in Appendix D. In all experiments, we assume that the adversary's goal is to reroute queries to the strong model. In Appendix L, we evaluate rerouting to the weak model.
Target routers. We evaluate the four prescriptive routing algorithms from Ong et al. (2024), which provides open-source code and trained parameters for a representative variety of approaches: similarity-based classification (Stripelis et al., 2024; Lee et al., 2024), an MLP constructed via matrix factorization (Stripelis et al., 2024), BERT-based classification (Ding et al., 2024; Stripelis et al., 2024; Šakota et al., 2024), and fine-tuned LLM.
Underlying LLMs. Ong et al. (2024) trained the routers with GPT-4-116-ppreveer (Achiam et al., 2023) as the strong model and Mixtral 8x7B (Jiang et al., 2024) as the weak model. They report successful generalization between the underlying LLMs, stating that their routers trained for a particular strong-weal LLM pair can be used with other pairs as well.
To allow our evaluation to scale, we use as the strong model Mg the open-sourced Llama- 3.1-8B (Meta, 2024b) and as M, the 4-bit quantized version of Mixtral 8x7B (for efficiency reasons). This reduced the cost of our experiments by avoiding expensive GPT API calls and lowering the computational costs of Mixtral. Unless mentioned otherwise, all of our results are for this router pair (LLM Pair 1). In Appendix M.2, we perform less extensive experiments with the original strong/ weak model pair (LLM pair 4). We additionally performed experiments for the case where the weak model produces much worse responses than the strong model see Appendix M.
5

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Upgrade
Strong
Upgrade
Strong
Upgrade
Strong
Rsw
100 0
100
90 =1
94
98 0
-
RMF
100
+ 100 =
78
4 90 +
100
54 4 100 =
RCLS
100 =
67 7 100 +
=0
47 7 100 +
100 =0
56 4 100 +
RLLM
73 5
57 > 88 + 9
53 >
98 1
94 3
53 + 97 1
Table The white-box attack's rerouting success rate. "Upgrade" the percentage of "Weak" queries successfully rerouted to the strong model by adding confounder gadget. "Strong" the change in the percentage of queries that are routed to the strong LLM Mg.
Evaluation datasets. We will evaluate our attacks using three standard LLM benchmarks as workloads: MT-Bench (Zheng al., 2023), dataset of 160 open-ended questions, MMLU (Hendrycks et al., 2021), dataset of 14,042 multi-choice questions, and GSM8K (Cobbe et al., 2021), dataset of 1,319 scade-school math problems. We provide more details about benchmark evaluation in Appendix D.
Router calibration. For each workload, we must calibrate each router by setting the threshold T to achieve some target fraction e of queries routed to the strong model. For our experiments here, we set c 0.5, meaning the goal is have about half the queries routed to the strong model. We provide more details about calibration in Appendix D and evaluate other values of e in Appendix E.
=
Rerouting Open-Source Routers
Unless otherwise specified, our evaluation focuses on the setting where the attacker first finds fixed, query-independent set of gadgets and uses them to attack all queries. In Ap- pendix N, we evaluate query-specific gadgets which are more effective but also more expen- sive generate. In Appendix J, we evaluate optimization-fre alternatives for generating confounder gadgets. We discuss defense mechanisms in Section 7.
White-box confounder gadget generation. Following our attack framework described in Section 3, we construct query-independent control-plane gadget designed to confuse each router. For the white-box setting, we set batch size B = 32 and iteration number T 100, ignoring thresholds. We generate four sets of и = 10 gadgets per router (see Appendix C for examples). We discuss runtime and optimization convergence in Appendix F.
When reporting the scores below, we average over n gadgets used with all 72 MT-bench queries, and sets of 100 randomly selected queries from MMLU and GSMSK. None of these test queries were used in the training of the routers or their calibration.
Rerouting success rates. We measure the percentage of upgraded queries, ie., queries that were originally routed to the weak model and after being modified with the confounder gadget, were rerouted to the strong model. Table 1 shows that our attack successfully reroutes almost all weak queries to the strong model. No queries are "downgraded, " i.e., rerouted from the strong to weak model. We also show the increase in the percentage of queries routed to the strong model before and after modifying queries with our gadgets.
Quality of responses. A successful gadget must reroute queries to the strong model while preserving or improving quality of responses to the modified query. We use perplexity and benchmark-specific scores to evaluate response quality. Table 2 shows that for LLM pair 1, average response perplexity does not significantly change. To the extent that it does, it usually somewhat decreases, indicating more "natural" responses. Table shows that responses to confounded queries' average scores (1-10 quality scores for MT-bench, % correct MMLU / /GSM8K) are typically no worse (sometimes better) than responses to the original queries. We provide full details and further discussion in Appendix G.

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Original
Confounded
Original
Confounded
Original
Confounded
Rsw
13.8
122
20.4
20.1 0.1
17.1
1.3
MM
12.6
12.3 0.2
20.0
20.3 = 0.1
17.0
15.2
RCLS
13.1
12.1 + 0.2
20.2
20.5 = 0.1
17.0
15.0 0.2
RLLM
12.7
12.7 + 0.4
21.0
19.6 = 0.1
16.4
15.2 + 0.3
Table 2: Average perplexity of responses to the original and confounded queries, in the white-box setting for LLM pair 1.
MT-Bench
MMLU
GSM8K
Original
Confounded
Original
Confounded
Original
Confounded
ssw
8.4
8.3 + 0.0
61
66 + 0
46
64 +
R MF
8.4
8.4
64
64 +
50
67 +
RCLS
8.4
8.3 +.0
63
65
50
63 +
RILM
8.3
8.2
67
66 =
64 +
Table 3: Average benchmark-speccffcc scores of responses to the original and confounded queries, in the white-box setting for LLM pair 1.
Black-box attack results. Consider an attacker that does not know the algorithm used by the target router but has access to surrogate router to mount cansferability-baset attack. We evaluate all combinations of surrogate and target routers. Upgrade rates are lower than in the white-box setting, with an average of 80%, 66%, 71,, and 62% for Rsw, RMF, RCLS, and RLLM respectively, but still high, i.e., the attack transfers. The LLM-based algorithm RLLM has the lowest rates, perhaps because is the most complex of the four. The downgrade rate is 1.2% on average and 0% in most cases.
We additionally evaluate the response quality and show that as in the white-box setting, the attack does not increase the average perplexity for LLM pair 1 nor does it decrease the average benchmark-speciffc scores, most cases. We provide full results in Appendix H.
6 Rerouting Commercial Routers
We evaluate our rerouting attack on several commercial routers: Unify (Unify), NotDiamond (NotDiamond), OpenRouter (OpenRouter), and Martian (Martian). As these are only available via black-box APIs, we use our black-box attack with the 40 gadgets optimized for the open-sourced routers Rsw, RMF, RCLS, and RLLM (10 per router). We evaluate using the MT-bench benchmark.
Unify. This router lets users specify a list of models from different providers and metric configuration for routing decisions. The available metrics are quality, time to first token, inter-token latency, and cost. We configure the router to choose between GPT-4o (Open 2024b) as the strong model and Mixtral 8x7B as the weak model. We manually calibrate the router such that 49% of the original, unmodified queries being routed to the strong model and 51% to the weak model, resulting in a total cost of $0.13 for the 72 MT-bench queries.
Adding confounder gadgets generated for the four open-source routers results in upgrade rates of 79%, 88%, 91%, and 89,, respectively, averaged across 10 gadgets. Downgrade rates are all zero. Average cost across the 10 gadgets increased to $0.22, $0.23, $0.22, and $0.21, respectively reflecting an average cost increase by a factor of 1.7x.
NotDiamond. This router lets users route their queries to a list of predefined models. Available objectives are to maximize quality, or balance quality and cost, or balance quality and latency. Exact details of routing logic are not specified. We focus on "cost-aware" routing, for which the does state "NonDiamond will automatically determine when a query is simple enough to cheaper model without degrading the quality of the response."
7

---

Published as a conference paper at COLM 2025
Again use GPT-4o (strong) and Mixtral-8x7b (weak). router sends 18% of the original queries to the weak model. Our gadgets for Rsw, RMP, RCis, and RIIM upgrade the LLM at rates of 21%, 18%, 21% 11, and 15%, respectively, Downgrade rates are 1-3%.
To begin with, NotDiamond aggressively routes to the stronger model even for unmodified queries, which may be the reason the attack's success rate is not as high as in other settings. We explored using NotDiamond with various other LLM pairs, and observed similarly unbalanced routing and similar upgrade and downgrade rates.
This API offers routing between Llama-3-70b (weakest, cheapest), Claude- 3.5-Sonnet (middle option), and GPT-4o (strongest, most expensive). Queries are routed "depending on their size, subject, and complexity."
OpenRouter.
With OpenRouter, 96% of the original queries are routed to the weaker Llama, 4% to the stronger GPT, and none to Claude. The total cost for all original queries is $0.03. After adding confounder gadgets, queries originally routed to Llama (96% of queries) present an interesting pattern: for some gadgets, all of those queries are rerouted to GPT, and for other gadgets, there is no impact whatsoever. The universally effective gadgets are generated at roughly similar rates across most of the attacker's 's surrogate routers (20%-40% of gadgets generated using Rsw, RMF, RLLM, 0% using RCLS). Using any of the universally effective gadgets increases the overall cost by 8x, to an average of $0.25.
Martian. This router is supposed to let the user provide list of models and to specify the maximum amount the user willing to pay for query or for 1M tokens. Unfortunately, as of November, 2024, the router appears to forward queries to the same LLM regardless of the user's list. We notified Martian about this behavior and excluded it from our evaluation.
7 Defenses
Perplexity-based filtering. P'erplexity a measure of how "natural" the text looks (see 5), and it can be used to detect adversarial manipulation (Jain et al., 2023; Alon & Kamfonas, 2023). In this defense, we assume the defender has access to a set of unmodified queries. The defender computes their perplexity to establish baseline threshold; new queries whose perplexity exceed the threshold (i.e. are abnormally " "unnatural") are flagged as adversarial.
To evaluate the efficacy of this defense, we compare the perplexity values of the original GSMSK queries and queries confounded with one randomly selected gadget produced using Rsw. Perplexity is calculated using GP'T-2. Figure 2 presents histograms of perplexity values of original vs. confounded queries, and the ROC curve for the defense that uses a perplexity threshold to distinguish between original and confounded queries. These results indicate that confounded queries are readily distinguishable from the original queries using perplexity values (ROCAUC*1). Results are similar for other gadgets and benchmarks, which we omitted due to space constraints.
Unfortunately, this defense can be evaded if an adversary incorporates a perplexity con- straint into the gadget generation process, such that it maximizes the score of the routing algorithm R and simultaneously aligns the gadget's perplexity to some predefined value. We find that this adaptive attack attains comparable success rates to the original attack, but renders perplexity values of confounded queries similar those of original queries, with distinguisher's ROCAUC values of 0.5-0.7. Appendix O provides full details of this adaptive attack and its evaluation.
LLM-based filtering. Even though adversarially modified queries cannot be easily detected using perplexity, they may still be "unnatural." possible defense is employ an oracle LLM to determine if the query is natural or not. This defense requires the router to invoke an additional LLM for every processed query, which is computationally expensive. In fact, this would undermine the very purpose of routing, which is to save costs (Section 1).
Ihttps:// htt.://rra..tt....aooporrrrr//aa
8

---

Published as a conference paper at COLM 2025
Original Contoundes
80
62
0.0
20
05
AJC 1.00
0
100
150 Perplexity
250
250
0.
62
24
1.
FPR
(a) Rsw
(b) Rsw
Figure 2: Perplexity the original queries in the GSMSK benchmark compared to the perplexity of confounded queries using single uniformly sampled gadget. We additionally present the ROC curve of the defense that detects confounded queries by checking they cross a perplexity threshold, and it's corresponding ROCAUC score. Confounded queries have significantly higher perplexity values, and are thus easy to recognize and filter out.
MI-Bench N = 2
MMLU N
GSM8K N = 2
N = 1
==
=
N 3
N N=1
N =
Rsw
80 10 4
96 2
91 33
68 44
85
7
84
97 =1
95 +
M
81 4
99 1
97 +1
66 +3
+1
82 33
86 +
98 +0
96 + 1
RCLS
78 5
22
89 4
5
84
69 45
+
91
+
83
+
L
70
92 2
98 +
52 +
+5
48 +8
4
87
+
78
+
Table 4: Average upgrade rate when evaluating against N + routers, including the router used for optimization (represented by the left column). For a given value of N all possible combinations were evaluated and averaged. Attack effectiveness decreases in comparison to the standard white-box setting in Table yet significant upgrade rate persists.
Furthermore, is possible optimize gadgets so that they both have low perplexity and appear "natural" to LLM evaluators (Zhang et al., 2024a).
Paraphrasing. We can consider "active" defense that paraphrases queries using oracle LLM before dispatching them to the router. The paraphrased query might not contain the original gadget, and might not confound the router. This defense is likely impractical. First, as with LLM-based filtering, it requires an expensive LLM invocation for each query. Second, may degrade the quality of LLM responses, which are sensitive to phrasing.
Detecting anomalous user workloads. Another possible defense requires the router to monitor individual user workloads, identify users whose queries are routed the strongest model with an abnormally high frequency, and impose user-specific thresholds dynamically calibrated to route consistent fraction of queries to the strong model. Such sser-specific routing would complicate implementations and make inaccurate decisions for user until there is sufficient data about their queries. The defense could still be circumvented in settings where attackers can create large number of Sybil users.
Multiple routers. Another possible defense can utilize multiple routers by making the routing decision based on the majority vote of these routers. We evaluate this defense in the white-box setting, where we use gadgets optimized against a single router and the routing decision is based on the router used for optimization and N other routers (where N ranges from to 3, given 4 total routers), testing all combinations. Table shows that this method decreases attack effectiveness, yet significant upgrade (i.e., successful attack) rate persists.
As a possible countermeasure, we evaluate the effect of optimizing the gadgets against M routers. The optimization objective in this case is to maximize the average score, with M

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU 3
GSM8K
= =
= 4
M=
M 4 M =
M
M =4
85 2 100 100 0 74
91 #1
88
90
99
99 +
Table 5: Average upgrade rate when optimizing against M > 1 routers and evaluating using M routers. For given value of M all possible combinations were evaluated and averaged. This degrades the performance of the multi-router defense although at the cost of slower gadget generation
ranging from to 4. Table 5 indicates that optimizing against multiple routers degrades the performance of using N routers as defense mechanism, although at the cost of slower gadget generation.
Related Work
There is a large body of research on adversarial examples Goodfellow al. (2015); Papernot et al. (2016; 2017). For example, HotFlip Ebrahimi et al. (2017) is an early attack against text classifiers that employed token-swapping optimization similar to ours. More recent prompt injection adversarial examples specifically target LLMs to extract information or bypass safety guardrails Liu et al. (2023); Schulhoff et al. (2023); Zou et al. (2023); Wei et al. (2023); Zhu et al. (2023); Chu et al. (2024). As explained in Section 1, the purpose of adversarial examples our case to change the control flow while maintaining or improving the quality of the system's responses. See Appendix P for survey of other attacks against LLM-based systems.
Conclusion
LLM routers balance quality and cost of inference by routing different queries to different LLMs. They are an example of broader, emerging class systems we call "LLM control planes" that orchestrate multiple LLMs to respond to queries.
We introduced and defined new safety property, LLM control plane integrity. Informally, this property holds if an adversarial user cannot influence the routing decisions made by the control plane. To show that existing LLM routers do not satisfy this property, we designed, implemented, and evaluated black-box optimization method for generating query-independent "confounder gadgets." " When added to any query, the gadget confuses the router into routing the query to the adversary-chosen LLM, yet preserves the quality of that LLM's response to the query.
We demonstrated the efficacy of confounder gadgets multiple open-source and commer- cial LLM routers, discussed defenses, and indicated directions for future research.
Acknowledgments
This research was supported in part by the Google Cyber NYC Institutional Research Program, the Israel Science Foundation (Grant No. 1336/22) and the European Union (ERC, FTRC, 101043243). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them.
10

---

Published as a conference paper at COLM 2025
Ethics Statement
As LLMs approach the limitations of scaling laws, researchers and practitioners are adopting inference architectures that leverage multiple LLMs response user queries. It is important to proactively identify potential security vulnerabilities in these architectures.
Our work initiates the study of integrity vulnerabilities in multi-LLM orchestrations, how they can be exploited, and how these attacks can be mitigated. The purpose of this work is to motivate research on protecting control-flow integrity in multi-LLLM systems and help the community design more robust and trustworthy multi-LLM systems.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report. arXiv preprint ar arXiv:2303.08774, 2023.
Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, et al. Automix: Automatically mixing language models. arXiv preprint arXiv:2310.12963 2023.
Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity,
arxiv preprint Xitx:2308.14132, 2023.
Bloomberg. OpenAl, Google and Anthropic are struggling to build more https://ww/ http:///wwwwbbooobrreee bloomberg....momo comm//eew//artiile///00---1-1111 openai-googles ant.aadttrropiicreeeeettt-e- CrGXSfHu, 2024. Published: 2024-11-13.
advanced AL.
to-build-more advanced -iceeedd-iiiae ai? ?sref=
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4), 1952.
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan
Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, al. Stealing part of production language model. ar preprint ar Xii::2403306633 2024.
Harsh Chaudhari, Giorgio Severi, John Abascal, Matthew Jagielski, Christopher A Choquette-Choo, Milad NasI, Cristina Nita-Rotaru, and Alina Oprea. Phantom: General trigger attacks on retrieval augmented language generation. arXiv preprint arXiv:2405.2045 2024.
Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05172 2023.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. Chatbot arena: An open platform for evaluating LLMs by human preference. In Forty-first International Conference on Machine Learning (ICML), 2024.
Sukmin Cho, Soyeong Jeong, Jeongyeon Seo, Taeho Hwang, and Jong C Park. Typos that broke the RAG's back: Genetic attack RAG pipeline by simulating documents in the wild via low-level perturbations. ar Xie preprint ar Xiv:2404.111 13948, 2024.
Junjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen, Michael Backes, and Yang Zhang, Com- prehensive assessment of jailbreak attacks against L.MMs. arxiv preprint ar arXin:2402.0556 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichrr Nakano, et al. Training verifiers to solve math word problems. Xix preprint ar xxXo:21100.4163 2021.
11

---

Published as a conference paper at COLM 2025
Nilesh Dalvi, Pedro Domingos, Mausam, Sumit Sanghai, and Deepak Verma. Adversarial classification. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, 2004.
Jacob Devlin, Ming-War Wing-WWeiee Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019.
Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. Hybrid LLM: Cost-efficient and quality-aware query routing. In Infernational Conference on Learning Representations (ICLR), 2024.
Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. How robust is Google's Bard to adversarial image attacks? arxiv preprint (xXii:2309.115:: 2023.
Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. Glam: Efficient scaling of language models with mixture-of-experts In International Conference Machine Learning (ICML), 2022.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. arXiv preprint ar Xr1::1772.06751, 2017.
Hugging Face. Chatbot Arena LLM Leaderboard: Community-drivee evalua-
tion for best LLM and AI chatbots. https://huggingfaant co/spaces/Lmarena-aii chatbot-arens arrna-leaderboard, 2024. Accessed: 2024-11-14.
William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion
parameter models with simple and efficient sparsity. Journal of Machine Learning Research (JMLR), 2022.
Tao Feng, Yanzhen Shen, and Jiaxuan You. Graphrouter: A graph-based router for LLM selections. ar preprint arXiv:2410003834 2024.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you' ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. In ACM AlSec, 2023.
Jamie Hayes, Tlia Shumailov, and Itay Yona. Buffer overflow in mixture of experts. Xiv preprint ar Xiv:2402.05526, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), 2021.
IBM. What is control plane? https://www. http://wwwwwb...ooo/tm..//////cs Published: 2024-10-31.
2024.
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping- yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023.
Frederick Jelinek. Interpolated estimation of Markov source parameters from sparse data. 1980. URL https: / /api. semanticscholar /t10///rpuSD::6112000 2010.
12

---

Published as a conference paper at COLM 2025
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7B. arXiv preprint ar XrXio:2310.06825 2023a.
Albert Q Jiang, rlexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, al. Mixtral of experts. ar preprint arXio:2401.04085 2024.
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-Blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023b.
Chia-Hsuan Lee, Hao Cheng, and Mari Ostendorf. OrchestraLLM: Efficient orchestration of
language models for dialogue state tracking. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024.
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Kailong Wang, and Yang Liu. Jailbreaking ChatGITT via prompt engineering: An empirical study. arXiv preprint ar Xiv:2305.11860, 2023.
Daniel Lowd and Christopher Meek. Adversarial learnnnng In ACM International Conference on Knowledge Díscovery in Data Mining (SIGKDD), 2005.
Martian. Martian LLM router. https://withmarr com/
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference Learning Representations (ICLR), 2016.
Meta. Introducing Meta Llama 3: The most capable openly available LLM to date. https:
//ai, meta. ceaeta..co/bbbi
11amat/3 2024a. Published: 2024-04-18.
Meta. Introducing Llama 3.1: Our most capable models to date. https: http://aaaamaaa...eeee www.a.commmm/......mmmmmm... blog/meta- 11ama-1----------- 3-1/, 2024b. Published: 2024-07-23.
Surya Narayanan Hari and Matt Thomson. Tryage: Real-time, intelligent routing of user prompts to large language models. arXiv e-prints, 2023.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo Eric Wallace, Florian Tramer, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXii:2311.17035, 2023.
NotDiamond. Notdiamond LLM router. https://ww.nnotdiammos ai/.
Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and lon Stoica. RouteLLM: Learning to route LLMs with preference data. ar Xiv preprint Xix:::240 18665, 2024.
OpenAI. New embedding models and API updates. https:/ www. com/index/ 2024a. Published: 2024-01-25.
new new-eebbedinng-modee -ngp:dddinn--mde--------------a-
OpenAl. Hello gpt-4o. http:////ppenaaa.cmmo//eee/eeeee
2024b. Published: 2024-
05-23.
OpenRouter. Openrouter LLM router. https::/openroouter .ii.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In IEEE European symposium on security and privacy (EuroS&P), 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-bux attacks against machine learning. In Proceedings of the 2017 ACM Asia conference computer and communications security, 2017.
13

---

models/ Published as a conference paper at COLM 2025
Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. In NeurIPS ML Safety Workshop, 2022.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. https://cdn /ttps://ccd.oppeni. 2019.
openai com/ .dff,
unsupervised.mllitssteee
Reuters. OpenAl and others seek new path to smarter AT as current methods hit limitations. https://www.reuterrs. sennai-rvval--see-nnee--- smarter -at.rrrrr--....rrr.ttttttttaaaa- 2024. Published: 2024-11-15.
htpps://wwww
reuters.eeeeeeee com/
2024-11-11,,
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Marija Šakota, Maxime Peyrard, and Robert West. Fly-swat cannon? cost-effective language model choice via meta-modeling. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, 2024.
Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, and Jordan Boyd- Graber. Ignore this title and HackAPrompt Exposing systemic vulnerabilities of LLMs through global prompt hacking competition. in EMNLP, 2023.
Avital Shafran, Roei Schuster, and Vitaly Shmatikov. Machine against the RAG: Jamming retrieval-uugmented generation with blocker documents. arviv preprint (Xiv:2406.05870 2024.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture- of-experts layer. In International Conference on Learning Representations, 2016.
Tal Shnitzer, Anthony Ou, Mírian Silva, Kate Soule, Yuekai Sun, Justin Solomon, Neil Thompson, and Mikhail Yurochkin. Large language model routing with benchmark datasets. Xr preprint ar XrXt::2309.1570 2023.
KV Srivatsa, Kaushal Kumar Maurya, and Ekaterina Kochmar. Harnessing the power of multiple minds: Lessons learned from LLM routing. ar preprint arXiv:2405.0046) 2024.
Dimitris Stripelis, Zijian Hu, Jipeng Zhang, Zhaozhuo Xu, Alay Shah, Han Jin, Yuhang Yao, Salman Avestimehr, and Chaoyang He. Tensoropera router: A multi-model router for efficient L.M inference. arXiv preprint ar trX:::2408.1222 123300 2024.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dunitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.61199, 2013.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: family of highly capable multimodal models. Xrv preprint ar XrXiv:2312.11855, 2023.
The Information.
OpenAI
shifts
strategy as rate of 'GPT" AI im- https://www. theinformation. com/articles/
provements slows.
2024. Published:
2024-11-9.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et Llama Open foundation and fine-tuned chat models. Xrv preprint Xrx::22307 (tto23077.0988, 09288, 2023.
14

---

Published as a conference paper at COLM 2025
Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, et al. Tensor Trust: Interpretable prompt injection attacks from an online game. In International Conference on Learning Representations (ICLR), 2023.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction APIs. In USENIX Security Symposium, 2016.
Unify. Unify LLM router. https://unii ai//
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safety training fail? In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Itay Yona, Ilia Shumailov, Jamie Hayes, and Nicholas Carlini. Stealing user prompts from mixture of experts. arviv preprint ar arXiv:2410.22884, 2024.
Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture thought representations for cost-efficient reasoning. In International Conference on Learning Representations (ICLR), 2024.
Collin Zhang, Tingwei Zhang, and Vitaly Shmatikov. Adversarial decoding: Generating readable documents for adversarial objectives. ar preprint drXi::2410.02163, 2024a.
Yiming Zhang, Nicholas Carlini, and Daphne Ippolito. Effective prompt extraction from language models. In First Conference Language Modeling, 2024b.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging LLM-as-a-judge with MT-Bench and chatbot arena. Advances Neural Information Processing Systems (NeurIPS), 2023.
Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. AutoDAN: Automatic and interpretable adversarial attacks large language models. arXiv preprint ar axXic:2310.15140, 2023.
Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint 1iXi::230771504: 2023.
15

---

Published as a conference paper at COLM 2025
Routers
Notation
weeigte ranking
Rsw
Matrix factorization
RMF
BERT classifier
RCLS
LLM scoring
RLLM
LLM pair
Strong (M,)
Weak (M.)
Llama-3.1-8B
4-bit Mixtral 8x7B
2
Llama-3.1-8B
Desstall-B-Instruct-vve
3
Llama-3.1-88
Llama-2-7B-chat-hf
4
GPT-4-1106-prees
4-bit Mixtral 8x7B
Benchmark
Description
MT-Bench (Zheng et al., 2023)
160 open-ended questions
MMLU (Hendrycks et al., 2021)
14,042 multi-choice questions
GSMSK (Cobbe et al., 2021)
1,319 grade-school math problems
Figure 3: Summary of our setup for routers, underlying LLMs, and benchmark datasets used in the experiments.
A LLM Control Plane Integrity
In Section 2.1 we provided high-level definition of LLM control planes, inference flows, and the control plane integrity adversary. In this section we provide a more formal and extensive discussion of these definitions.
Formalizing control planes. An LLM control plane R is a potentially randomized algo- rithm, parameterized by a string called the parameters. It utilizes some number n of LLMs denoted by M. We will mostly focus on the case = 2, and, for reasons that will be clear in moment, use Ms ("strong") and M. ("weak")) to denote the underlying LLMs. Then inference on an input x E X for some set X' of allowed queries is performed by computing +-s R.M (x). Here we use f- to denote running R with fresh random coins; we use + when is deterministic. We focus inference for single query, but it is straightforward to extend our abstraction to sessions: the controller would maintain state across inv invocations, potentially adapting its behavior as function of a sequence of queries and responses.
LLM control planes should, in general, be computationally lightweight, at least compared to the underlying LLMs. When control plane deployed to reduce inference costs by using a cheaper L.LM for some queries, an expensive control plane would eat into the savings. For example, predictive binary LLM routers use relatively simple classifiers determine whether to use Mg or M, to respond to query.
Router
Strang Model
Queres
VeS
Resporses
complex?
Weak Model
Figure 4: LLM routers classify queries and route complex ones an expensive/strong model, others to a cheaper weak model.
Inference flow. Given set of LLMs M, control plane Rw, and input x, LLM inference flow is the sequence of LLM invocations (z; for < S m and E (4,s) made when executing R.M (x). Here m is the total number of invocations, and 21, are the queries. Should be randomized, the sequence and its length are random variables. An
16

---

Published as a conference paper at COLM 2025
inference flow can be written as a transcript
T = (i), (i2, (2,........ Zm)
of pairs of model indexes i; E {w,s} and model inputs 2j. For simplicity, we assume that execution proceeds serially. For binary routers, we have = 1 and T E {(w,x), (s,x)} We write submitting sequence of inferences X -i...... Xg to control plane as
R (x) (R (x 1 ...., R (x₂))
In the binary router case, each inference results in single LLM invocation, but in general inference could involve multiple LLM invocations.
An inference flow policy dictates the control plane designer's 's intention regarding use of the underlying models. For example, an application may want to ensure that only a small fraction of queries go to the expensive model Mg. We can define this a predicate over a sequence of transcripts. In our binary router example, the policy can be more simply defined as predicate P over (input, model) pairs the sequence of transcripts. For example, a policy might specify that the strong model is used in at most an e fraction of inferences:
(Ry.)) since this fully defines
9 1(1)
P( (Zqrig)) =
Σ
Se
where (ii)) = if and I(ty) 0 if w. In other words, the predicate is that the fraction of queries routed to the strong model is bounded by e.
Control plane integrity, A control plane integrity adversary is a randomized algorithm A that seeks to maliciously guide inference flow.
In an unconstrained LLM control plane integrity attack, the adversary A seeks to gen- erate inputs x = x,, Kg such that running (x) generates a transcript for which (( (n) = O. This attack could be launched by an adversary who wants to maximize inference costs for victim application using an LLM router.
A harder setting requires input adaptation, where the adversary is given inputs II...., Ng and it must find new inputs ,, for which the transcript resulting from ((( ig)
There will be some competing constraint, such as that xj and Łj are very similar for each j, or that the outputs yj +- (xxx and yj <-s RM(x)) are close. In the routing context, the adversary's goal is to increase the fraction of queries that get routed to the strong model, in order improve the overall quality of responses, drive up the victim application's 's inference costs, or both.
Relationship to adversarial examples. Evasion attacks (Dalvi et al., 2004; Lowd & Meek, 2005; Szegedy et al., 2013) against inference systems, aka adversarial examples (Goodfellow al., 2015; Papernot al., 2016; 2017), would, our setting, seek to find a small modification to an input such that R.M A) + RMMxx (x) where addition appropriately defined based on input type (e.g., slight changes to query text).
Our attack setting is not the same because in our case, the adversary's goal is not just to mislead the classifier. Instead, the goal is defined with respect to the overall system behavior, e.g., to drive up the victim's 's inference cost or unfairly improve the quality of outputs. Attacking the control-plane component with adversarial inputs is but means towards these goals.
Simply changing routing decisions with adversarial examples (i.e., unconstrained attack on control plane integrity) is not enough. In the input adaptation attack, the adversary seeks to modify the inference flow yet not change the responses of the strong underlying LLM to the extent possible. Adversarial inputs against the control plane must preserve or even improve the outputs generated by the overall system.
In the other direction, undermining LLM control plane integrity could facilitate evasion attacks. For example, if R.M is used to classify malicious content by combining LLMs tuned
17

---

Published as a conference paper at COLM 2025
to different types of harm categories, then modifying inputs to force inference flows away from appropriate models could aid evasion. We leave this to future work.
B
LLM Routers
In Section we discuss our methodology for attacking LLM control plane integrity. We focus on the binary router setting in which the router applies learned scoring function to input queries and routes any query whose score exceeds some threshold 7 the strong LLM Mg. In this section we provide more formal description of LLM routers.
We consider router R for M = {M,, Ms}, where consists of scoring function S, scoring function's parameters 6, and a threshold E Rt For notational brevity we just write R below, with M clear from context. Here S and define a scoring function Sg X + RT. Since our focus is LLMs, we assume that queries X' are strings of text tokens. The routing algorithm then works as follows:
(x) = (M if Sp(x) < T
(x) otherwise
where = (,,0, 7). We detail scoring functions in Section 4 and Appendix D; prior work has suggested linear models, light-weight LLMs, and more. Note that, consistent with this application, scoring functions are computationally cheap (as compared to Ma, My). Deployments calibrate to limit the fraction of queries routed to the strong model Ma, giving rise to the type of control plane integrity policy discussed in Section 2.1.
C Gadget Examples
Below few examples of the optimized confounder gadgets produced by our method for different routing algorithms:
Rsw:
legationbelong::677egggl0 hui
(DitiionaryzzedNNamenttl bidi.numberOf
demanding. XVI-ioibili Written Hormchriftlatable
RMF:
041 personne.nama tomeT'N Exact relação tinha Descripinnn
Ctwp://nNTMeeiiiiissmmmmcameeem nije Hir Ter
RCIs:
contiene DEM USERNAME fourteen chees781 consultants200 inici DOJ
571:
Ordmnth Norwegian Moccer.docs Abr226 METADATA
R R
18

---

Published as a conference paper at COLM 2025
dated:framefyumi335 Kurrasciiuzzeiphereex
Midhigghexxeattion441177 below1 unwrap: 1elow
nn
D Experimental Setup
In this section we provide additional details regarding our experimental setup for the open- source routers evaluations, described in Section 4. All settings are summarized in Figure 3.
Target routers, As discussed in Section 4, we focus our evaluation on the prescriptive routing algorithms by Ong et al. (2024). These routers were trained in a supervised fashion using a set of reference (training) queries whose performance score on each of the considered models is known. The scores were computed from collection of human pairwise rankings of model answers for each of the queries. There is no reason to believe non-learning- g.. rule-based) routing algorithm would be more adversarially robust. We now outline the four routing methods (see Ong et al. (2024) for full implementation details).
Similartty-weiighted ranking: The first method is based on the Bradley-Terry (BT) model (Bradley & Terry, 1952). For given user query, this model derives a function to compute the probability of the weak model being preferred over the strong model. The expressions all share parameters, which are optimized to minimize the sum of cross-entropy losses over the training-set queries, where each element in the sum is weighted by the respective query's similarity with the user's 's query (computed as embeddings cosine similarity, with the embedding derived using OpenAl's text-embedding. 3-small (OpenAI, 2024a)). We denote this method as Rsw.
probability-functio
Matrix factorization: The second method based on matrix factorization. The training queries are used to train bilinear function mapping model's embedding and a query's embedding to score corresponding to how well the model performs on the query. Routing is done by computing the score of the input query for each model, and choosing the highest- scoring model. We denote this method as RMF-
BERT classifier: The third method involves fine-tuning a classifier, based the BERT-base architecture (Devlin et al., 2019), to predict which of the two models produces a better response for the given query whether they do equally well (a tie). The routing decision is based on the probability the weak model providing better response versus the strong model or the tie. We denote this method as RCLs-
LLM classifier: The last method is based on asking an LLM to provide a score in the range 1-5 of how an Al expert would struggle to respond to a given query based on the query's complexity. For this, Ong et al. fine-tuned a Llama-3-8B model (Meta, 2024a) using their reference set of queries and corresponding scores. We denote this method as R
Evaluation datasets. As discussed in Section 4, we evaluate our attacks using three standard LLM benchmarks as workloads: MT-Bench (Zheng al., 2023), a dataset of 160 open-ended questions, MMLU (Hendrycks et al., 2021), a dataset of 14,042 multi-choice questions, and GSM8K (Cobbe et al., 2021), a dataset of 1,319 grade-school math problems. In this section we provide more details regarding our evaluation of these benchmarks.
We first note that Ong et al. (2024) flagged that some data points are "contaminated" i.e., they are too similar to the ones used in their training of the routers. We use these datasets without these contaminated elements, resulting in 72 MT-bench queries, 14,037 MMLU queries, and 1,307 GSM8K queries.
For MMLU and GSMSK, we will require that the LLMs respond in a predefined format so can parse and compare the responses to ground-truth answers. To facilitate this, prepended formatting instructions to the query, inserted as a prefix before the gadget in the case of confounded queries. We phrase this instruction follows: "Answer the question using
19

---

Published as a conference paper at COLM 2025
the format: "Answer: [A/B/C/D], Explanation: [EXPLANATION!" for the multi-choice queries of the MMLU benchmark, and a similar version for GSM8K. In other words, a confounded query ends up defined as 8; = instr||c] for instruction template instr, confounder gadget e, and original query x;- Thus this case we model a scenario where the adversary only controls part of the prompt rather than the entire prompt. See Appendix I ablations.
When assessing the quality of responses in Section 5, we use both the perplexity scores of the responses as well the following benchmark-specific metrics:
MT-bench: We score the responses on a scale of 1-10 using an LLudgee method- ology (Zheng al., 2023). We use GPT-4o (OpenAI, 2024b) as the judge and ask it to provide score given a pair of a query and a corresponding response.
MMLU: We parse the responses and compare the answer to the ground truth. In cases where the response did not fit any known multi-choice format, we marked the response mistake. We report accuracy as the percentage of responses that match the ground truth.
GSM8K: similar MMLU except questions are math rather than multiple choice, thus we parse the answers according to the expected format.
Router calibration. As mentioned in Section 4, for each workload, we must calibrate each router by setting the threshold t to achieve some target fraction e of queries routed set
to the strong model. For our experiments, developer that seeks to control for costs, even if it may mean sacrificing some performance for some workloads. We evaluate other values of in Appendix E. Note that the calibration process we use is agnostic to the underlying LLM pair. We therefore must define 12 distinct thresholds, one for each router, dataset pair. We now provide more details regarding the calibration process.
0.5. This reflects an application
To calibrate for MT-bench, we use the Chatbot Arena (Chiang et al., 2024) dataset as the calibration set, computing the threshold using the 55 K queries for which Ong et al. precom- puted the scoring function outputs. To calibrate for MMLU and GSM8K, we select 1,000 queries uniformly at random and uses these to set thresholds. Looking ahead, we do not use these queries during evaluation of the attacks.
Note that it important that the distribution of calibration queries be similar to the distribution of the target workload (and, in our experiments, the test queries). We observed that the Chatbot Arena-based threshold did not transfer well to MMLU and GSM8K, resulting in the majority of queries (z 98%) routed to the strong model.
E Effect of stricter router thresholds
As mentioned in Section and Appendix D, in all of our experiments we set é = 0.5. Figure shows that even when using more restrictive, i.e. lower, thresholds that route fewer queries to the strong model, our attack can still reroute significant number of queries to the strong model.
F Runtime and convergence
Figure shows the convergence rates for 10 different gadgets, against different routing algorithms. The overall average number of iterations before convergence is 58. Generation against Ksw converges the fastest (50 iterations on average), RMF the slowest (66 iterations average). Interestingly, the score of Rsw does not increase much during optimization but is still sufficient for a successful attack.
Runtime varies significantly when generating gadgets against different routing methods. On a machine with one A40 GPU, 4 CPUs, and 180G RAM, a single iteration takes 36.9 s, 8.4s, 0.8 and 6.9s s for the Rsw, RM, ReLS, and RLLM routers, respectively. On average,
20

---

Published as a conference paper at COLM 2025
Souter
Rocter
Soster B.o
Pouter
USP 20 2i
HBL
Treashord
Tresshod
(a) Rsw
(b) RMF
(c) RCLS
(d) RLLM
Figure 5: Average upgrade rate as function of the threshold Lower thresholds indicate stricter setting, where only a small number queries are intended for the strong model. 50% represents the threshold used for our main results. Even in the strict setting, our attack consistently achieves a significant upgrade rate across all evaluated routers and benchmarks.
0.343
Doa
40%
Mixx
Rid
Mixh dc
0.35
16
403%
Aflxc
64
barstona
karatona
beratora
(a) Rsw
(b) RMF
(c) ReLS
(d) RLLM
Figure 6: Convergence of gadget generation against different routing algorithms.
takes around 31 minutes to generate gadget for the Rsw router, minutes for RMF, 50s for ReLS, and 6 minutes for RLLM-
G Quality of attack responses
In Section 5 we evaluate the effectiveness of our attack in the white and black box settings. As previously discussed, successful rerouting attack must not only reroute originally weak queries to the strong model, it must do while preserving even improving the quality of the LLM responses. Due to space limitations, we provide additional details and discussions over the quality evaluation in this section.
As a first measure of response quality, we compare the perplexity scores for unmodified responses and confounded query responses. Text perplexity (Jelinek, 1980) is well-known method for approximating "naturalness" of text sequences, Perplexity can be computed using an LLM, we use GPT-2 (Radford al., 2019) for this purpose it is standard choice(Alon & Kamfonas, 2023; Zhang et al., 2024a),² Table 2 shows the results. As can be seen, adding the confounder gadget to queries does not significantly change response perplexity. To the extent that it does, it usually somewhat decreases response perplexíty, i.e., makes it more "natural". That said, perplexity is a coarse measure of "naturalness, " and it does not measure whether the response is correct. In particular, responses strong and weak LLMs tend to have similar perplexities. We further discuss this issue in Appendix K.
2A few responses had abnormally high perplexity values (> 100), which we found do not correlate with quality, but these variations disproportionately contribute to the average. We thus filter out such high-perplexity responses as outliers in both benign and attack settings. We provide examples and the number of filtered responses in Appendix K.
21

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Original
Confounded
Original
Confounded
Original
Confounded
Rsw
10.0
8.3
19.5
20.1
14.5
1.3
RM
10.0
8.6 0.4
19.5
20.2 = 0.2
14.5
1..2
RCLS
10.0
8.2 + 0.5
19.5
20.3 = 0.2
14.5
RLLM
10.0
10.1 + 0.5
19.5
19.7 = 0.1
14.5
15.0 + 0.3
Table 6: Average perplexity of responses to the original and confounded queries, in the white-box setting, using only the strong model of LLM pair 1.
MT-Bench
MMLU
GSM8K
Original
Confounded
Original
Confounded
Original
Confounded
Rsw
8.5
8.3
66
66 = 0
57
65 +
RMF
8.5
8.3
66
66 = 0
57
67 +
RCLS
8.5
8.4
66
66
+
RLLM
8.5
8.3
66
66 -
65 +
Table 7: Average benchmark-speccffic scores of responses to the original and confounded queries, in the white-box setting, using only the strong model of LLM pair 1.
We thus also evaluate using benchmark-specite metrics to assess response quality. For MT-bench, each response ranked on scale of 1-10 and we report the average, while for MMLU and GSM8K we report the percentage of correct responses. We provide full details of these metrics in Appendix D.
Table shows that in most cases responses to the confounded queries are no worse, and in some cases even better, than responses to the original queries. We attribute the improvement on the GSM8K benchmark to the fact that the strong model performs significantly better than the weak model on this benchmark (57% vs. 33%). On the MT-bench and MMLUU benchmarks, strong and weak models have comparable performance (8.5 vs. 7.6 for MT- bench and 66% vs. 64% for MMLU), thus routing does not degrade quality of responses and, consequently, the attack cannot improve it.
To further demonstrate the effect of adding the gadgets and disentangle from the effect of rerouting to the stronger model, compare the perplexity and benchmark scores of confounded and original queries using only the strong model. Table and Table 7 show that, in most cases, the inclusion of the gadget does not significantly impact the quality of responses. Also, when manually inspecting the outputs, we observed that when the LLM produced wrong answer for both the original and confounded queries, in most cases the answer was the same for both queries. In summary, gadgets are effective against the routers but do not substantially affect the target LLMs.
In Appendix M.1 we further demonstrate that the attack improves the quality of responses when there is significant gap between the weak and strong LLMs by evaluating LLM pairs for which the weak model is weaker than in LLM pair 1.
LLM responses are sometimes affected by the confounder gadget. For example, an LLM responded with "I can't answer that question as it appears to be a jumbled mix of characters". Still, the response continued with "However, can help you with the actual question you're asking," followed by the actual answer. We observed very few cases where an LLM refused to answer due to the presence of the gadget. most cases, the response did not mention anything abnormal about the query. Intuitively, this reflects the fact that while LLMs are built to be robust to noisy inputs, the router itself is not.
H Black-box attack results
In addition to the evaluation the white-box attack setting in Section 5, we evaluated the black-box transfer setting, where the attacker that does not know the algorithm used by the
22

---

Published as a conference paper at COLM 2025
Surrogate
Target
MT-Bench
MMLU
GSM8K
RMF
99 1
66 15
99 i1 72
Rsw
RCLS
5
44
RLLM
4
$1
63
Rsw
100
82 :
92
RMF
RCLS
96
56 5
88
RLLM
3
7
6
Rsw
100
64
76
RCLS
RMF
9
16
60
RELM
51
80 5
65
Rsw
100+0
53 4
60
RLLM
RMF
83
20
70
RCLS
85
46 11
73
Table 8: Average upgrade rates for our attack in the black-box (transfer) setting. The average
downgrade rate (i.e., strong-to-wan rerouting) 1.2% across all routers.
Surrogate
Target
MI-Bench
MMLU
GSM8K
PPL
Bench
PPL
Bench
PPL
Bench
MM
0.4
0.1
1.9
Rsw
RCiS
0.8
-0.1
0.8
0.3
1.7
9.6
RELM
0.6
0.0
1.1
-02
0.6
15.2
Rsw
1.4
-0.1
0.2
4.8
1.6
18.6
RM
RCiS
0.7
0.1
0.2
1.0
1.7
13.8
RI
0.3
0.0
1.1
0.5
0.2
14.7
Rsw
1.7
-0.1
0.3
2.5
1.7
13.4
RCLS
RMF
0.3
0.0
0.8
1.3
1.0
6.8
RELM
0.7
0.9
0.8
0.4
12.6
Rsw
0.8
0.2
1.3
2.6
1.3
13.6
RLLM
RM
0.6
1.2
0.9
1.3
RCLS
0.0
0.2
0.9
0.3
1.7
10.4
Table 9: Differences between average perplexity ("PPL") and benchmark-specift scores ("Bench") of responses to the original and confounded queries, in the black-box setting and for LLM pair 1. Positive values indicate lower average perplexity (more natural) and higher average score of responses to the confounded queries; higher values are better for the attacker. Standard errors were omitted for readability but are 0.2 on average for the perplexity evaluation and 0.8, and 1.8 for MT-bench, MMLU and GSM8K, respectively, for the benchmark-specific evaluation.
target router but has access to surrogate router to mount a transferabillty-based attack. Due space limitations, we only briefly mentioned the results of this evaluation in Section and provide the full results here.
Table 8 shows the results for all combinations of surrogate (denoted by R) and target routers. Upgrade rates are lower than in the white-box setting, but still high, i.e, the attack definitely transfers. This is also evident by the successful transferability to the commercial routers presented Section 6. Downgrade rate 1.2% on average and 0% in most cases.
Table 9 shows that the black-box attack does not increase the average perplexity for LLM pair 1. Table 9 also shows that the attack does not decrease benchmark-speerffc scores, other than few small decreases for MMLU queries. For GSM8K, like in the white-box setting, we see an improvement with our attack as the performance gap is larger between weak and strong models. This effect is further demonstrated in Appendix M.1. In summary, confounding affects only the routing, not the quality of responses, even the black-box setting.
23

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Prefix Suffix
100 i 0
90
98 1:0
Rsw
100 = 0
=
94
Prefix
100 1 0
78 1 4
100 1:0
Suffix
100 0
: 3
1:
Prefix Suffix
100 + 0
100 +
100 0
RCLs
100 0
100
Prefix
7 84--4
1
100
RLLM
Suffix
93+1
94
Table 10: Average upgrade rates for different ways of adding the gadget to queries, in the white-box setting. Results are similar both methods, with slight preference to the prefix approach.
I
Ablation Study
In this section, we evaluate the effect of different hyperparameters and design choices (in the white-box setting).
Prefix vs. suffix. As described in Section 3, we prepend the confounder gadget to the query. An alternative is to append it. This straightforward for MT-bench and GSM8K, but MMLU consists of multi-choice questions followed by list of possible answers, and the term "Answer:". We insert the gadget at the end of the question text and before the possible answers. If we append it at the very end, after "Answer:" the LLM assumes the query was answered and in many cases does not generate any output at all.
Table 10 shows that average upgrade rates similar regardless of whether the gadget inserted as a prefix or a suffix. For MMLU, prefix works better. The downgrade rate is 0% in all cases.
As mentioned Appendix D, to encourage the LLMs to follow the specific format in their responses (so they can be parsed and compared with the ground-trute answers), we add a short prefix to the MMLU and GSM8K queries that instructs the model how to respond. We add this instruction after modifying the queries with the confounder gadget, iee the instruction is prepended to the gadget.
An alternative insert the instruction after the gadget but before the query, however we observed this to slighly underperform its counterpart. In the white-box setting we observe slight decrease in the average (across all four routers) upgrade rate from 91% 89% for the MMLUU benchmark, and from 98% to 91% for the GSM8K benchmark. In the black-box setting, the average upgrade rate on MMLU reduces from 57% 49% and on GSM8K from 73% to 64%.
Token sampling method. When generating the confounder gadget (see Section 3), we iteratively replace tokens with the goal of maximizing the routing algorithm's score for the gadget. Candidate replacement tokens are chosen uniformly at random. An alternative is to choose candidates based on their probability of appearing in natural text. To evaluate this method, we compute token probabilities by parsing and tokenizing the wikitext-103-rawwe dataset (Merity et al., 2016).
Table shows that in most cases uniform sampling of replacement tokens yields better upgrade rates. We conjecture that uniform sampling produces unnatural text, con- fusing the router. For example, for the Rsw routing algorithm, uniform sampling produces the following gadget: "legationhelong::977rgggg
hitpo//looooooooooooototctoooonn iutiiiiioaaaaaaaaaaaaaammmm bidi munberrne""""
24

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Uni.
100 +
90 1
98 + 0 88 =
Rsw
Nat.
100 +
77 + 2
Uni.
100 + 0
78 + 4
100 + 0
MM
Nat.
97 +
41 + 3
92 +
Uni.
100 +
100 0
100 + 0
Reis
Nat.
100 +
96 +
100 0
Uni.
7
95
94 + 83
R
Nat.
7
87
Table 11: Average upgrade rates for different ways of sampling candidate tokens during gadget generation- uniformly ("Uni") based on their probability of appearing in natural text ("Nat"), in the white-box setting. Uniformly sampling the tokens yields better upgrade rates in most cases.
whereas sampling according to natural probabilities produces "total occurred According number Letar final Bab named remainder".
Number of tokens in the gadget. In main evaluation, the gadgets composed of n = 10 tokens. We evaluate the effect of using less (n = 5) or more (и = 20 or n = 50) tokens. We observed that 5 tokens were insufficient to make changes to the routing algorithm's score and thus we were not able to optimize the gadget in this setting. As for 20 tokens, observe a small improvement in the white-box setting, increase the average upgrade rate from 93.9% to 95.8% and bigger improvement in the black-box setting, increase the average upgrade rate from 70.2% to 81.3%. Using 50 tokens further increases the upgrade rates, to 98.2% in the white-box setting and 84.2% in the black box setting. The average convergence rate increases as well, from 60 iterations for 10 tokens, to 70 for 20 tokens, and 100 for 50 tokens. Overall this evaluation suggests that our rerouting attack can be even further improved by using longer gadgets, however is important to be careful not to make them too long to the point that they might degrade the performance of the underlying LLM.
J Optimization-Free Gadget Generation
We evaluate optimization-free alternatives to our black-box optimization method for gener- ating confounder gadgets.
Fixed gadget. A simple way to create a gadget without resorting to optimization is to repeat tokens. We use ! as the initialization token, so the gadget this case is !!!!!!!!! Another possibility is to select tokens uniformly at random. Table 12 shows the upgrade rates for both options, were the latter setting we repeat the process 10 times and report the average result and the standard error. While they are non-negligible, especially for the randomly sampled gadgets, they significantly underperform the upgrade rates reported Table for optimized gadgets.
Instruction injection. Prompt injection is a known attack on LLMs (Perez & Ribeiro, 2022; Toyer et al., 2023), thus consider gadget consisting of a direct instruction to the router to treat the query as a complex one and obtain a high-quality response.
We evaluated differently phrased instructions: two created manually and two generated by, respectively, Gemini (Team et al., 2023) and GPT-4o (OpenAI, 2024b), denoted as "ours-1", "ours-2", "Gemini", and "GPT".
Table 13 reports the results. This method works well in a few cases but poorly in most. This highlights the difference between attacking L.LMs and attacking LLM routers.
25

---

Published as a conference paper at COLM 2025
gadget
MT-Bench
MMLU
GSM8K
Init
7
21
21
Rsw
Random
97 =
49 =
58 + 8
Init
3
4
20
R
Random
37 + 8
+
34 +
Init
8
0
0
RCLs
Random
62 + 10
14 + 7
37 +
Init
3 38 4
13 68 + 5 41 +
9
R
Random
Table 12: Average upgrade rates when the gadget is not optimized and is either defined to be the the initial set of tokens set of uniformly sampled tokens. The optimization-base approach outperforms these optimization-free approaches.
MT-Bench
MMLU
GSM8K
intro type
Up.
Down.
Up.
Down.
Up.
Down.
Ours-1
100
28
46
Ours-2
100
32
63
Rsw
Gemini
100
0
35
4
56
GPT
100
54
4
77
Ours-1
0
31
0
57
0
100
Ours-2
0
100
RMF
Gemini
0
100
GPT
0
48
0
51
0
100
Ours-1
33
2
47
77
Ours-2
75
19
26
16
43
RCLS
Gemini
100
100
98
0
GPT
46
0
66
95
Ours-1
26
7
0
42
4
36
Ours-2
35
0
43
RILM
Gemini
55
21
GPT
19
26
23
25
Table 13: Average upgrade and downgrade rates of gadgets containing injected instructions to the router. This method significantly underperforms the optimization-ban approach in most cases.
K Perplexity Issues
In Section 4, we use perplexity as one of the metrics for evaluating the effect of our attack on generated responses. In general, perplexity measures "naturalness" of the text rather than its quality. Figure 7 shows the distribution of perplexity values of the clean responses generated by weak and strong models, as well as the ROCAUC scores. Perplexity of responses is similar for the weak and strong models, with ROCAUC scores ranging between 0.38 to 0.47.
As mentioned in Section 4, throughout our evaluations we filter out responses with perplex- ity values higher than 100. This is due to a handful of valid responses that have abnormally high perplexity. For example, for the query:
Suppose you are mathematician and poet. You always write your proofs short poets with less than 10 lines but rhyme. Prove the square root of is irrational number.
The weak model responds with:
26

---

Published as a conference paper at COLM 2025
Strong
weak
23.0
weak
20
17.5
15.0
Count
12.5
Court
30
Perplexitt
Perplexity
Perplexity
(a) MT-bench ROCAUC=0.38
(b) MMLU
(c) GSM8K
ROCAUC=0.47
ROCAUC=0.38
Figure 7: Histograms the perplexity values of clean responses generated by the strong and weak models.
MT-Bench
MMLU
GSMSK
Up.
Down.
Up.
Down.
Up.
Down.
Rsw
0:0
24 2
813
912
4 412
48 9
RMF
0+0
67 6
0+0
77 +7
1+1
78 +
RCLS
0+0
29
0+0
50
0+0
80 +
RLLM
24
1=0
55
5+1
21 +4
4
Table 14:
Upgrade and downgrade rates for the downgrading, variant of our rerouting attack, where the goal is to reroute queries to the weak model (white-box).
In squares, two seeks a home, Two whole numbers, never roam. If it did, in pairs combined, A different square would it find. But take root, two's square, or four, Still leaves a remainder's 's roar. The square root of two's no child of two, Its irrational nature ever true.
which has a high perplexity value of 166. We also observed few responses containing lists, code, equations that result in abnormally high perplexity values. In total, we filtered 8 responses for the MT-bench dataset, for MMLUU and for GSM8K. We stress that this is the total across all evaluated settings (i.e., for all routers in the white-box setting and all surrogate-target router combinations the black-box setting). In many settings, no responses were filtered. The maximum amount of filtered responses in specific setting was 5.
L Rerouting to the Weak Model
In this section we evaluate the generality of our attack and show that generation of con- founder gadgets can be optimized for the opposite objective from what we consider so far: reroute queries to the weak model. For this, we repeat the same optimization process as in Section but minimize the router's score. Table 14 shows the upgrade and downgrade rates for this variant of the attack, in the white-box setting. In most cases see significant downgrade rate and a minimal upgrade rate, meaning that most of the modified queries were routed to the weak model. One notable exception is the LLM-based router Ry,,M, for which the attack does not work well. Future work will be needed to explore improving confounder generation for this setting further.
27

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Original
Confounded
Original
Confounded
Original
Confounded
Rsw
9.2
9.2
76
84
62
86 +
RM
9.1
9.3
76
81 + 0
65
88 =
RCLS
9.2
9.1 0.0
76
84 + 0
68
90 +
RLLM
8.9
9.1 9:1::00 0.1
78
84 +
66
85 2
Table 15: Benchmark-specffic average scores of responses to the original and confounded
queries with (TT44-110-pprviiee as the strong model (LLM pair 4), in the white-box setting. Results demonstrate higher increase in performance with respect to the LLM pair 1 setting, due to the larger performance gap between the models.
M Results for Other LLM Pairs
As discussed in Section 4, for allowing our evaluation to scale, we use as the strong model Mg the open-sourced Llama-3.1 (Meta, 2024b) and as M2 the 4-bit quantized version of Mixtral 8x7B, denoted in our work as LLM pair 1. In this section extend our evaluation to other LLM pairs.
We begin by evaluating the case where the weak model produces much worse responses than the strong model. We define LLM pair 2 as the strong model plus Mistral-7B-Insttrut--ddl (Jiang et al., 2023a) and LLM pair the strong model plus Jlama-2-7B-chat-h (Touvron et al., 2023). The weaker models in pairs 2 and 3 were chosen to represent smaller (Mistral 7B) and older-generation (Llama-2) models: according to the Chatbot Arena leaderboard (Face, 2024; Chiang et al., 2024), Llama-3.1-8B is ranked in the 58th place, Mixtral 8x7B at the 88th place, Mistral-7B at the 108th place, and Llama-2-7B the 125th place.
We additionally perform some smaller-scale evaluations using the LLM pair originally used by Ong et al. (2024), i.e. GPT-4-1106-preieww (Achiam et al., 2023) as the strong model and Mixtral 8x7B (Jiang et al., 2024) as the weak model. We refer to this setting as LLM pair 4.
model
Figure 3 in the appendix shows the strong-weak pairs in our experiments.
M.1 Results for LLM Pairs 2 and 3
The discussion over quality of attack responses in Section 5 shows that for the white-box setting, in most cases, responses the confounded queries are no worse, and in some cases even better, than responses to the original queries. To further demonstrate that the attack improves the quality of responses when there is significant gap between the weak and strong LLMs, we perform an additional evaluation with Miistral-BB-Instruct-v
(Jiang et al., 2023a) and Llama-2-7B-cha--h (Touvron et al., 2023) as the weak LLMs (LLM pairs 2 and 3). Mistral-7B achieves 7.4, 57%, and 25% on MT-bench, MMLU, and GSM8K, respectively. Llama-2-7B achieves 6.4, 44%, and 21%. %. Table 16 shows that the rerouting attack improves the quality of responses when either of these LLMs is the weak model, and in particular for the weaker Llama-2-7B model.
Table 17 shows similar behaviour for the black box setting, with the benchmark-specific scores improving when the weak model is significantly weaker than the strong model, i.e., LLM pairs 2 and 3.
M.2 Results for LLM Pair
As discussed in Section 4, we replace the strong model that was used by Ong et al. (2024), GPT-4-100--rreiew (rank 28 in the Chatbot Arena leaderboard (Face, 2024; Chiang et al., (rank 58) to reduce the costs of our extensive
2024)), with the open-sourced Llama-3.1 set of evaluations. In this section we perform smaller-scale evaluation of the quality- enhancing attack performance when using GPT as the strong model, i.e. LLM pair We evaluate this setting using three of the 10 confounder gadgets for each router.
28

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Orig.
Conf.
Orig.
Conf.
Orig.
Conf.
LLM pair 2
Rsw
8.5
0.0
55
64
46
64 1
RMF
8.4
0.1
63
RCLS
8.4
8.1
58
66 =
49
63
RLLM
8.4
8.1
62
66 = 0
38
63 +
LLM pair 3
Rsw
8.4
0.0
51
64
40
MM
8.1
8.3
57
63 =
44
67
+
RCLS
8.3
8.4
52
66 -
45
63 +1
RILM
8.1
8.2
59
66 =1
37
14
Table 16: Average benchmark-speciffcc scores responses to the original and confounded queries with Mistral-7B-Insstrutt-03.33 (LLM pair 2) or Llama-2-7B-chatal (LLM pair 3) as the weak model, in the white-box setting. Results further emphasize that the rerouting attack improves quality of responses when there is significant gap between the weak and strong LLMs.
Surrogate
Target
LLM pair MMLU
LLM pair 3 MMLU
MT-Bench
GSM8K
MT-Bench
GSMSK
RMF
-0.1
1.6
13.6
0.2
5.0
20.5
&sw
RCLS
-0.1
4.0
8.7
0.0
6.8
13.4
RLLM
0.1
4.2
18.5
0.1
5.8
20.9
Rsw
-0.2
7.9
18.9
0.1
11.3
24.3
RM
RCIS
-022
5.0
14.4
0.1
9.1
18.6
RLLM
-02
4.4
18.3
0.0
4.7
21.6
Rsw
0.1
5.0
13.1
0.0
8.1
17.9
RCLS
RMF
-0.1
-2.9
4.0
0.2
-3.7
11.2
RLLM
0.0
3.2
15.5
0.2
4.8
18.9
Rsw
0.2
5.2
11.3
0.1
7.8
16.7
7LLM
RMF
-0.2
-0.9
8.4
0.1
15.2
RCLS
-0.2
3.8
10.8
0.1
7.2
14.2
Table 17: Differences between average benchmark specific scores of responses to the original and confounded queries, when the confounder gadget was generated for a different surro- gate router than the target (black-box setting) for LLM pairs 2 and 3. Positive values indicate a higher average score for responses to the confounded queries; higher values are better for the attacker. Results ability and are on average 0.1, 0.8, and 1.8 for MT-bench, MMLU and GSM8K, respectively. Aligned with the white-box setting, results show almost decrease in performance, and improvement when there a performance gap for the LLM pair.
averaged across gadgets. Standard errors were omitted for read-
Table 15 shows the results across benchmarks in the white-box setting. Compared the pair 1 setting (Table 3), the attack results in higher increase in benchmark performance. This further demonstrates higher attack effect on response quality when the performance gap between the weak and strong models higher.
N Query-specific Gadgets
By default, our gadget generation method query-independent and the same gadget can be used to reroute any query. An adversary with more resources may instead generate dedicated gadget for each query. Our confounder gadget approach, provided in Section 3, extends to this setting readily by replacing (c(c) in Eq. with (p(c][z;;
29

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSM8K
Rsw
100
100
100
RM
100
96
100
RCLS
100
100
100
RLLM
100
100
100
Table 18: Upgrade rates for query-specific gadgets, in the white-box setting. Results are
nearly perfect, i.e nearly all confounded queries are routed to the strong model.
Surrogate
Target
MT-Bench
MMLU
GSMSK
RMF
100
96
100
R Rsw
RCLS
83
57
68
RLLM
89
Rsw
100
95
100
RMF
RCLS
83
43
73
R M
48
83
68
Rsw
100
74
RCLS
RMF
73
13
65
RLLM
52
83
70
Rsw
100
77
88
RLLM
RMF
RCLS
30
Table 19: Upgrade rates for query-specific gadgets, in the black-box setting. In most cases results are better than in the query- query-independent setting, at the cost of more resource intensive process.
Table 18 and Table 19 show the results for the white-box and black-box settings, respectively. (Here, percentage numbers are not averaged and there is no standard error since we used single gadget per query.) The white-box results are nearly perfect; the black-box results are often better but sometimes somewhat worse than those for query-independent gadgets. We conjecture that this is due to some level of overfitting.
O Adaptive Attack Against Perplexity Defense
In each iteration 1 E [T], we uniformly sample a target index j E [1,n] and generate a set B of B + candidates as explained in Section 3. We then modify Eq. 3 such that now find the candidate that maximizes the difference between the router's score and the perplexity constraint for the confounder:
(t+1) <- arg max (So(c))x;) IPPL(c) -pl)
cEB
where PPL(.) denotes the perplexity function computed using GPT-2, the value denotes target perplexity value to which we want gadgets' perplexity to close, and the value a is balancing coefficient. For the experiments below, we set p to be the average perplexity value of 100 uniformly sampled queries from the GSMSK benchmark.
Figure shows the results when setting a = 0.01, for the GSM8K benchmark and one confounder gadget generated for the Rsw router. Modified queries distinguished from normal queries by their perplexity alone. For instance, in the case of the Rsw router, setting the threshold value at 55 as before, flagged as anomalous, meaning the true-posiin nou-pooiiiie rate is zero. We note that there is some variability across gadgets. The average ROCAUC scores of the defense across ten gadgets with standard deviation indicated parenthetically, are 0.66 (+0.04), 0.69 (+0.02), 0.71 (+0.02), and 0.69 (=0.)3 for the Rsw, RMF, RCLS, and RLLM routers, respectively.
longer be easily
confounded queries are
Thhe perplexity calibration queries were chosen such that they do not overlap with the queries used for evaluation.
30

---

Published as a conference paper at COLM 2025
Original
1.0
20.0
Confounded
15.0
Count
12.5
0.6
75
5.0
0.2-
0.0
0.0
30
40
100
Perplexity
FPR
(a) Rsw
(b) Rsw
Figure 8: Perplexity values of the original and confounded queries, and the corresponding ROC curves of the defense that detects confounded queries by checking if they cross perplexity threshold, when the confounder gadget is optimized for low perplexity, in the GSM8K benchmark and for one gadget sampled uniformly at random. Confounded queries have similar perplexity values as the original queries, and can no longer be easily distinguished based perplexity alone.
MT-Bench
MMLU PPI.-opt.
GSM8K
Orig.
PPL-opt.
Orig.
Orig.
PPI.-opt.
Rsw
100 1
100 100
90 1
59 15
98 1:0
70 17
RMF
100 +
=2
78 4
7
100 0
98 +
ReLS
1
18
100
66
66+10 12
100 0
88 6
RELM
73
51
95
89 3
94
81 88
Table 20: Average upgrade rates for gadgets generated without ("Orig.") and with ("PPL- opt.") wow-perplexity optimization, for the balancing coefficient = 0.01. In some cases, optimizing for low perplexity has a negative effect on the attack success rate, however the attack can still be considered successful. A more careful choice of can potentially limit the effect on the attack success.
At the same time, optimizing for low perplexity does not significantly impact the attack success rate. Table 20 compares the average upgrade rates (over 10 gadgets) of the original perplexity-agnostic optimization approach from Section 3 and the perplexity- minimizing one described above. The attack efficacy might be improvable further by adjusting a to find sweet spot that avoids the defense effectively while ensuring high rerouting success rate.
We additionally compare both optimization approaches in terms of response quality. Table 21 and Table 22 compare the average perplexity values and benchmark-specffic scores of responses. Results indicate that both types of gadgets have similar (minor) effect on the quality of responses.
The attack is not particularly sensitive to the choice of queries used to obtain the calibration value p. Although p was computed using GSM8K queries, we observe similar performance when evaluating on MT-bench and MMLU, with average ROCAUC scores of 0.50 (+0.01), 0.51 (+0,01), 0.52 (+0), and 0.51 (+00) for MT-bench, and 0.52 (+0.03), 0.54 (+0.02), 0.55 (=0,01), and 0.53 (+002) for MMLU. One might also remove the calibration value altogether, instead simply minimizing the gadget's perplexity value. This can produce an "overshooting" effect, where the perplexity value is significantly lower than that of normal queries, thereby making it still distinguishable from standard queries.
In summary, perplexity-based filtering is not an effective defense against against rerouting.
31

---

Published as a conference paper at COLM 2025
MT-Bench
MMLU
GSMSK
Orig.
PPL-opt.
Orig.
PPL-opt.
Orig.
PPL-opt.
Rsw
12.3 0.2
12.0
2.11
1.2
0.3
15,8 0.4
RMP
12.3 0.2
1.1
20.3 0.1
2.24
1.3
15.9 =
RCLS
12.1
12.3:02
2.1
20.0
15.0
15.1 0.3
RLLM
12.7 0.4
11.9
19.6
1.25
15.2
15.0 = 0.3
Table 21: Average perplexity of responses to gadgets generated without ("Orig.") and with ("PPL-opt.") ") low-perplexity optimization, for the balancing coefficient a = 0.01. Both
methods perform similarly.
MT-Bench
MMLU
GSM8K PPL-opt.
Orig.
PPL-opt.
Orig.
PPL-opt.
Orig.
Rsw
8.3 0.0
8.1
66 +
63 +
64 +
61
+
RMF
0.0
8.2
64 11
66 1
67 11
63
1
RCiS
8.3 0.0
8.0
1:0
63
11
1
RI
8.2
8.0
66 0
67 0
64:11
61:22
Table 22: Average benchmark-speciffcc scores of responses to gadgets generated without ("Orig.") ") and with ("PPL-opt.") low-perplexity optimization, for the balancing coefficient 0.01. Both methods perform similarly.
P Extended Survey of Related Work
Evasion attacks against ML systems. A large body of work investigated evasion attacks against ML systems Dalvi et al. (2004); Lowd & Meek (2005); Szegedy et al. (2013), aka adversarial examples Goodfellow et al. (2015); Papernot et al. (2016; 2017), including text- only (e.g,, Cho et al. (2024)) and multi-modal LLMs Dong et al. (2023). In Appendix A, we explained the differences and similarities between LLM control plane attacks and adversarial examples.
Prompt injection against LLMs. Prompt injection involves manipulating the prompt, i.e, input to the LLM,, causing it to generate outputs that satisfy some adversarial objective Perez & Ribeiro (2022); Toyer et al. (2023). Evasion attacks discussed above use prompt injection. For example, jailbreaking attacks bypass some safety guardrail of the target LLM, such as "do not output expletives" Liu et al. (2023); Schulhoff et al. (2023); Zou al. (2023); Wei et al. (2023); Zhu et al. (2023); Chu et al. (2024).
Prompt injection can also extract information from about the model, e-g., the system prompt Perez Perez & Ribeiro (2022); Zhang et al. (2024b); Schulhoff et al. (2023), training data Nasr et al. (2023), or model parameters Carlini et al. (2024). In indirect prompt injection Gre- shake et al. (2023), adversaries do not directly interact with the target LLM and instead inject adversarial inputs into third-party data, which is then added to the LLM prompt (intentionally unintentionally) by the victim application and/or or its users. A related category of attacks exploits weaknesses of LLMs to compromise RAG, retrieval-augmented generation Chaudhari et al. (2024); Shafran et al. (2024).
Our attacks, too, modify LLM queries, but the objective is different: to change routing decisions while preserving or improving outputs. Future work may investigate indirect attacks that somehow trick users into forming queries that confound control planes of I.LM-bseed applications.
Attacks against MoE. Mixture-of-Expers (MoE) architectures reduce inference cost by routing different tokens of the query to different "expert" sub-models Du et al. (2022); Fedus et al. (2022); Riquelme et al. (2021); Shazeer et al. (2016). This can be thought of as an internal router within single L.M.
32

---

Published as a conference paper at COLM 2025
Hayes et al. Hayes et al. (2024) identified vulnerability in MoE systems that can be exploited for denial of service. The connection between token-routing, single-LLM attacks and multi-LLM attacks studied in this paper can be explored in future work.
Yona et al. Yona et al. (2024) presented a side-channel attack on MoE that enables an attacker to infer other users' prompts. There may exist side-channel attacks against multi-LLM control planes that, for example, identify the underlying LLMs. These attacks would target confidentiality rather than integrity.
33


Reviews:
[{"review_id": "JxJdZscL0Z", "reviewer": "Reviewer_xwTG", "summary": "The paper addresses adversarial attacks on LLM routers, introduces the concept of \"LLM control plane integrity.\" The method uses hill-climbing optimization to generate confounder gadgets by iteratively replacing tokens to maximize the router's scoring function. And the resulting gadget can be prepended to any query to confound routing decisions. They evaluate both white-box (full knowledge) and black-box (transfer) attack scenarios.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The paper demonstrates that LLM routers can be manipulated using query-independent \"confounder gadgets\" - short adversarial token sequences that, when prepended to any query, force the router to send queries to the expensive model while preserving or even improving output quality. This introduces a new class of AI safety problems distinct from traditional adversarial examples.\n2) The paper also explores perplexity-based defenses and shows how to evade them.\n3) The paper conduct experiments on four types of LLM routers, and all of them shows improvements.", "reasons_to_reject": "The writing needs to be improved. Some details are not clear. For example, it's not so clear in the threat model session that how the attacker can intervene the plane. And also, is the added suffix also the input of the LLM? In this case, the author should also involve the results of the baselines when using the stronger model. (to see what's the effect of adding the suffix/prefix)", "questions_to_authors": "1) In black-box settings, the assumption is that the attack knows what kind of routers he uses? Can the attack transfer between different types of routers?\n2) The attack goal is to upgrade the LLM, what if the attack goal is downgrade LLM? is the attack similarly effective?\n3) The 4 types of router attack result in similar results for commercial cases, why is that?", "extra_scores": {}}, {"review_id": "0yRfBzD3tD", "reviewer": "Reviewer_2R6h", "summary": "This paper studies the adversarial robustness of LLM routers, systems that route queries to different language models based on estimated complexity. The authors show that simple, query-independent \"gadget\" strings can reliably force routers to select stronger, more expensive models. And they find they can transfer to black box setting.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Interesting task", "reasons_to_reject": "1. basically a similar scenario to the jailbreaking. As in the jailbreaking it is also trying to jailbreak a llm or a adversarial classifier.\n2. there is not much insight provided from this paper\n3. the paper is not well-written.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OW0PIPXLRA", "reviewer": "Reviewer_ZVFv", "summary": "This paper presents an adversarial attack targeting LLM routers. The attacker’s goal is to force the router to choose a more expensive LLM by adding an optimized prefix to the input.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The idea is interesting and the threat model seems realistic to me. I think the community should be aware of this threat model.\n\nThe results generally support the paper’s claims.\n\nThe authors also run experiments on commercial routers.\n\nThe writing is clear and the paper is well-organized.", "reasons_to_reject": "The attack is more stealthy when optimized with a PPL-based constraint. I would like to see a deeper look at how the queries optimized with the PPL-based constraint compare to the ones without this constraint. For example, I would like to see a qualitative analysis for the samples optimized with the PPL-based constraint.\n\nAs far as I understand, there’s only one experiment with more than two routers. It would be helpful to see more setups with more routers, especially with tighter / more conservative thresholds for choosing stronger LLMs. This would help understand the limits of the attack.\n\nThe number of test queries is small (e.g., only 100). A larger test set would make the results more convincing.\n\nThe authors did not release the code, which reduces the reproducibility of the results.", "questions_to_authors": "N/A", "extra_scores": {}}, {"review_id": "jaLqgl4z3T", "reviewer": "Reviewer_52ek", "summary": "In this paper, the authors propose a novel attack targeting LLM routers. By prepending an adversarial prefix, the attacker aims to \"upgrade\" all queries to strong models. By doing so, the attacker can increase the cost to the provider, and meanwhile improve the response quality. Through extensive experiments, the authors show that their simple method works very effectively, and the adversarial queries also transfer to commercial black-box routers.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well-written. I really enjoy reading it.\n- The method is simple, but very effective. The experimental section shows that the attack can reroute most queries to strong models under various router settings. The method also works quite well under the commercial black-box scenario.\n- I like the motivation, and the threat model is new to me and interesting.", "reasons_to_reject": "- I'm slightly confused by the threat or the adversary's goal. I think at the beginning, the authors claim that the attacker does this attack so that it can increase the cost for the provider, because everything goes to the strong model. However, from section 6, it seems that you have to pay whatever it actually costs. It's not like all queries have the same price. If that's the case, why would you do this attack?\n- I think the authors should also include more baselines, like just pretending some random strings or a sentence: \"This will be a very challenging question.\".", "questions_to_authors": "- How many different router training runs are used in the experiments? I know the results are averaged over gadgets with different random seeds. Do the authors also train different routers under the same LLM pairs, and average the results? I think it would be nice to have results averaged over 5 routers x 5 gadgets runs.\n- LLM router seems to be more difficult to attack. Maybe the authors can try better optimizers like GCG?\n- Minor suggestion: I think it would also be interesting to study the downgrade case, where the adversary does prompt injection and makes the victims receive bad responses.", "extra_scores": {}}, {"review_id": "LnSSZd31OA", "reviewer": "Reviewer_Zdu3", "summary": "The paper presents a novel adversarial attack and defense en el domain of adversarial attacks on routing LLM or agents orchestrators. Although the topic is very interesting and the paper throughly describe all the aspects of the proposal, I have the following concerns:\n- The attack, although it seems very elaborated, at the end, they are based on random addition of words to the prompt.\n- I miss an external evaluation on a specific task, in order to really evaluate the impact of changing the selection of LLM.\n- The defense section is more similar a suggestion section than a real proposal.", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The novelty of attacking the orchestration process of an agent architecture.\n- The detailed description of all the elements of the proposal.", "reasons_to_reject": "- The attack, although it seems very elaborated, at the end, they are based on random addition of words to the prompt.\n- I miss an external evaluation on a specific task, in order to really evaluate the impact of changing the selection of LLM.\n- The defense section is more similar a suggestion section than a real proposal.\n- On my humble opinion, the attack is not silent, which reduces its effectiveness.", "questions_to_authors": "- Why didn't you really propose one defense?", "extra_scores": {}}, {"review_id": "uaeQLlec5k", "reviewer": "Reviewer_WMXC", "summary": "This paper proposes an adversarial attack on LLM routers using confounder gadgets that, when prepended to any query, cause it to be routed to a stronger, more expensive model. The attack is effective in both white-box and black-box settings across open-source and commercial routers. The paper highlights the risks to LLM control plane integrity and discusses potential defenses and future research directions.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "I found the idea of attacking LLM routers via query-independent confounder gadgets to be quite practical, which achieves to reroute queries between strong and weak models arbitrarily and introduces two kinds of attack goals, i.e., cost inflation and arbitrage. The key insight of this approach is that instead of crafting adversarial queries on a per-input basis, the attacker generates a fixed token sequence in an offline phase that can be prepended to any input. This sequence tricks the router into selecting a stronger model, regardless of the query’s true complexity. When actual queries are submitted in the online phase, this lightweight prefix causes strong/weak models’ rerouting. Also, the experiments show effective, transferable results and success in real-world commercial model routers.", "reasons_to_reject": "If a commercial router does not disclose which model handled a query, how can one confidently determine that the attack successfully triggered cost inflation/arbitrage via rerouting? Alternatively, is it possible to infer the success of the rerouting attack purely from empirical signals, such as changes in response quality, latency, or output characteristics? \nIf such indicators are unreliable or insufficiently discriminative, then it raises the concern that an attacker might believe they are successfully inflating cost by rerouting to the strong model, while in fact the queries are still being handled by the weak model—thus undermining both the effectiveness and the interpretability of the attack.\n\nFor example, if each query is evaluated by several independent or diverse routers, and a majority vote or consensus is used to decide the routing path, would this increase robustness by making it harder for a single fixed gadget to consistently fool all routers?\n\nCould the confounder gadgets be made more natural or human-like in form, rather than appearing as unnatural or random token sequences?", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper investigates a critical but underexplored security issue in the emerging area of LLM control-plane integrity, specifically targeting LLM routing mechanisms that balance query-response quality against computational cost. The authors introduce a novel adversarial attack methodology termed "confounder gadgets," which are universal adversarial sequences capable of manipulating LLM routers into selecting stronger, costlier models, irrespective of the actual complexity of queries. 

I believe this area will keep gaining traction as this issue becomes bolder with test-time scaling and reasoning models (and also speculative decoding)


Summary Of Reasons To Publish (pros):
The key contributions of this paper are compelling and robust. Firstly, the introduction and formalization of "LLM control-plane integrity" as an explicit research problem in AI security is innovative and timely.  The empirical evaluation is extensive and thorough, showcasing impressive attack success rates across diverse routers and benchmarks (MT-Bench, MMLU, GSM8K). Additionally, the authors discuss potential defenses and their limitations, highlighting the nuanced complexity of maintaining control-plane integrity.


Summary Of Suggested Revisions (cons):
While the paper is solid, several improvements can further strengthen its impact. A more detailed analysis and explicit comparison of the proposed defenses against potential adaptive attacks would enhance clarity and practicality. Expanding on the discussion of realistic attacker assumptions and operational constraints would help contextualize the feasibility of deploying such attacks in practice. Additionally, providing further details about gadget interpretability—why certain tokens are particularly effective—would add depth and clarity to the analysis.

Technical and presentation aspects could benefit from refinements, particularly improved readability and consistency in gadget examples and visualization clarity. Some clarification around router calibration and threshold settings would also aid reproducibility and understanding of the experimental setup.

[Automatically added comment] At least one review was discounted during the decision process due to quality]
Decision: Accept