Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Authors: Yannis Montreuil
Axel Carlier
Lai Xing Ng
Wei Tsang Ooi
Venue: ICML 2025 poster (2025)
Keywords: Learning to defer
learning with abstention
robustness
learning theory
Primary area: theory->learning_theory
TL;DR: Adversarial robustness in the context of two-stage Learning-to-Defer: Introducing novel attacks and establishing defenses with guarantees

Abstract: Two-stage Learning-to-Defer (L2D) enables optimal task delegation by assigning each input to either a fixed main model or one of several offline experts, supporting reliable decision-making in complex, multi-agent environments. However, existing L2D frameworks assume clean inputs and are vulnerable to adversarial perturbations that can manipulate query allocation—causing costly misrouting or expert overload. We present the first comprehensive study of adversarial robustness in two-stage L2D systems. We introduce two novel attack strategies—untargeted and targeted—which respectively disrupt optimal allocations or force queries to specific agents. To defend against such threats, we propose SARD, a convex learning algorithm built on a family of surrogate losses that are provably Bayes-consistent and $(\mathcal{R}, \mathcal{G})$-consistent. These guarantees hold across classification, regression, and multi-task settings. Empirical results demonstrate that SARD significantly improves robustness under adversarial attacks while maintaining strong clean performance, marking a critical step toward secure and trustworthy L2D deployment.

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Yannis Montreuil 123
Axel Carlier 45
Lai Xing Ng25
Wei Tsang Ooi 15
Abstract
task to the decision maker with the highest confidence (Mao et al., 2023a; 2024d; Mao, 2025; Montreuil et al., 2024; 2025b). L2D is particularly valuable in high-stakes applica- tions where reliability and performance are critical (Mozan- nar & Sontag, 2020; Verma et al., 2023). In healthcare, L2D systems integrate an AI diagnostic model with human spe- cialists, delegating routine tasks to Al and deferring edge cases-such as difficult images to specialists for nuanced evaluation (Mozannar et al., 2023; Strong et al., 2024). By dynamically assigning tasks to the most suitable agent, L2D ensures both accuracy and reliability, making it ideal for safety-critical domains.
Two-stage Learning-to-Deff (L.2D) enables op- timal task delegation by assigning each input to either fixed main model or one of several offline experts, supporting reliable decision-making in complex, multi--gent environments. However, ex- isting L2D frameworks assume clean inputs and are vulnerable to adversarial perturbations that can manipulate query allocation causain costly misrouting or expert overload. We present the first comprehensive study of adversarial robust- ness in two-stage L2D systems. We introduce two novel attack mnaieggees-nttaggeee and tar- geted-which respectively disrupt optimal alloca- tions or force queries to specific agents. To defend against such threats, we propose SARD, a con- vex learning algorithm built on a family of surro- gate losses that are provably Bayes-consistent and (R,9) (..c))coossstent These guarantees hold across classification, regression, and multi-task settings. Empirical results demonstrate that SARD sigmifi- cantly improves robustness under adversarial at- tacks while maintaining strong clean performance, marking a critical step toward secure and trustwor- thy L.2D deployment.
Robustness in handling high-stakes decisions is, therefore, essential for such systems. However, existing L2D frame- works are typically designed under the assumption of clean, non-adversaria input data, leaving them highly suscepti- ble to adversarial perturbations suuurbattioosuuuulee subtle input manipulations that disrupt task allocation and alter decision boundaries. Unlike traditional machine learning systems, where adver- sarial attacks primarily affect prediction outputs (Goodfel- low al., 2015; Szegedy et al., 2014; Awasthi et al., 2023), L.D systems are susceptible to more sophisticated adver- sarial threats, including query redirection to less reliable agents intentional agent overloading. These vulnerabili- ties severely impact performance, drive up operational costs, and compromise trust.
1. Introduction
This paper addresses the important yet unexplored challenge of adversarial robustness in L2D systems. Inspired by ad- versarial attacks on classification (Goodfellow al., 2015; Madry et al., 2018; Gowal al., 2020), introduce two novel attack strategies tailored to two-stage L2D: untargeted attacks, which disrupt agent allocation, and targeted attacks, which redirect queries to specific agents. To counter these attacks, we propose a robust family of surrogate losses based on cross-entropy (Mao al., 2023a; 2024d; Montreuil et al., 2025c), designed ensure robustness in classification, re- gression, and multi-task settings. Building upon advances in consistency theory for adversarial robustness (Bao et al., 2020; Awasthi et al., 2022; 2023; Mao et al., 2023b), we establish both Bayes-consistency and () consistency for our surrogate losses, enabling reliable task allocation even under adversarial scenarios. Our algorithm, SARD, leverages these guarantees while preserving convexity.
Leaming-to-Defer (L2D) is a powerful framework that en- ables decision-making systems to optimally allocate queries among multiple agents, such as Al models, human experts, or other decision-makers (Madras et al., 2018). In the two- stage framework, agents are trained offline to harness their domain-specific expertise, enabling the allocation of each
"chooo of Computing, National University of Singapore, Singa- pore "Institute for Infocomm Research, Agency for Science, Tech- nology and Research, Singapore *CNRS @ @REATE LTD, I Create Way. Singapore 'IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France 'IPAL, IRL2955, Singapore. Correspon- dence to: Yannis Montreuil <yannis.montreui montreuil @ @u.nus.edu>>
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Our key contributions are:
was tackled by Mao et al. (2025a; 2024e), who extended the theory cover the realizable setting under specific condi- tions.
1. We introduce two new adversarial attack strategies tai- lored to two-stage L.2D: targeted attacks that manip- ulate routing toward specific agents, and untargeted attacks that disrupt optimal allocation.
In the two-stage regime, where experts and the main model are fixed and trained offline, Narasimhan et al. (2022) intro- duced a post-hoe deferral strategy. Building on this, Mao et al. (2023a) provided the first Bayes- and H-consistent formulation for classification, which was later extended to regression by Mao et al. (2024d). Finally, Montreuil et al. (2025c) generalized the two-stage framework to multi-task learning scenarios and Montreuil et al. (2025a) to top-k experts.
2. We propose robust family of cross-entropy based surrogate losses that are Bayes-consistent and (R, 9)- consistent, enabling reliable allocation in classification, regression, and multi-tast settings. These losses are the foundation of our convex and efficient algorithm, SARD.
3.
We empirically demonstrate that our attacks reveal crit- ical vulnerabilities in saate-of-the-eee two-stage L2D systems, while our proposed method SARD consis- tently withstands these attacks and achieves robust performance across diverse tasks.
Adversarial Robustness: The robustness of neural net- works against adversarial perturbations has been extensively studied, with foundational work highlighting their vulner- abilities (Biggio et al., 2013; Szegedy et al.. 2014; Good- fellow al., 2015; Madry al., 2018). A key focus in recent research has been on developing consistency frame- works for formulating robust defenses. Bao et al. (2020) proposed Bayes- surrogate loss tailored for ad- versarial training, which was further analyzed and extended in subsequent works (Meunier et al., 2022; Awasthi et al., 2021). Beyond Bayes-consistency, H-consistency has been explored to address robustness in diverse settings. Notably, Awasthi et al. (2022) derived H-consistency bounds for sev- eral surrogate families, and Mao et al. (2023b) conducted an in-depth analysis of the cross-entropy family. Building on these theoretical advancements, Awasthi et al. (2023) introduced a smooth algorithm that leverages consistency guarantees to enhance robustness in adversarial settings.
This work lays the theoretical and algorithmic foundations for adversarially robust Learning-to- to-fefer systems.
2. Related Works
Learning-to-Dete Learning-to-Ded addresses the prob- lem of allocating decisions between an automated model and offfine diccision-makers In contrast to classical selective cndliction-wiiee simply rejects uncertain inputs without further action (Chow, 2003; Bartlett & Wegkamp, 2008; Cortes et al., 2016; Geifman & EI- Yaniv, 2017)- ex- plicitly learns to defer to experts based on their distribution. Existing approaches fall into two broad categories: one- stage methods, where the predictor and deferral mechanism are trained jointly, and two-stage methods, where the defer- ral policy is optimized post hoc, assuming access to frozen predictions from pre-trate preitrined agents. This paper focuses on the two-stage setting.
Our work builds upon recent advancements in consistency theory to further improve adversarial robustness in two-stage L.2D.
3. Preliminaries
The one-stage formulation was initiated by Madras et al. (2018), who jointly trained a predictor and rejector in the binary setting. A major theoretical milestone was achieved by Mozannar & Sontag (2020), who established the first Bayes-consisten L2D method for multiclass classi- fication, ensuring asymptotically optimal delegation. This was followed by Verma et al. (2023), who introduced a one- versus-all surrogate formulation that also achieves Bayes- consistency and was later generalized to broader surrogate families by Charusaie al. (2022). To improve calibration across agents, Cao et al. (2023) proposed an asymmetric softmax surrogate that addresses the shortcomings identified in both Mozannar & Sontag (2020) and Verma et al. (2023). However, Mozannar et al. (2023) showed that these methods are not realizable-7H-conssstee C-nnnssst, potentially degrading per- formance when the hypothesis class is restricted. This issue
Multi-task scenario. We consider a multi-task setting that addresses both classification and regression problems simul- taneously. Let X' denote the input space, ) {1,..., ,n] represent the set of n distinct classes for classification, and T S R denote the target space for regression. Each data point is represented by a triplet 2 = (z, y, t) E Z, where 2 = X' x y x 'T. We assume the data is drawn inde- pendently and identically distributed (i.i.d) from an un- derlying distribution D over Z. To model this mulki-task problem, we introduce backbone w E W, which acts as a shared feature extractor. The backbone maps inputs æ E X to a latent feature representation y E Q, via the function w : X + Q. Building upon this backbone, we define a classifier h E H. representing all possible classification heads. Formally, h : Q x y + is a scoring function, with predictions computed as h(g) = arg max h(g. v).
2

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
Similarly, we define a regressor F, which maps latent features to real-valued targets, f 2 + T. These compo- nents are integrated into single multi-head network g E G. defined as 9 = {g: g(x)= = (h o w o w(x)) w E W. h E H. f F}.
et al., 2023c). To characterize consistency with partic- ular hypothesis set, Awasthi et al. (2022) introduced H- consistency bounds, which rely non-decreasing func- tion r R + R$ and take the following form:
Eog (h) - Egg, (H) Uugg (H) 2
(4)
Consistency in classification. In classification, the pri- mary objective is to identify a classifier h E H in the specific case where wxx) is the identity function, such that h(x)= =
& T(Eta
r
c (h) cos (H) + Mio
where the minimizability gap Mi (H) quantifies the dif- ference between the best-in-class excess risk and the ex- pected pointwise minimum error: Uea (H) = E (H) y)]] H. = Hall (Steinwart, 2007; Awasthi et al., 2022; Mao et al., 2024b; 2023e; 2024a). In the asymptotic limit, inequality (4) ensures recovery of Bayes-consistency (3).
arg max h(x,, y). This classifier should minimize the true error Ex., (h), defined as E (h) (a Lon (h, a, y)].
The Bayes-optimal error is expressed as E (H) = infnen Etg, (h). However, minimizing E (h) directly is challenging due to the non-differentaability of the true multi- class 0-1 loss (Zhang, 2002; Steinwart, 2007; Awasthi et al., 2022; Mao et al., 2025c; Zhong, 2025; Mao et al., 2024e; Cortes et al., 2025). To address this challenge, surrogate losses are employed convex, non-negative upper bounds on for A notable family of multiclass surrogate losses is the comp-sin sum (Mohri et al., 2012; Mao et al., 2023b), which we refer to as a family of multiclass surrogate losses:
Ex [infReи By [Lo1 x)]] The gap vanishes when
Adversarial robustness. Adversarially robust classifica- tion aims to train classifiers that are robust to small, im- perceptible perturbations of the input (Goodfellow et al., 2015; Madry et al., 2018). The objective to minimize the true multiclass loss for evaluated an adversarial input x' = 5 (Gowal et al., 2020; Awasthi et al., 2022). A perturbation ô is constrained by its magnitude, and we de- fine the adversarial region around x as Bp = {x' || - =|Ipp S 7), where ||Ipp is the p-norm and y E (0, 1) specifies the maximum allowed perturbation.
Pi (h, y) = T Σ Aリ
v.(h(z,))) -M(M (1)
where (v) exp(-u), which defines the cross-entropy family. For >> O, the transformation is given by:
Definition 3.1 (Adversarial True Multiclass Loss). The adversarial true multiclass loss toi H x X x + {0, 1} is given by:
log(1 + v)) - [(1 ((--))--- 1] if
1,
("(v)=
0 (2)
1]
Fothh,,,) sup for (h(2))))
This formulation generalizes several well-known loss func- tions, including the sum-exponential loss (Weston & Watkins, 1998). logistic loss (Ohn Aldrich, 1997), gen- eralized cross-entropy (Zhang & Sabuncu, 2018), and mean absolute error loss (Ghosh et al., 2017). The cor- responding true error for &% is defined as Eag, (h) = E( [ (h, x. v)], with its optimal value expressed as Egg, (H) = infREm Egg, (h).
Similarly to classification, minimizing eoz is computation- ally infeasible (Zhang & Agarwal, 2020; Bartlett et al., 2006; Awasthi et al., 2022). To address this, we introduce the fol- lowing surrogates.
Definition 3.2 (Adversarial Margin Surrogates). The family of adversarial margin surrogate losses T61" from the comp- sum p-margin family, which approximate the adversarial true multiclass loss Eoı is defined as:
A key property of surrogate loss is Bayes-consistency, which ensures that minimizing the surrogate excess risk leads to minimizing the true excess risk (Zhang, 2002; Bartlett et al., 2006; Steinwart, 2007: Tewari & Bartlett, 2005; Cortes et al., 2024; Mao et al., 2025b). Formally, &% is cayss-con with respect to f0 if, for any sequence (heJeer C H. the following implication holds:
Te."" (ht, x,,) = sup Vh 01 (',))(a.7) v'
Σ Voll((a's)) - lo(a',)))) .u))
Here, dO and VP represent transformations that charac- terize the behavior of the family. The piecewise-linear and non-convex transformation is defined as V,(c)
Eog, (h&) Eig, (H) k k+200 O
= Elo, (hx)
cB Egg (H) k-20 O.
min { max (0,1 (0,1-²) .1). Recent studies have shown that algorithms based on smooth, regularized variants of the comp-sum p- p-margin losses achieve H-consistency, thereby offering strong theoretical guarantees (Awasthi et al., 2022; 2023; Mao et al., 2023b).
This property typically assumes H. = Han- which may not hold for restricted hypothesis classes such as Hin or HReLu (Long& & Servedio, 2013; Awasthi et al., 2022; Mao
3

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Two-Stage Learning-to-Defer The Learning-to-Defer (L.2D) framework assigns each input X' to the most cost- effective agent, leveraging the strengths of multiple agents to improve task performance. Agents consist of a fixed primary model and a set of J offline experts. We denote the full agent set as A = {0} U [J], where j = O refers to the primary model g defined in Section 3. Each expert My outputs prediction pair (x) (mn) (z), m/ (z)), with m/ E J a categorical prediction and m/ (a)) T regression output. The complete set of expert predictions is denoted mxx) = (m (m(()))))((... E M. In the two-stage setting, all agents are trained offline, and only the allocation function is learned, keeping agent parameters fixed.
mal decision-making (Narasimhan et al., 2022; Mao et al., 2023a; 2024d; Montreuil et al., 2025c). Despite its effective- ness, we show that this framework is inherently vulnerable to adversarial attacks that exploit its reliance on the rejector function-a key component responsible for query alloca- tion. Given this critical role, our analysis concentrates on adversarial attacks and corresponding defenses targeting the rejector r E R. rather than individual agents. This emphasis is warranted by the fact that adversarial defenses for specific agents can typically be deployed offline within the two-stage 1.2D setup, whereas the rejector operates as a router at inference time, serving as the first layer in the archi- tecture to direct queries to the appropriate agent. Moreover, evaluating the robustness of individual agents under adver- sarial conditions reduces to selecting the most robust agent, whereas ensuring robustness the system level constitutes fundamentally different challenge.
A rejector R. defined trained to select the most suitable agent by computing r(x) = arg maxje j), as proposed by Mao et al. (2024d; 2023a); Montreuil et al. (2025c).
X' A IR, is
Untargeted Attack. In classification, the goal of an untar- geted attack is to find a perturbed input x' E Bp 7) that causes the classifier h E '. to misclassify the input (Good- fellow et al., 2015; Akhtar & Mian, 2018). Specifically, for clean input æ E X such that the classifier correctly predicts h(z) y. the attacker seeks a perturbed input x' such that h(x') # y-
Definition 3.3 (Two-Stage L2D Losses). Let x E X and r E R. The true deferral loss is defined as:
J Σ ( 2)))))) 1=0
(det(T, m, 2)
Its convex, non-negative, upper- bound surrogate family is given by:
In the context of Baarning-to-Dfferr the attack extends be- yond misclassification to compromising the decision allo- cation mechanism. Given an optimal agent j E AA, the attacker aims to find an adversarial input x' E Bp(x, 7) such that ra=') * jr, thereby resulting in deferral to a suboptimal agent j E A (i"), leading to an increased loss. This gives rise to the following untargeted attack formula- tion:
Tu der (r 9, m) = Σ Ty(o(s), 2) (1,2,,),
where Gj denotes the cost of assigning query x agent j E A (Madras et al., 2018). If r(z) = O, the decision is handled by the primary model g, which returns prediction g(x) = (h(w(x)), (w(())) and incurs cost (o(g(z), z) = ((((x),)) 2). where (:)) E R+ is a general loss function.
Definition 4.1 (Untargeted Attack in L2D). Let x' (p. be an adversarial input, where Bp(a,, 7) denotes the p-norm ball of radius 7 centered at r. The untargeted attack that maximizes misallocation in L2D is defined as:
If r(x) = j for some j > 0, the query is deferred to expert j, incurring a cost (m (z), 2) = (x)y(xz), 2) + By. where 3 2 is the consultation cost. The complementary cost function used in the surrogate is:
x' = arg max x'E j=0
Σ
(r,, 2')
Σ (L((()), m=(x),
Definition 4.1 characterizes an attack in which the adversary maximizes the overall surrogate loss incurred by misalloca- tion, thereby causing the system to defer to an unintended agent. The adversarial input x' is designed to maximize these surrogate losses. As a result, the attack increases the system's 'ss overall loss, leading to significant degradation in performance. An illustration of this attack is provided in Appendix Figure 2.
which matches the formulation from Mao et al. (2024d); Montreuil et al. (2025c). In the case of classification, vé corresponds to the standard 0-1 loss.
We illustrate how Leaming-to-Deffe works at inference in Figure 1.
4. Novel Adversarial Attacks on L2D
Targeted Attack.
Targeted attacks are often more impact- ful than untargeted ones, as they exploit specific system vulnerabilities to achieve precise adversarial goals (Akhtar
The two-stage L2D framework is designed to route queries to the most accurate agents, ensuring opti-
Motivation.
4

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
& Mian, 2018; Chakraborty et al., 2021). For example, in autonomous driving, targeted attack could deliberately misclassify a stop sign a speed limit sign, potentially leading to hazardous consequences. Such attacks exploit asymmetries in the task by forcing the classifier h E H.. to predict a specific target class y/2 E V/ thereby amplifying the risk of incorrect or dangerous decisions.
define the j-th adversarial true multiclass loss as
po (r,
sup foi (oi(r(2)).5)) 2)EB,(an)
which captures the worst-case misclassification loss when deferring to agent j under adversarial conditions. The ad- versarial true deferral loss is defined in Lemma 5.2.
In the context of L2D, the attacker seeks to manipulate the system assign a query z' Bp(a, 7) to a predetermined expert E A. rather than the optimal expert j" E A. This targeted attack can be formally expressed as follows:
Lemma 5.2 (Adversarial True Deferral Loss). Let x E X denote the clean input, Cj the cost associated with agent A, and Tj the aggregated cost. The adversarial true deferral loss Ede is defined as:
Definition 4.2 (Targeted Attack in L2D).. Let E ()) where (2, 7) denotes the p-norm ball of radius " centered at x. The targeted attack that biases the allocation of query x toward predetermined expert je E A is defined as
Eag(r, g, m)
Σ m())
j=0
+ (1 J) Σ e ()(()))
x' = arg min Ty. (g(x), m(x), 2) (r, .()). (')))(an)
See Appendix D.I for the proof of Lemma 5.2. The at- tacker's objective is to compromise the allocation process by identifying perturbations 5y that maximize the loss for each agent E A. Notably, the costs cj and Ty are evalu- ated on the clean input x. the agents" predictions remain unaffected by the perturbations (see Section 4).
The adversarial input x' minimizes the surrogate loss associ- ated with the target agent jr. thereby biasing the allocation process toward agent js (Definition 4.2).
For instance, an attacker may have an affiliated partner Jr among the system's is agents. Suppose the system operates under a pay-per-quens model-such as a specialist doctor in a medical decision-maker makinn system or third-party service provider in an Al platform. By manipulating the allocation mechanism systematically route more queries to jt, the attacker artificially inflates the agent's 's workload, resulting in unjustified financial gains shared between the attacker and the affiliated expert through direct payments, revenue- sharing, or other collusive arrangements. An illustration of this attack provided in Appendix Figure 3.
Remark 5.3. Simply applying adversarial evaluation to the entire two-stage loss (Definition 3.3) does not account for the worst-case scenario captured in Lemma 5.2. This point is further discussed in Appendix D.1.
As with standard classification losses, minimizing the adver- sarial true deferral loss in Lemma 5.2 is NP-hard (Zhang, 2002; Bartlett et al., 2006; Steinwart, 2007: Awasthi et al., 2022). Therefore, as in classification problems, we approxi- mate this discontinuous loss using surrogates.
5. Adversarially Consistent Formulation for Two-Stage Learning-to-Defer
Adversarial Margin Deferral Surrogate Losses. In the formulation of the adversarial true deferral loss (Lemma 5.2), discontinuities arise from the use of the indi- cator function in its definition. To approximate this discon- tinuity, we build on recent advancements consistency the- ory for adversarially robust classification (Bao et al., 2020; Awasthi et al., 2022; 2023; Mao et al., 2023b) and propose continuous, upper-bound surrogate family approximate it.
In this section, we introduce an adversarially consistent formulation of the two-stage L2D framework that ensures robustness against attacks while preserving optimal query allocation.
5.1. Novel Two-stage Learning-to- Formulation
Adversarial True Deferral Loss. To defend against the novel attacks introduced in Section 4, we define the worst- case adversarial true deferral loss, Eaet : R x G x M x Z + RT, which quantifies the maximum incurred loss under adversarial perturbations. Specifically, for each j E A, an adversarial perturbation d; is applied, resulting in a perturbed input x/j + ōy. where 2/ E (2) lies within an no-norm ball of radius 7-
Definition 5.4 (j-th Adversarial Margin Surrogates). We define the j-th adversarial margin surrogate family as
He.n (r, sup Th 2.EBp(um))
Σ - - r(a), f'
where Th" and 1 are defined in Equation (3.2). Based on this definition, we derive the adversarial margin deferral surrogate losses as:
Definition 5.1 (j-th Adversarial True Multiclass Loss). We
5

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
Lemma 5.5 (Adversarial Margin Deferral Surrogate Losses). Let E X denote the clean input and Tj the aggregated cost. The udversarial margin deferral surrogate losses Vou are then defined as:
aggregated cost. Then, the smooth adversarial surrogate family (SAD) & Wa is defined as:
Tamth def (ri g. m, = Σ T)((()), 00
amuth (r, x.
Tay (r,g, m, = Σ
a (r,
The proof is provided in Appendix D.4. While the SAD family offers a smooth and computationally efficient approx- imation of the adversarial true deferral loss, the question of its consistency remains a central concern.
The proof is provided in Appendix D.2. A key limitation of the adversarial margin deferral surrogate family lies in the non-convexity of the j-th adversarial margin surrogate loss 4g.%,, induced by the piecewise nature of Yp p()), which complicates efficient optimization.
5.2. Theoretical Guarantees
To establish the theoretical foundations of SAD, we first prove that the family of adversarial margin deferral sur- rogates B is Bayes-consistent and (R, -coonsisstt (Lemma 5.5). Wang & Scott (2023) introduced sufficient conditions for surrogate loss to satisfy Bayes-consistency, (Definition 3.1) and Phi- NDZ (Definition 3.2) are jointly sufficient to apply their general Theorem 2.6.
Adversarial Smooth Deferral Surrogate Losses. Prior work by Awasthi al. (2023); Mao al. (2023b) shows that the non-convex adversarial margin surrogate family can be approximated by a smooth and convex surrogate. Building on their results, we introduce the smooth adversarial surro- gale family, denoted as @01 smtt,, R x X' x A + RR, which approximates the supremum term in Lemma 5.2. Impor- tantly, &.mmth," serves as convex, non- non-negative upper bound for the j-th adversarial margin surrogate family, satisfying T.... < u We derive this surrogate in Lemma 5.6.
These
While our surrogate satisfies the Gamma-PD property, fails to meet the Phi-NDZ criterion because it is not dif- ferentiable on all of IR. As a consequence, Theorem 2.6 from Wang & Scott (2023) cannot be invoked directly to es- tablish clasificattonnccalirttio. This limitation motivates a dedicated analysis of the consistency properties of our sur- rogates. We show that both Bayes- and (R, -consisstyyyyy can nonetheless be established for the class T.... these guarantees extend to the regularized empirical vari- ant used in our algorithmic implementation, referred to as SARD.
Lemma 5.6 (Smooth Adversarial Surrogate Losses). Let E X denote the clean input and hyperparameters p > and 0. The smooth adversarial surrogate losses are defined as:
def and that
T 01
" (r,
(01(- x. P
+ v sup |1A, (2, 2)pB)(
(R. g)-consistency bounds of G.... A key step toward establishing theoretical guarantees is to demonstrate the R- consistency of the j-th adversarial margin surrogate losses T under the assumption that R is symmetric and that there exists a rejector r E R that is locally p-consistent.
The proof is provided in Appendix D.3. For x E X. define (z(z,i.3') r(z.j) - r(z,j"), and let (z, E IR denote the vector of pairwise dif- ferences (Ar ((,,5,,,,(,,,,)
1), (z, + 1),..., (,, The first term D81 (5.2..) corre- sponds to the multiclass surrogate losses modulated by the coefficient p. The second term introduces a smooth adver- sarial component over perturbations 1/ E Bp(x, 7), scaled by the coefficient v (Awasthi al., 2023; Mao al., 2023b). The hyperparameters (p, v) are typically selected via cross- validation to balance allocation performance and robustness.
Definition 5.8 (Locally p-consistent). A hypothesis set R is locally p-consistent if, for any x E X, there exists a hypothesis r E R such that:
|r(2', ((x1,)) 2p. x'EBp(u,7)
inf
where p> > 0, fj E A, and x' By 7) Moreover, the ordering of the values fr(a', j)} is preserved with respect to (r(x,))} for all x' B))
Using this surrogate, we detine the smooth adversarial defer- ral surrogate (SAD) family T b.mahh.. RRxxxMx2 + R+, which convex, non-negative, and upper-bounds Zest by construction. Its formal definition is as follows:
As shown by Awasthi et al. (2022); Mao et al. (2023b); Awasthi et al. (2023), common hypothesis classes-includdngg linear models, neural networks, and the set of all measurable functions are locally p-consistent for some p > Consequently, the guarantees established
Lemma 5.7 (SAD: Smooth Adversarial Deferral Surrogate Losses), Let x E X denote the clean input and Tj the
6

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
in Lemma 5.9 are broadly applicable in practice. See Ap- pendix D.5 for the proof.
thereby establishing both Bayes-consistency and (R, consistency of the surrogate losses &
Lemma 5.9 (R-consistency bounds for ) Assume R. is symmetric and locally p-consistent. Then, for the agent set A. any hypothesis r E R, and any distribution P with probabilities p= = (Po....,PP) E AlAl, the following inequality holds:
We now introduce the smooth adversarial regularized de- ferral (SARD) algorithm, which extends the standard SAD framework by incorporating regularization term. SARD retains the theoretical guarantees of & including consis- tency and minimizability under the same assumptions (Mao et al., 2023b; Awasthi et al., 2023).
Σ jEA
Pn Ps 01 (1,0,))
inf Σ FE
Pj (r,
jEA
Guarantees for SARD. We now extend our theoretical guarantees to the smooth adversarial deferral surrogate (SAD) family Ta leveraging the inequality T Z Lo.us Under the same assumptions as in Theorem 5.10, we obtain the following bound:
VI(1)
Pj T0 2,)
inf
Σ jEA
Pj ffn (1,2,,,) (1)2,)))
rER
jEA
Lemma 5.9 establishes the consistency of the j-th adversar- ial margin surrogate family T under adversarial pertur- bations defined for each j E A, weighted by distributional probabilities P5 AIAi. This result sets work apart from prior approaches (Mao et al., 2023b; Awasthi al., 2023; 2022), which do not account for adversarial inputs at the per-agent distributional level. By addressing this limi- tation, Lemma 5.9 provides a crucial theoretical guarantee: shows that the surrogate losses T align with the true adversarial losses Ei under appropriate conditions.
Corollary 5.11 (Guarantees for SAD). Assume R. is sym- metric and locally p-consistent. Then, for the agent set A, any hypothesis r E R, and any distribution D, the following holds for any multi-task model g E G:
Et (r:g) - 8 (R,9) Mi (R.
Eng
S v0)
(ggmn.(rr) - Eg (R) + +gga (R)
ad
+ Ecg (g)
(g) + Uco (9)- cB
Building on this result, we establish the Bayes- and (R, consistency of the adversarial margin deferral surrogate losses. The proof of Theorem 5.10 is provided in Ap- pendix D.6.
Corollary 5.11 shows that SAD satisfies similar consistency guarantees under the same assumptions as its non-smoon counterpart. In particular, the minimizability gap vanishes when R- Rall and G = Gan, further justifying the use of SAD in practice.
Theorem 5.10 ((R, -coosisstny bounds of ( Let R be symmetric and locally p-consistent. Then, for the agent set A, any hypothesis r E R, and any distribution DD the following holds for a multi-task model E G:
Motivated by this theoretical foundation, we introduce a learning algorithm that minimizes a regularized empirical approximation of SAD, recovering guarantees established in both Theorem 5.10 and Corollary 5.11. We refer to this algorithm as SARD (Smooth Adversarial Regularized Deferral).
(r,9)) -
E/y (R,9) + Uz (R,9)= S
1 (o(1) Egen (r) E san
(R) + Ulgg (R)
+ Eeg (g) - B (g) + Ueg (9).
Proposition 5.12 (SARD: Smooth Adversarial Regularized Deferral Algorithm). Assume R is symmetric and locally p-consistent. Let Ω: : R - RR+ be a regularizer and rj > O a hyperparumeter. The regularized empirical risk minimiza- tion objective solved by SARD is:
Theorem 5.10 establishes that the adversarial margin defer- ral surrogate family T is consistent with the adversarial true deferral loss Eaer- When the minimizability gaps van- ish-i.e., when R- Rall and g Gal (Steinwart, 2007: Awasthi et al., 2022) the bound reduces to:
K
T mmth (r,g, m, 2x) + 17 (())
Egn, (r.g) B (R, 9) < (g) e. (g) Ead
min PER
K
def
k=1
+ v1) (Egg. (
E E (R))
The pscudo-code for SARD is provided in Appendix C.
Assume that after offline training, the multi-task model satis- fies Erg (9) EB (G) I Eg. and the trained rejector satisfies Equ.n (r) T (R) < é₁. Then, the excess adversarial
5.3. Computational complexity of SARD
deferral risk is bounded as:
Having established statistical guarantees, we now quantify the complexity cost of SARD. Because training time can become a bottleneck for large-scale models, especially when
Eg (r.g) 9) - 8 ted
(R, S 6o + (1)e))
7

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
inner maximizations are involved, we spell out the precise forward/backwart counts so that future work can reproduce or improve our implementation.
6.1. Multiclass Classification Task
We compare our robust SARD formulation against the method introduced by Mao et al. (2023a) the CIFAR-100 dataset (Krizhevsky, 2009).
Proposition 5.13 (Epoch cost of SARD). Process n training examples in mini-batches of size B. Let A {0} U [J] be the set of deferral outcomes ([A] = J + 1) and let Ta E N be the number of projected- graiidentgrdienttttt steps used in the inner maximizations (PGDTa). Write Ctwa and Chwwa for the costs of one forward and one backward pass through the rejector network rq. Then one epoch of SARD minimization incurs
Setting: Categories were assigned to three experts with a correctness probability = 0.94, while the remaining probability was uniformly distributed across the other cat- egories, following the approach (Mozannar & Sontag, 2020; Verma et al., 2023; Cao al., 2023). To further evaluate robustness, we introduced weak expert M3, with only few assigned categories, and assumed that the at- tacker is aware of this weakness. Agent costs are de- fined as ()(h(z),)) = foi (h(x), v) for the model and cy>o(m) (z), y) = for (m,) (z),
n (Aaa)aa + Crowd)
network traversals, while the peak memory equals that of single forward-bakwant pass plus the storage of one adversarial copy of each input.
(z), V). aligned with (Mozannar & Sontag, 2020; Mozannar et al., 2023; Verma et al., 2023; Cao et al., 2023: Mao et al., 2023a). Both the model and the rejector were implemented using ResNet-4 (He et al., 2015). The agents' performance, additional training details, and experimental results are provided in Appendix E.
Proof. Per mini-batch SARD executes (i) one clean forward re: (ii) for each j A and each of the Tú PGD steps, one forward and one backward of ro; (iii) one backward pass to update 8. This totals 2(1 + | T.) network traversals. Multiplying by the [n/B] B] batches yields (5.13). Because PGD steps are processed sequentially, only the activations of the clean forward plus those of the current PGD iteration must reside in memory, giving the stated peak footprint.
Baseline
Clean
Untarg.
Targ. Mi
Targ. Ma
Targ. Ma
Mao et al. (2023a)
72.8 4 0.4
17.2 +: 0.2
0..4
45.4
13.4 D.1
67.0 0.4
49.8
6.34
62.1 1 0.2
64.8 0.3
Table I. Comparison of accuracy results between the proposed SARD and the bascline (Mao al., 2023a) the CIFAR-100 validation set, including clean and adversarial scenarios.
Comparison with Two-Stage L2D losses. For the stan- dard convex deferral loss & Σ j Tj Lū (r) the inner maximizations is absent (T, = 0), so its epoch cost is simply n (Ctwa + Cbwa). Equation 13)) therefore shows that SARD introduces a purely multiplicative overhead fac- tor + T..
The results in Table 1 underscore the robust- ness of our proposed SARD algorithm. While the baseline achieves a higher clean accuracy (72.8% vs. 67.0%). this comes at the cost of extreme vulnerability to adversarial attacks. In contrast, SARD prioritizes robustness, signif- icantly outperforming the baseline under adversarial con- ditions. Specifically, in the presence of untargeted attacks, SARD retains accuracy of 49.8%, 2.9 times improve- ment over the baseline's 's sharp decline to 17.2%. Similarly, under targeted attacks aimed at the weak expert M3. our method achieves 64.8% accuracy, a stark contrast to the baseline's 13.4%, highlighting SARD's ability to counteract adversarial exploitation of weak experts. These findings validate the efficacy of SARD in preserving performance across diverse attack strategies.
Results:
Although the factor in (5.13) appears large, the absolute overhead is modest on contemporary hardware. Moreover, the overhead scales linearly with both the number of experts |A| and the attack depth To. providing clear knobs for future work to trade robustness for wall-time. These observations guide our experimental choices in the next section.
6. Experiments
We evaluate the robustness of SARD against state-of-ther the-art two-stage L.2D frameworks across three tasks: classifica- tion, regression, and multi-task learning. Our experiments reveal that while existing baselines achieve slightly higher performance under clean conditions, they suffer from se- vere performance degradation under adversarial attacks. In contrast, SARD consistently maintains high performance, demonstrating superior robustness to both untargeted and targeted attacks. To the best of our knowledge, this is the first study to address adversarial robustness within the con- text of Learning-to -ooeene.t---efer....
6.2. Regression Task
We evaluate the performance of SARD against the method proposed by Mao et al. (2024d) using the California Housing dataset involving median house price prediction (Kelley Pace & Barry, 1997).
Setting: We train three experts, each implemented as an ML.,, specializing in specific subset of the dataset based
8

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
on predefined localization criterion. Among these, expert M3 is designed specialize in smaller region, resulting in comparatively weaker overall performance. Agent costs for regression are defined as co (2)) = (M))(f(a)) for the model and ( (z), t) RMSE(m; (ar), t), aligned with (Mao et al., 2024d). Both the model and the rejector are trained on the full dataset using MLPs. We pro- vide detailed agent performance results, training procedures, and additional experimental details in Appendix E.2.
Baseline
Clean
Untarg.
Targ. M₁
Targ. Mg
Montreuil al. (2025c)
44.4 + 0.4
9.7 0.1
17.4 + 0.2
20.4 + 0.2
Our
43.9 0.4
39.0 0.3
39.7 + 0.3
39.5 + 0.3
Table 3. Performance comparison of SARD with the bascline (Montreuil al., 2025c) the Pascal VOC dataset. The table reports mean Average Precision (mAP) under clean and adversarial scenarios.
Baseline
Clean
Untarg.
Targ. M;
Targ.
Targ.
mAP of 44.4 compared to 43.9 for SARD. However, under adversarial scenarios, the baseline experiences significant performance drop, with mAP decreasing to 9.7 in untar- geted attacks and 17.4 in targeted attacks on M1. In con- trast, SARD demonstrates strong robustness, maintaining mAP scores close to the clean setting across all attack types. Specifically, SARD achieves an mAP of 39.0 under untar- geted attacks and 39.7 when targeted M,, highlighting its resilience to adversarial perturbations.
Mao al. (20244) + 0.01
0.20 0.3
0.40 + 0.02
0.21 0.01
0.41 0.05
Our
0.17 +: 0.01
0.17 +: 0.0L
0.18 4: 0.01
0.18 4 0.01
0.18 4 0.01
Table 2. Performance comparison of SARD with the baseline (Mao et al., 2024d) on the California Housing dataset. The table reports Root Mean Square Error (RMSE).
Results: Table 2 presents the comparative performance of the baseline and SARD under clean and adversarial condi- tions. Under clean settings, both approaches achieve similar performance with an RMSE of 0.17. However, under ad- versarial attacks- both untargeted and targeted at specific experts (e.g., Ma)-SARD demonstrates significant robust- ness, maintaining an RMSE of 0.18 across all conditions. In contrast, the baseline's performance degrades substan- tially, with RMSE values increasing 0.29 and 0.41 under untargeted and M3-targeted attacks, respectively.
7. Conclusion
In this paper, we address the critical and previously under- explored problem of adversarial robustness in two-stage Learning-to-Defe systems. We introduce two novel ad- versarial attack strategies-uutargette and targeted- that exploit inherent vulnerabilities in existing L2D frameworks. To mitigate these threats, we propose SARD, robust defer- ral algorithm that provides theoretical guarantees based on Bayes consistency and (R, (R..)cccoiisttenc.. We evaluate our approach across classification, regression, and multi-task scenarios. Our experiments demonstrate the effectiveness of the proposed adversarial attacks in significantly degrad- ing the performance of existing two-stage L2D baselines. In contrast, SARD exhibits strong robustness against these attacks, consistently maintaining high performance.
6.3. Multi-Task
We evaluate the performance of our robust SARD algorithm against the baseline introduced by Montreuil al. (2025c) on the Pascal VOC dataset (Everingham et al., 2010), benchmark for object detection tasks combining both inter- dependent classification and regression objectives.
Setting: We train two Faster R-CNN models (Ren al., 2015) as experts, each specializing in a distinct subset of the dataset. Expert M, is trained exclusively on images containing animals, while expert M2 focuses on images with vehicles. Agent costs are defined as co(g(z),)) 2) = mAP(g(z)) 2) for the model and ()>a(m,(r)) 2) = mAP(m,(*)) z). aligned with (Montreuil et al., 2025c).
A Acknowledgment
This research is supported by the National Research Founda- tion, Singapore under its Al Singapore Programme (AISG Award No: AISG2-PhD--0222 20220-000-2--11111-111 and by A STTAR and is part of the programme DesCartes which is supported by the National Research Foundation, Prime Minister's Of- fice, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme.
The primary model and the rejector are implemented as lightweight versions of Faster R-CNN using MobileNet (Howard et al., 2017). We provide detailed performance results, training procedures, and additional experimental details in Appendix E.3.
Impact Statement
This paper introduces methods to improve the adversar- ial robustness of two-stage Learning-to-Defer frameworks, which allocate decision-making tasks between AI systems and human experts. The work has the potential to advance the field of Machine Learning, particularly in high-stakes
Table 3 presents the performance comparison between SARD and the baseline under clean and adversar- ial scenarios. Both methods perform comparably in clean conditions, with the baseline achieving a slightly higher
Results:

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
domains such as healthcare, finance, and safety-critical sys- tems, where robustmess and reliability are essential. By mit- igating vulnerabilities to adversarial attacks, this research ensures more secure and trustworthy decision decision-making pro- cesses.
Bartlett, P., Jordan, M., and McAuliffe, J. Convexity, classification, and risk bounds. Journal of the Amer- ican Statistical Association, 101:138-156, 2006, doi: 10.1198/0162 0447800060455000000000000000
Bartlett, P. L. and Wegkamp, M. H. Classification with a reject option using a hinge loss. Journal of Machine Learning Research, 98),, 2008
The societal implications of this work are largely positive, as it contributes to enhancing the reliability and fairness of Al systems. However, as with any advancement in adversarial robustness, there is potential for misuse if adversarial strategies are exploited for harmful purposes. While this paper does not directly address these ethical concerns, we encourage further exploration of safeguards and responsible deployment practices in future research.
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Smniić, N., Laskov, P., Giacinto, G., and Roli, F. Evasion Attacks against Machine Learning at Text Time, pp. 387-402. Springer Berlin Heidelberg. 2013. ISBN 9783642387098. doi: 00.1007997--36422--00
No immediate or significant ethical risks have been iden- tified in this work, and its societal impacts align with the well-established benefits of improving robustness in Ma- chine Learning systems.
Cao, Y., Mozannar, H., Feng, L., Wei, H., and An, B. In defense of softmax parametrization for calibrated and consistent learning to defer. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information P Processing Sys- lems 36: Annual Conference on Neural Information Pro- cessing Systems 2023, NeurlPS 2023, New Orleans, LA. USA, December 10 - 16, 2023, 2023.
References
Akhtar, N. and Mian, A. Threat of adversarial attacks on deep learning in computer vision: A survey. lece Access, 6:14410-1440 44300, 2018.
Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., and Mukhopadhyay, D. survey on adversarial attacks and defences. CAAI Transactions Intelligence Technology, 6(1):25-45, 2021.
Awasthi, P., Frank, N., Mao, A., Mohri, M., and Zhong, Y. Calibration and consistency of adversarial surrogate losses. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neu- ral Information Processing Systems 34: Annual Confer- ence on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 9804 9815, 2021
Charusaie, M., Mozannar, H., Sontag, D. A., and Samadi, S. Sample efficient learning of predictors that comple- ment humans. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), Interna- tional Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 2972- 3005. PMLR, 2022.
Awasthi, P., Mao, A., Mohri, M., and Zhong. Y. Multi-class h-consistency bounds. In Proceedings of the 36th Inter- national Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088.
Chow, C. On optimum recognition error and reject tradeoff. IEEE Transactions on information theory, 16(1):41-45 2003.
Awasthi, P., Mao, A., Mohri, M., and Zhong, Y. Theoreti- cally grounded loss functions and algorithms for adver- sarial robustness. In Ruiz, J. R., Dy, J. G., and van de Meent, J. (eds.), International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning Reseurch, pp. 10077-10094. PMLR, 2023.
Cortes, C., DeSalvo, G., and Mohri, M. Learning with rejection. In Ortner, R., Simon, H. U., and Zilles, S. (eds.), Algorithmic Learning Theory, pp. 67-82, Cham, 2016. Springer International Publishing. ISBN 978-3- 319-46379-7.
Cortes, Mao, A., Mohri, C., Mohri, M., and Zhong, Y. Cardinality-aware set prediction and top-k classification. Advances neural information processing systems, 37: 18265-18309, 2024.
Bao, H., Scott, C., and Sugiyama, M. Calibrated surrogate losses for adversarially robust classification. In Aber- nethy, J. D. and Agarwal, S. (eds.), Conference on Learn- ing Theory, COLT 2020, 9-12 July 2020, Virtual Event [Graz, Austria), volume 125 of Proceedings of Machine Learning Research, pp. 408 451, PMLR, 2020.
Cortes, C., Mao, A., Mohri, M., and Zhong, Y. Balancing the scales: A theoretical and algorithmic framework for learning from imbalanced data. In Forty-second Interna- tional Conference on Machine Learning, 2025.
10

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
Everingham, M., Van Gool, L.,, Williams, C., Winn, J., and Zisserman, A. The paseal visual object classes (voe) challenge. International Journal of Computer Vision, 88: 303-338, 2010. doi: 10.1007/s 1263-2096222000 009-0275-4.
June 2013, volume 28 of JMLR Workshop and Confer- ence Proceedings, pp. 801-809. JML..org. 2013.
Madras, D., Pitassi, T., and Zemel, R. S. Predict responsibly: Improving fairness and accuracy by learning to defer. In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa-Bianchi. N., and Garnett, R. (eds.), Advances Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018 December 3-8, 2018, Montréal, Canada, pp. 6150-6160, 2018.
Geifman, Y. and EI- Yaniv, R. Selective classification for deep neural networks. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural In- formation Processing Systems 30: Anmual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 4878 4887, 2017.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3. 2018, Conference Track Pro- ceedings. OpenReview.net, 2018.
Ghosh, A., Kumar, H., and Sastry, P. S. Robust loss func- tions under label noise for deep neural networks. In Singh, S. P. and Markovitch. S. (eds.). Proceedings of the Thirty-First AAA! Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pp- 1919-1925, AAAI Press, 2017.
Mao, A. Theory and Algorithms for Learning with Multi- Class Abstention and Multi-Expert Deferral. PhD thesis, New York University, 2025.
Goodfellow, J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference Learn- ing Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Mao, A., Mohri, C., Mohri, M., and Zhong, Y. Two-stage learning to defer with multiple experts. In Oh, A., Nau- mann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Pro- cessing Systems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023a.
Gowal, S., Qin, C., Uesato, J., Mann, T. A., and Kohli, P. Uncovering the limits of adversarial training against norm-bounded adversarial examples. ArXiv preprint, abs/2010.03593, 2020.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. соrr abs/1512.0333 (2015), 2015.
Mao, A., Mohri, M., and Zhong, Y. Cross-entropy loss func- tions: Theoretical analysis and applications. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Ma- chine Learning, ICML. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 23803-22005 23828. PMLR, 2023b.
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications, 2017.
Kelley Pace, R. and Barry, R. Sparse spatial autoregres- sions. Statistics and Probability Letters, 33(3):291-29) 1997. ISSN 0167-7152. doi: https://doi.or/110. 1016/
Mao, A., Mohri, M., and Zhong, Y. H-consistency bounds: Characterization and extensions. In Oh, A.. Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference
50167-7115296)0014403X
Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans,
Kingma, D. P. and Ba. J. Adam: A method for stochastic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Confer- ence Track Proceedings, 2015.
LA, USA, December 10 - 16, 2023c.
Mao, A., Mohri, M., and Zhong, Y. h-consistency guarantees for regression. In Salakhutdinov. R.. Kolter, Z., Heller, K., Weller, A., Oliver, N., Scar- lett, J. and Berkenkamp, F. (eds.). Proceedings of the 41st International Conference on Machine Learning. volume 235 of Proceedings of Machine Learning Research, pp. 34712-34733 PMLR, 21-27 Jul 2024a. URL https: /pppceeeiiigggsseeeeeeepp.e...ae mmlr. press/ press/v235// h//////3//3m2222/ 4g.. html.
Krizhevsky, A. Learning multiple layers of features from tiny images, 2009.
Long, P. M. and Servedio, R. A. Consistency versus re- alizable h- h-consistency for multiclass classification. In Proceedings of the 30th International Conference on Ma- chine Learning, ICML, 2013, Atlanta, GA, USA, 16-21

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Mao, A., Mohri, M., and Zhong, Y. Multi-label learning with stronger consistency guarantees. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U.. Tomezak, J. M.. and Zhang, C. (eds.), Advances in Neural Infor- mation Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024b.
Montreuil, Y., Yeo, S. H., Carlier, A., Ng, L. X., and Ooi, W. T. Optimal query allocation in extractive qa with Ilms: A learning-to-defer framework with theoretical guaran- tees, 2024.
Montreuil, Y., Carlier, A., Ng. L. X., and Ooi, W. T. Why ask one when you can ask k? two-stage learning-to-defer to the top-k experts, 2025a.
Mao, A., Mohri, M., and Zhong, Y. Realizable h-consistent and bayes-consistent loss functions for learning to defer. In Globersons, A., Mackey, L.,, Belgrave, D., Fan, A., Pa- quet, U., Tomczak, M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10- - 15, 2024, 2024c.
Montreuil, Y., Carlier, A., Ng, L. X., and Ooi, W. T. One- stage top-k learning-to-defer: Score-based surrogates with theoretical guarantees, 2025b.
Montreuil, Y., Heng, Y. S., Carlier, A., Ng. L. X., and Ooi, W. T. A two-stage learning-to-defer approach for multi- task learning. In Forty-second second International Conference on Machine Learning, 2025c.
Mao, A., Mohri, M., and Zhong, Y. Regression with multi- expert deferral. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024d.
Mozannar, H. and Sontag, D. A. Consistent estimators for
learning to defer to an expert. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 7076- 7087. PMLR, 2020.
Mao, A., Mohri, M., and Zhong, Y. A universal growth rate for learning with smooth surrogate losses. In Globersons, A. Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tom- czak, J. M., and Zhang, C. (eds.), Advances in Neural In- formation Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024e.
Mozannar, H., Lang, H., Wei, D., Sattigeri, P., Das, S., and Sontag, D. A. Who should predict? exact algorithms for learning to defer to humans. In Ruiz, F. J. R., Dy, J. G., and van de Meent, J. (eds.), International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning Research, pp. 10520- 10545. PMLR, 2023.
Mao, A., Mohri, M., and Zhong, Y. Mastering multiple- expert routing: Realizable ShS-consistency and strong guarantees for learning to defer. In Forty-second Interna- tional Conference on Machine Learning, 2025a.
Narasimhan, H., Jitkrittum, W., Menon, A. K., Rawat, A. S., and Kumar, S. Post-hoc estimators for learning to defer to an expert. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- -Decembber 9, 2022, 2022.
Mao, A., Mohri, M., and Zhong, Y. Enhanced $h$- consistency bounds. In 36th International Conference on Algorithmic Learning Theory, 2025b.
Mao, A., Mohri, M., and Zhong, Y. Principled algorithms for optimizing generalized metrics in binary classification. In Forty-second International Conference on Machine Learning, 2025c.
Ohn Aldrich, R. A. Fisher and the making of maximum likelihood 1912-1922. Statistical Science, 12(3):162-179 1997.
Meunier, L., Ettedgui, R., Pinot, R., Chevaleyre, Y., and Atif, J. Towards consistency in adversarial classification. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Infor- mation Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Ren, S., He, K., Girshick, R. B., and Sun, Faster R-CNN: towards real-time object detection with region proposal networks. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, Decem- ber 7-12, 2015, Montreal, Quebec, Canada, pp. 91-99, 2015.
Mohri, M., Rostamizadeh, A., and Talwalkar, A. Founda- tions of machine learning. MIT Press, 2012.
12

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Steinwart, How compare different loss functions and their risks. Constructive Approximation, 26:225-287, 2007.
In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 8792-8802, 2018
Strong, J., Men, Q.. and Noble, A. Towards human-Al collaboration in healthcare: Guided deferral systems with large language models. In ICML 2024 Workshop on LLMs and Cognition, 2024.
Zhong, Y. Fundamental Novel Consistency Theory: H-
Consistency Bounds. New York University, 2025.
Szegedy, C., Zaremba, W. Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. Intriguing properties of neural networks. Bengio, and LeCun, Y. (eds.), 2nd International Conference on Learning Representa- tions, ICLR 2014, Bansf, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
Tewari, A. and Bartlett, P. L. On the consistency of multi- class classification methods. In Auer, P. and Meir, R. (eds.), Learning Theory, 18th Annual Conference on Learning Theory, COLT 2005, Bertinoro, Italy, June 27- 30, 2005, Proceedings, volume 3559 of Lecture Notes in Computer Science, pp. 143-157. Springer, 2005. doi: 10.1007/1 15034151
Verma, R., Barrejón, D., and Nalisnick, E. T. Learning to defer to multiple experts: Consistent surrogate losses, confidence calibration, and conformal ensembles. In Ruiz, F. J. R., Dy, J. G., and van de Meent, J. (eds.), International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Va- lencia, Spain, volume 206 of Proceedings of Machine Learning Research, pp. 11415-11434, PMLR, 2023.
Wang, Y. and Scott, C. On classification callirattioni of gamma-phi losses. In Neu, G. and Rosasco, I.. (eds.), Proceedings of Thirty Sixth Conference on Learning The- ory, volume 195 of Proceedings of Machine Learning Research, pp. 4929-4951, PMLR, 2023.
Weston, J. and Watkins, C. Multi-class support vector ma- chines. Technical report, Citeseer, 1998.
Zhang, M. and Agarwal, S. Bayes consistency vs. h- consistency: The interplay between surrogate loss func- tions and the scoring function class. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, vir- tual, 2020.
Zhang. T. Statistical behavior and consistency of classifica- tion methods based convex risk minimization. Annals of Statistics, 32, 2002. doi: 10. ///// 0/0/111//////079120030
Zhang, Z. and Sabuncu, M. R. Generalized cross entropy loss for training deep neural networks with noisy labels.
13

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
A. Notation and Preliminaries for the Appendices
summarize the key notations and concepts introduced in the main text:
Input Space and Outputs:
A': Input space for X.
@ Latent representation q E Q.
y nn) Categorical output space for classification tasks.
T IR: Continuous output space for regression tasks.
Z = X y x T: Combined space of inputs and labels.
Learning-to-DDe to-ffr Setting:
- 0 [J]: Set of agents, where refers the primary model g (h, f), and J denotes the number of experts.
my(x) (m)) (Ir), m/ (rr):: Predictions by expert j, where m/ (a)) E E is categorical prediction and m
(x) E T is
regression estimate.
colg(z), ()(()), 2): The cost associated the multi-task model.
(>o(m(x), = (m)()) + Sy: The cost associated to the expert j with query cost 53 2 0.
b: x
Quantify the prediction's quality.
Hypothesis Sets:
W: Set of backbones :' Q
H: Set of classifiers h: Q y +
F: Set of regressors f Q T
G: Single multi-head network G {g: g(x) (h wee.hheeee.
R: Set of rejectors r X - A.
Adversarial Definitions:
d E Bp(x, 7): the adversarial input for the agent j E the p-norm ball (, = (2/ e e ||= - æ|1p S {}}
xr) = e ln (r, x.y. f): j-th Adversarial multiclass loss.
Tgin (r, x) supaçeB,(u,) a Ex V (r((ss,,")) j-th Adversarial margin surrogate losses,
providing differentiable proxy for Po
This notation will be consistently used throughout the appendices ensure clarity and coherence in theoretical and empirical discussions.
14

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
B. Illustration
B.1. Learning-to- in Inference
Allocate to the most confident
expert
((a)
Input
Router
Patient undergoing X-ray
r(x))
Poor classifier
Figure 1. Learning--oo Boaiig-tt--eeeeee Defer in inference. The input x is routed to the most cost-effective agent in the system by the rejector E R: in this case, the human expert.
B.2. Untargeted Attack
Allocate the poor classifier
He) Hel
expert
Input
Malicious Attacker é (ttt r(x)
Adversarial input 62
Router r(x')
Patient undergoing X-ryy
Poor classifier
Figure 2. Untargeted Attack: The malicious attacker perturbs the input to increase the probability that the query is assigned to less accurate expert, thereby maximizing classification errors. Rather than targeting specific expert, the attack injects adversarial noise disrupt the expert allocation process, leading to erroncous routing and degraded decision maasin..
B.3. Targeted Attack
Expert j. and the Attacker are associated
$
Allocate to the expert jt
Expert j
rer)
Input æ
Malicious Attacker targetting
Adversarial input
Router r(a')
Patient undergoing X-ray
x 6,
6, r(x') =
Poor classifier
Figure 3. Targeted Attack: The malicious attacker perturbs the input to increase the probability that the query is assigned to its associated agent. By manipulating the L2D system systematically route queries to this associate, the adversary ensures that the associate receives higher volume of queries, thereby increasing its cariings.
15

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
C. Algorithm
Algorithm 1 SARD Algorithm
Input: Dataset {T yke fa) 1==k:= multi-task model G, experts E M, rejector E R, number of epochs EPOCH, batch size BATCH, adversarial parameters (p, v), regularizer parameter learning rate X.
Initialization:
Initialize rejector parameters
for i to EPOCH do
Shuffle dataset {(akk yk, t=)
for each mini-batch B ((0k, yk, (4)) of size BATCH do Extract input-output pairs 2 (1, E B. Query model (xr) and experts mxx)
{ (Agents have been trained offline and fixed
Evaluate costs co (xx)) 2) and ()>0(m(z),22)
{Compute costs}
for = 0 to do
Evaluate rejector score r(z,j).
{Rejection score agent j (lg-ball perturbation for agent j
Generate adversarial input snj + 8g with 83 E ()),,)).
Run PGD attack on z/j
Bu || (1', (2,
{Smooth robustness evaluation]
Compute Adversarial Smooth surrogate losses Tor x).
end for
Compute the regularized empirical risk minimization:
ER (r; 0) BATCH >
Tamith. (r,g, 9, m, =) + 1752(1).
Update parameters 0: EQ 0 - - XV Pad (r;
{Gradient update)
end for end for Return: trained rejector model A.
D. Proof Adversarial Robustness in Two-Stage Learning-to-Defer
D.1. Proof Lemma
Lemma 5.2 (Adversarial True Deferral Loss). Let E X denote the clean input, Gj the cost associated with agent j E A.
and Tj the aggregated cost. The adversarial true deferral loss las is defined as:
Ere(T, g, m, m, = Σ ((() (r,
5=0
+ (1- Σ ((g(())
j=0
Proof. In adversarial training, the objective is to optimize the worst-case scenario of the objective function under adversarial inputs z' E (3(x,)) 7). For our case, we start with the standard L2D loss for the two-stage setting (Mao et al., 2024d; Montreuil et al., 2025c):
(oe(r, 9, m. x) = Σ ((())
1=0
(5)
J
Σ
(0((s)))(x),,) =) + + (1- JJ Σ cj ()((()),
0=
1=0
16

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
using
E Eimi()))))) co(g(x),2) + (x), 244 (g(()), m())
= 0
(6)
otherwise
Next, we derive an upper bound for Equation (5) by considering the supremum over all adversarial perturbations x' E (2, 7), under the fact that the attack is solely on the rejector E R:
( Et(lo()) m(x), z) ( + (1- Σ m()(()) xf (BBpp))) ==0
Pae((, g. m, 2) S
sup
==
However, the formulation in Equation (7) does not fully capture the worst-case scenario in L2D. Specifically, this formulation might not result robust approach, as it does not account for the adversarial input a²j Bp(r, 7) that maximizes the loss for every agent j E A. Incorporating this worst-case scenario, we obtain:
faa(r.g: m,
m, m) S Σ (l())) efeBp(a.))
sup
Infes) + J) Σ (())
(8)
Thus, formulating with the margin loss (=)
= r(x,)) maxy ((r, leads to the desired result:
(T 9, m, m2 2) Σ ()((()),
(0 + (1- Σ (((
sup efeBp(an)
(9)
Σ
(1- Σ
1=0
1=0
with (r, = SuPptEB, (=
D.2. Proof Lemma 5.5
Lemma 5.5 (Adversarial Margin Deferral Surrogate Losses), Let x E X denote the clean input and Tj the aggregated cost. The adversarial margin deferral surrogate losses def are then defined as:
def (r, g. m,
Σ m((), 2) 2) (r, x)
Proof. Referring to adversarial true deferral loss defined in Lemma 5.2, we have:
Tasllr 9, m, 2)
Σ
(l((x)
sup 2)B,
1e.(3,,0s0 + (1- J) Σ ((((()),,(s)))
1=0
j=0
( (tt) + (1- Σ
c(0(())
==U
By definition, Hemus upper bounds the j-th adversarial classification loss Pyn leading to:
Tanlr.g.m, < Σ (o(a)) m(x), m())
2) (r, + J) Σ ()(o(z),
(10)
3=0
==0
Then, dropping the term that does not depend on E R, leads to the desired formulation:
T.u def
g, m, 2) = Σ
Ty(o(t), m(xx, =) Da (r, j)
(11)
1=0
17

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
D.3. Proof Lemma 5.6
Lemma 5.6 (Smooth Adversarial Surrogate Losses). Let x E X denote the clean input and hyperparameters p > 0 and 0. The smooth adversarial surrogate losses are defined as:
uutk. (r, = 11 (-,x,,)) xz 01
+ sup [ (x, z zfEB,(rn)
Proof. Let x E denote an input and n E By 7) an adversarially perturbed input within an f--norm ball of radius for each agent. Let E be rejector. We now define the composite-sum p-margin losses for both clean and adversarial scenarios:
46 (r, 1
Σ (r(((a,)") -((,))) 1'*1
(12)
Jo.u (r, x.
sup (x,()
You
where Mc(v) = exp(-u). For u >> 0, the transformation 4 is defined as:
m-=1 (v) = log(1 + b), (v) 1
[(1 (1-0)=-
follows that for all and 2 O, have de (v) et al., 2023b).
1 S 1 ensuring that 4 I-Lipschitz over R (Mao (I+eja
Define (x))) = r( r(x,j') j') and let A, (xx j) denote the J-dimensional vector:
(x 1, ()...,,,,...........))
(x(I..)))
For any a > 0, with 4 non-decreasing and 1- I-Lipschitz:
Tot (r, 181 (r,
sup d)EBplan)) f'H
[pp( Ar Vp( 1000-A(a..)) (x,,,.))))
(13)
Since (z(2) !-Lipschitz, by the Cauchy-Schwarz inequality and for > vn
p.
To.. (r, I "f"" (r, +
(A, (z.j)ll2
sup afEBp(mm))
(14)
Using & (r,00)) 4/ ( Σ Ey Ve(r(xr)) r(x,,(()))) with Ve(v) exp(-r) and the fact that v ve(v/)) 2 v Vp(v), we
obtain:
(r, < tū (5 (5.2.s) +
sup (E,
(t)x))) - A,
(15)
Finally, we have the desired smooth surrogate losses upper-bounding
T Temth HauJ.
Th smmth. f = Φ (r. 22.5
(r,
+
sup
(A, - (x,
(16)
o (w,))
18

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
D.4. Proof Lemma 5.7
Lemma 5.7 (SAD: Smooth Adversarial Deferral Surrogate Losses). Let x E X denote the clean input and Tj the aggregated cost. Then, the smooth adversarial surrogate family (SAD) t mu is defined as:
def
$ 3.miih (r, 9, m,
Σ T)(a(z),
01 o (r, 20,
Proof. Using Lemma 5.5, we have:
del (r,g,, m. Σ
(r,
(17)
5=0
Since Tg Tanhh. by Lemma 5.6, we obtain:
4 Bamth (r, (,,g,,,,) 2)
7)(((r). 20 (r,
(18)
D.5. Proof Lemma 5.9
Lemma 5.9 (R-consistency bounds for Do...)) Assume R is symmetrrc and locally p-consistent. Then, for the agent set A. any hypothesis r E R, and any distribution P with probabilities (po....,PJ) E A (AI, the following inequality holds:
Σ
(r,
inf Σ (,
FER
SEA
jEA
(1)
Py 01
Far (r. - inf rER
Σ
Ps Po (1,
JEA
JEA
Proof. We define the margin as Pr (x. [) r(x,j) my rxx, j'), which quantifies the difference between the score of the j-th dimension and the highest score among all other dimensions. Starting from this, can define space (x) (r R inf ((,,)) Pr r(x))) 0} for Bp(a,, 7) for || =1Ip S >} representing hypothesis that correctly classifies the adversarial input. By construction, we have that x'S B2(
In the following, we will make use of several notations. Let p(r) = (p(x, O)... p(x, denote the probability distribution over A at point x E X. We sort these probabilities (p(x,j) : E .4} in increasing order (x) S Pmn(2) A ... S P. (x). Let R be hypothesis class for the rejector E with r X' x - R. We assume this hypothesis class to be symmetric implying that for any permutation of A and any R, the function Td defined by (=x = r(x, x(j) is also in for j E A. We similarly have r E R, such that r(z, (0)5),(( (1)5), r(, (J)E) sorting the scores (r(z,j) j E A} in increasing order.
For R symmetric and locally p-consistent, there exists r* E R with the same ordering of the A. regardless of any =tJ E B, 7). This implies (nf (r'(z', g) r (z'. d')l 2 for Vg # E A. Using the symmetry of R, we can find a r" with the same ordering of j E A.c. p(z, (k)rr) = (z) for any & E A:
VA r (a (0)))))) < r (c), (1)) sr (1).
(19)
We introduce new notation & = 2'10) corresponding to the k-th ordered adversarial input. For instance, if we have an ordered list (1/2 2),r (= (z( 1)}. using the notation we have {r- (0) r (1))),r"( (2)
19

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
We define conditional risk Cagnus parameterized by the probability Pj A along with its optimum:
Σ
Ti (r,
Py
JEA
(20)
(R,
Σ rER
inf
Py
Bol (r,
01
JEA
The optimum C Cggn is challenging to characterize directly. To address this, we instead derive an upper bound by analyzing Cagiuo (r", .2). In what follows, the mapping from to i is defined based on the rank of p(x. j) within the sorted list (2)]
(R, Ca (R, x) S Cagius (r", xz)
Σ p(x,))
sup
Vp"
Σ
V, -,(((16,)0-
JEA
J'EA 1'Fj
1-1
= Σ sup
p(z, plz, v²"
Σ
(r" (5% (4)
1= (2,))
Σ Up r (5%s (1)g)) - (St (SI.)))))) f=i+1
p(a, {0] 1V" " (r'(E) (1))) r- (() (o())) +J --) (v,(t)=1,wz
sup
(2,)))
sup
p(a, fi) v0 i) (v = Ve
ve 2 and
inf xfEBp(u,))
1n -
1=0 (2,))
Σ ( (f" and p(x) same ordering of j E A)
(21)
Then, assuming (x) # Ø and R symmetric, we have:
Aog ACjoms (r) = Cagus (r,x) Cageus (R, x) 2 (foms(r,2) - -Caams("",
p(a, (i)g Σ v (r(E%) (1ee)) r(E
+ +--1)
sup ()p(an)
(22)
Ple (J) 0
Then, for V, non negative, (p(v) 1 for S and Tha non-decreasing, we have that:
AC ACions (r,z) V(() +
sup p(I, - EP - (EB)(
2 r()
Σ Pn -) i)+ + Σ p(z, (J-
==0
((u (i))) 2 (1H)
Then RHS is equal to the above,
20

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
(D) (1(1)
plx, (J))
plu, [J - 13
(Cgg ("())((, (1) + v²(1) JEA
max - p(arr(o)))
v²(2)
p(a, (J - 2]1)
(
p(a, (0}))
(23)
(1)
(1)
(1)1)(10)
v"(2)
(())
(24)
:
(J)
(1) (3x)
Rearranging terms for (1) & (1) S (2 S ... S (J) and similarly for probabilities Pl)] 2 ... 2 (1))
leads to:
AC (1, r(()) + V1(1))
1maxp(x,)) - p(0, p(2))(x))
JEA
(25)
= -1) JEA
maxp(x.j) - (())
for any r E R. we have:
AC (r,0) = CCy (1,2) - -c (R
22
10
= =Epto,)] p(x,))
sup 1 10-00),)500-
inf FER
Σ p(u,)) sup 1 p.
SEA
cfEBp(un))
JEA
(1 p(z,r(()) ( + (x)
inf
[[1 p(a,,(()))) + (o))
FER
= (1 (1-)( + + (xx,(s) 101-(1111 (1-maxp(e,)) JEA
(R is symmetric and R2 * Ø)
maxp(x, - p(r, p( (x)
(26)
JEA
We therefore have proven that:
ACy (1,²) S V2(1)
AC ACgams
(27)
En - inf Σ
fo S v0(1)
Σ
pj 01 (r,2,5) - inf
Σ
pj By ( (r,
rER jEA
FER
jEA
jEA
jEA
D.6. Proof Theorem 5.10
Theorem 5.10 ((R9)) consistenccy bounds of ToH). Let R be symmetric and locally p-consistent. Then, for the agent set A, any hypothesis E R. and any distribution D, the following holds for multi-task model g E G:
Eg (r.9)
cB (R, Exs
+ 21 cd
(R,
v'(1) (Eg.. (r) E (R) + Uz (R)
+ Ec, - E0 (g) ) + Mcy (g).
21

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Proof. Using Lemma 5.5, have:
dr 9, m, = Σ m)x), (r,
(28)
We define several important notations. For a quantity R, we note w (g(x), w" (2) geo (w(g,))) =)], and the combination w² (xx) ins (w z)]:
B (2. f))], an optimum
())))())=) = (=()) = inf infgeslcolo(x).)) = (z))
(29)
otherwise
Furthermore,
Ex ak (m,,,) 2) (m)(),)) = =) inf m()) 2)] cz(2) +
(30)
otherwise
Next, we define the conditional risk C₂
associated to the adversarial true deferral loss.
C (r,g. = -Ey,tar
-=dd J) Σ ((0(x),
j=0
j=0
(31)
m()) {
Σ ()(((())
Now, we assume r E symmetric and define the space (x) = fr E R A > 0}. Assuming (x) * Ø. it follows:
(
Σ (()) beR, + ( +(1 (1- - J) Σ (4))
(32)
Intuitively, if & Rx)x), this means that there no that correctly classifies x' Bp(x, inducing an error of 1.
follows at the optimum:
CB (R. (x)) = inf del
+ +1 (( + (1 - J)
gEG.ERR
j-n
j=
inf nE
Σ (r + 1, + (1 - J) Σ ())()xx),x
inf Σ (m(() + (1- Σ
(m, (R,(x) #0, then 3r E (x)
PER 1=0
1=0
Σ (m)(z))
sup 1x())=)) + (1 - J) Σ (z))
1=0
HER
1=0
Σ 7) (m((),,)) maxF (1 J) Σ
こ (my (z),z)
JEA
0=
j=0
(33)
22

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
We can still work on making the last expression simpler:
F)(m(x), Σ (z) + Σ (z)z) +
Σ
Tx (me (m)((())) (z))
k=1
= -c)(x>> +
(m) (2), +
Tx (
(z) + (2) x)) +
Σ T (m)(())))(( (((1))))))))))
(34)
Je)()) +
Th (mz(z))
J (x)()) + Σ
3 (z)
==1
Then, reinjecting (34) in (33) gives:
J
C (R = Σ (m)())) - max
max (m)(),,
(35)
SEA
1=0
j = 0:
J
CB (R,
Σ (m), - -o((m(x),
1=0
(36)
Σ (t))
Σ (m)((()))
1=
==1
(xx)
if if 0:
c (R, xx)
Σ (m) - -)))((((x).
Edd
1=0
(37)
Σ (m)) - (m(()) + Σ Cmm x))p1))
1=0
Therefore, it can be reduced to:
C Ess
(R,9,2) min (m min ( jEA
(38)
JCA
We can write the calibration gap as AC; dr
(r,g,z) == Ci (r)g,)) cc (R, G. 2 0, it follows:
AC (r, = Cial (r:, g)
min
e (m, (2)) x))
jEA
= (r, (r,
minn 2 (g(z), (x),))))))))))))))))))) +
min mmin (0(())((()))))))))))) m,(x), x) - min (m²((),,))
(39)
jEA
jEA
jEa
1
23

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
Term B: Let's first focus on B. We can write the following inequality:
B min (g((()))))))))))))(())) m)(), - min JEA
5 (c(((())) -z)()))
(40)
JEA
Indeed, we have the following relationship:
1. (f min (, = B = ==](g(x) (x)
2. if co(((z)) > min ( and c7(x)) S minjels (z))
B = (m) (x)x S (o(g(()) - (x)
Term A: Then using the term A:
A = Car (r, g.
min (m))
%EA
Cid (1,9,
inf Cz (rig,))
rER
(41)
Σ j=0
m 2
inf Σ FR
(T)((()), m(x), x (r, (1,
Now, we introduce a change of variables to define a probability distribution = (Po, .P)) E A A accounting for the fact that Ty does not inherently represent probabilities. Consequently, for each j E A, we obtain the following expression:
7, (o((), Pj Σ T m)
(for T {ry 2 (}}ea)
(42)
|
We then, have:
Σ (p5th1((,21)))))
inf
E (p)())).22))))
Σ
(43)
j=
Then, using Lemma 5.9, leads to:
pr pg (r, (0..0..1)
A (1)
Py Tems (r,
inf
Σ
FER
j=0
= (1)
(g((()), m(x), x) (r, (v.20.1)
Σ FER 1=0
inf
(r
(r, s..2..)))]
|T|
j=0
(44)
J
v1)
Σ (((((x) 2) T (r.
inf
Σ
(T)(9(x). m(x), x) (r, (....
FR
j=0
= -V"(1) (extt)(Caan(1 Cannarr (r,z) - Ca Man (R,
Then, adding B leads to:
AC Rid (r.g.z) A A (using Eq. 39)
(45)
S (I) Caan (r, (r,x) - C Ca.. (R, + (o(o())) - (6())
By construction, have 2))xx),)) (oo 2)] with co(g(x), v()(x),)) and <5(x)
inggeg (ysxxlc((0(x),).) =)]. Therefore, we can write for E G and the cost op:
ACc, (g, x) co(g(x), (x)
(46)
24

---

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
Then,
A 4d (r.g.z) v²(1) Coer (r, -z. (R, (R, + ACoo (2(x),))
(47)
-1) ACgg (T, Acro ())
Therefore, by definition:
E (r.g) - 8 (R. +1;
11 (R,9) = E, [AC Fest
(r, g. g.2)]
(L) AC A (1: BB c (g(x), x)
v'(1) (Egar (r) Eg (R) + Uzan (R)
+ Eg (9()))))
(48)
-w"(1) (Eggr (r) -g (R) + Ugga (R)
+ +E, 1o (o))) - -E,(0(x)
-1"(1) (ggz (r) - -E S (R) + 22 20 (R)
+ (g)
where aE (g) (g) & Uas (4. In the special case of the log-softmaa (u 1), we have that (1) log(2).
E. Experiments details
We present empirical results comparing the performance state-of-the-ar Two-Stage Learning-to Searning-to-Defee to-feerr frameworks (Mao et al., 2023a; 2024d; Montreuil et al., 2025c) with our robust SARD algorithm, To the best of our knowledge, this is the first study to address adversarial robustness within the context of Learning-to-Deefrr
All baselines the log-softmax surrogate for oo with (y==(()) log(1 + u) and (.( cxp(-v). Adversarial attacks and supremum evaluations over the perturbation region (2, evaluated using Projected Gradient Descent (Madry et al., 2018). For each experiment, we report the mean and standard deviation over four independent trials to account for variability in results. Experiments are conducted on one NVIDIA H100 GPU, Additionally, we make our scripts publicly available.
E.1. Multiclass Classification Task
Experts: We assigned categories to three distinct experts: expert M₃ is more likely to be correct on 58 categories, expert M2 on 47 categories, and expert M3 on 5 categories. To simulate a realistic scenario, we allow for overlapping expertise, meaning that for some x E X, multiple experts can provide correct predictions. On assigned categories, an expert has a probability p= = 0.94 to be correct, while following uniform probability if the category not assigned.
for (h(x). y) for the model and (m) (x), foi (m²) (z:), v), consistent with (Mozannar & Sontag, 2020; Mozannar et al., 2023; Verma et al., 2023; Cao et al., 2023; Mao et al., 2023a). We report respective accuracies of experts in Table 4.
Agent costs are defined co(h(x),
Model: We train the classifier offline using a ResNet-4 architecture (He et al., 2015) for 100 epochs with the Adam optimizer (Kingma & Ba, 2015), a learning rate of 0.1, and a batch size of 64. The checkpoint corresponding to the lowest empirical risk on the validation set is selected. Corresponding performance is indicated in Table 4.
Baseline (Mao et al., 2023a): We train a rejector using a 4 (He et al., 2015) architecture for 500 epochs, a learning rate 0.005, a cosine scheduler, Adam optimizer (Kingma & Ba, 2015), and a batch size 2048. We report performance of the checkpoints corresponding to the lower empirical risk on the validation set.
25

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
Model
Expert M1
Expert M2
Expert M3
Accuracy
61.0
53.9
45.1
5.8
Table 4. Agent accuracies the CIFAR-1O1 100 validation Since the training and validation sets pre-determined in this dataset, the agents' knowledge remains fixed throughout the evaluation.
We train rejector using the ResNet-4 architecture (He et al., 2015) for 1500 epochs with a learning rate of 0.005, cosine scheduler, the Adam optimizer (Kingma & Ba, 2015) with L. weight decay of 10-4 acting regularizer, and batch size of 2048. The hyperparameters are set to 1 and 0.01. The supremum component from the adversarial inputs is estimated using PGD40 (Madry et al., 2018) with e = 8/ 8/255, the &o norm, and a step size of e/40, following the approach in (Mao et al., 2023b; Awasthi et al., 2023).
SARD:
Baseline
Clean
Untarg.
Targ. Model
Targ. M1
Targ. M2
Targ. M3
Mao et al. (2023a)
72.8 + 0.4
17.2 +: 0.2
61.1 0.1
54.4 + 0.1
45.4 + 0.1
13.4 + 0.1
Our
67.0 = 0.4
49.8 + 0.3
64.8 + 0.2
62.4 + 0.3
62.1 + 0.2
64.8 + 0.3
Table Comparison of accuracy results between the proposed SARD and the baseline (Mao al., 2023a) on the CIFAR-100 validation set, including clean and adversarial scenarios.
E.2. Regression Task
Experts: We train three experts offline with three layers MLPs (128, 64, 32), each specializing in specific subset of the dataset based on predefined localization criterion. The first expert M1 trains Southern California (latitude lower than 36), the second expert M2 on Central California (latitude between 36 and 38.5), and the last in Northern California (otherwise) representing smaller area. MLPs are trained using a ReLU, an Adam optimizer (Kingma & Ba, 2015), a learning rate of 0.001, and 500 epochs. Agent costs are defined as (f(f(z), t) = RMSE(g(2)) t) for the model and (ms0(m) (x), t) (M))(m,(z),))) t), consistent with (Mao et al., 2024d). We report respective RMSE of experts in Table E.2.
Model: We train a regressor using a two-layer MLP with hidden dimensions (64, 32) on the full training set. The model uses ReLU activations, the Adam optimizer (Kingma Ba, 2015), learning rate of 0.001, and is trained for 500 epochs. We report performance of the checkpoints corresponding the lower empirical risk the validation set. The model performance is reported in Table E.2.
Model
Expert Mj
Expert Mg
Expert M3
RMSE
0.27 + .01 1.23 + .02
1.85 + .02
0.91 + .01
Table 6. Agent RMSE on the California Housing validation set (20% of the dataset).
Baseline (Mao et al., 2024d); We train a rejector using a MLP (8,16) for 100 epochs, a learning rate of 0.01, a cosine scheduler, Adam optimizer (Kingma & Ba, 2015), and a batch size of 8096. We report performance of the checkpoints corresponding to the lower empirical risk on the validation set.
We train rejector using MLP (8, 16) for 400 epochs, learning rate of 0.01, cosine scheduler, Adam optimizer (Kingma & Ba, 2015) with L2 weight decay of 10 acting as regularizer, and a batch size of 8096. The hyperparameters are set to p = 1 and v = 0.05. The supremum component from the adversarial inputs is estimated using PGD10 (Madry et al., 2018) with e equal to 25% of the variance of dataset's features, the foo norm, and a step size of c/10, following the approach in (Mao et al., 2023b; Awasthi et al., 2023).
SARD:
26

---

Adversarial Robustness in Two-Stage Learning-to-Defer Algorithms and Guarantees
Baseline
Clean
Untarg.
Targ. Model
Targ. M
Targ. M2
Targ. M3
Mao (2024d)
0.17 + 0.01
0.29 = 0.3
0.19 0.01
0.40 + 0.02
0.21 0.01
0.41 0.05
Our
0.17 + 0.01
0.17 0.01
0.17 0.01
0.18 0.01
0.18 0.01
0.18 0.01
Table Performance comparison of SARD with the baseline (Mao et al., 2024d) on the California Housing dataset. The table reports Root Mean Square Error (RMSE) under clean and adversarial scenarios.
E.3. Multi Task
Experts: We train two specialized experts using a Faster R-CNN (Ren et al., 2015) architecture with MobileNet (Howard al., 2017) backbone. The first expert, M1, is trained on images containing animals, while the second expert, M2, is trained on images containing vehicles. Both experts are trained using the Adam optimizer (Kingma & Ba, 2015) with learning rate of 0.005, batch size of 128, and trained for 50 epochs. Agent costs are defined as (o(g(z) z) mAP(g(z); =) for the model and (y>o(my (x), 2) mAP(m (x), 2), consistent with (Montreuil et al., 2025c). We report respective mAP of experts in Table E.3.
Model: We train an object detection model using a larger Faster R-CNN (Ren et al., 2015) with ResNet-50 FPN (He et al., 2015) backbone. We train this model with Adam optimizer (Kingma & Ba, 2015), a learning rate of 0.005, batch size of 128, and trained for 50 epochs. We report performance of the checkpoints corresponding to the lower empirical risk on the validation set. The model performance is reported in Table E.3.
Model
Expert M₁
Expert M2
MAP
39.5
17.2
20.0
Table 8. Agents mAP Pascal VOC validation set. Since the training and validation sets are pre-determined in this dataset, the agents' knowledge remains fixed throughout the evaluation.
Baseline (Montreuil et al., 2025c): We train rejector using a Faster R-CNN (Ren et al., 2015) with MobileNet backbone (Howard et al., 2017) and classification head. We train this rejector for 70 epochs, learning rate 5e-4, a cosine scheduler, Adam optimizer (Kingma & Ba, 2015), and a batch size of 256. We report performance of the checkpoints corresponding to the lower empirical risk on the validation set.
SARD: We train a rejector using a Faster R-CNN (Ren et al., 2015) with a MobileNet backbone (Howard et al., 2017) and a classification head for 70 epochs with a learning rate of 0.001, a cosine scheduler, the Adam optimizer (Kingma Ba, 2015) with L2 weight decay of 10-4 acting as regularizer, and batch size of 64. The hyperparameters are set to p = and 0.01. The supremum component from the adversarial inputs is estimated using PGD20 (Madry et al., 2018) with 8/255, the loo norm, and step size of e/20, following the approach in (Mao al., 2023b; Awasthi et al., 2023).
Baseline
Clean
Untarg.
Targ. Model
Targ. M₁
Targ. M2
Montreuil et al. (2025c)
44.4 0.4
9.7 + 0.1
39.5 + 0.1
17.4 + 0.2
20.4 + 0.2
Our
43.9 0.4
39.0 0.3
39.5 - 0.1
39.7 0.3
39.6 =
Table 9. Performance comparison of SARD with the baseline (Montreuil et al., 2025c) on the Pascal VOC dataset. The table reports mean Average Precision (mAP) under clean and adversarial scenarios.
27


Reviews:
[{"review_id": "KzwGa5vO1f", "reviewer": "Reviewer_dgrk", "summary": "This paper investigates the two-stage learning to defer (L2D) frameworks under adversarial attacks. The authors introduce two novel attacks: untargeted and targeted, that exploit structural weaknesses in L2D systems. Then the authors propose the SARD algorithm, a robust, convex deferral mechanism that is both Bayes and (R,G)-consistency. SARD ensures optimal task allocation under adversarial perturbations and demonstrates robust performance across classification, regression, and multi-task benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful and constructive feedback. We are glad that they\nfound our contributions meaningful, and we appreciate their recognition of the soundness\nof both our theoretical and empirical results. \n> In the experiments, [...] with real-world expert predictions (e.g., CIFAR-10H) can be included.\n\n**We want to emphasize that the primary contribution of our paper is theoretical**. The experiments presented are primarily intended to empirically support and illustrate these theoretical results. \n    \nWithin the L2D community, synthetic experts are widely preferred for theoretical evaluation precisely because they allow controlled, systematic analyses across diverse conditions (e.g., specialized expert behaviors, and critical edge-case scenarios) that are generally not reproducible with currently available real-world expert labels [2,3,4,5,6]. For instance, we introduce synthetic experts to rigorously expose vulnerabilities to our novel targeted attack—a scenario that is difficult to construct using existing real-world dataset, yet remains plausible in practical deployments.\n\nNonetheless, we acknowledge the value of evaluating real-world datasets and consider this for future empirical exploration.\n> j-th adversarial margin surrogate closely resembles the structure of Gamma-Phi losses [...] relevant to this study.\n   \nVery good question! We confirm that our surrogate can be rewritten in a Gamma-Phi form $\\widetilde{\\Phi}^{\\rho,u,j}_{01}(r, x, j)= \\sup\\_{x\\_j' \\in B\\_p} \\gamma ( \\sum\\_{j' \\neq j} \\phi \\big( r(x\\_j', j') - r(x\\_j', j) \\big))$ with $\\gamma(v)=\\log(1+v)$ (assuming $v=1$) and $\\phi(v)=\\min(\\max(0,1 - v/\\rho), 1)$. However, despite satisfying the Gamma-PD condition (Definition 3.1 in [1]), our surrogate does not satisfy Definition 3.2 (Phi-NDZ), as $\\phi(v)$ is not differentiable on $\\mathbb{R}$. Thus, we cannot directly leverage the conclusions of Theorem 2.6 from [1] to prove classification-calibration.\n    \nFurthermore, the classification-calibration provided by [1] implicitly assumes the hypothesis class includes all measurable functions $\\mathcal{R}_{\\text{all}}$, a strong assumption that does not hold in our analysis. **Additionally, while classification-calibration implies Bayes-consistency—an asymptotic property—our Theorem 5.7 provides stronger finite-sample guarantees via explicit inequalities, without highly restricting the hypothesis class $\\mathcal{R}$**.\n    \nWe will explicitly discuss these distinctions in the revised manuscript. \n> The training [...] post-hoc estimator for L2D [2].\n\nThanks for the suggestion—we will include it. In fact, [5] can be viewed as an extension to the multi-expert setting, whereas [2] addresses only the single-expert case.\n> There are some minor formatting issues in the citations [...] preprints.\n\nThank you for pointing this out.  We will correct this in the revised manuscript\n> This paper primarily focuses on attacks targeting the deferral rule, whereas attacks on the classification rule  [...] combination as a direction for future work.\n\nWe agree that investigating combined attacks represents an interesting direction for future research.\n\nHowever, we would like to emphasize that the motivation for our work stems from the two-stage L2D setting [2,5,6], where the rejector is solely responsible for allocating each query to an external agent. In this setup, the external agents are fixed and not accessible beyond their output predictions. That is, we do not have access to their internal parameters, decision boundaries, or training pipelines; moreover, they may not even be describable as functions (e.g. human decision-makers).\n        \nAs such, while we acknowledge that attacks on the classification and deferral rules are not mutually exclusive in general, the structure of our problem precludes joint attacks. We cannot meaningfully design or evaluate perturbations that target the expert prediction, as we lack the ability to interact with or analyze the internal behavior of the experts. This naturally restricts our adversarial analysis to the deferral mechanism, which is the only component under the learner’s control.\n        \n**Moreover, robustness in classification has been extensively studied. In contrast, robustness in L2D systems remains unexplored. Our work aims to address this gap**. \n\nWe will clarify this in the revised manuscript.\n### References\n[1] Wang et al. (2023). On classification-calibration of gamma-phi losses. COLT23\n\n[2] Narasimhan et al. (2022) Post-hoc estimators for learning to defer to an expert. NeurIPS22\n\n[3] Mozannar et al. (2021). Consistent estimators for learning to defer to an expert. ICML21\n\n[4] Verma et al. (2023). Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles. AISTATS23\n\n[5] Mao, et al. (2023). Two-Stage Learning to Defer with Multiple Experts. NeurIPS23\n\n[6] Mao, et al. (2024). Regression with multi-expert deferral. NeurIPS24", "claims_and_evidence": "All the theoretical claims are supported by detailed proofs.", "methods_and_evaluation_criteria": "The evaluation criteria and selected baseline methods are appropriate for assessing the two types of attacks.", "theoretical_claims": "I reviewed the proof of Lemma 5.1 and did not find any apparent flaws.", "experimental_designs_or_analyses": "In the experiments, various type of expert predictions are simulated for different datasets, while the soundness of the experimental design can be improved if results on datasets with real-world expert predictions (e.g., CIFAR-10H) can be included.", "relation_to_broader_scientific_literature": "While the authors focuses on the problem of adversarial robustness, the authors further discuss the H-consistency of the proposed methods.", "essential_references_not_discussed": "1. (1) and the j-th adversarial margin surrogate closely resembles the structure of Gamma-Phi losses, whose consistency and construction are thoroughly analyzed in [1]. A detailed discussion of this work is encouraged, examining whether the conclusions in [1] can simplify the proofs in this paper or inspire new insights relevant to this study.\n\n2. The training of the allocation rule follows a post-hoc approach, similar to the framework of the post-hoc estimator for L2D [2]. \n\n[1]. Wang, Y. and Scott, C. On classification-calibration of gamma-phi losses. In Conference on Learning Theory, pages 4929–4951, 2023.\n\n[2]. Narasimhan, H., Jitkrittum, W., Menon, A., Rawat, A., Kumar, S. Post-hoc estimators for learning to defer to an expert. Advances in Neural Information Processing Systems, 2022.", "other_strengths_and_weaknesses": "1. The setting of this paper is comprehensive, encompassing classification, regression, and their multitask variants. By addressing multiple learning paradigms, the study ensures broad applicability and relevance across diverse machine learning tasks.\n\n2. The proposed attacks are intuitive and align naturally with real-world adversarial scenarios.", "other_comments_or_suggestions": "There are some minor formatting issues in the citations. For example, citations [1] and [2] should refer to their published versions rather than preprints.\n\n[1]. He, K., Zhang, X.,Ren, S., and Sun, J. Deep residual learning for image recognition, 2015. URLhttps://arxiv.org/abs/1512.03385. \n\n[2]. Mao, A., Mohri, M., and Zhong, Y. Realizable h-consistent and bayes-consistent loss functions for learning to defer, 2024c. URLhttps://arxiv.org/abs/2407.13732.", "questions_for_authors": "This paper primarily focuses on attacks targeting the deferral rule, whereas attacks on the classification rule are more common in the multiclass classification setting. In general, these two types of attacks are not mutually exclusive. It is recommended to explore their combination as a direction for future work.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w2Eqf5yqLf", "reviewer": "Reviewer_689i", "summary": "This paper identifies that Learning-to-Defer frameworks are vulnerable to adversarial attacks and introduces two attack strategies: untargeted attacks that disrupt allocation and targeted attacks that redirect queries to specific agents. The authors propose SARD, a robust algorithm with theoretical guarantees based on Bayes-consistency and (R,G)-consistency. Experiments that while existing frameworks suffer severe performance degradation under attacks, SARD maintains consistent performance in both clean and adversarial conditions.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. We are grateful for your recognition of the rigor in our theoretical contributions and the strength of our empirical validation. \n\n> The computational complexity [...] practical deployment considerations.\n\nThank you for highlighting this important consideration. Let $\\mathcal{F}$ denote the computational cost of performing a single forward–backward pass for the rejector model $r \\in \\mathcal{R}$. For standard L2D approaches involving $|\\mathcal{A}| = J + 1$ agents, the complexity is $\\mathcal{O}(\\mathcal{F} + J)$. SARD, involves performing adversarial training through a PGD, which requires $T$ forward–backward passes per agent. Consequently, the computational complexity of SARD is $\\mathcal{O}\\bigl((J+1) T \\mathcal{F}\\bigr)$. We will explicitly clarify this complexity comparison in the revised manuscript.\n\n> The paper doesn't explore whether [...] as an alternative solution.\n\n**We demonstrate that naively applying standard adversarial training to existing L2D baselines (Definition 3.1) fails to minimize the desired worst-case deferral loss (as shown in Lemma 5.1)**. As a result, an attacker could still exploit the allocation mechanism if adversarial training is applied to the non-worst-case deferral loss defined in Definition 3.1.\n        \nThis motivates our design and optimization of a distinct loss function—one that fundamentally departs from the standard adversarial training objective. We will further clarify this in the main body of the paper.\n\n> Minor typo on page 5: [...] missing the complete definition.\n\nThank you for pointing this out. We will correct this.\n\n> How does SARD's performance change [...] compare to standard adversarial training approaches?\n\nWe did not observe any unexpected behavior from SARD compared to standard adversarial training [7,8,9]. Intuitively, as the adversarial budget increases, the problem becomes inherently more challenging to defend against. From a theoretical standpoint, SARD does not exhibit any particular breakdowns beyond those already known for standard adversarial training.\n\n> Did you explore the effectiveness [...] with simpler adaptations of existing methods.\n\nYes, we explored this explicitly in our experiments. In each considered scenario, we evaluated baseline approaches both on clean datasets and under our novel attacks with a PGD [13]. **We demonstrate that existing baseline methods are highly vulnerable to both our attacks, while our proposed approach, SARD, consistently outperforms these baselines by a significant margin under adversarial conditions**. \n\n> How sensitive is SARD [...] practical guidelines for implementation?\n\nIndeed, SARD's performance depends on these hyperparameters, similar to smoothing approach in adversarial training [7,8]. Unlike previous observations in multiclass classification [7, 8], SARD generally achieves better performance with relatively smaller values of $\\nu$. This suggests that our L2D formulation inherently requires a lighter adversarial regularization term, likely due to the increased complexity of the decision boundaries involved.\n        \nWe will explicitly clarify these practical guidelines.\n\n> For real-world deployment [...] benefits would clearly outweigh the slight decrease in clean performance?\n\n**L2D frameworks are specifically designed for scenarios where decision-making carries important risks, thereby making robustness an essential consideration**. Consider a hospital deploying L2D for cancer diagnosis. Ideally, L2D would allocate cancer detection queries to the most suitable agent available. Suppose the hospital has several agents: a neurologist, a dermatologist, a general practitioner, and a properly trained AI model, each with distinct consultation costs (e.g., neurologist: $5\\beta_1$, dermatologist: $5\\beta_1$, general practitioner: $\\beta_1$, AI model: $0$). \n\nIn this realistic scenario, an adversary might deliberately manipulate the L2D system (rejector) through the attacks we have introduced: our untargeted attack (Definition 4.1) could cause misallocation of queries, leading to critical cases, such as complex skin cancer detection, being incorrectly routed from the dermatologist to a less specialized agent like the AI model, increasing the risk of making a mistake. **Alternatively, using our targeted attack (Definition 4.2), an adversary might intentionally route straightforward cases to expensive experts (e.g., neurologist) to unnecessarily increase costs ($5\\beta_1$ instead of $0$), or maliciously redirect consultations to a collaborating agent motivated by financial gain**.\n\nGiven these potential vulnerabilities, we firmly believe that the benefits of robustness clearly outweigh minor decreases in clean-data performance in such high-stakes settings. This motivates our approach, especially as L2D frameworks are increasingly adopted in critical applications [10, 11].\n\nSee discussion with reviewer @BFHK for references.", "claims_and_evidence": "The main claims are well-supported by evidence。 The vulnerability of existing L2D frameworks is convincingly demonstrated through empirical results. And the robustness of SARD is supported by consistent performance across clean and adversarial conditions in all three tasks.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are appropriate for the problem with diverse tasks and sufficiently large dataset.", "theoretical_claims": "The theoretical claims are sound and well-supported by detailed proofs:\n\nLemma 5.6 establishes R-consistency bounds for the j-th adversarial margin surrogate losses, building on consistency theory for adversarially robust classification.\nTheorem 5.7 extends these bounds to the full adversarial margin deferral surrogate losses, showing how they relate to the adversarial true deferral loss.", "experimental_designs_or_analyses": "The experiments overall validate the theoretical claims and demonstrate the practical benefits of SARD across different application domains.", "relation_to_broader_scientific_literature": "It provides an valueable perspective.", "essential_references_not_discussed": "It covers most essential reference", "other_strengths_and_weaknesses": "- The computational complexity and training overhead of SARD compared to baseline methods are not discussed, which is important for practical deployment considerations.\n- The paper doesn't explore whether standard adversarial training approaches could be directly applied to the baseline models as an alternative solution.", "other_comments_or_suggestions": "- There is a noticeable performance trade-off - SARD consistently achieves slightly lower performance on clean data compared to baselines in exchange for robustness. While this is a common challenge in adversarial robustness research, a more explicit discussion of this trade-off would be valuable.\n- There is a noticeable performance trade-off - SARD consistently achieves slightly lower performance on clean data compared to baselines in exchange for robustness. While this is a common challenge in adversarial robustness research, a more explicit discussion of this trade-off would be valuable.\n- Minor typo on page 5: \"we define the j-th adversarial true multiclass loss\" appears to be missing the complete definition.", "questions_for_authors": "- How does SARD's performance change as the perturbation budget increases? Is there a point at which the theoretical guarantees break down, and if so, how does this compare to standard adversarial training approaches?\n- Did you explore the effectiveness of standard adversarial training approaches (e.g., PGD-AT) applied directly to the baseline models? This would help clarify whether the benefits come specifically from your novel formulation or could be achieved with simpler adaptations of existing methods.\n- How sensitive is SARD to the choice of hyperparameters ρ and ν? Did you observe any consistent patterns during hyperparameter tuning that could serve as practical guidelines for implementation?\n- For real-world deployment in critical applications, how would you recommend practitioners balance the trade-off between clean performance and robustness? Are there specific scenarios where you believe the robustness benefits would clearly outweigh the slight decrease in clean performance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}, {"review_id": "mtLPMXrqa5", "reviewer": "Reviewer_BFHK", "summary": "This paper addresses adversarial robustness in two-stage Learning-to-Defer (L2D) frameworks by introducing two new attack strategies: untargeted attacks, which disrupt agent allocation, and targeted attacks, which redirect queries to specific agents. To counter these attacks, the authors propose SARD, a robust, convex deferral algorithm grounded in Bayes and (R, G)-consistency. Experimental results validate both the effectiveness of the attacks and the robustness of the proposed defense.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful evaluation and appreciate the recognition of our experiments and theoretical claims. In the following, we clarify our motivations, provide justification for our design choices (e.g., the multi-task emphasis in Section 3), and highlight the technical challenges addressed in deriving SARD. We also discuss how our proposed attacks go beyond existing methods and why they are theoretically and practically significant in the L2D context.\n> One issue with the paper is that Section 3 begins with the multi-task [...] discussing multi-task learning as an extension.\n\nOur motivation for initially framing Section 3 within the multi-task setting was not to imply that L2D is predominantly studied or especially relevant only to multi-task problems. Rather, we aimed to emphasize the generality of our theoretical results, demonstrating explicitly that the agent costs $c_j$ can take any positive form—classification [5], regression [6], or any multi-task metrics. **We intended to highlight that our proofs, losses, attacks, and defense methods hold broadly across these different problem formulations, a fact that is non-trivial [5,6] and significant for demonstrating the versatility of our approach**. \n\nIn simple words, our approach (SARD) can be used in any setting. We thank you for pointing out this potential misunderstanding and will clarify this in the final version. \n> The paper does not sufficiently highlight the challenges and technical difficulty of the problem [...] The technical difficulty of deriving SARD beyond this theoretical stacking is unclear.\n\nWhile the base attack strategies we adapt were initially developed for multiclass classification, a key contribution of our work lies in their novel extension to the fundamentally different setting of L2D. **Specifically, we are the first to show explicitly how these attack methods can be repurposed to strategically corrupt query routing decisions, which we believe is not trivial**. Importantly, our attacks target the rejector itself (responsible for allocation decisions), rather than the agents performing the tasks, which is an important distinction to multiclass classification. This design choice stems from a core characteristic of the two-stage L2D setting: we do not have access to the internal structure or training procedure of the agents, as also assumed in [4,5].  \n    \nWe provide a concrete example and detailed discussion (see response to reviewer @689i) highlighting why robustness against such novel attacks is critical, given the high-stakes decisions typically addressed by L2D systems.\n    \nRegarding technical complexity, we emphasize that the losses, attack strategies, and theoretical results introduced in our paper are entirely novel and require dedicated analysis, as formalized in Theorem 5.7. While previous works (e.g., [7,8,9]) have studied adversarial surrogate losses, their analyses are confined to multiclass classification and do not extend naturally to the L2D. **As discussed with reviewer @689i, applying standard adversarial training directly to the L2D loss (Definition 3.1) overlooks the worst-case allocation scenario—captured in Lemma 5.1—and thus leaves the system vulnerable to attacks**. Moreover, even in the absence of adversarial perturbations, proving consistency guarantees for L2D is nontrivial, as demonstrated in recent literature [3,4,5,6,12]. Furthermore, Lemma 5.6 and its proof are entirely original and differ from prior results (e.g., [7,8,9]). Notably, our theoretical framework explicitly addresses worst-case deferral loss by introducing adversarial inputs tailored for each individual agent $j \\in \\mathcal{A}$—a novel and technically challenging aspect that is not present in standard multiclass classification analyses.\n\nWe hope this clarifies the challenges involved, and we would be glad to elaborate further if needed. \n\n### References\n\n[3] Mozannar et al. (2021). Consistent estimators for learning to defer to an expert. ICML21\n\n[4] Verma et al. (2023). Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles. AISTATS23\n\n[5] Mao, et al. (2023). Two-Stage Learning to Defer with Multiple Experts. NeurIPS23\n\n[6] Mao, et al. (2024). Regression with multi-expert deferral. NeurIPS24\n\n[7] Awasthi, et al. (2023). Theoretically Grounded Loss Functions and Algorithms for Adversarial Robustness. AISTATS23\n\n[8] Mao et al. (2023). Cross-entropy loss functions: theoretical analysis and applications. ICML23\n\n[9] Bao, et al. (2021) Calibrated surrogate\nlosses for adversarially robust classification. COLT21\n\n[10] Strong et al. (2025). Towards Human-AI Collaboration in Healthcare: Guided Deferral Systems with Large Language Models. AAAI25\n\n[11] Joshi et al. (2021). Learning-to-defer for sequential medical decision-making under uncertainty. TMLR21\n\n[12] Mao et al. (2024). Principled Approaches for Learning to Defer with Multiple Experts. ISAIM24", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Theoretical claims look sound.", "experimental_designs_or_analyses": "The experimental results seem to be sound.", "relation_to_broader_scientific_literature": "The results of this paper are related to the literature of learning-to-defer and adversarial robustness.", "essential_references_not_discussed": "Yes.", "other_strengths_and_weaknesses": "- One issue with the paper is that Section 3 (Preliminaries) begins with the multi-task scenario, which may misleadingly suggest that the paper primarily focuses on multi-task learning. This raises questions: Is Learning-to-Defer (L2D) predominantly studied in the multi-task setting, or is adversarial robustness particularly relevant to multi-task learning? If neither is true, the choice to emphasize multi-task learning from the outset needs clearer justification. A possible improvement would be to restructure the section by first introducing classification, then regression, and finally discussing multi-task learning as an extension.\n\n- The paper does not sufficiently highlight the challenges and technical difficulty of the problem. One of the main contributions—introducing two new attack strategies—feels somewhat straightforward, as it mainly exploits existing vulnerabilities rather than proposing novel attack methods. Similarly, while the proposed defense incorporates consistency guarantees, it appears to be largely a combination of existing theories from learning-to-defer consistency and H-consistency bounds for adversarial robustness. The technical difficulty of deriving SARD beyond this theoretical stacking is unclear.", "other_comments_or_suggestions": "N/A.", "questions_for_authors": "N/A.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eK6H3rpfmK", "reviewer": "Reviewer_DEy6", "summary": "This paper studies adversarial robustness in the L2D paradigm. Based on rigorous theoretical results, they propose a novel method called SARD to improve the adversarial robustness of L2D models.\n\nConcretely, this paper first presents untargeted and targeted attacks on the L2D, based on optimization on the commonly adopted adversarial loss, tailored to the L2D setting. They then develop approximations of the worst-case loss and then apply smoothing (lemma 5.3) on the proposed loss to ease optimization. Consistency bounds are then developed for the losses.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their encouraging and constructive feedback. We appreciate their recognition of the rigor of our theoretical contributions and the strength of our experimental results in supporting our claims. We also acknowledge and value their observation that, to the best of their knowledge, this is the first work to study adversarial robustness in L2D. Please find our clarifications below.\n\n> The authors checked the performance of the proposed methods in classification, regression, and multi-task settings, which sufficiently supported the generalization of their methods. However, it would be better if they could evaluate more datasets for each setting; currently, each task involves a single dataset.\n\n> Could you include more datasets in the evaluation of each task?\n\nWe would like to emphasize that the primary objective of our paper is to establish a rigorous theoretical foundation for defending against adversarial attacks in the Learning-to-Defer  setting. Our experiments primarily serve to empirically validate the proposed theoretical framework, rather than to benchmark performance across a wide range of datasets. Given the theoretical focus and current standards within the L2D community, we believe our existing experiments across classification, regression, and multi-task scenarios sufficiently demonstrate generality (see discussion with reviewer @dgrk). \n            \nNonetheless, exploring more datasets remains a valuable direction for future empirical work.\n\n> While the authors do not present many reasoning about why this is important, I believe that evaluating from adversarial perspective for a possibly deployed paradigm is useful. \n\nThank you for recognizing the importance of evaluating Learning-to-Defer from an adversarial perspective. We strongly agree that such robustness evaluation is crucial. **L2D models are increasingly deployed in high-stakes applications, including healthcare [10, 11], or more broadly autonomous decision-making, where adversarial manipulation leading to incorrect deferral decisions can have severe real-world consequences**. Specifically, targeted attacks can strategically force the rejector to allocate more queries to a particular agent, thereby causing a harmful bias in the allocation process. On the other hand, untargeted attacks degrade the overall performance and reliability of the entire system, making its behavior unpredictable by maximizing the occurrence of errors. \n        \nWe further discuss potential misuse of L2D in the discussion with reviewer @689i and will clarify these critical points in our revised manuscript.\n\n\n### References\n[10] Strong et al. (2025). Towards Human-AI Collaboration in Healthcare: Guided Deferral Systems with Large Language Models. AAAI25\n\n[11] Joshi et al. (2021). Learning-to-defer for sequential medical decision-making under uncertainty. TMLR21", "claims_and_evidence": "The claims are sufficiently supported.\n\nConcretely, the authors claim the development of novel attack and adversarial training methods on the L2D, and claim consistency bounds on the correponsding loss functions.", "methods_and_evaluation_criteria": "The method makes sense, and the evaluation has no evident flaw.", "theoretical_claims": "I did not check the formal proofs; nevertheless, the derived method based on their theories work quite well.", "experimental_designs_or_analyses": "The authors checked the performance of the proposed methods in classification, regression, and multi-task settings, which sufficiently supported the generalization of their methods. However, it would be better if they could evaluate more datasets for each setting; currently, each task involves a single dataset.", "relation_to_broader_scientific_literature": "To the best of my knowledge, this paper is the first to evaluate adversarial robustness and effectively propose an adversarial training method for the L2D paradigm. \n\nWhile the authors do not present many reasoning about why this is important, I believe that evaluating from adversarial perspective for a possibly deployed paradigm is useful.\n\nThis work is not of general interest to the adversarial machine learning community because it only studies a specific setting, but it is of interest to those studying L2D.", "essential_references_not_discussed": "The literature is sufficiently discussed.", "other_strengths_and_weaknesses": "Since this work is the first to discuss adversarial robustness for a specific family of models (L2D), I recommend accept.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Could you include more datasets in the evaluation of each task?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: In the Learning to Defer (L2D) paradigm, a classifier is allowed to defer to an expert on samples it deems to be of low confidence. This paper studies the vulnerability of two-stage L2D approaches to adversarial attacks, and proposes a family of surrogate losses for L2D that are robust to such attacks. They establish consistency guarantees for the proposed losses and demonstrate their efficacy through experiments.

The reviewers are generally positive about the paper, and the authors satisfactorily addressed the questions raised during the rebuttal process.  Aside from concerns that the paper addresses a niche area, and requires a trade-off between performance and robustness, the reviewers were appreciative of the theoretical contributions. 

We are happy to accept this paper, and strongly encourage the authors to incorporate the promised clarifications in their response, including discussions on the computational complexity, on the technical challenges involved, on additional related papers, and on practical use-cases in high-stakes application.
Decision: Accept (poster)