SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Rattana Pukdee
rpukdee@cs.cmu.edu
dravyans@cs.cmu.edu
The problem of designing learners that provide guarantees that their predictions are provably correct is of increasing importance in machine learning. However, learning theoretic guarantees have only been considered in very specific settings. In this work, we consider the design and analysis of reliable learners in challenging test-time environments as encountered in modern machine learning problems: namely 'adversarial' test-time attacks (in several variations) and 'natural' distribution shifts. In this work, we provide a reliable learner with provably optimal guarantees in such settings. We discuss computationally feasible implementations of the learner and further show that our algorithm achieves strong positive performance guarantees on several natural examples: for example, linear separators under log-concave distributions or smooth boundary classifiers under smooth probability distributions.
The question of providing reliability guarantees on the output of learned classifiers has been studied previously in the classical learning setting where the training and test data are independent and identically distributed (i.i.d.) draws from the same distribution [RS88, EYW10, EYW12]. Conceptually, a reliable learner outputs a prediction and may output a correctness guarantee. We know that the learner is correct on all points with the guarantee as long as the learning-theoretic assumptions hold, e.g., realizability. While a trivial model that abstains from providing any guarantee is also a reliable learner, we are interested in a reliable learner that provides the guarantee on as many points as possible (useful in the sense of Rivest and Sloan [RS88]). [EYW10] provides a characterization of optimal reliable learners in this classical learning setting.
However, the assumption that the training and test data are drawn from the same distribution is often violated in practice. The mismatch may take the form of a 'natural distribution shift' when the test distribution is different from the training distribution or 'adversarial attacks' when there is an adversary that can perturb a test data point with the goal of changing the model prediction. This is frequently accompanied by a significant performance drop, as well as the inability to guarantee the usefulness of the algorithm. As a result, there is a significant interest in the study of test-time attacks [GSS15, CW17, MMS $^{+}$ 18] and distribution shift [LWS18, RRSS19, MTR $^{+}$ 21] among the applied machine learning community. Furthermore, recently there has been growing interest in the theoretical machine learning community for designing approaches with provable guarantees under test time attacks [AKM19, MHS19, MHS22] as well as renewed interest in distribution shift [BDBCP06, MMR08, HK19]. All the prior theoretical work in the literature has mainly focused on the effect of attacks or distribution shift on average error rate (e.g. [DBCP06, AKM19]). However,
this neglects a major relevant concern for users of machine learning algorithms, namely the ability to provide correctness guarantees for individual predictions: i.e., reliability.
In this work, we advance this line of work by developing a general understanding of how to learn reliably in the presence of corruptions or changes to the test set, specifically under adversarial test-time attacks as well as distribution shift between the training (source) and test (target) data.
Our results. We consider algorithms that provide robustly-reliable predictions which are guaranteed to be correct under standard assumptions from statistical learning theory, for both test-time attacks and distribution shift. Our first main set of results tackles the challenging case of adversarial test-time perturbations. For this setting, we introduce a novel compelling reliability criterion on a learner that particularly captures the challenge of reliability under the test-time attacks. Given a test point  $z$ , a robustly-reliable classifier either abstains from prediction, or outputs both a prediction  $y$  and a reliability guarantee  $\eta$  with the guarantee that  $y$  is correct unless one of two bad events has occurred: 1) the true target function does not belong to the given hypothesis set  $\mathcal{H}$  or, 2) a test-point  $z$  is perturbed from its original point by adversarial strength of at least  $\eta$  (measured in the relevant metric). In the case of distribution shift, we provide novel analysis and a complexity measure that extends the classical notion of reliable learning to the setting when the test distribution is allowed to be an arbitrary new distribution.
Conceptual advances over prior work. Prior works on certified robustness [SKL17, CRK19, WLF22] have examined pointwise consistency guarantees. The certified robustness guarantee is only that a prediction does not change with an adversarial perturbation, but it does not guarantee that the
prediction is correct (neither for the original point nor the perturbation); in particular, a constant function is always certified robust but it may not be useful. In contrast, our notion of robustly-reliable learner guarantees that, for any test point  $x$  and perturbation  $z$ , if  $z$  has a distance less than  $\eta$  to  $x$  ( $\eta =$  reliability radius), then the prediction will be "correct" (robust loss zero) in a sense informed by which robust loss we are addressing; we discuss this idea for several different losses, leading to different interpretations of this guarantee. In particular, for the stability loss, the prediction being "correct" means that it predicts the true label of the original point  $x$ ; this implies certified robustness, but is even stronger, since it also guarantees the correct label. Prior work [BBHS22] introduces the notion of a robustly-reliable learner for poisoning attacks which is different from our definition that is tailored to test-time attacks with a guarantee in terms of a reliability radius. In distribution shifts setting, we are the first to assess the transferability of reliability guarantees which differ from a widely-studied metric of average error rate. For additional related work, we refer to Appendix A.
2 Preliminaries and problem formulation
Let  $\mathcal{X}$  denote the instance space and  $\mathcal{Y} = \{0,1\}$  be the label space. Let  $\mathcal{H}$  be a hypothesis class. The learner  $\mathcal{L}$  is given access to a labeled sample  $S = \{(x_i,y_i)\}_{i=1}^m$  drawn from a distribution  $\mathcal{D}$  over  $\mathcal{X} \times \mathcal{Y}$  and learns a concept  $h^{\mathcal{L}}: \mathcal{X} \to \mathcal{Y}$ . In the realizable setting, we assume we have a hypothesis (concept) class  $\mathcal{H}$  and target concept  $h^* \in \mathcal{H}$  such that the true label of any  $x \in \mathcal{X}$  is given by  $h^*(x)$ . In particular,  $S = \{(x_i,h^*(x_i))\}_{i=1}^m$  in this setting. Given the 0-1 loss function  $\ell: \mathcal{H} \times \mathcal{X} \to \{0,1\}$ , define  $\mathrm{err}_S(h,\ell) = \frac{1}{m}\sum_{(x,y) \in S}\ell(h,x)$ . We use  $\mathcal{D}_{\mathcal{X}}$  to denote the marginal distribution over  $\mathcal{X}$ . We use  $\mathbb{I}[\cdot]$  to denote the indicator function that takes values in  $\{0,1\}$ . We also define  $B_{\mathcal{D}}^{\mathcal{H}}(h^*,r) = \{h \in \mathcal{H} \mid \operatorname*{Pr}_{\mathcal{D}}[h(x) \neq h^*(x)] \leq r\}$  as the set of hypotheses in  $\mathcal{H}$  that disagree with  $h^*$  with probability at most  $r$ . During test-time, the learner makes a prediction on a test-point  $z \in \mathcal{X}$ . We consider the following settings
1. Adversarial test-time attack. We consider adversarial attacks with perturbation function  $\mathcal{U}: \mathcal{X} \to 2^{\mathcal{X}}$  that can perturb a test point  $x$  to an arbitrary point  $z$  from the perturbation set  $\mathcal{U}(x) \subseteq \mathcal{X}$  [MHS19]. We assume that the adversary has access to the learned concept  $h^{\mathcal{L}}$  as well as the test point  $x$ , and can perturb this data point to any  $z \in \mathcal{U}(x)$  and then provide this perturbed data point to the learner at test-time. We want to provide pointwise robustness and reliability guarantees in this setting. We will assume that  $x \in \mathcal{U}(x)$  for all  $x \in \mathcal{X}$ . For any point  $z$ , we have  $\mathcal{U}^{-1}(z) := \{x \in \mathcal{X} | z \in \mathcal{U}(x)\}$ , the set of points that can be perturbed to  $z$ . We use perturbation to refer to a point  $z \in \mathcal{U}(x)$  and the perturbation sets  $\mathcal{U}(x)$  interchangeably.
2. Distribution shift. We consider when a test point  $z$  is drawn from a different distribution from the training samples. In this case, we want to provide a pointwise reliability guarantee. We will discuss more on this in Section 5.
2.1 Robust loss functions
In the applied and theoretical literature, various definitions of adversarial success have been explored, each dependent on the interpretation of robustness; depending on whether the perturbed point must have the same label as the original point, or in lieu of this, whether the algorithm should predict the true label of the perturbed point, or the same label as the original point. To capture these, we formally consider the following loss functions.
Definition 1 (Robust loss functions). For a hypothesis  $h$ , a test point  $x$ , and a perturbation function  $\mathcal{U}$ , we consider the following adversarially successful events.
1. Constrained Adversary loss  $[SZS^{+}14, BBSZ23]$ . There exists a perturbation  $z$  of  $x$  that does not change the true label of an original point  $x$  but  $h(z)$  is incorrect.
$$
\ell^{h^{*}}_{CA}(h,x) = \sup_{\substack{z\in \mathcal{U}(x)\\ h^{*}(z) = h^{*}(x)}}\mathbb{I}[h(z)\neq h^{*}(z)].
$$
For a fixed perturbation  $z\in \mathcal{U}(x)$  , define  $\ell_{CA}^{h^*}(h,x,z) = \mathbb{I}[h(z)\neq h^* (z)\wedge h^* (z) = h^* (x)].$
2. True Label loss [ZL19, GKKW21]. There exists a perturbation  $z$  of  $x$  such that  $h(z)$  is incorrect.
$$
\ell_ {T L} ^ {h ^ {*}} (h, x) = \sup  _ {z \in \mathcal {U} (x)} \mathbb {I} [ h (z) \neq h ^ {*} (z) ].
$$
Figure 1: Different perturbation sets considered in  $\ell_{\mathrm{TL}}$ ,  $\ell_{\mathrm{ST}}$  (left),  $\ell_{\mathrm{CA}}$  (mid) and  $\ell_{\mathrm{IA}}$  (right). The dashed line represents the decision boundary of  $h^*$  and the background color of red and blue represents the label 0 and 1 respectively. The ball around each point describes the possible perturbation set  $\mathcal{U}(x)$  and the shaded area inside each ball is the allowed perturbation. In  $\ell_{\mathrm{TL}}$ ,  $\ell_{\mathrm{ST}}$ , we consider all perturbation in  $\mathcal{U}(x)$  while in  $\ell_{\mathrm{CA}}$ , we consider perturbations that do not change the true label of the perturbed point. Lastly, in  $\ell_{\mathrm{IA}}$ , an adversary only perturbs points where the original true label is 0.
In this case, if the true label of the  $z$  changes, then the learner need to match its prediction with the new label. For a fixed perturbation  $z \in \mathcal{U}(x)$ , define  $\ell_{TL}^{h^*}(h,x,z) = \mathbb{I}[h(z) \neq h^*(z)]$ .
3. Stability loss [AKM19, MHS19, MHS22]. There exists a perturbation  $z$  of  $x$  such that  $h(z)$  is different from  $h^{*}(x)$ .
$$
\ell_ {S T} ^ {h ^ {*}} (h, x) = \sup  _ {z \in \mathcal {U} (x)} \mathbb {I} [ h (z) \neq h ^ {*} (x) ].
$$
In this case, we focus on the consistency aspect where we want the prediction of any perturbation  $z$  to be the same as the prediction of  $x$  and this has to be correct w.r.t.  $x$  i.e. equals to  $h^{*}(x)$ . For a fixed perturbation  $z \in \mathcal{U}(x)$ , define  $\ell_{ST}^{h^{*}}(h,x,z) = \mathbb{I}[h(z) \neq h^{*}(x)]$ .
4. Incentive-aware Adversary loss [ZC21]. We take inspiration from economics application where we assume that the label 1 is a more favorable outcome e.g. loan approval for which an adversary has no incentive to make any perturbation when the original label is 1. Define the perturbation set
$$
\mathcal {U} _ {I A} (x, h ^ {*}) = \left\{ \begin{array}{l l} \mathcal {U} (x) & ; h ^ {*} (x) = 0 \\ \{x \} & ; h ^ {*} (x) = 1 \end{array} \right.
$$
and define an incentive-aware adversary loss as
$$
\ell_ {I A} ^ {h ^ {*}} (h, x) = \sup  _ {z \in \mathcal {U} _ {I A} (x, h ^ {*})} \mathbb {I} [ h (z) \neq h ^ {*} (x) ].
$$
For a fixed perturbation  $z\in \mathcal{U}(x)$  , define  $\ell_{IA}^{h^*}(h,x,z) = \mathbb{I}[h(z)\neq h^* (x)\wedge z\in \mathcal{U}_{IA}(x,h^*)]$
We say that  $h$  is robust to a perturbation  $\mathcal{U}$  at  $x$  w.r.t. a robust loss  $\ell$  if  $\ell^{h^*}(h, x) = 0$ .
Remark.  $\ell_{\mathrm{ST}}, \ell_{\mathrm{IA}}$  are robust losses that we can always evaluate in practice on the training data since we are comparing  $h(z)$  with  $h^{*}(x)$  which is known to us on the training data. For  $\ell_{\mathrm{CA}}, \ell_{\mathrm{TL}}$ , we are comparing  $h(z)$  with  $h^{*}(z)$  for which  $z$  may lie outside of the support of the natural data distribution and we may not have access to  $h^{*}(z)$ . We illustrate the relationship between these losses by making a few useful observations.
- In the robustly-realizable case [MHS20] when the perturbation function  $\mathcal{U}$  does not change the true label of any  $x$  in the training or test data, then all the losses  $\ell_{\mathrm{CA}}, \ell_{\mathrm{TL}}, \ell_{\mathrm{ST}}$  are equivalent. This corresponds to a common assumption in the adversarial robustness literature, that the perturbations are "human-imperceptible", which is usually quantified as the set of perturbations within a small metric ball around the data point.
- We provide an illustration of the perturbation set considered in various robust losses in Figure 1. By considering these perturbation set, we have the following implication  $\ell_{\mathrm{TL}} \to \ell_{\mathrm{CA}}$ ,  $\ell_{\mathrm{ST}} \to \ell_{\mathrm{CA}}$  and  $\ell_{\mathrm{ST}} \to \ell_{\mathrm{IA}}$  where  $\ell_1 \to \ell_2$  means robustness w.r.t.  $\ell_1$  implies robustness w.r.t.  $\ell_2$ .
3 Robustly-reliable learners w.r.t. metric ball attacks
Although our robust losses are defined for any general perturbation set, we first consider the case where the perturbation sets are balls in some metric space. Such attacks are widely studied in the literature, in particular, for balls with bounded  $L_{p}$ -norm. Moreover, the radius of the metric ball serves as a natural notion of adversarial strength that allows us to quantify the level of robustness. We will later (Theorem C.1) present results for general perturbation sets as well.
Let  $\mathcal{M} = (\mathcal{X},d)$  be a metric space equipped with distance metric  $d$ . We use the notation  $\mathbf{B}_{\mathcal{M}}(x,r) = \{x^{\prime}\in \mathcal{X}\mid d(x,x^{\prime})\leq r\}$  (resp.  $\mathbf{B}_{\mathcal{M}}^{o}(x,r) = \{x^{\prime}\in \mathcal{X}\mid d(x,x^{\prime}) < r\}$ ) to denote a closed (resp. open) ball of radius  $r$  centered at  $x$ . We will sometimes omit the underlying metric  $\mathcal{M}$  from the subscript to reduce notational clutter. We formally define a metric ball attack as follows.
Definition 2. Metric-ball attacks are defined as the class of perturbation functions  $\mathcal{U}_{\mathcal{M}} = \{u_{\eta} : \mathcal{X} \to 2^{\mathcal{X}} \mid u_{\eta}(x) = \mathbf{B}_{\mathcal{M}}(x, \eta)\}$ , induced by the metric  $\mathcal{M} = (\mathcal{X}, d)$  defined over the instance space.
At test-time, given a test-point  $z \in \mathcal{X}$ , we would like to make a prediction at  $z$  with a reliability guarantee. We consider this type of learner, a robustly-reliable learner defined formally as follows.
Definition 3 (Robustly-reliable learner w.r.t.  $\mathcal{M}$ -ball attacks). A learner  $\mathcal{L}$  is robustly-reliable w.r.t.  $\mathcal{M}$ -ball attacks for hypothesis space  $\mathcal{H}$  and robust loss function  $\ell$  if, for any target concept  $h^* \in \mathcal{H}$ , given  $S$  labeled by  $h^*$ , the learner outputs functions  $h_S^\ell : \mathcal{X} \to \mathcal{Y}$  and  $r_S^\ell : \mathcal{X} \to [0, \infty) \cup \{-1\}$  such that for all  $x, z \in \mathcal{X}$  if  $r_S^\ell(z) = \eta > 0$  and  $z \in \mathbf{B}_\mathcal{M}^o(x, \eta)$  then  $\ell^{h^*}(h_S^\ell, x, z) = 0$ . Further, if  $r_S^\ell(z) = 0$ , then  $h^*(z) = h_S^\ell(z)$ .
Note that  $\mathcal{L}$  outputs a prediction and a real value  $r$  (the "reliability radius") for any test input.  $r = -1$  corresponds to abstention (even in the absence of perturbation) i.e. when the learner is incapable of giving a reliability guarantee for that prediction), and  $r = \eta >0$  is a guarantee from the learner that if the adversary's attack is in  $\mathbf{B}_{\mathcal{M}}^o (x,\eta)$  then we are correct i.e. if an adversary changes the original test point  $x$  to  $z$ , the attack will not succeed if the adversarial budget is less than  $\eta$ . Lastly, when  $r = 0$ , the learner provides a guarantee that the learner's prediction at  $z$  is correct.
Definition 4 (Robustly-reliable region w.r.t.  $\mathcal{M}$ -ball attacks). For a robustly-reliable learner  $\mathcal{L}$  w.r.t.  $\mathcal{M}$ -ball attacks for sample  $S$ , hypothesis space  $\mathcal{H}$  and robust loss function  $\ell$  defined above, the robustly-reliable region of  $\mathcal{L}$  at a reliability level  $\eta$  is defined as  $RR^{\mathcal{L}}(S,\eta) = \{x\in \mathcal{X}\mid r_S^\mathcal{L}(x)\geq \eta \}$  for sample  $S$  and  $\eta \geq 0$ .
The robustly-reliable region contains all points with a reliability guarantee of at least  $\eta$ . We use  $\mathbb{R}\mathbb{R}_{W}^{\mathcal{L}}$  to denote robustly-reliable regions with respect to losses  $\ell_{W}$  for  $W \in \{\mathrm{CA}, \mathrm{TL}, \mathrm{ST}, \mathrm{IA}\}$ . A natural goal is to find a robustly-reliable learner  $\mathcal{L}$  that has the largest robustly-reliable region possible. First, we note that predictions that are known by the learner to be correct are still known to be correct even when the test points are attacked. Therefore, a test point  $z$  lies in the robustly-reliable region w.r.t.  $\ell_{\mathrm{CA}}, \ell_{\mathrm{TL}}$ , as long as we can be sure that  $h_S^{\mathcal{L}}(z)$  is correct. This is equivalent to  $z$  being classified perfectly, i.e. according to the true label. Therefore, the robustly-reliable region w.r.t.  $\ell_{\mathrm{CA}}, \ell_{\mathrm{TL}}$  is given by the agreement region of the version space, which is the largest region where we can be sure of what the correct label is in the absence of any adversarial attack [EYW0]. We recall the definition of version space [Mit82] and agreement region [CAL94, BBL06].
Definition 5. For a set  $H \subseteq \mathcal{H}$  of hypothesis, and any set of samples  $S$ , let  $\mathrm{DIS}(H) = \{x \in \mathcal{X} : \exists h_1, h_2 \in \mathcal{H} \text{ s.t. } h_1(x) \neq h_2(x)\}$  be the disagreement region and  $\operatorname{Agree}(H) = \mathcal{X} \setminus \operatorname{DIS}(H)$  be the agreement region. Let  $\mathcal{H}_0(S) = \{h \in \mathcal{H} | \operatorname{err}_S(h) = 0\}$  be a version space: the set of all hypotheses that correctly classify  $S$ . More generally,  $\mathcal{H}_{\nu}(S) = \{h \in \mathcal{H} | \operatorname{err}_S(h) \leq \nu\}$  for  $\nu \geq 0$ .
We can also characterize the robustly-reliable region with respect to other robust losses in terms of the agreement region in the following Theorem.
Theorem 3.1. Let  $\mathcal{H}$  be any hypothesis class. With respect to  $\mathcal{M}$ -ball attacks and  $\ell_W$ , for  $\eta \geq 0$
(a) there exists a robustly-reliable learner  $\mathcal{L}$  such that  $RR_W^{\mathcal{L}}(S,\eta)\supseteq A_W$ , and
(b) for any robustly-reliable learner  $\mathcal{L}$ ,  $RR_W^{\mathcal{L}}(S,\eta)\subseteq A_W$ .
Specifically, for the robust loss  $\ell_W$ , the optimal robustly-reliable region  $A_W$  are
Robust loss ℓW	Optimal robustly-reliable region AW
ℓCA, ℓTL	{z | z ∈ Agree(H0(S))}
ℓST	{z | Bo(z, η) ⊆ Agree(H0(S)) ∧ h(z) = h(x), ∀x ∈ Bo(z, η), ∀h ∈ H0(S)}
ℓIA	(AST ∩ {z | h*(z) = 1}) ∪ {z | z ∈ Agree(H0(S)) ∧ h*(z) = 0}
Proof. (Sketch) We provide the construction of the optimal robustly-reliable learner  $\mathcal{L}_{\mathrm{opt}}$  such that  $\mathrm{RR}_W^{\mathcal{L}_{\mathrm{opt}}}(S,\eta)\supseteq A_W$  and later show that for any robustly-reliable learner  $\mathcal{L}$ , we must also have  $\mathrm{RR}_W^{\mathcal{L}}(S,\eta)\subseteq A_W$ . We start with  $\ell_{\mathrm{CA}},\ell_{\mathrm{TL}}$ , consider a learner  $\mathcal{L}_{\mathrm{opt}}$  that predicts using an ERM classifier and outputs  $\eta = \infty$  for all points in the agreement region of  $\mathcal{H}_0(S)$ . Any prediction in  $\mathrm{Agree}(\mathcal{H}_0(S))$  is reliable because it also agrees with  $h^{*}$  ( $h^{*}\in \mathcal{H}_{0}(S)$  by realizability). On the other hand, for  $z\in \mathrm{DIS}(\mathcal{H}_0(S))$ , there exist  $h_1,h_2\in \mathcal{H}_0(S)$  that disagree on  $z$ . For any learner  $\mathcal{L}$ , it is not possible to guarantee that  $h^{\mathcal{L}}(z)$  is correct as we may have  $h^{*} = h_{1}$  or  $h^{*} = h_{2}$ .
Now, for  $\ell_{\mathrm{ST}}$ , consider a learner  $\mathcal{L}_{\mathrm{opt}}$  that classifies using an ERM but the reliability radius is now the largest  $\eta > 0$  for which  $\mathbf{B}^o(z,\eta) \subseteq \operatorname{Agree}(\mathcal{H}_0(S))$  and  $h(x) = h(z)$ ,  $\forall x \in \mathbf{B}^o(z,\eta)$ ,  $h \in \mathcal{H}_0(S)$ , else  $\eta = 0$  if  $z \in \operatorname{Agree}(\mathcal{H}_0(S))$ , and  $-1$  otherwise. The first condition guarantees that  $h(x) = h^*(x), \forall x \in \mathbf{B}^o(z,\eta)$ . Combined with the second condition we have  $h(z) = h(x) = h^*(x), \forall x \in \mathbf{B}^o(z,\eta)$ . Thus,  $\mathcal{L}_{\mathrm{opt}}$  is a robustly-reliable learner. On the other hand, for a robustly-reliable learner  $\mathcal{L}$ , consider  $z \in \mathrm{RR}_{\mathrm{ST}}^\mathcal{L}(S,\eta)$  for  $\eta > 0$ . We must have  $h^\mathcal{L}(z) = h^*(x), \forall x \in \mathbf{B}^o(z,\eta)$ . Using a similar argument to the case for  $\ell_{\mathrm{CA}}, \ell_{\mathrm{TL}}$ , we have  $z \in \operatorname{Agree}(\mathcal{H}_0(S))$ . If there exists  $x \in \mathbf{B}^o(z,\eta)$  that  $x \notin \operatorname{Agree}(\mathcal{H}_0(S))$ , there exists  $h_1, h_2 \in \mathcal{H}_0(S)$  that  $h^\mathcal{L}(z) \neq h_1(x)$  or  $h^\mathcal{L}(z) \neq h_2(x)$ . It is not possible to guarantee that  $h^\mathcal{L}(z) = h^*(x)$  as we may have  $h^* = h_1$  or  $h^* = h_2$ . Therefore, we must have  $\mathbf{B}^o(z,\eta) \subseteq \operatorname{Agree}(\mathcal{H}_0(S))$ . Finally, we cannot have  $x \in \mathbf{B}^o(z,\eta)$  that  $h(z) = h^*(z) \neq h^*(x)$  since this contradicts with  $h(z) = h^*(x)$ . Therefore, we must have  $h^*(x) = h^*(z)$ . Since we have  $x \in \operatorname{Agree}(\mathcal{H}_0(S))$ , this implies that  $h(x) = h(z)$  for  $h \in \mathcal{H}_0(S)$ . For  $\ell_{\mathrm{IA}}$ , the construction is similar to  $\ell_{\mathrm{ST}}$ . For full proof, we refer to Appendix B.
For  $\ell_{\mathrm{ST}}$ , the learner is able to certify a subset of the agreement region, which satisfies two additional conditions:  $h_S^{\mathcal{L}}$  must be correct on all possible points  $x$  that could be perturbed to an observed test point  $z$ , and the true label of  $z$  should match the true label of  $x$ . We denote the second condition of  $h(z) = h(x), \forall x \in \mathbf{B}^o(z,\eta), \forall h \in \mathcal{H}_0(S)$  as the label consistency condition. For  $\ell_{\mathrm{IA}}$ , since robustness w.r.t.  $\ell_{\mathrm{ST}}$  implies robustness w.r.t.  $\ell_{\mathrm{IA}}$ , we have  $A_{\mathrm{ST}} \subset A_{\mathrm{IA}}$ . In addition, with an incentive-aware adversary, whenever  $h(z) = 0$ , we must have  $h^*(x) = 0$  since the adversary does not perturb a data point with the original label 1. Therefore, we can additionally provide a guarantee on  $z$  that lies in the agreement region and  $h(z) = 0$ . We remark that we can identify the term  $h^*(z)$  for any point  $z \in \operatorname{Agree}(\mathcal{H}_0(S))$  due to the realizability assumption.