SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here is the exact description of the example that was introduced in Figure 1b. We will use colors to simplify the distinction between points. The points have the following coordinates:  $x_{1} = (0,1), x_{2} = (-2.7,1.1), x_{3} = (2.7,1.1), x_{4} = (0,-2)$ . For this extended analysis, it will be easier to work with classes in  $\{-1,1\}$ , so we have labels  $y_{1} = 1$  and  $y_{2} = y_{3} = y_{4} = -1$ . We set  $\epsilon = 1$ .
Let  $f = (w, b)$  be a generic linear classifier in  $\mathbb{R}^2$ , where  $w = (w_1, w_2)$  is the normal vector of the hyperplane and  $b$  is the bias term. The classification is done using the rule  $\text{sign}(f(x)) = \text{sign}(w^T x + b)$ , as in standard binary classification with labels  $[-1, 1]$ .
In this setting, a point  $(x,y)\in \mathbb{R}^2\times \{-1,1\}$  is correctly classified if  $y\cdot f(x) > 0$ . Moreover, for linear classifiers, the optimal adversarial perturbation is known and robustness at a point can be easily studied. Note that given a perturbation  $\delta$ , we can check if it induces a misclassification by simply computing  $y\cdot f(x + \delta)$ . We can say  $f$  is robust at  $(x,y)$  if, for every perturbation of norm at most  $\epsilon$ ,
we have that  $y\cdot f(x + \delta) > 0$  . This can be developed to get
$$
\begin{array}{l} y \cdot f (x + \delta) > 0 \iff y \left(w ^ {T} (x + \delta) + b\right) > 0 \\ \Longleftrightarrow y \left(w ^ {T} x + b\right) + y \left(w ^ {T} \delta\right) > 0 \tag {17} \\ \Longleftrightarrow y \left(w ^ {T} x + b\right) > - y \left(w ^ {T} \delta\right) \\ \end{array}
$$
For  $f$  to be robust at  $(x, y)$ , Equation (17) must hold for all  $\delta$ , in particular for the quantity maximizing the RHS. This happens for  $\delta^{*} = -y\epsilon \frac{w}{\|w\|}$ . Inserting this worst case perturbation, we obtain a simple condition for robustness in the linear case:
$$
y \left(w ^ {T} x + b\right) > \epsilon \| w \| \tag {18}
$$
Without loss of generality, we can assume  $w$  has norm 1 so the last equation further simplifies. We can use these inequalities to show that no linear classifier can be robust at  $x_{1}$ ,  $x_{2}$  and  $x_{4}$  at the same time. Indeed, being robust at each point gives us the following inequalities:
$$
\begin{array}{l} (x _ {1}): w _ {2} + b > 1 \quad \Longrightarrow b > 1 - w _ {2} \\ \left(x _ {2}\right): 2. 7 w _ {1} - 1. 1 w _ {2} - b > 1 \quad \Longrightarrow b <   2. 7 w _ {1} - 1. 1 w _ {2} - 1 \\ \left(x _ {4}\right): 2 w _ {2} - b > 1 \quad \Longrightarrow b <   2 w _ {2} - 1 \\ \end{array}
$$
Using the inequalities for  $x_{1}$  and  $x_{4}$ , we get the condition  $w_{2} > \frac{2}{3}$ . Using the inequalities for  $x_{1}$  and  $x_{2}$  gives us that  $2.7w_{1} - 0.1w_{2} > 2$ . Together with the bound on  $w_{2}$ , we get the bound  $w_{1} > \frac{62}{81}$ . These two bounds make it impossible for  $w$  to have norm 1, contradicting our hypothesis. An analogous reasoning shows that no linear classifier can be robust at  $x_{1}, x_{3}$  and  $x_{4}$  at the same time.
Knowing that a linear classifier can't be robust at  $x_{1}$ ,  $x_{2}$  and  $x_{4}$  simultaneously, we can further ask what is the best adversarial risk one can get. One can easily check that it is possible to be robust on any of the pairs  $(x_{1}, x_{2})$ ,  $(x_{2}, x_{4})$  or  $(x_{1}, x_{2})$ . At each time, robustness on the two selected points comes at the expense of non-robustness on the third point. This fact discards the strategy of being robust at the pair  $(x_{2}, x_{4})$ , because being non-robust at  $x_{1}$  implies a risk of  $\frac{1}{2}$ , way higher than the risk of  $\frac{1}{6}$  one would have to pay for being non-robust at  $x_{2}$  or  $x_{4}$ . With this being said, the other two solutions are optimal in terms of adversarial risk for linear classifiers, with an adversarial risk of  $\frac{1}{3}$ . Nevertheless, for building a robust mixture, only one of them will be useful.
C.4.2 Matching pennies situation.
In order to show that the matching pennies situation can exist on  $x_4$ , the simplest thing is to propose two linear classifiers for which it happens. We propose the following linear classifiers:
-  $f_{1} = (w_{1}, b_{1})$  with  $w_{1} = (0.825, 0.565132728)$  and  $b_{1} = 0.536876091$ .
-  $f_{2} = (w_{2}, b_{2})$  with  $w_{2} = (-0.825, 0.565132728)$  and  $b_{2} = 0.536876091$ .
Using Equation (18) one can check that these classifiers match the situation that is illustrated in Figure 5, i.e. they are both robust at  $x_{1}$ , they are robust at  $x_{3}$  and  $x_{2}$  respectively, they are non-robust at  $x_{4}$  but they can't be attacked on the same region, as their intersections with the 1-ball around  $x_{4}$  are disjoint (gray circular sectors in Figure 5). Let us denote  $\mathbf{h}$  the REC  $f_{1}$  and  $f_{2}$  with probabilities  $\left(\frac{1}{2},\frac{1}{2}\right)$ .
C.4.3 Adversarial risk calculation for the mixture of  $f_{1}$  and  $f_{2}$ .
For  $x_{2}$ ,  $f_{1}$  is not robust, meaning its adversarial 0-1 loss on  $x_{2}$  is 1. On the other hand,  $f_{2}$  is robust on  $x_{2}$ , as no perturbation of norm at most  $\epsilon$  will make it predict the wrong class. For these reasons, the adversarial 0-1 loss on  $x_{2}$  for  $\mathbf{h}$  is  $\frac{1}{2}$ . The same goes for  $x_{3}$ , and therefore they each add  $\frac{1}{12}$  to the total adversarial risk (mass of each point times the 0-1 loss).
At the point  $x_4$ , given the matching pennies situation, the adversarial 0-1 for  $\mathbf{h}$  is  $\frac{1}{2}$ , for a total adversarial risk of  $\frac{1}{12}$ . As both  $f_1$  and  $f_2$  are robust on  $x_1$ , the adversarial risk on  $x_1$  is 0, and we can conclude that the total adversarial risk of  $\mathbf{h}$  is  $\frac{3}{12} = \frac{1}{4}$ , which is less than the optimal adversarial risk when considering linear classifiers only, that is  $\frac{1}{3}$ .
C.5 On the toy example in Trillos et al. [45, Section 5.2]
The example shown in Trillos et al. [45, Section 5.2] involves three points of different classes. We are interested in Case 4-(i), in which each pair of points can be merged into one by the adversary, but the three cannot be merged because their three  $\epsilon$ -balls have an empty intersection, as depicted in Figure 6. For simplicity, we assume each point has a probability  $\omega_{i}$  of  $\frac{1}{3}$ , which satisfies the condition  $\omega_{1} < \omega_{2} + \omega_{3}$  on Case 4 - (i) in [45].
Figure 6: Toy example in Trillos et al. [45, Section 5.2], Case 4-(i). Best viewed in color.
Authors show that in this case, the optimal attack consists on distributing the mass of the original points into the two neighbors in such a way that, for each of the barycenters  $\bar{x}_{ij}$ , the mass coming from each of the original points  $x_{i}$  and  $x_{j}$  is the same. In our simplified example, this means that each original point sends  $\frac{1}{6}$  of its mass to each neighboring barycenter, as depicted in Figure 7a. When it comes to an optimal classifier, the one presented in Figure 7b is optimal and coincides with the one described in the original work of [45]. This classifier predicts the original class for each  $x_{i}$  inside the part of the ball  $B_{\epsilon}(x_{i})$  that is not intersecting any of the other balls. For each intersection  $B_{\epsilon}(x_{j}) \cap B_{\epsilon}(x_{k})$ , the classifier predicts class  $j$  and  $k$  with probability  $\frac{1}{2}$  each, and the third class  $i$  with probability 0. This classifier achieves an adversarial risk of  $\frac{1}{2}$ , because at any point of the new adversarial distribution that is supported on  $\{\bar{x}_{12}, \bar{x}_{13}, \bar{x}_{23}\}$ , this classifier has probability  $\frac{1}{2}$  of predicting the correct class.
(a) Optimal attack.
(b) Optimal classifier.
Figure 7: Solution for toy example in Trillos et al. [45, Section 5.2], Case 4-(i). Best viewed in color.
From the perspective of our work, it is interesting that the optimal probabilistic classifier shown in Figure 7b can be built as a uniform REC of 6 deterministic classifiers  $f_{ijk}$  that predict  $i$  in  $B_{\epsilon}(x_i)$ ,  $j$  in  $B_{\epsilon}(x_j) \setminus B_{\epsilon}(x_i)$  and  $k$  in  $B_{\epsilon}(x_k) \setminus (B_{\epsilon}(x_i) \cup B_{\epsilon}(x_j))$  (see Figure 8). Each  $f_{ijk}$  has standard accuracy of 1, and adversarial accuracy of  $\frac{1}{3}$  (risk of  $\frac{2}{3}$ ) against an optimal attack, like the constant
classifiers, because it can only be robust at one point at most. They are optimal when restricting to deterministic classifiers. Additionally, the uniform mixture of these 6 classifiers coincides with the classifier presented in Figure 7b, which has standard accuracy of 1, and adversarial accuracy of  $\frac{1}{2}$ , meaning an increase in performance of  $\frac{1}{2} - \frac{1}{3} = \frac{1}{6}$  with respect to the deterministic  $f_{ijk}$ .
(a) Deterministic classifier  $f_{123}$
(b) Deterministic classifier  $f_{231}$
Figure 8: Examples of deterministic classifiers that allow the construction of the optimal classifier as a REC
One can compute the matching penny gap of this mixture of 6 classifiers in this dataset as follows: at each original point  $x_{i}$ , there are only two classifiers that are robust ( $f_{ijk}$  and  $f_{ikj}$ ), which means  $\mu(\mathcal{H}_{vb}(x_i, i)) = \frac{4}{6} = \frac{2}{3}$ . On the other hand, a single perturbation will not fool all the four vulnerable models. Starting from  $x_{i}$ , the attacker might move this point towards  $x_j$  or  $x_k$ . Any of these perturbations will fool three of the six classifiers. For example, the attack  $x_1 \rightarrow \bar{x}_{12}$  will fool the subset  $\{f_{213}, f_{231}, f_{321}\}$ , while the other three classifiers will correctly predict the class 1 at  $\bar{x}_{12}$ . In conclusion,  $\mu^{\max}(x_i, i) = \frac{3}{6} = \frac{1}{2}$ . We conclude that at each point, the matching penny gap is exactly  $\frac{2}{3} - \frac{1}{2} = \frac{1}{6}$ , for a total expected matching penny gap of  $\frac{1}{6}$  (recall every point had the same mass). The average risk is  $\frac{2}{3}$ , and the risk of the mixture can be found using Equation (6) from Theorem 3.2 as  $\frac{2}{3} - \frac{1}{6} = \frac{1}{2}$ .
Appendix D Proofs
Proof. For any  $\mu \in \mathcal{P}(\mathcal{H}_b)$ , we have the following.
$$
\begin{array}{l} \mathcal {R} _ {\epsilon} (\mathbf {h} _ {\mu}) = \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \ell^ {0 - 1} \left(\left(x ^ {\prime}, y\right), \mathbf {h} _ {\mu}\right) \right] = \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \underset {h \sim \mu} {\mathbb {E}} \left[ \mathbb {1} \left\{h \left(x ^ {\prime}\right) \neq y \right\} \right] \right] \\ \leq \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \underset {h \sim \mu} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] \right] \\ = \underset {h \sim \mu} {\mathbb {E}} \left[ \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] \right] ^ {2} \\ = \underset {h \sim \mu} {\mathbb {E}} \left[ \mathcal {R} _ {\epsilon} (h) \right]. \\ \end{array}
$$
Taking infimum with respect to  $\mu$  on both sides of the above inequality, we get the following.
$$
\inf  _ {\mu \in \mathcal {P} \left(\mathcal {H} _ {b}\right)} \mathcal {R} _ {\epsilon} \left(\mathbf {h} _ {\mu}\right) \leq \inf  _ {\mu \in \mathcal {P} \left(\mathcal {H} _ {b}\right)} \mathbb {E} _ {h \sim \mu} \left[ \mathcal {R} _ {\epsilon} (h) \right] \tag {19}
$$
For any  $h \in \mathcal{H}_b$ , we may choose the Dirac measure  $\mu_h$  that assigns probability 1 to  $h$  in order to obtain  $\inf_{\mu \in \mathcal{P}(\mathcal{H}_b)} \mathbb{E}_{h \sim \mu}[\mathcal{R}_\epsilon(h)] \leq \mathbb{E}_{h \sim \mu_h}[\mathcal{R}_\epsilon(h)] = \mathcal{R}_\epsilon(h)$ . Taking infimum over  $h \in \mathcal{H}_b$ , we get the following.
$$
\inf  _ {\mu \in \mathcal {P} \left(\mathcal {H} _ {b}\right)} \mathbb {E} _ {h \sim \mu} \left[ \mathcal {R} _ {\epsilon} (h) \right] \leq \inf  _ {h \in \mathcal {H} _ {b}} \mathcal {R} _ {\epsilon} (h). \tag {20}
$$
The remaining assertion of the theorem follows by combining (19) with (20).
Proof. Observe that for any  $h \in \mathcal{H}_b$ ,  $\sup_{x' \in B_\epsilon(x)} \mathbb{1}\{h(x') \neq y\} = 1$  if and only if  $h \in \mathcal{H}_{vb}(x, y)$ . Hence,
$$
\underset {h \sim \mu} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] = \underset {h \sim \mu} {\mathbb {E}} [ \mathbb {1} \{h \in \mathcal {H} _ {v b} (x, y) \} ] = \mu (\mathcal {H} _ {v b} (x, y)).
$$
Taking expectation on both sides with respect to  $(x,y)\sim \rho$ , we get
$$
\underset {(x, y) \sim \rho} {\mathbb {E}} [ \mu (\mathcal {H} _ {v b} (x, y)) ] = \underset {(x, y) \sim \rho} {\mathbb {E}} \underset {h \sim \mu} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] = \underset {h \sim \mu} {\mathbb {E}} [ \mathcal {R} _ {\epsilon} (h) ], \tag {21}
$$
where the second equality follows from switching the order of the two preceding expectations.
For any  $x^{\prime}\in B_{\epsilon}(x)$
$$
\underset {h \sim \mu} {\mathbb {E}} \left[ \mathbb {1} \left\{h \left(x ^ {\prime}\right) \neq y \right\} \right] = \mu (\left\{h \in \mathcal {H} _ {b}: h \left(x ^ {\prime}\right) \neq y \right\}) \leq \mu^ {\max } (x, y),
$$
where the above inequality holds because the  $\mu$  measure of any subset of  $\mathcal{H}_b$  that is simultaneously vulnerable at some  $x' \in B_{\epsilon}(x)$  is at most  $\mu^{\max}(x, y)$ . Taking supremum over all  $x' \in B_{\epsilon}(x)$  in the above inequality, we get the following.
$$
\sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {E} _ {h \sim \mu} \left[ \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] \leq \mu^ {\max } (x, y).
$$
We will now show that the above inequality also holds in the other direction.
Let  $\{\mathcal{H}^k\}_{k=1}^\infty$  be a sequence of sets,  $\mathcal{H}^k \in \mathfrak{H}_{svb}(x,y)$ , such that  $\lim_{k \to \infty} \mu(\mathcal{H}^k) = \mu^{max}(x,y)$ . For each  $\mathcal{H}^k$ , we have by definition of  $\mathfrak{H}_{svb}(x,y)$  that there exists some  $x^k \in B_\epsilon(x)$  such that all classifiers  $h \in \mathcal{H}^k$  are fooled by  $x^k$ . In other words, the total mass of classifiers that are fooled by  $x^k$  is greater or equal to  $\mu(\mathcal{H}^k)$ . This gives
$$
\sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {E} _ {h \sim \mu} \left[ \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] \geq \mathbb {E} _ {h \sim \mu} \left[ \mathbb {1} \{h \left(x ^ {k}\right) \neq y \} \right] \geq \mu (\mathcal {H} ^ {k}).
$$
This is true for every  $k$ , so taking  $k \to \infty$  we get that  $\sup_{x' \in B_{\epsilon}(x)} \mathbb{E}_{h \sim \mu}[\mathbb{1}\{h(x') \neq y\}] \geq \lim_{k \to \infty} \mu(\mathcal{H}^k) = \mu^{max}(x, y)$ .
$$
\mathcal {R} _ {\epsilon} \left(\mathbf {h} _ {\mu}\right) = \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \underset {h \sim \mu} {\mathbb {E}} \left[ \mathbb {1} \{h \left(x ^ {\prime}\right) \neq y \} \right] \right] = \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \mu^ {\max } (x, y) \right]. \tag {22}
$$
Combining (22) and (21) yields (6).
2We can swap expectations by Fubini-Tonelli's theorem because the function  $\sup_{x^{\prime}\in B_{\varepsilon}(x)}\mathbb{1}\{h(x^{\prime})\neq y\}$  is universally measurable if  $h$  is measurable (See [16, 40]) and both measure spaces  $\mathcal{X}\times \mathcal{Y},\mathcal{H}_b$  are assumed  $\sigma$  -finite. We also assume the measurability w.r.t  $h$  . See Appendix A for details.
Proof. Suppose (9) holds for some  $\mu' \in \mathcal{P}(\mathcal{H}_b)$ . Then,
$$
\begin{array}{l} \inf  _ {\mu \in \mathcal {P} (\mathcal {H} _ {b})} \mathcal {R} _ {\epsilon} \left(\mathbf {h} _ {\mu}\right) \leq \mathcal {R} _ {\epsilon} \left(\mathbf {h} _ {\mu^ {\prime}}\right) = \underset {h \sim \mu^ {\prime}} {\mathbb {E}} \left[ \mathcal {R} _ {\epsilon} (h) \right] - \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \pi_ {\mathbf {h} _ {\mu^ {\prime}}} (x, y) \right] \\ <   \inf  _ {h \in \mathcal {H} _ {b}} \mathcal {R} _ {\epsilon} (h) + \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \pi_ {\mathbf {h} _ {\mu^ {\prime}}} (x, y) \right] - \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \pi_ {\mathbf {h} _ {\mu^ {\prime}}} (x, y) \right] \\ = \inf  _ {h \in \mathcal {H} _ {b}} \mathcal {R} _ {\epsilon} (h), \\ \end{array}
$$
where the first equality follows from Theorem 3.2 and the second inequality follows from the assumption in (9). Suppose (9) does not hold for  $\mu' \in \mathcal{P}(\mathcal{H}_b)$ . Then,
$$
\begin{array}{l} \mathcal {R} _ {\epsilon} \left(\mathbf {h} _ {\mu^ {\prime}}\right) = \underset {h \sim \mu^ {\prime}} {\mathbb {E}} \left[ \mathcal {R} _ {\epsilon} (h) \right] - \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \pi_ {\mathbf {h} _ {\mu} ^ {\prime}} (x, y) \right] \\ \geq \inf  _ {h \in \mathcal {H} _ {b}} \mathcal {R} _ {\epsilon} (h) + \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \pi_ {\mathbf {h} _ {\mu^ {\prime}}} (x, y) \right] - \underset {(x, y) \sim \rho} {\mathbb {E}} \left[ \pi_ {\mathbf {h} _ {\mu^ {\prime}}} (x, y) \right] \\ = \inf  _ {h \in \mathcal {H} _ {b}} \mathcal {R} _ {\epsilon} (h). \\ \end{array}
$$
Suppose (9) does not hold for any  $\mu' \in \mathcal{P}(\mathcal{H}_b)$ , then taking infimum with respect to  $\mu' \in \mathcal{P}(\mathcal{H}_b)$  in the above inequality, we get  $\inf_{\mu' \in \mathcal{P}(\mathcal{H}_b)} \mathcal{R}_{\epsilon}(\mathbf{h}_{\mu'}) \geq \inf_{h \in \mathcal{H}_b} \mathcal{R}_{\epsilon}(h)$ .
Proof. As both functions only take the values 0 and 1, it suffices to show that if the RHS is equal to 1, then so is the LHS. Suppose  $\sup_{x' \in B_{\epsilon}(x)} \mathbb{1}\{\mathbf{h}(x') \succ \alpha\} = 1$ . As the function  $\mathbb{1}\{\mathbf{h}(x') \succ \alpha\}$  takes only a finite number of values, this implies that there exists some  $x^* \in B_{\epsilon}(x)$  such that  $\mathbb{1}\{\mathbf{h}(x*) \succ \alpha\} = 1$ . This means that  $\mathbf{h}(x*) \succ \alpha$ , and therefore  $\sup_{x' \in B_{\epsilon}(x)} \mathbf{h}(x') \succ \alpha$ . This makes the LHS equal to 1.
If we further assume that  $\mathbf{h}$  is continuous or takes a finite number of values, then if the LHS is equal to one,
Proof. We begin the proof by rewriting the adversarial risk of  $\mathbf{h}$  in terms of the adversarial risk for each class. To alleviate notation, we denote  $\ell_{\epsilon}^{0 - 1}((x,y),\mathbf{h}) = \sup_{x^{\prime}\in B_{\epsilon}(x)}\ell^{0 - 1}((x^{\prime},y),\mathbf{h})$ . Recall from Equation (1) that
$$
\mathcal {R} _ {\epsilon} (\mathbf {h}) = \nu (0) \cdot \mathcal {R} _ {\epsilon} ^ {0} (\mathbf {h}) + \nu (1) \cdot \mathcal {R} _ {\epsilon} ^ {1} (\mathbf {h}) \tag {23}
$$
Let us now develop the terms  $R_{\epsilon}^{y}(\mathbf{h})$ . For any  $\succ \in \{>, \geq \}$  we obtain the following:
$$
\begin{array}{l} R _ {\epsilon} ^ {y} (\mathbf {h}) = \underset {x \sim p _ {y}} {\mathbb {E}} \left[ \ell_ {\epsilon} ^ {0 - 1} ((x, y), \mathbf {h}) \right] \\ = \mathbb {E} _ {x \sim p _ {y}} \left[ \int_ {0} ^ {1} \mathbb {1} \left\{\ell_ {\epsilon} ^ {0 - 1} ((x, y), \mathbf {h}) \succ \alpha \right\} d \alpha \right] \tag {24} \\ = \int_ {0} ^ {1} \underset {x \sim p _ {y}} {\mathbb {E}} \left[ \mathbb {1} \left\{\ell_ {\epsilon} ^ {0 - 1} ((x, y), \mathbf {h}) \succ \alpha \right\} \right] d \alpha \\ \end{array}
$$
In the last equation, we were able to interchange the two integrals using Tonnelli's theorem. Indeed, the function  $(x,\alpha)\mapsto \mathbb{1}\{\ell_{\epsilon}^{0 - 1}((x,y),\mathbf{h})\succ \alpha \}$  is Lebesgue measurable in  $\mathcal{X}\times \mathbb{R}$  if  $\ell_{\epsilon}^{0 - 1}((\cdot ,0),\mathbf{h})$  is Lebesgue measurable, which is the case [40, Theorem 1].
The next step is to use Lemma 4.1 to interchange the supremum operator and the indicator function. This will make the adversarial risk of the  $h^\alpha$  appear. For  $y = 0$  and replacing the operator  $\succ$  by  $>$ , we obtain the following:
$$
\begin{array}{l} R _ {\epsilon} ^ {0} (\mathbf {h}) = \int_ {0} ^ {1} \underset {x \sim p _ {0}} {\mathbb {E}} \left[ \mathbb {1} \{\ell_ {\epsilon} ^ {0 - 1} ((x, 0), \mathbf {h}) > \alpha \} \right] d \alpha \\ \geq \int_ {0} ^ {1} \underset {x \sim p _ {0}} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \left\{\mathbf {h} \left(x ^ {\prime}\right) > \alpha \right\} \right] d \alpha \tag {25} \\ \geq \int_ {0} ^ {1} \mathcal {R} _ {\epsilon} ^ {0} (h ^ {\alpha}) d \alpha \\ \end{array}
$$
Now we do a similar development for  $y = 1$  and replacing the operator  $\succ$  by  $\geq$  to obtain:
$$
\begin{array}{l} R _ {\epsilon} ^ {1} (\mathbf {h}) \geq \int_ {0} ^ {1} \underset {x \sim p _ {1}} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \{1 - \mathbf {h} \left(x ^ {\prime}\right) \geq \alpha \} \right] d \alpha \\ \geq \int_ {0} ^ {1} \underset {x \sim p _ {1}} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \mathbb {1} \{1 - \alpha \geq \mathbf {h} \left(x ^ {\prime}\right) \} \right] d \alpha \\ \geq \int_ {0} ^ {1} \underset {x \sim p _ {1}} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} 1 - \mathbb {1} \left\{\mathbf {h} \left(x ^ {\prime}\right) > 1 - \alpha \right\} \right] d \alpha \tag {26} \\ \geq \int_ {0} ^ {1} \mathbb {E} _ {x \sim p _ {1}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} 1 - \mathbf {h} ^ {1 - \alpha} \left(x ^ {\prime}\right) \right] d \alpha \\ \geq \int_ {0} ^ {1} \underset {x \sim p _ {1}} {\mathbb {E}} \left[ \sup  _ {x ^ {\prime} \in B _ {\epsilon} (x)} \ell^ {0 - 1} ((x, 1), \mathbf {h} ^ {1 - \alpha}) \right] d \alpha \\ \geq \int_ {0} ^ {1} \mathcal {R} _ {\epsilon} ^ {1} (\mathbf {h} ^ {1 - \alpha}) d \alpha = \int_ {0} ^ {1} \mathcal {R} _ {\epsilon} ^ {1} (\mathbf {h} ^ {u}) d u \\ \end{array}
$$
In the last equation, the change of variable  $u = 1 - \alpha$  allows us to complete the calculations and obtain the desired result. Putting everything together, we obtain the following result about the original probabilistic classifier  $\mathbf{h}$ :
$$
\mathcal {R} _ {\epsilon} (\mathbf {h}) \geq \int_ {0} ^ {1} \nu (0) \mathcal {R} _ {\epsilon} ^ {0} \left(h ^ {\alpha}\right) + \nu (1) \mathcal {R} _ {\epsilon} ^ {1} \left(h ^ {\alpha}\right) d \alpha = \int_ {0} ^ {1} \mathcal {R} _ {\epsilon} \left(h ^ {\alpha}\right) d \alpha \tag {27}
$$
In particular, Equation (27) implies that  $\mathcal{R}_{\epsilon}(\mathbf{h}) \geq \min_{\alpha} \mathcal{R}_{\epsilon}(h^{\alpha})$ , meaning that for any  $\mathbf{h}$  probabilistic binary classifier, there is always a deterministic  $\alpha$ -threshold classifier with better or equal adversarial risk.
Proof. Theorem 4.1 applied to RECs (see Section 4.2) tells us that  $\overline{\mathcal{H}_b} (\mathcal{P}_M(\mathcal{H}_b))$  is the set of all weighted ensembles built from  $\mathcal{H}_b$ . It is clear that for any  $h\in \mathcal{H}_b$ ,  $\delta_h\in \mathcal{P}_M(\mathcal{H}_b)$ , so  $\mathcal{H}_b\subseteq \overline{\mathcal{H}_b} (\mathcal{P}_M(\mathcal{H}_b))$ . Let us now show the other inclusion.