SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As a proof-of-concept, we provide a concrete realization of the first scheme above on the Malimg dataset. Namely, consider the setting where we want to distinguish between "clean" binaries and 24 classes of malware. A malware designer seeks to maliciously perturb the bytes in their binary to fool a classifier into falsely predicting that the malware is "clean." We therefore consider a cascading architecture where first a feature-convex classifier answers the "clean or malware" question, and then a subsequent classifier (not necessarily feature-convex) predicts the particular class of malware in the case that the feature-convex classifier assigns a "malware" prediction. Note that, in the initial step, we can either certify the "clean" binaries or the collection of all 24 malware classes, simply by negating the feature-convex classifier output logit. We logically choose to certify the malware classes as done in our experiments of Section 4; these certificates provide guarantees against a piece of malware going undetected.
We use the same feature-convex architecture and training details as described in Appendix E. For the cascaded malware classifier, we use a ResNet-18 architecture trained with Adam for 150 epochs with a learning rate of  $10^{-3}$ . The confusion plot for the multiclass classifier is provided in Figure 3, with an overall accuracy of  $96.5\%$ . With the exception of few challenging classes to distinguish, the classifier achieves reasonable performance despite the unbalanced class sizes.
Figure 4 visualizes the distribution of certified radii for the four most common malware classes in the dataset, excluding the "Yuner.A" class which featured duplicated images. Note that certification performance varies between classes, with high correlation across different norms for a particular malware class. Classes which tend to have larger certificates can be interpreted as clustering further away from the clean binaries, requiring larger perturbations to fool the classifier.
B Feature Map Motivation
This section examines the importance of the feature map  $\varphi$  with a low-dimensional example. Consider the binary classification setting where one class  $X_{2} \subseteq \mathbb{R}^{d}$  is clustered around the origin and the other class  $X_{1} \subseteq \mathbb{R}^{d}$  surrounds it in a ring. Here, the pair  $(X_{1}, X_{2})$  is convexly separable (see Definition 3.3) as an  $\ell_{2}$ -norm ball decision region covering  $X_{2}$  is convex (Figure 5a). Note that the reverse pair  $(X_{2}, X_{1})$  is not convexly separable, as there does not exist a convex set containing  $X_{1}$  but excluding  $X_{2}$ . A standard input-convex classifier with  $\varphi = \mathrm{Id}$  would therefore be unable to
Figure 3: The row-normalized confusion plot for the Malimg multiclass classifier. The overall accuracy of the composite classifier is  $96.5\%$ . The various malware classes (1-24) are circumscribed with a black rectangle. These are certified against the class of "clean" binaries. See Section 4 for more details on the mock clean binaries.
discriminate between the classes in this direction (Proposition 3.5), i.e., we would be able to learn a classifier that generates certificates for points in  $X_{1}$ , but not  $X_{2}$ .
The above problem is addressed by choosing the feature map to be the simple concatenation  $\varphi(x) = (x, |x|)$  mapping from  $\mathbb{R}^d$  to  $\mathbb{R}^q = \mathbb{R}^{2d}$ , with associated Lipschitz constants  $\mathrm{Lip}_1(\varphi) \leq 2$ ,  $\mathrm{Lip}_2(\varphi) \leq \sqrt{2}$ , and  $\mathrm{Lip}_{\infty}(\varphi) \leq 1$ . In this augmented feature space,  $X_1$  and  $X_2$  are convexly separable in both directions, as they are each contained in a convex set (specifically, a half-space) whose complement contains the other class. We are now able to learn a classifier that takes  $X_2$  as the sensitive class for which certificates are required (Figure 5b). This parallels the motivation of the support vector machine "kernel trick," where inputs are augmented to a higher-dimensional space wherein the data is linearly separable (instead of convexly separable as in our case).
C Proofs and Supplemental Theoretical Results
Proof. Suppose that  $\nabla g(\varphi(x)) \in \mathbb{R}^q$  is a nonzero subgradient of  $g$  at  $\varphi(x)$ , so that  $g(y) \geq g(\varphi(x)) + \nabla g(\varphi(x))^{\top}(y - \varphi(x))$  for all  $y \in \mathbb{R}^q$ . Let  $\delta \in \mathbb{R}^d$  be such that  $\| \delta \|_p < r(x)$ . Then it holds that
$$
\begin{array}{l} g (\varphi (x + \delta)) \geq g (\varphi (x)) + \nabla g (\varphi (x)) ^ {\top} (\varphi (x + \delta) - \varphi (x)) \\ \geq g (\varphi (x)) - \| \nabla g (\varphi (x)) \| _ {p, *} \| \varphi (x + \delta) - \varphi (x) \| _ {p} \\ \geq g (\varphi (x)) - \| \nabla g (\varphi (x)) \| _ {p, *} \operatorname {L i p} _ {p} (\varphi) \| \delta \| _ {p} \\ > 0, \\ \end{array}
$$
Figure 4: Certified radii distributions for four malware classes in the Malimg dataset.
so indeed  $f(x + \delta) = 1$ .
Figure 5: Experiments demonstrating the role of the feature map  $\varphi = (x, |x|)$  in  $\mathbb{R}^2$ , with the output logit shaded. Certified radii from our method are shown as black rings. (a) Certifying the outer class (dark red points). This is possible using an input-convex classifier as a convex sublevel set contains the inner class (dark blue points). (b) Certifying the inner class (dark red points). This would not be possible with  $\varphi = \mathrm{Id}$  as there is no convex set containing the outer class (dark blue points) but excluding the inner. The feature map  $\varphi$  enables this by permitting convex separability in the higher dimensional space. Note that although the shaded output logit is not convex in the input, we still generate certificates.
We now introduce a preliminary lemma for the results in Section 3.2.
Lemma C.1. For any nonempty closed convex set  $X \subseteq \mathbb{R}^d$ , there exists a convex function  $g: \mathbb{R}^d \to \mathbb{R}$  such that  $X = g^{-1}((-\infty, 0]) = \{x \in \mathbb{R}^d : g(x) \leq 0\}$ .
Proof. Let  $X \subseteq \mathbb{R}^d$  be a nonempty closed convex set. We take the distance function  $g = d_X$  defined by  $d_X(x) = \inf_{y \in X} \| y - x \|_2$ . Since  $X$  is closed and  $y \mapsto \| y - x \|_2$  is coercive for all  $x \in \mathbb{R}^d$ , it holds that  $y \mapsto \| y - x \|_2$  attains its infimum over  $X$  [10, Proposition A.8]. Let  $x^{(1)}, x^{(2)} \in \mathbb{R}^d$  and let  $\theta \in [0, 1]$ . Then there exist  $y^{(1)}, y^{(2)} \in X$  such that  $g(x^{(1)}) = \| y^{(1)} - x^{(1)} \|_2$  and  $g(x^{(2)}) = \| y^{(2)} - x^{(2)} \|_2$ . Since  $X$  is convex, it holds that  $\theta y^{(1)} + (1 - \theta) y^{(2)} \in X$ , and therefore
$$
\begin{array}{l} g \left(\theta x ^ {(1)} + (1 - \theta) x ^ {(2)}\right) = \inf  _ {y \in X} \| y - \left(\theta x ^ {(1)} + (1 - \theta) x ^ {(2)}\right) \| _ {2} \\ \leq \left\| \theta y ^ {(1)} + (1 - \theta) y ^ {(2)} - (\theta x ^ {(1)} + (1 - \theta) x ^ {(2)}) \right\| _ {2} \\ \leq \theta \| y ^ {(1)} - x ^ {(1)} \| _ {2} + (1 - \theta) \| y ^ {(2)} - x ^ {(2)} \| _ {2} \\ = \theta g (x ^ {(1)}) + (1 - \theta) g (x ^ {(2)}). \\ \end{array}
$$
Hence,  $g = d_X$  is convex. Since  $X = \{x \in \mathbb{R}^d : \inf_{y \in X} \| y - x \|_2 = 0\} = \{x \in \mathbb{R}^d : d_X(x) = 0\} = \{x \in \mathbb{R}^d : d_X(x) \leq 0\} = \{x \in \mathbb{R}^d : g(x) \leq 0\}$  by nonnegativity of  $d_X$ , the lemma holds.
Proposition 3.4. For any nonempty closed convex set  $X \subseteq \mathbb{R}^d$ , there exists  $f \in \mathcal{F}_{\mathrm{Id}}$  such that  $X = f^{-1}(\{2\}) = \{x \in \mathbb{R}^d : f(x) = 2\}$ . In particular, this shows that if  $(X_1, X_2)$  is a convexly separable pair of subsets of  $\mathbb{R}^d$ , then there exists  $f \in \mathcal{F}_{\mathrm{Id}}$  such that  $f(x) = 1$  for all  $x \in X_1$  and  $f(x) = 2$  for all  $x \in X_2$ .
Proof. Let  $X \subseteq \mathbb{R}^d$  be a nonempty closed convex set. By Lemma C.1, there exists a convex function  $g \colon \mathbb{R}^d \to \mathbb{R}$  such that  $X = \{x \in \mathbb{R}^d : g(x) \leq 0\}$ . Define  $f \colon \mathbb{R}^d \to \{1,2\}$  by  $f(x) = 1$  if  $g(x) > 0$  and  $f(x) = 2$  if  $g(x) \leq 0$ . Clearly, it holds that  $f \in \mathcal{F}_{\mathrm{Id}}$ . Furthermore, for all  $x \in X$  it holds that  $g(x) \leq 0$ , implying that  $f(x) = 2$  for all  $x \in X$ . Conversely, if  $x \in \mathbb{R}^d$  is such that  $f(x) = 2$ , then  $g(x) \leq 0$ , implying that  $x \in X$ . Hence,  $X = \{x \in \mathbb{R}^d : f(x) = 2\}$ .
If  $(X_{1},X_{2})$  is a convexly separable pair of subsets of  $\mathbb{R}^d$ , then there exists a nonempty closed convex set  $X\subseteq \mathbb{R}^{d}$  such that  $X_{2}\subseteq X$  and  $X_{1}\subseteq \mathbb{R}^{d}\setminus X$ , and therefore there exists  $f\in \mathcal{F}_{\mathrm{Id}}$  such that  $X_{2}\subseteq X = f^{-1}(\{2\})$  and  $X_{1}\subseteq \mathbb{R}^{d}\setminus X = f^{-1}(\{1\})$ , implying that indeed  $f(x) = 1$  for all  $x\in X_1$  and  $f(x) = 2$  for all  $x\in X_2$ .
Proof. For all  $x \in \mathbb{R}^d$ , it holds that  $f(x) = 2$  if and only if  $g(x) \leq 0$ . Since  $f \in \mathcal{F}_{\mathrm{id}}$ ,  $g$  is convex, and hence,  $X = \{x \in \mathbb{R}^d : g(x) \leq 0\}$  is a (nonstrict) sublevel set of a convex function and is therefore a closed convex set.
In order to apply the universal approximation results in Chen et al. [16], we now introduce their parameterization of input-convex ReLU neural networks. Note that it imposes the additional constraint that the first weight matrix  $A^{(1)}$  is elementwise nonnegative.
Definition C.2. Define  $\tilde{\mathcal{F}}_{\mathrm{Id}}$  to be the class of functions  $\tilde{f}\colon \mathbb{R}^d\to \{1,2\}$  given by  $\tilde{f} (x) = T(\tilde{g} (x))$  with  $\tilde{g}\colon \mathbb{R}^d\to \mathbb{R}$  given by
$$
\begin{array}{l} x ^ {(1)} = \operatorname {R e L U} \left(A ^ {(1)} x + b ^ {(1)}\right), \\ x ^ {(l)} = \operatorname {R e L U} \left(A ^ {(l)} x ^ {(l - 1)} + b ^ {(l)} + C ^ {(l)} x\right), l \in \{2, 3, \dots , L - 1 \}, \\ \tilde {g} (x) = A ^ {(L)} x ^ {(L - 1)} + b ^ {(L)} + C ^ {(L)} x, \\ \end{array}
$$
for some  $L \in \mathbb{N}$ ,  $L > 1$ , and some consistently sized matrices  $A^{(1)}, C^{(1)}, \ldots, A^{(L)}, C^{(L)}$ , all of which have nonnegative elements, and some consistently sized vectors  $b^{(1)}, \ldots, b^{(L)}$ .
The following preliminary lemma relates the class  $\hat{\mathcal{F}}_{\mathrm{Id}}$  from Definition 2.2 to the class  $\tilde{\mathcal{F}}_{\mathrm{Id}}$  above.
Lemma C.3. It holds that  $\hat{\mathcal{F}}_{\mathrm{Id}}\subseteq \hat{\mathcal{F}}_{\mathrm{Id}}$
Proof. Let  $\tilde{f} \in \tilde{\mathcal{F}}_{\mathrm{Id}}$ . Then certainly  $A^{(l)} \geq 0$  for all  $l \in \{2,3,\dots,L\}$ , so indeed  $\tilde{f} \in \hat{\mathcal{F}}_{\mathrm{Id}}$ . Hence,  $\tilde{\mathcal{F}}_{\mathrm{Id}} \subseteq \hat{\mathcal{F}}_{\mathrm{Id}}$ .
Theorem 1 in Chen et al. [16] shows that a Lipschitz convex function can be approximated within an arbitrary tolerance. We now provide a technical lemma adapting Theorem 1 in Chen et al. [16] to show that convex functions can be underapproximated within an arbitrary tolerance on a compact convex subset.
Lemma C.4. For any convex function  $g \colon \mathbb{R}^d \to \mathbb{R}$ , any compact convex subset  $X$  of  $\mathbb{R}^d$ , and any  $\epsilon > 0$ , there exists  $\hat{f} \in \hat{\mathcal{F}}_{\mathrm{Id}}$  such that  $\hat{g}(x) < g(x)$  for all  $x \in X$  and  $\sup_{x \in X} (g(x) - \hat{g}(x)) < \epsilon$ .
Proof. Let  $g \colon \mathbb{R}^d \to \mathbb{R}$  be a convex function, let  $X$  be a compact convex subset of  $\mathbb{R}^d$ , and let  $\epsilon > 0$ . Since  $g - \epsilon / 2$  is a real-valued convex function on  $\mathbb{R}^d$  (and hence is proper), its restriction to the closed and bounded set  $X$  is Lipschitz continuous [55, Theorem 10.4], and therefore Lemma C.3 together with Theorem 1 in Chen et al. [16] gives that there exists  $\hat{f} \in \tilde{\mathcal{F}}_{\mathrm{Id}} \subseteq \hat{\mathcal{F}}_{\mathrm{Id}}$  such that  $\sup_{x \in X} |(g(x) - \epsilon / 2) - \hat{g}(x)| < \epsilon / 2$ . Thus, for all  $x \in X$ ,
$$
\begin{array}{l} g (x) - \hat {g} (x) = \left(g (x) - \frac {\epsilon}{2}\right) - \hat {g} (x) + \frac {\epsilon}{2} \\ > \left(g (x) - \frac {\epsilon}{2}\right) - \hat {g} (x) + \sup  _ {y \in X} \left| \left(g (y) - \frac {\epsilon}{2}\right) - \hat {g} (y) \right| \\ \geq \left(g (x) - \frac {\epsilon}{2}\right) - \hat {g} (x) + \left| \left(g (x) - \frac {\epsilon}{2}\right) - \hat {g} (x) \right| \\ \geq 0. \\ \end{array}
$$
$$
\begin{array}{l} \sup  _ {x \in X} (g (x) - \hat {g} (x)) = \sup  _ {x \in X} | g (x) - \hat {g} (x) | \\ = \sup  _ {x \in X} \left| \left(g (x) - \frac {\epsilon}{2}\right) - \hat {g} (x) + \frac {\epsilon}{2} \right| \\ \leq \sup  _ {x \in X} \left| (g (x) - \frac {\epsilon}{2}) - \hat {g} (x) \right| + \frac {\epsilon}{2} \\ <   \epsilon , \\ \end{array}
$$
which proves the lemma.
We leverage Lemma C.4 to construct a uniformly converging sequence of underapproximating functions.
Lemma C.5. For all  $f \in \mathcal{F}_{\mathrm{Id}}$  and all compact convex subsets  $X$  of  $\mathbb{R}^d$ , there exists a sequence  $\{\hat{f}_n \in \hat{\mathcal{F}}_{\mathrm{Id}} : n \in \mathbb{N}\} \subseteq \hat{\mathcal{F}}_{\mathrm{Id}}$  such that  $\hat{g}_n(x) < \hat{g}_{n+1}(x) < g(x)$  for all  $x \in X$  and all  $n \in \mathbb{N}$  and  $\hat{g}_n$  converges uniformly to  $g$  on  $X$  as  $n \to \infty$ .
Proof. Let  $f \in \mathcal{F}_{\mathrm{Id}}$  and let  $X$  be a compact convex subset of  $\mathbb{R}^d$ . Let  $\{\epsilon_n > 0 : n \in \mathbb{N}\}$  be a sequence such that  $\epsilon_{n+1} < \epsilon_n$  for all  $n \in \mathbb{N}$  and  $\epsilon_n \to 0$  as  $n \to \infty$ . Such a sequence clearly exists, e.g., by taking  $\epsilon_n = 1/n$  for all  $n \in \mathbb{N}$ . Now, for all  $n \in \mathbb{N}$ , the function  $g - \epsilon_{n+1}$  is convex, and therefore by Lemma C.4 there exists  $\hat{f}_n \in \hat{\mathcal{F}}_{\mathrm{Id}}$  such that  $\hat{g}_n(x) < g(x) - \epsilon_{n+1}$  for all  $x \in X$  and  $\sup_{x \in X} ((g(x) - \epsilon_{n+1}) - \hat{g}_n(x)) < \epsilon_n - \epsilon_{n+1}$ . Fixing such  $\hat{f}_n, \hat{g}_n$  for all  $n \in \mathbb{N}$ , we see that  $\sup_{x \in X} ((g(x) - \epsilon_{n+2}) - \hat{g}_{n+1}(x)) < \epsilon_{n+1} - \epsilon_{n+2}$ , which implies that
$$
\hat {g} _ {n + 1} (x) > g (x) - \epsilon_ {n + 1} > \hat {g} _ {n} (x)
$$
for all  $x \in X$ , which proves the first inequality. The second inequality comes from the fact that  $\hat{g}_{n+1}(x) < g(x) - \epsilon_{n+2} < g(x)$  for all  $x \in X$ . Finally, since  $g(x) - \hat{g}_n(x) > \epsilon_{n+1} > 0$  for all  $x \in X$  and all  $n \in \mathbb{N}$ , we see that
$$
\sup  _ {x \in X} | g (x) - \hat {g} _ {n} (x) | = \sup  _ {x \in X} \left(g (x) - \hat {g} _ {n} (x)\right) <   \epsilon_ {n} \rightarrow 0 \text {a s} n \rightarrow \infty ,
$$
which proves that  $\lim_{n\to \infty}\sup_{x\in X}|g(x) - \hat{g}_n(x)| = 0$ , so indeed  $\hat{g}_n$  converges uniformly to  $g$  on  $X$  as  $n\to \infty$ .
With all the necessary lemmas in place, we now present our main theoretical results.
Proof. Let  $f \in \mathcal{F}_{\mathrm{Id}}$  and let  $X$  be a compact convex subset of  $\mathbb{R}^d$ . By Lemma C.5, there exists a sequence  $\{\hat{f}_n \in \hat{\mathcal{F}}_{\mathrm{Id}} : n \in \mathbb{N}\} \subseteq \hat{\mathcal{F}}_{\mathrm{Id}}$  such that  $\hat{g}_n(x) < \hat{g}_{n+1}(x) < g(x)$  for all  $x \in X$  and all  $n \in \mathbb{N}$  and  $\hat{g}_n$  converges uniformly to  $g$  on  $X$  as  $n \to \infty$ . Fix this sequence.
For all  $n\in \mathbb{N}$  , define
$$
E _ {n} = \{x \in X: \hat {f} _ {n} (x) \neq f (x) \},
$$
i.e., the set of points in  $X$  for which the classification under  $\hat{f}_n$  does not agree with that under  $f$ . Since  $\hat{g}_n(x) < g(x)$  for all  $x \in X$  and all  $n \in \mathbb{N}$ , we see that
$$
\begin{array}{l} E _ {n} = \{x \in X: \hat {g} _ {n} (x) > 0 \text {a n d} g (x) \leq 0 \} \cup \{x \in X: \hat {g} _ {n} (x) \leq 0 \text {a n d} g (x) > 0 \} \\ = \{x \in X: \hat {g} _ {n} (x) \leq 0 \text {a n d} g (x) > 0 \}. \\ \end{array}
$$
Since  $g$  is a real-valued convex function on  $\mathbb{R}^d$ , it is continuous [55, Corollary 10.1.1], and therefore  $g^{-1}((0,\infty)) = \{x\in \mathbb{R}^d:g(x) > 0\}$  is measurable. Similarly,  $\hat{g}_n^{-1}((- \infty ,0]) = \{x\in \mathbb{R}^d:\hat{g}_n(x)\leq 0\}$  is also measurable for all  $n\in \mathbb{N}$  since  $\hat{g}_n$  is continuous. Furthermore,  $X$  is measurable as it is compact. Therefore,  $E_{n}$  is measurable for all  $n\in \mathbb{N}$ . Now, since  $\hat{g}_n(x) < \hat{g}_{n + 1}(x)$  for all  $x\in X$  and all  $n\in \mathbb{N}$ , it holds that  $E_{n + 1}\subseteq E_n$  for all  $n\in \mathbb{N}$ . It is clear that to prove the result, it suffices to show that  $\lim_{n\to \infty}m(E_n) = 0$ . Therefore, if we show that  $m\left(\bigcap_{n\in \mathbb{N}}E_n\right) = 0$ , then the fact that  $m(E_1)\leq m(X) < \infty$  together with Lebesgue measure's continuity from above yields that  $\lim_{n\to \infty}m(E_n) = 0$ , thereby proving the result.
It remains to be shown that  $m\left(\bigcap_{n \in \mathbb{N}} E_n\right) = 0$ . To this end, suppose for the sake of contradiction that  $\bigcap_{n \in \mathbb{N}} E_n \neq \emptyset$ . Then there exists  $x \in \bigcap_{n \in \mathbb{N}} E_n$ , meaning that  $g(x) > 0$  and  $\hat{g}_n(x) \leq 0$  for all  $n \in \mathbb{N}$ . Thus, for this  $x \in X$ , we find that  $\lim \sup_{n \to \infty} \hat{g}_n(x) \leq 0 < g(x)$ , which contradicts the fact that  $\hat{g}_n$  uniformly converges to  $g$  on  $X$ . Therefore, it must be that  $\bigcap_{n \in \mathbb{N}} E_n = \emptyset$ , and thus  $m\left(\bigcap_{n \in \mathbb{N}} E_n\right) = 0$ , which concludes the proof.
Proof. Throughout this proof, we denote the complement of a set  $Y \subseteq \mathbb{R}^d$  by  $Y^c = \mathbb{R}^d \setminus Y$ .
Suppose that  $X_{1} = \{x^{(1)},\ldots ,x^{(M)}\} \subseteq \mathbb{R}^{d}$  and  $X_{2} = \{y^{(1)},\dots ,y^{(N)}\} \subseteq \mathbb{R}^{d}$  are such that  $(X_{1},X_{2})$  is convexly separable. Then, by definition of convex separability, there exists a nonempty closed convex set  $X^{\prime}\subseteq \mathbb{R}^{d}$  such that  $X_{2}\subseteq X^{\prime}$  and  $X_{1}\subseteq \mathbb{R}^{d}\setminus X^{\prime}$ . Let  $X = X^{\prime}\cap \operatorname {conv}(X_2)$ . Since  $X_{2}\subseteq X^{\prime}$  and both sets  $X^{\prime}$  and  $\mathrm{conv}(X_2)$  are convex, the set  $X$  is nonempty and convex. By finiteness of  $X_{2}$ , the set  $\mathrm{conv}(X_2)$  is compact, and therefore by closedness of  $X^{\prime}$ , the set  $X$  is compact and hence closed.
By Proposition 3.4, there exists  $f \in \mathcal{F}_{\mathrm{Id}}$  such that  $f^{-1}(\{2\}) = X$ . Since  $\operatorname{conv}(X_1 \cup X_2)$  is compact and convex, Lemma C.5 gives that there exists a sequence  $\{\hat{f}_n \in \hat{\mathcal{F}}_{\mathrm{Id}} : n \in \mathbb{N}\} \subseteq \hat{\mathcal{F}}_{\mathrm{Id}}$  such that  $\hat{g}_n(x) < \hat{g}_{n+1}(x) < g(x)$  for all  $x \in \operatorname{conv}(X_1 \cup X_2)$  and all  $n \in \mathbb{N}$  and  $\hat{g}_n$  converges uniformly to  $g$  on  $\operatorname{conv}(X_1 \cup X_2)$  as  $n \to \infty$ . Fix this sequence.
Let  $x \in X_2$ . Then, since  $X_2 \subseteq X'$  and  $X_2 \subseteq \operatorname{conv}(X_2)$ , it holds that  $x \in X' \cap \operatorname{conv}(X_2) = X = f^{-1}(\{2\})$ , implying that  $f(x) = 2$  and hence  $g(x) \leq 0$ . Since  $\hat{g}_n(x) < g(x)$  for all  $n \in \mathbb{N}$ , this shows that  $\hat{f}_n(x) = 2$  for all  $n \in \mathbb{N}$ . On the other hand, let  $i \in \{1, \dots, M\}$  and consider  $x = x^{(i)} \in X_1$ . Since  $X_1 \subseteq \mathbb{R}^d \setminus X' = \mathbb{R}^d \cap (X')^c \subseteq \mathbb{R}^d \cap (X' \cap \operatorname{conv}(X_2))^c = \mathbb{R}^d \cap X^c = \mathbb{R}^d \cap f^{-1}(\{1\})$ , it holds that  $f(x) = 1$  and thus  $g(x) > 0$ . Suppose for the sake of contradiction that  $\hat{f}_n(x) = 2$  for all  $n \in \mathbb{N}$ . Then  $\hat{g}_n(x) \leq 0$  for all  $n \in \mathbb{N}$ . Therefore, for this  $x \in X_1$ , we find that  $\lim_{n \to \infty} \hat{g}_n(x) \leq 0 < g(x)$ , which contradicts the fact that  $\hat{g}_n$  uniformly converges to  $g$  on  $\operatorname{conv}(X_1 \cup X_2)$ . Therefore, it must be that there exists  $n_i \in \mathbb{N}$  such that  $\hat{f}_{n_i}(x) = 1$ , and thus  $\hat{g}_{n_i}(x) > 0$ . Since  $\hat{g}_n(x) < \hat{g}_{n+1}(x)$  for all  $n \in \mathbb{N}$ , this implies that  $\hat{g}_n(x) > 0$  for all  $n \geq n_i$ . Hence,  $\hat{f}_n(x) = \hat{f}_n(x^{(i)}) = 1$  for all  $n \geq n_i$ .
Let  $n^{\star}$  be the maximum of all such  $n_i$ , i.e.,  $n^{\star} = \max \{n_i : i \in \{1, \ldots, M\}\}$ . Then the above analysis shows that  $\hat{f}_{n^{\star}}(x) = 2$  for all  $x \in X_2$  and that  $\hat{f}_{n^{\star}}(x) = 1$  for all  $x \in X_1$ . Since  $\hat{f}_{n^{\star}} \in \hat{\mathcal{F}}_{\mathrm{Id}}$ , the claim has been proven.
Theorem 3.10. Consider  $M, N \in \mathbb{N}$ . Let  $X_{1} = \{x^{(1)}, \ldots, x^{(M)}\} \subseteq \mathbb{R}^{d}$  and  $X_{2} = \{y^{(1)}, \ldots, y^{(N)}\} \subseteq \mathbb{R}^{d}$  be samples with all elements  $x_{k}^{(i)}, y_{l}^{(j)}$  drawn independently and identically from the uniform probability distribution on  $[-1, 1]$ . Then, it holds that
Proof. Throughout the proof, we denote the cardinality of a set  $S$  by  $|S|$ . For the reader's convenience, we also recall that, for  $n \in \mathbb{N}$ , the symmetric group  $S_{n}$  consists of all permutations (i.e., bijections) on the set  $\{1, 2, \ldots, n\}$ , and that  $|S_{n}| = n!$ . If  $\sigma \colon \{1, 2, \ldots, n\} \to \{1, 2, \ldots, n\}$  is a permutation in  $S_{n}$ , we denote the restriction of  $\sigma$  to the domain  $I \subseteq \{1, 2, \ldots, n\}$  by  $\sigma|_{I} \colon I \to \{1, 2, \ldots, n\}$ , which we recall is defined by  $\sigma|_{I}(i) = \sigma(i)$  for all  $i \in I$ , and is not necessarily a permutation on  $I$  in general.
Consider first the case where  $d \geq M + N$ . Let  $b \in \mathbb{R}^{M + N}$  be the vector defined by  $b_{i} = 1$  for all  $i \in \{1, \dots, M\}$  and  $b_{i} = -1$  for all  $i \in \{M + 1, \dots, M + N\}$ . Then, since  $x_{k}^{(i)}, y_{l}^{(j)}$  are independent uniformly distributed random variables on  $[-1, 1]$ , it holds that the matrix
$$
\left[ \begin{array}{c} x ^ {(1) ^ {\top}} \\ \vdots \\ x ^ {(M) ^ {\top}} \\ y ^ {(1) ^ {\top}} \\ \vdots \\ y ^ {(N) ^ {\top}} \end{array} \right] \in \mathbb {R} ^ {(M + N) \times d}
$$
has rank  $M + N$  almost surely, and therefore the linear system of equations
$$
\left[ \begin{array}{c} x ^ {(1) ^ {\top}} \\ \vdots \\ x ^ {(M) ^ {\top}} \\ y ^ {(1) ^ {\top}} \\ \vdots \\ y ^ {(N) ^ {\top}} \end{array} \right] a = b
$$
has a solution  $a \in \mathbb{R}^d$  with probability 1, and we note that from this solution we find that  $X_2$  is a subset of the nonempty closed convex set  $\{x \in \mathbb{R}^d : a^\top x \leq 0\}$  and that  $X_1$  is a subset of its complement. Hence,  $(X_1, X_2)$  is convexly separable with probability 1 in this case.
Now let us consider the general case:  $d \in \mathbb{N}$  and in general it may be the case that  $d < M + N$ . For notational convenience, let  $P$  be the probability of interest:
$$
P = \mathbb {P} \left(\left(X _ {1}, X _ {2}\right) \text {i s c o n v e x l y s e p a r a b l e}\right).
$$
Suppose that there exists a coordinate  $k \in \{1,2,\ldots,d\}$  such that  $x_{k}^{(i)} < y_{k}^{(j)}$  for all pairs  $(i,j) \in \{1,2,\ldots,M\} \times \{1,2,\ldots,N\}$  and that  $a \coloneqq \min \{y_{k}^{(1)},\dots,y_{k}^{(N)}\} < \max \{y_{k}^{(1)},\dots,y_{k}^{(N)}\} =: b$ . Then, let  $X = \{x \in \mathbb{R}^d : x_k \in [a,b]\}$ . That is,  $X$  is the extrusion of the convex hull of the projections  $\{y_{k}^{(1)},\dots,y_{k}^{(N)}\}$  along all remaining coordinates. The set  $X$  is a nonempty closed convex set, and it