SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Samuel Pfrommer* Brendon G. Anderson* Julien Piet Somayeh Sojoudi
{sam.pfrommer, bganderson, piet, sojoudi}@berkeley.edu
Real-world adversarial attacks on machine learning models often feature an asymmetric structure wherein adversaries only attempt to induce false negatives (e.g., classify a spam email as not spam). We formalize the asymmetric robustness certification problem and correspondingly present the feature-convex neural network architecture, which composes an input-convex neural network (ICNN) with a Lipschitz continuous feature map in order to achieve asymmetric adversarial robustness. We consider the aforementioned binary setting with one "sensitive" class, and for this class we prove deterministic, closed-form, and easily-computable certified robust radii for arbitrary  $\ell_p$ -norms. We theoretically justify the use of these models by extending the universal approximation theorem for ICNN regression to the classification setting, and proving a lower bound on the probability that such models perfectly fit even unstructured uniformly distributed data in sufficiently high dimensions. Experiments on Maling malware classification and subsets of the MNIST, Fashion-MNIST, and CIFAR-10 datasets show that feature-convex classifiers attain substantial certified  $\ell_1$ ,  $\ell_2$ , and  $\ell_{\infty}$ -radii while being far more computationally efficient than competitive baselines. $^1$
Although neural networks achieve state-of-the-art performance across a range of machine learning tasks, researchers have shown that they can be highly sensitive to adversarial inputs that are maliciously designed to fool the model [11, 61, 52]. For example, the works Eykholt et al. [22] and Liu et al. [42] show that small physical and digital alterations of vehicle traffic signs can cause image classifiers to fail. In safety-critical applications of neural networks, such as autonomous driving [12, 69] and medical diagnostics [1, 71], this sensitivity to adversarial inputs is clearly unacceptable.
A line of heuristic defenses against adversarial inputs has been proposed, only to be defeated by stronger attack methods [14, 36, 7, 64, 46]. This has led researchers to develop certifiably robust methods that provide a provable guarantee of safe performance. The strength of such certificates can be highly dependent on network architecture; general off-the-shelf models tend to have large Lipschitz constants, leading to loose Lipschitz-based robustness guarantees [29, 23, 73]. Consequently, lines of work that impose certificate-amenable structures onto networks have been popularized, e.g., specialized model layers [63, 77], randomized smoothing-based networks [41, 18, 76, 72, 3], and ReLU networks that are certified using convex optimization and mixed-integer programming [68, 67, 54, 4, 45]. The first category only directly certifies against one specific choice of norm, producing poorly scaled radii for other norms in high dimensions. The latter two approaches incur serious computational challenges: randomized smoothing typically requires the classification of thousands
$^{1}$ Code for reproducing our results is available on GitHub.
of randomly perturbed samples per input, while optimization-based solutions scale poorly to large networks.
Despite the moderate success of these certifiable classifiers, conventional assumptions in the literature are unnecessarily restrictive for many practical adversarial settings. Specifically, most works consider a multiclass setting where certificates are desired for inputs of any class. By contrast, many real-world adversarial attacks involve a binary setting with only one sensitive class that must be robust to adversarial perturbations. Consider the representative problem of spam classification; a malicious adversary crafting a spam email will only attempt to fool the classifier toward the "not-spam" class—never conversely [20]. Similar logic applies for a range of applications such as malware detection [28], malicious network traffic filtering [56], fake news and social media bot detection [19], hate speech removal [27], insurance claims filtering [24], and financial fraud detection [15].
The important asymmetric nature of these classification problems has long been recognized in various subfields, and some domain-specific attempts at robustification have been proposed with this in mind. This commonly involves robustifying against adversaries appending features to the classifier input. In spam classification, such an attack is known as the "good word" attack [44]. In malware detection, numerous approaches have been proposed to provably counter such additive-only adversaries using special classifier structures such as non-negative networks [25] and monotonic classifiers [32]. We note these works strictly focus on additive adversaries and cannot handle general adversarial perturbations of the input that are capable of perturbing existing features. We propose adding this important asymmetric structure to the study of norm ball-certifiably robust classifiers. This narrowing of the problem to the asymmetric setting provides prospects for novel certifiable architectures, and we present feature-convex neural networks as one such possibility.
1.1 Problem Statement and Contributions
This section formalizes the asymmetric robustness certification problem for general norm-bounded adversaries. Specifically, we assume a binary classification setting wherein one class is "sensitive"—meaning we seek to certify that, if some input is classified into this sensitive class, then adversarial perturbations of sufficiently small magnitude cannot change the prediction.
Formally, consider a binary classifier  $f_{\tau} \colon \mathbb{R}^d \to \{1, 2\}$ , where class 1 is the sensitive class for which we desire certificates. We take  $f_{\tau}$  to be a standard thresholded version of a soft classifier  $g \colon \mathbb{R}^d \to \mathbb{R}$ , expressible as  $f_{\tau}(x) = T_{\tau}(g(x))$ , where  $T_{\tau} \colon \mathbb{R} \to \{1, 2\}$  is the thresholding function defined by
$$
T _ {\tau} (y) = \left\{ \begin{array}{l l} 1 & \text {i f} y + \tau > 0, \\ 2 & \text {i f} y + \tau \leq 0, \end{array} \right. \tag {1}
$$
with  $\tau \in \mathbb{R}$  being a user-specified parameter that shifts the classification threshold. A classifier  $f_{\tau}$  is considered certifiably robust at a class 1 input  $x \in \mathbb{R}^d$  with a radius  $r(x) \in \mathbb{R}_+$  if  $f_{\tau}(x + \delta) = f_{\tau}(x) = 1$  for all  $\delta \in \mathbb{R}^d$  with  $\| \delta \| < r(x)$  for some norm  $\| \cdot \|$ . Thus,  $\tau$  induces a tradeoff between the clean accuracy on class 2 and certification performance on class 1. As  $\tau \to \infty$ ,  $f_{\tau}$  approaches a constant classifier which achieves infinite class 1 certified radii but has zero class 2 accuracy.
For a particular choice of  $\tau$ , the performance of  $f_{\tau}$  can be analyzed similarly to a typical certified classifier. Namely, it exhibits a class 2 clean accuracy  $\alpha_{2}(\tau) \in [0,1]$  as well as a class 1 certified accuracy surface  $\Gamma$  with values  $\Gamma(r,\tau) \in [0,1]$  that capture the fraction of the class 1 samples that can be certifiably classified by  $f_{\tau}$  at radius  $r \in \mathbb{R}_{+}$ . The class 1 clean accuracy  $\alpha_{1}(\tau) = \Gamma(0,\tau)$  is inferable from  $\Gamma$  as the certified accuracy at  $r = 0$ .
The full asymmetric certification performance of the family of classifiers  $f_{\tau}$  can be captured by plotting the surface  $\Gamma(r, \tau)$ , as will be shown in Figure 1a. Instead of plotting against  $\tau$  directly, we plot against the more informative difference in clean accuracies  $\alpha_{1}(\tau) - \alpha_{2}(\tau)$ . This surface can be viewed as an asymmetric robustness analogue to the classic receiver operating characteristic curve.
Note that while computing the asymmetric robustness surface is possible for our feature-convex architecture (to be defined shortly), it is computationally prohibitive for conventional certification methods. We therefore standardize our comparisons throughout this work to the certified accuracy cross section  $\Gamma(r, \tau^{*})$  for a  $\tau^{*}$  such that clean accuracies are balanced in the sense that  $\alpha_{2}(\tau^{*}) = \alpha_{1}(\tau^{*})$ , noting that  $\alpha_{1}$  monotonically increases in  $\tau$  and  $\alpha_{2}$  monotonically decreases in  $\tau$ . We discuss finding such a  $\tau^{*}$  in Appendix E.4. This choice allows for a direct comparison of the resulting certified accuracy curves without considering the non-sensitive class clean accuracy.
With the above formalization in place, the goal at hand is two-fold: 1) develop a classification architecture tailored for the asymmetric setting with high robustness, as characterized by the surface  $\Gamma$ , and 2) provide efficient methods for computing the certified robust radii  $r(x)$  used to generate  $\Gamma$ .
Contributions. We tackle the above two goals by proposing feature-convex neural networks and achieve the following contributions:
1. We exploit the feature-convex structure of the proposed classifier to provide asymmetrically tailored closed-form class 1 certified robust radii for arbitrary  $\ell_p$ -norms, solving the second goal above and yielding efficient computation of  $\Gamma$ .
2. We characterize the decision region geometry of convex classifiers, extend the universal approximation theorem for input-convex ReLU neural networks to the classification setting, and show that convex classifiers are sufficiently expressive for high-dimensional data.
3. We evaluate against several baselines on MNIST 3-8 [37], Malimg malware classification [50], Fashion-MNIST shirts [70], and CIFAR-10 cats-dogs [35], and show that our classifiers yield certified robust radii competitive with the state-of-the-art, empirically addressing the first goal listed above.
All proofs and appendices can be found in the Supplemental Material.
1.2 Related Works
Certified adversarial robustness. Three of the most popular approaches for generating robustness certificates are Lipschitz-based bounds, randomized smoothing, and optimization-based methods. Successfully bounding the Lipschitz constant of a neural network can give rise to an efficient certified radius of robustness, e.g., via the methods proposed in Hein and Andriushchenko [29]. However, in practice such Lipschitz constants are too large to yield meaningful certificates, or it is computationally burdensome to compute or bound the Lipschitz constants in the first place [65, 23, 73]. To overcome these computational limitations, certain methods impose special structures on their model layers to provide immediate Lipschitz guarantees. Specifically, Trockman and Kolter [63] uses the Cayley transform to derive convolutional layers with immediate  $\ell_2$ -Lipschitz constants, and Zhang et al. [77] introduces a  $\ell_{\infty}$ -distance neuron that provides similar Lipschitz guarantees with respect to the  $\ell_{\infty}$ -norm. We compare with both these approaches in our experiments.
Randomized smoothing, popularized by Lecuyer et al. [38], Li et al. [41], Cohen et al. [18], uses the expected prediction of a model when subjected to Gaussian input noise. These works derive  $\ell_2$ -norm balls around inputs on which the smoothed classifier remains constant, but suffer from nondeterminism and high computational burden. Follow-up works generalize randomized smoothing to certify input regions defined by different metrics, e.g., Wasserstein,  $\ell_1$ -, and  $\ell_{\infty}$ -norms [39, 62, 72]. Other works focus on enlarging the certified regions by optimizing the smoothing distribution [76, 21, 5], incorporating adversarial training into the base classifier [57, 78], and employing dimensionality reduction at the input [53].
Optimization-based certificates typically seek to derive a tractable over-approximation of the set of possible outputs when the input is subject to adversarial perturbations, and show that this overapproximation is safe. Various over-approximations have been proposed, e.g., based on linear programming and bounding [68, 67], semidefinite programming [54], and branch-and-bound [4, 45, 66]. The  $\alpha$ ,  $\beta$ -CROWN method [66] uses an efficient bound propagation to linearly bound the neural network output in conjunction with a per-neuron branching heuristic to achieve state-of-the-art certified radii, winning both the 2021 and the 2022 VNN certification competitions [8, 48]. In contrast to optimization-based methods, our approach directly exploits the convex structure of input-convex neural networks to derive closed-form robustness certificates, altogether avoiding any efficiency-tightness tradeoffs.
Input-convex neural networks. Input-convex neural networks, popularized by Amos et al. [2], are a class of parameterized models whose input-output mapping is convex. The authors develop tractable methods to learn input-convex neural networks, and show that such models yield state-of-the-art results in a variety of domains where convexity may be exploited, e.g., optimization-based inference. Subsequent works propose novel applications of input-convex neural networks in areas
such as optimal control and reinforcement learning [16, 75], optimal transport [47], and optimal power flow [17, 79]. Other works have generalized input-convex networks to input-invex networks [58, 51] and global optimization networks [80] so as to maintain the benign optimization properties of input-convexity. The authors of Siahkamari et al. [59] present algorithms for efficiently learning convex functions, while Chen et al. [16], Kim and Kim [34] derive universal approximation theorems for input-convex neural networks in the convex regression setting. The work Sivaprasad et al. [60] shows that input-convex neural networks do not suffer from overfitting, and generalize better than multilayer perceptrons on common benchmark datasets. In this work, we incorporate input-convex neural networks as a part of our feature-convex architecture and leverage convexity properties to derive novel robustness guarantees.
1.3 Notations
The sets of natural numbers, real numbers, and nonnegative real numbers are denoted by  $\mathbb{N}$ ,  $\mathbb{R}$ , and  $\mathbb{R}_+$  respectively. The  $d \times d$  identity matrix is written as  $I_d \in \mathbb{R}^{d \times d}$ , and the identity map on  $\mathbb{R}^d$  is denoted by  $\operatorname{Id} \colon x \mapsto x$ . For  $A \in \mathbb{R}^{n \times d}$ , we define  $|A| \in \mathbb{R}^{n \times d}$  by  $|A|_{ij} = |A_{ij}|$  for all  $i, j$ , and we write  $A \geq 0$  if and only if  $A_{ij} \geq 0$  for all  $i, j$ . The  $\ell_p$ -norm on  $\mathbb{R}^d$  is given by  $\| \cdot \|_p \colon x \mapsto (|x_1|^p + \dots + |x_d|^p)^{1/p}$  for  $p \in [1, \infty)$  and by  $\| \cdot \|_p \colon x \mapsto \max \{|x_1|, \ldots, |x_d|\}$  for  $p = \infty$ . The dual norm of  $\| \cdot \|_p$  is denoted by  $\| \cdot \|_{p,*}$ . The convex hull of a set  $X \subseteq \mathbb{R}^d$  is denoted by  $\operatorname{conv}(X)$ . The subdifferential of a convex function  $g \colon \mathbb{R}^d \to \mathbb{R}$  at  $x \in \mathbb{R}^d$  is denoted by  $\partial g(x)$ . If  $\epsilon \colon \Omega \to \mathbb{R}^d$  is a random variable on a probability space  $(\Omega, \mathcal{B}, \mathbb{P})$  and  $P$  is a predicate defined on  $\mathbb{R}^d$ , then we write  $\mathbb{P}(P(\epsilon))$  to mean  $\mathbb{P}(\{\omega \in \Omega : P(\epsilon(\omega))\})$ . Lebesgue measure on  $\mathbb{R}^d$  is denoted by  $m$ . We define  $\mathrm{ReLU} \colon \mathbb{R} \to \mathbb{R}$  as  $\mathrm{ReLU}(x) = \max\{0, x\}$ , and if  $x \in \mathbb{R}^d$ ,  $\mathrm{ReLU}(x)$  denotes  $(\mathrm{ReLU}(x_1), \ldots, \mathrm{ReLU}(x_d))$ . We recall the threshold function  $T_\tau \colon \mathbb{R} \to \{1, 2\}$  defined by (1), and we define  $T = T_0$ . For a function  $\varphi \colon \mathbb{R}^d \to \mathbb{R}^q$  and  $p \in [1, \infty]$ , we define  $\mathrm{Lip}_p(\varphi) = \inf\{K \geq 0 : \| \varphi(x) - \varphi(x')\|_p \leq K \| x - x' \|_p$  for all  $x, x' \in \mathbb{R}^d\}$ , and if  $\mathrm{Lip}_p(\varphi) < \infty$  we say that  $\varphi$  is Lipschitz continuous with constant  $\mathrm{Lip}_p(\varphi)$  (with respect to the  $\ell_p$ -norm).
2 Feature-Convex Classifiers
Let  $d, q \in \mathbb{N}$  and  $p \in [1, \infty]$  be fixed, and consider the task of classifying inputs from a subset of  $\mathbb{R}^d$  into a fixed set of classes  $\mathcal{Y} \subseteq \mathbb{N}$ . In what follows, we restrict to the binary setting where  $\mathcal{Y} = \{1, 2\}$  and class 1 is the sensitive class for which we desire robustness certificates (Section 1). In Appendix A, we briefly discuss avenues to generalize our framework to multiclass settings using one-versus-all and sequential classification methodologies and provide a proof-of-concept example for the Malimg dataset.
We now formally define the classifiers considered in this work. Note that the classification threshold  $\tau$  discussed in Section 1.1 is omitted for simplicity.
Definition 2.1. Let  $f\colon \mathbb{R}^d\to \{1,2\}$  be defined by  $f(x) = T(g(\varphi (x)))$  for some  $\varphi \colon \mathbb{R}^d\to \mathbb{R}^q$  and some  $g\colon \mathbb{R}^q\to \mathbb{R}$ . Then  $f$  is said to be a feature-convex classifier if the feature map  $\varphi$  is Lipschitz continuous with constant  $\mathrm{Lip}_p(\varphi) < \infty$  and  $g$  is a convex function.
We denote the class of all feature-convex classifiers by  $\mathcal{F}$ . Furthermore, for  $q = d$ , the subclass of all feature-convex classifiers with  $\varphi = \mathrm{Id}$  is denoted by  $\mathcal{F}_{\mathrm{Id}}$ .
As we will see in Section 3.1, defining our classifiers using the composition of a convex classifier with a Lipschitz feature map enables the fast computation of certified regions in the input space. This naturally arises from the global underestimation of convex functions by first-order Taylor approximations. Since sublevel sets of such  $g$  are restricted to be convex, the feature map  $\varphi$  is included to increase the representation power of our architecture (see Appendix B for a motivating example). In practice, we find that it suffices to choose  $\varphi$  to be a simple map with a small closed-form Lipschitz constant. For example, in our experiments that follow with  $q = 2d$ , we choose  $\varphi(x) = (x - \mu, |x - \mu|)$  with a constant channel-wise dataset mean  $\mu$ , yielding  $\mathrm{Lip}_1(\varphi) \leq 2$ ,  $\mathrm{Lip}_2(\varphi) \leq \sqrt{2}$ , and  $\mathrm{Lip}_{\infty}(\varphi) \leq 1$ . Although this particular choice of  $\varphi$  is convex, the function  $g$  need not be monotone, and therefore the composition  $g \circ \varphi$  is nonconvex in general. The prediction and certification of feature-convex classifiers are illustrated in Figure 1b.
(a)
Figure 1: (a) The asymmetric certified accuracy surface  $\Gamma(r, \tau)$  for MNIST 3-8, as described in Section 1.1. The "clean accuracy difference" axis plots  $\alpha_{1}(\tau) - \alpha_{2}(\tau)$ , and the black line highlights the certified robustness curve for when clean accuracy is equal across the two classes. (b) Illustration of feature-convex classifiers and their certification. Since  $g$  is convex, it is globally underapproximated by its tangent plane at  $\varphi(x)$ , yielding certified sets for norm balls in the higher-dimensional feature space. Lipschitzness of  $\varphi$  then yields appropriately scaled certificates in the original input space.
In practice, we implement feature-convex classifiers using parameterizations of  $g$ , which we now make explicit. Following Amos et al. [2], we instantiate  $g$  as a neural network with nonnegative weight matrices and nondecreasing convex nonlinearities. Specifically, we consider ReLU nonlinearities, which is not restrictive, as our universal approximation result in Theorem 3.6 proves.
Definition 2.2. A feature-convex ReLU neural network is a function  $\hat{f} \colon \mathbb{R}^d \to \{1,2\}$  defined by  $\hat{f}(x) = T(\hat{g}(\varphi(x)))$  with  $\varphi \colon \mathbb{R}^d \to \mathbb{R}^q$  Lipschitz continuous with constant  $\mathrm{Lip}_p(\varphi) < \infty$  and  $\hat{g} \colon \mathbb{R}^q \to \mathbb{R}$  defined by
$$
\hat {g} (x ^ {(0)}) = A ^ {(L)} x ^ {(L - 1)} + b ^ {(L)} + C ^ {(L)} x ^ {(0)}, \quad x ^ {(l)} = \mathrm {R e L U} \left(A ^ {(l)} x ^ {(l - 1)} + b ^ {(l)} + C ^ {(l)} x ^ {(0)}\right),
$$
for all  $l \in \{1, 2, \ldots, L - 1\}$  for some  $L \in \mathbb{N}, L > 1$ , and for some consistently sized matrices  $A^{(l)}, C^{(l)}$  and vectors  $b^{(l)}$  satisfying  $A^{(l)} \geq 0$  for all  $l \in \{2, 3, \ldots, L\}$ .
Going forward, we denote the class of all feature-convex ReLU neural networks by  $\hat{\mathcal{F}}$ . Furthermore, if  $q = d$ , the subclass of all feature-convex ReLU neural networks with  $\varphi = \mathrm{Id}$  is denoted by  $\hat{\mathcal{F}}_{\mathrm{Id}}$  which corresponds to the input-convex ReLU neural networks proposed in Amos et al. [2].
For every  $\hat{f} \in \hat{\mathcal{F}}$ , it holds that  $\hat{g}$  is convex due to the rules for composition and nonnegatively weighted sums of convex functions [13, Section 3.2], and therefore  $\hat{\mathcal{F}} \subseteq \mathcal{F}$  and  $\hat{\mathcal{F}}_{\mathrm{Id}} \subseteq \mathcal{F}_{\mathrm{Id}}$ . The "passthrough" weights  $C^{(l)}$  were originally included by Amos et al. [2] to improve the practical performance of the architecture. In some of our more challenging experiments that follow, we remove these passthrough operations and instead add residual identity mappings between hidden layers, which also preserves convexity. We note that the transformations defined by  $A^{(l)}$  and  $C^{(l)}$  can be taken to be convolutions, which are nonnegatively weighted linear operations and thus preserve convexity [2].
3 Certification and Analysis of Feature-Convex Classifiers
We present our main theoretical results in this section. First, we derive asymmetric robustness certificates (Theorem 3.1) for our feature-convex classifiers in Section 3.1. Then, in Section 3.2, we introduce the notion of convexly separable sets in order to theoretically characterize the representation power of our classifiers. Our primary representation results give a universal function approximation
theorem for our classifiers with  $\varphi = \mathrm{Id}$  and ReLU activation functions (Theorem 3.6) and show that such classifiers can perfectly fit convexly separable datasets (Theorem 3.7), including the CIFAR-10 cats-dogs training data (Fact 3.8). We also show that this strong learning capacity generalizes by proving that feature-convex classifiers can perfectly fit high-dimensional uniformly distributed data with high probability (Theorem 3.10).
3.1 Certified Robustness Guarantees
In this section, we address the asymmetric certified robustness problem by providing class 1 robustness certificates for feature-convex classifiers  $f \in \mathcal{F}$ . Such robustness corresponds to proving the absence of false negatives in the case that class 1 represents positives and class 2 represents negatives. For example, if in a malware detection setting class 1 represents malware and class 2 represents non-malware, the following certificate gives a lower bound on the magnitude of the malware file alteration needed in order to misclassify the file as non-malware.
Theorem 3.1. Let  $f \in \mathcal{F}$  be as in Definition 2.1 and let  $x \in f^{-1}(\{1\}) = \{x' \in \mathbb{R}^d : f(x') = 1\}$ . If  $\nabla g(\varphi(x)) \in \mathbb{R}^q$  is a nonzero subgradient of the convex function  $g$  at  $\varphi(x)$ , then  $f(x + \delta) = 1$  for all  $\delta \in \mathbb{R}^d$  such that
$$
\| \delta \| _ {p} <   r (x) := \frac {g (\varphi (x))}{\operatorname {L i p} _ {p} (\varphi) \| \nabla g (\varphi (x)) \| _ {p , *}}.
$$