SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yan Scholten<sup>1</sup>, Jan Schuchardt<sup>1</sup>, Aleksandar Bojchevski<sup>2</sup> & Stephan Gunnemann<sup>1</sup>
{y.scholten, j.schuchardt, s.guennemann}@tum.de, a.bojchevski@uni-koeln.de
$^{1}$ Dept. of Computer Science & Munich Data Science Institute, Technical University of Munich
$^{2}$ University of Cologne, Germany
Real-world data is complex and often consists of objects that can be decomposed into multiple entities (e.g. images into pixels, graphs into interconnected nodes). Randomized smoothing is a powerful framework for making models provably robust against small changes to their inputs – by guaranteeing robustness of the majority vote when randomly adding noise before classification. Yet, certifying robustness on such complex data via randomized smoothing is challenging when adversaries do not arbitrarily perturb entire objects (e.g. images) but only a subset of their entities (e.g. pixels). As a solution, we introduce hierarchical randomized smoothing: We partially smooth objects by adding random noise only on a randomly selected subset of their entities. By adding noise in a more targeted manner than existing methods we obtain stronger robustness guarantees while maintaining high accuracy. We initialize hierarchical smoothing using different noisings distributions, yielding novel robustness certificates for discrete and continuous domains. We experimentally demonstrate the importance of hierarchical smoothing in image and node classification, where it yields superior robustness-accuracy trade-offs. Overall, hierarchical smoothing is an important contribution towards models that are both – certifiably robust to perturbations and accurate.<sup>1</sup>
Machine learning models are vulnerable to small input changes. Consequently, their performance is typically better during development than in the real world where noise, incomplete data and adversaries are present. To address this problem, robustness certificates constitute useful tools that provide provable guarantees for the stability of predictions under worst-case perturbations, enabling us to analyze and guarantee robustness of our classifiers in practice.
Randomized smoothing (Lecuyer et al., 2019; Cohen et al., 2019) is a powerful and highly flexible framework for making arbitrary classifiers certifiably robust by averaging out small discontinuities in the underlying model. This is achieved by guaranteeing robustness of the majority vote when randomly adding noise to a model's input before classification. This typically yields a robustness-accuracy trade-off – more noise increases certifiable robustness but also reduces clean accuracy.
Recent adversarial attacks against image classifiers (Su et al., 2019; Croce and Hein, 2019; Modas et al., 2019) and graph neural networks (Ma et al., 2020) further motivate the need for more flexible robustness certificates against threat models where adversaries are bounded by both: the number of perturbed entities and perturbation strength. For example in social networks, adversaries typically cannot control entire graphs, but the perturbation of the controlled nodes may be (imperceptibly) strong.
When adversaries perturb only a subset of all entities, existing smoothing-based approaches sacrifice robustness over accuracy (or vice versa): They either add noise to entire objects (e.g. images/graphs) or ablate entire entities (e.g. pixels/nodes) even if only a small subset is perturbed and requires noising.
<sup>1</sup>Project page: https://www.cs.cit.tum.de/daml/hierarchical-smoothing
Figure 1: Hierarchical randomized smoothing: We first select a subset of all entities and then add noise to the selected entities only. We achieve stronger robustness guarantees while still maintaining high accuracy - especially when adversaries can only perturb a subset of all entities. For example in social networks, adversaries typically control only a subset of all nodes in the entire graph.
The absence of appropriate tools for decomposable data prevents us from effectively analyzing the robustness of models such as image classifiers or graph neural networks – raising the research question: How can we provably guarantee robustness under adversaries that perturb only a subset of all entities? We propose hierarchical randomized smoothing: By only adding noise on a randomly selected subset of all entities we achieve stronger robustness guarantees while maintaining high accuracy (Figure 1).
Our hierarchical smoothing framework is highly flexible and can integrate the whole suite of smoothing distributions for adding random noise to the input, provided that the corresponding certificates exist. Notably, our certificates apply to a wide range of models, domains and tasks and are at least as efficient as the certificates for the underlying smoothing distribution.
We instantiate hierarchical smoothing with well-established smoothing distributions (Cohen et al., 2019; Bojchevski et al., 2020; Levine and Feizi, 2020b; Scholten et al., 2022), yielding novel robustness certificates for discrete and continuous domains. Experimentally, hierarchical smoothing significantly expands the Pareto-front when optimizing for accuracy and robustness, revealing novel insights into the reliability of image classifiers and graph neural networks (GNNs).
In short, our main contributions are:
- We propose a novel framework for strong robustness certificates for complex data where objects (e.g. images or graphs) can be decomposed into multiple entities (e.g. pixels or nodes).
- We instantiate our framework with well-established smoothing distributions, yielding novel robustness certificates for discrete and continuous domains.
- We demonstrate the importance of hierarchical smoothing in image and node classification, where it significantly expands the Pareto-front with respect to robustness and accuracy.
Despite numerous research efforts, the vulnerability of machine learning models against adversarial examples remains an open research problem (Hendrycks et al., 2021). Various tools emerged to address adversarial robustness: Adversarial attacks and defenses allow to evaluate and improve robustness empirically (Szegedy et al., 2014; Goodfellow et al., 2015; Carlini et al., 2019; Croce and Hein, 2019). While attacks provide an upper bound on the adversarial robustness of a model, robustness certificates provide a provable lower bound. The development of strong certificates is challenging and constitutes a research field with many open questions (Li et al., 2020).
Robustness certification via randomized smoothing. Randomized smoothing (Lecuyer et al., 2019; Cohen et al., 2019) represents a framework for making arbitrary classifiers certifiably robust while scaling to deeper architectures. There are two main approaches discussed in the literature: Additive noise certificates add random noise to continuous (Cohen et al., 2019) or discrete (Lee et al., 2019) input data. Ablation certificates "mask" out parts of the input to hide the potentially perturbed information (Levine and Feizi, 2020b). Since the input is masked, ablation certificates can be applied to both continuous and discrete data. We are the first to combine and generalize additive noise and ablation certificates into a novel hierarchical framework: We first select entities to ablate, but instead of ablating we add noise to them. By only partially adding noise to the input our framework is more flexible and allows us to better control the robustness-accuracy trade-off under our threat model.
Further research directions and orthogonal to ours include derandomized (deterministic) smoothing (Levine and Feizi, 2020a, 2021; Horváth et al., 2022b; Scholten et al., 2022), input-dependent smoothing (Súkeník et al., 2022), and gray-box certificates (Mohapatra et al., 2020; Levine et al., 2020; Schuchardt and Gunnemann, 2022; Scholten et al., 2022; Schuchardt et al., 2023a).
Robustness-accuracy trade-off. The robustness-accuracy trade-off has been extensively discussed in the literature (Tsipras et al., 2019; Raghunathan et al., 2019; Zhang et al., 2019; Yang et al., 2020b; Xie et al., 2020), including works for graph data (Gosch et al., 2023b). In randomized smoothing the parameters of the smoothing distribution allow to trade robustness against accuracy (Cohen et al., 2019; Mohapatra et al., 2020; Wu et al., 2021) where more noise means higher robustness but lower accuracy. Our framework combines two smoothing distributions while introducing parameters for both, which allows us to better control the robustness-accuracy trade-off under our threat model since we can control both – the number of entities to smooth and the magnitude of the noise. Further advances in the trade-off in randomized smoothing include the work of Levine and Feizi (2020a) against patch-attacks, Schuchardt et al. (2021, 2023b) for collective robustness certification, and Scholten et al. (2022) for graph neural networks. Other techniques include consistency regularization (Jeong and Shin, 2020), denoised smoothing (Salman et al., 2020) and ensemble techniques (Müller et al., 2021; Horváth et al., 2022a,c). Notably, our method is orthogonal to such enhancements since we derive certificates for a novel, flexible smoothing distribution to better control such trade-offs.
Robustness certificates for GNNs. Research on GNN certification is still at an early stage (Gunnemann, 2022), most works focus on heuristic defenses rather than provable robustness guarantees. But heuristics cannot guarantee robustness and may be broken by stronger attacks in the future (Mujkanovic et al., 2022). Most certificates are limited to specific architectures (Zügner and Gunnemann, 2020; Jin et al., 2020; Bojchevski and Gunnemann, 2019; Zügner and Gunnemann, 2019). Randomized smoothing is a more flexible framework for certifying arbitrary models while only accessing their forward pass. We compare our method to additive noise certificates for sparse data by Bojchevski et al. (2020), and to ablation certificates by Scholten et al. (2022) that ablate all node features to derive certificates against strong adversaries that control entire nodes. While we implement certificates against node-attribute perturbations, our framework can in principle integrate smoothing-based certificates against graph-structure perturbations as well.
3 Preliminaries and background
Our research is focused on data where objects can be decomposed into entities. W.l.o.g. we represent such objects as matrices where rows represent entities. We assume a classification task on a discrete or continuous  $(N\times D)$ -dimensional space  $\mathcal{X}^{N\times D}$  (e.g.  $\mathcal{X} = \{0,1\}$  or  $\mathcal{X} = \mathbb{R}$ ). We propose certificates for classifiers  $f:\mathcal{X}^{N\times D}\to \mathcal{V}$  that assign matrix  $\pmb {X}\in \mathcal{X}^{N\times D}$  a label  $y\in \mathcal{V} = \{1,\dots ,C\}$ . We model evasion settings, i.e. adversaries perturb the input at inference. We seek to verify the robustness  $f(\pmb {X}) = f(\tilde{\pmb{X}})$  for perturbed  $\tilde{\pmb{X}}$  from a threat model of admissible perturbations:
$$
\mathcal {B} _ {p, \epsilon} ^ {r} (\mathbf {X}) \triangleq \left\{\tilde {\mathbf {X}} \in \mathcal {X} ^ {N \times D} \mid \sum_ {i = 1} ^ {N} \mathbb {1} [ \mathbf {x} _ {i} \neq \tilde {\mathbf {x}} _ {i} ] \leq r, | | \operatorname {v e c} (\mathbf {X} - \tilde {\mathbf {X}}) | | _ {p} \leq \epsilon \right\}
$$
where  $\mathbb{1}$  denotes an indicator function,  $||\cdot ||_p$  a  $\ell_p$ -vector norm and  $\mathrm{vec}(\cdot)$  the matrix vectorization. For example,  $||\mathrm{vec}(\cdot)||_2$  is the Frobenius-norm. Intuitively, we model adversaries that control up to  $r$  rows in the matrix, and whose perturbations are bounded by  $\epsilon$  under a fixed  $\ell_p$ -norm.
3.1 Randomized smoothing framework
Our certificates build on the randomized smoothing framework (Lecuyer et al., 2019; Li et al., 2019; Cohen et al., 2019). Instead of certifying a base classifier  $f$ , one constructs a smoothed classifier  $g$  that corresponds to the majority vote prediction of  $f$  under a randomized smoothing of the input. We define the smoothed classifier  $g: \mathcal{X}^{N \times D} \to \mathcal{Y}$  for a smoothing distribution  $\mu_{\mathbf{X}}$  as:
$$
g(\boldsymbol {X})\triangleq \operatorname *{argmax}_{y\in \mathcal{Y}}p_{\boldsymbol {X},y}\qquad p_{\boldsymbol {X},y}\triangleq \operatorname *{Pr}_{\boldsymbol {W}\sim \mu_{\boldsymbol{X}}}  [f(\boldsymbol {W}) = y]
$$
where  $p_{\mathbf{X},y}$  is the probability of predicting class  $y$  for input  $\mathbf{X}$  under the smoothing distribution  $\mu_{\mathbf{X}}$ . Let  $y^{*} \triangleq g(\mathbf{X})$  further denote the majority vote of  $g$ , i.e. the most likely prediction for the clean  $\mathbf{X}$ .
${}^{2}$  Multi-output classifiers can be treated as multiple independent single-output classifiers.
To derive robustness certificates for the smoothed classifier  $g$  we have to show that  $y^{*} = g(\mathbf{X}) = g(\tilde{\mathbf{X}})$  for clean matrix  $\mathbf{X}$  and any perturbed matrix  $\tilde{\mathbf{X}} \in \mathcal{B}_{p,\epsilon}^{r}(\mathbf{X})$ . This is the case if the probability to classify any perturbed matrix  $\tilde{\mathbf{X}}$  as  $y^{*}$  is still larger than for all other classes, that is if  $p_{\tilde{\mathbf{X}},y^*} > 0.5$ . However, randomized smoothing is designed to be flexible and does not assume anything about the underlying base classifier, i.e. all we know about  $f$  is the probability  $p_{\mathbf{X},y}$  for the clean matrix  $\mathbf{X}$ . We therefore derive a lower bound  $p_{\tilde{\mathbf{X}},y} \leq p_{\tilde{\mathbf{X}},y}$  under a worst-case assumption: We consider the least robust classifier among all classifiers with the same probability  $p_{\mathbf{X},y}$  of classifying  $\mathbf{X}$  as  $y$ :
$$
\underline {{\underline {{p}}}} _ {\boldsymbol {\tilde {X}}, y} \triangleq \min  _ {h \in \mathbb {H}} \Pr_ {\boldsymbol {W} \sim \mu_ {\boldsymbol {\tilde {X}}}} [ h (\boldsymbol {W}) = y ] \quad s. t. \quad \underset {\boldsymbol {W} \sim \mu_ {\boldsymbol {X}}} {\operatorname * {P r}} [ h (\boldsymbol {W}) = y ] = p _ {\boldsymbol {X}, y} \tag {1}
$$
where  $\mathbb{H} \triangleq \{h : \mathcal{X}^{N \times D} \to \mathcal{Y}\}$  is the set of possible classifiers with  $f \in \mathbb{H}$ . To ensure that the smoothed classifier is robust to the entire treat model we have to guarantee  $\min_{\tilde{\mathbf{X}} \in \mathcal{B}(\mathbf{X})} p_{\tilde{\mathbf{X}}, y^*} > 0.5$ . One can also derive tighter multi-class certificates, for which we refer to Appendix D for conciseness.
Probabilistic certificates. Since computing  $p_{\mathbf{X},y}$  exactly is challenging in practice, one estimates it using Monte-Carlo samples from  $\mu_{\mathbf{X}}$  and bounds  $p_{\mathbf{X},y}$  with confidence intervals (Cohen et al., 2019). The final certificates are probabilistic and hold with an (arbitrarily high) confidence level of  $1 - \alpha$ .
3.2 Neyman-Pearson Lemma: The foundation of randomized smoothing certificates
So far we introduced the idea of robustness certification using randomized smoothing, but to compute certificates one still has to solve Equation 1. To this end, Cohen et al. (2019) show that the minimum of Equation 1 can be derived using the Neyman-Pearson ("NP") Lemma (Neyman and Pearson, 1933). Intuitively, the least robust model classifies those  $\mathbf{W}$  as  $y$  for which the probability of sampling  $\mathbf{W}$  around the clean  $\mathbf{X}$  is high, but low around the perturbed  $\tilde{\mathbf{X}}$ :
Lemma 1 (Neyman-Pearson lower bound). Given  $\mathbf{X}, \tilde{\mathbf{X}} \in \mathcal{X}^{N \times D}$ , distributions  $\mu_{\mathbf{X}}, \mu_{\tilde{\mathbf{X}}}$ , class label  $y \in \mathcal{Y}$ , probability  $p_{\mathbf{X},y}$  and the set  $S_{\kappa} \triangleq \left\{\mathbf{W} \in \mathcal{X}^{N \times D}: \mu_{\tilde{\mathbf{X}}}(\mathbf{W}) \leq \kappa \cdot \mu_{\mathbf{X}}(\mathbf{W})\right\}$ , we have
$$
\underline {{p _ {\tilde {\boldsymbol {X}} , y}}} = \underset {\boldsymbol {W} \sim \mu_ {\tilde {\boldsymbol {X}}}} {\operatorname * {P r}} \left[ \boldsymbol {W} \in S _ {\kappa} \right] \quad \text {w i t h} \kappa \in \mathbb {R} _ {+} \text {s . t .} \quad \underset {\boldsymbol {W} \sim \mu_ {\boldsymbol {X}}} {\operatorname * {P r}} \left[ \boldsymbol {W} \in S _ {\kappa} \right] = p _ {\boldsymbol {X}, y}
$$
Proof. See (Neyman and Pearson, 1933) and (Cohen et al., 2019).
3.3 Existing smoothing distributions for discrete and continuous data
Our hierarchical smoothing framework builds upon certificates for smoothing distributions that apply noise independently per dimension (Lecuyer et al., 2019; Cohen et al., 2019; Lee et al., 2019). Despite being proposed for vector data  $\boldsymbol{x} \in \mathcal{X}^D$ , these certificates have been generalized to matrix data by applying noise independently on all matrix entries (Cohen et al., 2019; Bojchevski et al., 2020; Chu et al., 2022; Schuchardt and Gunnemann, 2022). Thus, given matrix data we define the density  $\mu_{\boldsymbol{X}}(\boldsymbol{W}) \triangleq \prod_{i=1}^{N} \prod_{j=1}^{D} \mu_{\boldsymbol{X}_{ij}}(\boldsymbol{W}_{ij})$  given the density of an existing smoothing distribution  $\mu$ .<sup>3</sup>
Gaussian randomized smoothing. For continuous data  $\mathcal{X} = \mathbb{R}$ , Cohen et al. (2019) derive the first tight certificate for the  $\ell_2$ -threat model. Given an isotropic Gaussian smoothing distribution  $\mu_{\boldsymbol{x}}(\boldsymbol{w}) = \mathcal{N}(\boldsymbol{w}|\boldsymbol{x},\sigma^2\boldsymbol{I})$ , Cohen et al. (2019) show that the optimal value of Equation 1 is given by  $\underline{p_{\tilde{\boldsymbol{x}},y}} = \Phi \left(\Phi^{-1}(p_{\boldsymbol{x},y}) - \frac{||\delta||_2}{\sigma}\right)$ , where  $\Phi$  denotes the CDF of the normal distribution and  $\delta \triangleq \boldsymbol{x} - \tilde{\boldsymbol{x}}$ .
Randomized smoothing for discrete data. Lee et al. (2019) derive tight certificates for the  $\ell_0$ -threat model. Certificates for discrete domains are in general combinatorial problems and computationally challenging. To overcome this, Lee et al. (2019) use the NP-Lemma for discrete random variables (Tocher, 1950) and show that the minimum in Equation 1 can be computed with a linear program (LP):
Lemma 2 (Discrete Neyman-Pearson lower bound). Partition the input space  $\mathcal{X}^D$  into disjoint regions  $\mathcal{R}_1, \ldots, \mathcal{R}_I$  of constant likelihood ratio  $\mu_{\boldsymbol{x}}(\boldsymbol{w}) / \mu_{\tilde{\boldsymbol{x}}}(\boldsymbol{w}) = c_i \in \mathbb{R} \cup \{\infty\}$  for all  $\boldsymbol{w} \in \mathcal{R}_i$ . Define  $\tilde{\boldsymbol{r}}_i \triangleq \operatorname*{Pr}_{\boldsymbol{w} \sim \mu_{\tilde{\boldsymbol{x}}}}(\boldsymbol{w} \in \mathcal{R}_i)$  and  $\boldsymbol{r}_i \triangleq \operatorname*{Pr}_{\boldsymbol{w} \sim \mu_{\boldsymbol{x}}}(\boldsymbol{w} \in \mathcal{R}_i)$ . Then Equation 1 is equivalent to the linear program  $\underline{p_{\tilde{\boldsymbol{x}},y}} = \min_{\boldsymbol{h} \in [0,1]^l} \boldsymbol{h}^T \tilde{\boldsymbol{r}}$  s.t.  $\boldsymbol{h}^T \boldsymbol{r} = p_{\boldsymbol{x},y}$ , where  $\boldsymbol{h}$  represents the worst-case classifier.
Proof in (Tocher, 1950). The LP can be solved efficiently using a greedy algorithm that consumes regions with larger likelihood ratio first (Lee et al., 2019). If the number of regions is small and  $\boldsymbol{r}_i$ ,  $\tilde{\boldsymbol{r}}_i$  can be computed efficiently, robustness certificates for discrete data become computationally feasible.
<sup>3</sup>We write  $\mu(Z)$  when referring to the density of the distribution  $\mu$ .
Figure 2: We derive flexible and efficient robustness certificates for hierarchical randomized smoothing by certifying classifiers on a higher-dimensional space where the indicator  $\tau$  is added to the data.
4 Hierarchical smoothing distribution
We aim to guarantee robustness against adversaries that perturb a subset of rows of a given matrix  $\mathbf{X}$ . To achieve this we introduce hierarchical randomized smoothing: We partially smooth matrices by adding random noise on a randomly selected subset of their rows only. By adding noise in a more targeted manner than existing methods we will obtain stronger robustness while maintaining high accuracy. We describe sampling from the hierarchical (mixture) distribution  $\Psi_{\mathbf{X}}$  as follows:
First, we sample an indicator vector  $\pmb{\tau} \in \{0,1\}^{N}$  from an upper-level smoothing distribution  $\phi(\pmb{\tau})$  that indicates which rows to smooth. Specifically, we draw each element of  $\pmb{\tau}$  independently from the same Bernoulli distribution  $\pmb{\tau}_i \sim \mathrm{Ber}(p)$ , where  $p$  denotes the probability for smoothing rows. Second, we sample a matrix  $\pmb{W}$  by using a lower-level smoothing distribution  $\mu$  (depending on domain and threat model) to add noise on the elements of the selected rows only. We define the overall density of this hierarchical smoothing distribution as  $\Psi_{\mathbf{X}}(\mathbf{W},\pmb{\tau}) \triangleq \mu_{\mathbf{X}}(\mathbf{W}|\pmb{\tau})\phi(\pmb{\tau})$  with:
$$
\mu_ {\boldsymbol {X}} (\boldsymbol {W} | \boldsymbol {\tau}) \triangleq \prod_ {i = 1} ^ {N} \mu_ {\mathbf {x} _ {i}} (\mathbf {w} _ {i} | \boldsymbol {\tau} _ {i}) \qquad \text {f o r} \qquad \mu_ {\boldsymbol {x} _ {i}} (\mathbf {w} _ {i} | \boldsymbol {\tau} _ {i}) \triangleq \left\{ \begin{array}{l l} \mu_ {\boldsymbol {x} _ {i}} (\mathbf {w} _ {i}) & \text {i f} \boldsymbol {\tau} _ {i} = 1 \\ \delta (\boldsymbol {w} _ {i} - \boldsymbol {x} _ {i}) & \text {i f} \boldsymbol {\tau} _ {i} = 0 \end{array} \right.
$$
where  $N$  is the number of rows and  $\delta$  the Dirac delta. In the case of a discrete lower-level smoothing distribution,  $\Psi_{\mathbf{X}}$  is a probability mass function and  $\delta$  denotes the Kronecker delta  $\delta_{\pmb{w}_i,\pmb{x}_i}$ .
5 Provable robustness certificates for hierarchical randomized smoothing
We develop certificates by deriving a condition that guarantees  $g(\mathbf{X}) = g(\tilde{\mathbf{X}})$  for clean  $\mathbf{X}$  and any perturbed  $\tilde{\mathbf{X}} \in \mathcal{B}_{p,\epsilon}^{r}(\mathbf{X})$ . Certifying robustness under hierarchical smoothing is challenging, especially if we want to make use of certificates for the lower-level smoothing distribution  $\mu$  without deriving the entire certificate from scratch. Moreover, Lemma 1 can be intractable in discrete domains where certification involves combinatorial problems. We are interested in certificates that are (1) flexible enough to make use of existing smoothing-based certificates, and (2) efficient to compute.
Our main idea to achieve both - flexible and efficient certification under hierarchical smoothing - is to embed the data into a higher-dimensional space  $W \hookrightarrow \mathcal{Z}^{N \times (D + 1)}$  by appending the indicator  $\tau$  as an additional column:  $Z \triangleq W|\tau$  ("l" denotes concatenation, see Figure 2). We construct a new base classifier that operates on this higher-dimensional space, e.g. by ignoring  $\tau$  and applying the original base classifier to  $W$ . In our experiments we also train the classifiers directly on the extended data.