Center Smoothing: Certified Robustness for Networks with Structured Outputs

The study of provable adversarial robustness has mostly been limited to classification tasks and models with one-dimensional real-valued outputs . We extend the scope of certifiable robustness to problems with more general and structured outputs like sets , images , language , etc . We model the output space as a metric space under a distance/similarity function , such as intersection-over-union , perceptual similarity , total variation distance , etc . Such models are used in many machine learning problems like image segmentation , object detection , generative models , image/audio-to-text systems , etc . Based on a robustness technique called randomized smoothing , our center smoothing procedure can produce models with the guarantee that the change in the output , as measured by the distance metric , remains small for any norm-bounded adversarial perturbation of the input . We apply our method to create certifiably robust models with disparate output spaces – from sets to images – and show that it yields meaningful certificates without significantly degrading the performance of the base model . 1 Introduction . The study of adversarial robustness in machine learning ( ML ) has gained a lot of attention ever since deep neural networks ( DNNs ) have been demonstrated to be vulnerable to adversarial attacks . These attacks are generated by making tiny perturbations of the input that can completely alter a model ’ s predictions [ 56 , 46 , 23 , 35 ] . They can significantly degrade the performance of a model , like an image classifier , and make it output almost any class of the attacker ’ s choice . However , these attacks are not limited just to classification problems . They have also been shown to exist for DNNs with structured outputs like text , images , probability distributions , sets , etc . For instance , automatic speech recognition systems can be attacked with 100 % success rate to output any phrase of the attackers choice [ 10 ] . Similar attacks can cause neural image captioning systems to produce specific target captions with high success-rate [ 11 ] . Quality of image segmentation models have been shown to degrade severely under adversarial attacks [ 2 , 27 , 30 ] . Facial recognition systems can be deceived to evade detection , impersonate authorized individuals and even render them completely ineffective [ 59 , 55 , 20 ] . Image reconstruction models have been targeted to introduce unwanted artefacts or miss important details , such as tumors in MRI scans , through adversarial inputs [ 1 , 50 , 8 , 12 ] . Super-resolution systems can be made to generate distorted images that can in turn deteriorate the performance of subsequent tasks that rely on the high-resolution outputs [ 14 , 63 ] . Deep neural network based policies in reinforcement learning problems also have been shown to succumb to imperceptible perturbations in the state observations [ 21 , 29 , 4 , 48 ] . Such widespread presence of adversarial attacks is concerning as it threatens the use of deep neural networks in critical systems , such as facial recognition , self-driving vehicles , medical diagnosis , etc. , where safety , security and reliability are of utmost importance . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Adversarial defenses have mostly focused on classification tasks [ 34 , 6 , 26 , 17 , 44 , 25 , 22 ] . Certified defenses based on convex-relaxation [ 61 , 49 , 53 , 13 , 54 ] , interval-bound propagation [ 24 , 28 , 18 , 47 ] and randomized smoothing [ 15 , 36 , 42 , 51 ] that guarantee that the predicted class will remain the same in a certified region around the input point have also been studied . Compared to empirical robustness methods that are often shown to be broken by stronger attacks [ 9 , 3 , 58 ] , procedures with provable robustness guarantees are of special importance to the study of robustness in ML as their guarantees hold regardless of improvements in attack strategies . Among these approaches , certified defenses based on randomized smoothing have been show to scale up to high-dimensional inputs , such as images , and does not need to make assumptions about the underlying model . The robustness certificates produced by these defenses are probabilistic , meaning that they hold with high probability and not absolute certainty . Unlike classification problems , where certificates guarantee that the predicted class remains unchanged under bounded-size perturbations , it is not immediately obvious what the goal of robustness should be for problems with structured outputs like images , text , sets , etc . While accuracy is the standard quality measure for classification , more complex tasks may require other quality metrics like total variation for images , intersection over union for object localization , earth-mover distance for distributions , etc . In general , neural networks can be cast as functions of the type f : Rk → ( M , d ) which map a k dimensional real-valued space into a metric space M with distance function d : M ×M → R≥0 . In this work , we design a randomized smoothing based technique to obtain provable robustness for functions of this type with minimal assumptions on the distance metric d. We generate a robust version f̄ such that the change in its output , as measured by d , is small for a small change in its input . More formally , given an input x and an ` 2-perturbation size 1 , we produce a value 2 with the guarantee that , with high probability , ∀x′ s.t . ‖x− x′‖2 ≤ 1 , d ( f̄ ( x ) , f̄ ( x′ ) ) ≤ 2 . Our contributions : We develop center smoothing , a procedure to make functions like f provably robust against adversarial attacks . For a given input x , center smoothing samples a collection of points in the neighborhood of x using a Gaussian smoothing distribution , computes the function f on each of these points and returns the center of the smallest ball enclosing at least half the points in the output space ( see figure 1 ) . Computing the minimum enclosing ball in the output space is equivalent to solving the 1-center problem with outliers ( hence the name of our procedure ) , which is an NP-complete problem for a general metric [ 52 ] . We approximate it by computing the point that has the smallest median distance to all the other points in the sample . We show that the output of the smoothed function is robust to input perturbations of bounded ` 2-size . We restrict the input perturbations to be inside an ` 2-ball as the main focus of this work is on the output space of f . However , our method does not critically rely on the ` 2 threat model or Gaussian smoothing noise , and can be adapted to other perturbations types and smoothing distributions . Although we define the output space as a metric , our proofs only require the symmetry property and triangle inequality to hold . Thus , center smoothing can also be applied to pseudometric distances that need not satisfy the identity of indiscernibles . Many distances defined for images , such as total variation , cosine distance , perceptual distances , etc. , fall under this category . Center smoothing steps outside the world of ` p metrics , and certifies robustness in metrics like IoU/Jaccard distance for object localization , and total-variation , which is a good measure of perceptual similarity for images . In our experiments , we show that this method can produce meaningful certificates for a wide variety of output metrics without significantly compromising the quality of the base model . Related Work : Randomized smoothing has been extensively used for provable adversarial robustness in the classification setting to defend against different ` p [ 15 , 36 , 51 , 57 , 43 , 41 , 37 , 40 ] and non- ` p [ 38 , 39 ] threat models . Beyond classification , it has also been used for certifying the median output of regression models [ 62 ] and the expected softmax scores of neural networks [ 33 ] . Smoothing a bounded vector-valued function by taking the mean of the output vectors has been shown to have a bounded Lipschitz constant when both input and output spaces are ` 2-metrics [ 60 ] . Center smoothing does not require the base function to be bounded because the minimum enclosing ball is resistant to outliers . Moving an outlier point away from this ball does not affect the output of the smoothed function . On the other hand , smoothing techniques that compute the mean of the output samples are more susceptible to outliers as changing any of the samples can alter the mean . Recently , a provable defense for segmentation tasks was developed by certifying each individual pixel of the output using randomized smoothing [ 19 ] . Due to the accumulating uncertainty over individual certifications , it is difficult to produce guarantees for large images , often leading to certified outputs with ambiguous pixels . Center smoothing bypasses this challenge by directly certifying the similarity between a clean segmentation output and an adversarial one under a metric such as intersection over union . 2 Preliminaries and Notations . Given a function f : Rk → ( M , d ) and a distribution D over the input space Rk , let f ( D ) denote the probability distribution of the output of f in M when the input is drawn from D. For a point x ∈ Rk , let x + P denote the probability distribution of the points x + δ where δ is a smoothing noise drawn from a distribution P over Rk and let X be the random variable for x+P . For elements in M , define B ( z , r ) = { z′ | d ( z , z′ ) ≤ r } as a ball of radius r centered at z . Define a smoothed version of f under P as the center of the ball with the smallest radius in M that encloses at least half of the probability mass of f ( x+ P ) , i.e. , f̄P ( x ) = argmin z r s.t . P [ f ( X ) ∈ B ( z , r ) ] ≥ 1 2 . If there are multiple balls with the smallest radius satisfying the above condition , return one of the centers arbitrarily . Let r∗P ( x ) be the value of the minimum radius . Hereafter , we ignore the subscripts and superscripts in the above definitions whenever they are obvious from context . In this work , we sample the noise vector δ from an i.i.d Gaussian distribution of variance σ2 in each dimension , i.e. , δ ∼ N ( 0 , σ2I ) . 2.1 Gaussian Smoothing . Cohen et al . in 2019 showed that a classifier h : Rk → Y smoothed with a Gaussian noise N ( 0 , σ2I ) as , h̄ ( x ) = argmax c∈Y P [ h ( x+ δ ) = c ] , where Y is a set of classes , is certifiably robust to small perturbations in the input . Their certificate relied on the fact that , if the probability of sampling from the top class at x under the smoothing distribution is p , then for an ` 2 perturbation of size at most , the probability of the top class is guaranteed to be at least p = Φ ( Φ −1 ( p ) − /σ ) , ( 1 ) where Φ is the CDF of the standard normal distribution N ( 0 , 1 ) . This bound applies to any { 0 , 1 } -function over the input space Rk , i.e. , if P [ h ( x ) = 1 ] = p , then for any -size perturbation x′ , P [ h ( x′ ) = 1 ] ≥ p . We use this bound to generate robustness certificates for center smoothing . We identify a ball B ( f̄ ( x ) , R ) of radius R enclosing a very high probability mass of the output distribution . One can define a function that outputs one if f maps a point to inside B ( f̄ ( x ) , R ) and zero otherwise . The bound in ( 1 ) gives us a region in the input space such that for any point inside it , at least half of the mass of the output distribution is enclosed in B ( f̄ ( x ) , R ) . We show in section 3 that the output of the smoothed function for a perturbed input is guaranteed to be within a constant factor of R from the output of the original input .

Summary: This paper proposed a smoothing technique to perform adversarial training, such that the trained model is robust to small $l_2$ perturbations in the input. The major contribution is that the proposed method is applicable to a wide range of distance metrics. The idea of computing the minimum enclosing ball is also a very interesting idea, and I believe similar idea can be used to smoothing based optimization.