Center Smoothing: Certified Robustness for Networks with Structured Outputs

The study of provable adversarial robustness has mostly been limited to classification tasks and models with one-dimensional real-valued outputs . We extend the scope of certifiable robustness to problems with more general and structured outputs like sets , images , language , etc . We model the output space as a metric space under a distance/similarity function , such as intersection-over-union , perceptual similarity , total variation distance , etc . Such models are used in many machine learning problems like image segmentation , object detection , generative models , image/audio-to-text systems , etc . Based on a robustness technique called randomized smoothing , our center smoothing procedure can produce models with the guarantee that the change in the output , as measured by the distance metric , remains small for any norm-bounded adversarial perturbation of the input . We apply our method to create certifiably robust models with disparate output spaces – from sets to images – and show that it yields meaningful certificates without significantly degrading the performance of the base model . 1 Introduction . The study of adversarial robustness in machine learning ( ML ) has gained a lot of attention ever since deep neural networks ( DNNs ) have been demonstrated to be vulnerable to adversarial attacks . These attacks are generated by making tiny perturbations of the input that can completely alter a model ’ s predictions [ 56 , 46 , 23 , 35 ] . They can significantly degrade the performance of a model , like an image classifier , and make it output almost any class of the attacker ’ s choice . However , these attacks are not limited just to classification problems . They have also been shown to exist for DNNs with structured outputs like text , images , probability distributions , sets , etc . For instance , automatic speech recognition systems can be attacked with 100 % success rate to output any phrase of the attackers choice [ 10 ] . Similar attacks can cause neural image captioning systems to produce specific target captions with high success-rate [ 11 ] . Quality of image segmentation models have been shown to degrade severely under adversarial attacks [ 2 , 27 , 30 ] . Facial recognition systems can be deceived to evade detection , impersonate authorized individuals and even render them completely ineffective [ 59 , 55 , 20 ] . Image reconstruction models have been targeted to introduce unwanted artefacts or miss important details , such as tumors in MRI scans , through adversarial inputs [ 1 , 50 , 8 , 12 ] . Super-resolution systems can be made to generate distorted images that can in turn deteriorate the performance of subsequent tasks that rely on the high-resolution outputs [ 14 , 63 ] . Deep neural network based policies in reinforcement learning problems also have been shown to succumb to imperceptible perturbations in the state observations [ 21 , 29 , 4 , 48 ] . Such widespread presence of adversarial attacks is concerning as it threatens the use of deep neural networks in critical systems , such as facial recognition , self-driving vehicles , medical diagnosis , etc. , where safety , security and reliability are of utmost importance . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Adversarial defenses have mostly focused on classification tasks [ 34 , 6 , 26 , 17 , 44 , 25 , 22 ] . Certified defenses based on convex-relaxation [ 61 , 49 , 53 , 13 , 54 ] , interval-bound propagation [ 24 , 28 , 18 , 47 ] and randomized smoothing [ 15 , 36 , 42 , 51 ] that guarantee that the predicted class will remain the same in a certified region around the input point have also been studied . Compared to empirical robustness methods that are often shown to be broken by stronger attacks [ 9 , 3 , 58 ] , procedures with provable robustness guarantees are of special importance to the study of robustness in ML as their guarantees hold regardless of improvements in attack strategies . Among these approaches , certified defenses based on randomized smoothing have been show to scale up to high-dimensional inputs , such as images , and does not need to make assumptions about the underlying model . The robustness certificates produced by these defenses are probabilistic , meaning that they hold with high probability and not absolute certainty . Unlike classification problems , where certificates guarantee that the predicted class remains unchanged under bounded-size perturbations , it is not immediately obvious what the goal of robustness should be for problems with structured outputs like images , text , sets , etc . While accuracy is the standard quality measure for classification , more complex tasks may require other quality metrics like total variation for images , intersection over union for object localization , earth-mover distance for distributions , etc . In general , neural networks can be cast as functions of the type f : Rk → ( M , d ) which map a k dimensional real-valued space into a metric space M with distance function d : M ×M → R≥0 . In this work , we design a randomized smoothing based technique to obtain provable robustness for functions of this type with minimal assumptions on the distance metric d. We generate a robust version f̄ such that the change in its output , as measured by d , is small for a small change in its input . More formally , given an input x and an ` 2-perturbation size 1 , we produce a value 2 with the guarantee that , with high probability , ∀x′ s.t . ‖x− x′‖2 ≤ 1 , d ( f̄ ( x ) , f̄ ( x′ ) ) ≤ 2 . Our contributions : We develop center smoothing , a procedure to make functions like f provably robust against adversarial attacks . For a given input x , center smoothing samples a collection of points in the neighborhood of x using a Gaussian smoothing distribution , computes the function f on each of these points and returns the center of the smallest ball enclosing at least half the points in the output space ( see figure 1 ) . Computing the minimum enclosing ball in the output space is equivalent to solving the 1-center problem with outliers ( hence the name of our procedure ) , which is an NP-complete problem for a general metric [ 52 ] . We approximate it by computing the point that has the smallest median distance to all the other points in the sample . We show that the output of the smoothed function is robust to input perturbations of bounded ` 2-size . We restrict the input perturbations to be inside an ` 2-ball as the main focus of this work is on the output space of f . However , our method does not critically rely on the ` 2 threat model or Gaussian smoothing noise , and can be adapted to other perturbations types and smoothing distributions . Although we define the output space as a metric , our proofs only require the symmetry property and triangle inequality to hold . Thus , center smoothing can also be applied to pseudometric distances that need not satisfy the identity of indiscernibles . Many distances defined for images , such as total variation , cosine distance , perceptual distances , etc. , fall under this category . Center smoothing steps outside the world of ` p metrics , and certifies robustness in metrics like IoU/Jaccard distance for object localization , and total-variation , which is a good measure of perceptual similarity for images . In our experiments , we show that this method can produce meaningful certificates for a wide variety of output metrics without significantly compromising the quality of the base model . Related Work : Randomized smoothing has been extensively used for provable adversarial robustness in the classification setting to defend against different ` p [ 15 , 36 , 51 , 57 , 43 , 41 , 37 , 40 ] and non- ` p [ 38 , 39 ] threat models . Beyond classification , it has also been used for certifying the median output of regression models [ 62 ] and the expected softmax scores of neural networks [ 33 ] . Smoothing a bounded vector-valued function by taking the mean of the output vectors has been shown to have a bounded Lipschitz constant when both input and output spaces are ` 2-metrics [ 60 ] . Center smoothing does not require the base function to be bounded because the minimum enclosing ball is resistant to outliers . Moving an outlier point away from this ball does not affect the output of the smoothed function . On the other hand , smoothing techniques that compute the mean of the output samples are more susceptible to outliers as changing any of the samples can alter the mean . Recently , a provable defense for segmentation tasks was developed by certifying each individual pixel of the output using randomized smoothing [ 19 ] . Due to the accumulating uncertainty over individual certifications , it is difficult to produce guarantees for large images , often leading to certified outputs with ambiguous pixels . Center smoothing bypasses this challenge by directly certifying the similarity between a clean segmentation output and an adversarial one under a metric such as intersection over union . 2 Preliminaries and Notations . Given a function f : Rk → ( M , d ) and a distribution D over the input space Rk , let f ( D ) denote the probability distribution of the output of f in M when the input is drawn from D. For a point x ∈ Rk , let x + P denote the probability distribution of the points x + δ where δ is a smoothing noise drawn from a distribution P over Rk and let X be the random variable for x+P . For elements in M , define B ( z , r ) = { z′ | d ( z , z′ ) ≤ r } as a ball of radius r centered at z . Define a smoothed version of f under P as the center of the ball with the smallest radius in M that encloses at least half of the probability mass of f ( x+ P ) , i.e. , f̄P ( x ) = argmin z r s.t . P [ f ( X ) ∈ B ( z , r ) ] ≥ 1 2 . If there are multiple balls with the smallest radius satisfying the above condition , return one of the centers arbitrarily . Let r∗P ( x ) be the value of the minimum radius . Hereafter , we ignore the subscripts and superscripts in the above definitions whenever they are obvious from context . In this work , we sample the noise vector δ from an i.i.d Gaussian distribution of variance σ2 in each dimension , i.e. , δ ∼ N ( 0 , σ2I ) . 2.1 Gaussian Smoothing . Cohen et al . in 2019 showed that a classifier h : Rk → Y smoothed with a Gaussian noise N ( 0 , σ2I ) as , h̄ ( x ) = argmax c∈Y P [ h ( x+ δ ) = c ] , where Y is a set of classes , is certifiably robust to small perturbations in the input . Their certificate relied on the fact that , if the probability of sampling from the top class at x under the smoothing distribution is p , then for an ` 2 perturbation of size at most , the probability of the top class is guaranteed to be at least p = Φ ( Φ −1 ( p ) − /σ ) , ( 1 ) where Φ is the CDF of the standard normal distribution N ( 0 , 1 ) . This bound applies to any { 0 , 1 } -function over the input space Rk , i.e. , if P [ h ( x ) = 1 ] = p , then for any -size perturbation x′ , P [ h ( x′ ) = 1 ] ≥ p . We use this bound to generate robustness certificates for center smoothing . We identify a ball B ( f̄ ( x ) , R ) of radius R enclosing a very high probability mass of the output distribution . One can define a function that outputs one if f maps a point to inside B ( f̄ ( x ) , R ) and zero otherwise . The bound in ( 1 ) gives us a region in the input space such that for any point inside it , at least half of the mass of the output distribution is enclosed in B ( f̄ ( x ) , R ) . We show in section 3 that the output of the smoothed function for a perturbed input is guaranteed to be within a constant factor of R from the output of the original input .

Summary: The authors extends randomized smoothing, originally developed for classifiers, to functions which map inputs to a general metric space. The extension seems intuitive but is technically challenging. The authors develop tractable, statistical methods both for finding the smooth prediction as well as computing the certificates. Various experiments are conducted to highlight its flexibility in different metrics and scenarios, such as Jaccard distance w.r.t. face detection, total variation distance w.r.t. dimension reduction, and angular distance for image reconstruction.