Improving Robustness using Generated Data

1 Introduction . Neural networks are being deployed in a wide variety of applications ranging from ranking content on the web [ 15 ] to autonomous driving [ 5 ] via medical diagnostics [ 22 ] . It has become increasingly important to ensure that deployed models are robust and generalize to various input perturbations . Unfortunately , the addition of imperceptible adversarial perturbations can cause neural networks to make incorrect predictions [ 9 , 10 , 27 , 44 , 64 ] . There has been a lot of work on understanding and generating adversarial perturbations [ 1 , 4 , 10 , 64 ] , and on building defenses that are robust to such perturbations [ 27 , 49 , 60 , 82 ] . We note that while robustness and invariance to input perturbations is crucial to the deployment of machine learning models in various applications , it can also have broader negative impacts to society such as hindering privacy [ 63 ] or increasing bias [ 68 ] . The adversarial training procedure proposed by Madry et al . [ 49 ] feeds adversarially perturbed examples back into the training data . It is widely regarded as one of the most successful method to train robust deep neural networks [ 30 ] , and it has been augmented in different ways – with changes in the attack procedure [ 25 ] , loss function [ 50 , 82 ] or model architecture [ 76 , 85 ] . We highlight the works by Carmon et al . [ 11 ] , Najafi et al . [ 51 ] , Uesato et al . [ 72 ] , Zhai et al . [ 80 ] who simultaneously proposed the use of additional unlabeled external data . While the addition of external data helped boost robust accuracy by a large margin , progress in the setting without additional data has slowed 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . ( see Fig . 1 ) . On CIFAR-10 [ 42 ] against ` ∞ perturbations of size = 8/255 , the best known model obtains a robust accuracy of 65.87 % when using additional data . The same model obtains a robust accuracy of 57.14 % without this data [ 30 ] . As a result , we ask ourselves whether it is possible to leverage the information contained in the original training set to a greater extent . This manuscript challenges the status-quo . To the contrary of standard training where it is widely believed that generative models lack diversity and that the samples they produce can not be used to train better classifiers [ 59 ] , we demonstrate both theoretically and experimentally that these generated samples can be used to improve robustness ( using the approach described in Fig . 2 and Sec . 3.3 ) . We make the following contributions : • We demonstrate in Sec . 3.2 that it is possible to use low-quality random inputs ( sampled from a conditional Gaussian fit of the training data ) to improve robust accuracy on CIFAR-10 against ` ∞ perturbations of size = 8/255 ( +0.93 % on a WRN-28-10 ) and provide a justification and sufficient conditions in Sec . 4 . • We leverage higher quality generated inputs ( i.e. , inputs generated by generative models solely trained on the original data ) , and study four recent generative models : the Denoising Diffusion Probabilistic Model ( DDPM ) [ 36 ] , StyleGAN2 [ 40 ] , BigGAN [ 7 ] and the Very Deep Variational Auto-Encoder ( VDVAE ) [ 14 ] ( Sec . 5 ) . We show that DDPM samples cover most closely the real data distribution ( as measured by the distance to the test set in the Inception feature space ) . • Using images generated by the DDPM allows us to reach a robust accuracy of 66.10 % on CIFAR-10 against ` ∞ perturbations of size = 8/255 ( an improvement of +8.96 % upon the stateof-the-art ) . Notably , our best CIFAR-10 models beat all techniques that use additional data ( see Sec . 6 ) and constitutes one of the largest improvements ever made in the setting without additional data . As a consequence , we demonstrate that it is possible to avoid the use of 80M-TI [ 65 ] which has been withdrawn due to presence of offensive images.1 2 Related work . Adversarial ` p norm-bounded attacks . Since Biggio et al . [ 4 ] , Szegedy et al . [ 64 ] observed that neural networks which achieve high accuracy are highly vulnerable to adversarial examples , the art of crafting increasingly sophisticated adversarial examples has received a lot of attention . Goodfellow et al . [ 27 ] proposed the Fast Gradient Sign Method ( FGSM ) which generates adversarial examples 1https : //groups.csail.mit.edu/vision/TinyImages/ with a single normalized gradient step . It was followed by R+FGSM [ 67 ] , which adds a randomization step , and the Basic Iterative Method ( BIM ) [ 44 ] , which takes multiple smaller gradient steps . Adversarial training as a defense . Adversarial training [ 49 ] is widely regarded as one of the most successful methods to train deep neural networks robust to such attacks . It has received significant attention and various modifications have emerged [ 25 , 50 , 76 ] . A notable work is TRADES [ 82 ] , which balances the trade-off between standard and robust accuracy , and achieved state-of-the-art performance against ` ∞ norm-bounded perturbations on CIFAR-10 . More recently , the work from Rice et al . [ 60 ] studied robust overfitting and demonstrated that improvements similar to TRADES could be obtained more easily using classical adversarial training with early stopping . Finally , Gowal et al . [ 30 ] highlighted how different hyper-parameters ( such as network size and model weight averaging ) affect robustness . Data-driven augmentations . Works , such as AutoAugment [ 18 ] and related RandAugment [ 19 ] , learn augmentation policies directly from data . These methods are tuned to improve standard classification accuracy and have been shown to work well on multiple datasets . DeepAugment [ 34 ] explores how perturbations of the parameters of pre-trained image-to-image models can be used to generate augmented datasets that provide increased robustness to common corruptions [ 32 ] . Similarly , generative models can be used to create novel views of images [ 37 , 39 , 57 ] by manipulating them in latent space . When optimized and used during training , these novel views reduce the impact of spurious correlations and improve accuracy [ 28 , 73 ] . Most recently , Laidlaw et al . [ 45 ] proposed an adversarial training method based on bounding a neural perceptual distance ( i.e. , an approximation of the true perceptual distance ) . While these works make significant contributions towards improving generalization and robustness to semantic perturbations , they do not improve robustness to ` p normbounded perturbations . Robustness to ` p norm-bounded perturbations using generative modeling . Finally , we highlight works , such as Defense-GAN [ 61 ] or ME-Net [ 77 ] , which leverage data modeling techniques to create stronger defenses against ` p norm-bounded attacks . Unfortunately , these techniques are not as robust as they seem and are broken by adaptive attacks [ 2 , 16 , 66 ] . Overall , to the best of our knowledge , there is little [ 48 ] to no evidence that data augmentations or generative models can be used to improve robustness to ` p norm-bounded attacks . In fact , generative models mostly lack diversity and it is widely believed that the samples they produce can not be used to train classifiers to the same accuracy than those trained on original datasets [ 59 ] . We differentiate ourselves from earlier works by leveraging additional generated samples for training rather than modifying the defense procedure , and by establishing sufficient conditions under which such samples improve robustness . 3 Adversarial training using generated data . The rest of this manuscript is organized as follows . In this section , we provide an overview of adversarial training , demonstrate using a motivational example that low-quality generated data can be leveraged to improve robustness to adversarial examples , and describe our method . In Sec . 4 , we detail sufficient conditions that explain why generated samples can improve robustness and explore the limitations of our approach . In Sec . 5 , we analyze four complementary and recent generative models in the context of our method . Finally , we provide experimental results in Sec . 6 . 3.1 Adversarial training . For classification tasks , Madry et al . [ 49 ] propose to find model parameters θ that minimize the adversarial risk : argmin θ E ( x , y ) ∼D ( max δ∈S [ f ( x+ δ ; θ ) 6= y ] ) ( 1 ) where D is a data distribution over pairs of examples x and corresponding labels y , f ( · ; θ ) is a model parametrized by θ , [ · ] is the Iverson bracket notation and corresponds to the 0− 1 loss , and S defines the set of allowed perturbations . For ` p norm-bounded perturbations of size , the perturbation set is defined as Sp = { δ | ‖δ‖p ≤ } . Hence , for ` ∞ norm-bounded perturbations S = S∞ and for ` 2 norm-bounded perturbations S = S2 . In the rest of this manuscript , we use p to denote ` p norm-bounded perturbations of size ( e.g. , ∞ = 8/255 ) . In practice , given a training set Dtrain , the adversarial training procedure replaces the 0− 1 loss with the cross-entropy loss lce and is formulated as argmin θ E ( x , y ) ∈Dtrain ( max δ∈S lce ( f ( x+ δ ; θ ) , y ) ) . ( 2 ) 3.2 Generated data can improve robust generalization . Data augmentations can reduce the generalization error of standard ( non-robust ) training [ 18 , 19 , 23 , 81 ] . However , to the contrary of standard training , augmentations beyond random flips and crops [ 31 ] – such as Cutout [ 23 ] , mixup [ 81 ] , AutoAugment [ 18 ] or RandAugment [ 19 ] – have been unsuccessful in the context of adversarial training [ 30 , 60 , 75 ] . The gap in robust accuracy between models trained with and without additional data suggests that common augmentation techniques , which tend to produce augmented views that are close to the original image they augment , are intrinsically limited in their ability to improve robust generalization . In other words , augmented samples are diverse ( if the training set is diverse ) , but not complementary to the training set . This phenomenon is particularly exacerbated when training adversarially robust models which are known to require an amount of data polynomial in the number of input dimensions [ 62 ] . We hypothesize that , to improve robust generalization , it is critical to use additional training samples ( augmented or generated ) that are diverse and that complement the original training set ( in the sense that these new samples should ideally come from the same underlying distribution as the training set but should not duplicate the training set ) . To test this hypothesis , we propose to use samples generated from a simple class-conditional Gaussian fit of the training data . By construction , such samples ( shown in Fig . 3 ( b ) ) are extremely blurry but diverse . We proceed by fitting a multivariate Gaussian to each set of 5K training images corresponding to each class in CIFAR-10 . For each class , we sample 100K images resulting in a new dataset of 1M datapoints ( no further filtering is applied ) . In Fig . 3 ( a ) , we show the performance of various robust models trained by decreasing the proportion of real samples present in each batch from 100 % ( original data only ) to 0 % ( generated data only ) . Decreasing this proportion reduces the importance of the original data . We observe that all proportions between 50 % and 90 % provide improvements in robust accuracy . Most surprisingly , the optimal proportion of 80 % provides an absolute improvement of +0.93 % , which is an improvement comparable in size to the ones provided by model weight averaging or TRADES [ 30 ] . As we show in Sec . 4 , the drop in robust accuracy for proportions below 50 % is expected in the capacity-limited regime . This experiment directly motivates our method .

Summary: This paper shows experimentally that training on generated data (from a generative model not trained on external data) can improve adversarial robustness.  Surprisingly, even a class-conditional Gaussian generative model can improve adversarial robustness. The paper additionally provides sufficient conditions for generated data to help adversarial robustness.