RANDOM MASK: Towards Robust Convolutional Neural Networks

1 INTRODUCTION . Deep learning ( LeCun et al. , 2015 ) , especially deep Convolutional Neural Network ( CNN ) ( LeCun et al. , 1998 ) , has led to state-of-the-art results spanning many machine learning fields , such as image classification ( He et al. , 2016 ; Hu et al. , 2017b ; Huang et al. , 2017 ; Simonyan & Zisserman , 2014 ) , object detection ( Redmon et al. , 2016 ; Girshick , 2015 ; Ren et al. , 2015 ) , image captioning ( Vinyals et al. , 2015 ; Xu et al. , 2015 ) and speech recognition ( Bengio et al. , 2003 ; Hinton et al. , 2012 ) . Despite the great success in numerous applications , recent studies have found that deep CNNs are vulnerable to some well-designed input samples named as Adversarial Examples ( Szegedy et al. , 2013 ) ( Biggio et al. , 2013 ) . Take the task of image classification as an example , for almost every commonly used well-performed CNN , attackers are able to construct a small perturbation on an input image to cause the model to give an incorrect output label . Meanwhile , the perturbation is almost imperceptible to humans . Furthermore , these adversarial examples can easily transfer among different kinds of CNN architectures ( Papernot et al. , 2016b ) . Such adversarial examples raise serious concerns on deep neural network models as robustness is crucial in many applications . Just as Goodfellow ( 2018 ) suggests , both robustness and traditional supervised learning seem fully aligned . Recently , there is a rapidly growing body of work on this topic . One important line of research is adversarial training ( Szegedy et al. , 2013 ; Madry et al. , 2017 ; Goodfellow et al. , 2015 ; Huang et al. , 2015 ) . Although adversarial training gains some success , a major difficulty is that it tends to overfit to the method of adversarial example generation used at training time ( Buckman et al. , 2018 ) . Xie et al . ( 2017 ) and Guo et al . ( 2017 ) propose defense methods by introducing randomness and applying transformations to the inputs respectively . Dhillon et al . ( 2018 ) introduces random drop during the evaluation of a neural network . However , Athalye et al . ( 2018 ) contends that such transformation and randomness only provide a kind of “ obfuscated gradient ” and can be attacked by taking expectation over transformation ( EOT ) to ∗Equal contribution . get a meaningful gradient . Papernot et al . ( 2016a ) and Katz et al . ( 2017 ) consider the non-linear functions in the networks and try to achieve robustness by adjusting them . There are also detection-based defense methods ( Metzen et al. , 2017 ; Grosse et al. , 2017 ; Meng & Chen , 2017 ) , which add a process of detecting whether an input is adversarial . In this paper , different from most of the existing methods , we take another approach to tackle the adversarial example problem . In particular , we aim to design a new CNN architecture which by itself enjoys robustness , without appealing to techniques such as adversarial training . To this end , we introduce Random Mask as a new ingredient of CNN . To be specific , we randomly select a set of neurons and remove them from the network before training . Then the architecture of the network is fixed during the training and testing process . Note that Random Mask is different from dropout which randomly masks out neurons in each step during training . In addition , Random Mask can be applied very easily to common CNN structures such as ResNet with only a few changes of code . We find that applying Random Mask to the shallow layers of the network is crucial for robustness . CNNs with properly designed Random Mask are far more robust than standard CNNs . In fact , our experimental results demonstrate that CNNs with Random Mask achieve state-of-the-art results against black-box attacks even when comparing with defense methods using adversarial training . Furthermore , CNNs with Random Mask maintain a high accuracy on normal test data , while low test accuracy is often regarded as a major weakness in many methods designed for achieving robustness . We next take a closer look at the adversarial examples generated particularly against CNNs with Random Mask . We investigate the adversarial examples that can “ fool ” our proposed architecture , i.e. , the examples that are perturbed version of the original image , but are classified to a different label by the network . Surprisingly , we find that the adversarial examples which can “ fool ” a CNN with Random Mask often “ fool ” humans as well . It is difficult for humans to correctly classify these adversarial example , and in many cases humans make the same “ incorrect ” prediction as our network . Figure 1 shows a few adversarial examples generated by PGD ( Basic iterative method ) ( Kurakin et al. , 2016 ) with respect to a CNN with Random Mask as well as the labels the network outputs . ( Please also see Figure 12 in Appendix E.1 for the original images and labels from CIFAR-10 . ) They are different from the typical adversarial examples generated against commonly used CNNs , which usually look like noisy versions of the original images and are easy to be correctly classified by humans . These observations raise important questions : 1 ) How should we define adversarial examples ? 2 ) How should we define robustness ? Currently , an adversarial example is usually defined as a perturbed datum which lies in the neighborhood of the original data but has a different classification output by the network ; and the robustness of a method is measured according to the proportion of these adversarial examples . However , if an adversarial example can also fool humans , it is more appropriate to say that the example does change the semantics of the datum to a certain extent . After all , why two images close to each other in terms of some distance ( e.g. , ` ∞ ) must belong to the same class ? How close should they be so that they belong to the same class ? Without complete answers to these questions , one should be very careful when measuring the robustness of a model merely according to currently-defined adversarial examples . Robustness is a subtle issue . We argue that one needs to rethink the robustness and adversarial examples from the definitions . Our main contributions are summarized as follows : • We develop a very simple but effective method , Random Mask . We show that combining with Random Mask , existing CNNs can be significantly more robust while maintaining high generalization performance . In fact , CNNs equipped with Random Mask achieve state-of-the-art performance against several black-box attacks , even when comparing with methods using adversarial training ( See Table 1 ) . • We investigate the adversarial examples generated against CNNs with Random Mask . We find that adversarial examples that can “ fool ” a CNN with Random Mask often fool humans as well . This observation requires us to rethink what are the right definitions of adversarial examples and robustness . 2 RANDOM MASK . We propose Random Mask , a method to modify existing CNN structures . It randomly selects a set of neurons and removes them from the network before training . Then the architecture of the network is fixed during the training and testing process . To apply Random Mask on a selected layerLayer ( j ) , suppose the input isXj and the output is convj ( Xj ) ∈ Rmj×nj×cj . We randomly generate a binary mask mask ( j ) ∈ { 0 , 1 } mj×nj×cj by sampling uniformly within each channel . The drop rate of the sampling process is called the ratio ( or drop ratio ) of Random Mask . Then we mask the neurons in position ( x , y , c ) of the output of Layer ( j ) if the ( x , y , c ) element of mask ( j ) is zero . More specifically , after Random Mask , we will not compute these masked neurons and make the next layer regard these neurons as having value zero during computation . A simple visualization of Random Mask is shown in Figure 2 . The Random Mask in fact decreases the computational cost in each epoch since there are fewer effective connections . Note that the number of parameters in the convolutional kernels remains unchanged , since we only mask neurons in the feature maps . In the standard setting , convolutional filter will be applied uniformly to every position of the feature map of the former layer . The success of this implementation is due to the reasonable assumption that if one feature is useful to be computed at some spatial position ( x , y ) , then it should also be useful to be computed at a different position ( x′ , y′ ) . Thus the original structure is powerful for feature extraction . Moreover , this structure leads to parameter sharing which makes the training process more efficient . However , the uniform application of filter also prevents the CNN from noticing the distribution of features . In other words , the network focuses on the existence of a kind of feature but pays little attention to how this kind of feature distributes on the whole feature map ( of the former layer ) . Yet the pattern of feature distribution is important for humans to recognize and classify a photo , since empirically people would rely on some structured feature to perform classification . With Random Mask , each filter may only extract features from partial positions . More specifically , for one filter , only features which distribute consistently with the mask pattern can be extracted . Hence filters in a network with Random Mask may capture more information on the spatial structures of local features . Just think of a toy example : imagine Random Mask for a filter masks all the neurons but one row in the channel , if a kind of feature usually distributes in a column , it can not have strong response because the filter can only capture a small portion of the feature . We do a straightforward experiment to verify our intuition . We sample some images from ImageNet which can be correctly classified with high probability by both CNNs with and without Random Mask . We then randomly shuffle the images by patches , and compare the accuracy of classifying the shuffled images ( See Appendix A ) . We find out that the accuracy of the CNN with Random Mask is consistently lower than that of normal CNN . This result shows that CNNs without Random Mask cares more about whether a feature exists while CNNs with Random Mask will detect spatial structures and limit poorly-organized features from being extracted . We further explore how Random Mask plays its role in defending against adversarial examples . Recent observation ( Liu et al. , 2018 ) of adversarial examples found that these examples usually change a patch of the original image so that the perturbed patch looks like a small part of the incorrectly classified object . This perturbed patch , although contains crucial features of the incorrectly classified object , usually appears at the wrong location and does not have the right spatial structure with other parts of the image . For example ( See Figure 11 in Liu et al . ( 2018 ) ) , the adversarial example of a panda image is misclassified as a monkey because a patch of the panda skin is perturbed adversarially so that it alone looks like the monkey ’ s face . However , this patch does not form a right structure of a monkey with other parts of the images . By the properties of detecting spatial structures and limiting feature extraction , Random Mask can naturally help CNNs resist such adversarial perturbations . In complement to the observation we mentioned above , we also find that most adversarial perturbations generated against normal CNNs look like random noises which do not change the semantic information of the original image . In contrast , adversarial examples generated against CNNs with Random Mask tend to contain some well-organized features which sometimes change the classification results semantically ( See Figure 3 and Figure 1 ) . This phenomenon also supports our intuition that Random Mask helps to detect spatial structures and extract well-organized features via imposing limitations . While the features that can be learned by each masked filter is limited , the randomness helps us get plenty of diversified patterns . Our experiments show that these limited filters are enough for learning features . In other words , CNNs will maintain a high test accuracy after being applied with Random Mask . Besides , adding convolutional filters may help our CNN with Random Mask to increase test accuracy ( See Section 3.3 ) . Furthermore , our structure is naturally compatible to ensemble methods , and randomness makes ensemble more powerful ( See Section 3.3 ) . However , it might not be appropriate to apply Random Mask to deep layers . The distribution of features is meaningful only when the location in feature map is highly related to the location in the original input image , and the receptive field of each neuron in deep layers is too large . In Section 3.3 , there are empirical results which support our intuition .

Summary: The authors propose a simple method for increasing the robustness of convolutional neural networks against adversarial examples. This method is simple but seems to achieve surprisingly good results. It consist in randomly remove neurons from the network architecture. The deleted neurons are selected before training and remain deleted during the training and test phase.  The authors also study the adversarial examples that still fool the network after applying their method and find than those examples also fool human. This finding raises the question of what is an adversarial example if both humans and networks are fooled by the same example.