Defective Convolutional Layers Learn Robust CNNs

1 INTRODUCTION . Deep learning ( LeCun et al. , 2015 ) , especially deep Convolutional Neural Network ( CNN ) ( LeCun et al. , 1998 ) , has led to state-of-the-art results spanning many machine learning fields ( He et al. , 2016 ; Ren et al. , 2015 ) . Despite the great success in numerous applications , recent studies show that deep CNNs are vulnerable to some well-designed input samples named as Adversarial Examples ( Szegedy et al. , 2013 ; Biggio et al. , 2013 ) . Take the task of image classification as an example , for almost every commonly used well-performed CNN , attackers are able to construct a small perturbation on an input image . The perturbation is almost imperceptible to humans but can make the model give a wrong prediction . The problem is serious as some designed adversarial examples can be transferred among different kinds of CNN architectures ( Papernot et al. , 2016b ) , which means a machine learning system can be easily attacked even if the attacker does not have access to the model parameters . There is a rapidly growing body of work on how to obtain a robust convolutional neural network , mainly based on adversarial training ( Szegedy et al. , 2013 ; Madry et al. , 2017 ; Goodfellow et al. , 2015 ; Huang et al. , 2015 ) . However , those methods need lots of extra computation to obtain adversarial examples at each time step and may tend to overfit the attacking method used in training ( Buckman et al. , 2018 ) . In this paper , different from most existing methods , we tackle the problem from another perspective . In particular , we explore the possibility of designing new CNN architectures which can be trained using standard optimization methods on standard benchmark datasets but by themselves enjoy robustness , without appealing to other techniques . Recently , studies ( Geirhos et al. , 2017 ; 2018 ; Baker et al. , 2018 ) show that the predictions of CNNs mainly depend on the texture of objects but not the shape . Also , Liu et al . ( 2018 ) finds attack methods usually perturb patches to contain textural features of incorrect classes . They suggest that the wrong prediction by CNNs for adversarial examples comes from the change on the texture-level information . The small perturbation of adversarial examples will change the textures and eventually affect the features extracted by the CNNs . Therefore , a natural way to avoid adversarial examples is to let the CNN make prediction relying less on textures but more about other information which will not be severely affected by small perturbations , such as shape . In real practice , sometimes a camera might have mechanical failures which cause the output image to have many defective pixels ( such pixels are always black in all images ) . Nonetheless , humans can still recognize objects in the image with defective pixels but have to classify the objects by other information as some local textural information is missing . Motivated by this , we introduce the concept of defectiveness into the convolutional neural networks : We call a neuron a defective neuron if its output value is fixed to zero no matter what input signal is received , and a convolutional layer a defective convolutional layer if it contains defective neurons . Before training , we replace the standard convolutional layers with the defective version on a standard CNN and train the network in the standard way . As defective neurons of the defective convolutional layer contain no information and are very different from their spatial neighbors , the textural information can not be accurately extracted from the bottom defective layers to top layers . Therefore , we destroy local textural information to a certain extent and prompt the neural network to learn more other information for classification . We call the architecture deployed with defective convolutional layers as Defective CNN . We find that applying the defective convolutional layers to the bottom1 layers of the network and introducing various patterns for defective neurons arrangement across channels are crucial for robustness . According to the experimental results , we find 2 RELATED WORK . Various methods have been proposed to defend against adversarial examples . One line of research is to derive a meaningful optimization objective and optimize the model by adversarial training ( Szegedy et al. , 2013 ; Madry et al. , 2017 ; Goodfellow et al. , 2015 ; Huang et al. , 2015 ) . The high-level idea of these works is that if we can predict the potential attack to the model during optimization , then we 1In this paper , bottom layer means the layer close to the input and top layer means the layer close to the output prediction . can give the attacked sample a correct signal and use it during training . Another line of research is to take an adjustment to the input image before letting it go through the deep neural network ( Liao et al. , 2017 ; Song et al. , 2017 ; Samangouei et al. , 2018 ; Sun et al. , 2018 ) . The basic intuition behind this is that if we can clean the adversarial attack to a certain extent , then such attacks can be defended . Although these methods achieve some success , a major difficulty is that it needs a large extra cost to collect adversarial examples and hard to apply on large-scale datasets . Several studies ( Geirhos et al. , 2017 ; 2018 ; Baker et al. , 2018 ) show that the prediction of CNNs is mainly from the texture of objects but not the shape . Also , Liu et al . ( 2018 ) found that adversarial examples usually perturb a patch of the original image so that the perturbed patch looks like the texture of incorrect classes . For example , the adversarial example of the panda image is misclassified as a monkey because a patch of the panda skin is perturbed adversarially so that it alone looks like the face of a monkey ( see Figure 11 in Liu et al . ( 2018 ) ) . All previous works above suggest that the CNN learns textural information more than shape and the adversarial attack might come from textural-level perturbations . This is also correlated with robust features ( Tsipras et al. , 2018 ; Ilyas et al. , 2019 ; Hosseini et al. , 2019 ) which has attracted more interest recently . Pixels which encode textural information contain high redundancy and may be easily deteriorated to the distribution of incorrect classes . However , shape information is more compact and might be a more robust feature . 3 DEFECTIVE CONVOLUTIONAL NEURAL NETWORK . 3.1 DESIGN OF DEFECTIVE CONVOLUTIONAL LAYERS . In this subsection , we introduce our proposed defective convolutional neural networks and discuss the differences between our proposed method and related topics . First , we briefly introduce the notations . For one convolutional layer , denote x as the input and z as the output of neurons in the layer . Note that x may be the input image or the output of the last convolutional layer . The input x is usually a M ×N ×K tensor in which M/N are the height/width of a feature map , and K is the number of feature maps , or equivalently , channels . Denote w and b as the parameters ( e.g. , the weights and biases ) of the convolutional kernel . Then a standard convolutional layer can be mathematically defined as below . Standard convolutional layer : . x′ = w ⊗conv x+ b , ( 1 ) z = f ( x′ ) , ( 2 ) where f ( · ) is a non-linear activation function such as ReLU2 and ⊗conv is the convolutional operation . The convolutional filter receives signals in a patch and extracts local textural information from the patch . As mentioned in the introduction , recent works suggest that the prediction of standard CNNs strongly depends on such textural information , and noises imposed on the texture may lead to wrong predictions . Therefore , we hope to learn a feature extractor which does not solely rely on textural features and also considers other information . To achieve this goal , we introduce the defective convolutional layer in which some neurons are purposely designed to be corrupted . Define Mdefect to be a binary matrix of size M ×N ×K . Our defective convolutional layer is defined as follows . Defective convolutional layer : . x′ = w ⊗conv x+ b , ( 3 ) z′ = f ( x′ ) ( 4 ) z = Mdefect ∗ z′ , ( 5 ) where ∗ denotes element-wise product . Mdefect is a fixed matrix and is not learnable during training and testing . A simple visualization of a defective convolutional layer is shown in Figure 2 . From the figure , we can see that Mdefect plays a role of “ masking ” out values of some neurons in the layer . This disturbs the distribution of local textural information and decouples the correlation among neurons . With the masked output z as input , the feature extractor of the next convolutional layer can not accurately capture the local textural feature from x . As a consequence , the textural information 2Batch normalization is popularly used on x′ before computing z . Here we simply omit this . is hard to pass through the defective CNN from bottom to top . To produce accurate predictions , the deep neural network has to find relevant signals other than the texture , e.g. , the shape . Those corrupted neurons have no severe impact on the extraction of shape information since neighbors of those neurons in the same filter are still capable of passing the shape information to the next layer . In this paper , we find that simply settingMdefect by random initialization is already helpful for learning a robust CNN . Before training , we sample each entry in Mdefect using Bernoulli distribution with keep probability p and then fix Mdefect during training and testing . More discussions and ablation studies are provided in Section 4 . As can be seen from Equation ( 3 ) - ( 5 ) , the implementation of our defective convolutional layer is similar to the dropout operation ( Srivastava et al. , 2014 ) . To demonstrate the relationship and differences , we mathematically define the dropout as below . Standard convolutional layer + dropout : Mdp ∼ Bernoulli ( p ) ( 6 ) x′ = w ⊗conv x+ b ( 7 ) z′ = f ( x′ ) ( 8 ) z = Mdp ∗ z′ . ( 9 ) The shape of Mdp is the same as Mdefect , and the value of each entry in Mdp is sampled in each batch using some sampling strategies at each step during training . Generally , entries in Mdp are independent and identically sampled in an online fashion using Bernoulli distribution with keep probability p. DropBlock ( Ghiasi et al. , 2018 ) dropouts a connected block in each channel . In SpatialDropout ( Tompson et al. , 2015 ) , the dropout masks apply to whole channels . Note that the masked unit of our proposed method is a single neuron . There are several differences between dropout and defective convolutional layer . First , the motivations behind the two methods are quite different . Dropout tries to reduce overfitting by preventing coadaptations on training data . As the neurons in the feature maps still have full access to local textural features in testing , the model does not have to learn shape features . However , in our proposed architecture , defective neurons are fixed to be corrupted , and such neurons can not contribute to local features . Second , the binary matrix Mdp is sampled online during training and is removed during testing , while the binary matrix Mdefect in defective convolutional layers is predefined and keeps fixed in both training and testing . Third , places to apply and values of the keep probability p for two methods are different . Dropout methods are usually applied to top layers , and p is set to be large ( e.g. , 0.9 ) ( Tompson et al. , 2015 ; Ghiasi et al. , 2018 ) . For defective convolutional layer , we find using a small p ( e.g. , 0.1 ) and applying it to bottom layers are more effective .

Summary: The paper proposes defective convolutional layers as a measure of defense against adversarial attacks on deep neural networks. This layer sets the outputs of a randomly sampled but *fixed* set of neurons in the convolutional layers to zero during training and testing. The authors claim that defective convolutional layers encourage the model to pick up features other than local textures, e.g. shape information. The shape-vs-texture tradeoff is supported by experiments showing that defective CNNs perform worse than normal CNNs on images with permuted patches and that adversarial examples with larger epsilons exhibit more semantic shapes. The detailed experiment section evaluates the method on transfer-based, gray-box and black-box adversarial attacks,	including Gaussian noise. Additionally, it provides ablation studies on the keep-probability and position of the defective layer.