Defective Convolutional Networks

1 INTRODUCTION . Deep learning ( LeCun et al. , 1998 ; 2015 ) , especially deep Convolutional Neural Network ( CNN ) ( Krizhevsky et al. , 2012 ) , has led to state-of-the-art results spanning many machine learning fields ( Girshick , 2015 ; Chen et al. , 2018 ; Luo et al. , 2020 ) . Despite the great success in numerous applications , recent studies show that deep CNNs are vulnerable to some well-designed input samples named as Adversarial Examples ( Szegedy et al. , 2013 ; Biggio et al. , 2013 ) . Take the task of image classification as an example , for almost every commonly used well-performed CNN , attackers are able to construct a small perturbation on an input image , which is almost imperceptible to humans but can make the model give a wrong prediction . The problem is serious as some well-designed adversarial examples can be transferred among different kinds of CNN architectures ( Papernot et al. , 2016b ) . As a result , a machine learning system can be easily attacked even if the attacker does not have access to the model parameters , which seriously affect its use in practical applications . There is a rapidly growing body of work on how to obtain a robust CNN , mainly based on adversarial training ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ; Madry et al. , 2017 ; Buckman et al. , 2018 ; Mao et al. , 2019 ) . However , those methods need lots of extra computation to obtain adversarial examples at each time step and tend to overfit the attacking method used in training ( Buckman et al. , 2018 ) . In this paper , we tackle the problem in a perspective different from most existing methods . In particular , we explore the possibility of designing new CNN architectures which can be trained using standard optimization methods on standard benchmark datasets and can enjoy robustness by themselves , without appealing to other techniques . Recent studies ( Geirhos et al. , 2017 ; 2018 ; Baker et al. , 2018 ; Brendel & Bethge , 2019 ) show that the predictions of standard CNNs mainly depend on the texture of objects . However , the textural information has a high degree of redundancy and may be easily injected with adversarial noise ( Yang et al. , 2019 ; Hosseini et al. , 2019 ) . Also , Cao et al . ( 2020 ) ; Das et al . ( 2020 ) finds adversarial attack methods may perturb local patches to contain textural features of incorrect classes . All the literature suggests that the wrong prediction by CNNs for adversarial examples mainly comes from the change in the textural information . The small perturbation of adversarial examples will change the textures and eventually affect the features extracted by the CNNs . Therefore , a natural way to avoid adversarial examples is to let the CNN make predictions relying less on textures but more on other information , such as the shape , which can not be severely distorted by small perturbations . In practice , sometimes a camera might have mechanical failures which cause the output image to have many defective pixels ( such pixels are always black in all images ) . Nonetheless , humans can still recognize objects in the image with defective pixels since we are able to classify the objects even in the absence of local textural information . Motivated by this , we introduce the concept of defectiveness into the convolutional neural networks : we call a neuron a defective neuron if its output value is fixed to zero no matter what input signal is received ; similary , a convolutional layer is a defective convolutional layer if it contains defective neurons . Before training , we replace the standard convolutional layers with the defective version on a standard CNN and train the network in the standard way . As defective neurons of the defective convolutional layer contain no information and are very different from their spatial neighbors , the textural information can not be accurately extracted from the bottom defective layers to top layers . Therefore , we destroy local textural information to a certain extent and prompt the neural network to rely more on other information for classification . We call the architecture deployed with defective convolutional layers as defective convolutional network . We find that applying the defective convolutional layers to the bottom1 layers of the network and introducing various patterns for defective neurons arrangement across channels are critical . In summary , our main contributions are : • We propose Defective CNNs and four empirical evidences to justify that , compared to standard CNNs , the defective ones rely less on textures and more on shapes of the inputs for making predictions . • Experiments show that Defective CNNs has superior defense performance than standard CNNs against transfer-based attacks , decision-based attacks , and additive Gaussian noise . • Using the standard training method , Defective CNN achieves state-of-the-art results against two transfer-based black-box attacks while maintaining high accuracy on clean test data . • Through proper implementation , Defective CNNs can save a lot of computation and storage costs ; thus may lead to a practical solution in the real world . 2 RELATED WORK . Various methods have been proposed to defend against adversarial examples . One line of research is to derive a meaningful optimization objective and optimize the model by adversarial training ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ; Huang et al. , 2015 ; Madry et al. , 2017 ; Buckman et al. , 2018 ; Mao et al. , 2019 ) . The high-level idea of these works is that if we can predict the potential attack to the model during optimization , then we can give the attacked sample a correct signal and use it during training . Another line of research is to take an adjustment to the input image before letting it go through the deep neural network ( Liao et al. , 2017 ; Song et al. , 2017 ; Samangouei et al. , 2018 ; Sun et al. , 2018 ; Xie et al. , 2019 ; Yuan & He , 2020 ) . The basic intuition behind this is that if we can clean the adversarial attack to a certain extent , then such attacks can be defended . Although these methods achieve some success , a major difficulty is that it needs a large extra cost to collect adversarial examples and hard to apply on large-scale datasets . Several studies ( Geirhos et al. , 2017 ; 2018 ; Baker et al. , 2018 ; Brendel & Bethge , 2019 ) show that the prediction of CNNs is mainly from the texture of objects but not the shape . Also , Cao et al . ( 2020 ) ; Das et al . ( 2020 ) found that adversarial examples usually perturb a patch of the original image to contain the textural feature of incorrect classes . For example , the adversarial example of the panda image is misclassified as a monkey because a patch of the panda skin is perturbed adversarially so that it alone looks like the face of a monkey ( see Figure 11 in ( Cao et al. , 2020 ) ) . All previous works above suggest that the CNN learns textural information more than shape and the adversarial attack might come from textural-level perturbations . This is also correlated with robust features ( Tsipras et al. , 2018 ; Ilyas et al. , 2019 ; Hosseini et al. , 2019 ; Yang et al. , 2019 ) which has attracted more interest recently . Pixels which encode textural information contain high redundancy and may be easily deteriorated to the distribution of incorrect classes . However , shape information is more compact and thus may serve as a more robust feature for predicting . 1In this paper , bottom layer means the layer close to the input and top layer means the layer close to the output prediction . 3 DEFECTIVE CONVOLUTIONAL NEURAL NETWORK . 3.1 DESIGN OF DEFECTIVE CONVOLUTIONAL LAYERS . In this subsection , we introduce our proposed defective convolutional neural networks and discuss the differences between the proposed method and related topics . First , we briefly introduce the notations . For one convolutional layer , denote x as the input and z as the output of neurons in the layer . Note that x may be the input image or the output of the last convolutional layer . The input x is usually aM×N×K tensor in whichM/N are the height/width of a feature map , and K is the number of feature maps , or equivalently , channels . Denote w and b as the parameters ( e.g. , the weights and biases ) of the convolutional kernel . Then a standard convolutional layer can be mathematically defined as below . Standard convolutional layer : . x′ = w ∗ x+ b , ( 1 ) z = f ( x′ ) , ( 2 ) where f ( · ) is a non-linear activation function such as ReLU2 and ∗ is the convolutional operation . The convolutional filter receives signals in a patch and extracts local textural information from the patch . As mentioned in the introduction , recent works suggest that the prediction of standard CNNs strongly depends on such textural information , and noises imposed on the texture may lead to wrong predictions . Therefore , we hope to learn a feature extractor which does not solely rely on textural features and also considers other information . To achieve this goal , we introduce the defective convolutional layer in which some neurons are purposely designed to be corrupted . Define Mdefect to be a binary matrix of size M ×N ×K . Our defective convolutional layer is defined as follows . Defective convolutional layer : . x′ = w ∗ x+ b , ( 3 ) z′ = f ( x′ ) ( 4 ) z = Mdefect ◦ z′ , ( 5 ) where ◦ denotes element-wise product . Mdefect is a fixed matrix and is not learnable during training and testing . We can see that Mdefect plays a role of “ masking ” out values of some neurons in the layer . This disturbs the distribution of local textural information and decouples the correlation among neurons . With the masked output z as input , the feature extractor of the next convolutional layer can not accurately capture the local textural feature from x . As a consequence , the textural information is hard to pass through the defective CNN from bottom to top . To produce accurate predictions , the deep neural network has to find relevant signals other than the texture , e.g. , the shape . Those corrupted neurons have no severe impact on the extraction of shape information since neighbors of those neurons in the same filter are still capable of passing the shape information to the next layer . In this paper , we find that simply setting Mdefect by random initialization is already helpful for learning a robust CNN . Before training , we sample each entry in Mdefect using Bernoulli distribution with keep probability p and then fix Mdefect during training and testing . More discussions and ablation studies are provided in Section 4 . As can be seen from Equation ( 3 ) - ( 5 ) , the implementation of our defective convolutional layer is similar to the dropout operation ( Srivastava et al. , 2014 ) . To demonstrate the relationship and differences , we mathematically define the dropout as below . Standard convolutional layer + dropout : Mdropout ∼ Bernoulli ( p ) ( 6 ) x′ = w ∗ x+ b ( 7 ) z′ = f ( x′ ) ( 8 ) z = Mdropout ◦ z′ . ( 9 ) 2Batch normalization is popularly used on x′ before computing z . Here we simply omit this . The shape of Mdropout is the same as Mdefect , and the value of each entry in Mdropout is sampled in each batch using some sampling strategies at each step during training . Generally , entries inMdropout are independent and identically sampled in an online fashion using Bernoulli distribution with keep probability p. There are several significant differences between dropout and defective convolutional layer . First , the binary matrix Mdropout is sampled online during training and is removed during testing , while the binary matrix Mdefect in defective convolutional layers is predefined and keeps fixed in both training and testing . The predefined way can help Defective CNNs save a lot of computation and storage costs . Second , the motivations behind the two methods are quite different and may lead to differences in the places to applying methods , the values of the keep probability p , and the shape of the masked unit . Dropout tries to reduce overfitting by preventing co-adaptations on training data . When comes to CNNs , those methods are applied to top layers , p is set to be large ( e.g. , 0.9 ) , and the masked units are chosen to be a whole channel in Tompson et al . ( 2015 ) and a connected block in Ghiasi et al . ( 2018 ) . However , our method tries to prevent the model extract textural information of inputs for making predictions . We would apply the method to bottom layers , use a small p ( e.g . 0.1 ) , and the masked unit is a single neuron . Also , in our experiments , we will show that the proposed method can improve the robustness of CNNs against transfer-based attacks and decisionbased attacks , while the dropout methods can not .

Summary: The paper proposes a method to improve adversarial robustness of the current convolutional networks. The method is based on dropping outputs of a fraction of neurons. However, unlike in dropout the masks are kept fixed throughout training/inference and applied to the bottom layers of the network. This shifts the focus of the network away from texture and towards shape features resulting in the adversarial examples being fooling humans as well.