Improving Adversarial Robustness via Channel-wise Activation Suppressing
Authors: Yang Bai
Yuyuan Zeng
Yong Jiang
Shu-Tao Xia
Xingjun Ma
Yisen Wang
Venue: ICLR 2021 Spotlight (2021)
Keywords: Adversarial robustness
channel suppressing
activation strategy.

Abstract: The study of adversarial examples and their activations have attracted significant attention for secure and robust learning with deep neural networks (DNNs).  Different from existing works, in this paper, we highlight two new characteristics of adversarial examples from the channel-wise activation perspective:  1) the activation magnitudes of adversarial examples are higher than that of natural examples; and 2) the channels are activated more uniformly by adversarial examples than natural examples. We find that, while the state-of-the-art defense adversarial training has addressed the first issue of high activation magnitude via training on adversarial examples, the second issue of uniform activation remains.  This motivates us to suppress redundant activations from being activated by adversarial perturbations during the adversarial training process, via a Channel-wise Activation Suppressing (CAS) training strategy.  We show that CAS can train a model that inherently suppresses adversarial activations, and can be easily applied to existing defense methods to further improve their robustness. Our work provides a simplebut generic training strategy for robustifying the intermediate layer activations of DNNs.

Published as a conference paper at ICLR 2021
ROBUSTNESS CHANNEL- WISE ACTIVATION SUPPRESSING
IMPROVING ADVERSARIAL
VIA
Yang Bai'. Yuyuan Zeng25 Yong Jiang! Shu-Tao Xia Xingjun Ma* Yisen WangSt
TTinnguu Berkeley Shenzhen Institute, Tsinghua University, China
Tsinghua Shenzhen International Graduate School, Tsinghua University, China
3 Key Lab. of Machine Perception (MoE), School of EECS, Peking University, Beijing. China
A "School of Information Technology, Deakin University, Geelong, VIC, Australia
$ SPCL Research Center of Networks and Communications, Peng Cheng Laboratory, Shenzhen, China
MXXiv:2103.88000 [cs.LG] 11 Mar 2021
ABSTRACT
The study of adversarial examples and their activation has attracted significant attention for secure and robust learning with deep neural networks (DNNss. Dif- ferent from existing works, in this paper, we highlight two new characteristics of adversarial examples from the channel-wise activation perspective: 1) the ac- tivation magnitudes of adversarial examples are higher than that of natural ex- amples; and 2) the channels are activated more uniformly by adversarial exam- ples than natural examples. We find that the state-of-the-art defense adversarial training has addressed the first issue of high activation magnitudes via training adversarial examples, while the second issue of uniform activation remains. This motivates us to suppress redundant activation from being activated by ad- versarial perturbations via Channel-wise Activation Suppressing (CAS) strat- egy. We show that CAS can train a model that inherently suppresses adversarial activation, and can be easily applied existing defense methods to further im- prove their robustness. Our work provides a simple but generic training strategy for robustifying the intermediate layer activation of DNNs. Code is available https::/ /ttp://gggtut.../// com/bymavis/CAS ISS__CCRR221111
INTRODUCTION
Deep neural networks (DNNs) have become standard models for solving real-world complex prob- lems, such as image classification (He et al. 2016), speech recognition (Wang et al. 2017 and natural language processing (Devlin et al. 2019). DNNs can approximate extremely complex func- tions through a series of linear (e.g. convolution) and non-linear (e.g. ReLU activation) operations. Despite their superb learning capabilities, DNNs have been found to be vulnerable to adversarial examples (or attacks) (Szegedy et al. [014]] Goodfellow et al. 2015). where small perturbations on the input can easily subvert the model's prediction. Adversarial examples can transfer across dif- ferent models (Liu et al., 2017]] Wu et al. [220aa] Wang et al. (202]) and remain destructive even in the physical world (Kurakin et al. [0166] Duan et al.) 2020), raising safety concerns autonomous driving (Eykholt et al.) 2018) and medical diagnosis (Ma et al. 2021).
Existing defense methods against adversarial examples include input denoising (iao et al. 2018
Bai et 2019), defensive distillation (Papernot et al., 2016), gradient regularization (Gu & Rigazio 2014). model compression (Das al. 2018) and adversarial training (Goodfellow et al. 2013
Madry et al. 2018 Wang et al., 2019) amongst which adversarial training has demonstrated the most reliable robustness (Athalye et al.) 2019, Croce & Hein, 20206). Adversarial training is data augmentation technique that trains DNNs adversarial rather than natural examples. In adversarial training, natural examples are augmented (or perturbed) with the worst-case perturbations found within small norm ball around them. This augmentation has been shown effectively smooth out the loss landscape around the natural examples, and force the network to focus more on the pixels that are most relevant to the class. Apart from these interpretations, it is still not well understood,
"Equll contribution.
Correspondence to: Yisen Wang wrisen.wangee (ww.nnuuuu..uuccn). Shu- Xia (xiast @ ssinggua.eduuccn)

---

Published as a conference paper at ICLR 2021
from the activation perspective, how small input perturbations accumulate across intermediate layers to subvert the final output, and how adversarial training can help mitigate such an accumulation. The study of intermediate layer activation has thus become crucial for developing more in-depth understanding and robust DNNs.
In this paper, we show that, if studied from channel-wise perspective, strong connections between certain characteristics of intermediate activation and adversarial robustness can be established. Our channel-wise analysis is motivated by the fact that different convolution filters (or channels) learn different patterns, which when combined together, describe a specific type of object. Here, adver- sarial examples are investigated from a new perspective of channels in activation. Different from the existing activation works assuming different channels are of equal importance, we focus on the relationship between channels. Intuitively, different channels of an intermediate layer contribute differently to the class prediction, thus have different levels of vulnerabilities (or robustness) to ad- versarial perturbations. Given an intermediate DNN layer, we first apply global average pooling (GAP) to obtain the channel-wise activation, based on which, we show that the activation magni- tudes of adversarial examples are higher than that of natural examples. This means that adversarial perturbations generally have the iignal-bosstnn effect channels. We also find that the channels are activated more uniformly by adversarial examples than that by natural examples. In other words, some redundant (or low contributing) channels that not activated by natural examples, yet activated by adversarial examples. We show that adversarial training can effectively address the high magnitude problem. yet fails to address the uniform channel activation problem, that is, some redun- dant and low contributing channels are still activated. This to some extent explains why adversarial training works but its performance is not satisfactory.
Therefore, we propose a new training strategy named Channel-wise Activation Suppressing (CAS), which adaptively learns (with an auxiliary classifier) the importance of different channels to class prediction, and leverages the learned channel importance to adjust the channels dynamically. The robustness of existing state-of-the-art adversarial training methods can be consistently improved applied with our CAS training strategy. Our key contributions are summarized as follows:
We identify, from a channel-wise activation perspective, two connections between DNN activation and adversarial robustness: 1) the activation of adversarial examples are of higher magnitudes than that of natural examples; and 2) the channels are activated more uniformly by adversarial examples than that by natural examples. Adversarial training only addresses the first issue of high activation magnitudes, yet fails to address the second issue of uniform channel activation.
We propose novel training strategy to train robust DNN intermediate layers via Channel- wise Activation Suppressing (CAS). In the training phase, CAS suppresses redundant chan- nels dynamically by reweighting the channels based on their contributions the class pre- diction. CAS is a generic intermediate-layee robustification technique that can be applied to any DNNs along with existing defense methods.
We empirically show that CAS training strategy can consistently improve the robust- ness of current state-of-the-as adversarial training methods. is generic, effective, and can be easily incorporated into many existing defense methods. We also provide complete analysis on the benefit of channel-wise activation suppressing to adversarial robustness.
2 RELATED WORK
Aevessaiial Defense. Many adversarial defense techniques have been proposed since the discovery of adversarial examples. Among them, many were found to have caused obfuscated gradients and can be circumvented by Back Pass Differentiable Approximation (BPDA), Expectation over Trans- formation (EOT) Reparameterization (Athalye al.; 2019)) Adversarial training (AT) has been
demonstrated to be the most effective defense (Madry et al. 2018 Wang et al. 2019 2020b), which
solves the following min-max optimization problem:
min
max (a)
((F(x(0))
(I)
where, F is a DNN model with parameters 8, x is a natural example with class label y, x' is the adversarial example within the nno-m ball B, (m)
{x' :ll x² ك e) centered at xc,
2

---

Published as a conference paper at ICLR 2021
F(a',0) is the output of the network, and C is the classification loss (e.g. the cross-entropy loss). The inner maximization problem is dependent on the adversarial examples x' generated within the e-ball, while the outer minimization problem optimizes model parameters under the worst-case perturbations found by the inner maximization. There are other variants of adversarial training with some new objective function regularization. For example, TRADES (Zhang al.. (019) optimized a trade-off' objective of adversarial robustness and accuracy. MART (Wang et al. 2020c) applied a distinctive emphasis on misclassified versus correctly classified examples. AWP (Wu et al., 2020b) incorporated the regularization on the weight loss landscape. However, apart from
these improvements, it is still not well understood how adversarial training can help produce state- of-the-art robustness from the activation perspective.
Activation Perspective of Adversarial Robustness. Some previous works have investigated the adversarial robustness from the architecture perspective, such as skip connection (Wu et al., 2020a) and batch normalization (Galloway et al., 2019). As for intermediate activation, Ma et al. (2018) characterized that adversarial activation forms adversarial subspace that has much higher intrinsic dimensionality, /Zhang et al. (2018) certified the robustness of neural networks with different activa- tion functions. Xu et al. (2019) explored the influence of adversarial perturbations activation from suppression, promotion and balance perspectives. Other works developed new activation operations with the zation techniques (Rakin et al. (2118). While these works directly modify the activation functions, there are also works focusing on the activation outputs. For instance, k-Winner-Takes- (Xiao et al., (020) taked the largest & feature values in each activation layer to enhance adversar- al robustness. However, this has recently been shown not robust against adaptive attacks (Tramer) et al. 2020). Stochastic Activation Pruning (SAP) (Dhillon et al. 2018) taked the randomness and the value of features into consideration. Each activation is chosen with a probability proportional to its absolute value. Adversarial Neural Pruning (ANP) (Madaan & Hwang 2020) pruned out the fea- tures that are vulnerable to adversarial inputs using Bayesian method. Prototype Conformity Loss (PCL) (Mustafa et al.) 2019) was proposed to cluster class-wist features and push class centers away from each other. Feature Denoising (FD) (Xic et al. (2019) added denoising layers to the network for sample-wiss denoising
data-dependent function (Wang et al. 2020a) and adaptive quanti-
(kWTA)
feature maps. However, these methods were developed based on obser- vations on the full output (e-g. the entire feature or activation map that does not distinguish different channels) of DNN intermediate layers. In contrast, our CAS explores both channel importance and channel correlations, and the suppressing is done with the guidance of the labels.
3 CHANNEL-WISE ACTIVATION AND ADVERSARIAL ROBUSTNESS
In this part, we investigate two characteristics of DNN intermediate activation from a channel-wise perspective, and show two empirical connections between channel-wise activation and adversarial robustness. Specifically, we train ResNet-1s 18 (He et al., 2016) and VGG16 (Simonyan & Zisserman 2014) CIFAR-10 (Krizhevsky et al. 2009) using both standard training and adversarial training under typical settings. We then apply global average pooling to extract the channel-ve wise activation from the penultimate layer. We investigate the extracted channel-wise activation of both natural and adversarial examples from two perspectives: 1) the magnitude of the activation, and 2) the activation frequency of the channels.
Channel-wise Activation Magnitude. Figure I illustraaes the averaged activation magnitudes for both natural test examples and the corresponding adversarial examples crafted by PGD-20 attack (Madry et al., 2018). For standard models (trained on natural examples), the activation magni- tudes of adversarial examples are generally higher than that of natural examples as shown in Figure [C]] Adversarial perturbation exhibits a clear signal-boosting
effect on the channels, which leads to the accumulation of adversarial distortions from the input to the output layer of the net- work. As shown in Figure [ [dd] adversarial training can effectively narrow the magnitude gaps between natural and adversarial examples, interestingly by decreasing the activation magnitudes of adversarial examples. This is because adversarial training can restrict the Lipschiz constant of the model at deeper layers (i.e. layers close to the output). which reduces the magnitude gaps caused by adversarial perturbations (Finlay al. 2018 Sinha et al.) (2019). Note that network architecture also influences activation magnitudes. Figure [ show that magnitudes in VGG have much more zero values than those in ResNet, i.e. VGG produces more sparse channels than ResNet.
3

---

Published as a conference paper at ICLR 2021
1.0
1.0
1.0
1.0
ronrrl esamples
examples shamp
naural examples
examples examples
0.
0.0
0.0
0.0
0.4
0.4
0.4
0.4
Mege
0.0
155
200
320
505
0.0
200
400
0.0
100
300
530
0.0
100
200 455
Channel
Chonnel
(a) R8 (STD)
(b) ResNett-18 (ADV)
(c) VGGI6 (STD)
(d) VGG16 (ADV)
Figure 1: The magnitudes (y-axis) of channel-wise activation at the penultimate layer (512 channels at x-axis) for both standard ('STD') and adversarially trained ('ADV") models. In each plot, the magnitudes are averaged and displayed separately for natural and adversarial test examples. The 512 channels are sorted in descending order of the magnitude.
1090
1000
1000
INturnl eaRmptte
netural earmples
natural excmpppe adverratt eommples
TARTA POWTROEE
advemurtsl eccmplls
Number Accivatiin
Number Auchvvation
Number offrrrttttte Acchvatton
600
600
500
200
200
200
300
150
300
400
100
330
402
Channal
Channel
(a) 18 (STD)
(b) ResNet-18 (ADV)
(c) ResNet-18 (our CAS)
The activation frequency (y-axis) of channel-wise activation at the penultimate layer (512 channels at x-axis) of ResNet-18 18 trained using (a) standard training ('STD'). (b) adversarial train- ing ('ADV',, and (c) our CAS-based adversarial training ('CAS'). The activation frequencies are counted separately for the natural test examples and their PGD-20 adversarial examples. Channels are sorted in a descending order of activation frequency of natural examples.
Figure 2:
Channel-wise Activation Frequency. Given specific class, different convolution filters learn dif- ferent patterns associated with the class. Similar to the robust vs. non-robust feature differentiation in adversarial training (Ilyas al., 2019), the intermediate filters (or channels) can also be robust or non-robust. Intuitively, for natural examples in the same class, robust channels produce more generic patterns and should be activated more frequently, yet the non-robust ones should be acti- vated less frequently. As such, non-robust channels can cause more variations to the next layer if activated by adversarial perturbations, increasing the vulnerability to adversarial examples. To investigate this, we visualize the activation frequency of the channel-wise activation in Figure 2 Here, we take one specific class (e.g. class 0) of CIFAR-10 as an example. A channel is determined as activated if its activation value is larger than a threshold (e.g. 1% of the maximum activation value over all 512 channels). We count the activation frequency for each channel by natural exam- standard or adversarially trained ResNet- 18
ples and their PGD adversarial examples separately models, and sort the channels in a descending order according to the activation frequency by natu- ral examples. As can be observed in Figure 2(a) adversarial examples activate the channels more uniformly, and they tend to frequently activate those that are rarely activated by natural examples (e.g. the right region in Figure 2(a). This observation is consistent across different classes. The low frequency channels are non-robust channels, and correspond to those redundant activation that are less important for the class prediction. It can also be observed that adversarial perturbations also inhibit those high frequency channels of natural examples (the left region of Figure 2(a)). Figure 2(b))sows shows that, by training on adversarial examples, adversarial training can force the channels to be activated in similar frequency by both natural and adversarial examples. However, there are still a certain proportion of the redundant channels (e.g. channels #350 #500) that are activated by adversarial examples. This motivates us to propose Channel-wise Activation Suppressing (CAS) training strategy to avoid those redundant channels from being activated by adversarial examples. Figure 2(c) shows the effectiveness of our CAS strategy applied with adversarial training, that is, our CAS can suppress all channels, especially those low frequency ones on natural examples. More visualizations of channel-wise activation frequency can be found in Appendix C
4

---

Published as a conference paper at ICLR 2021
4 PROPOSED CHANNEL-WISH ACTIVATION SUPPRESSING
In this section, we introduce our Channel-wise Activation Suppressing (CAS) training strategy, which dynamically learns and incorporates the channel importance (to the class prediction) into the training phase to train a DNN model that inherently suppresses those less important channels.
CAS
Cl Loae
Ixpue
18ck
Figure 3: Framework of proposed Channel-wise Activation Suppressing (CAS).
Overview. Figure 3 illustrates our CAS training strategy. The CAS module consists of global average pooling operation (i.e. GAP in the CAS module) to obtain the channel-wise activation, and an auxiliary classifier (i.e. FC in the CAS module) to learn the channel importance. The learned channel importance then multiplied back to the original activation for adjustment, and the adjusted activation are then passed into the next layer for model training. The entire network and the auxiliary classifier are trained simultancously using a combination of the CAS loss and the CE loss. The CAS module can be attached to any intermediate layer of DNN.
4.1 CAS MODULE
Denote the /-th activation layer output of network F as f RRxxxK where H, W, K represent the height, width, channel of the activation map, respectively. In CAS module, we first apply the GAP operation on the raw activation f to obtain the channel-wise activation f E R Formally, for the k-th channel,
f. - HXW M (1.5))
(2)
The channel-wise activation f is then passed into the auxiliary classifier to perform multi-class classification with fully-connected (FC) layer. For C' classes, the parameters of the auxiliary classifier can be written as A/' [M M,,,,, E KCC which can identify the importance of each channel to a specific class, and will be applied to reweight the original activation f in channel wise manner. In the training phase, the ground-truth label y is utilized the index to determine the channel importance, í.e. Mr E R While in the test phase, since the ground-truth label is not available, we simply take the weight component M,, E RRK that is associated to the predicted class ÿ' as the channel importance (detailed analysis can found in Section 5. The computed channel importance is then applied to reweight the original activation map f follows:
& M & M
(training phase) (test phase)
fl
where represents the channel-wise multiplication. The adjusted f will be passed into the next layer via forward propagation. Note that, so far, neither the auxiliary nor the network is trained, just computing the channel importance and adjusting the activation in channel-wise manner.
4.2 MODEL TRAINING
We can insert S Channel-wise Activation Suppressing (CAS) modules into S different intermediate layers of DNNs. The CAS modules can be considered as auxiliary components of the network, and be trained using standard training different types of adversarial training. Here, take the

---

Published as a conference paper at ICLR 2021
original adversarial training (Madry et al. 2018) example, and define the loss functions to simultaneously train the network and our CAS modules. Each of our CAS modules has a FC layer. Taking inserted CAS module after the I-th activation layer of network F for examples, the CAS loss function can then be defined as,
Lcss(p (, M),u) = Σ fcc - y) log (2)),
(4)
where C' is the number of classes, p' softmax M') RC the prediction score the classifier in CAS module, and x' is the adversarial example used for training. Note that CCAS is the cross entropy loss defined on the auxiliary classifier. Similarly, it can also be extended to multiple CAS modules. The overall objective function for adversarial training with our CAS strategy is:
L(x', y; 0, M) CcE(p(#' .0).)) B
Cp (x', M))
(5)
where B is a tunable parameter balancing the strength of CAS. Besides the original adversarial training (AT) (Madry et al. 20188) we can also combine CAS with other defense techniques such as TRADES (Zhang et al., 2019) and MART (Wang al. 2020c). In Appendix B summarize the loss functions of the original AT, TRADES, MART, and their combined versions with our CAS.
The complete training procedure of our CAS is described in Algorithm in Appendix
EXPERIMENTS
In this section, first provide comprehensive understanding of our CAS training strategy, then evaluate its robustness on benchmark datasets against various white-box and black-box attacks.
5.1 EMPIRICAL UNDERSTANDING OF CAS
In this part, we first show the suppressig effect and robustness of our CAS, then analyze the effectiveness of our CAS when applied at different layers of DNN. The parameter analysis of CAS can found Appendix D In Appendix we show that CAS can also help representation learning and natural training.
Experimental Settings. We adversarially train ResNet-18 for 200 epochs on CIFAR-I0 using SGD with momentum 0.9, weight decay 2e-4, and initial learning rate 0.1 which is divided by 10 at 75-th and 90-th epoch. We use PGD-10 (c = 8/255 and step size 2/255) with random start for training. The robustness (accuracy adversarial examples) is evaluated under attacks: FGSM (Goodfellow
et al. 2015). PGD-20 (Madry et al. 2018), and CW (Carlini & Wagner 2017) optimized by PGD.
1000
2000
Number Activation
600
advercaral
Number of Activttion
800
Numbarr Astivation
800
Numbber of Autivaatio
800
600
800
600
600
400
400
400
403
200
200
200
200
502 Channel
400
300
200
250
332
400
330 Channel
402
300
200 100
200 405
Channel
Charnel
(a) AWTA
(b) SAP
(c) PCL
(d) CAS
Figure 4: Comparisons of activation frequency distribution between adversarial and natural exam- ples on different activation or feature oriented defense methods (kWTA, SAP, PCL and our CAS).
Channel Suppressing Effect. We compare CAS with three activation- or feature-based defense methods: AWTA (Xiao et al.) 2020), SAP (Dhillon et al., 2018) and PCL (Mustafa et al.) 2019). Here, we train &WTA with 20% sparse largest values in each activation layer, SAP with typical random pruning and PCL with warm-up training by CE loss and then fine-tuning with the added PCL loss. Figure 4soows the activation frequencies at the penultimate layer of ResNet- 18 trained by different methods. While &WTA, SAP and PCL demonstrate certain level of channel suppressing, their effects are not as significant as our CAS training. kWTA and SAP hardly have the channel suppressing effect (e.g. channel #350 #500 in Figure 4(a) and channel #380 #500 in Figure

---

Published as a conference paper at ICLR 2021
Table Robustness (%) of ResNet-18 trained by different defense (KWTA, SAP, PCL and our CAS)
on CIFAR-10. Avg-PGD- 100 denotes 100-step averaged PGD attack Tramer et al. 2020
Defense
Natural
FGSM
PGD-20
CW
Avg-PGD-100
EOT
&WTA
76.48
59.56
50.72
46.84
16.72
SAP
79.13
59.04
46.35
46.65
19.98
PCL
88.15
46.47
24.68
37.50
CAS
86.79
61.23
48.88
53.33
53.20
56.47
Table 2: Effectiveness of the channel suppressing operation in CAS module on CIFAR-10 with ResNet-18. 18. CAS is inserted Block4 of ResNet-18. Without suppressing means the CAS mod- ule inserted, however, the channel suppressing operation not applied during either training or testing. In this case, CAS is just simple auxiliary classifier.
Defense
Natural
FGSM
PGD-20
CW
AT
84.27
60.46
46.50
48.97
AT+CAS (without suppressing) AT+CAS (with suppressing)
83.42
59,81
44.20
46.27
86.79
61.23
48.88
53.33
4(b),, for the reason that they improve robustness mainly by introducing certain randomness into the activation, and thus can be easily attacked by some adaptive attacks (Tramer et al. 2020 Athalye
et al. 2019). PCL still frequently activates many redundant channels (e.g. channel #130 #250 in
Figure [C).. This because PCL does not directly enforce channel suppression. Different from these methods, our CAS demonstrates the most effective channel suppression. As side note, due to the dynamic thresholding, the frequency distributions should be compared within the same model between natural and adversarial examples not across different models. Within the same model, the closer the frequency distribution of the adversarial examples is to that the natural examples, the better the adversarial robustness (i.e. the adversarial accuracy is closer to the natural accuracy).
From this point view, our CAS can effectively reduce the activation frequency gaps between nat- ural and adversarial examples, producing superior robustness. The natural accuracy and robustness of these methods are reported in Table []] Due to the randomness introduced in &WTA and SAP, they not robust against average PGD (Avg-PGD) using margin loss (Tramer al. 2020) Expecta- tion Over Transformation (EOT) (Athalye et al., 2019) attacks. Our CAS training strategy does not rely on randomness, thus robust even against Avg-PGD or EOT attacks.
We next verify that explicit Channel Suppressing (CS) is indeed essential to the improved robustness of CAS. Specifically, we remove CS defined in Equation 3 from CAS, then retrain ResNet- using adversarial training with the CAS loss defined in Equation Table shows that the robustness can not be improved without explicit channel suppressing.
CAS at Different Layers. We insert the CAS module into different blocks of ResNet-18, and show the different robustness improvements in Table Intuitively, deeper layer activation are more correlated to the class prediction, thus should benefit more from our CAS training. Shallow layers, however, may suffer from inaccurate channel importance estimations. As demonstrated in Table it is indeed the case: the largest improvement is obtained when applying CAS at Block4 (e.g. after the ReLU output of Block4). The robustness can also be improved when inserting CAS into Block3 or both the Block3 and Block4 ('B ("Block3+4") though notably less significant than that at Block4.
Table Effectiveness of our CAS module at different blocks of ResNet- 18 on CIFAR-10.
Defense
Block
Natural
FGSM
PGD-20
CW
Block2
71.89
49.69
40.26
46.46
Block3
83.05
59.20
47.84
48.19
AT+CAS
Block4
86.79
61.23
48.88
53.33
Block3+4 83.77
58.32
48.27
54.62
Robustness of the CAS Module. Since our CAS module suppresses channel-wise activation ac- cording to the label (during training) the prediction (during testing), it might raise the concerns of
7

---

Published as a conference paper at ICLR 2021
whether the CAS module itself is robust or how the misclassification in CAS module would affect the final results. One observation is that, when the CAS module is inserted to deep layers that are close to the final layer of the network, it can learn to make very similar predictions with the final layer. For an empirical analysis, we test the robustness of the CAS module against different attacks in Table 4 The results indicate that our CAS module is robust by itself, leading to both higher natural accuracy and adversarial robustness. More evaluations can be found in Appendix F.5
Table 4: Robustness of defense ResNet-1 18 models trained with (+CAS) or without CAS module on CIFAR- 10 against different attacks. For +CAS models, we only apply the attack on the CAS module using the CAS loss (Equation 4). For bascline defenses, we attack the final layer of the model.
Defense
Attack Place
Natural
FGSM
PGD-20
CW
AT / +CAS
Final CAS
84.27/84.95
60.46/61.40
46.50/47.99
48.97/57.79
TRADES / +CAS
Final CAS
83,50/83.84
63.68/64.31
52.80/54.11
50.90/64.01
+
Final CAS
1//8488
63.91/65.14
52.67/54.48
49.44/66.92
5.2 ROBUSTNESS EVALUATION
In this section, we evaluate our CAS on CIFAR-10 10 (Krizhevsky et al. 2009) and SVHN Netzer jet al. 2011) datasets with ResNet-18 (He et al. 2016). We apply our CAS training strategy to several state-of-the-an art adversarial training approaches: 1) AT (Adversarial Training) (Madry et al. 2018). 2) TRADES (Zhang et al. 2019). and 3) MART (Wang et al. 2020c). We follow the default settings stated in their papers. More results 34deResNet-34- 10 (Zaoruyko & Komodakis 2016) and VGG16 (Simonyan & Zisserman 2014) can be found in [F. and F22
Experimental Settings. The training settings for CIFAR-10 are the same as Section 5.1 For SVHN, we adversarially train ResNet-1 88 using SGD with momentum 0.9, weight decay 5e-4, initial learning rate 0.01 which is divided by 10 at 75-th and 90-th epoch, and training attack PGD-10 e = 8/255 and step size 1/255) with random start.
White-box Robustness. We evaluate the robustness of all defense models against three types of white-box attacks: FGSM, PGD-20 (step size c/10) and CW oo (optimized by PGD). To fairly com- pare our method with baselines, we use adaptive white-box attack for our CAS models, i.e. the attacks are performed on the joint loss of CF and CAS. Here, we report the robustness of the models obtained at the last training epoch in Table (5) As shown in Table 5 our CAS can improve both the natural accuracy and the robustness of all baseline methods, resulting in noticeably better ro- bustness. The improvement against CW oo attack is more significant than against FGSM or PGD-20 attacks. This is because the prediction margins are enlarged by our CAS training with the channel suppression. As shown in Figure 9 (Appendix [E). the deep representations learned by CAS-trained models more compact within each class, while are more separated across different classes. This makes margin-based attacks like CW more difficult to success.
Robustness results obtained at the best checkpoint throughout the entire training process and the learning curves are provided in Appendix F33 Our CAS can also improve the robustness of the best checkpoint model for each baseline defense. Thus the improvement of our CAS is reliable and consistent, and is not caused by the effect of overfitting (Rice et al., 2020). We have also evaluated our CAS training strategy under AutoAttack (Croce & Hein 20206) and different attack perturbation budget in Appendix and F55
Black-box Robustness. We evaluate the black-box robustness of CAS and the baseline methods against both transfer and query-based attacks. For transfer attack, the adversarial examples are generated on CIFAR-10/SVHN test images by applying PGD-20 and CW oo attacks on a naturally trained ResNet-50. For query-based attack, we adopt N Attack (Li et al.] 2019). Since N AAttack requires a lot of queries, we randomly sample 1,000 images from -O/SA-- test set and limit the maximum query to 20,000. We test both black-box attacks the models obtained at the last training epoch. The results are reported in Table 6 For both transfer and qusry-based attacks, our CAS can improve the robustness of all defense models by considerable margin. Especially against the A Attack, our CAS training strategy can improve AT, TRADES and MART by ~ 30% 40%. One reason why CAS is particularly more effective against N Attack is that V AAttack
8

---

Published as a conference paper at ICLR 2021
Table 5:
White-box robustness (aceuracy (%) various white-box attacks) CIFAR-10 and
SVHN, based on the last checkpoint of ResNet-18. '+CAS" indicates applying our CAS training strategy existing defense methods. The best results are boldfaced.
SVHN
CIFAR-I
Defense
Natural
FGSM
PGD-20
CW
Natural
FGSM
PGD-20
CW
AT
89.62
65.09
42.55
50.96
84.27
60,46
46.50
48.97
AT+CAS
90.39
67.51
51.98
53.53
86.79
61.23
48.88
53.33
TRADES
91.16
69.85
50.90
50.85
83.50
63.68
52.80
50.90
TRADES+CAS
91.69
70.97
55.26
60.10
85.83
65.21
55.99
67.17
MART
91.16
67.31
48.72
50.52
82.16
63.91
52.67
49.44
MART+CAS
93.05
70.30
51.57
53.38
86.95
63.64
54.37
63.16
utilizes similar margin objective function CW oc. which be effectively blocked by channel suppressing.
Table 6: Black-box robustness (accuracy (%) various black-box attacks) of ResNet- 18 on SVHN and CIFAR-10. '+CAS' indicates applying our CAS training strategy to existing defense methods. The best results are boldfaced.
SVHN
CIFAR-I 10
Defense
PGD-20
CW
N NAttack
PGD-20
CW
N NAttack
AT
64.42
70.60
40.36
79.13
79.87
45.05
AT+CAS
65,50
72.35
75.72
85.80
86.54
83.32
TRADES
67.81
74.43
44.16
78.18
78.95
49.25
TRADES+CAS
68.48
75.66
81.82
84.77
85.54
79.42
MART
66.85
73.90
41.66
76.93
77.49
49.05
MART+CAS
68.45
75.45
80.12
85.68
86.59
81.93
CONCLUSION
In this paper, investigated intermediate activation of deep neural networks (DNNs) from novel channel-win -iiee perspective, the context of adversarial robustness and adversarial training. We high- light two characteristics of the channels of adversarial activation: higher magnitude, and 2) more uniform activation frequency. We find that standard adversarial training improves robustness by addressing the first issue of the higher magnitude, however, fails address the second issue of the more uniform activation frequency. To overcome this, we proposed the Channel-wise Activation Suppressing (CAS), which dynamically learns the channel importance and leverages the learned channel importance to suppress the channel activation in the training phase. When combined with adversarial training, we show that, CAS can train DNNs that inherently suppress redundant channels from being activated by adversarial examples. Our CAS is simple but generic training strategy that can be easily plugged into different defense methods to further improve their robustness, and can be readily applied to robustify the intermediate layers of DNNs.
ACKNOWLEDGEMENT
Yisen Wang is partially supported by the National Natural Science Foundation of China under Grant 62006153, and CCF-Baidu Open Fund (OF2020002). Shu-Tao Xia is partially supported by the National Key Research and Development Program of China under Grant 2018 1800204, the National Natural Science Foundation of China under Grant 61771273, the R&D Program of Shen- zhen under Grant B0YT20000000811220440
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICMI., 2019.

---

Published as a conference paper at ICLR 2021
Yang Bai, Yan Feng, Yisen Wang. Tao Dai, Shu-Tao Xia, and Yong Jiang. Hilbert- based generative defense for adversarial examples. In ICCV, 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In S&P. 2017.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. ICML, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML, 2020b.
Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Siwei Li, Li Chen, Michael E Kounavis, and Duen Horng Chau. Compression the rescue: Defending from adversarial attacks across modalities. In KDD, 2018.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial de- fense. ICLR, 2018.
Ranjie Duan, Xingjun Ma, Yisen Wang, James Bailey, A Kai Qin, and Yun Yang. Adversarial camouflage: Hiding physical-worlbe world attacks with natural styles. In CVPR, 2020.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-worid attacks on deep learning visual classification. In CVPR, 2018.
Chris Finlay, Adam Oberman, and Bilal Abbasi. Improved robustness to adversarial examples using lipschitz regularization the loss. arXiv preprint arXiv:1810.000 1810.0095. 2018.
Angus Galloway, Anna Golubeva, Thomas Tanay, Medhat Moussa, and Graham W Taylor. Batch normalization a cause of adversarial vulnerability. ICML Workshop, 2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. Computer Science, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.
Andrew lyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In NeurIPS, 2019.
Linxi Jiang, Xingjun Ma, Zejia Weng, James Bailey, and Yu-Gang Jiang. Imbalanced gradients: A new cause of overestimated adversarial robustness. arXiv preprint arXiv:2006. 13726, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world, 2016.
Yandong Li, Lijun Li, Liqiang Wang, Tong Zhang, and Boqing Gong. Nattack: Learning the dis- tributions of adversarial examples for an improved black-box attack on deep neural networks. ICML, 2019.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In CVPR, 2018
10

---

Published as a conference paper at ICLR 2021
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. ICLR, 2017.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In ICLR, 2018.
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu. Under- standing adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition, 10:107332, 2021.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9Novv: 2605, 2008
Divyam Madaan and Sung Ju Hwang. Adversarial neural pruning with latent vulnerability suppres- sion. ICML, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. ICLR, 2018.
Aamir Mustafa, Salman Khan, Munawar Hayat, Roland Goecke, Jianbing Shen, and Ling Shao. Adversarial defense by restricting the hidden space of deep neural networks. In ICCV, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In S&P, 2016.
Adnan Siraj Rakin, Jinfeng Yi, Boqing Gong, and Deliang Fan. Defend deep neural networks against adversarial examples via fixed anddynamic quantized activation functions. arXiv preprint arXiv:1807.06714. 2018.
Leslie Rice, Eric Wong, and J Zico Kolter. Overfitting in adversarially robust deep learning, ICML, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. Computer Science, 2014.
Abhishek Sinha, Vineeth N Balasubramanian, Harshitha Machiraju, et al. Harnessing the vulnera- bility of latent layers in adversarially trained models. AAAI, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. ICLR, 2014.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002.088 08340007.7847 2020.
Bao Wang, Alex T Lin, Zuoqiang Shi, Wei Zhu, Penghang Yin, Andrea L Bertozzi, and Stanley Os- her. Adversarial defense via data dependent activation function and total variation minimization. Inverse Problems and Imaging, 2020a.
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuei Hu, Ji Liu, and Zhangyang Wang. Once-for- all adversarial training: In-situ tradeoff between robustness and accuracy for free. arXiv preprint arXiv:2010. 11888 2020b.
Xin Wang, Jie Ren, Shuyun Lin, Xiangming Zhu, Yisen Wang, and Quanshi Zhang. A unified approach to interpreting and boosting adversarial transfcrability. In ICLR, 2021.
Yisen Wang, Xuejiao Deng, Songbai Pu, and Zhiheng Huang. Residual convolutional ctc networks for automatic speech recognition. arXiv preprint arXiv: 170.00770,,,7793, 2017.
Yisen Wang, Xingjun Ma James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.

---

Published as a conference paper at ICLR 2021
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020c
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In ECCV, 2016.
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In ICLR, 2020a.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener-
alization. NeurIPS, 33, 2020b.
Chang Xiao, Peilin Zhong, and Changxi Zheng. Resisting adversarial attacks by k-winners-take-all. ICLR, 2020.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In CVPR, 2019.
Kaidi Xu, Sijia Liu, Gaoyuan Zhang, Mengshu Sun, Pu Zhao, Quanfu Fan, Chuang Gan, and Xue Lin. Interpreting adversarial examples by activation promotion and suppression. arXiv preprint 10X00::904.02057 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. BMVC, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. ICML, 2019.
Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In NeurIPS, 2018.
12

---

Published as a conference paper at ICLR 2021
A ALGORITHM OF CAS TRAINING
Algorithm Robust Training with CAS.
Input: Training data { 1:1,,,..... DNN F(0), CAS modules with parameters M, maximum training epochs
Output: Robust Network JF
1: for in [1, 2, 7" do
2: for minibatch {z1.... 2b) do
3: Generate ad adversarial examples using PGD attack on Equation 5
4: Compute the CAS loss in Equation 4 using the channel-wise activation f
5: Reweight the original features f' f My using the parameters M in CAS
6: Forward with the adjusted f' and compute the CE loss at the output layer
7: end for
8: Optimize all parameters (8, by Equation 5 Jusing gradient descent
9: end for
B A SUMMARY OF THE ADVERSARIAL LOSS FUNCTIONS USED WITH CAS
When combined with our CAS, the training loss is combination of the original adversarial loss and our CAS loss. Table defines the exact loss functions used for AT (Madry et al., 2018), TRADES Zhang al. 2019). MART (Wang et al. 2020c) and their CAS enhanced versions. Here, assume the CAS module is attached to number of layers of the network. p" and M denote the prediction score and weights of the auxiliary classifier in the s-th CAS module, respectively.
Table 7: A summary of the loss functions used for standard adversarial training (AT), TRADES, MART, and their corresponding versions with our CAS ('+CAS").
Defense
Loss Function
Chip(a' 0), + 3 CE(p (ær M)).)
+CAS
TRADES S
CE(p(æ, 0), A (L(p(z 0) [p(p(
CF (an, M.)) +B ( - KT (a,, M") ( (.)))
MART
BCE(p(a' 9).2) + A (L(p(æ, 0)]pp(( a', M)) (1 (a, 8))
+CAS
S
BCE (ar, (m',0, M.). +B E. KL.(" (m, 0, M (ar""("",, M²)) " (1 (,(,,,,M"))
C CHANNEL-WISE ACTIVATION SUPPRESSING ON MORE DATASETS AND DEFENSE MODELS
Here, we visualize the channel suppressing effect of our CAS training strategy on more defense models: TRADES (Zhang et al., 2019) and MART (Wang al. 2020c). We train ResNet-18 (He et al. 2016) on CIFAR-IO (Krizhessyy et al. 2009). The CAS modules are attached to the final block of 18.
Figure illustrates the channel activation frequencies of the original defense models and their im- proved versions with our CAS. can be observed, our CAS can suppress the redundant channels consistently in both defense models. This restricts the channel-wise activation to be more class- correlated. More importantly, the channel activation are suppressed similar frequency distribu- tion between the natural and the adversarial examples.
We further compare the channel-wise activation natural versus adversarial examples on SVHN and ImageNet in Figure 6 and Figure 7 For SVHN, we choose examples of class O and construct the adversarial examples by PGD-20 with = 8/255. We count the channel activation frequency of naturally trained, adversarially trained and CAS trained models. As shown in Figure neurons #200 #500 in Figure [6())aam and neurons #250 #400 in Figure]6(b) are frequently activated by adversarial examples while Figure 6(c) our CAS can effectively suppress these redundant activation.
13

---

Published as a conference paper at ICLR 2021
For ImageNet, it is extremely time consuming to perform adversarial training, we adopt the stan- dard pre-trained ResNet-157 152 from torchvision.models | and an adversarially trained ResNet-152 from the Github repository We select 5,900 dog images from the validation set of ImageNet (class #151-#269) and generate the adversarial examples using PGD-30 (c=16/255, a=1/255). We count the channel activation the penultimate layer of ResNet-152. can be observed in Figure 7(a) that adversarial examples activate more uniformly than natural examples. While in Figure ((b) adver- sarially trained models can align the activation distribution of the natural examples with that of the adversarial examples to some extent, however, the activation frequencies of the adversarial exam- ples are still higher than that of the natural examples. This indicates that there are still redundant channels and non-robust features in this adversarially trained ResNet-152 model.
netural ourppee
sutural champes
FUCLED coumpkca
natural exampis
Number of Altivationn
800
odre lal ciamples
Activation
1000
acverrsrrial exumples
200
advcrsar coImp
302
adversand chanplss
400
-00
400
400
200
400
100
230
400
150
300
402
202
400
302
Cha
Channel
(a) TRADES
(b) TRADES
(c) MART
(d) MART+CAS
Figure 5: The distributions of channel activation frequency of both natural and adversarial examples in different defense models (e.g. TRADES and MART). The frequency distribution gap between natural and adversarial examples is effectively narrowed down by our CAS training, and the redun- dant channels (channel #200 #512) are significantly suppressed by CAS.
Number of Activation
1500
adversarial examples
natural examples
Number of Activation
1500
adversarial examples
natural
Number of Activattio
1500
adversarial examples
natural examples
1250
1250
2250
1000
1000
2000
750
750
750
500
500
500
250
250
250
0
100
200
300
400
300
0
100
200
300
400
500
0
100
200
300
400
500
Channel
Channel
Channel
(a) 18 (STD)
(b) RessNet-18 (ADV)
(c) RRsNeet-1 (our CAS)
Figure 6: The distributions of channel activation frequency of both natural and PGD-20 adversarial
examples for 18 on SVHN
6000
6000
natural examples adversarial examples
natural examples
Number Activvation
5000
Number Activaatio
5000
adversarial examples
4000
4000
3000
3000
2000
2000
1000
1000
0
500
1000
1500
2000
0
500
1000
1500
2000
Channel
Channel
(a) ResNet-152 (STD)
(b) ResNet- 152 (ADV)
Figure 7: The distributions of channel activation frequency of both natural and PGD-30 adversarial examples for ResNet-152 on ImageNet.
http:///wweeaaaaoooooccccccllees http://ithhcceomeecooeeeceeeeeee Trainngn
14

---

Published as a conference paper at ICLR 2021
FSSH
PGD2a CW
15
005a
Figure 8: Robustness of AT+CAS against white-box attacks FGSM, PGD-20 and CW under dif- ferent 3. As 3 increases, the robustness also improved, especially against the CW oW attack.
D SENSITIVITY OF CAS TO PARAMETER B
As mentioned in Section 4 the parameter 33 on the CAS loss controls the suppressing strength. To test the sensitivity of CAS training under different 3, we insert a CAS module Block4 of ResNet- 18, and train the network on CIFAR- using AT+CAS under B E [0, 0.5. 1, 2, 5, 10, 20). Note that = indicates the standard adversarial training (AT). We test the robustness of the models in white-box setting against FGSM, PGD-20 and CW cz. As shown in Figure 8 the models trained with larger are generally more robust, especially against the CW oc attack. This is because larger increases the strength of channel suppressing, leading to larger inter-class margins. As we further show in Figure the representations learned with CAS are more separated between different classes, and are more compact within the same class. This tends to increase the difficulty of margin- based attacks like woxte-box CW attack and black-box / attack.
E CAS IMPROVES REPRESENTATION LEARNING
The representations learned by natural adversarial training with without CAS strategy illustrated in Figure The t-ŠNE (Maaten & Hinton 2008) 2D embeddings are computed on deep features extracted at the penultimate layer of ResNet- 18 on CIFAR-10. As can be observed, CAS training improves representation learning for both natural training and adversarial training. This is largely attributed to the strong channel suppressing capability of our CAS training. Channel suppressing helps learn high-quality representations with high inter-class separation and intra-class compactness. Interestingly, our CAS training can even improve the performance natural training from 92.75% to 94.56%. This implies that our CAS is a generic training strategy that can benefit both model training and representation learning. Although CAS is not direct regularization technique, it can achieve a similar representation regularization effect existing representation enhancing techniques like the center loss (Wen et al. 2016).
F MORE EXPERIMENTAL RESULTS
F.1 WIDERESNET RESULTS ON CIFAR-10
The white-box robustness of WideResNeet34-10 ZZaoruykoo & Komodakis 2016) models trained using AT, AT+CAS, TRADES, TRADES+CAS, MART and MART+CAS are reported in Table g We attach the CAS module to the last two convolution layers of the network, and set 3 = 2. The training settings are the same as used for ResNet-18, except that, here we use weight decay 5e-4. The 'best' and 'last' results indicate the best and last checkpoints, respectively. For our CAS, we generate the attacks using the same adaptive white-box attacks as used in Section 5.2 The robustness of AT, TRADES and MART can be consistently improved by CAS training. The most improvements are achieved on AT. These results confirm that our CAS training can lead to consistent improvements to different adversarial training methods more complex models.
15

---

Published as a conference paper at ICLR 2021
pare
der
02g
65g
E2
Duck
(a) ADV (Natural Examples)
(b) ADV+CAS (Natural Examples)
Mane
CX:
200r
-22
1002
Fosse
ahp
EucC
-40
(c) ADV (Adversarial Examples)
(d) ADV+CAS (Adversarial Examples)
114
bid
tc:
Eng
[ng
Tore
(e) STD (Natural Examples)
(1) STD+CAS (Natural Examples)
Figure 9: The t-SNE 2D embeddings of deep features extracted at the penultimate layer of ResNet- 18 models trained using natural ('STD') or adversarial training ('ADV') on CIFAR-10. The em- beddings are shown separately for natural versus adversarial examples. Our CAS training can help improve inter-class separation and intra-class compactness for both types of training.
Table 8: White-box robustness (accuracy (%) on various white-box attacks) of WideRessNet-34-11 on CIFAR-10 10 dataset. '+CAS' indicates applying our CAS training to existing defense methods. The best results are boldfaced.
CIFAR-10
Defense
Natural Best
FGSM
PGD-20
CW
Last
Best
Last
Best
Last
Best
Last
AT
84.16
84.23
66.62
61.27
53.75
49.53
50.52
47.68
AT+CAS
88.25
89.06
67.14
66.98
56.28
53.22
58.54
54.77
TRADES
86.23
86.41
66.41
65.63
54.42
52.64
53.45
52.43
TRADES+CAS
87.07
87.15
66.92
66.15
55.43
53.15
61.46
57.07
MART
84.09
85.69
67.24
66.41
57.56
54.49
54.42
52.55
MART+CAS
87.87
89.20
68.09
67.96
58.24
54.95
61.48
57.11
16

---

Published as a conference paper at ICLR 2021
F.2 VGG16 RESULTS ON CIFAR-10
Similar to the above WideResNet experiments, here we report the results of VGG16 (Simonyan & Zisserman 2014) CIFAR-10 in Table For VGGI6, we attach the CAS module its last three convolution layers, and set 8 = 3. We can see that our CAS training can enhance the robustness of 'best' models by remarkable margin 7% 90%.0. Note that the accuracy of natural training is also improved by considerable margin. Compared with complex models like WideResNet, our CAS training even more beneficial for small capacity models (e.g. VGG16).
Table 9: White-box robustness (accuracy (%) on various white-box attacks) of VGG16 on CIFAR- 10. *+CAS' indicates applying our CAS training strategy to existing defense methods. The best results are boldfaced.
CIFAR-10
Defense
Natural
FGSM
PGD-20
CW
Best
Last
Best
Last
Best
Last
Best
Last
AT
70.32
70.38
51.87
51.85
42.23
42.01
43.63
43.81
AT+CAS
82.25
82.61
59.77
57.56
49.22
42.73
53.03
49.68
TRADES
74.98
76.67
52.10
52.99
41.53
41.13
45.50
45.25
TRADES+CAS
83.45
83.25
61.36
61.51
49.86
49.56
52.60
52.47
MART
71.70
72.20
54.72
54.90
46.55
46.53
44.52
44.72
MART+CAS
81.53
83.28
61.89
61.70
52.01
49.13
50.97
49.10
F.3 ROBUSTNESS RESULTS AT THE BEST CHECKPOINT
We report the white-box robustness of the 'best' (iee the best checkpoint) models of ResNet-18 on SVHN and CIFAR-IO Table I0, as a supplementary to the 'last' (i.e. the last checkpoint) results in Table Again, our CAS can also improve the robustness of the 'best' checkpoint models consistently. This proves that our CAS can improve both the robustness and the natural accuracy throughout the entire training process.
Table 10: White-box robustness (accuracy (%) various white-box attacks) of ResNet- 18 on CIFAR-10 and SVHN on the to existing defense methods. The best results are boldfaced.
best checkpoint. '+CAS' indicates applying our CAS training strategy
SVHN
CIFAR-10
Defense
Natural
FGSM
PGD-20
CW
Natural
FGSM
PGD-20
CW
AT
91.41
69.52
53.07
50.38
84.20
63.32
52.01
50.97
AT+CAS
92.69
71.83
54.78
54.13
86.72
62.92
54.40
62.72
TRADES
90.40
71.22
57.71
54,49
82.68
63.15
53.05
50.46
TRADES+CAS
91.41
71.39
59.32
66.94
84.75
63.94
57.20
67.77
MART
87.37
68.58
57.72
51.48
78.90
63.29
54.92
50.31
MART+CAS
91.04
72.32
60.29
57.05
85.99
64.23
58.21
69.91
F.4 ROBUSTNESS AGAINST ALTOATTACK
We report the white-box AutoAttack (Croce Hein 2020b) evaluation results of ResNet-18 on CIFAR-10 10-1 in TT AutoAtttcck is an ensemble of two proposed Auto-PGID attacks and the other two complementary attacks (Croce Hein 2020a). AutoAttack has been shown can produce more accurate robustness evaluations on wide range of adversarial training defenses. As shown in Table II although less significant than against regular attacks like PGD and CW, CAS with proper training still improve the robustness of AT, TRADES and MART by noticeable margin. This confirms that the improvements brought by our CAS training is 'real' and substantial, rather than obfuscated gradients nor improper evaluation.
17

---

Published as a conference paper at ICLR 2021
Table 11: White-box robustness (accuracy (%) against AutoAttack) of ResNet-18 CIFAR-I0.
+CAS indicates applying our CAS training strategy to existing defense methods. The best results boldfaced.
AutoAttack
Defense
Best
Last
AT
46.58
41.90
AT+CAS
47.40
44.74
TRADES
48.28
47.46
TRADES+CAS
48.40
48.38
MART
47.06
45.45
MART+CAS
48.45
46.39
F.5 MORE ROBUSTNESS EVALUATIONS OF CAS
The robustness of the CAS module further evaluated in this section. By producing large margins, the CAS module and the channel suppression operation make more boundary adversarial amples be correctly classified than that in the baseline models. However, this may also increase the risk of causing imbalanced margin and imbalanced gradients. To test this, evaluate CAS against a recent Margin Decomposition (MD) attack (Jiang et al. 2020). We use the default setting of MD attack as stated in the original paper. The results are reported in Table 12] where it shows our CAS is robust against the MD attack. We also test our CAS against PGD-20 with step size e/ 10 and various attack strengths. This result presented in Table I3 The robustness of CAS decreases with the increase of which proves that the robustness of CAS is not a result of obfuscated gradients.
Table 12: Robustness of CAS against Margin Decomposition (MD) attack. This experiment was conducted with ResNet-18 on CIFAR-10.
Adversarial Loss Type
Defense
MD (CE)
MD (CAS)
MD (CE+CAS)
AT
43.35
AT+CAS
46.27
49.56
TRADES
48.47
CAADES
53.51
55.53
MART
46.99
MART+CAS
49.89
55.75
Table 13: Robustness (%) of CAS with ResNet- on CIFAR- 10 against PGD-20 under different c.
Attack strength
2/255
4/255
8/255
16/255
32/255
AT+CAS
79.15
70.26
48.88
16.64
2.48
CRADES+CAS
80.09
72.69
55.99
27.94
5.10
MART+CAS
81.01
73.56
54.37
20.49
2.19
F.6 CHANNEL ACTIVATION FREQUENCY UNDER DIFFERENT THRESHOLDS
In Figure 2 we have shown the activation frequency of standard training (STD). adversarial training (AT) and CAS-based adversarial training (AT+CAS) on a ResNet-18 model with the threshold of 1%. That is, channel is determined as activated if its activation count is larger that 1% of the maximum activation count over all 512 channels. In Figure 10, we show the distribution of activation frequency under different thresholds (e.g. 0.5%. 1%, 5%). Larger threshold will truncate more low-count activation. The plots show that different thresholds will introduce some noise to the distributions but the general patterns do not change.
18

---

Published as a conference paper at ICLR 2021
examples adversarial examples
natural examples pormppe
natural examples adversarial examples
Number of Activation
Number of Activation
800
Number of Aut rediioo
800
800
400
400
400
200
200
100
200
300
500
100
200
400
500
100
300
500
Channel
Charnel
Channel
(a) 18 (AT 0.5%
(b) ResNet- 18 (AT 1%)
(c) ResNet- 18 (AT 5%)
1000
1000
1000
examples
natural examples soampea
examples
Number of Activatton
800
adversarial examples
Number of Activatton
800
Number of Activation
800
adversarial examples
600
400
400
400
200
0
100
200
300
400
500
100
200
300
400
502
100
200
300
400
500
Channel
Charnel
Channel
(d) ResNet-18 (AT+CAS 0.5%)
(c) ResNet-18 (AT+CAS 1%)
(f) ResNet-18 (AT+CAS 5%)
Figure 10: The activation frequency (y-axis) of channel-wise activation at the penultimate layer of adversarial training (AT) and our CAS-based adversarial training (AT+CAS) for ResNet-18 on CIFAR-10. 10. The activation frequencies are visualized with respect different thresholds (e.g. 0.5%,
1% and 5%).
19


Reviews:
[{"review_id": "uC4zX3qy49r", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper investigates the adversarial robustness from the activation perspective. Specifically, the authors analyzed the difference in the magnitude and distribution of activation between adversarial examples and clean examples: the activation magnitudes of adversarial examples are higher and the activation channels are more uniform by adversarial examples. Based on the above interesting findings, the authors claim that different channels of intermediate layers contribute differently to the class prediction and propose a Channel-wise Activation Suppressing (CAS) method to suppress redundant activations, which can improve the DNN robustness. \n\nSome highlights in this paper:\n+ The CAS strategy is simple and can be easily applied to existing models. Combining CAS with the existing adversarial training methods leads to better DNN robustness.\n+ The experiments are well-conducted and convincing. The authors not only provided ablation experiments to verify the effectiveness of CAS, but also provided both the performance of the last epoch and the performance of early stop, which confirmed that CAS can improve the DNN robustness.\n+ The paper is well-written and the idea is easy to follow.\n\nHowever, there are some downsides. I’d like more details about:\n- Adversarial training inhibits the magnitude of activation, what is the connection between this and network robustness?\n- The closer the activation distribution of the adversarial example is to that of the clean example, the better the robustness of the network. It would be good to provide more discussions and explanations here.\n\nOverall the paper is easy to understand and interesting.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "mk2Fgx2cqCk", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors studied the behavior of adversarial examples from the channel view of activations, which is very novel. They focused on the magnitude and frequency of activations and found that state-of-the-art adversarial defense (adversarial training) only addressed the magnitude issue but the frequency distribution issue remains. This provided a novel perspective for us to understand why state-of-the-art adversarial training method works to a certain extent but not so good. Then, the authors proposed a Channel-wise Activation Suppressing (CAS) to address the frequency distribution to further improve the adversarial robustness. CAS is generic, effective, and can be easily incorporated into many existing defense methods. \n\nPros:\n1. The authors studied adversarial examples from a new perspective of channels in activations. Previous works focusing on activations usually assumed that each channel is of equal importance, while the authors focused on the relationship between channels. From two aspects of activation magnitude and frequency, the authors found two novel characteristics of adversarial examples: adversarial examples have higher activation magnitude and more uniformly activated channels compared to natural examples. The findings were convincingly evaluated on different neural network architectures and different training methods. This hints at a very interesting phenomenon.\n\n2. The proposed method is generic. The authors found that the activated channels are still uniform under adversarial training, that is, some redundant and low contributing channels are still activated. To suppress the redundantly activated channels, the authors proposed Channel-wise Activation Suppressing (CAS) training strategy. It dynamically learns and incorporates the channel importance (to the class prediction) into the training process. The motivation is very clear and the method is easy to follow. More importantly, CAS can be widely applied to strengthen existing adversarial training approaches since it suppresses those less important channels.\n\n3. Lots of experiments are provided to understand and evaluate the proposed methods. The experiments covered lots of aspects, including channel suppressing effect of CAS, representation learning, ablation studies, and extensive robustness evaluation on white-box and black-box attacks. The authors also tested the adaptive attacks, strongest auto-attack, and the optimization-based black-box attack, which definitely convinced me of the effectiveness of the proposed method. \n\nOverall, the paper hints at an interesting phenomenon and inspires an in-depth understanding of adversarial training. The proposed method is elegant and generic. The empirical evidence is solid and extensive. \n\nCons:\n1. How does the activation threshold effect Figure 2?\n2. In the testing phase, the predicted class of the auxiliary classifier is used for the channel importance. Is it vulnerable to attacks? if the predicted label is incorrect, how it will affect the final performance?\n3. How well the auxiliary classifier works. With the limited information from the output of GAP, it is likely that the classifier performs poorly, and thus results in bad channel importance weighting.\n4. CAS could both improve natural acc and adversarial robustness, why CAS could achieve this both? and how is the overhead of CAS?", "extra_scores": {}, "extra_text": {}}, {"review_id": "XFTdrbWPKCA", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the use of channel suppression in improving robustness to adversarial examples. The authors make a convincing illustration in section 3 on how adversarial examples tend to activate more channels compared to natural examples, and adversarial training is not effective in reducing them. This provides a convincing motivation to their design of the Channel-wise Activation Suppression (CAS) module. Their CAS module is also effective in improving adversarial robustness when used in conjunction with different adversarial defense methods, including adversarial training, TRADES, and MART. \n\nI think this paper is of high quality, but I do have several questions on the details: \n1. In section 4.1 there is a difference in how the mask M is produced in training and test phase. How important is it to have the correct y available for the mask, as oppose to \\hat{y} from the channel predictions? For example it might be difficult to predict the target class from the low-level features (e.g. block 2 channel features), leading to inaccurate \\hat{y} for channel suppression. Could this be a reason for lower performance of inserting CAS into block 2 in Table 3?  \n2. Just to confirm, are both losses (CE and CAS) in Eq 5 taken into account in the generation of adversarial perturbations with FGSM and PGD? \n3. In Table 2, what does it mean to have CAS without channel suppression? Is it effectively just a CNN with predictions made from features in different layers? \n4. Do the authors have any intuitions on why having CAS module alone on Block 4 is better than having it on both Block 3 and 4 in general? \n\nI am leaning towards acceptance of this paper if the authors can address the above questions sufficiently. \n\nAfter author response: the authors have sufficiently addressed my questions and also the other reviewers' questions. I am keeping my score of acceptance. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "DTAk5yTsn08", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\nSummary:  \n\nThis paper uncovers interesting phenomenons of adversarial training, i.e., more uniformly distributed adversarial data activations than those of natural data. To force the behaviors (in this paper, channels activations) of adversarial data to be similar to those of natural data, the authors explicitly suppress the redundant channels by reweighing the channel activations. \n\n##########################################################################\nReason for score. \n\nOverall, I vote for accepting. I like the uncovered phenomenons of larger and more uniformly distributed activations of adversarial data than those of natural data. \nTechnically, this paper proposed effective training strategies (i.e., channel-wise activation suppressing (CSA)) to enhance adversarial training. \n\n##########################################################################\nPros: \n\n1 This paper provides the understanding of adversarial training from the channel activation perspective, showing that adversarial training can reduce the magnitude of the activation of the adversarial data, but fail to break the uniform activations by the adversarial data. \n\n2 Figure 2 shows the efficacy of the proposed CSA methods for breaking the adversarial data's uniform activations. Compared with standard adversarial training, CSA can further suppress the redundant channel activations. \n\n3 The experiment evaluations are comprehensive, showing CSA strategies' efficacy across various adversarial training methods, network structures, and attack methods. \n\n##########################################################################\nCons: \n\n1 What is the side effect of redundant channel activations? Specifically, what is the side effect of uniform activations of the adversarial data? Would you mind explaining more?\n\n2 Although CSA successfully suppresses the redundant channels of the adversarial data, CSA also seems to suppress the activations of natural data? Is this the reason for the improvement on natural accuracy?\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on two new characteristics of adversarial examples from the channel-wise activation perspective, namely the activation magnitudes and the activated channels. The philosophy behind sounds quite interesting to me, namely, suppressing redundant activations from being activated by adversarial perturbations. This philosophy leads to a novel algorithm design I have never seen, i.e., Channel-wise Activation Suppressing (CAS) training strategy.

The clarity and novelty are clearly above the bar of ICLR. While the reviewers had some concerns on the significance, the authors did a particularly good job in their rebuttal. Thus, all of us have agreed to accept this paper for publication! Please carefully address all comments in the final version.
Decision: Accept (Spotlight)