Towards Understanding Dual BN In Hybrid Adversarial Training
Authors: Chenshuang Zhang
Chaoning Zhang
Kang Zhang
Axi Niu
Junmo Kim
In So Kweon
Venue: Accepted by TMLR ()

Abstract: There is a growing concern about applying batch normalization (BN) in adversarial training (AT), especially when the model is trained on both adversarial samples and clean samples (termed Hybrid-AT). With the assumption that adversarial and clean samples are from two different domains, a common practice in prior works is to adopt Dual BN, where BN$_{adv}$ and BN$_{clean}$ are used for adversarial and clean branches, respectively. A popular belief for motivating Dual BN is that estimating normalization statistics of this mixture distribution is challenging and thus disentangling it for normalization achieves stronger robustness. In contrast to this belief, we reveal that disentangling statistics plays a less role than disentangling affine parameters in model training. This finding aligns with prior work (Rebuffi
et al., 2023), and we build upon their research for further investigations. We demonstrate that the domain gap between adversarial and clean samples is not very large, which is counter-intuitive considering the significant influence of adversarial perturbation on the model accuracy. We further propose a two-task hypothesis which serves as the empirical foundation and a unified framework for Hybrid-AT improvement. We also investigate Dual BN in test-time and reveal that affine parameters characterize the robustness during inference. Overall, our work sheds new light on understanding the mechanism of Dual BN in Hybrid-AT and its underlying justification.

Published in Transactions on Machine Learning Research (03/2024)
Towards Training
Understanding Dual BN In Hybrid Adversarial
Chenshuang Zhang
2cs15@kaist.com oc..
KAIST
Chaoning Zhang
chaoringghaug999000gmma com
Kyung Hee University
Kang Zhang
shangkang@haist.oc..
KAIST
Axi Niu Northwestern Polytechnical University
wuzpmamil.mupp.ucduuu edu..cm
Junmo Kim
junmo.kim@kaist.co
KAIST
In So Kweon KAIST
iskweon W7@.kistt.k..... ac.
Reviewed on OpenReview:
https: // openreview. net/ forum? id= PQKHMSE4SH
Abstract
There is growing concern about applying batch normalization (BN) in adversarial training (AT), especially when the model is trained both adversarial samples and clean samples (termed Hybrid-AT). With the assumption that adversarial and clean samples are from two different domains, a common practice in prior works is to adopt Dual BN, where BN and BNetean are used for adversarial and clean branches, respectively. popular belief for motivating Dual BN is that estimating normalization statistics this mixture distribution challenging and thus disentangling it for normalization achieves stronger robustness. In contrast this belief, we reveal that disentangling statistics plays less role than disen- tangling affine parameters in model training. This finding aligns with prior work (Rebuffi et al. 2023), and we build upon their research for further investigations. We demonstrate that the domain gap between adversarial and clean samples is not very large, which is counter-intuitive considering the significant influcnce of adversarial perturbation on the model accuracy. We further propose a two-task hypothesis which serves as the empirical foundation and a unified framework for Hybrid-AT improvement. We also investigate Dual BN in test-time and reveal that affine parameters characterize the robustness during infer- ence. Overall, our work sheds new light on understanding the mechanism of Dual BN in Hybrid-AT and its underlying justification.
1 Introduction
Adversarial training (AT) Ganin et al. 2016 Madry et al. 2018 Shafahi et. al. 2019 Andriushchenko
& Flammarion 2020 Bai et al. 2021) that optimizes the model on adversarial examples is a time-tested
and effective technique for improving robustness against adversarial attack (Qiu et al. 2019 Xu Yang.
* *Correspondngg author

---

Published in Transactions on Machine Learning Research (03/2024)
2020
Dong et al. 2018 Zhang et al.
Beyond classical AT (also termed Madry-AT) Madry et al, common AT uetup is to train the model on both adversarial samples and clean samples (termed
2021b).
2018
Hybrid-AT) (Goodfellow et al.] 2015
Kannan et al.] 2018 Xie & Yuille] 2020 Xie et al. 2020a). Batch
normalization (BN) (Ioffe & Szegedy 2015 Santurkar et al. 2018 Bjorck et al. 2018 Li et al. 2017) has
become a de facto standard component. in modern deep neural networks (DNNs) (He et al. 2016: Huang
ct al. 2017 Zhang et al. 2019a; 2021a), however, there is a notable concern regarding how to use BN in the Hybrid-AT setup. This concern mainly stems from Xie & Yuille| 22200); Xie et al. (2020a), which claims the adversarial and clean samples are from two different domains, and thus a separate BN should be used for each domain. This technique applying different. BN for different domains has been adopted in multiple works with different names, e.g., Dual BN (Jiang et al.) 2020 Wang et al. 2020; 2021) and mixture BN (Xie & Yuille, 2020). With different names, however, they refer to the same practice of adopting BNado and BNetean for adversarial and clean samples, respectively. To avoid confusion, we use Dual BN for the remainder of this work.
Despite the increasing popularity of Dual BN, the mechanism of how Dual BN helps Hybrid-AT remains not fully clear. Toward a better understanding of this mechanism, we revisit a long- held belief in Xie & Yuille (2 1: Xie ct al. (0020a). Specifically, it justifies the nccessity of Dual BN in Hybrid-AT with the following claim (quoted from the abstract of Xie & Yuille (020))):
" "Estimating normalization statistics of the miature distribution is challenging and "disentangling the misture distribution for normalization, i.e., applying seporate BNs to clean and adversarial images for statistics estimation, achieves much stronger robustness."
The above claim (Xie & Yuille 2020) emphasizes the necessity of discntangling the normalization statistics (NS) in Hybrid-AT. The underlying motivation for the above claim is that BN statistics calculated on the clean domain are incompatible with training the model on the adversarial domain, and vice versa. Therefore, Hybrid-AT with a single BN suffers from such incompatibility with BN statistics caleulated from the mixed distribution, while Dual BN can avoid the incompatibility through training the clean and adversarial samples with two BN branches separately. As a preliminary investigation, our work experiments with a new variant of AT with Cross-BN, namely training the adversarial samples with BNetean and vice versa. Interestingly, we find that using BN from another domain only has limited influence on the performance. This observation inspires us to have a closer look at how Dual BN works in Hybrid-AT. Through untwining normalization statistics (NS) and affine parameters (AP) in Dual BN to include one effect while excluding the other, we demonstrate that two AP sets can achieve comparable performance to the original Dual BN, which is consistent with the finding in Rebuffi et al. (2023). We also reveal that disentangled NS can achieve similar performance to Dual BN under certain conditions like small perturbations (E = 8/255). These findings refute the prior claim emphasizing the role of disentangled NS in Dual BN (Xie & Yuille 2020 Xie et al. 2020a) and also inspires us to investigate whether the motivation for Dual BN holds, i.c., the two-domain hypothesis
in Xie & Yuille 2020)) Xie et al. (020aa).
As the motivation for adopting Dual BN, the two-domain hypothesis assumes that "clean images and adver- sarial images are drawn from bwo different domains' " (quoted from Xie & Yuille (2020)). This hypot is verified in Xie & Yuille| (2020) mainly by the visualization of NS, which highlights a large adversarial-cleat domain gap. However, we point out that their visualization has a hidden flaw, which makes their claim regarding the domain gap between adversarial and clean samples deserve a closer look. Specifically, the vi- sualization in Xie & Yuille (2020) ignores the influence of different AP when calculating NS. After fixing this hidden flaw, we demonstrate that the adversarial-clean domain gap is not as large claimed in prior work. Interestingly, under the same perturbation, /nosse magnitude, we show that there is no significant difference between adversarial-clent domain gap and noisy-clean counterpart.
Inspired by the above findings, we propose a two-task hypothesis to replace the two-domain hypothesis in Xie & Yuille (2020): : Xie et al. (2020a) for justification on how Dual BN works in Hybrid-AT. Specifically, we claim that there are two tasks in Hybrid-AT: one task for clean accuracy and the other for robustness. Our two-task hypothesis offers empirical foundations and unified framework for Hybrid-AT improvements, which generalizes Hybrid-AT with Dual BN to various model designs, including the adapter method in Rebufii et al. (2023) and Trades-AT (Zhang et al. 2019b). In addition to exploring BN for training Hybrid-
2

---

Published in Transactions on Machine Learning Research (03/2024)
AT models, study delves into Dual BN at test time, uncovering that affine parameters characterize robustness during inference.
We summarize our main contributions as follows.
Our work thoroughly investigates how disentangled normalization statistics (NS) and affine param- eters (AP) in Dual BN impact the training Hybrid-Ai AT models, leading to comprehensive and solid refutation of prior claims about the significance of NS.
Our work investigates the adversarial-clean domain gap. We point out hidden flaw of NS visual- ization in prior work, and demonstrate the adversarial-lean domain gap is not as large as expected both visually and quantitatively.
Our work proposes a two-task hypothesis as an empirical foundation and unified framework for enhancing Hybrid-ATT, connecting diverse methods like Dual BN, Dual Linear, Adapters and Trades- AT. This hypothosis may bring new inspirations to Hybrid-AT improvements from new perspoctive.
Our study examines Dual BN at test time, exploring various NS and AP types with pretrained model and revealing that AP determines robustness during inference.
2 Problem overview
2.1
Adversarial training
Adversarial training. Adversarial training (AT) Ganin et al. 2016 Madry et al.) 2018 Shafahi et al. 2019 Andriushchenko Flammarion, 2020 Bai et al. 2021) has boen the most powerful defense mothod against adversarial atlacks, among which Madry-AT (Madry et al. 2018) is a typical method detailed as follows. Let's assume D is a data distribution with (a, pairs and f(-,0) is a model parametrized by 0. indicates cross-entropy loss in classification. Instead of directly feeding clean samples from D to minimize the risk of (( (x. v)l, Madry et al. (2018) formulates saddle problem for finding model parameter 0 by optimizing the following adversarial risk:
arg min (( max
max ([<x
(1)
8ES
where S denotes the allowed perturbation budget which is a typically ip norm-bounded e. We term the above adversarial training framework as Classical-AT. It adopts two-step training procedure (inner maximization + outer mininization), and trains the robust model with only adversarial samples. Following the same procedure, Xie & Yuille )2020) : Xie et al. (2020a) propose to train the robust model with both clean and adversarial samples, termed as Hybrid-AT. The loss of Hybrid-AT is defined as follows:
Lllbrrdd = al( + -1- - a) 8:00),
(2)
where x and x + 5 indicate clean and adversarial samples, respectively. a is hyper-parameter for balancing the clean and adversarial branches, is set to 0.5 in this work following Goodfellow et al. (2015) Xie & Yuille b
2.2 Batch normalization in AT
Batch normalization (BN). We briefly summarize how BN works in modern networks. For a certain layer in the DNN, we denote the feature layers of mini-batch in the DNN as B {a ...., x²) The feature layers are normalized by mean y and standard deviation
11 2 +
(3)
3

---

Published in Transactions on Machine Learning Research (03/2024)
80
Clean Accuracy(S)
70
80
Robustnesst
30
40
30
Grea-At Cres--hh,
20
GressAl Cross
Buscine AT with Ser-ini c-
Bisciin with Se c-16
USST with Cross-n c c-15
Cress-Al with Crass-Bhi.
Epoch
Epoch
Figure 1: Clean accuracy and robustness (PGD10 Accuracy) of Cross- AT during training. In Cross-AT, the adversarial samples are normalized by the BN statistics calculated by clean samples. Interestingly, Cross-AT yield comparable robustness to original SBlf-BN(BNadv).
Table 1: Test accuracy of Cross- Hobos--ybri--TT (e 16/255)- In Cross-Hybrid-AT the adversarial branch is
normalized by BN BNttean: and the clean branch is normalized by BNade- Experimental results show that Cross-
Hybrid-AT achieves comparable results to Hybrid-AT with vanilla Dual BN.
Model
Training
Test
Clean
PGD-10
AA
Hybrid-AT
Dual BN
BNade
61.84
31.67
22.51
Cross- Pross-Hyyrrid-T
Dual BN
BNelaa
59.56
31.25
22.40
Hybrid-AT
Single BN
93.70
29.86
0.48
where and 3 indicate the weight and bias in BN, respectively. To be clear, we refer and a as normalization statistics (NS), y and B as affine parameters (AP). During training, NS calculated on the current mini- batch statistics for the update of model weights. Meanwhile, a runming average of NS is recorded in the whole training process, which applied for inference after training ends.
Dual BN in AT. There is increasing interest in investigating BN in the context of adversarial robust-
ness (Aaiss et al. 2020: Cheng et al. 2020 Nandy et al. 2021 Sitawarin et al. Gong et al. 2022) This
work focuses on Hybrid-AT with Dual BN Xie & Yuille 2020
Xie et al. 2020a which applies BNalean and
BNado to clean branch and adversarial branch, respectively.
2.3 Experimental setups
Pang et al. 2020 demonstrates that AT's basic training settings significantly impact model performance and recommended specific parameters for fair comparison of AT' methods. If not specified, we adhere to the suggested settings in Pang et al. (2020).
Experimental setups. In this work, we perform experiments on CIFAR10 (Krizhevsky et al. 2009 An- driushchenko & Flammarion, 2020 Zhanng al. 2022) with ResNet18 AAndrisshhhenoo & Flammarion 2020 Targ et al. 2016); Wu al. 2019 Li et al. 2016; Zhang et al. 2022 Specifically, train the model for 110 epochs. The learning rate set to 0.1 and decays by a factor of 0.1 at the epoch 100 and 105. We adopt an SGD optimizer with weight decay 5 x 10-4, For generating adversarial examples during training, we use Eoo PGD attack with 10 iterations and step size 2/255. For the perturbation constraint, is set to Ex 8/255 (Pang et al. 2020 or 16/255 (Xie Yuille 2020 Following Pang et al. (2020), we evaluate the model robustness under PGD-10 attack (PGD attack with steps) and AutoAttack (AA) (Croce
Hein 2020).
3
On the BN induced misalignment
In Hybrid-AT, the model is trained with two branches: clean branch and an adversarial branch. These two branches share all model weights but are found require independent BN modules, i.., Dual BN (Xie & Yuille 2020 Xie et al. 2020a). At test time, only single branch can be used by choosing either BN
4

---

Published in Transactions on Machine Learning Research (03/2024)
Table 3: Test accuracy (%) of untwining NS and AP in Dual BN. For NSs, indicates mixed distribution and 2 indicates discntanglod distribution for normalization. For APs, indicates a single set and 2 indicates double sets of APs. The subscripts of AP add and eeeen indicate the input data lype used during training. Setupl with two sets of APs achieves comparable results with Dual BN.
= 8/255
16/255
Setups
NS
AP
Clean
PGD-10
AA
Clean
PGD-10
AA
Single BN
1
88.06
49.75
7.03
93.70
29.86
0.48
Dual BN (BNado)
2
2
82.77
51.33
46.19
61.84
31.67
23.14
Dual BN (BNcleam)
2
94.91
0.32
0.10
94.18
0.00
0.00
Setupl (AP adv)
2
81.86
50.99
44.63
60.02
30.89
23.43
Sctupl (APalean)
2
94.74
0.10
0.04
94.30
0.00
0.00
Sctup2 (NSade)
1
85.49
49.39
42.96
55.91
21.92
10.64
Setup2 (NSaten))
2
1
89.22
49.48
42.95
86.35
1.08
0.00
or BNetcan- The adversarial branch (with BNodv) adopted in model robustness, while BNeteon is adopted in Xie et al. (2020a)
Xie & Yuille (2020) for prioritizing high for only considering clean accuracy.
However, swapping the BN during inference, i.e., adopting BNataa for robustness and BNado for clean accuracy, leads to a significant performance drop. As shown in Table 2 Beeeennn leads to almost zero robustness during inference. This interest- ing phenomenon inspires us to investigate the following ques- tion: will BNelean achieve robustness if it trained with the adversarial branch, and vice versa? To facilitate discussion of the above misalignment, we introduce new term Cross-BN which refers to adopting BNetean for the adversarial branch or BN for the clean branch. With similar
Table 2: Test accuracy (%) of Hybrid-AT
with Dual BN. BN leads to almost zero robustness under both perturbation bud- gets (c): 8/255 and 16/255.
Setups
Clean
PGD-10
AA
8/255
Dual BN (BN...)
82.77
51.33
46.19
Dual BN (BNatem))
94.91
0.32
0.10
16/255
Dual BN (BNaas)
61.84
31.67
23.14
Dual BN (BNetem))
94.18
0.00
0.00
terminology rule, BNeleon for the clean branch or BNado for the adversarial branch is termed as
BN inside the network
Cross- AT: a preliminary investigation. Before inves- tigating Hybrid-AT with Cross-BN. we first investigate & setting where only adversarial samples are used for model training. Note that it is ad adversarial branch, and the base- line model with Sclf-BN adopts BNade. Cross-AT is con- ducted by replacing the default BNads with Cross-BN BN, i.., BNetton (see Figure 2). Specifically, the adversarial samples are normalized by the BN statistics calculated by clean samples. It should be noted that Cross-AT: the clean samples are used only for forward propagation to get the BN statistics, and the model weights are updated only by the adversarial branch. Interestingly, although the adversarial branch is normalized by BNclean, Figure formance as the baseline model with Self-BN(BN
Cleen carple
NET
sample
NFT
Loss
x- f
Forward
a
Cross-AT: Replacing BN with BN BNoteon in the adversarial branch. The adversar- ial samples are normalized by the BN statistics calculated by clean samples. shows that
Figure
AT achieves comparable per-
Cross- Hoos--ybbiid-T: Hybrid-AT with Cross-BN. Here, for the Dual BN in Hybrid-AT, we replace the default Self-BN with Cross-BN and term it Cross-Hybrid-ATT. In Cross-Hybrid-nt the adversarial branch is normalized by BNelean and the clean branch is normalized by BN Baddv As shown in Table 1 BNetean Cros--Hyrrid-AT achieves comparable results to BN in Hybrid-AT. The finding in Cross- Mooss-yyriddAA is consistent with that in Cross-AT, which indicates that Cross-BN BN achieves comparable results to Self-BN,
Implication of the above results. As discussed above, training the model with Cross-BN lcads to a comparable performance as with Self-BN in Hybrid-AT. However, this finding appears counter-intuitive considering the results of Hybrid-AT with Single BN. As shown in Table [ Single BN leads to almost zero robustness (0.48%) under AA attack. Note that a single BN is calculated by a mixture of clean and adversarial samples. If calculating BN statistics on either clean examples or adversarial examples can lead to high robustness, how come training on BN calculated hybrid samples leads to an AA robustness close
5

---

Published in Transactions on Machine Learning Research (03/2024)
to zero? This finding also conflicts with the claims in Xie & Yuille (2020); Xie et al (2020a) that emphasize
the importance of NS, and inspire us to investigate how Dual BN works in Hybrid-AT.
4 Understanding how Dual BN works in training of Hybrid-AT
The failure of prior claims Xie & Yuille (020): Xie al. (2000a) to explain our observations in Section inspires us to investigate how Dual BN works in Hybrid-AT. On top of the single BN as default case,
Dual BN introduces an auxiliary BN component and causes two changes: (i) disentangling the mixture distribution for normalization statistics (NS) and (ii) introducing two sets affine parameters (AP). To
fully understand Dual BN in Hybrid-AT, we delve into its mechanisms.
Untwining NS and AP in Dual BN. As discussed above, compared with Hybrid-AT with Single-BN, Dual BN brings two effects: disentangled NSs and two sets of APs. To determine the influence of each effect on the model performance, we design two sctups of experiments to include only one effect while excluding the other. In Setupl, we only in- clude the effect of two sets of APs, by applying two different sets of APs (Sadv/Thade and Setean/ Datean/Tdeenn) in the adversarial and clean branches while using the default mixture distribution for normalization. In Setup2, we only include the effect of two sets of NSs by disentangling this mixture distribution with two different sets of NSs while making clean and BN ade share the same set of APs. The above sctups of BNs are summarized in Figure and we discuss the experimental results in Table as follows.
ReLU
ReLU
ReLU
ReLU
celeas
Telean
YYods
Tadrtoloon
Bads refeam
Sakenn
Puteun
Basio
Padesvaioon
Pado cleun
Melcon
Puat
Mude--hiun
Melson
Muddo
Codutction
Oelvam
Cast
Tody clcon
Oylnon
Cauty
conv
conv
conv
coIv
+
Faddey Telloa
Taddy Zeeleom
Fades sFelena
Fadie sTeloar
(a] Hybrid AT (Detault)
(b)
(c) Setup1
(d) Setup2
Illustration of different BN setups for un- twining NS and AP in Dual BN of Hybrid-AT.
Figure 3:
Role of disentangled NS and AP. As shown in Table 3 Dual BN (with BNadv during inference) brings significant robustness improvement over the Single BN bascline, which is consistent with findings in Xie & Yuille 22220). Interestingly, under the atlack of PGD-10, their robustness gap is not significant, how- ever, under AA, the Single BN achieves very low robustness (7.03% and 0.48% for = 8/255 and 16/255, respectively). Morcover, Sctupl (APado) achieves comparable robustness as that of Dual BN (BNadv) for 8/25 and 16/255, suggesting two sets of APs alone achieve similar performance as Dual BN for yielding higher robustness adv) than single BN setting. The effect of two sets of NSs is more nuanced: for a small perturbation 8/23 8/255, disentangling mixture distribution is beneficial for boosting the robustness under strong AA; for a large perturbation = 16/255, this benefit is less significant. This can be explained by the fact that training under e = 16/255 is much harder than c 8/255.
Overall, we have two conclusions. First, two sets of AP achieve comparable performance to Dual BN, aligning with the findings Rebulfi et al. (2023). Moreover, our research extends beyond Rebufli et al. (2023) by not only disentangling AP but also exploring the disentanglement of NS for a more thorough examination of Dual BN. Although not as effective disentangling AP, disentangling NS can also achieve comparable robustness to Dual BN under certain conditions even against the strong AutoAttack, such the small perturbation(e = 8/255). However, the benefit of disentangling NS narrows significantly for large perturbation.
Conclusions.
5
On the domain gap between clean and adversarial samples
A model trained on a source domain performs poorly on new target domain when there is domain shift (Dauné III
2007 Sun et al.) 2017). With BN as the target, it is common in the literature (Li et al. 2017 Benz et al.
202 Schneider et al. 2020; Xie & Yuille 2020 Xie et al., 2020a) to indicate the domain gap by the difference of NS between two domains. In adversarial machine learning, prior work (Xic & Yuille,
2020 Xie et al.
2020a Jiang et al., 2020) perceive the adversarial domain as new domain. Specifically, Xie & Yuille (2020) highlights the ady-clean domain gap by visualizing the difference of NS in BN and BN eleen
6

---

Published in Transactions on Machine Learning Research (03/2024)
0.055
NS (NSpa)
0.000
050
Stean (N5
075
NSchen
channel
0.02
0.00
channel
Figure Visualization of normalization statisties (NS) by randomly choosing channels and displaying the NS calculated with different APs. The superscript and subscript of NS refer to the AP and input images when calculating NS, respectively. For example, NS is computed calculated by the same AP are close to each other, such as NSads
clean samples with AP NSs adv and NS clean calculated by AP advs so is
similar NSdean and NS calculated by APclean:
adv
AP
APessn
-0.1
channel
0.00
APaly adv
AP
00.0
-020
channel
Figure 5: Visualization of affine paramcters (AP). Randomly chose 20 channels for visualizing AP clean and ado. There exists a gap between AP clean and AP addo-
(see Figure 5 of (Xie & Yuille 2020 ). The large domain gap visualized in (Xie & Yuille 2020 is somewhat
in confiict with the finding in Section ! that disentangling NS plays a less role for high performance. We further investigate the adv-clean domain gap for a comprehensive understanding.
5.1 A hidden flaw leads to misleading visualization
With our analysis in Section. and adversarial branch still have different weights, i.e., AP, even though the same set of convolutional filters are shared. Therefore, the NS difference between BN and BN is characterized by two factors: (a) AP inconsistency and (b) different domain inputs. We discuss the influence of these two factors on the NS difference as follows.
know that the AP in BNeleen and BNads different. The clean branch
Re-calibrated NS for disentangled analysis. In the default setup of Dual BN, NS calculated clean samples with AP ateena while NS is calculated adversarial samples with AP In order to analyze the influence of different AP and domain inputs the NS, additionally calculate the NS on clean samples with AP adv (denoted as NSade deen) and calculate the NS on adversarial samples with AP (denoted as Ngclean). These Iwo NS are termed re-calibrated NS since the AP and inputs are from different branches. Following NS and NS to indicate AP choice with the superscript and indicate sample choice with the subscript, we can also denote vanilla NS as NSelean and denote NS as NSads Both NSelean and NSado are termed as vanilla NS for differentiation. Details of obtaining various NS is reported in Section A.T of the appendix.
adv

---

Published in Transactions on Machine Learning Research (03/2024)
0.14
Wasserstein distance
0.12
Figure 6: Layer-wise discrepancy vi- sualization. For all layers, there exists a large distance (higher than zero) be- tween AP clean and AP addr see Y and B in the figure. However, with the same ady the gap between NSadu and NS clean stays almost zero in all layers, see and in the figure.
0.1
0.00
0.06
0.04
0.00
Depth of layer
A hidden flaw of NS visualization in Xie & Yuille (2020). To exclude the influence of AP inconsis- tency, we intend to compare NS between clean and adversarial samples with the same AP (the superscript in NS). In other words, the domain gap is characterized by the difference between NS and NSclea or that between NSads and N Following the procodures in Xie & Yuille 2020), we plot different types of NS Figure g by randomly sampling 20 channels of the second BN layer in the first residual block. Fig. [ shows that there exists gap between Ne and NSede which consistent with the findings in Xie Yuille| (2020). Moreover, there are two other observations from Figure 4 First, we fix the input samples and calculate NS with different AP, there exists large gap, ie., the gap between NS clean and NS well as the gap between NSede and NSeleen. Second, those NSs with the same APs are very close to each other: NSade and NS are very similar to each other, and the same applies to NSdlan and NSclear. We report the visualization results of AP in Figure 5 for comparison, which shows a significant gap between Alcioon and APdde.
ado
Conclusions. We point out flaw Xic Yuille (2020) that the large adv-clean gap visualized in Xie & Yuille (2020) is caused by the AP consistency. When adopting the same AP, the adv-clean domain gap significantly narrows. Our investigations suggest that the visualization and conclusions in Xie & Yuille (2020) might convey a mislcading message. Our findings update the understanding on the adv-clcan domain gap.
5.2 Adv-clean domain gap is not as large as expected
Quantitative measurement of domain gap. Figure 4 investigates the adv-clean domain gap qualita- tively. For a quantitative comparison, we measure the Wasserstein distance between clean and adversarial branches in different layers in Figure g As shown in Figure 0 the Wasscrstein distance of NS between clean and adversarial branches is much smaller than the difference of AP for a certain layer. This finding is consistent with that Figure and Figure
Adv-clean versus noisy-clean domain gap. As sug- gested in Benz et al. (2021a); Schneider et al. (2020). noisy samples (images corrupted by random noise) can be seen as a domain different from clean samples. Adver- sarial perturbation is a worst-case noise for attacking the model. Taking a ResNet118 model trained on clean samples for example, we report the performance under adversarial perturbation and random noise (with the same magnitude) in Table 4 As expected, the model accuracy drops zero with adversarial perturbation. Under random noise of the same magnitude, we find that the model performance only drops by small margin. Given that the influence of adversarial pertur- bation the model performance is significantly larger than that of random noise, it might be tempting to believe that the adversarial-clean domain gap is much larger than noisy-clean domain gap.
Table 4: Test accuracy (%) under random noise and adversarial perturbation during inference.
Noise/ Noist/perturbation perturaation Size Random noise
8/255
16/255
94.0
92.7
86.6
Adversarial perturbation
94.0
0.00
0.00
With Wasserstein distance NS between different domains as the metric, we compare the adversarial-clent domain gap with noisy-clean counterpart on the above ResNet18 model trained on clean samples, as shown in Figure The perturbation and noise magnitude are set to 16/255. Interestingly, we observe that there is no significant difference between the adversarial-clean domain gap and noisy-clean counterpart.
8

---

Published in Transactions on Machine Learning Research (03/2024)
0.02
0.01
12
18
Depth
Figure Visualization of adversarial- clean domain gap and noisy-clean do- main gap (perturbation, /oisse magni- tude is set to 16/255)
0.02
0.00
Depth
Conclusions. Based on the visualization and quantitative results, we reveal that the adversarial-clean domain gap is not as large as many might expect, considering the strong performance drop caused by adversarial perturbation.
5.3 Interpreting Hybrid-AT from a two-task perspective
New empirical foundation for Hybrid-AT im- provement. Prior work (Xie & Yuille 2020 takes the two-domain hypothesis as the empirical foun-
Table 5: Test accuracy of Hybrid-AT with Dual linear,
e = 16/255. Hybrid-AT with Dual Linear achieves a
similar trend and comparable results with Dual BN.
dation for applying Dual BN in Hybrid-AT. Our investigations in Section [.1 and Section 5.2 iden- tify flaw the two-domain hypothesis, and point out that the adv-clean domain gap is not as large expected. These findings suggest discarding the two-domain hypothesis in Hybrid-AT improvement. Hybrid-AT improvenent: the two-task hypothesis. model weights are trained for two tasks: one for clean accuracy and the other for robustness. A common approach for handling two tasks with a shared backbone is to make the top layers unshared. Here, we experiment with a shared encoder of single BN but with dual linear classifiers. The results in Table 5 show that Dual Linear results in similar behavior as Dual BN, validating the two-task hypothesis. Rebufhh et al. (023)) proposes use adapters for each type of input in Hybrid-AT model, which matches the classification performance of Dual BN with significantly fewer paramcters. The success of adapters in Rebuffi et al. 2023 also validates our two-lask hypothesis.
Setups
Branch
Clean
PGD10
AA
BN
61.84
31.67
22.51
Dual BN
BN
94.18
0.00
0.00
Linearade
60.72
28.84
16.50
Dual Linear
Linear
91.43
2.21
1.30
Instead, we propose new empirical foundation for Intuitively, with the two branches in Hybrid-AT, the
A unified framework for Hybrid-AT improve- ments. Our new perspective on Hybrid-AT enables alternative solutions mitigale the two-task con- flict without resorting to two sets of APs. Here, we experiment with including additional regulariza- tion loss, which is introduced to minimize the gap between two tasks. As concrete example, we add KL loss on the basic loss of Hybrid-AT in Eq The extra loss is designed to explicitly minimize the discrepancy between the outputs adversarial and clean branches. Interestingly, this simple change improves the AA result Single BN significantly from 0.48% 23.60 23.60% Compared to BN (the default branch during inference). Hybrid-AT with KL loss achieves superior performance on both clean accuracy and robustness. Interestingly, the Hybrid-AT with KL loss reminds of another AT framework termed Trades-AT (Zhang et al. 2019b), which is also trained hybrid samples and has a KL loss. This might provide an explanation for the effectiveness of Trades-AT (Zhang et al. 20195) by analyzing the KL term. Admittedly, KL loss on the output is just a naive attempt, but its promising result. invites future works to explore other solutions.
Table 6: Test accuracy of Hybrid-AT (Single BN) with KL. loss, 6 16/255.
Sctups
Clean
PGD10
AA
Single BN
93.70
29.86
0.48
Single BN (with KL loss)
68.86
33.61
23.60
Dual BN (BNadv)
61.84
31.67
22.51
Dual BN (BNalean)
94.18
0.00
0.00
Conclusions. Our investigations on the adv-clean domain gap suggest discarding the prior two-domain hypothesis. Instead, we propose two-task hypothesis as the new empirical foundation for Hybrid-AT improvement. The two-task hypothesis also serves a unified framework that collaborates different methods
9

---

Published in Transactions on Machine Learning Research (03/2024)
Table 7: Evaluation results of various NS and AP pairs, <= 16/255.
During inference, re-calibrated NS
achieves comparable performance the default sctting.
Setups
<= 8/255
<= 16/255
NS
AP
PGD10
AA
PGD10
AA
BNade
NS
AP adu
51.33
46.19
31.67
22.51
Default
BNetton
NS
AP clomm
0.32
0.10
0.00
0.00
Setupl
NS
AP Pado
17.1
9.16
10.02
9.80
Swap
Setup2
0.00
0.00
0.45
0.00
odb
Sctup3
NS
AP odu
51.75
46,55
32.73
24.40
Re-calibration
Setup4
Ngdeen
AP atom
0.00
0.00
0.00
0.00
together, such as Dual BN, adapters in Rebuffi al. (2023) and Trades-Al to be unrelated at first sight.
(Zhang al. 2019b that seem
5.4 AP characterizes robustness in test-time
Investigation on BN in test-time has been widely discussed topic in other areas (Li et al. 2017 Benz et al.
2021a; Schneider et al. 2020). Different from Section and Rebulfi et al. (2023) that investigates model training, we further invest igate how Dual BN works in test-time. Specifically, we compare the robustness with various NS and AP combinations, as reported in Table Details of obtaining various NS are reported in Soction I. of the appendix.
Robustness with various NS and AP pairs. Table 7 shows that given AP re-calibrated NS elean achieves a robustness of 51.75%, which is comparable to 51.33% with NSade. Note that the only difference between NSade clean and NSade adv is that they are calculated by clean and adversarial samples, respectively. More- over, given AP clean, both NS and NSclean yield zero robustness. The results of swapping NS and NSade when AP is fixed also given in Table 7 for comparison. We find that directly replacing NS with ade
NSclean in BNadv (Setupl) results in lower robustness (17.1%) than original BNado (51.33%).
Conclusions. We conclude that AP characterizes the large robustness gap between BNelean and BNade during inference. When AP is consistent for both NS computation and robustness evaluation, the robustness gap between the NS calculated on clean adversarial samples is limited.
6 Related work
Adversarial training. Since the advent of Classical-AT [Madry et al. 2018) and Hybrid-AT (Xie & Yuille 2020 Xie et al. 2020a), numerous works have attempted to improve AT from various perspectives. From the data perspective, Uesato et al.] (2019) ; Carmon el. al. (2019): Zhang et al. (2019c) have independently shown that unlabeled data can be used improve the robustness. From the model perspective, AT often benefits from the increased model capacity of models (Uesato et al. 2019: Xie & Yuille 2020). Xie et al.)
(2020b); Pang et al.) 22200; Gowal et al. (2020) have investigated the influence and suggested that smooth activation function, like parametric softplus, is often but not always (Gowal et al. 2020 helpful for AT.
Another branch of studies aims to improve the training efficiency of adversarial training based on PGD attack, termed as FAST AT (de Jorge et al.) 2022 Jia et al.] 2022b Park & Lee
2021 Wong et al.) 2020 Andriushchenko & Flammarion 20200 Jia et al. 2022a). Specifically, FGSM attack adopted in Wong
et al. (2020); Andriushchenko Flammarion (2020); de Jorge et al. (2022) to replace PGD attack during
training, which achieves promising robustness with catastrophic overfitting problem tackled.
Dual BN in AT. Prior work [Xie et al. 2020a shows that adversarial samples can be used to improve recognition (accuracy) by adversarial training where adversarial samples are normalized by an independent BN Moreover, Xie & Yuille (2020) has shown that adding clean images in adversarial training (AT) can significantly decrease robustness performance, where such negative effects can be alleviated to a large extent by simply normalizing clean samples with an independent BN BNeeean- Inspired by their finding, [Jing let al. (2020) also adopts Dual BN in adversarial contrastive learning, showing that single BN performs significantly worse than Dual BN. Beyond Dual BN, triple BN has been attempted in
Fan et al. (2021) for
10

---

Published in Transactions on Machine Learning Research (03/2024)
incorporating another adversarial branch. Wang et al. 22021) has also combined Dual BN with Instance Normalization to form Dual batch-and-Instance Normalization for improving robustness. Prior work (Xie) & Yuille 2020 interprets the necessity of Dual BN from the perspective of an inherent large adversarial- clean domain gap. By contrast, Rebuffi et al. (0023) demonstrates that separate batch statistics are not necessary for Hybrid-AT and it is sufficient to use adapters with few domain-specific parameters for each lype of input. Extensive experimental results Rebuffi et al. (2023) show that the proposed model matches Dual BN's performance and the adversarial model soups perform better on ImageNet. variants than the advanced masked into-encoders. Our investigations extends the scope of Rebuffi et al. (2023) by reporting new findings previously unaddressed topics, including but not limited to the role of disengaged NS in model training and the new understanding of adversarial-clean domain gap. Our two-task hypothesis extends the adapter method proposed in Rebulli et al. (2023) by not only underpimingg the adapter method with empirical evidence but also serving as a foundation for various methods utilizing domain-specific trainable parameters. This hypothesis also links the adapter method broader Hybrid-AT enhancement strategics, such as Trades-AT (Zhang et al. (2019b). Additionally, the adapter's success in Rebulli et al. (2023) reinforces our hypothesis, highlighting how work complements and expands upon the contributions of Rebuffi et al.) (2023).
BN applications beyond AT. Prior work investigates BN in various fields beyond AT, such as for ad- versarial transferability [Benz et al.] 2021b Dong et al. 2022) Benz et al. (2021b) investigates BN from the non-robust feature perspective. Specifically, [Bnzz et al. (2021b) empirically reveals that BN shifts model towards being more dependent on the non-robust features. Based on this finding, [enz et al.) (201bb) suggests strategies like removing BN early stopping during the training of substitute models to improve adversarial transfcrability. Another work Dong et al. (2022) provides both empirical and theo- retical evidence which shows that the upper bound of adversarial transferability is influenced by the types and parameters of normalization layers. Based on this observation, (Dong et al. (022) proposes Random Normalization Aggregation (RNA) module to replace original normalization layers and create a combination of different sampled normalization. Extensive experiments demonstrate that the proposed RNA module achieves superior performance on different datasets and models. Another branch of work adopts Dual BN domain adaptation. AdaBN (Li et al. 2017) leverages different statistics for two domains but loses source domain information by using only target domain statistics during inference. DSBN (Chang et al. 2019) introduces separate BN branch for unsupervise ised domain adaptation, extendable to multisource scenarios. Huang et al. (2023) proposes reciprocal normalization that structurally aligns the source and target domains by conducting reciprocity across domains. In this work, we mainly focus on investigating Dual BN in the Hybrid-AT.
7 Conclusion
We experiment with Cross-AT and demonstrate the compatibility of clean samples' BN statistics with the adversarial branch, which inspires to doubt the claims of prior work for justifying the necessity of Dual BN in Hybrid AT. We investigate the effect of disentangled NS and AP training Hybrid-AT model, leading to a thorough refutation of prior claims about the significance of NS. Our work further identifics a visualization flaw of the prior two-domain hypothesis, and points out that the adversarial-clean domain gap is not as large as expected. In addition, we propose a new interpretation of Hybrid-AT with Dual BN from the two-task perspective. Finally, we investigate different types of NS and AP in test-time, revealing that AP characterizes robustness during inference. Our study provides a comprehensive understanding of Dual BN as well the adversarial examples.
Acknowledgments
This work was conducted by Center for Applied Research in Artificial Intelligence (CARAI) grant funded by DAPA and ADD (UD230017TD).
11

---

Published in Transactions on Machine Learning Research (03/2024)
References
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. NewrIPS, 2020.
Muhammaa Awais, Fahad Shamshad, and Sung-Ho Bae. Towards an adversarially robust normalization approach. arXiv preprint 2r06v:2066 11007, 2020.
Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. Recent advances in adversarial training for adversarial robustness. arXic preprint arXiv:2102.0133 01356, 2021.
Philipp Benz, Chaoning Zhang, Adil Karjauv, and Tn So Kweon. Revisiting batch normalization for improving corruption robustness. WACV, 2021a.
Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization incrcases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In ICCV, 2021b.
Nils Bjorck, Carla P Gomes, Bart Selman, and Kilian Q Weinberger. Understanding batch normalization. Advances in neurol information processing systems, 31, 2018.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. NeurIPS, 2019.
Woong-Gi Chang, Tackgeun You, Soonguk Seo, Suha Kwak, and Bohyung Han. Domain-specific batch normalization for unsupervised domain adaptation. In CVPR, 2019.
Minhao Cheng, Zhe Gan, Yu Cheng, Shuohang Wang, Cho-Jui Hsich, and Jingjing Liu. Adversarial masking: Towards understanding robustness trade-off for generalization. 2020.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In TCML, 2020.
Hal Daumé III. Frustratingly easy domain adaptation. In Association of Computational Linguistics (ACL), 2007.
Pau de Jorge, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip HS Torr, Grégory Rogez, and Puneet K Dokania. Make some noise: Reliable and efficient single-step adversarial training. arXiv preprint arXiv:2202.011 01188, 2022.
Minjing Dong, Xinghao Chen, Yunhe Wang, and Chang Xu. Random normalization aggregation for adver- sarial defense. Advances Neural Information Processing Systems, 35:33676 33688, 2022.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018.
Lijie Fan, Sijia Liu, Pin-Y Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining finetuning? NewrIPS, 2021.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarti training of neural networks. Journal of Machine Learning Research, 2016.
Xinyu Gong, Wuyang Chen, Tianlong Chen, and Zhangyang Wang. Sandwich batch normalization:
drop-in replacement for feature distribution heterogeneity. In WACV, 2022.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXio preprint S51::2000005999 2020.
12

---

Published in Transactions on Machine Learning Research (03/2024)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. Densely connected convolu- tional networks. In CVPR, 2017.
Zhiyong Huang, Kekai Sheng, Ke Li, Jian Liang, Taiping Yao, Weiming Dong, Dengwen Zhou, and Xing Sun. Reciprocal normalization for domain adaptation. Pattern Recognition, 140:109533, 2023.
Sergey Ioffe and Christian Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 1CML, 2015.
Xiaojun Jia, Yong Zhang, Xingxing Wei, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Prior-guided adversarial initialization for fast adversarial training. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviu, Israel, October 23 27, 2022, Proceedings, Port IV, pp. 567 584. Springer, 2022a.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Jue Wang, and Xiaochun Cao. Boosting fast adversarial training with learnable adversarial initialization. IEEE Transactions on Image Processing, 2022b.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. NewrIPS, 2020.
Harini Kannan, Alexcy Kurakin, and Ian Goodfellow. wrXiv:1803.06375 2018.
Adversarial logit pairing. arXiv preprint
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. ", 2009.
Sihan Li, Jiantao Jiao, Yanjun Han, and Teachy Weissman. Demystifying resnet. arXiv preprint arXiv:1611.011 2016.
Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. ICLR workshop, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning modcls resistant to adversarial attacks. In ICLR, 2018.
Jay Nandy, Sudipan Saha, Wynne Hsu, Mong Li Lee, and Xiao Xiang Zhu. Adversarialyy trained models with test-time covariate shift adaptation. Xive preprint orXiv:2102 00096, 2021.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. arXiv preprint XrXix:2000000 0.00467, 2020.
Geon Yeong Park and Sang Wan Lec. Reliably fast adversarial training via latent adversarial perturbation. Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 7758 7767, 2021.
Shilin Qiu, Qihe Liu, Shijie Zhou, and Chunjiang Wu. Review of artificial intelligence adversarial attack and defense technologics. Applied Sciences, 2019.
Sylvestre-llvise Rebuffi, Francesco Croce, and Sven Gowal. Revisiting adapters with adversarial training. In The Eleventh International Conference on Learning Representations, 2023.
Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry, How does batch normalization help optimization? In NewrIPS, 2018.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wicland Brendel, and Matthias Bethge. Improving robust against common corruptions by covariate shift adaptation. NewrIPS, 2020.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NewrIPS, 2019.
Chawin Sitawarin, Arvind Sridhar, and David Wagner. Improving the accuracy-robustness trade-off for dual-domain adversarial training.
13

---

Published in Transactions on Machine Learning Research (03/2024)
Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adaptation. In Domain Adaplation in Computer Vision Applications. 2017.
Sasha Targ, Diogo Almeida, and Kevin Lyman. Resnet in resnet: Generalizing residual architectures. arxiv preprint 1r0: 00022, 2016.
Jonathan Uesato, Jean- Cann-aptistte Alayrac, Po-Sen Huang, Robert Stanforth, Alhusscin Fawzi, and Pushmoct Kohli. Are labels required for improving adversarial robustness? NeurIPS, 2019.
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kun Kuci Hu, Ji Liu, and Zhangyang Wang. Once-for-all adver- sarial training: In-situ tradeoff between robustnnss and accuracy for free. arXie preprint or Xiu:2010. 11828, 2020.
max: Adversarial composition of random augmentations for robust training. NewrIPS, 2021. Haotao Wang, Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Anima Anandkumar, and Zhangyang Wang. Aug-
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. ICLR, 2020.
Zifeng Wu, Chunhua Shen, and Anton Van Den Hengel. Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 2019.
Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. ICLR, 2020.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoc Le. Adversarial examples improve image recognition. In CVPR, 2020a.
Cihang Xie, Mingxing Tan, Boqing Gong, Alan Yuille, and Quoc V Le. Smooth adversarial training, arXio preprint arXi::2006. 14536, 2020b.
Cong Xu and Min Yang. Adversarial momantum-contrastivve pre-training, arXiv preprint arXhv:2012.19144 2020.
Chaoning Zhang, Francois Rameau, Seokju Lee, Junsik Kim, Philipp Benz, Dawit Mureja Argaw, Jean- Charles Bazin, and In So Kweon. Revisiting residual norks with nonlinear shortcuts. In BMVC, 2019a.
Chaoning Zhang, Philipp Benz, Dawit Mureja Argaw, Seokju Lee, Junsik Kim, Francois Rameau, Jean- Charles Bazin, and In So Kweon. Resnet or densenet? introducing dense shorteuts to resnet. In WACV, 2021a.
Chaoning Zhang, Philipp Benz, Chenguo Lin, Adil Karjauv, Jing WI, and In So Kweon. survey on universal adversarial attack. IJCAI, 2021b.
Chaoning Zhang, Kang Zhang, Chenshuang Zhang, Axi Niu, Jiu Feng, Chang D Yoo, and In So Kweon, Decoupled adversarial contrastive learning for sclf-supervised adversarial robustness. In ECCV, pp. 725- 742. Springer, 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019b.
Jingfeng Zhang, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. Where is the bottleneck of adversarial learning with unlabeled data? arXio preprint or 191 1.08696, 2019c.
14

---

Published in Transactions on Machine Learning Research (03/2024)
A Appendix
A.1 Experimental Setups
Experimental setups. We train and evaluate all models in the paper using the same setups following Pang
et al. (2020). In this work, we perform experiments on CIFAR10 (Krizhevsky et al. 2009 Andriushchenko
2020 Zhang et al., 2022) with ResNet18 AAdrrisshchenko & Flammarion 2020 Targ et al. 2016 Wu al. 2019 Li et al. 2016 Zhang al. 2022) and follow the suggested training sctups in Pang jet al. (2020) unless specified. Specilically, we train the model for 110 epochs. The learning rate set to 0.1 and decays by factor of 0.1 at the epoch 100 and 105. We adopt SGD optimizer with weight. decay use foo PGD attack with 10 iterations and step size = 2/255. For the perturbation constraint, e is set to foo 8/255 (Pang et al.] 2020) or 16/255 (Xie) & Yuille 2020 Following Pang et al. (2020). we evaluate the model robustness under PGD-10 attack (PGD
& Flammarion
5 x 10 For generating adversarial examples during training,
attack with 10 steps) and AutoAttack (AA) (Croce Hein 2020).
Re-calibrate NS. The Neee and NSad correspond original Dual BN's NSclean and NS respectively, For re-calibrated NS, we simulate vanilla NS calculations, specifying the input domain following the subscript and AP following the superscript. For example, NS is computed by processing clean images through the branch with APadu- To simulate the computation of running means in the original Dual BN, forward the samples for multiple epochs for converged results. In Table 7 we first calculate various types of NS and then evaluate robustness by combining NS with AP.
Training details of models in Section 5.3. For the Dual Linear model, we add two linear layers above its penultimate layer and train it using the Hybrid loss outlined Equation 1. For models with regularization loss, we train using single BN and augment the original Hybrid-AT loss with an additional KL-divergence between the predictions on adversarial and clean inputs.
A.2 Further investigations beyond BN
Inspired by finding that two sets of AP can achieve comparable results to Dual BN, we further investigate whether this holds in cases beyond BN where disentangling NS is not applicable. For example, layer nor- malization (LN) adopts sample-wise NS, and therefore it is not applicable to disentangle distribution-wise NS between two domains. We experiment with dual AP on ResNet with LN and the results are reported in Table We observe that with Dual AP, LN performs similarly with BN in either setup (b) and (c) in Figure We also investigate other normalization methods and model architectures (such as ViT) in Table S Table shows that across various normalization methods and architectures, two AP sets achieve comparable performance Dual BN while single AP set fails to achieves high robustness against AA.
Table 8: Effect of dual AP on various types of normalizations and models(c = 16/255), where LN, GN and IN denote Layer Normalization, Group Normalization and Instance Normalization respectively.
Norm
Norm
Setups
Branch
Clean
PGD10
AA
ResNet
BN
Single BN
93.70
29.86
0.48
Dual BN
BNado
61.84
31.67
22.51
Dual BN
BN
94.18
0.00
0.00
LN
Single AP
75.12
18.81
11.80
Dual AP
APado
62.56
26.98
16.90
Dual AP
AP eleea
88.41
0.00
0.00
GN
Single AP
81.85
21.94
14.50
Dual AP
AP
70.27
29.36
18.30
Dual AP
AP Peeeo
91.82
0.00
0.00
IN
Single AP
92.55
23.06
1.20
Dual AP
AP
52.29
25.27
16.10
Dual AP
AP elenm
92.35
0.00
0.00
VIT
LN Single AP
92.21
33.60
1.84
Dual AP
APatean
58.02
30.08
12.44
Dual AP
APadv
91.60
0.00
0.00
15


Reviews:
[{"review_id": "aQU3t6ucf6", "reviewer": "Reviewer_iNQL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. **Clear message**: The paper has a solid and clean message which is conveyed very clearly. The problem under study is very technical and niche, but could still be of broad interest to people working on adversarial robustness.\n2. **Strong empirical verification**: I find that the paper follows a very good scientific methodology to verify all its claims. The ablation studies decoupling the effect of normalization statistics and trainable parameters in Dual BN are very clear and solid.\n\n**Weaknesses**\n1. **MAJOR: Same results were previously published in ICLR2023**: Unfortunately, the exact same observation and explanation of Dual BN has already been published by [Croce et al. (2023)](https://openreview.net/pdf?id=HPdxC1THU8T). Although this paper uses different experiments to arrive at the same conclusion, it conveys exactly the same message, and goes, in fact, beyond this observation to construct a new method to interpolate between adversarial and clean behaviors during inference by mixing the classification token of an adversarially trained ViT. I invite the authors to read that work (specifically Section 3.2) and let me know if they think their work provides something else on top of Croce et al. (2023), but at the moment I am very inclined to reject this work on the basis of lack of novelty.\n2. **Minor: Overly repetitive text**: I find the first few pages of the paper a bit repetitive. The dichotomy between the old hypothesis of Xie & Yuille (2020) and the one of this work is repeated too much in my opinion. The writing could be streamlined and go more directly to the experimental validations. Overall, however, I find the paper is well written and easy to follow.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates the reasons for the success of dual batch normalization (Dual BN) in adversarial training, i.e., using different batch normalization layers to process clean and adversarial samples. In particular, the authors debunk the previous hypothesis that Dual BN works because it mitigates the distribution shift introduced by the adversarial perturbations during training, and show, instead, that Dual BN works because it introduces parallel sets of trainable parameters that can be used to independently learn to solve the clean and adversarial tasks. The claims of the authors are verified using different empirical ablation studies that look at different architecture variants of batch normalization (and other normalization layers) and by inspecting the activation statistics of a network under different settings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Due to the major overlap with prior work I cannot unfortunately recommend the acceptance of this manuscript and will not be requesting any changes over it.", "broader_impact_concerns": "I don't see any major societal concerns stemming from this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "M8SW1sS1zK", "reviewer": "Reviewer_Dqef", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths \n\n* This work provides multiple novel and interesting findings on Dual BN. For example, it is novel and important to prove that disentangled affine parameters play the main role in Dual BN instead of the normalization statistics claimed in prior work. \n* This paper points out a clue to refute the previous two-domain hypothesis, and proposes the new two-task hypothesis. Following the two-task hypothesis, multiple methods can achieve comparable performance to Dual BN. This finding is novel and may inspire more studies in the future.\n*  The analysis and experiments are solid to validate the conclusions. The paper is well organized.\n\nWeaknesses\n\n* In related works, this paper only discusses how to apply Dual BN in adversarial training. Discussions of applying Dual BN in other applications are needed.\n* Section 5.3 includes multiple models such as Dual Linear model and model with regularization loss. How are these models trained? \n* The experimental setups in Table 5 are unclear.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work investigates Dual BN in hybrid adversarial training (AT) with extensive experiments. As Hybrid AT model takes both adversarial and clean images as inputs, it is popular to adopt Dual BN for higher performance. According to prior two-domain hypothesis, Dual BN’s effectiveness is attributed to its disentangled normalization statistics. By contrast, this work finds that the disentangled affine parameters play the most important role in Dual BN instead of the statistics. This paper also refutes prior two-domain hypothesis by pointing out a visualization flaw. It further introduces the two-task hypothesis that demonstrates the potential of other methods. The topics and findings are of interest to the community.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the weaknesses part:\n* Provide more discussions of applying Dual BN in other applications rather than adversarial training.\n* Provide more training details for section 5.3 and more detailed experimental setups for Table 5.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZB9LrkS0Zc", "reviewer": "Reviewer_eqrk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. This paper is well-written and easy to follow.\n\n2. This paper focuses on an important topic of applying Dual BN in Hybrid AT. Given the zero robustness of $BN_{clean}$ in original Dual BN, it is novel and impressive to find that $BN_{clean}$ can achieve robustness if it is trained with the adversarial branch.\n\n3. This is the first paper to refute the two-domain hypothesis in prior studies. The authors point out that the adversarial-clean domain gap is not as large as expected. This finding is novel and strongly supported by solid experiments.\n\n4. The proposed two-task hypothesis shows the possibility of alternative solutions beyond Dual BN to mitigate the adv-clean conflict in Hybrid AT. This update of understanding may inspire future studies on Hybrid AT from a new perspective.\n\n\nWeaknesses:\n\n1. Section 4 discusses the ResNet model with various normalizations such as LN. How about other architectures such as the ViT model?\n\n2. It is unclear how different types of NS are computed in Section 5. More details are needed.\n\n3. There exist several papers studying the role of normalization layers in adversarial robustness, which have not been discussed in this paper, such as [a, b].\n\n[a]. Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective. ICCV 2021.\n\n[b]. Random Normalization Aggregation for Adversarial Defense. NeurIPS 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new perspective to understand Dual BN in Hybrid AT. Previous research has commonly supported the idea that there exists a significant domain disparity between adversarial and clean images. Moreover, prior studies have suggested that the implementation of Dual BN in Hybrid AT is effective due to disentangled statistics. However, this paper strongly challenges these assertions through extensive experimentation. One conclusion is that the key factors in Dual BN are two sets of affine parameters instead of disentangled statistics. Furthermore, the paper points out that the adversarial-clean domain gap is not as large as many might expect, and proposes a new interpretation of Hybrid-AT with Dual BN from the two-task perspective.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As mentioned in the weaknesses.\n\n1. Please discuss other architectures, such as ViT models.\n2. Please provide more details of different types of NS in Section 5.\n3. Please discuss relevant papers.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel dual batch normalization method tailored for adversarial training. The reviewers unanimously commend the manuscript for its significant contribution, highlighting intriguing insights into Dual BN, robust empirical validation, and lucid hypotheses. Minor concerns raised during the rebuttal phase have been effectively addressed through additional results and analysis. Furthermore, the consensus among the reviewers is that this paper will captivate readers, garnering positive feedback.

Overall, this manuscript fits the TMLR acceptance criteria and I recommend accepting this paper.
Decision: Accept (poster)