A Closer Look at Dual Batch Normalization and Two-domain Hypothesis In Adversarial Training With Hybrid Samples
Authors: Chaoning Zhang
Kang Zhang
Chenshuang Zhang
Axi Niu
Chang D. Yoo
In So Kweon
Venue: Submitted to ICLR 2023 (2023)
Keywords: Adversarial training
batch normalization

Abstract: There is a growing concern about applying batch normalization (BN) in adversarial training (AT), especially when the model is trained on both \textit{adversarial} samples and \textit{clean} samples (termed Hybrid-AT). With the assumption that \textit{adversarial} and \textit{clean} samples are from two different domains, a common practice in prior works is to adopt dual BN, where BN$_{adv}$ and BN$_{clean}$ are used for adversarial and clean branches, respectively. A popular belief for motivating dual BN is that estimating normalization statistics of this mixture distribution is challenging and thus disentangling it for normalization achieves stronger robustness. In contrast to this belief, we reveal that what makes dual BN effective mainly lies in its two sets of affine parameters. Moreover, we demonstrate that the domain gap between adversarial and clean samples is actually not very large, which is counter-intuitive considering the significant influence of adversarial perturbation on the model. Overall, our work sheds new light on understanding the mechanism of dual BN in Hybrid-AT as well as its underlying two-domain hypothesis. 

Under review as a conference paper at ICLR 2023
A CLOSER LOOK AT DUAL BATCH NORMALIZATION AND TWO-DOMAIN HYPOTHESIS IN ADVERSARIAL TRAINING WITH HYBRID SAMPLES
Anonymous authors Paper under double-blind review
ABSTRACT
There is a growing concern about applying batch normalization (BN) in adversarial training (AT), especially when the model is trained on both adversarial samples and clean samples (termed Hybrid-AT), With the assumption that adversarial and clean samples are from two different domains, common practice in prior works is to adopt dual BN, where BNadv and BNetean are used for adversarial and clean branches, respectively. A popular belief for motivating dual BN is that estimating normalization statisties of this mixture distribution is challenging and thus disentangling it for normalization achieves stronger robustness. In contrast this belief, we reveal that what makes dual BN effective mainly lies in its two sets of affine parameters. Moreover, we demonstrate that the domain gap between adversarial and clean samples is actually not very large, which is counter-intuitive considering the significant influence of adversarial perturbation on the model. Overall, our work sheds new light on understanding the mechanism of dual BN in Hybrid-AA AT as well as its underlying two-domain hypothesis. Recommended practices are summarized as takeaway insights for future practitioners.
INTRODUCTION
Adversarial training (AT) (Madry et al.) 2018) that optimizes the model on adversarial examples is time-tested and effective technique for improving robustness against adversarial attack. Beyond classical AT (also termed Madry-AT) (Madry al. 2018), a common AT setup is to train the model on both adversarial samples and clean samples (termed Hybrid-AT) (Goodfellow et al. [015 Kannan cct al. 2018) et al. (2020a). Batch normalization (BN) (loffe & SZegedy| (015)) has become a de facto standard component in modern deep neural networks (DNNs) (He et al.[2016 Huang et al. Zhang et al. 2019a 2021). however, there is a notable concern regarding how to use BN in the Hybrid-AT setup. The concern mainly stems from a two-domain hypothesis: "clean images and adversarial images are drawn from two different domains" (Xie & Yuille| (2020). Guided by this hypothesis, technique has been proposed disentangle the mixture distribution of the two domains by applying separate BN for each domain (Xie & Yuille 2020).
The above technique has been adopted in multiple works with different names, such as auxiliary BN (Xie et al. 2020a), mixture BN (Xie & Yuille 2020). Dual BN (Jiang et al. 20200] Wang et al.) 2020|[002] Despite different names, they refer to the same practice of adopting BN Nade and BN clean for adversarial and clean samples, respectively. To avoid confusion, we stick to using Dual BN for the remainder of this work. Despite its increasing popularity, the mechanism how dual BN helps Hybrid-AT remains not fully clear. Towards a better understanding of the underlying mechanism, we first revisit a long-held heed belief motivated by the two-domain hypothesis (Xie Yuille, 2020). Specifically, (Xie & Yuille (2200) justifies the necessity of dual BN in hybrid AT with the following claim (quoted from the abstract of (Xic & Yuille 2020
"Estmmating normalization statistics of the mixture distribution is challenging" and "disentangling the mixture distribution for normalization, i.e., applying separate BNs to clean and adversarial images for statistics estimation, achieves much stronger robustness."
The underlying motivation for the above claim is that BN statistics calculated on clean domain are incompatible with training the model on adversarial domain, and vice versa. Therefore, Hybrid-AT

---

Under review as a conference paper at ICLR 2023
with single BN suffers from such incompatibility with the mixed distribution for caleulating the normalization statistics. Meanwhile, it is claimed in (Xie & Yuille, 2020 that this incompatibility can be avoided by Dual BN through training the clean branch on BN and the adversarial branch on BN As a preliminary investigation of this incompatibility. our work experiments with new variant of AT with cross-domain BN, namely training the adversarial branch with BNeteon. Interestingly, we find that using BN from another domain only has limited influence on the performance. This observation inspires us to have closer look at what component in dual BN makes it more effective than single BN in Hybrid-AT. Through untwining normalization statistics (NS) and affine parameters (AP) in dual BN to include one effect while excluding the other, we demonstrate that disentangled AP instead of NS plays the main role in the merit of dual BN in Hybrid-AT. Moreover, we find that different APs in dual BN also well explain the performance discrepancy caused by the BN choice (either BN, odv or BN clean) at test time, which refutes prior claim which mainly attributes it to the role of NS (Xie & Yuille 2020).
The motivation for introducing Dual BN is inspired by two-domain hypothesis that "clean images and adversarial images are drawn from two different domains" " (Xie & Yuille, 2020). After showing their motivation does not hold, we further revisit this two-domain hypothesis itself. We reveal that the domain gap between adversarial and clean samples is not as large as claimed in prior work (Xie) & Yuille 2020). We point out hidden flaw when prior work visualizes the NS from two domains for highlighting a large adversarial-cleat domain gap. Specifically, their visualization ignores the influence of different AP, After fixing this hidden flaw, we demonstrate that this domain gap is small. Interestingly, under the same perturbation/noise magnitude, we show that there is no significant difference between adversarial-clean domain gap and noisy-clean counterpart. Therefore, we propose two-task hypothesis to replace the two-domain hypothesis in (Xie & Yuille 2020 Xie et al. 2020a) for theoretical justification on the necessity of dual BN in Hybrid AT. We design dual linear classifier experiment to verify this two-domain hypothesis which also motivates us to apply dual AP to architectures with other normalization modules. Beyond vanilla Hybrid-AT, AT, we further experiment with Trades-Ar AT (another variant of Hybrid-ATh (Zhang et al. [019b) which by default adopts single BN. We point out an NS inconsistency issue in their original implementation and demonstrate that fixing it can significantly improve performance. Moreover, we find that the KL. regularization loss in Trades-AT can also be introduced to improve vanilla Hybrid-AT in the single BN setting.
The model robustness under PGD-10 attack (PGD attack with 10 steps) and AutoAttack (AA) (Croce & Hein 2020) are evaluated in our analysis as the basic experimental settings, with more details reported in Section of the appendix and a more specific setup discussed in the context. Overall, considering the increasing interest in adopting dual BN in Hybrid-AT, our work comes timely by taking closer look at dual BN in Hybrid-AT as well as its underlying hypothesis for theoretical justification. The main findings of our investigation are summarized as follows:
In contrast to prior work that attributes the merit of dual BN in Hybrid-AT to disentangling NS, we demonstrate that what plays the major role lies in its two sets of AP.
The claimed large domain gap in prior work is caused by a hidden flaw of ignoring the impact of AP, which motivates two-task hypothesis for interpreting the task of Hybrid-AT.
As takeaways, we recommend NOT disentangling NS in Hybrid-AT, since disentangling NS has little influence on performance with dual AP and actually harms performance in the single AP setting. Moreover, with careful choice of training details, a single BN might be sufficient for achieving competitive performance.
2 PROBLEM OVERVIEW AND RELATED WORK
2.1 DEVELOPMENT OF ADVERSARIAL TRAINING
Adversarial training. Adversarial training (AT) has been the most powerful defense method against adversarial attacks, among which Madry-AT (Madry et al. 2018) is a typical method detailed as follows. Let's assume D is data distribution with (z. y) pairs and f(., 9) is a model parametrized by 8. indicates cross cross-entropy loss in classification. Instead of directly feeding clean samples from D to minimize the risk of () [f(x,, v)}, (Madry et al.] 2018) formulates a saddle problem for
2

---

Under review as a conference paper at ICLR 2023
finding model parameter 0 by optimizing the following adversarial risk:
arg min B((,)))> max ((f +0= +6;0), BES
(I)
0
where S denotes the allowed perturbation budget which is typically lp norm above adversarial training framework as Classical-AT, It adopts a two-step training procedures(inner maximization + outer minimization), and trains the robust model with only adversarial samples. Following the same procedure, (Xie Yuille 2020) proposes to train the robust model with both clean and adversarial samples, termed Hybrid-AT. The loss of Hybrid-AT is defined as follows;
norm-bounded bounded e. We term the
Liybrid = al(f(z;)) y) + (1 - (f((( + 0),))
(2)
where æ and x + ó indicate clean and adversarial samples, respectively. a is a hyper-paramettt for balancing the clean and adversarial branches, is set to 0.5 in this work following (Goodfellow et al. 2013 Xie & Yuille 2020).
Development of AT. Since the advent of Mardy-AT and Hybrid-AT, numerous works have attempted to improve AT from various perspectives. From the data perspective, (Uesato et al., 2019 Carmon 2019: Zhang al.. 2019c) have independently shown that unlabeled data can be used to improve the robustness. From the model perspective, AT often benefits from the increased model
et al.,
capacity of models (Uesato et 2019 Xie & Yuille 2020). Xii al.. 2020b Pang al. 2020
Gowal et al., 2020) have investigated the influence and suggested that a smooth activation function, like parametric softplus, is often but not always (Gowal et al. (020) helpful for AT. It has been shown in (Pang et al. 2020) that the basic training settings in AT can have a significant influence on the model performance, and suggested a set of parameters for fair comparison of AT methods. If not specified, we follow their suggested parameter settings.
2.2 BATCH NORMALIZATION IN AT
Batch normalization (BN), We briefly summarize how BN works in modern networks. For a certain layer in the DNN, denote the feature layers of a mini-batch in the DNN as B {x "--, x J. The feature layers are normalized by mean and standard deviation o as:
7 8
(3)
where 7 and / indicates the weight and bias in BN, respectively. To be clear, we refer and a as normalization statisties (NS), y and |3 as affine parameters (AP). During training, NS is calculated on the current mini-batch statistics for the update of model weights. Meanwhile, running average of NS recorded in the whole training process, which is applied for inference after training ends.
Dual BN in AT. There is an increasing interest in investigating BN in the context of adversarial robustness (Awass et al. 2020 Cheng et al. 2020 Nandy et al. 202T; Sitawarin et al.) Gong et al.) Xie
Hybrid-AT with dual BN (Xie & Yuille, 22200) Xie et al. (020a) which applics BNelcon and BN to clean branch and adversarial branch, respectively. Prior work (Xic cet al. 2020a) shows that AEs can be used to improve recognition (accuracy) by adversarial training where AES are normalized by an independent BNade. Moreover, (Xie & Yuille 2020) has shown that adding clean images in adversarial training (AT) can significantly decrease robustness performance, where such negative effects can be alleviated to a large extent by simply normalizing CEs with an independent BN stean- Inspired by their finding. (Jiang al. 2020) also adopts Dual BN in adversarial contrastive learning, showing that single BN performs significantly worse than Dual BN. Beyond Dual BN, triple BN has been attempted in (Fan al.) (202I) for incorporating another adversarial branch. Recently (Wang et al., ( has also combined Dual BN with Instance Normalization to Normalization for improving robustness. drawback of applying dual BN in Hybrid-AT AT lies in the unknown source of samples during inference, which makes it difficult to choose the test BN. Prior work (Xie & Yuille, 2020) interprets the necessity of dual BN from the perspective of an inherent large adversarial-cleat domain gap, which implicitly suggests disentangling NS (via dual BN) might be the only solution. Our work revisits how dual BN works in Hybrid-AT AT and finally proposes new interpretation from new two-task perspective, which encourages new directions of overcoming the two-task conflict in Hybrid-AT with appropriate regularization instead of dual BN.
2022). This work focuses
form Dual batch-and-Instane
3

---

Under review as a conference paper at ICLR 2023
3 EXISTING MOTIVATION AND PRELIMINARY INVESTIGATION
Existing motivation for dual BN. is hypothesized in (iie & Yuille 2020) that clean and adversarial samples from two domains, thus Dual BN should be introduced to disentangle the mixed distributions in Hybrid-AT. The basic assumption in (Xie & Yuille 2020) is that there is a clean- adversarial domain gap that separate BN is required for each branch. Compared with single BN to handle the mixed distribution, dual BN allows the two branches to be normalized by their corresponding BN only. In other words, the underlying motivation is that the BN statistics from different domain has negative influence on the training which can be avoided with Dual BN by only using statistics calculated on its own domain, i.e. BNetean for the clean branch and BN for the adversarial branch. It is reported in (Jiang et al. (020) that the model robustness with BNelean adopted at test time close to zero, which is also confirmed in our work (see Table T.
Can BNelean be compatible with the adversarial branch to yield robust model during inference?
Cross-AT. We are interested in the answer to the above question because it has significant impli- cations on whether domain -specffc BN statis- ties is better than the mixed one for Hybrid-AI There is a possibility that this incompatibility caused by the fact that the adversarial branch of Hybrid-AT is trained with BNado instead of BNelean. To test the above possibility, we per- form cross-A7' by replacing classical BN with the module Figure [ where the adversarial
inside network
NET
Scrple
Advenans
X
Artras Patess
NET
NET
Loss
Forward
x
47
Figure 1: Cross-AT: Replacing BNado with
BNeleon in the adversarial branch of Hyrbid-AT. branch is normalized by BN clean- Specifically, the cross-AT still keeps its clean branch in the Hybrid-AT for the forward propagation to get BNelean but the model weights are updated only by the adversarial branch (as shown in Figure D Such setup guarantees that BN is used and only used for the ad adversarial branch. An interpretable meaning of swapping the BN statistics from adversarial clean is as follows: it constitutes using full cross-domain BN statistics. the merit of dual BN over single BN lies in replacing mixed BN statistics with domain-specifee one, iee totally avoiding cross-domain statistics, Cross-AT with full cross-domain BN statistics might be expected to yield very low robustness. Interestingly, the results in Figure 2 show that Cross-AT achieves comparable performance as Hybrid-AT with BNado adopted during inference.
20
Acuragyht
mn.
Accurecytt)
10
8.
PytridAl.cmm Alucet
Hybnc AT. e-10
Crea 1-1 Crea AT 1-16
Crew Mi,
Gues
Epach
Epach
(a) Clean Accuracy
(b) Robustness (PGD10 Accuracy)
Figure 2: Clean and robust accuracy of Cross-AT during training. Hybird-AT is trained with dual BN, while the results of BNadv are reported for comparison since we are mainly interested in whether BN Bllean can be compatible with the adversarial branch to achieve robustness.
Remark on the preliminary investigation results. Such a compatibility of BNelean with the adversarial branch in the above preliminary Cross-AT investigation inspires us to suspect that the merit of dual BN over single BN in Hybrid-AT might not lie in disentangling the mixed distribution for normalization so that each branch is normalized by the statistics calculated on its own domain.
4 A CLOSER LOOKER AT DUAL BN IN HYBRID-AT
Introducing an auxiliary BN component, Dual BN causes two changes: (i) disentangling the mixture distribution for normalization statistics (NS) and (ii) introducing two sets of affine parameters (AP).
4

---

Under review as a conference paper at ICLR 2023
Prior works (Xie & Yuille 2020; Xie et al. 2020a) mainly highlight the effect of disentangled NS but pay little attention to that of two sets of AP.
Cross-Hybrid-AT. Inspired by the compatibility of BNelean with the adversarial branch in Cross-AT, perform Hybrid-AT with cross BN statistics.
NS
AP
Clean
PGD-10
AA
Hybnd-AT
NSade
APedo
61.34
31.67
N8daam
APrron
94.18
0.00
0.00
Choss Hybnu-AT
NSaaan
AP
59.56
31.35
22,40
Different from the setup in Section. 3 that only uses single BN, dual BN has two sets of APs. Therefore, we let each branch use its own AP but train the two branches with NS from another branch. In other words, the clean branch is trained (and tested) with NSd and the adversarial branch is trained (and tested) with NSatean. We term this setup Mooss-Hybrid-AT and its results are reported Table [ [ that swapping NS has little influence on the performance on the two branches, e.g. NSrtean in Cross Hybrid-AT achieves similar performance with NS in Hybrid-AT. Comparing standard Hybrid-AT and Cross-Hybrid-AN we highlight that they achieve comparable performance for both adversarial branch and clean branch. The original motivation of disentangling NS is to avoid the influence of NS calculated on partial (half) samples from a different domain, however, different domain distribution actually has no significant influence on the performance. Motivated by this observation, in contrast prior works that attribute the merit of dual BN over single BN in Hybrid-AT to disentangled NS, we establish the following hypothesis:
93.86
0.00
0.00
Table 1: Cross Hybrid-AT (E 16/255)).
show that NS calculated on full samples from
Conjecture 1. We conjecture that what makes Dual BN more effective than single BN in Hybrid-AT is mainly caused by two sets of AP instead of disentangled NS.
4.1 UNTWINING NS AND AP IN DUAL BN
Training setup design. As discussed above, compared with the default Hybrid-AT baseline, Dual BN brings two effects: disentangled NSs and two sets of APs. To determine the influence of each effect on the model performance, we de- sign two setups of experiments to include only one effect while excluding the other. In Setup1, we only include the effect of two sets of APs, by applying two different sets of APs (aadv/Tade and (edean)))cdan)) in the adversarial and clean branches while using the default mixture distri- bution for normalization. In Setup2, we only include the effect of two sets of NSs by only disentangling this mixture distribution with two different sets of NSs while making BN cleen and BN odv share the same set of APs. The above setups of BNs are summarized in Figure 3
ReLU
Trads
"uado
"addo chean
Pooee oboms
Patuse
Ruste
Beseun
Pedesciian
Made
Mudts--claan
Melean
Mhods
Oeteon
Taleany Caadd.
coIv
conv
corv
conv
Zuddr Zollsm
Zades Telloss
[al Hyboii AT (Default)
(b) Dual BN
(c) Setup1
(d) Setup2
Figure 3: Illustration of different BN setups.
8/255
16/2255
Setups
NS
AP
Clean
PGD-10
AA
Clean
PGD-10
AA
Single
93.70
29.86
0.44
Dual BN (BNats)
2
2
82.77
51.35
46.19
61.84
31.67
23.14
Dual BN (BNaten)
0.32
94.18
0.00
0.00
Sctup1 (APats)
2
81.86
50.99
44.63
60.02
30.89
23.43
Sctupl (AP dese)
94.74
94.30
0.00
0.00
Sctup2 (NS
2
1
85.49
49.39
42.96
55. 55.91
21.92
10.64
Setup2 (No))
1
89.22
49.48
42.96
86.35
1.08
0.00
Table 2: Test accuracy (%). For NSs, I indicates mixture distribution and 2 indicates disentangled distribution for normalziaiton. For APs, indicates single set and indicates double sets of APs. The subscripts of APade and Aieeen indicate the input data type used during training.
Results. As shown in Table 2 Dual BN (with BNad, during inference) brings significant robustness improvement over the Single BN baseline, which is consistent with findings in (Xie & Yuille, 2020). Interestingly, under the attack of PGD-10, their robustness gap is not significant, however, under AA, the Single BN achieves very low robustness (7.03% and 0.48% for = 8/255 and = 16/255, respectively). Moreover, Setupl (APado) achieves comparable robustness as that of Dual BN (BNadw)
5

---

Under review as a conference paper at ICLR 2023
for с 8/255 and 16/255, suggesting two sets of APs alone achieve similar performance as dual BN for yielding higher robustness (APade) than single BN setting. The results collaborate our conjecture that two sets of APs are key factor in improving Hybrid-AT. The effect of two sets of NSs is more nuanced: for a small perturbation = 8 8/255, disentangling mixture distribution is beneficial for boosting the robustness under strong AA; for large perturbation
16/255 this benefit is less significant. This can be explained by the fact the domain distribution mismatch is much smaller for 8/255 than that for 16/255. Note that Setup2 aNS and Setup2 (NS achieve comparable robustness for c 8/255, while their gap is significant for c = 16 Overall, we conclude two sets of APs are sufficient for avoiding the issue of low robustness against AA in Default Hybrid-AT, and achieve comparable robustness as Dual BN. Moreover, we experiment with using two sets of AP in either first half or second half of the model, which achieves slightly inferior robustness performance (see Table 11 in appendix).
4.2 DOES BN NORMALIZATION STATISTICS CHARACTERIZE DIFFERENT MODEL PERFORMANCE DURING INFERENCE?
Even though dual BN is adopted during training, only one branch of BN parameters can be adopted
during inference, and prior works (Xie & Yuille, 2020 Jiang et al. 2020) have reported a large performance gap between eN and BNadv during inference. Regarding this phenomenon, prior
work claims that "BN normalization statistics (NS) characterizes different model performance" (Xie
& Yuille 2020).
In the following, will refute this claim and show that APs play major role.
AP plays a main role in robustness evaluation. As dis- cussed above, Dual BN not only introduces two sets of NS but also two sets of affine parameters (AP). To investigate the individual role of NS and AP during inference, we conduct a swap experiment during inference as shown in
Sctups
AP
8/265
K10i0
NSa
AFais
Swap
NS
Table The model with original Dual BN achieves robustness of 51.33% and 0.10 % with BNado (NSadv and )dd and BNelcon (NSelean and AP clean), respectively. If we keep NS fixed and swap AP, the robustness increases from 0.32% to 17.1% % with the APelean replaced by APadv However, when we keep APeleon fixed, both NS.caa and NS achieve almost zero robustness. These phenomena are strong evidence demonstrating that the APs play a larger part in robustness evaluation. However, the results seem to suggest that NS is also important, For example, the robustness with the configuration of NSetean and AP is still much lower than that of NS add and APadv for which our analysis in the following section provides an explanation (See Table 4.
Table 3: Swap NS and NS in Hybrid-AT during inference.
5 A CLOSER LOOKER AT THE TWO-DOMAIN HYPOTHESIS
model trained on source domain performs poorly on new target domain when there is domain
shift (Daumé III 2007; Sun et al., 2017). With BN as the target. it is common in the literature (Li
al. 2017] Benz et al.) 2021 Schneider et al. 2020 Xie & Yuille 2020 Xie et 2020a) to indicate the domain gap by the difference of NS between two domains. For example, an early work (Li al., (017) has shown that adapting NS from the target domain during inference can improve the performance on a new target domain without retraining the model. This test-time BN adaptation has also been adopted in (Benz et al., 202T Schneider et al. 2020) for improving the model robustness against common corruptions by perceiving them (random noise for instance) new domain. With such an understanding, it is straightforward for prior works (Xic & Yuille 2020 Xie et al. 2020a Jiang et al. 2020) also perceive adversarial domain new domain.
5.1 A HIDDEN FLAW OF VISUALIZING NS IN PRIOR WORK
To highlight the two-domain gap, prior work (Xie & Yuille 2020) visualizes the difference of NS in BNadv and BNolean cleen (see Figure of (Xie & Yuille 2020)). We quote the following sentence from (Xie & Yuille 2020): "We observe that clean images and adversarial images induce significantly different rumning statistics, though these images share the same set of convolutional filters for feature extraction". With our analysis in Section. we know that the AP in BN and BNads ado is different. The clean branch and adversarial branch still have different weights, Lee. AP, even though the same

---

Under review as a conference paper at ICLR 2023
set of convolutional filters are shared. In other words, the significant difference between eleenn and NSade is induced by not only the difference between image inputs (clean images v.s. adversarial images) but also different model (AP) weights. To summarize, the NS difference between deenn and BN is characterized by two factors: (a) AP inconsistency and (b) different domain inputs.
NS (NS
-3.03
NS
-3051
NS (NSetac))
-3.07
NS
channel
0.04
channel
Figure 4: f and visualization. Randomly choose 20 channels and display the NS calculated with different APs.
In the default setup of dual BN, NSeleen is calculated on clean samples with AP Pleean while NSado is calculated on adversarial samples with AP We additionally calculate the NS on clean samples with APade (denoted as de and calculate the NS on adversarial samples with AP Aleleon (denoted as No Following 2dd and deeen to indicate AP choice with the superscript and indicate sample choice with the subscript, we can also denote NS.lea as NSclean and denote NS.do as NS To exclude the influence of AP inconsistency, we intend to compare NS between clean and adversarial samples with the same AP. In other words, the domain gap is characterized by the difference between cleon and NS elean or that between NS and NSclean Following the procedures in (Xie & Yuille 2020j, we plot different types of NS Figure by randomly sampling 20 channels of the second BN layer in the first residual block. Fig. 4 shows that there exists a gap between NSclean clean and NS adu which is consistent with the findings in (Xie & Yuille 2020). Moreover, there are two other observations from Figure First, if we fix the input samples and calculate NS with different AP, there exists a large gap, i.e. the gap between NS and NSctean, as well as the gap between NSede and NSclean. Second, those NSs with the same APs are very close to each other, e.g., NS adv and NS clean are very similar to each other, and the same applies for NS and NSclean The visualization results highlight the significance of AP in Dual BN, and is consistent with the finding in Section 4 Without considering the influence of AP, the visualization and conclusions in (Xie) & Yuille 2020) might convey a misleading message. We report the visualization results of AP in Figure which shows significant gap between AP clean and AP APade- For a quantitative comparison, we measure the Wasserstein distance between clean and adversarial branches in different layers in the appendix (see Figure 6 in the appendix), which corroborates our above finding.
An interesting phenomenon in Table 3is is that the robust- ness with eten and AP adv achieves much lower bustness(1 (tnesss(17.)) 17.1% 1%) than the original BNado (51.33%). This is caused by the fact that the NS. is calculated on AP cleann and thus there is a mismatch between NSclean and APodv. Following the notations in Fig. 4 we evaluate pretrained Hybrid-AT model by replacing its original NS.tean and NS with different NS, as shown Table Table 4 shows that given APadu. to 31.33% with NSade. Moreover, given AP both NSclean and NSelean yield almost zero clean robustness. We conclude that AP characterizes the large robustness gap between BNatean and BN during inference, instead of NS claimed in (Xie & Yuille, (020). When AP is fixed, the robustness gap between the NS calculated on clean or adversarial samples is quite subtle.
NS
AP
POD10
= 8/200
AA
PSD00
16/250
Dellank
Ng
AF.a.
APson
31.33
0.32
46.10
0.80
31.0
0/00
2131
0.00
Chose-drtsen
NS25
AP
51.33
66.55
52.71
24.30
AP
6.00
0.00
0.00
0.00
Table 4: Cross-domain but re-calibrated NS achieves comparable performance.
add. NSade clean achieves a robustness of 51.75%, which is comparable
5.2 COMPARING ADVERSARIAL-CLEAN AND NOISY-CLEAN DOMAIN GAP
As suggested in (Benz al. 2021 Schneider et al. 2020). noisy samples (im- ages corrupted by random noise) can be seen as a domain different from clean sam- ples. Adversarial perturbation is worst-case noise for attacking the model. Tak-
7

---

Under review as a conference paper at ICLR 2023
ing ResNet18 model trained on clean samples for example, we report the performance under adversarial perturbation and random noise (with the same magnitude) in Table [ As expected, the model accuracy drops to zero with adversar- ial perturbation. Under random noise of the same magnitude, find that the model performance only drops by a small margin. Given that the influence of adversarial perturbation on the model performance is significantly larger than that of random noise, it might be tempting to believe that the adversarial-clest domain gap is much larger than noisy-clean
Noise/perturbution Size
8/255
16/255
Random noise
94.0
92.7
86.6
Adversarial perturbation
94.0
0.00
0.00
Table 5: Accuracy (%) under random noise and adversarial perturbation. domain gap.
With Wasserstein distance of NS between different domains as the metric, we compare the adversarial-clenn domain gap with noisy-clean counterpart on the above ResNer18 model trained on clean samples, as shown in Figure The perturbation and noise magnitude are set to 16 255 (see
9.0L
Deptr
Figure 9 for the results of 8/255). Interestingly, we observe that there is significant difference between adversarial- clean domain gap and noisy-clean counterpart. In other words, the adversarial-clean domain gap is not as large as many might believe considering the strong performance drop caused by adversarial perturbation.
Desteree
0.02
0.00
10
D4p0: of layer
Figure 5: Comparison of adversarial- clean and noisy-clean domain gap.
5.3 INTERPRETING HYRBID-AT AT FROM A TWO-TASK PERSPECTIVE
From two-domain two-task hypothesis. Considering the adversarial-cles domain gap is similar to noisy-clean domain gap as well as a strict constraint on allowable perturbation budget, future works investigating Hybrid AT are suggested to discard the two-domain hypothesis.
Sctups
Branch
Clean
PGD10
AA
BN
61.84
31.67
22.51
Dual BN
BN deam
94.18
0,00
0.00
Dual Lineur
Linearade
00.72
28.84
16.00
I LineaTatoa
91.43
2.21
L33
Akin to prior work justifying the role of disentangling NSs Table 6: Dual linear model, c= 16/255. with the two-domain hypothesis, we provide a two-task hypothesis for justifying the importance of disentangled APs. Intuitively, with the two branches in Hybrid-AT, the model weights are trained for two tasks: one for clean accuracy and the other for robustness. Intuitively, it is difficult for a single same set of parameters realize two tasks. A common approach for handling two tasks with a shared backbone is to make the top layers unshared. Here, we experiment with a shared encoder of single BN but with dual linear classifiers. The results in Table 6 show that this setup results in similar behavior as dual BN. Such a phenomenon corroborates that disentangling APs is equivalent to making partial learnable network weights not shared between the two tasks.
Beyond BN. In contrast to the two-domain hy- pothesis, our two-task hypothesis highlights the necessity of disentangling AP instead of NS.
Model ResNer
Norn LN
Sctups Single AP
Branch
Clean
PGD10
AA
AP
75.12
18.81
11.80
APadi
26.98
16.90
Dual AP
AP
88.41
0.00
0.00
This motivates us to apply dual AP into mod-
VIT
T.
Single AP
92.11
33,00
els with other normalization modules when dis-
Dual AP
APaa
58.02
30.08
12.44
APrncn
91.60
0.00
0.00
entangling NS is not applicable. For example, layer normalization (LN) adopts sample-wise NS, and therefore it is not applicable to disen- tangle distribution-wiss NS between two domains.
Table 7: Model with Layer Normalization (LN),
c c= 16/255. We experiment with dual AP ResNet and ViT with LN and the results are reported in Table 7 (more results on other normalization are reported in Table 1 We observe that LN with dual AP results in similar behavior as BN with dual AP, either (b) or (c) in Figure 3, (see Table2). This further corroborates our two-task hypothesis which interprets the main role of dual BN as providing unshared parameters for two tasks. In Section we will show that such two-task conflict might alternatively be mitigated with an appropriate regularization.
6 BEYOND VANILLA HYBRID-AT AND TAKE-AWAY INSIGHT
During inference, whether the test sample is clean adversarial is unknown and therefore only single BN can be adopted. Prioritizing the robustness, prior work (Xie & Yuille 2020) adopts BNade at test time at the cost of clean accuracy drop. However, with single BN, it shown in (Zhang et al.)
8

---

Under review as a conference paper at ICLR 2023
2019b) that another variant of Hybrid-AT yields competitive robustness well as accuracy. Different from the basic loss in Eq [2] the adversarial branch in (Zhang et al. (019b) is trained by a KL loss and this variant of Hybrid-AT is termed Trades-AT for differentiation. Under the perturbation of < <= 16/255 (1 unlike vanilla Hybrid AT, the performance of Trade-AT with the default single BN maintains reasonably high robustness but still falls behind its counterpart adopting Dual BN (BNads during inference). Interestingly, this robustness gap can be significantly mitigated by fixing a small casily-overlooked training detail, which pointed out and discussed in the following.
A closer look at Trades-AT. With the original imple- mentation of Trades-AT (see Algorithm [ in Appendix D). clean and adversarial samples are fed to the model independently, which results in NSadu and NSetean being disentangled during training. During inference, however, the moving- -verage NS can be seen as a mixture of AS and NSctean. This mixed NS is termed NSmiz. This causes a unintended inconsistency among NSadv. NS and NSmi Even though the gap between aS and NSctean is not as large as suggested in prior works, their gap still exists especially when the perturbation size is relatively large (16/255). The unintended inconsistency caused by their gap might have a negative effect on the performance. To this end, we experiment with sending clean and adversarial samples together to train the model for removing the NS disentangling effect (see Algorithm in Appendix D), thus NS Mimz used for training both branches as well as test. Somewhat surprisingly, the performance is improved by a large margin after fixing this inconsistency problem (see the performance gap between Single BN (Algorithm and Single BN (Algorithm D ). For more
Trades-AT
Clean
PGD10
Single BN (Algorithm]
69.97
26.8
18:04
Single BN (Algorithm
75.14
29.41
21.32
Dual BN (BNode) Dual (BNoioon)
62.15
33.16
25.67
78.03
16.17
Table 8: A closer look at Trades-AT with different BN settings, = 16/255.
discussion, see Appendix D. For completeness, we also report the results of Trades-AT with dual BN in Table
After being fixed, AT (with single BN) adopts the same structure as Hybrid-AT with single BN (see Figure 3 (a)). Their difference lies in whether KI. regularization is adopted to train the adversarial branch. Therefore, we conjecture that KL regularization in Trades-AT might be
Hybrid-AT AT
Clean
PGD10
AA
Single BN (Default)
93.70
29.86
0.48
Single BN (KL loss)
68.86
33.61
23.60
Dual BN (BNadv)
61.84
31.67
22.51
Dual BN (Boo)
94.18
0.00
0.00
a key component for AT on the hybrid samples under the single BN setting. We introduce the KL loss to the Eq[2] and the results in Table
Table 9: Vanilla Hybrid-AT (Single BN)
with KL loss, < D show that vanilla Hybrid-AT (Single BN) with KL loss indeed achieves competitive performance for both robustness as well as accuracy the Dual BN setting.
16 16/255
Takeaway insight on disentangling NS and solutions for addressing two-task conftict. Our work reveals that the gap is not large as previously visualized (Xie & Yuille, 2020) but it does exist and can cause unintended inconsistency if overlooked by practitioners. As for the recommended practice, the two-domain hypothesis (Xie & Yuille 2020 Xie et al. 2020a) advocates the practice of disentangling NS. By contrast, we find that disentangling NS has little influence on performance when two sets of AP are used (see Table 2 In the case of a single AP, disentangling NS actually harms the performance. As a takeaway, we recommend the practice of NOT disentangling NS for simplicity. In addition, We show that, with a careful choice of training details, a single BN might be sufficient for achieving competitive performance in the setup of AT with hybrid samples (see Table X and Table 9. This suggests promising directions for solving the two-task conflict beyond dual BN.
7 CONCLUSION
We experiment with Cross-AT and demonstrate the compatibility of BN statistics of clean samples with the adversarial branch, which inspires to doubt the motivation in prior work for justifying the necessity of dual BN in Hybrid AT. We take a closer look at dual BN and its underlying theoretical hypothesis, which yields two intriguing findings. First, what makes dual BN effective lies in two sets of affine parameters instead of disentangled normalization statistics. Second, the adversarial-clean domain gap is not as large many might expect and it is similar its noisy-counterpart under the same perturbation/noin dual BN from the two task perspective which is shown to generalize to architectures (like ViT) using other variants of normalization modules. Finally, we investigate Hybrid-AT beyond its vanilla version and summarize recommended practices as takeaway insight for future practitioners.
magnitude. In addition, we propose a new interpretation of Hybrid-AT with

---

Under review as a conference paper at ICLR 2023
REFERENCES
Muhammad Awais, Fahad Shamshad, and Sung-Ho Bae. Towards an adversarially robust normaliza- tion approach. arXiv preprint arXiv:2006. 10000000.1007.. 2020.
Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. Revisiting batch normalization for improving corruption robustness. WACV, 2021.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John Duchi. Unlabeled data improves adversarial robustness. NewrIPS, 2019.
Minhao Cheng, Zhe Gan, Yu Cheng, Shuohang Wang, Cho-Jui Hsieh, and Jingjing Liu. Adversarial masking: Towards understanding robustness trade-off for generalization. 2020.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In ICML, 2020.
Hal Daumé III. Frustratingly easy domain adaptation. In Association of Computational Linguistics (ACL), 2007.
Lijie Fan, Sijia Liu, Pin-Yo Chen, Gaoyuan Zhang, and Chuang Gan, When does contrastive learning preserve adversarial robustness from pretraining to finetuning? NeurIPS, 2021.
Xinyu Gong, Wuyang Chen, Tianlong Chen, and Zhangyang Wang. Sandwich batch normalization: A drop-in replacement for feature distribution heterogeneity. In WACV, 2022.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03591, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In CVPR, 2017,
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. NeurlPS, 2020.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv: /803.0633 2018.
Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. ICLR workshp, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Jay Nandy, Sudipan Saha, Wynne Hsu, Mong Li Lee, and Xiao Xiang Zhu, Adversarially trained models with test-time covariate shift adaptation. arXiv preprint arXiv:2 (02::2002000066 2021.
Tianyu Pang, Xiao Yang. Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. arXiv preprint arXiv:2010.00467 2020.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. NeurIPS, 2020.
Chawin Sitawarin, Arvind Sridhar, and David Wagner. Improving the rocuracy-robustness trade-off for dual-domain adversarial training.
10

---

Under review as a conference paper at ICLR 2023
Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adaptation. In Domain Adaptation in Computer Vision Applications. 2017.
Jonathan Uesato, Cann-Bapisste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? NeurIPS, 2019.
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuet Hu, Ji Liu, and Zhangyang Wang. Once-for- all adversarial training: In-situ tradeoff between robustness and accuracy for free. arXiv preprint arXiv:2010.11882 2020.
Haotao Wang. Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Anima Anandkumar, and Zhangyang Wang. Augmax: Adversarial composition of random augmentations for robust training. NeurIPS, 2021.
Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. ICLR, 2020.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, and Quoe V Le. Adversarial examples improve image recognition. In CVPR, 2020a.
Cihang Xie, Mingxing Tan, Boqing Gong, Alan Yuille, and Quoe Le. Smooth adversarial training. arXiv preprint arXiv:2006. 11200.14550 14536, 2020b.
Chaoning Zhang, Francois Rameau, Seokju Lee, Junsik Kim, Philipp Benz, Dawit Mureja Argaw, Jean-Charles Bazin, and In So Kweon. Revisiting residual networks with nonlinear shortcuts. In BMVC, 2019a.
Chaoning Zhang, Philipp Benz, Dawit Mureja Argaw, Scokju Lee, Junsik Kim, Francois Rameau, Chanleh Bazin, and In So Kweon. Resnet or densenet? introducing dense shortcuts resnet. In WACV, 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019b.
Jingfeng Zhang, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. Where the bottleneck of adversarial learning with unlabeled data? arXiv preprint arXiv:1911.0m 2019c.
A EXPERIMENTAL SETUPS
In this work, we perform experiments on CIFARIO with ResNet18 and follow the suggested training setups in (Pang et al., 2020) unless specified. Specifically, we train the model for 110 epochs. The learning rate set to 0.1 and decays by a factor of 0. at the epoch 100 and 105. We adopt an SGD optimizer with weight decay 5 x 10-4 For generating adversarial examples during training, use Loo PGD attack with 10 iterations and step size a = 2/ 2/255. For the perturbation constraint, e is to foo 8/255 (Pang al. 2020) or 16/ 66/25 (Xie & Yuille (020). Following (Pang al. 20200), we evaluate the model robustness under PGD-10 attack (PGD attack with 10 steps) and AutoAttack (AA) (Croce & Hein 2020).
B VISUALIZATION OF NORMALIZATION STATISTICS AND AFFINE PARAMETERS
B.1 VISUALIZATION AFFINE PARAMETERS
Vioreesttnn distrrce
As a counterpart of the NS visualization (Figure in Section we visualize AP cleen and AP Aaado Figure 7 which shows a significant difference between them. For quantitative comparison, we measure the Wasserstein distance in different layers, as shown in Figure It shows a large distance between AP Palan and AP adv in all layers. APadu. the gap between NSadv adv and NSáde clean
However, with the same AP adts stays almost zero in all layers.
Degit Aapas
Figure Layer-wise discrep- ancy visualization.

---

Under review as a conference paper at ICLR 2023
sur
0.5
channel
0.30
AP
AP deat
-010
0.15
Figure 7: y and 3. Randomly chosen 20 channels for visualizing AP and APadu adv.
B.2 VISUALIZATION OF NS WHIEN AFFINE PARAMETERS ARE DISABLED
Figure S visualizes the distribution discrepancy between NS.tean and NSadu when affine parameters disabled during training and inference. The difference between NSciean and NSadv is not large as visualized in (Xie & Yuille 2020), which is consistent with the discussions in Section 5..
NS
channel
20-
04
channel
Figure 8: μ and a when AP disabled during training and inference. Randomly chosen 20 channels for visualizing NS and NSado-
B.3 COMPARISON WITH NOISY-CLEAN DOMAIN GAP
Figure shows the adversarial-clean domain gap with noisy-clean counterpart when perturba- tion/noise magnitude is 8/255, which shows the same trend with Figure 5 when perturbation/noise
magnitude is 16/255.
Distance
0.010
Marv clean
Meaty clean
0.005
0.000
3
8
10
11
12
13
15
16
17
18
19
Depth layer
0.03
Distance
Cody Craisy
0.02
0.01
13
15
Depth layer
Figure 9: Visualization of adversarial-clean domain gap and noisy-clean domain gap (perturba- tion/noise magnitude set to 8/255.
12

---

Under review as a conference paper at ICLR 2023
C OTHER RESULTS
C.1 DUAL AP EXPERIMENTS WITH DIFFERENT NORMALIZATION
In addition to the Layer Normalization (LN) results in Table we report the Dual AP results with Group Normalization (GN) and Instance Normalization (IN) Table T0 show consistent trend with that of BN and LN.
The results in Table TO
Model
Norm
Setups
Branch
Cleun
PGDIO
AA
Net
GN
Single
81.85
21.94
14.50
Dual AP
AP adde
70.27
29.36
18.30
Dual AP
APetean
91.82
0.00
0.00
Single
92.55
23.06
1.20
Dual AP
APade
52.29
25.27
16.10
Dual AP
AP
92.35
0.00
0.00
Table 10: Dual AP with Group Normalization (GN) and Instance Normalization (IN), 16/255.
C.2 FURTHER DISENTANGLING AFFINE PARAMETERS
We report the results of using two sets of AP in either first half second half of the model in Table 11 which achieves slightly inferior robustness performance than Setupl in Figure 3
Sctups
(first half)
AP (sccond half)
Clean
PGD-10
AA
Setupl (AP..de)
2
60.02
30.89
23.43
Sctup1 olPalcn
2
94,30
0.00
Sctup3 (APado
ado)
2
58.14
29.45
19.80
Sctup3 (AP
1
2
93.66
0.00
0.00
Sctup4 (APads)
2
1
59.91
30.62
22.10
Setup4 (APalcan)
2
94.05
0.00
0.00
Table 11: Test accuracy (%) with 16/255. With 1 set of NS, different sets of AP in the first and second half of the model. The subseripts of APado and Alean indicate the input data type used during training.
C.3 EVALUATE HYBRID-AR AT AND CROSS HYBRID-AT WITH DIFFERENT NS DURING INFERENCE
We evaluate Hybrid-AT and Cross Hybrid-AT with different NS during inference, as in Table 12 Table shows that the effect of more data to estimate BN by mixing NSado and NS (noted NS miiz) is limited. Under the AP celeann- the robustness with NSmir min stays zero, which is expected. adu- the robustness with NSmas is slightly lower than the corresponding NS used in training. Note that the gap between NS and NS not as large suggested in prior work but still exists, which explains the performance drop of mixed NS during inference (when disentangled NS applied during training).
Under the AP APadu-
Model
NS
AP
Clean
PGD--0
AA
Hybrid-AT
NS
APado
61.84
31.67
22,51
NSaican
AP
94.18
0.00
0.00
NSmiz
ado
61.43
18.80
NSmis
APaleun
92.00
0.00
0.00
Cross Hybrid-Al
NS
APadv
39.36
31.25
22.40
NS
APatenr
NS
AT ado
60.09
26.41
18.36
NS
AP Atoons
90.60
0.00
0.00
Table 12: Evaluate Hybrid-AT and Cross Hybrid-AT with different NS (e 16/255).
D ON THE IMPLEMENTATION DETAILS OF TRADES-AT
13

---

Under review as a conference paper at ICLR 2023
In the original implementation of Trades-AN AT, As shown in AlgorithmL] [ the clean samples and adversarial samples are sent to the model independently, which results in NSciea and NS being disentangled during training. Moreover, the NS during inference is moving average of NScca and NSceaan and we term it NSmiz for simplicity. As pointed in the main manuscript, the discrepancy among NSade. NSclean and NS minr causes unintended NS consistency. To this end, we experiment with mixing the two types of inputs before feeding them the model, which is shown in Algorithm 2 To further verify this improvement comes from feeding the clean and adversarial samples together to the model, we slightly modify Algorithm to its disentangled version (shown in Algorithm 3). We observe that Algorithm achieves significantly worse performance than that of Algorithm 2 Algorithm performs similarly as Algorithm
Tradks
Clean
PGD10
Defanlt (Algoribm)
69.97
26.8
18.04
Single BN (Algorimm)))
75.14
Single BN (Algorithm)
68.03
27.31
19.10
Table 13: A closer look Trades-AT with different BN settings, = 16/255.
2
| which is expected because both algorithms feed clean and adversarial samples independently to the model.
Algorithm Original implementation of Trades-AT
model: model (o.g.r RosNet)
yenerate_ads srverate_adv_Funtee _unner Functien of generating adversarisl sumples according to certain parametezs (e.g.r pertuzbation size) gross_entropy..o ki_fune: function of calculating
function of calculating cross-entropy loss
losa.
for (x_clean, label) in loader: load minibatch (X_clean, label) with samples
X_adv çenerate_adv__fmng (X_clea, model, arqs)) generate the adversarial samoles
calculate cress-ent.opy loss on clean samples
logits_elean model (X_clea)) cass_elean crass_enttroy__fun (logits_clean, labul)
calculate XL loss on beth clean samples and adversarial samples loss_ki kl = Fun__uun (log_softmax (model (x_adv) softmax (model (x_elean))
loas_clean
loss_k1
loss. backward
back-propagate
updacs (model)
update model weight
Algorithm 2 Our implementation of Trades-AT by feeding the clean and adversarial samples together to the model
model: model (e.g.r ResNet)
satsrate_adv__uue function of generating adversarisl samples according to certain
parsmecezs (e.g., perturbation size)
crosa_entropy__umg function of calculating cross- cross-entropy loss
fl_fund: function of calculating the loss.
for: (x_clean, label) in loader: load minibatoh (X_clean, label) with bacch_size samples
a_av generate _ave___dv___ee___eeee_eeeeeeeeee fune occlea,, model, args)) generace the adversarial samples
mix x_clean and x_adv before calculating their logits (model output)
X_mix coneat (X_clean, x_adv]
feed the clean and adversarial samples together to the model
logita_mix (x_mii)
split the loqit output for clean branch and adversarial branch logits_clcan legits_six/thttchssize logits_edv
calculate cruss-ent.opy
samples
losa_clean exose_entropy_fuee (logita_clean, label)
calculate XL loss on beth clean samples and adversarial samples losa_kl kl (log__oftmax (logits___y))) softmax (1ogita_cleann))) loss loss_clean beta loss_k]
loss. backwarrd back-pronaqate updaca (modal) t update model weight
14

---

Under review as a conference paper at ICLR 2023
Algorithm 3 Our implementation of Trades-AT by feeding the clean and adversarial samples inde- pendently to the model
model: model (e.g.r ResNet)
satrrate_adv__unee function of generating adversarial samples according to certain parsmecers (e.g., perturbation size)
gess__entrooy_funee function of ca calculating cross-antropy loss
funco function of calculating the XL. loss.
for (x_clean, 1abel) in loader: load minibatch (X_clean, label) with basch_size samples
generate_adv_ (x clean, model, args)) generate the adversarial samples
feed the clean and adversarial aamples independently to the model logits_olean model (x_cleam)
logits_adv
ca calculate cross-ent-oppy loss on clean samp' es
lass_clean crass_entrroyy_uue (logits_clenn, label)
calculate XL loss un buth closn samples and adversarial samples loss_kl (log__soooooell (logits_adv) softmax (logits_elean) loss loas_clean beta * loss_k1
lose. (ascwwackaard back-propasee updaze (model) update model weight
15


Reviews:
[{"review_id": "qIXOvqG1du", "reviewer": "Reviewer_UFTM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper firstly attempts at training a model with cross-domain BN statistics. Through untwining NS and AP in dual BN, it demonstrates that what makes it effective lies in two sets of AP instead of disentangled NS (as claimed in prior work).\nIt points out a hidden flaw in prior work for visualizing the NS to highlight a (large) domain gap between adversarial and clean samples. After fixing it, it reveals that the adversarial-clean domain gap is not as large as prior work suggests.", "summary_of_the_review": "It mainly compares with the work (Xie & Yuille, 2020) and points out some key insights different from Xie & Yuille, 2020. It performs comprehensive experiments to check certain assumptions. But it mainly use existing techniques. The technical contribution may be limited. And the application may be limited. It may still have problems to apply this method in practice. ", "strength_and_weaknesses": "\\+ Through untwining NS and AP in dual BN, it demonstrates that what makes it effective lies in two sets of AP instead of disentangled NS (as claimed in prior work).\n\n\\+ It points out a hidden flaw in prior work for visualizing the NS to highlight a (large) domain gap between adversarial and clean samples. After fixing it, it reveals that the adversarial-clean domain gap is not as large as prior work suggests.\n\n\\- The technical contribution may be limited. This paper perform a lot of experiments with the dual BN for Hybrid-AT. It tests many different AT and BN settings. But it mainly uses Hybrid-AT techniques and variants of BN. The technical contribution may be limited. \n\n\\- The application of the proposed method may be limited. The model must have BN layers so that it can be applied. Besides, it basically have two paths in the model for adversarial and clean data. During training, it knows how to activate each path since the model knows whether the data is clean or adversarial. But during inference without knowing the data is clean or adversarial, how can the model know which path to activate?  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DhP-SHuuGJb", "reviewer": "Reviewer_dNrE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper makes the interesting observation that the disentangled affine transformation is what really matters in the Mixture BN, instead of the disentangled BN statistics as previously believed by the community.", "summary_of_the_review": "please see above", "strength_and_weaknesses": "Overall, I think the message conveyed by this paper is simple yet important: The disentangled affine transformation is what really matters in the Mixture BN, instead of the disentangled BN statistics as previously believed by the community. The experimental results effectively support the conclusions. The paper is also well-written and is easy to follow for readers without expert knowledge in Mixture BN. \n\nI have one curious question for the authors:\n\nIn order to provide more evidence that clean and adversarial samples have similar BN statistics when we don't use disentangled affine transformation, could you please show similar visualization results as those in Figure 4, but remove all affine transformations in all BN layers? This can be achieved by disabling affine transformation when defining PyTorch BN layers. Usually, in normal training, disabling the affine transformations in BN layers won't harm performance much. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "8A8xa2XWUPf", "reviewer": "Reviewer_uM2g", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the motivation of applying dual BN in a hybrid adversarial training scene, where the model is trained on both adversarial samples and clean samples. In contrast to the popular belief that normalization statistics should be estimated separately for clean samples and adversarial samples to achieve stronger robustness, this paper reveals that what makes dual BN effective lies in two sets of affine parameters. Moreover, the paper demonstrates that the adversarial-clean domain gap is not as large as many might expect and it is similar to its noisy-counterpart under the same perturbation/noise magnitude. Overall, this paper raises an interesting topic but the algorithm part is a bit weak.", "summary_of_the_review": "The contribution of the paper is limited and the reason why exploring the reasonableness of the motivation for applying dual batch normalization proposed in the previous work is not clear.", "strength_and_weaknesses": "Strength:\nThis paper reconsiders the motivation of applying dual BN in adversarial training with hybrid samples and explores the effectiveness of normalization statistics and affine parameters separately.\n\nThis paper provides a closer looker at dual BN: disentangling the mixture distribution for normalization statistics and introducing two sets of affine parameters (AP). Then they find that the performance improvement mainly comes from more affine parameters.\n\nThe authors point out a visualization flaw in previous work: when AP is fixed, the robustness gap between the NS calculated on clean or adversarial samples is quite subtle.\n\n\nWeakness:\n\nThis paper lacks a description of the existing development and problems in the field of applying batch normalization in adversarial training. It is not clear why they want to explore the reasonableness of the motivation for applying dual batch normalization proposed in the previous work. For example, is there some bad phenomenon with the application of dual batch normalization? or some limitations not considered in previous work? In a word, the authors do not give a reason why they carry out this work. \n\nThe cross BN model looks like a mixture of ordinary BN and Dual BN. There is not much to take in this model. Although the authors provided two variants: setup1 and setup2, the contribution still looks incremental.\n\nIn Figure 2, Cross-AT method has a similar performance to Hybrid AT. However, from the statement that “the cross-AT still keeps its clean branch in the Hybrid-AT.., but the model weights are updated only by the adversarial branch’’, Cross-AT does not use clean samples to update the model weights, how does its performance on clean samples achieve the similar with Hybrid AT?\n\nIn the last paragraph of section 5, the paper claims that “future works utilizing dual BN in Hybrid AT are suggested to discard the two-domain hypothesis and embrace the two-task hypothesis”, the authors emphasize they have two intriguing findings. However, the authors have not clarified what contributions this work will bring to the development of the related field. \n\nIn Page 2 Line 13, the authors claim that “This conjecture is motivated by an interesting observation This conjecture is motivated by an interesting observation that swapping NS in the Hybrid-AT has little influence on the model performance”. Although the experiment results shown in Figure 2 may prove the little influence of Normalization statistics, the experiment setting itself is hard to interpret. That is, the authors may not illustrate what motivates them to this experiment or if is there any interpretable meaning about the operation of swapping NS.  \n\nIn the last paragraph of section 3, the authors claim that “Such a success inspires us to suspect that the merit of dual BN over single BN in Hybrid-AT …”. However, as the results shown in Figure 2, although there is no significant drop in performance after “Cross-AT”, there is also no significant gain to prove the effectiveness of “Cross-AT”. Therefore, the claim of “success” may not be accurate.\n\nIn table 2, although the test accuracy of Setup1(AP_{adv}) has improved compared to Single BN and Dual BN in PGD-10 and AA, the test accuracy has obviously dropped in the clean dataset. So the claim on Page 2 Line 2 “two sets of APs alone significantly improve the robustness.” may be weakly supported by the experiment.\n\nThe paper conducts a lot of experiments but lacks some necessary theoretical analysis in dual BN and Hybrid-AT. Moreover, a novel effective method is expected according to the theoretical finding.\n\n\nMinor issues:\n\nIn the second contribution listed in section 1: “We point out that it also introduces two sets of AP”, what“it”refers to is not clear.\n\nIn section 2.1, the part named “Experimental setups” is expected to be close to the experiment result in section 4 or section 5 of the article.\n\nIn Table 2, Table 3, there is no description of symbolic subscripts of AP_adv  AP_clean,.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2XcD38GHUHC", "reviewer": "Reviewer_8GPb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work reviewed the existing solution of Dual BN for adversarial robustness by focusing on the affine parameters in BN. It is argued that the stronger adversarial robustness in Dual BN is mainly attributed to additional affine parameters. Extensive ablation studies are presented to validate the argument. Moreover, it is argued that domain gap between adversarial and clean samples are not significantly larger than between noisy and clean samples, suggesting new explanations are required. \n", "summary_of_the_review": "Overall, this work presents interesting empirical insight into the effectiveness of Dual BN. However, the major weakness lies in the lack of novel solutions. It is hard to grasp useful practical guidelines for improving adversarial robustness from this work.", "strength_and_weaknesses": "strength:\n\n1. There are many empirical evaluations to support the hypotheses\n\n2. The logic and experiments are easy to follow.\n\nWeakness:\n\n1. This is more like a commentary paper pertaining to an existing paradigm, i.e. Dual BN. Despite some insights into the effectiveness of affine parameters, there is no novel algorithm proposed.\n\n2. From practical point of view, is there any recommended practice that one could adopt based on the analysis? Answering this quesiton could substantially improve the quality of this paper. \n\n3. I am wondering whether this analysis could generalize to modern architectures, e.g. ViT, where BN no longer exists.\n\nMinor:\n\n4. It is recommended to evaluate mixing up NS_adv and NS_clean in Table 1 such that the impact of using more data to estimate BN is eliminated.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject