Adversarial Feature Augmentation and Normalization for Visual Recognition
Authors: Tianlong Chen
Yu Cheng
Zhe Gan
Jianfeng Wang
Lijuan Wang
Jingjing Liu
Zhangyang Wang
Venue: Accepted by TMLR ()

Abstract: Recent advances in computer vision take advantage of adversarial data augmentation to improve the generalization of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings, instead of relying on computationally-expensive pixel-level perturbations. We propose $\textbf{A}$dversarial $\textbf{F}$eature $\textbf{A}$ugmentation and $\textbf{N}$ormalization (A-FAN), which ($i$) first augments visual recognition models with adversarial features that integrate flexible scales of perturbation strengths, ($ii$) then extracts adversarial feature statistics from batch normalization, and re-injects them into clean features through feature normalization. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks, including ResNets and EfficientNets for classification, Faster-RCNN for detection, and Deeplab V3+ for segmentation. Extensive experiments show that A-FAN yields consistent generalization improvement over strong baselines across various datasets for classification, detection, and segmentation tasks, such as CIFAR-10, CIFAR-100, ImageNet, Pascal VOC2007, Pascal VOC2012, COCO2017, and Cityspaces. Comprehensive ablation studies and detailed analyses also demonstrate that adding perturbations to specific modules and layers of classification/detection/segmentation backbones yields optimal performance. Codes and pre-trained models are available in: https://github.com/VITA-Group/CV_A-FAN.

Published in Transactions on Machine Learning Research (07/2022)
Adversarial Feature Augmentation and Normalization for Visual Recognition
Tianlong Chen
tienlong.chennuutemss.d
University of Tezos at Austin
Yu Cheng Microsoft Research
Ju..chenggmmicrroosft.ccmm
Zhe Gan
zhe.gansmmicroooft.com
Microsoft Cloud & AT
Jianfeng Wang
jannfu@micrrooof..com
Microsoft Cloud &
Lijuan Wang Microsoft Cloud & AT
kijuamw@microsoft..com
Jingjing Liu Tsinghua University
JJLiuuBair.singhuuuedeu
Zhangyang Wang University of Tepos at Austin
atlaswong@uteras..dm edu
Reviewed OpenReview: https: openreview. net/ formm? id- 2VEUI
Abstract
Recent advances in computer vision take advantage of adversarial data augmentation to improve the generalization of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings, instead of relying on computationall--xxeesiii pixel-level perturbations. We propose Adversarial Feature Augmentation and Normalization (A-FAN), which (i) first augments visual recognition models with adversarial features that integrate flexible scales of pertur- bation strengths, (ii) then extracts adversarial feature statistics from batch normalization, and re-injects them into cloan features through feature normalization. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks, including ResNets and Efficient.Nets for classification, Faster-RCNe for detec- tion, and Deeplab V3+ for segmentation. Extensive experiments show that A-FAN yields consistent generalization improvement over strong baselines across various datasets for classi- fication, detection and segmentation tasks, such CIFAR-10, CIFAR-100, ImageNet, Pascal VOC2007, Pascal VOC2012, COCO2017, and Cityspaces. Comprehensive ablation studies and detailed analyscs also demonstrate that adding perturbations to specific modules and layers of cnaiificaaiio/deeeetttin /etpi/////etecctoo/eeetttttttt /egmeentation backbones yields optimal performance. Codes and pre-trained models are available in: https: /tt///gttttt github....... www.///oo////eeeeee VVO//ITTTGGGO//III B:0////VVAAAAAAA A-FAN.
1 Introduction
Adversarial vulnerability critical issue in the practical application of neural networks. Various attacks have been proposed challenge visual recognition models of classification, detection and segmentation (Szegedy et al., 2013; Goodfellow et al., 2014; Li et al., 2018c;b; Lu al., 2017; Liu et al., 2018a; Lu et. al., 2017;

---

Published in Transactions on Machine Learning Research (07 (07/2022)
Xie et al., 2017; Wei et al., 2018; Arnab et al., 2018; Shen et al., 2019). Such susceptibility has motivated abundant studies on adversarial defense mechanisms for training robust neural networks (Schmidt et al., 2018; Sun et al., 2019; Nakkiran, 2019; Stutz et al., 2019; Raghunathan et al., 2019; Hu et al., 2019; Chen et al., 2020; 2021; Jiang et al., 2020), among which adversarial training based methods (Madry et al., 2017; Zhang el al., 2019a), 19), leveraging augmented adversarial examples, have achieved consistently superior robust.ness than others. However, crafting high-quality ad adversarial examples is computationally costly, and such adversarial training often results in a negative impact on performance over clean data (Zhang et al., 2019a).
Interestingly, few advanced studies turn to in- vestigate the possibility of closing networks' gen- eralization gap (Kawaguchi et al., 2017) via ad- versarial training. Recent progress shows that using adversarial perturbations to augment input data/cm /entediin can effectively alleviate overfit- ting issues and lead to belter generalization in mul- tiple domains, including image classification (Xie et al., 2020b), language understanding (Wang et al., 2019; Zhu et al., 2020), and vision-languag, model- ing (Gan et al., 2020). However, it still suffers from expensive computational cost due to the genera- tion of pixel-level perturbations when applied to image classification. We raise the following natural, yet largely open questions:
Feedforvand
Cinesifir
Prodiction
Generaae Advenanial
Upsample
Perturbations
ROI
A-FAN
Dosader
ASPP
Classifier
Rackhane
Clossd ficetion
Detection
Segmerttzioon (eg. Decplab V3+)
(ogg. ResNess)
Faster RCNN)
Figure 1: Overview of adversarial feature augmentation and normalization
A-FAN) for enhanced image classification (left), object delection (oenler) and semantic segmentattion (right). We take ResNets (He al., 2016), Faster RCNN (Ren et al., 2016), and DeepLab V3+ (Chen et al., 2018) pipelines examples. Our proposed A-FAN mechanisms are plugged into backbone networks and /or ROT/decoder modules for classification/ www.onnneeeeeteoeeeeooooooooceet inrpcction//egmenattiooon respectively.
Q1: Can adversarial training, as data augmen- tation, broadly boost the performance of various visual recognition tosks on clean data, not only image classification, but also object delection, se-
mantic segmentation or so?
Q2: If the above answer is yes, can have more efficient and effective options for adversarial data augmenlation, e.g., avoiding the high cost of finding inpul-level adversarial perturbalions?
In this paper, we propose A-FAN (Adversarial Feature Augmentation and Normalization). a novel algorithm to improve the generalization for visual recognition models. Our method perturbs the representations of intermediate feature space for both task-specific modules (e.g., Classifiers for ResNets, ROT for Faster RCNN, and Decoder for Deeplab V3+) and generic backbones, as shown Figure Specifically, A-FAN generates adversarial feature perturbations efficiently by one-step projected gradient descent, and fastly computes adversarial features with other perturbation strengths from weak to strong via interpolation. This strength-spectrum coverage allows models to consider wide range of attack strengths simultaneously, to fully unleash the power of implicit regularization of adversarial fcatures.
Furthermore, A-FAN normalizes adversarial augmented features in a "Mixup" fashion. Unlike previous work (Zhang et al., 2017; Li et al., 2020) that fuses inputs or features from different samples, we amalgamate adversarial and clean features by injecting adversarial statistics extracted from batch normalization into clean features. Such re-normalized features serve implicit label-preserving data augmentation, which smooths the learned decision surface (Li et al., 2020). Our main contributions summarized as follows:
We introduce new adversarial feature augmentation approach to enhancing the generalization of image classification, object detection, and semantic sogmentation models, by incorporating scaled perturbation strength from weak to strong simultancously.
We also propose a new feature normalization method, which extracts the statistics from adversarial perturbed features and ro-injects them into the original clean features. It can be regarded as implicit prese-ires data augmentation that smooths the learned decision boundary (illustrated in Figure later on).
2

---

Published in Transactions on Machine Learning Research (07/2022)
We conduct comprehensive experiments to verify the effectiveness of our proposed approach over diverse tasks (CIFAR-10, CIFAR-100, ImageNet for image classification; Pascal VOC2007 and COC02017 for object detection; Pascal VOC2007, Pascal VOC2012 and Cityspaces for semantic segmentation). The substantial and consistent performance lift demonstrates the superiority of our A-FAN framework.
2 Related Work
Adversarial Attacks and Defenses. When presented with adversarial samples, which are maliciously
designed by imperceptible perturbations (Goodfellow et al., 2014; Kurakin et al., 2016; Madry et al., 2017), deep neural networks often suffer from severe performance deterioration, e.g., Szegedy et al. (2013); Goodfellow et al. (2014); Carlini & Wagner (2017); Croce & Hein (2020) for classification models and Li et al. (2018c;b); Lu et al. (2017); Liu et al. (2018a); Xie el al. (2017); Wei et al. (2018); Zhang et al. (2020); Arnab et al. (2018); Shen et al. (2019) for detection/segnne /sggmenttionnn models. To address this notorious vulnerability, numerous defense mechanisms (Zhang ct al., 2019a; Schmidt et al., 2018; Sun ct al., 2019; Nakkiran, 2019; Stutz et al., 2019; Raghunathan et al., 2019) have been proposed, such as input transformation (Xu et al., 2017; Liao et al., 2018; Guo et al., 2017; Dziugaite et al., 2016), randomization (Liu et al,, 2018c;b; Dhillon et al., 2018), and certified defense approaches (Cohen et al., 2019; Raghumathan et al., 2018). Among these, adversarial- training- based methods show superior robustness in defending state-of-tho-art adversarial attacks (Goodfellow et al., 2014; Kurakin et al., 2016; Madry et al., 2017). Although adversarial training substantially enhances model robustness, it usually comes at the price of compromising the standard accuracy (Tsipras et al., 2019), which has been demonstrated both empirically and thoorctically (Zhang et al., 2019a; Schmidt et al., 2018; Sun al., 2019; Nakkiran, 2019; Stutz et al., 2019; Raghunathan al., 2019).
Adversarial Training Ameliorates Generalization. It is unexpected, but reasonable that recent works (Xie et al., 2020b; Zhu et al., 2020; Wang et al., 2019; Gan et al., 2020; Wei & Ma, 2019) present an opposite perspective: adversarial training can be leveraged to enhance models" generolization if harnessed in the right manner. For example, Xie et al. (2020b) shows that image classification performance the clean dataset can be improved by using adversarial samples with pixel-level perturbation generation. Zhu et al. (2020) and Wang et al. (2019) apply adversarial training to natural language understanding and language modeling, both successfully achieving better standard accuracy. Gan et al. (2020) achieves similar success on various vision-and-lanme aad-language tasks. Parallel studies (Wei & Ma, 2019; Ishii & Sato, 2019) employ handcrafted or auto-generated perturbed features to ameliorate gencralization. However, adversarial training in latent feature space as a more efficient and effective alternative has, to our best knowledge, not been studied in depth, even for classification tasks. Our work comprehensively explores this possibility not only for image classification, but also for object detection and semantic segmentation which are more challenging prediction tasks and usually require much more sophisticated model structure, posing obstacles to casily exploit adversarial information for enhanced generalization.
Feature Augmentation and Normalization. Pixel-level data augmentation techniques have been widely adopted in visual recognition models,e.g. Simard et al. (1993); Schölkopf et al. (1996); Cubuk et al. (2018); Hendrycks et al. (2019) for classification, Girshick et al. (2018); Liu et al. (2016); Zoph et al. (2019) for detection and segmentation. They are generic pipelines for augmenting training data with image-level information. Adversarial samples can also serve as a data augmentation method (Xie et al., 2020b). However, feature space augmentations have not received the same level of attention. A few pioncering works propose cenerative-based feature augmentation approaches for domain adaptation (Volpi et al., 2018), imbalanced classification (Zhang et al., 2019b), and few-shot learning (Chen et al., 2019).
Feature normalization plays an important role in neural network training (Toffe & Szegedy, 2015; Li et al., 2020; Montavon et al., 2012; Li & Zhang, 1998). Ioffe & Szegedy (2015) proposes batch normalization to remove biases in the dataset, which can substantially shrink model's generalization gaps. Xie et al. (2020b) utilizes dual batch normalization to calculate statistics of adversarial and clean samples separately, therefore obtaining promising standard accuracy. Recent investigations (Ba et al., 2016; Ulyanov et al., 2016; Wu & He, 2018; Li et al., 2019; 2020) devote particular attention to normalizing features of each training instance
3

---

Published in Transactions on Machine Learning Research (07 (0222)
Adversarial Feature Augmentation
Increased Sirength of Pentubations (ic.e)
Feedforward Prediction
Generate Adversarial P Perturbations
A-FAN
(1) adv
f(2) advv
Ck 1) adv
(A)
Adversarial Feature
f
felean
Helean
(5)
adv
+
adv
Blt Price
Normalization
Celean
Backbone
felean
c
LA FAN = feelaan +
adv
Networks
1=1
Figure 2: The pipeline of A-FAN, which contains adversarial feature augmentation and adversarial feature normalization. From top to bottom, a series of adversarial feature perturbations with different strengths are generated to augment. the intermediate clean features. Then, the statistics (i.e., Padv and ondy) of perturbed features fadv are extracted and re-injected into the original clean features felean. In the end, the normalized features fmix are taken as inputs by the rest of the network, and optimized by LA-FAN with standard (Lelean) and adversarial (Ladv) training objectives.
individually, As an illustration, Li et al. (2020) leverages the first and second-order moments of extracted features and re-injects these moments into features from another instance by feature normalization. Different from them, we propose utilize feature normalization techniques to combine adversarial and clean features to smooth the learned decision surface and improve model generalization.
3 Methodology
3.1 Preliminaries: Rationale of A-FAN
Theoretical Insights. For linear classifiers, a large output margin, the gap between predictions the true label and the next. most confident label, implies good generalization (Bartlett & Mendelson, 2002; Koltchinskii et al., 2002; Hofmann et al., 2008; Kakade et al., 2008). Although this relationship is less clear for non-lincar deep neural networks, Wei & Ma (2019) establishes a similar generalization bound associated with the "all-layer margin' which depends on Jacobian and intermediate layer norms. Furthermore, Wei & Ma (2019) derives thcoretical analyses that appropriately injecting perturbations intermediate features encourages a large layer margin and leads improved generalization. A parallel study (Wang et al., 2019) presents theoretical intuitions from new perspective that introducing adversarial noises encourages the diversity of the embedding vectors, mitigates overfitting, and improves generalization for neural language models. These obscrvations make the main cornerstone for our A-FAN approach valid.
Empirical Evidences. There exist advanced studies (Xie et al., 2020b; Zhu et al., 2020; Wang et al., 2019; Gan et al., 2020; Wei & Ma, 2019) revealing that appropriately utilizing adversarial perturbations boosts generalization of deep neural networks on diverse applications. Note that these designed approaches are not defense mechanisms for adversarial robustness; instead, they serve as special data augmentation for improved performance clean samples. Different from input perturbations (Xie et al., 2020b), our work leverages adversarial perturbation in latent feature space. To further unleash the power of adversarial augmented features, we asymmetrically fuse them with clean features, which allows the model to capture and smooth out different directions of the decision boundary (Ti et al., 2020). Accordingly, A-FAN-angmented models obtain flatter loss landscape (i.e., smaller norms of Hessian with respect to model weights) and improved generalization, as supported in Table 1 and Figure 3.
4

---

Published in Transactions on Machine Learning Research (07 (022)
ResNet-566
ResNei-Ses A-FAN
Table 1: Performance and Hessian properties of ResNet-56ss with without A-FAN CIFAR-10. A smaller spectral norm or trace of Hessian indicates flatter loss landscape w.r.t, model weights.
Settings
ResNet-56s
RosNet-50s A-FAN
Standard Accuracy
93.58 0.16
94.83 +: 0.11
Spectral Norm of Hessian
23.36 1.63
12.56
Trace of Hessian
245.24 10.87
208.94 7.40
Figure 3: Loss landscape (Li et al., 2018a) of ResNet-
56s with or without A-FAN on CIFAR-10.
3.2 Notations
Our proposed A-FAN framework includes two key components: (i) adversarial feature augmentation; and (ti) advessaiial feature normalization, as shown Figure 2. Note that we introduce adversarial perturbations the intermediate feature space, instead of manipulating raw image pixels as in common practice.
Let D {x, y) denotes the dataset, where is the input image and y is the corresponding ground-truth (e.g., one-hot classification labels, bounding boxes or segmentation maps). Let f'x; 0) with = (0,, O,) represent the predictions of neural networks, where 0g and 0, are the parameters of the backbone networks and lask-specific modules, respectively. For example, 0, denotes the parameters of ResNets' classifiers; or the parameters RPN, ROI, and classifier in Faster RCNN; or the parameters of ASPP and Decoder Deeplab V3+. Adversarial training (Madry al., 2017) can be formulated as follows:
min E (x.j
max (f (, 8, [|
(1)
where 5 is the crafted adversarial perturbation constrained within lp norm ball centered at with radius The radius is the maximun magnitude of generated adversarial perturbations, which roughly indicates the strength of perturbations (Madry et al., 2017). (xxyyjD takes the expectation over the empirical objective over the dataset D. The perturbation can be reliably created by multi-step projected gradient descent (PGD) (Madry et al., 2017) (taking || lloe perturbation for example):
8.41 11 [8, + sgu((vZudv (f 8,, 0):
(2)
where a is the step size of inner maximization, sgn the sign function, and Ladv is the adversarial training objective calculated over perturbed images.
3.3 Adversarial Feature Augmentation
In this scction, we present the proposed adversarial feature augmentation mechanism. Specifically, pertur- bations are generated in the intermediate feature space via PGD (taking features from backbone O, for example):
min EI.DE
Celean max Ladv (f(x, 6:0.;) 11
where the type of Lelean = L(f(x, 0): and Ladv are determined by tasks (c.g., detection models adopt regression and classification loss). À is hyperparameter to control the influence of adversarial feature augmentation. Perturbations 8 are generated by PGD, shown in Equation 2, but on the features f(x, 0b) from the backbone network (0,) rather than on raw input images. Note that the formulation in Equation 3 only considers single perturbation strength.
To fully unleash the powerful of adversarial augmentation in the feature space, we propose an enhanced technique that utilizes a scrics of adversarially perturbed features with strength from weak to strong simultancously. In particular, we integrate the adversarial training objective with respect the feature perturbation strength interval instead of single point.
5

---

Published in Transactions on Machine Learning Research (07/2022)
Approximation. Unfortunately, the integral is intractable due to the lack of an explicit functional representation for deep neural networks. We provides approximate solution by uniformly sampling e [0, and subsequently generating augmented features U adv shown in Figure 2.
Specifically,
Σ
max 10 Selt) Laav( 0b) + 8: 0ny),
(4)
where c(() adv = f(x, 0b) + -,)(())) is the adversarial augmented feature embedding.
3.4 Adversarial Feature Normalization
In this section, introduce the proposed adversarial feature normalization. Inspired by Zhang et al. (2017); Yun et al. (2019); Li et al. (2020), we fuse clean (felean) and adversarially (fadv) perturbed features for each training sample. Specifically, normalized features fmix are crafted by normalizing clean features with adversarial feature moments. This asymmetric composition across clean and adversarial features assists networks to smooth out decision boundaries and obtain improved generalization (Li al., 2020).
Let Melean and Madv denote the first-order moment of clean feature and the i-th augmented adversarial (t) feature. Similarly, dllan and a denote the correspouding second-order moment. Their feature statistics are calculated in the routine of batch normalization (loffe & Szegedy, 2015). Note that the statistics can also
derive from other normalization approaches (Ba al., 2016; Ulyanov et al., 2016; Wu & He, 2018; Li et al., 2019), such as instance-norm. The detailed formulation is defined follows:
f == = (1) Selean Melean
Madv:
(i)
(5)
Telean
where i e (1,2,... .k} refers to the index of strength levels and k is the number of augmented features. Normalized features fmix are fed to the networks and computed as the adversarial training objective L.dv-
Algorithm Adversarial Feature Augmentation and Normalization (A-FAN).
Initialize: f(x, 0) the visual recognition model, and 0 (01, 02). f(x, 02) are intermediate fcatures. # Generate adversarial augmented features
Uniformly sample & different perturbation strength (c(t),... .... (1)) from [0, Generate adversarial perturbations 6, (e(")) with PGD, according to Equation 2 and 3. Apply (e())) the intermediate features and obtain adversarial features f(1)
for c19) E (c(2),.... ((k) do
7: Generate other augmented features faav Sadv via the efficient implementation in Section 3.3.
8: end for
9: # Generate adversarial normalized features
10: Calculate the feature statistics /elean, Geleun and (r adv 1 with batch normalization (Toffe &
Szegedy, 2015).
11: for E (1.2......4 .b} do
(i) 12: Inject adversarial feature statistics Madv: ddy into clean features felean via the normalization, and
obtain normalized features f(() according to Equation 5.
13: end for
14: Feed normalized features to the model and compute the complete objective of A-FAN Equation 6.
15: Return Training objective CA-FAN
3.5 Overall Framework of A-FAN
As presented in Figure 2, we first generate a sequence of adversarial perturbations with diverse strengths augment. the intermediate features. Then, inject perturbed feature statistics into clean features by feature

---

Published in Transactions on Machine Learning Research (07/2022)
normalization. In the end, the augmented and normalized features fii logether with clean features Setean are both utilized in the network training. In this way, adversarial training can be formulated as an effective regularization to improve the generalization of visual recognition models. The full algorithm is summarized in Algorithm 1.
After incorporating adversarial feature augmentation and normalization, the complete training objective of A-FAN can be computed as follows:
fA-FAN == Celean + +>Eis1 adv
(6)
where x = 1 is tuned by grid search'.
4 A-FAN on Image Classification
Datasets and Backbones. We consider three representative datasets for image classification: CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009), and ImageNet (Deng et al., 2009). In our experiments, the original training datasets are randomly split into 90% training and 10% validation. The early stopping technique is applied to find the top- top-performing checkpoints on the validation set. Then, the selected checkpoints are evaluated on the test set to report the performance. The hyperparameters are tuned by grid search, which are quite stable from validation to test scts based on our observations, including PGD steps, step size a, the layers to introduce adversarial perturbations, and the number of perturbations with different strength levels. We evaluate large backbone networks (ResNet-18/500 11/////////////// (He al., 2016), EfficientNet-BOO (Tan & Le, 2019)) ImageNet, and test smaller backbones (ResNct-20s/ /56)) as well on CIFAR-10 and CIFAR-100.
Training Details and Evaluation Metrics. For network training on CIFAR-10 and CIFAR-100, we adopt an SGD optimizer with momentum of 0.9, weight decay of 5 x 10-4, and batch size of 128 for 200 epochs. The learning rate starts from 0.1 and decays to one-tenth at 50-th and 150-th epochs. We also perform a linear learning rate warm-up in the first 200 iterations. For ImageNet experiments, following the official setting in Pytorch repository," 2 we train deep networks for 90 epochs with a batch size of 512, and the learning rate decay at 30-th and 60-th epoch. The SGD optimizer is adopted with a momentum of 0.9 and a weight decay of 1 10-4, We evaluate the generalization of net with Standard Testing Accuracy (SA), which represents image classification accuracy on the original clean test dataset.
Table 2: Standard testing accuracy (SA%) of ResNet- 20s/56s CIFAR-10 and CIFAR-100. Baseline de- notes the standard training without A-FAN. 1 in- dicates the improvement over SA compared to the corresponding baseline in standard training.
Table 3: Standard testing accuracy (SA%) of ResNet- 18/50/101/552 and Efficient ImageNet.
ImageNet
Settings
Bascline
Bascline
A-FAN
ResNet-18
69.38
70.25 (1 0.87)
Settings
CIFAR-10
CIFAR-100
ResNet-50
75.21
76.33 (f 1.12)
Baseline
A-FAN
Baseline
A-FAN
ResNet-101
77.10
78.14 (1 1.04)
ResNet-152
78.31
78.69 (1 0.38)
ResNet-20s
91.25
92.52
1.27)
66.92
67.89 (1 0.97)
ResNet-56s
93.59
94.82
1.23)
71.22
72.36 1.14)
Neti
77.04
77.50 (f 0.46)
CIFAR and ImtgeNet Results. We apply PGD-5 and PGD-1 to augment the feature embeddings in the last block with adversarial perturbations for CIFAR and ImageNet models, respectively. A series of adversarial augmented features are crafted with three different strengths uniformly sampled from [0,a]. where the step size = 2.5/ Table 2 and Table 3 present the standard testing accuracy of diverse models on CIFAR-10, C1FAR-100 and ImageNet. Comparing the standard training (i.e., Baseline) with our proposed A- A-FAN, here are the main observations:
A-FAN obtains a consistent and substantial improvement over standard accuracy, e.g., 1.27% on CIFAR-10 with ResNet-20s, 1.14% on CIFAR-100 with ResNet-56s, 1.12% and 0.46% on ImageNet with Re and EfficientNet-BD BO. This suggests that training with augmented and normalized
The grid is {0.01, 0.1, 0.5, 0.6 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 5, 10).
2 chttps://githin wtt////t/////eeee hwtpp/orrreeeemee//////////arrea
7

---

Published in Transactions on Machine Learning Research (07/ (222)
Table 5: Details of training and evaluation. We use the standard implementations and hyperparameters in Ren et al. (2015); Chen et al. (2018). The evaluation metrics are also follow standards in Ren et al. (2015); Chen et al. (2018). Linear learning rate warm-up for 100 iterations is applied.
Detection
Segmentation
Datasets
Pascal VOC2007
COC02017
Pascal VOC2007
Pascal VOC2012
Cityspaces
Batch Size
Iterations
11, 250
180, 000
10, 000
30, 000
30,000
Init. Learning Rate
0.008
0.01
0.01
0.01
0.1
Learning Rate Deray
x0.1 250, 750
x0.1 at 120, 000, 160, 000
Polynomial power 0.9
Polynomial power 0.9
Optimizer
SGD with momentum 0.9 and weight decay 5 10-
SGD with momentum 0.9 and weight decay 10
Eval. Metric
mAP
AP, APso. AP
mIOU
mIOU
mIOU
features generated by A-FAN effectively enhances the generalization of deep networks. We hypothesize that is because adversarial perturbed features are treated as an implicit regularization, leading to better solutions for network training.
Shallow NesNets benefit more from A-FAN than deep ResNets (e.g., 1.12% on ResNet-50 vs. 0.38% on ResNet-1522. A possible reason is that the performance of standard trained deep ResNets is alrcady saturated, leaving little room for improvement.
Furthermore, notice that A-FAN advocates different steps of PGD to achieve superior performance on diverse datasets. More ablation analyses can be found in Section 7. Meanwhile, although the robust testing accuracy is not the focus of A-FAN, we report it for completeness in Section B.1
A-FAN vs. AdvProp. We compare A-FAN with AdvProp (Xie et al., 2020b) on CIFAR-10 with ResNet-18, and ImageNet with EfficientNet- BO (Tan & Le, 2019), presented in Table 43. CIFAR-10 models are trained on single GTX1080 Ti GPU, ImageNet (batch size 256) experiments are conducted on 2x Quadro RTX 6000 GPUs with 24Gx2 memory in lotal. Since for generating feature-level perturbations, only a partial back- propagation to the target intermediate layer is needed which brings computational saving. The results also confirm our intuition that proposed A-FAN as an effective and efficient alternative for pixel-level adversarial augmentations (e.g., AdvProp), achieves competitive performance with much more less computational cost. (i.e., less running time).
Table 4: Running time per epoch and standard testing accuracy (SA%) comparison across Bascline, AdvProp, and A-FAN.
ResNet-18 CIFAR-10
Efficieut on ImageNet
Settings
SA
Time
SA
Time
Baseline AdvProp
91.30
23s
77.00
2628s
94.52 (1 0.22)
123
77.60 (1 0.60)
13352s
A-FAN
94.67 (f 0.37)
56s
77.50 (1 0.50)
6237s
5 A-FAN on Object Detection
Datascts and Backbones. We evaluate A-FAN on Pascal VOC2007 (Everingham et al., 2010) and COC02017 (Lin et al., 2014) for object detection. COC02017 is a large-scale dataset with more than ten times of data than Pascal VOC2007. Specifically, in Pascal VOC2007, we use the train and validation sets for training, and evaluate on test set; in COCO2017, we train models on the train set and evaluateon the validation set. All other implementation and hyperparameters are provided in Table 5. Our experiments choose the widely-used framework, Faster RCNN (Ren et al., 2015), for detection tasks. It is worth mentioning that the proposed A-FAN approach can be directly plugged into other detection frameworks without any change, which is left to future work. We conduct experiments with both ResNet-50 (He el al., 2016) and ResNet-101 (He et al., 2016) as backbone networks.
Pascal VOC and COCO Results. Results are presented in Table 6. All hyperparamoters of A-FAN are tuned by grid search, including PGD sleps, step size a, the layers to introduce adversarial feature
Note that the number of PGD iterations is the same for AdvProp and A-FAN, which 5 for ResNet. on CIFAR-10 and for Efficient ImageNet, since these configurations lead the best performing AdyProp (Xie et (2020a)'s Table 2).
8

---

Published in Transactions on Machine Learning Research (07 (07/2022)
Table 6: Performance of object detection on Pascal VOC2007 and COC02017 datasets. Faster RCNN is equipped with ResNet-50//eeNeet110 backbone networks, respectively. Robustness is evaluated the ResNet-101
adversarial perturbed images (Li et al., 2018e; Xie et al., 2017) via PGD-10.
Re
ResNet-101
COCO2017
Baseline
A-FAN
Baseline
A-FAN
Pascal VOC2007
ResNet-50
MosNet-101
AP (%6
33.20
33.85
36.21
37.05
Baseline
A-FAN
Baseline
A-FAN
APsa (9E)
53.92
54.73
56.90
57.31
mAP (%)
73.96
75.38
74.32
75.71
APrs (%)
35.83
36.54
39.40
40.22
Robust mAP (%)
0.86
2.43
1.71
3.85
Robust AP (%)
0.00
0.50
0.20
0.66
augmentations, and the number of perturbations with different strength levels. We find that utilizing PGD-1 to generate adversarial feature perturbations in the last layer of backbone and ROI networks of Faster RCNN, achieves the most promising performance. We adopt a = 0.3/255 for Pascal VOC2007 and a 255 for COCO2017. For both datascts, a scries of adversarial augmented features are crafted with five different strengths uniformly sampled from [0,a]. To evaluate the robustness (L.e., robust AP) of detection model (Li et al., 2018c; Xie et al., 2017), PGD-10 attack with (55 and 2.0/255 is applied.
Table 6 summarizes the results of the baseline (i.e., standard training) and A-FAN. More results with different training scttings are provided in Section B.2. Comparing standard training with our proposed A-FAN mechanism, several major observations can be drawn:
A-FAN consistently achieves substantial performance improvement across multiple backbones on di- verse datasets. Specifically, A-FAN gains 0.63%/00844 AP with ResNet-50/ ResNVet-50/ReeNee--100 RessNet-1111 on COCO2007, and 1.42% mAP with CosNeet-5/RReseette training with adversarially augmented and normalized features crafted via A-FAN significantly boosts the generalization of detection models. possible reason that utilizing adversarially perturbed features as an implicit regularization for training leads to better generalization.
on Pascal VOC2007. This demonstrates that
Detectors trained on small-scale datasct benefits more from A-FAN. For example, Faster RCNN with ResNet-50 backbone obtains an almost double mAp4 boost (i.e., 1.42% vs. 0.81%) on VOC2007 than on COCO2017. It comes as no surprise that adversarially augmented and normalized features can be regarded as data augmentation in the embedding space and therefore perform more effectively on small-scale datasets (Shorten & Khoshgoftaar, 2019). We also notice that Faster RCNN with both shallow and deep ResNets gets a similar degree of improvement.
Besides the enhanced generalization, detectors trained with A-FAN also receive better robustness, improved by 0.46% ~ 0.50% robust AP on COC02017 and 1.57% ~ 2. 1.4 robust mAP on Pascal VOC2007. Although the improved robustness still cannot hold a candle to adversarially trained models (Dai et al., 2016; Ren et al., 2016; Lin et al., 2017), it is an extra bonus from A-FAN.
A-FAN can achieve similar improvements, compared to other sivrious/identiall data augmentations (e.g., (Zoph et al., 2019)).
Table 7: Performance of object detection on Pascal VOC2007 datasets. Faster RCNN is equipped with ResNet-101 backbone networks. Robustness are eval- uated on the adversarial perturbed images (Li al., 2018c; Xie al., 2017) via PGD-10 with a 0.3/255
Comparison with Learned Data Augmenta- tion (LDA) for Object Detection. A recent work (Zoph et al., 2019) presents learned, specialized dala augmentation policies to improve generalization performance for detection models. Although it is independent of our proposed feature-level adversarial augmentation, we still provide comparison experi- ments for comprehensive investigation, as shown in Table Note that, for a fair comparison, we follow
and 2.0/255.
ResNet-101 on Pascal VOC2007
Metrics
Baseline
Bascline
A-FAN
LDA
mAP (%)
76.00
79.66
78.70
Robust mAP (%)
2.59
5.05
4 APPs shares the same meaning as mAP VOC datasets (Zhang et. al., 2020)
9

---

Published in Transactions on Machine Learning Research (07/ (07/2022)
Table 8: Performance of object detection on Pascal VOC2007 and COC02017 datasets. Faster RCNN is equipped with ResNet-50/ CeaNott55//esNet--11 ResNet-101 backbone networks. Robustness is evaluated adversarially perturbed
images (Shen et al., 2019) via PGD-10.
ResNet-50
ResNet- 101
Pascal VOC2007
Cityspaces
Pascal VOC2012
ResNet-50
Baseline
A-FAN
Baseline
A-FAN
Baseline
A-PAN
Baseline
A-PAN
mIQU (%)
71.20
72.21
73.65
74.91
mIOU (%)
61.51
62.83
76.00
76.43
Robust mIOU (%)
10.84
12.07
9.75
11.01
Robust mIOU (%)
6.77
7.06
0.51
1.11
the exact same setting Zoph et al. (2019)'s. In addition to the detailed parameters, we combine the training sets of Pascal VOC 2007 and Pascal VOC 2012, and test the trained models on the Pascal VOC 2007 test set (4953 images). From Table 7, we observe that both A-FAN and LDA obtain performance improvements by 3.66% mAP and 2.70% mAP, respectivcly. Achieved superior performance further validates the effectiveness of our proposed A-FAN.
6 A-FAN on Semantic Segmentation
Datasets and Backbones. We validate the effectiveness of A-FAN on Pascal VOC2007 (Everingham et al., 2010), Pascal VOC2012 (Everingham et al., 2015), and Cityspaces (Cordts et al., 2016) for semantic segmentation. Among these commonly used datasots, Cityspaces is large-scale datasets with more than ten times of data than Pascal 70C2007/2012 Specifically, in Pascal VOC2007, we use the train and validation sets for training, and evaluate ontest set; in Pascal VOC2012 and Cityspaces, we train models on the train set and evaluate on the validation set. All other implementation and hyperparameters are provided in Table 5. Our experiments adopt the popular framework DecpLab V3+ (Chen et al., 2018) with ResNet-50 (He et al., 2016) and ResNett-101 (He el al., 2016) as backbone net works, for segmentation lasks. Note that A-FAN can also be directly plugged into other segmentation frameworks without any change, which is left to future work.
Pascal VOC and Cityspaccs Results. Results are collected in Table 8. We adopt PGD-1 to craft adversarially augmented features with three different perturbation strengths (sampled from [0,al) in the last layer of backbone and the decoder networks of DeepLab V3+ with a = 1.0/255, 0.4/255, 0.3/255 for Pascal VOC2007, Pascal VOC2012 and Cityspaces, respectively. All hyperparameters are tuned by grid search. PGD-10 with a = 1.0/255 and e = 8.0/255 is employed to measure robustness (i.e., Robust mIOU) of segmentation models (Shen al., 2019).
From the results in Table we observe that Deeplab V3+ gains substantial performance improvement from A-FAN, which is consistent with the obser- vations on detection models. First, A-FAN en- hances the generalization of segmentation models by 1.01% 1.26% mIOU with ResNeet-50/ResNerr Ree//eReNNee 101 on Pascal VOC2012, 1.32% with ResNet-50 on Pas- cal VOC2007, and 0.43% mIOU with ResNet-50 Cityspaces. Second, A-FAN improves Deeplab V3+ more Pascal VOC2007 /212 than Cityspaces (i.e., 1.01% ~ 1.32% vs. 0.43%), where the former two datasets only have one-tenth amount of data. compared to Cityspaces. Third, Training with A-
Table 9: Ablation study of A-FAN on CIFAR-10 with ResNet-56s. AFA: adversarial feature augmentation; AFN: adversarial feature normalization (í.e., A-FAN = AFA + AFN). The settings of AFA and AFN are the same as the ones of classificatio experiments (Sec- tion 4), where AFA and AFN are applied to the features from the last residual block of ResNet-56s.
Classification
Settings
Standard Accuracy (%)
Baseline
93.59
+ AFA
94.45 (1 0.86)
+ AFA + AFN
94.82 (1 1.23)
FAN yields moderate robustness improvement (i.e., 0.29% ~ 1.25% robust mIOU) for segmentation models.
7 Ablation Study and Analyses
A Augmentation v.s. Normalization. To verify the effects of adversarial feature augmentation (AFA) and adversarial feature normalization (AFN) in A-FAN, we incrementally evaluate each module on CIFAR-10 for image classification, Pascal VOC2007 for object detection, and Pascal VOC2012 for semantic segmentation.
10

---

Published in Transactions on Machine Learning Research (07 (222)
As shown in Table 9 and Table 10, AFA improves the baseline by 0.86% SA/1.13% AP A./0.89% mIOU for classification, detection and segmentation, respectively. The combination of the two modules, AFA and AFN, gains a further performance boost by 0.37% 3A/0.29% AP/0.12% mIOu on CIFAR-10, Pascal VOC2007 and VOC2012. These results demonstrate that each proposed component contributes to improving the generalization of detection and segmentation models, and AFA plays a dominant role in boosting performance.
(4) PGD Steps
(b) Feature Numbers
(a) PGD Steps
(b) Feature Numbers
Baseline 95
Baseline
Baseline
Baseline
PGD-10
PGD-1
10
k=1
PGD-10
PGD-1
93
PGD-7
PGD-3
kr3
PGD-7
PGD-3
k-7
PGD-5
1-5
PGD-5
k-5
(c) Block Index
Step Size 94
(c) Block Index
(d) Step Size
Raseline 95.8
Baseline
Baseline
B3,
94.9
4.0/55
0,5/55
B1,B2,33
B1
1.5/255
0.1/255
B1,B3B44
92
2.02555
1.0/55
B2,B3
B2
B1,B4
1.0/255
0.3/255
B4
1.5/555
B3
0.5/255
Figure 4: Ablation study on the location and strength of introducing A-FAN to classification models. Re- sults are on CIAFR-10 dataset with ResNet- (a) PGD steps used in the generation of adversarial per-
Figure 5: Ablation study on the location and strength of introducing A-FAN to detection models. Results are on Pascal VOC2007 datasct. (a) PGD steps usod in the generation of adversarial perturbations; (b) The number of augmented features, (A in Equation 4); (c) The location to apply A-FAN, e.g., B1 means that A-FAN is applied to features from the first residual blocks in the ResNet backbone; (d) Step size a that controls the strength of crafted perturbations. The red points represent settings with top performance.
turbations; (b) The number of augmented features, (k
in Equation 4); (c) The location to apply A-FAN, e.g., B1 mcans that A-FAN is applicd to features from the first residual blocks in the ResNet backbone; (d) Step size that controls the strength of crafted perturba- tions. The red points indicate the top performance.
(a) PCD Steps Bescllne
(b) Feature Numbers Haseline 22
Table 10: Ablation study of A-FAN on Pascal VOC2007 and Pascal VOC2012 for detection and segmentation, respectively. AFA: adversarial feature augmentation; AFN: adversarial feature normaliza- tion (i.e., A-FAN = AFA + AFN). ResNet-50 is used here and our methods are applied to the last block of Backbone/ROT, RROI/eeooor Docodor. Classification in Table
PGD-10
PGD-1 1--10
k-1
66
70
PGD-7
PGD-3
PGD-5
k-5
(c) Block Index Basellnee
(d) Step Size
Settings
Detection
Segmentation
Raseline
AP (%)
mIOU (%)
Baseline
73.96
71.20
B1,B2,33
1.5/255
0.1/255
AFA APA + AFN
75.09 (1 1.13)
72.09 0.89)
75.38 (1 1.42)
72.21 (1 1.01)
TO
A-FAN on Backbone
75.06 (1 1.10)
71.98 (1 0.78)
82,83
B2
1,0/55
0.4/555
A-FAN ROI/Decoder
74.68 (1 0.72)
71.71 0.51)
A-FAN on Both
75.38 (1 1.42)
72.21 (1 1.01)
R3
0.8/255
Figure 6: Ablation studies of A-FAN's location and strength on segmentation with Pascal VOC2012.
11

---

Published in Transactions on Machine Learning Research (07/2022)
Effects Backbone v.s. ROI/Decoder. In general, detection and segmentation models can be divided into backbone and task-specific modules (e-g., RPN/RO) in Faster RCNN (Ren et al., 2016) and ASPP/Decoder in Deeplab V3+). Our proposed A-FAN can be introduced to either or both modules as shown in detailed ablations in Table 10. We observe that applying A-FAN to backbone networks (11.10% 70.7 mIOU) gains more generalization improvement. than ROT/Decoder modules (+0.72% AP/10.51% mIOU) for detection and segmentation. Incorporating A-FAN on both backbone and task-specific modules always enjoys extra performance boost, compared applying either one alone.
Effects of Location and Strength. The performance gain from A-FAN is determined by the location and strength of generated adversarial perturbations. Figure 4, 5 and 6 illustrate a comprchensive control study to investigate these relevant factors. Without losing generality, these ablation experiments and analyses are performed on backbone networks. When studying one of the factors, we choose the best configuration for the other factors.
To identify the proper location for A-FAN operation we inject feature perturbations to different blocks (e.g., B1) some combination of blocks (e.g., B2,B3), presented in Figure 4 (c) for classification, 5 (c) for detection, and 6 (c) for segmentation. We notice that applying A-FAN to features from the last block (f.e. B3 or B4) obtains the best performance, while introducing A-FAN to multiple blocks degrades generalization.
The strength of A-FAN includes the number of PGD steps and the step size a for generating adversarial features, and the number of augmented features with different perturbation strengths (k in Equation 4), as shown in Figure (a).(b),(c) for classification, (a).(b).cc for detection, and (a).(b)(cc for segmentation. Experiments show that (ResNet-18, Faster RCNN, Deeplab V3+} gains more from A-FAN with {PGD- 0.3 0.4.. and (3,5,) augmented features with different perturbation strength. These systematic evaluations reveal that: (i) weak (e.g., 0.1 (25)) adversarial perturbed features contribute marginal generalization improvements; (ii) excessively strong (e.g.. PGD-10, a = 4.0/255) A-FAN incurs performance deterioration. In summary, we observe that a proper configuration for A-FAN usnally produces high-quality augmented and normalized features, realizing enhanced visual recognition models.
5,PGD-1,PGD-I),, step size 255 255
Input Images Clean Festures
Augmented Fastures
Normalived Festures
Ingat Image
Features
Augmested
Cleam Fesauure
Features
Normalized
Figure 7: Visualization of adversarially augmented and normalized features for classification models with A-FAN, using trained ResNet-18. The fifth and sixth columns are normalized features of the third and forth columns, respectively.
Imput Image
Augimesred Features
Clean I'cature
Nand Haad
Normalized Features
Inpul Images Featunes
Augmented Features
Normalized
Figure 9: Visualization of adversarial augmented and normalized features for detection models with A-FAN, using a trained Faster RCNN. The left column shows the input image and the corresponding clean feature. The remaining four columns, from left to right, present features with an increased perturbation strength; from up to bottom, it shows augmented and normalized features alternatively.
Figure 8: Visualization of adversarially augmented and normalized features for segmentation models with A-FAN, using a trained Deeplab V3+. The fifth and sixth columns are normalized features of the third and forth columns, respectively.
12

---

Published in Transactions on Machine Learning Research (07/2022)
Comparing A-FAN with Random Noise. One straight-froward approach to augment feature embed- dings is injecting random noise, Here we replace the generated adversarial noise in our proposed mech- anism with a randomly sampled noise from Gaus- sian distribution [N0,, a²). As shown Table 11, AFA+AFN (i.e., A-FAN) achieves larger perfor- mance gain than Random Noise+AFN,, suggesting that gradient-based crafted feature augmentation benefits more to the generalization of visual recogni- tion models.
Table 11: Performance comparison between adversarial feature perturbations with the strength a and random noise sampled from a Gaussian distribution (00, a²). Results are reported on CIFAR-10 (with ResNet-566)) Pascal VOC2007, and Pascal VOC2012 for classifica- tion, detection, and segmentation, respectively. 1/4 indicates performance ppprovement//eeradddaee
CIFAR-10 10
VOC2007
vOC2212
Settings
SA (%)
AP (8)
mIOu (8)
Randonu Noire + AFN
93.36 (4 0.25)
73.91 (1 0.05)
71.23 [1 0.03)
APN A-FAN)
(1 1.23)
75.38 (1 1.42)
72:21 (1 1.01)
Visualization. Figure 7, and provide visualization of adversarially augmented features and normalized features generatod by A-FAN. Features are collected via applying A-FAN classification, detection and segmentation models ImageNet, Pascal VOC2007 and VOC2012 datasets, respectively. For better visualization, use features from the first block of backbone networks and further enlarge the magnitude of adversarial perturbations by x20 times. We notice that normalizing features by injecting adversarial statistics into clean features, seems to neutralize the excessively generated adversarial noise. It offers an explanation for the extra performance improvement by adversarial feature normalization.
8 Conclusion and Discussion
In this paper, we present A-FAN, an enhanced adversarial training method to improve image classification, object ion,ction, and semantic segmentation. By generating a series of adversarial perturbations with different strengths on feature embeddings, and fusing adversarial feature statistics with clean features, A-FAN substantially boost the recognition performance of varions models across multiple representative datasots, such as CIPAR-10/100 ImageNet, Pascal VOC2007/ /2012, COC02017 and Cityspaces. For future work, we would like extend A-FAN to more tasks and provide theoretical understanding of A-FAN.
References
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. Advances in Neural Information Processing Systems, 33:16048 16059, 2020.
Anurag Arnab, Ondrej Miksik, and Philip HS Torr. On the robustness of semantic segmentation models to adversarial attacks. In CVPR, pp. 888 897, 2018.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. arXiv preprint Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities Risk bounds and structural results. Journal of Machine Learning Research, 3(Now):463 ::433448,,- 2002.
arXiv:1607 10000044706 2016.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, pp. 801-818, 2018.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From sulf-supe pre-training to fine-tuning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pottern Recognition, pp. 7099708,44 2020.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In International Conference on Learning Representations, 2021.
13

---

Published in Transactions on Machine Learning Research (07/2022)
Zitian Chen, Yanwei Fu, Yinda Zhang, Yu-Gang Jiang, Xiangyang Xue, and Leonid Sigal. Multi-level semantic feature augmentation for one-shot learning. IEEE Transactions on Image Processing, 28(9):4594-45 4604-4605, 2019.
Jeremy M Cohen, Elan Rosenfeld, and J Zico Kolter. Cortified adversarial robustness via randomized smoothing, arXiv preprint arXiv:1902.088 02918, 2019.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, pp. 3213-3223, 2016.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In TCML, 2020.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint Xix:11880. 09501, 2018.
Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-fen: Object detection via region-based fully convolutional networks. In NewrIPS, pp. 379 387, 2016.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248 255, 2009.
Guncet S Dhillon, Kamyar Azizzadenosheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial defense. Xiv preprial orXiv:1803.0144 2018.
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. study of the effect of jpg compression on adversarial images. arXic preprint 1r0.:16608.0088, 2016.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303-33) 338,, 2010.
Mark Everingham, SM Ali Eslami, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. International journal of computer vision, 111(1):98 136, 2015.
Zhe Gan, Yen- Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for vision-and-languuage representation learning. In NewrIPS, 2020.
Ross Girshick, Tlija Radosavovic, Georgia Gkioxari, Piotr Dollár, and Kaiming He. Detectron. https: 2018
// htt.:////.ccccoooeeeeeerrreeeeee
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. arXiv preprint xf11077711.001117, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual lcarning for image recognition. In CVPR, pp. 770-778, 2016.
Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Takshminarayanan. Augmix: simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations, 2019.
Thomas Hofmann, Bernhard Schölkopf, and Alexander Smola. Kernel methods in machine learning. The annals of statistics, pp. 1171-1220 2008.
14

---

Published in Transactions on Machine Learning Research (07/2022)
Ting-Kuet Kucii Hu, Tianlong Chen, Haotao Wang, and Zhangyang Wang. Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference. Tn International Conference on Learning Representations, 2019.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint Xrx:::102..00 101,0667, 2015.
M. Ishii and A. Sato. Training deep neural networks with adversarially augmented features for small-scale training datasets. In 2019 International Joinl Conference on Neural Networks (IJCNN), pp. 1 1-8, 2019. doi: 10.1109/LJCNI 0....00.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. arXiv preprint arXiv:2010.1333 2020.
Sham M Kakade, Karthik Sridharan, and Ambuj Tewari. On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. 2008.
Kenji Kawaguchi, Leslie Pack Kaelbling, and Yoshua Bengio. Generalization in deep learning, arviv preprint DXXiv:710.00566. 2017.
Vladimir Koltchinskii, Dmitry Panchenko, et al. Empirical margin distributions and bounding the generaliza- tion error of combined classifiers. Annals of statistics, 30(1):1 10,1 2002.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXic preprint ar trXic:1607.0223 02533,, 2016.
Boyi Li, Felix Wu, Kilian Q Weinberger, and Serge Belongie. Positional normalization. In Advonces in Neural Information Processing Systems, pp. 1622 1634, 2019.
Boyi Li, Folix Wu, Scr-Nam Lim, Serge Belongic, and Kilian Q Weinberger. On feature normalization and data augmentation. arXiv preprinl ar xixx:20002.1110 2020.
Guoying Li and Jian Zhang, Sphering and its properties. Sankhyō: The Indian Journal of Slatislics, Series A, pp. 119-133, 1998.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Neural Information Processing Systems, 2018a.
Yuezun Li, Xiao Bian, Mfing-Ching Chang, and Siwei Lyu. Exploring the vulnerability of single shot module in object detectors via imperceptible background patches. arXiv preprint arXiv:1809.055 05966, 2018b.
Yuezun Li, Daniel Tian, Ming-Ching Chang, Xiao Bian, and Siwei Lyu. Robust adversarial perturbation deep proposal-based models. arXiv preprinl arXiv: 100:::800.000 1809.0566, 2018c.
Fangzhou Liao, Ming Liang, Yimpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In CVPR, pp. 1778-1787, 2018.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pp. 740-755. Springer, 2014.
Tsung-Yo Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, pp. 2117 2125, 2017.
Wei Liu, Dragomir Anguclov, Dumitru Erhan, Christian Szogedy, Scott Reed, Cheng-Yane Fu, and Alexan- der C Berg. Ssd: Single shot multibox detector. In ECCV, pp. 21 37. Springer, 2016.
Xin Liu, Huanrui Yang, Ziwei Liu, Linghao Song, Hai Li, and Yiran Chen. Dpatch: An adversarial patch attack object detectors. arXiv preprint Xiv: 806. 02299, 2018a.
15

---

Published in Transactions on Machine Learning Research (07/2022)
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsich. Towards robust neural networks via random self-ensemble. In ECCV, pp. 369-385, 2018b.
Xuanqing Liu, Yao Li, Chongruo Wu, and Cho-Jui Hsieh. Ady-bnn: Improved adversarial defense through robust bayesian neural network. aix preprint arXim:1810.01270 2018c.
Jiajun Lu, Hussein Sibai, and Evan Fabry. Adversarial examples that fool detectors. ar preprint arXiv:112..02494 2017.
Alcksander Madry, Alcksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial atlakks. arXiv preprint arXic: 06083, 2017.
Grégoire Montavon, G Orr, and Robur- Müller. Neural networks-tricks of the trade second edition. 2012.
Preetum Nakkiran. Adversarial robustness may be at odds with simplicity. ar preprinl or Xxx::1901.00522 2019.
Aditi Raghumathan, Jacob Steinhardt, and Percy S Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In NearIPS, pp. 10877 10887, 2018.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C Duchi, and Percy Liang. Adversarial training can hurt generalization. In ICMLW, 2019.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NewrIPS, pp. 91 91-99, 2015.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cmn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6): 1137 1149, 2016.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. In NeurIPS, pp. 5014 5026, 2018.
Bernhard Schölkopf, Chris Burges, and Vladimir Vapuik. Incorporating invariances in support vector learning machines. In ICANN, pp. 47-52. Springer, 1996.
Guangyu Shen, Chengzhi Mao, Junfeng Yang, and Baishakhi Ray. Advspade: Realistic unrestricted attacks for semantic segmentation. ar preprint wrXi::1910.023/,, 2019.
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):60, 2019.
Patrice Simard, Yann LeCun, and John S Denker. Efficient pattern recognition using new transformation distance. In NewrIPS, pp. 50 58, 1993.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and generalization. In CVPR, pp. 6976 6987, 2019.
Ke Sun, Zhanxing Zhu, and Zhouchen Lin. Towards understanding adversarial examples systematically: Exploring data size, task and model factors. arXiv preprint orXiv:1902. 1019, 2019.
Christian Szegedy, Wojeiech Zaremba, Tlya Sutskever, Joan Bruna, Dumitru Erhan, Tan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Mingxing Tan and Quoe V Lc. Efficientnct: Rethinking model scaling for convolutional neural networks. arXiv preprinl arXiv:1903 1905. 1tt.:1999..94,, 2019.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. Tn ICI.R, 2019.
16

---

Published in Transactions on Machine Learning Research (07/2022)
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arviv preprint (eec160070802, 2016.
R. Volpi, Morerio, S. Savarese, and V. Murino. Adversarial feature augmentation for unsupervised domain adaptation. In CVPR, pp- 5495-5504, 2018.
Dilin Wang, Chengyue Gong, and Qiang Liu. Improving neural language modeling via adversarial training. arXiv preprint arXii:1906.0050 2019.
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kuei Hu, Ji Liu, and Zhangyang Wang. Once-for-all adver- sarial training: In-situ tradeoff between robustness and accuracy for free. ar preprint arXiv:2010.11828 2020.
Colin Wei and Tengyu Ma. Improved sample complexities for deep networks and robust classification via all-layer margin. arXiv preprinl 1rXiv:1910.044 042044 2019.
Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial atlacks for image and video object detection. arXiv preprint arXic:1811.12611 2018.
Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference on compuler vision (ECCV), pp. 19, 2018.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic sogmentation and object detection. In ICCV, pp. 1369-1378, 2017.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, and Quoe V Le. Adversarial examples improve image recognition. In CVPR, 2020a.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoc V Lc. Adversarial examples improve image recognition. In CVPR, pp. 819 828, 2020b.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. Xiv preprint arxiv: 700::7004.0115 2017.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV, pp. 6023 6023-6032, 2019.
H. Zhang, W. Zhou, and H. Li. Contextual adversarial attacks for object detection. Tn 2020 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2020. doi: 10.1109 A1100/TCMM442244.200 202.09988...220002222222
Hantao Zhang, Wengang Zhou, and Houqiang Li. Contextual adversarial attacks for object detection. In ICME, pp. 1-6. IEEE, 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan.
Theoretically principled trade-off between robustness and accuracy, ar preprint arXiv:1901.08879, 2019a.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk mininization. arXiv preprint 1rXxi:1710.0000 12, 2017.
Yinghui Zhang, Bo Sun, Yongkang Xiao, Rong Xiao, and YunGang Wei. Feature augmentation for imbalanced classification with conditional mixture wgans. Signal Processing: Image Communication, 75:89-99, 99, 2019b.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding, In ICLR, 2020.
Barret Zoph, Ekin D Cubuk, Golnaz Ghiasi, Tsung-Y Yi Lin, Jonathon Shlens, and Quoc V Le. Learning data augmentation strategies for object detection. Xrxiv preprint arXiv:1906.111 11172, 2019.
17


Reviews:
[{"review_id": "ImLgH9MeXP", "reviewer": "Reviewer_tMLU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "* Strengths\n\n- The paper is overall well-written, and it contains all necessary details  for understanding the design of the proposed algorithm.\n\n- It is nice to see that three quite different vision tasks (classification, detection, segmentation) can both be benefited from the proposed method.\n\n- It is, while being obvious, a sensible choice to lift up the sample-wise (input-wise) data augmentation to feature level, so I think the proposed method is technically correct.\n\n* Weakness\n\n- My main concern is whether the proposed method would save time in a well designed training system. \n\nI would accept the claim on page 8, \"A-FAN vs. AdvProp\", where the author stated that \"since for generating feature-level perturbations, only a partial backpropagation to the target intermediate layer is needed which brings computational saving.\" \n\nHowever, generating input-level perturbations can be parallelized, i.e., we can use a separated worker (with periodic model sync) handling the generation of adversarial samples, so the actual training time is the same as training the same model (i.e. baseline in Table 4). \n\nThis, of course, doubles the need of computational resources. On the other hand, the proposed method can be implemented in the same way, as such, the actual training time would be the same as baseline in Table 4 (if we ignore the time cost of model sync and communication overhead).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this submission, the author proposed a feature-level data augmentation method, realized by adversarial training. It was demonstrated to be effective on three common vision tasks including classification, detection, and segmentation. Several ablation studies were conducted and showed different impacts over the choice of location (which layer's / block's feature would be augmented) and strength.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "* The paper used a number of text-wrapped tables, but the gap between tables and texts is too small.\n\n* Page 4, there should be standard deviations in Table 1.\n\n* Page 5, it is not necessary to present Eq. (4) first, as it was not used anyway. Eq. (5) is not correct as the approximated term does not have the subinterval term. Since the authors, essentially, sampled the strength terms (\\epsilon) uniformly, Sec 3.3 can be simplified without the logic of Eq. (4) --> numerical integration.\n\n* Page 6, Eq. (6) is somewhat confusing, the author may want to clarify that index \"i\" does not refer to feature dimension, it actually refers to the index of strengths.\n\n* Page 6. is the (re)normalization for adversarial feature a drop-in replacement for batch normalization, if so, does it omit the rescale operation, which is usually following BN?\n\n* Page 6. what is the grid for \\lambda?\n\n* For all experiments, it would be nice to specify the location of feature augmentation.\n\n*  Page 7, where is \"Section B.1\"?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "epiAG8RKfl", "reviewer": "Reviewer_aZf3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\n1) This paper introduces a general feature augmentation method to improve the performance of various models across different tasks, e.g., image classification, object detection, and semantic segmentation.\n2) The proposed feature normalization replaces the original feature statistics with a series of adversarial statistics with different levels of perturbation strength. Then, the method will optimize the model to ensure the same prediction across the various statistics in the normalization layer, so it may learn a more robust decision boundary against the distribution shift.\n3) The comprehensive experiments on CIFAR-10, CIFAR-100, ImageNet for image classification, Pascal VOC2007 and COCO2017 for object detection, and Pascal VOC2007, Pascal VOC2012, and Cityspaces for semantic segmentation demonstrate the effectiveness of the proposed method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a training framework called Adversarial Feature Augmentation and Normalization (A-FAN) to improve the visual recognition models. A-FAN firstly generates a series of adversarial augmentations on intermediate feature embeddings with different levels of perturbation strength. Then, it applies the adversarial feature normalization to replace the original batch normalization statistics with the adversarial ones. The augmented new features will be fed to the network and optimized as auxiliary losses. The proposed A-FAN can boost the performance of various models across multiple representative datasets and tasks, such as CIFAR-10/100, ImageNet, Pascal VOC2007/2012, COCO2017.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "\nSeveral concerns and suggestions for the paper:\n\nC1) This paper claims to improve the generalization ability of models, which I, unfortunately, cannot agree with. From my perspective, the proposed A-FAN only improves the performance of models rather than the generalization ability, because it's just evaluated on the conventional metrics and I.I.D. test environments. Specifically, the generalization ability usually represents the robustness under the distribution shift, domain adaptation, transfer learning, or adversarial examples. However, these settings are not evaluated by this paper, so it's improper to claim an improvement in the generalization ability.\n\nC2) This efficiency of the proposed method remains to be discussed. Although the proposed method only requires general adversarial samples on the feature level rather than the pixel level, it needs to generate multiple samples with different levels of the perturbation strength, which is still quite time-consuming.\n\nS1) The title of Section 3 is supposed to be Approach rather than Preliminaries because it contains the entire proposed approach. In fact, the Preliminaries should only be the title of Sub-section 3.1.\n\nS2) The proposed A-FAN seems to just extend the AdvProp (Adversarial Examples Improve Image Recognition, Cihang Xie, et al.) from pixel-level adversarial augmentations to feature-level adversarial augmentations. Therefore, more experimental analyses between them should be included in this paper. Yet, the performance of AdvProp is only reported in Table 4.  Is there any specific reason to explain why the performances of AdvProp are not reported in other experiments?", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "62n2Q30PBZ", "reviewer": "Reviewer_pGx2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Strengths\n1. The paper is well organized, which is easy to follow.\n2. The experimental results are somewhat good compared with the baselines.\n\n\n*Weaknesses\n1. The contribution of this paper is somewhat incremental. For example, there is no significantly technical difference between the proposed AFN and MoEx [Li et al. 2020]. The authors don’t clearly illustrate the difference between them. Importantly, the motivation of using AFN and why AFN can boost the performance on clean data should be explained much clearer in an intuitive or theoretical way.  \n2. The setting of AdvProp in Table 4 is not clear. That is to say, the numbers of iterations for PGD are same for both AdvProp and A-FAN? \n3. Some closely related works in Section of “Adversarial Training Ameliorates Generalization” should be selected as comparison methods in the experimental part.\n4. The illustrations of AFA and AFN in Table 9 are not clear.  Does AFA also use two batch norms like AdvProp? \n5. The results in Tables 10, 6 and 8 are doubtful and confused. For example, the results of “A-FAN on Both” are same as “Baseline + AFA + AFN” (i.e., Baseline + A-AFN). However, the authors claimed that A-AFN is performed only on the feature level to accelerate the training process. It seems that the results in Tables 6 and 8 are obtained by performing A-FAN on the entire network, which is the same as the commonly used pixel-level adversarial augmentation.\n6. In fact, some works have employed a single-step FGSM to make the adversarial training more computationally efficient, such as [1][2][3], which also enjoy the similar effect as multi-step PGD. Therefore, how about using these works to achieve the goal in this work. \n7. The details of random Noise are not clear. Note that for A-FAN, for each clean example, multiple different perturbations are used. How many noise examples are used for each clean example when using the random noise? In addition, it seems that Random Noise cannot improve the performance of Baseline in Table 11. This conclusion is somewhat not consistent with [4].\n8. More different adversarial attacks, such as FGSM, C&W, DeepFool, are recommended to be adopted to enhance the experiments.\n \n[1] Fast is better than free: Revisiting adversarial training. ICLR 2020.\n[2] Understanding and Improving Fast Adversarial Training. NeurIPS 2020.\n[3] Make Some Noise: Reliable and Efficient Single-Step Adversarial Training. arXiv 2022.\n[4] Dataset Augmentation In Feature Space. ICLR Workshop 2017.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an A-FAN method by using adversarial examples to boost the performance of various visual recognition tasks on clean examples. Specifically, adversarial features, i.e., adversarial examples generated on the intermediate feature embeddings, are employed instead of the commonly used image-level adversarial examples to accelerate the training process. Also, a simple normalization strategy is proposed.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Please see the above weaknesses, where the first seven comments are recommended to be answered.", "broader_impact_concerns": "There is no negative ethical implications of this work. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper describes a new data augmentation technique that perturbs features (instead of inputs) adversarially by injecting various normalization statistics. The approach is general in the sense that it can be applied to several different vision tasks since the augmentation is done at the backbone output.  This is a valuable contribution that advances the state of the art for generalization in computer vision.  The reviewers unanimously recommend acceptance.  Well done! The authors are encouraged to incorporate the promised changes that have not been incorporated yet in the final version of the paper.  
Decision: Accept (poster)