Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: In this paper, the authors suggest a method for adversarial feature-level augmentation, mainly framed as an approach to improve the clean-set accuracy rather than adversarial example robustness. The authors also propose a learnable version (LALFA), to automatically learn the location and strength of the perturbations.
Overall, I think this is an interesting paper that can be considered for publication at ICLR. The following elaborates it further:

Strengths:
* Even though the adversarial feature augmentations at the feature level are not unprecedented, but the learnable tuning of the location/strength is an interesting approach that can potentially avoid expensive hyperparameter optimization. 
* The shown improvements of the best-achieved models seem consistent over the baselines, across different datasets and models.

Weaknesses:
* The obtained improvements are moderate on smaller networks and become marginal with deeper counterparts.
* The performance and the offered advantages seem quite sensitive to the choice of hyperparameters in ALFA (Tables 4 and 5). I wonder, at least, how stable/conclusive the comparisons are when transferring the selected model from one set to another, e.g. validation to test. 

Further detailed comments:
* " L-ALFA saves toilsome tuning by automatically adjusting the strength and locations of adversarial feature augmentations" => It is fair to note that this is achieved at the expense of introducing a new hyper-parameter to tune, namely \alpha. 
* An ablation study on the L1 regularization could have been useful.
* In Table 1 and some of the other numerical comparisons, except for Table 6, where standard deviations are reported: I wonder how significant are these comparisons? Are the differences meaningful when considering the intra-experiment variations?
* Insights on why MoEx, one of the three baselines compared against, is performing worse than both random noise and normal training, would be helpful.
* Equation (1) defines \delta \in B_\epsilon(x) and uses it as an offset to x: f(x+\delta;\theta). Later B_\epsilon(x) is defined as "The norm ball B_\epsilon is centered at x with radius \epsilon": These are not consistent, if \delta is used as an offset, it cannot be sampled around x.
* \mathcal{L}_{at} has been used inconsistency across equation (1) and equations (3) and (4), on the first argument; f_i(x;\theta^{(i)}+\delta^{(i)}) in equation (3) and (4) misses the second part of the network, transforming the intermediate feature maps to the required predictions.
* \mathcal{L}_{at} is used in equation 1, but is elaborated after equation 2.
* Defining both F(x;\theta) and f(x;\theta) is referring to the neural network, and its output seems unnecessary and confusing; besides, F(x;\theta) is never referenced before.
* f(x+\delta,\theta) => f(x+\delta;\theta)
* Adding a row in table 7, representing the results from ALFA, will make the direct comparison between ALFA and L-ALFA easier.
* "ResNet-18 has ... and twenty convolutional layers" => It has seventeen convolutional layers, I think. Please clarify on dim(\eta) = 20 in Table 8. Are some of the pooling layers outputs also taken?
* typo: "a unduly"
review_title:  

final_score: rating: 4: Ok but not good enough - rejection
review_content: Pros:

Method is clearly stated, and the learning adaptive perturbation strength seems novel.

Provided experimental results for large datasets like ImageNet.

Major Concerns:

The method proposes crafting adversarial perturbation at different layers of the network. A small change in features space may correspond to a large change in input space for neural networks, hence it is doubtful whether the perturbations crafted will be meaningful. The method which attempts to cause perturbations in the feature space also ensures that perturbed image is similar to original image
(imperceptibility constraint) [3].

The authors mention use of unbounded perturbation size in section 4.2 which goes against the goal of adversarial training (i.e. same prediction in the neighborhood of the image). Also, if the perturbation is unbounded then there is no requirement of Projection step in PGD (Projection is done for enforcing the boundedness of perturbation). Could the authors please clarify this?

The intuition why this method works is also not too clear, as it has been shown in [1] that adversarial training leads to drop in standard accuracy. But in this paper adversarial training has been shown to increase standard accuracy even without using any adaptive parameters (like batch normalization layers used in AdvProp [2]). This simply goes against the established facts. 

It is hard to distill what settings work well across different datasets. As seen in Table 5, for CIFAR-10, changing from 1 step PGD to 5 step PGD increases accuracy, whereas it decreases in case of ImageNet. Also there exist only a certain range of step size values for which standard accuracy increases. Could the authors kindly offer additional theoretical or intuitive explanations to clarify the same? This would
help substantially improve the submission.

Other questions:

The authors could  provide info on fooling achieved by crafted adversaries which is required for confirming the success of adversary creation. Also, the adversarial training method proposed in this paper shows minimal increase in adversarial robustness which also supplements the need for details on fooling rate. Typically adversarial training  increases its robustness [5], which is not observed with the proposed method. Could the authors clarify this?

The accuracy reported for the ImageNet models is lower than Torchvision models [4] as the authors use the same code as PyTorch repo it is unexpected. This may be due to the use of 10% training data for validation which is not required in ImageNet, as it already has a validation set. (This is important as the magnitude of improvements are not very large over the standard accuracy in case of ImageNet and is
even smaller when compared with Torchvision models). The Torchvision ResNet 50 model obtains 76.15% accuracy which is comparable to ALFA (76.23%) (present in Table 3).


As the perturbations are calculated for each layer, the complexity of the method would increase with large networks like ResNet-152. So it is unclear if the proposed method will continue to be cost efficient in comparison to pixel-based methods.

Also, could the authors clarify the network architecture used for comparison of the time complexity in Section 4.3 for ALPHA and AdvProp [2]?

Overall, this seems a very complex method without any theoretical grounding to increase the generalization performance. 

[1] Zhang, H., Yu, Y., Jiao, J., Xing, E., Ghaoui, L.E. & Jordan, M.. (2019). Theoretically Principled Trade-off between Robustness and Accuracy. Proceedings of the 36th International Conference on Machine Learning, in PMLR 97:7472-7482

[2]Xie, C., Tan, M., Gong, B., Wang, J., Yuille, A. L., & Le, Q. V. (2020). Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 819-828).

[3]Ganeshan, A., & Babu, R. V. (2019). FDA: Feature disruptive attack. In Proceedings of the IEEE International Conference on Computer Vision (pp. 8069-8079).

[4]https://pytorch.org/docs/stable/torchvision/models.html

[5] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018, February). Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations.
review_title: This paper proposes the use of adversarial training at the feature level layers to improve generalization of Neural Networks. In particular,the authors propose to use PGD (Projected Gradient Descent) adversarial training at intermediate layers to increase the standard accuracy on clean images belonging to the test set.

final_score: rating: 4: Ok but not good enough - rejection
review_content: --Paper summary--

The authors propose Adversarial Feature Augmentation (ALFA), which augment features at hidden layers by adding adversarial perturbations. Where and how strongly the augmentation is conducted is automatically optimized via training. Experimental results show that the proposed method consistently improves the performance of baselines over several datasets and network architectures.

--Review summary--

Although the motivation of this study is clear, the proposed method is not appropriately designed along with the motivation. Moreover, its novely is merginal. I vote for rejection.

--Details--

Strength

- The motivation is clear and seems reasonable. Training with adversarial perturbations is known to be effective but computationally expensive. It can be problematic when the model or training data is large-scale.
- The proposed method consistently improves the performance of baselines over several datasets and network architectures.

Weakness and concerns

- Is the computational complexity of the proposed method really small? Since the adversarial perturbation is computed for every layer, its computational complexity should be almost same with that of standard adversarial training.
- The training objective shown in Eq. (3) is not reasonable. Since the norm of \delta is upper-bounded by a certain constant \epsilon, the effect of the adversarial perturbation can be reduced just by increasing the scale of features. Are features normalized ones?
- The optimization of \eta in L-ALFA is not reasonable. Since min_\eta comes after max_\delta, L-ALFA should choose the layer that corresponds to the smallest increase of loss by adding adversarial perturbation. Therefore, this design minimizes the effect of the augmentation, which is contradictive to the motivation of introducing \eta. Moreover, since \epsilon is common for all layers, the optimal \eta should be sensitive to the scale of features, which indicates that the performance of the proposed method would heavily depend on both how to initialize the model and whether any normalization is conducted in the model or not. 
- Marginal novelty. An idea of adversarially augmented features has been already presented in [R1].   
[R1] "Training Deep Neural Networks with Adversarially Augmented Features for Small-scale Training Datasets," IJCNN 2019.

review_title: Although the motivation of this study is clear, the proposed method is not appropriately designed along with the motivation.

final_score: rating: 5: Marginally below acceptance threshold
review_content: Overview of paper: this work tackles the task of adversarial augmentation for better generalization. Instead of augmentation the pixels space, which is expensive and potentially harder, they augment the intermediate feature representation. As the choice of the particular layer for application of the perturbations affects performance, the authors, optimize it jointly with the rest of the parameters. Experiments show this method  improves accuracy over standard training.

Novelty: although adversarial training on raw image is known to improve generalization, doing so on image features is novel as far as I am aware. Additionally, jointly choosing the layers to be perturbed is also new in my understanding (although the main benefit is in the analysis, as the fixed strategy of perturbing the last block seems comparable).

Evaluation: The proposed feature adversarial training seems to consistently improve generalization on several popular datasets, however there are a few limitations: i) the gap is not huge. ii) it is not clear that the difference is significant from Xie et al. iii) nearly all results are on Cifar10, including the baseline comparisons iv) the speed up due to operating on features rather than pixels is cited as the main motivation but limited effort exists to evaluate it.

Presentation: the paper is nicely written, and is easy to follow.

Other questions: did you use auxiliary BNs like Xie et al? In what experiments did ALFA-L beat ALFA on the last block? (Sec 4.4 is a bit hazy there)

Overall: Adversarial feature perturbation for generalization is an interesting ideas and was shown to have some benefits. The evaluation of accuracy and runtime against other reasonable methods (particularly doing the same on pixels) is limited. I am positively inclined towards this paper and hope the authors can address by concerns in the rebuttal.

##########################################################################################

The response addressed some of my concerns, but I am concerned about L-ALFA taking such a large part of the paper and then shown to not help so much over just picking the final laye nor being much faster than the baseline. The more important ALFA hyperparameters that would most benefit from automatic tuning are not sufficiently treated. Although I do like the objective of this paper and some of the approaches, I think it might need to be revised and resubmitted, incorporating the extensive discussion presented by all the reviewers.   
review_title: Review

Meta-review: Adversarial training is usually done on the image space by directly optimizing the pixels. This paper suggests the adversarial training over intermediate feature spaces in the neural network. The idea is very simple. The authors have done extensive experiments to justify its performance. But the performance gain though this idea seems to be marginal. Further, the layer to conduct the adversarial training can be optimized within the framework, which aligns with the general autoML idea. The new version L-ALFA has been well introduced, but unfortunately, the practical result can be very straightforward, that is just to select the final layer. The more important ALFA hyperparameters that would most benefit from automatic tuning are not sufficiently treated.  There have been extensive discussions between the authors and the reviewers. After incorporating the reviewers' comments, the paper will have a good chance to be accepted at another venue. 


Decision: Reject