Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposed distributed adversarial training (DAT) for robust models. The method is a combination of PGD-like adversarial training, LARS-like large batch training, and quantizing gradients for communication efficiency in distributed training. The authors show convergence of adversarial training with LARS-like learning rate under layer-wise assumptions, and empirical results of DAT can scale to 6x6=36 GPUs and batch size 6*512 for ImageNet.

Pros

+ The paper is well written and easy to follow. 
+ The convergence rate looks reasonable. 
+ It is good to show that adversarial training can be accelerated through distributed settings.
+ Extensive experiments on CIFAR-10 and ImageNet. 

Cons

- The general contribution of the paper is a bit incremental. The main message seems to be that LARS can help large-batch adversarial training?
- The convergence rate contribution seems incremental given the known adversarial training convergence proof and LARS-like convergence proof. 
- As far as I know, scaling up learning rate (maybe with warmup) performs good for large batch training, we only need LARS for a certain regime, the ImageNet setting in table 1 where the batch size is 6*512 seems to fall in the regime where scaling LR works. Did the authors scale up LR for baselines without LARS?
- The Fast FGSM method (Wong et al. 2020) uses cyclic LR etc. to make convergence fast. Are those tricks applied here? Wong et al. reported CIFAR training in about 10 min, which is faster than the baseline in table 1 (52 sec/epoch * 100 epochs). I am worried the authors may target a less significant problem than we expected.
- [Xie et al. 2019 Feature Denoising for Improving Adversarial Robustness] and Kannan et al. 2018 use more GPUs than reported in the paper. And Xie et al. achieves better robustness on ImageNet. What is the practical advantage of this paper compared to Xie et al? What are the insights from the authors’ reimplementation of Xie’s method on CIFAR? ( DAT-LSGD in table 1)
- Why does communication time decrease when more GPU machines (24) are used?



==================== post rebuttal ==============================

I do not think my concerns are addressed by the discussion. However, I also think this is a well-written paper in general and I will not be upset if it is accepted. 

My main concerns,
(1) The authors fail to show that the proposed method is non-trivial. I think this concern is raised by multiple reviewers. The authors keep clarifying the technical difficulty (especially the theory) of applying LARS etc, but my main concern is the necessity of these knobs added by authors. After a few rounds of discussion, we reach to a conclusion that adversarial training is different from standard training, but I do not think that could be considered insights from this paper. I would strongly suggest authors consider explaining why LARS is necessary by either theory or intuitive insights, and make it clear what exactly the difference is. 
(2) The authors claim contributions for large scale setting (ImageNet with large number of available GPUs), but the experiments are somewhat worse than previous results. Lacking computation resources is a good excuse, but since the authors claimed they can use larger batch size with smaller number of GPUs, I do not see a technical reason why they cannot use their method to re-run the large scale experiments to directly compare with previous results.
review_title: Clearer message on when to use this method

final_score: rating: 5: Marginally below acceptance threshold
review_content: This work introduces a framework, called DAT, to scale out adversarial training to distributed settings. DAT combines three techniques: one-shot fast gradient sign method for more efficient inner maximization, gradient quantization for reduced communication volume, and layer-wise adaptive learning rate optimizer for large-batch training.  Equipped with the proposed techniques, the authors obtain promising speedups to perform adversarial training against ResNet18 and ResNet50 on CIFAR-10 and ImageNet with multi-node and multi-GPU, while achieving comparable (roughly under 2% difference) accuracy.
 
 
Pros:
- By combing all the techniques, the proposed framework yields promising results for distributed adversarial training.
- The paper shows that sparse gradients and large-batch training scheme also apply to adversarial training and provide some theoretical analysis to show the convergence rate of the composite of these schemes. 
 
Cons:
- The technical contribution of the work seems to be limited. All techniques used are from existing works. Therefore, the main contribution is on applying these techniques in the context of adversarial training. 
- Several optimizations employed by the framework have a subtle impact on model convergence. Therefore, although the training time for a specific run might be reduced, the overall development time and complexity might increase. 
 
Comments: 

Adversarial training is slow, and the paper looks into how to accelerate AT through distributed training. The paper correctly identifies several techniques, such as large-batch, gradient compression, and FGSM, to improve the computation efficiency on single GPU as well as reducing the communication volume across GPUs and machines. The evaluation is thorough and convincing. The main concern is on the novelty side. All techniques are from existing work and have been studied heavily: FGSM reduces the amount of computation for inner maximization; gradient quantization reduces the communication volume; large-batch training improves GPU utilization. Therefore, the contribution is mainly on creating a framework that combines all the techniques and demonstrates empiricdally that they help scale out AT.
 
The reviewer appreciates the theoretical analysis of the convergence rate of DAT. However, similar convergence proofs have been given in each of the individual techniques in the past, as in [1-3]. Why isn't the convergence of DAT a simple composition of the three, or is there a more specialized aspect in the convergence proof for DAT?
 
Another main concern is the actual convergence impact of the composed framework. Each of the proposed techniques would have an impact on model convergence and is known to make training more difficult, e.g., each may lead to model divergence, slow convergence, or convergence to suboptimal local minima. How does DAT help avoid these challenges or does DAT actually impose more challenges because it involves the interaction of all three methods?
  
Question:

The evaluation does not include a comparison of Fast-AT under the 6x6 setting. Is it because there is no performance gain when training Fast-AT across multiple servers, or is it because Fast-AT does not support multi-node training? If it is the latter case, then it seems to be better to add Fast-AT (6x6) as a baseline since it does not seem to be too difficult to extend AT with DDP in PyTorch or multi-node training in TensorFlow, and it would show a better gap between the current multi-GPU multi-node training and DAT.  
 
[1] Wang et. al. "On the Convergence and Robustness of Adversarial Training ", http://proceedings.mlr.press/v97/wang19i/wang19i.pdf

[2] You et. al., "Large Batch Optimization for Deep Learning: Training BERT in 76 minutes", https://arxiv.org/abs/1904.00962

[3] Alistarh et.al., "The Convergence of Sparsified Gradient Methods", https://papers.nips.cc/paper/7837-the-convergence-of-sparsified-gradient-methods.pdf
review_title: Official Blind Review #1

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Adversarial training is a principled approach towards robust neural networks against adversarial attacks, but it is extremely computing intensive. This work tackles the problem by leveraging the general distributed training method, and addressing the problems of direct application by several effective innovations. 

Strength:

+ The proposed method is practical.

+ The proposed DAT can deal with both labeled data (supervised learning) and partial unlabeled data (semi-supervised).

+ I like the idea to use gradient quantization/compression.

+ Make theorectical conribution by convergence analysis for DAT with LALR and gradient quantization.

Additional comments and questions:

1. One conclusion is the paper can speed up by 3 times with 6 times resource. Please comment on what's the typical speedup of distributed training with n times of resources. Do you think you can further speedup?

2. In formula (2), additional regularization term by lamba is used, making it different from the direct generation of (1) into multiple workers. Why we must introduce the regularization term in DAT?

3. Please provide more details about how to measure communication time please? any profiler used?

4. In table 2, additional unlabeled data can improve accuray. Why?

5. Which deep learning framework is used to support gradient quantization?
review_title: The paper proposes a novel and comprehensive distributed learning method for speeding up adversarial training with multiple computing nodes.

final_score: rating: 4: Ok but not good enough - rejection
review_content: Hello authors,

Thank you for the submission.  I found it to be well-written and I enjoyed reading it.  The generic description of DAT given in Equation 2 is nice.  However, there are some concerns on my end that I hope you can help me alleviate that are keeping me from giving the paper a positive review.  I have an open mind and can be convinced.

1. Why is the solution of DAT not obvious?

The authors claim that "the direct solution [for scaling AT] of distributing the data batch across multiple machines may not work", but I cannot understand why when Algorithm 1 looks like exactly that.  The adversarial samples are generated on each machine at the beginning of each iteration so as to avoid computing them on the server at the beginning of each iteration.  To me, this is an unsurprising and obvious optimization.  Other than that minor variation, this looks exactly like the standard parameter server approach.

2. Are the experiments really fair here?

The most clear claim appears to be from Table 1 and is mentioned in the introduction, where using DAT-PGD with 6 nodes gives a 3x speedup over AT while preserving standard test accuracy and almost preserving robust accuracy.  However, compare with when LALR is *not* used: then, the accuracy is significantly degraded.  So a natural question that I would like to understand is what LALR does when used with the original AT algorithm.  In fact, to get a clear apples-to-apples comparison, I would think that it would be best to use AT with a 6*512 batch size, plus LALR.  Basically, what I mean is that I am not sure if the preservation of accuracy is entirely due to LALR.  If it is, this implies that LALR can be used off-the-shelf with AT in a *non*-distributed setting with larger batch size, and that could result in a situation where DAT-PGD + LALR significantly underperforms AT + LALR---which would contradict the result that DAT is able to preserve accuracy, instead giving the result that LALR improves accuracy.

Alternately, I might expect that DAT-PGD without LALR but with a batch size of 512/6 on each of the 6 nodes (giving the same as the original batch size) would give comparable performance.

I hope the gist of my comment is clear here: it seems like the comparison is not really a fair comparison.  I think it muddles improvements from DAT and improvements from LALR, and makes it hard for the reader to understand what is really going on.

Some additional small things:

 - In (1) boldface y is used, but in (2), non-boldface y is used.  Perhaps these should be made the same?
 - "Server" instead of "Sever" in Algorithm 1
 - In Section 4 there seems to be redundancy: "see the meta-form of DAT in Algorithm 1 and details in Algorithm A1. " but I believe it is meant that both references should point at the same thing.
 - In Section 4, it is written that "DAT takes a M times fewer iteration than AT".  (Also there are some grammar issues in that sentence.)  It may be clearer to say "M fewer gradient updates on the server".
 - In Section 4, it is claimed that the computation time of DAT with iterative PGD is K times more than that of DAT with FGSM.  I know that I am being pedantic but this is not correct---to make it correct, the computation time of *the adversarial perturbation generation step* is K times more, not the overall computational time.
 - Use \citet not \citep at the end of the sentence starting with "Indeed, we will show ..." at the top of page 5.
 - What is the unit for training time in Table 1?  Does that include communication time?

Overall, I am open to discuss my thoughts, but in my current understanding, I am not able to make a strong enough case for DAT to be published at ICLR.

## Post-rebuttal comments

I am impressed at the amount of time the authors have spent trying to clarify the different concerns.  But unfortunately my concerns are not fully addressed, and also a new concern arises: if this much space had to be spent clarifying different confusions of the reviewers, I think the paper could do with a complete overhaul.  I would suggest that the authors take into account the general confusions that arose in this review process and rewrite the paper such that those particular confusions are alleviated.  For instance, I struggled for clarity on what makes this contribution non-trivial, whether the contributions are primarily theoretical or primarily empirical (and how I should understand the balance between the two), and the fairness of the empirical comparison.  Rewriting the paper such that those three concerns---and the others pointed out by the other reviewers---are discussed clearly would be a significant improvement.
review_title: I have some concerns about differences from existing work and the experiments

Meta-review: In this paper, the authors claim to propose a distributed large-batch adversarial training framework to robustify DNN.
Although the authors made efforts to clarify reviewers' concerns,  it is clear that the authors still cannot convince some reviewers in several points after several rounds of discussion between reviewers and authors.

The reviewers were not in consensus on acceptance and some concerns were still not clearly addressed in the rebuttal phase.
Hence, I recommend acceptance only if there is a room. 

Decision: Reject