Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose introducing adversarial perturbation in the late phase of finetuning of sequence models to enable large mini-batch. The proposed method can speed up the finetuning by almost three times.
main_review: The main idea of this paper is to introduce adversarial perturbations to improve the convergence rate of large batch fine-tuning of sequence models. The proposed idea is very simple and effective. In this paper, the authors demonstrate a nearly three times speedup in BERT and RoBERTa fine-tuning on the GLUE dataset. Although the observations made in the paper such as only one PGD step is sufficient for generating perturbation, the novelty of the idea is limited. As mentioned in the paper, both aspects of the idea have been proposed and validated for training convolutional neural networks. Therefore, a throughout evaluation of sequence models at different scale on several NLP datasets is critical. My major concerns are listed blow.

major:
1) Empirical results on other datasets: Since the goal of the paper is to speed up finetuning, it is important to validate whether the proposed method can be generalized to different datasets such as SQuAD v2.

2) Empirical results on large models: The paper is motivated by the increasing cost of finetuning large sequence models. However, the paper only includes results on BERTBase model. It is important to include more results on large sequence models such as BERTLarge, Google T5 Large [1], and GPT-2/3 models. 

3) Concurrent adversarial learning: A recent work in [2] on improving the convergence rate of very large small batch training also suggests adding adversarial perturbations. More interestingly, they retain a delayed model to generate adversarial perturbations, which makes the process of training and generating adversarial perturbations completely parallel.

minor:
1) caption of Figure 2: with and with

[1] Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
[2] Concurrent Adversarial Learning for Large-Batch Training
summary_of_the_review: The authors proposed to add adversarial perturbation to enable large mini-batch finetuning of  sequence models. However, the empirical study is limited to BERTBase model on GLUE datasets.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose ScaLA to speed up the fine-tuning of large pre-trained transformer language models. Specifically, ScaLA employs adversarial training to solve the worse convergence in large-batch training. Several techniques are proposed to reduce the computational overhead. Experiments on the GLUE benchmark show faster convergence compared to existing methods.
main_review: Pros:
1. Firstly, the paper is generally well written and easy to follow.
2. The proposed method contains several techniques to reduce the computational overhead from adversarial training, leading to significant training time savings.
2. The experimental results seem satisfactory. The convergence is faster compared to the baseline. 

Cons:
1. Firstly, the core contribution, using adversarial training to improve generalization and convergence under large-batch training, has been adopted in NLP (FreeLb), which limits the novelty of the proposed paper. (A minor point, the comparison in Table 5 does not seem fair; ScaLA uses LARS optimizer for large-batch training, while FreeLb does not. This optimizer is not part of the technical contribution, and should be kept the same). 
2. FreeLb can also be used for training from scratch, which is another important setting. It is unclear whether ScaLA can also be applied to this setting.
3. The authors argued that ScaLA can help to escape sharp minimum. Other techniques like the SAM optimizer [a] also have the same effect and proved to be working for ViTs [b]. It would be great to compare against these methods.
4. It is a bit unclear to me what are the training settings for each method. It seems that due to the different convergence speeds, different methods use different numbers of training epochs/iterations. How do you choose when to halt training to report the results (e.g., in Table 1)?


[a] Foret et al., Sharpness-aware minimization for efficiently improving generalization
[b] Chen et al., When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations

summary_of_the_review: Generally, I feel this paper presents an interesting solution to solve the convergence and generalization problem of large-batch fine-tuning. However, the proposed method is somewhat similar to existing work, which limits the novelty. I would like to hear back from the authors for the final opinion.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an adversarial training algorithm to speed up the fine-tuning of large Transformer models while retaining strong performance. The authors also incorporate several tricks such as delayed perturbation to lower the computational overhead.
main_review: **Strong Points:**
- This paper is well organized and it's good to see that we can also use large batch training for the fine-tuning process.
- The authors did plenty of ablation studies to break down the impact of each component in ScaLA, and it's interesting that using delayed perturbation just has minor side effects.

**Weak Points and Questions:**
- From Table 1, ScaLA even matches or surpass the performance of small batch training, I'm curious about the performance of similar adversarial training algorithm when applied on small batch fine-tuning.
- In Algorithm 1, should line 9 also be different for classification and regression task?
- It would be better if the author can show the results of baselines when using the same fine-tuning time. Table 1 shows that ScaLA performs similar to small batch baselines but is faster, Table 2 shows that ScaLA achieves higher accuracy but is slower than large batch baselines. I would like to know the result of baseline under the same training cost (e.g., using a batch size of 512).
summary_of_the_review: In summary, this paper is well-written and i can see the efforts of the authors to make it rigorous and well-organized, the ablation studies and analysis are also very comprehensive. But I'm not sure about the novelty of this paper, ScaLA to me is more like a combination of several methods, incorporating adversarial perturbation and layer-wise optimizer. Moreover, the benefits of adversarial training on normal batch size in both NLP and CV domains are well-studied, so I think the novelty is restricted. I vote for a weak accept for the written quality and the rigorous analysis of this paper.
review_title: null

Meta-review: The submission considers a method involving adversarial training to speed up the fine-tuning of large pre-trained transformer language models. Reviewers consider it to be a borderline paper.  Many suggestions are made by the reviewers which will help improve the presentation and substance and make it more useful for the community.
Decision: Reject