On the Computational Efficiency of Adapting Transformer Models via Adversarial Noise
Authors: Minjia Zhang
U.N. Niranjan
Yuxiong He
Venue: NeurIPS 2022 Submitted (2022)
Keywords: Efficient Training Methods
Pre-trained Transformer Networks
Distributed Training

Abstract: Pretraining Transformer-based language models followed by adapting the pre-trained models to a downstream task is an effective transfer mechanism in NLP. While it is well-known that the pretraining stage is computationally expensive, even the adaptation starts to become time-consuming for many downstream tasks as Transformers grow in size rapidly. 
Prior work focuses on reducing the pretraining wall-clock time via increasing the batch size to obtain higher training throughput on multiple processors. However, few studies have explored how such a scheme may benefit the adaptation phase. On the other hand, adversarial training has shown improved generalization for adapting Transformer models on many NLP tasks, but it is often treated as a separate line of research, where its effectiveness under the large-batch regime is not well understood. 
In this paper, we show that adversarial training obtains promising model accuracy even with a considerably larger batch size. However, the computational complexity associated with this approach, due to the high cost of generating adversaries, prevents it from reducing adaptation costs with an increasing number of processors. As such, we systematically study adversarial large-batch optimization for adapting transformers from a computational complexity perspective. Our investigation yields efficient and practical algorithms for adapting transformer models. We show in experiments that our proposed method attains up to 9.8$\times$ adaptation speedups over the baseline on BERT$_{base}$ and RoBERTa$_{large}$, while achieving comparable and sometimes higher accuracy than the state-of-the-art large-batch optimization methods.

On the Computational Efficiency of Adapting Transformer Models via Adversarial Training
Anonymous Author(s)
Affiliation
Address
email
Abstract
Pretraining Transformer-basee language models followed by adapting the pre- trained models downstream task is an effective transfer mechanism in NLP. While it is well-known that the pretraining stage is computationally expensive, even the adaptation starts to become time-consuming for many downstream tasks Transformers grow in size rapidly. Prior work focuses on reducing the pretraining wall-clock time via increasing the batch size obtain higher training throughput on multiple processors. However, few studies have explored how such scheme may benefit the adaptation phase. On the other hand, adversarial training has shown improved generalization for adapting Transformer models on many NLP tasks, but it is often treated separate line of research, where effectiveness under the large-batch regime is not well understood. In this paper, show that adversarial training obtains promising model accuracy even with a considerably larger batch size. However, the computational complexity associated with this approach, due to the high cost generating adversaries, prevents it from reducing adaptation costs with increasing number of processors. As such, systemat- ically study adversarial large-baud batch optimization for adapting transformers from computational complexity perspective. Our investigation yields efficient and practical algorithms for adapting transformer models. We show in experiments that our proposed method attains up to 9.8 adaptation speedups over the baseline BERT and RoBERTajarge while achieving comparable and sometimes higher accuracy than the state-of-the-ar large-batch optimization methods.
10
12
14
16
18
20
21
Introduction
22
23 In the past few years, we have witnessed the success of transformer models 2 such as BERT (9) RoBERTa 30 T5 (35, and GPT-3 [3. These models are trained on massive open-domain data
24
25
and subsequently adapted to various downstream tasks, which have led to accuracy breakthroughs in many NLP applications 44]. Despite their remarkable performance in accuracy, training these models extremely time-consuming given their huge model sizes, ranging from few hundred million parameters to over billions of parameters. As result, optimizations for faster training speed with high accuracy are the focus of a highly active research area and have a clear, practical impact.
27
29
30 To accelerate the training speed of large models, one of the most popular approaches is to leverage distributed training, where a mini-batch is partitioned across multiple processors (c.g., GPUs) to
31
compute gradients locally in parallel and then aggregate the local updates [27 30 20
39
40) 37
33
Under such a paradigm, increasing the batch size clearly has the benefit of improved training throughput per iteration. However, increasing the batch size has a non-trivial impact
model 35 convergence and generation practice. Early studies observed that increasing the batch sizes often
34
leads to slow convergence and/or poor generation under the same training iteration budget [25). To close the accuracy gap from large batch optimizations, prior works proposed either increase the
37
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

38
number of training iterations, which limits the performance benefits of large- batch optimizations 17 or variants of adaptive optimizers such as LARS [5T] and LAMB [52]. It has been empirically observed that LAMB 152 is able to speed up BERT pre-training by using considerably larger batch sizes on massive GPUs Despite showing promising results, prior work primarily focuses
39
40
41
42 on large-batch optimizations for accelerating pre-training Transformers. However, as the size of Transformers increases rapidly, reducing the training overhead at the adaption stage starts to become more prominent, e.g., with the active research that has been pushing the training time of BERT models to only a few hours or less than one hour [52 5570. it takes tens of hours to fine-tune these models on MNLI 30] Furthermore, since the adaptation of these large transformer models has been used by major players in the industry, many model scientists have to perform adaptation more frequently than pre- pre-training training the Transformers. As result, the excessive long adaptation time hinders the turnaround time, and the aggregated training cost for adaptation is also quite high.
43
44
45
47
49
50 We aim to accelerate the adaption of pre-trained Transformer models. For this purpose, we introduce ScaLA, a method that achieves similar model adaptation quality but with significantly shorter optimization time. Especially, the contributions of our paper consist of (1) We look into projected gradient descent based adversarial training, which has shown promising accuracy results in fine-tuning Transformer models. We find that adversarial training still leads to improved generalization under the large- batch regime, which we denote as adversarial large-bate large-batte batch optimization helps improve generalization but makes each individual processor slower, making it difficult to actually reduce training time even with a large number of processors. As such, we perform a systematic study of how different training strategies of adversarial large batch optimization affect the computational efficiency and generalization for adapting Transformers. We find that many computations in adversarial training are redundant and only have small impact on the final model accuracy. (3) Based on our studies, we present a novel algorithm ScaL.A that injects lightweight adversaries into large batch optimization speed up the adaptation of pre-trained transformer networks. (4) We theoretically quantify the convergence rate of adversarial large-batch optimization using techniques for analyzing non-convex saddle-point problems. (5) We conduct extensive evaluation, and our results show that ScaLA accelerates the adaptation of pre-trained Transformer- Trensokrmer-networks networks by up to 9.8 times over the baseline on BERT ( and RoBERTa (30] over a wide range of natural language understanding (NLU) tasks. We conduct ablation studies to assess the impact of our approach on both generalization and computational efficiency under various conditions.
51
53
54
55
optimization.
(2) Adversarial
57
59
60
61
62
63
64
65
86
87
68
Background and Related Work
89
70 Despite the great success of pre-trained transformer networks such as BERT 9., big challenge,
71 in general, comes from the training efficiency even with self-attention and parallelizable recur- hardware (24], training transformer networks can still take a 73 significant amount of time. One effective approach to reducing training time is through data par-
72 rence 43 and
74 allelism [9 30 40), which motivates studies on large-batch stochastic non-convex optimizations
75 for transformer networks [52]. These studies have raised concerns with respect to its convergence, generalizability, and training stability by observing that training with large batch could be dif-
76
77
ficult (25 17 33). Different from prior works, which mostly focus on reducing the pre-training
time
522 56 12 6), this work shows effective approach to accelerate the adaptation of pre- traitrd models while preserving the accuracy of downstream tasks.
79
80
There has also been an increasing interests in developing efficient adaptation methods of pre-trained Transformer models [T8 46 19 16). For example, [8] inserts small modules called adapters to each layer of the pre-trained model, and only the adapters are trained during adaptation. [19] adds low-rank matrices to approximate parameter updates. (34] shows that it is possible to quickly adapt
81
83
84 to new tasks by collectively learning knowledge from multiple tasks. These methods have achieved
85
comparable performance to standard fine-tuning on different sets of tasks. However, their focus is on reducing memory consumption of adaptation by reducing the trainable parameters needed per task.
87 Unlike these methods, which still incur full forward/backward computation cost during adaptation, we investigate how to accelerate the adaptation speed through adversarial large- batch optimization.

---

Minimize loss
Output byer
Maximize
(mm)
103
170
Baseline
Adversaria
35000
30000
Training time Accurary
915
prediction
21000
Context embeddings
deviation
Training time
203
Accuracy
Transformer encoder
(Delayed
PGA-1]
35000
Input embeddings
Adv. nolse
20000
Lexicon Lexocer
Noise generator
100D
sentence inputs
Figure 1: The architecture of the pro- Figure 2: posed method.
Time breakdown Figure 3: Impact of perturba- tion steps.
without and with PGA-I.
The Proposed Method
89
90 Motivated by the challenges in accelerating the adaptation, in this section, we present a principled 91 large-batch optimization method via lightweight adversarial noise for improved adaptation speed while maintaining the quality of the solutions as measured by task-appropriate accuracy metrics.
92
93
3.1
Adversarial Training Preliminaries
Adversarial training has been proposed and studied extensively in the computer vision literature mainly for improving the robustness against adversarial attacks [13 31. The key idea is to apply small perturbation to input images that maximizes the adversarial loss:
95
min (ra)) max l(f(x 8)0)
[
97
While adversarial training has been successfully mitigating adversarial attacks, traditional understand- ing is that adversarial training could hurt generalization performance. However, there has been an increasing amount of attention paid to leverage adversarial training for improving accuracy clean data performance [48] 58 [O]. In particular, there are some studies show that adversarial training helps improve the generalizability of language modeling [S] 45 22 29). However, very few works examine how adversarial learning works helps improve the adaptation speed of pre-trained Trans- former models. (58) studies adversarial training under the large-batch regime, observing improved accuracy by accumulating the gradient of the parameters from each of the ascent steps and updating the parameters only once after K inner ascent steps with the accumulated gradients. However, (1) (58) still requires multiple iterations to generate adversaries and injects adversaries in the full training process, leaving the major performance bottleneck from adversaries not reduced, (2) its implementation does not support multi-GPU training. In fact, end-to-ent training time reduction or speedup is reported in (58). putting question on how useful it is in accelerating Transformer training time.
98
99
100
101
103
105
107
109
111
Basic setup. Since language expressions are quite sensitive to individual words or clauses, where noises against those would likely generate incorrect or biased training data with wrong labels (55] follow prior success in applying adversarial training to NLP models (8) have an adversarial
113
training setup by applying noises to the continuous word embeddings instead of directly to discrete
115
words or tokens.
min Ee---g(L,, {)] = min max Ee [L (x. Ar(o, y) xEX
(2)
zEX yET
116
where X - denotes the overall training objective, X - IR denotes the augmented regu- larization and & denotes samples drawn from Q (for simplicity, we slightly abuse the notation using
117
E to denote the random variable, e.g. [g(x, {)], or its empirical realizations, e.g. π >>' g(x, &x) for any K). The overall (outer) training objective involves minimization problem in the parameter space while being stochastic with respect to the data space. The adversarial regularization (inner) term a deterministic maximization problem operating in the data space conditioned fixed parameter configuration. We emphasize that this formulation is a two-player sequential (23), not simultaneous, game wherein the goal is to optimize transformer network that is insensitive to adversarial noise.
118
120
122
3

---

124
3.2
Improving Adaptation Throughput via Large-Batch Optimizations
125
We are interested in extending the large-batch optimization to the Transformation adaptation phase using pre-trained base model on GLUE as an example. This part presents several studies that motivate the design of the lightweight adversarial large-batch optimization approach in Section 3.3
128
127
128
The detailed hardware/software setup described in Section 4
129
Scalability analysis. First, we carry out a scalability test by varying the number of GPUs from I to
32, with and without communication. Different from pre-training, the adaptation stage often employs
131
a much smaller batch size (e.g., 16, 32) than pre-training (e.g., 4096) [9 30). We choose a batch size
132
32, as suggested by most literature for BERT fine-tuning (9] 30, and we divide the samples in the
133
mini-batch among P= 1,,32) GPUs. If the per-worke batch size (e.g.. 16) is larger than the maximum admissible per-worker batch size (e.g., 8). we use local gradient accumulation [14] to avoid running out of memory. Figure 4(a) shows the scalability results. For batch size 32, the training time decreases when P increases from I to 4. However, it quickly plateaus and even decreases with more GPUs. We find that this is because when the batch size is small, the communication overhead dominates the total execution time (e.g.B=332 vs. B=32 (no comm)). The communication overhead is huge, especially when there is cross mooos-e communication (e.g., from 16 to 32), hindering the scalability of multi-GPU training. In contrast, by increasing the batch size (c.g., to IK), the training time keeps decreasing as the number of GPUs increases because an increased batch size reduces the number of all-reduce communications to process the same amount of data and also increases the compute resource utilization per GPU (i.e., increased computation-vs-cot vommmmmunnn ratio).
134
135
136
137
138
139
140
142
148
85.0
30000
B=32
32
83.6
84.4
04.5
84.8
84.5
82.9
80.0
32.7
32.7
32.7
35.4
1e5
Traiiing time G3
33999
B-1K
B-32 (no commn
78.7
822
83.2
83.6
83.7
84.1
84.3
84.4
83.7
84.1
80.1
-8..
7,5
commnn
2048
78.5
81.0
82.4
83.0
83.5
63.7 83.7
84.1 84.1
83.8
81.8
-88.0
5.0
12000
-8..5
4096
73.0
8
81.2
82.3
83.0
83.6 83.7
83.8 84.0
83.7
82.6
2.5
5932
8192
86.0
78.0
79.7 60.7
83.4
-11.5
81.0
0.0
D-14
B-18 (SCKAA)
RGPUs
16-05
36
08
S6-08
76-05
1e-04
04
9e-04
1e-03
3e-03
(a) Scalability
(b) Generalizabilitty
(c) Sharpness
Figure 4: Scalability, generalizability, and curvature analysis results by adapting BERT base to the MNLI task.
144
Generalizability analysis. Increasing the batch size leads to accelerated per-epoch execution time due to the efficient utilization of hardware. However, how would increasing the batch size affect the generalizability in adapting transformer networks? Since prior works on batch size scaling often focus on computer vision tasks and pre-training 44 14 4T 52], we conduct an analysis of large-batch adaptation on pre-trained Transformers by performing a hyperparameter sweep on batch sizes {IK. 2K, 4K, 8K} and learning rates {Ie-4, 3e-4,5c-4, 7e-4, 9c-4, 1e-3, 3e-3), where the learning rate range covers both linear scaling [14] and sqrt scaling (52). We report the validation accuracy in Figure [ We make two observations: (1) the learning rate scales roughly with the square root of the increase of the mini-batch size, although the best learning rates do not always follow the
145
146
147
148
149
150
151
152
153
sqrt rule; (2) there is a generalization gap between the small batch and large batch accuracies, and the gap becomes larger when the batch size increases. Furthermore, methods, such as LAMB (52). works well on pre-training with extremely large batch sizes (log, = { (15, 16)) but do not close the generalization gap in adaptation (as shown in Section D. These results pose the question: can we increase the batch size during adaptation in the interest of making adaptation more efficient but preserving generalization?
154
155
156
157
159
Curvature analysis. To further examine the generalization gap, we resort to the curvature analysis. Prior work [25 [54] correlate the low generalization with sharp minima (which are characterized by positive curvature of large magnitude in the parameter space). The indication is that sharp local minimum also reflects a higher sensitivity of the loss even within the neighborhood of training data points and can attribute to the difficulty in generalization. Their hypothesis was that larger noise due to the higher variance in gradient estimates computed using small mini-batches, in contrast to gradient estimates computed using large mini-batches, encourages the parameter weights to exit out
160
161
162
163
164
185
of the basin of sharp minima and towards flatter minima which have better generalization.

---

167
To verify this hypothesis, we quantitatively measure the steepness of loss landscape by loading the checkpoint of an adapted model and computing the curvature, i.c., properties of the second derivative of the model, with respect to its parameters, for a fixed batch of samples. Following [4], for model (x)), we compute the largest eigenvalue of the model's Hessian, Tmmax [V (x)] using the
188
169
170
171
Hessian-vector product primitive and the power method. We use the largest eigenvalue as measure
172
of sharpness since the corresponding (top) eigenvector characterizes the direction of the largest change in gradient at a given point in the parameter space. From Figure 4(c) the largest eigenvalue of the model trained with a large batch (e.g., IK) is much larger (e.g., 2.6x) than the small-batch
174
175
baseline and with higher deviations (e.g., 3.9x). This result confirms that large-batch adaptation
178
makes the loss landscape of the model more prone to ill-conditioning and less robust to perturbation, which helps explain the loss in generalization.
178
3.3
Improving the Generalization via Lightweight Adversarial Large-Batch Optimization
179
Our analysis indicates that although injecting adversarial noise into large-batch optimization helps im- prove the generalizability; it may not reduce the adaptation time because the generation of adversarial noises can take large fraction of time. This section provides an analysis of the computational cost and then describes two approaches to reduce the time spent in generating adversarial noise, thereby further reducing the overall adaptation time.
180
181
182
184
The generation of adversarial noise requires an extra PGA inner loop that standard training does not have. Figure 2 provides the time breakdown of optimization using PGA with T = (denoted as PGA-1). PGA-I performs the perturbation and takes approximately the same time as making three forward passes (Fwd) through the network. This is because one step of PGA requires to make one forward and backward pass (Bwd) over the entire network. The backward pass of the optimization takes roughly twice the amount of time as the standard backward step because the back propaaaatn triggered twice to calculate the noise and the gradients. The time spent on the optimizer step function remains the same. In total, the optimization would slow down training by at least 2 times, even with T=I. This motivates us to look at the effectiveness of different perturbation steps as well as the usefulness of perturbation from the initial epochs.
185
188
187
188
189
190
192
194
One-shot perturbation. Prior works often do multiple gradient computation steps (T >> 1)
195
and take several times longer training time to produce adversaries [31 58) likely because their focus is on generalization instead of computational efficiency. Subsequently, researchers presented
198
197
Curriculum Adversarial Training (CAT) 4 and Annealing based Adversarial Training [50), which progressively increase the perturbation with various strengths, cutting the adversarial training cost while maintaining good accuracy. To investigate how CAT and similar methods affect large-scale NLP problems involving transformers, we evaluate the final accuracy and training cost of QNLI, varying the number of perturbation steps T and report the results in Figure 3 Interestingly, although using a large T helps to produce stronger noises, we find that this does not lead to improved accuracy, despite the fact that the training overhead still increases almost linearly. In fact, the best accuracy is achieved with T = 1.
198
199
200
202
204
205
We note that the model has two components, namely, the parameter space and data space. First, unlike the minimization in the parameter space, which is stochastic, the maximization in the data space is deterministic. Second, with respect to the testing phase, the numerical convergence in the model's 's parameter space is of primary importance rather than the numerical convergence in the data space, i.e., the maximization is an auxiliary procedure that augments the training phase to make the parameter space "aware" of effects of the batch size across epochs. Due to these two points, at certain epoch, for a given batch, the marginal utility of an additional PGA step is low, and we are able to get away with inexact deterministic maximization. Therefore, we apply PGA-1 in our larch-batch optimization scheme, given that it produces sufficiently good solutions while being much more computationally efficient.
206
207
208
210
212
214
215
Delayed perturbation injection. Given that PGA-1 still adds an overhead factor of 2, we are motivated to further reduce the overhead of adversarial noise. In particular, we investigate how useful adversarial noises are in the whole large- batch optimization process. We conduct additional experiments to measure the final accuracy corresponding to starting from a regular fine-tuning and then enabling PGA-1 for I. 2 l, where 4, E [T]. Our observation is that enabling PGA-1 1 from the beginning does not offer much improvement in accuracy, whereas adversarial noise becomes
216
217
218
220
more potent the model begins to stabilize towards the end of training (more detailed results in
5

---

222
Appendix A.I). In general, at initialization, the model's parameters are relatively far from their final values and are less likely get stuck at local minima. Therefore the adversarial noises generated in the initial training iterations are quite different from the noises towards the end of training because they would not maximize the adversarial loss in Equation 2 This hypothesis suggests that we might be able to inject adversarial noise in the later training process while still leveraging it to improve
223
225
226
227
generalizability. We remark that this phenomenon has been observed by prior work on computer vision tasks [T..
229
Putting together. Combining the formulation with the above investigations, the full procedure of ScaLA is provided in Algorithm whose convergence rate is characterized in Theorem 3.1
230
Algorithm
ScaLA
1: Input: Epochs T, delay 4, perturbation (inner) step size clipping radius regularization strength A, (outer) learning rate 77
2: Output: f--aayer transformer model & with converged robust parameters I TT
3: for E [T'] do
4:
for worker p E [P] do for mini-batch Ep ~ Q do
5:
6:
r(xr) +- 7 < o(2, Ep), select yo
7:
4 then
8:
Check delay condition D. Generate adversarial noise with PGA-1
9:
< (w + ((
10:
[(xx)= - KLaym (7, 2 (xx-11111) B1))
end if
12:
g( Sp) +- [1 Ep) + Xr(a;)
13:
Vzglat. Sp) +- Backward pass on +
14:
end for end for
15:
16:
Vag(It) + 1 Ep vag(z()b))
17
rt ./-1 - 11 (x)())
18:
end for
Theorem 3.1 (Complexity of Algorithm D Informal - Details in Appendix D), Consider the problem
232 in Equation 2 Let t = Setting the outer learning rate as 1 = O (1/VT) and scaling batch
size as b (T), for Algorithm 7 we have E (1 S 50(c+ Ko/VT) where I is the estimator obtained from running I steps of Algorithm | and picking It uniformly at random for
233
234
235
t E T Here, e is the error due to the approximate inner maximization oracle, characterizes the
236
smoothness of f(x,.), 91/2a the Moreau-enve envelope of g and Ka max, as/ min, oz.
4 Evaluation
We evaluate the effectiveness of ScaLA in adapting pre-trained transformer networks over set of NLP tasks.
239
240
Hardware. We conduct the evaluation using 2 NVIDIA DGX-2 nodes. Each node consists of
241
16 NVIDIA V100 GPUs. The nodes are connected with InfiniBand using 648-port Mellanox MLNX-OS -SS CS7500 switch. Model/Dataset. We study adaptation on pre-trained BERT mode and RoBERTaarge hosted by HuggingFace 47 We use the GLUE benchmark (4), which is a collection of sentence or sentence- natural language understanding tasks including question answering, sentiment analysis, and textual entailment. We exclude tasks that have very small datasets CoLA, RTE), We report the details about the hyperparameters in Appendix B.
242
243
245
4.1
Main Results Adaptation Time Acceleration
We first compare the following schemes: (1) Single GPU + SB: This is the existing PyTorch imple-
mentation of Transformer fine-tuning from HuggingFace (HF), using small batch (SB) sizes (e.g., 32).
249
(2) Multi-GPU SB: This is multi-GPU PyTorc implementation using DistributedDataParrlly [27].

---

251
(3) Multi-GPU + LB- + FreeLB:; this is the work described in [58] using large minibatches (LB), e.g. IK, and perturbation step K for adaptation, and (4) Multi-GPU + LB ScaLA: This is our approach as described in [ Table results on MNLI, QNLI, QQP, and SST2, which are larger datasets and less sensitive to random seeds. n X g refers to P. nodes each with Pg GPUs for total of P = P. homogeneous workers (e.g., 32 GPUs on 2 NVIDIA DGX-2 nodes). For fair comparison, we reproduce BERT and RoBERTa baseline. Our reproduced baseline achieves the same or slightly higher accuracy than the originally reported results in [ and [30]. We now discuss our results and observations.
252
254
255
256
258
Table 1: The adaptation time and accuracy results on GLUE benchmark. ScaLA achieves the same average accuracy as the baseline while providing up to 18x speedups than single GPU, and up to 9.8x x speedups with the same amount of hardware.
MNLI-m
ONLI
QQP
SST-2
BERT base
nxg
boz
Steps
Time
Acc.
Steps
Time
Acc.
Steps
Time
Acc/F1
Steps
Time
Acc.
Avg.
Devlin et al. 2019
84.4
88.4
92.7
Baseline (2=
1xl
32
73632
19635
84,8
19644
5535
90.6
68226
16494
91/88.0
126.30
2736
93.1
89.4
Baseline (8=32)
2x16
32
73632
8848
84,8
19644
2408
68226
11311
91/88.0
126.30
1494
93.1
89.4
Freel h (B=1K)
2x16
1K
2301
5953
85.2
615
1944
90.3
21.33
19030
91.2/88.2
2
396
680
92.8
89.5
ScaLA (B=IK)
2x16
IK
2301
1323
185.1
G15
432
90.0
2133
4229
90,9/87.7
396
151
93.5
89.4
MNLI-m
ONLI
QQP
SST-2
RoBERTawr
nxg
boz
Steps
Time
Acc.
Steps
Time
Acc.
Sttps
Time
Acc/F1
Steps
Time
Acc.
Avg.
Liu et al. 2020
90.2
94.7
92.27-
96.4
Baseline (B=32)
1xl
32
73632
43090
90.5
19644
14188
94.7
68226
40945
92.0/89.4
12630
4940
96.4
92.5
Baseline (B=32)
2x16
32
73632
18114
90.5
19644
4842
94.7
68226
16614
92.0/89.4
12630
3072
96.4
92.5
Freel h (B=1K)
2x16
1K
2301
15133
91.2
615
5256
95.2
2133
10818
92.5/9900
396
1804
96.9
93.3
ScaLA (B- 1K)
2x16
1K
2301
3363
90.9
615
1168
95.
2133
2404
92.3/89.8
396
401
96.7
92.9
259
Adaptation time analysis. Compared with single-GPU training. the multi-GPU baseline leads to only modest training speedup improvements, e.g., with 1.5 2.4x faster training speed for both BERT and RoBERTa, even with 32x more compute resources. The speedup is limited because of the small mini-batches (e.g.. 32) used for adaptation, which do not provide a sufficient workload to fully utilize the underlying hardware. Thus, communication overhead becomes the dominant part, and the adaptation often struggles to obtain speedups even with more workers. In contrast, ScaLA achieves
260
262
264
up to 18x speedups over the single-GPU bascline with 32 GPUs. When using the same number of
286
GPUs g.. 32), ScaLA is 2.7-9.8x faster. The speedups come from three aspects: (1) the improved
hardware efficiency for each worker from increased per-worker micro-batch size; (2) the reduced all-reduce communication overhead since it takes fewer communication rounds to process the same number of samples in one epoch; (3) the lightweight adversarial noise incurs only small portion of the total training overhead. Finally, ScaLA obtains the speedups while achieving the same accuracy (88.4 vs. 88.4) average accuracy for BERT and higher accuracy (92.9 vs. 92.5) for RoBERTa as the baselines. ScaL is 4.5 times faster than FreeL.b while achieving similar accuracy on BERT (89.4 vs. 89.5) and RoBERTa (92.9 vs. 93.5). ScaLA faster than FreeL because FreeLb does not consider much about the training cost and performs multiple ascent steps to calculate adversaries across the full training process. As a matter of fact, FreeLb even slower to run than vanilla baseline (c.g.. QNLI on RoBERTa). In contrast, ScaLA analyzes the computational efficiency of adversarial large-batch optimization and introduces several simple yet effective approaches to reduce the adversarial noise cost, which leads to overall improved computational efficiency.
268
269
270
271
272
273
274
275
278
277
278
279
Generalizability analysis. Since there are very few works on large-batch adaptation, we create
280
several baselines to compare with ScaLA: (1) Multi-GPU + LB + Tuning LR: This configuration uses large mini-batches (e.g.. IK), and applies beuristic-based scheduling rule (e.g., square root) combined with an extensive grid search for learning rates; (2) Multi-GPU + LB + LAMB: Uses LAMB (52) optimizer for large-batch adaptation. We make several observations from the results in Table First, compared with the baseline accuracy reported in the paper, the accuracy of Multi-GPU + LB drops by close to 1 point (88.4 vs. 89.4, and 92.1 vs. 92.9) in average and close (e.g., QQP on BERT), indicating that it is challenging to obtain on-par accuracy with large-batch optimizations for adaptation despite with heavy hyperparameter tuning. Second, since LAMB is
281
282
283
284
285
points for some tasks
286
287
288
designed primarily for improving the convergence of pre-training instead of the adaptation, its ability to accelerate the adaptation has yet to be proven. In our experiments, LAMB leads to only marginal improvements (88.6 vs. 88.4 and 9 vs. 92.1) than the baseline and is 0.8 points lower than the small-batce baseline. This is because LAMM does not directly minimize the sharpness of the loss
289
291
292
landscape, it can still lead to poor generalizability during adaptation. With ScaLA, we are able
7

---

293
to close the generalization gap from lurge-batch optimization (89.4 vs. 89.4, and 92.5 vs. 92.9) and achieve 0.8 points higher accuracy (89.4 vs. 88.6, 92.9 vs. 92.1) than LAMB on both BERT and RoBERTa. Scal AA improves generalizability because it introduces adversarial noise in the large-batch optimization process, which serves as a regularizer. By training the network to be robust to such perturbations, the model loss landscape is smoothed out, leading to improved generalization.
294
296
297
Table 2: The comparison results between SeaLA and alternative methods for large-batch adaptation on the GLUE benchmark, which show that ScaLA achieves higher accuracy than baselines after training the same number of samples and steps.
MNLI-m
ONLI
QQP
SST-2
nxg
Batch Size
Steps
Time
Acc.
Steps
Time
Acc.
Steps
Time
Acc/F1
Steps
Time
Acc.
Avg-
Vanilla (B=IK)
2x16
1K
2301
1148
84.3
615
349
89.3
2133
2892
89.6/86.
396
134
88.4
LAMB (B=IK)
2x16
1K
2301
1180
84.1
615
359
89.6
2133
2978
90.5/87.0
396
139
92.4
88.6
ScaLA (B=1K)
2x16
1K
2301
1323
85.1
615
432
90.0
2133
4229
90.9/87.7
396
151
93.5
89.4
MNLI-m
QNLI
QQP
SST-2
RoBERTauge
nxg
Batch
Steps
Time
Acc,
Steps
Time
Acc.
Steps
Time
Acc/F1
Steps
Time
Acc.
Avg.
Vamilla (B=1K)
2x16
1K
2301
2514
90.1
613
936
94.3
2133
1874
9/87
396
317
95.9
92.1
LAMB (B-1K) 2x16
1K
2301
2646
90.5
613
973
94.5
2133
1998
91.3/88.5
396
324
96.2
92.1
ScaLA (B= [K] 2x16
1K
2301
3363
90.9
615
1168
95.1
2133
2404
92.3/89.8
396
401
96.7
92.9
298
4.2
Experiment - Analysis Results
299
Ablation analysis: We study the importance of components in ScaLA. We set E, to O, which denotes
300
as w/o Delaying P. We replace the outer minimization to use ADAM [26), which is noted as w/o Groupwise LRR. We set O, which denotes as w/o PGA-I. The results are reported in Table 3
301
Table 3: Ablation study of ScaLA using BERT
on GLUE tasks.
base
MNLI-m
QNLI
QQP
SST-2
Avg.
Speedup
Time
Acc.
Time
Acc.
Time
Acc/Fl
Time
Acc.
Baseline
19635
84.8
5535
90.6
16494
91/88.0
2736
93.1
89.4
ScaL
1323
85.1
432
90
4229
90.9/87.7
151
93.5
89.4
12.4
w/o Delaying PGA-1
2503
85.2
726
90.2
6407
91.3/88.3
272
93.1
89.5
7.0
w/o Groupwise LR
1290
85.0
422
89.9
4212
90.7/87.6
146
93.0
89.2
12.7
w/o PGA-1
1180
84.1
359
89.6
2978
90.5/87.0
139
92.4
88.6
14.3
302
The results in Table show that the removal of either design element would result in performance drop. For example, removing PGA-1 leads 0.8 points accuracy drop (88.6 vs. 89.4), indicating that adversarial noise is crucial for improving the generalizability of large-batch adaptation. Moreover, if
303
304
305
we perform PGA-1 without delayed injection, the average accuracy increases by 0.1 points (89.5 vs. 89.4). but the execution time is increased by 1.5-1.9x, indicating the importance of having lightweight adversarial noise for speeding up the adaptation. Finally, removing group-wise learning rates leads to small 0.2 points accuracy drop (89.2 vs. 89.4), indicating that ScaL.A still achieves benefits without group-wise learning rates (89.2 vs. 88.6), but they are complementary to each other.
306
307
309
Table 4: Alternatives to generate perturbations using random noise, ground-truth, and label probability.
Model MNLI-m
QNLI
QQP
SST-2
Avg
Bascline
84.3
89.3
89.6/86.1
93
88.4
Gaussian noise
84.5
89.4
90.3/87.0
92.6
88.7
ScaLA (GT)
84.1
89.6
90.7787.6
93.2
89.0
ScaLA (LP)
85.1
90
90.9/87.7
93.5
89.4
310
Curvature analysis. We measure the steepness of the loss landscape again after applying ScaLA. As shown in Fig. 4(c) the largest eigenvalue of the model becomes much smaller (6.9x) with lower deviations with ScaL.A and is slightly better than the small batch baseline, which is a strong indication that our approach enforces the smoothness of the model that leads to the accuracy improvement.
312
314
Comparison with random noise. We have performed additional experiments by adding Gaussian noise to the embeddings. Table that random noise indeed can improve the accuracy for MNLI-m (84.3 vs. 84.5), QNLI (89.3 vs. 89.4), and QQP (90.3/87.0 vs. 89.6/86. 1) over the baseline, but it also leads to worse results SST-2 (93, vs. 92.6). Compared with ScaLA, random noise consistently falls
316
8

---

1250
65C
53.
3550
Seaeine (B-32)
32
8
86.8
86
64.8
86
98.0
95.4
Training time
1000
LAMD (D-20)
ScalA (1-23))
accJracy
K
04 3
85
05.1
04.2
1K
32
924
$9.0
-3.0
SuubA wel dally (Sw3R)
64.5
2000
2500
2K
64.1
84.1
04.7
2K
50.1
92.0
90.4
Acour
1000
-
Baseline (8=2K) ScoLA (8m2K)
4K
34
63.6
63 2
01.
8K
83.4
AGPUs
#rpochs
Baseline
LANB
Scal.A
Bosellen
LAMB
ScaiLA
V1M
Figure 5: Comparison of Figure 6: Comparison of scalability using different test accuracy by training large-batch optimization the baseline longer. methods on SST-2.
(a) MNLI-m
(b) SST-2
Figure 7: Comparison of accuracy under even larger batch sizes.
318
behind ScaLA in its ability to reduce the generalization error on all tested tasks and is on average 0.7 points lower than ScaLA (88.7 vs. 89.4). These results indicate that ScaLA's approach of explicitly enforcing the smoothness of the loss landscape can result in better improvement.
320
321
Perturbations via ground-trutt vs. label probability. We also create one-hot labels and use those to generate perturbations instead of using label probability generated by the network. Table shows that using label probability (LP) consistently leads to higher accuracy than using the ground-truth (GT), e.g., 89.4 vs. 89.0 on average. Label probability leads to better generalization, probably because
322
323
324
it provides better measurement of the adversarial direction, which is the direction in the input space in which the label probability the model is most sensitive to small perturbations.
326
327
Scalability analysis varying GPUs. Figure 5 shows the scalability comparison on SST-2 after optimizations. While the speedup still plateaus at 4 GPUs with a small batch size (e.g., B 32), the four large-batch configurations are able to scale well up to 32 GPUs and take similar amount of time with 32 GPUs ScaLA scales better than ScaLA without delaying PGA-1, and achieves a much faster training speed, especially in the 1-16 GPU range.
328
329
330
Train longer, generalize better? Despite improved adaptation speed, one may still wonder whether simply performing large-batch adaptation longer would also close the generalization gap. Figure shows the comparison between ScaLA and the baseline on batch size of 2K. ScaLA obtains an accuracy of 85.2 after 6 epochs of training, whereas the baseline has difficulty to reach 84 after
333
334
335
336
training twice longer (e.g., 12 epochs). ScaLA achieves better accuracy because explicitly penalizes model weights from getting stuck at sharp minima, leading to better generalizability.
337
Generalizability under different batch sizes. We also evaluate how different batch sizes affect the generalizability of adapting transformers. Figure 7 shows the results on MNLI-m and SST-2. We make two major observations: (1) The accuracy tends to drop as the batch size increases. (2) While both the baseline and LAMB suffer from significant accuracy drop by drastically increasing the batch size (e.g., from 32 to 8K), ScaLA is able to mitigate the generalization gap and consistently achieves higher accuracy than the baseline (e.g.. 84.4 vs. 83.5 for MNLI, and 92.6 vs. 91.3 for SST-2 at batch size 8K) and LAMB (e.g.. 84.4 83.9 for MNLI, and 92.6 vs. 91.7 for SST-2 at batch size 8K). These results indicate the benefit of ScaL. is maintained by further increasing the batch size, which could bring even greater speedups when increasing the data parallelism degree.
339
340
341
343
344
345
348
Conclusions and Future Directions
347
348
In this paper, we study how accelerate the adaptation speed of pre-trained Transformer models for NLU tasks. We introduce ScaLA, an efficient large-batch adaptation method using carefully injected lightweight adversarial noises. The experiment results show that ScaLA obtains up to 9.8x speedups on adapting transformer networks and outperforms state-of-the-art large-batch optimization methods
350
352
in generalizability. Given the promising results of ScaLA on accelcrating the adaptation speed, opens new research opportunities on applying ScaLA to accelerate the more expensive pre-traie pre-training tasks as well as emerging pre-trained transformer networks for computer vision domains tasks.
354

---

References
355
356
[1] Microsoft Blog. Microsoft DeepSpeed achieves the fastest BERT training time.
https
7/www. deepspeed attpppeed..i//eeeeeee att////aaa/000000///////stttee-1 05-27-2020.
html Accessed:
358
[2] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von
360
Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint 2tt.::2:08.022588. 2021.
362
[3] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Hersert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. CoRR, abs/2005.1 14165, 2020.
364
386
367
368
369
[4] Qi-Zhi Cai, Chang Liu, and Dawn Song. Curriculum adversarial training. In Jérôme Lang, editor, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pages 3740-3747 ijeai.org, 2018
370
372
[5] Yong Cheng, Lu Jiang, and Wolfgang Macherey. Robust neural machine translation with doubly adversarial inputs. In Anna Korhonen, David R. Traum, and Lluís Mårquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL ACI 2019, Florence, Italy, July 28- August 2, 20/9, Volume I: Long Papers, pages 5322-4333 Association for Computational Linguistics, 2019.
373
374
375
378
[6]
Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: pre-training text encoders as discriminators rather than generators. In 8th International Con- ference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OeenReview.net, 2020.
378
380
Damek Davis and Dmitriy Drusvyatskiy. Stochastic subgradient method converges at the rate ( on weakly convex functions. arXiv preprint /8020080,,,2988, 2018.
382
[8] Damek Davis and Dmitriy Drusvyatskiy. Stochastic model-based minimization of weakly convex functions. SIAM Journal on Optimization, 29(1)2207-239, 2019.
384
[9] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019), pages 4171-4186, 2019.
386
388
[10] Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for iision-and-language representation learning. In Hugo Larochelle, Marc Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Ad- vances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Processing Systems 2020, NeurlPS 2020. December 6- 2020. virtual, 2020.
390
392
394
[11] Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization, 23(
2013.
396
[12] Linyuan Gong, Di He, Zhuohan Li, Tao Qin, Liwei Wang, and Tie-Yan Liu. Efficient training of BERT by progressively stacking. In Proceedings of the 36th International Conference on Muchine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pages 2337-2346 2019.
398
400
[13] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
402
10

---

404
[14] Priya Goyal, Piotr Dollár, Ross B. Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch. Yangqing Jia, and Kaiming He. Accurate, large minibatch SGD: training imagenet in I hour. CoRR, 0121106602677,2017
405
[15] Sidharth Gupta, Parijat Dube, and Ashish Verma. Improving the affordability of robustness training for dnns. In 2020 JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2020, Seattle, WA, USA, June 14-19, 2020, pages 3383-3392. IEEE, 2020.
408
410
[16] Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Kirg-KKrrpattick,, and Graham Neubig. To- wards unified view of parameter-efficin transfer learning. CoRR, abs/2110.04433 2021.
[17] Elad Hoffer, Itay Hubara, and Daniel Soudry. Train longer, generalize better: closing the generalization gap in large batch training of neural networks. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA, pages 1731-1741, 2017.
413
415
417
418
[18]
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly, Parameter-efficient transfer learning for NLP. In Kamalika Chaudhuri and Ruslan Salakhutdinov. editors. Proceedings of the 36th International Conference on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California. USA, volume 97 of Proceedings of Machine Learning Research, pages 2790-2799. PMLR, 2019.
419
420
421
422
423
424
[19] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhee Yuanzhi Li. Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. CoRR, abs/2106.09685, 2021.
426
427
[20]
Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Xu Chen, HyoukJoong Lee, Jiquan Ngiam, Quoe V. Le, Yonghui Wu, and Zhifeng Chen. Gpipe: Efficient training of giant neural networks using pipeline parallelism. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, pages 103-112, 2019.
428
429
430
431
[21] Prateek Jain and Purushottam kar. Non-convex optimization for machine learning. arXiv preprint arXiv:1712.07889) 2017.
433
434
[22] Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao SMART: robust and efficient fine-tuning for pre-trained natural language models through prin- cipled regularized optimization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2020, Online, July 5-10, 2020, pages 2177-2190. Association for Computational Linguisties, 2020.
435
436
437
438
439
440
[23] Chi Jin, Praneeth Netrapalli, and Michael Jordan. What local optimality in nonconvex- nonconcave minimax optimization? In International Conference on Machine Learning, pages 4880-4889, PMLR, 2020.
442
443
[24] Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami. Rajendra Gottipati. William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg, John Hu, Robert Hundt, Dan Hurt, Julian Ibarz, Aaron Jaffey, Alek Jaworski, Alexander Kaplan, Harshit Khaitan, Daniel Killebrew, Andy Koch, Naveen Kumar, Steve Lacy, James Laudon, James Law, Diemthu Le, Chris Leary, Zhuyuan Liu, Kyle Lucke, Alan Lundin, Gordon MacKean, Adriana Maggiore, Maire Mahony, Kieran Miller, Rahul Nagarajan, Ravi Narayanaswami, Ray Ni, Kathy Nix, Thomas Norrie, Mark Omernick, Narayana Penukonda, Andy Phelps, Jonathan Ross, Matt Ross, Amir Salek, Emad Samadiani, Chris Severn, Gregory Sizikov, Matthew Snelham, Jed Souter, Dan Steinberg, Andy Swing, Mercedes Tan, Gregory Thorson, Bo Tian, Horia Toma, Erick Tuttle, Vijay Vasudevan, Richard Walter, Walter Wang, Eric Wilcox, and Doe Hyun Yoon. In In-Datacenter Performance Analysis of a Tensor Processing
445
448
447
448
449
450
451
452
454
11

---

456
Unit. In Proceedings of the 44th Annual International Symposium on Computer Architecture, ISCA '17, pages 1-12, 2017.
457
[25]
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On targe-batch training for deep learning: Generalization gap and sharp minima. In 5th International Conference on Leurning Representations, ICLR 2017, Toulon, France, April
459
460
461
24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
462
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In 3rd [26]
International Conference on Learning Representations, ICLR 2015, 2015.
464
[27]
Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, and Soumith Chintala. Pytorch distributed: Experiences on accelerating data parallel training. Proc. VLDB Endow., 13(12):30053300 2020.
465
488
467
[28] Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning, pages 6083-6093. PMLR, 2020.
489
[29] Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang. Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. CoRR, abs/2004.08994, 2020.
472
[30]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907, 1692, 2019.
474
475
476
[31] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018
477
479
480
[32]
Takeru Miyato, Andrew M. Dai, and Ian Goodfellow. Adversarial training methods for semi- supervised text classification. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.nett 2017.
481
482
483
484
[33] Zachary Nado. Justin Gilmer, Christopher J. Shallue, Rohan Anil, and George E. Dahl. A large batch optimizer reality check: Traditional, generic optimizers suffice across batch sizes. CoRR, 2021.
486
487
[34] Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. In Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty, editors, Proceedings of the 16th Conference of the European Chupter of the Association for Computational Linguistics: Muin Volume, EACL 2021, Online, April 19- 23, 2021, pages 487-503. Association for Computational Linguistics, 2021.
489
491
482
[35] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. CORR, abs/1910. 10600.......1.. 2019.
493
494
[36] Hassan Rafique, Mingrui Liu, Qihang Lin, and Tianbao Yang. Weally-convex-coneee min- max optimization: provable algorithms and applications in machine learning. Optimization Methods and Software, pages 1-35, 2021.
496
497
498
[37] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimiza- tion towards training A trillion parameter models. CoRR, hbbs/191000254. 2019.
499
500
[38]
Ralph Tyrell Rockafellar. Convex analysis. Princeton university press, 2015.
501
[39]
Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanan- takool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, and Blake A. Hechtman. Mesh-tensorflow: Deep learning for supercomputers. In Advances in Neu- ral Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, pages 10435-10444, 2018.
502
504
12

---

506
[40] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-Img Im: Training multi-biliion parameter language models using model parallelism. CoRR, ahs/1100.00053, 2019.
507
508
[41] Leslie N. Smith. A disciplined approach to neural network hyper-parameters: Part 1- learning rate, batch size, momentum, and weight decay. CoRR, abs/ aabs/1803.00 888.008300 2018.
510
511
[42]
Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, and Quoe V. Le. Don't 't decay the learning rate, increase the batch size. In 6th International Conference Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
513
[43] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pages 5998-6008, 000-66088222 2017.
516
518
519
[44] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, 2019.
520
521
522
[45] Dilin Wang, ChengYus Gong, and Qiang Liu. Improving neural language modeling via adversarial training. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 6555-6565. PMLR, 2019.
523
524
525
526
527
[46]
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang. Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. K-adapter: Infusing knowledge into pre-trained models with adapters. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors. Findings of the Association for Computational Linguistics: ACL/JJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/UCNLP 2021 of Findings of ACL, pages 1405 1418.14181... Association for Computational Linguistics, 2021.
528
529
530
531
532
533
[47] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the- art natural language processing. In Qun Liu and David Schlangen, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstra- tions, EMNLP 2020 Demos, Online, November 16-20, 2020, pages 38 45. Association for Computational Linguisties, 2020.
535
537
539
[48] Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang. Alan L. Yuille, and Quoc V. Le. Adversarial examples improve image recognition. In 2020 IEEE/CVE Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 816-825. Computer Vision Foundation / JEEE, 2020.
542
543
544
545
[49] Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, and Michael W. Mahoney. Hessian- based analysis of large batch training and robustness to adversaries. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, edi- tors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 4954 4964, 2018.
546
547
548
549
550
551
[50] Nanyang Ye, Qianxiao Li, Xiao Yun Zhou, and Zhanxing Zhu. Amata: An annealing mechanism for adversarial training acceleration. CoRR, abs/2012.08112, 2020.
552
553
[51] Yang You, Igor Gitman, and Boris Ginsburg. Large batch training of convolutional networks. arXiv preprint arXiv: 1008. 03888, 2017.
554
555
[52] Yang You, Jing Li, Jonathan Hseu, Xiaodan Song, James Demmel, and Cho-Jui Hsieh. Reducing BERT pre-training time from 3 days to 76 minutes. CoRR, 1009999940,,,,, 2019.
13

---

557
[53] Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training bert in 76 minutes. arXiv preprint arXiv:/9044 00962, 2019.
558
560
[54] Yang You, Yuhui Wang, Huan Zhang, Zhao Zhang. James Demmel, and Cho-Jui Hsich. The limit of the batch size. CoRR, abs/2006.08517, 2020.
561
[55] Dongxu Zhang and Zhichao Yang. Word embedding perturbation for sentence classification. CoRR, abs/1804.0S$ 166, 2018.
563
584
[56]
Minjia Zhang and Yuxiong He. Accelerating training of transformer-based language models with progressive layer dropping. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Floon Frori-Florinaaa Balcan, and Hsuan- Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
585
586
587
588
[57] Shuai Zheng, Haibin Lin, Sheng Zha, and Mu Li. Accelerated large batch optimization of BERT pretraining in 54 minutes. CoRR, abs/2006. 13484, 2020.
570
[58) Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. In 8th International Conference on Learning Representutions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view.net, 2020.
572
574
Checklist
575
576
1. For all authors...
577
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
578
contributions and scope? [Yes]
579
(b) Did you describe the limitations of your work? [Yes]
580
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
582
583
2. If you are including theoretical results...
584
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
5e5
(b) Did you include complete proofs of all theoretical results? [Yes] The complete proof in in Appendix D
587
3. If you ran experiments...
5B8
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
589
590
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We have very detailed training details in Appendix B.
582
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [N/A] Our results are reported based on the same set of hyperparameters for all configurations, so it does not rely on the randomness.
593
594
595
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
597
4. If you are using existing assets (e.g., code, data, models) or curating/releasnng new assets...
598
(a) If your work uses existing assets, did you cite the creators? [Yes] We cited all dataset and github repository used in the paper.
599
600
(b) Did you mention the license of the assets? [N/A]
601
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
602
603
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
14

---

605
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
608
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
609
610
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
612
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "vXOB3bIptDX", "reviewer": "Reviewer_oBzn", "summary": "This paper aims to improve the efficiency of finetuning pretrained large transformers through adversarial training. It resorts to large mini-batch training and proposes several methods to reduce the computational overhead and address the optimization challenges of large mini-batch training. The paper provides an extensive empirical analysis confirming similar challenges raised by recent works. The proposed ScaLA algorithm combines several optimization tricks such as delayed and one-shot perturbations. On the BLUE benchmark, ScaLA substantially reduces the overhead of finetuning BERT and RoBERTA without hurting accuracy. Convergence analysis of the algorithm is provided. The ablation study is sound.\n\nThe paper reads like a collection of techniques mostly proposed by previous works. For example, as the paper points out, several key ideas are already established in computer vision applications, and now are adopted to textual data by this work. To be clear, I have no doubt that such efforts are of great use in real-world applications of large models. Just that I am not sure this paper is a right fit for the NeurIPS community. \n\n====\n\nAfter author response: with the additional results and analysis, as well as the experiments planned out, the paper is definitely in a better shape. I have increased my rating to 6.\n", "questions": "- The notation around Eqs. 1 and 2 is confusing to me. What are $x$, $y$, $Q$, etc.? Shouldn’t $r$ be $\\mathbb{X}\\times\\mathbb{Y}\\rightarrow\\mathbb{R}$? Why is g’s domain $\\mathbb{X}\\times\\mathbb{Y}$?\n- Could the authors explain what PGA is?\n", "limitations": "The paper does not explicitly discuss the limitations and potential negative societal impact.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Addressing the challenges of large mini-batch training is important and timely.\n- Technical aspects of the paper are reasonably clear.\n- The proposed method is simple and effective.\n\nWeaknesses:\n- The paper heavily builds on the findings of existing works. It is hard to argue for technical novelty.\n- I think the paper could be better organized. The current presentation has several separate ideas floating around lacking a clear central theme. Besides, I find the detailed discussion of previous work in the middle of the method section very distractive. \n- The paper should include experiments on other pretrained transformers, as well as other datasets.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fYJ12KM8tzR", "reviewer": "Reviewer_tfV1", "summary": "This paper addresses the problem of speeding up adaptation training by introducing an adversarial training algorithm that is compatible with large batch training. Previously, while being able to significantly speed up adaptation training, large batch training has been shown to harm the model's ability to generalize. Adversarial training algorithm helps, but the inner optimization steps brings a major computation overhead over other optimization algorithms. Motivated by these insights, this paper proposes ScaLA, which (1) sticks to the adversarial solution but reduces gradient computation steps to 1, and (2) introduces a delay in perturbation injection, in order to bring down the computation overhead. Experiments show that the proposed training regime is faster than previously-proposed solutions like FreeLb and LAMB, while also improving the performance of the model, which indicates stronger generalization capabilities.", "questions": "Please respond to the points made in the weaknesses & detailed comments part above.", "limitations": "Adequately addressed to the best of my knowledge.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The research problem is well-motivated by the related work survey and the preliminary analyses.\n- The presentation of the solution is mostly clear, with the benefit of each part being clearly stated.\n- Experiments show clear benefit over existing methods in terms of adaptation training speed, while also showing strong performance.\n\nWeaknesses:\n\n- It is unclear to me if any of the modification here is specifically targeted to speed-up the adaptation training process (as opposed to pre-training).\n- There are a few non-ideal presentation choices, see below.\n\nSome more detailed comments:\n\n- Eqn. 2: the choice of using $x$ and $y$ as model parameters and perturbed embeddings is a very confusing one, especially since they were used as instances from the training data in (1).\n- L130: It would be nice to introduce what exactly with and without communication means, or provide some citations for it. I have trouble understanding how would the non-communication case work.\n- Algorithm 1 L9: I don't understand $\\prod_{\\omega}$ means here.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Uiy5r73Lzoo", "reviewer": "Reviewer_eYN9", "summary": "This paper proposes a ScaLA method to improve the computational efficiency of the fine-tuning of pre-trained language models while preserving generalization. ScaLA uses a large batch size to lower the total training iterations and employs adversarial training to mitigate the generalization drop brought by using a large batch size. This idea is quite counter-intuitive to me as it aims to lower the computational overhead, but introduces a more time-consuming adversarial training part. However, by using techniques such as one-shot adversarial examples and delayed perturbation injection, ScaLA is able to accelerate the adversarial training part and thus lower the overall computational overhead, which is empirically justified by experiments over a range of natural language understanding tasks.", "questions": "Kindly refer to my suggestion in the weakness part.", "limitations": "I do not see any potential negative social impact.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written and the method is clearly presented.\n2. The empirical result looks strong.\n\nWeakness:\n1. The proposed method looks like a naive combination of two existing techniques and thus to some extent incremental.\n2. It would help if the authors can include a more in-depth analysis of why large batch size does harm to generalization and how this issue can be addressed (or even best addressed) by adversarial training. The current version focuses more on empirical validation.\n\n===========After Rebuttal===========\n\nThank the authors for the response. After reading all the reviews and responses, I decided to keep my rating unchanged.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject