Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models
Authors: Zirui Wang
Yulia Tsvetkov
Orhan Firat
Yuan Cao
Venue: ICLR 2021 Spotlight (2021)
Keywords: Multi-task Learning
Multilingual Modeling

Abstract: Massively multilingual models subsuming tens or even hundreds of languages pose great challenges to multi-task optimization. While it is a common practice to apply a language-agnostic procedure optimizing a joint multilingual task objective, how to properly characterize and take advantage of its underlying problem structure for improving optimization efficiency remains under-explored. In this paper, we attempt to peek into the black-box of multilingual optimization through the lens of loss function geometry. We find that gradient similarity measured along the optimization trajectory is an important signal, which correlates well with not only language proximity but also the overall model performance. Such observation helps us to identify a critical limitation of existing gradient-based multi-task learning methods, and thus we derive a simple and scalable optimization procedure, named Gradient Vaccine, which encourages more geometrically aligned parameter updates for close tasks. Empirically, our method obtains significant model performance gains on multilingual machine translation and XTREME benchmark tasks for multilingual language models. Our work reveals the importance of properly measuring and utilizing language proximity in multilingual optimization, and has broader implications for multi-task learning beyond multilingual modeling.

Published as a conference paper at ICLR 2021
GRADIENT VACCINE: INVESTIGATING AND IMPROV- ING MULTI-TASK OPTIMIZATION IN MASSIVELY MUL- TILINGUAL MODELS
Zirui Wang- 2:2 Yulia Tsvetkov', Orhan Firat', Yuan Cao2 1 Carnegie Mellon University, "Google Al {ziruiw, ytsvetko) cmu. edu, forhanf, wwwaccaoeggooge....... con
ABSTRACT
Massively multilingual models subsuming tens or even hundreds of languages pose great challenges to multi-task optimization. While it is common practice to apply a language-agnostic procedure optimizing a joint multilingual task objec- tive, how to properly characterize and take advantage of its underlying problem structure for improving optimization efficiency remains under-explored. In this paper, attempt to peek into the black-hox of multilingual optimization through the lens of loss function geometry. We find that gradient similarity measured along the optimization trajectory an important signal, which correlates well with not only language proximity but also the overall model performance. Such observa- tion helps us to identify critical limitation of existing gradient- gradient-based multi-task learning methods, and thus we derive a simple and scalable optimization proce- dure, named Gradient Vaccine, which encourages more geometrically aligned pa- rameter updates for close tasks. Empirically, our method obtains significant model performance gains on multilingual machine translation and XTREME benchmark tasks for multilingual language models. Our work reveals the importance of prop- erly measuring and utilizing language proximity in multilingual optimization, and has broader implications for multi-task learning beyond multilingual modeling.
INTRODUCTION
Modern multilingual methods, such multilingual language models (Devlin et al. 2018 Lample
& Conneau 2019 Conneau et al. (219)) and multilingual neural machine translation (NMT) (Firat
et al. 2016 Johnson et al. 2017] Aharoni et al. 2019 AArivazhagan al.] 2019), have been showing success in processing tens or hundreds of languages simultaneously in a single large model. These models are appealing for two reasons: (1) Efficiency: training and deploying single multilingual model requires much less resources than maintaining one model for each language considered, (2) Positive cross- -ingull transfer: by transferring knowledge from high-resource languages (HRL), multilingual models are able to improve performance on low-resource languages (LRL) on wide variety of tasks (Pires et al. 2019 Wu & Dredze, 2019 Siddhant et al. 2020; Hu et al. (020).
Despite their efficacy, how to properly analyze or improve the optimization procedure of multilingual models remains under-explored. particular, multilingual models are multi-task learning (MTL) (Ruder] 2017) in nature but existing literature often train them in a monolithic manner, naively using a single language-agnostic objective on the concatenated corpus of many languages. While this approach ignores task relatedness and might induce negative interference (Wang et al.] 20200), its optimization process also remains a black-box, muffling the interaction among different languages during training and the tross-linguse transferring mechanism.
In this work, we attempt to open the multilingual optimization black-box via the analysis of loss geometry. Specifically,
aim to answer the following questions: (1) Do typologically similar languages enjoy more similar loss geometries in the optimization process of multilingual models? (2) If so, in the joint training procedure, do more similar gradient trajectories imply less interference between tasks, hence leading to better model quality? (3) Lastly, can we deliberately encourage
Work done during an intersship Gongle.

---

Published as a conference paper at ICLR 2021
more geometrically aligned parameter updates improve multi-task optimization, especially in real-world massively multilingual models that contain heavily noisy and unbalanced training data?
Towards this end, we perform comprchensive study massively multilinguul neural machine translation tasks, where each language pair is considered as a separate task. We first study the correlation between language and loss geometry similarities, characterized by gradient similarity along the optimization trajectory. We investigate how they evolve throughout the whole training process, and glean insights on how they correlate with cross-lingual transfer and joint performance. In particular, our experiments reveal that gradient similarities across tasks correlate strongly with both language proximities and model performance, and thus we observe that typologically close languages share similar gradients that would further lead to well-aligned multilingual structure (Wul et al., 2019) and successful cross-lingual transfer. Based on these findings, we identify a major limitation of popular multi-task learning method (Yu et al. 2020) applied in multilingual models and propose a preemptive method, Gradient Vaccine, that leverages task relatedness to set gradient similarity objectives and adaptively align task gradients to achieve such objectives. Empirically, our approach obtains significant performance gain over the standard monolithic optimization strategy and popular multi-task baselines on large-scale multilingual NMT models and multilingual language models. To the best of our knowledge, this is the first work to systematically study and improve loss geometries in multilingual optimization scale.
2 INVESTIGATING MULTI-TASK OPTIMIZATION IN MASSIVELY MULTILINGUAL MODELS
While prior work have studied the effect of data (Arivazhagan al., 2019; Wang et al. 2020a
architecture (Blackwood et al. 2018 Sachan & Neubig 2018 Vázquez et al. 2019 Escolano et al,, 2020) and scale (Huang et al., [2199] Lepikhin et al. 2020 on multilingual models, their opti- mization dynamics are not well understood. We hereby perform a series of control experiments on massively multilingual NMT models investigate how gradients interact in multilingual set- tings and what are their impacts on model performance, as existing work hypothesizes that gradient conflicts, defined as negative cosine similarity between gradients, can be detrimental for multi-task learning (Yu et al. 2020) and cause negative transfer (Wang et al. 2019).
2.1 EXPERIMENTAL SETUP
For training multilingual machine translation models, we mainly follow the setup in Arivazhagan al. (2019). In particular, we jointly train multiple translation language pairs single sequence- lo-sequence (seq2seq) model (Sutskever et al. 2014). We use the Transformer-Bi Big (Vaswani et al.) 2017] architecture containing 375M parameters described in (Chen et al., 201 8a). where all param- eters are shared across language pairs. We use an effective batch sizes of 500k tokens, and utilize data parallelism to train all models over 64 TPUv3 chips. Sentences are encoded using a shared source-target Sentence Piece Model (Kudo & Richardson] 2018) with 64k tokens, and a <2xx> token is prepended to the source sentence to indicate the target language (Johnson et al. 2017). The full training details can be found in Appendix B
To study roal-world multi-task optimization on a massive scale, we use an in-house training cor- pus (Arivazhagan et al.. 2019) generated by crawling and extracting parallel sentences from the web (Uszkoreit et af. 2010), which contains more than 25 billion sentence pairs for 102 languages to and from English. We select 25 languages (50 language pairs pivoted on English), containing over 8 billion sentence pairs, from 10 diverse language families and 4 different levels of data sizes (detailed in Appendix A). We then train two models on two directions separately, namely Any- and En->Any. Furthermore, to minimize the confounding factors of inconsistent sentence seman- ties across language pairs, we create multi-way aligned evaluation set of 3k sentences for all languages"] Then, for each checkpoint at an interval of 1000 training steps, we measure wise cosine similarities of the model's gradients on this dataset between all language pairs. We examine gradient similarities at various granularities, from specific layers to the entire model.
We also experiment on publicly available dataset of WMT and obtain similar observations in Appendix In other words, 3k semantically identical sentences are given in 25 languages.
2

---

Published as a conference paper at ICLR 2021
Turkic
Manue-Congo Slavic
Slavic
Uralic
West Western European
Malayo-Ponl Polynesiin
de
0.8
pl
c5
uk
be
0.4
nu
et
0.2
ne
KX
-02
ky
Xh
kd
Figure 1: Cosine similarities of encoder gradients between xx-en language pairs averaged across all training steps. Darker cell indicates pair-wise gradients are more similar. Best viewed color
2.2 OBSERVATIONS
We make the following three main observations. Our findings consistent across different model architectures and settings (see Appendix C and D for more results and additional discussions).
Gradient similarities reflect language proximities. We first examine close tasks enjoy simi- lar loss geometries and vice versa. Here, we use language proximity (defined according to their memberships in linguistic language family) to control task similarity, and utilize gradient sim- ilarity measure loss geometry. We choose typological similarity because it is informative and popular, and we leave the exploration of other language similarity measurements for future work. In Figure we use symmetric heatmap visualize pair-wise gradient similarities, averaged across all checkpoints at different training steps. Specifically, we observe strong clustering by membership closeness in the linguistic family, along the diagonal of the gradient similarity ma- trix. In addition, all European languages form large cluster in the upper-left corner, with an even smaller fine-grained cluster of Slavic languages inside. Furthermore, we also observe simi- larities for Western European languages gradually decrease in West Slavic Slavic-> Slavic, illustrating the gradual continuum of language proximity.
Gradient similarities correlate positively with model quality. As gradient similarities correlate well with task proximities, it is natural to ask whether higher gradient similarities lead to better multi-task performance.
Figure 2(a) we train a joint model of all language pairs in both En->Any and Any-En directions, and compare gradient similarities between these two. While prior work has shown that En- is harder and less amenable for positive transfer (Arivazhagan et al. 2019), we find that gradients of tasks in En->Any are indeed less similar than those in Any--Enn On the other hand, while larger batch sizes often improve model quality, we observe that models trained with smaller batches have less similar loss geometries (Appendix D). These all indicate that gradient interference poses great challenge to the learning procedure.
To further verify this, we pair En-*Fi with different language pairs (e.g. En-+Es or En-HHi, and train set of models with exactly language pairs We then evaluate their performance on the En- test set, and compare their BLEU scores versus gradient similarities between paired two tasks. As shown in Figure 2(b) gradient similarities correlate positively with model performance, again demonstrating that dissimilar gradients introduce interference and undermine model quality.
3. Gradient similarities evolve across layers and training steps. While the previous discussion focuses on the gradient similarity of the whole model averaged over all checkpoints, we now
WWestern European includes Romance and Germanic.
"oo remove confounding factors, fix the sampling strategy for all these models.

---

Published as a conference paper at ICLR 2021
X-n Bn-X
Cosine Similerity
(a)
(h)
Figure 2: Comparing gradient similarity versus model performance. (a): Similarity of model gradi- ents between xx-en (left) and en-xx (right) language pairs single Any->Any model. (b): BLEU scores en-fr of set of trilingual models versus their gradient similarities. Each model is trained on en-fr and another en-xx language pair.
study it across different layers and training steps. Figure 4(c) shows the evolution of the gradient similarities throughout the training. Interestingly, we observe diverse patterns for different gradi- subsets. For instance, gradients between En->Fr and En->Hi gradually become less similar (from positive to negative) in layer I of the decoder but more similar (from negative to positive) in the encoder the same layer. On the other hand, gradient similarities between En-*Fr and En-*Es are always higher than those between En En-FFr and En En-HHi in the same layer, consistent with prior observation that gradients reflect language similarities.
In addition, we evaluate the difference between gradient similarities the multilingual encoder and decoder in Figure 4(a) We find that the gradients more similar in the decoder (positive values) for the Any- En direction but less similar (negative values) for the En->Any direction. This is in line with needs to handle one single language. Moreover, we visualize how gradient similarities evolve across layers in Figure 4(b) We notice that similarity between gradients increase/decrease move up from bottom to top layers for the Any->En/En-AAn direction, and hypothesize that this due to the difference in label space (English-only tokens versus tokens from many languages).
intuition that gradients should be consistent when the decoder only
These results demonstrate that the dynamics of gradients evolve over model layers and training time.
Our analysis highlights the important role of loss geometries in multilingual models. With these points in mind, we next turn to the problem of how to improve multi-task optimization in multilin- gual models in a systematic way.
PROPOSED METHOD
Following our observations that inter-task loss ge- ometries correlate well with language similarities and model quality, a natural question to ask next take advantage of such gradient dy- namics and design optimization procedures superior to the standard monolithic practice. Since train large-scale models on real-world dataset consisting of billions of words, of which tasks are highly unbal- anced and exhibit complex interactions, we propose an effective approach that not only exploits inter-task structures but also is applicable to unbalanced tasks and noisy data. To motivate our method, we first review state-of-the- multi-tast learning method and show how the observation in Section helps us to identify its limitation.
how
Num of GradN::
Taaning Stap
Bariing 3100
Figure 3: Counts of active PCGrad (left) and Gra (right) during the training process.
3.1 GRADIENT SURGERY
An existing line of work (Chen et al. 2018b; Sener & Koltun 2018 Yu et al. 2020) has success- fully utilized gradient-based techniques improve multi-task models. Notably, Yu al. (2020)
4

---

Published as a conference paper at ICLR 2021
030
X.tn trX
1:0
052
Cosine Similarity
34
3.38
-366
032
0M6
-00 038
-M4
Training Step
(a)
(b)
(c)
Figure 4: Evaluating gradient similarity across model architecture and training steps. (a): Differ- ence between gradient similarities in the encoder and decoder. Positive value (darker) indicates the encoder has more similar gradient similarities, (b): Gradient similarities across layers. (c): Gradient similarities of different components and tasks across training steps.
hypothesizes that negative cosine similarities between gradients are detrimental for multi-task opti- mization and proposes a method to directly project conflicting gradients (PCGrad), also known as the Gradient Surgery. As illustrated in the left side of Figure [(a] the idea is to first detect gradient conflicts and then perform a "surgery" to deconflict them if needed. Specifically, for gradients g; and gj of the i-th and j-th task respectively at specific training step, PCGrad (1) computes their cosine similarity to determine if they are conflicting, and (2) if the value is negative, projects g, onto the normal plane of gj as:
g
- g:
By
g
The altered gradient g' replaces the original g, and this whole process is repeated across all tasks in a random order. For more details and theoretical analysis, we refer readers to the original work.
Now, we can also interpret PCGrad from a different perspective: notice that the gradient cosine similarity will always be zero after the projection, effectively setting target lower bound. In other words, PCGrad aims to align gradients to match a certain gradient similarity level, and implicitly makes the assumption that any two tasks must have the same gradient similarity objective of zero. However, as we shown in Section 2 different language proximities would result in diverse gradient similarities. In fact, many language pairs our model share positive cosine similarities such that the pre-condition for PCGrad would never be satisfied. This is shown in the left of Figure 5(b) where PCGrad is not effective for positive gradient similarities and thus it is very sparse during training in the left of Figure 3 Motivated by this limitation, we next present our proposed method.
3.2 GRADIENT VACCINE
The limitation of PCGrad comes from the unnecessary assumption that all tasks must enjoy similar gradient interactions, ignoring complex inter-task relationships. To relax this assumption, a natural idea is to set adaptive gradient similarity objectives in some proper manner. An example is shown in the right of Figure ((b) where two tasks have a positive gradient similarity of cos(0) dig- While PCGrad ignores such non-negative case, the current value of Φij may still be detrimentally low for more similar tasks such as French versus Spanish. Thus, suppose we have some similarity goal of cos = o7 > Φij (c.g. the "normal" cosine similarity between these two tasks), we alter both the magnitude and direction of g: such that the resulting gradients match such gradient similarity objective. In particular, we replace 9i with a vector that satisfies such condition in the vector space spanned by g, and g,, i.e. a1 gg + a2 gj- Since there are infinite numbers of valid combinations of and ag, for simplicity, we fix a and by applying Law of Sines in the plane of g: and gj. we solve for the value of a2 and derive the new gradient for the i-th task as b.
[g / 0 2isy ())
gj-
(2)
[g
(xT))
"See Appendix for derivation detail, implementation practice and thenretical analysis.
5

---

Published as a conference paper at ICLR 2021
My. Bj
Fi
Ri
8
8
(a)
(b)
Figure 5: Comparing PCGrad (left) with GradVac (right) in two cases. (a): For negative similarity, both methods are effective but Grad Vac can utilize adaptive objectives between different tasks. (b): For positive similarity, only GradVac is active while PCGrad stays "idle".
This formulation allows us arbitrary gradient similarity objective of 1-] The remaining question is how to set such objective properly. In the above analysis, we have seen that gradient interactions change drastically across tasks, layers and training steps. To incorporate these three factors, we exploit an exponential moving average (EMA) variable for tasks i.j and parameter group (e.g. the k-th layer) as:
1 = -1- 1) + Bo ljk
(3)
where Pijk the computed gradient similarity at training step t, is a hhyper-paameete and Pigk The full method is outlined Algorithm (Appendix E Notice that gradient surgery a special case of our proposed method such that / 0. As shown in the right of Figure ((a) and 5()) our method alters gradients more preemptively under both positive and negative cases, laking more proactive measurements in updating the gradients (Figure 3). We therefore refer to it Gradient Vaccine (GradVac). Notice that the resulting models will have the same numbers of parameters for deploying typical MNMT models and thus enjoy the same benefits for memory efficiency, while the proposed method will have the same order of complexity with the original multi-task training paradigm of computational efficiency.
(1)
20)
EXPERIMENTS
We compare multi-task optimization methods with the monolithic approach in multilingual settings, and examine the effectiveness of our proposed method on multilingual NMT and multilingual lan- guage models.
4.1 GENERAL SETUP
We choose three popular scalable gradient-based multi-task optimization methods as baselnes: GradNorm (Chen et al. (2018)), MGDA (Sener & Koltun| 2018), and PCGrad (Yu et al. 2020). For fair comparison, language-speeife gradients are computed for samples in each batch. The sam- pling temperature is also fixed at T=5 unless otherwise stated. For the baselines, the default settings and training procedures for aapepparaetter selection as explained in their spective papers. For our method, to study how sensitive GradVac is to the distribution of tasks, additionally examine a variant that allows us to control which languages are considered for GradVac. Specifically, we search the following hyper-parameters our large-scale dataset: tasks considered for GradVac (HRL_oonyy, LRL_only, all all_task), parameter granularity {whole model, enc dec, all layer, all matrix). EMA decay rate [1c-1, 1e-2, Ie-3]. We find onny..oll, all layer, le-2) to work generally well and use these the following experiments (see Appendix F for more details and results).
mainly follow
small-scale WMT dataset and transfer to
4.2 RESULTS AND ANALYSIS
WMT Machine Translation. We first conduct comprehensive analysis our method and other baselines small-scale WMT task. We consider two high-resource languages (WMT14 en- fr. WMT19 en-cs) and two low-resource languages (WMT14 en-hi, WMT18 en-tr), and train two models for both to and from English. Results shown in Table 1

---

Published as a conference paper at ICLR 2021
Ei--ARy en-hi
Any=
en-fr
en-cs
en-tr
avg
fr-en
cS-en
hi-en
tr-en
avg
Monolithic
Training
(1) Bilingual Model
41.80
24.76
5.77
9.77
20.53
36.38
29.17
8.68
13.87
22.03
(2) Multilingual Model
37.24
20.22
13.69
18.77
22.48
34.29
27.66
18.48
22.01
25.61
Mult-task
Training
[3] Norm (Chen et al. 20186
37.02
18.78
11.57
15,44
20.70
34.58
27.85
18.03
22.37
25.71
(4) MGDA (Seeer & Kollun 2018
38.22
17.54
12.02
13.69
20.37
35.05
26.87
18.28
22.41
25.65
(5) PCGrad at. 2020
37.72
20.88
13.77
18.23
22.65
34.37
27.82
18.78
22.20
25.79
(6) PCGrad W. ayyr
38.01
21.04
13.95
18,46
22.87
34.57
27.84
18.84
22.48
25.93
Our Approach
(7) Grad Vac w. fixed_ohj
38.41
21.12
13.75
18.68
22.99
34.35
27.97
18.72
22.14
25.85
(8) Grad model
38.76
21.32
14.22
18.89
23.30
34.84
28.01
18.85
22.
25.99
(9) GradVac w. all layer
39,27*
21.67
14.88
19.73
23.89
35.28*
28.42
19,07
22.58
26.34
Table BLEU scores on the WMT dataset. The best result for multilingual model is bolded while underline signifies the overall best, and means the gains over baseline multilingual models are statistically significant with p < 0.05.
First, we observe that while the naive multilingual baseline outperforms bilingual models on low- resource languages, it performs worse on high-resource languages due to negative interference (Wang et al., 2020b) and constrained capacity (Arivazhagan et al.) 2019). Existing baselines fail to address this problem properly, as they obtain marginal or even improvement (row 3, 4 and 5). In particular, we look closer at the optimization process for methods that utilize gradient signals to reweight tasks, i.e. GradNorm and MGDA, and find that their computed weights are less meaningful and noisy. For example, MGDA assigns larger weight for en-fr in the en-xx model, that results in worse performance on other languages. This is mainly because these methods are designed under the assumption that all tasks have balanced data. Our results show that simply reweighting task weights without considering the loss geometry has limited efficacy.
By contrast, our method significantly outperforms all baselines. Compared to the naive joint train- ing approach, the proposed method improves over not only the average BLEU score but also the individual performance on all tasks. We notice that the performance gain on En-tAny is larger com- pared Any->En. This is in line with our prior observation that gradients are less similar and more conflicting in En->Any directions.
We next conduct extensive ablation studies for deeper analysis: (1) GradVac applied to all layers vs. whole model (row 8 vs. 9): the all layer variant outperforms whole model, showing that setting fine-grained parameter objectives is important. (2) Constant objective vs. EMA (row 7 vs. 9): we also examine variant of Grad Vac optimized using a constant gradient objective for all tasks (e.g. iy = 0.5, Vi,j) j) and observe performance drop compared using EMA variables. This highlights the importance of setting task-aware objectives through task relatedness. (3) GradVac vs. PCGrad (row 8-9 vs. 5-6): the two GradVac variants outperform their PCGrad counterparts, validating the effectiveness of setting preemptive gradient similarity objectives.
TT
Massively Multilingual Machine Transla- tion. We then scale up experiments and transfer the best setting found on WMT to the same massive dataset used in Section 2 We visualize model performance in Figure 6 and average BLEU scores are shown Table We additionally compare with models trained with uniform language pairs sampling strategy (T=1) and find that our method outperforms both multilingual models. Most notably, while uniform sampling favor high-resource language pairs more than low-resource ones, Grad Vac is able to improve both consistently across all tasks. We observe larger performance gain on high-resource languages, illustrating that addressing gradient conflicts mitigate negative interfer- ence on these head language pairs. On the other hand, our model still perform worse on resourceful languages compared to bilingual baselines, most likely limited by model capacity.
Any En
High
Med
Low
All
T=1
28.56
28.51
19.57
24.95
T=5
28.16
28.42
24.32
26.71
Grad Vac
28.99
28.94
24.58
27.21
En-+Any
High
Med
Low
All
T=1
22.62
21.53
12.41
18.18
T=5
22.04
21.43
13.07
18.25
Grad
24.20
21.83
13.30
19.08
Table 2:
Average BLEU scores of 25 language pairs on our massively multilingual dataset.
7

---

Published as a conference paper at ICLR 2021
Blsseines Mubtiinusl
Biligual Baselines
Muttiinguel
Mohilnguel T=5
Multilnguel Gradkac
GradVacc
ABL
Language Pairs
Language Pairs
(a) X-Fn
(b) En-X
Figure 6: Comparing multilingual models with bilingual baselines dataset. Language pairs are listed in the order of training data sizes (high-resource languages on the left).
de
hi
jv
kk
my
te
t]
yo
avg
mBERT
83.2
77.9
87.5
82.2
77.6
87.6
82.0
75.8
87.7
78.9
83.8
90.7
82.9
GradNorm
83.5
77.4
82.7
78.4
87.9
81.2
73.4
85.2
78.7
83.6
82.6
.MGA
82.1
74.2
81.5
77.8
87.8
81.9
78.2
87.5
91.7
82.4
PCGrra
83.7
78.6
88.2
81.8
79.6
87.6
81.8
74.2
85.9
78.5
85.6
92.2
83.1
+ Vre
83.9
79.4
88.2
81.8
80.5
87.4
82.1
73.9
87.8
79.3
87.8
93.0
83.8
Table F1 on the NER tasks of the XTREME benchmark.
XTREME Benchmark. We additionally apply method multilingual language models and evaluate on the XTREME benchmark (Hu et al. 2020). We choose tasks where training data are available for all languages, and finetune pretrained multilingual BERT model (mBERT) (Devlin iet al. (2018) on these languages jointly (see Appendix G for experiment details and additional re- sults). As shown in Table 3 our method consistently outperforms naive joint finetuning and other multi-task baselines. This demonstrates the practicality of our approach for general multilingual tasks.
RELATED WORK
Multilingual models train multiple languages jointly (Firat et al. 2016 Devlin al. 2018 Lample
Conneau 2019: Conneau al. 2019 Johnson et al. 2017; 7; Aharoni et al.) 2019) Arivazhagan al. 2019). Follow-up work study the cross lingull ability of these models and what contributes to (Pirss et al. 2019 Wu Dredze 2019 Wu et al.] 2019 AAtetxe et al.) [2119] Kudugunta et al. al.] (2000), the limitation of such training paradigm (Arivazhagan al. [019 Wang et al. 2020b), and how to further improve it by utilizing post-hoc alignment (Wang et al, 2020c Cao et al. 2020). data balancing (Jean al. 2019 Wang al. [2020a) or calibrated training signal Mulcaire el al. 2019 Huang et 2019)). In contrast to these studies, we directly investigate language interactions across training progress using loss geometry and propose language-aware
2019 Karthikeyan
method to improve the optimization procedure.
On the other hand, multilingual models can be treated multi-task learning methods (Ruder 2017 Zamir et al.] 2018). Prior work have studied the optimization challenges of multi-task training (Hes-
sel al. a 201 [chau) et 2019). while others suggest to improve training quality through learning task relatedness (Zhang & Yeung 2012), routing task- specifc paths (Rusu et al.) 2016 Rosenbaum ; Lin et al. 2019). However, while these methods are often evaluated on balanced task distributions, multilingual datasets are often unbalanced and noisy. As prior work have shown training with unbalanced tasks can be prone to negative interference (Ge et al. 2014;) Wang Carbonell] 2018), we study how mitigate it in large models trained with highly unbalanced and massive-scale dataset.
al. 2019), altering gradients directly (Kendall et al. 20018 Chen et al. 2018b; Du et al.) 2018 Yu et al. 2020), or searching pareto solutions (Sener & Koltun 201 8
8

---

Published as a conference paper at ICLR 2021
6 CONCLUSION
In this paper, we systematically study loss geometry through the lens of gradient similarity for mul- tilingual modeling, and propose a novel approach named GradVac for improvement based on our findings. Leveraging the linguistic proximity structure of multilingual tasks, we validate the as- sumption that more similar loss geometries improve multi-task optimization while gradient conflicts can hurt model performance, and demonstrate the effectiveness of more geometrically consistent updates aligned with task closeness. We analyze the behavior of the proposed approach on mas- sive multilingual tasks with superior performance, and we believe that approach is generic and applicable beyond multilingual settings.
ACKNOWLEDGMENTS
We want to thank Hieu Pham for tireless help to the authors different stages of this project. We also would like to thank Zihang Dai, Xinyi Wang, Zhiyu Wang, Jiateng Xie, Yiheng Zhou, Ruochen Xu, Adams Wei Yu, Biao Zhang, Isaac Caswell, Sneha Kudugunta, Zhe Zhao, Christopher Fifty, Xavier Garcia, Ye Zhang, Macduff Hughes, Yonghui Wu, Samy Bengio and the Google Brain team for insightful discussions and support the work. This material is based upon work supported in part by the National Science Foundation under Grants No. IIS2007960 and IIS2040926, and by the Google faculty research award.
REFERENCES
Roee Aharoni, Melvin Johnson, and Orhan Firat. Massively multiliigua neural machine translation. CoRR, abs/1903.00089, 2019. URI. htt.:/ttp//aaxx..oo///////////// 00089
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, et al. Massively multilingual neural machine translation in the wild: Findings and challenges. arXiv preprint arXiv: 190.0010,,,0011, 2019.
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the cross lonss-l transferability of mono- lingual representations. arXiv preprint
2019.
Graeme Blackwood, Miguel Ballesteros, and Todd Ward. Multilingual neural machine transla- tion with task-specific attention. CoRR, abs/1806.03290 2018. URL http::/arxiv..or/
abs/ / 180688860000 03280
Steven Cao, Nikita Kitaev, and Dan Klein. Multilingual alignment of contextual word representa- tions. In International Conference on Learning Representations, 2020.
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Niki Parmar, Mike Schuster, Zhifeng Chen, The best of both worlds: Combin- ing recent advances in neural machine translation. arXiv preprint arXiv: 1804.:880.0990 09849, 2018a.
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International Conference on Machine Learning. pp. 794-803, PMLR, 2018b.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Un- supervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02111 2019.
Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04880, 2018.
Yunshu Du, Wojeiech M Czarnecki, Siddhant M Jayakumar, Razvan Pascanu, and Balaji Lakshmi- narayanan. Adapting auxiliary losses using gradient similarity, arXiv preprint arXiv: wrXiv:1812.02224 2018.

---

Published as a conference paper at ICLR 2021
Carlos Escolano, Marta Costa-jussä, José AR Fonollosa, and Mikel Artetxe. Multilingual ma- chine translation: Closing the gap between shared and sanguage-speciffe encoder-decoders. arXiv preprint arXiv:2004.06575, 2020.
Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. Multi-way, multilingual neural machine transla- tion with a shared attention mechanism. arXiv preprint arXiv: 660100103, 2016.
Liang Ge. Jing Gao, Hung Ngo, Kang Li, and Aidong Zhang. On handling negative transfer and im- balanced distributions in multiple source transfer learning. Statistical Analysis and Data Mining: The ASA Data Science Journal, 7(4))254-21) 271, 2014.
Matteo Hessel, Hubert Soyer, Lasse Espeholt, Wojeiech Czarnecki, Simon Schmitt, and Hado van Hasselt. Multi-task deep reinforcement learning with popart. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 33, pp.
2019.
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. Xtreme: A massively multilingual multi-task benchmark for evaluating cross- moss-linngall generaliza- tion. arXiv preprint urXiv:2003. 1080, 2020.
Haoyang Huang, Yaobo Liang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, and Ming Zhou. Unicoder: A universal language encoder by pre-training with multiple cross- cross-lingual tasks. arXiv preprint arXiv: 909. 0096009 2019a.
Yanping Huang, Youlong Cheng. Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al. Gpipe: Efficient training of giant neural networks using pipeline parallelism. In Advances in neural information processing systems, pp- 103-112, 2019b.
Sébastien Jean, Orhan Firat, and Melvin Johnson. Adaptive scheduling for multi-task learning, 2019.
Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, al. Google's multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339-351, 2017.
K Karthikeyan, Zihan Wang, Stephen Mayhew, and Dan Roth. Cross-lingual ability of multilingual bert: empirical study. In International Conference on Learning Representations, 2020.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7482-7491, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6980, 2014.
Taku Kudo and John Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint
2018.
Sneha Reddy Kudugunta, Ankur Bapna, Isaac Caswell, Naveen Arivazhagan, and Orhan Firat. In- vestigating multilingual nmt representations at scale. arXiv preprint arXiv:1909.02197 2019.
Guillaume Lample and Alexis Conneau. Cross-lingual language model pretraining. arXiv preprint 1xXiv:9011.0291, 2019.
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu. Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006. I 16668, 2020.
Xi Lin, Hui- Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-task learning. In Advances in Neural Information Processing Systems, pp. 12060-12070, 2019.
10

---

Published as a conference paper at ICLR 2021
Phoebe Mulcaire, Jungo Kasai, and Noah A Smith. Polyglot contextual representations improve crosslingual transfer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume 1 (Long and Short Papers), pp. 3912-3918, 2019.
Joakim Nivre, Mitchell Abrams, Željko Agié, Lars Ahrenberg, Lene Antonsen, Maria Jesus Aran- zabe, Gashaw Arutie, Masayuki Asahara, Luma Ateyah, Mohammed Attia, et al. Universal de- pendencies 2.2. 2018.
Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. Cross- lingual name tagging and linking for 282 languages. In Proceedings of the 55th Anmual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp- 1946-19555 2017.
Addison Phillips and Mark Davis. Tags for identifying languages. Technical report, BCP 47, RFC 4646, September, 2006.
Telmo Pires, Eva Schlinger, and Dan Garrette. How multilingual is multilingual bert? arXiv preprint arXiv:1906.01500 2019.
Clemens Rosenbaum, Ignacio Cases, Matthew Riemer, and Tim Klinger. Routing networks and the challenges of modular and compositional computation. arXiv preprint arXiv:1904.12774 2019.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint wrXiv:1706.05095 05098, 2017.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 160.:66666.4661,, 2016.
Devendra Sachan and Graham Neubig. Parameter sharing methods for multilingual self-attentional translation models. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 261-271, Brussels, Belgium, October 2018, Association for Computational Lin- guistics. doi: 10.18653/1//18--6222 URL https: /wwww www.:alwwwb...e...eee wwt..rroo/attoooggo...oooo
W18-6327 6327
Tom Schaul, Diana Borsa, Joseph Modayil, and Razvan Pascanu. Ray interference: a source of plateaus in deep reinforcement learning. arXiv preprint arXiv: /904. 1455, 2019.
Ozan Sener and Vladlen Koltun. Multi-task learning multi-objective optimization. In Advances in Neural Information Processing Systems, pp. 527-538, 2018.
Aditya Siddhant, Melvin Johnson, Henry Tsai, Naveen Ari, Jason Riesa, Ankur Bapna, Orhan Firat, and Karthik Raman. Evaluating the cross-lingual effectiveness of massively multilingual neural machine translation. In AAAI, pp. 8854-8861, 2020.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances neural information processing systems, pp. 3104-3112 12, 2014.
Jakob Uszkoreit, Jay Ponte, Ashok Popat, and Moshe Dubiner. Large scale parallel document mining for machine translation. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010). pp. 1 1101-1109, 2010.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 1998-6008, 2017.
Raúl Vázquez, Alessandro Raganato, Jörg Tiedemann, and Mathias Creutz. Multilingual NMT with hunguage-independer attention bridge. In Proceedings of the 4th Workshop on Representa- tion Learning for NLP (RepL4NLP-2011) pp. 33-39, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 10.18653/v 1/W19-435. URL https: / www aclweb.
bzq./anthelogy/iii 9- 4305
Xinyi Wang, Yulia Tsvetkov, and Graham Neubig. Balancing training for multilingual neural ma- chine translation. arXiv preprint arXiv:2004.0674 2020a.

---

Published as a conference paper at ICLR 2021
Zirui Wang and Jaime Carbonell. Towards more reliable transfer learning. In Joint European Con- ference on Machine Learning and Knowledge Discovery Databases, pp. 794-810. Springer, 2018.
Zirui Wang, Zihang Dai, Barnabás Póczos, and Jaime Carbonell. Characterizing and avoiding nega- tive transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11293-11302, 2019.
Zirui Wang. Zachary C' Lipton, and Yulia Tsvetkov. negative interference in multilingual models: Findings and a meta-learning treatment. In EMNLP, 2020b.
Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang. Graham Neubig, and Jaime Carbonell. Cross- lingual alignment vs joint training: A comparative study and a simple unified framework. In International Conference Learning Representations, 2020c.
Shijie Wu and Mark Dredze. Beto, bentz, becas: The surprising cross-lingual effectiveness bert.
arXiv preprint arXiv:1904.09077, 2019.
Shijie Wu, Alexis Conneau, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. Emerging cross- lingual structure in pretrained language models. arXiv preprint 19110116 2019.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.
Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.06782, 2020.
Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 3712-3722, 2018.
Yu Zhang and Dit-Yan Yeung. A convex formulation for learning task relationships in multi-task learning, arXiv preprint arXiv: 1203.3330,333 2012.
12

---

Published as a conference paper at ICLR 2021
A
DATA STATISTICS
Data Distribution over Language Pairs
Number Examples (log)
10a
10a
10"
50
Language Pairs
Figure 7: Per language pair data distribution of the dataset used train multilingual model. The yaxis depicts the number of training examples available per language pair on a logarithmic scale.
We select 25 languages (50 language pairs) from our dataset to be used in our multilingual models for more careful studies on gradient trajectory. For such purpose, we pick languages that belong different language families (typologically diverse) and with various levels of training data sizes. Specifically, we consider the following languages and their details are listed in 4 French (frr, Span- ish (es), German (de), Polish (pl), Czech (cs), Macedonian (mk), Bulgarian (bg), Ukrainian (uk), Belarusian (be), Russian (ru), Latvian (Iv), Lithuanian (It), Estonian (et), Finnish (fi), Hindi (hi), Marathi (mr). Gujarati (gu), Nepali (ne), Kazakh (kk), Kyrgyz (ky), Swahili (sw), Zulu (zu), Xhosa (xh), Indonesian (id), Malay (ms).
Our corpus has languages belonging to a wide variety of scripts and linguistic families. The selected 25 languages belong 10 different language families (e.g. Turkic versus Uralic) or branches within language family (e.g. East Slavic versus West Slavic), as indicated in Figure Table Families groups of languages believed share a common ancestor, and therefore tend to have similar vocabulary and grammatical constructs. We therefore utilize membership of language family to define language proximity.
In addition, our language pairs have different levels of training data, ranging from 10th to 10 sen- tence pairs. This shown Figure We therefore have four levels of data sizes (number of languages in parenthesis): High (7), Medium (8), Low (5), and Extremely Low (5). In particular, we consider tasks with more than 10$ to be high-resource, 10" 10 be medium-resoure, and rest to be low-resource (with those below 5 million sentence pairs to be extremely -wwrrssurcee. Therefore, our dataset is both heavily unbalanced and noisy, as it is crawled from the web, and thus introduces optimization challenges from multi-task training perspective. These characteristics of our dataset make the problem that we study as realistic as possible.
B TRAINING DETAILS
For both bilingual and multilingual NMT models, utilize the encoder-decoder Transformer [Vaswani et al. 2017) architecture. Following prior work, we share all parameters across all lan- guage pairs, including word embedding and output softmax layer.
To train each model, we use a single Adam optimizer (Kingma & Ba 2014) with default decay hyper-parameters. We warm up linearly for 30K steps to a learning rate of le-3, which then decayed with the inverse square root of the number of training steps after warm-up. At each training step, we sample from all language pairs according to a temperature based sampling strategy as in prior work (Lample & Conneau, 2019 AAriazhhaan et al. (201)). That is, at each training step, we sample each sentence from all language pairs to train proportionally to P. = ( ) where T,, is the size the training corpus for language pair and T is the temperature. We set T=5 for most of experiments.
13

---

Published as a conference paper at ICLR 2021
Language
Language Family
Data Size
Language Id
Language Family
Size
French
fr
Western European
High
Finnish
fi
Uralic
High
Spanish
Wester European
High
Hindi
hi
Indo- Indo-Iranian
Medium
German
de
Westem European
High
Marathi
mr
Indo-Iranian
Ex-Low
Polish
West Slavic
High
Gujarati
gu Indo-Iramian
Low
Czech
West Slavic
High
Nepali
ne
Indo-Iranian
Ex-Low
Macedonian
mk
South Slavic
Low
Kazakh
kk
Turkic
Low
Bulgarian
bg
South Slavic
Medium
Kyrgyz
ky
Turkic
Ex-L.ow
Ukrainian
uk
East Slavic
Medium
Swahili
sw
Benue-Conge Conuo
Low
Belarusian
Russian be
East Slavic
Low
Zulu
Benue-Congo
Fx-Low
East Slavic
High
Xhosa
xh
Benue-Conge
Ex-L.ow
Latvian
Baltic
Medium
Indonesian
Malayo-Puu Palayo-oolynesian
High
Lithuanian
Baltic
Medium
Malay
ms
Malayo-Polynesie
Medium
Estonian
et
Uralic
Medium
Table 4: Details of all languages considered our dataset. Notice that since German (Germanic) particularly similar French and Spanish (Romance), we consider larger language branch for them named "Western European". "Ex-Low" indicates extremely low-resource languages in our dataset. We BCP-47 language codes labels (Phillips & Davis 2006).
Language
Id
Language Family
Data Size
Validation Set
French
fr
Romance
4IM
newstest2013
Spanish
es
Romance
15M
newstest2012
Russian ru
Slavic
38M
newstest2018
Czech CS
Slavic
37M
newstest2018
Latvian lv
Baltic
6M
newstest2017
Lithuanian It
Baltic
6M
newstesr2019
Estonian et Uralic
2M
newstesr2018
Finnish fi
Uralic
6M
newstest2018
Table 5: Details of all languages selected from WMT for gradient analysis.
C ADDITIONAL RESULTS ON WMT
C.1 DATA
We experiment with WMT datasets that are publicly available. Compared to our dataset, they only contain relatively small subsets of languages. Therefore, select 8 languages (16 language pairs) of 4 language families to conduct the same loss geometries analysis Section 2 These languages detailed Table }5 French (fr), Spanish (es), Russian (ru), Czech (cs), Latvian (lv), Lithuanian (It), Estonian (et), Finnish (fi). We collect all available training data from WMT 13 to WMT 19, and then perform deduplication process to remove duplicated sentence pairs. We then use the validation sets to compute gradient similarities. Notice that unlike our dataset, WMT validation sets are not aligned Therefore, the semantic structures of these sentences may introduce an extra degree of noise.
C.2 VISUALIZATION
As in Section we compute gradients on the validation sets on all checkpoints and averaged across all checkpoints to visualize our results. We use similar setups to our previous analysis, including model architectures, vocabulary sizes, and other training details. The main results are shown in Figure 8 Similar to findings in Section 2 gradient similarities cluster according to language proximities, with languages from the same language family sharing the most similar gradients on the diagonal. Besides, gradients in the English to Any directions less similar compared to the other direction, consistent with our above findings. Overall, despite the scale being much smaller in terms of number of languages and sizes of training data, findings mostly consistent.
14

---

Published as a conference paper at ICLR 2021
16
04
15
:14
-13
02
-00
(a) X-En
(b) En-X
Figure 8: Cosine similarities on WMT dataset averaged across training steps.
25
64
-00
ct
Figure 9: Cosine similarities (on Transformer- Base models) of xx-en language pairs on WMT dataset averaged across all training steps.
C.3 VISUALIZATION SMALLER MODELS
Prior work has shown languages fighting for capacity in multilingual models (Arivazhagan al. 2019 Wang et al., 20206). Therefore, we are also interested to study the effect of model sizes on gradient trajectory. Since our larger dataset contains language pairs Lransformer-LLarge model, we additionally train Transformer-Bane Base model using the 8 language pairs of WMT. We visualize it in Figure 9 and find that our observed patterns are more evident in smaller models. This finding is consistent across other experiments we ran and indicates that languages compete for capacity with small model sizes thereby causing more gradient interference. also shows that our analysis in this work is generic across different model settings.
D ADDITIONAL. RESULTS ON OUR DATASET
In Figure we show visualization on models trained using Any->En language pairs. Here, we also examine models trained in the other direction, En-tAny. As shown in Figure TO] we have similar observations made in Section 2 such that gradient similarities cluster strongly by language proximities. However, the en-xx model has smaller scales in cosine similarities and more negative values. For example, Nepali shares mostly conflicting gradients with other languages, except for those belonging to the same language family. This is in line with above discussion that gradient interference may a source of optimization challenge, such that the en-xx model is harder to train than the xx-en model.
15

---

Published as a conference paper at ICLR 2021
Western European
Indo-Iranian
Benue- Benue-Conge Slavic
Maleyy-PPolyneiian
Uralic
East
Slavic
d6
pl
05
uk
0.4
be
0.2
et
0.0
ne
KX
20
Figure 10: Cosine similarities of decoder gradients between en-xx language pairs averaged across all training steps. Darker cell indicates pair-wise gradients are more similar.
European
Malayo Polynneia
Slavic
Benue-Congo
South Slavic
Uralic
de
pl
cs
uk
0.4
ba
nu
0.2
et
0.0
ne
KX
X0
Cosine similarities of decoder gradients between en-xx language pairs averaged across all training steps. Darker cell indicates pair-wise gradients are more similar. Model trained with smaller batch sizes.
Figure 11:
Moreover, while our previous models are trained using large batch size for better performance (as observed in prior work (Arivazhagan al.] 2019)), also evaluate gradients in model trained with smaller batches (125k tokens) in Figure 11 Compared to model trained with larger batch sizes, this model enjoy similar patterns but with smaller gradient cosine similarity values, indicating that gradients are less similar. This presents an additional potential explanation of why larger batch sizes can be more effective for training large models: they may better reflect the correct loss geometries such that gradients are less conflicting in nature. For our case, this means larger batches better reflect language proximities hence gradients better quality.
Finally, these results also reveal that gradient similarities are mostly dependent on task relatedness, as even sentence pairs with identical semantic meanings can have negative cosine similarities due to language differencees
16

---

Published as a conference paper at ICLR 2021
Algorithm Vaa Update Rule
1: Require: EMA decay 3, Model Components M {0x}. Tasks for Grad G =Ti}
2: Initialize model parameters
3: Initialize EMA variables Piyk 0, Vi
4: Initialize time step =
5: while not converged do
6: Sample minibatch of tasks 1 {Ti}
7: for 02 E M do
8: Compute gradients gik Vo, LT. VT:
9: Set BiN +- gik
10:
for Ti E G ( do
11:
for T, B Ti in random order do
12:
Compute iykk
<
II, IIlR
kic 8ik
if (0) (1)
20)) Gj 1jk
then
ll 11 1- (0
(6 (3k)))
Set gi gi +
gjk
115 V 1 1- )a
end if
((++1) Update Pijk =(1 -1- (1- + B end for
16:
17:
18:
end for
19:
Update Og with gradient Eg'r end for
20:
21:
Update t ++
22:
end while
17

---

Published as a conference paper at ICLR 2021
E PROPOSED METHOD DETAILS
In this part, we provide details of our proposed method, Gradient Vaccine (GradVac). We first show how to derive formulation in Eq. _ followed by how instantiate in practice. And last, also study its theoretical property.
E. METHOD DERIVATION
As stated in Section the goal of our proposed method is to align gradients between tasks to match a pre-set target gradient cosine similarity. An example is shown in Figure 12 where we have two tasks i.j and their corresponding gradients gi and gj have cosine similarity of Φij- ie. cos(0) Gi lig: ||l Then, we want to alter their gradients, such that the resulting new gradients have gradient similarity of some pre-set value o.. To do so, replace gi with new vector in the vector space spanned by g; and gj. g d1 g; 02 Sj- Without loss of generality, we set a1 = 1 and solve for 42 i.c. find the d2 such that cos(r) = g; 1. Q.j- By using Laws of Sines, must have that:
|B:ll
02 12
(4)
sin
sin - 7)
and thus we can further solve for 02
|g:
0g
||||,|1
sin(?)
sin
- 7)
42 |g]
sin
(f) cos(7)
cos (0) sin
Ig:
02 0211g,1
(oT)²
67 1 di - 0i5 1 (oT)²
Ig: (0
03 - Ois 1 (0)
02
|lg
(of))
We therefore arrive at the update rule in Eq. Our formulation allows to set arbitrary target values for any two gradients, and thus we can better leverage task relatedness by setting individual gradient similarity objective for each task pair. Notice that can rescale the gradient such that the altered gradients will have the same norm as before. But in our experiment we find is sufficient to ignore this step. On the other hand, note that when of O, have that:
Ig 100
Di5
( ))
||
1ll
[0
-01)
[|g]
gi gg
||g:
gg:
|Il g
g
g:
g
g 2
This is exactly the update rule of PCGrad in Eq. T method.
Thus, PCGrad special case of our proposed
E.2 ALGORITHM IN PRACTICE
Our proposed method detailed in Algorithm I
In our experiments on multilingual NMT and
multilingual BERT, utilize set of exponential moving average (EMA) variables to set proper
18

---

Published as a conference paper at ICLR 2021
8
g
Figure 12: Pictorial description of our method.
pair-wise gradient similarity objectives, as shown in Eq. 3 This is motivated by our observations Section [2] such that gradients of different languages in Transformer model evolve across layers and training steps. Therefore, we conduct VradVac on different model components independently. For example, we can do one GradVac on each layer in the model, or just perform a single Grad Vac on the entire model. In addition, we also introduce an extra degree of freedom by controlling which tasks to perform GradVac. This corresponds to selecting of tasks and only alter gradients for tasks within this set, as shown in line 10 in Algorithm [ Empirically, we find performing Grad Vac by layers and on low-resource languages to work generally the best (See Appendix for detailed discussion).
E.3 THEORETICAL PROPERTY
Finally, analyze the theoretical property of our method. Supper only have two tasks, and their losses are L1 and L2, and we denote their gradient cosine similarity at a given step as ø12- When Φ12 is negative,
method is largely equivalent to PCGrad and enjoy PCGrad's convergence analysis. Thus, here we consider the other case when Φ12 is positive and show that:
Theorem 1. Suppose L1 and L2 convex and differentiable, and that the gradient of Lipschitz continuous with constant L > 0. Then, the GradVac update rule with step size < L[1+a7] and
I < + where a sin(d12- -6fa) (612 > and oT 2 Φ12 some target cosine similarity), will sinio(s)
converge to the optimal value ((
Proof. Let gi = VL, and g2 V V2 be gradicnts for task 1 and task 2 respectively. Thus we have ||aal|
+ g2 as the original gradient and g' = + a liga gi the Grad Vac update rule, such that:
||g: [igall g2 as the altered gradient by
sin(212 cos(( (of2) cos(612) sin(eTe)
(5)
sin(
where oT, is some pre-set gradient similarity objective and 072 2 Φ12 (thus a 2 0 since we only consider the angle between two gradients the range of to n))
Then, we obtain the quadratic expansion of L as:
c(O+) < C(0)+ VCC)) -6) v2 ((0))e+ - 0le
(6)
and utilize the assumption that VL is Lipschitz continuous with constant I., we have:
=(0t) < ((0) + V 0) + + 4I -
011
NNttcee that in reality a² will almost always be smaller than and thus it sufficient to assume <,
19

---

Published as a conference paper at ICLR 2021
Thus, plug in the update rule of Grad Vac obtain:
L(0+) - (()) - t. g.....ggo gg +
B2
81
+
8i
82)
+
Lt +
82
81
+
0
B1
82
81
g2
gi
82
(Plug in in gi + 82 and re-arrange terms)
1 + a²
= - (t
Lt + aQ12 Lt ( + |g2l(²)
2
- -(2a(t Lt
+ 012 (2t Lt + a ))) ||2al])
C()) (1 1 + a²
1 a
Ll ) g 112 + ||g2| 2001((
Lt (())ll ||gall)
2
- (aad1z(t - (I 2| ||>)| (2a(t ([ggi) ||2ll
(Remove non-iiie terms)
((0) (t 1 + a²
1 a² (())l llaall
Lt (lg || + ||aa| 2012(t
2
(()) - (t 1 a²
Lt (1g 1| + || + 2012||B111
2
((0) - (t + a²
LL (1]:ll + || + 2g1 82)
2
+
a²
C()) (t
[[g) 12
2
a²
(()) (t 2
Lt |gll
we have that /. 1+o 2 0 and thus L((+) L()) (unless the gradient has norm). This tells us 2 applying update rule of GradVac can reach the optimal value L(0*) since the objective function strictly decreases.
The last line implies that if we choose learning rate to be small enough t M(I+a=j>
2
20

---

Table 8: Comparing EMA decay rate 33 for GradVac. Parameter granularity fixed at all layer and Published as a conference paper at ICLR 2021
en-fr
en-cs
en-hi
en-tr
avg
Vac w. only
39.07
21.51
14.92
19.63
23.78
GradVac w. LRL_only
39.27
21.67
14.88
19.73
23.89
Vra w. all task
38.85
21.47
14.48
19.75
23.64
Table 6: Comparing which tasks to be included for GradVac. Parameter granularity fixed at all layer while 3=1e-2.
en-fr
en-cs
en-hi
en-tr
avg
VradVac w. whole model
38.76
21.32
14.22
18.89
23.30
VradVac enc_dec
39.05
21.73
14.54
19.33
23.66
GradVac w. layer
39.27
21.67
14.88
19.73
23.89
Grad all matrix
38.95
21.56
14.57
19.01
23.52
Table 7: Comparing parameter granularity for GradVac. Grad tasks fixed at LRL_,olyy while 8=1e-2.
en-fr
en-cs
en-hi
en-tr
avg
GradVac w. B=le-1
38.72
20.74
14.52
19.25
23.31
GradVac 3=1e-2
39.27
21.67
14.88
19.73
23.89
VradVac w. f=1e-3
38.85
20.96
14.85
19.68
23.59
GradVac tasks fixed at LRL_only.
F HYPER-PARAMETER SETTINGS
Here, we show how we choose the best hyper-paramedd parameter setting for our method. As discussed in Appendix(E) there are three hyper-paramete settings for our implementation: (1) which tasks be considered for GradVac, (2) which layers measure EMA and perform GradVac, (3) EMA decay rate. Due to the scale our model on the larger dataset, we use the smaller scale WMT dataset to find the optimal setting and transfer to other experiments. We do this by grid search using average perplexity on the validation set. Below, we demonstrate part of our results for each hhper--yrramee parameter to choose from.
First, we examine the effect of what tasks to include for GradVac, i.e. G in Algorithm T We consider three options: (1) HRL only: only perform GradVac high-resource languages, (2) LRL LRL,only: only perform GradVac on low-resource languages, (3) all task: perform GradVac on all languages. Results shown in Table We find that only conducting GradVac subset of languages obtain better performance while it is the best to conduct Grad Vac on low-resource language only. This is probably because the effective batch sizes of low-resource languages are usually smaller due the sampling strategy.
Next, compare the effect of parameter granularity model quality. This corresponds set- ting different model components for GradVac (M in Algorithm T.. We consider four possibilities, from coarse to fine-graid -ieeierined:: (1) whole model: only perform GradVac once the entire model, (2) enc dec: perform separately for encoder and decoder, (3) all layer: perform individually for each layer in encoder and decoder, (4) all_matrix: perform for each parameter matrix in the model. As shown in Table 7 we find that choosing proper parameter granularity is important, neither too coarse nor too fine-grained perform the best. This is consistent with our observation made Sec- note that settings are based NLP tasks and Transformer networks, and therefore the best overall setting for problems of other domains may vary.
tion 2 However,
Finally, we study how sensitive method is the hyper- parameer 3, i.c. the EMA decay rate. Results in Table g illustrate that setting an effective "window" of 100 training steps work best for problem setups. This is expected, setting larger 3 value corresponds to conduct Grad
21

---

Published as a conference paper at ICLR 2021
ar
bg
C11
11r
la
te
avg
mBERT
84.2
94.7
92.7
91.0
93.8
93.3
88.0
91.9
83.3
80.3
90.4
79.2
88.6
+ GradNorm
83.5
94.7
92.3
91.0
93.6
93.2
88.2
91.4
83.0
80.5
90.6
79.1
88.4
.GGA
84.4
94.5
92.3
90.4
93.5
92.7
88.
92.3
83.4
80.5
90.2
78.7
88.4
+ PCGrra
83.7
94.8
92.6
91.5
94.2
92.8
88.5
91.7
83.7
80.5
90.8
79.4
88.7
+ Vac
84.1
95.0
93.6
91.7
94.4
93.9
88.5
92.4
83.5
79.8
90.9
79.5
88.9
Table 9: on the POS tasks of the XTREME benchmark.
more aggressively, and vice versa. In general, we find our best settings be consistent across tasks in this paper.
G XTREME EXPERIMENTS
G.1 FINETUNING DETAILS
We also conduct experiments the XTREME benchmark (Hu et al. 2020) for cross-lingual trans- fer tasks. While other work mostly focus on zero-shot cross-lingual transfer (finetune on English training data and then evaluate on the target language data), we use different setup of multi- task learning such that we finetune multiple languages jointly and evaluate on all languages. Notice that our goal not compare with state-of-the -ortth--rrt results on this benchmark but rather examine the effectiveness of our proposed method on pre-trained multilingual language models. We therefore only consider tasks that contain training data for languages: named entity recognition (NER) and part-of-speech tagging (POS).
The NER task from the WikiAnn (Pan et al. 2017) dataset, which built automatically from Wikipedia. A linear layer with softmax classifier is added on top of pretrained models to predict the label for each word based on its first subword. We report the Fl score. Similar NER, POS also sequence labelling task but with focus synthetic knowledge. In partieular, the dataset used from the Universal Dependencies treebanks (Nivre et al.) 2018). Task-specific layers are the same in NER and report F1. We select 12 languages for each task randomly.
We use the multilingual BERT (Devlin et al. 2018) as our base model, which Transformer model pretrained the Wikipedias of T04 languages using masked language modelling (MLM). contains 12 layers and 178M parameters. Following Hu et al. (2020), we finetune the model for 10 epochs for NER and POS, and search the following hyperparameters: batch size {16, 32}; learning rate 5, 3e-5, 5e-5).
G.2 POS RESULT
We evaluate all multi-task baselines on the POS tasks in Table 9 We find that our proposed method outperforms other methods average, consistent with results in other settings (Section 4
22


Reviews:
[{"review_id": "asPunsSAari", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary \nTaking multilingual NMT (MNMT) into account, this work, investigates better model optimization alternative, that is in part can be attributed as a multi-task optimization problem. MNMT's are quite beneficial from different perspectives (improving low-resource languages, efficiency, etc). However, their inherently multi-task nature requires more focus on how to gist out the best possible learning for each of the languages pairs. With a potential impact on the optimization of other multi-task models, this work asks how model the similarity between model gradients is crucial in multi-task settings, and how to best optimize MNMT models focusing on the typologically similarity of languages. By analyzing the geometry of the NMT model objective function, authors indicate that computing similarity along gradient provides information on the relationship between languages and the overall model performance. Authors argue the analysis of the gradient helps to identify the point of limitation in multi-task learning, which the work aims to address, by focusing the parameter updates for tasks that are similar or close in terms of geometrical alignment (also known as Gradient Vaccine /GradVac/). \n\nExperimental results are provided from multilingual tasks involving 10^9 magnitude model training examples and several languages pairs. Mathematical proof and theoretical details of the proposed optimization approach GradVac are detailed in comparison with previous approach (such as Gradient Surgery). Experimental results shows the proposed GradVac to contribute for the improvement of model performance. These findings underline the importance of taking into account language proximity for a better optimization approach and model improvements in general. \n\n\n\nPros / Reason for the Score \nAfter my assessment of the proposed approach and the visible advantage of GradVac, I am voting for an accept score. Below my points of the pros and cons of this work. It's my hope authors will address the cons and the questions raised in the rebuttal period.  \n\n- This work raises an important question of optimization in a multi-task model, particularly for multilingual NMT models where an optimization approach is quite rare and recent progress in MNMT mainly focuses on improving performance. Hence the findings in this work, can provide further insight on how to best optimize an MNMT model and potentially set a new standard training mechanism for future works in MNMT. \n\n- From the experimental results, particularly its quite interesting to see how the proposed approach (GradVac) improves the high-resource languages (on the left side of Figure 6 (b)). I think in massive MNMT models while there is huge gain (naturally) for low-resource cases, the high-resource pairs tends to degrade. This work shows an interesting mechanism to address performance degradation for certain pairs in an MNMT model and to maintain an improvement trend  for all of the language pairs involved. \n\n\nCons and Questions \n- Considering the language similarity, this work focused on typologically similarity (that deals with the characteristics of the language structure), is there any consideration for genetic similarity, or any other similarity measure between languages the authors considered? Or why is the typological similarity the primary/only choice for this work? \n\n- As in Yu et al. 2020, where the PCGrad approach is used to project the gradient of task i to the plane of task j, was there any motivation behind not to adapt or asses this approach in MNMT before going / do the authors have any comment why this approach does lag behind from the GradVacc.? - Perhaps this is related to the assumption PCGrad is not fit a positive gradient similarities - a case in this work?\n\n- One of the advantages of MNMT model is efficiency (as also mentioned in this work), however, when we deal with model training or even inference the paper does not mention the complexity that can be introduced by the application of GradVac, can the author provide the details on this? \n\n- Page (P) 1: mentions one of the motivations for the work is to investigate ways to optimize the single language-agnostic objective for training an MNMT model that leverages training data of multiple pairs - if this work is aiming at optimizing based on task relatedness - did it consider for instance training MNMT models that are language family specific and see how that relatedness correlates with the approach in this work and the baseline MNMT models (such as Any->En or En->Any)?\n\n- What is the impact of training only two MNMT models Any->En and En->Any, why not Any<>Any? Wouldn't this make a lot more sense from the point of having multiple tasks (in terms of observing different language characteristics both at the encoder and decoder side of the model)?\nSimilarly, the Any<>Any that is employed (shown in Figure 2.) gradient similarities correlates positively with model quality. In other words authors clearly demonstrated the En>Any direction gradient similarity is quite low with respect to Any>En, in my understanding using Any<>Any model throughout the experiment makes more sense by constructing a real multi-tasking MNMT model, where we can also see the proposed approaches effectiveness. \n\n- Not sure if I am missing it, if it correct that we do not have comparison of the proposed optimization approaches with other optimizations from the results in Figure 6? At least with PCGrad ?\n\n\nComments\n- In an ideal case, I would go for evaluating a multilingual model that is not English centric to properly construct a real multilingual model. I understand the experimental design here, specifically this is the data (En<>Any) in general or available in-house for the authors. Yet, with recent progresses in multilingual NMT and zero-shot NMT approaches, its becomes realistic now to incrementally augment data for the non English pairs (can leverage monolingual data of the Any languages too), hence, resulting in more pairs. Such a setting of Any-Any can even further reflect how the optimization is beneficial. \n\n- please re-arrange the figures, I see discussion about Figure 5 while there is Figure 3 and 4 beforehand - if possible. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "_qCn5THrkcV", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper conducts comprehensive analyses and a method to the multi-task training in multilingual models. By analyzing the gradient similarity of two tasks in multilingual NMT, this paper reveals that gradient similarities reflect language proximities, correlate with model quality, and also evolve with layers and training steps. Furthermore, this paper proposes a method called GradVac to improve the multi-task training over standard monolithic training. Experiment results show GradVac achieves better accuracy than other multi-task optimization methods. \n\nThe paper is well-written and easy to follow. The motivation is clear, the analyses on the gradient similarity are interesting, and method proposed is effective according to experiments comparison.  While I like the investigation on multilingual model in Section 2, I have some questions on the proposed method in Section 3:\n1) How do you choose the task to alter the gradient? By random? For example, we always change g1 (by random) for any two gradients g1 and g2. If g1 in current batch is estimated more accurate than g2 or g1 can contribute more on the model optimization than g2, do we still alter g1 while leave g2 unchanged?  How about first choose a not good gradient than alter it instead of alter by random?\n2) Does it harm the optimization of a task if we change its gradient? How can we ensure that the benefits brought by solving the conflicts of two gradients is worthy compared to the drawbacks brought by altering the gradient of a task?\n3) How about consider more tasks at a time instead of only 2 tasks? Can the proposed method be extended smoothly? Since it is more practical as there are multiple tasks trained at the same time.\n4) In equation 3, with EMA, the gradient similarity objective may be more stable than the computed gradient similarity at each step. What is the intuition behand? I do not see GradVac is more preemptively in this sense.\n\nI expect the author can answer the points above, and then I can adjust my final score.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "QZCE3bakV-S", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies the behaviour of gradient similarities across languages in multilingual NMT models. They find gradient similarities mirror language similarity. Hence, they look at method to gradient-based methods for multilingual NMT. They apply PCGrad to the multilingual NMT task and also extend this method to address the cases when the task gradients have only weak similarity. \n\n**Strengths**\n- I find the analysis of gradients and language similarities interesting and adds to the understanding of multilingual NMT models. \n- The GradVac method addresses the case of positive cosine values and thus allowing for faster mitigation of interference while training. \n- The experimentation and analysis are strong. Particularly, the study of hyper-parameters to discover the best settings is very useful. \n\n**Weaknesses**\n- The proposed method is only a minor extension of the PCGrad method and the novelty is limited. \n- The gains against corresponding PCGrad variant is very limited - probably won't be statistically significant. So, it is not clear if there is a major benefit to the GradVac procedure over PCGrad. A statistical significance test of the BLEU scores using standard bootstrap sampling will be useful. \n\n**Questions for the authors**\n- Did you explore setting gradient objectives based on language similarity? \n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "6D-NG_9jZJT", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a novel method, GradientVaccine, to improve multi-task optimization on a massive multilingual translation and named entity recognition model. They investigate the loss function geometry on many language pairs and use the idea to encourage more geometrical parameter updates. This approach extends Gradient Surgery (a.k.a PCGrad) by adding an exponential moving average with a term $\\phi^{(t)}_{ijk}$ and $\\beta$ to address the limitation of PCGrad. I can say this approach is straightforward but very useful. The presented idea is very elegant and has been convinced by the theoretical foundation. \n\nStrengths:\n- The paper proposes a multi-task training method by calculating the gradient similarity and used them as a trajectory of the optimization.\n- The method is beneficial in massive multilingual NMT and XTREME benchmarks. \n\nWeaknesses:\n- The results of the proposed method seems consistent. However, the authors may only run the experiment once. It would be great if the authors can also provide a significant test of their results. And interestingly, in POS, the method is not as effective as NMT and XTREME.\n\nI have some questions:\n- How does $\\beta$ affect the gradient similarity objective? Can you also provide an ablation study on this?\n\nOverall, I enjoy reading this paper, and the experimental results are strong, and the paper is solid. The authors compare this method with other important baselines. In summary, this is a good paper. It presents a straightforward and useful idea for multi-task learning. \n\n**Post-rebuttal**\n\n> I want to thank the author for addressing my concerns. Overall, this is an exciting paper with comprehensive ablation studies and analysis. It provides an effective multi-task training method for multilingual tasks. The authors also support the method with a strong mathematical foundation. Thus, I would like to keep my positive rating. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a scalable optimization method for multi-task learning in multilingual models. 

Pros:
1) Addresses a problem which has not been explored much in the past
2) Presents very good analysis to show the limitations of existing methods. 
3) Good results. 
4) Well written

Cons:
1) Some missing details about various choices made in the experiments (mostly addressed in the rebuttal)

This is a very interesting and useful work and I recommend that it should be accepted.

Decision: Accept (Spotlight)