Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary 
Taking multilingual NMT (MNMT) into account, this work, investigates better model optimization alternative, that is in part can be attributed as a multi-task optimization problem. MNMT's are quite beneficial from different perspectives (improving low-resource languages, efficiency, etc). However, their inherently multi-task nature requires more focus on how to gist out the best possible learning for each of the languages pairs. With a potential impact on the optimization of other multi-task models, this work asks how model the similarity between model gradients is crucial in multi-task settings, and how to best optimize MNMT models focusing on the typologically similarity of languages. By analyzing the geometry of the NMT model objective function, authors indicate that computing similarity along gradient provides information on the relationship between languages and the overall model performance. Authors argue the analysis of the gradient helps to identify the point of limitation in multi-task learning, which the work aims to address, by focusing the parameter updates for tasks that are similar or close in terms of geometrical alignment (also known as Gradient Vaccine /GradVac/). 

Experimental results are provided from multilingual tasks involving 10^9 magnitude model training examples and several languages pairs. Mathematical proof and theoretical details of the proposed optimization approach GradVac are detailed in comparison with previous approach (such as Gradient Surgery). Experimental results shows the proposed GradVac to contribute for the improvement of model performance. These findings underline the importance of taking into account language proximity for a better optimization approach and model improvements in general. 



Pros / Reason for the Score 
After my assessment of the proposed approach and the visible advantage of GradVac, I am voting for an accept score. Below my points of the pros and cons of this work. It's my hope authors will address the cons and the questions raised in the rebuttal period.  

- This work raises an important question of optimization in a multi-task model, particularly for multilingual NMT models where an optimization approach is quite rare and recent progress in MNMT mainly focuses on improving performance. Hence the findings in this work, can provide further insight on how to best optimize an MNMT model and potentially set a new standard training mechanism for future works in MNMT. 

- From the experimental results, particularly its quite interesting to see how the proposed approach (GradVac) improves the high-resource languages (on the left side of Figure 6 (b)). I think in massive MNMT models while there is huge gain (naturally) for low-resource cases, the high-resource pairs tends to degrade. This work shows an interesting mechanism to address performance degradation for certain pairs in an MNMT model and to maintain an improvement trend  for all of the language pairs involved. 


Cons and Questions 
- Considering the language similarity, this work focused on typologically similarity (that deals with the characteristics of the language structure), is there any consideration for genetic similarity, or any other similarity measure between languages the authors considered? Or why is the typological similarity the primary/only choice for this work? 

- As in Yu et al. 2020, where the PCGrad approach is used to project the gradient of task i to the plane of task j, was there any motivation behind not to adapt or asses this approach in MNMT before going / do the authors have any comment why this approach does lag behind from the GradVacc.? - Perhaps this is related to the assumption PCGrad is not fit a positive gradient similarities - a case in this work?

- One of the advantages of MNMT model is efficiency (as also mentioned in this work), however, when we deal with model training or even inference the paper does not mention the complexity that can be introduced by the application of GradVac, can the author provide the details on this? 

- Page (P) 1: mentions one of the motivations for the work is to investigate ways to optimize the single language-agnostic objective for training an MNMT model that leverages training data of multiple pairs - if this work is aiming at optimizing based on task relatedness - did it consider for instance training MNMT models that are language family specific and see how that relatedness correlates with the approach in this work and the baseline MNMT models (such as Any->En or En->Any)?

- What is the impact of training only two MNMT models Any->En and En->Any, why not Any<>Any? Wouldn't this make a lot more sense from the point of having multiple tasks (in terms of observing different language characteristics both at the encoder and decoder side of the model)?
Similarly, the Any<>Any that is employed (shown in Figure 2.) gradient similarities correlates positively with model quality. In other words authors clearly demonstrated the En>Any direction gradient similarity is quite low with respect to Any>En, in my understanding using Any<>Any model throughout the experiment makes more sense by constructing a real multi-tasking MNMT model, where we can also see the proposed approaches effectiveness. 

- Not sure if I am missing it, if it correct that we do not have comparison of the proposed optimization approaches with other optimizations from the results in Figure 6? At least with PCGrad ?


Comments
- In an ideal case, I would go for evaluating a multilingual model that is not English centric to properly construct a real multilingual model. I understand the experimental design here, specifically this is the data (En<>Any) in general or available in-house for the authors. Yet, with recent progresses in multilingual NMT and zero-shot NMT approaches, its becomes realistic now to incrementally augment data for the non English pairs (can leverage monolingual data of the Any languages too), hence, resulting in more pairs. Such a setting of Any-Any can even further reflect how the optimization is beneficial. 

- please re-arrange the figures, I see discussion about Figure 5 while there is Figure 3 and 4 beforehand - if possible. 
review_title: This work takes aim at an interesting problem of optimizing multilingual neural machine translation (MNMT) model. Although MNMT is inherently a multi-task modeling approach, less emphasis have been given on achieving an optimal performance on all of the tasks involved. The proposed approach (GradVac) takes into account similarity between tasks and demonstrates better performance can be achieved by focusing on parameter updates that are geometrically aligned.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper conducts comprehensive analyses and a method to the multi-task training in multilingual models. By analyzing the gradient similarity of two tasks in multilingual NMT, this paper reveals that gradient similarities reflect language proximities, correlate with model quality, and also evolve with layers and training steps. Furthermore, this paper proposes a method called GradVac to improve the multi-task training over standard monolithic training. Experiment results show GradVac achieves better accuracy than other multi-task optimization methods. 

The paper is well-written and easy to follow. The motivation is clear, the analyses on the gradient similarity are interesting, and method proposed is effective according to experiments comparison.  While I like the investigation on multilingual model in Section 2, I have some questions on the proposed method in Section 3:
1) How do you choose the task to alter the gradient? By random? For example, we always change g1 (by random) for any two gradients g1 and g2. If g1 in current batch is estimated more accurate than g2 or g1 can contribute more on the model optimization than g2, do we still alter g1 while leave g2 unchanged?  How about first choose a not good gradient than alter it instead of alter by random?
2) Does it harm the optimization of a task if we change its gradient? How can we ensure that the benefits brought by solving the conflicts of two gradients is worthy compared to the drawbacks brought by altering the gradient of a task?
3) How about consider more tasks at a time instead of only 2 tasks? Can the proposed method be extended smoothly? Since it is more practical as there are multiple tasks trained at the same time.
4) In equation 3, with EMA, the gradient similarity objective may be more stable than the computed gradient similarity at each step. What is the intuition behand? I do not see GradVac is more preemptively in this sense.

I expect the author can answer the points above, and then I can adjust my final score.

review_title: Review comments for paper 2546

final_score: rating: 7: Good paper, accept
review_content: The paper proposes a novel method, GradientVaccine, to improve multi-task optimization on a massive multilingual translation and named entity recognition model. They investigate the loss function geometry on many language pairs and use the idea to encourage more geometrical parameter updates. This approach extends Gradient Surgery (a.k.a PCGrad) by adding an exponential moving average with a term $\phi^{(t)}_{ijk}$ and $\beta$ to address the limitation of PCGrad. I can say this approach is straightforward but very useful. The presented idea is very elegant and has been convinced by the theoretical foundation. 

Strengths:
- The paper proposes a multi-task training method by calculating the gradient similarity and used them as a trajectory of the optimization.
- The method is beneficial in massive multilingual NMT and XTREME benchmarks. 

Weaknesses:
- The results of the proposed method seems consistent. However, the authors may only run the experiment once. It would be great if the authors can also provide a significant test of their results. And interestingly, in POS, the method is not as effective as NMT and XTREME.

I have some questions:
- How does $\beta$ affect the gradient similarity objective? Can you also provide an ablation study on this?

Overall, I enjoy reading this paper, and the experimental results are strong, and the paper is solid. The authors compare this method with other important baselines. In summary, this is a good paper. It presents a straightforward and useful idea for multi-task learning. 

**Post-rebuttal**

> I want to thank the author for addressing my concerns. Overall, this is an exciting paper with comprehensive ablation studies and analysis. It provides an effective multi-task training method for multilingual tasks. The authors also support the method with a strong mathematical foundation. Thus, I would like to keep my positive rating. 
review_title: A good paper

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper studies the behaviour of gradient similarities across languages in multilingual NMT models. They find gradient similarities mirror language similarity. Hence, they look at method to gradient-based methods for multilingual NMT. They apply PCGrad to the multilingual NMT task and also extend this method to address the cases when the task gradients have only weak similarity. 

**Strengths**
- I find the analysis of gradients and language similarities interesting and adds to the understanding of multilingual NMT models. 
- The GradVac method addresses the case of positive cosine values and thus allowing for faster mitigation of interference while training. 
- The experimentation and analysis are strong. Particularly, the study of hyper-parameters to discover the best settings is very useful. 

**Weaknesses**
- The proposed method is only a minor extension of the PCGrad method and the novelty is limited. 
- The gains against corresponding PCGrad variant is very limited - probably won't be statistically significant. So, it is not clear if there is a major benefit to the GradVac procedure over PCGrad. A statistical significance test of the BLEU scores using standard bootstrap sampling will be useful. 

**Questions for the authors**
- Did you explore setting gradient objectives based on language similarity? 
 
review_title: An extension of existing approach, benefits not clear

Meta-review: This paper proposes a scalable optimization method for multi-task learning in multilingual models. 

Pros:
1) Addresses a problem which has not been explored much in the past
2) Presents very good analysis to show the limitations of existing methods. 
3) Good results. 
4) Well written

Cons:
1) Some missing details about various choices made in the experiments (mostly addressed in the rebuttal)

This is a very interesting and useful work and I recommend that it should be accepted.

Decision: Accept (Spotlight)