Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: In the paper, Rotograd is proposed as a new gradient-based approach for training multi-task deep neural networks based on GradNorm. GradNorm is first formulated as a Stackelberg game, where the leader aims at normalizing the gradient of different tasks and the follower aims at optimizing the collective weighted loss objective. Under this formulation, one can utilize theoretical guarantees of the Stackelberg game by making the leader have a learning rate that decays to zero faster than the follower. To further account for the different gradient directions, a learnable rotation and translation are applied to the representation of each task, such that the transformed representation match that of the single-task learning. By adding an additional term accounting for learning this rotation, the leader in the Stackelberg game will minimize the loss to homogenize both the gradient magnitude and match the representation to single-task learning as close as possible. 

In general, I find the direction of gradient homogenization for multi-task learning very important and interesting. The paper provides an interesting perspective through the Stackelberg game formulation, which provides a framework for selecting the learning rate of GradNorm type of gradient homogenization methods. The other contribution of the paper is a learnable task-specific rotation that aligns the task gradients with single-task learning. The proposing of a learnable rotation matrix seems an interesting idea, although I am not sure if it has been proposed previously for multi-task learning. 

I find the first contribution of formulating the problem as a Stackelberg game to be interesting and novel. However, in terms of the second contribution, I have some concerns about whether it makes the most sense by aligning the transformed representation with that of single-task learning. For MTL, one of the key benefits is learning a better representation by sharing it across different tasks to encourage helpful transfer between the tasks; by constraining the transformed representation to be as close to the single-task learning representation, it might limit the transfer between tasks since the representation are constrained to be equivalent to that learned by single-task learning. I think it is helpful to think about using rotation invariant representations for aligning the gradient directions, but it is questionable to align it to that of the single-task learning. 

Another major concern is about the experimental results, full experiments are only conducted on one real-world dataset. The experiment on the second dataset seems to be very preliminary, which might not be sufficient to justify the proposed method empirically. Also on the second dataset, it seems the two different implementations of Rotograd have a large discrepancy in the results, which might need more investigation about why this happens. Meanwhile, many ablation studies seem to be missing. I am mostly interested to see experiments that validate the Stackelberg game formulation, for example by using different learning rates for the leader and the follower. Also, it would be interesting to see how the proposed Rotograd compares with pure GradNorm on gradient direction alignment. Overall, I feel the experiments are not complete for validating the effectiveness of the method. 

Some minor points: the description of d-grad method seems to be missing. Also, Yu et. al [2020] also deals with gradient aligning for MTL which could be considered as a baseline to compare with. 

Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., & Finn, C. (2020). Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.06782. 

--------After author's response----------

I am not fully convinced by the explanation of the motivation behind rotation matrix, in particular why it is aligning with the single-task learning, which is counter-intuitive. The authors provided more ablation studies, however, the evaluation on datasets is still quite preliminary with some questions remaining (such as why there is a discrepancy between the two versions of Rotograd on the second dataset). Therefore I am keeping my original score.

review_title: Interesting idea but weak experiment implementation and lack of motivation for the proposed method

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper presents an extension of Gradnorm to address task conflicting due to discordant gradient direction. Specially it introduces a rotation matrix to rotate the hidden representation from the last shared layer. The authors put the proposed method in the context of game theory to show stability and convergence of the training, which might be of merit. 

The writing of the paper doesn’t meet the publication standard, needing major work to improve. There are many typos and awkward sentences, hindering understanding of their work. Also, there are many places that need clarification, for example, in Proposition 4.1, the inverse of the gradient of Z with respective to \theta needs to be calculated. So, what is the shape of this gradient matrix? How it is necessarily to be a square matrix? What ||\Delta_{\theta} Z|| represents? the F-norm? There is lack of adequate explanation of the motivation behind the objective in Eq. (6). By reading the paper, I have no idea about the two oracle functions, and why they are defined in the way shown in Eq. (8). 

Eq. (3) is inaccurate, not aligning with that proposed in the GradNorm paper for the computation of L_{grad}^k.

Eq. (9) is problematic. Why R_k z_i^t does not appear in the objective function of the first optimization problem? If this is because z_i^{k,t} = R_k z_i^t + d_k, then the objective in the second optimization problem would be just 0. 

Why operating on z instead of the gradient in Gradnorm can resolve the discordant gradient issue among tasks is not properly justified. 

The reported empirical results are weak and do not support this method works as claimed.  

review_title: The writing needs improvement. The proposed idea is not well justified. The empirical results are weak.

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:

This paper proposes an MTL method that encourages the gradients on shared parameters to have similar directions across different tasks. The motivation is to reduce conflicts between gradients of different tasks, so that training can proceed more smoothly, and fit multiple tasks more easily. The paper introduces a new way of thinking about this kind of method, i.e., through the lens of Stackelberg games, which could be useful in reasoning about the convergence of such methods. The method is shown to perform favorably against related methods, especially in regression settings.



Strong points:

Minimizing gradient conflict is a well-motivated way to reduce negative transfer.

The algorithm description is detailed, and should be straightforward for others to implement.

Stackelberg games are an interesting framework for thinking about methods like GradNorm and Rotograd that adaptively guide MTL training.


Weak points:

The theory is interesting at a high-level, but it is not clear that it provides insights on what makes Rotograd work. In the paper, one main takeaway from the Stackelberg games framework is that the methods converge if the leader’s learning rate is asymptotically smaller than the follower’s. This takeaway is implemented by decaying the leader’s learning rate, but it is not shown that this is a key point required for Rotograd to work. I would not be surprised if the results were unaffected if this decay were removed. If this point is really important, it should be illustrated in ablation studies. More broadly, since the point does not only apply to Rotograd, this ablation could also be done on Gradnorm and other methods. Such ablations would be one way to connect the theory to the methods.

Another main takeaway from the theory is that the rotation matrices and translation vectors should be updated with gradient descent, instead of simply replacing them each step. Intuitively, the algorithm would still make sense and be simpler if R and d were simply replaced. Experiments showing that the gradient-descent update rule is necessary would help show the value of the theory.

Similarly, the value of Proposition 4.1 is not clear. Is it to prove stability? Does this have some particular connection to Rotograd, or is it a useful fact about hard parameter-sharing methods in general?

There is one ablation “rotograd-sgd”, but it is not clear how exactly it works: Can it simply update R and d however it wants, or is Eq. 9 still used to regularize the updates in some way?

By adding the rotation matrices, it’s possible that information that would be useful to share across tasks is instead stored in these task-specific matrices. That is, conflict between tasks can beneficially lead to more general representations. Restricting R to be a rotation instead of any matrix is one step towards limiting the amount of information leakage into task-specific parameters. Is there a conceptual reason to expect that the benefits from reducing conflicts will outweigh this leakage?

The experiments are on an intentionally very small architecture, where one of the main issues is expressivity, which gives Rotograd an edge over methods that do not include an additional task-specific matrix. 

In Section 5.1, does the method without Rotograd do poorly because there are no task-specific networks in that case?

Although Rotograd is motivated to reduce negative transfer, Table 1 shows that Rotograd does not reduce negative transfer, but rather improves positive transfer. That is, uniform does better than rotograd in the tasks where single-task is better than multi-task, but rotograd does better than uniform in the tasks where uniform is already better than single-task. This makes me think that the benefits of Rotograd are not coming from reducing negative transfer, but from somewhere else.

Is there an explanation for why Rotograd does not work as well for multi-class classification tasks (i.e., performs worse than all other methods for Left and Right)? Is it because the task-specific heads have larger output sizes? E.g., could it be better to have a separate rotation matrix for each class? Figure 4 in A.3 confirms that there is an issue here: the cosine similarity is not higher for rotograd for the classification tasks.

Overall, from the limited scope of the experiments it is not clear that Rotograd would provide practical advantages over competing methods. The ChestXray experiments show that although Rotograd does not hurt much, it does not help overall compared too uniform.

That said, it would be still be interesting to see whether insights from Stackelberg games could lead to practical improvements for this problem.


Minor comments:

The writing has some issues. These issues don’t make the work unclear, but they are a bit distracting. Some example suggestions for fixing distracting word choice: “palliate” -> “alleviate”, “spoiled” -> “noted”, “we have not being able to propose Rotograd, but also to derive” -> “we have proposed Rotograd, and derived”. There is also frequent non-standard mixing of em dashes with spaces and commas.

“$[r_k(t)]^\alpha$ is a hyperparameter” -> “$\alpha$ is a hyperparameter” The hyperparameter is \alpha, correct?

----

Update: 

I am very happy to see the new experiments that validate the implications of the Stackelberg games theory. The main drawback of the paper is that it is not clear that direction homogenization could lead to practical improvements for multi-task learning. The additional experiments in Table 2 are useful, and suggest that much of the benefit comes from the greater expressivity due to task-specific matrices.

review_title: Review of Rotograd

Meta-review: The paper is proposing a novel representation of the GradNorm. GradNorm is presented as a Stackelberg game and its theory is used to understand and improve the convergence of the GradNorm. Moreover, in addition to the magnitude normalization, a direction normalization objective is added to the leader and a rotation matrix and a translation is used for this alignment. The paper is reviewed by three knowledgable reviewers and they unanimously agree on the rejection. Here are the major issues raised by the reviewers and the are chair:
- The motivation behind the rotation matrix layers is not clear. It should be motivated in more detail and explained better with additional illustrations and analyses.
- Empirical study is weak. More state of the art approaches from MTL should be included and more realistic datasets should be included.
- The proposed method is not properly explained with respect to existing methods. There are MTL methods beyond GradNorm like PCGrad and MGDA (MTL as MOO). These methods also fix directions. Hence, it is not clear what is the relationship of the proposed method with these ones.

I strongly recommend authors to improve their paper by fixing these major issues and submit to the next venue.
Decision: Reject