Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces a training algorithm for neural networks that reduces conflict between multiple output tasks that utilize a shared portion of the network.  Considering a common shared representation from which multiple task-specific subnetworks branch, the approach aims to homogenize task-specific gradient magnitudes and directions at this branch point.  The proposed strategy is to introduce additional parameterized rotation matrices, each of which modifies the shared representation before it is passed to a corresponding task-specific branch.  The parameters of these rotation matrices are optimized to maximize gradient similarity between different tasks at the branch point; this optimization step is interlaced with standard updates of other network parameters to minimize total task loss.
main_review: The proposed optimization scheme appears to be a unique technical approach among a larger set of recent work that attempt to deconflict gradients in multi-task learning.  Prior work has, e.g., focused on gradient projection operations [Yu et al., 2020], which, while similar in motivation, may lack the flexibility of the proposed approach's learnable rotations.

Experiments demonstrate proof-of-concept on synthetic optimization problems, as well as good results on real datasets (CIFAR10, CelebA, NYUv2).  Results on NYUv2 (semantic segmentation, depth estimation, normal estimation) look especially promising in comparison to both a baseline system and recently-published multi-task optimization procedures.

Overall, the proposed RotoGrad approach seems quite promising and could become a standard tool for multi-task neural network training.  It would be interesting to see results on larger-scale problems (both larger networks and larger datasets), as well as examine how RotoGrad might be applicable in transfer-learning and meta-learning scenarios.

summary_of_the_review: This paper presents a parameterization scheme and optimization procedure for improved training of shared multi-task networks.  The technical approach is concise, well-motivated, and novel with respect to existing work.  Experiments across synthetic and real datasets show convincing results.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel algorithm to tackle negative transfer in MTL optimization, by homogenizing the task gradients’ magnitude and direction. The method first homogenize gradients’ norm towards the norm of the task that converged the least. Next, it employs rotation matrices to align task gradients with the direction of the average gradient.

main_review: **Strengths:**
- The paper is well written and well structured.
- The paper tackles an important problem and proposes an intuitive approach for alleviating the problem. The motivation for the method is clear.
- The strong experimental section presents a clear improvement over previous MTL approaches.

**Weaknesses:**
- It would be useful to fully observe and understand the tradeoff between $m$ and runtime, as the scalability of the approach for large values of $d$ is limited.
- It appears the proposed update step may increase the loss (objective) for some tasks. While intuitive, it’s not clear to me if the average gradient is the right choice for the target vector ($v_n$). Have you experimented with other choices for $v_n$, for example, the common descent direction in MGDA?
- Missing citation to relevant works: [1]  propose a method to handle conflicting gradients in MTL. [2] theoretically discuss challenges of MTL optimization. Specifically, while different significantly than the proposed method, their covariance alignment approach appears to relate to the rotation matrices in the proposed method (although being applied to the input instead of the shared feature space).

Experiments:
- Please report standard errors or some other measure of variability for the results in Table 1. How many random seeds were considered?
- According to C1, you modified the implementation of the SOTA approach PCGrad to compute gradient w.r.t $z$, instead of all shared parameters as in the original paper. It is unclear how this modification affects PCGrad’s performance and if this choice maintains a fair comparison.
- Missing details of hyperparameters tuned for each method.


[1] Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models, ICLR 2021.

[2] Understanding and Improving Information Transfer in Multi-Task Learning, ICLR 2020.



summary_of_the_review: Overall it is a good paper with clear motivation and an intuitive approach. It tackles a major challenge in MTL optimization. Strong empirical results highlight the benefit of the approach compared to previous related methods.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: - This paper proposes a new way of dealing with negative transfer or catastrophic interference in multitask learning. The main idea is to avoid conflicting gradients by making them homogeneous both in terms of magnitude and direction. Gradient Magnitude are normalized by taking into account the convergence rate of all tasks. Directions are homogenised by smoothly rotating the feature-space introducing rotation matrices $R_k \in SO(d)$ parametrized via exponential maps on the Lie algebra of $SO(d)$. 
- Rotation matrices are optimized to reduce the direction conflicts by maximizing the cosine similarity within the the batch, while the resto of the network is trained to minimize the multitask loss. The training algorithm result in an alternated optimization procedure that can be interpreted as a Stackelberg game with a leader and a follower alternatively minimizing their own losses.

main_review: - **Questions:**
    1. Why do you need to specifically enforce $R_k$ to be a rotation matrix? Why it cannot be free to be a (non)-linear transformation? Have you tried experiments in this direction? Would that make sense?
    2. Regarding the comparison with other methods: From the supplementary, it looks that the PCGrad projection has been computed differently (wrt features rather than weights), can you elaborate more on this difference? 
    3. By looking at the PCgrad paper (which seems to be your main "competitor") I found that they apply pcgrad in combination with other multitask methods, e.g. MTAN. Can ROTOGRAD work together with other methods? Did you try these experiments?
    4. It would be great to go beyond supervised learning experiments and include multitask RL experiments in your analysis.
- **Minors**
    - I suggest guiding the reader a bit more on table1, by highlighting the best and worst results as you do in the supplementary.

summary_of_the_review: The paper is solid and it looks has received several iterations. It is a nice contribution to the multi-task learning field, although I think there are still some questions to be addressed first. I'm leaning towards acceptance and I'll be more than happy to increase my rate after the rebuttal and discussion with other reviewers.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work offers an approach to improve the training dynamics of multi-task neural networks by making two assumptions:

1. Magnitude differences among task gradients inhibit convergence.
2. Directional differences (often termed "conflicting gradients") in the gradients among tasks inhibit convergence. 

It adapts the optimization process by:

1. Ensuring all gradients have equal norm. 
2. Introducing a geometric rotation in SO(d) transformation after the final shared feature map and before the task-specific layers.

The intuition underpinning this method is supported with several illustrative (toy) examples as well as evaluation on MNIST, CelebA, CIFAR-10, and NYUv2.
main_review: Early recent work (2015-2018) in multi-task learning focused on dynamic task reweighing methods (i.e. how to change the loss scale throughout training), but it seems that the community has shifted to focusing on direct gradient modifications (2020-2021) with a plethora of methods being developed that modify both the scale and direction of the gradient update on the shared parameters (PCGrad, GradDrop, GradVac, IMTL, CAGrad, etc.). With at least 5 somewhat conceptually similar methods having been released in the past year, it's more important than before for new approaches in this space to offer a new perspective or approach this challenge from a different direction.

It is my opinion this work meets the requisite threshold. The concept of applying (and learning) a geometric rotation to align the optimization trajectories is novel; although, the concept of gradient-magnitude homogenization has been well-studied, with the presentation offered by this work feeling very reminiscent of GradNorm. Returning to the former point, I have not seen a rotation transformation in SO(d) applied to MTL -- the closest analogue are modulation modules which filter/modify information in either the forward and/or backward pass -- and I think the dissemination of this perspective may lead to other interesting and powerful work. 

Nevertheless, the current conceptualization of rotation to maximize cosine similarity may have some problems. Two identical tasks with identical initialization will have perfect alignment, and results in an effective doubling of the learning rate for the shared parameters, but does not lead to the benefits that may be conveyed from multi-task learning models (i.e. inductive bias on different hypothesis classes, regularization, etc.). Moreover, the (hyper)parameter free approach which enforces a strict constraint on equal norm among all gradients seems highly prohibitive (and perhaps undesirable since certain tasks should naturally train at different rates?). That said, many of the methods I mention in this first paragraph adopt a similar perspective with regards to a correlation between positive cosine similarity and modeling performance, and one can invert the question to posit that two tasks with perfectly opposite gradients will not even begin to train.

Strengths:

1. The approach is well motivated with illustrated examples and intuition.
2. Extremely well written and polished paper. 
3. Empirical methodology is sound. It appears as if all methods are fairly compared and assessed.
4. Experimental results are compelling.

Weaknesses:

1. I don't see any significant flaws in this work.

Nitpicks:

1. *to the shard feature z* => to the shared feature representation z.
2. The results in the table are a bit difficult to read as nothing stands out. Perhaps rerun the NYUv2 experiments 3-5 times and bold significant results? 
3. *While recent work has acknowledged that negative transfer is a two-fold problem, existing approaches fall short as they only focus on either homogenizing the gradient magnitude across tasks; or greedily change the gradient directions, overlooking future conflicts.* => This sentence is grammatically off. Consider breaking it into two smaller sentences. 
4.  *Previous works have tracked down this issue to the disparities in gradient magnitudes and directions across tasks, when optimizing the shared network parameters.* => remove the comma or bring the dependent clause (when optimizing the shared network parameters) to the beginning of the sentence (i.e. When optimizing the shared network parameters, prior wok has indicated discrepancies in gradient magnitude among tasks may inhibit model performance or something like this).
5. Several *very* recent papers have been overlooked that should probably be included at different points in the related work section.

[1] Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models, ICLR 2021 (spotlight)

[2] Conflict-Averse Gradient Descent for Multi-task learning, NeurIPS 2021

[3] Efficiently Identifying Task Groupings for Multi-Task Learning, NeurIPS 2021 (spotlight)

[4] Variational Multi-Task Learning with Gumbel-Softmax Priors, NeurIPS 2021
summary_of_the_review: I think this is a good paper. It is clearly written, the empirical evaluation is fair, and presents a novel perspective (rotation) to reduce inter-task conflict in multi-task learning paradigms. 

I recommend it for acceptance and have no additional or follow-up questions.
review_title: null

Meta-review: The paper addresses the problem of inconsistent gradients in multi-task learning, proposing ways to handle both their magnitude nd direction. Gradient directions are aligned by introducing a rotation layer between the shared backbone and task-specific branches. 
Reviewers appreciated the technical approach, higlighting the novelty of the rotation layers in this context. The empirical evaluations are systematic fair and insightful, and the presentation is polished. Reviewers unanimously supported accepting the paper.
Decision: Accept (Spotlight)