Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new method for resolving the gradient conflicts encountered in multi-task learning problems. Gradient conflict means when two gradients have a negative cosine similarity. The idea is based on [1] to project gradients of tasks to be orthogonal to all the other tasks if there is any gradient conflict between the two tasks. The difference is that in [1] gradient is projected iteratively based on pairs of tasks. In this work, the gradient is projected to the subspace that is orthogonal to the span of all other gradients. The paper also explores tuning the weight of gradient aggregation after resolving all gradient conflicts, so that it allows controlling the balance over different tasks. The experiments show that the proposed method is able to optimize for different Pareto optimal solutions and the results are competitive.

[1] Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824–5836, 2020.
strength_and_weaknesses: Strength: 
1. The method proposed is guaranteed to resolve all gradient conflicts. 
2. The method allows an easy way to control the weights of gradients from different tasks, which is nice to have to explore different Pareto optimal solutions to the problem. 

Weaknesses:
1. The idea of projection is not new but based on Yu et al. 2022. 
2. Not sure if strictly enforcing no gradient conflicts by projecting to the orthogonal subspace loses too much information about the gradient. In the worst-case scenario where there are a lot of gradient conflicts, this method will destroy a lot of the original gradient information by iterative projecting the gradients to a new orthogonal subspace. I would be more convinced if there is some evidence on how this procedure destroys or keeps most of the gradient information as compared to Yu et al. 
3. I would love to see how Yu et al. performs with a similar kind of gradient re-weighting, though it would be non-trivial since there could be gradient conflicts in Yu et al.’s method (i.e. not all cosine similarity between g_i and G are positive, maybe use an exponential of the cosine similarity). As it seems like when alpha = 0, (i.e. gradient averaging after resolving the gradient conflicts), the proposed method does not have a clear advantage over Yu et al. It is the flexible re-weighting that makes it interestingly competitive. I wonder if that is also true by applying a similar re-weighting idea to Yu et al.
clarity,_quality,_novelty_and_reproducibility: The paper is well-written and the idea is clearly illustrated with visualizations and descriptions of how the algorithm works. The experiments are quite extensive, the method is compared with several state-of-the art baselines. The idea of projecting gradient is not completely new, but more based on Yu et al. and extending it to projection to orthogonal subspace. 
summary_of_the_review: Overall, I think the paper presents an interesting perspective on how to do flexible re-weighting of gradients based on the cosine similarity of the independent task gradient and the aggregated gradient. The idea of gradient projection is not completely new but inherited from Yu et al. I have some reservations about whether it makes the most sense to project gradient to be orthogonal to all other gradients to resolve the conflict, as it can lose a lot of the gradient information.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: To address the task interference in multi-task learning, this paper presents an innovative method to obtain deconflicted gradients among tasks. This is done through projecting the gradient of a task to the subspace that is orthogonal to that spanned by the gradients of all other tasks. The authors provide the convergence analysis of the method and tested the method using two benchmark datasets and one large-scale recommendation dataset. 
strength_and_weaknesses: Strength
+ The proposed method produces deconflicted gradients that are not in confliction with any original gradients. 
+ The deconflicted gradients make it much easy to define an overall updating gradient that does not conflict with any individual task specific gradient while weighting tasks during learning. 
+ Code will be provided
+ Presentation of the manuscript is clear

Weakness
-  Regarding the discussion underneath Eq. (2), since g_i’ is located in the subspace that is orthogonal to that spanned by all g_j’s, should be g_i’• g_j = 0? 
- The underlying assumption of existence of all g_i’ is that all g_i’s should be linearly independent among them. A discussion of such assumption in reality would strength the approach.
-  Resulting from using different \alpha values on test set are provided. However, no discussion is made how \alpha would be tuned in practice. Use results from the \alpha value leading to the best test performance to compare with other methods is not a fair comparison. 
- Recent related work should be discussed and compared, for example:
Javaloy, A., & Valera, I. (2022). RotoGrad: Gradient Homogenization in Multitask Learning. ICLR, 1–24. http://arxiv.org/abs/2103.02631
- My biggest concern of this paper is the weak empirical results, with minimal difference in the results among compared the methods (especially, table 1 and 3), which does not support the advantage of the proposed method claimed by the authors.

clarity,_quality,_novelty_and_reproducibility: Good
summary_of_the_review: The idea is interesting and the method is technically sound. However, several weaknesses as listed, especially, the weak empirical results dampened my enthusiasm of this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This manuscript proposes a novel projection-based method for tackling the issues of gradient conflict in the multi-task learning problem, GradOPS. The work improves previous projection-based methods like PCGrad by projecting conflict gradients onto an orthogonal subspace. In this way, it can not only resolve all conflicts in more than two tasks but also effectively searches for diverse solutions with different trade-off preferences among different tasks.
strength_and_weaknesses: Strengths:

1. The underlying idea of this work is well-motivated. It hypothesizes a reasonable theory about the “conflicting gradients” issue in multi-task learning and follow-up with a simple algorithm.
2. The concept definition of “strong non-conflicting” and “weak non-conflicting” provide a clear description of the core issue, which in turn strongly serves as the basis for method design.
3. The proposed method is simple and general, it can be easily applied to various gradient-based approaches. Furthermore, the paper writing is professional and rigorous.

Weaknesses:

1. Although this work provides a novel projection-based approach, the core idea does not improve too prominently from previous work (Yu et al., 2020; Wang et al., 2020), in fact, PCGrad can also be generalized to the case of T > 2 as well (and the experimental results did not significantly exceed PCGrad, shown in Table 1, 2). Moreover, compared to PCGrad, the theoretical analysis of this paper is relatively weak.
2. The experimental results are somewhat disappointing, which makes the proposed method less convincing. For instance, the results in Table 1 and Table 2 are not exactly better than the baseline method (MGDA). In addition, the performance of GradOPS shown in Table 3 is highly dependent on a good choice of α values, which may mean that the method is not so general for different types of datasets.
3. There is no comparison with the state-of-the-art algorithm, in fact, the NashMTL (Navon et al., 2022) has surpassed the previous method, suggesting that the authors supplement the corresponding comparison experiments.
clarity,_quality,_novelty_and_reproducibility: This paper is well-organized and somewhat novel; however, there are some weaknesses required to address.
summary_of_the_review: This paper is well-organized and somewhat novel; however, there are some weaknesses required to address.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Training multitask learning models can be difficult at times due to the existence of conflicting gradients. That is, when computing the gradient of the model's parameters, different tasks present different gradients which can differ in magnitude and direction, counteracting each other when added up, and leading to parameter updates that obtain unsatisfactory results. 

In this paper, the authors argue the necessity and importance of the absence of gradient conflict during training, and provide a method to explicity ensure this setting. The method, GradOPS, generates a new set of gradients by first transforming the original gradients into an orthogonal set of vectors, and then subtracting their contribution (i.e. projection) from the original gradients, making sure that the resulting gradients do not conflict the original ones. Finally, these gradients are added up, weighted by their contribution to the final update direction (and a hyperparameter).

The authors then conduct three experiments on a multitask binary-classification, NYUv2, and a recommendation system, proving their method effective compared with other MTL approaches.

strength_and_weaknesses: **Strengths**
- S1: I like the distinction between strong and weak non-conflicting gradients, although I find the explanations a bit too hand wavy for my taste.
- S2: The proposed method does not depend on the task ordering, which I find quite important.
- S3: Using scalar projections to measure the "dominance" of one task in the training process is quite interesting.
- S4: I really like the use of Mean Rank to compare methods in the experiments.

**Weaknesses**

Proposed method
- W1: I have serious concerns understanding the proposed method and the intuition behind it:
  - As of Eq. 1, $u_i$ is not defined for $i \neq 1$.
  - For $i \neq 1$, $u_2 = 0$ always.
  - It is not clear to me why this method should work. For example, if there are more tasks than parameters, then the basis generated by GS would span the entire parameter space. Then Eq. (2) would produce $g_i = 0$ for all tasks, unless for some subtle reason in the algorithm, $u_i$ is always different from 0. 
   - I fail to understand the reason behind the double normalization in Eq. 4 and 5, rather than weighting $R_i$ directly in Eq. 4.
- W2: One of the advantages of GradOPS is that "it solves all conflicts among tasks," meaning that the resulting vectors $g'_i$ hold $\langle g'_i, g_j \rangle \geq 0$ for all $i$ and $j$. I fail to see this as a milestone by itself, since this could very well be achieved by multiplying all gradients by 0. Why is this important? And why are the new update directions meaningful?
- W3: While the introduction of the parameter $\alpha$ is introduced with the advantage of exploring more solutions, there is no control or intuition for the practitioner on which values of $\alpha$ set to get to a desired trade-off.

Experiments
- W4: While the experiments are repeated ten times, there are no standard deviations and no best averaged results were marked in bold, rather than running any statistical test to check for significant changes.
- W5: Results on the NYUv2 dataset are significantly worse than those reported in the literature, for example, in RotoGrad [1] and RLW [2].

Literature review: 
- L1: Literature review is missing some conflicting gradient methods that do not fit in the MOO vs. projection discussion: e.g. GradDrop [3] randomly drops elements of the task gradients, and RotoGrad [1] applies rotation matrices in the heads to align task gradients.
- L2: Some comments feel a bit off. For example, saying that IMTL-G is a state-of-the-art method is rather questionable.

**Questions:**
- Q1. How were the MR and $\Delta m$ computed? Individually for each run, or over the averaged results?
- Q2: Which values of $\alpha$ were selected for CAGrad and GradNorm?

[1] - Spotlight, ICLR 2022 - [RotoGrad: Gradient Homogenization in Multitask Learning](http://arxiv.org/abs/2103.02631)

[2] - ArXiv - [A Closer Look at Loss Weighting in Multi-Task Learning](http://arxiv.org/abs/2111.10603) 

[3] -  NeurIPS 2021 - [Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout](https://proceedings.neurips.cc//paper_files/paper/2020/hash/16002f7a455a94aa4e91cc34ebdb9f2d-Abstract.html)
clarity,_quality,_novelty_and_reproducibility: **Clarity** I find the paper well written in general. However, writing could be improved a bit, e.g., by replacing expressions such as "What's more."

Moreover, I find quite concerning the usage of the word "gradient" along the entire text, as it is not clear to me that this modified gradients are still gradients. I also find some statements too strong for my taste (without further justification or citation), such as "Existing MOO methods only seek weak non-conflicting G towards certain trade-offs, _which is directly responsible for task performance._"

**Quality** Disregarding all the concerns I've presented in the section above, I find the quality of the paper ok.

**Novelty** The proposed method is rather novel.

**Reproducibility** The experiments are not reproducible since there is no code provided and the proposed method is not well-defined.
summary_of_the_review: Overall, I find the idea of the paper (robustifying, in some sense, projection-based methods to improve conflicting gradients) appealing, and it could be a nice direction to pursue. The paper also have some cool reflections regarding conflicting gradients.

However, the proposed method has some questionable design choices and technical flaws that need further investigation/clarification, including the motivation of why the proposed procedure leads to sensible directions to follow during optimization.

This, added to some concerns on the experimental part, makes me believe that the manuscript needs a bit more of work, and hence leans me towards rejection.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a simple and effective gradient deconfliction algorithm, called GradOPS, for multi-task learning (MTL). Concretely, GradOPS projects the gradient associated with one task onto the subspace orthogonal to the span of the other task-specific gradients, achieving non-conflicting gradients for different tasks. Theoretical analysis of the convergence of GradOPS is provided, followed by extensive experiments on several multi-task learning datasets. Overall, this paper demonstrates the effectiveness of the proposed novel algorithms, both from theoretically and practically.
strength_and_weaknesses: Strengths:
1. The proposed algorithm is simple and effective. I am very happy to see that such simple algorithm can achieve non-conflicting gradients and improved performance for multi-task learning problems.
2. Figure 1 is a good example to illustrate the differences between the proposed GradOPS and existing MTL algorithms. 
3. Theoretical analysis of the convergence of GradOPS is provided, ensuring that the output of GradOPS can converge to a Pareto stationary point.
4. Authors also give a full discussion of the advantages of the proposed GradOPS in Section 3.3.
5. I find almost no typos in the paper.

Weaknesses and Suggestions:
1. More clarity on Figure 1. I appreciate the illustration of Figure 1. However, I believe that it can be improved in terms of the clarity and beauty. For instance, the caption of Figure 1 in Page 2 should include the calculation of the aggregated gradient G = \sum_{i=1}^{3}g_{i}, instead of letting readers seek the detailed aggregation rule of G in Subsection 3.1 in Page 4. Besides, different arrows with different colors in Figure1 represents different gradients, and it will be better to clarify them in the caption. Finally, the notation of GradOPS-modified g_{1}^{‘} looks not good enough. It should be g_{1}^{‘} instead of g^{‘}_{1} in Figure 1(b)(c)(d), to keep notation consistence with g_{1} in Figure 1(a).
2. The theoretical results should be placed in the main paper, and in particular, the assumptions of the convergence of the proposed algorithm in Theorem 1 & 2 should be expressed explicitly in the main paper (e.g. in the bottom of page 4). For example, the convergence of GradOPS in Theorem 1 requires the Lipschitz properties of the gradient and the small step size of GD. Such assumptions are quite essential to derive the final theoretical results, and should be clarified in the main paper (e.g. at least with the claim “the convergence of GradOPS is provided in Theorem 1, under mild assumptions of neural network and the step size of GD”). A counterexample is that, if we use ReLU activation function in deep neural network, the smooth assumption of neural network could not be satisfied and the convergence of GradOPS in Theorem 1 could not be guaranteed.
3. As far as I can tell, the proof of Theorem 1 and Theorem 2 is easy (and hence is correct), so the derived results should not be stated as Theorem. It will be better to call them as Proposition.
4. More explanations on the architecture of the deep neural networks (DNN) in experiment Part. As far as I can see, this paper uses different deep neural networks as backbones for different datasets, and authors also point out the name of these networks in the main paper. However, the components of DNN is very important in our experiments, and so I will suggest the authors to give more explanations on the components of DNN in the Appendix C (e.g. the kernel size of convolutional network, and the activation function in DNN).
5. Other minor suggestions: (1) Eqs.4 and 5 can be merged and put in one line for concise. (2) Avoid orphan line before Section 3.1. in Page 4 and the orphan line at the top of Page 9. (3) Typos: in caption of Figure 3, “the final updated direction”.

clarity,_quality,_novelty_and_reproducibility: Clarity: not bad. 
Quality: the writing can be improved. 
Novelty and Reproducibility: good.

summary_of_the_review: Thank authors for their detailed responses. In terms of others’ review and authors’ feedback, I remain my score as “weak accept.”
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: In this paper, the authors proposed a new multi-task learning method to avoid conflicts of gradients among tasks and search for diverse solutions towards different trade-off preferences among them. Though the proposed idea is potentially interesting, there are some major concerns about some technical details of the proposed method. These concerns remain after rebuttal. Moreover, the experimental results are not convincing to demonstrate the superior performance of the proposed method over existing methods. 

In summary, this work is potentially promising work, but is not ready to be published based on its current shape. 
justification_for_why_not_higher_score: There are some concerns about the technical design of the proposed method. Experiments are not convincing.
Decision: Reject