Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an algorithm to optimize in a Pareto set in order to reach a solution or solutions that can minimize an extra criterion. The proposed algorithm is highly relevant in the context of multitask learning. The proposed algorithm takes both 1) optimizing the extra criterion w/o considering the original objectives and 2) multiple gradient descent (MGD) as its special cases. It has the flexibility to switch between two cases depending on the magnitude of the gradients w.r.t. the original tasks. The authors provide strong theoretical analysis for the proposed algorithm along with some empirical results to show its effectiveness.
main_review: Strengths
1) Intro and background sections refer to many prior works.
2) The proposed algorithm is quite intuitive, and the theoretical analysis is rigorous.

Weaknesses
1) Many important details are left out, esp. in the experiment section. Granted, some of these important details can be found in the appendices, but I personally find them more important than some existing contents of the paper. For example, the practical implementation of the proposed algorithm and how its hyperparameters are chosen. Among the 3 experiment subsections, the 1st one shows a toy example. The 2nd and 3rd have slightly more practical or realistic settings. The extra criterion $F(\theta)$ is critical for the readers to properly interpret the experiment results. The discussion about why and how to select such kind of $F(\theta)$ is lacking. For the 3rd experiment, it was mentioned in the appendix, but even there, it's not very clear. Moreover, no comparison of computational cost between different methods can be found.

A detail: At the end of line 93, the inequality should be $\ell(\theta') \succ \ell(\theta)$.
summary_of_the_review: The algorithm proposed in this paper is intuitive and highly relevant in the context of multitask learning, but I find the experiment section didn't show the effectiveness of the proposed method clearly. Many missing important details result in many question marks while I read the paper. I think a clear and convincing experiment section is, to some extent, more important than the theoretical analysis given the gap between those regularity conditions required and the practical use cases. The experiment section of this paper definitely has room for improvement.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper deals with multi-objective optimization. In particular, given loss functions $\ell_1,\dots,\ell_m$ and $F$, the goal is to minimize $F$ in the Pareto set of $\ell_1,\dots,\ell_m$, i.e. $\min_{\theta} F(\theta)$ under the constraint that for all $\theta'$ there exists $j$ with $\ell_j(\theta') \geq \ell_j(\theta)$. This is useful for multi-task learning, where we are interested in training a model with robust performance under a variety of tasks.

The main contribution is an algorithm that attempts to approximate this optimization problem. The main idea is to take steps in some direction $v$ that correlates as much as possible with $-\nabla F(\theta)$, while making sure that $\langle v, -\nabla \ell_j(\theta)\rangle \geq \alpha$ for all $j$, and some scalar $\alpha$.

The theoretical results show that the above continuous process will converge to a solution $\theta$ that, if $F$ and $\ell_j$'s are locally convex at $\theta$, cannot be locally improved in the sense that any direction that decreases $F$ gives a solution that is Pareto-dominated by $\theta$.

The experimental results show how the algorithm performs when $F$ is designed to 1) balance the ratios between the losses $\ell_j$ based on some pre-specified numbers and 2) maximize diversity of a bunch of solutions $\theta_1,\dots,\theta_k$ and 3) domain generalization.
main_review: The objective studied is quite general and it is conceptually nice that you can plug in any loss $F$ to optimize over the pareto set (it seems that this framework is quite versatile). The paper is carefully written and nice to read. The theoretical results aren't too surprising, and somewhat limited because they work in the continuous setting and don't give specific step sizes. In the experimental results we see that the performance is marginally better than previous work (but it's more general). It would be good to mention how long the algorithm takes and compare it to the other methods.

The authors mention that the problem with the scalarization approaches (where the losses are combined into a single objective by taking their convex combination) is that they only find solutions on the convex envelope of the Pareto frontier, and thus can miss some solutions. But is there any evidence that the authors' approach does any more than that (i.e. find solutions that are not on the convex envelope)? In the figures that I saw in the paper the Pareto frontier is convex.

Regarding the diversity experiments, the authors should explain a bit better how they computed the metrics. It is mentioned in the appendix that instead of sampling from the Pareto set they sample from the solutions of the various algorithms that are being compared but they could expand on that.

The authors mention that one drawback of JiGen is that it requires a careful grid search for the weighting of the two objectives. I am not sure I agree with this, since in the JiGen paper this parameter doesn't seem to be extremely critical for performance. What weighting did the authors use in their JiGen experiments and how would it compare to e.g. always setting the weights to (1,0.5)?

I have a slight doubt about the approach of trying to make the distributions of the feature embeddings of image classification and jigsaw the same. By adding this distribution discrepancy module in the objective it seems to me that this approach is inadvertently trying to achieve patch permutation-invariance, which is not necessarily something desirable in general (and could also make the classification problem harder). 

Can the authors' algorithm be framed of as some kind of approximate projected gradient descent? I.e. if $g \leq \epsilon$, then take a gradient step in $F$, otherwise make Pareto-improving steps to decrease $g$. I wonder how this would compare to the authors approach, since this is a more principled way to go about it.

There is a typo in Theorem 2: $g(\theta_t)$ should be $g(\theta_s)$.
summary_of_the_review: The paper is well written and presents a nice general framework, although it is not clear to me that there is a significant performance improvement compared to previous work. Therefore I lean towards acceptance.

---------------------------

After reading Reviewer's 6MZF comments, I agree that there are major similarities to Algorithm 2 in Kamani et al that need to be acknowledged. I disagree with the authors' claim that Kamani et al only makes sense for a specific choice of $F$, since their algorithm can be stated for general $F$. Therefore, I decrease my score by 1 point.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper summarizes a new gradient descent procedure for finding Pareto optimal points, while simultaneously optimizing with a "non-informative" function F that depends on the whole set of points.
main_review: The paper provides a good overview of Pareto optimality and of previous works, although a thorough discussion of scalarizations (including the recent hypervolume scalarizations used for non-convex Pareto frontiers) would make sense, since there is extensive use of the linear scalarization throughout. The main interesting setting of the paper seems to be the addition of a "non-informative" function that would like to be maximized along the Pareto front. Examples of these functions are given; however, it seems like such a problem has already been considered.

Furthermore, it is unclear if the proposed gradient descent approach to solve the main problem is novel or if it has good convergence guarantees in finite time (not in the limit). Under certain assumptions, the algorithm have descent guarantees, but it is unclear if these assumptions are discussed or are realistic. Ultimately, the algorithm also fails to find points on the Pareto frontier, just points that are Pareto "optimal" according to a given definition (when the Pareto frontier is non-convex, there are no clear guarantees).



 
summary_of_the_review: The paper presents an interesting setting for multi-objective optimization, but suffers from a lack of discussion of scalarizations, assumptions, why non-convex Pareto frontiers can be discarded, and the novelty/convergence of the optimization.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper intends to introduce an algorithm for finding points on the Pareto frontier with some conditions defined by the user. In this proposal, the condition defined by the user is added to the main objectives and solves a new multiobjective optimization problem. They provide empirical comparisons to show the effectiveness of their approach.
main_review: Finding points on the Pareto front is the main motivation of many proposals in recent years. The two main motivations of the current proposal are based on resolving issues of the existing methods for large-scale models and non-convex and non-linear models in deep learning. Clearly, these claims that existing methods are not suitable for these situations are not accurate. First, in most recent works based on MGDA methods such as [A] and [B], there is either convergence analysis for both convex and non-convex objectives or no convexity assumption. Hence, this statement seems to be inaccurate, and there is no experimental comparison with other existing methods other than EPO. 

Moreover, the claim that they are proposing an algorithm for large-scale models seems to be inaccurate again. Since all the analysis is based on having access to the deterministic oracle of gradients and not stochastic ones, it is not clear how they can be scalable to large-scale models. The non-stochastic analysis of convergence is provided by other algorithms as well, hence, the superiority of the current approach over them is not clear.

Above all that, my main concern is the similarity of the current proposal with Algorithm 2 in [B] without a clear reference in the current work, which is clearly not acceptable. Especially since they have cited this paper in the related work and are aware of this work. In [B], similar to the current proposal the goal of finding preference-based solutions on the Pareto set has been added as a KL-divergence to the main objective. Again, similar to the current proposal, before reaching the Pareto set, they are enforcing the main and side objectives at the same time, and when they reach the Pareto set, they only pursue the side objective since the main objective common descent direction becomes zero. Compare condition (8) in the paper with Case I and II in Algorithm 2 in [B]. It is even more clear how close these proposals are when you compare their empirical results. Comparing Figure 1a and 1b in this paper and Figure 5b and 5c in [B] show the similarities. The difference of the current work with the one in [B] is generalizing the side objectives to other objectives such as diversity in addition to the preference-based. Despite this, the lack of acknowledging the existing similar work and comparing with them is clearly unacceptable.

[A] Mahapatra, Debabrata, and Vaibhav Rajan. "Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization." International Conference on Machine Learning. PMLR, 2020.

[B] Kamani, Mohammad Mahdi, et al. "Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing." arXiv preprint arXiv:2104.01634 (2021).


summary_of_the_review: There are some concerns regarding the motivations of this work. Also, similarities with the current work and existing methods not referenced appropriately, which seems to reduce the novelty of this work.
review_title: null

Meta-review: The authors propose the OPT-in-Pareto algorithm that considers multi-objective optimization, and includes an extra "non-informative" reference metric for choosing between different Pareto-optimal solutions.

The reviewers generally agreed that the work was compelling. However, one reviewer (6MZF) brought up the fact that the proposal is extremely similar to one proposed by a different arXiv paper, and convincingly argued that the authors of this paper were aware of the other before submission.

This is a difficult situation. On the one hand, for the purposes of establishing priority, an arXiv paper "doesn't count". On the other hand, I believe that authors are obligated to appropriately credit all relevant work of which they are aware, in *any* form: this includes journals, conference proceedings, preprints, emails, personal conversations, stackoverflow posts, tweets, etc. In this case, it seems that the authors did not adhere to this second condition, and while they have updated their manuscript, two reviewers said that they were unsatisfied by the changes on this point.

I want to emphasize that this isn't a question of priority: the first to publish "wins", and nobody has published this work, yet. However, other researchers working on the same problem, and proposing similar solutions, *must* be appropriately credited, even by the eventual winners (if they are aware of them).
Decision: Reject