Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper assumes there exist Pareto Subspaces, i.e., weight subspaces where multiple optimal functional solutions lie, and develops a weight-ensembling method named Pareto Manifold Learning that casts multi-task problems as learning an ensemble of single-task predictors by interpolating among members during training.  Multiple single-task predictors are trained in conjunction to produce a subspace formed by their convex hull, and endowed with desirable Pareto properties. Each single-task model infuses and benefits from representational knowledge and the other members. The losses are weighted in tandem with the interpolation when training.  The ensemble as a whole engenders a weight subspace that explicitly encodes tradeoffs and results in a continuous parameterization of the Pareto Front.
strength_and_weaknesses: Strength:
This paper castes multi-task problems as learning an ensemble of single-task predictors by interpolating among members during training, and designs weight subspaces where multiple optimal functional solutions lie. Based on this weight space, an ensembling method that can produce a continuous Pareto Front in a single training run is proposed. This allows practitioners to modulate the performance on each task during inference on the fly.  

Weaknesses:

1.Similar methods have already been proposed for multi-task learning and has not been disccussed in this paper [1].

1.When sampling on the convex hull parameterization, authors choose to adopt the Dirichlet distribution since its support is the T-dimensional simplex. Does this distribution have other properties.   Why using this distribution?  If p≫1，how the ensemble will change. 

2.When training, a mono tonic relationship is imposed between the degree of a single-task predictor participation and the weight of the corresponding task loss. As a result, the ensemble engenders a subspace that explicitly encodes tradeoffs and results in a continuous parameterization of the Pareto Front.  Whether the mono tonic relationship can be replaced by other relationships? Explaining this point may be better.

[1]Navon A, Shamsian A, Fetaya E, et al. Learning the Pareto Front with Hypernetworks[C]//International Conference on Learning Representations. 2020.
clarity,_quality,_novelty_and_reproducibility: Quality:
This paper is well written and easy to read. The organization is clear.

Clarity:
Some issue of motivations and method details need more clarification. Please see weakness section.

Originality:
fair. 

summary_of_the_review: This paper produces a subspace with multiple Pareto stationary points in the multi-task learning based on the geometry of the loss landscape in single task machine learning where the local optimal are connected by simple paths. However, the representation and more illustrations are needed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a novel approach to seek a continuous Pareto Front for multi-task learning. The key idea is to train multiple single-task predictors and performs convex hull operations on different models in the weight space to produce a subspace with multiple Pareto stationary points. A continuous Pareto Front can be produced through only a single run in this method.
strength_and_weaknesses: Strengths:

1. This paper improves the method in Ma et al. (2020) and proposes a novel method to produce continuous Pareto stationary points, which is simple and easy to utilize. In addition, the method is efficient as it requires only a single run.
2. This paper is inspired by the theory of single-task machine learning and the motivation is quite interesting. Meanwhile, the concept of Pareto manifold learning is insightful and worth exploring.
3. The visual explanation provided in Figure 3 is easy to understand, while the visual analysis of the article (Figure 4 and Figure 5) is comprehensive.

Weaknesses:

1. It must be acknowledged that the conjecture about the loss landscape from the single-task to multi-task is insightful. However, it is better to give a toy example to explain it. More importantly, even though multiple valley intersections in the low loss region in the multi-task scenario, this observation is not strongly related to Pareto manifold learning. I suggest the author should explain it more clearly.
2. In the last paragraph of the introduction, the author announces that “the algorithm produces a subspace of Pareto Optimal solutions”, It should be pointed out that multi-task learning methods can only ensure convergence to Pareto optimal when assuming it holds a convex loss, in other words, we can only approximate rather than obtain the Pareto optimal solutions.
3. The proposed method fails to exceed the baseline methods such as Nash-MTL and PCGrad on several datasets(Table 1, Table 5, and Table 6), which makes the method less convincing. In addition, the ablation experiment demonstrated in Tabel 2 takes a value of $\lambda$ equal to 0 on the MultiMNIST dataset, which may indicate that the regularization is not effective. Most importantly, there is a lack of detailed analysis of the experimental results.
4. Paper writing should be polished, For instance, Figure 1 and Figure 2 are not mentioned in the context, which is not friendly to read.
5. In addition, some typos exist in the paper. In line 11 of Algorithm 1, the citation of Figure 4.2 is wrong, the same problem is found in the corresponding text above Claim 3, I guess it should be Figure 3.

clarity,_quality,_novelty_and_reproducibility: This paper is well organized and easy to follow.
summary_of_the_review: This paper focus on an interesting problem and provides some new ideas. However, there are some technical flaws in this work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposes a Pareto manifold learning approach to produce a continuous Pareto front for a given multi-task learning (MTL) problem in a single run. The MTL problem is formulated as a multi-objective optimization problem, and the proposed method can be treated as an ensemble approach for multiple single-task models with linear weight interpolation. In the proposed method, each task (objective) has its own neural network model (with identical structure), and their linear weight combination will be explicitly associated with the corresponding trade-off losses (with linear scalarization). Experimental results show that this method can successfully find the Pareto subspaces for different MTL problems and outperforms other MTL methods.
strength_and_weaknesses: **Strengths:**

+ This paper is well-written and easy to follow.

+ Multi-task learning and multi-objective optimization are both important research topics, and this work tries to further bridge these two fields.  

+ The proposed method has reasonable performance on different MTL problems (but with many concerns listed below).

**Weaknesses**

I have some major concerns on both the two claimed advantages of the proposed method, namely, a) learning the Pareto front for multi-task learning and b) better generalization performance.

**1. Contribution and Closely Related Works**

This work proposes to learn the Pareto front for a multi-task learning problem. However, similar Pareto front learning approaches have already been proposed for multi-task learning [1,2,3,4]. Those methods all allow practitioners to choose their preferred solutions at inference time. All these closely related works are not discussed or compared in this paper. The proposed linear interpolation approach could be similar to using a simple linear hypernetwork in [1,2]. 

Without a thorough discussion and comparison with these closely related works, it is very hard to judge the novelty and actual contribution of the proposed PML method.

**2. Pareto Solution for Better Generalization Performance**

The motivation to learn the Pareto solution for better generalization performance is not strong. Recent works [5, 6] on multi-task learning have shown that a single fixed solution found by simple linear scalarization can have similar or even better performance than those found by multi-objective optimization methods. The effect of careful hyperparameters finetune rather than the Pareto properties could be crucial for good generalization performance.   

In addition, the experimental results on the found Pareto front and better per-task performance are somehow not solid. The major issue is that, for an MTL problem with m tasks, the proposed PML model is m times larger than the other baselines, which could lead to unfair comparison. For example, in Figure 4, some baseline models can have comparable or even better performance than the PML's Pareto front. If we can double their model capacity to match the one for PML, it could be likely that some of them (a single solution) can totally dominate the whole Pareto front by PML. In this case, the "Pareto front" could be useless since all tradeoffs are suboptimal and dominated.

Similarly, in Table 1, the overall per-task performance of PML is already nondominated with other MTL baselines. If the model capacity of all the baselines can be doubled, their performance could be significantly improved. On the other hand, the single-task learning baseline (STL) indeed has the same model capacity with PML. In this problem, it seems that simply having separate models for each task already significantly outperforms PML.   

**3. Ensemble Learning Method**

Since the proposed PML method requires learning multiple models and can be treated as an ensemble method, it should be compared with the closely related ensemble learning methods. A naive baseline could be to train m models with the same fixed weight among tasks (e.g., simple LS) or other multi-objective optimization (e.g., MGDA, CAGrad), and then a simple ensemble learning method [7] can be applied to achieve better overall MTL performance. An ensemble of models with different optimization methods, such as LS with different weights or MGDA + CAGrad, could be another choice.

The recent work on ensemble learning could have a smaller model size or faster inference time [8,9], and the weight average methods [10, 11] are also strong alternatives with a single final model. Some related works on mode connectivity have been briefly discussed but not compared in the paper. What is the advantage of the proposed PML model for generalization over those ensemble learning or weight average methods?    

**4. Other Flat Minima Method**

One main motivation of this work is the connection between low-loss subspace (e.g., flat minima valley) and better generalization performance. From this viewpoint, it is interesting to know its relation to the entropy-sgd [12] and sharpness-aware minimization methods [13, 14, 15]. Can we simply use these methods to train an MTL model with fixed weights for each task? What is the advantage of PML over those methods? 

**Other Questions**

1. In Algorithm 1, should the input size of \theta be m rather than m?

[1] Learning the Pareto Front with Hypernetworks. ICLR 2021.

[2] Controllable Pareto Multi-Task Learning. arXiv:2010.06313.

[3] Scalable Pareto Front Approximation for Deep Multi-Objective Learning. ICDM 2021.

[4] Controllable Dynamic Multi-Task Architectures. CVPR 2022.

[5] In Defense of the Unitary Scalarization for Deep Multi-Task Learning. arXiv:2201.04122, 2022.

[6] Do Current Multi-Task Optimization Methods in Deep Learning Even Help? arXiv:2209.11379, 2022.

[7] Simple and scalable predictive uncertainty estimation using deep ensembles. NeurIPS 2017.

[8] BatchEnsemble: an alternative approach to efficient ensemble and lifelong learning. ICLR 2020.

[9] Training independent subnetworks for robust prediction. ICLR 2021.

[10] Averaging Weights Leads to Wider Optima and Better Generalization. UAI 2018.

[11] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. ICML 2022.

[12] Entropy-SGD: Biasing gradient descent into wide valleys. ICLR 2017.

[13] Sharp Minima Can Generalize For Deep Nets. ICML 2017.

[14] Fantastic generalization measures and where to find them. ICLR 2020.

[15] Sharpness-Aware Minimization for Efficiently Improving Generalization. ICLR 2021.

clarity,_quality,_novelty_and_reproducibility: **Clarity:** This paper is well-written and easy to follow.

**Quality:** The paper has a good overall quality, but there are also some major concerns on its main contributions as listed in the weaknesses.

**Novelty:** Some very closely related works are not discussed or compared in this paper, which makes it very hard to judge the novelty and actual contribution of this work.  

**Reproducibility:** The proposed model structure seems simple and straightforward. But the experimental results could be not robust, given 1) the current findings on MTL with multi-objective optimization, and 2) the unsolid experimental setting as discussed in the weaknesses.
summary_of_the_review: This paper is well-written, and it studies an important research topic that could further bridge the field of multi-task learning and multi-objective optimization. I personally enjoy reading this work. However, due to the major concerns on main contributions, relation to closely related work, and the experimental setting, it is hard for me to vote to accept the current manuscript.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper is within the family of handling multi-task learning as multi-objective optimization family. The main contribution is posing the problem as learning a space of solutions such that this linear space will correspond to a Pareto efficiency set. Since the space is assumed to be linear, its extreme points are learned directly within optimization. The method is later evaluated extensively on various benchmarks. The submission was reviewed by 3 experts and they all shared the same sentiment: The paper is very interesting however both discussion and experimental study misses a very significant set of works, hypernetworks. The closest family of related work to the submission is the hypernetworks and the method neither discusses nor compares with them. This issue has been partially addressed in rebuttal with additional discussion and a small experiment on MultiMNIST; however, paper still needs a major revision of including hypernetworks as baseline in all experiments and rewriting related work and introduction sections by carefully positioning the paper with respect to hypernetworks. It is clear to me that authors need longer time to complete this revision and should submit to the next conference deadline.
justification_for_why_not_higher_score: The paper is quite interesting in terms of idea and method. However, authors simply missed the closest set of related work (hypernetworks) from the literature in their discussion and experiments. The paper needs a major revision of extensive comparison and additional discussion to be presented.
Decision: Reject