Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes a method for multi-objective optimization. The key idea is to learn the entire Pareto front at once by training a hypernetwork that takes preference vector as an inputs and outputs network parameters, which corresponds to a point on the Pareto set with the desired trade-off specified by the preference vector. Specifically, the hypernetwork is a multi-head network where each head outputs a weight tensor of a module in the target network. The method improves HV from the baselines, in several multi-task learning problems, including image classification, regression and, mixed classification and regression.

+) The main contribution of this work is to learn a continuous function that maps a preference vector to network parameters that corresponds to the desired trade-off. The trained hypernetwork generalizes to preference vectors unseen during training so that the required training time for getting models of an arbitrary trade-off is reduced.

+) Compared to prior work, CPMTL, which starts from a Pareto optimal point and extends Pareto front locally around the point, the proposed method attempts to train a single hypernetwork that represents the entire Pareto front.

+) Pareto front provides insights on the trade-off relation between tasks but it usually requires repetitive training of models under different preference settings. I think this method can be useful when understanding the relations among tasks by significantly reducing the training time to obtain the Pareto front.

-) One concern is the scalability to the number of tasks. The amount of reduce in the training time of the proposed method over the base methods depends on how much the method generalize to the preference vectors unseen during training. For example, ideally, the model trained using the preference vectors [0,1] and [1,0] generalizes to [0.5,0.5]. In other words, it is desired that a hypernetwork is trained using less number of samples while generalizes well to arbitrary trade-offs, relying on the smoothness of the hypernetwork. Otherwise, the training requires more number of iterations to sample sufficient number of preference vectors that covers the whole preference vector space to match the original performance, where its size grows exponentially to the number of tasks. Current manuscript lacks experimental or theoretical analysis on this generalization performance.

-) Relating to the above point, I think evaluation measure HV and uniformity are not sufficient to evaluate the overall behavior. HV measures the quality of exploration, but higher HV does not necessarily mean that every model of method A dominates method B. For example, in the left figure of Figure 2, some points of EPO dominates PHN-EPO though HV is higher for PHN-EPO (Table 1). This implies that the proposed method reduces training time at cost of possible degrading movement of Pareto front. In other words, depending on the hyperparameter alpha used for the Dirichlet distribution, under the comparable training time to the baseline, the baseline method may dominate its PHN counter part at certain preferences.

-) And therefore, I think one needs to investigate the accuracy plot (as Figure 2) together with HV and uniformity to better understand the behavior of the method. Some are missing in the current manuscript (e.g. NYUv2)

-) Another concern is the scalability to the target model capacity. In the current manuscript, all the experiments are performed using architectures with small number of parameters (up to \~0.37M of ENNet), which is much smaller than popular architectures such as ResNet50 (\~26M). In addition, the required size of the hypernetwork grows at least proportional to the target model capacity,
and the amount of training data and the training time likely increase accordingly.

-) HyperNetwork require some overhead of memory and computation cost during the test time.

-) Comparison with important prior work CPMTL is necessary.

I would recommend 'accept'. Though I have concerns about the scalability and lack of analysis, I think this work has some advantages stated above.

Questions:
1. Discussion on the generalization performance to unseen preference vectors
2. Comparison with CPMTL
3. Accuracy plot for NYUv2

___
Thanks for the response. After reading the authors' response and other reviews, I would like to keep my recommendation.

review_title: Learning the Pareto Front

final_score: rating: 6: Marginally above acceptance threshold
review_content: Thanks for the efforts of the authors. After reading the response and other reviews,  I raise my score from 5 to 6. However, the proposed algorithm lacks analysis. So I cannot improve my score further. 
___________________________________________________

Summary: 
The paper proposes using a hypernetwork to learn the entire Pareto front of a multi-objective optimization.  It develops two approaches using linear scalarization and exact Pareto optimal (EPO), respectively. Experiments on several multi-task learning tasks show its advantage in terms of hypervolume and uniformity.

Strengths: 
+ It is very interesting and useful that the paper tries to learn the entire Pareto front directly.
+ The paper proposes using hypernetworks to learn the Pareto front for multi-task learning. 

Weaknesses: 
- Algorithm 1 is the main contribution. The pseudocode and its descriptions are not clear. For example, it is better to use two algorithms to descript PHN-LS and PHN-EPO, respectively. r is the weights for LS, while it is the ray for EPO. However, r should be pre-given. How and why PHN could optimize the multi-objective optimization for any r? What does “Dir(\alpha)” mean?
-PHN is a solving method for MOO problems. So the paper should verify that it could learn the entire Pareto front. The paper may test with MOO problems with known Pareto fronts. The results in Figure 2 cannot testify to this claim. This is more important than the experiments on the multi-task learning.

Minor comments:
1.	The legend in Figure2 should be colored.
2.	The captions of the subfigures in Figure 4 should be under them. 
3.	Typos:
sampled “form” the m-dimensional-> sampled “from” the m-dimensional, “minimas”->”minima”, longer “then”-> longer “than”, “follwoing" ->“following"

review_title: The idea of learning the entire Pareto front is interesting. However, the organization and experiments need more elaboration.

final_score: rating: 7: Good paper, accept
review_content: The paper proposed a novel algorithm for MOO, which combines preference-based methods with hypernetworks, in order to encapsulate the preferences in the network input.
The paper is well written, the proposed method is clear, and the experiments are sufficient. 
My concerns are the following.

First, in the abstract you say that
"Recent optimization algorithms can target a specific desired ray in loss space,
but still face two grave limitations: (i) A separate model has to be learned for
each point on the front; and (ii) The exact trade-off must be known prior to the
optimization process. "

This is not entirely true. In Reinforcement Learning (RL) manifold-based approach such as

Parisi et al, "Multi-objective Reinforcement Learning through Continuous Pareto Manifold Approximation"
Yang et al, "A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation"

do not need neither a separate model, nor to know the trade-off a priori. The work of Parisi et al learns a parametrization producing infinitely many solutions at the same time using a specific loss, while Yang et al train a neural network which takes preferences over the objectives as input, and generalizes over them. 
Despite being tested only on MORL, these algorithms can easily be extended to MOO.
Therefore, I suggest you to rephrase your abstract and introduction to mention these methods.

In particular, the work of Yang et al is very reminiscent of your algorithm, since both include the preference vector in the network input. 

Nonetheless, I feel the paper has enough novely thanks to the use of hypernetworks, EPO, and its application to MOO.
The evaluation is sufficient, but I would suggest to move the evaluation of evolutionary algorithms to the main section. Evolutionary algorithms are extremely popular in MOO and thanks to a wide variety of fitness functions can learn any frontier, even though they may require longer run time (as your experiments clearly show).

Finally, a bit more analysis on concave frontier would be beneficial. In Figure 1 you show that LS fails in finding concave frontiers, but it seems to me that all experiments have convex frontier. To my understanding, EPO should address this limitation ("The recent EPO algorithm can guarantee convergence to a local Pareto optimal point on a specified ray r, addressing the theoretical limitations of LS"). Am I correct? A bit more discussion and experiments with concave frontiers would be a nice addition to the paper.

Overall, I am leaning to accept it but the authors should address the above issues.

**EDIT **
The authors have addressed my concerns, and I have increased my score.
review_title: Good paper with minor issues

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper tracks the problem of learning the entire Pareto front to allow the user to select a desired Pareto optimal solution by one inference procedure without retraining the model. The high-level idea is to learn the entire Pareto front simultaneously using a single hyper network, which receives as input the desired preference vector and returns a Pareto-optimal solution whose loss vector is in the desired direction. The paper gives an early trial to build a toolbox to allow users to get a desired solution by a single inference procedure.

Strength:
1.	This paper is an early trial to use a hyper network to directly approximate the Pareto Optimal front, allowing practitioners to flexibly choose Pareto solutions conditioned on different preference vectors.
2.	The paper is well-written and a substantial number of experiments are conducted, promising good results of the proposed method. 

Feedbacks:
1.	My major concern is that there is a similar work [1] that shares the same spirit with the work.  The authors might want to clarify the differences or conduct some experimental comparisons.
2.	It is better to attach more details about the EPO algorithm.
3.	There are some typos or grammar issues, such as 1) Page 6, the title of the subsection ‘Hyperparamter tuning’ should be ‘Hyperparameter tuning’. 2) Page 6, ‘We therefor’ should be ‘We therefore’. 3) Page 6, ‘as follow’ should be ‘as follows’. 4) Page 7, ‘are train and evaluate’ should be ‘are trained and evaluated’.

[1] X. Lin, Z. Yang, Q. Zhang, and S. Kwong, “Controllable Pareto multi-task learning,” arXiv preprint arXiv: 2010.06313, 2020.

review_title: Learning the Pareto Front with Hypernetworks

Meta-review: The paper proposes a hyper-net method for multi-objective optimization, which trains a neural network that maps preference vector to the corresponding Pareto solution. The proposed idea is interesting and useful, although the evaluation of the work is not overwhelming convincing. The writing of the work can be further improved. 

Also, the basic idea of the work is the almost the same as a concurrent work "Lin et al 2020. controllable pareto multi-task learning" which is also submitted to this conference. The paper cited that paper briefly, "... The proposed method is conceptually similar to our approach...",  which is too vague and brief. We urge the author to provide a through discussion on the detailed difference and similarity of the works, including empirical comparisons when necessary. 
Decision: Accept (Poster)