Controllable Pareto Multi-Task Learning
Authors: Xi Lin
Zhiyuan Yang
Qingfu Zhang
Sam Kwong
Venue:  (2021)
Keywords: Multi-Task Learning
Multi-Objective Optimization

Abstract: A multi-task learning (MTL) system aims at solving multiple related tasks at the same time.  With a fixed model capacity, the tasks would be conflicted with each other, and the system usually has to make a trade-off among learning all of them together. Multiple models with different preferences over tasks have to be trained and stored for many real-world applications where the trade-off has to be made online. This work proposes a novel controllable Pareto multi-task learning framework, to enable the system to make real-time trade-off switch among different tasks with a single model. To be specific, we formulate the MTL as a preference-conditioned multiobjective optimization problem, for which there is a parametric mapping from the preferences to the Pareto stationary solutions. A single hypernetwork-based multi-task neural network is built to learn all tasks with different trade-off preferences among them, where the hypernetwork generates the model parameters conditioned on the preference. At the inference time, MTL practitioners can easily control the model performance based on different trade-off preferences in real-time. Experiments on different applications demonstrate that the proposed model is efficient for solving various multi-task learning problems. 

Under review as a conference paper at ICLR 2021
CONTROLLABLE PARETO MULTI-TASK LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
A multi-task learning (MTL) system aims solving multiple related tasks the same time. With a fixed model capacity, the tasks would be conflicted with each other, and the system usually has to make a trade-off among learning all of them together. Multiple models with different preferences over tasks have to be trained and stored for many real-world applications where the trade-off has be made online. This work proposes a novel controllable Pareto multi-task learn- ing framework, to enable the system to make real-time trade-off switch among different tasks with single model. To be specific, we formulate the MTL as a peference-conddiiones multiobjective optimization problem, for which there a parametric mapping from the preferences to the Pareto stationary solutions. A sin- gle hypernetwork-base multi-task neural network is built to learn all tasks with different trade-off preferences among them, where the hypernetwork generates the model parameters conditioned on the preference. For inference, MTL prac- titioners can easily control the model performance based on different trade-off preferences in real-time. Experiments on different applications demonstrate that the proposed model is efficient for solving various multi-task learning problems.
INTRODUCTION
Multi-task learning (MTL) is important for many real-world applications, such as computer vision (Kokkinos) 2017), nat- ural language processing (Subramanian et al. (2018). and re- inforcement learning (Van Moffaert & Nowé 2014). these problems, multiple tasks are needed to be learned at the same time. MTL. system usually builds a single model to learn several related tasks together, in which the positive knowledge transfer could improve the performance for each task. In ad- dition, using one model to conduct multiple tasks is also good for saving storage costs and reducing the inference time, which
Task 2 Performance
can switch real-time
Pareto Solution
could be crucial for many applications (Standley et al.) (2200).
Pareto Front
However, with fixed learning capacity, different tasks could be conflicted with each other, and can not be optimized simul- taneously (Zamir et al.] 2018). The practitioners might need carefully design and train the tasks into different groups to achieve the best performance (Standley et al. 2020). A con- siderable effort is also needed to find a of suitable weights to balance the performance of each task (Kendall et al. 2018 Chen al. 20186 Sener & Koltun 2018). For some applica- tions, it might need to train and store multiple models for dif- ferent trade-off preferences among the tasks (Lin al. 2019).
Task 1 Performance
Controllable Pareto MTT. allows practitioners to con- trol the trade-offs among tasks in real time with a single model, which could be desirable for many MTL applications.
Figure 1:
In many real-world L applications the system will need to make a trade-off among different tasks in real time, and it is desirable to have the whole set of optimal trade-off solutions. For example, in self-driving system, multiple tasks must be conducted simultaneously but also compete for fixed resource (e.g., fixed total inference time threshold), and their preferences could change in real time for different scenarios (Karpathy 2019). A recommendation system needs to balance multiple criteria among different stakcholders simultanee smuutaneously, and making trade-off adjustment would a crucial component (Milojkovie et al. 2019). Consider the huge storage cost, it is far from ideal to

---

Under review as a conference paper at ICLR 2021
train and store multiple models to cover different trade-off preferences, which is also not good for real-time adjustment.
In this paper, we propose novel controllable Pareto multi-task learning framework, to learn the whole trade-off curve for all tasks with a single model. As shown in Fig. T MTL practitioners can easily control the trade-off among tasks based on their preferences. To our best knowledge, this is the first approach to learn the MTL trade-off curve. The main contributions are:
We formulate solving an MTL problem a weeference-conditione multiobjective opti- mization problem, and propose a novel Pareto solution generator to learn the whole trade- off curve of Pareto stationary solutions. The proposed Pareto solution generator is also novel contribution to multiobjectivve optimization.
We propose a general hypernetwork-based multi-task neural network framework, and de- velop an end-to-end optimization algorithm to train a single model concerning different trade-off preferences among all tasks simultancously. With the proposed model, MTL practitioners can control the trade-off's among different tasks in real time.
RELATED WORK
Multi-Task Learning. The current works on deep multi-task learning mainly focus on designing
novel network architecture and constructing efficient shared representation among tasks (Zhang &
Yang 2017: Ruder 2017). Different deep MTL networks, with hard or soft parameters sharing structures, haven been proposed in the past few years (Misra et al., 2016 Long et al. 2017 Yang & Hospedales 2017). However, how to properly combine and learn different tasks together remains basic but challenging problem for MTT applications. Although it has been proposed for more than two decades, the simple linear tasks scalarization approach is still the current default practice to combine and train different tasks in MTL problems (Caruana 1997).
Some adaptive weight methods have been proposed to better combine all tasks in MTL problems with a single model (Kendall al. 2018: Chen al.] [0188] Liu et al. 2019 Yu al. 2020). However, analysis on the relations among tasks in transfer learning (Zamir et al. 2018 and multi- task learning (Standley et al. 2020) show that some tasks might conflict with each other and can not be optimized at the same time. Sener and Koltun (Sener & Koltun 2018) propose to treat MTL as a multiohjective optimization problem, and find a single Pareto stationary solution among different tasks. Pareto MTI. (Lin et al.) 2019) generalizes this idea, and proposes to generate set of Pareto stationary solutions with different trade-off preferences. Recent works focuses on generating diverse and dense Pareto stationary solutions (Mahapatra & Rajan 2020
Ma et al. 2020).
Multiobjective Optimization. Multiobjective optimization itself is a popular research topic in the optimization community. Many gradient- based and gradient-fres algorithms have been proposed in the past decades (Fliege& & Svaiter 2000; Désidéri 2202 Miettinen| 2012). In addition to MTL, they also can be used in reinforcement learning { (Van Moffaert & Nowé 2014) and neural architecture search (NAS) (Elsken et al., 2011)). However, most methods directly use or modify well-studied multiobjective algorithms find a single solution or finite number of Pareto stationary solutions.
Recently, Lu et al. (2020) proposed a novel supernet-based multi-objective neural architecture trans- fer framework. This method simultaneously optimizes set of neural networks, which are sampled from a large supernet, to approximate the optimal trade-off curve among different objectives via a single run. The obtained supernet also supports fast adaption to new tasks and domains. Some ef- forts have been made to learn the entire trade-off curve for different machine learning applications. Parisi et al. (0166) proposed to learn the Pareto manifold for a multi-objective reinforcement learn- ing problem. However, since this method does not consider the preference for generation, it does not support real-time preference- based adjustment. Very recently, some efforts have been made to learn sefference-basel solution adjustment for multi-objective reinforcement learning (Yang et al.) 2019) and image generation (Dosovitskiy & Djolonga (020) with simple linear combinations. There is concurrent work (Anonymous] 2021) that also proposes to leam the entire trade-off curve for MTL problems by hypemnetork. While this work emphasizes the runtime efficiency on training for multiple preferences, highlight the benefit of scalability and real-time preference adjustment.
2

---

Under review as a conference paper at ICLR 2021
1,
Task Preference
generator
Task Preference
((
a) Pareto Solution Generator
b) Pareto Set
c) Pareto Front
Figure 2: Controllable Pareto MTL can directly generate corresponding Pareto stationary solu- tion based on a given preference among tasks. (a) A practitioner adjusts the m-dimensional pref- erence vector p to specify trade-off preference among m tasks. (b) A trained Pareto solution generator directly generates Pareto solution 0p conditioned should have objective values L(Op) that satisfies the trade-off preference among m tasks. The num- ber of possible preference vectors and the corresponding Pareto stationary solutions could be infinite for an MTL problem. We develop the idea in Section and propose the MTL model in Section 5
p. (c) The generated solution 0,
HyperNetworks. The hypernetwork approach is initially proposed for dynamic modeling and model compression (Schmidhuber 1992. Ha al.] 2017). It also leads to various novel appli- cations such as hyperparameter optimization (Brock et al. 2018:; MacKay et al.] 2019), Bayesian inference (Krueger et al.) 2018:: Dwaracherla al. 2020), and transfer learning (von Oswald al., 2020 Meyerson & Müikkulainen 2019). Recently, some discussions have been made on its initial- ization method (Chang et al., 2020). the optimization dynamic (Littwin al. 2020), and its relation to other multiplicative interaction methods (Jayakumar et al. 2020). This paper uses a hypernetwork to generate the weights the main multi-task neural network conditioned on different preferences.
MTL AS MULTI-OBJECTIVE OPTIMIZATION
An MTL problem involves learning multiple related tasks at the same time. For training deep multi-task neural network, it is also equal to minimize the losses for multiple tasks:
ming (())= = (0),
(0)),
where f the neural network parameters and (.) the empirical loss of the i-th task. For learning all m tasks together, an MTL algorithm usually aims to minimize all the losses at the same time. However, in many MTL problems, is impossible to find a single best solution to optimize all the losses simultaneously. With different trade-offs among the tasks, the problem (T) could have set of Pareto solutions which satisfy the following definitions (Zitzler Thiele 1999::
Pareto dominance. Let 0a, 0g be two solutions for problem ( Og is said to dominate 0g (0. 02) and only if (O.)) < (0.). Vi E {l....,m) m} and (0a) < C, (0.). {1...., m.
Pareto optimality. 8* is Pareto optimal solution if there does not exist such that ê < 8". The set of all Pareto optimal solutions called the Pareto set. The image of the Pareto set in the objective space is called the Pareto front.
Approximation with Finite Pareto Stationary Solutions. The number of Pareto solutions could be infinite, and their objective values are on the boundary of the valid value region (Boyd & Van- denberghe (2004). Under mild conditions, the whole Pareto set and Pareto front would be (m 1) dimensional manifolds in the solution space and objective space, respectively (Miettinen) 2012 In addition, for a general multiobjective optimization problem, no method can guarantee to find Pareto optimal solution. Traditional multiobjective optimization algorithms aim at finding set of finite Pareto stationary solutions to approximate the whole Pareto set and Pareto front:
S -0,,0,. ... 0x). F. (C(O.), ((0)). L(Ok)}.
where the approximated Pareto set of K estimated solutions, and is the set of correspond- ing objective vectors in the objective space. The current multiobjective optimization based MTL methods aim to find a single (K = 1) (Sener & Koltun, 2018) or a set of multiple (K > 1) Pareto stationary solutions (Lin et 2019 Mahapatra & Rajan
2020 Ma al. 2020) for given problem.
3

---

Under review as a conference paper at ICLR 2021
4 PREFERENCE PREFERENCE-CONNDTTONEDD PARETO SOLUTION GENERATOR
Instead of finite Pareto set estimation, we propose to directly learn the whole trade-off curve for a MTL problem in this paper. As shown in Fig. 2 want to build Pareto solution generator to map a preference vector p to its corresponding Pareto stationary solution 0p. If an optimal generator 0p g(p)")) is obtained, MTL practitioners can assign their preference via the preference vector p. and directly obtain the corresponding solution Op with the specific trade-off among tasks.
With the Pareto solution generator, we can obtain the approximate Pareto set/front
= (0p = "(pl)'))))))))))) Ip P). F = ( 0p E $).
where P is the set of all valid preference vectors, g Φ3) is the Pareto generator with optimal pa- rameters 6s. Once we have a proper generator g(pl we can reconstruct the whole approximated set of Pareto stationary solutions and the corresponding trade-off curve F by going through all possible preference vector p from P. In the rest of this section, we discuss two approaches define the form of preference vector p E P and its connection to the corresponding solution 0p = )
Dnfference-Conditionnee Linear Scalarization: A simple and straightforward approach define the preference vector and the corresponding solution 0p via the weighted linear scalarization:
8p )(plo") arg min P.L,
(4)
0
where the preference vector = (p1. P2. ... Pm) is the weight for each task, and the corresponding solution 0p is the minimum of the weighted linear scalarization. If we further require Σ p. 1, the set of all valid preference vector P is an (m dimmeensson manifold in Rm.. Therefore, the valid preference set and the Pareto set both (m 1) dimensional manifolds.
Although this approach is straightforward, it is not optimal for multiobjective optimization. Linear scalarization can find any Pareto solution on the non-convex part of the Pareto front (Das & Dennis 1997 Boyd & Vandenberghe/ 2004).. In other words, unless the problem has a convex Pareto front, the generator defined by linear scalarization cannot cover the whole Pareto set manifold.
Multi- Objei-Objecctvee Optimization: To bet- ter approximate the manifold of Pareto set, we generalize the idea of decomposition-basee multiobjective optimization (Zhang & Li 2007 et a (2014) and Pareto MTL (Lin et al., 2019) to connect the preference vector and the corresponding Pareto solution. To be specific, we define the preference as an m dimensional unit vector in the loss space, and the corresponding solution O, is the one on the Pareto front which has the smallest angle with p.
400
My
The idea is illustrated in in Fig. With set of randomly generated and the preference vector p. an MTL problem is decomposed into different regions the loss space. We call the region closest to p its preferred region. The corresponding Pareto solution Op is the solution that always belongs to the preferred region and on the Pareto front. Formally, we can define the corresponding Pareto solution as:
unit reference vectors U (u ()) .... u(K3
Figure 3: An MTL problem is decomposed by a set of unit vectors in the loss space.
0p = (((p = arg min L(0) s.t. L(0) E a(p,U),
(5)
where ((0) is the loss vector and p(p, U) is the constrained preference region conditioned on the preference and reference vectors 0(p, U) {u E RTI p) < L(v, w0)), Vj 1...,,""' K).
The constraint is satisfied if the loss vector L()) has the smallest angle with the preference vector p. The corresponding solution Op is restricted Pareto optimal in R(p, U). Since we require the preference vectors should be unit vector |p| 1 in the m-dimennioonll space, the set of all valid preference vectors P is an (m - 1) manifold in T.. Similar to the linear scalarization case, preference vector might have more than corresponding solution, and Pareto solution can correspond to more than one preference vector. For simplicity, we assume there a one-to-one mapping between the preference vector and the corresponding Pareto solution in this paper.
4

---

Under review as a conference paper at ICLR 2021
P
p
HyperNet Ø
g(p (
p.t
gfp (
embd1,
HyperNer
embd2: 0g
p, g(p
Task
fi(x
X
Shared
embik:
Cx
Bottom
Task (x(1))
g0p.ck xxp0
(a) Controllable Pareto MTL Network
(b) Chunked Hypernetwork
Figure 4: The Controllable Pareto Multi- Bast Network: (a) The main MTL network has fixed structure, and all its parameters are generated by the hypernetwork conditioned on the preference vector. (b) With the chunking method, the hypernetwork can iteratively generate parameters for different parts of the main network.
CONTROLLABLE PARETO MULTI-TASK LEARNING
In this section, we propose a hypernetwork-ban deep multi-task neural network framework, along with efficient end-to-end optimization procedure to solve the MTL problem.
5.1 HYPERNETWORK-BAR BBSED DEEP MULTI-TASE NETWORK
The proposed controllable Pareto multi-task network shown Fig. 4 a). As discussed in the previous section, we use a preference vector to represent practitioner's trade-off preference among different tasks. The hypernetwork takes the preference vector as its input, and generates 0p g(p Φ) as the corresponding parameters for the main MTL network. the only trainable parameters to be optimized are the hypernetwork parameters Φ. Practitioners can easily control the MTL network's s parameters, and hence the performances on different tasks in real time, by simply adjusting the preference vector p.
Model Structure and Regularization: In our proposed model, the main MTL network always has fixed structure, and the parameters Op are generated by the hypernetwork. We consider the main MTL network with hard-shared encoder (Ruder [017] Vandenhende et al.) 2020) as shown in Fig. aa). Once the parameters Op are generated, an input to the main MTL network will first go through the shared encoder, and then all task-specific heads to obtain the outputs f((((()). Different tasks share the same encoder, and they regularized by each other in the traditional MTL model. Our proposed model puts the cross-tasks regularization on the hypernetwork, where should generate set of good encoder parameters that work well for all tasks with given preference.
There is also a regularization effect across different trade-off preferences. Our goal is to train single hypernetwork-based model that performs well for all preferences. In other words, the generated main MTL network with a specific preference is regularized by the other networks with different preferences through the hypernetoork. This regularization effect might be best explained by the batched preferences update discussed in Appendix B.3 In this respect, et al. (2020) that uses hypernetwork with explicit regularization also good for knowledge transfer among different preferences. We further discuss how to use our model with pretrained feature extractor in Appendix
similar von Oswald separate tasks in the continual
learning setting.
Chunked Hypernetwork: With the model compression ability powered by the hypernetwork, our proposed model can scale well to large scale problems. Chunking (Ha et al.) 2017 von Oswald ct al. 2020) is a commonly used method to reduce the number of parameters for the hypernetwork. As shown in Fig. 4b). hypernetwork can iteratively generate small parts of the main network 8, = (0..1,0.2. 1p.2.-. ... 0p.k], with reasonable model size and multiple trainable chunk embedding (cx [c where 6p. = g(p, ex cx) In this way, we can significantly compress the model size and make it scalable for large MTL models. In this paper, use comneccod networks as the hypernetwork, and the main MTL neural networks have the same structures as the models used in current MTL literature (Liu et al. 2019 Sener & Koltun 2018 Vandenhende et al. 2020).
5

---

Under review as a conference paper at ICLR 2021
p e- 2, P.C.
FC
W
ppt.j
C2
Figure 5: Parameter Generation: The hypernetwork takes a preference vector p as input and gener- ates the parameter Pp... for part of the main MTL network, where Ei= p;e, is the preference embedding, Ck is the chunk embedding, FC is the fully connected layers, a is the generated vector, and W is parameter tensor. All blocks in color contain trainable parameters for the hypernetwork.
Parameter Generation: We follow the hypernetwork design proposed by Ha et al. (0117) with additional chunk embedding (von Oswald et al. (020) to construct our hypernetwork -based model. An illustration the process to generate chunk of parameters for the main MTL network is shown in Fig. 5 We first assign m trainable g-dimensional embedding e, for each task, and use the sefference-eiighee embedding
Σ. p,e, the input to the hypernetwork. In this way, the input dimension increases from a small m (e.g.. 2) to a larger g (e.g. 100). We simply concatenate of fully connected layers, and obtain the generated vector a E R²d, Then the hypernetwork uses a linear operation project the vector a into the generated parameters Op W, where W, E R x contains nj X n2 X d trainable parameters and p....j E 1835,Xm555 The hypernetwork generates all parameters 0,- [6p.1, pp.2.-.. ... 0p.k] ] by iteratively conducting this generation process.
the preference embedding with chunk embedding C as the input to
Scalablity and Inference: In the proposed hypernetwor,, the fully connected layers are reusable to generate different based on different chunk embedding. Most hypernetwork parameters are in the parameter tensors. By sharing the same W, to generate different chunks of parameters with different a, the number of parameters for the hypernetwork can be further reduced (Ha et al.) (017). which makes it suitable to generate the parameters for large scale models. In this work, we keep our hypernetwork based model to have comparable size with the corresponding MTL model. For inference, the fully connected layers can make batch inference for multiple chunk embedding to generate different a, and most computation is on the linear projection. It leads to an acceptable inference latency overhead, and supports real-time trade-off preference adjustment.
5.2 END-TO-END OPTIMIZATION: LEARNING PARETO GENERATOR VIA SOLVING MTL
Since we want to control the trade-off preference at the inference time, we need to train a model suitable for all preference vector p in the valid set P rather than single preference. Suppose have a probability distribution on all possible preference vector p ~ Pp. a general goal would be It is hard optimize the parameters within the expectation directly. We use Monte Carlo method to sample the preference vector, and use the stochastic gradient descent algorithm train the deep neural network. At each step, we can sample one preference vector p and optimize the loss function:
C(gpp)
mingC(g(plo). p p~P,
At each iteration t, if we have valid gradient direction d, for the loss ((g(plo))) simply update the parameters with gradient descent Φ = Φ - -ndd. For the preference-conditionn lincar scalarization case in problem 4 the calculation of d, is straightforward:
de = Σ p.V. C.Noo.C((l(llo)) p~ P Pp
For the should simultaneously reduce all losses and activated constraints. One valid descent direction can be written linear combination of all tasks with dynamic weight (t):
multiobjective optimization problem (5) a valid descent direction
d; Σ as (t)V
p~ Pp. U Pu.
(8)
where the coefficients at; (t) is depended on both the loss functions L (plp)) and the constraints (0)p, U). We give the detailed derivation in the Appendix due page limit.

---

Under review as a conference paper at ICLR 2021
5.3 ALGORITHM FRAMEWORK
Training Loss MultiNNiST Problem
Algorithm Controllable Pareto Multi-Tast Learning
1: Initialize the hypernetwork parameters Φo
183
2: for t = 1 to I do
Loss
1130
3: Sample preference vector p ~ Pp. U Pu
cou
4: Obtain valid gradient direction d, from 7
5: Update the parameters Φ1+1 = Φb r/d,
6: end for
7: Output: The hypernetwork parameters Ør
337
021
Task
Figure 6: The algorithm framework and the continually learned trade-off curve for the training loss on MultiMNIST problem: Our proposed algorithm optimizes the hypernetwrr parameters for all valid preferences, and continually learns the trade-off curve during the optimization process.
The algorithm framework is shown in Algorithm I We use a simple uniform distribution on all valid preferences and references vector in this paper. At each iteration, we calculate a valid gra- dient direction and update the hypernetwork parameters. The proposed algorithm simultaneously optimizes the hypernetwork for all valid preference vectors that represent different valid trade-offs among all tasks. As shown in Fig- it continually learns the trade-off curve during the optimization process. We obtain a Pareto generator the end, and can directly generate solution 9n = (((p)rr) from any valid preference vector p. By taking all preference vectors input, construct the whole trade-off curve.
SYNTHETIC MULTT--BIJECTIV OPTIMIZATION PROBLEM
Fanto TImE
Fanto FIEE Sande
5/01
550
(a) Approx. Pareto Set
(b) Approx. Pareto Front (c) Generated Pareto Set (d) Generated Pareto Front
Figure 7: Point Set Approximation and Generated Pareto Front. (a) & (b): Traditional multi- objective optimization algorithms can only find finite point set to approximate the Pareto set and Pareto front. addition, the simple linear scalarization method can not cover most part of the Pareto set/front when the ground truth Pareto front is concave curve. (c) & (d): Our proposed controllable Pareto MTL. method can successfully generate the whole Pareto front from the preference vectors. In contrast, the linear scalarization method is only accurate for Pareto solutions near the endpoints.
In the previous section, we propose to use hypernetwork to generate the parameters for the main deep multi-task network based on a preference vector. The network parameters are in high- dimensional decision space, and the optimization landscape would be complicated. To better analyze the behavior convergence performance of the proposed algorithm, we first use to generate the Pareto solutions for dow-dimensional multiobjective optimization problem.
The experimental results are shown in Fig. 7 This synthetic problem has a sin-curve-like Pareto set in the solution space, and its corresponding Pareto front is concave curve in the objective space. The detailed definition is given in the appendix. Our proposed controllable Pareto MTL method cover and reconstruct the whole manifold of Pareto front from the preference vectors, while the traditional methods can only find a finite point set approximation. It should be noticed that the simple linear scalarization method has poor performance for the problem with concave Pareto front.

---

Under review as a conference paper at ICLR 2021
7 EXPERIMENTS
0.50
Task 2: Drptth (3: -Robahee Enortt
7. AEC.PnE Zeltom
6915
2.08
LINAP Sealerst
Task
1834 TUE
Tank L: Accuracy Top Laft
Task Segimentation
Task Segmentation
(a) MultiMNIST
(b) CityScapes
(c) NYUv2
Figure 8: Results on MultiMISST, CityScapes and NYUv2: Our proposed algorithm can generate the Pareto front for each problem with single model respectively. For CityScapes and NYUv2, report pixel accuracy for segmentation and relative error) for depth estimation.
720
T2
Ave. Ace
Uniform Linear
79.45 + 0.23
T1
T5
Uncertainty
79.35 + 0.43
1.70
DWA
80.52 0.24
Li Bore Liieet
MGDA
80.14 + 0.19
125
Balanced Hypernet
80.53 0.16
Ctrl Linear
81.17 1 0.08
Ctrl Pareto MTL
82.36 + 0.06
T10
Figure 9: The results of CIFAR-100 with 20 Tasks: The results for each baseline method is single solution. Our proposed algorithms switch to the corresponding preference when making prediction for each task (e.g., preference on task 1 for making prediction on task 1).
In this section, we validate the performance of the proposed controllable Pareto MTL to generate the trade-off curves for different MTL problems. We compare it with the following MTL algorithms: 1) Linear Scalarization: simple linear combination of different tasks with fixed weights; 2)
tainty (Kendall al. 2018): adaptive weight assignments with balanced uncertainty: 3)DWA (Liu
jet al. 2019): dynamic weight average for the losses; 4) MGDA (Sener & Koltun 2018): finds one Pareto stationary solution; 5) Pareto MTL (Lin et al., 2019): generates set of wildly distributed
Pareto stationary solutions; and 6) Single Task: the single task baseline.
We conduct experiments on the following widely-used MTL problems: 1) MultiMNIST (aboor et al., 2017): This problem is to classify two digits on one image simultancously. 2) CityScapes (Cordts et al. 2016): This dataset has street-view RGB images, and involves two tasks to be solved, which are pixel-wise semantic segmentation and depth estimation. 3) NYUv2 (Silber- man al. 2012): This dataset for indoor scene understanding with two tasks: 13-class semantic segmentation and indoor depth estimation. 4) CIFAR-100 with 20 Tasks: Follow similar setting
(Rosenbaum et al. 2018), we split the original CIFAR- --00 dataset (Krizhevsky & Hinton 2009) into 20 five-class classification tasks. Details for these problems can be found in the Appendix C
The experimental results are shown in Fig. g where all models are trained from scratch. In all problems, proposed algorithm learn the trade-off curve with single model, while the other methods need to train multiple models and cannot cover the entire curve. In addition, infferecc-conditionn
Result Analysis:
the proposed
multiobjective optimization method always find better trade-off curve that dominates the curve found by the simple linear scalarization. These results validate the effectiveness of the proposed model and the end-to-end optimization method.
For all experiments, the single-tast task models are a strong baseline in performance with larger model size (m full models). However, it cannot dominate most of the approximated Pareto front learned by
8

---

Under review as a conference paper at ICLR 2021
Table 1: Overveiw of the models we used in different MTL problems.
Problem
Tasks
Network
Model
Params
(m))
Single MTL Model
84K
1.49 0.12
0.12
MultiMNIST
LeNet
Hypernetwork- Model
83K
1.96
Single MTL Model
25M
65.0 1.23
CityScape
2
SegNet
Hypernetwork -baseorkk--eedddddo Model
26M
88.6 = 4.85
Single MTL Model
25M
179 13.1
NYUv2
2
SegNet
Hypernetwork-basee Model
26M
198 = 11.5
Single MTL Model
1.4M
9.95 + 0.17
CIFARI00
20
ConvNet
Hypernetwork-basee Model
5.9M
23.4 + 0.34
0.34
Single MTL Model
56M
236
11.2
NYUv2
ResNet-50
Hypernetwork-based Model
120M
384
18.0
our model. Our models provide diverse optimal trade-offs among tasks (e.g., in the upper left and lower right area) for different problems and support real-time adjustment among them.
The experimental result on the 20-tasks CIFAR100 problem is shown in Fig. 9 Our proposed model can achieve the best overall performance among different tasks by making real-time trade-off adjustment. It also outperforms the balanced hypernet approach, which uses the same hypernetwork- based model to optimize all tasks simultancously. This result shows the benefit of our proposed model on preference- based modeling and real-time preference adjustment.
Scalability and Latency: The models we in the exper- iments are summarized in [able As discussed in the pre- vious sections, build the hypernetwork such that the pro- posed models have a comparable number of parameters with the corresponding MTL model. Given the current works (Lin
0/439
Task 2: Depth (mos))
0-022
et al. 2019: Mahapatra & Rajan 2020 Ma et al.) 2020) need to train and store multiple MTL models to approximate the Pareto front, our proposed model is much more -fffcciet while learning the whole trade-off curve. In this sense, it is more scalable than the current methods.
0,075
Credber
The inference latency for all models are also reported. The latency is measured on 1080Ti GPU with the training batch size for each problem. We report the averaged mean and stan- dard deviation over 100 independent runs. For proposed model, we randomly adjust the preference for each batch. Our model has an affordable overhead on the inference latency, which is suitable for real-time preference adjustment.
Task 1: Segmentation (mIDU)
Figure 10: Results on NYUv2 with ResNet-50 Backbone: The proposed model can generate rea- sonable trade-off curves.
Experiment with Pretrained Backbone: We conduct
periment on the NYUv2 problem with a large-scale ResNet-50 backbone (He et al. 2016) following the experimental setting in Vandenhende et al. (2020). Each model has ResNet-50 backbone pre- trained on ImageNet, and task-specific heads with ASPP module (Chen et al. 2018a). The results
shown in Fig TO where the results for MTL models are from Vandenhende et al. (2020) with the best finetuned confiuuratioss. Our proposed model also uses the pretrained ResNet-50 backbone, and generates the encoder's last few layers and all task-specific heads. Our model can learn the trade-off curve with comparable performance, which might be further improved with task balancing methods and better hypernetwork architecture design as Vandenhende et al. (022)..
8 CONCLUSION
In this paper, we proposed novel controllable Pareto multi-task learning framework for solving MTL problems. With a hypernetwork- leam the whole trade-off curve for all tasks with single model. It allows practitioners easily make real-tin time trade-off adjustment among tasks at the inference time. Experimental results on various MTL. applications demonstrated the usefulness and efficiency of the proposed method.
Pareto solution generator, our method can directly

---

Under review as a conference paper at ICLR 2021
REFERENCES
Anonymous.
Learning the pareto front with hypernetworks. In Submitted to International Confer- ence on Learning Representations, 2021. URL /tttps::/openrevieww net/ teet/foorum forum?
NJF772F4 ZZR,
under review.
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutional encoder- decoder architecture for image segmentation. JEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 39(12):2481-2495 2017.
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Andrew Brock, Theo Lim, J.M. Ritchie, and Nick Weston. Smash: One-shot model architecture search through hypernetworks. In ICLR 2018: International Conference on Learning Represen- tations 2018, 2018.
Rich Caruana. Multitask learning. Machine learning, 28(1):41-75, 1997.
Oscar Chang, Lampros Flokas, and Hod Lipson. Principled weight initialization for hypernetworks. In ICLR 2020 Eighth International Conference on Learning Representations, 2020.
Liang Chich Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In European Con- feeencee Computer Vision, 2018a.
Zhao Chen, Vijay Badrinarayanan, Chen- Yu Lee, and Andrew Rabinovich, Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In Proceedings of the 35th International Conference on Machine Learning, pp. 794-803, 2018b.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Indraneel Das and J.E. Dennis. A closer look at drawbacks of minimizing weighted sums of objec- tives for pareto set generation in multicriteria optimization problems. Structural Optimization, 14 (1):63-69, 69, 1997.
Jean-Antoine Désidéri. Mutiple-gradient descent algorithm for multiobjective optimization. In European Congress on Computational Methods in Applied Sciences and Engineering (ECCOMAS 2012), 2012.
Loss-conditional training of deep Alexey Dosovitskiy and Josip Djolonga. You only train once: networks. In International Conference Learning Representations, 2020. URL https: //
openreview net www.neet/fooooe hatprrrveeeenee//oooumii------
Vikranth Dwaracherla, Xiuyuan Lu, Morteza Ibrahimi, Ian Osband, Zheng Wen, and Benjamin Van Roy. Hypermodels for exploration. In ICLR 2020: Eighth International Conference on Learning Representations, 2020.
Thomas Elsken, Jan Metzen, and Frank Hutter. Efficient multi multi-objective neural architecture search via lamarckian evolution. In International Conference on Learning Representattonss, 2019.
Jörg Fliege and Benar Fux Svaiter. Steepest descent methods for multicriteria optimization. Mathe- matical Methods of Operations Research, 51(():479-4944 2000.
Bennet Gebken, Sebastian Peitz, and Michael Dellnitz. A descent method for equality and inequality constrained multiobjective optimization problems. In Numerical and Evolutionary Optimization, pp. 29-61. 61. Springer, 2017.
David Ha, Andrew M. Dai, and Quoc V. Le. Hypernetworks. ICLR 2017 International Confer- on Learning Representations 2017, 2017.
10

---

Under review as a conference paper at ICLR 2021
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Martin Jaggi. Revisiting frank-wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th international conference machine learning, pp. 427-435, 2013.
Siddhant M. Jayakumar, Jacob Menick, Wojciech M. Czarnecki, Jonathan Schwarz, Jack Rae, Si- mon Osindero, Yee Whye Teh, Tim Harley, and Razvan Pascanu. Multiplicative interactions and where find them. In ICLR 2020 Eighth International Conference Learning Representa- tions, 2020.
Andrej Karpathy. Multi-task learning in the wilderness. AMTL Workshop, ICML2019, 2019.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
lasonas Kokkinos. Ubernet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. In Proceedings of the IEEE Confer- ence on Computer Vision and Paltern Recognition, pp. 6129-6138, 2017.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, htpp://www.cs.toronlle edu/ kriz/cifarhtml, 2009.
David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, and Aaron Courville. Bayesian hypernetworks. arXiv: Machine Learning, 2018.
Xi Lin, Huiling Zhen, Zhenhua Li, Qingfu Zhang, and Sam Kwong. Pareto multi-task learning. In Advances in Neural Information Processing Systems (NeurIPS). accepted, 2019.
Etai Littwin, Tomer Galanti, and I Lior Wolf. On the optimization dynamics of wide hypernetworks. arXiv preprint arXiv:2003.12199 2020.
Hai-Lin Liu, Fangqing Gu, and Qingfu Zhang. Decomposition of multiobjective optimization problem into number of simple multiobjective subproblems. IEEE Trans. Evolutionary Compu- tation, 18(3):500-45, 2014.
Shikun Liu, Edward Johns, and Andrew J. Davison. End-to-end multi-task learning with attention. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1871 1880, 2019.
Mingsheng Long, Zhangjic Cao, Jianmin Wang, and Philip Yu. Learning multiple tasks with multilinear relationship networks. In Advances in Neural Information Processing Systems, pp. 1594-1603, 2017.
Zhichao Lu, Gautam Sreekumar, Erik Goodman, Wolfgang Banzhaf, Kalyanmoy Deb, and Vishnu Naresh Boddeti. Neural architecture transfer. arXiv preprint arXiv:2005.05839, 2020.
Pingchuan Ma, Tao Du, and Wojciech Matusik. Efficient continuous pareto exploration multi-task learning. Thirty-seventh International Conference Machine Learning (ICML 2020), 2020.
Matthew MacKay, Paul Vicol, Jonathan Lorraine, David Duvenaud, and Roger Grosse. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. In ICLR 2019: : 7th International Conference on Learning Representations, 2019.
Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient de- scent with controlled ascent in pareto optimization. Thirty-seventh International Conference on Machine Learning (ICML 2020), 2020.
Elliot Meyerson and Risto Mikkkulainen. Modular universal reparameterization: Deep multi-task learning across diverse domains. Advances in Neural Information Processing Systems, pp- 7903-7914, 2019.
11

---

Under review as a conference paper at ICLR 2021
Kaisa Miettinen. Nonlinear multiobjectiive optimization, volume 12. Springer Science & Business Media, 2012.
Nikola Milojkovic, Diego Antognini, Giancarlo Bergamin, Boi Faltings, and Claudiu Musat. Multi- gradient descent for multi-objective recommender systems. arXiv preprint arXiv:2001.00866 2019.
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3994 4003, 2016.
Simone Parisi, Matteo Pirotta, and Marcello Restelli. Multi-objective reinforcement learning through continuous pareto manifold approximation. Journal of Artificial Intelligence Research, 57:187-227, 2016.
Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. Routing networks: Adaptive selection of non-linear functions for multi-task learning. In ICLR 2018 International Conference on Learning Representations 2018, 2018.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.050990 2017.
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Advances in Neural Information Processing Systems, pp. 3856-3866, 2017.
Jürgen Schmidhuber. Learning to control fast-weight memories: alternative to dynamic recurrent networks. Neural Computation, 4(1):131-139, 1992.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, pp. 525-536, 2018.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and sup- port inference from rgbd images. In European conference on computer vision, pp. 746-760. Springer, 2012.
Trevor Standley, Amir Roshan Zamir, Dawn Chen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? International Confer- on Machine Learning, 2020.
Sandeep Subramanian, Adam Trischler, Yoshua Bengio, and Christopher J Pal. Learning general purpose distributed sentence representations via large scale multi-task learning. In International Conference on Learning Representations, 2018.
Kristof Van Moffaert and Ann Nowé. Multi-objective reinforcement learning using sets of pareto dominating policies. The Journal of Machine Learning Research, 15(():3483-33) 2014.
Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Lue Van Gool. Multi-task learning for dense prediction tasks: A survey, 2020.
Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F. Grewe. Continual learning with hypernetworks. In ICLR 2020 Eighth International Conference Learning
Representations, 2020.
Runzhe Yang, Xingyuan Sun, and Karthik Narasimhan, A generalized algorithm for multi multi-objective reinforcement learning and policy adaptation. In Advances in Neural Information Processing Systems, pp. 14636-14647, 2019.
Yongxin Yang and Timothy M. Hospedales. Deep multi-task representation learning: A tensor factorisation approach. In ICLR 2017 International Conference on Learning Representations 2017, 2017.
Tianhe Yu. Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. arXiv preprint arXiv:2001 006882, 2020.
12

---

Under review as a conference paper at ICLR 2021
Amir R. Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. 2018 IEEE/CVF Conference on Com-
puter Vision and Pattern Recognition, pp. 3712-3722, 2018. URL attps://academ::
mi mirosoot. com haaam//appr/294411 85501
Qingfu Zhang and Hui Li. Moea/d: A multiobjective evolutionary algorithm based on decomposi- tion. IEEE Transactions on evolutionary computation, 11(6):712-71, 2007.
Yu Zhang and Qiang Yang. A survey on multi-task learning. arXiv preprint arXiv:1707.08114 2017.
Eckart Zitzler and Lothar Thiele. Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE transactions on Evolutionary Computation, 3(4):557- 271, 1999.
13

---

Under review as a conference paper at ICLR 2021
APPENDIX
We provide more discussion and analysis this appendix, which can be summarized as follows:
Model with Pretrained Feature Extractor: We discuss how to use pretrained feature extractor our proposed model in Appendix A
Preference-based Multiobjective Gradient Descent: We give the details of preference- based multiobjective gradient descent for model training in Appendix B
Experimental Setting: The detailed experimental settings are provided Section
A MODEL WITH PRETRAINED FEATURE EXTRACTOR
4,00
7 HyperNet Ø
0
Task
fx,100000))
Preiriin
0g
Generated
Enoodirr
Task 2
/5(x100g 0,,0,)
1,(0)
(a) Pareto MTL Net with Pretrain Encoder
(b) Generated MTL Net with Different Preference
Figure 11: (a) Controllable Pareto MTL network with a pretrained feature extractor: The pretrained encoder is preference-agnoste and the hypernetwork generates the rest parts of the main MTL. network. (b) The generated MTL models share the same parameters for part the encoder.
One powerful and efficient way to improve the MTL model performance is to use feature extractor pretrained on large scale dataset (Ruder) 2017] Vandenhende al.) 2020p. This approach can be easily incorporated into proposed model. As shown in Fig
model can be built top of a shared pretrained feature extractor, and the hypernetwork generates the parameters for the rest part of the encoder and all task-specific heads. The main MTL model parameters O = [0,, 0pl now have a wreference-agnostic part 0, and preference preference-condiiionnee part 6, g
In other words, matter what preferences given, all generated MTL models will share the parameters for the feature extractor as shown in Fig (bb is a structure constraint across all generated solutions, and the trade-off curve learned by model would be front of restricted Pareto stationary solutions, with constraints on the sharing parameters. The shared encoder brings knowledge from the datasets pretrained on, and it also has regularization effect since shared by models with different preferences. The experiment on models with ResNet-50 backbone validates that our proposed model can still learn the trade-off curve with pretrained encoder.
14

---

Under review as a conference paper at ICLR 2021
(x
(0)
(2(
(e)
(9)
(
(,)
(a) iter
(b) iter 2
(c) iter 3
(d) iter 4
Figure 12: The training process of the Pareto solution generator. At each iteration, the proposed
algorithm randomly samples preference vector and set of reference vectors decompose the loss space, and calculates a valid gradient direction to update the Pareto solution generator. The algorithm iteratively learns and improves the generated manifold during the optimization process.
B PREFERENCE-BASED MULTIOBIECTIVE GRADIENT DESCENT
In this section, give the details of preference-based multiobjective gradient descent for training the hypernetwork-based model.
B. PREFERENCE- PREFERENCE-CONDITIONEE CONDITIONED MULTIOBJECTIVE GRADIENT DESCENT
The Pareto solution generator generates a solution 6, = g(p) (b) from the preference vector p. Under P (which is an (m 1)-dimensional manifold) as input, the output would be another (m - 1) -ddimensionaa manifold. At the end of the algorithm, hope the generated manifold is close the manifold of Pareto set.
ideal condition, if take all valid preference vectors
In the proposed algorithm, we randomly sample preference vector p at each iteration to update the generator parameters As illustrated in Fig. 12 continually learns and improves the current man- ifold around the preference vector at each iteration. It should be noticed that the sampled preference vector and all reference vectors different at each iteration, as the decomposition.
As mentioned in the Algorithm in the main paper, we use simple gradient descent to update the Pareto solution gener- ator at each iteration. For the case of linear scalarization, calculating the gradient direction straightforward. In this subsection, we discuss how to obtain valid gradient direc- tion for the preference-conditioned multiobjective optimiza- tion. Similar to the previous work (Sener & Koltun 2018 Lin eet al., 2019), we use multiobjective gradient descent (Fliege & Svaiter 2000 Désidéri 2012) to solve the MTL problem. However, in our algorithm, the optimization parameter is ø for the Pareto solution generator rather than 0 for single solution.
simple illustration of the multiobjective gradient descent idea for problem with two tasks is shown in Fig. [3] At each iteration, the algorithm samples a preference vector p and ob- tains its current corresponding solution Or- A valid gradient direction should reduce all the losses and guide the generated solution 0, g(l) Φ) toward the preference region 2(p, U) around the preference vector p.
(
Figure 13: For a preference vector, the valid gradient direction should reduce all losses and activated con- straints for the generated solution.
At iteration I, for given preference vector p. the preference- conditioned multiobjective problem with the Pareto solution generator can be written as:
... s.t. G.(p(p)o))ppp U) (u -( 50 so,j K.....
&
15

---

Under review as a conference paper at ICLR 2021
Since the parameter Pp is generated by the Pareto solution generator g(p)))). the generator's pa- rameter Φ is the only trainable parameters to be optimized. What we need is to find a valid gradient direction d, to reduce all the losses (p((())))) and activated constraints Gy(g(plo)) p. U) with respect to Φi.
We follow the methods proposed in (Fliege & Svaiter 2000 Gebken al. 2017). and calculate valid descent direction d, by solving the optimization problem:
(di, ar) aarg min deR'
+ _
(10)
8.1. V (..p((()))) s a l,..,,
V ( S Sa,j I(bi).
where d, is the obtained gradient direction, is auxiliary parameter for optimization, and [(0) {j E [gg(((()))))) 2 0 the index set all activated constraints. By solving the above problem, the obtained direction d, and parameter a2 will satisfy the following lemma (Gebken et al. 2017):
Lemma 1: Let (de, ax) be the solution of problem 10
we either have:
1. A non-zero d, and 04 with
an < (1/2)]]]41 < 0, V (ooplo))) d. ar, = 1,..,, V. (g(()())) d. an, j E I(b)).
(11)
2. or d, = 0 R", G2 = 0, and 0, - (p) is local Pareto critical restricted on 22 U).
In case we obtain a valid descent direction d: which has negative inner products with all V (g((((p))))) and V (ol((((pl for E I())). With suitable step size we can update Φ Φi +dd reduce all losses and activated constraints. In case we cannot find any nonzero valid descent direction, and obtain d, O. In other words, there is no valid descent direction to simultaneously reduce all losses and activated constraints. Improving the performance for one task would deteriorate the other(s) violate some constraints. Therefore, the current solution Op g Φ.) is local restricted Pareto optimal solution on (tp,U).
B.2 ADAPTIVE LINEAR SCALARIZATION
rewrite the gradient direction d, a dynamic linear combi- nation with the gradient of all losses V C (g(((o)) Similar to the approach in (Fliege & Svaiter Lin et al. 2019), we reformulate the problem (10) in its dual form:
As mentioned in the main paper,
2000
m
max X.
V filoppo)))))
+
Σ
AV (((o(()))
2
jel()))
(12)
m
s.t.
Σ
B1=1,
A 2 0, 33
mj E I(Φ.),
jEI(
where d, Em X h. (s(pp))))) Eje((h))) B A.a. Gj (z((p()) is the obtained valid gradient
direction, A and B, are the Lagrange multipliers for the linear inequality constraints in problem I0
Based on the definition in problem 9 a constraint (g(p))))) can be rewritten as linear combi- nation of all losses:
()((()))) = (w - p) L(g(plo)) = (u p.)_.(((pp
(13)
The gradient of the constraint also a linear combination of those for the losses:
Vo.l)(o(plo.) = =m(() Vo. C(g(p(o)))
(u - V.)V C.(((p)))))))
(14)
16

---

Under review as a conference paper at ICLR 2021
1
P
1,00
1,00
1
(a)
(b)
(c) iter
(d)
Figure 14: The training process of the Pareto solution generator with multiple preferences. At each iteration, the proposed algorithm randomly samples multiple preference vectors, and calculates valid gradient direction which can improve the performance for all subproblems.
Therefore, we can rewrite the valid descent direction as a linear combination of the gradients for all tasks with dynamic weight az (t):
d, Σ ((
a;(1)= x
Σ jEl.(0)
By
pi),
(15)
where A and Bj are obtained by solving the dual problem 12)
It should be noticed that the primal problem (TO) directly optimizes the gradient direction d: which could be millions of dimensions for training a deep neural network. In contrast, the dimension of dual problem (12) is the number of all tasks and activated constraints (up to few dozens), which is much smaller than the primal problem. Therefore. in practice, we obtain the valid gradient direction d, by solving the dual problem
12) instead of the primal problem (TO).
We use the Frank- Wolfe algorithm (Jgggi 2013) to solve the problem 122 as in the previous work (Sener & Koltun 2018 Lin et al. 2019). We use simple uniform distribution sample both the unit preference vector and unit reference vectors U in this paper. The number of preference vector is in the previous discussion, and the number of reference vectors is hyperparameter, which we set it as 3 in this paper. In the next subsection, we will discuss how to use more than one preference vector at each iteration update the Pareto solution generator.
B.3 BATCHED PREFERENCES UPDATE
To obtain an optimal Pareto generator s(p( we need to find:
Φ & ((g(plo).
(16)
In the main paper, we sample one preference vector each iteration calculate a valid direction d, to update the Pareto generator. A simple and straightforward extension is to sample multiple preference vectors to update the Pareto solution generator, as shown in Fig. 14
At each iteration, simultancously sample and optimize multiple subproblems with respect to the Pareto generator:
min L(g(p) ()), C(g(p2(0)). L(o(px()))) P1.P2, .PK Pp
(17)
where Pt, ...... PK are K randomly sampled preference vectors and each L(g(ps Φ)) a multi- objective optimization problem. Therefore, we now have a hierarchical multiobjective optimization problem. If we do not have specific preference among the sampled preference vectors, the above problem can be expanded problem with Km objectives:
min (L (g(p) ()), L. (g(p) ())) .... (g((p(())) L. (e(px())))
(18)
For the linear scalarization case, the calculation of valid gradient direction each iteration straightforward:
K
K m
d, = Σ (o,p(p(palo.))
p.v. C.V.C((( PL, P1.P2, ... .PK ~ Pp, k=
ΣΣ
(19)
17

---

Under review as a conference paper at ICLR 2021
where we assume all sampled subproblems equally important.
is more interesting to deal with the preference-conditione multiobjective optimization problem. For each preference vector Pk. suppose the rest preference vectors as its reference vectors, the obtained preference-condiiionee multiobjective optimization problem would be:
min(Cr(g(p::l0o))) (m ... LI(g(Pk())) (..p 41
(n(((pk())))
G1yl9(pi (.)) =(p p1)² L(g(pil s0,ie (I....,KVEI
G25l9(p2)05 o.)) = =p, - -P2)² L(g(p2 ( s0,vj (44.,,,)221,,
(20)
(k)(o(Pk(o.) (p, - (o--K)) "(g""(((( S S.Vj (1....,K]] KY(K)
There are (K 1) constraints for each preference vector, hence total K(K 1) constraints for the preference-conditoons Similar to the single point case, we can also calculate the valid gradient direction in the form of adaptive linear combination as:
multiobjective problem, although some of them could be inactivated.
d B.(1VV C(o(plo))))
(21)
where the adaptive weight (t(t) depends on all loss functions (((( (1)) and activated con- straints (s)lo(P)))).
18

---

Under review as a conference paper at ICLR 2021
C
EXPERIMENTAL SETTING
Synthetic Example: The synthetic example we use in Section 6 is defined as:
min f1(0) = 11 exp -(01 1)²
20. 2m
0. sin(551))
(22)
min f2(0) exp (-(01 + 1)2).
We set n = 10 and use a simple two-layer MLP network with 50 hidden units on each layer to generate the Pareto solutions based on the preference vectors.
MultiMNIST (Sabour et al., 2017): In this problem, the goal is to classify two overlapped digits in an image at the same time. The size of the original MNIST image is 28 x 28. We randomly
choose two digits from the MNIST dataset, and move digit to the upper-left and the other one to the bottom right with up to 4 pixels. Therefore, the input image is in size 36 x 36.
LeNet-based neural network with task-specific fully connected layers as the main MTL model, and use simple MLP as the hypernetorrk. Since the LeNet model is small, do not use any preference/chunk embedding. We let the hypernetwork-bast For all methods, the optimizer is Adam with learning rate 3e the batch size is 256, and the number of epochs is 200.
Similar to the previous work (Sener & Koltun 2018 Lin et al., 2019).
model has a similar number of parameters with a single MTL model.
CityScapes (Cordts et al., 2016): This dataset has street-view RGB images, and involves two tasks to be solved, which are pixel-wise semantic segmentation and depth estimation. We follow the setting used in (Liu et a 2019), and resize all images into 128 x 256. For the semantic segmentation, the model predicts the 7 coarser labels for each pixel. We use the L1 loss for the depth estimation. We report the experimental results on the Cityscapes validation set.
We use the MTL network proposed (Liu et al., 2019), which has SegNet (Badrinarayanan et al., 2017) the shared representation encoder, and two task-specific lightweight convolution layers. In our hypernetwork-basee model, the hypernetwork contains three 2-layer ML.s with 100 hidden units each layer, and most parameters are stored in the parameter tensors for linear projection. The preference embedding and chunk embedding are all 644-dimensional vectors. We also let the hypernetwork-basse bbsed model has similar number of parameters with the single MTL model. For all experiments, we use Adam with learning rate Ir = 3e 4 as the optimizer, and the batch size is 12. We train the model from scratch with 200 epochs.
NYUv2 (Silberman et al. 2012): This dataset is for indoor understanding with two tasks: 13-class semantic segmentation and indoor depth estimation. Similar to Liu et al. (2019), we resize all images into 288 384. For the training from scratch experiment, we a similar MTL network and hyperparameter setting as for the CityScapes problem, except the batch size is 8 in this problem.
We also test the performance on models with pretrained encoder on the NYUv2 Dataset. We follow the setting in the recent MTL survey paper (Vandenhende et al. (2020), all models have a ResNet-50 backbone (He et al. (016) pretrained on ImageNet, and two-specilic heads with ASPP module (Chen) et al.) 20) In our hypernetwork- based model, the hypernetwork has three different 2-layer MLPs with 200 hidden unit on each layer. One MLP is for generating shared convolution layers on top of the ResNet backbone, and the other two are for each task-specific head. The shared parameters for backbone are unfrozen, and will be adapted during training. use 100-dimensional vectors as preference and chunking embedding. We Adam with learning rate Ir le as the optimizer, the batch size is 8, and the total epoch is 200.
CIFAR100 with 20 Tasks: To validate the algorithm performance on MTL problem with many tasks, split the CIFAR- 100 dataset (Krizhevsky & Hinton| 2009) into 20 tasks, where each task is a 5-class classification problem. Similar setting has been used in MTL learning (Rosenbaum et al. 2018) and continual learning (von Oswald et al. 2020).
19

---

Under review as a conference paper at ICLR 2021
The MTL neural network has four convolution layers the shared architecture, and 20 task task-specific FC layers. In our proposed model, we have MLPs and the preference and chunking embedding are both 32-dimensional vectors. The optimizer is Adam with learning rate Ir = 3e +, the batch size 128, and the number of epochs is 200. We report the test accuracy for all 20 tasks.
20


Reviews:
[{"review_id": "hTG-at2xLlQ", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method to controllably generate models on the trade-off front of multi-task learning problems. The key idea is to use a hypernetwork that will generate the MTL parameters on demand conditioned on the desired trade-off. The hypernetwork can be trained along with the MTL in an end-to-end manner.\n\nStrengths:\n+ Paper addresses an important practical problem. Instead of training many models independently to span the trade-off front, it can generate an MTL model on demand for any desired trade-off point.\n+ The idea of using a hypernetwork in the context of MTL is new. \n\nWeaknesses:\n- Conceptually and technically the improvements over [1] are minimal. The main contribution of the paper is to use a hypernetwork that will generate the weights. Although this is new in the context of MTL, it is in fact a common idea in the context of Neural Architecture Search. In NAS we have a supernet which is equivalent to the hypernet and then perform reference based multi-objective optimization over it. See [2] for an example.\n- Method does not seem to be very scalable in terms of MTL model size. The experimental evaluation is on a small scale. For larger models the hypernet needs to generate a large output for the desired trade-off. In that case training the hypernet gets more challenging.\n- In terms of the exposition, firstly the main paper has very few details apart from the high level idea, there is a lot of repetition of the same points. Secondly, a lot of the actual paper and the appendix itself is background material that is already well known in the multi-objective optimization literature. \n- The experiments are a bit disappointing. The advantage of the proposed method in comparison to linear scalarization would be on concave pareto fronts as shown in Fig. 6. Unless I am missing something, looks like in none of the actual experiments is the trade-off a concave curve.  \n\nOverall, in the current form the paper looks more like a proof of concept. I would encourage the authors to demonstrate or discuss the scalability of the solution.\n\nClarifications:\n1) I guess one of the benefits of this approach over training multiple models is in terms of the total number of parameters and computational complexity. But there is no discussion of these aspects. How does the HyperNet compare in size wrt to the MTL model part? I would imagine the HyperNet is much bigger since it has to learn to predict the MTL parameters. \n2) How about inference? How long does it take for the HyperNet to generate the parameters since you still have to solve an optimization problem? \n3) The paper claims real-time but there is no discussion of this claim.\n4) In Fig.7 there seem to be some solutions that are better than the trade-off front obtained by the proposed method. Any comments on what is limiting the current approach from reaching/surpassing those solutions?\n\n[1] Pareto Multi-Task Learning\n[2] Neural Architecture Transfer", "extra_scores": {}, "extra_text": {}}, {"review_id": "MZjm7AIw6dZ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel controllable Pareto multi-task learning framework, which aims to learn the whole Pareto optimal front for all tasks with a single model. The motivation is straight forward and the proposed method is inspiring. However, the proposed technique does not match with the announced contribution.\n1. This paper announces that it proposes a novel Pareto solution generator that can learn the whole Pareto front for MTL. However, actually, this paper adopts fixed shared parameters (pretrained feature extractor) and only optimizes a part of parameters of a MTL, which degrade the solution space and the solutions may not be the Pareto solutions. The Pareto front generated by such method may not be the real Pareto front for MTL.\n2. Using fixed shared parameters conflicts with the essence Pareto MTL. In MTL, the tasks mutually regularized by the feature extractor, which improve the generalization ability of each task. \n\nThis paper is globally well organized and clearly written. However, some important details are missing.\n1. The details about the hypernetwork are unclear.\n2. The paper lacks of analysis on the experimental result.\n3. Some notations are not clear, e.g., does the loss used in the paper denotes empirical loss?", "extra_scores": {}, "extra_text": {}}, {"review_id": "KzBKK-evLe", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a controllable Pareto multi-task learning model by generating the Pareto-stationary solutions.\n\nEven though the idea to generate a Pareto-stationary solution seems interesting, the proposed method is overstated. It is well known that the MGDA can find Pareto-stationary solutions, which however are NOT Pareto-optimal solutions. The steepest gradient descent method to solve problem (10) is just the primal form of the MGDA method and hence it cannot find the Pareto-optimal solution. To the best of my knowledge, there is no method which can guarantee to find a Pareto-optimal solution for general multi-objective optimization problems. Authors claim that their method can generate Pareto-optimal solutions but actually they can generate only Pareto-stationary solutions at most. Authors confuse the Pareto-optimum with Pareto-stationary solutions and this is misleading. In this sense, the proposed method is not so appealing as Pareto-stationary solutions are easy to obtain in many methods.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper first aims to propose a new controllable Pareto multi-task learning framework to find pareto-optimal solutions. But after the revision according the comments, the paper claims to find finite Pareto stationary solutions. But the paper still can not prove their proposed method can find the Pareto stationary solutions. Even if they can find the  Pareto stationary solutions, they can not guarantee find the pareto front which is conflict with the experiments and claims. There are major flaws in the paper.
Decision: Reject