Multi-Objective Deep Learning with Adaptive Reference Vectors
Authors: Weiyu Chen
James Kwok
Venue: NeurIPS 2022 Accept (2022)
Keywords: Mutli-Task Learning
Multi-Objective Optimization

Abstract: Many deep learning models involve optimizing multiple objectives. Since objectives are often conflicting, we aim to get diverse and representative trade-off solutions among these objectives. Gradient-based multi-objective optimization (MOO) algorithms using reference vectors have shown promising performance. However, they may still produce undesirable solutions due to mismatch between the pre-specified reference vectors and the problem's underlying Pareto front. In this paper, we propose a novel gradient-based MOO algorithm with adaptive reference vectors. We formulate reference vector adaption as a bilevel optimization problem, and solve it with an efficient solver. Theoretical convergence analysis is also provided. Experiments on an extensive set of learning scenarios demonstrate the superiority of the proposed algorithm over the state-of-the-art.

Multi-Objective Deep Learning with Adaptive Reference Vectors
Weiyu Chen James T. Kwok
Department of Computer Science and Engineering
The Hong Kong University of Science and Technology Hong Kong
{wchenbx, jamesk) ust.hk
Abstract
Many deep learning models involve optimizing multiple objectives. Since ob- jectives are often conflicting, we aim to get diverse and representative trade-off solutions among these objectives. Gradient-based multi-objective optimization (MOO) algorithms using reference vectors have shown promising performance. However, they may still produce undesirable solutions due to mismatch between the pre-specified reference vectors and the problem's 's underlying Pareto front. this paper, propose novel gradient-based MOO algorithm with adaptive ref- erence vectors. We formulate reference vector adaption as a bilevel optimization problem, and solve it with an efficient solver. Theoretical convergence analysis is also provided. Experiments on an extensive set of learning scenarios demonstrate the superiority of the proposed algorithm over the sute-of-the-and
Introduction
Deep learning models often evaluated under multiple, potentially conflicting, criteria. For example, in multi-task learning [6], a single model is required to perform well on multiple tasks. In some scenarios, besides accuracy, model fairness is also important biased against gender and race. These problems can all be formulated as multi-objective optimization (MO0) [39] problems, and have attracted attention from various fields such energy resource optimization [9] and signal processing [4].
ensure that the model is not
Since the multiple objectives usually cannot be optimized simultaneously by single solution, the goal of MOO to find a set of solutions with different trade-offs to approximate the true Pareto front (PF). The past decades have witnessed the birth of a large number of gradient-fres MOO algorithms, such as evolutionary multi-objective optimization algorithms [11, 52] and Bayesian multi multi-objective algorithms [26, 3]. These algorithms perform well on small-scale problems but fail to provide useful solutions when facing the huge number of parameters deep learning models.
Recently, gradient-based MOO algorithms [14] have demonstrated promising performance in deep learning. Following the pioneering work in [45], Lin al. [31] and Mahapatra al. [37] propose to use set of reference vectors (RV), and generate multiple solutions on the PF each of which closest to an RV. Later, strategies are further proposed to improve the efficiency by training a hypernetwork [40, 30] refereeceeveectorccodiii
network [44]. However, note that set of uniformly distributed RVs may not lead to a set of uniformly distributed solutions. Hence, an important limitation of these algorithms is that solutions generated using fixed RVs may not cover some parts the PF, thus failing to provide enough information about the PF. As an illustration, in Figure la, solutions obtained by the fixed RVs are close to the ends of the PF. If the RVs are properly positioned, is possible that the obtained solution set can uniformly cover the PF (Figure 1b).
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Since it is impossible to know the true PF of the problem before optimization, how to generate proper set of RVs is big challenge. In this paper, we propose to learn the set of RVs simultaneously with the model parameters. This is formulated as a bilevel optimization problem [13], in which the lower-level optimization problem obtains the Pareto-optimal solutions with given set RVs, while the upper-level problem optimizes the RVs based on a quality measure. Moreover, we solve this optimization problem with an inexpensive solver, while still showing theoretically nice con- vergence properties. Experiments set learning scenarios demonstrate the superi- ority of the proposed algorithm.
fa
f
(b) Adaptive RVs. Figure I: Illustration of the difference between fixed and adaptive reference vectors. The dashed lines reference vectors, the blue curve is the PF, and the points are solutions on the PF closest to each reference vector.
(a) Fixed RVs.
extensive
Notations. In this paper, [m]] denotes the set .I..... m. and RT = {u E R bu 2 0, Vi E [m]]. For two vectors u, 0 E IRm, u 2 v means that uy 2 v for [m]] denotes the Euclidean norm for vectors and the Frobenius norm for matrix.
2 Related Work: Multi-Ohjective Optimization (MOO)
In multi-obeectivv optimization (MOO) [39], one aims to minimize m 2 objectives (Ji()), Sm (o)), equivalently, the vector-valued function:
min f(o)= (ff fm (#)]
R..
solution 61 is dominated by another solution Φ2 and only fa(01) f((d2) for [m]. and Bi E [m]. f:(11) > f(((2)) A solution 6² is Pareto-optimal if and only if it is not dominated by any other σ'. Pareto front (PF) the set of multi-objective values of all Pareto-optimal solutions. A PF is regular [48] if it is simplex-like (i.e., all vectors with positive directions intersect it when they start at the origin), and irregular otherwise. is shown that regular PF's are not very realistic [24].
Since the number of Pareto-optimal solutions is usually large or even infinite, a set of n Pareto- optimal solutions & [dl..... (n.. is often used approximate the PF. Let the corresponding multi-objective values be F = {f((1),.... f(ox)}. f(o)) The quality of 4 can be evaluated from two perspectives; convergence and diversity [27]. Convergence refers to the distance between F and the true PF, while diversity measures whether the solutions are well-distributed in the space of objectives. popular measure, which evaluates both convergence and diversity, the hypervolume (HV) [53]. Given reference point E IRm", the HV of F is:
MV(F;2) ((,,,
where [z = 1q E R' S 2 2), and A(-) is the Lebesgue measure of set.
reference vector (RV) [52], sometimes called the weight vector or preference vector, can be used to guide the optimization algorithm by indicating the preferred point on the PF. Usually, an algorithm is expected to obtain the Pareto-optimal solution closest the given RV the objective space. Given RV RT, the MOO problem in (1) can be converted to a single-objective problem by using scalarization function s(o; r). The most straightforward construction is linear scalarization:
s(b;r) = Σ (())f(())
(3)
1=1
which weights the m objectives with the elements r(j)'s 's of r. is known that the minimizer & of s(o;r) is also Pareto-optimal for the original problem in (1) (Ch 4.7, [5]). However, the vector of objectives fØ) obtained may be far away from the given RV. To encourage them to be closer to this vector, penalty term can be added to the linear scalarization function, leading to [44]:
T f(()
s(O:T) Σ (o)s(()) + |rl [[())]]
(4)

---

where y > 0 is a constant. Given a (discrete) set of solutions, Ma et al. [36] generate exploration directions to spawn new solutions on the PF, leading to a continuous PF.
set of solutions can be obtained to approximate the entire PF. Usually, R is simply set to R However, sometimes the decision-makee may only be
By using n RVs from a subset R. C IR
interested in a specific region of the PF [2]. For example, we may limit the angle between any RV and each coordinate axis. R can then be changed to:
r. u4
(r fri.....aa) COS P22 S
||rl|| ||u/]| S cos p1, Vi [m],j E [m]
(5)
where uy is the ith coordinate axis and (P,, are the maximum and minimum allowable angles, respectively. However, note that the set of pre-specified RVs may not fit the problem's PF (e.g., some of them may not intersect the underlying PF), leading to an undesirable solution distribution.
Gradient-Free MOO. Evolutionary MOO algorithms (c.g., NSGA-II[III, MOEA/D[52]) and Bayesian MOO algorithms (e.g.. BMOA[26], USeMO[3]) are widely used for small-scale black-box problems. These algorithms assume that gradient information is not available. Hence, they often fail to converge on deep learning problems where the solution space can be very large.
Gradient-Based MOO. Gradient-based MOO algorithms more efficient when problems have differentiable objectives and large number of parameters. Sener and Koltun [45] propose to apply Multiple-Gradien Descent Algorithm (MGDA) [14] to multi-task learning. Liu and Vicente [33] provide theoretical analysis of stochastic MGDA. MGDA can be further extended to incorporate RVs (c.g.. EPO [37, 38] and Pareto-MIT [31]). Some algorithms (e.g.. HOO-SVGD [34], HIGA [49]) can output a solution set without using RVs. However, they optimize several neural networks simultaneously, and are computationally expensive and need large GPU memory (especially when the neural network is large).
Proposed Algorithm
As mentioned in Section 1, the fixed in undesirable solution distributions. Instead of using fixed set of n RVs (R = [r....., .......... F'. E R), we propose to adapt them so that the resultant solution set is well-distributed. In Section 3.1, we first introduce a reference vector centoo-condiiooeee neural network so that reference vectors can be easily handled without using a lot more parameters, Section 3.2 then formulates reference vector adaptation as bilevel optimization problem, and an efficient solver is presented in Section 3.3. Its convergence properties are then studied in Section 3.4
RVs used in common practice may result
3.1 Reference Vendor-Conditioned Neural Network
In deep learning models, ó corresponds to the network parameters. Optimizing & = {d1..... on means optimizing n neural networks (as EPO [37] and MOO-SVGD [34]), which is highly inefficient. To alleviate this problem in deep MOO algorithms, Navon et al. [40] and Lin al. [30] propose to train a single hypernetwork [22] that can output neural network parameters based on the RV. In particular, the Pareto Hypernetwork (PHN) in [40] proposes two ways optimize the hypernetwork: (i) PHN-LS, which uses linear scalarization, and (ii) PHN-EPO, which uses EPO. However, the hypernetwork still incurs significant computational overhead. For example, the hypernetwork in [40] is around 100 times larger than the base neural network. In this paper, we use the more efficient conditioned network [15, 44]. Specifically, COSMOS [44] concatenates the RV and data sample, and treats this as a joint input to the network. On the other hand, YOTO [15] originally developed for use with a family of parameterized loss functions, in which the loss parameter is incorporated with the sample into the network via FilM layers [42]. It is theoretically shown that YOTO as powerful using neural networks [15]. In this work, we adapt the YOTO architecture by replacing the loss parameter with the RV. In this way, the proposed model only has small parameter overhead compared to the single deep network.
3.2 Reference Vector Adaption via Bilevel Optimization
Consider an RV-conditioned neural network (d) r) with parameter & and RV r as input. With set of RVs R. [ri...., Tn], the multi-objective values of a solution set can be written as
3

---

(f(d:ri)..... T.)]. We use a function Q--) to measure its quality. Two choices are con- sidered this paper. The first one encourages the (o):r:)'s be far away from each other (and thus more uniformly distributed in the space of objectíves):
1 (- ((ds;))))))
QR, = Σ exp
f,1=1
where h is constant. The second one is:
QR HV [(o (((ssrn))) ))));
(7)
which encourages the maximization of HV in (2). Note that the HV-optimal solution is usually not uniformly distributed [21,47].
To obtain the set of RVs R that generates o, we formulate it as bilevel optimization problem [13]. Recently, bilevel optimization has gained great popularity in many machine learning problems such
meta-lcarning [18], neural architecture search [32], and hyperparameter optimization [19]. We consider the following bilevel optimization problem:
mingeR s.t.
o(R,
(8)
(² arg min S(R, o).
(9)
where Q ((R, =\t -Q(R, (R), S(R, x) = Σ"= r();r;), and s(0rr) is the scalarization function in (3) or (4). 1 The lower-level optimization problem (9) obtains the Parcto-optimal solutions with the given set of RVs R, while the upper-level optimization problem (8) optimizes R to maximize the corresponding solution quality.
The idea of RV adaption is also used in some evolutionary algorithms [43, 29]. However, they update the RVs using information from the current population and archive, and cannot be directly used in gradient-based MOO algorithms. Moreover, they cannot scale to problems with a large number of parameters, as is typically the case in deep learning.
3.3 Solving the Bilevel Optimization Problem
There are various bilevel optimization solvers for (8). Many of them involve propagation through the inner loop [18, 20], which has large computational overhead compared to [44]. The proposed algo- rithm, which will be called Gradient- based Multi-Obe Majti-Objeectve Optimization with Adaptive Reference vectors (GMOOAR),
shown in Algorithm step in both the inner and outer loops, as in FO-MAML [18] and DARTS [32]. In each iteration k mini-batches & and गy are randomly sampled from the data and then used to estimate the stochastic gradients V Rg, <b:: Ex) and V Q Rg. ox+11 1; Tk),, respectively. projz(-) (-) is the Fuclidean pro- jection operator onto R that ensures that RVs are inside R. In the sequel, the algorithm using the uniformity-related quality function (6) will be denoted GMOOAR-U, while the one using HV in (7) will be denoted GMOOAR-HV. The gradient of (6) can be computed directly from the computation graph, while the gradient of (7) can be computed using an efficient dimension-sweeping algorithm [17].
performs only one stochastic gradient descent
The proposed algorithm has O(w + m) memory and O(w + m) time complexity per iteration, where the dimension of Φ. Since m < w in most cases. has comparable time and space complexity with COSMOS, which is Oww). In Section 3.4, we will show that this simple solver can still provide theoretical guarantees on its obtained solution.
Algorithm 1 Gradient- Multi-Objective Optimization with Adaptive Reference vectors (GMOOAR).
Input: learnable RVs Rt, learning rates {ak, 3e3, initial parameter 6, number of iterations K.
1: for k k= to K do
2: sample a mini-batch E of samples;
3: Φk+1
3g V&S(RA dvi Ex); /* optimize network parameters #/
4: sample a mini-batch Ta of samples;
5: R4+1 t proj (Rg a V (RR )x+1i /* optimize reference vectors */
6: end for
As the RV-conditioned network here, fo) (resp. f())) in (3). (4) becomes fs(d;r) (resp. f(drr)).

---

3.4 Convergence
In this section, we provide convergence analysis for Algorithm As in [23], we make the following assumptions on S(R, Φ) and Q(R, +).
Assumption 1. (i) S(R, Φ) is twice-differentiable in (R, (ä) VoS(R, Φ). RO Φ), (2, Φ). V Φ), and (R, 6) are Lipschitz continuous w.E. & with constants Lg Lg 122.2 Lig.,, and 2, respectively. (iii) S ( Φ). S $). and V VoQ(R, u) are Lipschitz continuous w.r.. R with constants Ls,3, and Lg, and Lgg. respectively, (iv) SRR o) is pg-strongly convex in 6. (v) S +)1l S C, and Q +)ll S Cg
Given o² (R), the gradient of the upper-level objectivv u R = QR, (R)) can be obtained as Vu(R) VRQ(R, ( (R) S(R (R)) SRR, x0 (R))
o" (R)), (10)
where V VRQ(RR x0 (R)) is the direct gradient with respect to R. In [23]. (u(R) is evaluated using a surrogate constructed by replacing o² (R) in (10) with 0R+1:
V RRQ(Rs. 1 = V R Q RR, 0x+1)
S Ox+ IV Si Ok+
+
(11)
However, this involves computing the Hessian and is expensive. On the other hand, the proposed
algorithm uses h = V Q #x+1; Tk)), which is the stochastic estimate of V R Rg, 0x+1). Note
that V R RR, oR+1) is the first-order approximation of the gradient in (11). Such an approximation is also used in [18, 32). It greatly reduces the time and space complexities, but leads to a bias that can
be bounded by a constant:
V VVRQRRR 0R+1) - Ezy (151l
6x I C.Cog/ps.
1)l 2g S(R,, 0x+ RQQRRRRRRR. 2x+ 1)ll
Next, we also make the following assumption similar to [23]. Let h = V&S(RK, Ex: Assumption 2. For any l 2 O, there exist constants J. Ta. and by such that:
Eg, [M-]] = Va.((R 2b), E [M = V RQQ(RA
Ru. ( + Bx, || $ by:
Es, h VoS(RK 2x 11 < o2
Ex [1 Bx VQ(RK, 0x+1) o
Let A = R., lb 35 (Ra-1)l the expected gap between de and the optimal network parameter given reference vectors Ra Similarly, denote the expected gap between R2 and the optimal reference vectors He in (8) by A = 11 []
Theorem 1. Assume further that u(R) pa-strongly convex, and the step sizes (ay, Bx) satisfy
04 S min
83/2
3x-1 <
1
3x/t/88
ak-1 & 11 300/44/
(12a)
Myg
3x
B. < min
c10kk
2/3
1
U4
8pA00r S 3uBk, vk 2 0.
(12b)
pbs
I
02)
48c
L.
where I., Tug are constants and co, 01 are free parameters. For any 2 1, the iterates generated by Algorithm I satisfy
k-1
I
C1 2
1 02
b2
A
II
(1)-a)
1 +
A
2/3 a--
2
113
q
[k-1
o2
og
T2
4
II (1
(1-B/4//44
4
+
+
0
2
3k
(13)
11
where S denotes that numerical constants omitted.
5

---

With diminishing step sizes ag; Ca/ ka), Bk = =ca/(k+ +kk))//2 where
Ki max {
35
Ls
3 (1 + of) (512) L2
Ca
8
KB
Ni
CB
32
14g
3/4g
4
3/1²
the following Corollary shows that the assumptions (12a) and (12b) in Theorem I are satisfied, and thus A converge to 0, i... oE converges to the optimal solution o² (Rx=1)).
Corollary 1. &n converges to the optimal x- (Ba-1) (9).
Experiments
In this section, extensive experiments are performed, including synthetic problems (Section 4.1), multi-task learning (Section 4.2), accuracy cecuracy-faireess trade-off (Section 4.3), and usage on larger networks (Section 4.4). Finally, ablation study is presented in Section 4.5, All experiments are conducted on an RTX-2080Ti with 1 IIGB memory.
4.1 Synthetic Problems
Experiments are performed on four commonly used multi-objective benchmark problems [11, 52, 34] with different numbers of objectives: (i) 2-objective DTLZ2 [12], (ii) 3-objective DTLZ2 [12]. (iii)
2-objective scaled-DTLZ2 [12], and (iv) 3-objective MaF1 [8]. Their detailed definitions are in Appendix B.1. The PFs of problems (i) and (ii) are regular, while those of (iii) and (iv) are irregular.
The number of inputs is set 30. We aim to get 15 non-dominated solutions for each 2-objective problem, and 36 non-dominated solutions for each 3-objective problem.
The proposed algorithm² (GMOOAR-U - using quality function (6) and GMOOAR-HV using quality function (7)) compared with the state-of-the-art COSMOS [44],which uses fixed reference vectors. For the 2-objective problems, reference vectors for COSMOS are generated by following their strategy in [44]. For the 3-objective problems, we generate reference vectors for COSMOS by the method in [10], which can obtain more uniform reference vectors. For GMOOAR, the reference vectors are initialized randomly, As in [40], neural network (with? hidden layers, each with 20 units) is used. More experimental details can be found in Appendix B.2.
Figure shows the solutions and HV values obtained. For the 2-objective and 3-objective DTLZ2 (Figures 2a and 2b), the solutions obtained by COSMOS are relatively uniform. However, on the scaled-DTLZ2 (Figure 2c) and MaF1 (Figure 2d) with irregular PFs, many of its solutions are near the boundary and close to each other. In contrast, solutions obtained by GMOOAR-U - are distributed more uniformly. Solutions obtained by GMOOAR-HV are not uniform, but have higher HV than COSMOS and GMOOAR-U.
Figure 3 shows the solution sets obtained by GMOOAR-U on the 3-objective DTLZ2 when the region of interest is a subspace R constrained as in (5). Note that imposing this constraint on COSMOS difficult as (i) it hard to generate a set of uniformly distributed RVs; and (ii) even with set of uniformly distributed RVs, they may not lead to uniformly distributed set of solutions.
4.2 Multi-Task Learning
In this experiment, we three benchmark datasets from [31]] Multi-MNIST, Multi-Fashion, and Multi- Multti-Fashion-MNNIT. In Multi-MNIST, each image is constructed by putting two different MNIST images together, one the bottom-right (BR) and the other at the top-left (TL). Similarly, Multi-Fashion images are constructed by combining images from FashionMNISTT [50]. while Multi- FaSSion+MNIST images are constructed by combining one MNIST image with FashionMNIST image. More details can be found [31]. J. The goal is to classify both the BR and Tl. images correctly, by minimizing the two cross-entropy losses using single neural network. As in [31, 37, 44),
the LeNet [28] with multi-head as base network. Details of the referencee-vectorccodiii
network
in Appendix B.3.
We compare the proposed algorithms (GMOOAR-U and GMOOAR-HV) with (i) EPO [37]. (ii) Pareto hypernetworks (PHN-LS and PHN-EPO) [40], (iii) MOO-SVGD [34], and (iv) COSMOS
We use linear scalarization with penalty term in (4).

---

COSMOS
GMCOKA-1
GMOOVRIF
04
66-
HV-2.935
HV-3.100
HV=3.123
02-
8.
33
60
33
22
60
24
24
64
64
04
(a) 2-objective DTLZ2.
GMOOX341
GMOOVP-IN
201234
NW
746
HV-7.004
299
(b) 3-objective DTLZ2.
0039055
UNGGALU
UMOOAK
1..
10-
22-
08
HV-2.470
HV-2.468
HV-2.489
32-
02
100
00
(c) 2-objective caaeddDTLL..
156
(d) 3-objective MaF1.
Figure 2: Solution sets (red) and HV values obtained on the synthetic datasets. The Pareto-optimal solutions is in blue.
84
64
54
(a) (P1 0, 602
4/2.
(b)
7/10, 3m/ 100 (c)
3x/ 31/10, p2 n/2
Figure 3: Solution sets obtained by GMOOAR-U U with different (191, P22) settings.
[44]. For EPO, PHN-LS, PHN-EPO and COSMOS, generate reference vectors following the strategy in [44]. For GMOOAR, the reference vectors are initialized randomly. The experiment is repeated 10 times with different random seeds.
in all Following common practice [44], we obtain a set solutions in each iteration (with experiments). They are evaluated on the validation set every epochs. We only keep the solutions of iteration kbest as the final solution set, where keest the iteration that yields the solution set with the largest validation HV. Note that the original implementation of MOO-SVGD (obtained from the

---

0.5
EPO PHNT-spO
6575
0.65
PHRLL
0.33
225
0.20
2.25
2.40
0.43
0.23
0.50
0.25
2,50
0.70
0.2
6.3
0.5
0.6
Josu for Tuak
LAG
Task TL
Low
(a) Multi-MNIST.
(b) Multi-Pashion.
(c) Mulli-FashonnMMNiSTT
Figure 4: BR and TL test losses obtained the multi learning datasets.
HV
# parameters
runtime (s)
Multi-MNIST
Multi-Fashion
Fashion-MNIST
EPO [37]
2.95:10.02
2.31
2.86=0.02
478,650
8,370
PHN-EPO [40]
2.82:10.04
2.16:10.55
2.74:1.055
3,243,410
1,515
PHN-LS [40]
2.79100.4
2.141:.04
2.67=10.06
3,243,410
644
MOO-SVGD [34]
2.67+0002
2.02=0.02
2.54=0.04
478,650
8,661
COSMOS [44]
2.95
2.31=0.03
2.82=#0.33
43,058
260
GMOOAR-U
3.02:10001
2.13
2.91
43,685
288
GMOOAR-HY -V
3.02
2.33
2.92=0.02
43,685
348
Table I: Average HV, number parameters and runtime on the multi-task learning datasets.
authors) stores all non-dominated solutions of each iteration in archive A. On termination, they try all size-n subsets of and select the subset with the largest HV on the validation set as the final solution set. As there are C/AAI such subsets and |A| is large when MOO-SVGD terminates, this can very expensive. In order be fair all algorithms being compared, we thus also use the aforementioned commonly practiced strategy on MOO-SVGD.
As in [7], Figure 4 shows the testing performance obtained by the solution with median HV (over the 10 runs). As can be seen, on Multi-MNIST and Multi-Fashion, the solution sets obtained by COSMOS are dense in the middle but sparse towards the ends, while the solution sets obtained by GMOOAR-U are more uniform. Moreover, compared to all other baselines, the solution sets obtained by GMOOAR are closer the bottom-right corner where the underlying true PF resides. For MOO-SVGD, many of its obtained solutions are much inferior, and only one of them is in the range shown in Figure 4. complete plot of all the MOO-SVGD solution sets is Appendix C.
Table shows the HV's of the solution sets (averaged over the 10 runs), the number of parameters and runtime. As be seen, GMOOAR has consistently higher HVs than the other baselines. Moreover, its number of parameters is comparable to that of COSMOS, and is much fewer than the other baselines. Compared to the base network LeNet, GMOOAR has only 37% of the runtime, GMOOAR is only slightly slower than COSMOS and is much faster than the other baselines. GMOOAR AH- is slower than GMOOAR-U U as the computations of HV and its gradient are more expensive [53].
parameters. In terms
4.3 Accuracy-Fairnes Trade-off'
In this experiment, follow [44] and aim to achieve both high accuracy and fairness three tabular datasets: Adult [16], Compass [1], and Default [51]. The accuracy measured by the cross-entropy loss, while fairness is measured by a hyperbolic tangent relaxation of the Difference of Equality of Opportunity (DEO) [41]. As in [40, 44]. a 2-hidden-layer multilayer perceptron is used as the base network. More details can be found in Appendix B.4. The experiment is repeated 10 times.
Figure shows the test loss and fairness measure obtained by the solution set with median HV over the 10 runs. Since the datasets are not difficult, the approximated PFs obtained by various algorithms are close. Solutions obtained by GMOOAR are uniformly distributed, while those obtained by EPO, PHN-LS, PHN-EPO and COSMOS are very dense in the top-left region. Solutions obtained by

---

2.25
EYO
6.12
3.10
0.012
PAR-C
PUBLLS
0:30
MDD SVGD
COSNOS
3.20
0.06
9.35
0.000
CNDDAX-IV
2.10
3.05
5.00
9.50
0.003
2.55
043
550
35
2.7
0.8
0.0
Cross
Sinayy Dessenimpyy
Srray Croee- Loos
(a) Adult.
(b) Compass.
(c) Default.
Figure 5: Test losses and faimess measures obtained on the faimess datasets.
MOO-SVGD are clustered and only cover parts of the PF. Table 2 shows the HV values obtained. can be seen that GMOOAR achieves better HVs than the baselines.
Adult
Compass
Default
ΕРO[37]
3.342++0.001
3.7092
3.119++0.001
PHN-EPO [40]
3.340=+0.006
3.709+00000
3.111=00.005
PHN-LS [40]
3.341#0.000
3.698
3.121
MOO-SVGD [34]
3.330:10.008
3.716110.011
3.110::0.00:
COSMOS [44]
3.336:10.006
3.7104100004
3.114::0.005
GMOOAR-U
3.344::0004
3.719
3.023:0.0004
GMOOAR-HV
3.345=00005
3.714=10.008
3.123
Table 2: Average HV and standard deviation the fairness datasets.
4.4 Larger Networks
To demonstrate that the proposed method can be used on larger networks, we apply GMOOAR on the EfficientNet-B4 [46] with about 17 million parameters. Following [44], we perform experiments on two easy tasks ("Goatee" and "Mustache") and two hard tasks ("Oval Face" and "Pointy Nose") selected from the 40 tasks CelebA [35]. Since EfficientNet-B4 is around 400 times larger than the LeNet used in previous experiments, PHN-LS, PHN-EPO, and MOO-SVGD cannot be run on our machine. For performance evaluation, the testing cross-entropy loss of each selected task is used. The experiment is repeated 5 times with different random seeds.
Figure 6 shows the test losses obtained by the solution set with median HV over the 5 runs. The corresponding HV values are shown in Table 3. On CelebA-Fasy, both GMOOAR-U and GMOOAR- HV outperform COSMOS in terms of HV. On CelebA-Hart all three algorithms achieve similar HVs, though that of GMOOAR-HV slightly better.
COS
COSNOS
50
GMOGAR-U GMCOAR-IN
GNNDOR-U CNDGAR-II
0.75
40
0.63
tuntime (h)
5.36
COSMOS
0.62
30
GMOOAR-U
2.22
GMODAR-HV
5.10
3.8
0.59
10
0.0
s12
als
0.575
0.875
(a) CelebA-Easy
(b) CelebA-Hard.
number of tasks
Figure 7:
Average runtime on CelebA with different numbers of tasks.
Figure 6: Test losses on the two easy tasks (left) and two hard tasks (right) of CelebA.
Next, we select more tasks from CelebA. Table 4 shows the average HV over 3 runs with different random seeds on CelebA with 3, 5, 10, and 15 tasks. Details of the selected tasks are Appendix B.4. As can be seen, the number of tasks increases, GMOOAR exhibits more significant performance

---

CelebA-Easy
CelebA-Har
COSMOS [44] GMOOAR-U GMOOAR-HI
3.700:10.003
2.217=:0002
3.710
2.217=0.002
3.711
2.222:10.006
Table 3: Average HV and standard deviation of solution sets obtained on CelebA with 2 tasks.
gains compared to COSMOS. In particular, GMOOAR-HV always achieves the highest HV, which is then closely followed by GMOOAR-U. However, the computation HV is NP-hard with respect to the number of objectives [53]. Hence, as can be seen from Figure 7, the runtime of GMOOAR-HH grows much faster with the number of tasks than COSMOS and GMOOAR-U On the other hand, GMOOAR- R is very scalable and its running time changes little with the number of tasks.
tasks
5 tasks
10 tasks
15 tasks
COSMOS [44] GMOOAR-U
6.925 + 0.005
11.56+0.009
118.1 + 1.0
1805+:52
6.953 + 0.004
15.1910.18
215.0 - 6.1
4057144
GMOOAR-HV
6.967 + 0.007
0..34
217.8 = 1.5
Table 4: Average HV and standard deviation of solution sets obtained on CelebA with 3, 10, and 15 tasks. GMOOAR-HV cannot be on 15 tasks as it takes more than month machine.
4.5 Ablation Study
In this experiment, we study the effects of the RV learning rate a and bandwidth à in (6) on the performance of GMOOAR-U, We use the same setting in Section 4.3. The experiment is repeated 10 times with different random seeds.
Figure 8a shows the variations of HV with (h fixed to 0.01). As can be seen, an too small results in almost no RV adaption and thus poor performance, while an too large may lead to unstable learning. Figure 8b shows the variation of HV with (a is fixed to 0.005). When á is too small (resp. too large), Q Φ) is close to zero (resp. n) and the gradient vanishes, making learning difficult.
3.70
3.70
1.58
3.64
3.54
362
(a) RV lcarning
(b) Bandwith h.
Figure 8: Averge HV and 95% confidence interval with different a's and h's 's on GMOOAR-U using the Compass dataset.
Conclusion
In this paper, we present a novel gradient- basde MOO algorithm with adaptive RVs. The proposed algorithm efficiently adapt the RVs during optimization and provide diverse solutions with a small overhead compared to single-objective optimization. Experiments show the ability of the proposed strategies to obtain well-distributed solutions based the specified quality function. In the future, we will consider the incorporation of [36] (to obtain continuous PF) and other state-of-the-art MOO algorithms.
Acknowledgements
This research was supported in part by the Research Grants Council of the Hong Kong Special Administrative Region (Grant 16200021).
10

---

References
[1] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. In Ethics of Data and Analytics, pages 254-264. Auerbach Publications, 2016.
[2] Slim Bechikh, Marouane Kessentini, Lamjed Ben Said, and Khaled Ghédira. Preference incorporation in evolutionary multiobjective optimization: A survey of the state-of-the-art, In Advances in Computers, volume 98, pages 141-207. 2015.
[3] Syrine Belakaria, Aryan Deshwal, Nitthilan Kannappan Jayakodi, and Janardhan Rao Doppa. Uncertainty-awant search framework for multi-objective Bayesian optimization. In AAAI Conference Artificial Intelligence, pages 10044-10052, 2020.
[4] Emil Bjorsson, Eduard Axel Jorswieck, Mérouane Debbah, and Bjorn Ottersten. Multiobjective signal processing optimization: The way to balance conflicting metrics in 5G systems. IEEE Signal Processing Magazine, 31(6): 10::::4-22 14-23, 23, 2014.
[5] Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
[6] Rich Caruana. Multitask learning, Machine Learning, 28(1):41-75, 1997.
[7] Ran Cheng, Yaochu Jin, Markus Olhofer, and Bernhard Sendhoff. A reference vector guided evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, 20(5):773-791, 2016.
[8] Ran Cheng, Miqing Li, Ye Tian, Xingyi Zhang, Shengxiang Yang, Yaochu Jin, and Xin Yao. benchmark test suite for evolutionary many-objective optimization. Complex & Intelligent Systems, 3(1):67-81, 2017.
[9] Yunfei Cui, Zhiqiang Geng. Qunxiong Zhu, and Yongming Han. Multi-objective optimization methods and application in energy saving. Energy, 15:661-704, 2017.
[10] I Das and JE Dennis. Normal-boundass intersection: A new method for generating Pareto- optimal points in multieriteria optimization problems. SIAM Journal on Optimization, 1996.
[11] K. Deb, Pratap. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-IL. IEEE Transactions on Evolutionary Computation, 612):182-197,2200
[12] Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. Scalable test problems for evolutionary multiobjective optimization. In Evolutionary Multiobjective Optimization, pages 105-145. Springer, 2005.
[13] Stephan Dempe. Foundations of Bilevel Programming. Springer Science & Business Media, 2002.
[14] Jean-Antoine Désidéri. Multiple-gradient descent algorithm (MGDA) for multiobjective opti- mization. Comptes Rendus Mathematique, 350(5-6):3133115 2012.
[15] Alexey Dosovitskiy and Josip Djolonga. You only train once: Loss-conditional training of deep networks. International Conference on Learning Representations, 2019.
[16] Dheeru Dua and Casey Graff. UCI machine learning repository. 2017.
[17] Michael Emmerich and André Deutz. Time complexity and zeros of the hypervolume indicator gradient field. In EVOLVE A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation III, pages 169-193, Springer, 2014.
[18] Chelsea Finn, Pieter Abbecl, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In International Conference on Machine Learning, pages 1126-11 135, 2017.
[19] Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. International Conference Machine Learning, pages 1568-1577, 2018.

---

[20] Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi. and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In International Conference on Machine Learning. pages 1568-1577, 2018.
[21] Tobias Friedrich, Frank Neumann, and Christian Thyssen. Multiplicative approximations, opti- mal hypervolume distributions, and the choice of the reference point. Evolutionary Computation, 23(1):131-159 2015.
[22] David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. In International Conference on Learning Representations, 2017.
[23] Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale frame- work for bilevel optimization: Complexity analysis and application to actor-critic. Preprint 2XXiv:2007005100 2020.
[24] Hisao Ishibuchi, Linjun He, and Ke Shang. Regular Pareto front shape is not realistic. In IEEE Congress on Evolutionary Computation, pages 2034-2041, 2019.
[25] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2015.
[26] Marco Laumanns and Jiri Ocenasek. Bayesian optimization algorithms for multi-objective optimization. In International Conference on Parallel Problem Solving from Nature, pages 298-307, 2002.
[27] Marco Laumanns, Lothar Thiele, Kalyanmoy Deb, and Eckart Zitzler. Combining convergence and diversity in evolutionary multiobjective optimization. Evolutionary Computation, 10(3):263- 282, 2002.
[28] Yann LeCun, Patrick Haffner, Léon Bottou, and Yoshua Bengio. Object recognition with gradient-based based learning. In Shape, Contour and Grouping in Computer Vision, pages 319-345. Springer, 1999.
[29] Miqing Li and Xin Yao. What weights work for you? Adapting weights for any Pareto front shape in decomposition--asel evolutionary multiobjective optimisation. Evolutionary Computation, 28(2):227-233, 2020.
[30] Xi Lin, Zhiyuan Yang. Qingfu Zhang, and Sam Kwong, Controllable Pareto multi-task learning. Preprint arXiv:2010.03313. 2020.
[31] Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-task learning. In Neural Information Processing Systems, 2019.
[32] Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. In International Conference on Learning Representutions, 2018
[33] Suyun Liu and Luis Nunes Vicente. The stochastic multi-gradies algorithm for multi-objective optimization and its application to supervised machine learning, Annals of Operations Research, pages 1-30, 2021.
[34] Xingchao Liu, Xin Tong, and Qiang Liu. Profiling Pareto front with multi-objective Stein variational gradient descent. In Neural Information Processing Systems, 2021.
[35] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In International Conference on Computer Vision, pages 3730-3738, 2015.
[36] Pingchuan Ma, Tao Du, and Wojciech Matusik. Efficient continuous Pareto exploration in multi-task learning. International Conference on Machine Learning, pages 6522-6531, 2020.
[37] Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in Pareto optimization. In International Conference on Machine Learning, pages 6597-6607, 2020.
[38] Debabrata Mahapatra and Vaibhav Rajan. Exact Pareto optimal search for multi-task learning: Touring the Pareto front. Preprint htt.:2208.005997 2021.
12

---

[39] Kaisa Miettinen. Nonlinear Multiobjective Optimization, volume 12. Springer Science & Business Media, 2012.
[40] Aviv Navon, Aviv Shamsian, Ethan Fetaya, and Gal Chechik. Learning the Pareto front with hypernetworks. In International Conference on Learning Representations, 2020.
[41] Kirtan Padh, Diego Antognini, Emma Lejal- Boi Faltings, and Claudiu Musat. Address- ing fairness in classification with a model- aodel-agnostic multi-objective algorithm. In Uncertainty in Artificial Intelligence, pages 600-609, 2021.
[42] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. FiLM: Visual reasoning with a general conditioning layer. In AAAI Conference on Artificial Intelligence, 2018.
[43] Yutao Qi, Xiaoliang Ma, Fang Liu, Licheng Jiao, Jianyong Sun, and Jianshe Wu. MOEA/D with adaptive weight adjustment. Evolutionary Computation, 22(2):231-266 2014.
[44] Michael Ruchte and Josif Grabocka. Scalable Pareto front approximation for deep multi- objective learning. In International Conference on Data Mining, pages 1306-1311, 2021.
[45] Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Neural Information Processing Systems, 2018.
[46] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pages 6105-6114, 14, 2019.
[47] Ryoji Tanabe and Hisao Ishibuchi. An analysis of quality indicators using approximated optimal distributions a 3-D objective space. IEEE Transactions on Evolutionary Computation, 24(5):853-86) 867, 2020.
[48] Ye Tian, Cheng He. Ran Cheng, and Xingyi Zhang. A multistage evolutionary algorithm for better diversity preservation multiobjective optimization. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 31(9):5880-5894 2019.
[49] Hao Wang, André Deutz, Thomas Bäck, and Michael Emmerich. Hypervolume indicator gradient ascent multi-objective optimization. In International Conference on Evolutionary Multi-Criterina Optimization, pages 654 669, 2017.
[50] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. Preprint arXiv: 1 708.07747,,
[51] I-Cheng Yeh and Che-hui Lien. The comparisons of data mining techniques for the predictive accuracy probability of default of credit card clients. Expert Systems with Applications, 36(2):2473-2488) 2009.
[52] Qingfu Zhang and Hui Li. Moea/d: A multiobjective evolutionary algorithm based on decom- position. IEEE Transactions on Evolutionary Computation, 116))712-733,, 2007.
[53] Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M Fonseca, and Viviane Grunert Da Fonseca. Performance assessment of multiojeective optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation, 7(2): 17-132,
13


Reviews:
[{"review_id": "szhZvq_pGvl", "reviewer": "Reviewer_bc4u", "summary": "This paper aims to obtain a set of Pareto-optimal solutions that can better uniformly cover the Pareto front by learning a set of appropriate reference vectors (RVs). The proposed method is formulated as a bilevel optimization problem where we can obtain the Pareto-optimal solution in the inner loop and optimize the RVs by maximizing the quality function in the outer loop. The convergence analysis of the proposed method is provided and empirical studies show the effectiveness of the proposed method.", "questions": "Refer to above.", "limitations": "Refer to above.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths：\n\n1. This paper aims to obtain a set of Pareto-optimal solutions that can better uniformly cover the Pareto front by learning a set of appropriate reference vectors (RVs), which is meaningful for multi-objective optimization.\n\n2. This paper is well-written and easy to follow.\n\nWeaknesses：\n\n1. The novelty of this paper is limited and it is just an ensemble of the existing works. The RV-conditioned network is adapted from [14] (as mentioned in Section 3.1). The solver (Section 3.3) of the bilevel problem in Eqs. (8)-(9) is more trivial, which is similar to MAML.  \n\n2. The description of Pareto domination in Lines 38-39 is problematic.\n\n3. How to compute the gradient w.r.t. $R$ in Eq. (7)?\n\n4. The Eq. (9) is equivalent to minimize $\\sum_{j=1}^m (\\sum_{i=1}^n r_i(j))f_j(\\phi)$ and can only obtain a Pareto-optimal solution based on a new reference vector $(\\sum_{i=1}^n r_i(1), \\cdots, \\sum_{i=1}^n r_i(m))$ rather than a set of Pareto-optimal solutions (as mentioned in Line 120). Thus, how the proposed method can obtain multiple Pareto-optimal solutions and how to guarantee those solutions can uniformly cover the PF.\n\n5. In Section 3.3 and Algorithm 1, it seems the hypergradient (i.e., the gradient of $\\phi_{k+1}$ w.r.t. $R$) is ignored, which may cause a large approximation error.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "qvkRxELgwK_", "reviewer": "Reviewer_efXh", "summary": "This paper propose a novel gradient-based multi-objective optimization (MOO) algorithm with adaptive reference vectors (RVs). They formulate reference vector adaption as a bilevel optimization problem, and solve it with an efficient solver. Experiments on an extensive set of learning scenarios demonstrate the superiority of the proposed algorithm over the state-of-the-art.", "questions": "I have some questions about this work:\n- As the authors claim in the Line 5 \" they may still produce undesirable solutions due to mismatch between the pre-specified reference vectors and the problem’s underlying Pareto front.\" Then why do we need to get the PF by utilizing the RVs ? I don't understand this point and hope the authors can make some explanations, since this is the major motivation of this work.\n- The authors propose to learn the set of RVs simultaneously with the model parameters and formulate the lower-level optimization problem obtains the Pareto-optimal solutions with the given set of RVs, while the upper-level problem optimizes the RVs based on some quality measure. Do the authors try the different level orders? That is the lower-level optimization problem optimizes the RVs while the upper-level optimization problem obtains the Pareto-optimal solutions. I'm just curious about that.", "limitations": "For limitations see Weaknesses and Questions. No social impact discussed. Given the effort NeurIPS has put into this effort, I believe something should be added to discuss this.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The motivation is clear and straightforward.  \n- This paper is well-written.\n- The authors provide theoretical results on the convergence of the proposed Algorithm.\n- Experiments on several learning scenarios, including synthetic problems, multi-task learning, accuracy-fairness trade-off, and usage on larger networks demonstrate the effectiveness of the method.\n\nWeaknesses:\n- The novelty of this paper is limited. They simply formulate how to generate a proper RV and the model's parameters as a bilevel optimization problem and solve it with an solver, which is limited novelty. Since the idea of RV adaption is commonly used in some evolutionary algorithms [40][26] and the bilevel optimization problem also is a common problem, the authors simply combine these two together to generate their method.\n- The solver proposed in this work (GMOOAR) to solve the Bilevel Optimization Problem seems exists some problems. I am not sure can solve the bilevel problem in this simple way, the algorithm needs to be explained more detail.\n- How the authors can guarantee the generated RVs are proper RVs?  There needs more experiments to demonstrate this point.\n- There miss some SOTA methods to compare, such as *Multi-Objective Meta Learning* (Ye et. al. , NeurIPS 2021).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XkH0SUvO7Hx", "reviewer": "Reviewer_WZqb", "summary": "The paper proposes a novel multi-objective optimization algorithm based on reference vectors (RV).\nIn contrast to prior works, the paper proposes learning these RVs using gradient-based optimization instead of using pre-specified and fixed RVs. \nThis is formulated as a bilevel optimization problem, where the lower level obtains the optimal solution given the RVs, and the upper level optimizes a quality measure w.r.t. the learned RVs (uniformity and the hypervolume indicator are considered as quality measures).  \nAn efficient solver for the bilevel optimization problem is proposed, requiring just one gradient descent step for lower and upper-level problems. A convergence analysis is also provided.\nThe algorithm is evaluated on both synthetic and real-world problems and achieves state-of-the-art performance. ", "questions": "I would appreciate it if the authors could address the weaknesses mentioned above, i.e., \n- how does the method compare in terms of efficiency with prior approaches\n- a discussion of the limitations of the method", "limitations": "A section with limitations is unfortunately missing and would definitely benefit the paper. ", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The approach to extend existing RV-based approaches with a mechanism to adaptively learn said RVs is sensible and appears to improve upon the baselines in all cases\n- The formulation as a bilevel optimization problem is elegant, and the provided solver is efficient, quite simple, and appears to work well in practice.  \n- The overall presentation is excellent. The technical presentation of the method and background material is sound and clear, and the method is also well-motivated.\n- The experiments cover a good number of benchmarks from more artificial/synthetic ones to more real-world applications. \n\nWeaknesses:\n- The experiments are most convincing on the somewhat artificial benchmarks around MNIST and FashionMNIST. The experiments might be more compelling with more real-world applications, e.g., multi-label image classification with more than two objectives. \n- Adding a comparison in terms of computational efficiency (e.g., in terms of runtime) to Table 1, for example, could be helpful for the readers. \n- A discussion of the limitations of the method appears to be missing. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Ct4_o4vEVe", "reviewer": "Reviewer_5onM", "summary": "Paper tackles multi objective optimization (MOO), and proposes a gradient based MOO using reference vectors(RVs). Previous RV based MOO algorithms use a fixed set of RVs, and they might not be able to cover some parts of the Pareto Front (PF). Authors propose to jointly learn RVs along with the solving the MOO problem by formulating it as a bilevel optimization problem. Authors present an algorithm for solving the bilevel optimization problem, and theoretical convergence analysis of the algorithm is also provided. ", "questions": "See strengths and weaknesses.", "limitations": "Authors should add some discussion about whether their algorithm can scale to large scale datasets (for e.g. large scale MTL tasks like image classification + object detection + semantic segmentation ).", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is novel and interesting. It seems to be more reasonable to learn the RVs rather than using a fixed set of RVs.\n\n1. Some more details about how the FilM conditioning is done using RVs should be added for completeness.\n\n2. Datasets in Section 4.1 and even the Multi-MNIST, Multi-Fashion and Multi-Fashion+MNIST are synthetically created. It would have been interesting to see results on more real world tasks such as the multi-objective tasks considered in YOTO [14] (image compression).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2LpMVh7g73T", "reviewer": "Reviewer_nfad", "summary": "This paper proposed a gradient-based MOO algorithm with adaptive reference vectors. The core idea is to view it as a bilevel optimization problem. Then the authors proposed an interactive solution for this bi-level optimization.", "questions": "- It is interesting to view this task as a bi-level optimization problem, but the solver seems not novel. The bi-level solving is similar with existing well-known Expectation–maximization algorithm or the way used in neural architecture search (NAS). So I have concerns about the technical novelty of this paper.\n\n- There are two assumptions proposed in this paper. In which case we should the uniform one (GMOOAR-U) rather than HV-optimal one (GMOOAR-HV)?\n\n- Another concern is the scalability of the proposed method. It can be seem from Table 1 and Table 7 that the performance gain is getting worse and worse when extending to large models. \n\n", "limitations": "The authors did not talk about limitations of their work.", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- A new way to reformulate the task to bi-level optimization \n- Better performance than state-of-the-art methods.\n\n\nWeakness:\n- It is interesting to view this task as a bi-level optimization problem, but the solver seems not novel. The bi-level solving is similar with existing well-known Expectation–maximization algorithm or the way used in neural architecture search (NAS). So I have concerns about the technical novelty of this paper.\n\n- There are two assumptions proposed in this paper. In which case we should the uniform one (GMOOAR-U) rather than HV-optimal one (GMOOAR-HV)?\n\n- Another concern is the scalability of the proposed method. It can be seem from Table 1 and Table 7 that the performance gain is getting worse and worse when extending to large models. \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept