Enhancing Meta Learning via Multi-Objective Soft Improvement Functions
Authors: Runsheng Yu
Weiyu Chen
Xinrun Wang
James Kwok
Venue: ICLR 2023 poster (2023)
Keywords: Meta Learning
Multi-Objective Optimization

Abstract: Meta-learning tries to leverage information from similar learning tasks. In the commonly-used bilevel optimization formulation, the shared parameter is learned in the outer loop by minimizing the average loss over all tasks. However, the converged solution may be comprised in that it only focuses on optimizing on a small subset of tasks. To alleviate this problem, we consider meta-learning as a multi-objective optimization (MOO) problem, in which each task is an objective. However, existing MOO solvers need to access all the objectives’ gradients in each iteration, and cannot scale to the huge number of tasks in typical meta-learning settings. To alleviate this problem, we propose a scalable gradient-based solver with the use of mini-batch. We provide theoretical guarantees on the Pareto optimality or Pareto stationarity of the converged solution. Empirical studies on various machine learning settings demonstrate that the proposed method is efficient, and achieves better performance than the baselines, particularly on improving the performance of the poorly-performing tasks and thus alleviating the compromising phenomenon.

Published as a conference paper at ICLR 2023
ENHANCING META LEARNING VIA MULTI-OBIECTIVE SOFT IMPROVEMENT FUNCTIONS
Runsheng Yu', Weiyu Chen', Xinrun Wang", James T. Kwok'
DDpartment of Computer Science and Engineering, Hong Kong University of Science and Technology Schooo of Computer Science and Engineering, Nanyang Technological University runshengyu@gma il. com, wchenbxécse. usk xinrun. waageaangeeenn nt.. edu. sg, jamoske csc. ust.hk
ABSTRACT
Meta-learning tries to leverage information from similar learning tasks. In the commonly-used bilevel optimization formulation, the shared parameter learned in the outer loop by minimizing the average loss over all tasks. However, the converged solution may be compromised in that only focuses on optimizing on a small subset of tasks. To alleviate this problem, we consider meta-learning as a multi-objective optimization (MOO) problem, in which each task is an objective. However, existing MOO solvers need to access all the objectives' gradients in each iteration, and cannot scale to the huge number of tasks typical meta-learning settings. To alleviate this problem, we propose scalable gradient- mudient-based solver with the use of mini-batch. We provide theoretical guarantees on the Pareto optimality or Pareto stationarity of the converged solution. Empirical studies on various machine learning settings demonstrate that the proposed method is efficient, and achieves better performance than the baselines, particularly on improving the performance of the porrly-performing tasks and thus alleviating the compromising phenomenon.
INTRODUCTION
Meta-learning, also known as "learning to learn", aims to enable models to learn more effectively by leveraging information from many similar learning tasks (Hospedales et al.) 2020). In recent
years, meta-lcarning has received much attention for its fast adaptation to new learning scenarios
with limited data (Kao et al. 2021; Finn et al. 20117] Snell et al. [2.7]]00]]] Lee et al.
2019 Nichol et al.,
2018, Deleu et al.) 2022 Rajeswaran al. 22099] Vilalta & Drissi 2002). It is usually formulated as
a bi-level optimization problem (Franceschi et al. 2018 Hong et al. (020)) which finds task-specific parameters in the inner level and minimizes the average loss over tasks in the outer level.
Recently, Wang et al. (2021) reformulate meta-learning as a multi-task learning problem. From this perspective, minimizing the average loss in the outer level using (stochastic) gradient descent may always be desirable. Specifically. it may suffer from the compromising (or conflicting) phenomenon, in which the converged solution only focuses on minimizing the losses of small subset of tasks while
ignoring the others (Yu et al. 2020: Liu et al. 2021a Sener & Koltun 2018). This compromised solution may thus lead poor performance.
To alleviate this problem, we propose reformulating meta-learning as multi multi-objeectie optimization (MOO) problem. in which each task is an objective. The performance of all tasks (objectives) are then considered during optimization (Fmmerich & Deutz] 2018). A popular class of MOO solvers is the gradient-based based approach (L.iu et al. [021a]] Yu et al. 2020 [Sener & Koltun 2018 Navon et al.) Liu et al.) 2021b), with prominent examples such as the multiple-gradicn descent algorithm (MGDA) (Désidéri) 2012, Sener & Koltun 2018), PCGard (Yu et al., (0200), and CAGard (Liu et al., a021a). In each iteration, they find common descent direction among all objective gradients, instead of simply optimizing the average performance over all objectives.
Existing gradient-based MOO methods require using gradients from all the objectives. However, when formulating meta-learning as a MOO problem with each task being an objective, computing all these gradients in each iteration can become very expensive, as the number of objectives (i.e., tasks)

---

Published as a conference paper at ICLR 2023
can be huge. For example, in 5-way 1-shot classification the minilmageNeer data, the total number of meta-training tasks is (5)) R2 7 x 10%
To address this challenge, we propose a scalable MOO solver by using the improvement function (Miettinen& & Mäkela 1995 Mäkelä al. 2016 Montonen al. (018 with the help of mini-batch, On the other hand, we show that a trivial extension of existing gradient-basee MOO methods with the use of mini-batch does not guarantee Pareto optimality and has poor performance in practice.
Our main contributions are as follows: (i) To alleviate the compromising phenomenon, we reformulate meta-learning multi-objective optimization problem in which each task is objective; (ii) To handle the possibly huge number of tasks, we propose a scalable gradient- basee solver. (ii) We provide theoretical guarantees the Pareto optimality or Pareto stationarity of the converged solution. (iv) Empirical studies on few-shot regression, few-shot classification, and reinforcement learning demonstrate that the proposed method achieves better performance, particularly in improving the performance of the
tasks and thus alleviating the compromising phenomenon.
BACKGROUND
Multi-Objective Optimization (MOO). In MOO (Marler & Arora 2004), one aims to minimizo m 2 2 objectives fi(
min (2) Sm (x)].
Definition 2.1. (Global Pareto optimality) (Miettinen 2012 optimal if there does not exist another x such that f-
Mäkelä al. 2016 x² global Pareto 2 fo (z) for all T E {l....,m m), and
fr(x)) for (z) for least one 7' ( m].
The Pareto front (PF) is the set of multi-objective values of all global Pareto-optimal solutions.
Definition 2.2. (Pareto stationarity) (Miettinen 2012 Désidéri 2012) x" is Pareto-stationary if u, 2 0 and E Uy 1.
there exist [u>)T= m such that Σm= U1,
Note that global Pareto optimal solutions also Pareto stationary (Désidéri 2012). Analogous to the extension from a stationary point to an e-stationary point (Lin et al. 2020), we extend Pareto stationarity to e-Pareto stationarity. Obviously, O-Pareto stationarity reduces to Pareto stationarity.
Definition 2.3. (e-Pareto stationarity). For a given 6, J i (-Pureto-stationrry iff there exist {w,, }
u, Definition 2.4. (Improvement function) Montonen et al. 2018 The fiucctttontttunntioo of problem 7 is: (a, xx) = max,=1...m {x) - f-(z')}.
such that , (z)] S u, 2 0 VT and Em U =
Note that x² satisfying x¹ = arg min, (x, x²) (intuitively, x² cannot be further improved) is Pareto stationary (Mntonen al. 2018). To find x², can perform steepest descent on H:
X ||d|
x 3d" d' = arg min (x= + dd,x))
where x is the iterate at iteration s, B is the learning rate satisfying [1 + 3d, x²) H (2 xx)) and X' is a perrameerreeerr... It can be shown that when + oc, x is Pareto stationary Montonen al. 2018).
In this paper, we focus gradient-based MOO methods, including MGDA Désidéri 2012 Sener & Koltun 2018). PCGard YYu et al.) (2020), and CAGard (Liu et al.) 2021a). They assign weights each objective's ' gradient and find a common descent direction that decreases the losses of all objectives. For example, MGDA finds the direction g²
(z()) m 7o (VVzJ(a)) each iteration, where
(+) = arg min (++)
Σ Va/+to)] S.t. >n" 1,77 2 0,V7.
(3)
Meta-Learning.
Meta-learning aims to achieve good performance with limited data and compu-
tation (Hospedales et al. 2020). Most of them are gradient- -based (Nichol et al. 2018 Deleu et al. Rajeswaran et al. 2019 Zhou et al. 2019 Shu et al. 2019) or metric-based Snell et al.
2022
Without loss of generality, we consider minimization in this paper.
2

---

Published as a conference paper at ICLR 2023
Lee al. 2019 Vinyals et al. 2016). Let T be the set of all tasks, and to be the shared model parameter. For a task 7 E 7, let D, be its dataset and C the corresponding loss function. It tries to obtain task-specifie parameter w, from the shared
w (w)). Meta-learning is usually formulated as the following bilevel optimization problem (Ji et al. 2021]:
minw Σ (w++) s.t. w, = ue (w).
The inner subproblem learns the task-specific parameter w, for each т, while the outer subproblem learns w by minimizing the average loss over tasks in T. As m can be very large, usually a mini-batch B of tasks are uniformly sampled from T, and is then updated w w $18 1 [BB] Eren L, (w("(()) (Finn et al. 2017).
By taking each L, (w (w) in (4 an objective, this can be regarded as a weighted sum in multi-task learning (Wang et al. 202T). As observed in (Sener & Koltun 2018 Yu al. (020), gradient descent on this weighted sum can suffer from the compromising phenomenon, in which the loss obtained on some task T' can be much larger than the losses on the other tasks.
SOFT IMPROVEMENT MULTI-OBJECTIV META -EATA-LE (SIMOL)
We take the view of meta-learning as multi-task learning in ( (Wang et al. 2021 one step further and consider the meta-learning problem as the following multi-objective optimization (MOO) problem:
min (Ci (w (w)), ...... ..... Lm (w (w))),
(5)
in which each task corresponds to an objective. This considers all the individual tasks instead of simply considering the total loss over all tasks (Liu et al., 202Ta). Recently, Ye et al. (202T) also use multi-objective learning into meta-learning. However, their focus is not addressing the compromising phenomenon and they do not treat each task as an objective. Instead, besides minimizing the average task loss in (4), they consider adding some other objectives such as robustness to adversarial attacks. Moreover, MGDA is still used to find the Pareto optimal solution. However, as
in other gradient-based MOO methods (Yu al., 2020 Liu al. 202 la), MGDA requires collecting gradients from all m objectives in each iteration (as can be seen from its optimization problem (3)
This is computationally feasible only when there are a small number of objectives When each task is treated as an objective, the number of objectives can easily be in the millions (as in performing 5-way 1 1-shot classification on minilmageNVet).
Another widely adopted MOO based methods are the Chebyshev methods (Micttinen 2012: Mao
al. 2020 Momma et al. 2022 which leverage the weighted Chebyshev problem to find the Pareto front. However, these methods also cannot handle a huge number of tasks, as the computational complexity per epoch for these methods is O(m²). where m is the number of tasks.
To alleviate this problem, one solution is to use only mini-batch of objectives in each iteration. For example, when subset B of objectives used, MGDA's optimization problem in becomes:
min Σ TEB 7, V VC (w:
s.t. Σ TEB 7 1, 0, VT.
However, the descent direction then only considers objectives in B, and the original normalization constraint DTh 7/- 1 in 3 is also changed to Eter ?r 1. The obtained solution may no longer Pareto optimal. In the tollowing, we demonstrate this by using a simple toy example with two objectives (fi (x:) and f2(1r), where x E TR22) from (Liu et al. 2021a Navon al. 2022 As can be seen from Figure] [ mini-batch MGDA (with mini-batei of 1) cannot converge to the Pareto front.
3.1 SOFT IMPROVEMENT FUNCTION
In this section, we propose scalable MOO solver with the use of the improvement function (Mon. tonen et al. 2018). The proposed solver is agnostic to the number of tasks, while still theoretically guaranteeing that the solution is Pareto optimal.
Using Definition 2.4, the improvement function for problem (5) is:
H (ww w') max,=l....m (w)(
(6)
"In the meta-learning experiments of (Ye et al. 2021), they only consider two objectives. Definitions for fi, fa and the environment setup are in Appendix
3

---

Published as a conference paper at ICLR 2023
Canee tolden
30
10
100
-15
-15
1041
(a) MGDA.
(h) mini-batch MGDA.
(c) proposed SIMOL.
Figure 1: Convergence on two-objective toy dataset with mini-batch size 1. The Pareto front is shown in black.
Consider the optimization problem
max Ey [Cr (w) (m
where π is a probability density function on т. The following Lemma shows that (6) and 7 are equivalent when maximum in (6). All the proofs are in Appendix C
the Dirac delta distribution concentrated the task corresponding to the
Lemma 3.1. H(w, w') = max, Erax [C, (w (w)) C, (w2(w)))
Using 2 and Lemma 3.1 can be updated as
u++1 w 3d"
(8)
d' = arg mina (max, E,S, [Cr +d) - (w ())) + |d]
(9)
Taking the first-order approximation
(w + +d) Lr (w" + V V (w,
(10)
the minimax theorem (Simons 1995) can be used swap the min and max operators in
9 as
(xt arg max., ming Br [C. (w, (w" d)) (w, (w )) + ||d||
(11)
The following Proposition shows that the inner minimization problem has a closed-form solution.
Proposition
3.2. ming E,sr [C+ (ww((w (w" + d)) L. (w= Eraa V C (w (w)l-=w-)
+
|d]
2X E, V (w (w)) [=)) 1|,, and the optimal is d'
The expectation in Proposition 3.2 requires sampling tasks from æ. An easier alternative is sample tasks from the uniform distribution (-) over the set TI of all tasks, and then weighting each sampled task with r(r) π (r))UUT)). Note that
Ex-ur(T) = Σ. (())(()))U(()) = - -
(12)
We further parameterize neural network ro with parameter O. Using Proposition then rewrite (T as
3.2
we can
X"
0- = arg max 0 2X
112 Vw ((( lww
2 (BrnUro(T) - 1)²,
(13)
where the last term (with another hyper-parameter X") is a penalty for enforcing the constraint in (12). For notational simplicity, we denote the objective in (13 by K(0).
In principle, 0* can be obtained from 13 by gradient ascent. However, problem (13 involves an expectation over tasks. Recall that have total of m tasks, and m be huge. Hence, using all of them to compute this expectation may not be feasible. Instead, Let B be a mini-batch of & tasks, and denote the the
version of the objective in [3) as:
2
KB(
1 B
V (wf(w))
Σ re(T) -1 r1(
2X
FEB
FEB
4

---

Published as a conference paper at ICLR 2023
where X, X" are another set of Syyper--arameterr (which will be set in Proposition 3.3 corresponding
to X, X" in (13). Note that (0) K (0). Let B be the set of all size-k nati-b (with l >> 1). The following Proposition bounds the difference between K (0), the original objective (3) and the version [8
EBB (0) based on mini- mini-batches.
Proposition 3.3. Set X
and
X"
1
We
have:
C1|8I
(K(O)
T EEE Ro()))
I
C2kIBIG1
+ C24|BIA'G2,
where
C
=
A
12 m (2-)) mm
and C2
=
m [
2
m mm-21 2)
k 2 ]
( ( m
k k-
m
k 15-2
.G =
we kw= 112 and G2
1 Σ Erenlro(1) -1}
B
BEB
BEB
Corollary 3.3.1. When is < m. (K(8) 1 EBecKK(0))
Gi
k x2
kx'
In the experiments, m 2 10 ko = 10², and G1, G2 S 10 When X ž 10?, X" - 10 20 we have
B& EBES K B E [-110"-- -1] during training, and (0) (& EBE kB))))² < 0.2 is small.
Thus, Proposition 3.3 shows that K (0) can be decomposed into hini-batches as
[8] ΣBEB (o))).
This allows to update 0 by SGD over the task mini-batches as:
- 0 (V
(14)
where 3¹ is the learning rate. Similarly, we approximate d' by its mini-batch approximation d' = x ETER (wF (w)) and update w as w= = w² + 3d"
The whole procedure, which will be called Soft Improvement Multi Multi-Objective Meta Learning (SIMOL), shown in Algorithm [T] Step trains the base learner. In the experiments. we use two
popular meta-learning algorithms: MAML (Finn et al. (017) and prototypical network (PN) (Snell
al. 2017). For MAML., the base learner updated as
w((w) w - av (,
(15)
where Q, is the set of query examples for task T, Nc is the number of classes per epoch, far is the model with parameter w, = 1S&1 Σ (er.yees fw(x)) and S₃ is the set of examples belonging to class k. Pseudo-codes for SIMOL-based MAML and PN are shown in Algorithms and 3 of Appendix respectively.
For the PN, wi (w) = TQ-INe Ezeq, Exrexpl expreppl-ll -xpp(_1l
1
Algorithm 1: Soft Improvement Multi-Objective Meta Learning (SIMOL)
Input: T, batch size k, learning rates 3 and f² for w, d' = 0 and 0, respectively. for s 1, 2, . do
Reset d² = 0:
for r = 1, 2..................... k do
obtain ræ- (T) V - C, (w (w=)) for task T;
d' = d' - -r- (T) V w- L (w)));
#11 wº :bd
gs+1 8 + 8¹ -1
Ve
ro- (r
2X'
V \" t ErEB re- 11)
2
CONVERGENCE ANALYSIS
Lemma 4.1. Define
X"
R(0, w) = EraE [L (Iu()) + A())
3 (Brwuro(F)- 1)2
(16)
5

---

Published as a conference paper at ICLR 2023
where A(6) - - (T) 1 I(C, (wf(w) + d²) C, (w;(ww) 1 (e d²) - C- (w [w)]], and 1 is the stop gradient operaton Then, R(0, w) -== -d", and VaR(0, w²)= (Q)(
This allows interpreting the updates in (4p and (8) as performing Gradient Descent Ascent (GDA) (Singh et al.) 2000) on (66.. Thus, we can leverage game theoretical tools (Lin et al. 2020 in the analysis.
Let R(0, w; B) =1 ExEebro(T)ICCC - BBj EveB re(T)[ [(C, + d²) (w 1 (म [ TeT) + " (wt))()
B
Eren Te(T) - 1)2
be the mini-batch version of R, and (B) be the uniform distribution over task mini-batches, The following Theorem shows that Algorithm | converges to an c-Pareto stationary point of (5).
Theorem 4.2. Assume that (i) (- is L-smooth and ro(-) is p-strongly concave. (ii) The domain of O is convex and bounded set with diameter D > (iii) (B) (VannBBB(Vii [Vol((0, w; B) BV,,,, w)]
O, and (B) |I V R(0, w. B) VoR(O, w)ll S o². Assume the first-order approximation in 410,, and take 3 = + (1/D2o² + o²)), ) 3 = ((1/Lo²), Algorithm 7 converges to an e-Pareto stationary point of 5 with a rate of (11/()) If C. (w, (w)) is also p'-comvex w.r.t. w. and = 0, the O-Pareto stationary point is also global Pareto optimal.
Assumption (i) is commonly used in the literature (Collins et al. 2020 Zhou al. 2021: Finn
ctt al. 2019); while (ii) and (iii) are from (Lin et al. 2020). Theorem 4.2 shows that the proposed method can obtain c- --Pareto stationary point (or global Pareto optimal point for convex objectives) regardless of m, the number of tasks/objectives.
Corollary 4.2.1. Consider the MAML base learner update in (IS). Assume that W (w) is Hessian- Lipschitz continuous, bounded, Lipschitz-continunn and w is bounded. Then, Algorithm 7 converges to an e-Pureto stationary point of 5) with a rate of (()/6))
Corollary 4.2.1 is an application of Theorem 4.2 revealing that SIMOL with MAML can also
converge to a Pareto point. Convergence of the outer loop is slower than the O(1/²)) rate of standard MAML (Fallah et al. 2020). However, standard MAML only guarantees convergence to stationary points of w but not to Pareto-stationary points. Moreover, as will be seen in Section [5.] empirically, the proposed method has comparable even slightly faster convergence speed than MAML and other meta learning baselines. Besides, most the gradient-basee based MOO approaches (except CAGrad (Liu) ct al. 202 do not provide convergence rate analysis; while CAGrad requires that all task gradients are available in each epoch, which is very expensive (as will be demonstrated in Section 5.2
EXPERIMENTS
few-shot classification (Section 5.2). and reinforcement learning (Section 5.3). J. All experiments are run on a GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) CPU E5- Our implementations based on the popular open-source meta-learning library Leam2l.earn (Amold et al. 2020).
In this section, we perform experiments on few-shot regression (Section 5.1
5.1 FEW-SHOT REGRESSION
Setup. We follow the setup in (Finn et al., 2017: Li et al. 2017). The target function for task r is y a, sin(x by), where ar and 0- sampled uniformly from [0.1, 5.0] and [0, x². respectively. We generate 160, 000 meta-training tasks and 1, 000 meta-testing tasks. A multilayer perceptron with fully-connected (FC) layers (each of size 32) and ReLU activation is used as meta-learner and re-weighting network. The re- re-weighting network uses all mini-batch instances as input. To ensure that the weighting network output positive, we take the square of its last layer's output as output.
The backbone meta-learning algorithm is MAML (Finn et al. 2017). We use Adam (Kingma & Ba 2014), with an initial learning rate of 0.01, to update the base learners for steps in the inner-loop. For the outer-loop. we compare (i) minimizing the (single objective) of overall task loss as in MAML; versus performing MOO with (ii) mini-batch MGDA (Désidéri) 2012 Ye et al., (002). (iii) mini-batch as in (Liu et al. 202 a), (iv) mini-batch PCGrad, using the
CAGrad, using the same
"The stop gradient operator satisfies 1. (h(z)) h(*), V. 1. (h(=)) 0, where h(-) is any differentiable function.

---

Published as a conference paper at ICLR 2023
same hyper- tipper-parameters as in Yu et al. 2020). (v) the proposed SIMOL, and (vi) updating of (6) with a mini-batch version of the improvement function in (6). Hyperparameters for MAML. mini-batch MGDA, and mini mini-batch CAGrad follow (Finn et al., 2017), while that for the proposed SIMOL are in Appendix D
The mini-batch size is 16. We do not compare with the batch versions of MGDA/CAGraad as computing all task gradients takes very large memory and time. The initial learning rate for the outer loop is 0.001. The experiment is repeated three times with different random seeds. For performance evaluation, we use the squrr--uured-error (MSE) over all meta-testing tasks. We also report the worst-10% MSE, which is the average MSE for the 10% porst-performing meta-testing tasks.
Results. Table 1 shows the MSE and its 95% confidence interval (computed as in (Finn et al. 2017 Li et al. 2017 ) As can be seen, SIMOL consistently outperforms MAML, and the mini-batch versions of MGDA, CAGrad, PCGrad and improvement function in terms of both the overall and worst-10% MSEs. Indeed, the mini-batchs versions of MGDA, CAGard, PCGrad and improvement function are even worse than the original MAML. Figure 2 shows the convergence of MSE with the number of training epochs. As can seen, SIMOL converges slightly faster than other baselines.
Table 1: MSE (with 95% confidence interval) for few-shot regression. The best results are in bold. The denotes that the improvement the second-best is statistically significant (at significance level of 0.1 using the paired t-test).
overall
worst-
5-shot
2-shot
5-shot
2-shol
min average loss (MAML)
0.43 0.11
1.70 + 0.11
2.13 + 0.21
7.75
0.48
mini-batch MGDA mini-batch CAGrad
0.002
1.13
2.62 + 0.10
7.04 + 0.51
1.90 + 0.24
1.82 0.44
8.18 + 0.63
8.23 - 2.33
mini-batch PCGrad
1.89 + 0.12
0.11
6.61 + 0.17
7.73 +: 0.54
mini-batch improvement function
0.69 + 0.03
1.79 0.09
2.63 + 0.21
8.23 +: 0.36
SIMOL
0.34 0.04
1.24
0.08
1.69 + 0.18
5.66
0.33
awerage
awtrage loss
miPe-batth FOGeN
mini-batth PCGmd
SPNCL
MOL
2000
4000
6000
8000
10000
0
4000
6000
8000
10000
Training Epach
Training Epoch
(a) 5-shot.
(b) 2-shot.
Figure 2: Convergence of MSE with the number of training epochs.
5.2 FEW-SHOT IMAGE CLASSIFICATION
Setup. In this section, we perform 5-way-1-shot and 5-way-5-shot classification on the minilma- geNet (Ravi & Larochelle| (0166) and tieredImageNNe data (Ren et al. 20188). Following (Finn et al., 2017), we split the minilmageNel dataset into a meta-training set with 64 classes, a meta-validation set with 16 classes, and meta-testing set with 20 classes. The total number of meta- training tasks is (64) R 7.6 x 105 Similarly, as in (Zhou et al. 2019), we split the tieredImageNet dataset into meta- mncaa-traiinn set with 351 classes, a meta-valrdation set with 97 classes, and meta-test set with For both datasets, we
160 classes. The total number of meta meta-trainin training tasks is (351) = 4.3 10 randomly select 1, 000 meta-testing tasks for evaluation.
We use two backbone meta-learning algorithms, MAMI. and prototypical network (PN) (Snell et 2017), with hyper-parameter following the original papers. Following (Finn et al. 2017 Li et al. 2017 Zintgraf et al. 2019), we use the CNN 5 LeCun et al. 2015) as the meta-learner, and CNN4
Thhe CNN4 consists of four 3 convolution networks with batch normalization, x max-pooling and ReI.U activation layer.
7

---

Published as a conference paper at ICLR 2023
with 3-layer FC as the reweighting network. The optimizer is Adam. The learning rates for the meta networks are 0.003 for MAML-based methods and 0.001 for PN-based methods, respectively. The learning rate of the reweighting network is 0.08 for SIMOL and 0.0008 for SIMOL+PN. The mini-batch size is 32. More details on the hyperparameters are in Appendix D
The proposed SIMOL is compared with (i) minimizing the overall task loss in standard MAML, (ii) mini-batch MGDA, and (iii) mini-batch CAGrad. We also compare with MAML variants including (iv) Reptile (Nichol & Schulman| 2018), (v) FOMAML (Finn al. 2017), (vi) Meta- MinibatchProx (Zhou et al. 2019). (vii) TSA-MAML (Zhou et al. 2021), (viii) IMAML (Rajeswaran al. 2019)) (ix) MTL (Wang et al. 2021 D. multi-task learning based maml approach, and the standard prototypical network (Snell et al. 2017). The evaluation metrics are similar to those in
Section 5.1 but with accuracy instead of MSE. Experiments are repeated three times with different
random seeds.
Results. Tables and 3 show the meta-testing accuraccies and 95% confidence intervals on mini- ImageNet and rieredImageNet, respectively. For MAML and its variants, SIMOL consistently outperforms all the other baselines terms of both the overall and worst-10% accuracies. The same is also observed on the meta-learning algorithm prototypical network. This demonstrates that SIMOL is useful for both gradient- based and metric-based meta-learning approaches.
Note that (mini-batch) MGDA and CAGrad do not perform good in terms of both overall and worst- 10% accuracies, showing that they cannot be straightforwardly extended to the use of mini-batch. On the other hand, the batch versions of MGDA and CAGrad are computationally impractical. Table 4 compares the per-epoch running time in training stage of standard MAML, SIMOL, and batch MGDA and CAgrad. Experiment performed on 5-way-5-shot classification with the MAML algorithm on minilmage/Net. As can be seen, while SIMOL has comparable per-epoch running time as MAML, batch MGDA and CAgrad are much more computationally expensive (around 432, 000 times slower).
Table 2: 5-way classification accuracies minilmageNer (with 95% confidence interval). Results of Reptile, FOMAML, and Meta-MinibatchProx are from (Zhou et al., (209),, IMAML from (Deleu al. 2022), and -AAM from (Zhou et al. 2021) are denoted "_"" The best results are in bold.
Results not reported in the original papers
overall
worst- 10%
1-shot
5-shot
1-shot
5-shot
(MAML.)
min average loss
49.24 0.78
62.13 + 0.72
13.33 + 1.07
41.71 1.02
mini-batch MGDA
46.08 + 0.78
60.15 + 0.41
10.60 + 1.33
39.67 = 0.55
mini-batch CAGrad
44.67 + 0.75
60.05 + 0.67
11.33 + 1.12
40.01 = 0.88
SIMOL
50.62* + 1.39
65.83* + 0.86
14.99* - 1.72
44.81* 0.58
(MAML
Reptile
47.07 + 0.26
62.74 0.37
variants)
FOMAML
45.53 + 1.58
61.02 + 1.12
IMAML
49.30 + 1.88
59.77 0.41
Meta-MinibatchProx
48.51 1 0.92
64.15 1 0.92
TS-MAML
48.44 0.91
65.52 1 0.68
MTL
49.87 1: 0.41
65.81 + 0.33
13.64 4: 1.45
43.42 = 0.47
(PN)
Standard SIMOL
48.25 + 0.95
65.29
0.48
13.10 + 1.32
45.03 = 0.69
50.45* 0.93
66.67* + 0.47
15.00 + 1.21
46.27* 0.70
5.3 REINFORCEMENT LEARNING
In this experiment, experiments are performed on the HalfCheetach-Dii and Walker-Dit environments in Mujoco (Todorov et al., 2012). In both environments, each task corresponds a random direction in the XY-plane, and the agent (Walker/ HalfCheetach) learns to run in that direction as far as possible. The reward is the average velocity minus control costs. We again use MAMI. as the meta-learning algorithm, and the base reinforcement learning algorithm is vanilla policy gradient (VPG) (Sutton) ct al. 1999). Following (Rothfuss et al.) 2018), the policy network has two 64 x 64 FC layers with tanh activation, while the critic is linear Prate-value function whose parameters are obtained by minimizing least-square. The weighigting network has two 64 x 64 FC layers with ReLU activation. Following (Zintgraf et al., 2019),
MAML the baseline.
Figure 3 shows the convergence of the accumulated reward with the number of training iterations. As can be seen, SIMOL consistently outperforms MAML.. Moreover, the convergence of MAML. is less
8

---

16.67 0.78 Published as a conference paper at ICLR 2023
Table 3: 5-way classification accuracies on fieredImageNer (with 95% confidence interval).
overall
worst- 10%
1-shot
5-shot
1-shot
5-shot
(MAML)
min average loss
50.58 + 1.44
69.33 + 0.74
11.60 + 1.96
46.95 1.14
bini-batch MGDA
22.92 1.04
53.41 + 0.74
7.12 = 2.11
32.79 0.88
mini-batch CAGrad
49.04 + 0.93
65.43 + 0.73
11.40 + 1.97
42.63 0.95
SIMOL
51.42* 1.50
70.13* 1 0.74
12.00 + 1.95
47.51 1 1.46
(MAML
Reptilc FOMAML
49.12 0.43
65.99 + 0.42
variants)
45.53 1: 1.58
61.02 1 1.12
IMAML
38.54 1.37
60.24 1 0.76
Meta-MinibatthProx
50.14 1.37
68.30 1 0.91
TS-MAN MAML
48.82 = 0.88
67.82 = 0.72
MTL
51.02 = 0.46
66.47 + 0.39
13.60 - 1.59
49.45 + 0.58
(PN)
Standard
47.63 + 0.93
68.92 + 0.52
14.40 + 1.25
SIMOL
50.09* 0.96
71.51 + 0.79
15.10* + 1.20
49.85 + 0.79
stable, as also reported by (Rothfuss et al. smoother and more stable.
2018). On the other hand, the convergence of SIMOL is
NAMU-FGG
320
150
200
Accumu motumaaddd faward
300
-159
-200
-300
-20
-455
-100
Rorations
Itorations
(a) HalfCheetah-
(b) Walker- Dir
Figure 3: Returns for SIMOL-VPG and MAML-VPG. Results averaged over 3 trials.
5.4 ABLATION STUDY
In this experiment, use the setup in Section 5.1 and vary the number of FC layers in SIMOL's meta-learner. The number of training epochs is always fixed to 10, 000, Table S shows the MSE's on 2-shot regression. MSE. the deeper networks may not be sufficiently trained with the fixed number of training epochs, leading to worse performance.
be seen, the of 2 FC layers has the best overall MSE and worst-10%
Table 4: Per-epoch running time for 5-way 5- T-Table 5: Performance for SIMOL with different shot classification on minilmageNee..
numbers of layers in 2-shot regression.
#layers
overall MSE
worst-10 % MSE
MAML
SIMOL
batch MGDA
batch CAGrad
2
1.24 + 0.08
5.66 = 0.33
2.0 see
2.3 sec
6.9 days
5.6 days
3
1.36 0.09
6.01 0.38
4
1.42 0.08
6.22 = 0.34
CONCLUSION
In this paper, we propose to avoid the compromising phenomenon in meta-learning by reformulating as a multi-objective optimization (MOO) problem, which each task an objective. However, current gradient-based MOO solvers cannot scale to large number of objectives. With the use of improvement function and mini-batch, we propose a scalable gradient- based solver with theoretical guarantees to Pareto-optimality Empirical studies few-shot regression, few-shot classification, and reinforcement learning demonstrate that the proposed method is efficient, and has good generalization in terms of both overall performance and performance the poorly-per mooll--peroonnng tasks.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
This research was supported in part by the Research Grants Council of the Hong Kong Special Administrative Region (Grant 16200021).
REFERENCES
Sébastien MR Arnold, Praateek Mahajan, Debajyoti Datta, Ian Bunner, and Konstantinos Saitas Zarkias. Leam2learn: A library for meta-learning research. Preprint arXiv:2008. 12284, 2020.
DW Bolton. The multinomial theorem. The Mathematical Gazette, 52(382):336-3444 342, 1968.
Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic meta- meta-learning. NeurIPS, 33: 18800-1888,,, 2020.
Tristan Deleu, David Kanaa, Leo Feng, Giancarlo Kerg, Yoshua Bengio, Guillaume Lajoie, and Pierre-Luc Bacon. Continuous-time meta-learning with forward mode differentiation. In ICLR, 2022.
Jean-Antoine Désidéri. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. CRM, 30M05353333333333333 2012.
Michael Emmerich and André H Deutz. A tutorial multiohjective optimization: fundamentals and cvolutionary methods. NC, 17(3):885-666 609, 2018.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient-basee model-agnostic meta- lett-le algorithms. In AISTATS, pp. 1082-1092, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, pp. 1126-1135, 2017.
Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning, In ICML, pp. 1920-1930, 2019.
Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In ICMI., pp. 1568 1568-1577, 2018.
Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic. Preprint arXiv:2007.055 0010000557.. 2020.
Timothy M Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. PAMI, 2020.
Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In ICML, pp. 4882-4892, 2021.
Chia Hsiang Kao, Wei-Chen Chiu, and Pin-Yu Chen. MAML. is a noisy contrastive learner in classification. In ICLR, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Preprint arXiv:1412.6980, 2014.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436 444, 2015.
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In CVPR, pp. 10657-10665, 2019.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few-shot
learning. Preprint arXiv:2008. 12284, 2017.
Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-comcat minimax problems. In ICML, pp. 6083-6093 2020.
10

---

Published as a conference paper at ICLR 2023
Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averss gradient descent for multi-task learning. NeurIPS, 2021a.
Liyang Liu, Yi Li, Zhanghui Kuang, J Xue, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang. Towards impartial multi-task learning. In ICLR, 2021b.
Marko M Mäkelä, Napsu Karmitsa, and Outi Wilppu. Proximal bundle method for nonsmooth and nonconvex multiobjective optimization. In MMOCS, pp. 191-204. 2016.
Yuren Mao, Shuang Yun, Weiwei Liu, and Bo Du. Tchebycheff procedure for multi-task text classification. In ACL, pp. 4217-4226, 2020.
Timothy Marler and Jasbir S Arora. Survey of multi-objecctvve optimization methods for engineering. Structural and Multddisciplinayy Optimization, 26(6):369-395, 2004.
K Miettinen and MM Mäkelä. Interactive bundle-based method for nondifferentiable multiobjeective optimization: NIMBUS. Optimization, 34(3):231-244) 1995.
Kaisa Miettinen. Nonlinear Multiobjective Optimization, volume 12. Springer Science & Business
Media, 2012.
Michinari Momma, Chaosheng Dong, and Jia Liu. A mulli--hhectivemmulii--a learning framework
induced by pareto stationarity. In International Conference Machine Learning, pp. 15997-
PMLR, 2022.
O Montonen, N Karmitsa, and MM Mäkelä. Multiple subgradient descent bundle method for convex nonsmooth multiobjective optimization. Optimization, 67(1):139-158; 2018.
Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning as bargaining game. In ICML, 2022.
Alex Nichol and John Schulman. Reptile: A scalable metalearning algorithm.
Preprint
arXiv:1803.009999,22 2018.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. Preprint arXiv:1803.02999, 2018.
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. NewrIPS, 2019.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. ICLR, 2016.
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard Zemel. Meta-learning for semi-supervised few-shot classification. In ICLR, 2018
Jonas Rothfuss, Dennis Lee, Ignasi Clavera, Tamim Asfour, and Pieter Abbeel. Promp: Proximal meta-policy search. In ICLR, 2018.
Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. NewrIPS, 2018.
Jun Shu, Qi Xie, Lixuan Yi. Qian Zhao, Samping Zhou, Zongben Xu, and Deyu Meng. Meta-weight- net: Learning an explicit mapping for sample weighting. NeurIPS, 2019.
Stephen Simons. Minimax theorems and their proofs. In Minimax and Applications, pp. 1-23. Springer, 1995.
Satinder Singh, Michael J Kearns, and Yishay Mansour, Nash convergence of gradient dynamics in general-sum games. In UAI, pp. 541-548, 2000.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. NewrIPS. 2017.
Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement leamnngg with function approximation. NewrlPS, 1999.
11

---

Published as a conference paper at ICLR 2023
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In ICIRS. pp. 5026-5033. 2012.
Ricardo Vilalta and Youssef Drissi. A perspective view and survey of meta-learning. AIJ, 18(2): 77-95, 2002.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for one shot learning. NeurlPS, 2016.
Haoxiang Wang, Han Zhao, and Bo Li. Bridging multi-task learning and meta-lcarning: Towards efficient training and effective adaptation. In ICML, pp. 10991-11002, 2021.
Feiyang Ye, Baijiong Lin, Zhixiong Yue, Pengxin Guo, Qiao Xiao, and Yu Zhang. Multi-objective meta learning. NeurIPS, 2021.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. NeurIPS, 2020.
Pan Zhou, Xiaotong Yuan, Huan Xu, Shuicheng Yan, and Jiashi Feng. Efficient meta learning via minibatch proximal update. NeurIPS, 2019.
Pan Zhou, Yingtian Zou, Xiao-Tong Yuan, Jiashi Feng, Caiming Xiong, and Steven Hoi. Task similarity aware meta learning: Theory-inspired improvement MAML. In UAA, pp. 23-33, 2021.
Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning. In ICMI, pp. 7693-7702, 2019.
12

---

Published as a conference paper at ICLR 2023
A TOY EXAMPLE
The definitions of /1 and /2 are:
fa( (t)) f2(2) = +
where
log (max (00. (-21 -7) tanh (-#2)), 6, (x) = log (max (0.. (-z) + +3) tanh (-22) + 2| 0.000005))
1 (x)
(1.00)00000 + 6,
(1x)) = (--z) + 7)2 0.1 * (-02 - -8)² /10 - 20,
92(x)) = (-=1-7)2 0.1 * (-x2 - 8)² /10 20,
ei (x)) mox(tanh(0..522).. 0),
(2(2) = max(tanh(-0.5rz 0).
For mini-batch MGDA, and SIMOL the probability sample task 1 is 2/3 and 1/3 for task 2. The learning rates for MGDA and mini-batch MGDA are both 0.01. The learning rate for SIMOL's meta-learner is 0.01, while that for its re-weighting network 0.1. Note that the meta-learner and the re-weighting network are represented by learnable vectors.
B PSEUDO-CODES
Algorithms and 3 show the pseudo-codes for SIMOL-based MAML and PN, respectively, The key differences between SIMOL and MAML/PN are highlighted in blue.
Algorithm 2: SIMOL for MAML.
Input: T. and total epoch S. B is the batch size, 33 and 3 are learning rates for w and A, a is the learning rates for inner loop.
for epoch 1, 2, ........ S do
Sample tasks 1, 2.
for Every task do Receive T'- rg- (r); Compute adapted parameters with gradient descent wz(w) w" - a (w
#+1 w
3d 1 Em= ng. ro (wt
0++1 = 0= + 3'' K B);
Algorithm 3: SIMOL for PN.
for epoch 2, ...,. do
Sample tasks 1, 2, ........ B; for Every task do for Every class do
Select the support S and query set Q+:
Compute prototype e = No Exxiness So (x;):
Calculate W (w) TQ.INo Exeq, E (
exp(- (c))=
Receive ro- (T);
#++1 w
g=1 DB ro. (o
0++1 0r + K B);
10
13

---

Published as a conference paper at ICLR 2023
C PROOFS
C.I PROOF OF LEMMA 3.1
Proof. Note that
(w L, (w [((T))(((((()) L. (wof(w)))
r=1...m
where
p(T) = {1 T = arg max, L (w) (-(w)(w())
{0
otherwise
First, we have
Bx(r) [ (w (w)) Ly (wf((()) - (mtr), (x ((0 0.
max m(r)
This can be done by setting (T) = p(T). Next, we show that
max Em(r) [c, (w)(ww) C, (w: - [u(t) - (uZ((w))) s O.
x(7)
This is established since
max (r) (r)
- Cr -
E - ((u(((w)))
$ Σ (T(T) -
- C,
[mm(7) (1000
(c
Σ (E+ L- (wf
rE rE(lommm)r
(
2 (x(7) - 1] (w) (u
Σ
(w) C, (wz
rE(l....m)\r
I ((()) (2r)) (wz - (w) (u
Σ
- (wt)
rE(I....m)\r!
(w)) - C,
(wt - C,, (wf (')) - [C, (wt - Cr
0,
where
7'
arg
max {Lr (wx()) - C, (w T=I..,m
(a) is due to the fact that 2 (w C, (w (w) )) Vr based on the property of r'. })] (w'))
[C,, (w), (ut (w Therefore, we have
(u (w)) (w, (w ))] 2 [C- (w)) (w(((w))
and
(w (w)) L. (w², (w }) - (w (x
Thus.
max
(+(w((w))) (wx()((
max Eywa(r) (w:()( L. (=
T(>)
C.2 PROOF PROPOSITION 3.2 Proof.
Using the first-order Taylor expansion,
arg (ain (ron d)) (w: + X
2 ||d|
X
arg aig (ro(7) (wlr(w(( To] + 2 ||d|1
(17)
14

---

Published as a conference paper at ICLR 2023
Taking the derivatives w.r.t. d,
(,Bu()) [Pa(T) (W dd]
X
2 ||d|| Bu( Brcg + Xd.
Setting the above to zero, we have ( V,
= 0, and
- X Bu
Putting d' back to the objective in 17 we have:
X
(r d)) (w
||]|
X/
1
[rr.urr d
B V L, (u
X
= CT
+
|| |
V
(w
2X'
= N (Br (1)
2X B VuL, (w (w)l
X T V Cw (w:(())\])ll
Next, we have the following two Lemmas.
Lemma C.I. When E. (w"(w" d)) = L. (u d, Eq. 19 is conver w.r.f. and concave. w.t. (T)...
Proof. Putting (r + dd) 2 (w( (w into Eq. 9
we have:
X
X"
min max Bu (r ( (w d)) (C r(r)
2 ||d| 2
E (EU()(((T) -
min max EuG (() + V L. (w (w=")) d r(r)
TT TTd (f., (w(("))
+ 2 |||| - X"
X
(Bu())((TT] - -1)%/
For V (w (w*) )) T d 2 IId]? since (wL-(w; (w )) d is both convex and concave w.r.t. d, and ||]| convex w.r.t. Then, (wLr(w; (w*)) X' [[d] is convex w.r.t d. Thus, the sum
2
of two convex functions V Cw w.r.t. d. Similarly, since concave w.r.t. ro(r).
(w](w T) Jdd+ + 2 ||d|2 is convex w.r.t d. Thus, Eq. 9 is convex
A" (ro()) 1)² is concave w.r.. re(r). Therefore, Eq. (9 is also
Lemma C.2. When L. (w" + d)) ~ L, (w= + V (w
X'
X" -/ (Exlro(F)] 1])
min max Burcs +d) L. (w + re(r)
l|]|
X'
X"
maxminEurrrrr(()C,, + + (r (w (w))) retr)
||dll (Ep(re(7)] - 1)
Proof. The above equation is established by the minimax theorem
(Simons convex w.r.t. and concave w.r.t. ro(r). This holds by using Lemma C.1
1995) when Eq. (9) is
15

---

Published as a conference paper at ICLR 2023
Lemma C.3. If every B B is set consists of unique selection of k (m > k>>>>>>> 1) tasks out of m (the total number of tasks) tasks without replacement. For any continuous function f(-), we have:
2
TR 1
Σ
f(T)
C1
Σ
Σ
f(T)
C₂ M Σ [f())
m
where C1
A² (x-2)
)
C2
[(=-]) (x=2)
((=1)
k-
Proof. Note that
S(T)
Σ
(
E EED BEB FEB
Σ ((+))()(
(n [x-]]
1 n 2)
Σ f(7)
2
-
2 B 2
TET
((=1)
(n-22)
ΣΣΣ 1(T7))())(())
BB
f(7)
n-2 B-2
(B=)
BEB DEBBr'BBEEL
BEBBEB
(B-1) 2 (B-2)
f(T)
Σ f(7)2
=
B (B-2 m²
B m²
BEB TEB
BEB
2
Gi
Σ f()
C2 Σ Σ [f(T))
BEB
LEEB
BEB TEB
(2 T
(a) is due to the multinomial theorem (Bolton 1968) and (T,,7') otherwise
(b) is due the
fact that every task r has occurred exactly n (A-1) times, and every tuple (T, has occurred exactly (n-2) times.
Lemma C.4. every B a set consists of a unique selection of k (m 1) tasks out of m (the total number of tasks) tasks without replacement. Let g(T) (7) (2 .... 9m 9m(r)), where ge(-)'s 's are continuous functions. We have
T7
g
g(7))
C
Σ
Σ TEB
g(())
C2 ΣΣΣ Σ 11
m
BEBTEB
where C1
and C2
(m==)
1 ((=22)
/(==1 (T-2)
(m k-
ma
Proof. Observe that [ = The remaining follows from Lemma C.3
16

---

Published as a conference paper at ICLR 2023
Proof. (of Proposition
3.3
Using Lemmas C.3
and C.4
we have:
1
(T)
K0(0)
B
B
3C1
+
|d
A'C1
X"
Σ
E(ra(t) (ra(T) - -1)
2X
2X|BB]
k
BEB
BEB
TEB
-C2
Σ Σ
(-
1
(w (ww -1
((ro(T) 11²)
BEB
ΣΣ BEB
C2 x
1| 12 112
where d
Σ Vwr) lum Recall X' and X" are defined in Sec. 3 Then,
BEB
(Kr(0)
1
Σ kn))))²
(18)
[B)
C2
ΣΣ
(19)
X
BEB
Σ (no(T) - 113 ))
+C
(20)
B
BEB
C2k|BIG + C21IBINGG2.
(21)
X
By noting that (0) is exactly K (0), we obtain the first claim.
Regarding the second claim, note that when m is large,
&
m
mm
m k
k
C2A/B
m
(x-22
k: m
~
m²
k Rk-
m
{mm k-
2) m m-1
K
T K-1
m²
(m
(k-2
m²
m
k-2
m
2
(k: - 1)m
k -
The first approximation is due to the fact that due to the property of combinatories ((T) the results.
(m-))
(m 2)
k 2
The third and forth equations are
m A
Putting k- back into Eq. (18 we obtain
C.3 PROOF OF LEMMA 411
Proof. We have:
VoR(O, w) = VoA(0) - Vo A" (Br.ero(T) 1)2
X P 1 [(- + d²) - L (w(
(B - (=F(w)))
X"
VollBnnee
|| Vo Vo -E (ErwUTO(7) 1)²
VoK(O).
17

---

Published as a conference paper at ICLR 2023
Also notice that:
Ww, w)
X" Br-uld (o Vio((0)))))))))))))))) ve (Br.uro(T)) 1)9]
Therefore, V Ri0, www = dd"
C.4 PROOF OF THEOREM
Definition C.5. (Weak Global Pareto Optimality) Määkelä et al. 2016) solution xf problem 5 is weakly global Pareto optimal if there does not exist another x such that f+ 2 fz) f for (I....,m]
Theorem C.6. (Theorem in (Mettinen & Mäkelä 1995)) For multi- multi-objective optimization problem min(fi(w) f2(w)..... Fm (w) and its corresponding improvement functioon (w, w²). A necessary
condition for w E R to be weakly global Pareto optimal is that w" = arg min (w, w*). Moreover, if fi(w) is convex Vi, then it is sufficient condition.
We observe the following.
By replacing fr C, u VT, the above theorem can be directly applied to the MAML setting.
2) f in our setting can be neural network. Thus, the convexity condition for fr can be hard to satisfy. Here, we give the more relaxed version of the above theorem:
Theorem C.7. w" == arg min (w, w²) is Pareto stationary.
Proof. Using Theorem 2 in (Miettinen & Mäkelä 1995), have
0 H (w, w*) C conv U, aw (fi(w) - fs )), where is reloaded as sub-gradient, and conv is the
convex
Note that E Hn, (w) due to the fact that the element in the convex set of the union of sub- gradients is still sub-gradient, and for {w[0 E H,, (w, w²)), the sub-gradient o,, H (w, w*) is zero.
Then, we always have w, (f((w( (w))) convex set, where wi 00 11. vi is real number. By simplifying the above term, we have Σ, w;O (fi(w) - fa (w*)) E, f((w 0. which exactly the definition of Pareto sta- tionary point.
0, E, w. based by definition of
We now show convergence of Algorithm
Lemma C.8. In each epoch of Algorithm 1 set X" = so Ex]ro(T]]] =1
for
we have:
otherwise
H d, (+) I (w"""""
When equality holds, w is a stationary point.
Proof. By Lemma C.2 the solutions of
maxo mina Bu( (ro(T) (w (w* d)) (w, (wo*)) + X 114112 8" (Ep(ro(7)] -1)
the same those from
mina maxy ( (ra(T) [C+ (we (w= + d)) (r (w)))] 1lda122 X (Fplro(T) 1)2].
Solving 0, the above minimax problem degenerates to
X'
min max [Cr (wr (w* + df) - L, (w (w*))] + 2 |d|2 m=....m
18

---

Published as a conference paper at ICLR 2023
Note that the above optimization problem is exactly ming (w d, w*) x [|d]2 where d is the steepest descent direction. Then, due to the property of steepest descent direction, we always have H (w" d, w*) S (w",, w*), and equality holds iff |d| O, where w" stationary point according to the definition.
Lemma C.9. Algorithm 0asiaimmmeeee converges to a Pareto stationary point. Moreover, if Cr (w(((w) is convex, Algorithm converges global weak Pareto optimal point.
Wim wimm)s is decreasing. Also obviously, H (w" + d, w², 8++ wi.m) has a lower bound. Thus, using the monotone convergence theorem, Algorithm [1 converges. Using Theorem C.7 is Pareto stationary. Together with Theorem C.6 is global Pareto optimal if (w)))) is convex.
Proof. From Lemma C.8. the sequence {w +d, w", w
Lemma C.10. (i) If L- + d)) is L-smooth and μl ul-coveex w.E. w and d, and ro(T) ) is p=- concave, (T) and L (=("((w++) are bounded (ie.., re(T) S Caux and L, (w S C' then R(0, w) is L + Cmaxlé convex w.r.l. w and 2L-smooth w.rl. w, 0 and maxx concave w.r.l. 0.
(ii) If L, (we (w + d)) is I-smooth w.r. w and d, and re(T) is p-concave, then R(0,, w, B) is L-smooth w.r.t. w, 0, and C² CaaxxH- w.r.t O.
Proof. For claim (i), we first prove that for any differentiable functions f.g. 1. [f(z)] 1 [g(x)]] is O-smooth and convex w.r.t x. Note that
V T [f()]] [s(=)] - V. 1 [f [g(x)) 112
due to the property of the stop gradient operator. Thus,
[fx))] 1 [a(=)] - [fx')) [ø(x')] = - (1 [f(=') (o()')))L [f(x]] L [o(x))) = O.
For convexity, by using the property of 0-smoothness, we have
I [f(=']] [ø(x') 1 [ lo(=)) (ar(1 [f()')] [o('))) [f()] [s(x))) Thus, 1 [f(x)] - [g(x)] is convex due to the definition of convexity. Therefore, 1 [fx) [s(x)] also 0-smooth and convex w.r.t. x.
A direct application of the above, we obtain L [f(x)] is O-smooth and convex w.r.t. by setting g(x) = 1.
Using the above, have 1 (w)(w) + d) L, (wf(w))) 1 [T d') C, (w)()))., 1 [CD, (w+(w) + d²) - LD, (w+ [w) 1 ETWUTO (T) (w)(w)
d²) (wr (w)) are also 0-smooth and convex. Therefore (T) 1 [LDD (w+(w) d²)
LDD (wi (w)) II (ET (w) + dd) fD, (w))] is convex and L-smooth (as L>0 0).
Also, we have 1 (x (w)) is Cmaxpt convex and L-smooth due to our assumption.
Then, E (ro(T)) [L. ( (w)) is fiax:/'-covver and L-smooth since the addition of convex functions implies R-- [1 (ra(T)) [C (w, is Cimx//''conee convex, and the adding of smooth func- tions implies ErwulL [L (ra (C (w" (w)]] is L-smooth, and taking the average does not affect the results. Thus, R(O0, w) is Cmaxké -convex and L-smooth.
For ro. since- ( (To - 1)2 also Ch Thus R(0, w)) C -conave
concave. X' Vo(DEproo (T)) - 1)2 + (0) is concave, A()) is ~ Then A()) 2
The proof of claim (ii) is similar.
Proof. (of Theorem 4.2 Recall Lemma C.10 on the properties of convex and smooth for R. Combine with the assumptions in Theorem 4.2, and use Theorem 4.9 in (Lin et al. 2020). we obtain the bound of 0 when B I.
When B > 1, note that using Lemma A.2 in Lin et al. 2020,, we have:
B R( w. B)
< | R(0, w)ll o²
E
B
B
1=1
19

---

Published as a conference paper at ICLR 2023
Therefore, let o² o² and use Theorem 4.9 in (Lin et al. 2020) shows the bound of (
The remaining part that the fixed point w of max, stationary) can be obtained by using Lemma C.9 which says that the fixed point of Algorithm is global Pareto optimal (resp. Pareto stationary).
R w, B) global Pareto optimal (resp. Pareto
Finally, we show that using SIMOL on MAML can guarantee convergence.
Proof. (of Corollary 4.2.1 First, we show that if v. (w) is Hessian-Lipschitz cistaiiiiicchhtzccttuuuuuuu continuous, bounded and Lipschitz-continuous, and ||l||| is bounded, then (w) is also Hessian-Lipschitz continuous, bounded and Lipschitz-contimuuu. Note that
V of (w) V (w''ll
a (V V. w Cr
| (w) - V w (w')l = V - V (V (w)))l
[
w aV- vv |w]| V [w>r(l)]|-
It is easy to that (w) is also Hessian Hessian-Lipschitz Lipschitz continuous, bounded and Lipschitz continuous. Applying Lemma (Collins et al. 2020 we obtain that L, (w)) is C-smooth, where C' is positive. Then setting C = L. and using Theorem 4.2, get the desire result.
D HYPER-PARAMETER SELECTION OF SIMOL
For the few-shot regression experiment (section 5.1). the regularization parameters X, Å are selected from f0.001, 0.01, 0.1, 1}, and learning rate 3' is selected from {0.01, 0.03, 0.1, 0.3, 1} based the validation set.
the X X combination selected from few-shot regression, while the learning rate 3' selected from {0.0003, 0.0008, 0.01, 0.03, 0.08, 0.1) for the 1-shot minilmageNet task based the validation this is then also used in the other sho-shot classification experiments.
For the few-shot classification experiments (section 5.2
20


Reviews:
[{"review_id": "ud-E4FhDTPc", "reviewer": "Reviewer_sFDS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to formulate the meta-learning as a multi-objective optimization (MOO) problem instead of the traditional single objective problem where the sum of each task’s loss is optimized. By doing do the compromising phenomenon can be avoided, but it also incurs a new problem — the current gradient-based MOO solvers cannot scale to a large number of objectives. Then the authors proposes a scalable gradient-based solver via the use of improvement functions. Besides, this algorithm is guaranteed to have theoretical convergence. Experimental results on few-shot regression, few-shot classification and reinforcement learning show that the proposed method, called SIMOL, outperforms several existing algorithms in terms of either efficiency or generalization. ", "summary_of_the_review": "The paper seems to be the first to reformulate the meta-learning problem as a MOO to avoid the compromising problem. Then a gradient-based algorithm guaranteed to converge to pareto optimal solution is proposed. Overall, this is a solid contribution. ", "strength_and_weaknesses": "Strength:\n\n1. This work is well motivated that reformulating the original problem to a MOO can avoid the compromising phenomenon.\n2. The overall logic of deriving the solution along the journey is quite clear: 1) The improvement function can provide a closed-form gradient for the original problem; 2) The closed-form requires  uniform sampling of the tasks, which can be parameterized as a neural network; 3) Optimizing the network requires another sampling of tasks to compute the gradient of the NN; 4) Theoretically, the gap between the full batch and mini-batch of tasks can be bounded, so we can safely adopt the SGD to optimize the NN and finally solve the MOO problem with pareto optimality.\n3. Empirical studies looks very promising that the proposed method works quite well on multiple tasks and is better than the competitors.\n\n\nWeaknesses:\n\n1. There are tons of math equations. The author may need to simplify or condense them a bit, and give timely interpretation for them. It is important to clearly extract and convey the message behind the equations to the readers.\n2. The fonts of Figure 1 and 2 are quite small that it is hard to read.\n3. Typos: e.g., “the converged solution may be comprised” —> “the converged solution may be compromised”\n4. Immediately after Equation (7), it is logically not correct to say “We first rewrite the above as:”, since Lemma 3.1 is to prove that (7) and (8) are equivalent, but before the proof, we don’t know if that’s true.\n5. Definition 2.1. and 2.2. may need to cite some classical literatures rather than those new ones, as these definitions should exist long time ago.\n6. In Table 3, SIMOL has much large variance than others in the MAML category. Would that be a concern?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "U4AEoBrb6z", "reviewer": "Reviewer_f9rZ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers solving meta-learning (formulated as bilevel optimization with the inner level being tasks) using techniques from multi-objective optimization (MOO). Previous work along this line required computing gradients for each task at each iteration, which is expensive. This paper proposes an algorithm to find an $\\epsilon$-Pareto stationary point using only mini-batches of tasks, built on the concept of improvement functions from MOO. Experiments are done on linear regression, few-shot classification, and meta-reinforcement learning. The proposed algorithm SIMOL generally improves over MAML and Prototypical Networks without significantly increasing computational burden, unlike previous work.", "summary_of_the_review": "This paper is a novel application of approaches from MOO to meta-learning, with the intuition that a Pareto optimality would also lead to faster adaptation at test time. The algorithm is mathematically motivated and analyzed, and shown to improve upon MAML with similar computational complexity. However, SIMOL does not always lead to statistically significant improvements and no ablation studies are carried out.", "strength_and_weaknesses": "# Strengths\n- The worst task performance of SIMOL improves over those of vanilla MAML and Prototypical Networks without increasing the computational burden drastically. The overall performance with a PN backbone is also better.\n- The algorithm is well mathematically motivated, and theoretical analysis of convergence is provided along with experiments.\n- The paper is well written and the arguments are straightforward to understand. \n\n# Weaknesses\n- There are no ablation studies, e.g. on the architecture of the network\n- For the MAML backbone, the improvement of SIMOL is often not statistically significant\n- The convergence theory may not be tight; the rate is slower than indicated from experiments\n\n# Questions\n- Are gradient signs missing from line 5 of Alg. 1?\n- How are the hyperparameters chosen?\n- Why is PCGard not included as a baseline?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "TNYswXeuEC", "reviewer": "Reviewer_4MPs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes SIMOL, a soft improvement based multi-objective optimization algorithm, for efficient meta learning. The proposed method first formulates meta learning as a multi-objective optimization problem, where each task is an objective to optimize. Then, to handle the huge number of tasks (e.g., can be up to 7 x 10^6) in meta-learning, it further leverages the soft improvement algorithm with mini-batch update as its scalable gradient-based optimizer for model training. Theoretical analyses and experimental studies have been conducted to show the good properties and practical performance of the proposed method.   \n", "summary_of_the_review": "To my understanding, this is the first work to consider meta learning as a multi-objective optimization problem that treats each task (the number could be huge) as a single objective to optimize. However, due to the major concerns on the multi-objective optimization formulation, algorithm performance, and theoretical analysis, I cannot vote to accept the current manuscript.\n", "strength_and_weaknesses": "**Strengths:**\n\n+ This paper is generally well-organized and easy to follow.\n\n+ To my understanding, this is the first work to consider meta learning as a multi-objective optimization problem that treats each task (the number could be huge) as a single objective to optimize. But I also have some concerns on this formulation (see weaknesses below). \n\n+ The proposed SIMOL method is simple and easy to implement. \n\n**Weaknesses:**\n\n**1. Novelty**\n\nThis work proposes to formulate meta learning as a multi-objective optimization problem. However, many similar works have already been done to leverage multi-objective optimization for multi-task learning [1,2,3]. Given the close relation between meta learning and multi-task learning [4], this formulation is straightforward.\n\nThe second contribution to dealing with the huge number of tasks is indeed interesting. However, the proposed scalable optimization method directly uses the improvement function [5] with the mini-batch update, which makes it like yet another multi-objective optimization method for MTL/meta learning. I also have some concerns on the proposed method's ability to deal with the huge number of objectives (see below).\n\n**2. Multi-Objective Optimization Formulation for Better Generalization**\n\nIt is unclear why the multi-objective optimization formulation could lead to a better generalization performance for meta learning.\n\n*i) Why Pareto solution on the training set is good for meta learning?* \n\nIn multi-task learning, the goal is to find a solution to maximize the performance for all tasks (objectives). Since the tasks could be conflicted with each other, it is reasonable to find Pareto solution(s) for the given problem. In meta-learning, however, the goal is to improve the fast adaption performance for new tasks that are not seen for the training (optimization) process. Why could a Pareto solution on the training set be good for better generalization performance for unseen tasks?\n\nIs it meaningful to formulate meta-learning as a multi-objective optimization problem with huge objectives, especially when the tasks we actually care about (e.g., new unseen tasks) are not in this huge set of objectives? A clear discussion on this motivation could be very helpful.\n\n*ii) Why could the simple linear scalarization approach be worse than other methods for meta learning?*\n\nTo my understanding, the simple linear scalarization approach can still find (stationary) Pareto solutions, although it is true that it cannot find any Pareto solution on the non-convex part of the Pareto front. Since the goal here is to find a single Pareto solution (but not to find the whole Pareto front), it is unclear why the solution found by linear scalarization will perform worse than the solutions found by other multi-objective optimization methods. Different Pareto solutions should be non-dominated with each other, and it is hard to tell which one should be better than the others, especailly for the performane of unseen tasks.\n\n*iii) Why the improvement function method could be better to handle meta learning with huge tasks?*\n\nThe improvement method used in this work is a non-scalarization multi-objective optimization algorithm, which aims to find a common descent direction for **all** objective functions. Many other multi-objective optimization algorithms also have the same goal but with different mechanisms. Given its min-max structure, the improvement method is closely relative to the Chebyshev method for multi-objective optimization [6] (now with equal preferences and adaptive reference point), which has been used for MTL [7,8]. Why could it have better performance for meta learning?\n\n*iv) Why a single Pareto solution could be good for all tasks?*\n\nClosely related to the previous points, since there is no single best solution for all objectives, most multi-objective optimization methods aim to find a good approximation (e.g., a set of representative solutions with different optimal trade-offs) for a given problem [6]. See the related approaches for multi-task learning [9, 10, 11] and also meta multi-objective reinforcement learning [12]. \n\nThe motivation for finding only one single Pareto solution to have the best overall performance for all objectives (and even the new objectives that the algorithm has never seen) is not solid. \n\n**3. Algorithm Performance**\n\nThe closely related work that takes meta-learning as multi-task learning [4] should be included as a baseline. Since this work is \"one step further\" from [4], why not directly develop the SIMOL method based on [4] which could be more efficient and straightforward than the bilevel formulation?\n\n**Much more importantly**, due to the opposite findings in some closely related work, a solid analysis could be needed to support the proposed algorithm's promising performance. The key motivation of this work is to use a multi-objective optimization algorithm to solve the meta-learning problem, and some promising performances have been reported for different problems. However, the current work on multi-task learning leads to an opposite finding. According to [13,14,15], a simple linear scalarization method can perform comparably or even better than all those multi-objective optimization methods for both supervised learning and reinforcement learning. The effect of fine-tuned hyper-parameters (rather than the multi-objective optimization formulation) could lead to significantly different algorithm performances. Due to the close relation between multi-task learning and meta-learning, strong and solid evidence and discussion should be provided to support the finding in this work. \n\n\n**4. Theory**\n\nThis work leverage the theoretical analysis from the improvement function method [5] to give the (Pareto stationary) convergence analysis, and Lin et al. [16] to give the convergence analysis for mini-batch update. However, the more important issues, such as the connection from the Pareto solution to better generalization performance for meta-learning, and the reason why the Pareto solution found by simple linear scalarization could have poorer generalization performance, are not discussed.\n\n**Other Comments**\n\n1. In Figure 1, if batch size =1, both mini-batch MGDA and SIMOL will only have one objective value (either f1 or f2) and its gradient at each step. There should be only a single term in (2) for SIMOL without any maximization comparison. Why did they perform quite differently in this case? \n\n2. It is also interesting to know the performance of the best-10% for different problems.\n\n3. It seems that a \")\" is missing in the definition of \\bar d^* two lines below (16).\n\n[1] Multi-task learning as multi-objective optimization. NeurIPS 2018.\n\n[2] Gradient surgery for multi-task learning. NeurIPS 2020.\n\n[3] Conflict-averse gradient descent for multi-task learning. NeurIPS 2021.\n\n[4] Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation. ICML 2021.\n\n[5] Interactive bundle-based method for nondifferentiable multiobjeective optimization: NIMBUS. Optimization 1995.\n\n[6] Nonlinear multiobjective optimization. Kluwer 1998.\n\n[7] Tchebycheff Procedure for Multi-task Text Classification. ACL 2020.\n\n[8] A Multi-objective / Multi-task Learning Framework Induced by Pareto Stationarity. ICML 2022.\n\n[9] Pareto multi-task learning. NeurIPS 2019.\n\n[10] Efficient Continuous Pareto Exploration in Multi-Task Learning. ICML 2020.\n\n[11] Learning the Pareto Front with Hypernetworks. ICLR 2021.\n\n[12] Meta-Learning for Multi-objective Reinforcement Learning. IROS 2019.\n\n[13] A closer look at loss weighting in multi-task learning. arXiv:2111.10603, 2021.\n\n[14] In Defense of the Unitary Scalarization for Deep Multi-Task Learning. arXiv:2201.04122, 2022.\n\n[15] Do Current Multi-Task Optimization Methods in Deep Learning Even Help? arXiv:2209.11379, 2022.\n\n[16] On gradient descent ascent for nonconvex-concave minimax problems. ICML 2020.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster