Multi-Objective Meta Learning
Authors: Feiyang Ye
Baijiong Lin
Zhixiong Yue
Pengxin Guo
Qiao Xiao
Yu Zhang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Meta Learning
Multi-objective Optimization
Bi-level Optimization

Abstract: Meta learning with multiple objectives has been attracted much attention recently since many applications need to consider multiple factors when designing learning models. Existing gradient-based works on meta learning with multiple objectives mainly combine multiple objectives into a single objective in a weighted sum manner. This simple strategy usually works but it requires to tune the weights associated with all the objectives, which could be time consuming. Different from those works, in this paper, we propose a gradient-based Multi-Objective Meta Learning (MOML) framework without manually tuning weights. Specifically, MOML formulates the objective function of meta learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta learner. To solve the MOBLP, we devise the first gradient-based optimization algorithm by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search. The source code of MOML is available at https://github.com/Baijiong-Lin/MOML.

Multi-Objective Meta Learning
Feiyang Ye Ye Baijiong Lin Zhixiong Yue!.?, Pengxin Guo', Qiao Xiao and Yu Zhang' 14.1 Department of Computer Science and Engineering, Southern University Science and Technology University of Technology Sydney
Eindhoven University of Technology
4 Peng Cheng Laboratory
(12060007, _1inbj,,,,. yuezx, 12032913) 11a1mmmmmmmmmmm.1mm1111111111111 wuw.suh.teseee.. edu. {qiaoxia07222, yu zhang. ust) com
Abstract
Meta learning with multiple objectives has been attracted much attention recently since many applications need to consider multiple factors when designing learning models. Existing gradient-hased works meta learning with multiple objectives mainly combine multiple objectives into a single objective in a weighted sum man- This simple strategy usually works but it requires to tune the weights associated with all the objectives, which could be time consuming, Different from those works, in this paper, we propose gradient-based Multi-Objectin Meta Learning (MOML) framework without manually tuning weights. Specifically, MOML formulates the objective function of meta learning with multiple objectives Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is solve several possibly conflicting objectives for the meta learner. To solve the MOBLP. we devise the first gradient- based optimization algorithm by alterna- tively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-basee multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient- gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search. The source code of MOML is available at httttss :eattps:/giiiee //github. com com/Bai joong--n-- Lin/MOML
Multi Oileeivv
Introduction
In the past few years, deep learning has achieved great success in various fields 43 because it can effectively and efficiently process massive and high-dimensional data. However, training deep learning model from scratch often requires large amount of labeled data to learn large number of model parameters and needs to choose hyperparameters by hand.
As way to address those problems by enabling models to learn how to learn, meta learning has attracted considerable attention recently 19 20) Meta learning gains knowledge from multiple meta training tasks so that the knowledge can be reused in new tasks new environments rapidly with few training examples. Taken broadly, objective functions of meta learning models are usually formulated as bi-level optimization problem where the lower-level subproblem represents the adaptation given task with learned meta parameters and the upper-level subproblem tries to optimize these meta parameters via meta objective [19]. Hence, from this view, meta learning has wide range of applications such as hyperparameter optimization [13, Neural Architecture Search (NAS) 29 and Reinforcement Learning (RL) [68]
Equaa contribution. Corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In many studies on conventional meta learning methods and applications, there is only single meta objective in the upper-level subproblem. For example, the Model-Agnostie Meta-Learning (MAML) method [12] only measures the performance on a validation dataset in the upper-level subproblem to evaluate the learned initialization of parameters. DARTS [2)]. a differentiable method for NAS, evaluates the performance of the searched architecture on the validation dataset. However, in real-world applications, there are usually more than one objective to be considered. For example, for MAML, we may need to consider not only the performance but also the robustness which can help adapt to new tasks with the learned initialization. Similarly, the network size and performance should be balanced in NAS, especially when the searched architecture will be deployed to devices with limited resources such as mobile phones. In those applications, we can see that there is a need to balance multiple possibly conflicting objectives in meta learning.
Meta learning with multiple objectives thus has drawn much attention in recent studies. Specifically, some works study specific meta learning problems in the multi-objective case, such as multi-objective
NAS 66 57
| 14 multi-ohjective RI. 44, and so on. However, those works either linearly combine
multiple objectives into single objective for the upper-level subproblem [65| 63 I or utilize multi-objective bi-level evolutionary algorithms [5 54 47] to handle it. The former approach needs to tune weights associated with all the objectives, which is time consuming, and its performance depends on the set of candidate weights in, for example, the cross validation method. The latter approach, whose computational complexity is even higher, has no convergence guarantee in the optimization process and is not easy to be integrated into gradient- based learning models such as deep neural networks, which limits its use in many learning models.
To alleviate those limitations existing works, in this paper we propose unified gradient-baser Multi- Sulti-Obbjective Meta Learning (MOML) framework with a convergence guarantee. The MOML framework formulates objective functions in meta learning with multiple objectives as a Multi- Objective Bi-Level optimization Problem (MOBLP), where the lower-level subproblem is to learn the adaptation to a task similar to vanilla meta learning and the upper-level subproblem minimizes a vector-valuee function corresponding to multiple objectives for the meta learner. To solve MOBLP, we devise the first gradient-bbsee optimization algorithm by alternatively solving the lower-level and upper-level subproblems via gradient descent method and gradient-ber based multi-objectinn optimization method such as [7]. respectively. We theoretically prove the convergence properties of the proposed based optimization method. To show the effectiveness of the MOML. framework, we apply it to several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and NAS, where multi-task learning is firstly formulated from the perspective of meta learning. In summary, the main contributions of this paper are four-fold.
We propose a unified MOML framework based on the MOBLP and devise a gradient-based optimization algorithm for the MOML framework.
We prove the convergence property of the proposed optimization algorithm.
We formulate several learning problems as instances of the MOML framework.
Experiments show that MOML achieves state-of-the-art performance on those learning problems.
2
Related Work
Meta Learning. Meta learning (a.k.a. learning to learn) learns knowledge from multiple tasks and then adapts it to new tasks with a few samples quickly. Many studies in meta learning mainly focus on solving the few-shot learning problem. From this view, meta learning can be divided into three main categories, including metric-based approach [55] [66.. model-based approach (28), and optimization-basee approach [22 39). For example, as an optimization-basee
method, MAML learns an initialization of model parameters so that a new task can be learned with a few training samples by fine-tuning the learned initialization. A widely-used formulation in meta learning can be cast as a level optimization problem, where the upper-level subproblem is to learn meta parameters by minimizing a meta objective and the lower-level subproblem is to quickly adapt to new tasks with meta parameters 45 [9]. For example, MAMI. adapts to a new task by using the associated training dataset and the learned initialization in the lower level, and then updates the initialization according to the validation performance in the upper level. From this perspective, meta learning is general

---

learning paradigm and has more general applications [9]. In this paper, we study meta learning from the perspective of the bi-level optimization.
Oulti-Objective Optimization. Multi-objective optimization to address the problem of simulta- neously minimizing multiple objectives, which may conflict with each other. The simplest way to handle multi multi-objective optimization is to convert to a singlc-objective optimization problem via the linear scalarization approach which minimizes the weighted sum of all the objectives. Actually, many machine learning algorithms adopt this approach. For example, minimizing the regularization term to control the model complexity and minimizing the training loss are two conflicting objectives, and learning model is usually to minimize a linear combination of those two terms. Recently, many kinds of advanced multi-objective optimization algorithms are proposed. such as evolutionary algorithms [73,, population- -based algorithms [15], gradient- based algorithms [ 37]. and so on. Some of them have been successfully applied solve machine learning problems [22] In this work, we focus on gradient-basee based algorithms because this approach can be easily integrated into gradient-based machine learning models such as deep learning models.
The MOML Framework
In this paper, the proposed MOML framework has unified objective function, which is formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP), as
min F(w, (x) = (Fi (w, F2(w, ot), Fm (w. a))² s.L. w S(a). aEA,EER
where function F: IP x R + - is vector-valued jointly continuous function for the m desired meta objectives and A is a nonempty compact subset of IR". In problem (I.. S(a) is defined as the set of optimal solutions minimize fw, a) w.r.t. w, i.e.
S(a) = arg min (w)
When equals 1, problem (T) reduces to a standard Bi-Level optimization Problem (BLP), which widely-used formulation in meta learning [19], and hence from this perspective, the MOML framework is generalization of meta learning. In problems ( and (2). F is called the Leper-Level (UL) subproblem and f I2 R" > IR is the Lower-Level (LL) subproblem. The LL subproblem can be considered as constraint to the UL subproblem. In MOML, F contains multiple meta objectives to be achieved for the meta learner and f defines the objective function for current task such as the training loss. In Section 5] will see the application of MOML in different learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search.
To solve problem [ there exist a type of works 5 54 which adopt multi-objective evolutionary algorithms. However, such methods have a high complexity without convergence guarantee and are not easy to be integrated with gradient-baset models such as deep neural networks. Hence, we do not include them in experiments. To the best of our knowledge, there is no gradient-based -baeed optimization algorithm with convergence guarantee to solve an MOBLP, which is what we will do the next section.
4
Optimization
In this section, we devise a general algorithm to solve the MOBLP (i.e., problem (1) and provide convergence analyses under certain assumptions.
4.1 Lower-Level Singleton Condition
Due to the complicated dependency between UL and LL variables, solving the MOBLP is challenging, especially when optimal solutions of the LL subproblem are not unique.
For BLP with single objective in the UL subproblem, many studies that the LL subproblem only admits a unique minimizer for every A. This condition can simplify both the optimization process and convergence analyses and first fomally introduced as Lever-Level Singleton (LLS) condition in [30].
13 52 potentially require
3

---

For the MOBLP, the LLS condition is necessary. If the LLS condition does not hold, the MOBLP is even ill-defined 9 To see this, suppose for fixed ag. we get a set of solutions S(ag) for the LL subproblem. Since F is vector-valued in problem (1) it is unclear that at which
((00) ) the UL. subproblem F should be evaluated. With the L.I. condition, problem ① can be simplified as
min ((a) (a) s.t. w (a) = arg min aw) aEA
4.2 Gradient-based Optimization Algorithm
Here we design gradient-basede based optimization algorithm to solve problem (3). Usually, there is no closed form for the solution w" (ax) of the L.L subproblem and so is difficult to optimize the UL subproblem directly. Another approach
use the optimality condition of the LL subproblem (i.e., [w, a) = 0) as equality constraints for the UL subproblem in a way similar to [4]]. However, this approach only works for LL subproblems with simple forms and cannot work for general learning models.
To solve problem (3). take strategy similar to the alternating optimization. In the first part of each iteration (corresponding to steps 3-6 in Algorithm [ we solve the LL subproblem via gradient descent methods. Specifically, with initialization wp for the L.L. variable where the index of the iteration is omitted for notation simplicity, the solution of the LL. subproblem can be updated for K steps as w (a)) = To (wm (a), a.) k 1,.... K 1, where Ta represents an operator update Here we consider a first-order gradient descent method for Ta such as the Stochastic Gradient Descent (SGD) method and Ta can be formulated explicitly as To (w)((a)), a) = Wa (x) vV f (a), a), where > 0 denotes the step size and V (ax), ar) denotes the derivative of f w.r.f. w at (a). In the second part of each iteration (corresponding to steps 7-9 in Algorithm [ by fixing
the value of w as the current solution wx (a) obtained in the first part, we solve the UL subproblem
as
min Φ (a) F F( (a).
Problem 4 is an unconstrained Multi-Objective optimization Problem (MOP) and we can use any multi- objective optimization algorithms to solve it. Here we choose gradient- based multi-objective optimization methods as they can be seamlessly inte-
Algorithm 1 Optimization Algorithm for MOML
Input: numbers of iterations (T, K). step size (H,, v) 1: Randomly initialized ox:
2:
for 1 do Initialize w((aa:;; for j to K do
3:
4:
grated into any gradient- based learning
5:
wa(a)) (j-) (VV end for
6:
framework. There are several gradient-
7:
Compute gradients V. (w (a.). a.) for the i's;
based multi-objective optimization al-
8:
Compute the gradient
gorithms [44 7 588 and they com-
d(wk (ax), an) = ( (wk (ac), at)
monly find an appropriate descent di-
9:
a141
vd(wkk (a.), an) with step size
rection d for all the objectives in F by aggregating their gradients w.r.t. a. So such process is denoted by d we adopt a simple gradient- based MOP method called Multiple Gradient Descent Algorithm (MGDA) 77. whose details are introduced in Appendix A.. due page limit.
10:
end for
(OOPSolver((Va.H)
(a), )= in Algorithm In this paper,
The entire algorithm to solve problem (3) is shown in Algorithm [ which to the best of our knowledge is the first gradient-based optimization algorithm for MOBLPs. In Algorithm [ we obtain only one solution for MOBLPs, which is different from multi-objective evolutionary algorithms that can find a population of solutions. How to obtain multiple nearly optimal solutions for MOBLPs is beyond the scope of this paper and we will study it in our future work.
4.3 Convergence Analysis
Algorithm[I is simple and intuitive, but its convergence properties are unknown, which is what we aim to study in this section. As an MOBLP, problem (3) cannot reduce to SOML with a scalar-valued objective function in the upper-level subproblem when using gradient-based Algorithm I solve it. Therefore, has different theoretical properties from SOMI. as we need to focus on the convergence properties of minimal point set instead of minimum scalar in the UL subproblem.

---

We first recall some notions about vector-valued functions. Consider vector valued function g(z) IR" + R (m, n E N, 2 2). We denote by Min g(2) the set of all the minimal points of function g(z). Min g(z) is also called the Pareto frontier or Pareto-optimal set. The corresponding
efficient solution or Pareto-optimal solution set of g(>) is denoted by Eff(g(()). The convexity of vector-valued functions is called the P-convex. The details of these definitions can be found in
Appendix A
To help analyze convergence properties of Algorithm with a basic assumption about the LLS condition which is mentioned in Section 4.1 and is widely adopted BLPs 52 we can obtain the following result.
Theorem 1. If f(w,a) a) is jointly continuous, arg min., f(w, a) is singleton for every a A, and w" (at is uniformly bounded on A, then the function (a), a) is continuous w.r.l. a.
Because A is compact set, Theorem immplis the existence of solutions. Theorem and the uniform convergence of wg( (a) can further imply the convergence for the solution of the LL subproblem, which is similar to that of the standard BLP problem [13]
For the convergence of Algorithm D we need to analyze minimal point sets of the images of perturbed functions şa (a:) and p(aa) which are defined in problems (4) and (3) respectively. We consider the most natural set convergence under this setting, i.e., the Kuratowski-Painkee
set-convergence, Due to page limit, please refer to these definitions in Appendix A Under certain assumptions which are we have the following convergence results.
used analyses of BL. and MOP 13 35
Theorem 2. In addition to Theorem 7 it is assumed that (i) F(., a) is uniformly Lipschitz continuous; (it) The iterative sequence {wx(a)) converges uniformly to w" (a) on A as K + -20; (tii) A is convex set; (iv) PK is P-convex and &p is strictly P-convex. Then, the Kuratowski-PPu Painkleveé set-convergence of both the minimal point set and efficient solution set in ( i.e.,
Min (x((a) Min p(a), Eff xpx (ax + Eil sp(a).
Thus, under some specific assumptions, Theorem provides a theoretical justification for the conver- gence of Algorithm to solve MOBLPs.
5 Applications of MOML
In this section, we introduce several use cases of the MOML framework in several learning problems, including few-shot learning, semi-supervised domain adaptation, multi-tas task learning, and neural architecture search.
5.1 Few-Shot Learning
Few-Shot Learning (FSL) aims to tackle the problem of training a model with only a few training samples 64
Recently, FSL is widely studied from the perspective of meta learning by using prior knowledge in the meta training process. Most studies in FSL only consider the classification performance. However, in real- world applications, the performance not the only important focus. For example. we expect FSL models to not only have good performance but also be robust to adversarial attacks (26), which may improve the generalization of FSL models. In the following, we can see that this setting can naturally be modeled by the proposed MOMI. framework.
Problem Formulation. Suppose there is a base dataset Drase with a category set Cruse and a novel dataset Doovel with a category set Cnovel. where Cbase 7 Cnovel = 0. The goal of FSL is to adapt the knowledge learned from Diase to help the learning of "nooett In the ith meta training episode, we generate from D Drasc an N-way k-shot classification task, which consists of a support set ()))) and query set ())()) For the robustness, we add perturbations generated by the Projected Gradient base generate perturbed query set base The objective function of the FSL model that considers both the performance and the robustness can be formulated
Descent (PGD) method [26) into each data point in De(()
adds
min (()) a, Dille ) (( a, base
Dali),ado)
s.t. (a) = arg min Cr a, o(i))
(5)
5

---

where w represents model parameters, a denotes the meta parameters to encode common knowledge that can be transferred to novel tasks, and Cr(w, a, D) denotes the average classification loss of a model with model parameters and meta parameters w on dataset 'D. In the UI. subproblem of problem 5 the first objective measures the classification loss on the query set based on w>() (a) obtained by solving the LL subproblem and the second objective measures the robustness via the classification performance on the perturbed query set. Problem (5) provides general formulation, which depends on what a represents, for FSL. We adapt problem (5) to three FSL methods (i.e..
MAML [12], ProtoNet [55] and BOIL [40] and the detailed formulations are put in Appendix [C] It is easy to see that problem (5) fits the MOMI. framework and we can use Algorithm ! to solve it.
For an MOBLP such as problem ( common approach is to transform the UL subproblem into a single objective problem via the linear scalarization approach (65 633 [T]. In contrast with MOMI,, this approach is named as Single-Objective Meta Learning (SOML) in this paper and used as an extra baseline method in our experiments. Specifically, with fixed weights {7 which satisfy E 7/- = and 7/2 2 O, problem {1) is transformed to a standard bi-level optimization problem as mill Σ 7: Fi(w, a). This approach heavily depends on the selection of weights (7% ) and inappropriate weights usually lead to poor performance.
Experiments. Experiments are con- ducted on two FSL benchmark datasets, ImageNe [6]] and CUB-200-2011 (referred to as CUB) (62]. Due to page limit, experimental settings and experi- mental results the CUB dataset put in Appendix[C The original baseline meth- ods (i.e., MAMI,, ProtoNet, and BOIL) are trained in a conventional way and we evaluate the clean accuracy and PGD ac-
Table 1: Classification accuracy (abbreviated as "Clean Acc.") ") and PGD accuracy (abbreviated as "PGD) Acc.") ") on the mini-ImageNNet dataset for 5-way k-shot FSL. The best result in each group of methods is highlighted in bold and the best result in each setting is annotated with underline.
Method
Clean Ace.
PGD Acc.
B-score
MAML MAML+SOML
45.24:0.81
1.18:10.15
2.10:0.52
00.78=0.75
23.91=0.67
99.8320.43
MAML+MOML (ours)
39.23=0.77
25.8010.67
31.12:0.70
curacy of all the methods, where the clean
I-shot
ProtoNet [55]
44.67--0.75
2.60++0.21
3.73++0.35
accuracy is measured on the original test query set and the PGD accuracy is tested on the query set perturbed by the PGD at-
ProMoNet-SOML
38.65-0.722
13.10+0.66
28.67-00.67
ProloNet-MMM.... MOMI.
35.06-+0.70
21.24--0.65
40.51-0.66
BOII. [0
10.6
3.05
1.63.10.61
BOIL+SOML
00.44:0.79
25.94:10.69
31.29:10.75
tack. As the clean accuracy and the PGD accuracy are reported to be conflicting to each other [70], inspired by the widely- used F1 Fc-re based on the precision and
BOIL+MOMI (ours)
41.22.10.83
10.75
32.98 1..79
MAML
6188110.77
2.82.10.1
5.01100.43
MAML+SOML
56.1620.722
34.8520.72
42.91=0.71
MAML+MOML (ours)
55.66.10.78
39.3810.77
15.89
5-hhot
ProtoNet [55]
66.55=0.70
0,68++0.09
1.31 18
recall, we propose new metric called Balance score (B-score), which is defined as B-score = 2 (CA x PA) (CA + PA) with CA and PA denoting the clean accu- racy and PGD accuracy, respectively, to fully measure the performance in terms of the clean accuracy and PGD accuracy.
ProtoNetSOML
59.11=0.71
30.41=0.73
46.93-20.71
ProtoNet+ (ours)
58.72=0.74
41.5990..75
48,59-+0.74
ROIL. [i]]
66.02-00.22
4.85+00.0
1.11+0.51
SOIL+
58.54110.76
34.78.10.75
42.94110.88
MOMT. (ours)
60.21+00.99
35.47+0.78
44.37-0.78
The average results over 600 testing tasks on the mini- Image-t dataset are presented in Table [ From the results, we can see that the adversarial robustness of the original FSL methods (i.e., MAML. ProtoNet, and BOIL) is poor, while SOMI. and the proposed MOMI. can significantly improve the PGD accuracy. For example, MOML. can improve the PGD accuracy of ProtoNet by about 40.91 % under the 5-shot setting. Since the classification accuracy and adversarial robustness are conflicting 770, the clean accuracy of SOML and MOML slightly drops when compared with the orignal FSL baselines. The B-score of MOML is higher than SOML, which indicates the multi-objective formulation in the UL subproblem of MOML is better than the single-objective formulation in SOML.
5.2 Supervsup Domain Adaptation
Semi-Supervised Domain Adaptation (SSDA) aims to address the domain shift between two domains so that the model trained in label-rich source domain can be adapted to target domain with limited labeled samples and abundant unlabeled samples (69 A widely used approach for SSDA is to align the distributions of two domains by finding some domain-invarian components. There are usually three objectives to be considered, including two classification losses on two domains and an alignment loss to measure the domain discrepancy. While existing works such as [49] [7] optimize

---

all the objectives by simply computing weighted sum of them, we formulate the SSDA problem as an MOBLP under the MOML framework.
Problem Formulation, Given a source domain S and a target domain T, the source domain has a large labeled dataset Ds and the target domain has limited labeled dataset D/ well large unlabeled dataset Di,, where Dy DJ U Dij denotes the entire dataset for the target domain. The average classification losses in the source and target domains are represented by (w, Ds) and LD (w, D) respectively, where w is the model parameter. The aligmment loss denoted by (a( a, Ds. D+) aims to learn
projection space by minimizing the local maximum mean discrepancy in DSAN (74) or domain-invarisnn prototypes by maximizing the entropy in MME 44), where is the initialization of Then can formulate the SSDA problem under the MOML framework as
components such as
min (Cn(w" (a): Ds), (a)) Dly), Ca(w, a, Ds, Dz))
s.t. w" (a) arg min L (wc, a, Ds. D4))
(6)
In the LL subproblem of problem (6). we aim to learn model to find a domain- component between two domains via the alignment loss Ca by optimizing w with an initialization a, and in the UL subproblem, we expect to improve the model further by updating a via minimizing the two classification losses together with the alignment loss. Here a acts similar to the parameter initialization in MAML (i.e., a in problem (5)) and helps learn w in the LL subproblem, but it does not require any adaptation on the testing process. We can follow two state-of-the-a art domain adaptation models (i.e., DSAN and MME) to design the alignment loss and due to page limit, the detailed formulation of the alignment loss L, put Appendix [
Although a bi-level objective function is also formulated Meta-MME (27]. there exist two sig-
nificant differences between Meta-MME and the proposed MOML method. Firstly, Meta-MME aims to learn the initialization of the network parameters by minimizing the source classification loss and the alignment loss in its LL subproblem and then validate on a few target labeled samples in
the UL subproblem, which is different from the proposed MOML formulation (i.e., problem (6)) Secondly, the proposed MOML method considers a multi-objeccivve optimization problem in the UL subproblem, which is different from Meta-MME. Empirically, MOML outperforms the Meta-MME method as shown in Table 2
Table 2: Classification accuracy (%) on the Office-31 dataset with ResNet-50 backbone for SSDA. it means the corresponding method is appropriately modified to adapt to our experimental setting according to the released code. $ indicates that the corresponding model is reimplemented by us. 1. and - in the subscript indicate an increase, a decrease, and no change, respectively, when compared with the original method in two groups of models based on DSAN or MME. The best results in each group of models (i.e., DSAN and MME) are highlighted in bold and the best results of each transfer task are annotated with underlines.
Method
AD
D-
A
W-A
D
W-
Avg
S+T
93.58
74.16
92.17
74.08
98.01
100
88.67
DANNÍ
14
93.09
74.52
91.60
75.07
98.58
100
88,81
ENT
16
93.33
74.16
94.44
75.03
97.86
100
89.13
APE
94.81
76.10
91.80
76.02
97.29
99.75
89.29
ADR 50
93.33
76.73
93.30
76.51
97.86
100
89.62
CDAN 32
94.32
75.25
92.88
76.98
98.43
100
89.64
DSAN 14
93.83
76.82
93.59
75.68
98.43
100
89.73
DSAN SOML
94.32,
76.91
94.16;
75.99,
97.72,
100
89.85,
DSAN MOML (ours)
94.08
77.13-
94.59,
75.96,
98.36
100
90.02,
MME 49
92.09
77.71
93.73
77.27
98.01
100
89.80
Meta-MME
27
92.10
77.01
94.30
76.87
98.29
100
89.76
MME+ SOML
92.09
77.57
94.30,
77.20,
98.29,
100-
89, 89.90,
MME+ MOML (ours)
94.32,
78.30-
94.44
77.71
98.43,
100
90.53,
Experiments.
Experiments are conducted on the Office-31 dataset
48 which has 3 domains:
Amazon (A), Webcam (W) and DSLR (D). By following [59] 33), we construct all six transfer
7

---

tasks. Each class in the target domain has three labeled images in the training process by following
49
Baseline models in comparison include a deep neural network (denoted by 'S+T') that is
trained on Ds U D4 and other eight state-of-the-arts domain adaptation methods: ENT [16), DANN 14. ADR (50), CDAN (22 MME [49], Meta-MME [27]. APE [23] and DSAN 74. As DSAN and MME are two state-of-the-art domain adaptation models, to improve their performance further, SOML and MOML adopt the alignment loss in DSAN and MME, respectively, leading to two groups of models, i.e., (DSAN, DSAN+SOML, MMAN+MOML) and (MME, Meta-MME, MME+SOML, MMM+MOML). All the methods except S+T are trained on Ds UDT. Due to page limit, details of baselines and experimental settings are put in Appendix D
Experimental results on the Office-31 dataset are shown in Table 2 The incorporation of SOML into DSAN and MME can consistently improve the performance of two baseline models and the proposed MOMI, method further improves the classification accuracy. For example, MME+ significantly improves the performance of the first five tasks and achieves the perfect performance (i.e., the 100% accuracy) on the last task as most baselines did. Moreover, compared with state-of- the-art baselines, DSAN+MOML achieves the best result (i.e., 94.59%) on transfer task A- W and MME+MOML has the best accuracy (i.e., 78.30% and 77.71%) on transfer tasks D- and W-A Besides, MOE+ achieves the best average classification accuracy of 90.53% and significantly outperforms all of the baselines, which indicates the effectiveness of the proposed MOML method.
5.3 Multi-Task Learning
Multi Task Learning (MTL) 72) aims to improve the performance of multiple tasks simultancously by leveraging useful information contained in these tasks. Learning the loss weighting is a challenge in MTL and there are some works 51 71] to solve this problem. Among those works. Sener and Koltun (51) formulate multi-task learning problems from the perspective of multi-objective optimization and implicitly learn the task weights via MGDA, Liu et al. [1] estimate the task weight of each task as the ratio of the training losses in the last two iterations for the corresponding task, and Yu el al. [71] project each task's gradient onto the normal plane of the other. Different from those works which are all based on single-level optimization problems on the entire training set, for the first time we formulate this problem as MOBLP based on the split of the entire training dataset and solve this problem based on the MOMI. framework.
Problem Formulation. Suppose there are m tasks. The ith task has dataset Dj for model training. Here each D, is partitioned into two subsets: the training dataset D.r and the validation dataset Drat, where Dfr is used to train a multi-task model and Draa is to measure the performance of the multi-task model on the ith task. (=:w), the learning function of the multi-task model parameterized by w, receives data points from the m tasks and outputs predictions. Let a; E [0, 1] denotes the loss weight for the ith task. The goal is to jointly learn the loss weights a (a1..... am,) and the model parameter W. The objective function of the proposed method under the MOML framework is formulated as
min (CMTL(W" (a), D()) CMTL(W" (a), Dral)) a.L.rrlw D.", 0 ai a. 1 Σ a =
a
s.t. w( = arg min m
m
where (MTL(W, D) - E(x,)eD ((f(x;)) v) denotes the average loss of f(:w) on a dataset DD with [D] denoting the size of D and ((.-)) denoting a loss function. In the L.I. subproblem of problem 77. when given weights in a, we aim to learn a MTL model to get optimal parameters wt on the training dataset and in the UL subproblem, we expect to update via minimizing the loss of the trained MTL model with parameters wn on the validation dataset of each task.
Experiments. Experiments are conducted on the NYUv2 53 Office-31 and Office-Home [60]
datasets. Baseline methods in the comparison include different loss weighting strategies such as Equal Weights (EW), Dynamic Weight Average (DWA) [3I] with the temperature parameter T as 2, MGDA (1), PCGrad Z and SOML, and they are built on two multi-task architectures, including Deep Multi-Task Learning (DMTL) that adopts the hard-sharing structure to share the first several layers and Multi-Task Attention Network (MTAN) [I]. The ResNet-50 is used as the backbone and we do not use data augmentation. Due to page limit, the introduction of datasets, details
8

---

experimental settings. and experimental results on the Office-31 and Office-Home datasets are put in Appendix A
Table 3: Performance on the NYUv2 dataset with three tasks: 13-class semantic segmentation, depth estimation, and surface normal prediction. The best combinations of the architecture and weighting strategy are highlighted in bold. The best results for each task on each measure are annotated with underlines. (1) means the higher (lower) the result, the better the performance.
Segmentation
Depth
Surface Normal
Weighting Strategy
Architecture
Angle Distance
Within t
mIoUt
Pix Ace
Abs Err
Rel Err!
Mean
Median 4
11.25
22.5 1
30
EW
52.71
74.78
0.3886
0.1581
23.8568
17.3537
34.57
60.45
71.63
DWA 11
$2.72
75.11
0.3931
0,1631
23.7894
17.3320
34.57
60.51
71.67
DMTL
MGDA
$2.89
74.87
0.3963
23.7513
17,2685
34.74
60.64
71.75
PCGrad 7
53.22
75.45
0.3920
0.1658
23.2904
16,8728
35.47
61.58
72.56
SOML
53,20
75.22
0.3923
0.1628
23.6888
17.0228
35.42
61.03
71.97
MOML (ours)
54.98
75.98
0.3877
0.1618
23.2401
16.7388
35.90
61.81
72.76
EW
53.97
75.90
0.3794
0,1580
22.8743
16.5502
36.54
62.52
73.31
54.12
75.79
0.3902
22.9691
16.6212
73.28
MGDA 61
54.38
75.55
0.3854
0,1583
22.9396
16,4670
36.70
62.55
73.23
MTAN
PCGrad 7A
54.40
76.13
0,3830
0.1581
23.0040
16.4636
36.67
62.65
73.34
SOML
54.03
75.48
0.3829
0.1581
22.8279
16.4259
36.74
62.67
73.46
MOMI (ours)
54.23
75.63
0.3843
0.1567
22.7530
16.2468
37,20
63.09
73.65
Experimental results on the NYUv2 dataset are shown in Table Firstly, SOML outperforms the EW. DWA, and MGDA strategies when using the DMTL architecture and achieves comparable performance with the four baselines with the MTAN architecture. It indicates the proposed bi-level formulation (i.e., problem (7) with weighted combined objectives in the UL subproblem used in SOML) can achieve very good performance when compared with the state-of-the-arr baselines. Secondly, the MOML method outperforms the SOML method under both DMTL and MTAN architectures, which means the multi-objective formulation in the UL subproblem is better than the single-objective formulation. Finally, it is noticeable that the proposed MOMI. method achieves state-of-the-art results in many metrics. For example, when training with the DMTL architecture, MOML can achieve 54.98% in terms of the mloU, which significantly higher than other baselines even with the advanced MTAN architecture. MOML with the MTAN architecture achieves the best results on six metrics. Experimental results on the NYUv2, Office-31, 1, and Office-Home datasets, the latter two of which have their results in Appendix E, show that the proposed MOML framework achieves state-of the-art performance for learning loss weights in MTL.
The loss weights learned by the proposed MOML are (0.3120, 0.3804, 0.3076) and (0.2541,0.44) 0.4860, 0.2599) for the three tasks (i.e., semantic segmentation, depth estimation, and surface normal prediction) when using the DMTL and MTAN architectures, respectively. It is in- teresting to find that when using different architectures, the weight of the depth estimation task commonly the highest, while the other two tasks have similar weights.
5.4 Neural Architecture Search
NAS aims to design the architecture of neural networks in an automated way. In architecture design, usually need to consider multiple factors. For example, we expect that the searched architecture has good performance, behaves robustly to noises, and consumes low resources. In this case, we formulate the NAS with multiple objectives case of MOML. Due page limit, the introduction of datasets, details on experimental settings, and experimental results are put in Appendix
Problem Formulation. By following the DARTS method 29), in operation space denoted by O, each element is an operation function o(-) and each cell is a directed acyclic graph with N nodes, where each node represents hidden representation and each edge (i.j) denotes candidate operation o(.) with a probability ab (t.j) Therefore, a = {ao ((-5) { FooD is a representation of the neural architecture, where E denotes the set of all the edges in all the cells. The entire dataset is split into training dataset denoted by D,r and a validation dataset denoted by D
The multi-objective NAS considers three objectives: the classification accuracy, adversar- ial robustness and the number of parameters. We formulate three corresponding losses as n (w, Deat), n (w, a. Dodv), and ( where denotes all the model parameters in the

---

neural network and D denotes the perturbed validation dataset by adding perturbations on each data point. Ev (w, a, D) denotes the average classification loss on dataset 'D of neural network with parameters w and an architecture
1
CN (w. a D)
Σ
[(w, x,), v),
(x.
where f(W,x,) x, denotes the loss function for each sample and n is the size of D.
To formulate Enxop(x). we denote by no the number of parameters associated with operation searched architecture a. Then Wop can
o and by Nn) the number of parameters be computed by (a) L(JjeR (((-)), where ( is the number of parameters of the searched operation on the edge (i.j). As we determine the operation of each edge by selecting the one with the largest probability, hence we have (a.)) arg max.eo a. As the arg max
operation is non-differentiable, use the softmax function approximate Noop((:) as (a) Ro. Therefore, to search a network architecture with expected size L, (c))((t) can be formulated as
exp(a!!.))
EGSEER EoEo Ev'co expla "apla
Lnop = |Noop T.--
(8)
Finally, the overall formulation for the multi-objective NAS is formulated min (Cn(w" oz, Deat), L (w" (ax), ar Dodi)) Loop s.t. w (a) arg min (N(ww, D.y).
(9)
In the LL subproblem of problem p when given the architecture a, we can train a model with optimal parameters
the training dataset and in the UL subproblem, we expect to update the architecture a by making trade-off among the validation loss, the adversarial robustness and the number of parameters. Obviously problem (9) matches the MOML framework. It is easy to that the DARTS method is special case problem (9) when its UL subproblem contains the first objective only and hence problem (9) generalizes the DARTS method by considering two more objectives.
Compared with the NSGANetV2 method [34] that utilizes a multi-objective bi-level evolutionary algorithm, MOML more efficient and has convergence guarantee. Moreover, NSGANetV2 uses ensembled surrogate models to predict the accuracy of an architecture, which may incur performance gap between the UI. and LL subproblems. The LL subproblem of NSGANetV? only chooses over 300 candidate architectures for evaluation with a supernet constructed for weight sharing, which may lead to suboptimal solutions.
Conclusions
As a generalization of meta learning based on the bi-level formulation, a simple MOML framework based on the multi-objective bi-level optimization is proposed in this paper. In the MOML framework, the upper-level subproblem takes multiple objectives of a learning problem into consideration. To solve the objective function of the MOML framework, gradient-based optimization algorithm is proposed and the convergence property of this algorithm is studied. Moreover, several use cases of the MOML framework are investigated demonstrate the effectiveness of the MOML framework. In our future work, we will apply the MOML framework to more learning problems.
Acknowledgements
This work is supported by NSFC grant 62076118. We thank Dr. Jin Zhang for helpful discussion.
References
[1] Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all all: Train one network and specialize it for efficient deployment. In Proceedings of the 8th International Conference on Learning Representations, 2020.
10

---

[2] Rich Caruana. Multitask learning, Machine Learning. 28(1):41-75, 1997.
[3] Liang- Chich Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. with atrous separable convolution for semantic image segmentation. In Proceedings of the 14th European Conference on Computer Vision, volume 11211. pages 833 851, 2018.
[4] Xi Chen, Ali Ghadirzadeh, Marten Björkman, and Patric Jensfelt. Meta-lcarning for multi-objective reinforcement learning- Pruceedings of the IEEE/RSJ International Conference on Intelligent Robots and Syxtems, pages 977-983, 2019.
[5] Kalyanmoy Deb and Ankur Sinha. Solving bilevel multi-objective optimization problems using evolu- tionary algorithms. In Proceedings of the 5th International Conference on Evolutionary Multi- Multi-Criterion Optimization, pages 10-124, 2009.
[6] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
[7] Jean-Antoine Désidéri. Multple-gradient descent algorithu (MGDA) for multiobjective optimization. Comptes Rendus Mathematique, 350(5):31331) 2012.
[8] Justin Domke. Generic methods for optimization- based modeling. Proceedings of the 15th Artificial Intelligence and Statistics, pages 3 318-326, 2012.
[9] Gabriele Eichfelder. Twenty years of continuous multiobjective optimization. 2020.
[10] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient-base agdel-ag meta-learning algorithms. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 1082 1092, 2020.
[11] Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Measuring and hamnessing transference in multi-task learning arXiv preprint arKiv:2010. 15413, 2020.
[12] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learnin for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning, pages 1126 11351113, 2017.
[13] Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In Proceedings of the 35th International Conference on Machine Learning, pages 1568-1577, 2018
[14] Yaroslav Ganin and Victor S. Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 1180-1189 2015.
[15] loannis Giagkiozis, Robin Purshouse, and Peter J Fleming. An overview of population- algorithms for multi-objective optimisation. International Journal of Systems Science, 4619):1572-1599, 2015.
[16] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems, pages 529-536, 2004.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778,
2016.
[18] Nathan Hilliard, Lawrence Phillips. Scott Howland, Artëm Yankov, Courtney D Corley. and Nathan O Hodas. Few-shot learning with metric-agnostic conditional embeddings. arXiv preprint 1802 04376. 2018.
[19] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.0545 2020.
[20] Mike Huisman, Jan N. van Rijn, and Aske Plaat. survey of deep meta-learning. Artificial Intelligence Review, 54(6):448-44541, 2021.
[21] Xiaojie Jin, Jiang Wang, Joshua Slocum, Ming Hsuan Yang. Shengyang Dai. Shuicheng Yan, and Jiashi Feng. Re-darts: Resource constrained differentiable architecture search. arXiv preprint arXiv:1912.1288 2019.

---

[22] Yaochu Jin and Bernhard Sendhoff. Pareto-based multiobjectivv machine learning: An overview and studies. JEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(3):397-415, 2008.
[23] Taekyung Kim and Changick Kim. Attract, perturb, and explore: Learning feature alignment network for semi-supervised domain adaptation. In Proceedings of 16th European Conference on Computer Vision, volume 12359, pages 591-607, 2020.
[24] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, 2015.
[25] Alex Krizhevsky, Geoffrey Hinton, et Learning multiple layers features from tiny images. 2009.
[26] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Proceedings of the 5th International Conference on Learning Representations, 2017.
[27] Da Li and Timothy M. Hospedales. Online meta-Icarning for multi-source and semi-supervised domain adaptation. In Proceedings of the 16th European Conference on Computer Vision, volume 12361, pages 382-403, 2020.
[28] Li and Jitendra Malik. Learning to optimize. arXiv preprint arXi: 2016.
[29] Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: diffcrentiable architecture search. Proceed- ings of the 7th International Conference on Learning Representations, 2019.
[30] Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. generic first-order algorithmic framework for bi-level programming beyond lower-level singleton. Proceedings of the 37th International Conference on Machine Learning, pages 6305-6315, 2020.
[31] Shikun Liu, Edward Johns, and Andrew J. Davison. End-to-end multi-task learning with attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1871-1880, 2019.
[32] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems, pages 1647-1657, 2018.
[33] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning, pages 2208-2211 2217, 2017.
[34] Zhichao Lu, Kalyammoy Deb, Erik Goodman, Wolfgang Banzhaf, and Vishnu Naresh Boddeti. NS- GANetv2: Evolutionary multi-ohjective surrogate-assisted neural architecture search. In Proceedings the 16th Eumpean Conference on Computer Vision, pages 35-51, 2020.
[35] RE Lucchetti and Enrico Miglicrina. Stability for convex vector optimization problems. Optimization, 53(5-6):517-528, 2004.
[36] Roberto Lucchetti. Convexity and well-posed problems. Springer Science & Business Media, 2006.
[37] Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. Proceedings of the 37th International Conference on Machine Learning, pages 6597-6607, 2020.
[38] Nikola Milojkovic, Diego Antognini, Giancarlo Bergamin, Boi Faltings, and Claudiu Musat. Multi-gradient descent for multi-objective recommender systems. arXiv preprint arXiv:2001 10000400 2019.
[39] Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-leaming algorithms. arXiv preprint arXiv:1803.029%0 2018.
[40] Jachoon Oh, Hyungjun Yoo, ChangHwan Kim, and Se- Youno Yun. Boil: Towards representation change for few-shot learning. In Proceedings of the 9th International Conference on Learning Representations, 2021.
[41] Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In Proceedings of the 33rd International Conference on Machine Learning, pages 737-746, 2016.
[42] Hieu Pham, Melody Y. Guan, Barret Zoph, Quoe V. Le. and Jeff Dean. Efficient neural architecture search via parameter sharing. In Proceedings of the 35th International Conference on Machine Learning. volume 80, pages 4092 4101, 2018
12

---

[43] Samira Pouyanfar, Saad Sadiq, Yilin Yan, Haiman Tian, Yudong Tao, Maria Presa Reyes, Mei-Ling Shyu, Shu-Ching Chen, and SS Iyengar. A survey deep learning: Algorithms, techniques. and applications. ACM Computing Surveys, 51(5):1-36. 2018.
[44] Shaojian Qu, Mark Goh, and Felix TS Chan. Quasi-Newton methods for solving multiobjective optimiza- tion. Operations Research Letters, 3965)3377-399 2011.
[45] Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, and Sergey Levine. Meta-learning with implicit gradients. In Advances in Neural Information Processing Systems, pages 113-124, 2019.
[46] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In Proceedings of the 5th International Conference on Learning Representations, 2017.
[47] Sauli Ruuska and Kaisa Miettinen, Constructing evolutionary algorithms for bilevel multiobjective optimization. In Proceedings of the IEEE Congress on Evolutionary Computation, pages 1-7, 2012.
[48] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Proceedings of the 6th European Conference on Computer Vision, pages 213 226. 2010.
[49] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Sacnko. Semi-supervised domain adaptation via minimax entropy. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8050-8058, 2019.
[50] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Sacnko. Adversarial dropout regularization. In Proceedings of the 6th International Conference on Learning Representations, 2018.
[51] Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, pages 525 536, 2018.
[52] Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots. Truncated back-propagation for bilevel optimization. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, pages 1723-1732, 2019.
[53] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Proceedings of the 8th European Conference on Computer Vision, pages 746-760, 2012.
[54] Ankur Sinha. Bilevel multi-objective optimization problem solving using progressively interactive EMO. In Proceedings of the 6th International Conference on Evolutionary Multi-Criterion Optimization, pages 269-284, 2011.
[55] Jake Snell, Kevin Swersky. and Richard S. Zemel. Prototypical networks for few-shot learning. Advances in Neural Information Processing Systems, pages
2017.
[56] Flood Sung. Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1208, 2018
[57] Mingxing Tan. Bo Chen, Ruoming Pang. Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoe Le. MnasNet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 2820-2828, 2019.
[58] Hiroki Tanabe, Ellen H Fukuda, and Nobuo Yamashita. Proximal gradient methods for multiobjective optimization and their applications. Computational Optimization and Applications, 72(2):339-361, 2019.
[59] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint 142.::1412..4444, 2014.
[60] Hemanth Venkateswara, Jose Eusebio. Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018-5027, 2017.
[61] Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances in Neural Information Processing Syxtems, pages 3630-3638, 2016.
[62] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
13

---

[63] Xinyi Wang, Yulia Tsvetkov, and Graham Neubig. Balancing training for multilingual neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4537- 2020.
[64] Yaqing Wang. Quanming Yao. James T Kwok, and Lionel M Ni. Generalizing from few examples: survey on few-shot learning. ACM Computing Surveys, 53(3)11334 2020.
[65] Zirui Wang. Zachary C. Lipton, and Yulia Tsvetkov. Findings and meta-learning treatment. Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing, pages 4438 4450, 2020.
On negative interference in multilingual models:
[66] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Patter Recognition, pages 10734-10742, 2019.
[67] Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. SNAS: stochastic neural architecture search. In Proceedings of the 7th International Conference on Learning Representations, 2019.
[68] Zhongwen Xu, Hado van Hasselt, and David Silver. Meta-gradient reinforeement learning, In Advances in Neural Information Processing Systems, pages 2402-24222 2423,243, 2018
[69] Qiang Yang. Yu Zhang. Wenyuan Dai, and Sinno Jialin Pan. Transfer Learning. Cambridge University Press, 2020.
[70] Yao Yuan Yang. Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy robustness. In Advances Neural Information Processing Systems, volume 33, 2020.
[71] Tianhe Yu, Saurabh Kumar, Abhishck Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-tase task learning. Advances in Neural Information Processing Systems, 2020.
[72] Yu Zhang and Qiang Yang. A survey multi-task learning. IEEE Transactions Knowledge and Data Engineering. 2021.
[73] Aimin Zhou, Yang Qu. Hui Li. Shi-Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, and Qingfu Zhang. Multiobjective evolutionary algorithms: A survey of the state the Swarm Evolutionary Computation, 1(1):32 49, 2011.
[74] Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Guolin Ke, Jingwu Chen, Jiang Bian, Hui Xiong, and Qing He. Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 32(4):1713-177) 2021.
14


Reviews:
[{"review_id": "x4LhfUl0IH", "reviewer": "Reviewer_MQP9", "summary": "The authors consider and propose a multiple objectives perspective for Meta-Learning. More specifically, they formulate the problem as a bilevel-optimization problem in which the lower-level problem is the within-task problem and the upper-level problem is a vector-valued problem aiming at minimizing a multi-objective meta-learning problem. Existing state-of-the-art methods for multiple objectives meta-learning mainly combine multiple objectives into a single objective in a weighted sum manner and they tune the weights associated with all the objectives, which could be time consuming. Others, use new evolutionary algorithms not relying on convergence guarantees and not easily applicable to deep learning methods. Differently from those works, the authors propose a gradient-based Multi-Objective-Meta-Learning (MOML) gradient-based method not requiring tuning weights. Their method alternatively solves the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Under specific assumptions, the authors guarantees an asymptotic convergence result for their method, assuming to be able to compute the meta-subgradients of the meta-objective. Finally, they show the advantage of applying their multi-objective meta-learning method in comparison to state-of-the-art methods on problems of few-shot learning, semi-supervised domain adaptation, multi-task learning, and neural architecture search.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors consider a multi-objective perspective for meta-learning in which one desires optimizing multiple objectives simultaneously at the meta-level. Such a problem seems to be interesting and the paper is quite well written.\n\nMaybe, the necessity of considering a multiple-objective perspective in Meta-Learning is not enough motivated. It would be nice to give an example of the multi-meta-objective F, immediately after Eq. 1.\n\nSince usually there is no closed-form solution to the lower-level problem, the authors solve such a problem approximately by applying Stochastic Gradient Descent (see Alg. 1). After this, they apply standard multi-objective optimization algorithms on the approximation of the upper-level problem obtained by replacing the lower-level solution with the last iteration of Stochastic Gradient Descent. The approximation error introduced by this substitution in not enough analyzed in my opinion. Is it possible to formally quantify such an approximation error as done in [1] below?\n\nIn order to study the convergence properties of their meta-algorithm in Alg. 1, the authors need to consider optimization tools for vector-valued functions and to make standard assumptions in literature, such as the existence and the uniqueness of the solution of the lower-level problem.  About the convergence result presented in Thm. 2: it would be nice to have a non-asymptotic convergence rate. \n\nWhat about the step in which the authors compute the meta-subgradients by a routine? This is not an exact computation, so there will be an approximation error. Can you estimate it as done in [1] below?\n\nThe authors say that, for MAML, one may need to consider not only the performance but also the robustness which can help adapt to new tasks with the learned initialization. If one motivation supporting the necessity of considering multiple-objective is controlling the robustness of the learned initialization, what about the connection to the conditional Meta-Learning approach introduced in [2,3] below?\n\nf in Eq. 2 is the within-task objective. Clarify this. In Eq. 1, w depends on \\alpha: indicate this dependency by denoting it with w_\\alpha.\n\nLines 181-182: “How to obtain multiple nearly optimal solutions for MOBLPs is beyond the scope of this paper and we will study it in our future work.” What do you mean? Why?\n\nThe experiments seem to be exhaustive and show the advantage of applying their multi-objective meta-learning method in comparison to state-of-the-art methods on problems of few-shot learning, semi-supervised domain adaptation, multi-task learning, and neural architecture search.\n\n1.   Online-Within-Online Meta-Learning, Denevi et al. 2019\n2.   A structured prediction approach for conditional meta-learning, Wang et al. 2020\n3.   The Advantage of Conditional Meta-Learning for Biased Regularization and Fine Tuning, Denevi et al. 2020.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"ethical_concerns": "no ethical issue in my opinion", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "48"}, "extra_text": {}}, {"review_id": "woiC7gRAXDB", "reviewer": "Reviewer_e5P4", "summary": "Multi-objective bi-level optimization problem is formulated to efficiently do meta learning with multiple objectives. By properly using Multiple Gradient Descent Algorithm (MGDA), the authors propose a unified gradient-based multi-objective meta learning (MOML) framework with showing convergence guarantee. Moreover, it has been appropriately applied to several meta learning problems with more than one objective. ", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- There have been many meta learning problems that optimize multiple objective functions, and the proposed multi-objective meta learning (MOML) framework is trying to make it easier by avoiding the hyper-parameter tuning for combing those objective functions.\n- Multiple Gradient Descent Algorithm (MGDA) seamlessly fits into gradient-based meta-learning frameworks, and this proposed method is straight forward. Moreover, it is good to have convergence analysis to properly understand the method.\n- The right applications have been shown in the experimental section, and it is interesting to see how this can be applied.\n- The single-objective meta learning (SOML) is used as a baseline, where the weighting parameters for objective functions have to be tuned. However, it is unclear, how this SOML was trained or how the performance varies by randomly choosing the weighting parameters?\n- When we have multiple objective functions, we would like to emphasize particular objective funcion rather than others. For SOML, we have chance to manually emphasize it, but could this kind of situation can be handled with the poposed MOML? \n \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- it would be better to contain NAS results in the main paper by shortening other experiment parts or moving partially to appendix. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}, {"review_id": "r3b3xeOl44T", "reviewer": "Reviewer_nv9K", "summary": "This paper claims to present a \"new\" framework for multi-objective meta-learning. Their methodology is straightforward and easy to understand, and they have empirically evaluated their methods on several tasks and showed good results.", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The title of this paper excites me in the first place, but the idea of this method quickly disappoints me as the method is a simple learning extension to the current method. Unfortunately, I did not learn any new insight from this paper, so I decide to reject it.\nPros:\neasy to understand\nmany experiments with good results\nCons:\nI suggest authors taking a broader investigation of literature in Multi-objective Optimization. There are many approaches to consolidate multiple objectives into one object to transform the problem into single-objective optimization. Here you only studied the scalarization, largely ignoring other methods such as hyper-volume, and etc.. The authors should conduct an in-depth literature review in this field. Frankly, even Wikipedia covers a few methods. https://en.wikipedia.org/wiki/Multi-objective_optimization\nThe idea is boring. Correct me if I'm wrong; I believe the method only parameterizes the weights for each objective at the upper level and learns these weights. That's the main contribution of this method. If this is the case, I can easily come up with thousands of variants of proposed methods, e.g., by learning the scalarization, by learning the construction of hyper-volume. I understand you have a good result, but why is this better than others? The idea should be a few lines (< 5) of changes to DARTS.\nThe evaluation seems to cover quite different tasks, but the improvement seems very limited (e.g. Table.2). Overall it cannot convince me that this method solves some problems that other methods cannot solve. I understand a few points of improvement in these tables, but these numbers are not significantly different.\n\n======to other reviewers======\n\nTo other reviewers who are not familiar with DARTS, I highly recommend taking a look at it here (https://arxiv.org/abs/1806.09055). The proposed method simply combines multiple objectives of DARTS, and performs the experiments on different downstream tasks. Therefore, this novelty and contribution of this paper is very limited. \n\nI read the authors' comments, but the authors did not address my novelty concerns, especially on its difference to DARTS. This work is clearly a trivial version of DARTS applied to a different domain. Here the trivial part is that\n1) the formulation of bi-level optimization is exactly same as DARTS.\n2) Darts goes to single objective optimization, which optimizes \n\nmin o1\n\nThe proposed method in this paper is simply:\n\nmin o1*w1+o2*w2+o3*w3.....\n\nwhere w1, w2, w3 are learnable weights.\n\nThe inner loop of both methods is exactly same. So I really don't think this is a valid contribution to the research community, especially to a top conference like ICLR.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See my comments above. \n\nI don't foresee any negative social impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "azKSPvSRcfe", "reviewer": "Reviewer_jJ4x", "summary": "This work leverages multi-objective optimization to augment meta learning algorithms. In particular, the inner optimization proceeds as normal; while the outer optimization uses Desideri's multiple-gradient descent algorithm to dynamically scale the per-task inner gradients. The paper concludes with a theoretical proof of convergence and empirical analysis on several applications of meta learning. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work is well-written, and for the most part, lucid. Certain sections are quite dense, which I attribute to the author's desire to include a comprehensive explanation of their methods/proofs/experiments, but I would recommend further elaboration in Section 3 and 4 to elucidate the proposed method. For instance, clearly defining $\\alpha$, clearly defining $w$, and elaborating on how the MOPSolver combines LL gradients would likely help readers better grasp the scope of the proposed method, especially if they are unfamiliar with multi-objective optimization. \n\nThe theoretical analysis on convergence supports the authors' proposal, and strong empirical results on a diverse range of applications further bolster the utility of MOML. Given this is a general purpose augmentation that can be applied to many Meta Learning problems to improve performance, I believe it is a significant contribution to the field. However, one well-known detriment associated with Multi-Objective Optimization methods relates to its slow runtime. It would be beneficial for the authors to provide comparison to SOML (single objective, weighted combination in the outer optimization) to be transparent on the limitations of Desideri's algorithm. \n\nIn summary, I believe this paper is well-written, incorporates prior work to make a novel contribution, provides theoretical analysis, offers comprehensive empirical results, and makes a meaningful contribution to the meta learning domain. In many ways, it feels like the Meta Learning analog of Sener and Koltun's Multi-Task Learning as Multi-Objective Optimization, which itself was a seminal work.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I would recommend a runtime comparison of MOML to SOML should be conducted and added to the paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a method for multi-objective meta-learning which can be used for various settings such as semi-supervised and multi-task learning. The paper is well written. The authors provide the theoretical analysis on convergence, and the experimental results demonstrate the effectiveness of the proposed method. More discussion on the relation with the existing methods is needed to clarify the technical contribution of the proposed method.
Decision: Accept (Poster)