Multi-Objective Meta Learning
Authors: Feiyang Ye
Baijiong Lin
Zhixiong Yue
Pengxin Guo
Qiao Xiao
Yu Zhang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Meta Learning
Multi-objective Optimization
Bi-level Optimization

Abstract: Meta learning with multiple objectives has been attracted much attention recently since many applications need to consider multiple factors when designing learning models. Existing gradient-based works on meta learning with multiple objectives mainly combine multiple objectives into a single objective in a weighted sum manner. This simple strategy usually works but it requires to tune the weights associated with all the objectives, which could be time consuming. Different from those works, in this paper, we propose a gradient-based Multi-Objective Meta Learning (MOML) framework without manually tuning weights. Specifically, MOML formulates the objective function of meta learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta learner. To solve the MOBLP, we devise the first gradient-based optimization algorithm by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search. The source code of MOML is available at https://github.com/Baijiong-Lin/MOML.

Multi-Objective Meta Learning
Feiyang Ye Ye Baijiong Lin Zhixiong Yue!.?, Pengxin Guo', Qiao Xiao and Yu Zhang' 14.1 Department of Computer Science and Engineering, Southern University Science and Technology University of Technology Sydney
Eindhoven University of Technology
4 Peng Cheng Laboratory
(12060007, _1inbj,,,,. yuezx, 12032913) 11a1mmmmmmmmmmm.1mm1111111111111 wuw.suh.teseee.. edu. {qiaoxia07222, yu zhang. ust) com
Abstract
Meta learning with multiple objectives has been attracted much attention recently since many applications need to consider multiple factors when designing learning models. Existing gradient-hased works meta learning with multiple objectives mainly combine multiple objectives into a single objective in a weighted sum man- This simple strategy usually works but it requires to tune the weights associated with all the objectives, which could be time consuming, Different from those works, in this paper, we propose gradient-based Multi-Objectin Meta Learning (MOML) framework without manually tuning weights. Specifically, MOML formulates the objective function of meta learning with multiple objectives Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is solve several possibly conflicting objectives for the meta learner. To solve the MOBLP. we devise the first gradient- based optimization algorithm by alterna- tively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-basee multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient- gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search. The source code of MOML is available at httttss :eattps:/giitee //github. com com/Bai joong--n-- Lin/MOML
Multi Oileeivv
Introduction
In the past few years, deep learning has achieved great success in various fields 43 because it can effectively and efficiently process massive and high-dimensional data. However, training deep learning model from scratch often requires large amount of labeled data to learn large number of model parameters and needs to choose hyperparameters by hand.
As way to address those problems by enabling models to learn how to learn, meta learning has attracted considerable attention recently 19 20) Meta learning gains knowledge from multiple meta training tasks so that the knowledge can be reused in new tasks new environments rapidly with few training examples. Taken broadly, objective functions of meta learning models are usually formulated as bi-level optimization problem where the lower-level subproblem represents the adaptation given task with learned meta parameters and the upper-level subproblem tries to optimize these meta parameters via meta objective [19]. Hence, from this view, meta learning has wide range of applications such as hyperparameter optimization [13, Neural Architecture Search (NAS) 29 and Reinforcement Learning (RL) [68]
Equaa contribution. Corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In many studies on conventional meta learning methods and applications, there is only single meta objective in the upper-level subproblem. For example, the Model-Agnostie Meta-Learning (MAML) method [12] only measures the performance on a validation dataset in the upper-level subproblem to evaluate the learned initialization of parameters. DARTS [2)]. a differentiable method for NAS, evaluates the performance of the searched architecture on the validation dataset. However, in real-world applications, there are usually more than one objective to be considered. For example, for MAML, we may need to consider not only the performance but also the robustness which can help adapt to new tasks with the learned initialization. Similarly, the network size and performance should be balanced in NAS, especially when the searched architecture will be deployed to devices with limited resources such as mobile phones. In those applications, we can see that there is a need to balance multiple possibly conflicting objectives in meta learning.
Meta learning with multiple objectives thus has drawn much attention in recent studies. Specifically, some works study specific meta learning problems in the multi-objective case, such as multi-objective
NAS 66 57
| 14 multi-ohjective RI. 44, and so on. However, those works either linearly combine
multiple objectives into single objective for the upper-level subproblem [65| 63 I or utilize multi-objective bi-level evolutionary algorithms [5 54 47] to handle it. The former approach needs to tune weights associated with all the objectives, which is time consuming, and its performance depends on the set of candidate weights in, for example, the cross validation method. The latter approach, whose computational complexity is even higher, has no convergence guarantee in the optimization process and is not easy to be integrated into gradient- based learning models such as deep neural networks, which limits its use in many learning models.
To alleviate those limitations existing works, in this paper we propose unified gradient-baser Multi- Sulti-Obbjective Meta Learning (MOML) framework with a convergence guarantee. The MOML framework formulates objective functions in meta learning with multiple objectives as a Multi- Objective Bi-Level optimization Problem (MOBLP), where the lower-level subproblem is to learn the adaptation to a task similar to vanilla meta learning and the upper-level subproblem minimizes a vector-valuee function corresponding to multiple objectives for the meta learner. To solve MOBLP, we devise the first gradient-bbsee optimization algorithm by alternatively solving the lower-level and upper-level subproblems via gradient descent method and gradient-ber based multi-objectinn optimization method such as [7]. respectively. We theoretically prove the convergence properties of the proposed based optimization method. To show the effectiveness of the MOML. framework, we apply it to several meta learning problems, including few-shot learning, domain adaptation, multi-task learning, and NAS, where multi-task learning is firstly formulated from the perspective of meta learning. In summary, the main contributions of this paper are four-fold.
We propose a unified MOML framework based on the MOBLP and devise a gradient-based optimization algorithm for the MOML framework.
We prove the convergence property of the proposed optimization algorithm.
We formulate several learning problems as instances of the MOML framework.
Experiments show that MOML achieves state-of-the-art performance on those learning problems.
2
Related Work
Meta Learning. Meta learning (a.k.a. learning to learn) learns knowledge from multiple tasks and then adapts it to new tasks with a few samples quickly. Many studies in meta learning mainly focus on solving the few-shot learning problem. From this view, meta learning can be divided into three main categories, including metric-based approach [55] [66.. model-based approach (28), and optimization-basee approach [22 39). For example, as an optimization-basee
method, MAML learns an initialization of model parameters so that a new task can be learned with a few training samples by fine-tuning the learned initialization. A widely-used formulation in meta learning can be cast as a level optimization problem, where the upper-level subproblem is to learn meta parameters by minimizing a meta objective and the lower-level subproblem is to quickly adapt to new tasks with meta parameters 45 [9]. For example, MAMI. adapts to a new task by using the associated training dataset and the learned initialization in the lower level, and then updates the initialization according to the validation performance in the upper level. From this perspective, meta learning is general

---

learning paradigm and has more general applications [9]. In this paper, we study meta learning from the perspective of the bi-level optimization.
Oulti-Objective Optimization. Multi-objective optimization to address the problem of simulta- neously minimizing multiple objectives, which may conflict with each other. The simplest way to handle multi multi-objective optimization is to convert to a singlc-objective optimization problem via the linear scalarization approach which minimizes the weighted sum of all the objectives. Actually, many machine learning algorithms adopt this approach. For example, minimizing the regularization term to control the model complexity and minimizing the training loss are two conflicting objectives, and learning model is usually to minimize a linear combination of those two terms. Recently, many kinds of advanced multi-objective optimization algorithms are proposed. such as evolutionary algorithms [73,, population- -based algorithms [15], gradient- based algorithms [ 37]. and so on. Some of them have been successfully applied solve machine learning problems [22] In this work, we focus on gradient-basee based algorithms because this approach can be easily integrated into gradient-based machine learning models such as deep learning models.
The MOML Framework
In this paper, the proposed MOML framework has unified objective function, which is formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP), as
min F(w, (x) = (Fi (w, F2(w, ot), Fm (w. a))² s.L. w S(a). aEA,EER
where function F: IP x R + - is vector-valued jointly continuous function for the m desired meta objectives and A is a nonempty compact subset of IR". In problem (I.. S(a) is defined as the set of optimal solutions minimize fw, a) w.r.t. w, i.e.
S(a) = arg min (w)
When equals 1, problem (T) reduces to a standard Bi-Level optimization Problem (BLP), which widely-used formulation in meta learning [19], and hence from this perspective, the MOML framework is generalization of meta learning. In problems ( and (2). F is called the Leper-Level (UL) subproblem and f I2 R" > IR is the Lower-Level (LL) subproblem. The LL subproblem can be considered as constraint to the UL subproblem. In MOML, F contains multiple meta objectives to be achieved for the meta learner and f defines the objective function for current task such as the training loss. In Section 5] will see the application of MOML in different learning problems, including few-shot learning, domain adaptation, multi-task learning, and neural architecture search.
To solve problem [ there exist a type of works 5 54 which adopt multi-objective evolutionary algorithms. However, such methods have a high complexity without convergence guarantee and are not easy to be integrated with gradient-baset models such as deep neural networks. Hence, we do not include them in experiments. To the best of our knowledge, there is no gradient-based -baeed optimization algorithm with convergence guarantee to solve an MOBLP, which is what we will do the next section.
4
Optimization
In this section, we devise a general algorithm to solve the MOBLP (i.e., problem (1) and provide convergence analyses under certain assumptions.
4.1 Lower-Level Singleton Condition
Due to the complicated dependency between UL and LL variables, solving the MOBLP is challenging, especially when optimal solutions of the LL subproblem are not unique.
For BLP with single objective in the UL subproblem, many studies that the LL subproblem only admits a unique minimizer for every A. This condition can simplify both the optimization process and convergence analyses and first fomally introduced as Lever-Level Singleton (LLS) condition in [30].
13 52 potentially require
3

---

For the MOBLP, the LLS condition is necessary. If the LLS condition does not hold, the MOBLP is even ill-defined 9 To see this, suppose for fixed ag. we get a set of solutions S(ag) for the LL subproblem. Since F is vector-valued in problem (1) it is unclear that at which
((00) ) the UL. subproblem F should be evaluated. With the L.I. condition, problem ① can be simplified as
min ((a) (a) s.t. w (a) = arg min aw) aEA
4.2 Gradient-based Optimization Algorithm
Here we design gradient-basede based optimization algorithm to solve problem (3). Usually, there is no closed form for the solution w" (ax) of the L.L subproblem and so is difficult to optimize the UL subproblem directly. Another approach
use the optimality condition of the LL subproblem (i.e., [w, a) = 0) as equality constraints for the UL subproblem in a way similar to [4]]. However, this approach only works for LL subproblems with simple forms and cannot work for general learning models.
To solve problem (3). take strategy similar to the alternating optimization. In the first part of each iteration (corresponding to steps 3-6 in Algorithm [ we solve the LL subproblem via gradient descent methods. Specifically, with initialization wp for the L.L. variable where the index of the iteration is omitted for notation simplicity, the solution of the LL. subproblem can be updated for K steps as w (a)) = To (wm (a), a.) k 1,.... K 1, where Ta represents an operator update Here we consider a first-order gradient descent method for Ta such as the Stochastic Gradient Descent (SGD) method and Ta can be formulated explicitly as To (w)((a)), a) = Wa (x) vV f (a), a), where > 0 denotes the step size and V (ax), ar) denotes the derivative of f w.r.f. w at (a). In the second part of each iteration (corresponding to steps 7-9 in Algorithm [ by fixing
the value of w as the current solution wx (a) obtained in the first part, we solve the UL subproblem
as
min Φ (a) F F( (a).
Problem 4 is an unconstrained Multi-Objective optimization Problem (MOP) and we can use any multi- objective optimization algorithms to solve it. Here we choose gradient- based multi-objective optimization methods as they can be seamlessly inte-
Algorithm 1 Optimization Algorithm for MOML
Input: numbers of iterations (T, K). step size (H,, v) 1: Randomly initialized ox:
2:
for 1 do Initialize w((aa:;; for j to K do
3:
4:
grated into any gradient- based learning
5:
wa(a)) w)-1(ar) (VV end for
6:
framework. There are several gradient-
7:
Compute gradients V. (w (a.). a.) for the i's;
based multi-objective optimization al-
8:
Compute the gradient
gorithms [44 7 588 and they com-
d(wk (ax), an) = ( (wk (ac), at)
monly find an appropriate descent di-
9:
a141
vd(wkk (a.), an) with step size
rection d for all the objectives in F by aggregating their gradients w.r.t. a. So such process is denoted by d we adopt a simple gradient- based MOP method called Multiple Gradient Descent Algorithm (MGDA) 77. whose details are introduced in Appendix A.. due page limit.
10:
end for
(OOPSolver((Va.H)
(a), )= in Algorithm In this paper,
The entire algorithm to solve problem (3) is shown in Algorithm [ which to the best of our knowledge is the first gradient-based optimization algorithm for MOBLPs. In Algorithm [ we obtain only one solution for MOBLPs, which is different from multi-objective evolutionary algorithms that can find a population of solutions. How to obtain multiple nearly optimal solutions for MOBLPs is beyond the scope of this paper and we will study it in our future work.
4.3 Convergence Analysis
Algorithm[I is simple and intuitive, but its convergence properties are unknown, which is what we aim to study in this section. As an MOBLP, problem (3) cannot reduce to SOML with a scalar-valued objective function in the upper-level subproblem when using gradient-based Algorithm I solve it. Therefore, has different theoretical properties from SOMI. as we need to focus on the convergence properties of minimal point set instead of minimum scalar in the UL subproblem.

---

We first recall some notions about vector-valued functions. Consider vector valued function g(z) IR" + RM (m, n E N, 2 2). We denote by Min g(2) the set of all the minimal points of function g(z). Min g(z) is also called the Pareto frontier or Pareto-optimal set. The corresponding
efficient solution or Pareto-optimal solution set of g(>) is denoted by Eff(g(()). The convexity of vector-valued functions is called the P-convex. The details of these definitions can be found in
Appendix A
To help analyze convergence properties of Algorithm with a basic assumption about the LLS condition which is mentioned in Section 4.1 and is widely adopted BLPs 52 we can obtain the following result.
Theorem 1. If f(w,a) a) is jointly continuous, arg min., f(w, a) is singleton for every a A, and w" (at is uniformly bounded on A, then the function (a), a) is continuous w.r.l. a.
Because A is compact set, Theorem immplis the existence of solutions. Theorem and the uniform convergence of wg( (a) can further imply the convergence for the solution of the LL subproblem, which is similar to that of the standard BLP problem [13]
For the convergence of Algorithm D we need to analyze minimal point sets of the images of perturbed functions şa (a:) and p(aa) which are defined in problems (4) and (3) respectively. We consider the most natural set convergence under this setting, i.e., the Kuratowski-Painkee
set-convergence, Due to page limit, please refer to these definitions in Appendix A Under certain assumptions which are we have the following convergence results.
used analyses of BL. and MOP 13 35
Theorem 2. In addition to Theorem 7 it is assumed that (i) F(., a) is uniformly Lipschitz continuous; (it) The iterative sequence {wx(a)) converges uniformly to w" (a) on A as K + -20; (tii) A is convex set; (iv) PK is P-convex and &p is strictly P-convex. Then, the Kuratowski-PPu Painkleveé set-convergence of both the minimal point set and efficient solution set in ( i.e.,
Min (x((a) Min p(a), Eff xpx (ax + Eil sp(a).
Thus, under some specific assumptions, Theorem provides a theoretical justification for the conver- gence of Algorithm to solve MOBLPs.
5 Applications of MOML
In this section, we introduce several use cases of the MOML framework in several learning problems, including few-shot learning, semi-supervised domain adaptation, multi-tas task learning, and neural architecture search.
5.1 Few-Shot Learning
Few-Shot Learning (FSL) aims to tackle the problem of training a model with only a few training samples 64
Recently, FSL is widely studied from the perspective of meta learning by using prior knowledge in the meta training process. Most studies in FSL only consider the classification performance. However, in real- world applications, the performance not the only important focus. For example. we expect FSL models to not only have good performance but also be robust to adversarial attacks (26), which may improve the generalization of FSL models. In the following, we can see that this setting can naturally be modeled by the proposed MOMI. framework.
Problem Formulation. Suppose there is a base dataset Drase with a category set Cruse and a novel dataset Doovel with a category set Cnovel. where Cbase 7 Cnovel = 0. The goal of FSL is to adapt the knowledge learned from Diase to help the learning of "nooett In the ith meta training episode, we generate from D Drasc an N-way k-shot classification task, which consists of a support set ()))) and query set ())()) For the robustness, we add perturbations generated by the Projected Gradient base generate perturbed query set base The objective function of the FSL model that considers both the performance and the robustness can be formulated
Descent (PGD) method [26) into each data point in De(()
adds
min (()) a, Dille ) (( a, base
Dali),ado)
s.t. (a) = arg min Cr a, o(i))
(5)
5

---

where w represents model parameters, a denotes the meta parameters to encode common knowledge that can be transferred to novel tasks, and Cr(w, a, D) denotes the average classification loss of a model with model parameters and meta parameters w on dataset 'D. In the UI. subproblem of problem 5 the first objective measures the classification loss on the query set based on w>() (a) obtained by solving the LL subproblem and the second objective measures the robustness via the classification performance on the perturbed query set. Problem (5) provides general formulation, which depends on what a represents, for FSL. We adapt problem (5) to three FSL methods (i.e..
MAML [12], ProtoNet [55] and BOIL [40] and the detailed formulations are put in Appendix [C] It is easy to see that problem (5) fits the MOMI. framework and we can use Algorithm ! to solve it.
For an MOBLP such as problem ( common approach is to transform the UL subproblem into a single objective problem via the linear scalarization approach (65 633 [T]. In contrast with MOMI,, this approach is named as Single-Objective Meta Learning (SOML) in this paper and used as an extra baseline method in our experiments. Specifically, with fixed weights {7 which satisfy E 7/- = and 7/2 2 O, problem {1) is transformed to a standard bi-level optimization problem as mill Σ 7: Fi(w, a). This approach heavily depends on the selection of weights (7% ) and inappropriate weights usually lead to poor performance.
Experiments. Experiments are con- ducted on two FSL benchmark datasets, ImageNe [6]] and CUB-200-2011 (referred to as CUB) (62]. Due to page limit, experimental settings and experi- mental results the CUB dataset put in Appendix[C The original baseline meth- ods (i.e., MAMI,, ProtoNet, and BOIL) are trained in a conventional way and we evaluate the clean accuracy and PGD ac-
Table 1: Classification accuracy (abbreviated as "Clean Acc.") ") and PGD accuracy (abbreviated as "PGD) Acc.") ") on the mini-ImageNNet dataset for 5-way k-shot FSL. The best result in each group of methods is highlighted in bold and the best result in each setting is annotated with underline.
Method
Clean Ace.
PGD Acc.
B-score
MAML MAML+SOML
45.24:0.81
1.18:10.15
2.10:0.52
00.78=0.75
23.91=0.67
99.8320.43
MAML+MOML (ours)
39.23=0.77
25.8010.67
31.12:0.70
curacy of all the methods, where the clean
I-shot
ProtoNet [55]
44.67--0.75
2.60++0.21
3.73++0.35
accuracy is measured on the original test query set and the PGD accuracy is tested on the query set perturbed by the PGD at-
ProMoNet-SOML
38.65-0.722
13.10+0.66
28.67-00.67
ProloNet-MMM.... MOMI.
35.06-+0.70
21.24--0.65
40.51-0.66
BOII. [0
10.6
3.05
1.63.10.61
BOIL+SOML
00.44:0.79
25.94:10.69
31.29:10.75
tack. As the clean accuracy and the PGD accuracy are reported to be conflicting to each other [70], inspired by the widely- used F1 Fc-re based on the precision and
BOIL+MOMI (ours)
41.22.10.83
10.75
32.98 1..79
MAML
6188110.77
2.82.10.1
5.01100.43
MAML+SOML
56.1620.722
34.8520.72
42.91=0.71
MAML+MOML (ours)
55.66.10.78
39.3810.77
15.89
5-hhot
ProtoNet [55]
66.55=0.70
0,68++0.09
1.31 18
recall, we propose new metric called Balance score (B-score), which is defined as B-score = 2 (CA x PA) (CA + PA) with CA and PA denoting the clean accu- racy and PGD accuracy, respectively, to fully measure the performance in terms of the clean accuracy and PGD accuracy.
ProtoNetSOML
59.11=0.71
30.41=0.73
46.93-20.71
ProtoNet+ (ours)
58.72=0.74
41.5990..75
48,59-+0.74
ROIL. [i]]
66.02-00.22
4.85+00.0
1.11+0.51
SOIL+
58.54110.76
34.78.10.75
42.94110.88
MOMT. (ours)
60.21+00.99
35.47+0.78
44.37-0.78
The average results over 600 testing tasks on the mini- Image-t dataset are presented in Table [ From the results, we can see that the adversarial robustness of the original FSL methods (i.e., MAML. ProtoNet, and BOIL) is poor, while SOMI. and the proposed MOMI. can significantly improve the PGD accuracy. For example, MOML. can improve the PGD accuracy of ProtoNet by about 40.91 % under the 5-shot setting. Since the classification accuracy and adversarial robustness are conflicting 770, the clean accuracy of SOML and MOML slightly drops when compared with the orignal FSL baselines. The B-score of MOML is higher than SOML, which indicates the multi-objective formulation in the UL subproblem of MOML is better than the single-objective formulation in SOML.
5.2 Supervsup Domain Adaptation
Semi-Supervised Domain Adaptation (SSDA) aims to address the domain shift between two domains so that the model trained in label-rich source domain can be adapted to target domain with limited labeled samples and abundant unlabeled samples (69 A widely used approach for SSDA is to align the distributions of two domains by finding some domain-invarian components. There are usually three objectives to be considered, including two classification losses on two domains and an alignment loss to measure the domain discrepancy. While existing works such as [49] [7] optimize

---

all the objectives by simply computing weighted sum of them, we formulate the SSDA problem as an MOBLP under the MOML framework.
Problem Formulation, Given a source domain S and a target domain T, the source domain has a large labeled dataset Ds and the target domain has limited labeled dataset D/ well large unlabeled dataset Di,, where Dy DJ U Dij denotes the entire dataset for the target domain. The average classification losses in the source and target domains are represented by (w, Ds) and LD (w, D) respectively, where w is the model parameter. The aligmment loss denoted by (a( a, Ds. D+) aims to learn
projection space by minimizing the local maximum mean discrepancy in DSAN (74) or domain-invarisnn prototypes by maximizing the entropy in MME 44), where is the initialization of Then can formulate the SSDA problem under the MOML framework as
components such as
min (Cn(w" (a): Ds), (a)) Dly), Ca(w, a, Ds, Dz))
s.t. w" (a) arg min L (wc, a, Ds. D4))
(6)
In the LL subproblem of problem (6). we aim to learn model to find a domain- component between two domains via the alignment loss Ca by optimizing w with an initialization a, and in the UL subproblem, we expect to improve the model further by updating a via minimizing the two classification losses together with the alignment loss. Here a acts similar to the parameter initialization in MAML (i.e., a in problem (5)) and helps learn w in the LL subproblem, but it does not require any adaptation on the testing process. We can follow two state-of-the-a art domain adaptation models (i.e., DSAN and MME) to design the alignment loss and due to page limit, the detailed formulation of the alignment loss L, put Appendix [
Although a bi-level objective function is also formulated Meta-MME (27]. there exist two sig-
nificant differences between Meta-MME and the proposed MOML method. Firstly, Meta-MME aims to learn the initialization of the network parameters by minimizing the source classification loss and the alignment loss in its LL subproblem and then validate on a few target labeled samples in
the UL subproblem, which is different from the proposed MOML formulation (i.e., problem (6)) Secondly, the proposed MOML method considers a multi-objeccivve optimization problem in the UL subproblem, which is different from Meta-MME. Empirically, MOML outperforms the Meta-MME method as shown in Table 2
Table 2: Classification accuracy (%) on the Office-31 dataset with ResNet-50 backbone for SSDA. it means the corresponding method is appropriately modified to adapt to our experimental setting according to the released code. $ indicates that the corresponding model is reimplemented by us. 1. and - in the subscript indicate an increase, a decrease, and no change, respectively, when compared with the original method in two groups of models based on DSAN or MME. The best results in each group of models (i.e., DSAN and MME) are highlighted in bold and the best results of each transfer task are annotated with underlines.
Method
AD
D-
A
W-A
D
W-
Avg
S+T
93.58
74.16
92.17
74.08
98.01
100
88.67
DANNÍ
14
93.09
74.52
91.60
75.07
98.58
100
88,81
ENT
16
93.33
74.16
94.44
75.03
97.86
100
89.13
APE
94.81
76.10
91.80
76.02
97.29
99.75
89.29
ADR 50
93.33
76.73
93.30
76.51
97.86
100
89.62
CDAN 32
94.32
75.25
92.88
76.98
98.43
100
89.64
DSAN 14
93.83
76.82
93.59
75.68
98.43
100
89.73
DSAN SOML
94.32,
76.91
94.16;
75.99,
97.72,
100
89.85,
DSAN MOML (ours)
94.08
77.13-
94.59,
75.96,
98.36
100
90.02,
MME 49
92.09
77.71
93.73
77.27
98.01
100
89.80
Meta-MME
27
92.10
77.01
94.30
76.87
98.29
100
89.76
MME+ SOML
92.09
77.57
94.30,
77.20,
98.29,
100-
89, 89.90,
MME+ MOML (ours)
94.32,
78.30-
94.44
77.71
98.43,
100
90.53,
Experiments.
Experiments are conducted on the Office-31 dataset
48 which has 3 domains:
Amazon (A), Webcam (W) and DSLR (D). By following [59] 33), we construct all six transfer
7

---

tasks. Each class in the target domain has three labeled images in the training process by following
49
Baseline models in comparison include a deep neural network (denoted by 'S+T') that is
trained on Ds U D4 and other eight state-of-the-arts domain adaptation methods: ENT [16), DANN 14. ADR (50), CDAN (22 MME [49], Meta-MME [27]. APE [23] and DSAN 74. As DSAN and MME are two state-of-the-art domain adaptation models, to improve their performance further, SOML and MOML adopt the alignment loss in DSAN and MME, respectively, leading to two groups of models, i.e., (DSAN, DSAN+SOML, MMAN+MOML) and (MME, Meta-MME, MME+SOML, MMM+MOML). All the methods except S+T are trained on Ds UDT. Due to page limit, details of baselines and experimental settings are put in Appendix D
Experimental results on the Office-31 dataset are shown in Table 2 The incorporation of SOML into DSAN and MME can consistently improve the performance of two baseline models and the proposed MOMI, method further improves the classification accuracy. For example, MME+ significantly improves the performance of the first five tasks and achieves the perfect performance (i.e., the 100% accuracy) on the last task as most baselines did. Moreover, compared with state-of- the-art baselines, DSAN+MOML achieves the best result (i.e., 94.59%) on transfer task A- W and MME+MOML has the best accuracy (i.e., 78.30% and 77.71%) on transfer tasks D- and W-A Besides, MOE+ achieves the best average classification accuracy of 90.53% and significantly outperforms all of the baselines, which indicates the effectiveness of the proposed MOML method.
5.3 Multi-Task Learning
Multi Task Learning (MTL) 72) aims to improve the performance of multiple tasks simultancously by leveraging useful information contained in these tasks. Learning the loss weighting is a challenge in MTL and there are some works 51 71] to solve this problem. Among those works. Sener and Koltun (51) formulate multi-task learning problems from the perspective of multi-objective optimization and implicitly learn the task weights via MGDA, Liu et al. [1] estimate the task weight of each task as the ratio of the training losses in the last two iterations for the corresponding task, and Yu el al. [71] project each task's gradient onto the normal plane of the other. Different from those works which are all based on single-level optimization problems on the entire training set, for the first time we formulate this problem as MOBLP based on the split of the entire training dataset and solve this problem based on the MOMI. framework.
Problem Formulation. Suppose there are m tasks. The ith task has dataset Dj for model training. Here each D, is partitioned into two subsets: the training dataset D.r and the validation dataset Drat, where Dfr is used to train a multi-task model and Draa is to measure the performance of the multi-task model on the ith task. (=:w), the learning function of the multi-task model parameterized by w, receives data points from the m tasks and outputs predictions. Let a; E [0, 1] denotes the loss weight for the ith task. The goal is to jointly learn the loss weights a (a1..... am,) and the model parameter W. The objective function of the proposed method under the MOML framework is formulated as
min (CMTL(W" (a), D()) CMTL(W" (a), Dral)) a.L.rrlw D.", 0 ai a. 1 Σ a =
a
s.t. w( =ag min m
m
where (MTL(W, D) - E(x,)eD ((f(x;)) v) denotes the average loss of f(:w) on a dataset DD with [D] denoting the size of D and ((.-)) denoting a loss function. In the L.I. subproblem of problem 77. when given weights in a, we aim to learn a MTL model to get optimal parameters wt on the training dataset and in the UL subproblem, we expect to update via minimizing the loss of the trained MTL model with parameters wn on the validation dataset of each task.
Experiments. Experiments are conducted on the NYUv2 53 Office-31 and Office-Home [60]
datasets. Baseline methods in the comparison include different loss weighting strategies such as Equal Weights (EW), Dynamic Weight Average (DWA) [3I] with the temperature parameter T as 2, MGDA (1), PCGrad Z and SOML, and they are built on two multi-task architectures, including Deep Multi-Task Learning (DMTL) that adopts the hard-sharing structure to share the first several layers and Multi-Task Attention Network (MTAN) [I]. The ResNet-50 is used as the backbone and we do not use data augmentation. Due to page limit, the introduction of datasets, details
8

---

experimental settings. and experimental results on the Office-31 and Office-Home datasets are put in Appendix A
Table 3: Performance on the NYUv2 dataset with three tasks: 13-class semantic segmentation, depth estimation, and surface normal prediction. The best combinations of the architecture and weighting strategy are highlighted in bold. The best results for each task on each measure are annotated with underlines. (1) means the higher (lower) the result, the better the performance.
Segmentation
Depth
Surface Normal
Weighting Strategy
Architecture
Angle Distance
Within t
mIoUt
Pix Ace
Abs Err
Rel Err!
Mean
Median 4
11.25
22.5 1
30
EW
52.71
74.78
0.3886
0.1581
23.8568
17.3537
34.57
60.45
71.63
DWA 11
$2.72
75.11
0.3931
0,1631
23.7894
17.3320
34.57
60.51
71.67
DMTL
MGDA
$2.89
74.87
0.3963
23.7513
17,2685
34.74
60.64
71.75
PCGrad 7
53.22
75.45
0.3920
0.1658
23.2904
16,8728
35.47
61.58
72.56
SOML
53,20
75.22
0.3923
0.1628
23.6888
17.0228
35.42
61.03
71.97
MOML (ours)
54.98
75.98
0.3877
0.1618
23.2401
16.7388
35.90
61.81
72.76
EW
53.97
75.90
0.3794
0,1580
22.8743
16.5502
36.54
62.52
73.31
54.12
75.79
0.3902
22.9691
16.6212
73.28
MGDA 61
54.38
75.55
0.3854
0,1583
22.9396
16,4670
36.70
62.55
73.23
MTAN
PCGrad 7A
54.40
76.13
0,3830
0.1581
23.0040
16.4636
36.67
62.65
73.34
SOML
54.03
75.48
0.3829
0.1581
22.8279
16.4259
36.74
62.67
73.46
MOMI (ours)
54.23
75.63
0.3843
0.1567
22.7530
16.2468
37,20
63.09
73.65
Experimental results on the NYUv2 dataset are shown in Table Firstly, SOML outperforms the EW. DWA, and MGDA strategies when using the DMTL architecture and achieves comparable performance with the four baselines with the MTAN architecture. It indicates the proposed bi-level formulation (i.e., problem (7) with weighted combined objectives in the UL subproblem used in SOML) can achieve very good performance when compared with the state-of-the-arr baselines. Secondly, the MOML method outperforms the SOML method under both DMTL and MTAN architectures, which means the multi-objective formulation in the UL subproblem is better than the single-objective formulation. Finally, it is noticeable that the proposed MOMI. method achieves state-of-the-art results in many metrics. For example, when training with the DMTL architecture, MOML can achieve 54.98% in terms of the mloU, which significantly higher than other baselines even with the advanced MTAN architecture. MOML with the MTAN architecture achieves the best results on six metrics. Experimental results on the NYUv2, Office-31, 1, and Office-Home datasets, the latter two of which have their results in Appendix E, show that the proposed MOML framework achieves state-of the-art performance for learning loss weights in MTL.
The loss weights learned by the proposed MOML are (0.3120, 0.3804, 0.3076) and (0.2541,0.44) 0.4860, 0.2599) for the three tasks (i.e., semantic segmentation, depth estimation, and surface normal prediction) when using the DMTL and MTAN architectures, respectively. It is in- teresting to find that when using different architectures, the weight of the depth estimation task commonly the highest, while the other two tasks have similar weights.
5.4 Neural Architecture Search
NAS aims to design the architecture of neural networks in an automated way. In architecture design, usually need to consider multiple factors. For example, we expect that the searched architecture has good performance, behaves robustly to noises, and consumes low resources. In this case, we formulate the NAS with multiple objectives case of MOML. Due page limit, the introduction of datasets, details on experimental settings, and experimental results are put in Appendix
Problem Formulation. By following the DARTS method 29), in operation space denoted by O, each element is an operation function o(-) and each cell is a directed acyclic graph with N nodes, where each node represents hidden representation and each edge (i.j) denotes candidate operation o(.) with a probability ab (t.j) Therefore, a = {ao ((-5) { FooD is a representation of the neural architecture, where E denotes the set of all the edges in all the cells. The entire dataset is split into training dataset denoted by D,r and a validation dataset denoted by D
The multi-objective NAS considers three objectives: the classification accuracy, adversar- ial robustness and the number of parameters. We formulate three corresponding losses as n (w, Deat), n (w, a. Dodv), and ( where denotes all the model parameters in the

---

neural network and D denotes the perturbed validation dataset by adding perturbations on each data point. Ev (w, a, D) denotes the average classification loss on dataset 'D of neural network with parameters w and an architecture
1
CN (w. a D)
Σ
[(w, x,), v),
(x.
where f(W,x,) x, denotes the loss function for each sample and n is the size of D.
To formulate Enxop(x). we denote by no the number of parameters associated with operation searched architecture a. Then Wop can
o and by Nn) the number of parameters be computed by (a) L(JjeR (((-)), where ( is the number of parameters of the searched operation on the edge (i.j). As we determine the operation of each edge by selecting the one with the largest probability, hence we have (a.)) arg max.eo a. As the arg max
operation is non-differentiable, use the softmax function approximate Noop((:) as (a) Ro. Therefore, to search a network architecture with expected size L, (c))((t) can be formulated as
exp(a!!.))
EGSEER EoEo Ev'co expla "apla
Lnop = |Noop T.--
(8)
Finally, the overall formulation for the multi-objective NAS is formulated min (Cn(w" oz, Deat), L (w" (ax), ar Dodi)) Loop s.t. w (a) arg min (N(ww, D.y).
(9)
In the LL subproblem of problem p when given the architecture a, we can train a model with optimal parameters
the training dataset and in the UL subproblem, we expect to update the architecture a by making trade-off among the validation loss, the adversarial robustness and the number of parameters. Obviously problem (9) matches the MOML framework. It is easy to that the DARTS method is special case problem (9) when its UL subproblem contains the first objective only and hence problem (9) generalizes the DARTS method by considering two more objectives.
Compared with the NSGANetV2 method [34] that utilizes a multi-objective bi-level evolutionary algorithm, MOML more efficient and has convergence guarantee. Moreover, NSGANetV2 uses ensembled surrogate models to predict the accuracy of an architecture, which may incur performance gap between the UI. and LL subproblems. The LL subproblem of NSGANetV? only chooses over 300 candidate architectures for evaluation with a supernet constructed for weight sharing, which may lead to suboptimal solutions.
Conclusions
As a generalization of meta learning based on the bi-level formulation, a simple MOML framework based on the multi-objective bi-level optimization is proposed in this paper. In the MOML framework, the upper-level subproblem takes multiple objectives of a learning problem into consideration. To solve the objective function of the MOML framework, gradient-based optimization algorithm is proposed and the convergence property of this algorithm is studied. Moreover, several use cases of the MOML framework are investigated demonstrate the effectiveness of the MOML framework. In our future work, we will apply the MOML framework to more learning problems.
Acknowledgements
This work is supported by NSFC grant 62076118. We thank Dr. Jin Zhang for helpful discussion.
References
[1] Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all all: Train one network and specialize it for efficient deployment. In Proceedings of the 8th International Conference on Learning Representations, 2020.
10

---

[2] Rich Caruana. Multitask learning, Machine Learning. 28(1):41-75, 1997.
[3] Liang- Chich Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. with atrous separable convolution for semantic image segmentation. In Proceedings of the 14th European Conference on Computer Vision, volume 11211. pages 833 851, 2018.
[4] Xi Chen, Ali Ghadirzadeh, Marten Björkman, and Patric Jensfelt. Meta-lcarning for multi-objective reinforcement learning- Pruceedings of the IEEE/RSJ International Conference on Intelligent Robots and Syxtems, pages 977-983, 2019.
[5] Kalyanmoy Deb and Ankur Sinha. Solving bilevel multi-objective optimization problems using evolu- tionary algorithms. In Proceedings of the 5th International Conference on Evolutionary Multi- Multi-Criterion Optimization, pages 10-124, 2009.
[6] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
[7] Jean-Antoine Désidéri. Multple-gradient descent algorithu (MGDA) for multiobjective optimization. Comptes Rendus Mathematique, 350(5):31331) 2012.
[8] Justin Domke. Generic methods for optimization- based modeling. Proceedings of the 15th Artificial Intelligence and Statistics, pages 3 318-326, 2012.
[9] Gabriele Eichfelder. Twenty years of continuous multiobjective optimization. 2020.
[10] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient-base agdel-ag meta-learning algorithms. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 1082 1092, 2020.
[11] Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Measuring and hamnessing transference in multi-task learning arXiv preprint arKiv:2010. 15413, 2020.
[12] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learnin for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning, pages 1126 11351113, 2017.
[13] Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In Proceedings of the 35th International Conference on Machine Learning, pages 1568-1577, 2018
[14] Yaroslav Ganin and Victor S. Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 1180-1189 2015.
[15] loannis Giagkiozis, Robin Purshouse, and Peter J Fleming. An overview of population- algorithms for multi-objective optimisation. International Journal of Systems Science, 4619):1572-1599, 2015.
[16] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems, pages 529-536, 2004.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778,
2016.
[18] Nathan Hilliard, Lawrence Phillips. Scott Howland, Artëm Yankov, Courtney D Corley. and Nathan O Hodas. Few-shot learning with metric-agnostic conditional embeddings. arXiv preprint 1802 04376. 2018.
[19] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.0545 2020.
[20] Mike Huisman, Jan N. van Rijn, and Aske Plaat. survey of deep meta-learning. Artificial Intelligence Review, 54(6):448-44541, 2021.
[21] Xiaojie Jin, Jiang Wang, Joshua Slocum, Ming Hsuan Yang. Shengyang Dai. Shuicheng Yan, and Jiashi Feng. Re-darts: Resource constrained differentiable architecture search. arXiv preprint arXiv:1912.1288 2019.

---

[22] Yaochu Jin and Bernhard Sendhoff. Pareto-based multiobjectivv machine learning: An overview and studies. JEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(3):397-415, 2008.
[23] Taekyung Kim and Changick Kim. Attract, perturb, and explore: Learning feature alignment network for semi-supervised domain adaptation. In Proceedings of 16th European Conference on Computer Vision, volume 12359, pages 591-607, 2020.
[24] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, 2015.
[25] Alex Krizhevsky, Geoffrey Hinton, et Learning multiple layers features from tiny images. 2009.
[26] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Proceedings of the 5th International Conference on Learning Representations, 2017.
[27] Da Li and Timothy M. Hospedales. Online meta-Icarning for multi-source and semi-supervised domain adaptation. In Proceedings of the 16th European Conference on Computer Vision, volume 12361, pages 382-403, 2020.
[28] Li and Jitendra Malik. Learning to optimize. arXiv preprint arXi: 2016.
[29] Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: diffcrentiable architecture search. Proceed- ings of the 7th International Conference on Learning Representations, 2019.
[30] Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. generic first-order algorithmic framework for bi-level programming beyond lower-level singleton. Proceedings of the 37th International Conference on Machine Learning, pages 6305-6315, 2020.
[31] Shikun Liu, Edward Johns, and Andrew J. Davison. End-to-end multi-task learning with attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1871-1880, 2019.
[32] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems, pages 1647-1657, 2018.
[33] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning, pages 2208-2211 2217, 2017.
[34] Zhichao Lu, Kalyammoy Deb, Erik Goodman, Wolfgang Banzhaf, and Vishnu Naresh Boddeti. NS- GANetv2: Evolutionary multi-ohjective surrogate-assisted neural architecture search. In Proceedings the 16th Eumpean Conference on Computer Vision, pages 35-51, 2020.
[35] RE Lucchetti and Enrico Miglicrina. Stability for convex vector optimization problems. Optimization, 53(5-6):517-528, 2004.
[36] Roberto Lucchetti. Convexity and well-posed problems. Springer Science & Business Media, 2006.
[37] Debabrata Mahapatra and Vaibhav Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. Proceedings of the 37th International Conference on Machine Learning, pages 6597-6607, 2020.
[38] Nikola Milojkovic, Diego Antognini, Giancarlo Bergamin, Boi Faltings, and Claudiu Musat. Multi-gradient descent for multi-objective recommender systems. arXiv preprint arXiv:2001 10000400 2019.
[39] Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-leaming algorithms. arXiv preprint arXiv:1803.029%0 2018.
[40] Jachoon Oh, Hyungjun Yoo, ChangHwan Kim, and Se- Youno Yun. Boil: Towards representation change for few-shot learning. In Proceedings of the 9th International Conference on Learning Representations, 2021.
[41] Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In Proceedings of the 33rd International Conference on Machine Learning, pages 737-746, 2016.
[42] Hieu Pham, Melody Y. Guan, Barret Zoph, Quoe V. Le. and Jeff Dean. Efficient neural architecture search via parameter sharing. In Proceedings of the 35th International Conference on Machine Learning. volume 80, pages 4092 4101, 2018
12

---

[43] Samira Pouyanfar, Saad Sadiq, Yilin Yan, Haiman Tian, Yudong Tao, Maria Presa Reyes, Mei-Ling Shyu, Shu-Ching Chen, and SS Iyengar. A survey deep learning: Algorithms, techniques. and applications. ACM Computing Surveys, 51(5):1-36. 2018.
[44] Shaojian Qu, Mark Goh, and Felix TS Chan. Quasi-Newton methods for solving multiobjective optimiza- tion. Operations Research Letters, 3965)3377-399 2011.
[45] Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, and Sergey Levine. Meta-learning with implicit gradients. In Advances in Neural Information Processing Systems, pages 113-124, 2019.
[46] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In Proceedings of the 5th International Conference on Learning Representations, 2017.
[47] Sauli Ruuska and Kaisa Miettinen, Constructing evolutionary algorithms for bilevel multiobjective optimization. In Proceedings of the IEEE Congress on Evolutionary Computation, pages 1-7, 2012.
[48] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In Proceedings of the 6th European Conference on Computer Vision, pages 213 226. 2010.
[49] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Sacnko. Semi-supervised domain adaptation via minimax entropy. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8050-8058, 2019.
[50] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Sacnko. Adversarial dropout regularization. In Proceedings of the 6th International Conference on Learning Representations, 2018.
[51] Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. In Advances in Neural Information Processing Systems, pages 525 536, 2018.
[52] Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots. Truncated back-propagation for bilevel optimization. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, pages 1723-1732, 2019.
[53] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Proceedings of the 8th European Conference on Computer Vision, pages 746-760, 2012.
[54] Ankur Sinha. Bilevel multi-objective optimization problem solving using progressively interactive EMO. In Proceedings of the 6th International Conference on Evolutionary Multi-Criterion Optimization, pages 269-284, 2011.
[55] Jake Snell, Kevin Swersky. and Richard S. Zemel. Prototypical networks for few-shot learning. Advances in Neural Information Processing Systems, pages
2017.
[56] Flood Sung. Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1208, 2018
[57] Mingxing Tan. Bo Chen, Ruoming Pang. Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoe Le. MnasNet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 2820-2828, 2019.
[58] Hiroki Tanabe, Ellen H Fukuda, and Nobuo Yamashita. Proximal gradient methods for multiobjective optimization and their applications. Computational Optimization and Applications, 72(2):339-361, 2019.
[59] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint 112.::1412..4444, 2014.
[60] Hemanth Venkateswara, Jose Eusebio. Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018-5027, 2017.
[61] Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances in Neural Information Processing Syxtems, pages 3630-3638, 2016.
[62] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
13

---

[63] Xinyi Wang, Yulia Tsvetkov, and Graham Neubig. Balancing training for multilingual neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4537- 2020.
[64] Yaqing Wang. Quanming Yao. James T Kwok, and Lionel M Ni. Generalizing from few examples: survey on few-shot learning. ACM Computing Surveys, 53(3)11334 2020.
[65] Zirui Wang. Zachary C. Lipton, and Yulia Tsvetkov. Findings and meta-learning treatment. Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing, pages 4438 4450, 2020.
On negative interference in multilingual models:
[66] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE Conference on Computer Vision and Patter Recognition, pages 10734-10742, 2019.
[67] Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. SNAS: stochastic neural architecture search. In Proceedings of the 7th International Conference on Learning Representations, 2019.
[68] Zhongwen Xu, Hado van Hasselt, and David Silver. Meta-gradient reinforeement learning, In Advances in Neural Information Processing Systems, pages 2402-24222 2423,243, 2018
[69] Qiang Yang. Yu Zhang. Wenyuan Dai, and Sinno Jialin Pan. Transfer Learning. Cambridge University Press, 2020.
[70] Yao Yuan Yang. Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy robustness. In Advances Neural Information Processing Systems, volume 33, 2020.
[71] Tianhe Yu, Saurabh Kumar, Abhishck Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-tase task learning. Advances in Neural Information Processing Systems, 2020.
[72] Yu Zhang and Qiang Yang. A survey multi-task learning. IEEE Transactions Knowledge and Data Engineering. 2021.
[73] Aimin Zhou, Yang Qu. Hui Li. Shi-Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, and Qingfu Zhang. Multiobjective evolutionary algorithms: A survey of the state the Swarm Evolutionary Computation, 1(1):32 49, 2011.
[74] Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Guolin Ke, Jingwu Chen, Jiang Bian, Hui Xiong, and Qing He. Deep subdomain adaptation network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 32(4):1713-177) 2021.
14


Reviews:
[{"review_id": "pN9UXBlIM4a", "reviewer": "Reviewer_7cN3", "summary": "This paper proposes a framework for multi objective bi-level optimization. They first prove (under assumptions) convergence of their iterative algorithm, then show several instances of problems in this framework.", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think the goal of multi task learning to meta-learning and has not been explored in depth as far as I am aware. The papers demonstrate this and introduce a general framework which I believe captures the problem.\n\nI think discussion around why we might care about multi-task meta-learning and alternatives would be appreciated. In what settings, for example, do we have a distribution of tasks to meta-train over but not necessarily know which objectives we want at inner-test time? How does this relate to something more naive like joint training? In some sense, the extra level of meta- feels unneeded -- e.g. at meta-test time one could simply condition on the type objective with which to do well on and treat the problem like meta-learning not multi objective meta-learning. On the other hand, this is an interesting problem framing and thus hard to tell where clear benefits will be obtained. Show casing this further / building targeted examples where only multi-objective meta-learning will work would be interesting,\n\nThe problems discussed do not show the multi-objective nature of this work. My understanding is multi-objective method seek to find points on the pareto frontier of the different objectives. This is not actually shown or tested in the experiments. This is also a simpler setting than finding multiple points on this frontier which I traditionally associate with multi-objective and is what evolutionary methods often explore. Without visualizing this frontier though, it's hard to know if this method is working as intended or simply finding a different tradeoff.\n\nThe few shot learning setup: I found this a bit weird. Why the PGD attacks / adversarial robustness? I feel adversarial robustness adds a lot of complication to a problem when, as far as I can tell, it is simply another problem to test against. Have you considered leveraging something like this (https://arxiv.org/abs/1903.12261)?\n\nIf I am understanding things correctly, the vanilla baseline is trained targeting clean data where as SOML and MOML both have access to PGD data. It would be great to see a baseline for the vanilla baseline trained only with PGD data.\n\nIt is hard to evaluate performance / if this proposed solution is on the efficient frontier. It looks like adding MOL hurts clean performance by a fair bit, but does do better on PGD accuracy. A B-score (combination of PGD and clean data) is used, and is the main signal that the method proposed does better. If we are evaluating targeting a single objective, however, why not simply meta-train targeting this instead?\n\nAlso, how are the weights for SOML found? From the looks of it I don't see any search procedure done in the appendix. Again, if we know the target we are evaluating against I feel these could be set against that. Given enough compute, will SOML match MOML? or does the dynamic nature matter?\n\n\nFor the SSDA and multi task learning I have the same question on whether or not MOML is actually on the frontier.\n\nTable 3 is also lacking error bars. All methods seem similar enough that these would be appreciated.\n\nTheory:\nThis is treated as a big selling point and the result (thrm 2) is compelling but comes at the cost of some big assumptions which are not discussed. First, and most critically in my mind, K is never set to a large number. Usually it's < 10. In this setting, these problems can hardly be viewed as bi-level at all! Having some discussion on this would be appreciated to not mislead readers.\n\nClarity: I found the paper dense and put important details into the appendix. \nIn particular, the core of the method presented -- the MOPSolver -- is not discussed at all. I think this should be in the main text. Details of this algorithm, and it's connection to choosing weights ahead of time is important imo!\nAdditionally, details in how the weightings are chosen are also not discussed and should be in the main text.\n\nThere where also a number of awkward sentences e.g. (88 --\"Multi-objective optimization is to address the problem of simultaneously minimizing multiple objectives\").\n\nOriginality: As far as I know this method is novel though is closer to a novel combination of well-known techniques rather than an entirely novel method.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "For limitations see main review.\nNo social impact discussed. Given the effort NeurIPS has put into this effort, I believe something should be added to discuss this.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "_sApPNz9QK", "reviewer": "Reviewer_kG42", "summary": "The authors propose a Multi-Objection Meta Learning (MOML) formulation, offer a gradient based solver, show that the solver converges, and try to argue that MOML is useful in solving (1) few shot learning, (2) Semi-Supervised Domain Adaptatio, (3) Multi-Task Learning, and (4) NAS.", "questions": "", "limitations": "", "rating": 3, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I find it very difficult to understand this paper, and I am not convinced of the contribution of this paper.\n\nIn equation 1, which is the main objective of this paper. What do the author mean by minimizing over a vector valued function? Are the authors trying to find the Pareto optimal solution? I wish that I could guess what the authors try to say by understanding their algorithm. However they again introduce a min over vector value function in equation (4). But the authors mention that \"we can use any multi-objective optimization algorithms to solve it.\"\n\nThe simulations are not convincing me that MOML is better than the baselines. For example in section 5.1, the authors' method considered robustness against adversarial attack, but clearly MAML does not. It is not fair to compare a adversarial-attack-aware method with one that does not. Also the classification accuracy seems to have a big drop.\n \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "0.5 "}, "extra_text": {}}, {"review_id": "YYB1ZQAYEQR", "reviewer": "Reviewer_CNYc", "summary": "This work introduces a new meta-learning approach for multi-objective settings. Compared to existing works, the proposal does not require combining these multiple objectives into a single summary via a difficult-to-tune weight parameter.", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: I'm not an expert on existing approaches to meta-learning when there are multiple objectives, and so am not well qualified to assess the originality of this work.\n\n\nQuality: \n\n1.  When reading Theorem 1, I was surprised that it didn't require any sort of continuity condition on the function $f$. From looking at the proof in the supplement, it seems that there may be an issue with the current argument unless a further regularity condition is imposed in the theorem statement. In particular, the current proof claims that the accumulation point $\\bar{\\omega}$ is equal to $\\omega^*(\\bar{\\alpha})$, where both of these quantities are defined within the proof.\n\nHowever, a simple counterexample that satisfies all of the conditions of the currently stated Theorem 1 shows this to be false. In particular, you could have $\\mathcal{A}=[0,1]$, $p=1$, and $f(\\omega,\\alpha)=1_{\\alpha\\not=\\omega,\\omega\\in\\mathbb{Q}} + 1_{\\alpha\\not=2\\omega,\\omega\\not\\in\\mathbb{Q}}$, where $\\mathbb{Q}$ is the set of rationals. Then, we can choose a set of rational numbers $\\alpha_n$ converging to an irrational number $\\bar{\\alpha}$, and it will be the case that $\\omega^*(\\alpha_n)=\\alpha_n\\rightarrow\\bar{\\alpha}$, and this limit is not equal to $\\omega^*(\\bar{\\alpha})=2\\bar{\\alpha}$. Granted the $f$ in this example is pathological, but that's the point -- the current conditions of the theorem don't disallow such an $f$.\n\n2. More generally, I think more discussion is needed in the paper about the extent to which the regularity conditions assumed in the theorems are actually satisfied by the neural network parameterizations used in the experiments --- in particular, Conditions (ii) and (iv) in Theorem 2 both seem potentially problematic in neural network settings. In my view, the current presentation is somewhat misleading since it seems to imply that there are convergence guarantees for the implemented methods, which it seems there are not. E.g., it seems to me that this sentence in the abstract requires some qualification:\n\n> Theoretically, we prove convergence properties of the proposed gradient-based optimization algorithm.\n\n3. It's notable that, in Table 1, your main comparison seems to focus on the B-score criterion that you yourself have defined. Is there any more widely accepted score that you could use to evaluate the performance of your method?\n\nIt could also be worth finding a way to indicate the settings where XX+MOML (yours) dominates XX+SOML. It looks like this occurs for 1-shot BOIL and 5-shot BOIL. In all other settings, it looks like you outperform SOML according to PGD Acc but underperform in terms of Clean Acc. This makes me wonder if increasing the weight that SOML places on PGD Acc would lead to an approach that dominates yours in some settings (i.e., outperforms both in terms of Clean Acc and PGD Acc).\n\n\nClarity: \n\nThe presentation of the problem of interest at the beginning of Section 3 could be improved. First, it's critical to formally define the meaning of $\\min g(z)$ for a vector-valued function before or immediately after Eq. 1. At present, this isn't introduced until 2 pages later (top of page 5). Second, I think it would help to briefly mention what role $\\alpha$ and $\\omega$ will play in settings of interest, e.g. few shot learning.\n\n\nSignificance: \n\nMulti-objective settings are common in practice, and so the possibility of a meta-learning procedure that does not require selecting a difficult-to-tune weight function is appealing.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "9Pv1fttx3lU", "reviewer": "Reviewer_3cBH", "summary": "This paper considers the problem of meta-learning with multiple objectives in the optimization problem for the meta-parameters.\nA novel algorithm is proposed for this problem, and the approach is evaluated on a wide set of tasks and applications.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The motivation is clear and straightforward. The problem discussed in the paper is of interest to many ML applications, as seen in the vast experiments covering FSL, MTL, NAS, and DA.  \n- The paper is well written, although many details are missing from the main text due to the page limit.\n- The authors provide theoretical results on the convergence of the proposed Alg., which further motivates the approach.\n\nQuestions/suggestions/Weaknesses:\n- The broader scope of MOO is to find a set of solutions with different trade-offs across the Pareto front. This is not discussed in this paper and the approach only generates a single solution. So what is the advantage/benefit or conceptual difference from using MTL methods in the UL optimization?\n- In relation to the previous point, in sections 5.1 and 5.2, it would be good to evaluate other MTL approaches for weighting the losses (aside from linear scalarization/SOML), such as GradNorm, DWA, PCGrad, etc.\n- In all experiments, it would have been beneficial to discuss or visualize the corresponding losses (in terms of dominated solutions w.r.t baselines, Pareto frontier, etc.).\n- In relation to the previous point, the standard MOO metric is hyper-volume (HV). It would be useful to report the HV of multiple runs and compare it to baseline methods.\n- In Tables 2 and 3, please report standard errors or some other measure of variability for the results.\n- Section 5.3: At least for the SOML approach, how come the alpha corresponds to the easiest task (lowest loss) is not learned to be one while all other weights are set to zero (since the alphas are directly optimized on the validation loss)? This will minimize the weighted validation loss. Could you please explain why this is not the case? \n- The improvements in all experiments are generally fairly limited compared to baselines.\n- If I understand correctly, there is no validation split for baseline methods. How did you select hyper-parameters (HP) for baselines methods and for your method? Was there any optimization/search of HPs?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors should provide some discussion on the societal impacts of the work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4-5 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Unfortunately, all reviewers believe that the paper is not ready for publication at this point. The response provided by the authors partially addressed some of the comments raised by the reviewers, but it did not convince them to raise their score. I encourage the authors to revise their paper based on the comments provided by the reviewers and consider submitting their work to upcoming venues.
Decision: Accept (Poster)