E NHANCING M ETA L EARNING VIA M ULTI -O BJECTIVE S OFT I MPROVEMENT F UNCTIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
Meta-learning tries to leverage information from similar learning tasks.
In the commonly-used bilevel optimization formulation, the shared parameter is learned in the outer loop by minimizing the average loss over all tasks.
However, the converged solution may be compromised in that it only focuses on optimizing on a small subset of tasks.
To alleviate this problem, we consider meta-learning as a multi-objective optimization (MOO) problem, in which each task is an objective.
However, existing MOO solvers need to access all the objectives’ gradients in each iteration, and cannot scale to the huge number of tasks in typical meta-learning settings.
To alleviate this problem, we propose a scalable gradient-based solver with the use of mini-batch.
We provide theoretical guarantees on the Pareto optimality or
Pareto stationarity of the converged solution.
Empirical studies on various machine learning settings demonstrate that the proposed method is efficient, and achieves better performance than the baselines, particularly on improving the performance of the poorly-performing tasks and thus alleviating the compromising phenomenon.
I NTRODUCTION
Meta-learning, also known as “learning to learn", aims to enable models to learn more effectively by leveraging information from many similar learning tasks (Hospedales et al., 2020).
In recent years, meta-learning (Vilalta & Drissi, 2002) has received much attention for its fast adaptation to new learning scenarios with limited data (Kao et al., 2021; Finn et al., 2017;
Snell et al., 2017; Lee et al., 2019; Nichol et al., 2018; Deleu et al., 2022; Rajeswaran et al., 2019).
Meta-learning is usually formulated as a bi-level optimization problem (Franceschi et al., 2018;
Hong et al., 2020), which finds task-specific parameters in the inner level and minimizes the average (expected) loss over different tasks in the outer level.
Recently, Wang et al.
(2021) reformulate meta-learning as a multi-task learning problem.
From this perspective, minimizing the average loss in the outer level using (stochastic) gradient descent may not always be desirable.
Specifically, it may suffer from the compromising (or conflicting) phenomenon (Liu et al., 2021a; Sener & Koltun, 2018), in which the converged solution only focuses on minimizing the losses of a small subset of tasks while ignoring the others (Yu et al., 2020; Liu et al., 2021a; Sener & Koltun, 2018).
This compromised solution may thus lead to poor performance.
To alleviate this problem, we propose reformulating meta-learning as a multi-objective optimization (MOO) problem (Sener & Koltun, 2018), in which each task is an objective.
The performance of all the tasks (objectives) are then considered during optimization (Emmerich & Deutz, 2018).
A popular class of MOO solvers is the gradient-based approach (Liu et al., 2021a; Yu et al., 2020; Sener & Koltun, 2018; Navon et al., 2022; Liu et al., 2021b), with prominent examples such as the multiplegradient descent algorithm (MGDA)
(Désidéri, 2012; Sener & Koltun, 2018), PCGard (Yu et al., 2020), and CAGard (Liu et al., 2021a).
In each iteration, they find a common descent direction among all objective gradients, instead of simply optimizing the average performance over all objectives.
Existing gradient-based MOO methods rely on using gradients from all the objectives.
However, when formulating meta-learning as a MOO problem with each task being an objective, computing all these gradients in each iteration can become very expensive, as the number of objectives (i.e., tasks) can be huge.
For example, in 5-way 1-shot classification on the MiniImagenet dataset, the total number of meta-training tasks is (cid:0) 645 (cid:1)
≈
7 × 10 6 .
To address this challenge, we design a scalable MOO solver by using the improvement function (Miettinen & Mäkelä, 1995; Mäkelä et al., 2016; Montonen et al., 2018) with the help of mini-batch.
On the other hand, a trivial extension of existing gradient-based MOO methods with the use of mini-batch does not guarantee Pareto optimality and has poor performance in practice.
Our main contributions are as follows: (i) To alleviate the compromising phenomenon, we reformulate meta-learning as a multi-objective optimization problem in which each task is an objective; (ii) To handle the possibly huge number of tasks, we propose a scalable gradient-based solver with the use of mini-batch.
(iii)
We provide theoretical guarantees on the Pareto optimality or Pareto stationarity of the converged solution.
(iv) Empirical studies on few-shot regression, few-shot classification, and reinforcement learning demonstrate that the proposed method achieves better performance, particularly in improving the performance of the poorly-performing tasks and thus alleviating the compromising phenomenon.
B ACKGROUND
Multi-objective Optimization (MOO).
In MOO (Marler & Arora, 2004), one aims to minimizem ≥ 2 objectives f 1 ( x ) , . . .
, f m ( x ) :
[Equation]
Definition 2.1.
(Global Pareto optimality) (Miettinen, 2012; Mäkelä et al., 2016)
x ∗ is global Pareto optimal if there does not exist another x such that f τ ( x ∗ ) ≤ f τ ( x ) for all τ ∈ { 1 , . . .
, m } , and f τ ′ ( x ∗ )
< f τ ′ ( x ) for at least one τ ′ ∈ { 1 , . . .
, m } .
The Pareto front (PF) is the set of multi-objective values of all global Pareto-optimal solutions.
Definition 2.2.
(Pareto stationarity) (Miettinen, 2012; Désidéri, 2012)
x ∗ is Pareto-stationary iff there exist { u i } mi =1 such that ∥ (cid:80) mτ
=1 u τ ∇ x f τ ( x ) ∥
= 0 , u τ ≥ 0 ∀ τ and (cid:80) mτ
=1 u τ = 1 .
Note that global Pareto optimal solutions are also Pareto stationary (Désidéri, 2012).
Analogous to the extension from a stationary point to an ϵ -stationary point (Lin et al., 2020), we extend Pareto stationarity to ϵ -Pareto stationarity.
Obviously, 0 -Pareto stationarity reduces to Pareto stationarity.
Definition 2.3.
( ϵ -Pareto stationarity).
A point x is ϵ -Pareto-stationary iff there exist { u i } mτ =1 such that ∥ (cid:80) mτ
=1 u τ ∇ x f τ ( x ) ∥ ≤ ϵ, u τ ≥ 0 ∀ τ and (cid:80) mτ =
u τ = 1 .
Definition 2.4.
(Improvement function) (Montonen et al., 2018)
The improvement function of problem (1) is:
[Equation]
Note that x ∗ satisfying x ∗ =
arg min x H ( x,
x ∗ )
(intuitively, x ∗ cannot be further improved) is a
Pareto stationary point (Montonen et al., 2018).
To find x ∗ , one can perform steepest descent on H :
[Equation]
[Equation]
where x s is the iterate at iteration s , β is the learning rate satisfying H ( x s + βd, x s ) <
H ( x s , x s ) , and λ ′ is a hyper-parameter.
It can be shown that when s → ∞ , x s is Pareto stationary (Montonen et al., 2018).
In this paper, we focus on gradient-based MOO methods, including MGDA (Désidéri, 2012; Sener &
Koltun, 2018), PCGard (Yu et al., 2020), and CAGard (Liu et al., 2021a).
They assign weights to each objective’s gradient and find a common descent direction that decreases the losses of all objectives.
For example, MGDA finds the direction g ∗ ( x )
= (cid:80) mτ
= [Equation] x f τ ( x ) in each iteration, where
[Equation]
Meta-Learning.
Meta-learning aims to achieve good performance with limited data and computation (Hospedales et al., 2020).
Most of them are gradient-based (Nichol et al., 2018; Deleu et al., 2022; Rajeswaran et al., 2019;
Zhou et al., 2019; Shu et al., 2019) or metric-based (Snell et al., 2017;
Lee et al., 2019; Vinyals et al., 2016).
Let T be the set of all m tasks, and w be the shared model parameter.
For a task τ ∈ T , let D τ be its dataset and L τ the corresponding loss function.
It tries to obtain task-specific parameter w τ from the shared w as w ∗ τ ( w ) .
Meta-learning is usually formulated as the following bilevel optimization problem (Ji et al., 2021):
[Equation]
The inner-subproblem learns the task-specific parameter w τ for each τ , while the outer-subproblem learns w by minimizing the average loss over tasks in T .
As m can be very large, usually a mini-batch B of tasks are uniformly sampled from T , and w is then updated as w s +1 =
w s − β 1 | B | (cid:80) τ ∈ B L τ ( w ∗ τ ( w ))
(Finn et al., 2017).
By taking each L τ ( w ∗ τ ( w )) in (5) as an objective, this can be regarded as a weighted sum in multi-task learning (Wang et al., 2021).
As observed in (Sener & Koltun, 2018; Yu et al., 2020), gradient descent on this weighted sum can suffer from the compromising phenomenon, in which the loss obtained on some task τ
′ can be much larger than the losses on the other tasks.
S OFT I MPROVEMENT M ULTI -O BJECTIVE META -L EARNING (SIMOL)
To mitigate the compromising phenomenon, we take the view of meta-learning as multi-task learning in (Wang et al., 2021)
one step further and consider the meta-learning problem as the following multi-objective optimization (MOO) problem:
[Equation]
in each each task corresponds to an objective.
This considers all the individual tasks instead of simply considering the total loss over all tasks (Liu et al., 2021a).
Recently, Ye et al.
(2021) also used multi-objective learning into meta-learning.
However, their focus is not on addressing the compromising phenomenon and they do not treat each task as an objective.
Instead, besides minimizing the average task loss in (5), they consider adding some other objectives such as robustness to adversarial attacks.
Moreover, MGDA is then used to find the Pareto optimal solution.
However, as in other gradient-based MOO methods (Yu et al., 2020; Liu et al., 2021a), MGDA requires collecting gradients from all m objectives in each iteration (as can be seen from its optimization problem (4)).
This is computationally feasible only when there are a small number of objectives.
2 When each task is treated as an objective, the number of objectives can easily be in the millions (as in performing 5-way 1-shot classification on MiniImagenet).
To alleviate this problem, one solution is to use only a mini-batch of objectives in each iteration.
For example, when a subset B of objectives is used, MGDA’s optimization problem (4) becomes:
[Equation]
However, the descent direction then only considers objectives in B , and the original normalization constraint (cid:80) mτ =1 γ τ = 1 in (4) is also changed to (cid:80)
τ ∈ B γ τ = 1 .
The obtained solution can no longer be guaranteed to be Pareto optimal.
In the following, we demonstrate this empirically by using a simple toy example with two objectives ( f 1 ( x ) and f 2 ( x ) , where x ∈ R 2 ) from (Liu et al., 2021a;
Navon et al., 2022).
3 As can be seen from Figure 1, mini-batch MGDA (with a mini-batch of 1) cannot converge to the Pareto front.
3. S OFT I MPROVEMENT F UNCTION
In this section, we design a scalable MOO solver with the use of the improvement function (Montonen et al., 2018).
The proposed solver is agnostic to the number of tasks, while still theoretically guaranteeing that the solution is Pareto optimal.
[Figure]
Using Definition 2.4, the improvement function for problem (6) is:
[Equation]
We first rewrite the above as (the correctness will be proven in Lemma 3.1 below):
[Equation]
where π is a probability density function on τ .
The following Lemma shows that (7) and (8) are equivalent, when π is the Dirac delta distribution concentrated on the task corresponding to the maximum in (7).
All the proofs are in Appendix B.
Lemma 3.1.
[Equation]
Using (3) and Lemma 3.1, w is updated as
[Equation]
Taking the first-order approximation
[Equation]
(11) the minimax theorem (Simons, 1995) can then be applied to swap the min and max operators in (10), leading to
[Equation]
The following Proposition shows that the inner minimization problem has a closed-form solution.
Proposition 3.2.
[Equation]
w
[Equation]
w
[Equation]
w
[Equation]
∥ [Equation]
and the optimal d is d [Equation]
[Equation]
The expectation in Proposition 3.2 requires sampling tasks from π .
An easier alternative is to sample tasks from the uniform distribution U ( · ) over the set T of all tasks, and then weighting each sampled task τ with
[Equation]
Note that
[Equation]
We further parameterize r as a neural network r θ with parameter θ .
Using Proposition 3.2, we can then rewrite (12) as
[Equation]
where the last term (with another hyper-parameter λ ′′ ) is a penalty for enforcing the constraint in (14).
For notational simplicity, we denote the objective in (15) by K ( θ ) .
In principle, θ ∗ can be obtained from (15) by gradient ascent.
However, problem (15) involves an expectation over tasks.
Recall that we have a total of m tasks, and m can be huge.
Hence, using all of them to compute this expectation may not be feasible.
Instead, Let B be a mini-batch of k tasks, and denote the the mini-batched version of the objective in (15) as:
[Equation]
where ˆ λ ′ , ˆ λ ′′ are another set of hyper-parameters (which will be set in Proposition 3.3) corresponding to λ ′ , λ ′′ in (15).
Note that ˜ K T ( θ )
=
K ( θ ) .
Let B be the set of all size- k mini-batches ( k > 1 ).
The following Proposition bounds the difference between K ( θ ) , the original objective in (15), and the version 1 |B| (cid:80) B ˜ K B ( θ ) based on mini-batches.
Proposition 3.3.
Set λ ′ C 1 |B| =
ˆ λ ′ , λ ′′ C 1 |B| k 2 = ˆ λ ′′ .
We have:
[Equation]
where C 1 ≡
[Equation]
C
[Equation]
k |B
[Equation]
(cid:88)
[Equation]
and
[Equation]
−
[Equation]
Corollary 3.3.1.
When k ≪ m , (cid:16) K ( θ ) − 1 |B| (cid:80)
[Equation]
In the experiments, m ≥ 10 6 , k ≈ 10 2 , and G 1 , G 2 ≤ 10 4 .
When λ ′ ≥ 10 2 , λ ′′ ≤ 10 − 2 , we have ( K ( θ ) − 1 |B| (cid:80) B ∈B ˜ K B ( θ )) 2 ≤ 0 .
2 , which is small.
Thus, Proposition 3.3 shows that K ( θ ) can be decomposed into mini-batches as 1 |B| (cid:80) B ∈B ˜ K B ( θ ) .
This allows us to update θ by SGD (with learning rate β ′ ) over the task mini-batches:
[Equation]
Similarly, we approximate d ∗ by its mini-batch approximation
[Equation]
−
[Equation]
| [ (cid:80)
[Equation]
B
[Equation]
L
[Equation]
( w ∗ τ ( w )) | w = w s ] and update w by: w s {++
+}
= w s + β ˜ d ∗ .
The whole procedure, which will be called Soft Improvement Multi-Objective Meta Learning (SIMOL), is shown in Algorithm 1.
Step 4 trains the base learner.
In the experiments, we use the two popular meta-learning algorithms MAML (Finn et al., 2017) and prototypical network (PN)
(Snell et al., 2017) as the backbone.
For MAML, the base learner is updated by
[Equation]
where α is the base learner’s learning rate.
For the PN,
[Equation]
where S k is the set of examples belonging to class k , Q τ is the set of query examples for task τ , N C is the number of classes per epoch, and f w is the model with parameter w .
Algorithm
[Equation]
[Equation]
[Equation]
[Equation]
C ONVERGENCE A NALYSIS
Define
[Equation] [Equation]
where ∆( θ ) ≡ − 1 λ
E τ ∼
U r θ ( τ ) ⊥
[( L τ ( w ∗ τ ( w ) + d ∗ ) − L τ ( w ∗ τ ( w ))]] · ⊥ [ E τ ∼ U r θ ( τ )[ L τ ( w ∗ τ ( w ) + d ∗ ) − L τ ( w ∗ tau ( w ))]] , and ⊥ is the stop gradient operator.
Lemma 4.1.
∇ w R ( θ, w ) | w = w s = − d ∗ , and ∇ θ R ( θ, w s )
= ∇ θ K ( θ ) .
This allows interpreting the updates in (16) and (9) as performing Gradient Descent Ascent (GDA) (Singh et al., 2000) on (17).
Thus, we can leverage game theoretical tools (Lin et al., 2020) in the analysis.
Let ˜ R
( θ, w ; B ) :
=
⊥
[Equation]
1) 2 be the mini-batch version of R , and U ( B ) be the uniform distribution over task mini-batches.
The following Theorem shows that Algorithm 1 converges to an ϵ -Pareto stationary point of (6).
Theorem 4.2.
Assume that (i) L τ ( w ∗ τ ( · )) is L-smooth and r θ ( · ) is µ -strongly concave.
(ii) The domain of θ is a convex and bounded set with diameter D > 0 ,
(iii) E B ∼ U ( B ) [ ∇ θ ˜ R
( θ, w ; B ) −∇ θ R ( θ, w )]
= 0 , and E B ∼ U ( B ) ∥∇ θ
˜ R
( θ, w, B )
− ∇ θ R ( θ, w ) ∥ 2 ≤ σ 2 .
Assume the first-order approximation in (11), and take β = Θ (cid:0) 1 /D 2 σ
( L 2 + σ 2 ) (cid:1) , β ′ =
Θ(1 /Lσ 2 ) .
Algorithm 1 converges to an ϵ -Pareto stationary point of (6) with a rate of O (1 /ϵ 8 ) .
If L τ ( w ∗ τ ( w )) is also µ ′ -convex w.r.t.
w and ϵ = 0 , the 0 -Pareto stationary point is also global Pareto optimal.
Assumption (i) is standard (Collins et al., 2020;
Zhou et al., 2021;
Finn et al., 2019); while (ii) and (iii) are from (Lin et al., 2020).
Theorem 4.2 shows that the proposed method can obtain an ϵ -Pareto stationary point (or global Pareto optimal point for convex objectives) regardless of m .
Corollary 4.2.1.
Consider the MAML base learner update w ∗ τ ( w ) =
w − α ∇ w L τ ( w ) .
Assume that ∇ w L τ ( w ) is Hessian-Lipschitz, bounded and Lipschitz-continuous, w is bounded.
then, Algorithmconverges to an ϵ -Pareto stationary point of (6) with a rate of O (1 /ϵ 8 ) .
Corollary 4.2.1 is an application of Theorem 4.2 revealing that SIMOL with MAML could also converge to Pareto point.
Convergence of the outer loop is slower than the O ( 1 ϵ 2 ) rate of standard MAML (Fallah et al., 2020).
However, standard MAML only guarantees convergence to stationary points but not to Pareto-stationary points.
Moreover, as will be seen in Section 5.1, empirically, the proposed method has comparable or even slightly faster convergence speed than MAML and other baselines.
Among the gradient-based MOO approaches, most of them, except CAGrad (Liu et al., 2021a), do not provide convergence rate analysis.
However, the analysis in CAGrad relies on the assumption that all the task gradients are used in every epoch, which can have a very high per-epoch complexity, as will be empirically demonstrated in Section 5.2.
E XPERIMENTS
In this section, we perform experiments on few-shot regression (Section 5.1), few-shot classification (Section 5.2), and reinforcement learning (Section 5.3).
All experiments are run on a single GeForce
RTX 2080 Ti GPU and Intel(R)
Xeon(R) CPU E5-2680.
Our implementations are based on the
Learn2Learn framework (Arnold et al., 2020), a popular open-source meta-learning library.
5. F EW - SHOT R EGRESSION
In this section, we follow the setup in (Finn et al., 2017; Li et al., 2017).
Experiments are performed on 5-shot and (the harder) 2-shot regression using a synthetic dataset.
The target function for task τ is
[Figure] y = a τ sin( x + b τ ) , where a τ and b τ are sampled uniformly from [0 . 1 , 5 . 0] and [0 , π ] , respectively.
We generate 160 , 000 meta-training tasks and 1 , 000 meta-testing tasks.
A multilayer perceptron with 2 fully-connected (FC) layers (each of size 32 ) and ReLU activation is used as meta-learner and re-weighting network.
The re-weighting network uses all instances in each mini-batch as input.
The backbone meta-learning algorithm is MAML (Finn et al., 2017).
We use Adam (Kingma & Ba, 2014), with an initial learning rate of 0 . 01 , to update the base learners for 5 steps in the innerloop.
Hyperparameters for the backbone meta-learning algorithms follow the original papers.
For the outer-loop, we compare (i) minimizing the (single objective) of overall task loss as in original
MAML; versus performing MOO with (ii) mini-batch MGDA
(Désidéri, 2012; Ye et al., 2021), (iii) mini-batch CAGrad, using the same hyper-parameters as (Liu et al., 2021a), and (iv)
the proposed
SIMOL (with β = 0 . 0008 , β ′ = 0 .
03 , ˆ λ ′′ = 0 .
01 , and ˆ λ ′ = 1 ).
The mini-batch size is always 16 .
We do not compare with the batch versions of MGDA/CAGrad, as computing all task gradients takes extremely large memory and time.
The initial learning rate for the outer loop is 0 .
The experiment is repeated three times with different random seeds.
For performance evaluation, we use the mean-squared-error (MSE) over all meta-testing tasks.
We also report the worst- 10% MSE, which is the average MSE for the 10% worst-performing meta-testing tasks.
Table 1 shows the results.
As can be seen, SIMOL consistently outperforms SGD, MGDA, and
CAGrad in terms of both the overall and worst-10% MSEs.
Indeed, (mini-batch) MGDA and CAGard are even worse than the original MAML.
The good performance of SIMOL in terms of the worst- 10%
MSE also lends support to the hypothesis that improving the poorly performing tasks can also aid overall performance.
[Table]
Figure 2 shows the convergence of MSE with the number of training epochs.
As can be seen,
SIMOL converges slightly faster than the other baselines.
Figure 3 shows the learned functions on two example meta-testing tasks 5 in the 5-shot and 2-shot settings.
As can be seen, SIMOL fits the ground-truth better than the baselines.
5. F EW - SHOT I MAGE C LASSIFICATION
In this section, we perform 5 -way- 1 -shot and 5 -way- 5 -shot classification on miniImageNet (Ravi &
Larochelle, 2016) and tieredImageNet (Ren et al., 2018).
Following (Finn et al., 2017) [Table] we split the miniImageNet dataset into a meta-training set with 64 classes, a meta-validation set with 16 classes, and a meta-test set with 20 classes.
The total number of meta-training tasks is (cid:0) 645 (cid:1) ≈ 7 . 6 × 10 6 .
Similarly, as in (Zhou et al., 2019), we split the tieredImageNet dataset into a meta-training set with 351 classes, a validation set with 97 classes, and a test set with 160 classes.
The total number of meta-training tasks is (cid:0) 3515 (cid:1) ≈ 4 .
3 × 10 10 .
for both datasets, we randomly select 1 , 000 meta-testing tasks for evaluation.
We use two backbone meta-learning algorithms, MAML and prototypical network (PN)
(Snell et al., 2017).
Hyperparameters for the backbone meta-learning algorithms follow the original papers.
Following (Finn et al., 2017; Li et al., 2017; Zintgraf et al., 2019), we use the CNN4 6 (LeCun et al., 2015) as the meta-learner.
The optimizer is Adam.
The learning rates are 0 .
003 for MAML-based methods and 0 .
001 for PN-based methods, respectively.
The mini-batch size is 32.
The other hyper-parameters are the same as in Section 5.1.
More details can be found in Appendix D.
The proposed SIMOL is compared with (i) minimizingt the overall task loss as in standard MAML (named as min average loss ), (ii) MGDA, and (iii) CAGrad.
We also compare with MAML variants including (iv) Reptile (Nichol & Schulman, 2018), (v) FOMAML (Finn et al., 2017), (vi) MetaMinibatchProx
(Zhou et al., 2019), (vii) TSA-MAML (Zhou et al., 2021), and (viii) IMAML (Rajeswaran et al., 2019)) and the standard prototypical network (Snell et al., 2017), IX) MTL (Wang et al., 2021), a multi-task learning based meta learning.
The evaluation metrics are similar to those in Section 5.1, but with accuracy instead of MSE.
Experiments are repeated three times with different random seeds.
Results on miniImageNet and tieredImageNet are shown in Tables 3 and 4, respectively.
As can be seen, with MAML, SIMOL consistently outperforms all the other baselines in terms of both overall and worst-10% accuracies.
The same is also observed when the backbone meta-learning algorithm is the prototypical network.
This demonstrates that SIMOL is useful in both gradient-based and metric-based meta-learning approaches.
Note that the performance of (mini-batch) MGDA and CAGrad are not good in terms of both overall and worst-10% accuracies, showing that they cannot be straightforwardly extended to the use of minibatch.
On the other hand, the batch versions of MGDA and CAGrad are computationally impractical.
Table 2 compares the per-epoch running time of standard MAML, SIMOL, and batch MGDA and
CAgrad.
experiment is performed on 5-way-5-shot classification with the MAML algorithm on miniImageNet.
As can be seen, while SIMOL has comparable per-epoch running time as MAML, batch MGDA and CAgrad are much more computationally expensive (around 432 , 000 times slower).
[Table]
5. R EINFORCEMENT L EARNING
In this experiment, we demonstrate that the proposed SIMOL can also be used in reinforcement learning.
Experiments are performed on the HalfCheetach-Dir and Walker-Dir environments
(Rothfuss et al., 2018).
In both environments, each task corresponds to a random direction in the XY-plane, and the agent (Walker/ HalfCheetach) learns to run in that direction as far as possible.
The reward is equal to average velocity minus control costs.
We again use MAML as the meta-learning algorithm, and the base RL algorithm is vanilla policy gradient (VPG) (Sutton et al., 1999).
Following (Rothfuss et al., 2018), the policy network has 64 × 64 FC layers with tanh as the activation function, while the critic is a linear state-value function whose parameters are obtained by minimizing least-squares.
Figure 4 shows the convergence of the accumulated reward with number of iterations.
As can be seen, outperforms MAML
Moreover, the of MAML is less stable, which [Table] [Table] is also reported by (Rothfuss et al., 2018).
On the other hand, as SIMOL focuses more on the poorly performing tasks, its convergence is smoother and more stable.
C ONCLUSION
In this paper, we propose to avoid the compromising phenomenon in meta-learning by reformulating it as a multi-objective optimization (MOO) problem, in which each task is an objective.
However, current gradient-based MOO solvers cannot scale to a large number of objectives.
With the use of improvement function and mini-batch, we propose a scalable gradient-based solver with theoretical guarantees to Pareto-optimality.
Empirical studies on few-shot regression, few-shot classification, and reinforcement learning demonstrate that the proposed method is efficient, and has good generalization in terms of both overall performance and performance on the poorly-performing tasks.
A T OY EXAMPLE AND P SEUDO C ODE A. T OY EXAMPLE
The definition of f 1 and f 2 are: f 1 ( x ) =
c 1 ( x ) l 1 ( x ) + c 2 ( x ) g 1 ( x ) and f 2 ( x ) =
c 1 ( x ) l 2 ( x ) + c 2 ( x ) g 2 ( x ) , where l 1 ( x ) = log (max ( | 0 . 5 ( − x 1 − 7) − tanh ( − x 2 ) | , 0 . 000005))
+ 6 , l 2 ( x ) = log (max ( | 0 .
5 ( − x
[Equation] [Equation]
[Equation]
x
[Equation] [Equation]
+ 6 , g 1 ( x ) = (cid:16) (
[Equation]
[Equation] [Equation]
[Equation] [Equation]
(cid:17) /
[Equation]
20 , g
[Equation] [Equation]
[Equation]
(cid:16) (
[Equation]
[Equation] [Equation]
[Equation] [Equation]
(cid:17)
[Equation] [Equation]
20 , c 1 ( x ) = max (tanh (0 .
5 x 2 ) , 0) and c 2 ( x ) = max (tanh ( − 0 .
5 x 2 ) , 0) .
For mini-batch MGDA, and SIMOL the probability to sample task 1 is 2 / 3 and 1 / 3 for task 2.
The learning rates for MGDA and mini-batch MGDA are 0 .
The learning rate for SIMOL meta learner is 0 .
01 , while for the reweight network is 0 .
A. P SEUDO C ODE
The code in blue is the major differences between standard methods and SIMOL based methods.
Algorithm between original method
our
Input: T , and
total epoch S
B is the batch size, β and β ′ are learning rates for w and θ .
α is the learning rates for inner loop.
1 for epoch s = 1 , 2 , 3 , . . .
, S do Sample tasks 1 ,
B ; for Every task τ do
Receive r θ s ( τ ) ;
Compute adapted parameters with gradient descent w ∗ τ ( w ) =
w s − α ∇ w L τ ( w s ) ; w s +1 =
w s − β s 1 B (cid:80) Bτ =1 r θ s ( τ ) ∇ w L τ ( w ∗ τ ( w )) ; θ s +
= θ s + β ′ s ˜ K
( θ, B ) ;
1 for epoch s = 1 , 2 , 3 , . . .
, S do
Sample tasks 1 , 2 , . . .
B ; for Every task τ do for Every class c do
Select the support S cτ and query set Q τ ;
Compute prototype c τ = 1 N C (cid:80) ( x i ,y i )
∈ S cτ f θ ( x i ) ;
Calculate w ∗ τ ( w ) = 1 | Q τ | N C (cid:80) x ∈ Q τ exp( −∥ f w ( x ) − c k ∥ 2 ) (cid:80) k ′ exp( −∥ f w ( x ) − c k ′ ∥ 2 ) ;
Receive r θ s ( τ ) ; w s +1 =
w s − β s 1 B (cid:80) Bτ =1 r θ s ( τ ) ∇ w L τ ( w ∗ τ ( w )) ; θ s +
= θ s + β ′ s ˜ K
( θ, B ) ;
B O MITTED P ROOFS B. P ROOFS IN S EC .
Proof.
(of Lemma 3.1) Observe that
[Equation]
0 otherwise.
We firstly show that
[Equation]
This can be done by setting π ( τ ) =
p ( τ ) .
We then show that
[Equation]
This is established since
[Equation]
where τ ′ = arg max τ =1 ...,m {L τ ( w ∗ τ ( w ))
− L τ ( w ∗ τ ( w ′ )) } , and (a) is due to the fact that [ L τ ′ ( w ∗ τ ′ ( w ))
[Equation]
L τ ′ ( w ∗ τ ′ ( w ′ ))
] ≥ [ L τ ( w ∗ τ ( w ))
− L τ ( w ∗ τ ( w ′ ))] , ∀ τ based on the property of τ ′ .
Therefore, both [ L τ ′ ( w ∗ τ ′ ( w )) − L τ ′ ( w ∗ τ ′ ( w ′ ))
] ≥ [ L τ ( w ∗ τ ( w ))
− L τ ( w ∗ τ ( w ′ ))] and [ L τ ′ ( w ∗ τ ′ ( w ))
− L τ ′ ( w ∗ τ ′ ( w ′ ))]
≤ [ L τ ( w ∗ τ ( w ))
− L τ ( w ∗ τ ( w ′ ))] are established.
Thus, max τ =
[Equation]
Proof.
(of Lemma 3.2)
According to the assumptions,
[Equation]
Taking the derivatives w
[Equation]
r
[Equation]
t d
[Equation]
Setting the above equation to zero E U ( τ ) [ r θ ( τ ) ∇ w L D τ ( w ∗ τ ( w s ))]
+ d =
and we find the optimum:
[Equation]
Putting d ∗ back to arg min d E U
[Equation]
L τ ( w
[Equation]
( w
[Equation]
d (cid:105)(cid:105) +
[Equation]
2 || d
|| 2 , we have:
[Equation]
The following two lemmas are not hard to obtain, and we put them here for completeness.
Lemma B.1.
When L τ ( w ∗ τ ( w s + d ))
≈ L τ ( w ∗ τ ( w s ) + ∇ w L τ ( w ∗ τ ( w s ))
⊤ d Eq.
(10) is convex w.r.t d and concave.
w.r.t r θ ( τ ) .
Proof.
Putting L τ ( w ∗ τ ( w s + d ))
[Equation]
w
[Equation]
( w
[Equation] [Equation]
d into Eq.
(10), we have:
[Equation]
Regarding d , for the ∇ w L τ ( w ∗ τ ( w s ))
⊤ d
+ λ ′ 2 ||
d || 2 since ∇ w L τ ( w ∗ τ ( w s ))
⊤ d is both convex and concave, and λ ′
||
d || 2 is convex.
Then the sum of two convex function ∇ w L τ ( w ∗ τ ( w s ))
⊤ d
+ λ ′ 2 || d ||is convex.
Thus, Eq.
(10) is convex regarding d .
Similarly, regarding r θ ( τ ) , since − λ ′′ 2 ( E U ( τ ) [ r θ ( τ )] − 1) 2 is concave.
Therefore, Eq.
(10) is also concave regarding r θ ( τ ) .
Lemma B.2.
When L
[Equation]
w
[Equation]
w
[Equation] [Equation]
Proof.
It is worth noting that it is established according to the minimax theoremm(Simons, 1995).
And the remaining is to show that the above equation is Eq.
(10) is convex w.r.t d and concave.
w.r.t r θ ( τ ) , which have been already proven in Lemma B.1.
The following we are going to proof the Prop. 3.3.
Lemma B.3.
If we construct a set B such that every element B ∈ B is a set consists of a unique selection of k ( m > k > 1 ) tasks out of m (the total number of tasks) tasks without replacement, then we have:
[Equation]
where C 1 =
[Equation]
, and C 2 =
[Equation]
n-dimension vector.
Proof.
Note that
[Equation]
(a) is due to the polynomial theorem and 1 ( τ, τ ′ )
= (cid:26) 2 τ ̸ = τ ′ 1 otherwise .
(b) is due to the fact that every task τ has occurred exactly (cid:0) n − 1 B − 1 (cid:1) times, and every tuple ( τ, τ ′ ) has occurred exactly (cid:0) n − 2 B − 2 (cid:1) times.
Lemma B.4.
If we construct a set B such that every element B ∈ B is a set consists of a unique selection of k ( m > k > 1 ) tasks out of m (the total number of tasks) tasks without replacement, then we have:
[Equation]
|
[Equation]
where C 1 = k 2 ( m − 2 k − 2 ) m 2 , and C
=
[Equation]
Proof.
This can be done by observing the fact that || g ( τ ) || 2 = (cid:80)
i g i ( τ ) 2 .
The remaining follows the Lemma B.3.
Proposition B.5.
Assume that ˜ K T ( θ )
=
K ( θ ) .
Set
[Equation]
, we have:
[Equation]
where C 1 :=
k 2 ( m
[Equation]
−
[Equation]
) m 2 , and C 2 :
= (cid:20)(cid:18) m −
[Equation]
−
[Equation]
(cid:19)
[Equation]
(cid:18)
[Equation]
k
[Equation]
(cid:19)(cid:21)
[Equation]
(cid:18)
[Equation]
, G
[Equation]
:= k |B| (cid:88)
[Equation]
r θ ( τ ) − 1]
[Equation]
When k ≪ m , we have ( K ( θ ) − 1 |B| (cid:80) B ˜ K B ( θ )) 2 ≤ G 1 k λ ′
+ G 2 k λ ′′ .
Proof.
(of Proposition 3.3) Invoking lemmas B.3 and B.4, we have:
[Equation]
where ˜ d ∗ := 1 | B | (cid:80) B ∈B r θ ( τ ) ∇ w L τ ( w ∗ τ ( w )) | w = w s .
Then, |
[Equation]
By noting that ˜ K T ( θ ) with K ( θ )
[Equation]
we obtain our first claim.
Regarding the second claim
[Equation]
note that when m is large,
[Equation]
The first approximation is due to the fact that (cid:0) m − 1 k − 1 (cid:1) ≫ 12 (cid:0) m − 2 k − 2 (cid:1) .
The third and forth equations are due to the property of combinatorics ( (cid:0) mk (cid:1) =
mk (cid:0) m − 1 k − 1 (cid:1) )
Putting 1 k − 1 back to the Eq.
(20), we obtain the results.
B. P ROOF OF L EMMA 4.
Proof.
(Lemma 4.1)
By directly calculating, we have:
[Equation]
Also notice that:
[Equation]
Therefore,
[Equation]
B. P ROOF OF T HEOREM 4.
In (Montonen et al., 2018)
[Equation]
the authors mention that when s → ∞ , x s and H is differential w.r.t x s , is the Pareto Stationary point without detailed proof.
In the following
[Equation]
we provide a way to proof that.
We firstly define
Definition B.6.
(Weak Global Pareto Optimality) (Mäkelä et al., 2016)
A solution x ∗ of problem (6) is weakly global Pareto optimal if there does not exist another x such that f τ ( x ∗ ) ≤ f τ ( x ) for τ ∈ { 1 , . . .
, m } .
Theorem B.7.
(Theorem 5 in (Miettinen & Mäkelä, 1995))
For a multi-objective learning min w [ f 1 ( w ) , f 2 ( w ) , . . .
, f m ( w )] and its corresponding improvement function H ( w, w ∗ ) .
A necessary condition for w ∗ ∈ R n to be a global weak Pareto optimum is that w ∗ =
arg min w ∈ R n H ( w, w ∗ ) .
And if further f i ( w ) , ∀ i is convex, then it is a sufficient condition.
Here we introduce two observations aid our following analysis: 1)
By replacing f τ =
L τ ◦ w ∗ τ ( w ) , ∀ τ , the above theorem can be directly applied to MAML setting.
2) f in our setting might be neural network, and thus the convex condition is hard to preserve.
Here, we give the local version of the above theorem:
Theorem B.8.
w ∗ := arg min w H ( w, w ∗ ) is also the Pareto stationary point.
Proof.
According to Theorem 2 in (Miettinen & Mäkelä, 1995), we have ∂ w H ( w, w ∗ ) ⊂ conv (cid:83) i ∂ w ( f i ( w ) − f i ( w ∗ )) , where ∂ is reloaded as sub-gradient, and conv is the convex set.
Note that 0 ∈ ∂ w H ( w, w ∗ ) due to the fact that the convex set of the unions of sub-gradients are still sub-gradients, and in extreme point, the sub-gradient is zero.
Then, we have (cid:80) i
w i ∂
w ( f i ( w ) − f i ( w ∗ ))
= 0 , (cid:80) w i = 1 , based on the property of definition of convex set.
By reducing the above term, we have (cid:80) i w i ∂ ( f i ( w ) − f i ( w ∗ ))
= (cid:80) i
w i
∂
w f i ( w ) = 0 .
which is exactly the definition of Pareto stationary point.
Theorems B.7 and B.8 discuss the global and local relationship between Pareto optimality and the improvement function respectively.
We firstly show the convergence properties of Alg. 1:
Lemma B.9.
By running Alg. 1, for each epoch s , when λ ′′ = (cid:26)
[Equation]
∞ , otherwise , we have:
[Equation]
If equality holds, it is a stationary point.
Proof.
By Lemma B.2, the solutions of
[Equation]
is equivalent to
[Equation]
Solving θ , the above minimax problem degenerates to
[Equation]
Note the above equation is min d H ( w s + d, w s ) + λ ′
||
d
|| 2 , where d is the steepest descent direction.
Then, due to the property of steepest descent direction, we always have H ( w s + d, w s ) ≤ H ( w s , w s ) and the equality holds iff the stationary point
||
d || 2 = 0 .
Lemma B.10.
Alg.
1 converges to Pareto stationary point.
If further L τ ( w ∗ τ ( w )) is convex, it is the global weak Pareto optilimity.
Proof.
According to Lemma B.9, the sequence { H ( w s + d, w s , w s +11: m , w s 1: m ) } s is deceasing.
Also obviously, H ( w s + d, w s , w s +11: m , w s 1: m ) has lower bound.
Thus, leveraging the Monotone convergence theorem, Alg.
1 converges.
According to Theorem B.8, it is the Pareto stationary.
Together with Theorem B.7, it is the global
Pareto optimality for convex case.
Lemma B.11.
i)
If L τ ( w ∗ τ ( w + d )) is L-smooth and µ ′ -convex w.r.t w and d , and r θ ( τ ) is µ -concave, r θ ( τ ) and L τ ( w ∗ τ ( w s +1 )) are bounded (i.e., r θ ( τ ) ≤ C max , L τ ( w ∗ τ ( w s +1 )) ≤ C ′ max ), then R ( θ, w ) is L + C max µ ′ convex w.r.t w and L smooth w.r.t w, θ and C ′ max µ concave w.r.t.
θ . ii)
And if L τ ( w ∗ τ ( w + d )) is L-smooth w.r.t w and d , and r θ ( τ ) is µ -concave, then
˜ R
( θ, w, B ) is L smooth w.r.t w, θ and C ′ max µ concave w.r.t.
θ .
Proof.
For the i) claim.
We firstly prove that ⊥ [ f ( x )] ⊥ [ g ( x )] is 0-smooth and convex w.r.t.
x for differential function f, g .
Note that
[Equation]
due to the property of stop gradient operator.
[Equation]
Regarding the convexity, by using the property of 0-smoothness, implies
[Equation]
Thus, it is convex due to the definition of convexity.
Therefore, we conclude that ⊥ [ f ( x )] ⊥ [ g ( x )] is also 0-smooth and convex w.r.t.
x .
A direct application of the above conclusion is that ⊥ [ f ( x )] is 0-smooth and convex w.r.t.
x by regarding g ( x ) ≡ 1 .
Then, using the above conclusion, we have ⊥
[Equation]
] are also 0-smooth and convex.
Therefore r
[Equation]
⊥
[Equation]
is
[Equation]
and L -smooth
[Equation]
L
[Equation]
Also, we have ⊥ ( r θ ( τ ))
[ L τ ( w ∗ τ ( w )]] is C max µ ′ convex and L -smooth due to our assumption.
Then E τ ∼ U [ ⊥ ( r θ ( τ ))
[ L
[Equation]
( w ∗ τ ( w )]] + ∆
[Equation]
−
[Equation]
( E
[Equation]
) − 1)
[Equation]
is C max µ ′ convex and 2 Lsmooth since the adding of convex functions implies C max µ ′ convex and the adding of smooth functions implies L -smooth, and the take the average does not affect the results.
Then, R ( θ, w ) is C max µ ′ convex and 2 L -smooth.
Regarding r θ , since − λ ′ 2 ∇ θ ( E p [ r θ s ( τ )] − 1) 2 is concave, ∆( θ ) is 2 µC ′ max concave.
Then ∆( θ ) − λ ′′ 2 ( 1 B (cid:80) τ
[ r θ ( τ )] − 1) 2 is also C ′ max µ -concave.
Thus R ( θ, w ) is C ′ max µ -concave.
The proof of our ii) claim can be similarly conducted by following the above proof.
Theorem B.12.
Assume that 1) L τ ( w ∗ τ ( · )) is L-smooth and r θ ( · ) is µ -strongly concave.
The domain of θ is a convex and bounded set with diameter D > 0 , 3) E B ∼ U ( B ) [ ∇ θ ˜ R
( θ, w ; B )
− ∇ θ R ( θ, w )]
= 0 , and E B ∼ U ( B ) ∥∇ θ
˜ R
( θ, w, B )
− ∇ θ R ( θ, w ) ∥ 2 ≤ σ 2 .
Assume the first-order approximation in Eq.
(11), and take β = Θ (cid:0) 1 /D 2 σ
( L 2 + σ 2 ) (cid:1) , β ′ =
Θ(1 /Lσ 2 ) .
Algorithmconverges to an ϵ -Pareto stationary point of (6) with a rate of O (1 /ϵ 8 ) .
If L τ ( w ∗ τ ( w )) is also µ ′ -convex w.r.t.
w and ϵ = 0 , the 0 -Pareto stationary point is also global Pareto optimal.
Proof.
Recall Lemma B.11 on the properties of convex and smooth for ˜ R .
Combining it with the assumptions mentioned above, and invoking Theorem 4.9 in (Lin et al., 2020), we can obtain the iteration bound of O ( 1 ϵ 8 ) when B = 1 .
When B > 1 , noting that in Lemma A.2 in (Lin et al., 2020), we have:
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
[Equation]
Therefore, denoting σ 2 ′ =
σ 2 B and reusing Theorem 4.9 in (Lin et al., 2020) shows iteration bound of
O ( 1 ϵ 8 ) .
The remaining part that the fixed point of w is the global Pareto optilimity (resp.
Pareto stationary point).
It can be directly testified by invoking Lemma B.10 that reveals the fixed point of Alg.
1 is the globally Pareto optimality (resp.
Pareto stationary point).
Finally, we show that using SIMOL to MAML can guarantee convergence based on the analysis above:
Corollary B.12.1.
Consider the MAML base learner update w ∗ τ ( w ) =
w − α ∇ w L τ ( w ) .
assume that ∇ w L τ ( w ) is Hessian-Lipschitz, bounded and Lipschitz-continuous, w is bounded.
then, Algorithmconverges to an ϵ -Pareto stationary point of (6) with a rate of O (1 /ϵ 8 ) .
Proof.
We firstly show that if ∇ w L τ ( w ) is Hessian-Lipschitz, bounded and Lipschitz continuous, and || w || is bounded, then w ∗ τ ( w ) is also Hessian-Lipschitz, bounded and Lipschitz continuous.
Note that ||
[Equation] [Equation]
[Figure]
[Equation]
Then, it is easy to find that w ∗ τ ( w ) is also Hessian-Lipschitz, bounded and Lipschitz continuous.
Applying Lemma 3 (Collins et al., 2020), we immediately get L τ ( w ∗ τ ( w )) is C -smooth, where C is a positive number.
Then setting C =
L and using theorem 4.2, we immediately get the desire result.
C G ENERALIZATION BETWEEN T RAINING DISTRIBUTION TO TESTING DISTRIBUTIONS .
Proposition C.1.
If the distribution over training tasks P ( T train ) and the distribution over testing tasks P ( T test ) are the same, when E τ ∼ P ( T train )
L τ ( w ∗ τ ( w ∗ )) ≤ ϵ , where w ∗ is the parameter learnt from SIMOL, we have E τ ∼ P ( T test )
L τ ( w ∗ τ ( w ∗ )) ≤ ϵ .
Similarly, when E τ ∼ P ( T train ) ||∇ w L τ ( w ∗ τ ( w ∗ ))
||
2 ≤ ϵ , we have E τ ∼ P ( T test ) ||∇ w L τ ( w ∗ τ ( w ∗ ))
||
2 ≤ ϵ . Proof. As the distribution over training tasks P ( T train ) and the distribution over testing tasks
P ( T test ) are the same, E τ ∼ P ( T train )
L τ ( w ∗ τ ( w ∗ )) =
E τ ∼ P ( T test )
L τ ( w ∗ τ ( w ∗ )) .
Therefore, it is obviously that E τ ∼ P ( T test )
L τ ( w ∗ τ ( w ∗ )) ≤ ϵ .
Similarly, E τ ∼ P ( T train ) ||∇ w L τ ( w ∗ τ ( w ∗ )) ||
2 =
E τ ∼ P ( T test ) ||∇ w L τ ( w ∗ τ ( w ∗ ))
|| 2 , we also have
E τ ∼ P
( T test )
||∇ w L τ ( w ∗ τ ( w ∗ ))
||
2 ≤ ϵ .
D E XTRA E XPERIMENTS D. M ORE DETAILED SETUP
Regarding the classification scenarios.
For the re-weighting network, we also use CNN4 as backbone.
Differently, we feed every instance in a task to CNN4 and then two layer FC with size 128 ×and 48 × 6 to generate the task embedding, and then, they are stacked together and be feed into a layer with size (6 ∗ # way ∗ # shot ) ∗ 1 (e.g., for 1-way-1-shot task, the layer size is (6 ∗ 5) ∗ 1 ).
The learning rate is 0 .
08 for SIMOL and 0 .
for SIMOL+PN.
Regarding how to select the hyper-parameters, the weighting for regularizers ˆ λ ′ , ˆ λ ′′ , and learning rate β ′ are selected by sweeping among { 0 . 001 , 0 . 01 , 0 . 1 , 1 } , and { 0 . 01 , 0 . 3 , 1 } on 2-shot regression tasks, and choose the best combinations (i.e., ˆ λ ′ = 0 . 01 , ˆ λ = 1 , β ′ = 0 . 03 ).
We directly use ˆ λ ′ = 0 . 01 , ˆ λ = 1 to other environments. Regarding the learning rate β ′ in classification environments, we select the best among { 0 . 0003 , 0 . 0008 , 0 . 01 , 0 . 03 , 0 . 08 , 0 . 1 } in 1 shot miniImageNet ( 0 . 08 for SIMOL and 0 .
for
SIMOL+PN) and extend it other classification environments. [Figure] [Figure] D. T OY E XAMPLE
Our extra toy examples show the convergence behavior of ours when finding a Pareto front.
We construct the synthetic as:
[Equation]
The task 1 and task 2 has the same probability to be sampled for SIMOL.
The initial point is uniformly sampled from [ − 0 .
5] following (Lin et al., 2019).
GD is used as the Optimizer.
The baselines are Linear Scalarization (with equal weights for each objective) (Jahn, 1985), Machine Learning based MGDA (MOOMTL (Sener & Koltun, 2018)), as well as ParetoMTL (Lin et al., 2019).
Results: As can be seen, ours have a more disperse points than other baselines.
D. R EGRESSION T ASKS .
Regarding the effectiveness of mini-batch size, as shown in Fig.
6, larger mini-batch size does improve the performance ( 1 . 43 for SIMOL-4 v.s. 1 . 24 for SIMOL-16).
However, the gain is deceasing with [Table] [Table] the increasing of the mini-batch size ( 1 . 23 for [-SIMOL-
-]SIMOL-64 v
[Table]
s. 1 . 24 for SIMOL-16).
Therefore, it is not necessary to maintain a large batch size.
The ablations on different layers’ structure is shown in 5. Results reveal that more layers with the same training time perform worse results. The extra results on PCGrad is shown in Tab. 9, showing that PCGrad does not suit regression tasks with large-scale tasks. D. E XTRA C LASSIFICATION R ESULTS
The best 10 % performance is shown in Tabs.
7 and 8.
Results show that ours have better performance over other baselines, but the improvement is smaller than that of the worse 10 % .
[Figure] [Table] [Table] [Table]