Title: A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm
Venue: NeurIPS 2023
Accepted: yes
Keywords: meta learning

robust fast adaptation

model agnostic meta learning
Abstract: Meta learning is a promising paradigm to enable skill transfer across tasks.
Most previous methods employ the empirical risk minimization principle in optimization.
However, the resulting worst fast adaptation to a subset of tasks can be catastrophic in risk-sensitive scenarios.
To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of tail task risk.
We take the two-stage strategy as heuristics to solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. 
Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.

A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm

Abstract

Meta learning is a promising paradigm to enable skill transfer across tasks. Most previous methods employ the empirical risk minimization principle in optimization. However, the resulting worst fast adaptation to a subset of tasks can be catastrophic in risk-sensitive scenarios. To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of expected tail risk. We take the two-stage strategy as heuristics to solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.

1 Introduction

The past decade has witnessed the remarkable progress of deep learning in real-world applications (LeCun et al., 2015). However, training deep learning models requires an enormous dataset and intensive computational power. At the same time, these pre-trained models can frequently encounter deployment difficulties when the dataset's distribution drifts in testing time (Lesort et al., 2021).

As a result, the paradigm of meta learning or learning to learn is proposed and impacts the machine learning scheme (Finn et al., 2017), which leverages past experiences to enable fast adaptation to unseen tasks. Moreover, in the past few

Probability

Frequencies

Probability

Frequencies

years, there has grown a large body of meta learning methods to find plausible strategies to distill common knowledge into separate tasks (Finn et al., 2017; Duan et al., 2016; Garnelo et al., 2018a).

Notably, most previous work concentrates merely on the fast adaptation strategies and employs the standard risk minimization principle, e.g. the empirical risk minimization, ignoring the difference between tasks in fast adaptation. Given the sampled batch from the task distribution, the standard meta learning methods weight tasks equally in fast adaptation. Such an implementation raises concerns in some real-world scenarios, when worst fast adaptation is catastrophic in a range of risk-sensitive applications (Johannsmeier et al., 2019; Jaafra et al., 2019; Wang et al., 2023b). For example, in

robotic manipulations, humanoid robots (Duan, 2017) can quickly leverage past motor primitives to walk on plain roads but might suffer from tribulation doing this on rough roads.

Research Motivations. Instead of seeking novel fast adaptation strategies, we take more interest in optimization principles for meta learning. Given the meta trained model, this paper stresses the performance difference in fast adaptation to various tasks as an indispensable consideration. As the concept of robustness in fast adaptation has not been sufficiently explored from the task distribution perspective, researching this topic has more practical significance and deserves more attention in meta learning. Naturally, we raise the question below:

Can we reconsider the meta learning paradigm through the lens of risk distribution, and are there plausible measures to enhance the fast adaptation robustness in some vulnerable scenarios?

Developed Methods. In an effort to address the above concerns and answer these questions, we reduce robust fast adaptation in meta learning to a stochastic optimization problem within the principle of minimizing the expected tail risk, e.g., conditional value-at-risk (CVaR) (Rockafellar et al., 2000) . To tractably solve the problem, we adopt a two-stage heuristic strategy for optimization with the help of crude Monte Carlo methods (Kroese and Rubinstein, 2012) and give some theoretical analysis. In each optimization step, the algorithm estimates the value-at-risk (VaR) (Rockafellar et al., 2000) from a meta batch of tasks and screens and optimizes a percentile of task samples vulnerable to fast adaptation. As illustrated in Fig. (1), such an operation is equivalent to iteratively reshaping the task risk distribution to increase robustness. The consequence of optimizing the risk function distributions ϑ k → ϑ k +1 is to transport the probability mass in high-risk regions to the left side gradually. In this manner, the distribution of risk functions in the task domain can be optimized toward the anticipated direction that controls the worst-case fast adaptation at a certain probabilistic level.

Outline & Primary Contributions. We overview related meta learning and robust optimization work in Section (2). Section (3) introduces general notations and describes meta learning optimization objectives together with typical models. The distributionally robust meta learning problem is presented together with a heuristic optimization strategy in Section (4). We report experimental results and analysis in Section (5), followed by conclusions and limitations in Section (6). Our primary contribution is two-fold:

We recast the robustification of meta learning to a distributional optimization problem. The resulting framework minimizes the conditional expectation of task risks, namely the tail risk, which unifies vanilla meta-learning and worst-case meta learning frameworks.

To resolve the robust meta learning problem, we adopt the heuristic two-stage strategy and demonstrate its improvement guarantee. Experimental results show the effectiveness of our method, enhancing fast adaptation robustness and mitigating the worst-case performance.

2 Literature Review

Meta Learning Methods. In practice, meta learning enables fast learning (adaptation to unseen tasks) via slow learning (meta training in a collection of tasks). There exist different families of meta learning methods. The optimization-based methods, such as model agnostic meta learning (MAML) (Finn et al., 2017) and its variants (Finn et al., 2018; Rajeswaran et al., 2019; Grant et al., 2018; Vuorio et al., 2019; Abbas et al., 2022), try to find the optimal initial parameters of models and then execute gradient updates over them to achieve adaptation with a few examples. The context-based methods, e.g. conditional neural processes (CNPs) (Garnelo et al., 2018a), neural processes (NPs) (Garnelo et al., 2018b) and extensions (Gordon et al., 2019; Foong et al., 2020; Wang and Van Hoof, 2020; Gondal et al., 2021; Wang and van Hoof, 2022; Shen et al., 2021; Wang et al., 2023a), learn the representation of tasks in the function space and formulate meta learning models as exchangeable stochastic processes. The metrics-based methods (Snell et al., 2017; Allen et al., 2019; Bartunov and Vetrov, 2018) embed tasks in a metric space and can achieve competitive performance in few-shot classification tasks. Other methods like memory-augmented models (Santoro et al., 2016; Xiao et al., 2021, 2022), recurrent models (Duan et al., 2016) and hyper networks (Zhao et al., 2020; Beck et al., 2023) are also modified for meta learning purposes.

Besides, there exist several important works investigating the generalization capability of methods. Bai et al. (2021) conduct the theoretical analysis of the train-validation split and connect it to

optimality. In (Chen et al., 2021), a generalization bound is constructed for MAML through the lens of information theory. Denevi et al. (2019) study an average risk bound and estimate the bias for improving stochastic gradient optimization in meta learning.

Robust Optimization. When performing robust optimization for downstream tasks in deep learning, we can find massive work concerning the adversarial input noise (Goodfellow et al., 2018; Goel et al., 2020; Ren et al., 2021), or the perturbation on the model parameters (Goodfellow et al., 2014; Kurakin et al., 2016; Liu et al., 2018; Silva and Najafirad, 2020). In contrast, this paper studies the robustness of fast adaptation in meta learning. In terms of robust principles, the commonly considered one is the worst-case optimization (Olds, 2015; Zhang et al., 2020; Tay et al., 2022). For example, Collins et al. (2020) conducts the worst-case optimization in MAML to obtain the robust meta initialization. Considering the influence of adversarial examples, Goldblum et al. (2019) propose to adversarially meta train the model for few-shot image classification. Wang et al. (2020) adopt the worst-case optimization in MAML to increase the model robustness by injecting adversarial noise to the input. However, distributionally robust optimization (Rahimian and Mehrotra, 2019) is rarely examined in the presence of the meta learning task distribution.

3 Preliminaries

Notations. Consider the distribution of tasks p ( τ ) for meta learning and denote the task space by Ω τ . Let τ be a task sampled from p ( τ ) with T the set of all tasks. We denote the meta dataset by D τ . For example, in few-shot regression problems, D τ refers to a set of data points { ( x i , y i ) } i m =1 to fit.

Generally, D τ are processed into the context set D C τ for fast adaptation, and the target set D T τ for evaluating adaptation performance. As an instance, we process the dataset D τ = D C τ ∪ D T τ with a fixed partition in MAML (Finn et al., 2017). D C τ and D T τ are respectively used for the inner loop and the outer loop in model optimization.

Definition 1 (Meta Risk Function) With the task τ ∈ T and the pre-processed dataset D τ and the model parameter ϑ ∈ Θ , the meta risk function is a map ℓ : D τ × Θ ↦→ R + .

In meta learning, the meta risk function ℓ ( D T τ , D C τ ; ϑ ) , e.g. instantiations in Example (1)/(2), is to evaluate the model performance after fast adaptation. Now we turn to the commonly-used risk minimization principle, which plays a crucial role in fast adaptation. To summarize, we include the vanilla and worst-case optimization objectives as follows.

Expected Risk Minimization for Meta Learning. The objective that applies to most vanilla meta learning methods can be formulated in Eq. (1), and the optimization executes in a distribution over tasks p ( τ ) . The Monte Carlo estimate corresponds to the empirical risk minimization principle.

Here ϑ is the parameter of meta learning models, which includes parameters for common knowledge shared across all tasks and for fast adaptation. Furthermore, the task distribution heavily influences the direction of optimization in meta training.

Worst-case Risk Minimization for Meta Learning. This also considers meta learning in the task distribution p ( τ ) , but the worst case in fast adaptation is the top priority in optimization.

The optimization objective is built upon the min-max framework, advancing the robustness of meta learning to the worst case. Approaches like TR-MAML (Collins et al., 2020) sample the worst task in a batch to meta train with gradient updates. Nevertheless, this setup might result in a highly conservative solution where the worst case only happens with an incredibly lower chance.

4 Distributionally Robust Fast Adaptation

This section starts with the concept of risk measures and the derived meta learning optimization objective. Then a heuristic strategy is designed to approximately solve the problem. Finally, we provide two examples of distributionally robust meta learning methods.

4.1 Meta Risk Functions as Random Variables

Assumption 1 The meta risk function ℓ ( D T τ , D C τ ; ϑ ) is β τ -Lipschitz continuous w.r.t. ϑ , which suggests: there exists a positive constant β τ such that ∀{ ϑ, ϑ ′ } ∈ Θ :

Let (Ω τ , F τ , P τ ) denote a probability measure over the task space, where F τ corresponds to a σ -algebra on the subsets of Ω τ . And we have ( R + , B ) a probability measure over the non-negative real domain for the previously mentioned meta risk function ℓ ( D T τ , D C τ ; ϑ ) with B a Borel σ -algebra. For any ϑ ∈ Θ , the meta learning operator M ϑ : Ω τ ↦→ R + is defined as:

In this way, ℓ ( D T τ , D C τ ; ϑ ) can be viewed as a random variable to induce the distribution p ( ℓ ( D T τ , D C τ ; ϑ )) . Further, the cumulative distribution function can be formulated as F ℓ ( l ; ϑ ) = P ( { ℓ ( D T τ , D C τ ; ϑ ) ≤ l ; τ ∈ Ω τ , l ∈ R + } ) w.r.t. the task space. Note that F ℓ ( l ; ϑ ) implicitly depends on the model parameter ϑ , and we cannot access a closed-form in practice.

Definition 2 (Value-at-Risk) Given the confidence level α ∈ [0 , 1] , the task distribution p ( τ ) and the model parameter ϑ , the VaR (Rockafellar et al., 2000) of the meta risk function is defined as:

Definition 3 (Conditional Value-at-Risk) Given the confidence level α ∈ [0 , 1] , the task distribution p ( τ ) and the model parameter ϑ , we focus on the constrained domain of the random variable ℓ ( D T τ , D C τ ; ϑ ) with ℓ ( D T τ , D C τ ; ϑ ) ≥ VaR α [ ℓ ( T , ϑ )] . The conditional expectation of this is termed as conditional value-at-risk (Rockafellar et al., 2000):

where the normalized cumulative distribution is as follows:

This results in the normalized probability measure (Ω α,τ , F α,τ , P α,τ ) over the task space, where Ω α,τ := ⋃ ℓ ≥ VaR α [ ℓ ( T ,ϑ )] [ M -1 ϑ ( ℓ ) ] . For ease of presentation, we denote the corresponding task distribution constrained in Ω α,τ by p α ( τ ; ϑ ) .

Rather than optimizing VaR α , a quantile, in meta learning, we take more interest in CVaR α optimization, a type of the expected tail risk . Such risk measure regards the conditional expectation and has more desirable properties for meta learning: more adequate in handling adaptation risks in extreme tails , more accessible sensitivity analysis w.r.t. α , and more efficient optimization .

Remark 1 CVaR α [ ℓ ( T , ϑ )] to minimize is respectively equivalent with the vanilla meta learning optimization objective in Eq. (1) when α = 0 and the worst-case meta learning optimization objective in Eq. (3) when α is sufficiently close to 1.

4.2 Meta Learning via Controlling the Expected Tail Risk

As mentioned in Remark (1), the previous two meta learning objectives can be viewed as special cases within the CVaR α principle. Furthermore, we turn to a particular distributionally robust fast

adaptation with the adjustable confidence level α to control the expected tail risk in optimization as follows.

Distributionally Robust Meta Learning Objective. With the previously introduced normalized probability density function p α ( τ ; ϑ ) , minimizing CVaR α [ ℓ ( T , ϑ )] can be rewritten as Eq. (3).

Even though CVaR α [ ℓ ( T , ϑ )] is a function of the model parameter ϑ , the integral in Eq. (3) is intractable due to the involvement of p α ( τ ; ϑ ) in a non-closed form.

Assumption 2 For meta risk function values, the cumulative distribution function F ℓ ( l ; ϑ ) is β ℓ -Lipschitz continuous w.r.t. l , and the implicit normalized probability density function of tasks p α ( τ ; ϑ ) is β θ -Lipschitz continuous w.r.t. ϑ .

Assumption 3 For any valid ϑ ∈ Θ and corresponding implicit normalized probability density function of tasks p α ( τ ; ϑ ) , the meta risk function value can be bounded by a positive constant L max :

Proposition 1 Under assumptions (1)/(2)/(3), the meta learning optimization objective E α ( ϑ ) in Eq. (3) is continuous w.r.t. ϑ .

Further, we use ξ α ( ϑ ) to denote the VaR α [ ℓ ( T , ϑ )] for simple notations. The same as that in (Rockafellar et al., 2000), we introduce a slack variable ξ ∈ R and the auxiliary risk function [ ℓ ( D T τ , D C τ ; ϑ ) -ξ ] + := max { ℓ ( D T τ , D C τ ; ϑ ) -ξ, 0 } . To circumvent directly optimizing the nonanalytical p α ( τ ; ϑ ) , we can convert the probability constrained function E α ( ϑ ) to the below unconstrained one after optimizing ξ :

It is demonstrated that E α ( ϑ ) = min ξ ∈ R φ α ( ξ ; ϑ ) and ξ α ∈ arg min ξ ∈ R φ α ( ξ ; ϑ ) in (Rockafellar et al., 2000), and also note that CVaR α is the upper bound of ξ α , implying

With the deductions from Eq.s (3)/(4), we can resort the distributionally robust meta learning optimization objective with the probability constraint into a unconstrained optimization objective as Eq.(5).

Sample Average Approximation. For the stochastic programming problem above, it is mostly intractable to derive the analytical form of the integral. Hence, we need to perform Monte Carlo estimates of Eq. (5) to obtain Eq. (6) for optimization.

Remark 2 If ℓ ( D T τ , D C τ ; ϑ ) is convex w.r.t. ϑ , then Eq.s (5)/(6) are also convex functions. In this case, the optimization objective Eq. (6) of our interest can be resolved with the help of several convex programming algorithms (Fan et al., 2017; Meng et al., 2020; Levy et al., 2020).

...

...

...

...

...

...

Estimation

Surrogate Function

Optimization

Improvement

Guarantee

4.3 Heuristic Algorithms for Optimization

Unfortunately, most existing meta learning models' risk functions (Finn et al., 2017; Garnelo et al., 2018a; Santoro et al., 2016; Li et al., 2017; Duan et al., 2016), ℓ ( D T τ , D C τ ; ϑ ) are non-convex w.r.t. ϑ , bringing difficulties in optimization of Eq.s (5)/(6).

To this end, we propose a simple yet effective optimization strategy, where the sampled task batch is used to approximate the VaR α and the integral in Eq. (5) for deriving Eq. (6). In detail, two stages are involved in iterations: (i) approximate VaR α [ ℓ ( T , ϑ )] ≈ ˆ ξ α with the meta batch values, which can be achieved via either a quantile estimator (Dong and Nakayama, 2018) or other density estimators; (ii) optimize ϑ in Eq. (6) via stochastic updates after replacing ξ α by the estimated ˆ ξ α .

Proposition 2 Suppose there exists δ ∈ R + such that | ξ α ( ϑ ) -ˆ ξ α ( ϑ ) | < δ with ˆ ξ α ( ϑ ) an estimate of ξ α ( ϑ ) . Then there exists a constant κ α = max { 2 -α 1 -α , α 1 -α } such that

The performance gap resulting from VaR α approximation error is estimated in Proposition (2). For ease of implementation, we adopt crude Monte Carlo methods (Kroese and Rubinstein, 2012) to obtain a consistent estimator of ξ α .

Theorem 1 (Improvement Guarantee) Under assumptions (1)/(2)/(3), suppose that the estimate error with the crude Monte Carlo holds: | ˆ ξ α t -ξ α t | ≤ λ β ℓ (1 -α ) 2 , ∀ t ∈ N + , with the subscript t the iteration number, λ the learning rate in stochastic gradient descent, β ℓ the Lipschitz constant of the risk cumulative distribution function, and α the confidence level. Then the proposed heuristic algorithm with the crude Monte Carlo can produce at least a local optimum for distributionally robust fast adaptation.

Note that Theorem (1) indicates that under certain conditions, using the above heuristic algorithm has the performance improvement guarantee, which corresponds to Fig. (2). The error resulting from the approximate algorithm is further estimated in Appendix Theorem (2).

4.4 Instantiations & Implementations

Our proposed optimization strategy applies to all meta learning methods and has the improvement guarantee in Theorem (1). Here we take two representative meta learning methods, MAML (Finn et al., 2017) and CNP (Garnelo et al., 2018a), as examples and show how to robustify them through the lens of risk distributions. Note that the forms of ℓ ( D T τ , D C τ ; ϑ ) sometimes differ in these methods. Also, the detailed implementation of the strategy relies on specific meta learning algorithms and models.

Example 1 (DR-MAML) With the task distribution p ( τ ) and model agnostic meta learning (Finn et al., 2017), the distributionally robust MAML treats the meta learning problem as a bi-level

optimization with a VaR α relevant constraint.

The gradient operation ∇ ϑ ℓ ( D C τ ; ϑ ) corresponds to the inner loop with the learning rate λ .

The resulting distributionally robust MAML (DR-MAML) is still a optimization-based method, where a fixed percentage of tasks are screened for the outer loop. As shown in Fig. (3) and Eq. (7), the objective is to obtain a robust meta initialization of the model parameter ϑ .

Example 2 (DR-CNP) With the task distribution p ( τ ) and the conditional neural process (Garnelo et al., 2018a), the distributionally robust conditional neural process learns the functional representations with a CVaR α constraint.

Fast Adaptation

Here h θ 1 is a set encoder network with θ 2 the parameter of the decoder network.

The resulting distributionally robust CNP (DR-CNP) is to find a robust functional embedding to induce underlying stochastic processes. Still, in Eq. (8), a proportion of tasks with the worst functional representation performance are used in meta training.

Moreover, we convey the pipelines of optimizing these developed distributionally robust models in Appendix Algorithms (1)/(2) .

5 Experimental Results and Analysis

This section presents experimental results and examines fast adaptation performance in a distributional sense. Without loss of generality, we take DR-MAML in Example (1) to run experiments.

Benchmarks. The same as work in (Collins et al., 2020), we use two commonly-seen downstream tasks for meta learning experiments: few-shot regression and image classification. Besides, ablation studies are included to assess other factors' influence or the proposed strategy's scalability.

Baselines & Evaluations. Since the primary investigation is regarding risk minimization principles, we consider the previously mentioned expected risk minimization , worst-case minimization , and expected tail risk minimization for meta learning. Hence, MAML (empirical risk), TR-MAML (worst-case risk), and DR-MAML (expected tail risk) serve as examined methods. We evaluate these methods' performance based on the Average , Worst-case , and CVaR α metrics. For the confidence level to meta train DR-MAML, we empirically set α = 0 . 7 for few-shot regression tasks and α = 0 . 5 image classification tasks without external configurations.

5.1 Sinusoid Regression

Following (Finn et al., 2017; Collins et al., 2020), we conduct experiments in sinusoid regression tasks. The mission is to approximate the function f ( x ) = a sin( x -b ) with K-shot randomly sampled function points, where the task is de-

fined by a and b . In the sine function family, the target range, amplitude range, and phase range are

respectively [ -5 . 0 , 5 . 0] ⊂ R , a ∈ [0 . 1 , 5 . 0] and b ∈ [0 , 2 π ] . In the setup of meta training and testing datasets, a distributional shift exists: numerous easy tasks and several difficult tasks are generated to formulate the training dataset with all tasks in the space as the testing one. Please refer to Appendix (J) for a detailed partition of meta-training, testing tasks, and neural architectures.

Result Analysis. We list meta-testing MSEs in sinusoid regression in Table (1). As expected, the tail risk minimization principle in DR-MAML can lead to an intermediate performance in the worstcase. In both cases, the comparison between MAML and DR-MAML in MSEs indicates that such probabilistic-constrained optimization in the task space even has the potential to advance average fast adaptation performance. In contrast, TR-MAML has to sacrifice more average performance for the worst-case improvement.

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

3

.

5

Mean Squared Error (MSE)

0

20

40

60

80

100

120

MAML

DR-MAML(Ours)

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

3

.

5

Mean Squared Error (MSE)

0

20

40

60

80

100

120

MAML

TR-MAML

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

Mean Squared Error (MSE)

0

20

40

60

80

100

120

140

MAML

DR-MAML(Ours)

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

Mean Squared Error (MSE)

0

20

40

60

80

100

120

140

MAML

TR-MAML

MSEs of Meta-Test Tasks on Sinusoid 5-shot

MSEs of Meta-Test Tasks on Sinusoid 10-shot

More intuitively, Fig. (4) illustrates MSE statistics on the testing task distribution and further verifies the effect of the CVaR α principle in decreasing the proportional worst-case errors. In 5-shot cases, the difference in MSE statistics is more significant: the worst-case method tends to increase the skewness in risk distributions with many task risk values gathered in regions of intermediate performance, which is unfavored in general cases. As for why DR-MAML surpasses MAML in terms of average performance, we attribute it to the benefits of external robustness in several scenarios, e.g. , the drift of training/testing task distributions.

5.2 Few-Shot Image Classification

Here we do investigations in fewshot image classification. Each task is an N-way K-shot classification with N the number of classes and K the number of labeled examples in one class. The Omniglot (Lake et al., 2015) and mini -ImageNet (Vinyals et al.,

2016) datasets work as benchmarks for examination. We retain the setup of datasets in work (Collins et al., 2020).

Result Analysis. The classification accuracies in Omniglot are illustrated in Table (2): In 5-way 1-shot cases, DR-MAML obtains the best average and CVaR α in meta-testing datasets, while TRMAMLachieves the best worst-case performance with a slight degradation of average performance compared to MAML in both training/testing datasets. In 20-way 1-shot cases, for training/testing datasets, we surprisingly notice that the expected tail risk is not well optimized with DR-MAML, but there is an average performance gain; while TR-MAML works best in worst-case/CVaR α metrics.

When it comes to mini -ImageNet, findings are distinguished a lot from the above one: in Table (3), DR-MAML yields the best result in all evaluation metrics and cases, even regarding the worst-case. TR-MAML can also improve all metrics in meta-testing cases. Overall, challenging meta-learning tasks can reveal more advantages of DR-MAML over others.

5.3 Ablation Studies

This part mainly checks the influence of the confidence level α , meta batch size, and other optimization strategies towards the distribution of fast adapted risk values. Apart from examining these factors of interest, additional experiments are also conducted in this paper; please refer to Appendix (K).

0

.

2

0

.

4

0

.

6

0

.

7

0

.

8

Confidence Level

0

.

6

0

.

8

1

.

0

1

.

2

1

.

4

1

.

6

Average

MAML

TR-MAML

DR-MAML(Ours)

0

.

2

0

.

4

0

.

6

0

.

7

0

.

8

Confidence Level

2

.

0

2

.

5

3

.

0

3

.

5

4

.

0

4

.

5

5

.

0

Worst

MAML

TR-MAML

DR-MAML(Ours)

0

.

2

0

.

4

0

.

6

0

.

7

0

.

8

Confidence Level

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

CVaR

MAML

TR-MAML

DR-MAML(Ours)

Sinusoid 5-shot

Sensitivity to Confidence Levels α . To deepen understanding of the confidence level α 's effect in fast adaptation performance, we vary α to train DR-MAML and evaluate models under previously mentioned metrics. Taking the sinusoid 5-shot regression as an example, we calculate MSEs and visualize the fluctuations with additional α -values in Fig. (5). The results in the worst-case exhibit higher deviations. The trend in Average/CVaR α metrics shows that with increasing α , DR-MAML gradually approaches TR-MAML in average and CVaR α MSEs. With α ≤ 0 . 8 , ours is less sensitive to the confidence level and mostly beats MAML/TR-MAML in average performance. Though ours aims at optimizing CVaR α , it cannot always ensure such a metric to surpass TR-MAML in all confidence levels due to rigorous assumptions in Theorem (1).

25

50

62

83

0

.

90

0

.

95

1

.

00

1

.

05

Mean Squared Error (MSE)

Sinusoid 5-shot

25

50

62

83

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

Mean Squared Error (MSE)

Sinusoid 10-shot

2

4

6

8

48

.

0

48

.

2

48

.

4

48

.

6

48

.

8

49

.

0

49

.

2

49

.

4

Classification Accuracies

mini

-ImageNet 5-way 1-shot

Influence of the Task Batch Size. Note that our optimization strategy relies on the estimate of VaR α , and the improvement guarantee relates to the estimation error. Theoretically, the meta batch in training can directly influence the optimization result. Here we vary the meta batch size in training, and evaluated results are visualized in Fig. (6). In regression scenarios, the average MSEs can decrease to a certain level with an increase of meta batch, and then performance degrades. We attribute the performance gain with a batch increase to more accurate VaR α estimates; however, a meta batch larger than some threshold can worsen the first-order meta learning algorithms' efficiency, similarly observed in (Nichol et al., 2018). As for classification scenarios, there appears no clear trend since the meta batch is smaller enough.

Comparison with Other Optimization Strategies. Note that instantiations of distributionally robust meta learning methods, such as DR-MAML and DR-CNPs in Examples (1)/(2) are regardless of optimization strategies and can be optimized via any heuristic algorithms for CVaR α objectives.

Additionally, we use DR-MAML as the example and perform the comparison between our two-stage algorithm and the risk reweighted algorithm (Sagawa et al., 2020). The intuition of the risk reweighted algorithm is to relax the weights of tasks and assign more weights to the gradient of worst cases. The normalization of risk weights is achieved via the softmax operator. Though there is an improvement guarantee w.r.t. the probabilistic worst group of tasks, the algorithm is not specially designed for meta learning or CVaR α objective.

Meanwhile, the weight of task gradients after normalization is a biased estimator w.r.t. the constrained probability p α ( τ ; ϑ ) in the task space. In other words, the risk reweighted method can be viewed as approximation w.r.t. the importance weighted method in Eq. (9). In the importance weighted method, for tasks out of the region of (1 -α ) -proportional worst, the probability of sampling such tasks τ b is zero, indicating ω b ( τ b ; ϑ ) = 0 . While in risk reweighted methods, the approximate weight is assumed to satisfy ω b ( τ b ; ϑ ) ∝ exp ( ℓ ( D T τ b , D C τ b ; ˆ ϑ ) τ ) , where ˆ ϑ means last time updated meta model parameters and the risk function value is evaluated after fast adaptation.

In implementations, we keep the setup the same as Group DRO methods in (Sagawa et al., 2020) for meta-training. As illustrated in Table (4)/(5), DR-MAML with the two-stage optimization strategies consistently outperform that with the Group DRO ones in both 5-shot and 10-shot sinusoid cases regarding all metrics. The performance advantage of using the two-stage ones is not significant in mini -ImageNet scenarios. We can hypothesize that the estimate of VaR α in continuous task domains, e.g. , sinusoid regression, is more accurate, and this probabilistically ensures the improvement guarantee with two-stage strategies. Both the VaR α estimate in two-stage strategies and the importance weight estimate in the Group DRO ones may have a lot of biases in few-shot image classification, which lead to comparable performance.

6 Conclusion and Limitations

Technical Discussions. This work contributes more insights into robustifying fast adaptation in meta learning. Our utilized expected tail risk trades off the expected risk minimization and worst-case risk minimization, and the two-stage strategy works as the heuristic to approximately solve the problem with an improvement guarantee. Our strategy can empirically alleviate the worst-case fast adaptation and sometimes even improve average performance.

Existing Limitations. Though our robustification strategy is simple yet effective in implementations, empirical selection of the optimal meta batch size is challenging, especially for first-order optimization methods. Meanwhile, the theoretical analysis only applies to a fraction of meta learning tasks when risk function values are in a compact continuous domain.

Future Extensions. Designing a heuristic algorithm with an improvement guarantee is non-trivial and relies on the properties of risk functions. This research direction has practical meaning in the era of large models and deserves more investigations in terms of optimization methods. Also, establishing connections between the optimal meta batch size and specific stochastic optimization algorithms can be a promising theoretical research issue in this domain.

Acknowledgments and Disclosure of Funding

This work is funded by National Natural Science Foundation of China (NSFC) with the Number # 62306326.

References

Abbas, M., Xiao, Q., Chen, L., Chen, P.-Y., and Chen, T. (2022). Sharp-maml: Sharpness-aware model-agnostic meta learning. In International Conference on Machine Learning , pages 10-32. PMLR.

Allen, K., Shelhamer, E., Shin, H., and Tenenbaum, J. (2019). Infinite mixture prototypes for few-shot learning. In International Conference on Machine Learning , pages 232-241. PMLR.

Antoniou, A., Edwards, H., and Storkey, A. (2019). How to train your maml. In Seventh International Conference on Learning Representations .

Bahadur, R. R. (1966). A note on quantiles in large samples. The Annals of Mathematical Statistics , 37(3):577-580.

Bai, Y., Chen, M., Zhou, P., Zhao, T., Lee, J., Kakade, S., Wang, H., and Xiong, C. (2021). How important is the train-validation split in meta-learning? In International Conference on Machine Learning , pages 543-553. PMLR.

Barabás, B. (1987). Estimation of density functions by order statistics. Periodica Mathematica Hungarica , 18(2):115-122.

Bartunov, S. and Vetrov, D. (2018). Few-shot generative modelling with generative matching networks. In International Conference on Artificial Intelligence and Statistics , pages 670-678. PMLR.

Beck, J., Jackson, M. T., Vuorio, R., and Whiteson, S. (2023). Hypernetworks in meta-reinforcement

learning. In Conference on Robot Learning , pages 1478-1487. PMLR.

Chen, Q., Shui, C., and Marchand, M. (2021). Generalization bounds for meta-learning: An information-theoretic analysis. Advances in Neural Information Processing Systems , 34:2587825890.

Chen, R. S., Lucier, B., Singer, Y., and Syrgkanis, V. (2017). Robust optimization for non-convex objectives. Advances in Neural Information Processing Systems , 30.

Collins, L., Mokhtari, A., and Shakkottai, S. (2020). Task-robust model-agnostic meta-learning. Advances in Neural Information Processing Systems , 33:18860-18871.

Denevi, G., Ciliberto, C., Grazzi, R., and Pontil, M. (2019). Learning-to-learn stochastic gradient descent with biased regularization. In International Conference on Machine Learning , pages 1566-1575. PMLR.

Dong, H. and Nakayama, M. K. (2018). A tutorial on quantile estimation via monte carlo. In International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing , pages 3-30. Springer.

Duan, Y. (2017). Meta learning for control . University of California, Berkeley.

Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P. (2016). Rl2: Fast reinforcement learning via slow reinforcement learning. arXiv preprint arXiv:1611.02779 .

Duchi, J. C. and Namkoong, H. (2021). Learning models with uniform performance via distributionally robust optimization. The Annals of Statistics , 49(3):1378-1406.

Fallah, A., Mokhtari, A., and Ozdaglar, A. (2021). Generalization of model-agnostic meta-learning algorithms: Recurring and unseen tasks. Advances in Neural Information Processing Systems , 34:5469-5480.

Fan, Y., Lyu, S., Ying, Y., and Hu, B. (2017). Learning with average top-k loss. Advances in neural information processing systems , 30.

Finn, C., Abbeel, P., and Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126-1135. PMLR.

Finn, C., Xu, K., and Levine, S. (2018). Probabilistic model-agnostic meta-learning. Advances in neural information processing systems , 31.

Foong, A., Bruinsma, W., Gordon, J., Dubois, Y., Requeima, J., and Turner, R. (2020). Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems , 33:8284-8295.

Gagne, C. and Dayan, P. (2021). Two steps to risk sensitivity. Advances in Neural Information Processing Systems , 34:22209-22220.

Garnelo, M., Rosenbaum, D., Maddison, C., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., Rezende, D., and Eslami, S. A. (2018a). Conditional neural processes. In International Conference on Machine Learning , pages 1704-1713. PMLR.

Garnelo, M., Schwarz, J., Rosenbaum, D., Viola, F., Rezende, D. J., Eslami, S., and Teh, Y. W. (2018b). Neural processes. arXiv preprint arXiv:1807.01622 .

Goel, A., Agarwal, A., Vatsa, M., Singh, R., and Ratha, N. K. (2020). Dndnet: Reconfiguring cnn for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , pages 22-23.

Goldblum, M., Fowl, L., and Goldstein, T. (2019). Robust few-shot learning with adversarially queried meta-learners.

Gondal, M. W., Joshi, S., Rahaman, N., Bauer, S., Wuthrich, M., and Scholkopf, B. (2021). Function contrastive learning of transferable meta-representations. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , volume 139 of Proceedings of Machine Learning Research , pages 3755-3765. PMLR.

Goodfellow, I., McDaniel, P., and Papernot, N. (2018). Making machine learning robust against adversarial inputs. Communications of the ACM , 61(7):56-66.

Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 .

Gordon, J., Bronskill, J., Bauer, M., Nowozin, S., and Turner, R. (2019). Meta-learning probabilistic inference for prediction. In International Conference on Learning Representations .

Grant, E., Finn, C., Levine, S., Darrell, T., and Griffiths, T. (2018). Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930 .

Hiraoka, T., Imagawa, T., Mori, T., Onishi, T., and Tsuruoka, Y. (2019). Learning robust options by conditional value at risk optimization. Advances in Neural Information Processing Systems , 32.

Hsieh, Y.-P., Mertikopoulos, P., and Cevher, V. (2021). The limits of min-max optimization algorithms: Convergence to spurious non-critical sets. In International Conference on Machine Learning , pages 4337-4348. PMLR.

Jaafra, Y., Laurent, J. L., Deruyver, A., and Naceur, M. S. (2019). Robust reinforcement learning for autonomous driving.

Jiang, Y., Li, C., Dai, W., Zou, J., and Xiong, H. (2021). Monotonic robust policy optimization with model discrepancy. In International Conference on Machine Learning , pages 4951-4960. PMLR.

Johannsmeier, L., Gerchow, M., and Haddadin, S. (2019). A framework for robot manipulation: Skill formalism, meta learning and adaptive control. In 2019 International Conference on Robotics and Automation (ICRA) , pages 5844-5850. IEEE.

Juditsky, A., Nemirovski, A., and Tauvel, C. (2011). Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems , 1(1):17-58.

Kroese, D. P. and Rubinstein, R. Y. (2012). Monte carlo methods. Wiley Interdisciplinary Reviews: Computational Statistics , 4(1):48-58.

Kurakin, A., Goodfellow, I., and Bengio, S. (2016). Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236 .

Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science , 350(6266):1332-1338.

Larochelle, S. (2017). Optimization as a model for few-shot learning. In International Conference on Learning Representations .

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature , 521(7553):436-444.

Lesort, T., Caccia, M., and Rish, I. (2021). Understanding continual learning settings with data distribution drift analysis. arXiv preprint arXiv:2104.01678 .

Levy, D., Carmon, Y., Duchi, J. C., and Sidford, A. (2020). Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems , 33:8847-8860.

Li, Z., Zhou, F., Chen, F., and Li, H. (2017). Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835 .

Liu, Q., Liu, T., Liu, Z., Wang, Y ., Jin, Y ., and Wen, W. (2018). Security analysis and enhancement of model compressed deep learning systems under adversarial attacks. In 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC) , pages 721-726. IEEE.

Meng, S. Y., Charisopoulos, V., and Gower, R. M. (2020). A stochastic prox-linear method for cvar minimization.

Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. (2009). Stochastic approximation approach to stochastic programming. SIAM J. Optim. Citeseer .

Nichol, A., Achiam, J., and Schulman, J. (2018). On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 .

Olds, K. C. (2015). Global indices for kinematic and force transmission performance in parallel robots. IEEE Transactions on Robotics , 31(2):494-500.

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E. Z., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019). Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , pages 8024-8035.

Pinto, L., Davidson, J., Sukthankar, R., and Gupta, A. (2017). Robust adversarial reinforcement learning. In International Conference on Machine Learning , pages 2817-2826. PMLR.

Puchert, P., Hermosilla, P., Ritschel, T., and Ropinski, T. (2021). Data-driven deep density estimation. Neural Computing and Applications , 33:16773-16807.

Quaranta, A. G. and Zaffaroni, A. (2008). Robust optimization of conditional value at risk and portfolio selection. Journal of Banking & Finance , 32(10):2046-2056.

Rahimian, H. and Mehrotra, S. (2019). Distributionally robust optimization: A review.

Rajeswaran, A., Finn, C., Kakade, S. M., and Levine, S. (2019). Meta-learning with implicit gradients. Advances in neural information processing systems , 32.

Ren, J., Zhang, D., Wang, Y., Chen, L., Zhou, Z., Chen, Y., Cheng, X., Wang, X., Zhou, M., Shi, J., et al. (2021). Towards a unified game-theoretic view of adversarial perturbations and robustness. Advances in Neural Information Processing Systems , 34:3797-3810.

Rockafellar, R. T., Uryasev, S., et al. (2000). Optimization of conditional value-at-risk. Journal of risk , 2:21-42.

Rosenblatt, M. (1956). A central limit theorem and a strong mixing condition. Proceedings of the national Academy of Sciences , 42(1):43-47.

Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. (2020). Distributionally robust neural networks. In International Conference on Learning Representations .

Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. (2016). Meta-learning with memory-augmented neural networks. In International conference on machine learning , pages 1842-1850. PMLR.

Shalev-Shwartz, S. and Wexler, Y. (2016). Minimizing the maximal loss: How and why. In International Conference on Machine Learning , pages 793-801. PMLR.

Shen, J., Zhen, X., Worring, M., and Shao, L. (2021). Variational multi-task learning with gumbelsoftmax priors. Advances in Neural Information Processing Systems , 34:21031-21042.

Silva, S. H. and Najafirad, P. (2020). Opportunities and challenges in deep learning adversarial robustness: A survey. arXiv preprint arXiv:2007.00753 .

Snell, J., Swersky, K., and Zemel, R. (2017). Prototypical networks for few-shot learning. Advances in neural information processing systems , 30.

Tay, S. S., Foo, C. S., Daisuke, U., Leong, R., and Low, B. K. H. (2022). Efficient distributionally robust bayesian optimization with worst-case sensitivity. In International Conference on Machine Learning , pages 21180-21204. PMLR.

Triantafillou, E., Zhu, T., Dumoulin, V., Lamblin, P., Evci, U., Xu, K., Goroshin, R., Gelada, C., Swersky, K., and Manzagol, P. A. (2019). Meta-dataset: A dataset of datasets for learning to learn from few examples.

Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al. (2016). Matching networks for one shot learning. Advances in neural information processing systems , 29.

Vuorio, R., Sun, S.-H., Hu, H., and Lim, J. J. (2019). Multimodal model-agnostic meta-learning via task-aware modulation. Advances in neural information processing systems , 32.

Wang, Q., Federici, M., and van Hoof, H. (2023a). Bridge the inference gaps of neural processes via expectation maximization. In The Eleventh International Conference on Learning Representations .

Wang, Q., Feng, Y., Huang, J., Lv, Y., Xie, Z., and Gao, X. (2023b). Large-scale generative simulation artificial intelligence: The next hotspot. The Innovation , page 100516.

Wang, Q. and Van Hoof, H. (2020). Doubly stochastic variational inference for neural processes with hierarchical latent variables. In International Conference on Machine Learning , pages 10018-10028. PMLR.

Wang, Q. and van Hoof, H. (2022). Learning expressive meta-representations with mixture of expert neural processes. In Advances in neural information processing systems .

Wang, R., Xu, K., Liu, S., Chen, P.-Y., Weng, T.-W., Gan, C., and Wang, M. (2020). On fast adversarial robustness adaptation in model-agnostic meta-learning. In International Conference on Learning Representations .

Wang, Z., Shen, Y., and Zavlanos, M. (2022). Risk-averse no-regret learning in online convex games. In International Conference on Machine Learning , pages 22999-23017. PMLR.

Wilder, B. (2018). Risk-sensitive submodular optimization. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32.

Xiao, Z., Shen, J., Zhen, X., Shao, L., and Snoek, C. (2021). A bit more bayesian: Domain-invariant learning with uncertainty. In International Conference on Machine Learning , pages 11351-11361. PMLR.

Xiao, Z., Zhen, X., Liao, S., and Snoek, C. G. (2022). Energy-based test sample adaptation for domain generalization. In The Eleventh International Conference on Learning Representations .

Zhang, J., Cheung, B., Finn, C., Levine, S., and Jayaraman, D. (2020). Cautious adaptation for reinforcement learning in safety-critical settings. In International Conference on Machine Learning , pages 11055-11065. PMLR.

Zhao, D., von Oswald, J., Kobayashi, S., Sacramento, J., and Grewe, B. F. (2020). Meta-learning via hypernetworks.

Contents

K Additional Experimental Results 29

L Platforms & Computational Tools 32

M Declaration of Author Contribution 32

A Frequently Asked Question

Here we collected technical questions and suggestions from researchers who helped check out the manuscript. We thank these researchers for precious questions and provide more details.

Novelty & primary findings of this work. Here we mainly summarize two points of novelty in this work:

Meta Learning Robustification Framework. Though the concept of the expected tail risk has emerged for several decades and has been widely employed in financial domains, the application to fast adaptation or robustification of meta learning remains limited in literature as far as we know.

Optimization Strategy & Theoretical Analysis. We theoretically analyze two-stage optimization strategies as the heuristic algorithm in optimizing the distributionally robust meta learning models and demonstrate the improvement guarantee under certain conditions.

As verified in experimental results in the main paper, placing a probabilistic constraint in the task space is meaningful. It circumvents the effect of over-pessimistic consideration (worst-case optimization), increases robustness in proportional cases, and mostly retains or even improves average performance.

Apart from the novelty in the framework and algorithm parts, we have several findings which bring crucial insights into meta learning: (i) Not all tasks are necessary to perform fast adaptation. (ii) Additional focus on the tail risk has the potential to enhance models' generalization capability. (iii) The tail risk instead of extreme worst-case risk can better advance robustness in challenging datasets.

Meta risk function values as random variables. In some few-shot learning related work, the context and the target dataset are equivalently called the support and query datasets. The definition of a task in meta learning is up to application scenarios and specific meta learning algorithms or models. The commonly-used sinusoid regression using model-agnostic meta learning (Finn et al., 2017) considers the fixed number of context points to induce tasks. In contrast, conditional neural processes (Garnelo et al., 2018a) for few-shot regression vary the number of context points to induce tasks. Once the context and the target are partitioned and the model parameter is specified, we can obtain a meta risk function value. However, the meta risk function values are not in a compact Euclidean subspace in several cases.

The continuity of the meta risk probability density function. This relates to the task distribution and meta learning problems. Throughout the few-shot regression task, the meta risk function value can be approximately viewed as a continuous random variable. However, when it comes to the few-shot image classification mission, the meta risk function value is impossible to cover the entire continuous interval. In these scenarios, the probable values of accuracies are finite. This makes the theoretical analysis, e.g. Theorem (1), no longer holds. For example, Assumption (2) will be unrealistic when there exists a constant gap between two accuracy values. Hence, we leave this part a future research direction in theoretical analysis. Regarding the meta risk function, ours shares the same setup as that in (Fallah et al., 2021). Admittedly, precisely modeling risk distributions can be crucial, and more techniques, e.g. , deep kernel estimators (Puchert et al., 2021) can be adopted to improve this point.

The selection of baselines in robust meta learning. A large body of prior work considers the robustness of meta learning in scenarios when the input of a data point, the model parameter, and the number of context points are trembled or modified. Robustness in presence of tasks distributions is seldom investigated except for the worst-case optimization in (Collins et al., 2020). Hence, we retain most of the setups in (Collins et al., 2020) for a fair comparison.

Influence of risk minimization principles. This paper is primarily devoted to studying the influence of the risk minimization principle on meta learning. The empirical risk minimization principle corresponds to reducing the Monte Carlo estimate of Average-case Meta Learning in this work. In comparison to TR-MAML (Collins et al., 2020), our approach has a couple of advantages as follows: (i) easier implementations. Note that min-max optimization is numerically unstable and requires a relaxation method for computationally intensive convex optimization, e.g. , robust stochastic mirror-prox algorithm used in TR-MAML (Collins et al., 2020). (ii) more flexible in terms of robustness concept. Theoretically, the worst-case meta learning corresponds to the extreme case of the distributionally robust risk minimization principle. (iii) empirically better performance in

most cases. The expected tail risk minimization preserves a particular property that minimizing proportional worst-case fast adaptation seldom sacrifices the average performance. These advantages are also why we call our approach simple yet effective .

Comparison with Other Heuristic Optimization Strategies. To sum up the proposed optimization strategy, we empirically highlight the following points regarding the adopted heuristic strategies. There exist a couple of approximate algorithms for CVaR α optimization. In comparison, the crude Monte Carlo one is the simplest for VaR α estimates. It has an improvement guarantee under certain conditions, leaving it easier to analyze. Besides, we also compare ours to the risk reweighted method (Sagawa et al., 2020) in previously used benchmarks, and please take a closer look at that in Section (K).

B Pseudo Algorithms of DR-MAML & DR-CNPs

Algorithm 1: DR-MAML

Algorithm 2: DR-CNP

C Properties of Risk Minimization Principles

C.1 Stochastic Optimization in the Constrained Form

In the section above, meta learning optimization objectives are discussed within three different principles, respectively the average-case in Eq. (1), the worst-case in Eq. (2) and CVaR α worst-case in Eq. (5). This subsection continues this topic and introduces the relaxation variable ξ in the optimization objective. In this way, the robust fast adaptation can be reframed in the case of stochastic optimization with the probabilistic constraint.

We can equivalently express the minimization of the worst-case problem (Shalev-Shwartz and Wexler, 2016) within the following constrained stochastic optimization framework as follows.

C.2 SGD Intractability of Meta Learning CVaR α Optimization Objective

This subsection shows that directly stochastic gradient descent is intractable for meta learning to optimize CVaR α . Note that the normalized density distribution function p α ( τ ; ϑ ) implicitly depends on ϑ and α , so we cannot access the exact form of such a distribution.

As illustrated in Eq. (11), the stochastic gradient estimate is not plausible since p α ( τ ; ϑ ) has no closed form. Hence, heuristic or convex programming algorithms for specific cases are mostly used as this domain's optimization strategy. However, designing optimization strategies for non-convex cases is non-trivial in this domain and requires more consideration of theoretical guarantees.

C.3 Risk-Sensitive Applications & Optimization Strategies

Related Applications. There are a number of applications concerning robust optimization with probabilistic constraints. Most of them are for the sake of safety. The risk principle CVaR α firstly occurs in the financial domain as a coherent principle (Rockafellar et al., 2000). It enjoys much popularity in portfolio optimization (Quaranta and Zaffaroni, 2008). Gagne and Dayan (2021) adopt CVaR α principles to improve the distributional reinforcement learning performance. To robustify robotic control, Pinto et al. (2017) varies hyper-parameters of Markov decision processes and optimizes proportional worst-case trajectories in policy optimization within the principle of CVaR α . In work (Wilder, 2018), a CVaR α related strategy is devised to solve submodular optimization problems. Such a risk measure is also included in producing robust options (Hiraoka et al., 2019). Regarding probabilistic robust meta learning within CVaR α principles, there exists scarce related work until now.

Optimization Strategies. Concerning the constrained stochastic optimization problem, we particularly overview related work in this subsection. In the past few decades, there emerge substantial optimization strategies together with theoretical analysis for convex risk functions in CVaR α optimization (Nemirovski et al., 2009; Fan et al., 2017; Meng et al., 2020; Levy et al., 2020; Duchi and Namkoong, 2021; Wang et al., 2022). As for the min-max risk minimization principle, which focuses on the worst case instead of a propositional worst cases, some researchers have designed relaxation or other heuristic algorithms to handle convex or non-convex risk functions (Chen et al., 2017; Collins et al., 2020; Jiang et al., 2021; Hsieh et al., 2021). Nevertheless, it remains challenging to design algorithms with convergence guarantees for non-convex risk function cases. One of the latest CVaR α work on non-convex risk functions is (Sagawa et al., 2020), where a risk reweighted algorithm for robust neural networks is proposed to handle distributional shifts. Moreover, most CVaR α optimization in non-convex risk functions follows this type of risk reweighted strategies in applications.

Remarks on Literature Work: Risk functions for robust fast adaptation cases are mostly non-convex, and it is non-trivial to design optimization strategies with a convergence guarantee. Meanwhile, we also conduct comparison experiments between work in (Sagawa et al., 2020) and ours in meta learning downstream tasks. We refer the reader to Appendix (K) for more details and analysis.

D Computational Complexity

The primary difference between distributionally robust meta learning and expected risk based meta learning lies in that a fixed probabilistic portion of task gradients are considered in meta updates. Such an operation drives the optimization procedure to focus more on proportional vulnerable scenarios, increasing the robustness in worst cases.

However, the iteration of the surrogate function φ ( ˆ ξ α t ; ϑ ) involves the screening of proportional worst cases since our strategies require extra evaluation of fast adaptation. This brings additional computational cost with complexity O ( B log( B ) ) , where B is the number of tasks in each batch. On the other hand, the proposed strategy performs sub-gradient updates instead of complete gradient updates. This helps reduce computational cost with complexity O ( α B|M| ) in each iteration, where |M| corresponds to the scale of parameters of meta learning models and α is the confidence level in CVaR α optimization.

Universally analyzing the computational complexity in meta learning is intractable since various meta learning methods exist. Some are gradient-based ones, while some are non-parametric ones. The exact number of iterations for convergence heavily relies on specific methods. In summary, given the same number of iterations and a fixed confidence level, the computational complexity difference for CVaR α optimization in meta learning scenarios is O ( B ( α |M| log( B )) ) .

To deepen understanding of our method, we explain more via specific examples. The main idea is to execute the sub-gradient operation over the batch of task gradients. Here we take the DR-MAML in Example (1) to show the operation and how the distributionally robust meta initialization is obtained:

where λ 1 is the outer loop learning rate, λ 2 is the inner loop learning rate, and δ ( τ i ) is the indicator variable. Here δ ( τ i ) = 1 in the case when the ℓ ( D T τ i ; ϑ τ i t ) falls into the (1 -α ) -probabilistic worst-case region otherwise δ ( τ i ) = 0 .

As for the optimal rate for convergence or the generalization bound, it is up to specific meta learning methods and risk minimization principles. For worst-case risk minimization for meta learning, there already exists theoretical analysis in previous work, e.g. , optimization-based meta learning (Collins et al., 2020) when fast adaptation functions hold the convexity property. When it comes to more universal cases, considering diverse meta learning methods and optimization strategies, it is still challenging to estimate the optimal rate for convergence. Also, our considered scenarios exist distributional drift between meta training and meta testing task distributions, which makes it tough to derive the generalization bound.

Finally, note that our developed optimization strategies for meta learning are regardless of meta learning methods, and DR-MAMLs and DR-CNPs are merely two examples. For the sake of convenience, we only implement DR-MAML to compare with TR-MAML in this work.

E Proof of Remark (2)

Remark (2). If ℓ ( D T τ , D C τ ; ϑ ) is convex w.r.t. ϑ , then Eq.s (5)/(6) are also convex functions. In this case, the optimization objective Eq. (6) of our interest can be resolved with the help of convex programming algorithms (Fan et al., 2017; Meng et al., 2020; Levy et al., 2020).

Proof: We at first show that [ ℓ ( D T τ , D C τ ; ϑ ) -ξ ] + := max { ℓ ( D T τ , D C τ ; ϑ ) -ξ, 0 } is convex w.r.t ϑ if ℓ ( D T τ , D C τ ; ϑ ) is convex w.r.t. ϑ : For ease of derivation, let us redenote two functions f 1 ( ξ ; ϑ ) := ℓ ( D T τ , D C τ ; ϑ ) -ξ and f 2 ( ξ ; ϑ ) := 0 .

With any constant λ ∈ [0 , 1] and any two parameters ϑ 1 ∈ Θ and ϑ 2 ∈ Θ , we can have:

which shows that the risk function with slack variables is convex w.r.t. ϑ .

As φ α ( ξ ; ϑ ) = ξ + 1 1 -α E p ( τ ) [ [ ℓ ( D T τ , D C τ ; ϑ ) -ξ ] + ] is the convex combination of the above mentioned convex function, the resulting φ α ( ξ ; ϑ ) is naturally convex w.r.t. ϑ .

F Proof of Proposition (1)

Assumption (1): The meta risk function ℓ ( D T τ , D C τ ; ϑ ) is β τ -Lipschitz continuous w.r.t. ϑ , which suggests: there exists a positive constant β τ such that ∀{ ϑ, ϑ ′ } ∈ Θ :

Assumption (2): For meta risk function values, the risk cumulative distribution function F ℓ ( l ; ϑ ) is β ℓ -Lipschitz continuous w.r.t. l , and the implicit normalized probability density function of tasks p α ( τ ; ϑ ) is β θ -Lipschitz continuous w.r.t. ϑ .

Assumption (3): For any valid ϑ ∈ Θ and corresponding implicit normalized probability density function of tasks p α ( τ ; ϑ ) , the meta risk function value can be bounded by a positive constant L max :

Proposition (1): Under Assumptions (1)/(2)/(3), the meta learning optimization objective E α ( ϑ ) in Eq. (3) is continuous w.r.t. ϑ .

Proof: Suppose that ∀{ ϑ, ϑ ′ } ∈ Θ and || ϑ -ϑ ′ || < δ , we can have the following inequality based on Assumption (2):

Meanwhile, we can have the following inequality based on Assumption (1):

Together with the boundness Assumption (3) of meta risk function value:

we can roughly estimate the probabilistic constrained expected meta risk function values as follows:

From the above inequality, we can see that E α ( ϑ ) - E α ( ϑ ′ ) is a ( β θ L max + β max ) -Lipschitz continuous w.r.t. ϑ . As a result, we demonstrate Proposition (1).

G Proof of Proposition (2)

Proposition (2): Suppose there exists δ ∈ R + such that | ξ α ( ϑ ) -ˆ ξ α ( ϑ ) | < δ with ˆ ξ α ( ϑ ) an estimate of ξ α ( ϑ ) . Then there exists a constant κ α = max { 2 -α 1 -α , α 1 -α } such that

Proof: Based on the direct deduction in work (Rockafellar et al., 2000), we know that for any ϑ ∈ Θ , the inequality holds: φ α ( ξ α ; ϑ ) ≤ φ α ( ˆ ξ α ; ϑ ) .

In the case when ˆ ξ α = ξ α + δ with δ ∈ R + , the probability space of the task Ω τ can be respectively partitioned into three disjoint probability space:

As a result, we can estimate the difference between the two terms as follows:

Similarly, in the case when ˆ ξ α = ξ α -δ with δ ∈ R + , the probability space of the task Ω τ can be partitioned into three disjoint space with the probability respectively:

As a result, we can estimate the difference between the two terms as follows:

Based on the inequalities (16)/(18), we can have κ α = max { 2 -α 1 -α , α 1 -α } such that the proposition is verified as φ α ( ˆ ξ α ; ϑ ) -κ α δ < φ α ( ξ α ; ϑ ) .

H Proof of Improvement Guarantee in Theorem (1)

Theorem (1): Under assumptions (1)/(2)/(3), suppose that the estimate error with the crude Monte Carlo holds: | ˆ ξ α t -ξ α t | ≤ λ β ℓ (1 -α ) 2 , ∀ t ∈ N + , with the subscript t the iteration number, λ the learning rate in stochastic gradient descent, β ℓ the Lipschitz constant of the risk cumulative distribution function, and α the confidence level. Then the proposed heuristic algorithm with the crude Monte Carlo can produce at least a local optimum for distributionally robust fast adaptation.

Proof: In the main paper, Fig. (2) provides the outline of the improvement guarantee proof. Note that ˆ ξ α t is an estimate of ξ α t with the help of Monte Carlo samples, and this depends on the model parameters ϑ t in optimization. Performing the gradient updates w.r.t. the surrogate function φ α ( ˆ ξ α ; ϑ ) , we can have the following equation with a small step-size learning rate λ :

To verify the improvement guarantee, we need to show that with the meta model parameters derived from the surrogate function:

With the previous deduction φ α ( ξ α t +1 ; ϑ t +1 ) ≤ φ α ( ξ α t ; ϑ t +1 ) from the property of CVaR α , the demonstration is equivalently reduced to show that:

We can perform the one order Taylor expansion with Peano's form of remainders w.r.t. φ α ( ξ α t ; ϑ ) around the point ϑ t :

In the case when ˆ ξ α = ξ α + δ with δ ∈ R + , we use the partitioned task probability space in Eq. (15) and can derive the gradient estimate:

With | F ℓ ( ˆ ξ α ; ϑ ) -F ℓ ( ξ α ; ϑ ) | ≤ β ℓ δ in the Assumption (2),

where µ defines the sup τ ∈ Ω τ ||∇ ϑ ℓ ( D T τ , D C τ ; ϑ ) | ϑ = ϑ t || .

Then we can derive the following inequality with the deduction from Eq. (24):

where ν 1 = E τ ∼ P ++ ( τ ) [ ∇ ϑ ℓ ( D T τ , D C τ ; ϑ ) | ϑ = ϑ t ] .

To ensure the existence of improvement guarantee, we need that the following inequality holds:

Similarly, in the case when ˆ ξ α = ξ α -δ with δ ∈ R + , we use the probability space of partitioned tasks in Eq. (17) and can derive the gradient estimate:

With | F ℓ ( ˆ ξ α ; ϑ ) -F ℓ ( ξ α ; ϑ ) | ≤ β ℓ δ in the Assumption (2), the following formula can be easily verified:

Once again, we can derive the following inequality with the deduction from Eq. (28):

where ν 2 = E τ ∼ P + ( τ ) [ ∇ ϑ ℓ ( D T τ , D C τ ; ϑ ) | ϑ = ϑ t ] .

To ensure the existence of improvement guarantee, we need that the following holds:

Considering the above two cases and estimated bounds Eq. (26)/(30), we can roughly estimate the upper bound of the required δ to guarantee performance improvement using our developed strategy in optimization:

where λ is the formerly mentioned learning rate, || ν m || 2 2 is max {|| ν 1 || 2 2 , || ν 2 || 2 2 } , and µ is supermum of the meta risk function derivatives in the task domain sup τ ∈ Ω τ ||∇ ϑ ℓ ( D T τ , D C τ ; ϑ ) | ϑ = ϑ t || .

With the help of Jensen inequality, || ν i || 2 2 can be roughly bounded as:

With Eq.s (31)/(32), we can finally obtain the necessary condition for improvement guarantee:

I Proof of Approximation Error in Theorem (2)

This section is to build up connections between the number of Monte Carlo samples in estimating VaR α and the gap of solutions between the approximately derived one and the theoretical one.

Theorem 2 (Gaps of Optimized Solutions) Suppose F ℓ ( l ; ϑ ) ∈ C 2 in l -domain. With meta trained ϑ ∗ , the crude Monte Carlo estimate of ˆ ξ α , the constant κ α in Proposition (2), and the sufficiently large number of the task batch B , and R B = O ( B -3 / 4 ln B ) , we have the expected error between the exact optimum and the approximate optimum:

Proof: As noted in (Bahadur, 1966), with assumptions that the cumulative distribution function F ℓ ( l ; ϑ ) 's second order derivative is continuous in l -domain, namely F ℓ ( l ; ϑ ) ∈ C 2 , and dF ℓ ( ξ ; ϑ ) dξ | ξ = ξ α , the quantile estimate with crude Monte Carlo can be asymptotically written in the form with the help of central limit theory (Rosenblatt, 1956):

where the empirical cumulative distribution function is computed as follows.

With the sampled meta risk function values L B = { ℓ ( D T τ i , D C τ i ; ϑ ) } B i =1 , we rank them by values as ˆ L B = { ℓ ( D T ˆ τ i , D C ˆ τ i ; ϑ ) } B i =1 , which means ℓ ( D T ˆ τ i -1 , D C ˆ τ i -1 ; ϑ ) ≤ ℓ ( D T ˆ τ i , D C ˆ τ i ; ϑ ) . Then the empirical cumulative distribution function with these order statistics (Barabás, 1987) can be written as:

Based on Proposition (2) and κ α = max { 2 -α 1 -α , α 1 -α } , we know the inequality holds:

With Eq. (34)/(36), the following inequality naturally holds when B is large enough:

J Experimental Set-up & Implementation Details

This section is to provide experimental details in this paper. For the implementation of few-shot sinusoid regression and few-shot image classification, we respectively refer the reader to TR-MAML's codes ( https://github.com/lgcollins/tr-maml ) in (Collins et al., 2020) and vanilla MAML's codes ( https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch ) in (Antoniou et al., 2019). And ours is built on top of the above codes except for simple modification of loss functions. The learning rates for the inner loop and the outer loop of all methods are the same as the above ones.

To facilitate the use of our heuristic optimization strategy, we leave the pytorch version of loss functions within the expected tail risk minimization. The example is provided in the case of mean square errors after MAML's inner loop, which is simple to implement yet effective in robustifying fast adaptation , as follows:

J.1 Meta Learning Datasets & Tasks

Sinusoid Regression. MAML(Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered in this experiment. We retain the setup in task generation and partition in (Collins et al., 2020). More specifically, there exists a distribution drift between the meta-training and the meta-testing function families { f m ( x ) = a m sin( x -b m ) } M m =1 .

Numerous easy tasks and a small proportion of difficult tasks are available in meta-training, while all tasks in the space are used in the evaluation. The range of the phase parameter is b ∈ [0 , π ] , and the amplitude range of the parameter is a ∈ [0 . 1 , 1 . 05] for easy tasks and a ∈ [4 . 95 , 5 . 0] for difficult tasks. It is noted that the sinusoid task is more challenging to fit with larger amplitudes as the resulting function is less smooth. The loss function corresponds to the mean-squared error between the predicted value f ( x ) and the ground truth value. The number of task batches is 50 for 5-shot and 25 for 10-shot. The optimal selection of the confidence level α is difficult since we need to trade off the worst and average performance. Our setup is to minimize CVaR α , which already considers the worst-case at some degree, so we watch the average performance in meta training results and set α = 0 . 7 for all few-shot regression tasks. There is no external configuration for this hyper-parameter. The maximum number of iterations in meta-training is 70000.

Few-shot Image Classification. MAML(Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered in this experiment. The N-way K-shot classification corresponds to an N -classification problem with K -labeled examples available to the meta learner.

The Omniglot dataset consists of 1623 handwritten characters from 50 alphabets, with each 20 examples. The task distribution is uniform for all task instances consisting of characters from one specific alphabet. The dataset split follows procedures in (Triantafillou et al., 2019). Finally, 25 alphabets are used for meta-training, with 20 alphabets for meta-testing. The number of task batches is 16. The confidence level α = 0 . 5 is selected with the same criteria as that in few-shot regression tasks. The maximum number of iterations is 60000 in meta-training. As the construction of the Omniglot meta dataset is related to specific alphabets and the scale of combination for tasks is huge, this indicates randomly sampled meta-training tasks in the evaluation of the main paper Tables may not be used in meta-training.

The mini -ImageNet dataset is pre-processed according to (Larochelle, 2017). In detail, 64 classes are used for meta-training, with the remaining 36 classes for meta-testing. Tasks are generated as follows: 64 meta-training classes are randomly grouped into 8 meta-train tasks with the class numbers { 6 , 7 , 7 , 8 , 8 , 9 , 9 , 10 } , and the 36 meta-testing classes are processed in the same way. Finally, each task is built by sampling 1 image from 5 different classes within one task, resulting in a 5-way 1-shot problem. The number of task batches is 4. The maximum number of iterations is 60000 in meta-training.

J.2 Neural Architectures & Optimization

Sinusoid Regression. MAML(Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered in this experiment. We retain the neural architecture for regression problems in (Finn et al., 2017; Collins et al., 2020). That is, we deploy a fully-connected neural network with two hidden layers of 40 ReLU nonlinear activation units. All methods use one stochastic gradient descent step as the inner loop.

Few-shot Image Classification. We retain the neural architecture for few-shot image classification problems in (Finn et al., 2017; Collins et al., 2020). In detail, a four-layer convolutional neural network is used for both Omniglot and mini -ImageNet datasets. All methods use one stochastic gradient descent step as the inner loop.

K Additional Experimental Results

More Quantitative Analysis. Due to the page limit in the main paper, we include the α 's sensitivity experimental result in sinusoid 10-shot regression. As illustrated in Fig. (7), the trend is similar to that in sinusoid 5-shot regression. Worst-case optimization degrades the average performance of TRMAML. DR-MAML is entangled with MAML in the average performance, while the performance gap between them is significant in the worst-case.

0

.

2

0

.

4

0

.

6

0

.

7

0

.

8

Confidence Level

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

Average

MAML

TR-MAML

DR-MAML(Ours)

0

.

2

0

.

4

0

.

6

0

.

7

0

.

8

Confidence Level

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

3

.

5

Worst

MAML

TR-MAML

DR-MAML(Ours)

0

.

2

0

.

4

0

.

6

0

.

7

0

.

8

Confidence Level

0

.

4

0

.

6

0

.

8

1

.

0

1

.

2

1

.

4

1

.

6

1

.

8

2

.

0

CVaR

MAML

TR-MAML

DR-MAML(Ours)

Sinusoid 10-shot

Regarding few-shot image classification in the mini -ImageNet dataset, we can observe that in Fig. (8), the standard error is relatively smaller than in previous regression cases. When the confidence level is over a particular value, e.g. α > 0 . 5 , there occurs a significant decline of performance in all metrics. Note that when α → 1 . 0 , the optimization objective approaches the worst-case optimization objective. Here we attach two possible reasons for the performance degradation phenomenon: (i) The adopted base optimization technique matters in nearly worst-case optimization. The stochastic mirror descent-ascent (Juditsky et al., 2011) is utilized in TR-MAML, which is more stable in deriving the optimal solution. In comparison, the stochastic gradient descent with sub-gradient operations works as the optimization method, and this method can be unstable when the scale of worst-case examples is small in the update. (ii) For few-shot image classification, estimates of VaR α can be less precise with limited batch sizes and higher α values since the meta risk function value is discontinuous. Consequently, we can also attribute the severe degradation of fast adaptation performance in higher α value cases to the approximation errors of quantile estimates.

More Visualization Results. Further, we explore the influence of the expected tail risk minimization principle in meta learning. Here the landscape of meta risk values, namely fast adaptation losses, is presented in the sinusoid regression problem.

As exhibited in Fig. (9), the evaluated meta risk values from one random trial are associated with hyper-parameters of tasks. The final optimized results can discover some tasks difficult in fast adaptations. Meta learning methods are difficult to adapt in task regions with higher amplitudes. MAML exhibits higher MSEs in regions with the amplitude a > 2 , while DR-MAML minimizes a proportion of risks in these regions. In contrast, TR-MAML reduces the risk around task regions with a > 2 to a certain extent; however, it shows relatively higher risk values in easy regions. Such

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

20

25

30

35

40

45

50

Average

MAML

TR-MAML

DR-MAML(Ours)

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

20

25

30

35

40

45

50

Worst

MAML

TR-MAML

DR-MAML(Ours)

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

20

25

30

35

40

45

50

CVaR

MAML

TR-MAML

DR-MAML(Ours)

mini

-ImageNet 5-way 1-shot

X

0

1

2

3

4

Y

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

1

0

1

2

3

4

DR-MAML

X

0

1

2

3

4

Y

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

2

-

1

0

1

2

3

MAML

X

0

1

2

3

4

Y

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

1

0

1

2

TR-MAML

5

0

.

0

5

0

.

0

5

0

.

0

evidence reflects the interpretability in optimization within the expected tail risk minimization, and the landscape of meta risk values is relatively flat and smooth than others.

Experimental Results with DR-CNPs. The implementation is the same as CNP (Garnelo et al., 2018a) and official Github files ( https://github.com/deepmind/neural-processes/blob/ master/conditional_neural_process.ipynb ); the Gaussian Process curve generator works as the benchmark, and we retain the neural architecture and optimization pipelines in Github files. Please check implementation details, e.g. , neural architectures, optimizers, epochs, batch sizes, etc. , from the mentioned GitHub repository.

Also, note that this is more challenging than sinusoid as there is more randomness in generating curves. Similar to few shot image classifications, we set α = 0 . 5 in meta training DR-CNPs. In meta-testing processes, we randomly sample 64 curves per run with random context points in GitHub files and examine the performance of CNP, TR-CNP (worst-case), and DR-CNP (distributionally robust). The log-likelihoods in 5 runs are reported as follows (the higher, the better).

From Table (6), we observe that the DR-CNP retains the average performance the same as the CNP and simultaneously achieves the highest worst and CVaR α log-likelihoods. In contrast, the TR-CNP sacrifices the average performance a lot and obtains the mediate worst result. Here we attribute

the failure of achieving the best worst performance using TR-CNPs to the sensitivity of min-max optimizers as mentioned in TR-MAML.

Particularly, some randomly sampled fitted curves in difficult scenarios, e.g. , setting the number of context points as 5, are visualized in Fig. (10). Observations are consistent with Table (6), where TRCNPs exhibit over estimated uncertainty and cannot well reveal the trend of curves. In comparison, DR-CNPs can well handle challenging cases, capturing more convincing uncertainty.

Experimental Results in Meta Reinforcement Learning. Wealso performed additional experiments in 2-D point robot navigation tasks. This is a meta reinforcement learning benchmark. All setups of the point robot are the same as MAML, and please refer to pytorch-maml-rl and cavia Github for details of the navigation environments. The neural network of policies is a two-hidden layer MLP with 64 hidden units and tanh activations. The fast learning rate is 0.1, and the number of batches is 500. Other details, e.g. , optimizers, step-wise rewards, horizons and etc. , can be found in MAML. The one-step gradient is performed for fast adaptation, and the trust region policy optimization works as the policy gradient algorithm. Similar to few shot image classifications, we set α = 0 . 5 in meta training point robots. In meta-testing processes, we randomly sample 100 navigation goals as tasks and examine the performance of MAML, TR-MAML, and DR-MAML. The histogram comparison is visualized in Fig. (11).

(a)

(b)

Unlike few-shot supervised learning, worst-case optimization is pretty challenging in RL, and the result of TR-MAML is unstable and cannot achieve the goal of worst-case optimality in Table (7).

This is due to the nature of large performance deviations when deploying policies in various tasks. Seeking optimizers for stable worst-case optimization is non-trivial. Also, stably generalization across skills in reinforcement learning is indeed difficult with a few interactions. Overall, DR-MAML can well control the proportional worst case performance.

L Platforms & Computational Tools

In this research project, we use NVIDIA 1080-Ti GPUs in computation. Pytorch (Paszke et al., 2019) works as the deep learning toolkit in implementing few-shot image classification experiments. Meanwhile, Tensorflow is the deep learning toolkit for implementing sinusoid few-shot regression experiments.

M Declaration of Author Contribution

The authors confirm their contribution to the paper as follows: Q.W. (Qi Wang) and Y.Q.L. (Yiqin Lv) conceptualized the idea of tail risk minimization in meta learning and developed simple yet effective strategies with mathematical demonstrations. Y.Q.L. performed most of the experiments and Q.W. examined the performance in meta reinforcement learning benchmark. Q.W. and Y.Q.L. wrote and prepared the manuscript, while Y.H.F. (Yanghe Feng) contributed crucial insights into the organization and checked the mathematical part of the draft in submission. J.C.H. (Jincai Huang) and Z.X. (Zheng Xie) contributed to the supervision and helped proofread the manuscript. All authors reviewed the results and approved the final version of the manuscript. Meanwhile, we also thank dr. Zhengge Zhou for helpful discussion.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address an existing gap in fast adaptation of meta-learning by developing a robustification framework that draws from the concept of expected tail risk and recasts robustification as a distributional optimization problem. They adopt a heuristic two-stage optimization strategy that is broadly useful toward most meta-learning methods. The applicability of this framework toward enhancing robustness against task distributions and mitigating worst-case performance is verified and discussed over a selection of benchmark tasks.\n\nSTRENGTHS:\n+ The paper motivates the importance of expanding work in distributionally robust optimization and presents a unique formulation of the robustness problem for meta learning. \n+ Concrete illustration of how the proposed optimization method can benefit existing meta learning work.\n+ Well-organized sections and visual structure.\n+ Nice FAQ — the authors should consider making their contributions clearer in the main text of the paper.\n\nWEAKNESSES:\n- Appendix J.1 states: \"The optimal selection of the confidence level α is difficult since we need to trade off the worst and average performance. Our setup is to minimize CVaRα, which already considers the worst-case at some degree, so we watch the average performance in meta training results and set α = 0.7 for all few-shot regression tasks.\" This isn't really tail risk (e.g., alpha > 0.9, alpha < 0.05). Given the tail risk problem, why does alpha range from 0.2 to 0.8 (Figure 5)?\n\n- The paper seems incremental to TR-MAML. It would have been better to see more detail in the main text (as opposed to Appendices) on the main contribution of taking a probabilistic approach to meta learning and using a cVaR probabilistic measure (e.g., applicability, convergence, bounds). Instead, the main text focused on implementing DR-MAML which may not work for true tail scenarios. \n\n- The authors should introduce at least a new dataset versus just reusing MAML/TR-MAML. Appendix J shows that the authors simply modified the loss function, reusing code from MAML and TR-MAML. It would be nice to see additional experiments in more challenging settings, e.g., RL tasks like point navigation or continual learning.\n\n- Appendix K states: \"Note that when α → 1.0, the optimization objective approaches the worst-case optimization objective. Here we attach two possible reasons for the performance degradation phenomenon:...\" This should be expanded and included in the main text as opposed to the Appendix.\n\n- It seems the main contribution is imposing a probabilistic constraint (cVaR) versus using worst-case TR-MAML, which necessarily only works for meta-learning over a large task space.\n\nQUESTIONS:\n- What is the justification for deriving results over only 3 runs? This makes empirical claims seem suspect, e.g. the conclusion that “DR-MAML surpasses MAML in terms of average performance”\n\n- It would be nice to see additional discussion of the impacts of assumptions 1/2/3. (The authors briefly touch on this around line 294.)\n\n- Pseudocode is in Appendix B but it would be more useful in the main text.\n\n* Experimental results for the proposed approach could be further substantiated by including a discussion of Tables 4/5 in the main text.\n\n* Minor:\n\t* line 34: “plaint” -> “plain” (note: this may be ignorance on my part.)\n\t* line 199: “sometime” -> “sometimes”\n\t* line 208: “a” -> “an”\n\t* Figure 4 caption line 2: “DR-MAMAL” -> “DR-MAML”\n\t* Appendix, line 859: “Risk-Rweighted” -> “Risk-Reweighted”\n\nLIMITATIONS:\n* The authors name the primary theoretical and practical limitations of their work.\n\n* Societal impacts are not directly applicable. The authors touch upon limitations of existing meta-learning methods for high-risk real-world scenarios, which is important.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-28"}

comments: SUMMARY:
The paper proposes a novel expected tail risk objective for meta-learning, which provides an intermediate of the worst-case and average risk minimization algorithms. The proposed algorithm can be adapted to existing meta-learning algorithms with improved robustness.

STRENGTHS:
* The paper is well-written and easy to follow.
* Considering expected tail risk in meta-learning is novel to me. Controlling the worst task performance at a certain probabilistic level can improve the robustness.
* The experimental results show the effectiveness of the proposed approach.

WEAKNESSES:
* The meta risk function $\ell(D_{\tau}^T, D_{\tau}^C;\nu)$ is an empirical risk estimation that depends on the number of data points $m$ of task $\tau$. It’s preferable to consider its generalization as in [1,2,3].
* Why only adopt eight meta-training tasks and four meta-testing tasks for the experiment of mini-ImageNet in Table.3?

>[1] Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, and Massimiliano Pontil. Learning-to-learn stochastic gradient descent with biased regularization. In International Conference on Machine Learning, pages 1566–1575. PMLR, 2019a.

>[2] Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoretic analysis. Advances in Neural Information Processing Systems, 34: 25878–25890, 2021.

>[3] Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, Jason Lee, Sham Kakade, Huan Wang, and Caiming Xiong. How important is the train-validation split in meta-learning? In International
Conference on Machine Learning, pages 543–553. PMLR, 2021.

QUESTIONS:
See the discussion in the weaknesses section.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considered a robust meta-learning framework by addressing the worst-tasks performance. Specifically, this paper adopted conditional value at risk (CVR) to filter poorly performed tasks. In order to effectively learn the CVR, this paper adopted a surrogate loss, a variant of hinge loss with a learnable threshold. The proposed framework exhibits empirical improvement in certain few-shot tasks.\n\nSTRENGTHS:\nIn general, this paper tackles an interesting problem in mete-learning. I would think the following points might be interesting. \n- Addressing robustness issue in meta-learning. \n- A principled and simple method through the tool of conditional value at risk.\n- The paper is quite clearly written.\n\nWEAKNESSES:\nThis reviewer found there are several concerns in the current version. The factors that make this reviewer suggest that a major revision is better. I will discuss these points in the following aspects. \n\n\n**Originality** and **Significance**\n\nThe main motivation of the proposed method is to address the robustness in meta-learning. However, this reviewer feels a bit confused about its motivation in choosing CVR based approaches. There are alternative methods in distribution robust optimization such as group dro, which can be naturally extended by adding the task weights as a learnable parameter. Or if we adopt Wasserstein distance, we could generate new samples as a data augmentation training. \n\nBased on these, I would think the proposed method may lack sufficient motivation or comparison to justify its benefits. \n\n\n**Quality**\n\n\nI have some technical concerns. \n\n- In equation (2), the worst performing group sometimes is quite risky since it would have quite arbitrary value. It should be better to consider the robustness in the context of distributional robustness optimization.\n- In sec 4.3, I would think there are better principles to learn the threshold value. I mean, since the learnable thresholding value is convex for the loss in (6), thus we could directly adopt several solvers to solve the optimal value after each epoch. This could construct a principled method to optimize the hyper-parameter. \n- In the few-shot context, the worst-performing groups in general have quite few data, does the accuracy really make some statistical sense?\n- The representation learning based meta-method seems not empirically validated?\n\n\n**Clarity**. \n\nMost parts within the paper are clear. Some parts are still a bit confusing, for example\n- Sec 4.1 128-129 For any \\theta, the meta learning.. It should be for any **fixed** \\theta, right? \n- The surrogate seems quite similar to the hinge loss, right? I would think a further discussion is better. \n- DN-CRP is better named as meta-representation learning.\n- Fig 2 seems a bit confusing because of \\alpah_{t}, actually \\alpha here is fixed. The threshold value was changing w.r.t. t rather than \\alpha.\n\nQUESTIONS:\nIn general I would think this paper has several merits. However, this reviewer has certain concerns on the novelties and technical parts. Thus currently I vote for a weak rejection. Authors are suggested to check weakness points to know details.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors present a method by which to improve the expected tail risk during meta learning via a two-stage optimisation strategy. In addition, the authors provide theoretical and empirical analysis of their proposed approach, demonstrating improvements in the average, worst-case and conditional value at risk (CVar) metrics.\n\nSTRENGTHS:\nThe paper is written and structured very well. In particular, Sections 2,3, and 4 provide great introductions to the meta-learning paradigm and the idea of risk-minimisation applied to meta learning. The theoretical analysis outlined in Section 4 is also quite clear. Looking at the supplementary material, proofs and assumptions are outlined well, and ablations are provided.\n\nWEAKNESSES:\n- This paper is quite strong, and many of my initial concerns are answered in the supplementary material. \n- Besides minor issues with regards to wording and grammar, this paper is quite well written. For example, on lines 34-35, 'suffering from tribulation' and 'plaint'. \n-  A minor concern, but one which might help practitioners. Choosing the optimal meta-batch size appears quite important to the final performance. Although the authors discuss the influence of task batch size, perhaps ablating across more (and a wider range of) batch sizes might be more informative. \n\nFor other weaknesses/comments, see questions.\n\nQUESTIONS:\n- In the paper, the Monte Carlo estimate allows for the theoretical analysis, but it is indeed quite crude. Do the authors have any comments on perhaps improving upon this (even if the theory isn't there yet)?\n- What optimizers did the authors use? In the existing limitations section, they suggest that the batch size selection has a greater effect for first-order optimizers. Did the authors attempt to use any higher order (or approximate) optimizers?\n- I believe the theoretical analysis breaks down for the image classification tasks? (please do correct me if I'm wrong) Do the authors have any further insights on this point, in particular as to why the tail risk is not well optimised for omniglot, but works well for mini-ImageNet? \n- The authors also state that challenging meta-learning tasks can reveal more advantages of DR-MAML (line 281). Have more challenging tasks been tried? This could, for example be the meta-dataset or something equivalent? \n- In Figure 5, there appears to be deviations from the trend when $\\alpha = 0.7$, and in general, the trend appears to be somewhat non-monotonic. Is this a dataset dependant issue?\n- Do the authors have any general guidelines for how the image classification performance of DR-MAML changes with $\\alpha$?\n\nLIMITATIONS:\nThe limitations of this work are outlined in Section 6 and assumptions are outlined throughout the paper. In addition, error bars and std's are provided for all experiments. \n\nOverall, I see no potential negative societal impacts of this work. If anything, robust meta-learning has a potentially positive societal impact for deployed meta-learning systems.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper proposes a novel meta-learning pipeline from the perspective of distributionally robust perspective to robustify fast adaptation. Through a two-stage strategy heuristically, the model could control the worst fast adaptation cases at a certain probabilistic level. And experiments show the effectiveness of the proposed model compared to other baselines, especially TR-MAML.

STRENGTHS:
1. The proposed idea is interesting, and it could generalize the expected risk minimization and worst-case risk minimization for meta learning together.
2. The presentation and structure of the paper is good.
3. The experiments are extensive for the proposed model DR-MAML.

WEAKNESSES:
1. It is a difficult to follow for section 4. Some motivations or intuitions are necessary. For instance:
- what is the goal to introduce $(R^{+}, B)$ above line 129? It does not appear in the following paper.
- what is the intuition to involve the slack variable $\xi$ below line 157?

2. Some notations are not clear. For instance:
- line 134: why $M^{-1}(\ell)$ instead of $M(\ell)$?
- For Eq.(1) and (3), is it better to add "min" after the ":="? 
- line 208: "a optimization..." -> "an opt..."

3. Do you consider diverse tasks during meta-training or just follow the common setting used in MAML? Since the goal is to robustify the fast adaptation, if you just use the common setting in MAML, how could you validate whether your model is helpful to keep robustness? Or the common setting is not robust?

4. Could you add some experiments for Example 2: DR-CNP?

QUESTIONS:
see weakness.

LIMITATIONS:
yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-11