A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm
Authors: Cheems Wang
Yiqin Lv
Yanghe Feng
Zheng Xie
Jincai Huang
Venue: NeurIPS 2023 poster (2023)
Keywords: meta learning
robust fast adaptation
model agnostic meta learning

Abstract: Meta learning is a promising paradigm to enable skill transfer across tasks.
Most previous methods employ the empirical risk minimization principle in optimization.
However, the resulting worst fast adaptation to a subset of tasks can be catastrophic in risk-sensitive scenarios.
To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of tail task risk.
We take the two-stage strategy as heuristics to solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. 
Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.

A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm
Qi Wang Yiqin Lv Yanghe Feng²
Zheng Xie
Jincai Huang²t
College of Science, National University of Defense Technology
*College of Systems Engineering, National University of Defense Technology ,uanngj incai Auudd edu
{wangqi15, lvyiqin98, hrg.gggg.......gggaaga.eeee
Abstract
Meta learning is promising paradigm enable skill transfer across tasks. Most previous methods employ the empirical risk minimization principle in optimization. However, the resulting worst fast adaptation a subset of tasks can be catastrophic in risk-sensitive scenarios. To robustify fast adaptation, this paper optimizes meta learning pipelines from a distributionally robust perspective and meta trains models with the measure of expected tail risk. We take the two-stage strategy as heuristics solve the robust meta learning problem, controlling the worst fast adaptation cases at a certain probabilistic level. Experimental results show that our simple method can improve the robustness of meta learning to task distributions and reduce the conditional expectation of the worst fast adaptation risk.
Introduction
The past decade has witnessed the remarkable progress of deep learning in real-world appli- cations (LeCun al., 2015). However, train- ing deep learning models requires an enormous dataset and intensive computational power. At the same time, these pre-trained models can fre- quently encounter deployment difficulties when the dataset's s distribution drifts in testing time (Lesort al., 2021).
Freperecee
Fregurmcter
Probabably
Prodesilly
Valt
Figure 1: Illustrations of Distributionally Robust Fast Adaptation. Shown are histograms of meta risk function values f(D?" Di, {) in the task distribution p(r). Given probability a. optimize meta leamn ing model parameters to decrease the risk quantity CVaR, in Definition (3).
As a result, the paradigm of meta learning or learning to learn proposed and impacts the ma- chine learning scheme (Finn et al., 2017), which leverages past experiences enable fast adapta- tion to unseen tasks. Moreover, in the past few years, there has grown large body of meta learning methods find plausible strategies to distill common knowledge into separate tasks (Finn et al., 2017; Duan et al., 2016; Garnelo et al., 2018a).
Notably, most previous work concentrates merely on the fast adaptation strategies and employs the standard risk minimization principle, e.g. the empirical risk minimization, ignoring the difference between tasks in fast adaptation. Given the sampled batch from the task distribution, the standard meta learning methods weight tasks equally in fast adaptation. Such an implementation raises concerns in some real-world scenarios, when worst fast adaptation is catastrophic in range of risk-sensitive applications (Johannsmeier et al., 2019; Jaafra et al., 2019; Wang et al., 2023b). For example, in
'These authors contributed equally.
I 'Zhnn Xie and Jincai Huang corresponding authors.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

robotic manipulations, humanoid robots (Duan, 2017) can quickly leverage past motor primitives to walk on plain roads but might suffer from tribulation doing this on rough roads.
Research Motivations. Instead of seeking novel fast adaptation strategies, we take more interest in optimization principles for meta learning. Given the meta trained model, this paper stresses the performance difference in fast adaptation to various tasks as an indispensable consideration. As the concept of robustness in fast adaptation has not been sufficiently explored from the task distribution perspective, researching this topic has more practical significance and deserves more attention in meta learning. Naturally, we raise the question below:
Can we reconsider the meta learning paradigm through the lens of risk distribution, and are there plausible measures to enhance the fast adaptation robustness in some vulnerable scenarios?
Developed Methods. In an effort to address the above concerns and answer these questions, we reduce robust fast adaptation in meta learning to a stochastic optimization problem within the principle of minimizing the expected tail risk, e.g., conditional value-at-risk (CVaR) (Rockafellar et al., 2000). To tractably solve the problem, we adopt a two-stage heuristic strategy for optimization with the help of crude Monte Carlo methods (Kroese and Rubinstein, 2012) and give some theoretical analysis. In each optimization step, the algorithm estimates the value-at-risk (VaR) (Rockafellar et al., 2000) from meta batch of tasks and screens and optimizes a percentile of task samples vulnerable to fast adaptation. As illustrated in Fig. (I), such an operation equivalent to iteratively reshaping the task risk distribution increase robustness. The consequence of optimizing the risk function distributions 0s + dx+1 is to transport the probability mass in high-risk regions to the left side gradually. In this manner, the distribution of risk functions in the task domain can be optimized toward the anticipated direction that controls the worst-case fast adaptation at a certain probabilistic level.
Outline & Primary Contributions. We overview related meta learning and robust optimization work in Section (2). Section (3) introduces general notations and describes meta learning optimization objectives together with typical models. The distributionally robust meta learning problem is presented together with heuristic optimization strategy in Section (4). We report experimental results and analysis in Section (5), followed by conclusions and limitations in Section (6). Our primary contribution is two-fold:
1. We recast the robustification of meta learning to a distributional optimization problem. The resulting framework minimizes the conditional expectation of task risks, namely the tail risk, which unifies vanilla meta-learning and worst-case meta learning frameworks.
2. To resolve the robust meta learning problem, we adopt the heuristic two-stage strategy and demonstrate its improvement guarantee. Experimental results show the effectiveness of our method, enhancing fast adaptation robustness and mitigating the worst-case performance.
2 Literature Review
Meta Learning Methods. In practice, meta learning enables fast learning (adaptation to unseen tasks) via slow learning (meta training in a collection of tasks). There exist different families of meta learning methods. The aptimization-baser methods, such as model agnostic meta learning (MAML) (Finn al., 2017) and its variants (Finn et al., 2018; Rajeswaran et al., 2019; Grant et al., 2018; 8; Vuorio et al., 2019; Abbas et al., 2022), try to find the optimal initial parameters of models and then execute gradient updates over them to achieve adaptation with a few examples. The bonsext-based methods, e.g. conditional neural processes (CNPs) (Garnelo et al., 2018a). neural processes (NPs) (Garnelo ct al., 2018b) and extensions (Gordon ct al., 2019; Foong et al., 2020; Wang and Van Hoof, 2020; Gondal et al., 2021: Wang and van Hoof, 2022; Shen et al., 2021; Wang et al., 2023a), learn the representation of tasks in the function space and formulate meta learning models as exchangeable stochastic processes. The metrics- based methods (Snell et al., 2017; Allen et al., 2019; Bartunov and Vetrov, 2018) embed tasks in a metric space and can achieve competitive performance in few-shot classification tasks. Other methods like memory-augmentt 2021, 2022), recurrent models (Duan et al., 2016) and hyper networks (Zhao et al., 2020; Beck et al., 2023) are also modified for meta learning purposes.
models (Santoro et al., 2016; Xiao et al.,
Besides, there exist several important works investigating the generalization capability of methods. Bai et al. (2021) conduct the theoretical analysis of the train-validatio split and connect it to

---

optimality. In (Chen et al., 2021), a generalization bound is constructed for MAML through the lens of information theory. Denevi al. (2019) study an average risk bound and estimate the bias for improving stochastic gradient optimization in meta learning.
Robust Optimization. When performing robust optimization for downstream tasks in deep learning, we can find massive work concerning the adversarial input noise (Goodfellow et al., 2018; Goel et al., 2020; Ren al., 2021), or the perturbation on the model parameters (Goodfellow et al., 2014; Kurakin et al., 2016; Liu et al., 2018; 8; Silva and Najafirad, 2020). In contrast, this paper studies the robustness terms of robust principles, the commonly considered one is
of fast adaptation in meta learning. the worst-case optimization (Olds, 2015; Zhang et al., 2020; Tay et al., 2022). For example. Collins al. (2020) conducts the worst-case optimization MAML obtain the robust meta initialization. Considering the influence of adversarial examples, Goldblum et al. (2019) propose to ad adversarially meta train the model for few-shot image classification. Wang et (2020) adopt the worst-case optimization in MAML to increase the model robustness by injecting adversarial noise to the input. However, distributionally robust optimization (Rahimian and Mehrotra, 2019) rarely examined in the presence of the meta learning task distribution.
3 Preliminaries
Notations. Consider the distribution of tasks p(r) for meta learning and denote the task space by Q,- Let be task sampled from p(T) with the set of all tasks. We denote the meta dataset by D, For example, in few-shot regression problems, D, refers to a set of data points { y: to fit.
Generally, D, are processed into the context set D for fast adaptation, and the target DJ' for evaluating adaptation performance. As an instance, we process the dataset D, D U D' with fixed partition in MAMI. (Finn al., 2017). D. and D are respectively used for the inner loop and the outer loop in model optimization.
Definition (Meta Risk Function) With the task E and the pre-processed dataset D, and the model parameter i E O, the meta risk function is map É: D, x + Rt
In meta learning, the meta risk function ((DT. :.: ;), e.g. instantiations in Example (1)(2), is to evaluate the model performance after fast adaptation. Now we turn to the commonly-used risk minimization principle, which plays crucial role in fast adaptation. To summarize, we include the vanilla and worst-case optimization objectives as follows.
Expected Risk Minimization for Meta Learning. The objective that applies most vanilla meta learning methods can be formulated in Eq. (1), and the optimization executes in distribution over tasks p(T). The Monte Carlo estimate corresponds the empirical risk minimization principle.
min E(0) Ep(r) [IDF) #E0
Here is the parameter of meta learning models, which includes parameters for common knowledge shared across all tasks and for fast adaptation. Furthermore, the task distribution heavily influences the direction of optimization in meta training.
Worst-case Risk Minimization for Meta Learning. This also considers meta learning in the task distribution p(r), but the worst case fast adaptation the top priority in optimization.
min max WEB TET (a
The optimization objective is built upon the min-max framework, advancing the robustness of meta learning to the worst case. Approaches like TR-MAML (Collins et al., 2020) sample the worst task in batch to meta train with gradient updates. Nevertheless, this setup might result in a highly conservative solution where the worst case only happens with an incredibly lower chance.

---

Distributionally Robust Fast Adaptation
This section starts with the concept of risk measures and the derived meta learning optimization objective. Then a heuristic strategy is designed to approximately solve the problem. Finally, we provide two examples of distributionally robust meta learning methods.
4.1 Meta Risk Functions Random Variables
Assumption The meta risk function E(D ; ;0) is 3, B,pLisschitz continuous w.t. 0, which suggests: there exists a positive constant 3, such that V(U, 0' G:
E()))) Df 0')l B-110 d'|l.
Let Fr, denote probability measure over the task space, where F, corresponds o-algebra on the subsets of Ω,, And we have (R+, .B) a probability measure over the non-negative real domain for the previously mentioned meta risk function I(D ; with B Borel o-algebra. For any & E O, the meta learning operator Ma 22 + R is defined as:
M T (DT.D.
T, 0) can be viewed as random variable to induce the distribution p([(D) D())). Further, the cumulative distribution function be formulated as Fe(4; 1) P((((D) 0) S 4 E Q2 RR))) w.r.. the task space. Note that Fe(1; 0) implicitly depends on the model parameter 0, and cannot access closed-form in practice.
In this way, 1(DT,
Definition 2 (Value-at-Risk Given the confidence level [0, 1], the task distribution p(r) and the model parameter 0, the VaR (Rockafellar et al., 2000) of the meta risk function is defined as:
VaR, (e(T, innf (11F((1))) 2 aTTT.T.
IER
Definition (Conditional Value-at-Risk) Given the confidence level [0, 1), the task distribu- tion p(T) and the model parameter 0, we focus on the constrained domain of the random variable E(D ;;;0) 0) with I(D ; Z VaR, [L(T, [)]. The conditional expectation of this is termed as conditional value-at-risk (Rockafellar et al., 2000):
CVaR, [M(T, 0]] = |0- (IF
where the normalized cumulative distribution is as follows:
0,
1 << VaR ((T, ((:))))aa VaR [[([
(I;
1-a
This results in the normalized probability measure (o..r.Fa.h Pa,r) over the task space, where Pa,r UE2 [M (4).. For case of presentation, denote the corresponding task distribution constrained in Do,r by (T)
Rather than optimizing VaR.., a quantile, meta learning, we take more interest CVaR,, optimiza- tion, type of the expected tail risk. Such risk measure regards the conditional expectation and has more desirable properties for meta learning: more adequate in handling adaptation risks in extreme tails, more accessible sensitivity analysis w.r.f. o,, and efficient optimization.
Remark 1 CVaR, [E(T, 8)] to minimize is respectively equivalent with the vanilla meta learning optimization objective in Eq. (1) when a = and the worst-case meta learning optimization objective in Eq. (3) when is sufficiently close I.
4.2 Meta Learning via Controlling the Expected Tail Risk
As mentioned in Remark (1), the previous two meta learning objectives can be viewed as special cases within the CVaR, principle. Furthermore, we turn to a particular distributionally robust fast

---

adaptation with the adjustable confidence level a to control the expected tail risk in optimization as follows.
Distributionally Robust Meta Learning Objective. With the previously introduced normalized probability density function (T; 0), minimizing CVaR, [E(T, 0)] can be rewritten Eq. (3).
min &, :- Ep. (T:0) (F: tg
(3)
Even though CVaR,, 0) is function of the model parameter i, the integral in Eq. (3) is intractable due the involvement (T; i) non-closed form.
Assumption For meta risk function values, the cumulative distribution function Fell; 0) is Be- Lipschitz continuous w.r.t. I, and the implicit normalized probability density function of tasks po (T; 9) is Be-Lipschitz continuous w.r.l. 0.
Assumption 3 For any valid i θ and corresponding implicit normalized probability density function of tasks Pa (T; the meta risk function value can be bounded by a positive constant Lmaxi
DT: S Cmax.
sup
TEQ...
Proposition 1 Under assumptions (1)(2)(3)) the meta learning optimization objective & (1) Eq. (3) is continuous w.r.t 0.
Further, we &o (1) to denote the VaR., 0)] for simple notations. The same as that in (Rockafellar et al., 2000), we introduce a slack variable & E R and the auxiliary risk function [e(D], ;)) E analytical Po (2); strained one after optimizing g:
max Di; 0) &, Oj. To circumvent directly optimizing the non- convert the probability constrained function E,, (1) to the below uncon-
(E: = -T--B Bp(r) [I(D] 00 - -s'
demonstrated that & (0) minŞeR Pa (5: i)) and &. E arg aiggingeR Po (E; () in (Rockafellar et al., 2000), and also note that CVaR, is the upper bound of &a, implying
Ea S (aas 0) S (E:0), Vε R and vD E e.
With the deductions from Eq.s (3)(4), we can resort the distributionally robust meta learning optimization objective with the probability constraint into a unconstrained optimization objective as (5).
+ 1 En(r) [10FF20;00 =1+]] -
min #EERER
Sample Average Approximation. For the stochastic programming problem above, it is mostly intractable derive the analytical form of the integral. Hence, need to perform Monte Carlo estimates of Eq. (5) to obtain Eq. (6) for optimization.
min + dEBRER
U1D D.:0)
(6)
(1- B
Remark 2 E :C:0) () is convex w.r.f. ô, then Eq.s (5)(6) are also convex functions. In this case, optimization objective Eq. (6) of our interest can be resolved with the help of several convex programming algorithms (Fan al., 2017; Meng et al., 2020; Levy et al., 2020).
5

---

Eox
Eova
Ecsua
Eap
Vara(((T, Estimation
Theorem (Improvement Guarantee) Under assumptions (1)322/(3), suppose that the estimate error with the crude Monte Carlo holds: 16... -...l Bπiー-|= Vt N+, with the subscript t the iteration number, X the learning rate stochastic gradient descent, Be the Lipschitz constant of (6 o))
(l ())
( (8)
((leagi +)
Surrogute Function Optimization
Osia
far-a
(p (o-o, 2 (/mm. 2 lamee 2 (om f Some
Improvement
Guarantee
Figure 2: Optimization Diagram of Distributionally Robust Meta Learning with Surrogate Functions. From left to right: the meta model parameters in the middle block are optimized w.st. the constructed surrogate lunction p(... marked in blue in (-th iteration. Under certain conditions Theorem (I), the distributionally robust meta learning objective ((Ea; i) marked pink can decreased monotonically until reaches convergence in the H-th itcration.
4.3 Heuristic Algorithms for Optimization
Unfortunately, most existing meta learning models' risk functions (Finn al., 2017; Gamelo et al., 2018a; Santoro et al., 2016; Li et al., 2017; Duan et al., 2016), {(D), 24; 0) are non-convex w.r.f. 1, bringing difficulties in optimization
Eq.s (5)(6).
To this end, we propose a simple yet effective optimization strategy, where the sampled task batch is used approximate the VaR, and the integral in Eq. (5) for deriving Eq. (6). In detail, two stages involved in iterations: (i) approximate VaR [P(,, 8]] se Éa with the meta batch values, which can be achieved via either a quantile estimator (Dong and Nakayama, 2018) or other density estimators; (ii) optimize 0 in Eq. (6) via slochastic updates after replacing &a by the estimated &..
Proposition 2 Suppose there exists E R such that ((a É. (v)) < 8 with (a) an estimate (0)). Then there exists constant max
1-a ?
1-a such that
(a(E 0) - K.d < (v) S
The performance gap resulting from VaR,, approximation error is estimated Proposition (2). For ease of implementation, we adopt crude Monte Carlo methods (Kroese and Rubinstein, 2012) to obtain a consistent estimator &x..
the risk cumulative distribution function, and a the confidence level. Then the proposed heuristic algorithm with the crude Monte Carlo can produce at least local optimum for distributionally robust fast adaptation.
Note that Theorem (I) indicates that under certain conditions, using the above heuristic algorithm has the performance improvement guarantee, which corresponds Fig. (2). The error resulting from the approximate algorithm is further estimated Appendix Theorem (2).
4.4 Instantiations & Implementations
Our proposed optimization strategy applies to all meta learning methods and has the improvement guarantee in Theorem (1). Here we take two representative meta learning methods, MAML (Finn et al., 2017) and CNP (Garnelo et al., 2018a), as examples and show how to robustify them through the lens of risk distributions. Note that the forms of É(D DG; ;f;00 0) sometimes differ in these methods. Also, the detailed implementation of the strategy relies specific meta learning algorithms and models.
Example (DR-MAML) With the task distribution p(T) and model agnostic meta learning (Finn et al., 2017), the distributionally robust MAML treats the meta learning problem bi-level

---

optinization with VaRa relevant constraint.
5 + 1 Py(r) [I(D]; - (V)(( -VVe((0)))(( -s]*]
min
dEG
EER
The gradient operation Val(Di; 0) corresponds to the inner loop with the learning rate X.
The resulting distributionallyy robust MAML (DR-MAML) method, where a fixed per- centage of tasks are screened for the outer loop. shown in Fig. (3) and Eq. (7), the objective is to obtain robust meta initialization of the model parameter 1.
Meta Learning
is still optimization-
Fast Adaptation
V4
0
V4
0g
Example 2 (DR-CNP) With the task distribution p(T) and the conditional neural process (Garnelo et al., 2018a), the distributionally robust conditional neural process learns the functional representations with a CVaRa con- straint.
44
Vta
0z
01
Figure 3: Diagram of Distributionally Ro- bust Fast Adaptation for Model Agnostic Meta Learning (Finn et al., 2017), For ex- ample, with the size of the meta batch and a 10%, 5 (1 - a) tasks in gray with the worst fast adaptation performance are screened for updating meta initialization. Here ho, is set encoder network with 0z the parameter of the decoder network.
1 1 Bp(r) [e(D 2. 22)) - =
min E + NCE SER
(8)
s.1. 2. ho, (2+) with 02)
The resulting distributionally robust CNP (DR-CNP) to find a robust functional embedding to induce underlying stochastic processes. Still, in Eq. (8), proportion of tasks with the worst functional representation performance are used in meta training.
Moreover, we convey the pipelines of optimizing these developed distributionally robust models in Appendix Algorithms (1).
5 Experimental Results and Analysis
This section presents experimental results and examines fast adaptation performance in a distributional sense. Without loss of generality, we take DR-MAML in Example (1) run experiments.
Benchmarks. The same as work in (Collins et al., 2020), we use two commonly-seen downstream tasks for meta learning experiments: few-shot regression and image classification. Besides, ablation studies are included to assess other factors' influence or the proposed strategy's 's scalability.
Baselines & Evaluations. Since the primary investigation is regarding risk minimization principles, we consider the previously mentioned expected risk minimization, worst-case minimization, and expected tail risk minimization for meta learning. Hence, MAML (empirical risk), TR-MAMI. (worst-case risk), and DR-MAML (expected tail risk) serve as examined methods. We evaluate these methods' performance based on the Average, Worst-case, and CVaR, metrics. For the confidence level to meta train DR-MAML, we empirically set a = 0.7 for few-shot regression tasks and a 0.5 image classification tasks without external configurations.
5.1 Sinusoid Regression
Following (Finn et al., 2017; Collins et al., 2020), conduct experiments in sinusoid regression tasks. The mission is approxi- mate the function f(r) sin(( - b) with K-shot randomly sampled function points, where the task is de-
Table 1: Test average mean square errors (MSEs) with reported standard deviations for sinusoid regression (5 runs). We respectively consider 5- 5-shot and 10-ahot with 0.7. The results evaluated across the 490 meta- test tasks, (Collins al., 2020). The best results are bold.
5-ehot Worst
10-shot Worst
Method
Average
CVaR.
Average
CVaR,
MAML (Finn et al. 2017)
1.02aam
3.89
2.25anu5
0.664416
2.574am 1.15
TR-MAML (Collins et 2020)
1.09
2.28
1.794000
0.77+0.11
1.68:943
1.27
DR-MAML (Ours)
0.89.am
2.9
1.765082
0.54:am
1.703617 0.96:am
fined by and b. In the sine function family, the target range, amplitude range, and phase range
7

---

respectively [-5.0, C R, a E [0.1, and b E [0, 2m.. In the setup of meta training and testing datasets, a distributional shift exists: numerous easy tasks and several difficult tasks are generated to formulate the training dataset with all tasks in the space as the testing one. Please refer to Appendix (I) for a detailed partition of meta-training, testing tasks, and neural architectures.
Result Analysis. We list meta-testing MSEs in sinusoid regression in Table (I). As expected. the tail risk minimization principle in DR-MAML can lead to an intermediate performance in the worst- case. In both cases, the comparison between MAML and DR-MAML in MSEs indicates that such bobbaailistic-cnsttaannn
optimization the task space even has the potential to advance average fast adaptation performance. In contrast, TR-MAML has to sacrifice more average performance for the worst-case improvement.
basta Metaa-Int Trodks Sinusoid Sobbot
Misti Metoclbet Thaks Strmaod Ilsbot
Mean Squared (MSE)
Mean Squared Froor (MSF)
Moan Squared (MSE)
Moon Squared (MSR)
Figure 4: Histograms Aleta-Testing Performance in Simusoid Regression Problems. With a = 0.7, we respectively visualize the comparison results, DR- MAMML-s--AMN and TR-MAME ME----M----MMALLE in 5-shot (Two Sub-figures Left Side) and 10-shot (Two Sub-figures Right Side) cases, for a sample trial.
More intuitively, Fig. (4) illustrates MSE statistics the testing task distribution and further verifies the effect of the CVaR, principle in decreasing the proportional worst-case errors. In 5-shot cases, the difference in MSE statistics more significant: the worst-case method tends increase the skewness in risk distributions with many task risk values gathered in regions of intermediate performance, which unfavored in general cases. As for why DR-MAML surpasses MAML in terms of average performance, we attribute it to the benefits of external robustness in several scenarios, e.g.. the drift of training/testing task distributions.
Table 2: Average N N-way K-shot classification accuracies in Omniglot with reported standard deviations (3 runs). With a = 0.5, the best results are in bold.
Meta: Training Alphabets
Meta: Mestinesting Alphabets
5-way 1 shot
20-way 1-ebos
5-way 1-shot
20-2ay
1-2hot
Method
Avenge Wonst
CVaR,
Average
Worst
CViR, |
Avemgg
Wonst
CVaR,
Axerage
Worst
CVaR,
MAML (Finn et al., 2017)
98.4:02 82.4am
96,9sss
99.2sa:
33,9aa0
80,9 93.5am2
82,5ana
91.6336
67.6e2a
49.7x33
60.4117
RRMM (Collims 2020)
97.4506
92
8244z)
93.1-11
85.3-19
91.3
74.614
584
68.5-12
DR DR-MAML (Ours)
17.s
84.0ama
45.s
99.62ns
57.921
84.8
93.7:84
84.120
92.1
74.6
51.0223
4644
5.2 Few-Shot Image Classification
Here we do investigations in few- shot image classification. Each task is an N-way K-shot clas- sification with N the number of classes and K the number of la- beled examples in one class. The Omniglot (Lake et al., 2015) and (Vinyals et al., 2016) datasets work as benchmarks for examination. We retain the setup of datasets in work (Collins et al., 2020).
Table 3: Average 5-way 1-shot classification accuracies in mini- TmtgeNet with reported standard deviations (3 runs). With 0.5, the best results are bold.
Eight Meta: Metainraining Tasks
Four Meta-Testing Testing Tasks CVaR,
Method
Average
Wonst
CVaR.
Average
MAML (Finn et 2017)
70.1
48.0.45
63.2aza
44.7a07 44.6207 48.5106
46.6104
TR-MAMI. (Collins al., 2020)
63.2_13 60.7216
12
45.9148 46.6204
DR MAML. (Ours)
70.2 42 63.4 m2 67.2 49.4:a1
47.1. 47.5.
Result Analysis. The classification accuracies in Omniglot are illustrated in Table (2): In 5-way 1-shot cases, DR-MAML obtains the best average and CVaR, in meta-testing datasets, while TR- MAML achieves the best worst-case performance with a slight degradation of average performance compared to MAML in both training/testing datasets. In 20 way 1-shot cases, for training/testing datasets, we surprisingly notice that the expected tail risk is not well optimized with DR-MAML, but there is an average performance gain; while TR-MAML works best in worst-case/CVaR. metrics.
When it comes to mini-ImageNNt, findings are distinguished a from the above one: in Table (3), DR-MAML yields the best result in all evaluation metrics and cases, even regarding the worst-case. MR-MAML can also improve all metrics in meta-testing cases. Overall, challenging meta-learning tasks can reveal more advantages of DR-MAML over others.
8

---

5.3 Ablation Studies
This part mainly checks the influence of the confidence level at meta batch size, and other optimization strategies towards the distribution of fast adapted risk values. Apart from examining these factors of interest, additional experiments are also conducted in this paper, please refer to Appendix (K).
Sinusoid S-shoe
MAME
MARE.
Avarugo
Woree
CVan
Confidence Level
Confidence Level
Confidence Level
Figure 5: Meta Testing MSEs of Vota-Trained DR-MAML with Various Confidence Levels a. MAML and TR-MAML are irrelevant with the variation of in meta-training. The plots report testing MSEs with standard error bars in shadow regions.
Sensitivity to Confidence Levels a. To deepen understanding of the confidence level a' 's effect in fast adaptation performance, we vary a to train DR-MAML and evaluate models under previously mentioned metrics. Taking the sinusoid s-shot regression as an example, we calculate MSEs and visualize the fluctuations with additional o-values in Fig. (5). The results in the worst-case exhibit higher deviations. The trend in Average/CVaR, metrics shows that with increasing a, DR-MAML gradually approaches TR- MRM in average and CV.RR, MSEs. With S 0.8, ours is less sensitive to the confidence level and mostly beats MAMI/TR-MAMA in average performance. Though ours aims at optimizing CVaR it cannot always ensure such a metric to surpass TR-MAML in all confidence levels due to rigorous assumptions in Theorem (1).
Sinusoid
Simwood
mmini-TmagpiNet Seway
Moss Squaedd (MSSE)
Moan Squared Eene (MS8))
Clowiiitaation Arcurmciiv
Bulch
Sime
Batch Size
Betch Size
Figure 6: Meta Testing Average Performance of Meta-Trained DR-MAML with Various Sizes of the Meta Batch. The plots report average results with standard error bars in shadow regions.
Influence of the Task Batch Size. Note that our optimization strategy relies on the estimate of VaR., and the improvement guarantee relates to the estimation error. Theoretically, the meta batch in training can directly influence the optimization result. Here we vary the meta batch size in training. and evaluated results are visualized in Fig. (6). In regression scenarios, the average MSEs can decrease to a certain level with an increase of meta batch, and then performance degrades. We attribute the performance gain with batch increase to more accurate VaR,, estimates; however, meta batch larger than some threshold can worsen the first-order meta learning algorithms' efficiency, similarly observed in (Nichol et al., 2018). As for classification scenarios, there appears no clear trend since the meta batch is smaller enough.
Comparison with Other Optimization Strategies. Note that instantiations of distributionally robust meta learning methods, such as DR-MAML and DR-CNPs in Examples (1 ((2 are regardless of optimization strategies and can be optimized via any heuristic algorithms for CVaR,, objectives.
Additionally,
MR-M as the example and perform the comparison between two wo-stage algorithm and the risk reweighted algorithm (Sagawa et al., 2020). The intuition of the risk reweighted algorithm is to relax the weights of tasks and assign more weights to the gradient of worst cases. The normalization of risk weights is achieved via the softmax operator. Though there is an improvement guarantee w.s. the probabilistic worst group of tasks, the algorithm is not specially designed for meta learning or CVaR, objective.
(P ((2) (DF:Df:O)) min (0) := Ep((:o) p(T; #
1 B
(9)

---

Table 4: Test average mean square (MSEs) with reported standard deviations for sinusoid regression
runs). We mainly compare DR-MAML with different optimization algorithms. 5-shot and 10-shot cases are respectively considered here. The results are evaluated across the 490 meta-test tasks, which is the same as in (Collins et al., 2020). With a = 0.7 for meta training. the best testing results are in bold.
5-shot Worst
10-shot
Method
Average
CVaR..
Average
Worst
CVaaa
DR RAML (Group DRO, (Sagawa et al., 2020)
0.006 10.06
3.55
1.83
0.61
1.90
1.132002
DR- DR-M (Two-Slage, Ours)
0.8910.M
2.91 10.6
1.76.00.2
0.54
1.70
0.961081
Table
Average 5-way 1-shot classification accuracies in mini- Image with reported standard devia- tions runs). We mainly compare DR-MAML with different optimization algorithms. With meta training, the best testing results are in bold.
for
Eight Meta- Meta-nraining Tasks
Four Meta-Testing Tasks
Method
Average
Worst
CVaR,
Average
Worst
CVaR,
DR-MAML (Group DRO, (Sagawa et al., 2020)) DR-MAMI (Two-Stage. Ours)
67. 17.0=a2
56,6=04
61.6=02
49.1
46,6*01
47.2
70.2
63.4 0.2
67.2.0.
49.410.1
47.1
47.5
Meanwhile, the weight of task gradients after normalization is a biased estimator w.r.t the constrained probability (T( 0) in the task space. In other words, the risk reweighted method can be viewed as approximation w.r.f the importance weighted method in Eq. (9). In the importance weighted method, for tasks out of the region of (1 a) propporrti worst, the probability of sampling such tasks T,, is zero, indicating wh (T); 0) 0. While in risk reweighted methods, the approximate weight
:d) where i means last time updated meta model parameters and the risk function value is evaluated after fast adaptation.
assumed to satisfy wb (Ta; 0) x exp
In implementations, we keep the setup the same as Group DRO methods in (Sagawa et al., 2020) for meta-training. As illustrated in Table (4)/(5) DR-MAML with the two-stage optimization strategies consistently outperform that with the Group DRO ones in both 5-ohot and 10-shot sinusoid cases regarding all metrics. The performance advantage of using the two-stage ones is not significant in mini- ImageNet scenarios. We can hypothesize that the estimate of VaR, in continuous task domains, e.g. sinusoid regression, is more accurate, and this probabilistically ensures the improvement guarantee with two-stage strategies. Both the VaR, estimate two-stage strategies and the importance weight estimate in the Group DRO ones may have lot of biases in few-shot image classification, which lead to comparable performance.
Conclusion and Limitations
Technical Discussions. This work contributes more insights into robustifying fast adaptation in meta learning. Our utilized expected tail risk trades off the expected risk minimization and worst-case risk minimization, and the two-stage strategy works the heuristic to approximately solve the problem with an improvement guarantee. Our strategy can empirically alleviate the worst-case fast adaptation and sometimes even improve average performance.
Existing Limitations. Though our robustification strategy is simple yet effective in implementations, empirical selection of the optimal meta batch size is challenging, especially for first-order optimization methods. Meanwhile, the theoretical amalysis only applies to a fraction of meta learning tasks when risk function values in compact continuous domain.
Designing heuristic algorithm with an improvement guarantee is non-trivial and relies on the properties of risk functions. This research direction has practical meaning in the era of large models and deserves more investigations in terms of optimization methods. Also, establishing connections between the optimal meta batch size and specific stochastic optimization algorithms can be a promising theoretical research issue in this domain.
Future Extensions.
Acknowledgments and Disclosure of Funding
This work is funded by National Natural Science Foundation of China (NSFC) with the Number # 62306326.
10

---

References
Abbas, M., Xiao, Q. Chen, L.,, Chen, P.-.,, and Chen, T. (2022). Sharp-maml: Sharpness-aware model-agnostic meta learning. In International Conference Machine Learning, pages 10-32. PMLR.
Allen, K., Shelhamer, E.,, Shin, H., and Tenenbaum, J. (2019). Infinite mixture prototypes for few-shot learning. International Conference on Machine Learning, pages 232-241. PMLR.
Antoniou, A., Edwards, H., and Storkey, A. (2019). How to train your maml. In Seventh International Conference Learning Representations.
Bahadur, R. R. (1966). A note on quantiles in large samples. The Annals of Mathematical Statistics, 37(3):577-580.
Bai, Y., Chen, M., Zhou, P., Zhao, T., Lee, J., Kakade, S., Wang, H., and Xiong, C. (2021). How important is the train-validation split in meta-learning? In International Conference on Machine Learning, pages 543-553. PMLR.
Barabás, B. (1987). Estimation of density functions by order statistics. Periodica Mathematica Hungarica, 18(2):115-122.
Bartunov, S. and Vetrov, D. (2018). Few-shot generative modelling with generative matching networks, In International Conference on Artificial Intelligence and Statistics, pages 670-678. PMLR.
Beck, J., Jackson, M. T., Vuorio, R., and Whiteson, S. (2023). Hypernetworks in meta-reinforcementt learning. In Conference Robot Learning, pages 1478-1487. PMLR.
Chen, Q. Shui, C., and Marchand, M. (2021). Generalization bounds for meta-learning: An thformition--hheortiiic analysis. Advances in Neural Information Processing Systems, 34:25878- 25890.
Chen, R. S., Lucier, B., Singer, Y., and Syrgkanis, V. (2017). Robust optimization for non-convex objectives. Advances in Neural Information Processing Systems, 30.
Collins, L., Mokhtari, A., and Shakkottai, (2020). Task-robust model model-agnostic meta-learning. Advances in Neural Information Processing Systems, 33: 18800-181..71
Denevi, G., Ciliberto, C., Grazzi, R., and Pontil, M. (2019). Learning-to-lest stochastic gradient descent with biased regularization. In International Conference on Machine Learning, pages 1566-1575 PMLR.
Dong, H. and Nakayama, M. (2018). A tutorial on quantile estimation via monte carlo. In International Conference Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, pages 3-30. Springer.
Duan, Y. (2017). Meta learning for control. University of California, Berkeley.
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P. (2016). RI2: Fast reinforcement learning via slow reinforcement learning. arXiv preprint orXiv:1611.02777
Duchi, J. C. and Namkoong, H. (2021). Learning models with uniform performance via distribution- ally robust optimization. The Annals of Statistics, 4778): 1406.
Fallah, A., Mokhtari, A., and Ozdaglar, (2021). Generalization of aodes-agnostic meta-learning algorithms: Recurring and unseen tasks. Advances in Neural Information Processing Systems, 34:5469-5480.
Fan, Y., Lyu, S., Ying, Y., and Hu, B. (2017). Learning with average top-k loss. Advances in neural information processing systems, 30.
Finn, C., Abbeel, P., and Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep
networks. In International conference on machine learning, pages 1126-1135. PMLR.

---

Finn, C., Xu, K., and Levine, S. (2018). Probabilistic model-agnostic meta-learning Advances in neural information processing systems, 31.
Foong, A., Bruinsma, W., Gordon, J., Dubois, Y., Requeima, J., and Turner, R. (2020). Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems, 33:8284-822 8295.
Gagne, C. and Dayan, P. (2021). Two steps risk sensitivity. Advances in Neural Information Processing Systems, 34:22209-22220.
Garnelo, M.. Rosenbaum, D., Maddison, C., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., Rezende, D., and Eslami, S. A. (2018a). Conditional neural processes. International Conference Machine Learning, pages 1 1704-17 1713. PMLR.
Garnelo, M., Schwarz, J., Rosenbaum, D., Viola, F., Rezende, D. J., Eslami, S., and Teh, Y. W. (2018b). Neural processes. arXiv preprint (tf7:/807.0122
Goel, A., Agarwal, A., Vatsa, M., Singh, R., and Ratha, N. (2020). Dndnet: Reconfiguring enn for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 22-23.
Goldblum, M., Fowl, L., and Goldstein, T. (2019). Robust few-shot learning with adversarially queried meta-learners.
Gondal, M. W., Joshi, S., Rahaman, N., Bauer, S., Wuthrich, M., and Scholkopf, B. (2021). Function contrastive learning of transferable reprrssteestn,,.nnn, In Proceedings of the 38th International Conference on Machine Learming, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 3755-3765 PMLR.
Goodfellow, I., McDaniel, P., and Papernot, N. (2018). Making machine learning robust against adversarial inputs. Communications of the ACM, 61(7):56-66.
Goodfellow, J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1412.657..
Gordon, J., Bronskill, J., Bauer, M., Nowozin, S., and Turner, R. (2019). Meta-leamnng probabilistic inference for prediction. In International Conference on Learning Representations.
Grant, E., Finn, C., Levine, S., Darrell, T., and Griffiths, (2018). Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:
Hiraoka, T., Imagawa, T., Mori, T., Onishi, T., and Tsuruoka, Y. (2019). Learning robust options by conditional value at risk optimization. Advances in Neural Information Processing Systems, 32.
Hsieh, Y.PP., Mertikopoulos, P., and Cevher, V. (2021). The limits of min-max optimization algorithms: Convergence to spurious non non-critical sets. International Conference on Machine Learning, pages 4337-4348. PMLR.
Jaafra, Y., Laurent, L., Deruyver, A., and Naceur, M. (2019). Robust reinforcement learning for autonomous driving.
Jiang, Y., Li. C., Dai, W., Zou, and Xiong, (2021). Monotonic robust policy optimization with model discrepancy. In International Conference on Machine Learning, pages 4951-4960. PMLR.
Johannsmeier, I.., Gerchow, M., and Haddadin, S. (2019). framework for robot manipulation: Skill formalism, meta learning and adaptive control. In 2019 International Conference on Robotics and Automation (ICRA), pages 5844-5850. IEEE.
Juditsky, A., Nemirovski, A., and Tauvel, C. (2011). Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1):17-58.
Kroese, D. and Rubinstein, R. Y. (2012). Monte carlo methods. Wiley Interdisciplinary Reviews: Computational Statistics, 4(1):48-58.
12

---

Kurakin, A., Goodfellow, I., and Bengio, S. (2016). Adversarial machine learning at scale. arXiv preprint arXiv:1611.01255
Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. (2015), Human-level concept learning through probabiliittic program induction. Science, 33066266):1332-13333
Larochelle, S. (2017). Optimization as model for few-shot learning. In International Conference on Learning Representations.
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. nature, 521(7553):436 444.
Lesort, T., Caccia, M., and Rish, L. (2021). Understanding continual learning settings with data distribution drift analysis. arXiv preprint arXiv:2104.01678.
Levy, D., Carmon, Y., Duchi, J. C., and Sidford, A. (2020). Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems, 33:8847-8860.
Li, Z., Zhou, F., Chen, F., and Li, H. (2017). Meta-sgd: Learning to learn quickly for few-shot 1707.0983.
learning. arXiv preprint
Liu, Q., Liu, T., Liu, Z., Wang. Y., Jin, Y., and Wen, W. (2018). Security analysis and enhancement of model compressed deep learning systems under adversarial attacks. In 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC), pages 721-726. IEEE.
Meng, S. Y., Charisopoulos, V., and Gower, R. M. (2020). stochastic prox-linear method for evar minimization.
Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. (2009). Stochastic approximation approach to stochastic programming. SIAM J. Optim. Citeseer.
Nichol, A., Achiam, J., and Schulman, J. (2018). On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999
Olds, K. C. (2015). Global indices for kinematic and force transmission performance in parallel robots. IEEE Transactions on Robotics, 31(2):494-500
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, 7., Gimelshein, N., Antiga, I.,, Desmaison, A., Köpf, A., Yang. E. Z., DeVito, Z.,, Raison, M., Tejani, A., Chil- amkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019). Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 8024-8035.
Pinto, L., Davidson, J., Sukthankar, R., and Gupta, A. (2017). Robust adversarial reinforcement learning. International Conference on Machine Learning, pages 2817-2826. PMLR.
Puchert, P., Hermosilla, P., Ritschel, T., and Ropinski. T. (2021). Data-driven deep density estimation.
Neural Computing and Applications, 33:16773-1680)
Quaranta, A. G. and Zaffaroni, A. (2008). Robust optimization of conditional value at risk and portfolio selection. Journal of Banking & Finance, 22400):0066
Rahimian, H. and Mehrotra, S. (2019). Distributionally robust optimization: A review.
Rajeswaran, A., Finn, C., Kakade, S. M., and Levine, S. (2019). Meta-learning with implicit gradients. Advances in neural information processing systems, 32.
Ren, J., Zhang, D.,, Wang, Y.. Chen, L., Zhou. Z., Chen, Y., Cheng, X., Wang, X., Zhou, M., Shi, J., et al. (2021). Towards a unified game-theoretic view of adversarial perturbations and robustness. Advances in Neural Information Processing Systems, 34:3797-3810
Rockafellar, R. T., Uryasev, S., et al. (2000). Optimization of conditional value-at-risk. Journal of risk. 2:21 42.
13

---

Rosenblatt, M. (1956). A central limit theorem and a strong mixing condition. Proceedings of the national Academy of Sciences, 42(1):43-47.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang. P. (2020). Distributionally robust neural networks. In International Conference on Learning Representations.
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap. T. (2016). Meta-learning with memory-augmenteed neural networks. In International conference on machine learning, pages 1842-1850. PMLR.
Shalev-Shwartz, S. and Wexler, Y. (2016). Minimizing the maximal loss: How and why. In International Conference on Machine Learning, pages 793-801. PMLR.
Shen, J., Zhen, X., Worring, M., and Shao, L. (2021). Variational multi-task learning with gumbel- softmax priors. Advances in Neural Information Processing Systems, 34:21031-21042
Silva, S. H. and Najafirad, P. (2020). Opportunities and challenges in deep learning adversarial robustness: A survey. arXiv preprint arXiv:2007.00753.
Snell, J., Swersky, K., and Zemel, R. (2017). Prototypical networks for few-shot learning. Advances in neural information processing systems, 30.
Tay, S. S., Foo, C. S., Daisuke, U., Leong, R., and Low, B. K. H. (2022). Efficient distributionally robust bayesian optimization with worst-case sensitivity. In International Conference on Machine Learning, pages 21 18--21204. PMLR.
Triantafillou, E., Zhu, T., Dumoulin, V., Lamblin, P., Evei, U., Xu, K., Goroshin, R., Gelada, C., Swersky, K., and Manzagol, P. A. (2019), Meta-dataset: A dataset of datasets for learning to learn from few examples.
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et (2016). Matching networks for one shot learning. Advances in neural information processing systems, 29.
Vuorio, R., Sun, S.-H., Hu, H., and Lim, J. (2019). Multimodal model-agnostic meta-learning via task-aware modulation. Advances in neural information processing systems, 32.
Wang, Q., Federici, M., and van Hoof, H. (2023a). Bridge the inference gaps of neural processes via
expectation maximization. In The Eleventh International Conference on Learning Representations.
Wang, Q. Feng, Y. Huang, J., Lv. Y., Xie, Z., and Gao, X. (2023b). Large-scale generative simulation artificial intelligence: The next hotspot. The Innovation, page 100516.
Wang, Q. and Van Hoof, H. (2020). Doubly stochastic variational inference for neural processes with hierarchical latent variables. In International Conference Machine Learning, pages 10018-10028. PMLR.
Wang, Q. and van Hoof, H. (2022). Learning expressive meta-represenntations with mixture of expert neural processes. In Advances in neural information processing systems.
Wang, R., Xu, K., Liu, S., Chen, P.-., Weng, T.-W., Gan, C., and Wang, M. (2020). On fast adversarial robustness adaptation in model-agnostic meta-learning. International Conference on Learning Representations.
Wang, Z., Shen, Y., and Zavlanos, M. (2022). Risk-averse no-regret learning online convex games. In International Conference on Machine Learning, pages 22999-23011 PMLR.
Wilder, B. (2018). Risk-sensitive submodular optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Xiao, Z., Shen, J., Zhen, X., Shao, I.., and Snoek, C. (2021). bit more bayesian: Domain-invariant learning with uncertainty. In International Conference on Machine Learning, pages 11351-11361. PMLR.
Xiao, 7.,, Zhen, X., Liao, S., and Snoek, C. G. (2022). Energy-based test sample adaptation for domain generalization. In The Eleventh International Conference Learning Representations.
14

---

Zhang, J., Cheung, B., Finn, C., Levine, S., and Jayaraman, D. (2020). Cautious adaptation for reinforcement learning in safety-crition settings. In International Conference Machine Learning. pages 1055-11065. PMLR.
Zhao, D., von Oswald, J., Kobayashi, S., Sacramento, J., and Grewe, B. F. (2020). Meta-learning via hypemetworks.
15

---

Contents
Introduction
2
Literature Review
Preliminaries
Distributionally Robust Fast Adaptation
4.1
Meta Risk Functions as Random Variables
4.2
Meta Learning via Controlling the Expected Tail Risk
4.3
Heuristic Algorithms for Optimization
4.4
Instantiations & Implementations.
5
Experimental Results and Analysis
5.1
Sinusoid Regression....
5.2
Few-Shot Image Classification
5.3 Ablation Studies
Conclusion and Limitations
10
Frequently Asked Question
18
Pseudo Algorithms of DR-MAML & DR-CNPs
19
C Properties of Risk Minimization Principles
20
C.I Stochastic Optimization in the Constrained Form
20
C.2 SGD Intractability of Meta Learning CVaR,, Optimization Objective
20
C.3 Risk-Sensitive Applications Optimization Strategies
20
D Computational Complexity
21
E Proof of Remark (2)
21
F Proof of Propasition (1)
22
G Proof of Proposition (2)
23
Proof of Improvement Guarantee in Theorem (1)
24
I Proof of Approximation Error in Theorem (2)
27
Experimental Set-up & Implementation Details
27
J.1 Meta Learning Datasets Tasks
28
J.2 Neural Architectures & Optimization
29
16

---

K Additional Experimental Results
29
Platforms & Computational Tools
32
M Declaration of Author Contribution
32
17

---

A Frequently Asked Question
Here we collected technical questions and suggestions from researchers who helped check out the manuscript. We thank these researchers for precious questions and provide more details.
Novelty & primary findings of this work. Here we mainly summarize two points of novelty in this work:
Meta Learning Robustification Framework. Though the concept of the expected tail risk has emerged for several decades and has been widely employed in financial domains, the application to fast adaptation or robustification of meta learning remains limited in literature far as we know.
Optimization Strategy & Theoretical Analysis. We theoretically analyze two-stage opti- mization strategies as the heuristic algorithm in optimizing the distributionally robust meta learning models and demonstrate the improvement guarantee under certain conditions.
As verified in experimental results in the main paper, placing probabilistic constraint in the task space is meaningful. It circumvents the effect of over-pessimistic consideration (worst-case optimization), increases robustness in proportional cases, and mostly retains or even improves average performance.
Apart from the novelty in the framework and algorithm parts, we have several findings which bring crucial insights into meta learning: (i) Not all tasks are necessary to perform fast adaptation. (ii) Additional focus on the tail risk has the potential to enhance models' generalization capability. (iii) The tail risk instead of extreme worst-case risk can better advance robustness in challenging datasets.
Meta risk function values as random variables. In some few-shot learning related work, the context and the target dataset are equivalently called the support and query datasets. The definition of a task in meta learning is up to application scenarios and specific meta learning algorithms or models. The commonly-used sinusoid regression using model-agnostic meta learning (Finn et al., 2017) considers the fixed number of context points to induce tasks. In contrast, conditional neural processes (Garnelo et al., 201 8a) for few-shot regression vary the number of context points to induce tasks. Once the context and the target are partitioned and the model parameter is specified, we can obtain a meta risk function value. However, the meta risk function values are not in a compact Euclidean subspace in several cases.
The continuity of the meta risk probability density function. This relates to the task distribution and meta learning problems. Throughout the few-shot regression task, the meta risk function value can be approximately viewed as a continuous random variable. However, when it comes to the few-shot image classification mission, the meta risk function value is impossible to cover the entire continuous interval. In these scenarios, the probable values of accuracies are finite. This makes the theoretical analysis, e.g. Theorem (1), no longer holds. For example, Assumption (2) will be unrealistic when there cxists constant gap between two accuracy values. Hence, we leave this part future research direction in thcoretical analysis. Regarding the meta risk function, ours shares the same setup as that in (Fallah et al., 2021). Admittedly, precisely modeling risk distributions can be crucial, and more techniques, e.g., deep kernel estimators (Puchert et al., 2021) can be adopted to improve this point.
The selection of baselines in robust meta learning. large body of prior work considers the robustness of meta learning in scenarios when the input of a data point, the model parameter, and the number of context points are trembled or modified. Robustness in presence of tasks distributions is seldom investigated except for the worst-case optimization in (Collins et al., 2020). Hence, we retain most of the setups in (Collins et al., 2020) for a fair comparison.
Influence of risk minimization principles. This paper is primarily devoted to studying the influence of the risk minimization principle on meta learning. The empirical risk minimization principle corresponds to reducing the Monte Carlo estimate of Average-case Meta Learning in this work. In comparison to TR-MAML (Collins al., 2020), our approach has a couple of advantages as follows: (i) easier implementations. Note that min-max optimization is numerically unstable and requires a relaxation method for computationally intensive convex optimization, e.g., robust stochastic mirror-prox algorithm used in TR-MAMI (Collins et al., 2020). (ii) more flexible in terms of robustness concept. Theoretically, the worst-case meta learning corresponds to the extreme case of the distributionally robust risk minimization principle. (ii) empirically better performance in
18

---

most cases. The expected tail risk minimization preserves a particular property that minimizing proportional worst-case fast adaptation seldom sacrifices the average performance. These advantages are also why we call our approach simple yet effective.
Comparison with Other Heuristic Optimization Strategies. To sum up the proposed optimization strategy, we empirically highlight the following points regarding the adopted heuristic strategies. There exist couple of approximate algorithms for CVaR, optimization. In comparison, the crude Monte Carlo one is the simplest for VaR,, estimates. It has an improvement guarantee under certain conditions, leaving it easier analyze. Besides, we also compare ours to the risk reweighted method (Sagawa et al., 2020) in previously used benchmarks, and please take closer look at that in Section (K).
B Pseudo Algorithms of DR-MAML & DR-CNPs
Algorithm 1: DR-MAML
Input :Task distribution P(T); Confidence level a; Task batch size B; Learning rates: x and
X2-
Output :Meta-trained model parameter d. Randomly initialize the model parameter 0 while not converged do Sample a batch of tasks {T:),= ~ p(r);
// inner loop via gradient descent for i to do
Evaluate the gradient: Vol(D); 0) in Eq. (7);
Perform task-specific gradient updates:
0. - V (
end
// estimate VaR., [C(T, 0)] R &.r
Evaluate performance LB ;
Estimate VaR, [((T, and set & = Ea in Eq. (7) with either percentile rank or density
estimators;
// outer loop via gradient descent
Screen the subset Lg ,,) with & for meta initialization updates;
- 8 No Σ i=1 ; 0.) in Eq. (7);
end
Algorithm 2: DR-CNP
Input Task distribution p(T); Confidence level a; Task batch size B; Learning rate A. Output:MMtaa-eee Meta-trained model parameter 9.
Randomly initialize the model parameter 0; while not converged do Sample batch of tasks {T,] [=1 ~ P(T);
// estimate [aR,[!((T,))
Evaluate performance LB = (1(D)) 22)
Estimate VaR, (e(T,))) ~ E. with either percentile rank or density estimators;
// execute gradient descent
Screen the subset Lg { 0)}K with &. for meta initialization updates;
- 9 - AV. Σ ( ()) in Eq. (8);
end
19

---

C Properties of Risk Minimization Principles
C.1 Stochastic Optimization in the Constrained Form
In the section above, meta learning optimization objectives are discussed within three different principles, respectively the average-case in Eq. (I), the worst-case in Eq. (2) and CVaR, worst-case in Eq. (5). This subsection continues this topic and introduces the relaxation variable & in the optimization objective. In this way, the robust fast adaptation can be reframed in the case of stochastic optimization with the probabilistic constraint.
We can equivalently express the minimization of the worst-case problem "Shalev-Shwards and Wexler, 2016) within the following constrained stochastic optimization framework as follows.
min S 0EA,EER+
(10)
s.t. ((D,, 2.:0) S
with E T
C.2 SGD Intractability of Meta Learning CVaR, Optimization Objective
This subsection shows that directly stochastic gradient descent is intractable for meta learning to optimize CV Note that the normalized density distribution function pa (T; 0) implicitly depends on and a, so we cannot access the exact form of such a distribution.
VoE (0) / pa((: ()(T:))(((())))))) [(D"" 2:00 Vo 0) Vol(D of dr
(1la)
1(2 Te D. 24, 0) () [V
Dr.
(11b)
K
Score Function
As illustrated in Eq. (11), the stochastic gradient estimate is not plausible since Pa(T:: 0) has no closed form. Hence, heuristic or convex programming algorithms for specific cases are mostly used as this domain's optimization strategy. However, designing optimization strategies for non-convex cases is non-trivial in this domain and requires more consideration of theoretical guarantees.
C.3 Risk-Sensitive Applications & Optimization Strategies
Related Applications. There are number of applications concerning robust optimization with probabilistic constraints. Most of them are for the sake of safety. The risk principle CVaR,, firstly occurs in the financial domain as coherent principle (Rockafellar al., 2000). It enjoys much popularity in portfolio optimization (Quaranta and Zaffaroni, 2008). Gagne and Dayan (2021) adopt CVaR, principles to improve the distributional reinforcement learning performance. To robustify robotic control, Pinto et al. (2017) varies hyper-parameterr of Markov decision processes and optimizes proportional worst-case trajectories in policy optimization within the principle of CVaR,, In work (Wilder, 2018), a CVaR, related strategy is devised to solve submodular optimization problems. Such risk measure is also included in producing robust options (Hiraoka al., 2019). Regarding probabilistic robust meta learning within CVaR principles, there exists scarce related work until now.
Optimization Strategies. Concerning the constrained stochastic optimization problem, we partic- ularly overview related work in this subsection. In the past few decades, there emerge substantial optimization strategies together with theoretical analysis for convex risk functions in CVaR, op- timization (Nemirovski al., 2009; Fan et al., 2017; Meng et al., 2020; Levy et al., 2020; Duchi and Namkoong, 2021; Wang et al., 2022). As for the min-max risk minimization principle, which focuses on the worst case instead of propositional worst cases, some researchers have designed relaxation or other heuristic algorithms to handle convex or non-convex risk functions (Chen et al., 2017; Collins al., 2020; Jiang et al., 2021; Hsieh et al., 2021). Nevertheless, it remains challenging to design algorithms with convergence guarantees for non-convex risk function cases. One of the latest CVaR, work on non-convex risk functions is (Sagawa et al., 2020), where a risk reweighted algorithm for robust neural networks is proposed to handle distributional shifts. Moreover, most CVaR, optimization in non-convex risk functions follows this type of risk reweighted strategies in applications.
20

---

Remarks on Literature Work: Risk functions for robust fast adaptation cases mostly non-co and it is non-trivial design optimization strategies with a convergence guarantee. Meanwhile, we also conduct comparison experiments between work in (Sagawa et al., 2020) and ours in meta learning downstream tasks. We refer the reader to Appendix (K) for more details and analysis.
D Computational Complexity
The primary difference between distributionally robust meta learning and expected risk based meta learning lies in that fixed probabilistic portion of task gradients are considered in meta updates. Such an operation drives the optimization procedure to focus more on proportional vulnerable scenarios, increasing robustness in worst cases.
However, the iteration of the surrogate function p((a,; i)) involves the screening of proportional worst cases since our strategies require extra evaluation of fast adaptation. This brings additional computational cost with complexity O log (B)). the other hand, the proposed strategy performs sub-gradient updates instead of complete gradient updates. This helps reduce computational cost with complexity O ((BB)M)) in each iteration, where [M] corresponds to the scale of parameters of meta learning models and a is the confidence level in CVaR, optimization.
(B)) where B is the number of tasks in each batch. On
Universally analyzing the computational complexity in meta learning is intractable since various meta learning methods exist. Some gradient-based ones, while some non-parametrie ones. The exact number of iterations for convergence heavily relies on specific methods. In summary, given the same number of iterations and fixed confidence level, the computational complexity difference
for CVaRa optimization in meta learning scenarios is O [Baa]M) log(
To deepen understanding of our method, we explain more via specific examples. The main idea is execute the sub-gradient operation over the batch of task gradients. Here we take the DR-MAML in Example (1) show the operation and how the distributionally robust meta initialization is obtained:
B
gmeta <-
gmeet
Volō(T,) ((DT. 17
(12)
with 07 J.meeta X2
0), P(T),
where A₁ is the outer loop learning rate, Ag is the inner loop learning rate, and 8(T,) is the indicator variable. Here 0(T;) in the case when the ((D), ( falls into the (1 a)) probabiiiistic worst-case region otherwise 8(T,) 0.
As for the optimal rate for convergence or the generalization bound, is up to specific meta learning methods and risk minimization principles. For worst-case risk minimization for meta learning, there already exists theoretical analysis in previous work, e.g., optimization- based meta learning (Collins et al., 2020) when fast adaptation functions hold the convexity property. When it comes to more universal cases, considering diverse meta learning methods and optimization strategies, is still challenging estimate the optimal rate for convergence. Also, considered scenarios exist distributional drift between meta training and meta testing task distributions, which makes it tough to derive the generalization bound.
Finally, note that our developed optimization strategies for meta learning are regardless of meta learning methods, and DR-MAMLS and DR-CNPs are merely two examples. For the sake of convenience, we only implement DR-MAML to compare with TR-MAML in this work.
E Proof of Remark (2)
Remark (2). If E(DT, D6:0) is convex w.r.f. O, then Eq.8 (5)(6) are also convex functions. In this case, the optimization objective Eq. (6) of our interest can be resolved with the help of convex programming algorithms (Fan al., 2017; Meng et al., 2020; Levy et al., 2020).
21

---

Proof: We at first show that [e(2; ; -I = m :4:0) {, is convex w.rl 0 if ((DT DC: () is convex w.r.t d: For ease of derivation, let us redenote two functions (i;
, ;;;0) and f2(E; () := 0.
With any constant X E [0, 1] and any two parameters di E θ and d2 E O we can have:
(1; + 2)02) =====]=== = Si(EAA0 + (1 2)122 [for some E [1,2]]
(13a)
_ (f: 01) (1 ():
(13b)
s (max + - -)
(13c)
$ [ - + x) [<(?] Df: 02)) - E (13d)
which shows that the risk function with slack variables is convex w.E.f. 0.
";; 29;0) - -] =]]] is the convex combination of the above As Pa (E: 0) E Ber) [[<[), mentioned convex function, the resulting (5: is naturally convex w.r.t 0.
F Proof of Proposition (1)
Assumption (I): The meta risk function E(DT, D; () is 3,-Lipschitz continuous w.e.t 0, which suggests: there exists a positive constant 3r such that V(0, 0') O:
- : = B+ - 0'|].
Assumption (2): For meta risk,funciie function values, the risk cumulative distribution function Fe(l; 0) is Be-Lipschitz continuous w.r.t. and the implicit normalized probability density function of tasks (T( is 39 Bo-Lipschitz continuous w.r.t. 0.
Assumption (3): For any valid 0 E θ and corresponding implicit normalized probability density function of tasks Po (T: f), the meta risk function value can be bounded by a positive constant Lmaxi
sup ((D), ((:)) S Lmax.
rEtp
Proposition (1): Under Assumptions (1)/22/(33, the meta learning optimization objective & (0) in Eq. (3) continuous w.r.. 9.
Proof:
Suppose that ViO, 0') θ and ||0 0 < 8, we have the following inequality based on Assumption (2):
Bol 0'11-
Meanwhile, have the following inequality based Assumption (1):
-( 8 1-
Together with the boundness Assumption (3) of meta risk function value:
sup 1(D2 Tr: DC 0) S Lmax;
TEQu
22

---

we can roughly estimate the probabilistic constrained expected meta risk function values as follows:
(d) - (
(T:0) [10), 29;0 Epa(T: [(0) 2,;
29;
(14a)
= ppp ((0);, 0)] - (T:01) (1D), 2-; ;0)
(14b)
+I +Rp 2;; 0) - 1(2), 2 00)
(14c)
$ pa 24: 2.:
+ (T:00) 1(D, ((D), :f:
(14d)
S S o'll sup {[(D) D:; 8)} + sup TEf2?
H1o e'
TEm2
(14c)
= Bo Lmax + 9max 110 8'11
(14f)
From the above inequality, we can see that (.) - & (00) is (BoLmax -mmax) ) continuous w.t. 0. As result, we demonstrate Proposition (1).
G Proof of Proposition (2)
Proposition (2): Suppose there exists 8 E R+ such that IEa (1)) (0)l of (0)) Then there exists constant Ka max (==a 1-a such that
with Ea (1) an estimate
Pa (E.(())): 0) Kad (v) S (0))
Proof: Based on the direct deduction in work (Rockafellar et al., 2000), we know that for any & O the inequality holds: (fa 1) < (a(Ea i)).
In the case when & = &. 8 with å E R+, the probability space of the task 22 can be respectively partitioned into three disjoint probability space:
P (T = (MM Sa, ( 1+ (+) = (1M5' (016a < (.DT S + 12r1
(15a)
(15b)
p++(7) P (IMM 2F 04:0) Ea 5,7 E (
(15c)
As result, we can estimate the difference between the two terms as follows:
Pa Pa.E: 0) Pa 1) = -c.
(16a)
+ E E)-r+(r)
((D) 2 2(;0) - Ea. + - E Erwe++(7) (T) Ea - Ea
(16b)
< + E, By-p+(r) (8)
2 a
E E
5 +
8 =
1 8.
(16c)
1-a
23

---

Similarly, in the case when Ea Ea with E Rt, the probability space of the task S2, can be partitioned into three disjoint space with the probability respectively:
(T) ( - -,TE
P ( (T) = P
1,.-6,7-
(17a)
- -. <0, E {Lr}
(17b)
(*( = =P00= 2i:0) z La, T E
(17c)
As a result, we can estimate the difference between the two terms as follows:
Ga (f: 1))
Sa (Sa; 1 = Ea - Ea
(18a)
E p (+) ((D) D-:0) - Eo/
1
E (xwpt(T)
(T)
La
Ea
(18b)
< 5-8+ 1 ETP (r) 0 + + -.-BBapttp Brapt(r)
1
x -
8
8.
(18c)
Based on the inequalities (16)/(18), we can have Kir max verified as Pa (Ea; kad < (S)
2-0 I-a' I-a
such that the proposition is
H Proof of Improvement Guarantee in Theorem (1)
Theorem (I): Under assumptions (1V(2M(3), suppose that the estimate error with the crude Monte Carlo holds: IEa, Saz Bel1-ofa: Vt N+, with the subscript t the iteration number, X the learning rate in stochastic gradient descent, Be the Lipschitz constant of the risk cumulative distribution function, and a the confidence level. Then the proposed heuristic algorithm with the crude Monte Carlo can produce at least local optimum for distributionally robust fast adaptation.
Proof: the main paper, Fig. (2) provides the outline of the improvement guarantee proof. Note that So,, an estimate of Eo, with the help Monte Carlo samples, and this depends on the model parameters 0g in optimization. Performing the gradient updates w.r.f. the surrogate function (a 19), we can have the following equation with a small step-size learning rate A:
Gradient Descent +1+1 = 01 - (Vo
(19)
Monotonic Sequence: "aa 0++1) S ( 0z).
To verify the improvement guarantee, we need to show that with the meta model parameters derived from the surrogate function:
Pa(Eamni I Pa(Ea, With the previous deduction Pa (Sa,11 1(+1) S Pa (Sa 1241) from the property of CVaR the
(20)
demonstration is equivalently reduced to show that:
(Em, 0e+1) (E.r;
(21)
We can perform the one order Taylor expansion with Peano's form of remainders w.nd sa (Eo.; around the point dy:
Pa (Sa 0(41) = Pa (Sa - >[Vora(Ea,;
( +0(10-11 - 0el) S (a,, 0t).
(22)
In the case when &a Ea + 8 with 8 Rt. use the partitioned task probability space in Eq. (15)
and can derive the gradient estimate:
1
V Vopa(Ea, 0)
E E (r)
V of
a
(23)
B
24

---

With (Ft(Ea; 0) Fe(Ea: 10) & 3e in the Assumption (2),
En 1,;0)1== d)lo=0) 1r ( D 001=01
(24a)
< (ENPt() Vol(D,, Di:
|1 11- Vol(DT,2, 1) 100=0
(24b)
S EENPI(() sup TES,
|| ; |ل1== (E.Pi
sup Leen,
|[V 24; ;
(24c)
$ 3 a) sup ||v,((D) D;: 0))==,,111) 3,5(1 a)p
(re
(24d)
where j defines the supren, || ( Then we can derive the following inequality with the deduction from Eq. (24):
0)
Paltan: 0c+1) -aalE 02) (25a)
-> (e.::
Vospo(En.. 0) 10000 + +0110++1 0zl) (25b)
Pa (La, U2) (25c)
T EmEri Vo((D, 0) 12
E EypPl (r)
(25d)
(1 - x)2
Vo( D.; 0)1==0 1"
BINP (r)
-ELptt()) VoL(D) D.: 0)1 (25e)
(1 ax2
+0(10++1 - 0.)) (25f)
2|I2111
(va(Enii (1 - a1) + 3,5(1 a)₁² + 0(10et1 - 0.l), (25g)
where v1 ET-p++()) [ol( Df 1)
To ensure the existence of improvement guarantee, we need that the following inequality holds:
11|2|112
Pa(Sa,; U.) (1 - a1= + 8,8(1-000 a)p² + O(18++ 0el) s ca(Ea.i U.).
(26)
Similarly, in the case when E. Ea with R+, we use the probability space of partitioned tasks in Eq. (17) and can derive the gradient estimate:
1
VoralEE, :0)
I- ETWP (7) Vol(D), D. 0))0=o,
(27)
+ +ETWEt(7) Vol(DT, D.. 01
25

---

With (e((E::) 0) - Fe(Sa: & 3y in the Assumption (2), the following formula can be easily verified:
EE (+) VollD,, 2;: 1):0) _" ( Vol(D,, D. d
(28a)
$ [[ Vol(D" (f; -1 ( Vol(D" 1a
(28b)
S R (1) sup FEN,
|| D. (.: ( FEQ,
sup
|V 0)|o=
(28c)
3,00(1 aa) sup of; 00: (0==0l)) BB5 az)
ren,
(28d)
Once again, we can derive the following inequality with the deduction from Eq. (28):
(aa l141)= = (Ea, 0,) > (VoppaEaa 0)._. Vasa(Ea, [o0=o,) 0110++1
1e()
(29a)
0z)
(29b)
(1 - (1-a)2 BrwD+(T) Vol(D) D4 (f: ]" EELDI(T Vol(D] 00:
X
(29c)
(1 a)2 [BaNT-(0) ( Vo((D",D :.000 T
-Ewpt(e) Vol(D? ( :.:0)0==0)
(29d)
+0(10++1
311)
(29c)
21|1211 SpaaEo,;
(1 a)2 3:5 a)2² 0(10++1 0el))
(29f)
where 1/2 (r) Vol(DT 24:0)
To ensure the existence of improvement guarantee, we need that the following holds:
A 1|1221 a)² + 3,5(1 a)μ² + O(10++ - -rl) S & (Err 0z).
(,,.;
(30)
Considering the above two cases and estimated bounds Eq. (26)/33)) we can roughly estimate the upper bound of the required 5 to guarantee performance improvement using developed strategy in optimization:
X AlIMml8
(31)
Br(1 12
where À is the formerly mentioned learning rate, ||mml is max 11, || [2], and is supermum of the meta risk function derivatives in the task domain supren. [ ? D. 01o==,
With the help of Jensen inequality, || can be roughly bounded as:
|| < ET-p+(r)
[Vol(D] D.: 0oo="1""
2 | Di 0)100000111 = (1- a)pp
(32)
< <(1-0) sup
ten,
26

---

With Eq.s (31)/(32), we can finally obtain the necessary condition for improvement guarantee:
X 8 8 Be(1 ((1-a)) a)2
(33)
I Proof of Approximation Error in Theorem (2)
This section is to build up connections between the number of Monte Carlo samples in estimating VaRa and the gap solutions between the approximately derived one and the theoretical one.
Theorem (Gaps of Optimized Solutions) Suppose Fe(1; 0) E C² in I-domain. With meta trained Os, the crude Monte Carlo estimate of &a. the constant Kax in Proposition (2), and the sufficiently large number of the task batch B, and Rg = = 3/4 1//+In/ B), we have the expected error between the exact optimum and the approximate optimum:
a PelÉai B,0.) (E:)) dg
(a() S (Sai 0.) + Ria
+ +RB
E=
Proof: As noted in (Bahadur, 1966), with assumptions that the cumulative distribution function Fe(1; 0)'s 's second order derivative is continuous I-domain, namely Fe(1: 9) E C?, and dF2((;0)) the quantile estimate with crude Monte Carlo can be asymptotically written in the form with the help of central limit theory (Rosenblatt, 1956):
de I==c.-
a FelE.;: B. + RB, with RB O(B-0(BBB30 3/4-3//40 Bn when + xo, (34)
E. - Ea
(E:00 $=
de
where the empirical cumulative distribution function is computed as follows.
With the sampled meta risk function values Lg = (((D" TO DG, 0)) we rank them by values as EB (L(D) Di, )) which means {(D) f:-1" DC 1 ; S ((D) f. ;0: Then the empirical cumulative distribution function with these order statistics (Barabás, 1987) be written as:
0, k
i ((D) ;0) B if (2 D ;0)
Fe(E:
<ESE Ph..:0) (k = 1.2,..., B)
(35)
1,
if (f De
Fg² 2.:00) <.
Based on Proposition (2) and Ka = max 2-a a }. we know the inequality holds: I-a 1-a Pa (f); 0) - (1) < K.
(36)
With Eq. (34)(36), the following inequality naturally holds when B is large enough:
a Pe( B, 0-) (F:())))) d5
Pa (E( 0.) s (U.) + Ka
+ RB
E=
Experimental Set-up & Implementation Details
This section is to provide experimental details in this paper. For the implementation of few-shot sinusoid regression and sew-shot image classification, we respectively refer the reader to TR-MAML's codes
com/lgco woggolllnsstrrrooooooo weiiinsstrraaaee mam1) in (Collins al., 2020) and vanilla MAML's codes (https: //ggttubt.t/tt/////// com/ /tt///aaennttniouuuuuu http:///ooTTTTTiooooooAAMMMMMA
in i (Antoniou et al., 2019). And ours is built on top of the above codes except for simple modification of loss functions. The learning rates for the inner loop and the outer loop all methods are the same as the above ones.
To facilitate the use our heuristic optimization strategy, we leave the pytorch version of loss functions within the expected tail risk minimization. The example is provided in the case of mean square emrors after MAML's inner loop, which is simple to implement yet effective in robustifying fast adaptation, follows:
27

---

import torch
from torch import NSELoss
def cvar_mse (y.nmsee(_pppeee ,ated,y-truee y-true,
(reducttion none!) batch_loss= (y pped y_
average risk values over non non-task dimensions batch_avg_loss=tonn mean (batch_ loss dim=-1)
10
12
crude Monte Carlo to estimate VaR and sub tasks topk_mes mse topk_ idxs=torchh.h.c...oooooo www.orcc..oppoooeeeo avg. int ((1- conf (( y-true size size () [0]))
13
15
return torch mean (topk_me )
Listing I: Loss Functions Two-Stage Heuristic Algorithm with Crude Monte Carlo
J.1 Meta Learning Datasets & Tasks
Sinusoid Regression, MAML (Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML considered in this experiment. We retain the setup in task generation and partition in (Collins et al., 2020). More specifically, there exists a distribution drift between the meta-tres veta-training and the meta-testing function families (fm(x) am sin(x b,. bm))
Numerous easy tasks and a small proportion of difficult tasks are available meta-training, while all tasks in the space are used in the evaluation. The range of the phase parameter is b E [0, m], and the amplitude range of the parameter is a E [0.1, 1.05] for easy tasks and a E [4.95, 5.0 for difficult tasks. It is noted that the sinusoid task is more challenging to fit with larger amplitudes the resulting function is less smooth. The loss function corresponds to the mean-squared error between the predicted value f(x) and the ground truth value. The number of task batches is 50 for 5-shot and 25 for 10-shot. The optimal selection of the confidence level a is difficult since we need to trade off the worst and average performance. Our setup is to minimize CVaRa. which already considers the worst-case at some degree, so we watch the average performance in meta training results and set 0.7 for all few-shot regression tasks. There is external configuration for this hyper-parameter. The maximum number of iterations in meta-training is 70000.
Few-shot Image Classification. MAML (Finn ct al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered this experiment. The N-way K-shot classification corresponds to an R-classification problem with K-labeled examples available to the meta learner.
The Omniglot dataset consists of 1623 handwritten characters from 50 alphabets, with each 20 examples. The task distribution is uniform for all task instances consisting of characters from one specific alphabet. The dataset split follows procedures in (Triantafillou et al., 2019). Finally, 25 alphabets are used for meta-training, with 20 alphabets for meta-testing. The number of task batches The confidence level a 0.5 selected with the same criteria as that in few-shot regression tasks. The maximum number of iterations is 60000 in meta-training. As the construction of the Omniglot meta dataset is related to specific alphabets and the scale of combination for tasks is huge, this indicates randomly sampled meta-training tasks in the evaluation of the main paper Tables may not be used in meta-training.
The mini- ImageNe dataset is pre-processed according to (Larochelle, 2017). In detail, 64 classes are used for meta-tras training. with the remaining 36 classes for meta-testing. Tasks are generated as follows: 64 meta- meta-traininn training classes are randomly grouped into 8 meta-train tasks with the class numbers (6. 7, 7, 8, 8, 9, 9, 10), and the 36 meta-testing classes are processed the same way. Finally, each task is built by sampling image from different classes within one task, resulting in 5-way 1-shot problem. The number of task batches The maximum number of iterations is 60000 in mnta-training.
28

---

J.2 Neural Architectures & Optimization
Sinusoid Regression. MAML (Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered in this experiment. We retain the neural architecture for regression problems in (Finn et al., 2017; Collins et al., 2020). That is, we deploy fully-connected neural network with two hidden layers of 40 ReLU nonlinear activation units. All methods use stochastic gradient descent step as the inner loop.
Few-shot Image Classification. We retain the neural architecture for few-shot image classification problems in (Finn et al., 2017; Collins et al., 2020). In detail, four-layer convolutional neural network is used for both Omniglot and mini-ImageNet datasets. All methods use one stochastic gradient descent step as the inner loop.
K
Additional Experimental Results
More Quantitative Analysis. Due to the page limit in the main paper, we include the a's sensitivity experimental result in sinusoid st-shot regression. As illustrated in Fig. (7), the trend is similar to that in sinusoid s-ohot regression. Worst-case optimization degrades the average performance of TR- MAML. DR-MAML is entangled with MAML in the average performance, while the performance gap between them is significant in the worst-case.
Simusoid 10-shot
SIAME
MANI
MANE TR-MANE
39
TR-ALASH 200-MAMEE/0AN))
31
15.
Avorage
Worst
8.-
65-
34
Confidence Level
Confideuce
Coufidence Level
Figure 7: Meta Testing Performance of Meta-Trained DR-MAML with Various Confidence Levels a. MAML and TR-MAML are irrelevant with the variation of a in meta-training. The plots report testing MSEs with standard error bars in shadow regions.
Regarding few-shot image classification in the mini-ImageNee dataset, we can observe that in Fig. (8), the standard error is relatively smaller than in previous regression cases. When the confidence level over a particular value, e.g. a > 0.5, there occurs significant decline of performance in all metrics. Note that when + 1.0, the optimization objective approaches the worst-case optimization objective. Here we attach two possible reasons for the performance degradation phenomenon: (i) The adopted base optimization technique matters in nearly worst-case optimization. The stochastic mirror descent-ascent (Juditsky et al., 201 1) is utilized in TR-MA which is more stable in deriving the optimal solution. In comparison, the stochastic gradient descent with sub-gradient operations works as the optimization method, and this method can be unstable when the scale of worst-case examples is small in the update. (ii) For few-shot image classification, estimates of VaR, be less precise with limited batch sizes and higher a values since the meta risk function value is discontinuous. Consequently, we can also attribute the severe degradation of fast adaptation performance in higher value cases to the approximation errors of quantile estimates.
More Visualization Results. Further, explore the influence of the expected tail risk minimization principle in meta learning. Here the landscape of meta risk values, namely fast adaptation losses, is presented in the sinusoid regression problem.
As exhibited in Fig. (9), the evaluated meta risk values from one random trial are associated with hyper-parametee parameters of tasks. The final optimized results can discover some tasks difficult in fast adaptations. Meta learning methods are difficult to adapt task regions with higher amplitudes. MAML exhibits higher MSEs in regions with the amplitude a > 2, while DR-MAML minimizes a proportion of risks in these regions. In contrast, TR-MAMI. reduces the risk around task regions with > 2 to certain extent: however, it shows relatively higher risk values in easy regions. Such
29

---

minss-maageNet S-way 1-shot
Worst 6.
Avorage
C.VR
15
Confidence Level
Confidence Level
Confidence Level
Figure 8: Meta Testing Classification Accuracies of MeainTr DR-MAML with Various Confidence Levels MAML and TR-MAML are irrelevant with the variation of in meta-training. The plots report testing accuracies with standard error bars in shadow regions.
DR-MAML
MAML
TR-M
Figure 9: The Fast Adaptation Risk Landscape of Meta-Trained DR-MAML, TR-MAML and MAML.
Shown is an example sinusoid 5-shot regression, which corresponds to the function space f(z) = a sin( 6). The X-axis denotes the amplitude parameter a, and the Vaxii is the phase parameter b. The confidence level is = 0.7 in meta-training. The plots report testing MSFs in the Z-axis with a random trial of generating tasks.
evidence reflects the interpretability in optimization within the expected tail risk minimization, and the landscape of meta risk values is relatively flat and smooth than others.
Experimental Results with DR-CNPs. The implementation is the same CNP (Garnelo et al., 2018a) and official Github files (https: //github. thub. cww/ (deeeminn/nnurueeeeeeeeeeeeeeeee cocll-procce wwwppporseeeeeeeeeeeeeeeee /blob/ master/ cestee/cooniiiiin conditional peeural_procese .ip.cssssss));; the Gaussian Process curve generator works as the benchmark, and we retain the neural architecture and optimization pipelines in Github files. Please check implementation details, e.g., neural architectures, optimizers, epochs, batch sizes, etc., from the mentioned GitHub repository.
Also, note that this is more challenging than sinusoid as there is more randomness in generating curves. Similar to few shot image classifications, we set a = 0.5 in meta training DR-CNPs. In meta-testing processes, we randomly sample 64 curves files and examine the performance of CNP, TR-CNP (worst-case), and DR-CNP (distributionally robust). The log- likelihoods in runs reported as follows (the higher, the better).
with random context points in GitHub
From Table (6), we observe that the DR-CNU retains the average performance the same as the CNP and simultaneously achieves the highest worst and CVaR, log-likelshoods. In contrast, the TR-CNP sacrifices the average performance a lot and obtains the mediate worst result. Here we attribute
Table 6: Meta-testing point-wist wise log- lideliidl results of Gaussian processes in 5 runs.
Method
Average
Worst
CVaR,
CNP (Gareeo et al., 2018a)
-0.22-0004
-1.35
-0,50
TR-CNP
-0.80=0001
-1.08+00.z
-0.87+0.01
OR-MAML
-0.21=0002
-0.74:10.08
-0.37-10.03
30

---

Figure 10: Curve Fitting Result Visualization in 5 Random Trials. From Up to Down are respectively DR-CNPs (Ours), CNPs, and TR- TR-CNPs. Shadow regions are three standard deviations.
the failure of achieving the best worst performance using TR-CNPs to the sensitivity of min-max optimizers mentioned in TR-MAMI.
Particularly, some randomly sampled fitted curves difficult scenarios, e.g., setting the number of context points 5, visualized in Fig. (10). Observations consistent with Table (6), where TR- CNPs exhibit over estimated uncertainty and cannot well reveal the trend of curves. In comparison, DR-CNPs well handle challenging cases, capturing more convincing uncertainty.
Experimental Results in Meta Reinforcement Learning. We also performed additional experiments in 2-D point robot navigation tasks. This is meta reinforcement learning benchmark. All setups of point robot are the same as MAML, and please refer to pytorch-maml and cavia Github for details of the navigation environments. The neural network of policies is a two-hidden layer MLP with 64 hidden units and tanh activations. The fast learning rate is 0.1, and the number of batches is 500. Other details, e.g., optimizers, step-wise rewards, horizons and etc., can be found in MAML. The one-step gradient is performed for fast adaptation, and the trust region policy optimization works set 0.5 in meta
the policy gradient algorithm. Similar few shot image classifications, training point robots. In meta-testing processes, we randomly sample 100 navigation goals as tasks and examine the performance of MAML, TR-MAML, and DR-MAML. The histogram comparison visualized Fig. (11).
Histogram Comparison
Histogram Comparison
14
14.
Frecusncy
10-
20
40
Step-otte Rewards
Resoants
(8)
(b)
Figure 11: Histograms of Meta-Testing Performance in Point Robot Navigation Problems. We report the negative average step-wise rewards in r-axis.
Unlike few-shot supervised learning, worst-case optimization is pretty challenging in RI, and the result of TR-MAML is unstable and camnot achieve the goal of worst-case optimality in Table (7).
31

---

Table 7: Meta-testing step-wise rewards in 4 runs in point robot navigation tasks.
Method
Average
Worst
CVaRa
CNP (Garnelo et al., 2018a)
-16.6=0.19 -36.8=0.12
-46.944.2
-24.2:00.9
TR-CNP
-81.344.35
-52, +0089
DR-MAML
-14.3=0.49
-46.7+3.24
-21.6:10.79
This is due the nature of large performance deviations when deploying policies in various tasks. Seeking optimizers for stable worst-case optimization is non-trivial. Also, stably generalization across skills reinforcement learning is indeed difficult with a few interactions. Overall, DR-MAML can well control the proportional worst case performance.
L Platforms & Computational Tools
In this research project, we use NVIDIA 1080-Ti GPUs computation. Pytorch (Paszke al., 2019) works as the deep learning toolkit in implementing few-shot image classification experiments. Meanwhile, Tensorflow is the deep learning toolkit for implementing sinusoid few-shot regression experiments.
M Declaration of Author Contribution
The authors confirm their contribution to the paper as follows: Q.W. (Qi Wang) and Y.Q.L. (Yiqin Lv) conceptualized the idea of tail risk minimization in meta learning and developed simple yet effective strategies with mathematical demonstrations. Y.Q.L. performed most of the experiments and Q.W. examined the performance meta reinforcement learning benchmark. Q.W. and Y.Q.L. wrote and prepared the manuscript, while Y.H.F. (Yanghe Feng) contributed crucial insights into the organization and checked the mathematical part of the draft submission. J.C.H. (Jincai Huang) and Z.X. (Zheng Xie) contributed to the supervision and helped proofread the manuscript. All authors reviewed the results and approved the final version of the manuscript. Meanwhile, we also thank dr. Zhengge Zhou for helpful discussion.
32


Reviews:
[{"review_id": "pybjvpTsMF", "reviewer": "Reviewer_YmWX", "summary": "This paper proposes a novel meta-learning pipeline from the perspective of distributionally robust perspective to robustify fast adaptation. Through a two-stage strategy heuristically, the model could control the worst fast adaptation cases at a certain probabilistic level. And experiments show the effectiveness of the proposed model compared to other baselines, especially TR-MAML.", "questions": "see weakness.", "limitations": "yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed idea is interesting, and it could generalize the expected risk minimization and worst-case risk minimization for meta learning together.\n2. The presentation and structure of the paper is good.\n3. The experiments are extensive for the proposed model DR-MAML.", "weaknesses": "1. It is a difficult to follow for section 4. Some motivations or intuitions are necessary. For instance:\n- what is the goal to introduce $(R^{+}, B)$ above line 129? It does not appear in the following paper.\n- what is the intuition to involve the slack variable $\\xi$ below line 157?\n\n2. Some notations are not clear. For instance:\n- line 134: why $M^{-1}(\\ell)$ instead of $M(\\ell)$?\n- For Eq.(1) and (3), is it better to add \"min\" after the \":=\"? \n- line 208: \"a optimization...\" -> \"an opt...\"\n\n3. Do you consider diverse tasks during meta-training or just follow the common setting used in MAML? Since the goal is to robustify the fast adaptation, if you just use the common setting in MAML, how could you validate whether your model is helpful to keep robustness? Or the common setting is not robust?\n\n4. Could you add some experiments for Example 2: DR-CNP?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank \\# Reviewer YmWX for insightful comments. The remainders focus on questions to answer.\n***\n**1. Goal to introduce $(R^+,B)$ and the slack variable in Line157**\n\nThanks for this question. \n\n(1) The goal of introducing $(R^+,B)$ is to describe the risk value distribution in the task space, and we need to induce the probability measure of risk values via the task distribution. This notation serves the CVaR definition. \n\n(2) The involvement of the slack variable is to relax the intractable unparameterized tail task distribution $p_{\\alpha}(\\tau;\\vartheta)$ to the whole task distribution. Since the optimal of the slack variable is the VaR, this motivates the two-stage strategy as heuristics.\n\n***\n**2. Notation meanings and typos**\n\nThanks for these comments and suggestions. \n\n(1) In line134, the notation $\\mathcal{M}^{-1}$ denotes the map from the risk value in the Euclidean space to the task space, as we need to screen the subspace of tasks in the tail risk for optimization. \n\n(2) We will modify “:=” after Equations in the final version and revise the typo “a optimization” to “an optimization” in line 208.\n\n***\n**3. Diverse tasks or the common setting in meta-training**\n\nThis is a crucial question in the literature. We use diverse tasks during meta-training, and this follows the same setup as TR-MAML. In TR-MAML, a large number of easy tasks and a few difficult tasks are used in meta-training. This setup in TR-MAML is better to validate the robustness as more difficult tasks appear in meta-testing processes. More explanations can be found in the setup section in TR-MAML. \n\n***\n**4. Additional experiments for DR-CNP** \n\nThanks for this nice suggestion. ***We’ve taken your advice and changed the manuscript by adding the DR-CNP results***. The implementation is the same as CNP [1] and official  deepmind/neural-processes Github files; the Gaussian Process curve generator works as the benchmark, and we retain the neural architecture and optimization pipelines in Github files. Please check implementation details, e.g., neural architectures, optimizers, epochs, batch sizes, etc., from the mentioned GitHub repository. \n\nAlso, note that this is more challenging than sinusoid as there is more randomness in generating curves. Similar to few shot image classifications, we set $\\alpha=0.5$ in meta training DR-CNPs. In meta-testing processes, we randomly sample 64 curves per run with random context points as generate_curves in GitHub files and examine the performance of CNP, TR-CNP (worst-case), and DR-CNP (distributionally robust). The log-likelihoods in 5 runs are reported as follows (the higher, the better):\n\n***Contents to insert in Appendix K: Table 6. Meta-testing point-wise log-likelihood results in 5 runs.***\n\n|  Methods     | Average            | Worst              | $\\text{CVaR}_{\\alpha}$ |\n|--------------|--------------------|--------------------|------------------------|\n| CNP          | $-0.22(\\pm0.04)$      | $-1.35(\\pm0.38)$      | $-0.50(\\pm0.11)$          |\n| TR-CNP       | $-0.80(\\pm0.01)$      | $-1.08(\\pm0.02)$      | $-0.87(\\pm0.01)$          |\n| DR-CNP(Ours) | $-0.21(\\pm0.02)$ | $-0.74(\\pm0.08)$ | $-0.37(\\pm0.03)$     |\n\nFrom Table 6, we observe that the DR-CNP retains the average performance the same as the CNP and simultaneously achieves the highest worst and $\\text{CVaR}_{\\alpha}$ log-likelihoods. In contrast, the TR-CNP sacrifices the average performance a lot and obtains the mediate worst result. Here we attribute the failure of achieving the best worst performance using TR-CNPs to the sensitivity of min-max optimizers as mentioned in TR-MAML.  \n\nParticularly, ***some randomly sampled fitted curves in difficult scenarios, e.g., setting the number of context points as 5, are visualized in the Global Rebuttal PDF file, Figure 10***. Observations are consistent with Table 6, where TR-CNPs exhibit over estimated uncertainty and cannot well reveal the trend of curves. In comparison, DR-CNPs can well handle challenging cases, capturing more convincing uncertainty.\n\n***\n**Reference**\n\n[1] Garnelo, M., Rosenbaum, D., Maddison, C., Ramalho, T., Saxton, D., Shanahan, M., ... & Eslami, S. A. (2018, July). Conditional neural processes. In International conference on machine learning (pp. 1704-1713). PMLR.\n\n***\nFinally, we hope these questions are well answered, and concerns are well addressed. Thanks again for your efforts. For any other question, we are happy to discuss it and make further clarifications. \n\n\n"}, {"review_id": "iaKAmiiklg", "reviewer": "Reviewer_ANpj", "summary": "In this paper, the authors present a method by which to improve the expected tail risk during meta learning via a two-stage optimisation strategy. In addition, the authors provide theoretical and empirical analysis of their proposed approach, demonstrating improvements in the average, worst-case and conditional value at risk (CVar) metrics.", "questions": "- In the paper, the Monte Carlo estimate allows for the theoretical analysis, but it is indeed quite crude. Do the authors have any comments on perhaps improving upon this (even if the theory isn't there yet)?\n- What optimizers did the authors use? In the existing limitations section, they suggest that the batch size selection has a greater effect for first-order optimizers. Did the authors attempt to use any higher order (or approximate) optimizers?\n- I believe the theoretical analysis breaks down for the image classification tasks? (please do correct me if I'm wrong) Do the authors have any further insights on this point, in particular as to why the tail risk is not well optimised for omniglot, but works well for mini-ImageNet? \n- The authors also state that challenging meta-learning tasks can reveal more advantages of DR-MAML (line 281). Have more challenging tasks been tried? This could, for example be the meta-dataset or something equivalent? \n- In Figure 5, there appears to be deviations from the trend when $\\alpha = 0.7$, and in general, the trend appears to be somewhat non-monotonic. Is this a dataset dependant issue?\n- Do the authors have any general guidelines for how the image classification performance of DR-MAML changes with $\\alpha$?", "limitations": "The limitations of this work are outlined in Section 6 and assumptions are outlined throughout the paper. In addition, error bars and std's are provided for all experiments. \n\nOverall, I see no potential negative societal impacts of this work. If anything, robust meta-learning has a potentially positive societal impact for deployed meta-learning systems.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is written and structured very well. In particular, Sections 2,3, and 4 provide great introductions to the meta-learning paradigm and the idea of risk-minimisation applied to meta learning. The theoretical analysis outlined in Section 4 is also quite clear. Looking at the supplementary material, proofs and assumptions are outlined well, and ablations are provided. ", "weaknesses": "- This paper is quite strong, and many of my initial concerns are answered in the supplementary material. \n- Besides minor issues with regards to wording and grammar, this paper is quite well written. For example, on lines 34-35, 'suffering from tribulation' and 'plaint'. \n-  A minor concern, but one which might help practitioners. Choosing the optimal meta-batch size appears quite important to the final performance. Although the authors discuss the influence of task batch size, perhaps ablating across more (and a wider range of) batch sizes might be more informative. \n\nFor other weaknesses/comments, see questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank \\# Reviewer ANpj for these helpful comments. The remainders focus on questions to answer.\n\n***\n**1. Potential improvement on crude Monte Carlo**\n\nThis is a crucial question. \n\n(1) We choose the crude Monte Carlo in quantile estimate due to its simplicity. However, accurate estimation of the quantile contributes to better convergence in a theoretical sense. We can also use some deep density estimators for VaR estimates, which may require additional modeling and computations. \n\n(2) Another factor related to the crude Monte Carlo is the task batch, which is quite empirical. The main paper reports the sinusoid investigation using larger task batches greater than 50. For even larger batch sizes, especially in image cases, our device suffers from cuda out-of-memory issues.\n\n***\n**2. More challenging tasks to investigate**\n\n(1) The 2-D point robot navigation is a meta reinforcement learning benchmark. All setups of the point robot are the same as MAML [1-2], and please refer to pytorch-maml-rl and cavia Github for environment and task details. The policy network is a two-hidden layer MLP with 64 hidden units and tanh activations. Other details, e.g., optimizers, step-wise rewards, horizons and etc., can be found in MAML [1]. The one-step policy gradient is performed for fast adaptation with the learning rate 0.1, and TRPO [3] works as the policy gradient algorithm. $\\alpha=0.5$ is set in meta training DR-MAML. In meta-testing, we uniformly sample 100 navigation tasks (goals’ locations follows the distribution $U[-0.5, 0.5]^2$) and examine the performance. The step-wise episodic rewards are reported as follows.\n\nNew results as Table 7. Meta-testing step-wise rewards in 4 runs in point robot navigation tasks (the higher, the better).\n\n|  Methods     | Average | Worst | $\\text{CVaR}_{\\alpha}$ |\n|--------------|---------|-------|------------------------|\n| MAML          | $-16.6(\\pm0.19)$     | $-46.9(\\pm4.22)$   | $-24.2(\\pm0.49)$                    |\n| TR-MAML       | $-36.8(\\pm0.12)$    | $-81.3(\\pm4.35)$   | $-52.1(\\pm0.89)$                    |\n| DR-MAML(Ours) | $-15.3(\\pm0.49)$     | $-46.7(\\pm3.24)$   | $-21.6(\\pm0.79)$                    |\n\nUnlike few-shot supervised learning, worst-case optimization is pretty challenging in RL, and the result of TR-MAML is unstable and cannot achieve the goal of worst-case optimality in Table 7. This is due to the nature of large performance deviations when deploying policies in various tasks. Seeking optimizers for stable worst-case optimization is non-trivial.  Also, stably generalization across skills in reinforcement learning is indeed difficult with a few interactions. Overall, DR-MAML can well control the proportional worst case performance.\n\n(2) We’ve also performed additional experiments in stochastic process modelling with DR-CNP. Please refer to Response to \\# Reviewer YmWX or Global Response for details.\nContents to insert in Appendix K: Table 6. Meta-testing point-wise log-likelihood results in 5 runs (the higher, the better). \n\n|  Methods     | Average            | Worst              | $\\text{CVaR}_{\\alpha}$ |\n|--------------|--------------------|--------------------|------------------------|\n| CNP          | $-0.22(\\pm0.04)$      | $-1.35(\\pm0.38)$      | $-0.50(\\pm0.11)$          |\n| TR-CNP       | $-0.80(\\pm0.01)$      | $-1.08(\\pm0.02)$      | $-0.87(\\pm0.01)$          |\n| DR-CNP(Ours) | $-0.21(\\pm0.02)$ | $-0.74(\\pm0.08)$ | $-0.37(\\pm0.03)$     |\n\nFrom Table 6, we observe that the DR-CNP retains the average performance the same as the CNP and simultaneously achieves the highest worst and $\\text{CVaR}_{\\alpha}$ log-likelihoods. In contrast, the TR-CNP sacrifices the average performance a lot and obtains the mediate worst result. Here we attribute the failure of achieving the best worst performance using TR-CNPs to the sensitivity of min-max optimizers as mentioned in TR-MAML.  \n\n***\n**3. Optimizers used in the experiments**\n\n(1) For a fair comparison, we keep all the experimental setups the same as TR-MAML. And first-order stochastic gradient descent is used in implementations for few-shot regression and classification scenarios. \n\n(2) As for higher order or more advanced optimizers, we run additional experiments in 2-D point robot navigation tasks, where the conjugate gradient method is used for trust region optimization. That is because standard first-order optimizers harm all baselines in convergence and stability.\n\n***\n**4. Other empirical observations and guidelines**\n\nThanks for your questions. \n\n(1) Regarding observed tail risk differences between Omniglot and mini-imageNet, we attribute this to a performance bottleneck. Note that Omniglot is relatively easy for all baselines. Hence all baselines achieve over 90% CVaR accuracies in meta-training. In contrast, DR-MAML can show more advantages in challenging tasks, e.g., mini-imageNet. \n\n(2) Regarding the deviations from the trend in Figure 5, we assume this is a data-dependent issue, and the quantile estimators may be less accurate in this scenario. \n\n(3) Regarding the general guidelines for $\\alpha$ selection, in the image classification, we presume the mediate value, e.g., 0.5, is good enough. However, this also depends on the task batch size, as observed in the Appendix experiments.\n\n**Reference**\n\n[1] Finn, C., Abbeel, P., & Levine, S. (2017, July). Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning (pp. 1126-1135). PMLR.\n\n[2] Zintgraf, L., Shiarli, K., Kurin, V., Hofmann, K., & Whiteson, S. (2019, May). Fast context adaptation via meta-learning. In International Conference on Machine Learning (pp. 7693-7702). PMLR.\n\n[3] Schulman, J., Levine, S., Abbeel, P., Jordan, M., & Moritz, P. (2015, June). Trust region policy optimization. In International conference on machine learning (pp. 1889-1897). PMLR.\n\n***\n*Finally, we hope your questions are well answered. And many thanks for your reviews.*\n"}, {"review_id": "JYyY9K3FtR", "reviewer": "Reviewer_NA6N", "summary": "This paper considered a robust meta-learning framework by addressing the worst-tasks performance. Specifically, this paper adopted conditional value at risk (CVR) to filter poorly performed tasks. In order to effectively learn the CVR, this paper adopted a surrogate loss, a variant of hinge loss with a learnable threshold. The proposed framework exhibits empirical improvement in certain few-shot tasks. \n", "questions": "In general I would think this paper has several merits. However, this reviewer has certain concerns on the novelties and technical parts. Thus currently I vote for a weak rejection. Authors are suggested to check weakness points to know details. \n", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "In general, this paper tackles an interesting problem in mete-learning. I would think the following points might be interesting. \n- Addressing robustness issue in meta-learning. \n- A principled and simple method through the tool of conditional value at risk.\n- The paper is quite clearly written. \n", "weaknesses": "This reviewer found there are several concerns in the current version. The factors that make this reviewer suggest that a major revision is better. I will discuss these points in the following aspects. \n\n\n**Originality** and **Significance**\n\nThe main motivation of the proposed method is to address the robustness in meta-learning. However, this reviewer feels a bit confused about its motivation in choosing CVR based approaches. There are alternative methods in distribution robust optimization such as group dro, which can be naturally extended by adding the task weights as a learnable parameter. Or if we adopt Wasserstein distance, we could generate new samples as a data augmentation training. \n\nBased on these, I would think the proposed method may lack sufficient motivation or comparison to justify its benefits. \n\n\n**Quality**\n\n\nI have some technical concerns. \n\n- In equation (2), the worst performing group sometimes is quite risky since it would have quite arbitrary value. It should be better to consider the robustness in the context of distributional robustness optimization.\n- In sec 4.3, I would think there are better principles to learn the threshold value. I mean, since the learnable thresholding value is convex for the loss in (6), thus we could directly adopt several solvers to solve the optimal value after each epoch. This could construct a principled method to optimize the hyper-parameter. \n- In the few-shot context, the worst-performing groups in general have quite few data, does the accuracy really make some statistical sense?\n- The representation learning based meta-method seems not empirically validated?\n\n\n**Clarity**. \n\nMost parts within the paper are clear. Some parts are still a bit confusing, for example\n- Sec 4.1 128-129 For any \\theta, the meta learning.. It should be for any **fixed** \\theta, right? \n- The surrogate seems quite similar to the hinge loss, right? I would think a further discussion is better. \n- DN-CRP is better named as meta-representation learning.\n- Fig 2 seems a bit confusing because of \\alpah_{t}, actually \\alpha here is fixed. The threshold value was changing w.r.t. t rather than \\alpha.\n\n\n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank \\# Reviewer NA6N for these insightful comments. The remainders focus on concerns to address.\n\n***\n**1. Group DRO baseline comparison, research motivation and data augmentation**\n\nThanks for this precious question. \n\n(1) Sorry for confusing you. ***The Group DRO baseline can be found in Appendix-K Line846-867, where it is called the risk reweighted method in Table 4/5***. The experimental analysis is attached there. \n\n(2) Regarding motivation, the choice of CVaR considers scenarios similar to group DRO. The difference is CVaR considers the tail risk while group DRO assigns the adjustable weight to tasks, and both are crucial for increasing robustness. Also, we build up connections between CVaR and group DRO in Appendix. Importantly, we contribute the improvement guarantee in a theoretical sense for analyzing CVaR objectives. \n\n(3) Since all baselines are not data augmentation-based methods, we only compare the risk minimization principles for fairness. We agree that data augmentation, like Wasserstein distance, has the potential for robustness improvement, and we leave it a future exploration.\nTo avoid confusion, we will rename the risk reweighted by group DRO in the appendix.\n\n***\n**2. Other principled method to optimize the hyper-parameter $\\xi$**\n\nThanks for this comment. \n\n(1) For the threshold value or slack variable $\\xi$, we agree that other methods exist to optimize, such as deep density learning models. However, our proposed one, a quantile estimate with crude MC, is the simplest. \n\n(2) Note that the risk function is non-convex in most scenarios; our proposed heuristic method is easy to implement and has an improvement guarantee. Based on these properties and findings, we take the simple yet effective way for optimization.\n\n***\n**3. Meta learning and statistical sense of few data**\n\nThanks for the question. Meta learning or few-shot learning indeed relies on a few data points for skill transfer. As for the statistical sense, it is considered in a task distribution sense. Though data are few, the task space is huge, and statistical sense is verified in most meta learning work, including MAML.\n\n***\n**4. Representation based meta learning to validate**\n\nThanks for this suggestion. We’ve taken advice and included the DR-CNPs as the additional experiments. Please refer to **Response to \\# Reviewer YmWX** or **Global Comments** for details.\n\n***\n**5. Other clarity issues**\n\nThanks for these points. \n\n(1) Regarding Line128-129, your understanding is correct, and it is fixed and bounded. \n\n(2) Regarding the hinge loss, our optimization objective indeed relates to that a bit. We'll add, “Note that our optimization objective also has a connection with hinge losses. Hinge losses are considered in a data point sense while the hinge operation in ours is examined in a risk value distribution sense.” in Line157. \n\n(3) Regarding DR-CNP, you are right, and it is a meta-representation learning method. We’ll further detail this in the final version. \n\n(4) Regarding $\\alpha_t$, here $t$ is associated with $\\xi$ and indexes the iteration number. Your suggestion is nice, and we will modify the location of indices in the Figure.\n\n***\nFinally, we hope your questions are well answered. And many thanks for your helpful reviews. For any other question, we are happy to discuss it and make further clarifications. \n"}, {"review_id": "pftqYXA1fO", "reviewer": "Reviewer_vUMr", "summary": "The paper proposes a novel expected tail risk objective for meta-learning, which provides an intermediate of the worst-case and average risk minimization algorithms. The proposed algorithm can be adapted to existing meta-learning algorithms with improved robustness.  ", "questions": "See the discussion in the weaknesses section.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* The paper is well-written and easy to follow.\n* Considering expected tail risk in meta-learning is novel to me. Controlling the worst task performance at a certain probabilistic level can improve the robustness.\n* The experimental results show the effectiveness of the proposed approach.\n", "weaknesses": "* The meta risk function $\\ell(D_{\\tau}^T, D_{\\tau}^C;\\nu)$ is an empirical risk estimation that depends on the number of data points $m$ of task $\\tau$. It’s preferable to consider its generalization as in [1,2,3].\n* Why only adopt eight meta-training tasks and four meta-testing tasks for the experiment of mini-ImageNet in Table.3?\n\n>[1] Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, and Massimiliano Pontil. Learning-to-learn stochastic gradient descent with biased regularization. In International Conference on Machine Learning, pages 1566–1575. PMLR, 2019a.\n\n>[2] Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoretic analysis. Advances in Neural Information Processing Systems, 34: 25878–25890, 2021.\n\n>[3] Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, Jason Lee, Sham Kakade, Huan Wang, and Caiming Xiong. How important is the train-validation split in meta-learning? In International\nConference on Machine Learning, pages 543–553. PMLR, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank \\# Reviewer vUMr for these constructive comments. The remainders focus on questions to answer.\n\n***\n**1. Consider and discuss generalization related work.**\n\nThanks for this suggestion. \n\n(1) Most previous work studies the generalization in the context of the complete task space within the expected risk minimization principle, while our work considers the expected tail risk. This set-up difference brings more difficulties in accurately quantifying the bound. *Instead, this work pays more attention to the estimated stepwise bound w.r.t. $\\alpha$ in Proposition 2, the improvement guarantee and convergence properties in Theorem1, and the gaps of optimized solution in Theorem2. As for generalization capability across tasks, we execute the quantitatively evaluation in experiments.*\n\n(2) In Literature Review section Line83, *we will cite and add more discussions on references [1-3] in the final version* as follows: \n\nInsert the contents in Line 84\n\n>Besides, there exist several important works investigating the generalization capability of methods. \\citet{bai2021important} [3] conduct the theoretical analysis of the train-validation split and connect it to optimality. In \\citep{chen2021generalization} [2], a generalization bound is constructed for MAML through the lens of information theory. \\citet{denevi2019learning} [1] study an average risk bound and estimate the bias for improving stochastic gradient optimization in meta learning.\n\n**Reference:**\n\n[1] Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, and Massimiliano Pontil. Learning-to-learn stochastic gradientdescent with biased regularization. In International Conference on Machine Learning, pages 1566–1575. PMLR,2019a.\n\n[2] Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoretic analysis. Advances in Neural Information Processing Systems, 34: 25878–25890, 2021.\n\n[3] Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, Jason Lee, Sham Kakade, Huan Wang, and Caiming Xiong. Howimportant is the train-validation split in meta-learning? In International Conference on Machine Learning, pages543–553. PMLR, 2021.\n \n***\n**2. Setup of tasks in mini-imageNet**\n\nThanks for this question. \n\n(1) In the original TR-MAML’s mini-imageNet setup, the concept of eight meta-training and four meta-testing is in the combinatorial class number sense. For example, since there 100 classes, the eight meta-training means the partition of 64 classes with task lists $\\{6,7,7,8,8,9,9,10\\}$, where the element corresponds to the number of classes in one task. Other M-way N-shot setting is standard in the domain. \n\n(2) For a fair comparison, we use the same setup as TR-MAML, where eight meta-training and four meta-testing tasks are created for mini-imageNet experiments.\n\n***\nFinally, we hope your questions are well answered. And many thanks for your helpful reviews. For any other question, we are happy to discuss it and make further clarifications. \n"}, {"review_id": "Gg80nSL4mb", "reviewer": "Reviewer_Wn6T", "summary": "The authors address an existing gap in fast adaptation of meta-learning by developing a robustification framework that draws from the concept of expected tail risk and recasts robustification as a distributional optimization problem. They adopt a heuristic two-stage optimization strategy that is broadly useful toward most meta-learning methods. The applicability of this framework toward enhancing robustness against task distributions and mitigating worst-case performance is verified and discussed over a selection of benchmark tasks. ", "questions": "- What is the justification for deriving results over only 3 runs? This makes empirical claims seem suspect, e.g. the conclusion that “DR-MAML surpasses MAML in terms of average performance”\n\n- It would be nice to see additional discussion of the impacts of assumptions 1/2/3. (The authors briefly touch on this around line 294.)\n\n- Pseudocode is in Appendix B but it would be more useful in the main text.\n\n* Experimental results for the proposed approach could be further substantiated by including a discussion of Tables 4/5 in the main text.\n\n* Minor:\n\t* line 34: “plaint” -> “plain” (note: this may be ignorance on my part.)\n\t* line 199: “sometime” -> “sometimes”\n\t* line 208: “a” -> “an”\n\t* Figure 4 caption line 2: “DR-MAMAL” -> “DR-MAML”\n\t* Appendix, line 859: “Risk-Rweighted” -> “Risk-Reweighted”\n", "limitations": "* The authors name the primary theoretical and practical limitations of their work.\n\n* Societal impacts are not directly applicable. The authors touch upon limitations of existing meta-learning methods for high-risk real-world scenarios, which is important.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The paper motivates the importance of expanding work in distributionally robust optimization and presents a unique formulation of the robustness problem for meta learning. \n+ Concrete illustration of how the proposed optimization method can benefit existing meta learning work.\n+ Well-organized sections and visual structure.\n+ Nice FAQ — the authors should consider making their contributions clearer in the main text of the paper.", "weaknesses": "- Appendix J.1 states: \"The optimal selection of the confidence level α is difficult since we need to trade off the worst and average performance. Our setup is to minimize CVaRα, which already considers the worst-case at some degree, so we watch the average performance in meta training results and set α = 0.7 for all few-shot regression tasks.\" This isn't really tail risk (e.g., alpha > 0.9, alpha < 0.05). Given the tail risk problem, why does alpha range from 0.2 to 0.8 (Figure 5)?\n\n- The paper seems incremental to TR-MAML. It would have been better to see more detail in the main text (as opposed to Appendices) on the main contribution of taking a probabilistic approach to meta learning and using a cVaR probabilistic measure (e.g., applicability, convergence, bounds). Instead, the main text focused on implementing DR-MAML which may not work for true tail scenarios. \n\n- The authors should introduce at least a new dataset versus just reusing MAML/TR-MAML. Appendix J shows that the authors simply modified the loss function, reusing code from MAML and TR-MAML. It would be nice to see additional experiments in more challenging settings, e.g., RL tasks like point navigation or continual learning.\n\n- Appendix K states: \"Note that when α → 1.0, the optimization objective approaches the worst-case optimization objective. Here we attach two possible reasons for the performance degradation phenomenon:...\" This should be expanded and included in the main text as opposed to the Appendix.\n\n- It seems the main contribution is imposing a probabilistic constraint (cVaR) versus using worst-case TR-MAML, which necessarily only works for meta-learning over a large task space.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank \\# Reviewer Wn6T for these insightful comments. The remainders focus on concerns to address.\n\n***\n**1. Misconception of the tail risk**\n\nThanks for the comment. \n\n(1) In our paper, ***the concept of the tail risk is from the literature work [1-2] and the standard definition in Wikipedia term “tail risk”***. The varying $\\alpha$ describes the tail risk level. As for $\\alpha$ ranging between 0.2 and 0.8, it is because we can only test limited scenarios in $\\alpha\\in(0,1)$, and $\\alpha\\to 0$ and $\\alpha\\to 1.0$ corresponds to the average and worst case. \n\n(2) Due to numerous tasks, the optimization strategies, including ours and group DRO, perform batch optimization in practice. \n\nHence, the concept of the tail risk in this paper is correct and optimization in a task batch way works in the tail cases.\n\n**Reference**\n\n[1] Sarykalin, S., Serraino, G., & Uryasev, S. (2008). Value-at-risk vs. conditional value-at-risk in risk management and optimization. In State-of-the-art decision-making tools in the information-intensive age (pp. 270-294). Informs Tutorials in Operations Research.\n\n[2] Hong, L. J., & Liu, G. (2009). Simulating sensitivities of conditional value at risk. Management Science, 55(2), 281-293.\n\n***\n**2. Incremental to TR-MAML**\n\nThanks for the question. We need to clarify that our work differs a lot from TR-MAML: \n\n(1) define distributionally robustness for fast adaptation rather than worst-case in TR-MAML; (2) design heuristic two-stage optimization algorithms with improvement guarantee; (3) show increased robustness without sacrificing the average performance in contrast to TR-MAML. \n\nHence, the method is new with theoretical contributions and balancing performance. Though the experimental set-up is incremental w.r.t. TR-MAML, we also provided new experimental results after taking advice.\n\n***\n**3. Include the point navigation**\n\nThanks for these constructive comments. We’ve taken advice and run experiments in point robots for navigation tasks, and will include results in the final version.\n\n(1) The 2-D point robot navigation is a meta reinforcement learning benchmark. All setups of the point robot are the same as MAML [1-2], and please refer to pytorch-maml-rl and cavia Github for environment and task details. The policy network is a two-hidden layer MLP with 64 hidden units and tanh activations. Other details, e.g., optimizers, step-wise rewards, horizons and etc., can be found in MAML [1]. The one-step policy gradient is performed for fast adaptation with the learning rate 0.1, and TRPO [3] works as the policy gradient algorithm. $\\alpha=0.5$ is set in meta training DR-MAML. In meta-testing, we uniformly sample 100 navigation tasks (goals’ locations follows the distribution $U[-0.5, 0.5]^2$) and examine the performance. The step-wise episodic rewards are reported as follows.\n\nNew results as Table 7. Meta-testing step-wise rewards in 4 runs in point robot navigation tasks (the higher, the better).\n\n|  Methods     | Average | Worst | $\\text{CVaR}_{\\alpha}$ |\n|--------------|---------|-------|------------------------|\n| MAML          | $-16.6(\\pm0.19)$     | $-46.9(\\pm4.22)$   | $-24.2(\\pm0.49)$                    |\n| TR-MAML       | $-36.8(\\pm0.12)$    | $-81.3(\\pm4.35)$   | $-52.1(\\pm0.89)$                    |\n| DR-MAML(Ours) | $-15.3(\\pm0.49)$     | $-46.7(\\pm3.24)$   | $-21.6(\\pm0.79)$                    |\n\nUnlike few-shot supervised learning, worst-case optimization is pretty challenging in RL, and the result of TR-MAML is unstable and cannot achieve the goal of worst-case optimality in Table 7. This is due to the nature of large performance deviations when deploying policies in various tasks. Seeking optimizers for stable worst-case optimization is non-trivial.  Also, stably generalization across skills in reinforcement learning is indeed difficult with a few interactions. Overall, DR-MAML can well control the proportional worst case performance.\n\n***\n**4. Whether method works for meta learning over a large task space**\n\nThanks for the question. You are right, and we consider the distribution of tasks. The combinatorial complexity of tasks is high, meaning a large task space in our experiments. \n\n***\n**5. Moving appendix contents to the main paper**\n\nThanks for the comment. We’ll move the appendix pseudo code, the appendix K “Note that when $\\alpha$ …”, discussions on Table 4/5 to the main paper.\n\n***\n**6. Other questions**\n\n(1) Regarding impacts of assumptions 1/2/3, these support the derivation of remarks, propositions, and theorems. Their roles are reflected in Appendix E-H.\n\n(2) Regarding 3 runs in image classification experiments, there are two reasons: (a) for fair comparison, we keep all the experimental setups the same as TR-MAML. (b) another comes from the relatively stable classification performance in different time-consuming runs.\n\n***\n**7. Typos**\n\nThanks for the comment. We revised the “plaint” to “plain” in Line34; “sometime” to sometimes” in Line 199; a to an in Line 208, Figure4 caption “DR-MAMAL” to “DR-MAML” in Figure4 caption Line 2; and Appendix “Risk-Rweighted” to “Risk Reweighted” in Line 859. \n\n**Reference**\n\n[1] Finn, C., Abbeel, P., & Levine, S. (2017, July). Model-agnostic meta-learning for fast adaptation of deep networks. ICML.\n\n[2] Zintgraf, L., Shiarli, K., Kurin, V., Hofmann, K., & Whiteson, S. (2019, May). Fast context adaptation via meta-learning. ICML.\n\n[3] Schulman, J., Levine, S., Abbeel, P., Jordan, M., & Moritz, P. (2015, June). Trust region policy optimization. ICML.\n\n***\n**8. Particular thanks**\n\nWe express particular gratitude to \\# Reviewer Wn6T for carefully proofreading throughout the main and the appendix. This indeed means a lot of effort and helps improve the quality of the manuscript. We hope your concerns are well addressed.\n"}]

Author rebuttal:
We sincerely thank all reviewers and area chairs for wonderful work. In this response, I will summarize reviews, address confusion/concerns, and report future changes in the manuscript.
&nbsp;
***
### **I. Positive Comments**

These include: (1) *a novel meta learning pipeline/objective* **[\# Reviewers YmWX/ANpj/vUMr]** to solve interesting problems **[\# Reviewers NA6N]**; (2) *a well-written paper* with good presentation **[\# Reviewers YmWX/ANpj/NA6N/vUMr/Wn6T]**; (3) *extensive experimental results* with good ablations **[\# Reviewers YmWX/ANpj/vUMr]**; (4) lemmas/theorem with well supported proofs **[\# Reviewer ANpj]**.
&nbsp;
***
### **II. Primary Confusion and Concerns**

**1. Require point navigation tasks and DR-CNP results**

Thanks for these constructive comments. 

(1) We’ve performed experiments in point robots for navigation tasks [1]. 
New results as **Table 7**. Meta-testing step-wise rewards in 4 runs in point robot navigation tasks (the higher, the better).

|  Methods     | Average | Worst | $\text{CVaR}_{\alpha}$ |
|--------------|---------|-------|------------------------|
| MAML          | $-16.6(\pm0.19)$     | $-46.9(\pm4.22)$   | $-24.2(\pm0.49)$                    |
| TR-MAML       | $-36.8(\pm0.12)$    | $-81.3(\pm4.35)$   | $-52.1(\pm0.89)$                    |
| DR-MAML(Ours) | $-15.3(\pm0.49)$     | $-46.7(\pm3.24)$   | $-21.6(\pm0.79)$                    |

Please refer to **Response to \# Reviewers Wn6T** and attached PDF for more details and analysis.

(2) We’ve performed experiments in Gaussian Process regression [2] using developed DR-CNPs. 
New results as **Table 6**. Meta-testing point-wise log-likelihood results in 5 runs (the higher, the better). 

|  Methods     | Average            | Worst              | $\text{CVaR}_{\alpha}$ |
|--------------|--------------------|--------------------|------------------------|
| CNP          | $-0.22(\pm0.04)$      | $-1.35(\pm0.38)$      | $-0.50(\pm0.11)$          |
| TR-CNP       | $-0.80(\pm0.01)$      | $-1.08(\pm0.02)$      | $-0.87(\pm0.01)$          |
| DR-CNP(Ours) | $-0.21(\pm0.02)$ | $-0.74(\pm0.08)$ | $-0.37(\pm0.03)$     |

Please refer to **Response to \# Reviewers YmWX** for implementation and discussions. *Visualized results are in attached PDF files*.

**2. Lack of the Group DRO baseline [3] and discussions [\# Reviewer NA6N]**

Thanks for this advice. ***The Group DRO was in the original submission, corresponding to Appendix K and the reference (Sagawa et al., 2020) [3].***
Please check the contents in *Line 843*:
> Additionally, we use DR-MAML as the example and perform the comparison between our two-stage algorithm and the risk reweighted algorithm (Sagawa et al., 2020)...

**(1) Experimental Results.** The group DRO (Sagawa et al., 2020) [3] experimental results were reported in Table4/5 together with connections. The implementation of the group DRO is the same as (Sagawa et al., 2020) [3]. We’ll rename the DR-MAML (Risk-Reweighted) with DR-MAML (Group DRO).

**(2) Research Motivations.** Ours is similar to the group DRO method, and the difference is that ours focuses on the tail risk only while the group DRO considers the worst group in a risk reweighted way.

*As missing the Group DRO is the major concern, we hope the misunderstanding can be well resolved this time.*

**3. Incremental to TR-MAML and Misconception of Tail Risk [\# Reviewer Wn6T]**

These are contribution clarification and tail risk misconception issues.

**(1) Concerning contributions, our work differs from TR-MAML a lot**: (a) define distributionally robustness for fast adaptation rather than worst-case in TR-MAML; (b) heuristic two-stage optimization algorithms with improvement guarantee; (c) increased robustness without sacrificing the average performance in contrast to TR-MAML. Hence, *the method is new with theoretical contributions and can improve adaptation robustness*. Though the experimental set-up is incremental w.r.t. TR-MAML, we provided extensive new experimental results, e.g., DR-CNP and Meta RL, after taking advice. 

**(2) Concerning the tail risk concept in this paper**, it is from the literature work [4-5] and the “tail risk” Wikipedia. We hope such a misconception is addressed.

Other answers to questions, clarifications, and misconceptions are attached in separate reviewers’ responses.

**Reference**

[1] Finn, C., Abbeel, P, & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. ICML.

[2] Garnelo, M., Rosenbaum, D, Maddison, C., Ramalho, T, Saxton, D., Shanahan, M., ... & Eslami, S. A. (2018). Conditional neural processes. ICML.

[3] Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. (2020). Distributionally robust neural networks. ICLR.

[4] Sarykalin, S., Serraino, G., & Uryasev, S. (2008). Value-at-risk vs. conditional value-at-risk in risk management and optimization. Informs Tutorials in Operations Research.

[5] Hong, L J, & Liu, G. (2009). Simulating sensitivities of conditional value at risk. Management Science.
&nbsp;
***
### **III. Future Changes in the Manuscript**

**1. Additional experimental results**

In Appendix K, *we’ll include the above results and discussions*.

**2. Rename DR-MAML (Risk-Reweighted) with DR-MAML (Group DRO) in Table4/5**

We’ll rename the baseline and move this result to the main paper. 

**2. More discussions in literature work [\# Reviewer vUMr]**

We’ll cite and discuss references, see Response to \# Reviewer vUMr for details.

**3. Move the appendix contents to the main paper**

We’ll move Pseudocode in Appendix B/Table4/Table5 result analysis to the main paper.

**4. Typo revisions**

We revised “plaint” to “plain” in Line34; “sometime” to sometimes” in Line 199; a to an in Line 208, Figure4 caption “DR-MAMAL” to “DR-MAML” in Figure4 caption; and Appendix “Risk-Rweighted” to “Risk Reweighted” in Line 859. 
&nbsp;
***
Once again, thank all reviewers and area chairs and your effort means a lot in improving our manuscript.


Meta-review: The paper addresses tail risk in meta-learning, i.e. the performance on difficult tasks, by optimizing an approximation of conditional value at risk. The practical algorithm contains two stages, one for estimating the threshold for conditional value at risk via meta-batches, followed by gradient descent. The practical experiments show the efficacy of the proposed method.

The reviewers initially raised several issues regarding the methods’ motivation and requested additional experiments on additional methods/settings. After rebuttal, most concerns are addressed and all reviewers are in favor of acceptance.
Decision: Accept (poster)