Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning
Authors: Yiqin Lv
Cheems Wang
Dong Liang
Zheng Xie
Venue: NeurIPS 2024 poster (2024)
Keywords: Meta learning
Few-shot learning
Fast adaptation robustness
Primary area: other

Abstract: Meta learning is a promising paradigm in the era of large models and 
task distributional robustness has become an indispensable consideration in real-world scenarios.
Recent advances have examined the effectiveness of tail task risk minimization in fast adaptation robustness improvement \citep{wang2023simple}.
This work contributes to more theoretical investigations and practical enhancements in the field.
Specifically, we reduce the distributionally robust strategy to a max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate.
In the presence of tail risk, we further derive the generalization bound, establish connections with estimated quantiles, and practically improve the studied strategy.
Accordingly, extensive evaluations demonstrate the significance of our proposal in boosting robustness.

Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning
Yiqin Lv Qi Wang" Dong Liang'
Zheng Xie
College of Science, National University of Defense Technology
Changsha, China
Email to: flvyiqin98, wangqi15, ionggiinuu xiezheng81 (annenngngnnuut..... Janudt. edu.
Abstract
Meta learning promising paradigm in the era of large models and task dis- tributional robustness has become indispensable consideration in real-world scenarios. Recent advances have examined the effectiveness of tail task risk mini- mization in fast adaptation robustness improvement [1]. This work contributes to more theoretical investigations and practical enhancements in the field. Specifically, we reduce the distributionally robust strategy to max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate. In the presence of tail risk, further derive the generalization bound, establish connections with estimated quantiles, and practically improve the studied strategy. Accordingly, extensive evaluations demonstrate the significance of our proposal and its scalability to multimodal large models in boosting robustness.
Introduction
The past few years have witnessed a surge of research interest meta learning due its great potential in the academia and industry [2-5]. By leveraging previous experience, such a learning paradigm can extract knowledge as priors and empower learning models with adaptability to unseen tasks from few examples [6].
Nevertheless, the investigation of the robustness needs be more comprehensive from the task distribution perspective. In particular, the recently developed large models heavily rely on the few- shot learning capability and demand robustness of prediction in risk-sensitive scenarios [7]. For example, when the GPT-like dialogue generation system [8-10] comes into medical consultancy domains, imprecise answers can cause catastrophic consequences to patients, families, and even societies in real-world scenarios. In light of these considerations, it is desirable to watch adaptation differences across tasks when deploying meta learning models and promote task robustness study for meeting substantial practical demands.
Recently, Wang et al. [1] proposes increase task distributional robustness via employing the tail risk minimization principle [11] for meta learning. In circumventing the optimization intractability in the presence of nonconvex risk functions, two-stage optimization strategy is adopted as the heuristic to solve the problem. In brief, the strategy consists of two phases in iteration, respectively: (i) estimating the risk quantile VaR,, [11] with the crude Monte Carlo method [12] in the task space; (ii) updating the meta learning model parameters from the screened subset of tasks. Such a strategy is simple in implementation, with an improvement guarantee under certain conditions, and empirically shows improved robustness when faced with task distributional shifts. Despite these advances, there remain several unresolved theoretical or practical issues in the field.
*Correspondence Authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Existing limitations. This paper also works on the robustness of fast adaptation in the task space and tries to fill gaps in [1]. Theoretically, we notice that in [1] (i) there constitutes no notion of solutions, (ii) it lacks an algorithmic understanding of the two-stage optimization strategy. (iii) the analysis on generalization capability is ignored in the tail risk of tasks. Empirically, the use of the crude Monte Carlo might be less efficient in quantile estimates and suffers from a higher approximation error of the VaR., degrading the adaptation robustness. These bottlenecks may weaken the versatility of the two-stage optimization strategy's 's use in practice and require more understanding before deployment.
Primary contributions. In response to the bbove-mentioned concerns, we propose translating the two-stage optimization strategy for distributionally robust meta learning [1] into max-min optimization problem [13]. Intrinsically, this work models the optimization steps as Stackelberg game, and task selection and the sub- -raadiidtent optimizer work as the leader and follower players in secision-making, respectively. The theoretical understanding is from two aspects:
1. We constitute the local Stackelberg equilibrium solution concept, estimate the convergence rate, and characterize the asymptotic behavior in learning dynamics.
We derive the generalization bound in the presence of the tail task risk, which connects quantile estimates with fast adaptation capability in unseen tasks.
Meanwhile, the empirical influence of VaR, estimators is examined, and we advance meta learners' robustness by comprising more accurate quantile estimators.
Literature Review
2.1 Meta Learning
Meta learning, or learning to learn, is an increasingly popular paradigm to distill knowledge from prior experience unseen scenarios with few examples [6]. Various meta learning methods have emerged in the past decade, and this section overviews some dominant families.
The baaee methods mainly use the encoder-decoder structure and represent tasks by latent variables. Typical ones are in the form of the conditional exchangeable stochastic processes and learn function distributions, such as neural processes [14], conditional neural processes [15] and their extensions [16-24]. The optimization- based approaches seek the optimal meta initialization model parameters and update models from a few examples. Widely known model agnostic meta learning [25] and related variants [26-29], such as MetaCurvature [30], which learns curvature information and transforms gradients in the inner-loop optimization. The metrics-based methods represent tasks in geometry and perform well in few-shot image classification [31-33]. For example, MetaOptNet [34] proposes to learn embeddings under a linear classifier and achieve SOTA few-shot classification performance. There also exist other methods, e.g., hyper-networks [35, 36), memory-augmenteed networks [37] and recurrent models [38].
2.2 Robustness Generalization
The robustness concept in meta learning attracts recent attention, particularly when deploying large models in real-world scenarios. Admittedly, previous literature works have investigated the scenarios where the meta dataset's ' input is corrupted [39, 40] or the model parameter is perturbed [29]. Studies regarding the fast adaptation robustness in task distribution remain limited. Wang et al. [41] explicitly generates task distribution for robust adaptation. Collins et al. [42] employs the worst-case optimization for promoting MAML's robustness to extreme worst cases. With the help of tail risk minimization, Wang al. [1] proposes two-stage optimization strategies to robustify the fast adaptation. This work centers around [1] but stresses more theoretical understandings and performance improvement points.
As for generalization capability, there are a couple of works in meta learning. Chen al. [43] exploits the information theory to derive the bound for MAML's like methods. From the data splitting perspective, Bai al. [44] formulates the theoretical foundation and connects it to optimality, an average risk bound is constructed with the bias for improving performance. Importantly, prior work [1] ignores the generalization analysis, and meta learner's generalization in tail risk cases has not been studied in the literature.
[45],

---

3 Preliminaries
General notations. Let p(r) be the task distribution in meta learning. We respectively express the lask space and the model parameter space as S2, and e We denote the complete task set by T and refer D- as the meta dataset.
For instance, 2, comprises a collection data points (( 34) )=1 in regression. D, is ususally prepared into the support set D for skill transfer and the query set D4 to assess adaptation perfor- mance. Take the conditional neural process [15] example, D = {(1,, y)) works for task representation with D {(1,, H the all data points to fit in regression.
The meta risk function corresponds to map D, x x + R evaluating fast adaptation performance. Given P(T) and meta learning model parameters 8, we can induce the cumulative distribution of the meta risk function value in the real space as Fe := ([{(()) 0) < 1; }), but there is no explicit parameterized form for Fg in practice as Fe is O-dependent.
T,7 E R R+}).
When it comes the tail risk minimization, commonly the conditional value-at-risk (CVaR,)) with the probability threshold a E [0, 1). The quantile of our interest is called the value-at-risk (VaR,) [1]] with the definition: VaR, [e(T, 0)] infier (11 2 a, T}. The resulting
normalized cumulative distribution F7 (t; 0) is defined as:
1< VaR,(((T,0) P (1;0) = Fall:)-aa
00
12 VaR. ((T,( 0)].
v0 E O, the meta learning operator Mo defines: :T 1(D D;; : Accordingly, the tail risk task subspace Paar == U22 VaR, ( [E[T.9]] [Mp'(C) with the task distribution constrained in Pa,r by (T( 0). Please refer to Fig. 7 for illustrations of risk concepts.
Assumption 1. To proceed, we retain most assumptions from for theoretical analysis, including:
1. The meta risk function E(D? D 8) is B, Lipschitt continuous w.r.t. 8;
2. The cumulative distribution Fe(1; 0) is -i-Lschchitz continuous w.r.t and the normalized density function Pa (T 0) is Be-Lipschitz continuous w.r.t. 0;
3. For arbitrary valid 0 E @ and corresponding Pa (T; ((D,, 00: is bounded: supren..r ((D9 D ; 0) s f.axx.
3.1 Risk Minimization Principles
This subsection revisits commonly used risk minimization principles in the meta learning field.
Expected risk minimization. The standard principle the expected/empirica risk minimization originated from statistical learning theory [46]. It minimizes meta risk based on the sampling chance of tasks from the original task distribution:
min E()) = : (:: 0)
#Ee
Worst-case risk minimization. Noticing that the worst fast adaptation can be disastrous in some risk sensitive scenarios, Collins et al. [42] proposes to conduct the worst-case optimization in meta learning:
min max Ew (0) L(D9 2 +) lE
(2) However, as observed from experiments [42]. such a principle inevitably sacrifices too much average performance for gains of worst-case robustness. Mcanwhile, it requires a couple of imple- mentation tricks and specialized algorithms stabilizing optimization.
Expected tail risk minimization (CVaR.,). To balance the average performance and the worst-case performance, Wang et al. [1] minimizes the expected tail risk, or equivalently CVaR, risk measure:
min (,) = Ep, (r:0) (000),07:0)
3

---

Due to no closed form of Pa (T; Wang et al. [1] introduces a slack variable & E IR and reformulates the objective as follows:
min Ea(0 EERRER
1- vod3 = a Bo(r) [1(09] ((:0)
1
1
(4)
where 0g = (B) denotes the quantile statistics and (109,25:0) - max 0. is the hinge risk.
The optimization objective involves the integral of quantiles in a continuous interval (a, 1]. which is intractable to precisely parameterize with neural networks. The form in Eq. (4) utilizes the duality trick [11], enabling tractable sampling from the complete task space.
3.2 Examples & Two-stage Heuristic Strategies
Before delving deeper into the theoretical issues, we first present DR-MAMI. [1] an instantiation to explain the expected tail risk minimization.
Example 1 (DR-MAML [T]. Given P(T) and vanilla MAML [25], the distributionally robust MAML within CVaRa can be written as a bi-level optimization problee::
min E + -
Pp(r)
- ]
OEG
where the gradient update w.r.t the support set Vol(D 0) indicates the inner loop with a learning rate X. The outer loop executes the gradient updates w.r.t. Eq. (5) and seeks the robust meta initialization in the parameter space.
Two-stage optimization strategies. Without loss of generality, we further detail the computa- tional pipelines of Example with two-stage optimization strategies. Note that MAMI. [25] is an optimization-hased meta learning method, and the implementation is to execute the sub-gradient descent over a batch of tasks when updating the meta initialization gmeta
07: O
A1 VoE(D 0), i = I,..., B
(6a)
(a),
(6b)
8(T,)
11( To: 07-) E) i i= 1,...,B B
(6c)
@
A2 Volō(T)) ((D
(6d)
Here, A and λ are the inner loop and the outer loop learning rates, and the subscript records the iteration number, with dT,,) the indicator variable.
Meto Lecriigg Adiphetion
PMCB is the empirical distribution with B Monte Carlo task samples. 8(T;) indicates the meta risk 1(D8; of.) after fast adaptation falls into the defined tail risk region, otherwise 8(T;) = O.
Mato Bunch
Seb abott for Mete Rarwerterr Spdan
Leader's Mone Girage- Qy. (2) silsert de actiaii Tuck
Pollwer's Meve (frage
Throughout optimizing DR-MAME Stage-I includes the fast adaptation w.r.t. individual task in Eq. (6a), and the quantile estimate in Eq. (6b). Stage-II ap- plies the sub-gradient updates the model parame- ters in Eq. (6c)/(6d). These two stages repeat until convergence is achieved.
Figure I: Illustration of optimization stages in distributionally robust meta learning from Stackelberg game. Given the DR- MAML example, the pipeline can be inter- preted as bi-level optimization: the leader's s move for characterizing tail task risk and the follower's 's move for robust fast adaptation.
Theoretical Investigations
This section presents theoretical insights into two-stage optimization strategies. We perform analysis from the algorithmic convergence, the asymptotic tail risk robustness, and the cross-task generalization capability in meta lcarning.

---

Main Empirical Results [Extensive Evaluation]
S++
gm
VaR, ()
Estmmatio
CDF Approx. Error with KDE [Theorem 4.4]
F(e-a, 0
Fi
Gemeralization Tail Risk [Theorem 4.3)
Surrogate Function Optimization
0 Equilibrium
Asymptotic Performance Gap [Theorem 4.2]
Convergence Rate [Theorem 4.1]
Fla-1.0lasss 2
Flam 0) a-d,
Improvement Guarantee
Existence & Solution Concept [Proposition 2]
Stackelbeerg Game for Meta Learning Robustification
The sketch of theoretical and empirical contributions in two-stage robust strategies. On the left side the two-stage distributionally robust strategy []]. The contributed theoretical understanding is right-down, with the right-up the empirical improvement. Arrows show connections between components.
Figure 2:
4.1 Distributionally Robust Meta Learning Stackelberg Game
Implementing the two-stage optimization strategy meta learning requires first specifying the stages' order. The default the minimization of the risk measure wet the parameter space after the maximization of the risk measure w.t.. the task subspace. Hence, we propose to connect it to max-min optimization [13] and the Stackelberg game [47].
Max-min optimization. With the pre-assigned decision- making orders, the studied problem can be characterized as:
max min F(g, 0) = Eg(r) ((D), (0 (G)ER.. AEB
where Qa = (q ST - constitutes collection of uncertainty sets [48] over task subspace Tg. and q(T) is the normalized probability density over the task subspace, Note that in the expected tail risk minimization principle, there is closed form of optimization objective Eq. (4) as the tail risk is 8-dependent. is approximately interpreted as the max-min optimization when applied to the distribution over the uncertainty set Q..
= -
a}
Proposition 1. The uncertainty set Qa is convex and compact in terms of probability measures.
Practical optimization is achieved via mini-batch gradient estimates and sub-gradient updates with the task size B in [1]: the feasible subsets correspond all combinations of size [B (1 (1-o) a Also, Eq. (7) is non-differentiable w.r.l q(T), leaving previous approaches [49-52] unavailable in practice.
Stackelberg game & best responses. The example computational pipelines in Eq. (6) can be under- stood as approximately solving a stochastic two-player zero-sum Stackelberg game. Mathematically, such game referred to as SG can be depicted as SG == (Pi, Pr; {g Q.). ). (0 e O); 0)).
Moreover, we translate the two-stage optimization as decisions made by two competitors, which are illustrated in Fig. 1. The maximization operator executes in the task space, corresponding to the leader Pr, in SG with the utility function F(q. 0). The follower PF attempts to execute sub-gradient
updates over the meta learners' parameters via maximizing -F(g, 0).
The two players compete to maximize separate utility functions in SG, which can be characterized as:
sg qt = arg max F. [,, qEQ.
Oct1 arg min min R.g. BE0
[(009
Leader Player
Follower Player
where the leader player PL specifies the worst case combinations from the uncertainty set Q.. and the follower Pr reacts to the resulting normalized tail risk for increasing fast adaptation robustness.
It is worth noting that the update rules in Eq. (8) are also called best rexponses of players in game theory. The above procedures can be deemed the bi-level optimization [53] since the update of the meta learner implicitly depends the leader's last time decision.
5

---

4.2 Solution Concept & Properties
The improvement guarantee has been demonstrated when employing two-stage optimization strategies for minimizing the tail risk in [1]. Furthermore, we claim that under certain conditions, there converges to solution for the proposed Stackelberg game SG. The sufficient evidence is:
1. The two-stage optimization [1] results in monotonic sequence:
Model Updates: ... ( + (q. 0 H Monotonic Improvement ..... 2 F(Q:11,01) F(q 02t1) 2 2.:;
(9a)
(9b)
As S Lmax, the objective E.g ((D9, 29:0) < Lmax naturally holds Vy E Qa and 0 E O.
Built on the boundness of risk functions and the theorem of improvement guarantee, such an optimization process can finally converge [54]. Then, a crucial question arises concerning the obtained solution: What the notion of the convergence point in the game?
To answer this question, we need to formulate the corresponding solution concept in Sg. Here, the global Stackelberg equilibrium is introduced as follows.
Definition 1 (Global Stackelberg Equilibrium). Let (g., 0.) E Qa x θ be the solution. With the leader q. E Qa and the follower 0, E O, (g+ ) is called global Stackelberg equilibrium the following inequalities are satisfied, Vq Qa and 40 E e,
inf F(q,6") S Flg., 0.) F). O'Ee
Proposition 2 (Existence of Equilibrium). Given the Assumption 1, there always exists the global Stackelberg equilibrium as the Definition 1 for the studied Sg.
Nevertheless, the existence of the global Stackelberg equilibrium can be guaranteed; NP-hard to obtain the equilibrium with existing optimization techniques. The same as that in [55], we turn to the local Stackelberg equilibrium as the Definition 2, where the notion the local Stackelberg game is restricted in neighborhood Qa x O' in strategies.
Definition 2 (Local Stackelberg Equilibrium). Let (g.. 0.) E Qa θ be the solution. With the leader q. E Qa and the follower E O, (q., f.) is called local Stackelberg equilibrium for the leader if the following inequalities hold, Aig E Q.,
F(g. 0), where (er(g) = 10 e'FF(q,0) V0 inf F(gr,0) #ESar(1) (q) (ESer(9-)
inf
The nature of nonconvex programming comprises the above local optimum, and we introduce concepts below for further analysis. It can be validated that F(q, 0) is a quasi-concave function w.t.t q. meaning that for any positive number E R., the set { E F(q, 0) > 1} is convex in Q. As result, deduce that there exists implicit function h(-) θ - Qa such that the condition holds h(() with = arg max F(q, 0). For the implicit function h, along with VoF(lg. f), make the Assumption below.
Assumption 2. The implicit function h(()) is Bi--ipschitz continuous w.rt. 0 E θ and V Fg, 0) is B Lipschitz continuous w.r.. q E 2..
4.3 Convergence Rate & Generalization Bound
Learning learn scales with the number of tasks, but the optimization process is computationally expensive [56-60], particularly when large language models are meta learners [3, 61, 62]. In training distributionally robust meta learners, estimating the convergence rate allows monitoring of the convergence and designing early stopping criteria to reach a desirable performance, reducing computational burdens [63]. Consequently, turn to another question regarding the solution concept: What is the convergence rate of the two-stage optimization algorithm?
The runtime complexity for the leader's move can be easily estimated from subset selection, while the analysis for the follower non-trivial. Under certain conditions, we can derive the following convergence rate theorem, where À is the learning rate in gradient descent w.E.t. 0.

---

Theorem 4.1 (Convergence Rate for the Second Player). Let the iteration sequence in op- timization be: + (9-11,0} + {q. 02413
{...8.), with the converged
equilibirum (g., 0.). Under the Assumption 2 and suppose that || oo 0.)1l2 <
1 - -33 we can have limt--oo 110--1-0.112 < 1, and the iteration converges with the rate 18:-6.112
(III XV Fle,, 02)l12 + AB)Bk) when approaches infinity.
Moreover, after executing the two-stage algorithm T time steps and given learned 0 we can establish a bound on the asymptotic performance gap w.r.l. CVaR,, in Theorem 4.2. For expositional clarity, we simplify E D.: 0.), ((D Di (meda). VaR., [e(T, 0.)]. and VaRa [e(T, (1drta as t gieeta VaR,, and VaR meta respectively.
Theorem 4.2 (Asymptotic Performance Gap in Tail Task Risk), Under the Assumption I and given batch of tasks {Ti) we can have
VaR (((T)) P(T2))
CV - (V.) S 3-1 0-11
(10)
a
where Ti = f:: VaRR, Emeta VaR fT : 10 VaR Emeto VaR
For sufficiently large T, the first term can be bounded by small number due to the convergence, and the second term vanishes since limy gmeta !" and lim VaR VaR respectively.
Another crucial issue regarding meta learning lies in the fast adaptation capability in unseen cases. This drives to answer the following question: How does the resulting meta learner generalize in the presence of tail task risk?
To this end, we first define R(O.) = Epu(T) [8*), R((o)) (8.) Σ (T.)
( 05,; and is within that of p(T). p(r.) (DD :.;0a,))) where T3 ~ p(T). Also note that the support of pa (T; 0.) the tail risk generalization.
namely (T; C supp(p(T)). Then we can induce Theorem 4.3 w.E.f.
Theorem 4.3 (Generalization Bound in the Tail Risk Cases). Given collection of task samples {T and corresponding meta datasets, we can derive the following generalization bound in the presence of tail risk:
2( I-a Craax + Vnipp (r) [((D DS 0.)) In (t)
R(O.) < P((0.)+
+
B
(1I)
mmax
2ln ( + 3aB
3(1
a)
where the inequality holds with probubility at least and e E (0, 1), V(1 denotes the variance operation, and Lmax is from the Assumption
In conjunction with the confidence and task batch B of significant size, Theorem 4.3 reveals the generalization bound given the meta-trained parameter 0,. It is also associated with the variance V (T) ( _ -2;00)) 0.)). Besides, also derive specific bound in the case of MAML, and details are attached in Appendix Theorem C.I.
4.4 Practical Enhancements & Implementations
Theorem 4.3 reveals that an accurate estimate of VaR,, yields precise variance (i.e.. V (T) ( T. D ;0),))))) leading to more reliable bounds. Accordingly. this section offers improvements over [i] via utilizing kernel density estimators (KDE) [64] for VaR, 's estimates. Compared to crude Monte Carlo (MC) methods, KDE can handle arbitrary complex distributions, capture local statisties well, and smoothen the cumulative function in a non moramarait way.
Specifically, we can construct KDE with a batch of task risk values {!(29)
T2B D 0))
1(D2 (D t
FEKDE(1;0) = Bhg i=1
dd
(12)
h

---

where K IR RR is kernel function, e-g., the Gaussian kernel, ((x)
expl exp[-||=||>/p> and J Texp(- |- 12/
hy the smoothing bandwidth. Once the KDE is built, it enables access to the quantile from the cumulative distribution functions or numeric integrals. The following Theorem 4.4 shows that KDE serves as reliable approximation for VaR...
Theorem 4.4. Let (i (ER 0)] and (a; 0) VaR 0)]. Suppose that (x) ) lower bounded by constant, Vz. For any E O, with probability at least e, we can have the following bound:
he vB
sup ege
(Fi-KDE 8) - FF (a; (a; So
(13)
log B
As implied, close the distribution approximation gap by adopting smaller, more flexible bandwidth. Additionally, KDE models offer smooth estimate of the cumulative distribution function and require prior assumptions.
Remark 1. In addition to smoothness, flexibility, and distribution agnostic traits, KDE in adoption can enhance the studied method's generalization capability. The crude Monte Carlo used in [1] typically incurs an error of approximately O ((BB) in estimating quantiles [65]. In contrast, that of
KDE is more than O h VB+log
from Theorem 4.4.
5 Empirical Findings
Prior sections mainly focus on the theoretical understanding of two-stage distributionally robust strategies. This section conducts extensive experiments broader range of benchmarks and examines the improvement tricks, e.g., the use of KDE for quantile estimates, from empirical results.
Benchmarks & baselines. We perform experiments on the few-shot regression, system identification, image classification, and meta reinforcement learning, where most of them keep setups the same as prior work [1, 42]. We evaluate the methods from risk minimization principles and corresponding indicators, including expected/empirical risk minimization (Average), worst-case risk minimization (Worst), and tail risk minimization (CVaR,).
MAML mainly works as the base meta learner, and we term the KDE aug-annmetted DR-MAML as DR-MAML+. Then we compare DR-MA MAML+ with several baselines, including vanilla MAML [25]. TR-MAML [42], DRO-MAML [66] and DR-MAML [1].
Smmand bolad
Stsmeed 10-ahe
Penduhum Boled
Paodiium Weeeet
Mincn Sigaoe Brnos
Amerage
Wazri
Anerage
Wisit
CVSR
Amerge
Wizse
Figure 3: Meta testing performance in sinu- soid regression problems (5 runs). The charts report testing mean square errors (MSEs) over 490 unseen tasks [42] with black vertical lines indicate standard error bars.
Figure 4: Meta testing performance in Pen- dulum 10-shot and 20 shot problems runs). Reported are testing MSEs over 529 un- seen tasks with 0.5, where black vertical lines indicate standard error bars.
0.7, where
5.1 Sinusoid Regression
The goal of the sinusoid regression [25] is to quickly fit an underlying function f(x) = A sin(x - B) from K randomly sampled data points, and tasks are specified by B)). The meta-training and testing setups are the same as that in [1, 42], where many easy functions with a tiny fraction of difficult ones are included in the training.
Result analysis. illustrated in Fig. 3, we can observe that DR-MAML+ consistently outper- forms all baselines across average and CVaR,, indicators in the 5-shot case. Though the average performance slightly lags behind DR-MAMI. in the 10-shot case, DR-MAML+ surpasses other baselines in both the Worst and CVaR, indicators. This implies that DR- MR-MAML+ exhibits more
8

---

robustness in challenging task distributions, e.g., 5-shot case. Furthermore, the standard error asso- ciated with our method is significantly smaller than others, underscoring the stability of DR-MAML+
5.2 System Identification
The system identification corresponds to learning a dynamics model from few collected transitions in physics systems. Here, we consider the Pendulum system and create diverse dynamical systems by varying its mass and length 1, with (m. () ~ 4( 1.6], [0.4, 1.6)). A random policy collects transitions for meta training, and 10 random transitions work as a support dataset.
Result & analysis. Fig. 4 shows no significant difference between 10-shot and 20-shot cases. DR-MAML+ dominates the performance across indicators in both cases. Due the min-max optimization, TR-MAML behaves well in the worst-case but sacrifices too much average performance. Within the studied strategies, DR-MAML+ exhibits an advantage over DR-MAML regarding CVaR...
5.3 Few-shot Image Classification
We perform few-shot image classification on the mini-ImageNer dataset [67], with the same setup in [42]. The task is a 5-way 1-shot classification problem. And 64 classes are selected for constructing meta-training tasks, with the remaining 32 classes for meta-testing.
Table 1: Average 5-way 1-shot classification accuracies in mini-ImageNe with reported standard deviations (3 runs). With a = 0.5, the best results are in bold.
Eight Meta- Detanira Tasks
Four Meta-Testing Tasks
Method
Average
Worst
CVaRa
Average
Worst
CVaR,
MAML [25]
70.1
48.01445
63.2.26
46.610.
44.710.7
44.61007
TR-MAMI. [42]
63.2
60.7
62.1
48.5100
45.9
46.640.5
DRO-MAML [66]
67.0=1.2
56.6104
61.6402
49.1+.02
46.620.1
47.240.
DR-MAML [I]
70.2:10.
63.4:10.
67.2:10.1
49.41:0.1
47.110.1
47.5-10.1
DR-MAML+(OUrs)
70.4.0.1
63.8.10.2
67.510.1
49.910.11
47.210.1
48.1:001
Result & analysis. In Table I, methods within two-stage distributionally robust strategy, namely DR-MAML and OR-MALL+, show superiority to others across all indicators in both training and testing scenarios, which is similar to empirical findings in [1]. Interstingly, DR DR-MAML+ and DR- MAML are comparable in most scenarios, and we attribute this to the small batch size in training, which weakens KDE's s quantile approximation advantage.
5.4 Meta Reinforcement Learning
Here, we take 2-D point robot navigation as the meta reinforce- ment learning benchmark in eval-
Table 2: Meta testing returns in point robot navigation (4 runs). The chart reports average return and CVaR, return with 0.5.
Method
Average
CVaRa
uation. The goal to reach the target destination with the help of a few exploration transitions for fast adaptation, and we retain the setup in MAML [25]. In meta
MAML [25]
-21.1 +
-29.2 :: 1.37
DRO-MAML [66]
-20.9 + 0.41
-29.0 + 0.66
DR-MAML [1]
-19.6 + 0.49
-28.9 + 1.20
DR-MAML
-19.2.1 0.44
-28.4.1 0.86
testing, we randomly sample 80 navigation goals and examine methods' navigation performance.
Result & analysis. As reinforcement learning methods fluctuate fiercely in worst-case indi- cators, we only report Average and CVaR, re- turns in Table 2. We observe that using studied strategies in DR-MAML enhances the returns.
Sinnslld Salet
Shassol 10-hot
Auprrttmeten Krvce
E313
E.0G
DR-MAML+ benefits from a more reliable quan- tile estimate and achieves superior performance. The application of distributional robustness to reinforcement learning yields improvements in
2
Teak
Teak
Figure 5: VaR, approximation errors with the crude MC and KDE. We compute the difference between the estimated VaR, and the Oracle VaR, in the absolute value /VaR.- VaRal.
returns.

---

5.5
Assessment of Quantile Estimators
With the meta trained model, e.g., DR-MAML+ sinusoid regression, we collect the testing task risk values with different task batch sizes to estimate the VaR,, from respectively the crude MC and KDE. As observed from Fig. the VaR, approximation error decreases with more tasks, and the KDE produces more accurate estimates with a sharper decreasing trend. 'hee above well verifies the conclusion Theorem 4.3.
5.6 Empricial Result Summarization
Here, we summarize two points from the above empirical results and associated theorems, (i) From Theorem 4.2/4.3 and Fig. 3/4/5: the VaR, estimate relates the reliable generalization bound, and cumulated tiny approximation errors along iterations potentially result in worse equilibrium. (ii) From Theorem 4.3/4.4, Remark Fig. and Table 1/2: with the studied strategy, the KDE is better choice of task risk distribution modelling than the crude MC in tougher benchmarks, e.g., 5-shot sinusoid regression, meta-testing mini- Imane-m classification, and point robot navigation.
5.7 Compatibility with Large Models
tigred-ImagoNe
ImsgoNetA
ImsgoNetSetthh
Clocsiticaion Accuracios
CVaR,
CVaR,
CVaR.
CLIP
MaPLe
DR.MalLee
DR-MaPLe-
Figure 6: Meta testing results on 5-way 1-shot classification accuracies with reported standard deviations runs). The charts respectively report classification accuracies We further conduct few-shot image classification experiments in the presence of large model. Note that CLIP [68] exhibits strong zero-shot adaptation capability; hence, we employ "ViT-B/16"-base CLIP as the backbone to enable few-shot learning in the same way as MaPLe with training setup N CTX and MAX EPOCH 30 [69], scaling to large neural networks in evaluation (See Appendix Section D for details).
150 unseen tasks.
Improved Robustness in Evaluation: As illustrated in Fig. DR-MaPLe and DR-MaPLe+ consistently outperform baselines across both average and indicators in cases, demonstrating the advantage of the wwo-stage strategy in enhancing the robustness of few-shot learning. DR-MaPLe+ achieves better results as KDE quantiles are more accurate with large batch sizes. These results confirm the scalability and compatibility of our method on large models.
Learning Efficiency as Limitations: terms of implementation time and memory cost, we retain the setup the same as that in [1]: use the same maximum number meta gradient updates for all baselines in training processes, which means given a 0.5, the tail risk minimization principle requires double task batches evaluate and screen sub-batches. It can be seen that both DR-MaPLe and DR-MaPLe+ consume more memories, and the extra training time over MaPLe arises from the evaluation and sub-batch screening in the forward pass. Such additional computations and memory costs raise computational and memory efficiency issues for exchanging extra significant robustness improvement fast adaptation.
Conclusion
To conclude, this paper proposes to understand the two-stage distributionally robust strategy from optimization processes, define the convergence solution, and derive the generalization bound in presence of tail task risk. Extensive experiments validate the studied improvement tricks and reveal more empirical properties of the studied strategy. We leave computational overhead reduction as promising topic for future exploration in robust fast adaptation.
10

---

Acknowledgement
This work funded by National Natural Science Foundation of China (NSFC) with the Number 62306326. We express particular gratitude to friends who guide large model-relevant experiments.
References
[1] Qi Wang, Yiqin Lv, Yanghe Feng, Zheng Xie, and Jincai Huang. A simple yet effective strategy to robustify the meta learning paradigm. In Teirth-seventh Conference on Neural Information Processing Systems, 2023.
[2] Yuanfu Lu, Yuan Fang, and Chuan Shi. Meta-learning on heterogeneous information networks for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1563 1573-157,, 2020.
[3] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[4] Yi Yuan, Gan Zheng, Kai-Kit Wong, and Khaled В Letaief. Meta-reiinforeement learning based resource allocation for dynamic v2x communications. IEEE Transactions on Vehicular Technology, 70(9):8964 8977, 2021.
[5] Brenden M Lake and Marco Baroni. Human-lite systematic generalization through a meta- learning neural network. Nature, pages 1-7, 2023.
[6] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44 (9):5149-5169, 2021.
[7] Qi Wang, Yanghe Feng, Jincai Huang, Yiqin Lv, Zheng Xie, and Xiaoshan Gao. Large-scale generative simulation artificial intelligence: The next hotspot. The Innovation, page 1005 16, 2023.
[8] Young-Jun Lee, Chae-Gyun Lim, and Ho-Jin Choi. Does gpt-3 generate empathetic dialogues? a novel in-context example selection method and automatic evaluation metric for empathetic dialogue generation. In Proceedings of the 29th International Conference Computational Linguistics, pages 669-683, 2022.
[9] Chen Tang, Hongbo Zhang, Tyler Loakman, Chenghua Lin, and Frank Guerin. Terminology- aware medical dialogue generation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
[10] Bharath Chintagunta, Namit Katariya, Xavier Amatriain, and Anitha Kannan. Medically aware gpt-3 as data generator for medical dialogue summarization. In Machine Learning for Healthcare Conference, pages 354-372. PMLR, 2021.
[11] R Tyrrlll Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk,
2000.
[12] Dirk P Kroese and Reuven Rubinstein. Monte carlo methods. Wiley Interdisciplinary Reviews: Computational Statistics, 4(1):48-55 58, 2012.
[13] John M Danskin. The theory of max-min, with applications. SIAM Journal on Applied Mathematics, 14(4):641-664, 1966.
[14] Marta Gamelo,, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola. Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint arXiv: / 2018.
[15] Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International Conference Machine Learning, pages 1704-1713, PMLR, 2018.
11

---

[16] Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard Turner. Meta-learning probabilistic inference for prediction. In International Conference on Learning Representations, 2018.
[17] Qi Wang, Marco Federici, and Herke van Hoof. Bridge the inference gaps of neural pro- cesses via expectation maximization. In The Eleventh International Conference on Learning Representations, 2023. URL. tttetps://ppenrvvieee wwt.:neettuuaaee wttf:oorummdddddaaaaeee
[18] Andrew Foong, Wessel Bruinsma, Jonathan Gordon, Yann Dubois, James Requeima, and Richard Turner. Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems, 33:8284-8295, 2020.
[19] Qi Wang and Herke Van Hoof. Doubly stochastic variational inference for neural processes with hierarchical latent variables. In International Conference on Machine Learning, pages 10018-10028. PMLR, 2020.
[20] Muhammad Waleed Gondal, Shruti Joshi, Nasim Rahaman, Stefan Bauer, Manuel Wuthrich, and Bernhard Scholkopf. Function contrastive learning of transferable melaarepresentatiion, In Proceedings of the 38th International Conference on Machine Learning. ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 1765-3765. PMLR, 2021.
[21] Qi Wang and Herke van Hoof. Learning expressive meta-repreesentation with mixture of expert neural processes. In Advances in neural information processing systems, 2022.
[22] Juho Lee, Yoonho Lee, Jungtaek Kim, Eunho Yang, Sung Ju Hwang, and Yee Whye Teh. Bootstrapping neural processes. Advances in neural information processing systems, 33:6606- 6615, 2020.
[23] Qi Wang and Herke Van Hoof. Model-based meta reinforcement learning using graph structured surrogate models and amortized policy search. In International Conference on Machine Learning, pages 23055-23077. PMLR, 2022.
[24] Jiayi Shen, Xiantong Zhen, Marcel Worring, et al. Episodic multi-task learning with heteroge- neous neural processes. arXiv preprini arXiv:2310.18713, 2023.
[25] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. International conference on machine learning, pages 1126-1135. PMLR, 2017.
[26] Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in neural information processing systems, 32, 2019.
[27] Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient-basee based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.089900, 2018.
[28] Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-leaming. Advances in neural information processing systems, 31, 2018.
[29] Momin Abbas, Quan Xiao, Lisha Chen, Pin- Yu Chen, and Tianyi Chen. Sharp-maml: Sharpness- aware model-agnostic meta learning. In International Conference on Machine Learning, pages 10-32, PMLR, 2022.
[30] Eunbyung Park and Junier B Oliva. Meta-curvature. Advances in neural information processing systems, 32, 2019.
[31] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances neural information processing systems, 30, 2017.
[32] Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. Infinite mixture prototypes for few-shot learning. In International Conference on Machine Learning, pages 232-241. PMLR, 2019.
12

---

[33] Sergey Bartunov and Dmitry Vetrov. Few-shot generative modelling with generative matching networks. International Conference on Artificial Intelligence and Statistics, pages 570-678. PMLR, 2018.
[34] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), June 2019.
[35] Jacob Beck, Matthew Thomas Jackson, Risto Vuorio, and Shimon Whiteson. Hypernetworks meta-reinforremmet learning. In Conference on Robot Learning, pages 1478-1487. PMLR, 2023.
[36] Dominic Zhao, Johannes von Oswald, Scijin Kobayashi, João Sacramento, and Benjamin Grewe. Meta-learning via hypemetworks. 4th Workshop Meta- Meta-Leaming NeurIPS 2020, 2020.
[37] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmentee neural networks. In International conference on machine learning, pages 1842-18 1850. PMLR, 2016.
[38] Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel. RI2: Fast reinforcement learning via slow reinforcement learning. arXiv preprint arXiv: /tt::6111.22 1.007779 2016.
[39] Ren Wang, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Tsui- Wei Weng, Chuang Gan, and Meng Wang. On fast adversarial robustness adaptation in model-agnostic meta-learning. In International Conference Learning Representations, 2020.
[40] Micah Goldblum, Liam Fowl, and Tom Goldstein. Adversarially robust few-shot learning: A meta-learning approach. Advances in Neural Information Processing Systems, 33: 13:1768--17895 2020.
[41] Cheems Wang, Yigin Lv, Yixiu Mao, Yun Qu, Yi Xu, and Xiangyang Ji. Robust fast adaptation from adversarially explicit task distribution generation. arXiv preprint arXiv:2407.19525 2024.
[42] Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic meta- learning. Advances in Neural Information Processing Systems, 33:18860-18871 2020.
[43] Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoreti analysis. Advances in Neural Information Processing Systems, 34: 15878-25899 2021.
[44] Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, Jason Lee, Sham Kakade, Huan Wang, and Caiming Xiong. How important the vain-validation split in meta-learning? In International Conference on Machine Learning, pages 543-553, PMLR, 2021.
[45] Giulia Denevi, Carlo Ciliberto. Riccardo Grazzi, and Massimiliano Pontil. Learning-to- lo-ammm stochastic gradient descent with biased regularization. In International Conference on Machine Learning, pages 1566-1575. PMLR, 2019.
[46] Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
[47] Tao Li and Suresh P Sethi. A review of dynamic stackelberg game models. Discrete & Continuous Dynamical Systems-B, 22(1):125, 2017.
[48] Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 59(2):3413377, 2013.
[49] Dimitri Bertsekas. Nonlinear programming. Journal of the Operational Research Society, 48 0):334-334 1997.
13

---

[50] Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for Senconvex-concayye minimax problems. In International Conference on Machine Learning, pages 6083-6093. PMLR, 2020.
[51] Pierre Loridan and Jacqueline Morgan. Weak via strong stackelberg problem: new results. Journal of global Optimization, 8:263-287, 1996.
[52] Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In International conference on artificial intelligence and statistics, pages 1552. PMLR, 2020.
[53] Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):10045-10067, 2021.
[54] Tom M Apostol. Mathematical analysis. 1974.
[55] Tanner Fiez, Benjamin Chasnov, and Lillian Ratliff. Implicit learning dynamics in stackelberg games: Equilibria characterization, convergence analysis, and empirical study. International Conference Machine Learning, pages 3133-3144. PMLR, 2020.
[56] Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 13693-13696, 2020.
[57] David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint 2rXiv:2104.13300 2021.
(58) Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv:2102.01293, 2021.
[59] Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang. Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, al. Summary of chatgpt-related research and perspective towards the future of large language models. Meta-Radiology, page 100017, 2023,
[60] Enkelejda Kasneci, Kathrin Seßler, Stefan Kücheeaan, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. Chatgpt for good? opportunities and challenges of large language models for education. Learning and individual differences, 103:102274 2023.
[61] Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the I 1th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021, pages 1816-3830 Association for Computational Linguistics (ACL), 2021.
[62] Ahmad Faiz, Sotaro Kaneda, Ruhan Wang, Rita Osi, Parteek Sharma, Fan Chen, and Lei Jiang. LImearbon: Modeling the end-to-end carbon footprint of large language models. arXiv preprint arXiv:2309.144 143993, 2023.
[63] Jordan Hoffmann, Sebastian Borgcaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203. 155556 2022.
[64] Mats Rudemo. Empirical choice of histograms and kernel density estimators. Scandinavian Journal of Statistics, pages 65-78, 1982.
[65] Hui Dong and Marvin K Nakayama. A tutorial quantile estimation via monte carlo. Monte Carlo and Quasi-Monte Carlo Methods: MCQMC 2018, Rennes, France, July 1-6, pages 3-30, 2020.
(66) Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference Learning Representations, 2020.
14

---

[67] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances neural information processing systems, 29, 2016.
[68] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PMLR, 2021.
[69] Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fa- had Shahbaz Khan. Maple: Multi- prompt learning. Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 19113-19122, 2023.
[70] Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, and Xiangyang Ji. Supported trust region optimization for offline reinforcement learning. In International Conference on Machine Learning, pages 23829-23851 PMLR, 2023.
[71] Hongchang Zhang, Yixiu Mao, Boyuan Wang, Shuncheng He, Yi Xu, and Xiangyang Ji. In- sample actor critic for offline reinforcement learning. In The Eleventh International Conference on Learning Representations, 2023.
[72] Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, and Xiangyang Ji. Supported value regularization for offline reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
[73] Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszezynski. Lectures stochastic programming: modeling and theory. Society for industrial Mathematics, 2009.
[74] Jianzhun Shao, Yun Qu, Chen Chen, Hongchang Zhang, and Xiangyang Ji. Counterfactual conservative q learning for offline multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
[75] Yun Qu, Boyuan Wang, Jianzhun Shao, Yuhang Jiang, Chen Chen, Zhenbin Ye, Liu Line, Yang Feng, Lin Lai, Hongyang Qin, et al. Hokoff: real game dataset from honor of kings and its offline reinforcement learning benchmarks. Advances in Neural Information Processing Systems, 36, 2024.
[76] Lorna Paredes and Chew Tuan Seng. Controlled convergence theorem for banach-valued hl integrals. Scientiae Mathematicae Japonicae, 56(2):347-358, 2002.
[77] Chi Jin, Praneeth Netrapalli, and Michael Jordan. What is local optimality in nonconvex- nonconcave minimax optimization? In International conference on machine learning, pages 4889. PMLR, 2020.
[78] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press. 2004.
[79] Frappier and QI Rahman. On an inequality of bernstein. Canadian Journal of Mathematics, 14(4):932-944 1982.
[80] Rong Liu and Lijian Yang. Kernel estimation of multivariate cumulative distribution function. Journal of Nonparametrie Statistics, (0(8):661-677 2008.
[81] Srh Larochelle. Optimization as a model for few-shot learning. In International Conference on Learning Representations, 2017.
[82] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenen- baum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv: 1003. 00666, 2018.
[83] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardr, and Dawn Song. Natural adversarial examples. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pages 15262-15271, 2021.
15

---

[84] Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global repre- sentations by penalizing local predictive power. Advances Neural Information Processing Systems, 32, 2019.
[85] Adam Paszke. Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas Köpf. Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Py- torch: An imperative style, high-performance deep learning library. In Advances in Neu- ral Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 8024 8035, 2019. URL wwaccts::/prooeeeeee
neurips......... wwterisscccppppppe hccppappr/2019/haaae abbc:28ffe/19222ff8877777777---3 2tt.:///77777---bttatttt....... Awstract. html..
[86] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. {TensorFlow J: a system for [Large-Scale) machine learning- In 12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265- 283, 2016.
16

---

Contents
Introduction
2
Literature Review
2.1 Meta Learning...
2.2 Robustness & Generalization
3
Preliminaries
3.1
Risk Minimization Principles
3.2
Examples Two-stage Heuristic Strategies
4
Theoretical Investigations
4.1
Distributionally Robust Meta Learning as Stackelberg Game
4.2
Solution Concept & Properties
4.3
Convergence Rate & Generalization Bound
4.4
Practical Enhancements Implementations
5
Empirical Findings
5.1
Sinusoid Regression...
5.2
System Identification
5.3
Few-shot Image Classification
5.4
Meta Reinforcement Learning
5.5
Assessment of Quantile Estimators
10
5.6
Empricial Result Summarization
10
5.7
Compatibility with Large Models
10
6
Conclusion
10
Quick Guide to This Work
19
A.1 Technical Comparison in Robust Fast Adaptation
19
A.2
Significance of Theoretical Understandings
19
A.3 Meanings Indicators and Terms
20
A.4 Computational Complexity
20
A.5 Broader Impact & Future Extensions
20
B Pseudo Algorithms
20
Expressions, Theorems & Proofs
C.I Characterization of Optimization Processes
21
C.2 Assumptions
................................
22
C.3 Proof of Proposition
23
C,4 Proof of Proposition 2
23
17

---

C.5 Proof of Quasi-concavity for F(g. 8) w.r.t. 9
24
C.6 Proof of Theorem 4.1
24
C.7 Proof of Theorem 4.2
................................
25
C,8 Proof of Theorem 4.3
26
C.9 Proof of Theorem 4.4
28
C.10 Additional Theorem
28
D Implementation Details
30
D.I Benchmark Details & Neural Architectures Opensource Codes
30
D.2 Modules in Python
E Additional Experimental Results
32
E.1
Evaluation with Other Robust Meta Learners
32
E.2 Numeric Results in Tables and Histograms
32
E.3 Sensitivity Analysis to Confidence Level
33
E,4 Further Exploration Adaptation
35
Computational Platforms & Softwares
35
18

---

A Quick Guide to This Work
This section mainly includes explanations and clarifications on this work.
A.1 Technical Comparison in Robust Fast Adaptation
Table 3: summary of robust fast adaptation methods. We take MAML as an example, list related methods, and report their characteristics in literature. We mainly report the statistics according to whether existing literature works include the generalization analysis and convergence analysis. The form of meta learner and the robustness type are generally connected.
Principle
Meta Learner
Generalization Convergencel
Robustness Type
MAML
minage Bp(r)
[109, 05:01
DRO-MAML [66]
(axy minsge
(y)r)
X
Uncertainty Set (Not tail risk)
TR-MAML [42]
minsgo mäX
V
Worst-Case Task
DR-MAML [1]
minggo (p.().))
Tail Task Risk
DR-MAML-+(Ur)) m0Xg()) mittese By(r) 05:0)
Tail Task Risk
Primary differences: As far as we know, literature work is quite limited regarding fast adaptation robustness the task space. TR-MAML and DR-MAML are the most recent and typical ones that can handle task distributional shift scenarios well. As reported in Table 3, TR-MAML only focuses on the worst-case, which considers bit extreme and rarely occurred cases. DRO-MAML is new baseline, where the uncertainty set Q is included for robust fast adaptation, hence there exists no theoretical analysis. As for the tail task risk, DR-MAML lacks generalization capability and convergence rate analysis w.r.f. the meta learner. The meta learner in DR-MAML+ is a more specific instantiation of that in DR-MALL. We claim that these theoretical understanding is necessary the presence of the robust fast adaptation due to its potential applications in large models.
Theoretical and empirical insights: In comparison, this work not only contributes to the Stackelberg game for estimates, but also derives the generalization and the asymptotic performance gap in iterations based on a normalized but non-differentiable probability density space. Note that we lean more focus on theoretical understanding and pursuing SOTA performance is not the ultimate purpose of this work. The connections between different quantile estimators and generalization bound highlighted in Theorem 4.3/4.4 and Remark reveal the theoretical advantage of KDEs over crude Monte Carlo methods. This motivates us to replace crude Monte Carlo with KDEs. Such replacement as a simple implementation trick is supported by rigorous theoretical analysis. The empirical results align with thcoretical understanding.
In terms of improving the studied strategy, investigations in extensive experiments seem meaningful for practical implementations, and some non-trivial discoveries together with improvement tricks are also reported, such as the relationship between quantile estimate errors and adaptation robustness, the batch size's influence on several benchmarks, etc.
In this work, the theoretical and empirical parts are connected in an implicit manner. The generaliza- tion capability is empirically examined from experimental results, and the performance gap between DR-MAML+ and DR-MAML can be attributed to the difference in generalization bounds. As for the convergence trait and asymptotic performance, the insight might guide the optimization process in training large models, such as early stopping criteria design.
A.2 Significance of Theoretical Understandings
As pointed out in [3, 61, 62], large language models are few-shot learners. When large model, such as a large diciision-maiing model in the future, comes into practice fast adaptation robustness can be crucial issue as real-world scenarios are indeed risk-sensitive.
This work takes the latest work [1] as an example, and the interest is in the theoretical aspect. Most of the assumptions in this work are from [1]. The baselines are typical and latest, while the benchmarks
19

---

cover diverse downstream tasks. In multimodal few-shot image classification experiments, our contributed points help guide the development of large models terms of training and robustness enhancement. Our investigations also provide insight into robust policy optimization, particularly when safety one necessary consideration [70-72].
Shapiro et al.'s book [73] is comprehensive resource that addresses stochastic modeling and optimization methods, but does not explore solution concepts in game theory define generalization bounds relevant to meta learning and deep learning. Instead, our work further enriches the stochastic programming theory in meta learning, connects it to the Stackelberg game, and contributes to tail risk generalization bounds, convergence rates, asymptotic properties, and so on. Therefore, the solution concept and the theoretical properties are specific our meta learning setup, distinctly from the scope covered by [73].
A.3 Mcanings of Indicators and Terms
Illustration of VaR.,. CDF and others: Fig. illustrates a typical probability distribution, cumula- tive distribution, and the resulting mean, VaR,, and CVaR,,, Given E [0. 1), VaR,, is the quantile of the risk distribution. Specially, VaRo.5 coincides with the mean. Upon the definition of VaRa.
CVaR,, can be define CVaR,, Ep(r) E/e VaR.] That is, CVaR, is the expectation of the risks of the 1 - a tail of the distribution. Relative to the original probability distribution, CVaR can be interpreted as certain distribution shift, which reweighs arbitrary risk exceeding VaRa, up to coefficient T-a'
Meaning of the asymptotic performance gap: We plot Fig. 8 display the gap between the CVaR, value in iterations and that in the convergence. The area difference depicts this gap.
A.4 Computational Complexity
Analyzing computational complexity across meta-learning methods inherently challenging due the diversity in methodological approaches within the field. Meta-learning encompasses a wide range of techniques, including gradient- based methods, which rely on iterative updates to model parameters, and non paramermetticc methods, which may instead focus instance-bssd learning or kernel-based approaches. Therefore, the space complexity is specific to the meta-learning method, while this work is agnostic it. Here, report the computational complexity for the DR-
MAML+ as O BBB alMl)) while using KDE with the Gaussian kernel, and that of DR-MAMI. is O B(log(B) - B(lo((B)-aMMi)) alM))).
A.5 Broader Impact & Future Extensions
This paper presents work whose goal to advance the field robust meta learning. There are many potential societal consequences of our work, which we detail as follows.
The fast adaptation robustness is an urgent concern, particularly in large models and risk-sensitive control. This work provides versatile insights for theoretical analysis and performance improvement the presence of tail task risk, and future explorations can be decision-making scenarios, such as multi-agent policy optimization [74, 75], and computationna//mmmory cost reduction.
B Pseudo Algorithms
For better understanding of the game theoretical optimization, we take DR-MAML+ and DR-CNP+ as examples and include the Pseudo Algorithms 1/2 in this section. Particularly, the algorithms specify the decision-making orders and highlight the use of KDE modules to build task risk value distributions and estimate the quantile.
20

---

Canslalle
Mcan VaR, CVaR,
B86k
Figure 7: Diagram of risk concepts this work. Here, the r-axis is the task risk value fast adaptation given specific O. The shadow-lined region illustrates the tail risk with a probability 1 in the probability density. The area of the shadow-lined region after - a normalization corresponds the expected tail risk CVaR..
Asyugrotic Perfirmance Geg level Optimizzties
Prequencis
{t: VaRg, puadu VaRijdia
Vall
VaRt
Rik !
2 VaR pucto pucto
VaRaicta
Figure 8: Illustration of the asymptotic behavior in approximating the equilibrium. Here, the r-axis is the feasible task risk value in fast adaptation. The dark blue region indicates the histogram of the task risk values in the local Stackelberg equilibrium (g-, 0.). The shallow blue region describes the histogram of the task risk values at some iterated point (qr-1, @ppii) The sets Ti and To respectively collect the tasks resulting the opposite order.
C Expressions, Theorems & Proofs
C.1 Characterization of Optimization Processes
Without loss of generality, we can also express the process of solving the studied Stackelberg game as:
max F(g, (x) s.t. (g) = arg min F(g, 0) 2
(13a: Leader's Decision-Making
min Flg 8EB
(13b: Follower's Decision-Making)
where the optimization w.r.. (q. 0) is the computation the best responses for two adversarial players, As bi-level optimization in Eq. (14a)/(14b)) the exact solution is intractable to obtain in theoretical sense, and the two-stage distributionallly robust optimization is a heuristic approach.
Meaning of the obtained equilibrium. Here, we can interpret the obtained solution (g., 0,) from solving Eq. (7) as follows. Given the follower's decision 0, and the induced task risk distribution Fe(1; 0.). the leader cannot further raise proposal of task subset with probability degradde the tailed expected performance. And this explains the meaning of robust fast adaptation solution w.E.. the tail task risk.
to
21

---

Algorithm 1: Meta-training DR-MAML+ as A Stackelberg Game
Input :Tsk distribution p(T); Confidence level a; Taas batch size Learning rates: A and λ2.
Output Meta-trained model parameter
Randomly initialize the model parameter 0; while not converged do Sample batch of tasks {Ti p(7);
# The Leader Player's Decision-Making for to do
// inner loop via gradient descent as the fast adaptation
Evaluate the gradient: Vol(D 0) in Eq. (5);
Perform spek-ie gradient updates:
0, 0 0 V of (D
end
// model the task risk distribution and estimate quantile Evaluate performance LB = {E(D), 0,))
Estimate VaR, [<(T, 0) and set Ea in Eq. (5) with kernel density estimators; Screen the subset fg = {1(D 2 0,) with &. for meta initialization updates; # The Follower Player's Decision-Making
Execute outer loop via gradient descent increase adaptation robustness:
- A2 Vo
122 0.) Eq. (5);
end
Algorithm 2: Meta Training OR-CNP+ as A Stackelberg Game
Input Task distribution p(T); Confidence level a; Task batch size B; Learning rate X. Output Meta-trained model parameter Randomly initialize the model parameter 8; while not converged do
Sample batch of tasks {T. ~ p(T); # The Leader Player's Decision-Making
// model the task risk distribution and estimate the quantile
Evaluate performance L8 i.)));
Estimate VaR. ((( x & with kernel density estimators;
Screen the subset Lg {E(D) ; 0) with & for meta initialization updates;
# The Follower Player's 's Decisi sion-Makingan Execute gradient descent increase adaptation robustness:
0 0 XVo D
((D) 7 2))
end
C.2
Assumptions
We list all of the assumptions mentioned in this work. These assumptions further serve the demon- stration of propositions and theorems the main paper.
Assumption 1. To proceed, we retain most assumptions from [l] theoretical analysis, including:
1. The meta risk function (DD.DD. is B, Lipschiit continuous w.r.t. 8;
2. The cumulative distribution Fe(1; 0) is L-piscccittz continuous w.r.t. I, and the normalized density function (T] 0) is Sg-Lipsehitz continuous w.n.t. 0;
3. For arbitrary valid 0 E @ and corresponding Pa (T; ((D 22:0) is bounded: sup 10 D 5 0) S Lmax-
Assumption 2. The implicit function h(.)) is Lip-iihhhtt continuous w.r.t. 0 E O and V VoF(g, LipLipschizz continuous w.r.t. q Q..
22

---

C.3 Proof of Proposition 1
Proposition The uncertainty set Qa convex and compact terms of probability measures.
Proof:
We firstly focus on the convexity of Qa- For any (q1 == (2 == 92(r)} E Ca. we partition these two task spaces with non-zero sampling probability mass respectively as Ti UT and T L Tc. As displayed in Fig. 9, To denotes the shared subset task between and q2. Below we show that AiyI X292 E 2. with A + X2 1. This true because
JEET p(T)dT
(15a)
SEETA UT22
p( + I (NeD(t) + >xpl()) dr S dep(T)dr+ Sters d2p(T)dr
JTETO
TET,
(15b)
+ (f TETO
p(T)dT++ + / p(T)ar) A2 ( p(r)dT+ f p(T)dr)
(15c)
FET,
JETTS
/ TET
X1
p(r)dt+ + X2
p(r)dT
(15d)
TET
1-a. a.
(15e)
We next demonstrate the compactness of Ca- The distance between two distributions V4n,, 92 E Qa can be defined as:
de (91, 92) = [ [(((T) 92(T) dr.
Since L' space is Banach space, the compactness is equivalent to the closedness and Boundedness of Q.. Considering a sequence (T)(T) E @.) with the resulting limitation is (r(,,, following the Controlled Convergence Theorem [76), we know that
lim ( Pa (T( (r(rddr= < lim / n-000
a-+0e FET [Pn(T) -p.(r) lim do (gn. 0. (16) n=000
TET
Due to the symmetry of the distance, we can have lim SrET Pa(T) - Pn (T)ddd S 0. Thus,
[ P.p..() (.(ddt=== lim n=
/ Pn(T)dr = ak.
JET
That is, (T) Q.. indicating that Q. is a closed set. As the boundedness is clear in the studied problem, this completes the proof of Proposition 1.
C.4 Proof of Proposition 2
Proposition (Existence of Equilibrium) Given the Assumption 1, there always exists the global Stackelberg equilibrium the Definition 1 for the studied SG.
Proof: Note that θ is compact subspace of the Euclidean space. And it is trivial to see that F(g. 0) = E, ((D). :E: 0) is continuous w.r.t. 0 θ satisfies the 3, -ipschitz continuity in the Assumption 1.
Here we need to show the continuity of F(g, 0) w.r. the collection of probability measures or probability functions e.. To this end, with v0 fixed, We consider two metric spaces (Q., do.) and (C, R.. The map of our interest is g(g) F(g.-) Qa + L C C B
23

---

T
T1
To
T2
Figure 9: Partition of the task subspace. Here we take two probability measure (41. 92) E Q. for illustration. Ti Tc and T2 U TC defines the corresponding task subspaces for q1 and d2 with non-zero probability mass in the whole space T.
Naturally, we can have the following inequality:
s((i)) s(12)
En ((D,, 25: 0]] - -E (DD...... 00
25:
(17a)
| TETO (a)(T) [2(T]](())))) : 0)dT
-
(17b)
+ +1 TEn
@1 (f;:0T
(2(T)
(17c)
TET
1 TETO
$
(1(T)) (2(T) (29,2);0)
(17d)
f JET (T((T) ( ( (f; 0)d
(I((T) 92(7) ( 25;
(17e)
JETS
JET (T((T) (o(()))((T((e dr 3L (41,
(17f)
which implies 3Cmax-Lipschit. continuity of g(4) w.r.t. 4q E Qa.
According to the Remark in [77], there always exists the global Stackelberg equilibrium as the Definition when Qa θ compact and F(g. 0) continuous. This completes the proof of Proposition 2.
C.5 Proof of Quasi-concavity for F(g, 0) w.r.f. q
can be validated that F(g. 0) is a quasi-concave function w.r.f. meaning that for any positive number E R4,, the set {glg- E F., 0) 1) is convex in Qa-
Proof: According to the conventional definition (i.e., the superlevel set is convex [78]), for all d A2 = 1, 91, 92 E (q)FF(g,0) 1}. we can have
F(A1q A292. 0) E Ex, Auga [(09, 2.:
(18a)
Euy (00,,D5: 0)
+ Aup [(0),
(18b)
A).((110)) X2
(18c)
> L.
(18d)
Thus, \iq + A292 E (q)F(g. 1} and the superlevel set convex, implying that F(g. 8) is quasi-concave w.r.t. q.
C.6 Proof of Theorem 4.1
Theorem 4.1 (Convergence Rate for the Second Player) Lei the iteration sequence in opti- mization be: ... (4- + {gr, 0++1} + + (g,, 0.), with the converged equilibirum (g.,0.) 0.). Under the Assumption and suppose that [I Alg 0+)112 <
24

---

1 Ahr can have lime-soo 110--1-0.1e 18.-8.112
< and the iteration converges with the rate
(III 2o 0.)112 + Sn).
Proof: Let the resulting stationary point be (qx, 0z]. we denote the difference terms by q = q q+ and = 0.. Then, according to the optimization step, we can have the following equations:
De+1 0, -V = O.+1 = 0, AVaF(ai
(19)
Now we perform the first-order Taylor expansion of the 0 related function VaF(qr; around 0, and can derive:
VoF(g;; VaF(ge: v 0.) 0(1 0.11) VoFlge 02) Vop
(20a)
Vo.F( 0.).
(20b)
Then we have the following result with the help of Assumption 2:
VoF( (VF((a;))) - VoF(q.;
(21a)
[VF((h()))))) ())(
(21b)
S Bode. (h(0e), h(0,))
(21c)
S 3/ 10: - 0-112-
(21d)
With Eq. (19), Eq. (20) and Eq. (21), we can derive the equation that:
u+fl = 0, AV
(22a)
= 0 veFlgr; 0.) + V For 0.)00 0.)0,
(22b)
[ - A A F 0.) 0. AVoF (9:: 0.)
(22c)
110+1112 S ||| X AV Vo (.0))111210 ||21000112 All VoF((g))))l
(22d)
$ (11 No 12 + 3r) |1|0/112
(22e)
Thus, when | - 2V Fa.;;
(q.; 0.)112 1 AB we have
|10+11l22
lim
< lim | AV Vo. 02)112 AB
(23a)
||0:12
= || AV Ao Foro; 0-)112 + ~A333
(23b)
(23c)
This completes the proof of Theorem 4.1.
C.7 Proof of Theorem 4.2
Theorem 4.2 (Asymptotics in the Tail Risk Cases) Under the Assumption and given a batch of tasks (7x )B we can have
VaR*, a (P(Ti) -P((2)).
CVaRo CVaR. S 3-100me 0-11
(24)
where T fr: VaR pueto > VaRmela }, To : VaR gmeta VaRmelaa
25

---

Proof. Given a batch of tasks {r1, (BB and according to the definition of CVaRa.. we have
CVaR (OT CVaR,. (0.)
(25a)
1 1-a I (1:2220R))
eueea ( p(T)
('((T)dr
(25b)
1 [T:ENm 2VaR
/ E' (T: emets Pa (T; (
(25c)
J. (mela (mecii (T: l' E'pa (TaaT::7 )oT + [. (T: pa (T:
(25d)
/ (mecaa 1") (a) Pal(::
ee (pa(T: Pa Pa(T;81) 0.))
(25e)
S B. 10T - 0.11 + STUUTAUTAUT e (palt;;07 PT( 0.) dr
(25h)
0.11 + f e" (palT; 0Tel) - -Pal:
B+1 11
(25g)
TUTT
B 110T - 0.11 + / "PalT:67em)dd / (Palt:ia)dr
(25h)
T1
To
(p(()) 1- / p(T))dr
Br 110T - 0,11 + 1-a Ji
(25i)
To
< Br 110TeTm - 0.11 + VaR
VaR
{ p(T)dt
p(T)dr 1- Ta
(25j)
.Ti
=B110pee 0.11 + VaR:
1 P(T1) VaR P(Tz).
(25k)
- a
In inequality (251), Ti (T Et < VaR gmeta > VaR VaRmaa J: To {T l > VaR fueta
VaR }, Ta {T 8" < VaR", guneta < VaRmen ). T4 { lr 2 VaR gueta 2 VaR meta Moreover, this inequality holds due to the B,- Lipsshiitz continuous of E(D 0). In Eq. (25g).
Pa (T; 0fpett) = (T; 0.) when E Ts. and (T; 0mppt) = Pa (T; 0.) = pir) when T4 Thus, we complete the proof of Theorem 4.2.
I-a
C.8 Proof of Theorem 4.3
Theorem 4.3 (Generalization Bound in the Tail Risk Cases) Given a collection of task samples {Ti) and corresponding meta datasets, we can derive the following generalization bound in the presence of tail risk:
2( - Laax + V Vnipa (r) [((D D;:(,))) In (2)
R(O.) P((0.)+ +
B
(26)
1 Lmax
1 (1)
2
30B
3(1
a) B
where the inequality holds with probability at least 1 and e E (0, 1), V[-1 denotes the variance operation, and L is from the Assumption
Proof. R(O-)- R((+)) can be decomposed two parts, i.e..
R(0.) - R((O.) = (R(0.) - (..( + (0.)
((n)))+((((0) ( R(0.)).
(27)
For the first part (i.e., R((0)) R. (0.)). we will adopt the Bernstein's inequality to provide an upper bound. Regarding Pa(T.) 1 7,. :²;0.) R(O=) as random variable with respect to 7; and
p(T.)
according to Assumption 1, we know that
(T( p(T:)
1
1
(1D 25..DDS R(O0))
{( DE 0.) R(O.) - 1-a
Lmax-
(28)
1-
26

---

Thus, following Bernstein's 's inequality [79], we know that
1 (T) 1 B p(T:)
112 75.
0.) I(00)) 26
(29a)
IP (O.))
(0.) - R(O,)) 25)
(29b)
BC²
$ exp
(29c)
2V n
Palt)) p(7:))
D 22 0.) - R(O.)
3 I--a Lenax
where
Pa (T)
(1D D.:00 - 1n(OW)
(30a)
((T))
(Ti
V=
((D Di, 0.))
(30b)
p(Ti)
Pa(Ti)
D 8.) - -(E,
(T(TT)
0..00)) 1(D D. 8.)) 0.)) 2
E,
ET.
(30c)
p(T:)
p(T.)
/
Palt)
).))) p(()dt-
(f. (( p(T)
Dú: 0.:
(30d)
p(T)
J
Pa(T)
(1. ((29 : (.))
( (aa
(30e)
P(T)
= I-a /
pp. (T)dT ( (19) D;; 0..0 )o (,;
2
[(D 0.
(30f)
< 1 a a -1) J. 109 (.)aa(t))) + V (r [())) [..).
(30g)
a
C2
maax + VTmpal))
(30h)
Setting match the upper bound in inequality (29c) shows that with probability least the following bound holds:
2( 1-a L2aax + Vy
In (2) (t)
2c In (!)
(0.) R(0.)| =
+ 3(1 a)
(31)
For the second part (i.e., (0,) R((O)), we have
88
(T(
(0.) - R((O.)
8(T.) ((D) D..
(32a)
B
p(T.)
1=1
Lmax
Pa((T))
8(T.))
(32b)
B
p(Ti)
Lmax
Pa((T))
1) ((())
(32c)
B
P(Ti)
B
L
8(T,).
(32d)
1
B
[=l
27

---

In summary, we can obtain an upper bound of R(O.) RR(O.). That is,
R(O.) R(O.) - (o. R(0.) RNo (0.) -R(0.) 2( a max + ()mpp((t) In (t)
1-a
max In (+)
Laxx
Σ 8(T))
B
3(1 aaB
B
2( a
+ (Vrpo(tt)
Lax
2ln (1)
max
+ 3aBB
B
3(1 a)
B
This completes the proof of Theorem 4.3.
C.9 Proof of Theorem 4.4
Theorem 4.4 Let (i: 0) VaR
(((T)EL((TT (UT,0) 0)] and FT; = (aRT 0)] Suppose that (xx) is lower bounded by constant, Vz. For any e O, with probability at least e, we can have the following bound:
(~/ he
sup OE
(FeRDE () (Fr 5o
(33)
/B * log B
Proof. For any constant M, we firstly notice that
P
Paata JB
sup (Fo 0Ee
(o; 0) Fe (a; S M 21
(34a)
< P. Pra... TT
sup (FERDE 0) - Fi (a; 2 M <c .EB
(34b)
+ P Pr.,...T
sup (Fe(t M;0) Fi-KDE(t(f)) 2 S t= F FERDe 0).
(34c)
.EB
For any A and t, we have
P P Fee( M;0) (t:xu(t:00) 2 <c
(35a)
P Pr (Pe(t- M; FAKDR (M) FLKDE(L M;0) FEKDE (1; 20) S
(35b)
+ P Pri.e .e (Fe(t 0: FE-DDE(t M;0) 2 FAKDE(t: 0) FE-DDE(t - M:()) Se (35c)
<> Pru (Fe(t:0) (E 0) 2 FE (1 + M; FE.DDE 0)) Se
(35d)
M
B Σ
((D), 2 :2, + o(M)) Se (35e)
< Pr Pry (F) (2 Bhe
K
hg
M.uin
+ P Pe (F) FE-DD((;;))
Se,
(35t)
beg
where Kmin is the lower bound of the kernel function K(x), i.e., K(x) 2 Kmiin Vr.
According to Theorem 3 of [80], we know that for any O, with probability least the following inequality holds:
C
P Pr....s
sup 0E0,120
(Fr(t:0)
FERDE 0)) 2
Sc.
(36)
/B log B
where C' is a constant. Let M of Theorem 4.4.
C Kmin / /Bllog B
Thus, the Eq. (35f) holds and we complete the proof
C.10 Additional Theorem
To gain more theoretical insights into a popular meta-learning method-MAN [25], we provide the following Theorem C.I. Before proceeding, introduce some notations. During meta-training,
28

---

finite number of task instances are observed by first sampling a task from the distribution p(T). Each task D,, comprises collection of m, data points { D
1. 1 which are distributed over Z with each data point drawn from distribution D;. For some risk E, define the family of functions Fz = {E(0 - AVol(D) 7, 2 0 E @}. For each task D the Rademacher complexity of F on mn samples is
Rmm, (F2) E. (D) (()))
sup BER my
1
Σ j=1
m2
c - (Vo
(37)
where the ej's are Rademacher random variables. Let (.) Ed, A AVol(D) (), (g).
F((0) - - Ei t(0 AV 0), D.y) Denote by 0 the optimal model parameter under the m.
two-stage algorithm. Theorem C.I provides generalization of the algorithm new tasks.
Theorem C.1 (Generalization Bound for MAML in the Tail Risk Cases). For a new task T841 with distribution D8+1, if DB+1 Ei= a, then with probability least 1 6 for any 5 > 0. we can have
B
log (B/5)
FB+1 S max 11)00)
Σ
2a, (Fz) + at
(38)
2m,
Proof. The proof consists of two parts. We first explore the generalization to new instances of previously-seen tasks. Then we solve the generalization to new tasks.
Step 1. For any sample set A = {(D
k.j } define ((A) = supeEFs P:(0) - (O)). Let
A and A'' == (P.,)
be two samples that differ by exactly one point. According to the fact sup, fx) sup, g(z) < (f - g(z)). we know that ((A') - (.A) <
due the difference exactly one point. Similarly, we can obtain $(AA) ((A4) S m thus
(AA) - ((4)) s +. Following McDiarmid's inequality, for any > 0, with probability at least 1 - 9. we have
log(2/6)
((A) S [A(((A)) +
(39)
2m;
We next bound the expectation of the right-hand side inequality (39) as follows:
(A(((A) = EA
sup IEFz
F((0) - -((0))(((0) A((0]]
=EA sup LEFz
E.y (F(((F(()) [(
(40)
$ EAA'
sup EA (F(()) P((0)
(41)
FEFz
= =EA.
sup IEFz
(1(D) k.j - (()
(42)
j=l
= =EA.A'C
( (. 21, 2(() (1)))
sup EEFz mi
< EAe
sup IETz mi
cll 0901 + +EA,
1 tEJ2 m4
sup
Σ (((D))))
m;
-,E
sup LEFz
( i.ji 29)
mi
= 2Rm, 2Fz)
Eq. (40) uses the law of total expectation. Inequality (41) holds by Jensen's inequality and the convexity of the supremum function. In Eq. (42), C(D D2g) = a(o AV)((DE,
29

---

where (Di). 20,) E A. Following inequality (39), we can know that
log(2/6)
0) S R((0) + 2R (F2) +
(43)
2m4
Step 2. Since the new distribution D8+1 is the convex combination of D,, Vi 1,...
... B, we have
F8+ (0) = Eim az a.P. Accordingly, with probability at least å over the choice of samples used to compute F(O).
B Σ 1=1
B Σ
l2g(2/6)
FB+)
ag F((00) S
at a:(3(0 + Ra; + dij
(44)
2m,
which yields that
log (2/5)
FB+ 1 (0*) S max P0(0)) +
Σ
20, 2a,R, (Fz) + a:j
(45)
2m,
In summary, the two steps complete the proof of Theorem C.I.
D Implementation Details
D.1 Benchmark Details & Neural Architectures & Opensource Codes
Here, we illustrate all meta learning benchmark purposes in Fig. 10, which includes sinusoid regression, pendulum system identification, few-shot image classification, and meta reinforcement learning. We no longer run experiments on the Omniglot dataset, as most baselines can achieve SOTA performance and cannot tell the difference well from the openreview of [1].
144 m Amn(s- Mg
mvlo.m)
support dataset
class:
[ Dj
[m,
QUIRY dataset
(a) Sinusold Regression
(b) System Identification
(c) Faw-Shat Image Classification
(d) Continuous Control
Figure 10: Typical meta learning benchmarks in evaluation.
Sinusoid regression: In [1, 42], lot of easy tasks and limited challenging tasks are sampled for meta-training, with the tasks from the whole space employed in evaluation. The default range of the phase parameter is B m). while those of the amplitude are A E [0.1, 1.05) for easy tasks and A E (4.95, 5.0] for challenging tasks. Generally, sinusoid functions with larger amplitudes are hard adapt from a few support data points. The mean square error works as the risk function to measure the gap between the predicted value f() and the actual value. We set the task batch 50 for 5-shot and 25 for 10oshot, and the maximum iteration number is 70000. We refer the reader MAMM and DR-MAML for all of the setups.
We retain the neural architectures [42, 1] for all MAML like methods. In detail, all methods take a multilayer perceptron with two hidden layers and 40 ReLU activation units in each layer. The inner loop achieved via one stochastic gradient descent step. As for CNP like methods, please refer to the vanilla set-up in [15] (The Github link is attached here: mrogle-deepminde www.ddeeeeeennuuaaeeaeeeeeeeeeee
com/
As the task space is hugh, there is no way to exactly estimate the risk quantile. Hence, the Oracle quantile in Fig. 5 is roughly computed from the sampled 100 tasks given the pretrained DR-MAML+ The rationale behind this operation is that increasing the population number in statistics reduces the quantile estimate bias.
System identification:
The pendulum system a classical environment in the OpenAl gym (en- wttp://ppnaa//gg//////////eeeeee
vironment details are: https /ggtttu... com/ opw.ai/eeaaa//eooo httpb//bbmm/tteerrre///////////e gywggggg/nnnn hwwd/env//cee cllssic.
30

---

control/ hppdd//lelummmmmm py), and it is an actuated joint with one fixed end. The goal of system iden- tification for the pendulum system is predict the state transition given arbitrary actions with several randomly collected transitions as the support dataset. The observation is tuple in the form (cos f sin 0,0'), where E [T, x]. The action is in the range a E [-2.0. 2.0] and the torque applied to the pendulum body. The mass m and the length of the pendulum follows a uniform distribution (m,)) ~ (00.44 1.6] x [0.4, I.6]), sampled variables configure a Markov decision process as the task. In each batch, there are 16 tasks, and each task comprises 200 data points. Specifically, 10 few-shot data points are randomly sampled to enable system identification per task, denoted as 10-shot. For 20-shot cases, the number of data points in support dataset is 20. And the maximum iteration number is 5000.
For all MAML-like methods, the neural architecture used here is multilayer perceptron with three hidden layers of 128 hidden units each and the activation function ReL.U. The learning rate for both the inner and outer loops is set at 1e-4.
Few-shot image classification: The few-shot image classification is mostly described as an N-way K-shot classification, where N classes with K-labeled instances for each are considered. The dataset organized in the same manner as that in [81, 42, 1]: These include 64 classes for meta-training, with the rest 36 classes for meta-testing. We generate each task in the way: 8 meta-training tasks from the class {6, 7, 8, 8, 9, 9, 10} are randomly generated from 64 meta-training classes; the remaining classes are organized similarly. As result, each task is constructed from sampling one image from five classes, corresponding to a 5 5-way 1-shot problem. The task batch is set with maximum number of iterations of 60000 in meta-training.
For all MAML-like methods, the neural architecture used here is four-layer convolutional neural network for the mini-ImageNet datasets. The inner loop is achieved via one stochastic gradient descent step. We refer the reader to TR-MAML and DR-MAML for all of the setups (The Github link is attached here https: / /gi thub. wtt. htmmm//gollnns/teee ttr//rrmmaaas maml).
Meta reinforcement learning: 2D Navigation is classical meta reinforcement learning benchmark where efficient explorations matter. The task in 2D Navigation is to guide the point robot to take move actions for a purpose of reaching specific goal location from the step-wise reward. The reward the agent receives from the environment is based on the distance to the goal, and 20 episodes work as the support dataset for navigation fast adaptation. In terms of the task distribution, we sample tasks from a uniform distribution ([.5, 0.5] x 0.5)) over goal locations.
As for the neural architecture for policy network set-ups, we refer the reader to vanilla MAML (Github com/ htttt//siinndeeeeeteteteeeee wttpeee/pptthhchhhaeeeeeee -r1) and CAVIA
link is attached here attps://gittee (The Github link is attached here https: //gi thub. cwmm coom/mmintggra//// http://aaaaateeaaeeeeeeeeeeeeeee And trust region policy optimization works for policy optimization.
Table 4: Computational and memory cost in MaPLe relevant experiments.
Method
MaPLe
DR-Me
DR-MaPL.e+ (Ours)
Implementation Time
2.1 h 41.57 G
+1.7 h +36.84G
+1.7 h +36.84G
Memory Usage
Few-Shot Image Classification with MaPLe [69]: The stochastic gradient descent is the default optimizer with the learning rate 0.0035, and A6000 GPUs work for computations. We examine tail task risk minimization effecctivness on three large datasets. The class number split setup in datasets (class number to train/validate/test)) is TigredlmageNet (351/97/160) [82], ImagenetA (128/32/40) [83], and ImagenetSketch (640/1 (000/) [84]. Table 4 reports the overall training time and memory, where the vanilla MaPLe serves as the anchor point, and + means additional costs from the two-stage operation. For details of experimental implementations and setups, feel free access our code at https://ghh thu.. thub. com/ http/vyiion wwwyyiiinDRMMM..
D.2 Modules in Python
This subsection includes the impelementation of KDE for the studied strategy. Here, the example of the hinge loss is illustrated as follows.
31

---

import numpy as up import torch
from scipy stats import gaussian_kde from Stapy.optimiz. import brentq
def loss (bstchh_ confidence_level). eatimate the VaR_alpha according kernel denaity eatimator
kde = gaussian_ (batch_ try:
10
target_fune lambda x: kde. intogratee_ox. (-np. inf, x)
confidence__evel
11
aaR_alpha
mp. (pain(baacc up. max
batch_loss))
12
except ValueError:
up linspaaecceeeecc (np min mbn(batch_ooss)) np maaxc 1000)
pdf kde evaluatte (x)
15
cdf up cumsum (pdf) up. sudd)
16
index = np- argmar (cdf >= confidence_
17
VaR_alpha x[index]
19
calculate the meta loss tail_loss
20
alpha if (i VaR_a)pha) else torch.
tensor (0.) . cuda () for i in batch_loas]
new ww__bacch_loo lose = torch. stack (tail_loss) mean ()
factor 1 (1 confidence_leve levell
23
loss_meta VaR_alpha factor rew_batch_los
24
return loss_meta
Listing 1: The calculation process of CVaRa objective.
E
Additional Experimental Results
Due to the page limit in the main paper, we include additional experiments and corresponding results in this section.
E.1 Evaluation with Other Robust Meta Learners
In addition to MAML, we apply a similar modification to CNP, which results in TR-CNP, DRO-CNP, DR-CNP, and DR-CNP+ DDRCNN with KDE for VaR, estimates). We report the meta testing results on sinusoid regression and pendulum system identification benchmarks.
As illustrated in Table 5/6, all methods achieve comparable average performance in sinusoid and pendulum system identification. Regarding CVaR,, DR-CNP's improvement is relatively marginal over others except DR-CNP+. Compared to MAML, CNP seems sensitive quantile estimate accuracies when meeting with the studied strategies.
Table MSEs for Sinusoid 5-shot with reported standard deviations (5 runs). With a = 0.7, the best results are in bold.
Method
Average
Worst
CVaRa
CNP [15]
0.09+0.00
20.1
0.24:1001
TR-CNP [42]
0.10+1001
1.51
0.22+1003
DRO-CNP [66]
0.09+0002
2.54+181
0,2 +:0.05
DR-CNP [1]
0.09
1.62.10.5
0.20.10.02
DR-CNP+(Ours)
0.08
1.47..0.00
0.17 1.002
E.2 Numeric Results in Tables and Histograms
As the improving tricks in this work are regarding the quantile estimators, here we particularly include the quantitative results to show the difference between DR-MAN 3R-MAML and DR-MAN DR-MAML+ in Table 7/8.
32

---

Table 6: MSEs for Pendulum 10-shot with reported standard deviations (5 runs), With 0.5, the best results are bold.
Method
Average
Worst
CVaR,,
CNP [15]
0.75
1.51
0.87
TR-CNP [42]
0.7610000
1.20
0.85
DRO-CNP [66]
0.7310001
1.51
0.85
DR-CNP [1]
0.75.10.01
1.40.1..6
0.86.10.01
DR-CNP+(Ours)
0.72:10.01
1.36++007
0.82:10.00
Note that the studied distributionally robust strategy on the tail risk minimization, and CVaR, the direct optimization indicator. As be seen, DR-MAML+* performance superiority over DR-MAML significant w.z.t CVaR, values in 5-shot sinusoid regression and four mini-Imagel meta-testing tasks. These scenarios adaptation is limited in 5-shot data points and (ii) the distributional shift severe in mmag--m meta-testing phase.
challenging than others (i) the context information for
Table 7: Test average mean square errors (MSEs) with reported standard deviations for sinusoid regression (5 runs). We respectively consider s-ohot and 10-shot cases with a 0.7. The results are evaluated across the 490 meta-test tasks, as in [42]. The best results are in bold.
5-shot Worst
10-shot Worst
Method
Average
CVaR,
Average
CVaRa
DR-MAML [1]
0.89+0044
2.91 2.78+t0.2 1.65++0.22
1.76+0.02
0.54
1.70+0.17
0.96=0.01 0.95=0.02
(OUS
0.87=0002
0.59
1.51+0.11
Table 8: Average 5-way 1-shot classification accuracies in mini-ImageNet with reported
standard deviations (3 runs). With a = 0.5, the best results are in bold. The higher, the better for all values.
Eight Meta-Training Tasks
Four Meta-Testing Tasks
Method
Average
Worst
CVaR,
Average
Worst
CVaR,,
DR-MAML [1]
70.2
63.4
67.2.101
49.4
47.1
47.5
DO-MAML+(Ous)
10.4
63.8
67.5
19.
47.2
48.1
We can attribute the performance differences of the two methods to the cumulative quantile estimation errors using the crude MC. Even though the quantile estimation error in Fig. difference is tiny in each step, the cumulative error indeed affects the converged equilibrium a lot. This reflects the advantage of the KDE's used in DR-MAML+ when the task batch size cannot be larger in practice.
We also investigate the task risk value distributions in pendulum system identification. To this end, visualize
testing results for all methods in Fig. 11. It seems DR-MAML+'S result is more skewed to the left than others.
Fig. 12 displays all methods' performance W.E.t. the average and CVaR, returns along the meta- training process. We exclude TR- -AMM in visualization due to worse performance and unstable training properties. We can find that the DR-MAML exhibits fast performance rise at the early stage but its capability to continuously improve diminishes over time. DR-MAML+ consistently outperforms other baselines in most cases. The above suggests that the KDE module achieves performance gains over the crude MC when implemented with the two-stage distributionally robust strategy for meta RL scenarios.
E.3 Sensitivity Analysis to Confidence Level
To reveal the impact of confidence levels on model performance, we perform a sensitivity analysis with respect to confidence levels. Since only DR-MAMI. and DR-MAML+ are influenced by the confidence levels during the distributionally robust optimization across all baselines, we only compare the performance of the two methods highlight the differences between them. As shown in Fig.
33

---

Averigget B0
Averoggotuul
Aureepedd CVil
Antiggoil E1
CYOK or
Kive (100))
Mive
(M/4))
Aquiret Froee (140)]
Une. Apand (100))
Min Spuord (U00))
Figure 11: Histograms of meta-testing performance in system identification. With a = 0.5, we visualize the comprision results of baselines and our DR-MAML+ in 10-shot prediction. The lower, the better for Average and CVaRa values.
Point Robot Navigation
-10
-10
Average Return
115
CVaa Redtun
-30
200)
22:1
300
400
300:
100
200
314)
4001
500
Steps
Steps
MAMI
DRO MAML
DR-MAME
DR-MAMIL-
Figure 12:
Learning curves for the point robot navigation task. Here, 20 trajectories work as the support set for adaptation. The curves report the normalized returns and are averaged over four random seeds, with a 0.5.
13/14, we can observe that in both sinusoid 5-shot and 10-shot tasks, as the confidence level varies, DR-MAML+ exhibits more stable performance than DR-MAMI, indicating that DR-MAMLA has a lower sensitivity to confidence levels. can be illustrated that the crude Monte Carlo used in DR-M OR-MAML is unstable in terms of quantile estimation than the kernel density estimator used in ARRMMLL+. This can be due to the fact that the crude Monte Carlo method is more likely to get stuck in the local optimal solution. In addition, it can be seen from Fig. 13/14 that the performance of our developed DR-MAML+ better than DR-MAML in most cases. DR-MAML+ exhibits lower mean squared errors than DR-MAML in the average, worst, and CVaR,, indicators, demonstrating the advantages more accurate quantile estimation in improving robustness.
Sinusoid 5-shot
DI-MANI
DR-MANS C00-MAASS++100
DOS ALAALA
DIL-MAIS---DA (Dan)
D08-A6ANLL- (Cun))
Average
Worst
CVVaRt
03
0.3
0.5
07
03
03
0.3
05
03
0.
0.5
0.5
07
0.5
Confidence Level
Confidence Level
Confidence Level
Figure 13: Meta testing performance of DR-MAML and DR-MAML+ with different confidence level on Sinusoid 5-shot tasks. the plots, the vertical axis the MSEs, the horizontal axis the confidence level, and the shaded area represents the standard deviation.
34

---

Sinusoid 10-shot
EHL MANIS
DRLALASS
ALAAIL
(Dhen)
30-
Worst
C.VaRt
0.5
LG-
0.-
05-
033
05
67
0.3
033
0d 0.1
0.5
0..7
0.5
Confidence Level
Confidence Level
Confidence Level
Figure 14: Meta testing performance of DR-MAML and DR-MAML+ with different confidence level on Sinusoid 10-shot tasks. In the plots, the vertical axis is the MSEs, the horizontal axis is the confidence level, and the shaded area represents the standard deviation.
The fast adaptation risk landscape of meta-trained MAML, TR-MAML, DRO- MAML,, DR-MAML and DR-MAML+. The figure illustrates 5-shot sinusoid regression example, mapping to the function space f(x) = A sin(x - B). The X-axis and Y-xxis represent the amplitude parameter a and phase parameter respectively. The plots exhibit testing MSEs on the Z-axis across random trials of task generation.
Figure 15:
E.4 Further Exploration on Adaptation
We demonstrate the adaptation risk landscape of meta-trained MAML [25], TR-MAML [42], DRO- MAML [66]. DR-MAML [1] and our DR-MAML+ in Fig. 15. The adaptation risk landscape shows the superiority of our method in optimizing within the expected tail risk minimization. Compared to other methods, DR-MAML+ exhibits smoother and smaller risk profiles, illustrating its robustness even in challenging tasks.
F
Computational Platforms & Softwares
This work employs Pytorch [85] as the default deep learning toolkit when implementing the developed methods. As for baselines, TR-MAMAL follows the standard implementation as work [42] and runs with Tensorflow [86]. Others implemented with Pytorch. All experimental results computed by NVIDIA RTX6000 GPUs and A800 GPUs.
35

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Sec 5.7.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
36

---

Justification:
Refer to Appendix C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Refer Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
37

---

Answer: [Yes]
Justification:
Refer to Appendix D.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
wwwssssssioiiiioi for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /niips wwtpccccpppccceeeecccc.cccccccct gwwc//ggg//////eeeee /CodeSummi ciooomissii ssisssssnoooees seiinnolccy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Refer to Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Refer to Fig. 3/4/6/12/13/14 and Table 1/2/5/6/7/8.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
38

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Refer Appendix F.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https http///eerrpp.ee/ee//ee cc
Answer:
[Yes]
Justification:
We adhere to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Refer Appendix A.5.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
39

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The question is not applicable to the paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The question is not applicable to the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcee thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
40

---

Answer: [NA]
Justification:
The question is not applicable to the paper.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The question is not applicable to the paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The question is not applicable to the paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
41


Reviews:
[{"review_id": "VT7jwyR7Jv", "reviewer": "Reviewer_ci3Y", "summary": "The paper provides theoretical investigations for better understanding of an existing method in literature that focuses on minimizing expected tail risk. Equivalence of the algorithm is shown to a Stackerlberg game, which allows to study its convergence rate and asymptotic bounds on performance and generalization are provided. Finally, a practical heuristic is suggested to obtain performance improvements over the base method.", "questions": "- What is the overhead induced by searching for the appropriate value of hyperparameter $\\alpha$, compared to other baselines?", "limitations": "N/A", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The presentation and structure of the paper is good.\n- The theoretical results, while limited in mathematical novelty, seem comprehensive and are clear to follow.\n- Sufficient experiments and ablations are provided.", "weaknesses": "Clarity:  \n- It is not clear how the expression for the expected tail risk minimization is obtained in (3); the reviewer had to refer to the corresponding reference to understand it and thus, including it would help make the paper self-contained.\n- Figure 1 is not clear; the font size needs to be increased.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank **# Reviewer ci3Y** for these insightful comments. The remainders focus on concerns and questions to address.\n\n---\n\n**1. Additional explanations on the expected tail risk minimization**\n\nThanks for this advice. **We'll include more explanations in Line 115** as follows:\n> Wang et al. [1] minimizes the expected tail risk, or equivalently $\\text{CVaR}\\_{\\alpha}$ risk measure:\n$$\n\\min\\_{\\theta\\in\\Theta}\\mathcal{E}\\_{\\alpha}(\\theta):=\\mathbb{E}\\_{p_{\\alpha}(\\tau;\\theta)}\n        \\Big[\\ell(\\mathfrak{D}\\_{\\tau}^{Q},\\mathfrak{D}\\_{\\tau}^{S};\\theta)\n        \\Big].\n$$\n>Due to no closed form of $p\\_{\\alpha}(\\tau;\\theta)$, [1] introduces a slack variable $\\xi\\in\\mathbb{R}$ and reformulates the objective as follows:\n$$\n\\min_{\\theta\\in\\Theta,\\xi\\in\\mathbb{R}}\n        \\mathcal{E}\\_{\\alpha}(\\theta,\\xi):=\n        \\frac{1}{1-\\alpha}\\int\\_{\\alpha}^{1}v\\_{\\beta}d\\beta\n        =\n        \\xi+\\frac{1}{1-\\alpha}\\mathbb{E}\\_{p(\\tau)}\n        \\left[\\left[\\ell(\\mathfrak{D}\\_{\\tau}^{Q},\\mathfrak{D}\\_{\\tau}^{S};\\theta)\n        -\\xi\\right]^{+}\n        \\right],\n$$\n>where $v\\_\\beta:=F\\_{\\ell}^{-1}(\\beta)$ denotes the quantile statistics and $\\left[\\ell(\\mathfrak{D}\\_{\\tau}^{Q},\\mathfrak{D}\\_{\\tau}^{S};\\theta)-\\xi\\right]^{+}:=\\max$ {$\\ell(\\mathfrak{D}\\_{\\tau}^{Q},\\mathfrak{D}\\_{\\tau}^{S};\\theta)-\\xi,0$} is the hinge risk.\n\nWe hope such descriptions make it easier to understand.\n\n**2. Larger font size in figure 1**\n\nThanks for the valuable feedback. We've increased the font size in Figure 1 and will update it in the final manuscript.\n\n**3. Questions about hyperparameter $\\alpha$**\n\nThanks for your insightful question. \n\n(1) In our experiments, we did not perform extensive hyperparameter $\\alpha$ searches. Instead, we adopted hyperparameter settings that are consistent with existing literature [1]. This ensures that our results are fair in comparison to previous studies without introducing additional overhead from hyperparameter tuning.\n\n(2) To further reveal the impact of confidence levels $\\alpha$ on model performance, we conducted ablation experiments, as shown in **Fig. 12/13**. We can observe that in both sinusoid 5-shot and 10-shot tasks, as the confidence level varies, DR-MAML+ exhibits more stable performance than DR-MAML, indicating that DR-MAML+ has a lower sensitivity to confidence levels.\n\n---\n\n**Reference**\n\n[1] Wang, Qi, et al. \"A simple yet effective strategy to robustify the meta learning paradigm.\" NeurIPS 2023.\n\n---\n\n*Finally, we hope these questions and concerns are well answered and addressed. Thanks again for your efforts. We are happy to discuss any other questions and provide further clarifications.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wJjQR49cVJ", "reviewer": "Reviewer_6Ej4", "summary": "This paper proposes an enhancement to the previous work termed DR-MAML by reformulating it as a Stackelberg game. Theoretical investigations regarding its solution concept, convergence rate, and generalization bound are provided. Numerical experiments demonstrate the improved robustness of the proposed method.", "questions": "See above.", "limitations": "No explicit discussion on limitations are provided.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Both theoretical guarantees and numerical validations are provided to justify the sought robustness. \n2. Reformulating DR-MAML's objective as a Stackelberg game enables analyzing the problem from the viewpoint of game theory.", "weaknesses": "1. The contribution of this work is incremental and exclusive to DR-MAML, which limits its scope and broader applicability. \n2. The writing is hard to parse, and several notions are pretty vague. For instance, in lines 95-96, $F_{\\ell}^{\\alpha}$, $\\Omega_{\\alpha,\\tau}$, and $p_{\\alpha}$ are defined through illustration in a figure instead of mathematical expressions. \n3. Experiments are limited and lack comparison to SOTA methods. It is recommended comparing DR-MAML+ with popular meta-learning methods (such as MetaCurvature and MetaOptNet) on open-source benchmark datasets including tieredImageNet, CUB-200-2011, and MetaDatasets. In addition, the improvement in Table 1 and Table 2 are rather marginal (no more than 0.6%). \n4. There is no comparison of time and space complexity, and the scalability to large NNs like ResNet is unknown.\n5. The last statement of Theorem 4.1 is imprecise. According to Eq. (22), this convergence rate solely holds true when $t$ approaches infinity. \n6. The paper contains numerous typos. For instance, in line 4, the citation is not compiled. In line 95, \"the resulling\" should be corrected to \"resulting.\" Additionally, in lines 96, 130, 240, 241, 244, 246, 254, 265, 283, 295, 303, 319, 328, 332, and 335-337, the cross-references are in the equation style instead of the correct format.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank **# Reviewer 6Ej4** for these insightful comments. The remainder mainly focuses on concerns to address.\n\n---\n\n**1. Application scopes and contribution clarifications**\n\nThanks for the comment. Sorry for confusing you in contributions, and we further summarize:\n\n(1) Regarding the application scopes, ***this work is meta-learning method agnostic***. Apart from MAML, CNP is also used in examinations. Besides, we've taken the advice to conduct experiments on more benchmarks with large models' augmented backbone. See **Point 3**.\n\n(2) Regarding the contribution, this work leans more on theoretical investigations of tail task risk minimization for meta-learning, which complements the empirical discoveries in Wang et al. (2023a). These include (i) the notion of solutions, (ii) understanding the Stackelberg game, (iii) generalization and asymptotic analysis in tail adaptation risk, and implementation tricks to enhance robustness. The practical enhancement is the theory's side product. See details in **Lines 35-39**, **Lines 602-604**, and **Table 3**.\n\n**2. More descriptions on notations**\n\nThank you for your suggestions. **We'll add more to Line 96 as follows**:\n\n>The normalized cumulative distribution $F_{\\ell}^{\\alpha}(l;\\theta)$ is defined as:\n$$F\\_{\\ell}^{\\alpha}(l;\\theta)=\n\\begin{cases}\n0, & l<\\text{VaR}\\_{\\alpha}[\\ell(\\mathcal{T},\\theta)]\\ or \\\n\\frac{F\\_{\\ell}(l;\\theta)-\\alpha}{1-\\alpha}, & l\\geq\\text{VaR}\\_{\\alpha}[\\ell(\\mathcal{T},\\theta)].\n\\end{cases}$$\n\n> $\\forall\\theta\\in\\Theta$, the meta learning operator $\\mathcal{M}\\_{\\theta}$ defines:$\\mathcal{M}\\_{\\theta}:\\tau\\mapsto\\ell(\\mathfrak{D}\\_{\\tau}^{Q},\\mathfrak{D}\\_{\\tau}^{S};\\theta).$ Accordingly, the tail risk task subspace $\\Omega\\_{\\alpha,\\tau}:=\\bigcup\\_{\\ell\\geq\\text{VaR}\\_{\\alpha}[\\ell(\\mathcal{T},\\theta)]}\\left[\\mathcal{M}\\_{\\theta}^{-1}(\\ell)\\right]$, with the task distribution constrained in $\\Omega\\_{\\alpha,\\tau}$ by $p_{\\alpha}(\\tau;\\theta)$.\n\n**3. Additional experiments and larger NNs**\n\nThanks for these precious suggestions. \n\n(1)  Comparison with MetaCurvature/MetaOptNet and more evaluation in larger NNs:\n\nBoth [1] and our work are agnostic to meta-learning methods, with baseline selections specifically related to distributional robustness in meta-learning. Instead, we directly run experiments with large models. CLIP [2] is a recent SOTA than MetaCurvature/MetaOptNet; hence, we employ \"ViT-B/16\"-based CLIP as the backbone to enable few-shot learning in the same way as MaPLe (N_CTX: 2, MAX_EPOCH: 30) [3], scaling to large NNs in evaluation. SGD is the default optimizer with LR 0.0035, and A6000 GPUs work for computations. We examine tail task risk minimization effectiveness on three large datasets. The class number split setup in datasets (num train/num val/num test) is TieredImageNet (351/97/160), ImagenetA [4] (128/32/40), and ImagenetSketch (640/160/200).\n\nWe'll include the **global rebuttal Figure15-17** and analysis below in the manuscript:\n>Results are reported in **Fig. 15-17**. DR-MaPLe and DR-MaPLe+ consistently outperform baselines across both average and ${\\rm CVaR}_\\alpha$ indicators in $\\texttt{5-way 1-shot}$ cases, demonstrating the advantage of the two-stage strategy in enhancing the robustness of few-shot learning. DR-MaPLe+ achieves better results as KDE quantiles are more accurate with large batch sizes. These results examine the scalability and compatibility of our method on large models.\n\n***We'll opensource codes to facilitate robust fast adaptation research from the updated paper.***\n\n(2) We've updated related work in **Section2** **Lines 61-63**:\n>Widely known are model agnostic meta learning and related variants, such as MetaCurvature, which learns curvature information and transforms gradients in the inner-loop optimization.\n>The metrics-based methods ... For example, MetaOptNet proposes to learn embeddings under a linear classifier and achieve SOTA few-shot classification performance.\n\n\n**Reference**\n\n[1] Wang, Qi, et al. \"A simple yet effective strategy to robustify the meta learning paradigm.\" NeurIPS 2023.\n\n[2] Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" ICML 2021.\n\n[3] Khattak, Muhammad Uzair, et al. \"Maple: Multi-modal prompt learning.\" CVPR 2023.\n\n[4] Hendrycks, Dan, et al. \"Natural adversarial examples.\" CVPR 2021.\n\n**4. More analysis and rephrase**\n\n(1) Marginal improvement in **Table 1** and **Table 2**:\n\nImage processing necessitates a small batch size, causing smaller differences. See analysis in **Lines 305-307**:\n>... we attribute this to the small batch size in training, which weakens KDE's quantile approximation advantage.\n\n(2) Time and space complexity comparison:\n\nSorry for missing this part. **We'll add the following to the manuscript:**\n>The space complexity is specific to the meta-learning method, while this work is agnostic to it. Hence, we report the computational complexity for the DR-MAML+ as $\\mathcal{O}\\big(\\mathcal{B}\\big(\\mathcal{B} - \\alpha|\\mathcal{M}| \\big)\\big)$ while using KDE with the Gaussian kernel, and that of DR-MAML is $\\mathcal{O}\\big(\\mathcal{B}\\big(\\log(\\mathcal{B}) - \\alpha|\\mathcal{M}|\\big)\\big)$.\n\n(3) Refine theorem 4.1 condition:\n\nYou are right, and **we’ll refine it as:**\n> Let the iteration sequence ... when $t$ approaches infinity.\n\n**5. Typos and equation-style reference**\n\nSorry for this, we'll correct the \"resullting\" typo to \"resulting\" and modify all cross-reference styles, e.g., Fig/Table/Theorem/Assumption/Example, by removing unnecessary brackets \"()\", such as change \"Fig. (6)\" to \"Fig. 6\", \"Theorem (4.2)\" to \"Theorem 4.2\", \"Assumption (1)\" to \"Assumption 1\". We hope there is no typo and cross-reference style issues this time.\n\n---\n\n*Thanks again for carefully reading our manuscript and proposing constructive suggestions. After clarifying the contribution and including more results, we hope the evaluation of this work can be reconsidered. Your help is precious to us.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bEYd8QE56v", "reviewer": "Reviewer_8vBx", "summary": "This work investigates the tail risk minimization in meta-learning from theoretical and practical perspectives. Overall, this work is well-written, novel and theoretically enriches TR-MAML[1]/DR-MAML[2].\n\n\nIn the realm of large models, meta-learning plays a crucial role due to the pressing concern of distributional robustness across various tasks, particularly in risk-sensitive applications. \nHere, I express my positive attitude based on this work's completeness, novelty, and workload.\n\n[1] Collins, L., Mokhtari, A., & Shakkottai, S. (2020). Task-robust model-agnostic meta-learning. Advances in Neural Information Processing Systems, 33, 18860-18871.\n\n[2] Wang, Q., Lv, Y., Xie, Z., & Huang, J. (2024). A simple yet effective strategy to robustify the meta learning paradigm. Advances in Neural Information Processing Systems, 36.", "questions": "With complete proofs and empirical evaluations, I do not have too many concrete questions except for some discussions. \n\n(1) Both group DRO and tail risk minimization handle the distribution shift in robust optimization; what are the advantages or disadvantages between them in meta-learning? \n\n(2) What difficulties will we encounter if we adopt the developed strategy in the optimization of large models?", "limitations": "None", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The novelty of this work lies in two aspects.\n\n(1) This work reduces the distributionally robust meta-learning to a max-min optimization and performs analysis on tail risk generalization, convergence rate estimation, and the role of quantile estimation; \n\n(2) This work enhances DR-MAML through more accurate quantile estimates with theoretical support.\n\nOverall, this work clarifies its contribution in Table3 and completes all claims together with detailed proofs. Extensive experiments further verify theoretical insights.", "weaknesses": "No concrete weakness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank **# Reviewer 8vBx** for these helpful comments. The remainders focus on questions to answer.\n\n---\n\n**1. Advantages and disadvantages between group DRO and tail risk minimization in meta-learning**\n\nThanks for this comment. The group DRO method employs risk reweighted algorithm to relax the weights of tasks and assign more weights to the gradient of worst cases. The tail risk minimization principle adopts two-stage optimization strategies to control the worst fast adaptation cases at a certain probabilistic level. (1) Group DRO is advantageous when predefined groups are available to guarantee robust performance across these groups, however, meta training is in a task episodic manner and weakens the applicability of group DRO. Tail risk minimization is more flexible, does not require clearly defined groups, and is suitable for risk-sensitive scenarios. (2) In the experiments, the tail risk minimization method consistently outperforms group DRO, demonstrating the advantages of the two-stage optimization strategy in improving robustness. \n\n\n**2. Application to large models**\n\nThanks for this constructive comment. To answer the scalability question in large NNs, e.g., large models, we directly run experiments with large models. \n\n**Experimental setup:** CLIP [1] is a recent large vision-language model; hence, we employ \"ViT-B/16\"-based CLIP as the backbone to enable few-shot learning in the same way as MaPLe (N_CTX: 2, MAX_EPOCH: 30) [2], scaling to large NNs in evaluation. SGD is the default optimizer with LR 0.0035, and A6000 GPUs work for computations. We examine tail task risk minimization effectiveness on three large datasets. The class number split setup in datasets (num train/num val/num test) is TieredImageNet (351/97/160), ImagenetA [3] (128/32/40), and ImagenetSketch (640/160/200).\n\n**Result and analysis:** We'll include the **global rebuttal Figure15-17** and analysis below in the manuscript:\n>Results are reported in **Fig. 15-17**. DR-MaPLe and DR-MaPLe+ consistently outperform baselines across both average and ${\\rm CVaR}_\\alpha$ indicators in $\\texttt{5-way 1-shot}$ cases, demonstrating the advantage of the two-stage strategy in enhancing the robustness of few-shot learning. DR-MaPLe+ achieves better results as KDE quantiles are more accurate with large batch sizes. These results examine the scalability and compatibility of our method on large models.\n\n**3. Opensource plan**\n\nEven though this work contributes to more theoretical points, we hope our empirical investigations can provide more insights in developing large model augmented few-shot learners.\n\n***We'll opensource codes of large models' experiments to facilitate robust fast adaptation research from the updated manuscript.***\n\n___\n**Reference**\n\n[1] Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" ICML 2021.\n\n[2] Khattak, Muhammad Uzair, et al. \"Maple: Multi-modal prompt learning.\" CVPR 2023.\n\n[3] Hendrycks, Dan, et al. \"Natural adversarial examples.\" CVPR 2021.\n\n---\n\n*Finally, we hope your questions are well answered. And thanks for your suggestions in improving the quality of this manuscript.*\n\n---", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
*We sincerely thank all reviewers and area chairs for their work. This global response summarizes reviews, addresses concerns, answers questions, and reports changes in the manuscript.*

---

### **I. Review Summary**

We thank all reviewers for their comments: 
1. a *well-constructed paper easy to follow* **[# Reviewer ci3Y and # Reviewer 8vBx]**; 
2. *comprehensive theoretical analysis aligned with numerical evaluations* **[# Reviewer ci3Y, # Reviewer 6Ej4, and # Reviewer 8vBx]**; 
3. *addresses the problem from Stackelberg game/max-min optimization* **[# Reviewer 6Ej4 and # Reviewer 8vBx]**.

---

### **II. Primary Concerns and Questions**

**1. Contributions and application scope clarifications [# Reviewer 6Ej4]**

(1) Regarding the contribution, this work leans more on theoretical points **[# Reviewer ci3Y and # Reviewer 8vBx]** throughout tail risk minimization for meta-learning, which complements the empirical discoveries in Wang et al. (2023a) [1]. These include (i) notion of solutions, (ii) algorithmic understanding from the Stackelberg game, (iii) generalization, asymptotic analysis in tail adaptation risk, quantile estimates' influence, and implementation tricks to enhance robustness as mentioned in **Lines 35-39** and **Table 3**. The practical enhancement is the side product of the theory. 

(2) Regarding the application scopes, *this work is meta-learning method agnostic*. Apart from MAML, CNP is also used in examinations. Importantly, we've taken the advice to ***conduct experiments on more benchmarks with large models' augmented backbone***. See **Point 2** and detailed **Response to # Reviewers 6Ej4/8vBx**.

**2. Comparison with MetaCurvature/MetaOptNet and more evaluation [# Reviewer 6Ej4], Scalability in large models [# Reviewer 6Ej4 and # Reviewer 8vBx]**

Both [1] and our work are agnostic to meta-learning methods, with baseline selections specifically related to distributional robustness in meta-learning. Instead, we directly run experiments with large models. CLIP [2] is a recent SOTA than MetaCurvature/MetaOptNet; hence, we employ "ViT-B/16"-based CLIP as the backbone to enable few-shot learning in the same way as MaPLe (N_CTX: 2, MAX_EPOCH: 30) [3], scaling to large NNs in evaluation. SGD is the default optimizer with LR 0.0035, and A6000 GPUs work for computations. We examine tail task risk minimization effectiveness on three large datasets. The class number split setup in datasets (num train/num val/num test) is TieredImageNet (351/97/160), ImagenetA [4] (128/32/40), and ImagenetSketch (640/160/200).

We've attached the results in **Figure15-17** and analysis below:
>Results are reported in **Fig. 15-17**. DR-MaPLe and DR-MaPLe+ consistently outperform baselines across both average and ${\rm CVaR}_\alpha$ indicators in $\texttt{5-way 1-shot}$ cases, demonstrating the advantage of the two-stage strategy in enhancing the robustness of few-shot learning. DR-MaPLe+ achieves better results as KDE quantiles are more accurate with large batch sizes. These results examine the scalability and compatibility of our method on large models.

**3. Eq. (3) descriptions, larger font size in Fig. 1, more analysis, one typo and cross-reference style [# Reviewer ci3Y and # Reviewer 6Ej4]**

(1)	We've added Eq. (3) and other notation descriptions. See individual responses.
(2)	We've enlarged Fig. 1's font size in the updated version.
(3)	We've refined Theorem 4.1 to make our statement more precise, and time/space complexity analysis sees **Response to # Reviewer 6Ej4**.
(4) Typos and equation-style reference: 
We've corrected the “resullting” typo to “resulting” and modified all cross-reference styles, e.g., Fig/Table/Theorem/Assumption/Example, by removing unnecessary brackets “()”, such as changing “Fig. (6)” to “Fig. 6”, “Theorem (4.2)” to “Theorem 4.2”, “Assumption (1)” to “Assumption 1”. We've proofread the manuscript many times, securing no typos and other informal reference style uses.

**4. Other questions and opensource plan**

Thanks and we leave other questions and answers in individual responses. *We'll opensource codes of large models' experiments to facilitate robust fast adaptation research in the updated manuscript.*

**Reference**

[1] Wang, Qi, et al. "A simple yet effective strategy to robustify the meta learning paradigm." NeurIPS 2023.

[2] Radford, Alec, et al. "Learning transferable visual models from natural language supervision." ICML 2021.

[3] Khattak, Muhammad Uzair, et al. "Maple: Multi-modal prompt learning." CVPR 2023.

[4] Hendrycks, Dan, et al. "Natural adversarial examples." CVPR 2021.

---
### **III. Manuscript Future Changes**

**1. Include new experimental results with large model backbones.**

**2. Add descriptions and discussions, and revise cross-reference styles as previously mentioned.**

---

*Once again, thank you to all the reviewers and area chairs. Your effort means a lot in improving our manuscript.*

Meta-review: This paper investigates tail risk minimization in meta-learning, providing theoretical analysis and a practical enhancement to the DR-MAML method. Reviewers generally found the paper technically sound with a good presentation and valuable theoretical contributions, including the Stackelberg game equivalence, convergence rate analysis, and generalization bounds. 
There were few initial concerns regarding the limited scope and incremental nature of the contribution (6Ej4), the fact that the practical benefits were also seen as tangential to the main theoretical contribution (ci3Y), and limited experimental evaluation that lacking comparisons to SOTA methods and larger models (6Ej4). These were subsequently addressed by authors during rebuttal stage by clarifying the broader applicability of their theoretical analysis and providing additional analysis / experiments. 
Overall, The paper presents a solid theoretical investigation of tail risk minimization in meta-learning and demonstrates its practical benefits. The authors effectively addressed initial concerns regarding scope, evaluation, and applicability to large models, leading to a consensus among reviewers on acceptance.
Decision: Accept (poster)