Title: Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Meta learning

Few-shot learning

Fast adaptation robustness
Abstract: Meta learning is a promising paradigm in the era of large models and 
task distributional robustness has become an indispensable consideration in real-world scenarios.
Recent advances have examined the effectiveness of tail task risk minimization in fast adaptation robustness improvement \citep{wang2023simple}.
This work contributes to more theoretical investigations and practical enhancements in the field.
Specifically, we reduce the distributionally robust strategy to a max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate.
In the presence of tail risk, we further derive the generalization bound, establish connections with estimated quantiles, and practically improve the studied strategy.
Accordingly, extensive evaluations demonstrate the significance of our proposal in boosting robustness.

Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning

Abstract

Meta learning is a promising paradigm in the era of large models and task distributional robustness has become an indispensable consideration in real-world scenarios. Recent advances have examined the effectiveness of tail task risk minimization in fast adaptation robustness improvement [1]. This work contributes to more theoretical investigations and practical enhancements in the field. Specifically, we reduce the distributionally robust strategy to a max-min optimization problem, constitute the Stackelberg equilibrium as the solution concept, and estimate the convergence rate. In the presence of tail risk, we further derive the generalization bound, establish connections with estimated quantiles, and practically improve the studied strategy. Accordingly, extensive evaluations demonstrate the significance of our proposal and its scalability to multimodal large models in boosting robustness.

1 Introduction

The past few years have witnessed a surge of research interest in meta learning due to its great potential in the academia and industry [2-5]. By leveraging previous experience, such a learning paradigm can extract knowledge as priors and empower learning models with adaptability to unseen tasks from a few examples [6].

Nevertheless, the investigation of the robustness needs to be more comprehensive from the task distribution perspective. In particular, the recently developed large models heavily rely on the fewshot learning capability and demand robustness of prediction in risk-sensitive scenarios [7]. For example, when the GPT-like dialogue generation system [8-10] comes into medical consultancy domains, imprecise answers can cause catastrophic consequences to patients, families, and even societies in real-world scenarios. In light of these considerations, it is desirable to watch adaptation differences across tasks when deploying meta learning models and promote task robustness study for meeting substantial practical demands.

Recently, Wang et al. [1] proposes to increase task distributional robustness via employing the tail risk minimization principle [11] for meta learning. In circumventing the optimization intractability in the presence of nonconvex risk functions, a two-stage optimization strategy is adopted as the heuristic to solve the problem. In brief, the strategy consists of two phases in iteration, respectively: (i) estimating the risk quantile VaR α [11] with the crude Monte Carlo method [12] in the task space; (ii) updating the meta learning model parameters from the screened subset of tasks. Such a strategy is simple in implementation, with an improvement guarantee under certain conditions, and empirically shows improved robustness when faced with task distributional shifts. Despite these advances, there remain several unresolved theoretical or practical issues in the field.

Existing limitations. This paper also works on the robustness of fast adaptation in the task space and tries to fill gaps in [1]. Theoretically, we notice that in [1] (i) there constitutes no notion of solutions, (ii) it lacks an algorithmic understanding of the two-stage optimization strategy, (iii) the analysis on generalization capability is ignored in the tail risk of tasks. Empirically, the use of the crude Monte Carlo might be less efficient in quantile estimates and suffers from a higher approximation error of the VaR α , degrading the adaptation robustness. These bottlenecks may weaken the versatility of the two-stage optimization strategy's use in practice and require more understanding before deployment.

Primary contributions. In response to the above-mentioned concerns, we propose translating the two-stage optimization strategy for distributionally robust meta learning [1] into a max-min optimization problem [13]. Intrinsically, this work models the optimization steps as a Stackelberg game, and task selection and the sub-gradient optimizer work as the leader and follower players in decision-making, respectively. The theoretical understanding is from two aspects:

We constitute the local Stackelberg equilibrium as a solution concept, estimate the convergence rate, and characterize the asymptotic behavior in learning dynamics.

We derive the generalization bound in the presence of the tail task risk, which connects quantile estimates with fast adaptation capability in unseen tasks.

Meanwhile, the empirical influence of VaR α estimators is examined, and we advance meta learners' robustness by comprising more accurate quantile estimators.

2 Literature Review

2.1 Meta Learning

Meta learning, or learning to learn , is an increasingly popular paradigm to distill knowledge from prior experience to unseen scenarios with a few examples [6]. Various meta learning methods have emerged in the past decade, and this section overviews some dominant families.

The context-based methods mainly use the encoder-decoder structure and represent tasks by latent variables. Typical ones are in the form of the conditional exchangeable stochastic processes and learn function distributions, such as neural processes [14], conditional neural processes [15] and their extensions [16-24]. The optimization-based approaches seek the optimal meta initialization of model parameters and update models from a few examples. Widely known are model agnostic meta learning [25] and related variants [26-29], such as MetaCurvature [30], which learns curvature information and transforms gradients in the inner-loop optimization. The metrics-based methods represent tasks in geometry and perform well in few-shot image classification [31-33]. For example, MetaOptNet [34] proposes to learn embeddings under a linear classifier and achieve SOTA few-shot classification performance. There also exist other methods, e.g., hyper-networks [35, 36], memory-augmented networks [37] and recurrent models [38].

2.2 Robustness & Generalization

The robustness concept in meta learning attracts recent attention, particularly when deploying large models in real-world scenarios. Admittedly, previous literature works have investigated the scenarios where the meta dataset's input is corrupted [39, 40] or the model parameter is perturbed [29]. Studies regarding the fast adaptation robustness in task distribution remain limited. Wang et al. [41] explicitly generates task distribution for robust adaptation. Collins et al. [42] employs the worst-case optimization for promoting MAML's robustness to extreme worst cases. With the help of tail risk minimization, Wang et al. [1] proposes two-stage optimization strategies to robustify the fast adaptation. This work centers around [1] but stresses more theoretical understandings and performance improvement points.

As for generalization capability, there are a couple of works in meta learning. Chen et al. [43] exploits the information theory to derive the bound for MAML's like methods. From the data splitting perspective, Bai et al. [44] formulates the theoretical foundation and connects it to optimality. In [45], an average risk bound is constructed with the bias for improving performance. Importantly, prior work [1] ignores the generalization analysis, and meta learner's generalization in tail risk cases has not been studied in the literature.

3 Preliminaries

General notations. Let p ( τ ) be the task distribution in meta learning. We respectively express the task space and the model parameter space as Ω τ and Θ . We denote the complete task set by T and refer to D τ as the meta dataset.

For instance, D τ comprises a collection of data points { ( x i , y i ) } n + m i =1 in regression. D τ is ususally prepared into the support set D S τ for skill transfer and the query set D Q τ to assess adaptation performance. Take the conditional neural process [15] as an example, D S τ = { ( x i , y i ) } n i =1 works for task representation with D Q τ = { ( x i , y i ) } n + m i =1 the all data points to fit in regression.

The meta risk function corresponds to a map ℓ : D τ × Θ ↦→ R + , evaluating fast adaptation performance. Given p ( τ ) and meta learning model parameters θ , we can induce the cumulative distribution of the meta risk function value in the real space as F ℓ ( l ; θ ) := P ( { ℓ ( D Q τ , D S τ ; θ ) ≤ l ; τ ∈ T , l ∈ R + } ) , but there is no explicit parameterized form for F ℓ in practice as F ℓ is θ -dependent.

When it comes to the tail risk minimization, we commonly use the conditional value-at-risk (CVaR α ) with the probability threshold α ∈ [0 , 1) . The quantile of our interest is called the value-at-risk (VaR α ) [11] with the definition: VaR α [ ℓ ( T , θ )] = inf l ∈ R + { l | F ℓ ( l ; θ ) ≥ α, τ ∈ T } . The resulting normalized cumulative distribution F α ℓ ( l ; θ ) is defined as:

∀ θ ∈ Θ , the meta learning operator M θ defines: M θ : τ ↦→ ℓ ( D Q τ , D S τ ; θ ) . Accordingly, the tail risk task subspace Ω α,τ := ⋃ ℓ ≥ VaR α [ ℓ ( T ,θ )] [ M -1 θ ( ℓ ) ] , with the task distribution constrained in Ω α,τ by p α ( τ ; θ ) . Please refer to Fig. 7 for illustrations of risk concepts.

Assumption 1. To proceed, we retain most assumptions from [1] for theoretical analysis, including:

The meta risk function ℓ ( D Q τ , D S τ ; θ ) is β τ -Lipschitz continuous w.r.t. θ ;

The cumulative distribution F ℓ ( l ; θ ) is β ℓ -Lipschitz continuous w.r.t. l , and the normalized density function p α ( τ ; θ ) is β θ -Lipschitz continuous w.r.t. θ ;

For arbitrary valid θ ∈ Θ and corresponding p α ( τ ; θ ) , ℓ ( D Q τ , D S τ ; θ ) is bounded: sup τ ∈ Ω α,τ ℓ ( D Q τ , D S τ ; θ ) ≤ L max .

3.1 Risk Minimization Principles

This subsection revisits commonly used risk minimization principles in the meta learning field.

Expected risk minimization. The standard principle is the expected/empirical risk minimization originated from statistical learning theory [46]. It minimizes meta risk based on the sampling chance of tasks from the original task distribution:

Worst-case risk minimization. Noticing that the worst fast adaptation can be disastrous in some risk sensitive scenarios, Collins et al. [42] proposes to conduct the worst-case optimization in meta learning:

However, as observed from experiments in [42], such a principle inevitably sacrifices too much average performance for gains of worst-case robustness. Meanwhile, it requires a couple of implementation tricks and specialized algorithms in stabilizing optimization.

Expected tail risk minimization (CVaR α ). To balance the average performance and the worst-case performance, Wang et al. [1] minimizes the expected tail risk, or equivalently CVaR α risk measure:

Due to no closed form of p α ( τ ; θ ) , Wang et al. [1] introduces a slack variable ξ ∈ R and reformulates the objective as follows:

The optimization objective involves the integral of quantiles in a continuous interval ( α, 1] , which is intractable to precisely parameterize with neural networks. The form in Eq. (4) utilizes the duality trick [11], enabling tractable sampling from the complete task space.

where v β := F -1 ℓ ( β ) denotes the quantile statistics and [ ℓ ( D Q τ , D S τ ; θ ) -ξ ] + := max { ℓ ( D Q τ , D S τ ; θ ) -ξ, 0 } is the hinge risk.

3.2 Examples & Two-stage Heuristic Strategies

Before delving deeper into the theoretical issues, we first present DR-MAML [1] as an instantiation to explain the expected tail risk minimization.

Example 1 (DR-MAML [1]) . Given p ( τ ) and vanilla MAML [25], the distributionally robust MAML within CVaR α can be written as a bi-level optimization problem:

where the gradient update w.r.t. the support set ∇ θ ℓ ( D S τ ; θ ) indicates the inner loop with a learning rate λ . The outer loop executes the gradient updates w.r.t. Eq. (5) and seeks the robust meta initialization in the parameter space.

Two-stage optimization strategies. Without loss of generality, we further detail the computational pipelines of Example 1 with two-stage optimization strategies. Note that MAML [25] is an optimization-based meta learning method, and the implementation is to execute the sub-gradient descent over a batch of tasks when updating the meta initialization θ meta :

Here, λ 1 and λ 2 are the inner loop and the outer loop learning rates, and the subscript t records the iteration number, with δ ( τ i ) the indicator variable. ˆ F MCB is the empirical distribution with B Monte Carlo task samples. δ ( τ i ) = 1 indicates the meta risk ℓ ( D Q τ i ; θ τ i t ) after fast adaptation falls into the defined tail risk region, otherwise δ ( τ i ) = 0 .

Throughout optimizing DR-MAML, Stage-I includes the fast adaptation w.r.t. individual task in Eq. (6a), and the quantile estimate in Eq. (6b). Stage-II applies the sub-gradient updates to the model parameters in Eq. (6c)/(6d). These two stages repeat until convergence is achieved.

4 Theoretical Investigations

Sub-gradient for Meta Parameter Update

Leader's Move (Stage-I):

(1) Risk Distribution Modeling with KDEs

(2) Optimal Subset Selection in the Task

Batch

Follower's Move (Stage-II):

Fast Adaptation

Meta Task Batch

Risk Density Modeling

This section presents theoretical insights into two-stage optimization strategies. We perform analysis from the algorithmic convergence, the asymptotic tail risk robustness, and the cross-task generalization capability in meta learning.

...

...

...

...

...

Estimation

Surrogate Function

Optimization

Improvement

Guarantee

Existence & Solution Concept

[

Proposition 2

]

[Theorem 4.1]

Generalization in Tail Risk

[Theorem 4.3]

CDF Approx. Error with KDE

[Theorem 4.4]

Asymptotic Performance Gap

[Theorem 4.2]

Equilibrium

Main Empirical Results

[Extensive Evaluation]

4.1 Distributionally Robust Meta Learning as a Stackelberg Game

Implementing the two-stage optimization strategy in meta learning requires first specifying the stages' order. The default is the minimization of the risk measure w.r.t. the parameter space after the maximization of the risk measure w.r.t. the task subspace. Hence, we propose to connect it to max-min optimization [13] and the Stackelberg game [47].

Max-min optimization. With the pre-assigned decision-making orders, the studied problem can be characterized as:

where Q α := { q ( τ ) |T q ⊆ T , ∫ τ ∈T q p ( τ ) dτ = 1 -α } constitutes a collection of uncertainty sets [48] over task subspace T q , and q ( τ ) is the normalized probability density over the task subspace. Note that in the expected tail risk minimization principle, there is no closed form of optimization objective Eq. (4) as the tail risk is θ -dependent. It is approximately interpreted as the max-min optimization when applied to the distribution over the uncertainty set Q α .

Proposition 1. The uncertainty set Q α is convex and compact in terms of probability measures.

Practical optimization is achieved via mini-batch gradient estimates and sub-gradient updates with the task size B in [1]; the feasible subsets correspond to all combinations of size ⌈B ∗ (1 -α ) ⌉ . Also, Eq. (7) is non-differentiable w.r.t. q ( τ ) , leaving previous approaches [49-52] unavailable in practice.

Stackelberg game & best responses. The example computational pipelines in Eq. (6) can be understood as approximately solving a stochastic two-player zero-sum Stackelberg game. Mathematically, such a game referred to as SG can be depicted as SG := ⟨P L , P F ; { q ∈ Q α } , { θ ∈ Θ } ; F ( q, θ ) ⟩ .

Moreover, we translate the two-stage optimization as decisions made by two competitors, which are illustrated in Fig. 1. The maximization operator executes in the task space, corresponding to the leader P L in SG with the utility function F ( q, θ ) . The follower P F attempts to execute sub-gradient updates over the meta learners' parameters via maximizing -F ( q, θ ) .

The two players compete to maximize separate utility functions in SG , which can be characterized as:

where the leader player P L specifies the worst case combinations from the uncertainty set Q α , and the follower P F reacts to the resulting normalized tail risk for increasing fast adaptation robustness.

It is worth noting that the update rules in Eq. (8) are also called best responses of players in game theory. The above procedures can be deemed the bi-level optimization [53] since the update of the meta learner implicitly depends on the leader's last time decision.

4.2 Solution Concept & Properties

The improvement guarantee has been demonstrated when employing two-stage optimization strategies for minimizing the tail risk in [1]. Furthermore, we claim that under certain conditions, there converges to a solution for the proposed Stackelberg game SG . The sufficient evidence is:

The two-stage optimization [1] results in a monotonic sequence:

Built on the boundness of risk functions and the theorem of improvement guarantee, such an optimization process can finally converge [54]. Then, a crucial question arises concerning the obtained solution: What is the notion of the convergence point in the game?

To answer this question, we need to formulate the corresponding solution concept in SG . Here, the global Stackelberg equilibrium is introduced as follows.

Definition 1 (Global Stackelberg Equilibrium) . Let ( q ∗ , θ ∗ ) ∈ Q α × Θ be the solution. With the leader q ∗ ∈ Q α and the follower θ ∗ ∈ Θ , ( q ∗ , θ ∗ ) is called a global Stackelberg equilibrium if the following inequalities are satisfied, ∀ q ∈ Q α and ∀ θ ∈ Θ ,

Proposition 2 (Existence of Equilibrium) . Given the Assumption 1, there always exists the global Stackelberg equilibrium as the Definition 1 for the studied SG .

Nevertheless, the existence of the global Stackelberg equilibrium can be guaranteed; it is NP-hard to obtain the equilibrium with existing optimization techniques. The same as that in [55], we turn to the local Stackelberg equilibrium as the Definition 2, where the notion of the local Stackelberg game is restricted in a neighborhood Q ′ α × Θ ′ in strategies.

Definition 2 (Local Stackelberg Equilibrium) . Let ( q ∗ , θ ∗ ) ∈ Q α × Θ be the solution. With the leader q ∗ ∈ Q α and the follower θ ∗ ∈ Θ , ( q ∗ , θ ∗ ) is called a local Stackelberg equilibrium for the leader if the following inequalities hold, ∀ q ∈ Q ′ α ,

The nature of nonconvex programming comprises the above local optimum, and we introduce concepts below for further analysis. It can be validated that F ( q, θ ) is a quasi-concave function w.r.t. q , meaning that for any positive number l ∈ R + , the set { q | q ∈ Q α , F ( q, θ ) > l } is convex in Q α . As a result, we deduce that there exists an implicit function h ( · ) : Θ →Q α such that the condition holds h ( θ ) = q with q = arg max ¯ q ∈Q α F (¯ q, θ ) . For the implicit function h , along with ∇ θ F ( q, θ ) , we make the Assumption below.

Assumption 2. The implicit function h ( · ) is β h -Lipschitz continuous w.r.t. θ ∈ Θ , and ∇ θ F ( q, θ ) is β q -Lipschitz continuous w.r.t. q ∈ Q α .

4.3 Convergence Rate & Generalization Bound

Learning to learn scales with the number of tasks, but the optimization process is computationally expensive [56-60], particularly when large language models are meta learners [3, 61, 62]. In training distributionally robust meta learners, estimating the convergence rate allows monitoring of the convergence and designing early stopping criteria to reach a desirable performance, reducing computational burdens [63]. Consequently, we turn to another question regarding the solution concept: What is the convergence rate of the two-stage optimization algorithm?

The runtime complexity for the leader's move can be easily estimated from subset selection, while the analysis for the follower is non-trivial. Under certain conditions, we can derive the following convergence rate theorem, where λ is the learning rate in gradient descent w.r.t. θ .

Theorem 4.1 (Convergence Rate for the Second Player) . Let the iteration sequence in optimization be: · · · ↦→ { q t -1 , θ t } ↦→ { q t , θ t +1 } ↦→ ··· ↦→ { q ∗ , θ ∗ } , with the converged equilibirum ( q ∗ , θ ∗ ) . Under the Assumption 2 and suppose that || I -λ ∇ 2 θθ F ( q ∗ , θ ∗ ) || 2 < 1 -λβ q β h , we can have lim t →∞ || θ t +1 -θ ∗ || 2 || θ t -θ ∗ || 2 ≤ 1 , and the iteration converges with the rate ( || I -λ ∇ 2 θθ F ( q ∗ , θ ∗ ) || 2 + λβ q β h ) when t approaches infinity.

Moreover, after executing the two-stage algorithm T time steps and given learned θ meta T , we can establish a bound on the asymptotic performance gap w.r.t. CVaR α in Theorem 4.2. For expositional clarity, we simplify ℓ ( D Q τ , D S τ ; θ ∗ ) , ℓ ( D Q τ , D S τ ; θ meta T ) , VaR α [ ℓ ( T , θ ∗ )] , and VaR α [ ℓ ( T , θ meta T )] as ℓ ∗ , ℓ meta , VaR ∗ α , and VaR meta α , respectively.

Theorem 4.2 (Asymptotic Performance Gap in Tail Task Risk) . Under the Assumption 1 and given a batch of tasks { τ i } B i =1 , we can have

For sufficiently large T , the first term can be bounded by a small number due to the convergence, and the second term vanishes since lim T →∞ ℓ meta = ℓ ∗ and lim T →∞ VaR meta α = VaR ∗ α , respectively.

Another crucial issue regarding meta learning lies in the fast adaptation capability in unseen cases. This drives us to answer the following question: How does the resulting meta learner generalize in the presence of tail task risk?

To this end, we first define R ( θ ∗ ) = E p α ( τ ) [ ℓ ∗ ] , ̂ R ( θ ∗ ) = 1 B ∑ B i =1 δ ( τ i ) ℓ ( D Q τ i , D S τ i ; θ ∗ ) , and ̂ R w ( θ ∗ ) = 1 B ∑ B i =1 p α ( τ i ) p ( τ i ) ℓ ( D Q τ i , D S τ i ; θ ∗ ) , where τ i ∼ p ( τ ) . Also note that the support of p α ( τ ; θ ∗ ) is within that of p ( τ ) , namely supp ( p α ( τ ; θ ∗ )) ⊆ supp ( p ( τ )) . Then we can induce Theorem 4.3 w.r.t. the tail risk generalization.

Theorem 4.3 (Generalization Bound in the Tail Risk Cases) . Given a collection of task samples { τ i } B i =1 and corresponding meta datasets, we can derive the following generalization bound in the presence of tail risk:

where the inequality holds with probability at least 1 -ϵ and ϵ ∈ (0 , 1) , V [ · ] denotes the variance operation, and L max is from the Assumption 1.

In conjunction with the confidence ϵ and a task batch B of significant size, Theorem 4.3 reveals the generalization bound given the meta-trained parameter θ ∗ . It is also associated with the variance V τ i ∼ p α ( τ ) [ ℓ ( D Q τ i , D S τ i ; θ ∗ )] . Besides, we also derive a specific bound in the case of MAML, and details are attached in Appendix Theorem C.1.

4.4 Practical Enhancements & Implementations

Theorem 4.3 reveals that an accurate estimate of VaR α yields a precise variance (i.e., V τ i ∼ p α ( τ ) [ ℓ ( D Q τ i , D S τ i ; θ ∗ )] ), leading to more reliable bounds. Accordingly, this section offers improvements over [1] via utilizing kernel density estimators (KDE) [64] for VaR α 's estimates. Compared to crude Monte Carlo (MC) methods, KDE can handle arbitrary complex distributions, capture local statistics well, and smoothen the cumulative function in a non-parametric way.

Specifically, we can construct KDE with a batch of task risk values { ℓ ( D Q τ i , D S τ i ; θ ) } B i =1 :

where K : R d → R is a kernel function, e.g., the Gaussian kernel, K ( x ) = exp( -|| x || 2 / 2) ∫ exp( -|| x || 2 / 2) dx , and h ℓ is the smoothing bandwidth. Once the KDE is built, it enables access to the quantile from the cumulative distribution functions or numeric integrals. The following Theorem 4.4 shows that KDE serves as a reliable approximation for VaR α .

Theorem 4.4. Let F -1 ℓ -KDE ( α ; θ ) = VaR KDE α [ ℓ ( T , θ )] and F -1 ℓ ( α ; θ ) = VaR α [ ℓ ( T , θ )] . Suppose that K ( x ) is lower bounded by a constant, ∀ x . For any ϵ > 0 , with probability at least 1 -ϵ , we can have the following bound:

As implied, one can close the distribution approximation gap by adopting a smaller, more flexible bandwidth. Additionally, KDE models offer a smooth estimate of the cumulative distribution function and require no prior assumptions.

Remark 1. In addition to smoothness, flexibility, and distribution agnostic traits, KDE in adoption can enhance the studied method's generalization capability. The crude Monte Carlo used in [1] typically incurs an error of approximately O ( 1 √ B ) in estimating quantiles [65]. In contrast, that of KDE is no more than ( h ℓ √ B∗ log B ) from Theorem 4.4.

O

5 Empirical Findings

Prior sections mainly focus on the theoretical understanding of two-stage distributionally robust strategies. This section conducts extensive experiments on a broader range of benchmarks and examines the improvement tricks, e.g., the use of KDE for quantile estimates, from empirical results.

Benchmarks & baselines. We perform experiments on the few-shot regression, system identification, image classification, and meta reinforcement learning, where most of them keep setups the same as prior work [1, 42]. We evaluate the methods from risk minimization principles and corresponding indicators, including expected/empirical risk minimization (Average), worst-case risk minimization (Worst), and tail risk minimization (CVaR α ).

MAML mainly works as the base meta learner, and we term the KDE-augmented DR-MAML as DR-MAML+. Then we compare DR-MAML+ with several baselines, including vanilla MAML [25], TR-MAML [42], DRO-MAML [66] and DR-MAML [1].

Average

Worst

CVaR

α

1

2

3

4

5

Sinusoid 5-shot

Average

Worst

CVaR

α

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

3

.

5

Sinusoid 10-shot

Mean Square Errors

MAML

TR-MAML

DRO-MAML

DR-MAML

DR-MAML+(Ours)

5.1 Sinusoid Regression

The goal of the sinusoid regression [25] is to quickly fit an underlying function f ( x ) = A sin( x -B ) from K randomly sampled data points, and tasks are specified by ( A,B ) . The meta-training and testing setups are the same as that in [1, 42], where many easy functions with a tiny fraction of difficult ones are included in the training.

Result & analysis. As illustrated in Fig. 3, we can observe that DR-MAML+ consistently outperforms all baselines across average and CVaR α indicators in the 5-shot case. Though the average performance slightly lags behind DR-MAML in the 10-shot case, DR-MAML+ surpasses other baselines in both the Worst and CVaR α indicators. This implies that DR-MAML+ exhibits more

Average

Worst

CVaR

α

0

.

5

1

.

0

1

.

5

2

.

0

Pendulum 10-shot

Average

Worst

CVaR

α

0

.

5

1

.

0

1

.

5

2

.

0

Pendulum 20-shot

Mean Square Errors

MAML

TR-MAML

DRO-MAML

DR-MAML

DR-MAML+(Ours)

robustness in challenging task distributions, e.g., 5-shot case. Furthermore, the standard error associated with our method is significantly smaller than others, underscoring the stability of DR-MAML+.

5.2 System Identification

The system identification corresponds to learning a dynamics model from a few collected transitions in physics systems. Here, we consider the Pendulum system and create diverse dynamical systems by varying its mass m and length l , with ( m,l ) ∼ U ([0 . 4 , 1 . 6] , [0 . 4 , 1 . 6]) . A random policy collects transitions for meta training, and 10 random transitions work as a support dataset.

Result & analysis. Fig. 4 shows no significant difference between 10-shot and 20-shot cases. DR-MAML+ dominates the performance across all indicators in both cases. Due to the min-max optimization, TR-MAML behaves well in the worst-case but sacrifices too much average performance. Within the studied strategies, DR-MAML+ exhibits an advantage over DR-MAML regarding CVaR α .

5.3 Few-shot Image Classification

We perform few-shot image classification on the mini -ImageNet dataset [67], with the same setup in [42]. The task is a 5-way 1-shot classification problem. And 64 classes are selected for constructing meta-training tasks, with the remaining 32 classes for meta-testing.

Result & analysis. In Table 1, methods within a two-stage distributionally robust strategy, namely DR-MAML and DR-MAML+, show superiority to others across all indicators in both training and testing scenarios, which is similar to empirical findings in [1]. Interstingly, DR-MAML+ and DRMAMLare comparable in most scenarios, and we attribute this to the small batch size in training, which weakens KDE's quantile approximation advantage.

5.4 Meta Reinforcement Learning

Here, we take 2-D point robot navigation as the meta reinforcement learning benchmark in evaluation. The goal is to reach the target destination with the help of a few exploration transitions for fast adaptation, and we retain the setup in MAML [25]. In meta

testing, we randomly sample 80 navigation goals and examine methods' navigation performance.

Result & analysis. As reinforcement learning methods fluctuate fiercely in worst-case indicators, we only report Average and CVaR α returns in Table 2. We observe that using studied strategies in DR-MAML enhances the returns. DR-MAML+benefits from a more reliable quantile estimate and achieves superior performance. The application of distributional robustness to reinforcement learning yields improvements in returns.

20

30

40

50

60

70

Task Batch Size

0

.

02

0

.

04

0

.

06

Sinusoid 5-shot

MC

KDE

20

30

40

50

60

70

Task Batch Size

0

.

005

0

.

010

0

.

015

0

.

020

0

.

025

Sinusoid 10-shot

MC

KDE

Approximation Error

5.5 Assessment of Quantile Estimators

With the meta trained model, e.g., DR-MAML+ in sinusoid regression, we collect the testing task risk values with different task batch sizes to estimate the VaR α from respectively the crude MC and KDE. As observed from Fig. 5, the VaR α approximation error decreases with more tasks, and the KDE produces more accurate estimates with a sharper decreasing trend. The above well verifies the conclusion in Theorem 4.3.

5.6 Empricial Result Summarization

Here, we summarize two points from the above empirical results and associated theorems. (i) From Theorem 4.2/4.3 and Fig. 3/4/5: the VaR α estimate relates to the reliable generalization bound, and cumulated tiny approximation errors along iterations potentially result in worse equilibrium. (ii) From Theorem 4.3/4.4, Remark 1, Fig. 3, and Table 1/2: with the studied strategy, the KDE is a better choice of task risk distribution modelling than the crude MC in tougher benchmarks, e.g., 5-shot sinusoid regression, meta-testing mini -ImageNet classification, and point robot navigation.

5.7 Compatibility with Large Models



CVaR



















CVaR























CVaR































Improved Robustness in Evaluation: As illustrated in Fig. 6, DR-MaPLe and DR-MaPLe+ consistently outperform baselines across both average and indicators in cases, demonstrating the advantage of the two-stage strategy in enhancing the robustness of few-shot learning. DR-MaPLe+ achieves better results as KDE quantiles are more accurate with large batch sizes. These results confirm the scalability and compatibility of our method on large models.

Learning Efficiency as Limitations: In terms of implementation time and memory cost, we retain the setup the same as that in [1]: use the same maximum number of meta gradient updates for all baselines in training processes, which means given α = 0 . 5 , the tail risk minimization principle requires double task batches to evaluate and screen sub-batches. It can be seen that both DR-MaPLe and DR-MaPLe+ consume more memories, and the extra training time over MaPLe arises from the evaluation and sub-batch screening in the forward pass. Such additional computations and memory costs raise computational and memory efficiency issues for exchanging extra significant robustness improvement in fast adaptation.

6 Conclusion

To conclude, this paper proposes to understand the two-stage distributionally robust strategy from optimization processes, define the convergence solution, and derive the generalization bound in the presence of tail task risk. Extensive experiments validate the studied improvement tricks and reveal more empirical properties of the studied strategy. We leave computational overhead reduction as a promising topic for future exploration in robust fast adaptation.

References

Qi Wang, Yiqin Lv, Yanghe Feng, Zheng Xie, and Jincai Huang. A simple yet effective strategy to robustify the meta learning paradigm. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Yuanfu Lu, Yuan Fang, and Chuan Shi. Meta-learning on heterogeneous information networks for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 1563-1573, 2020.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Yi Yuan, Gan Zheng, Kai-Kit Wong, and Khaled B Letaief. Meta-reinforcement learning based resource allocation for dynamic v2x communications. IEEE Transactions on Vehicular Technology , 70(9):8964-8977, 2021.

Brenden M Lake and Marco Baroni. Human-like systematic generalization through a metalearning neural network. Nature , pages 1-7, 2023.

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence , 44 (9):5149-5169, 2021.

Qi Wang, Yanghe Feng, Jincai Huang, Yiqin Lv, Zheng Xie, and Xiaoshan Gao. Large-scale generative simulation artificial intelligence: The next hotspot. The Innovation , page 100516, 2023.

Young-Jun Lee, Chae-Gyun Lim, and Ho-Jin Choi. Does gpt-3 generate empathetic dialogues? a novel in-context example selection method and automatic evaluation metric for empathetic dialogue generation. In Proceedings of the 29th International Conference on Computational Linguistics , pages 669-683, 2022.

Chen Tang, Hongbo Zhang, Tyler Loakman, Chenghua Lin, and Frank Guerin. Terminologyaware medical dialogue generation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1-5. IEEE, 2023.

Bharath Chintagunta, Namit Katariya, Xavier Amatriain, and Anitha Kannan. Medically aware gpt-3 as a data generator for medical dialogue summarization. In Machine Learning for Healthcare Conference , pages 354-372. PMLR, 2021.

R Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk , 2:21-42, 2000.

Dirk P Kroese and Reuven Y Rubinstein. Monte carlo methods. Wiley Interdisciplinary Reviews: Computational Statistics , 4(1):48-58, 2012.

John M Danskin. The theory of max-min, with applications. SIAM Journal on Applied Mathematics , 14(4):641-664, 1966.

Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint arXiv:1807.01622 , 2018.

Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International Conference on Machine Learning , pages 1704-1713. PMLR, 2018.

Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard Turner. Meta-learning probabilistic inference for prediction. In International Conference on Learning Representations , 2018.

Qi Wang, Marco Federici, and Herke van Hoof. Bridge the inference gaps of neural processes via expectation maximization. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=A7v2DqLjZdq .

Andrew Foong, Wessel Bruinsma, Jonathan Gordon, Yann Dubois, James Requeima, and Richard Turner. Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems , 33:8284-8295, 2020.

Qi Wang and Herke Van Hoof. Doubly stochastic variational inference for neural processes with hierarchical latent variables. In International Conference on Machine Learning , pages 10018-10028. PMLR, 2020.

Muhammad Waleed Gondal, Shruti Joshi, Nasim Rahaman, Stefan Bauer, Manuel Wuthrich, and Bernhard Scholkopf. Function contrastive learning of transferable meta-representations. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , volume 139 of Proceedings of Machine Learning Research , pages 3755-3765. PMLR, 2021.

Qi Wang and Herke van Hoof. Learning expressive meta-representations with mixture of expert neural processes. In Advances in neural information processing systems , 2022.

Juho Lee, Yoonho Lee, Jungtaek Kim, Eunho Yang, Sung Ju Hwang, and Yee Whye Teh. Bootstrapping neural processes. Advances in neural information processing systems , 33:66066615, 2020.

Qi Wang and Herke Van Hoof. Model-based meta reinforcement learning using graph structured surrogate models and amortized policy search. In International Conference on Machine Learning , pages 23055-23077. PMLR, 2022.

Jiayi Shen, Xiantong Zhen, Marcel Worring, et al. Episodic multi-task learning with heterogeneous neural processes. arXiv preprint arXiv:2310.18713 , 2023.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126-1135. PMLR, 2017.

Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances in neural information processing systems , 32, 2019.

Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930 , 2018.

Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. Advances in neural information processing systems , 31, 2018.

Momin Abbas, Quan Xiao, Lisha Chen, Pin-Yu Chen, and Tianyi Chen. Sharp-maml: Sharpnessaware model-agnostic meta learning. In International Conference on Machine Learning , pages 10-32. PMLR, 2022.

Eunbyung Park and Junier B Oliva. Meta-curvature. Advances in neural information processing systems , 32, 2019.

Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems , 30, 2017.

Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum. Infinite mixture prototypes for few-shot learning. In International Conference on Machine Learning , pages 232-241. PMLR, 2019.

Sergey Bartunov and Dmitry Vetrov. Few-shot generative modelling with generative matching networks. In International Conference on Artificial Intelligence and Statistics , pages 670-678. PMLR, 2018.

Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019.

Jacob Beck, Matthew Thomas Jackson, Risto Vuorio, and Shimon Whiteson. Hypernetworks in meta-reinforcement learning. In Conference on Robot Learning , pages 1478-1487. PMLR, 2023.

Dominic Zhao, Johannes von Oswald, Seijin Kobayashi, João Sacramento, and Benjamin F Grewe. Meta-learning via hypernetworks. 4th Workshop on Meta-Learning at NeurIPS 2020 , 2020.

Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In International conference on machine learning , pages 1842-1850. PMLR, 2016.

Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel. Rl2: Fast reinforcement learning via slow reinforcement learning. arXiv preprint arXiv:1611.02779 , 2016.

Ren Wang, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Chuang Gan, and Meng Wang. On fast adversarial robustness adaptation in model-agnostic meta-learning. In International Conference on Learning Representations , 2020.

Micah Goldblum, Liam Fowl, and Tom Goldstein. Adversarially robust few-shot learning: A meta-learning approach. Advances in Neural Information Processing Systems , 33:17886-17895, 2020.

Cheems Wang, Yiqin Lv, Yixiu Mao, Yun Qu, Yi Xu, and Xiangyang Ji. Robust fast adaptation from adversarially explicit task distribution generation. arXiv preprint arXiv:2407.19523 , 2024.

Liam Collins, Aryan Mokhtari, and Sanjay Shakkottai. Task-robust model-agnostic metalearning. Advances in Neural Information Processing Systems , 33:18860-18871, 2020.

Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoretic analysis. Advances in Neural Information Processing Systems , 34: 25878-25890, 2021.

Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, Jason Lee, Sham Kakade, Huan Wang, and Caiming Xiong. How important is the train-validation split in meta-learning? In International Conference on Machine Learning , pages 543-553. PMLR, 2021.

Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, and Massimiliano Pontil. Learning-to-learn stochastic gradient descent with biased regularization. In International Conference on Machine Learning , pages 1566-1575. PMLR, 2019.

Vladimir Vapnik. The nature of statistical learning theory . Springer science & business media, 1999.

Tao Li and Suresh P Sethi. A review of dynamic stackelberg game models. Discrete & Continuous Dynamical Systems-B , 22(1):125, 2017.

Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Management Science , 59(2):341-357, 2013.

Dimitri P Bertsekas. Nonlinear programming. Journal of the Operational Research Society , 48 (3):334-334, 1997.

Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning , pages 6083-6093. PMLR, 2020.

Pierre Loridan and Jacqueline Morgan. Weak via strong stackelberg problem: new results. Journal of global Optimization , 8:263-287, 1996.

Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In International conference on artificial intelligence and statistics , pages 1540-1552. PMLR, 2020.

Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(12):10045-10067, 2021.

Tom M Apostol. Mathematical analysis . 1974.

Tanner Fiez, Benjamin Chasnov, and Lillian Ratliff. Implicit learning dynamics in stackelberg games: Equilibria characterization, convergence analysis, and empirical study. In International Conference on Machine Learning , pages 3133-3144. PMLR, 2020.

Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research. In Proceedings of the AAAI conference on artificial intelligence , volume 34, pages 13693-13696, 2020.

David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 , 2021.

Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv:2102.01293 , 2021.

Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang, Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, et al. Summary of chatgpt-related research and perspective towards the future of large language models. Meta-Radiology , page 100017, 2023.

Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences , 103:102274, 2023.

Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021 , pages 3816-3830. Association for Computational Linguistics (ACL), 2021.

Ahmad Faiz, Sotaro Kaneda, Ruhan Wang, Rita Osi, Parteek Sharma, Fan Chen, and Lei Jiang. Llmcarbon: Modeling the end-to-end carbon footprint of large language models. arXiv preprint arXiv:2309.14393 , 2023.

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022.

Mats Rudemo. Empirical choice of histograms and kernel density estimators. Scandinavian Journal of Statistics , pages 65-78, 1982.

Hui Dong and Marvin K Nakayama. A tutorial on quantile estimation via monte carlo. Monte Carlo and Quasi-Monte Carlo Methods: MCQMC 2018, Rennes, France, July 1-6 , pages 3-30, 2020.

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations , 2020.

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems , 29, 2016.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19113-19122, 2023.

Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, and Xiangyang Ji. Supported trust region optimization for offline reinforcement learning. In International Conference on Machine Learning , pages 23829-23851. PMLR, 2023.

Hongchang Zhang, Yixiu Mao, Boyuan Wang, Shuncheng He, Yi Xu, and Xiangyang Ji. Insample actor critic for offline reinforcement learning. In The Eleventh International Conference on Learning Representations , 2023.

Yixiu Mao, Hongchang Zhang, Chen Chen, Yi Xu, and Xiangyang Ji. Supported value regularization for offline reinforcement learning. Advances in Neural Information Processing Systems , 36, 2024.

Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszczynski. Lectures on stochastic programming: modeling and theory . Society for industrial Mathematics, 2009.

Jianzhun Shao, Yun Qu, Chen Chen, Hongchang Zhang, and Xiangyang Ji. Counterfactual conservative q learning for offline multi-agent reinforcement learning. Advances in Neural Information Processing Systems , 36, 2024.

Yun Qu, Boyuan Wang, Jianzhun Shao, Yuhang Jiang, Chen Chen, Zhenbin Ye, Liu Linc, Yang Feng, Lin Lai, Hongyang Qin, et al. Hokoff: real game dataset from honor of kings and its offline reinforcement learning benchmarks. Advances in Neural Information Processing Systems , 36, 2024.

Lorna I Paredes and Chew Tuan Seng. Controlled convergence theorem for banach-valued hl integrals. Scientiae Mathematicae Japonicae , 56(2):347-358, 2002.

Chi Jin, Praneeth Netrapalli, and Michael Jordan. What is local optimality in nonconvexnonconcave minimax optimization? In International conference on machine learning , pages 4880-4889. PMLR, 2020.

Stephen P Boyd and Lieven Vandenberghe. Convex optimization . Cambridge university press, 2004.

C Frappier and QI Rahman. On an inequality of s. bernstein. Canadian Journal of Mathematics , 34(4):932-944, 1982.

Rong Liu and Lijian Yang. Kernel estimation of multivariate cumulative distribution function. Journal of Nonparametric Statistics , 20(8):661-677, 2008.

Srh Larochelle. Optimization as a model for few-shot learning. In International Conference on Learning Representations , 2017.

Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 , 2018.

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 15262-15271, 2021.

Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems , 32, 2019.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , pages 8024-8035, 2019. URL https://proceedings.neurips.cc/paper/2019/hash/ bdbca288fee7f92f2bfa9f7012727740-Abstract.html .

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. { TensorFlow } : a system for { Large-Scale } machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16) , pages 265-283, 2016.

Contents

A Quick Guide to This Work

This section mainly includes explanations and clarifications on this work.

A.1 Technical Comparison in Robust Fast Adaptation

Primary differences: As far as we know, literature work is quite limited regarding fast adaptation robustness in the task space. TR-MAML and DR-MAML are the most recent and typical ones that can handle task distributional shift scenarios well. As reported in Table 3, TR-MAML only focuses on the worst-case, which considers a bit extreme and rarely occurred cases. DRO-MAML is a new baseline, where the uncertainty set Q is included for robust fast adaptation, hence there exists no theoretical analysis. As for the tail task risk, DR-MAML lacks generalization capability and convergence rate analysis w.r.t. the meta learner. The meta learner in DR-MAML+ is a more specific instantiation of that in DR-MAML. We claim that these theoretical understanding is necessary in the presence of the robust fast adaptation due to its potential applications in large models.

Theoretical and empirical insights: In comparison, this work not only contributes to the Stackelberg game for estimates, but also derives the generalization and the asymptotic performance gap in iterations based on a normalized but non-differentiable probability density space . Note that we lean more focus on theoretical understanding and pursuing SOTA performance is not the ultimate purpose of this work. The connections between different quantile estimators and generalization bound highlighted in Theorem 4.3/4.4 and Remark 1 reveal the theoretical advantage of KDEs over crude Monte Carlo methods. This motivates us to replace crude Monte Carlo with KDEs. Such a replacement as a simple implementation trick is supported by rigorous theoretical analysis. The empirical results align with theoretical understanding.

In terms of improving the studied strategy, investigations in extensive experiments seem meaningful for practical implementations, and some non-trivial discoveries together with improvement tricks are also reported , such as the relationship between quantile estimate errors and adaptation robustness, the batch size's influence on several benchmarks, etc.

In this work, the theoretical and empirical parts are connected in an implicit manner. The generalization capability is empirically examined from experimental results, and the performance gap between DR-MAML+ and DR-MAML can be attributed to the difference in generalization bounds. As for the convergence trait and asymptotic performance, the insight might guide the optimization process in training large models, such as early stopping criteria design.

A.2 Significance of Theoretical Understandings

As pointed out in [3, 61, 62], large language models are few-shot learners. When a large model, such as a large decision-making model in the future, comes into practice, fast adaptation robustness can be a crucial issue as real-world scenarios are indeed risk-sensitive.

This work takes the latest work [1] as an example, and the interest is in the theoretical aspect. Most of the assumptions in this work are from [1]. The baselines are typical and latest, while the benchmarks

cover diverse downstream tasks. In multimodal few-shot image classification experiments, our contributed points help guide the development of large models in terms of training and robustness enhancement. Our investigations also provide insight into robust policy optimization, particularly when safety is one necessary consideration [70-72].

Shapiro et al.'s book [73] is a comprehensive resource that addresses stochastic modeling and optimization methods, but it does not explore solution concepts in game theory or define generalization bounds relevant to meta learning and deep learning. Instead, our work further enriches the stochastic programming theory in meta learning, connects it to the Stackelberg game, and contributes to tail risk generalization bounds, convergence rates, asymptotic properties, and so on. Therefore, the solution concept and the theoretical properties are specific to our meta learning setup, distinctly from the scope covered by [73].

A.3 Meanings of Indicators and Terms

Illustration of VaR α , CDF and others: Fig. 7 illustrates a typical probability distribution, cumulative distribution, and the resulting mean, VaR α , and CVaR α . Given α ∈ [0 , 1) , VaR α is the α quantile of the risk distribution. Specially, VaR 0 . 5 coincides with the mean. Upon the definition of VaR α , CVaR α can be define as CVaR α = E p ( τ ) [ ℓ | ℓ ≥ VaR α ] . That is, CVaR α is the expectation of the risks of the 1 -α tail of the distribution. Relative to the original probability distribution, CVaR α can be interpreted as a certain distribution shift , which reweighs arbitrary risk exceeding VaR α up to a coefficient 1 .

1 -α

Meaning of the asymptotic performance gap: We plot Fig. 8 to display the gap between the CVaR α value in iterations and that in the convergence. The area difference depicts this gap.

A.4 Computational Complexity

Analyzing computational complexity across all meta-learning methods is inherently challenging due to the diversity in methodological approaches within the field. Meta-learning encompasses a wide range of techniques, including gradient-based methods, which rely on iterative updates to model parameters, and non-parametric methods, which may instead focus on instance-based learning or kernel-based approaches. Therefore, the space complexity is specific to the meta-learning method, while this work is agnostic to it. Here, we report the computational complexity for the DR-

MAML+ as

(

(

α

)

)

while using KDE with the Gaussian kernel, and that of DR-MAML

O

B

B -

|M|

A.5 Broader Impact & Future Extensions

This paper presents work whose goal is to advance the field of robust meta learning. There are many potential societal consequences of our work, which we detail as follows.

The fast adaptation robustness is an urgent concern, particularly in large models and risk-sensitive control. This work provides versatile insights for theoretical analysis and performance improvement in the presence of tail task risk, and future explorations can be decision-making scenarios, such as multi-agent policy optimization [74, 75], and computational/memory cost reduction.

B Pseudo Algorithms

For a better understanding of the game theoretical optimization, we take DR-MAML+ and DR-CNP+ as examples and include the Pseudo Algorithms 1/2 in this section. Particularly, the algorithms specify the decision-making orders and highlight the use of KDE modules to build task risk value distributions and estimate the quantile.

Cumulative Distribution

Probability Density

1

a

0.5

Mean VaRaCVaRa

Risk

AsymptoticPerformanceGapinBi-level Optimization

Frequencies

VaR&

Riskt

J = {t: t ≥ VaRa,emeta < VaRmeta}

C Expressions, Theorems & Proofs

C.1 Characterization of Optimization Processes

Without loss of generality, we can also express the process of solving the studied Stackelberg game as:

where the optimization w.r.t. ( q, θ ) is the computation of the best responses for two adversarial players. As a bi-level optimization in Eq. (14a)/(14b), the exact solution is intractable to obtain in a theoretical sense, and the two-stage distributionallly robust optimization is a heuristic approach.

Meaning of the obtained equilibrium. Here, we can interpret the obtained solution ( q ∗ , θ ∗ ) from solving Eq. (7) as follows. Given the follower's decision θ ∗ and the induced task risk distribution F ℓ ( l ; θ ∗ ) , the leader cannot further raise a proposal of a task subset with a probability 1 -α to degradde the tailed expected performance. And this explains the meaning of robust fast adaptation solution w.r.t. the tail task risk.

Algorithm 1: Meta-training DR-MAML+ as A Stackelberg Game

Algorithm 2: Meta Training DR-CNP+ as A Stackelberg Game

C.2 Assumptions

We list all of the assumptions mentioned in this work. These assumptions further serve the demonstration of propositions and theorems in the main paper.

Assumption 1. To proceed, we retain most assumptions from [1] for theoretical analysis, including:

The meta risk function ℓ ( D Q τ , D S τ ; θ ) is β τ -Lipschitz continuous w.r.t. θ ;

The cumulative distribution F ℓ ( l ; θ ) is β ℓ -Lipschitz continuous w.r.t. l , and the normalized density function p α ( τ ; θ ) is β θ -Lipschitz continuous w.r.t. θ ;

For arbitrary valid θ ∈ Θ and corresponding p α ( τ ; θ ) , ℓ ( D Q τ , D S τ ; θ ) is bounded: sup τ ∈ Ω α,τ ℓ ( D Q τ i , D S τ i ; θ ) ≤ L max .

Assumption 2. The implicit function h ( · ) is β h -Lipschitz continuous w.r.t. θ ∈ Θ , and ∇ θ F ( q, θ ) is β q -Lipschitz continuous w.r.t. q ∈ Q α .

C.3 Proof of Proposition 1

Proposition 1. The uncertainty set Q α is convex and compact in terms of probability measures.

Proof: We firstly focus on the convexity of Q α . For any { q 1 := q 1 ( τ ) , q 2 := q 2 ( τ ) } ∈ Q α , we partition these two task spaces with non-zero sampling probability mass respectively as T 1 ∪ T C and T 2 ∪ T C . As displayed in Fig. 9, T C denotes the shared subset task between q 1 and q 2 . Below we show that λ 1 q 1 + λ 2 q 2 ∈ Q α with λ 1 + λ 2 = 1 . This is true because

We next demonstrate the compactness of Q α . The distance between two distributions ∀{ q 1 , q 2 } ∈ Q α can be defined as:

Since L 1 space is a Banach space, the compactness is equivalent to the closedness and Boundedness of Q α . Considering a sequence { q n ( τ ) ∈ Q α } with the resulting limitation is q ∗ ( τ ) , following the C ontrolled Convergence Theorem [76], we know that

Due to the symmetry of the distance, we can have lim n →∞ ∫ τ ∈T p ∗ ( τ ) -p n ( τ ) dτ ≤ 0 . Thus,

That is, p ∗ ( τ ) ∈ Q α , indicating that Q α is a closed set. As the boundedness is clear in the studied problem, this completes the proof of Proposition 1. ■

C.4 Proof of Proposition 2

Proposition 2 (Existence of Equilibrium) Given the Assumption 1, there always exists the global Stackelberg equilibrium as the Definition 1 for the studied SG .

Proof: Note that Θ is compact as a subspace of the Euclidean space. And it is trivial to see that F ( q, θ ) := E q [ ℓ ( D Q τ , D S τ ; θ ) ] is continuous w.r.t. θ ∈ Θ as ℓ satisfies the β τ -Lipschitz continuity in the Assumption 1.

Here we need to show the continuity of F ( q, θ ) w.r.t. the collection of probability measures or probability functions Q α . To this end, with ∀ θ ∈ Θ fixed, We consider two metric spaces ( Q α , d Q α ) and ( L , R L ) . The map of our interest is g ( q ) = F ( q, · ) : Q α ↦→L⊆ R + .

Naturally, we can have the following inequality:

which implies 3 L max-Lipschitz continuity of g ( q ) w.r.t. ∀ q ∈ Q α .

According to the Remark in [77], there always exists the global Stackelberg equilibrium as the Definition 1 when Q α × Θ is compact and F ( q, θ ) is continuous. This completes the proof of Proposition 2. ■

C.5 Proof of Quasi-concavity for F ( q, θ ) w.r.t. q

It can be validated that F ( q, θ ) is a quasi-concave function w.r.t. q , meaning that for any positive number l ∈ R + , the set { q | q ∈ Q α , F ( q, θ ) > l } is convex in Q α .

Proof: According to the conventional definition (i.e., the superlevel set is convex [78]), for all λ 1 + λ 2 = 1 , q 1 , q 2 ∈ { q |F ( q, θ ) > l } , we can have

Thus, λ 1 q 1 + λ 2 q 2 ∈ { q |F ( q, θ ) > l } and the superlevel set is convex, implying that F ( q, θ ) is quasi-concave w.r.t. q . ■

C.6 Proof of Theorem 4.1

Theorem 4.1 (Convergence Rate for the Second Player) Let the iteration sequence in optimization be: · · · ↦→ { q t -1 , θ t } ↦→ { q t , θ t +1 } ↦→ ··· ↦→ { q ∗ , θ ∗ } , with the converged equilibirum ( q ∗ , θ ∗ ) . Under the Assumption 2 and suppose that || I -λ ∇ 2 θθ F ( q ∗ , θ ∗ ) || 2 <

1 -λβ q β h , we can have lim t →∞ || θ t +1 -θ ∗ || 2 || θ t -θ ∗ || 2 ≤ 1 , and the iteration converges with the rate ( || I -λ ∇ 2 θθ F ( q ∗ , θ ∗ ) || 2 + λβ q β h ) .

Proof: Let the resulting stationary point be [ q ∗ , θ ∗ ] , we denote the difference terms by ˆ q = q -q ∗ and ˆ θ = θ -θ ∗ . Then, according to the optimization step, we can have the following equations:

Now we perform the first-order Taylor expansion of the θ related function ∇ θ F ( q t ; θ ) around θ ∗ and can derive:

Then we have the following result with the help of Assumption 2:

With Eq. (19), Eq. (20) and Eq. (21), we can derive the equation that:

Thus, when ∥ I -λ ∇ 2 θθ F ( q ∗ ; θ ∗ ) ∥ 2 < 1 -λβ q β h , we have

This completes the proof of Theorem 4.1.

C.7 Proof of Theorem 4.2

Theorem 4.2 (Asymptotics in the Tail Risk Cases) Under the Assumption 1 and given a batch of tasks { τ i } B i =1 , we can have

■

Proof. Given a batch of tasks { τ 1 , · · · , τ B } and according to the definition of CVaR α , we have

In inequality (25f), T 1 = { τ : ℓ ∗ < VaR ∗ α , ℓ meta ≥ VaR meta α } , T 2 = { τ : ℓ ∗ ≥ VaR ∗ α , ℓ meta < VaR meta α } , T 3 = { τ : ℓ ∗ < VaR ∗ α , ℓ meta < VaR meta α } , T 4 = { τ : ℓ ∗ ≥ VaR ∗ α , ℓ meta ≥ VaR meta α } . Moreover, this inequality holds due to the β τ -Lipschitz continuous of ℓ ( D τ ; θ ) . In Eq. (25g), p α ( τ ; θ meta T ) = p α ( τ ; θ ∗ ) = 0 when τ ∈ T 3 , and p α ( τ ; θ meta T ) = p α ( τ ; θ ∗ ) = p ( τ ) 1 -α when τ ∈ T 4 . Thus, we complete the proof of Theorem 4.2. ■

C.8 Proof of Theorem 4.3

Theorem 4.3 (Generalization Bound in the Tail Risk Cases) Given a collection of task samples { τ i } B i =1 and corresponding meta datasets, we can derive the following generalization bound in the presence of tail risk:

where the inequality holds with probability at least 1 -ϵ and ϵ ∈ (0 , 1) , V [ · ] denotes the variance operation, and L max is from the Assumption 1.

Proof. R ( θ ∗ ) -̂ R ( θ ∗ ) can be decomposed to two parts, i.e.,

For the first part (i.e., R ( θ ∗ ) -̂ R w ( θ ∗ ) ), we will adopt the Bernstein's inequality to provide an upper bound. Regarding p α ( τ i ) p ( τ i ) ℓ ( D Q τ i , D S τ i ; θ ∗ ) -R ( θ ∗ ) as a random variable with respect to τ i and according to Assumption 1, we know that

Thus, following Bernstein's inequality [79], we know that

where

Setting ϵ to match the upper bound in inequality (29c) shows that with probability at least 1 -ϵ , the following bound holds:

For the second part (i.e., ̂ R w ( θ ∗ ) -̂ R ( θ ∗ ) ), we have

In summary, we can obtain an upper bound of R ( θ ∗ ) -̂ R ( θ ∗ ) . That is,

This completes the proof of Theorem 4.3.

C.9 Proof of Theorem 4.4

Theorem 4.4 Let F -1 ℓ -KDE ( α ; θ ) = VaR KDE α [ ℓ ( T , θ )] and F -1 ℓ ( α ; θ ) = VaR α [ ℓ ( T , θ )] . Suppose that K ( x ) is lower bounded by a constant, ∀ x . For any ϵ > 0 , with probability at least 1 -ϵ , we can have the following bound:

Proof. For any constant M , we firstly notice that

For any θ and t , we have

where K min is the lower bound of the kernel function K ( x ) , i.e., K ( x ) ≥ K min , ∀ x .

According to Theorem 3 of [80], we know that for any ϵ > 0 , with probability at least 1 -ϵ , the following inequality holds:

where C is a constant. Let M = h ℓ C K min √ B∗ log B . Thus, the Eq. (35f) holds and we complete the proof of Theorem 4.4. ■

C.10 Additional Theorem

To gain more theoretical insights into a popular meta-learning method-MAML [25], we provide the following Theorem C.1. Before proceeding, we introduce some notations. During meta-training, a

finite number of task instances are observed by first sampling a task from the distribution p ( τ ) . Each task D τ i comprises a collection of m i data points { ( D S i,j , D Q i,j ) } m i j =1 , which are distributed over Z with each data point drawn from a distribution D i . For some risk ℓ , define the family of functions F Z := { ℓ ( θ -λ ∇ θ ℓ ( D S τ i ; θ ) , D Q τ i ) : θ ∈ Θ } . For each task D τ i , the Rademacher complexity of F on m i samples is

where the ϵ j 's are Rademacher random variables. Let F i ( θ ) = E D i ℓ ( θ -λ ∇ θ ℓ ( D S i,j ; θ ) , D Q i,j ) , ˆ F i ( θ ) = 1 m i ∑ m i j =1 ℓ ( θ -λ ∇ θ ℓ ( D S i,j ; θ ) , D Q i,j ) . Denote by θ ∗ the optimal model parameter under the two-stage algorithm. Theorem C.1 provides generalization of the algorithm to new tasks.

Theorem C.1 (Generalization Bound for MAML in the Tail Risk Cases) . For a new task τ B +1 with distribution D B +1 , if D B +1 = ∑ B i =1 a i D i , then with probability at least 1 -δ for any δ > 0 , we can have

Proof. The proof consists of two parts. We first explore the generalization to new instances of previously-seen tasks. Then we solve the generalization to new tasks.

Step 1. For any sample set A = { ( D S i,j , D Q i,j ) } m i j =1 , define Φ( A ) = sup ℓ ∈F Z F i ( θ ) -ˆ F i ( θ ) . Let A and A ′ := { (( D S i,j ) ′ , ( D Q i,j ) ′ ) } m i j =1 be two samples that differ by exactly one point. According to the fact sup x f ( x ) -sup x g ( x ) ≤ sup x ( f ( x ) -g ( x )) , we know that Φ( A ′ ) -Φ( A ) ≤ 1 m i due to the difference in exactly one point. Similarly, we can obtain Φ( A ) -Φ( A ′ ) ≤ 1 m , thus ∣ ∣ Φ( A ) -Φ( A ′ ) ∣ ∣ ≤ 1 m . Following McDiarmid's inequality, for any δ > 0 , with probability at least 1 -δ 2 , we have

We next bound the expectation of the right-hand side of inequality (39) as follows:

Eq. (40) uses the law of total expectation . Inequality (41) holds by Jensen's inequality and the convexity of the supremum function. In Eq. (42), ℓ ( D S i,j , D Q i,j ) := ℓ ( θ -λ ∇ θ ℓ ( D S i,j ; θ ) , D Q i,j ) ,

where ( D S i,j , D Q i,j ) ∈ A . Following inequality (39), we can know that

Step 2. Since the new distribution D B +1 is the convex combination of D i , ∀ i = 1 , · · · , B , we have F B +1 ( θ ) = ∑ B i =1 a i F i ( θ ) . Accordingly, with probability at least 1 -δ over the choice of samples used to compute ˆ F ( θ ) ,

which yields that

In summary, the two steps complete the proof of Theorem C.1.

D Implementation Details

D.1 Benchmark Details & Neural Architectures & Opensource Codes

Here, we illustrate all meta learning benchmark purposes in Fig. 10, which includes sinusoid regression, pendulum system identification, few-shot image classification, and meta reinforcement learning. We no longer run experiments on the Omniglot dataset, as most baselines can achieve SOTA performance and cannot tell the difference well from the openreview of [1].

（

a

）

Sinusoid Regression

（

d

）

1

2

support dataset

class:

class:

?

?

?

query dataset

（

c

）

Few-Shot Image Classification

（

b

）

System Identification

Sinusoid regression: In [1, 42], a lot of easy tasks and limited challenging tasks are sampled for meta-training, with the tasks from the whole space employed in evaluation. The default range of the phase parameter is B ∈ [0 , π ] , while those of the amplitude are A ∈ [0 . 1 , 1 . 05] for easy tasks and A ∈ [4 . 95 , 5 . 0] for challenging tasks. Generally, sinusoid functions with larger amplitudes are hard to adapt from a few support data points. The mean square error works as the risk function to measure the gap between the predicted value f ( x ) and the actual value. We set the task batch 50 for 5-shot and 25 for 10-shot , and the maximum iteration number is 70000. We refer the reader to TR-MAML and DR-MAML for all of the setups.

We retain the neural architectures [42, 1] in for all MAML like methods. In detail, all methods take a multilayer perceptron with two hidden layers and 40 ReLU activation units in each layer. The inner loop is achieved via one stochastic gradient descent step. As for CNP like methods, please refer to the vanilla set-up in [15] (The Github link is attached here: https://github.com/ google-deepmind/neural-processes ).

As the task space is hugh, there is no way to exactly estimate the risk quantile. Hence, the Oracle quantile in Fig. 5 is roughly computed from the sampled 100 tasks given the pretrained DR-MAML+. The rationale behind this operation is that increasing the population number in statistics reduces the quantile estimate bias.

System identification: The pendulum system is a classical environment in the OpenAI gym (environment details are: https://github.com/openai/gym/blob/master/gym/envs/classic_

■

control/pendulum.py ), and it is an actuated joint with one fixed end. The goal of system identification for the pendulum system is to predict the state transition given arbitrary actions with several randomly collected transitions as the support dataset. The observation is a tuple in the form (cos θ, sin θ, θ ′ ) , where θ ∈ [ π, π ] . The action is in the range a ∈ [ -2 . 0 , 2 . 0] and the torque is applied to the pendulum body. The mass m and the length l of the pendulum follows a uniform distribution ( m,l ) ∼ U ([0 . 4 , 1 . 6] × [0 . 4 , 1 . 6]) , sampled variables configure a Markov decision process as the task. In each batch, there are 16 tasks, and each task comprises 200 data points. Specifically, 10 few-shot data points are randomly sampled to enable system identification per task, denoted as 10-shot . For 20-shot cases, the number of data points in support dataset is 20. And the maximum iteration number is 5000.

For all MAML-like methods, the neural architecture used here is a multilayer perceptron with three hidden layers of 128 hidden units each and the activation function is ReLU. The learning rate for both the inner and outer loops is set at 1e-4.

Few-shot image classification: The few-shot image classification is mostly described as an N-way K-shot classification, where N classes with K -labeled instances for each are considered. The dataset is organized in the same manner as that in [81, 42, 1]: These include 64 classes for meta-training, with the rest 36 classes for meta-testing. We generate each task in the way: 8 meta-training tasks from the class { 6 , 7 , 7 , 8 , 8 , 9 , 9 , 10 } are randomly generated from 64 meta-training classes; the remaining classes are organized similarly. As a result, each task is constructed from sampling one image from five classes, corresponding to a 5-way 1-shot problem. The task batch is set 4 with a maximum number of iterations of 60000 in meta-training.

For all MAML-like methods, the neural architecture used here is a four-layer convolutional neural network for the mini -ImageNet datasets. The inner loop is achieved via one stochastic gradient descent step. We refer the reader to TR-MAML and DR-MAML for all of the setups (The Github link is attached here https://github.com/lgcollins/tr-maml ).

Meta reinforcement learning: 2D Navigation is a classical meta reinforcement learning benchmark where efficient explorations matter. The task in 2D Navigation is to guide the point robot to take move actions for a purpose of reaching a specific goal location from the step-wise reward. The reward the agent receives from the environment is based on the distance to the goal, and 20 episodes work as the support dataset for navigation fast adaptation. In terms of the task distribution, we sample tasks from a uniform distribution U ([ -0 . 5 , 0 . 5] × [ -0 . 5 , 0 . 5]) over goal locations.

As for the neural architecture for policy network set-ups, we refer the reader to vanilla MAML (Github link is attached here https://github.com/tristandeleu/pytorch-maml-rl ) and CAVIA (The Github link is attached here https://github.com/lmzintgraf/cavia/tree/master/rl ). And trust region policy optimization works for policy optimization.

Few-Shot Image Classification with MaPLe [69]: The stochastic gradient descent is the default optimizer with the learning rate 0.0035, and A6000 GPUs work for computations. We examine tail task risk minimization effectiveness on three large datasets. The class number split setup in datasets (class number to train/validate/test) is TieredImageNet (351/97/160) [82], ImagenetA (128/32/40) [83], and ImagenetSketch (640/160/200) [84]. Table 4 reports the overall training time and memory, where the vanilla MaPLe serves as the anchor point, and + means additional costs from the two-stage operation. For details of experimental implementations and setups, feel free to access our code at https://github.com/lvyiqin/DRMAML .

D.2 Modules in Python

This subsection includes the impelementation of KDE for the studied strategy. Here, the example of the hinge loss is illustrated as follows.

E Additional Experimental Results

Due to the page limit in the main paper, we include additional experiments and corresponding results in this section.

E.1 Evaluation with Other Robust Meta Learners

In addition to MAML, we apply a similar modification to CNP, which results in TR-CNP, DRO-CNP, DR-CNP, and DR-CNP+ (DR-CNP with KDE for VaR α estimates). We report the meta testing results on sinusoid regression and pendulum system identification benchmarks.

As illustrated in Table 5/6, all methods achieve comparable average performance in sinusoid and pendulum system identification. Regarding CVaR α , DR-CNP's improvement is relatively marginal over others except DR-CNP+. Compared to MAML, CNP seems more sensitive to quantile estimate accuracies when meeting with the studied strategies.

E.2 Numeric Results in Tables and Histograms

As the improving tricks in this work are regarding the quantile estimators, here we particularly include the quantitative results to show the difference between DR-MAML and DR-MAML+ in Table 7/8.

Note that the studied distributionally robust strategy is on the tail risk minimization, and CVaR α is the direct optimization indicator. As can be seen, DR-MAML+'s performance superiority over DR-MAML is significant w.r.t. CVaR α values in 5-shot sinusoid regression and four mini -ImageNet meta-testing tasks. These scenarios are more challenging than others as (i) the context information for adaptation is limited in 5-shot data points and (ii) the distributional shift is severe in mini -ImageNet meta-testing phase.

We can attribute the performance differences of the two methods to the cumulative quantile estimation errors using the crude MC. Even though the quantile estimation error in Fig. 5 difference is tiny in each step, the cumulative error indeed affects the converged equilibrium a lot. This reflects the advantage of the KDE's used in DR-MAML+ when the task batch size cannot be set larger in practice.

We also investigate the task risk value distributions in pendulum system identification. To this end, we visualize one run testing results for all methods in Fig. 11. It seems DR-MAML+'s result is more skewed to the left than others.

Fig. 12 displays all methods' performance w.r.t. the average and CVaR α returns along the metatraining process. We exclude TR-MAML in visualization due to its worse performance and unstable training properties. We can find that the DR-MAML exhibits a fast performance rise at the early stage but its capability to continuously improve diminishes over time. DR-MAML+ consistently outperforms other baselines in most cases. The above suggests that the KDE module achieves performance gains over the crude MC when implemented with the two-stage distributionally robust strategy for meta RL scenarios.

E.3 Sensitivity Analysis to Confidence Level

To reveal the impact of confidence levels on model performance, we perform a sensitivity analysis with respect to confidence levels. Since only DR-MAML and DR-MAML+ are influenced by the confidence levels during the distributionally robust optimization across all baselines, we only compare the performance of the two methods to highlight the differences between them. As shown in Fig.

0

.

5

1

.

0

1

.

5

2

.

0

Mean Squared Error (MSE)

0

20

40

60

80

100

Average=0.60

CVaR

α

=0.76

DR-MAML+(Ours)

0

.

5

1

.

0

1

.

5

2

.

0

Mean Squared Error (MSE)

0

20

40

60

80

100

Average=0.62

CVaR

α

=0.86

MAML

DR-MAML+(Ours)

0

.

5

1

.

0

1

.

5

2

.

0

Mean Squared Error (MSE)

0

20

40

60

80

100

120

Average=0.68

CVaR

α

=0.81

TR-MAML

DR-MAML+(Ours)

0

.

5

1

.

0

1

.

5

2

.

0

Mean Squared Error (MSE)

0

20

40

60

80

100

Average=0.61

CVaR

α

=0.82

DRO-MAML

DR-MAML+(Ours)

0

.

5

1

.

0

1

.

5

2

.

0

Mean Squared Error (MSE)

0

20

40

60

80

100

120

Average=0.61

CVaR

α

=0.78

DR-MAML

DR-MAML+(Ours)

0

100

200

300

400

500

Steps

-

30

-

25

-

20

-

15

-

10

Average Return

0

100

200

300

400

500

Steps

-

40

-

30

-

20

-

10

CVaR Return

MAML

DRO-MAML

DR-MAML

DR-MAML+(Ours)

13/14, we can observe that in both sinusoid 5-shot and 10-shot tasks, as the confidence level varies, DR-MAML+ exhibits more stable performance than DR-MAML, indicating that DR-MAML+ has a lower sensitivity to confidence levels. It can be illustrated that the crude Monte Carlo used in DR-MAML is more unstable in terms of quantile estimation than the kernel density estimator used in DR-MAML+. This can be due to the fact that the crude Monte Carlo method is more likely to get stuck in the local optimal solution. In addition, it can be seen from Fig. 13/14 that the performance of our developed DR-MAML+ is better than DR-MAML in most cases. DR-MAML+ exhibits lower mean squared errors than DR-MAML in the average, worst, and CVaR α indicators, demonstrating the advantages of more accurate quantile estimation in improving robustness.

0

.

1

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

1

.

1

1

.

2

Average

DR-MAML

DR-MAML+(Ours)

0

.

1

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

2

.

0

2

.

5

3

.

0

3

.

5

4

.

0

4

.

5

5

.

0

Worst

DR-MAML

DR-MAML+(Ours)

0

.

1

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

CVaR

DR-MAML

DR-MAML+(Ours)

Sinusoid 5-shot

0

.

1

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

0

.

4

0

.

6

0

.

8

1

.

0

1

.

2

Average

DR-MAML

DR-MAML+(Ours)

0

.

1

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

1

2

3

4

Worst

DR-MAML

DR-MAML+(Ours)

0

.

1

0

.

3

0

.

5

0

.

7

0

.

9

Confidence Level

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

3

.

5

CVaR

DR-MAML

DR-MAML+(Ours)

Sinusoid 10-shot

X

0

1

2

3

4

Y

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

2

-

1

0

1

2

3

MAML

X

0

1

2

3

4

Y

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

1

0

1

2

TR-MAML

X

0

1

2

3

4

Y

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

0

1

2

3

DRO-MAML

X

0

1

2

3

4

Y

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

1

0

1

2

3

4

DR-MAML

X

0

1

2

3

4

Y

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

Z

-

2

0

2

4

6

DR-MAML+

5

5

5

5

5

E.4 Further Exploration on Adaptation

We demonstrate the adaptation risk landscape of meta-trained MAML [25], TR-MAML [42], DROMAML[66], DR-MAML [1] and our DR-MAML+ in Fig. 15. The adaptation risk landscape shows the superiority of our method in optimizing within the expected tail risk minimization. Compared to other methods, DR-MAML+ exhibits smoother and smaller risk profiles, illustrating its robustness even in challenging tasks.

F Computational Platforms & Softwares

This work employs Pytorch [85] as the default deep learning toolkit when implementing the developed methods. As for baselines, TR-MAMAL follows the standard implementation as work [42] and runs with Tensorflow [86]. Others are implemented with Pytorch. All experimental results are computed by NVIDIA RTX6000 GPUs and A800 GPUs.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the claims.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of the work in Sec 5.7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Refer to Fig. 3/4/6/12/13/14 and Table 1/2/5/6/7/8.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We adhere to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Refer to Appendix A.5.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The question is not applicable to the paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The question is not applicable to the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The question is not applicable to the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The question is not applicable to the paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The question is not applicable to the paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work investigates the tail risk minimization in meta-learning from theoretical and practical perspectives. Overall, this work is well-written, novel and theoretically enriches TR-MAML[1]/DR-MAML[2].\n\n\nIn the realm of large models, meta-learning plays a crucial role due to the pressing concern of distributional robustness across various tasks, particularly in risk-sensitive applications. \nHere, I express my positive attitude based on this work's completeness, novelty, and workload.\n\n[1] Collins, L., Mokhtari, A., & Shakkottai, S. (2020). Task-robust model-agnostic meta-learning. Advances in Neural Information Processing Systems, 33, 18860-18871.\n\n[2] Wang, Q., Lv, Y., Xie, Z., & Huang, J. (2024). A simple yet effective strategy to robustify the meta learning paradigm. Advances in Neural Information Processing Systems, 36.\n\nSTRENGTHS:\nThe novelty of this work lies in two aspects.\n\n(1) This work reduces the distributionally robust meta-learning to a max-min optimization and performs analysis on tail risk generalization, convergence rate estimation, and the role of quantile estimation; \n\n(2) This work enhances DR-MAML through more accurate quantile estimates with theoretical support.\n\nOverall, this work clarifies its contribution in Table3 and completes all claims together with detailed proofs. Extensive experiments further verify theoretical insights.\n\nWEAKNESSES:\nNo concrete weakness.\n\nQUESTIONS:\nWith complete proofs and empirical evaluations, I do not have too many concrete questions except for some discussions. \n\n(1) Both group DRO and tail risk minimization handle the distribution shift in robust optimization; what are the advantages or disadvantages between them in meta-learning? \n\n(2) What difficulties will we encounter if we adopt the developed strategy in the optimization of large models?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an enhancement to the previous work termed DR-MAML by reformulating it as a Stackelberg game. Theoretical investigations regarding its solution concept, convergence rate, and generalization bound are provided. Numerical experiments demonstrate the improved robustness of the proposed method.\n\nSTRENGTHS:\n1. Both theoretical guarantees and numerical validations are provided to justify the sought robustness. \n2. Reformulating DR-MAML's objective as a Stackelberg game enables analyzing the problem from the viewpoint of game theory.\n\nWEAKNESSES:\n1. The contribution of this work is incremental and exclusive to DR-MAML, which limits its scope and broader applicability. \n2. The writing is hard to parse, and several notions are pretty vague. For instance, in lines 95-96, $F_{\\ell}^{\\alpha}$, $\\Omega_{\\alpha,\\tau}$, and $p_{\\alpha}$ are defined through illustration in a figure instead of mathematical expressions. \n3. Experiments are limited and lack comparison to SOTA methods. It is recommended comparing DR-MAML+ with popular meta-learning methods (such as MetaCurvature and MetaOptNet) on open-source benchmark datasets including tieredImageNet, CUB-200-2011, and MetaDatasets. In addition, the improvement in Table 1 and Table 2 are rather marginal (no more than 0.6%). \n4. There is no comparison of time and space complexity, and the scalability to large NNs like ResNet is unknown.\n5. The last statement of Theorem 4.1 is imprecise. According to Eq. (22), this convergence rate solely holds true when $t$ approaches infinity. \n6. The paper contains numerous typos. For instance, in line 4, the citation is not compiled. In line 95, \"the resulling\" should be corrected to \"resulting.\" Additionally, in lines 96, 130, 240, 241, 244, 246, 254, 265, 283, 295, 303, 319, 328, 332, and 335-337, the cross-references are in the equation style instead of the correct format.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nNo explicit discussion on limitations are provided.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

comments: SUMMARY:
The paper provides theoretical investigations for better understanding of an existing method in literature that focuses on minimizing expected tail risk. Equivalence of the algorithm is shown to a Stackerlberg game, which allows to study its convergence rate and asymptotic bounds on performance and generalization are provided. Finally, a practical heuristic is suggested to obtain performance improvements over the base method.

STRENGTHS:
- The presentation and structure of the paper is good.
- The theoretical results, while limited in mathematical novelty, seem comprehensive and are clear to follow.
- Sufficient experiments and ablations are provided.

WEAKNESSES:
Clarity:  
- It is not clear how the expression for the expected tail risk minimization is obtained in (3); the reviewer had to refer to the corresponding reference to understand it and thus, including it would help make the paper self-contained.
- Figure 1 is not clear; the font size needs to be increased.

QUESTIONS:
- What is the overhead induced by searching for the appropriate value of hyperparameter $\alpha$, compared to other baselines?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-23