SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 1 (Improvement Guarantee) Under assumptions (1)/(2)/(3), suppose that the estimate error with the crude Monte Carlo holds:  $|\hat{\xi}_{\alpha_t} - \xi_{\alpha_t}| \leq \frac{\lambda}{\beta_\ell(1 - \alpha)^2}, \forall t \in \mathbb{N}^+$ , with the subscript  $t$  the iteration number,  $\lambda$  the learning rate in stochastic gradient descent,  $\beta_\ell$  the Lipschitz constant of the risk cumulative distribution function, and  $\alpha$  the confidence level. Then the proposed heuristic algorithm with the crude Monte Carlo can produce at least a local optimum for distributionally robust fast adaptation.
Note that Theorem (1) indicates that under certain conditions, using the above heuristic algorithm has the performance improvement guarantee, which corresponds to Fig. (2). The error resulting from the approximate algorithm is further estimated in Appendix Theorem (2).
4.4 Instantiations & Implementations
Our proposed optimization strategy applies to all meta learning methods and has the improvement guarantee in Theorem (1). Here we take two representative meta learning methods, MAML (Finn et al., 2017) and CNP (Garnelo et al., 2018a), as examples and show how to robustify them through the lens of risk distributions. Note that the forms of  $\ell(\mathfrak{D}_{\tau}^{T}, \mathfrak{D}_{\tau}^{C}; \vartheta)$  sometimes differ in these methods. Also, the detailed implementation of the strategy relies on specific meta learning algorithms and models.
Example 1 (DR-MAML) With the task distribution  $p(\tau)$  and model agnostic meta learning (Finn et al., 2017), the distributionally robust MAML treats the meta learning problem as a bi-level
optimization with a  $\mathrm{VaR}_{\alpha}$  relevant constraint.
$$
\min  _ {\substack {\vartheta \in \Theta \\ \xi \in \mathbb {R}}} \xi + \frac {1}{1 - \alpha} \mathbb {E} _ {p (\tau)} \left[ \left[ \ell \left(\mathfrak {D} _ {\tau} ^ {T}; \vartheta - \lambda \nabla_ {\vartheta} \ell \left(\mathfrak {D} _ {\tau} ^ {C}; \vartheta\right)\right) - \xi \right] ^ {+} \right] \tag{7}
$$
The gradient operation  $\nabla_{\vartheta}\ell (\mathfrak{D}_{\tau}^{C};\vartheta)$  corresponds to the inner loop with the learning rate  $\lambda$
The resulting distributionally robust MAML (DR-MAML) is still a optimization-based method, where a fixed percentage of tasks are screened for the outer loop. As shown in Fig. (3) and Eq. (7), the objective is to obtain a robust meta initialization of the model parameter  $\vartheta$ .
Example 2 (DR-CNP) With the task distribution  $p(\tau)$  and the conditional neural process (Garnelo et al., 2018a), the distributionally robust conditional neural process learns the functional representations with a  $CVaR_{\alpha}$  constraint.
$$
\min  _ {\substack {\vartheta \in \Theta \\ \xi \in \mathbb {R}}} \xi + \frac {1}{1 - \alpha} \mathbb {E} _ {p (\tau)} \left[ \left[ \ell \left(\mathfrak {D} _ {\tau} ^ {T}; z, \theta_ {2}\right)\right) - \xi \right] ^ {+} \Bigg ] \tag{8}
$$
$$
s. t. z = h _ {\theta_ {1}} \left(\mathfrak {D} _ {\tau} ^ {C}\right) w i t h \vartheta = \left\{\theta_ {1}, \theta_ {2} \right\}
$$
Here  $h_{\theta_1}$  is a set encoder network with  $\theta_2$  the parameter of the decoder network.
Figure 3: Diagram of Distributionally Robust Fast Adaptation for Model Agnostic Meta Learning (Finn et al., 2017). For example, with the size of the meta batch 5 and  $\alpha = 40\%$ ,  $5*(1 - \alpha)$  tasks in gray with the worst fast adaptation performance are screened for updating meta initialization.
The resulting distributionally robust CNP (DR-CNP) is to find a robust functional embedding to induce underlying stochastic processes. Still, in Eq. (8), a proportion of tasks with the worst functional representation performance are used in meta training.
Moreover, we convey the pipelines of optimizing these developed distributionally robust models in Appendix Algorithms (1)/(2).
5 Experimental Results and Analysis
This section presents experimental results and examines fast adaptation performance in a distributional sense. Without loss of generality, we take DR-MAML in Example (1) to run experiments.
Benchmarks. The same as work in (Collins et al., 2020), we use two commonly-seen downstream tasks for meta learning experiments: few-shot regression and image classification. Besides, ablation studies are included to assess other factors' influence or the proposed strategy's scalability.
Baselines & Evaluations. Since the primary investigation is regarding risk minimization principles, we consider the previously mentioned expected risk minimization, worst-case minimization, and expected tail risk minimization for meta learning. Hence, MAML (empirical risk), TR-MAML (worst-case risk), and DR-MAML (expected tail risk) serve as examined methods. We evaluate these methods' performance based on the Average, Worst-case, and  $CVaR_{\alpha}$  metrics. For the confidence level to meta train DR-MAML, we empirically set  $\alpha = 0.7$  for few-shot regression tasks and  $\alpha = 0.5$  image classification tasks without external configurations.
5.1 Sinusoid Regression
Following (Finn et al., 2017; Collins et al., 2020), we conduct experiments in sinusoid regression tasks. The mission is to approximate the function  $f(x) = a\sin (x - b)$  with K-shot randomly sampled function points, where the task is de
Table 1: Test average mean square errors (MSEs) with reported standard deviations for sinusoid regression (5 runs). We respectively consider 5-shot and 10-shot cases with  $\alpha = 0.7$ . The results are evaluated across the 490 meta-test tasks, as in (Collins et al., 2020). The best results are in bold.
Method	5-shot	10-shot
Average	Worst	\( CVaR_{\alpha} \)	Average	Worst	\( CVaR_{\alpha} \)
MAML (Finn et al., 2017)	1.02±0.10	3.89±0.83	2.25±0.15	0.66±0.16	2.57±0.70	1.15±0.19
TR-MAML (Collins et al., 2020)	1.09±0.08	2.28±0.35	1.79±0.06	0.77±0.11	1.68±0.43	1.27±0.28
DR-MAML (Ours)	0.89±0.04	2.91±0.46	1.76±0.02	0.54±0.01	1.70±0.17	0.96±0.01
fined by  $a$  and  $b$ . In the sine function family, the target range, amplitude range, and phase range are
respectively  $[-5.0, 5.0] \subset \mathbb{R}$ ,  $a \in [0.1, 5.0]$  and  $b \in [0, 2\pi]$ . In the setup of meta training and testing datasets, a distributional shift exists: numerous easy tasks and several difficult tasks are generated to formulate the training dataset with all tasks in the space as the testing one. Please refer to Appendix (J) for a detailed partition of meta-training, testing tasks, and neural architectures.
Result Analysis. We list meta-testing MSEs in sinusoid regression in Table (1). As expected, the tail risk minimization principle in DR-MAML can lead to an intermediate performance in the worst-case. In both cases, the comparison between MAML and DR-MAML in MSEs indicates that such probabilistic-constrained optimization in the task space even has the potential to advance average fast adaptation performance. In contrast, TR-MAML has to sacrifice more average performance for the worst-case improvement.
Figure 4: Histograms of Meta-Testing Performance in Sinusoid Regression Problems. With  $\alpha = 0.7$ , we respectively visualize the comparison results, DR-MAML-vs-MAML and TR-MAML-vs-MAML in 5-shot (Two Sub-figures Left Side) and 10-shot (Two Sub-figures Right Side) cases, for a sample trial.
More intuitively, Fig. (4) illustrates MSE statistics on the testing task distribution and further verifies the effect of the  $\mathrm{CVaR}_{\alpha}$  principle in decreasing the proportional worst-case errors. In 5-shot cases, the difference in MSE statistics is more significant: the worst-case method tends to increase the skewness in risk distributions with many task risk values gathered in regions of intermediate performance, which is unfavored in general cases. As for why DR-MAML surpasses MAML in terms of average performance, we attribute it to the benefits of external robustness in several scenarios, e.g., the drift of training/testing task distributions.
Table 2: Average N-way K-shot classification accuracies in Omniglot with reported standard deviations (3 runs). With  $\alpha = 0.5$ , the best results are in bold.
Method	Meta-Training Alphabets	Meta-Testing Alphabets
5-way 1-shot	20-way 1-shot	5-way 1-shot	20-way 1-shot
Average	Worst	\( CVaR_{\alpha} \)	Average	Worst	\( CVaR_{\alpha} \)	Average	Worst	\( CVaR_{\alpha} \)	Average	Worst	\( CVaR_{\alpha} \)
MAML (Finn et al., 2017)	98.4±0.2	82.4±1.1	96.9±0.5	99.2±0.1	33.9±3.0	80.9±0.7	93.5±0.2	82.5±0.2	91.6±0.6	67.6±2.0	49.7±3.5	60.4±1.7
TR-MAML (Collins et al., 2020)	97.4±0.6	95.0±0.3	96.5±0.4	92.2±0.8	82.4±2.1	87.2±0.9	93.1±1.1	85.3±1.9	91.3±0.9	74.3±1.4	58.4±1.8	68.5±1.2
DR-MAML (Ours)	97.1±0.3	84.0±0.4	95.1±0.3	99.6±0.6	57.9±2.4	84.8±0.7	93.7±0.4	84.1±0.8	92.1±0.5	74.6±1.2	51.0±2.3	66.4±1.4
5.2 Few-Shot Image Classification
Here we do investigations in few-shot image classification. Each task is an N-way K-shot classification with N the number of classes and K the number of labeled examples in one class. The Omniglot (Lake et al., 2015) and mini-ImageNet (Vinyals et al.,
2016) datasets work as benchmarks for examination. We retain the setup of datasets in work (Collins et al., 2020).
Result Analysis. The classification accuracies in Omniglot are illustrated in Table (2): In 5-way 1-shot cases, DR-MAML obtains the best average and  $\mathrm{CVaR}_{\alpha}$  in meta-testing datasets, while TR-MAML achieves the best worst-case performance with a slight degradation of average performance compared to MAML in both training/testing datasets. In 20-way 1-shot cases, for training/testing datasets, we surprisingly notice that the expected tail risk is not well optimized with DR-MAML, but there is an average performance gain; while TR-MAML works best in worst-case  $\mathrm{CVaR}_{\alpha}$  metrics.
When it comes to mini-ImageNet, findings are distinguished a lot from the above one: in Table (3), DR-MAML yields the best result in all evaluation metrics and cases, even regarding the worst-case. TR-MAML can also improve all metrics in meta-testing cases. Overall, challenging meta-learning tasks can reveal more advantages of DR-MAML over others.
This part mainly checks the influence of the confidence level  $\alpha$ , meta batch size, and other optimization strategies towards the distribution of fast adapted risk values. Apart from examining these factors of interest, additional experiments are also conducted in this paper; please refer to Appendix (K).
Figure 5: Meta Testing MSEs of Meta-Trained DR-MAML with Various Confidence Levels  $\alpha$ . MAML and TR-MAML are irrelevant with the variation of  $\alpha$  in meta-training. The plots report testing MSEs with standard error bars in shadow regions.
Sensitivity to Confidence Levels  $\alpha$ . To deepen understanding of the confidence level  $\alpha$ 's effect in fast adaptation performance, we vary  $\alpha$  to train DR-MAML and evaluate models under previously mentioned metrics. Taking the sinusoid 5-shot regression as an example, we calculate MSEs and visualize the fluctuations with additional  $\alpha$ -values in Fig. (5). The results in the worst-case exhibit higher deviations. The trend in Average/CVaR $_{\alpha}$  metrics shows that with increasing  $\alpha$ , DR-MAML gradually approaches TR-MAML in average and CVaR $_{\alpha}$  MSEs. With  $\alpha \leq 0.8$ , ours is less sensitive to the confidence level and mostly beats MAML/TR-MAML in average performance. Though ours aims at optimizing CVaR $_{\alpha}$ , it cannot always ensure such a metric to surpass TR-MAML in all confidence levels due to rigorous assumptions in Theorem (1).
Figure 6: Meta Testing Average Performance of Meta-Trained DR-MAML with Various Sizes of the Meta Batch. The plots report average results with standard error bars in shadow regions.
Influence of the Task Batch Size. Note that our optimization strategy relies on the estimate of  $\mathrm{VaR}_{\alpha}$ , and the improvement guarantee relates to the estimation error. Theoretically, the meta batch in training can directly influence the optimization result. Here we vary the meta batch size in training, and evaluated results are visualized in Fig. (6). In regression scenarios, the average MSEs can decrease to a certain level with an increase of meta batch, and then performance degrades. We attribute the performance gain with a batch increase to more accurate  $\mathrm{VaR}_{\alpha}$  estimates; however, a meta batch larger than some threshold can worsen the first-order meta learning algorithms' efficiency, similarly observed in (Nichol et al., 2018). As for classification scenarios, there appears no clear trend since the meta batch is smaller enough.
Comparison with Other Optimization Strategies. Note that instantiations of distributionally robust meta learning methods, such as DR-MAML and DR-CNPs in Examples (1)/(2) are regardless of optimization strategies and can be optimized via any heuristic algorithms for  $\mathrm{CVaR}_{\alpha}$  objectives.
Additionally, we use DR-MAML as the example and perform the comparison between our two-stage algorithm and the risk reweighted algorithm (Sagawa et al., 2020). The intuition of the risk reweighted algorithm is to relax the weights of tasks and assign more weights to the gradient of worst cases. The normalization of risk weights is achieved via the softmax operator. Though there is an improvement guarantee w.r.t. the probabilistic worst group of tasks, the algorithm is not specially designed for meta learning or  $\mathrm{CVaR}_{\alpha}$  objective.
$$
\min  _ {\vartheta \in \Theta} \mathcal {E} _ {\alpha} (\vartheta) := \mathbb {E} _ {p (\tau ; \vartheta)} \left[ \frac {p _ {\alpha} (\tau ; \vartheta)}{p (\tau ; \vartheta)} \ell \left(\mathfrak {D} _ {\tau} ^ {T}, \mathfrak {D} _ {\tau} ^ {C}; \vartheta\right) \right] \approx \frac {1}{\mathcal {B}} \sum_ {b = 1} ^ {\mathcal {B}} \omega_ {b} (\tau_ {b}; \vartheta) \ell \left(\mathfrak {D} _ {\tau_ {b}} ^ {T}, \mathfrak {D} _ {\tau_ {b}} ^ {C}; \vartheta\right) \tag {9}
$$
Table 4: Test average mean square errors (MSEs) with reported standard deviations for sinusoid regression (5 runs). We mainly compare DR-MAML with different optimization algorithms. 5-shot and 10-shot cases are respectively considered here. The results are evaluated across the 490 meta-test tasks, which is the same as in (Collins et al., 2020). With  $\alpha = 0.7$  for meta training, the best testing results are in bold.
Method	5-shot	10-shot
Average	Worst	\( CVaR_{\alpha} \)	Average	Worst	\( CVaR_{\alpha} \)
DR-MAML (Group DRO, (Sagawa et al., 2020))	0.91±0.06	3.57±0.56	1.83±0.03	0.61±0.02	1.90±0.11	1.13±0.02
DR-MAML (Two-Stage, Ours)	0.89±0.04	2.91±0.46	1.76±0.02	0.54±0.01	1.70±0.17	0.96±0.01
Table 5: Average 5-way 1-shot classification accuracies in mini-ImageNet with reported standard deviations (3 runs). We mainly compare DR-MAML with different optimization algorithms. With  $\alpha = 0.5$  for meta training, the best testing results are in bold.
Method	Eight Meta-Training Tasks	Four Meta-Testing Tasks
Average	Worst	\( CVaR_{\alpha} \)	Average	Worst	\( CVaR_{\alpha} \)
DR-MAML (Group DRO, (Sagawa et al., 2020))	67.0±0.2	56.6±0.4	61.6±0.2	49.1±0.2	46.6±0.1	47.2±0.2
DR-MAML (Two-Stage, Ours)	70.2±0.2	63.4±0.2	67.2±0.1	49.4±0.1	47.1±0.1	47.5±0.1
Meanwhile, the weight of task gradients after normalization is a biased estimator w.r.t. the constrained probability  $p_{\alpha}(\tau; \vartheta)$  in the task space. In other words, the risk reweighted method can be viewed as approximation w.r.t. the importance weighted method in Eq. (9). In the importance weighted method, for tasks out of the region of  $(1 - \alpha)$ -proportional worst, the probability of sampling such tasks  $\tau_b$  is zero, indicating  $\omega_b(\tau_b; \vartheta) = 0$ . While in risk reweighted methods, the approximate weight is assumed to satisfy  $\omega_b(\tau_b; \vartheta) \propto \exp \left(\frac{\ell(\mathfrak{D}_{\tau_b}^T, \mathfrak{D}_{\tau_b}^C; \hat{\vartheta})}{\tau}\right)$ , where  $\hat{\vartheta}$  means last time updated meta model parameters and the risk function value is evaluated after fast adaptation.
In implementations, we keep the setup the same as Group DRO methods in (Sagawa et al., 2020) for meta-training. As illustrated in Table (4)/(5), DR-MAML with the two-stage optimization strategies consistently outperform that with the Group DRO ones in both 5-shot and 10-shot sinusoid cases regarding all metrics. The performance advantage of using the two-stage ones is not significant in miniImageNet scenarios. We can hypothesize that the estimate of  $\mathrm{VaR}_{\alpha}$  in continuous task domains, e.g., sinusoid regression, is more accurate, and this probabilistically ensures the improvement guarantee with two-stage strategies. Both the  $\mathrm{VaR}_{\alpha}$  estimate in two-stage strategies and the importance weight estimate in the Group DRO ones may have a lot of biases in few-shot image classification, which lead to comparable performance.
Technical Discussions. This work contributes more insights into robustifying fast adaptation in meta learning. Our utilized expected tail risk trades off the expected risk minimization and worst-case risk minimization, and the two-stage strategy works as the heuristic to approximately solve the problem with an improvement guarantee. Our strategy can empirically alleviate the worst-case fast adaptation and sometimes even improve average performance.
Existing Limitations. Though our robustification strategy is simple yet effective in implementations, empirical selection of the optimal meta batch size is challenging, especially for first-order optimization methods. Meanwhile, the theoretical analysis only applies to a fraction of meta learning tasks when risk function values are in a compact continuous domain.
Future Extensions. Designing a heuristic algorithm with an improvement guarantee is non-trivial and relies on the properties of risk functions. This research direction has practical meaning in the era of large models and deserves more investigations in terms of optimization methods. Also, establishing connections between the optimal meta batch size and specific stochastic optimization algorithms can be a promising theoretical research issue in this domain.
This work is funded by National Natural Science Foundation of China (NSFC) with the Number # 62306326.
Zhang, J., Cheung, B., Finn, C., Levine, S., and Jayaraman, D. (2020). Cautious adaptation for reinforcement learning in safety-critical settings. In International Conference on Machine Learning, pages 11055-11065. PMLR.
Zhao, D., von Oswald, J., Kobayashi, S., Sacramento, J., and Grewe, B. F. (2020). Meta-learning via hypernetworks.
1 Introduction 1
2 Literature Review 2
3 Preliminaries 3
4 Distributionally Robust Fast Adaptation 4
4.1 Meta Risk Functions as Random Variables 4
4.2 Meta Learning via Controlling the Expected Tail Risk 4
4.3 Heuristic Algorithms for Optimization 6
4.4 Instantiations & Implementations 6
5 Experimental Results and Analysis 7
5.1 Sinusoid Regression 7
5.2 Few-Shot Image Classification 8
5.3 Ablation Studies 9
6 Conclusion and Limitations 10
A Frequently Asked Question 18
B Pseudo Algorithms of DR-MAML & DR-CNPs 19
C Properties of Risk Minimization Principles 20
C.1 Stochastic Optimization in the Constrained Form 20
C.2 SGD Intractability of Meta Learning  $\mathrm{CVaR}_{\alpha}$  Optimization Objective 20
C.3 Risk-Sensitive Applications & Optimization Strategies 20
D Computational Complexity 21
E Proof of Remark (2) 21
F Proof of Proposition (1) 22
G Proof of Proposition (2) 23
H Proof of Improvement Guarantee in Theorem (1) 24
Proof of Approximation Error in Theorem (2) 27
J Experimental Set-up & Implementation Details 27
J.1 Meta Learning Datasets & Tasks 28
J.2 Neural Architectures & Optimization 29
K Additional Experimental Results 29
L Platforms & Computational Tools 32
M Declaration of Author Contribution 32
A Frequently Asked Question
Here we collected technical questions and suggestions from researchers who helped check out the manuscript. We thank these researchers for precious questions and provide more details.
Novelty & primary findings of this work. Here we mainly summarize two points of novelty in this work:
- Meta Learning Robustification Framework. Though the concept of the expected tail risk has emerged for several decades and has been widely employed in financial domains, the application to fast adaptation or robustification of meta learning remains limited in literature as far as we know.
- Optimization Strategy & Theoretical Analysis. We theoretically analyze two-stage optimization strategies as the heuristic algorithm in optimizing the distributionally robust meta learning models and demonstrate the improvement guarantee under certain conditions.
As verified in experimental results in the main paper, placing a probabilistic constraint in the task space is meaningful. It circumvents the effect of over-pessimistic consideration (worst-case optimization), increases robustness in proportional cases, and mostly retains or even improves average performance.
Apart from the novelty in the framework and algorithm parts, we have several findings which bring crucial insights into meta learning: (i) Not all tasks are necessary to perform fast adaptation. (ii) Additional focus on the tail risk has the potential to enhance models' generalization capability. (iii) The tail risk instead of extreme worst-case risk can better advance robustness in challenging datasets.
Meta risk function values as random variables. In some few-shot learning related work, the context and the target dataset are equivalently called the support and query datasets. The definition of a task in meta learning is up to application scenarios and specific meta learning algorithms or models. The commonly-used sinusoid regression using model-agnostic meta learning (Finn et al., 2017) considers the fixed number of context points to induce tasks. In contrast, conditional neural processes (Garnelo et al., 2018a) for few-shot regression vary the number of context points to induce tasks. Once the context and the target are partitioned and the model parameter is specified, we can obtain a meta risk function value. However, the meta risk function values are not in a compact Euclidean subspace in several cases.
The continuity of the meta risk probability density function. This relates to the task distribution and meta learning problems. Throughout the few-shot regression task, the meta risk function value can be approximately viewed as a continuous random variable. However, when it comes to the few-shot image classification mission, the meta risk function value is impossible to cover the entire continuous interval. In these scenarios, the probable values of accuracies are finite. This makes the theoretical analysis, e.g. Theorem (1), no longer holds. For example, Assumption (2) will be unrealistic when there exists a constant gap between two accuracy values. Hence, we leave this part a future research direction in theoretical analysis. Regarding the meta risk function, ours shares the same setup as that in (Fallah et al., 2021). Admittedly, precisely modeling risk distributions can be crucial, and more techniques, e.g., deep kernel estimators (Puchert et al., 2021) can be adopted to improve this point.
The selection of baselines in robust meta learning. A large body of prior work considers the robustness of meta learning in scenarios when the input of a data point, the model parameter, and the number of context points are trembled or modified. Robustness in presence of tasks distributions is seldom investigated except for the worst-case optimization in (Collins et al., 2020). Hence, we retain most of the setups in (Collins et al., 2020) for a fair comparison.