SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Numerous easy tasks and a small proportion of difficult tasks are available in meta-training, while all tasks in the space are used in the evaluation. The range of the phase parameter is  $b \in [0, \pi]$ , and the amplitude range of the parameter is  $a \in [0.1, 1.05]$  for easy tasks and  $a \in [4.95, 5.0]$  for difficult tasks. It is noted that the sinusoid task is more challenging to fit with larger amplitudes as the resulting function is less smooth. The loss function corresponds to the mean-squared error between the predicted value  $f(x)$  and the ground truth value. The number of task batches is 50 for 5-shot and 25 for 10-shot. The optimal selection of the confidence level  $\alpha$  is difficult since we need to trade off the worst and average performance. Our setup is to minimize  $\mathrm{CVaR}_{\alpha}$ , which already considers the worst-case at some degree, so we watch the average performance in meta training results and set  $\alpha = 0.7$  for all few-shot regression tasks. There is no external configuration for this hyper-parameter. The maximum number of iterations in meta-training is 70000.
Few-shot Image Classification. MAML (Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered in this experiment. The N-way K-shot classification corresponds to an N-classification problem with K-labeled examples available to the meta learner.
The Omniglot dataset consists of 1623 handwritten characters from 50 alphabets, with each 20 examples. The task distribution is uniform for all task instances consisting of characters from one specific alphabet. The dataset split follows procedures in (Triantafillou et al., 2019). Finally, 25 alphabets are used for meta-training, with 20 alphabets for meta-testing. The number of task batches is 16. The confidence level  $\alpha = 0.5$  is selected with the same criteria as that in few-shot regression tasks. The maximum number of iterations is 60000 in meta-training. As the construction of the Omniglot meta dataset is related to specific alphabets and the scale of combination for tasks is huge, this indicates randomly sampled meta-training tasks in the evaluation of the main paper Tables may not be used in meta-training.
The mini-ImageNet dataset is pre-processed according to (Larochelle, 2017). In detail, 64 classes are used for meta-training, with the remaining 36 classes for meta-testing. Tasks are generated as follows: 64 meta-training classes are randomly grouped into 8 meta-train tasks with the class numbers  $\{6,7,7,8,8,9,9,10\}$ , and the 36 meta-testing classes are processed in the same way. Finally, each task is built by sampling 1 image from 5 different classes within one task, resulting in a 5-way 1-shot problem. The number of task batches is 4. The maximum number of iterations is 60000 in meta-training.
J.2 Neural Architectures & Optimization
Sinusoid Regression. MAML (Finn et al., 2017), TR-MAML (Collins et al., 2020), and DR-MAML are considered in this experiment. We retain the neural architecture for regression problems in (Finn et al., 2017; Collins et al., 2020). That is, we deploy a fully-connected neural network with two hidden layers of 40 ReLU nonlinear activation units. All methods use one stochastic gradient descent step as the inner loop.
Few-shot Image Classification. We retain the neural architecture for few-shot image classification problems in (Finn et al., 2017; Collins et al., 2020). In detail, a four-layer convolutional neural network is used for both Omniglot and mini-ImageNet datasets. All methods use one stochastic gradient descent step as the inner loop.
K Additional Experimental Results
More Quantitative Analysis. Due to the page limit in the main paper, we include the  $\alpha$ 's sensitivity experimental result in sinusoid 10-shot regression. As illustrated in Fig. (7), the trend is similar to that in sinusoid 5-shot regression. Worst-case optimization degrades the average performance of TR-MAML. DR-MAML is entangled with MAML in the average performance, while the performance gap between them is significant in the worst-case.
Figure 7: Meta Testing Performance of Meta-Trained DR-MAML with Various Confidence Levels  $\alpha$ . MAML and TR-MAML are irrelevant with the variation of  $\alpha$  in meta-training. The plots report testing MSEs with standard error bars in shadow regions.
Regarding few-shot image classification in the mini-ImageNet dataset, we can observe that in Fig. (8), the standard error is relatively smaller than in previous regression cases. When the confidence level is over a particular value, e.g.  $\alpha > 0.5$ , there occurs a significant decline of performance in all metrics. Note that when  $\alpha \rightarrow 1.0$ , the optimization objective approaches the worst-case optimization objective. Here we attach two possible reasons for the performance degradation phenomenon: (i) The adopted base optimization technique matters in nearly worst-case optimization. The stochastic mirror descent-ascent (Juditsky et al., 2011) is utilized in TR-MAML, which is more stable in deriving the optimal solution. In comparison, the stochastic gradient descent with sub-gradient operations works as the optimization method, and this method can be unstable when the scale of worst-case examples is small in the update. (ii) For few-shot image classification, estimates of  $\mathrm{VaR}_{\alpha}$  can be less precise with limited batch sizes and higher  $\alpha$  values since the meta risk function value is discontinuous. Consequently, we can also attribute the severe degradation of fast adaptation performance in higher  $\alpha$  value cases to the approximation errors of quantile estimates.
More Visualization Results. Further, we explore the influence of the expected tail risk minimization principle in meta learning. Here the landscape of meta risk values, namely fast adaptation losses, is presented in the sinusoid regression problem.
As exhibited in Fig. (9), the evaluated meta risk values from one random trial are associated with hyper-parameters of tasks. The final optimized results can discover some tasks difficult in fast adaptations. Meta learning methods are difficult to adapt in task regions with higher amplitudes. MAML exhibits higher MSEs in regions with the amplitude  $a > 2$ , while DR-MAML minimizes a proportion of risks in these regions. In contrast, TR-MAML reduces the risk around task regions with  $a > 2$  to a certain extent; however, it shows relatively higher risk values in easy regions. Such
Figure 8: Meta Testing Classification Accuracies of Meta-Trained DR-MAML with Various Confidence Levels  $\alpha$ . MAML and TR-MAML are irrelevant with the variation of  $\alpha$  in meta-training. The plots report testing accuracies with standard error bars in shadow regions.
Figure 9: The Fast Adaptation Risk Landscape of Meta-Trained DR-MAML, TR-MAML and MAML. Shown is an example of sinusoid 5-shot regression, which corresponds to the function space  $f(x) = a\sin (x - b)$ . The  $X$ -axis denotes the amplitude parameter  $a$ , and the  $Y$ -axis is the phase parameter  $b$ . The confidence level is  $\alpha = 0.7$  in meta-training. The plots report testing MSEs in the  $Z$ -axis with a random trial of generating tasks.
evidence reflects the interpretability in optimization within the expected tail risk minimization, and the landscape of meta risk values is relatively flat and smooth than others.
Experimental Results with DR-CNPs. The implementation is the same as CNP (Garnelo et al., 2018a) and official Github files (https://github.com/deepmind/neural-processes/blob/master/conditional_neural_process.ipynb); the Gaussian Process curve generator works as the benchmark, and we retain the neural architecture and optimization pipelines in Github files. Please check implementation details, e.g., neural architectures, optimizers, epochs, batch sizes, etc., from the mentioned GitHub repository.
Also, note that this is more challenging than sinusoid as there is more randomness in generating curves. Similar to few shot image classifications, we set  $\alpha = 0.5$  in meta training DR-CNPs. In meta-testing processes, we randomly sample 64 curves per run with random context points in GitHub files and examine the performance of CNP, TR-CNP (worst-case), and DR-CNP (distributionally robust). The log-likelihoods in 5 runs are reported as follows (the higher, the better).
From Table (6), we observe that the DR-CNP retains the average performance the same as the CNP and simultaneously achieves the highest worst and  $\mathrm{CVaR}_{\alpha}$  log-likelihoods. In contrast, the TR-CNP sacrifices the average performance a lot and obtains the mediate worst result. Here we attribute
Table 6: Meta-testing point-wise log-likelihood results of Gaussian processes in 5 runs.
Method	Average	Worst	CVaRα
CNP (Garnelo et al., 2018a)	-0.22±0.04	-1.35±0.38	-0.50±0.11
TR-CNP	-0.80±0.01	-1.08±0.02	-0.87±0.01
DR-MAML	-0.21±0.02	-0.74±0.08	-0.37±0.03
Figure 10: Curve Fitting Result Visualization in 5 Random Trials. From Up to Down are respectively DR-CNPs (Ours), CNPs, and TR-CNPs. Shadow regions are three standard deviations.
the failure of achieving the best worst performance using TR-CNPs to the sensitivity of min-max optimizers as mentioned in TR-MAML.
Particularly, some randomly sampled fitted curves in difficult scenarios, e.g., setting the number of context points as 5, are visualized in Fig. (10). Observations are consistent with Table (6), where TR-CNPs exhibit over estimated uncertainty and cannot well reveal the trend of curves. In comparison, DR-CNPs can well handle challenging cases, capturing more convincing uncertainty.
Experimental Results in Meta Reinforcement Learning. We also performed additional experiments in 2-D point robot navigation tasks. This is a meta reinforcement learning benchmark. All setups of the point robot are the same as MAML, and please refer to pytorch-mamlrl and cavia Github for details of the navigation environments. The neural network of policies is a two-hidden layer MLP with 64 hidden units and tanh activations. The fast learning rate is 0.1, and the number of batches is 500. Other details, e.g., optimizers, step-wise rewards, horizons and etc., can be found in MAML. The one-step gradient is performed for fast adaptation, and the trust region policy optimization works as the policy gradient algorithm. Similar to few shot image classifications, we set  $\alpha = 0.5$  in meta training point robots. In meta-testing processes, we randomly sample 100 navigation goals as tasks and examine the performance of MAML, TR-MAML, and DR-MAML. The histogram comparison is visualized in Fig. (11).
(a)
Figure 11: Histograms of Meta-Testing Performance in Point Robot Navigation Problems. We report the negative average step-wise rewards in  $x$ -axis.
Unlike few-shot supervised learning, worst-case optimization is pretty challenging in RL, and the result of TR-MAML is unstable and cannot achieve the goal of worst-case optimality in Table (7).
Table 7: Meta-testing step-wise rewards in 4 runs in point robot navigation tasks.
Method	Average	Worst	CVaRα
CNP (Garnelo et al., 2018a)	-16.6±0.19	-46.9±4.22	-24.2±0.49
TR-CNP	-36.8±0.12	-81.3±4.35	-52.1±0.89
DR-MAML	-15.3±0.49	-46.7±3.24	-21.6±0.79
This is due to the nature of large performance deviations when deploying policies in various tasks. Seeking optimizers for stable worst-case optimization is non-trivial. Also, stably generalization across skills in reinforcement learning is indeed difficult with a few interactions. Overall, DR-MAML can well control the proportional worst case performance.
L Platforms & Computational Tools
In this research project, we use NVIDIA 1080-Ti GPUs in computation. Pytorch (Paszke et al., 2019) works as the deep learning toolkit in implementing few-shot image classification experiments. Meanwhile, Tensorflow is the deep learning toolkit for implementing sinusoid few-shot regression experiments.
M Declaration of Author Contribution
The authors confirm their contribution to the paper as follows: Q.W. (Qi Wang) and Y.Q.L. (Yiqin Lv) conceptualized the idea of tail risk minimization in meta learning and developed simple yet effective strategies with mathematical demonstrations. Y.Q.L. performed most of the experiments and Q.W. examined the performance in meta reinforcement learning benchmark. Q.W. and Y.Q.L. wrote and prepared the manuscript, while Y.H.F. (Yanghe Feng) contributed crucial insights into the organization and checked the mathematical part of the draft in submission. J.C.H. (Jincai Huang) and Z.X. (Zheng Xie) contributed to the supervision and helped proofread the manuscript. All authors reviewed the results and approved the final version of the manuscript. Meanwhile, we also thank dr. Zhengge Zhou for helpful discussion.