SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Huafeng Liu, Liping Jing, Jian Yu
Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University
The School of Computer and Information Technology, Beijing Jiaotong University
{hfliu1, lpjing, jianyu}@bjtu.edu.cn
Unlike traditional statistical models depending on hand-specified priors, neural processes (NPs) have recently emerged as a class of powerful neural statistical models that combine the strengths of neural networks and stochastic processes. NPs can define a flexible class of stochastic processes well suited for highly non-trivial functions by encoding contextual knowledge into the function space. However, noisy context points introduce challenges to the algorithmic stability that small changes in training data may significantly change the models and yield lower generalization performance. In this paper, we provide theoretical guidelines for deriving stable solutions with high generalization by introducing the notion of algorithmic stability into NPs, which can be flexible to work with various NPs and achieves less biased approximation with theoretical guarantees. To illustrate the superiority of the proposed model, we perform experiments on both synthetic and real-world data, and the results demonstrate that our approach not only helps to achieve more accurate performance but also improves model robustness.
Neural processes (NPs) [9, 10] constitute a family of variational approximation models for stochastic processes with promising properties in computational efficiency and uncertainty quantification. Different from traditional statistical modeling for which a user typically hand-specified a prior (e.g., smoothness of functions quantified by a Gaussian distribution in Gaussian process [25]), NPs implicitly define a broad class of stochastic processes with neural networks in a data-driven manner. When appropriately trained, NPs can define a flexible class of stochastic processes well suited for highly non-trivial functions that are not easily represented by existing stochastic processes.
NPs meta-learn a distribution over predictors and provide a way to select an inductive bias from data to adapt quickly to a new task. Incorporating the data prior into the model as an inductive bias, NPs can reduce the model complexity and improve model generalization. Usually, an NP predictor is described as predicting a set of data (target set) given a set of labeled data (context set). However, the number of noise in data introduces challenges to the algorithmic stability. In NPs, models are biased to the meta-datasets (a dataset of datasets), so small changes in the dataset (noisy or missing) may significantly change the models. As demonstrated in previous work [9, 10, 14, 11], existing NPs cannot provide stable predictions under noisy conditions, which may introduce high training error variance, and minimizing the training error may not guarantee consistent error reduction on the test set, i.e. low generalization performance [2]. In this case, algorithmic stability and generalization performance have strong connections and an unstable NP model has low generalization performance.
A stable model is one for which the learned solution does not change much with small changes in training set [2]. In general, heuristic techniques, such as cross-validation and ensemble learning, can be adopted to improve the generalization performance. Cross-validation needs to sacrifice the limited training data, while ensemble learning is computationally expensive on training sub-models. Recently, there are several improved NPs focused on considering model stability and
improving generalization performance empirically, such as hierarchical prior [27], stochastic attention mechanism [15], bootstrap [13], and Mixture of Expert [28]. However, most of them are unable to investigate the theoretical bound of the generalization performance of NPs. It is desirable to develop robust algorithms with low generalization error and high efficiency.
In this paper, we investigate NP-related models and explore more expressive stability toward general stochastic processes by proposing a stable solution. Specifically, by introducing the notion of stability into NPs, we focus on developing theoretical guidelines for deriving a stable NPs solution. We propose a method to find out subsets that are harder to predict than average, which is a key step for constructing this optimization problem. Based on it, a new extension of NPs with stable guarantees is formulated, which can be flexible to work with various NPs and achieves less biased approximation with theoretical guarantees. Considering the model adaptivity, an adaptive weighting strategy is proposed. To illustrate the superiority of the proposed stable solution, we perform experiments on synthetic 1D regression, system identification of physics engines, and real-world image completion tasks, and the results demonstrate that NPs with our stable solution are much more robust than original NPs.
In this section, we briefly review two different areas which are highly relevant to the proposed method, neural processes, and algorithmic stability.
Neural Processes Neural processes are a well-known member of the stochastic process family by directly capturing uncertainties with deep neural networks, which are not only computationally efficient but also retain a probabilistic interpretation of the model [9, 10, 14, 13]. Starting with conditional neural processes (CNP) [9], there have been several follow-up works to improve NPs in various aspects [6]. Vanilla CNP combines neural networks with the Gaussian process to extract prior knowledge from training data. NP [10] introduces a global latent variable to model uncertainty in a variational manner. Considering the problem of underfitting in the vanilla NP, Attentive NP [14] introduces the attention mechanism to improve the model's reconstruction quality. [11] introduced convolutional conditional neural process (CONVCNP) models translation equivariance in the data. Wang and Van Hoof [27] presented a doubly stochastic variational process (DSVNP), which combines both global and local latent variables. Lee et al. [18] extended NP using Bootstrap and proposed the bootstrapping neural processes (BNP). Kawano et al. [13] presented a group equivariant conditional neural process by incorporating group equivariant into CNPs in a meta-learning manner. Wang and van Hoof [28] proposed to combine the Mixture of Expert models with NPs to develop more expressive exchangeable stochastic processes. Kim et al. [15] proposed a stochastic attention mechanism for NPs to capture appropriate context information. Although there are many NP variants to improve the model performance, those do not consider stability to yield high generalization performance.
Algorithmic Stability Stability, as known as algorithmic stability, is a computational learning theory of how a machine learning algorithm is perturbed by small changes to its inputs [2]. Many efforts have been made to analyze various notions of algorithmic stability and prove that a broad spectrum of learning algorithms are stable in some sense [2, 3, 29, 12]. [3] proved that  $l_{2}$  regularized learning algorithms are uniformly stable and able to obtain new bounds on generalization performance. [29] generalized [3]'s results and proved that regularized learning algorithms with strongly convex penalty functions on bounded domains. Hardt et al. [12] showed that parametric models trained by stochastic gradient descent algorithms are uniformly stable. Li et al. [19] introduced the stability notation to low-rank matrix approximation. Liu et al. [22] proved that tasks in multi-task learning can act as regularizers and that multi-task learning in a very general setting will therefore be uniformly stable under mild assumptions. This is the first work to investigate the stability of NPs from theoretical guidelines and derive NPs solutions with high stability.
Let calligraphic letters (e.g.,  $\mathcal{A}$ ) indicate sets, capital letters (e.g.,  $A$ ) indicate scalars, lower-case bold letters (e.g.,  $\mathbf{a}$ ) denote vectors, and capital bold letters (e.g.,  $\mathbf{A}$ ) indicate matrices. Suppose there is a dataset  $\mathcal{D} = (\mathbf{X}, \mathbf{y}) = \{(x_i, y_i)\}_{i=1}^N$  with  $N$  data points  $\mathbf{X} = [\mathbf{x}_1, \mathbf{x}_2, \dots, \mathbf{x}_N]^\top \in \mathbb{R}^{N \times D}$ , and corresponding labels  $\mathbf{y} = [y_1, y_2, \dots, y_N] \in \mathbb{R}^N$ . Considering an arbitrary number of data points  $\mathcal{D}^{\mathcal{C}} = (\mathbf{X}^{\mathcal{C}}, \mathbf{y}^{\mathcal{C}}) = \{(x_i, y_i)\}_{i \in \mathcal{C}}$ , where  $\mathcal{C} \subseteq \{1, 2, \dots, N\}$  is an index set defining context information, neural processes model the conditional predictive distribution of the target values  $\mathbf{y}^{\mathcal{T}} = \{y_i\}_{i \in \mathcal{T}}$
at some target data points  $\mathbf{X}^{\mathcal{T}} = \{\mathbf{x}_i\}_{i\in \mathcal{T}}$  based on the context  $\mathcal{D}^{\mathcal{C}}$ , i.e.  $P(\mathbf{y}^{\mathcal{T}}|\mathbf{X}^{\mathcal{T}},\mathcal{D}^{\mathcal{C}})$ . Usually, target set is defined as  $\mathcal{T} = \{1,2,\dots ,N\}$ . Only in CNP [9],  $\mathcal{T}\subseteq \{1,2,\dots ,N\}$  and  $\mathcal{T}\cap \mathcal{C} = \emptyset$ . In this paper, we define  $\mathcal{T} = \{1,2,\dots ,N\}$  for all NPs, i.e. conditional predictive distribution is  $P(\mathbf{y}|\mathbf{X},\mathcal{D}^{\mathcal{C}}) = \prod_{i = 1}^{N}P(y_{i}|\mathbf{x}_{i},\mathcal{D}^{\mathcal{C}})$ .
Fundamentally, there are two NP variants: deterministic and probabilistic. Deterministic NP [9], i.e. CNP, models the conditional distribution as  $P(\mathbf{y}|\mathbf{X},\mathcal{D}^{\mathcal{C}}) = P(\mathbf{y}|\mathbf{X},\mathbf{r}^{\mathcal{C}})$ , where  $\mathbf{r}^{\mathcal{C}} \in \mathbb{R}^{d}$  is an aggregated feature vector processed by a function that maps  $\mathcal{D}^{\mathcal{C}}$  into a finite-dimensional vector space in a permutation-invariant way. In probabilistic NPs [10], a latent variable  $\mathbf{z} \in \mathbb{R}^{d}$  is introduced to capture model uncertainty and the NPs infer  $P_{\theta}(\mathbf{z}|\mathcal{D}^{\mathcal{C}})$  given context set using the reparameterization trick [16] and models such a conditional distribution as  $P_{\theta}(y_i|\mathbf{x}_i,\mathcal{D}^{\mathcal{C}}) = \int P_{\theta}(y_i|\mathbf{x}_i,\mathcal{D}^{\mathcal{C}},\mathbf{z})P_{\theta}(\mathbf{z}|\mathcal{D}^{\mathcal{C}})d\mathbf{z}$  and it is trained by maximizing an ELBO:  $\mathbb{E}_{\mathbf{z} \sim P_{\theta}(\mathbf{z}|\mathbf{X},\mathbf{y})}[\log P_{\theta}(\mathbf{y}|\mathbf{X})] - KL[P_{\theta}(\mathbf{z}|\mathbf{X},\mathbf{y})\| P_{\theta}(\mathbf{z}|\mathcal{D}^{\mathcal{C}})]$ .
Meta Training NP Prediction To achieve fast prediction on a new context set at test time, NPs meta-learn a distribution over predictors. To perform meta-learning, we require a meta-dataset (dataset of datasets). We consider an unknown distribution  $\mu$  on an instance space  $\mathcal{X} \times \mathcal{Y}$ , and a set of independent sample  $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^N$  drawn from  $\mu$ :  $(\mathbf{x}_i, y_i) \sim \mu$  and  $\mathcal{D} \sim \mu^N$ . Suppose meta-dataset contains  $M$  datasets  $\mathcal{D}_{1:M} = \{\mathcal{D}_m\}_{m=1}^M$  with  $\mathcal{D}_m = \{\mathcal{D}_m^C, \mathcal{D}_m^T\}$ , we assume that all  $M$  datasets drawn from a common environment  $\tau$ , which is a probability measure on the set of probability measures on  $\mathcal{X} \times \mathcal{Y}$ . The draw of  $\mu \sim \tau$  indicates the encounter of a specific learning task  $\mu$  in the environment  $\tau$ . For simplicity, we assume that each dataset has the same sample size  $N$ . Following the previous work related to multi-task learning [24] and meta learning [5], The environment  $\tau$  induces a measure  $\mu_{N,\tau}$  on  $(\mathcal{X} \times \mathcal{Y})^N$  such that  $\mu_{N,\tau}(A) = \mathbb{E}_{\mu \sim \tau}[\mu^N(A)]$ ,  $\forall A \subseteq (\mathcal{X} \times \mathcal{Y})^N$ . Thus a dataset  $\mathcal{D}_m$  is independently sampled from a task  $\mu$  encountered in  $\tau$ , which is denoted as  $\mathcal{D}_m \sim \mu_{N,\tau}$  for  $m \in [M]$ .
Suppose there exists a meta parameter  $\theta$  indicating the shared knowledge among different tasks. In this case, a meta learning algorithm  $\mathcal{A}_{meta}$  for NPs takes meta-datasets  $\mathcal{D}_{1:M}$  as input, and then outputs a meta parameter  $\theta = \mathcal{A}_{meta}(\mathcal{D}_{1:M}) \sim P_{\theta|\mathcal{D}_{1:M}}$ . When given a new test dataset  $\mathcal{D}$ , we can evaluate the quality of the meta parameter  $\theta$  by the following true risk:
$$
R _ {\tau} (\theta) = \mathbb {E} _ {\mathcal {D} \sim \mu_ {N, \tau}} \mathbb {E} _ {U \sim P _ {\theta | \mathcal {D} _ {1: M}}} [ R _ {\mu} (\theta) ] \tag {1}
$$
where  $R_{\mu}(\theta) = -\mathbb{E}_{(\mathbf{x}_i,y_i)\sim \mu}\log P_\theta (y_i|\mathbf{x}_i,\mathcal{D}^\mathcal{C})$ . Usually,  $\tau$  and  $\mu$  are unknown, we can only estimate the meta parameter  $\theta$  from the observed data  $\mathcal{D}_{1:M}$ . In this case, the empirical risk w.r.t  $\theta$  is:
$$
R _ {\mathcal {D} _ {1: M}} (\theta) = 1 / M \sum_ {m = 1} ^ {M} \mathbb {E} _ {\theta \sim P _ {\theta | \mathcal {D} _ {m} ^ {C}}} R _ {\mathcal {D} _ {m}} (\theta) \tag {2}
$$
where  $R_{\mathcal{D}_m}(\theta) = -(1 / N)\sum_{i = 1}^N\log P_\theta (y_i|\mathbf{x}_i,\mathcal{D}^C)$ .
NPs have various strengths: 1) Efficiency: meta-learning allows NPs to incorporate information from a new context set and make predictions with a single forward pass. The complexity is linear or quadratic in the context size instead of cubic as with Gaussian process regression; 2) Flexibility: NPs can define a conditional distribution of an arbitrary number of target points, conditioning an arbitrary number of observations; 3) Permutation invariance: the encoders of NPs use set property [32] to make the target prediction permutation invariant. Thanks to these properties, NPs are widely-used in lots of tasks, e.g., Bayesian optimization [8], recommendation [20, 21], physics engines controlling [27] etc. While there are many NP variants to improve the performance of NPs [9, 10, 14, 13, 15, 28], those do not take model's stability into consider account yet, which is the key to the robustness of the model.
Stability of NP A stable learning algorithm has the property that replacing one element in the training set does not result in a significant change to the algorithm's output [2]. Therefore, if we take the training error as a random variable, the training error of a stable learning algorithm should have a small variance. This implies that stable algorithms have the property that the training errors are close to the testing error [2]. Based on the defined risks, the algorithmic stability of approximate  $\{y_i\}_{i\in \mathcal{T}}$  in NPs is defined as follows.
Definition 4.1. (Algorithmic Stability of Neural Processes) For any measure  $\mu_{N,\tau}$  on  $(\mathcal{X}\times \mathcal{Y})^N$  such that  $\mu_{N,\tau}(A) = \mathbb{E}_{\mu \sim \tau}[\mu^{N}(A)],\forall A\subseteq (\mathcal{X}\times \mathcal{Y})^{N}$ , sample  $M$  datasets  $\mathcal{D}_{1:M}$  from  $\mu_{N,\tau}$  randomly. For a given  $\epsilon >0$ , we say that  $R_{\mathcal{D}_{1:M}}(\theta)$  is  $\delta$ -stable if the following holds:
$$
P \left(\left| R _ {\tau} (\theta) - R _ {\mathcal {D} _ {1: M}} (\theta) \right| \leq \epsilon\right) \geq 1 - \delta . \tag {3}
$$
The above stability for NPs has the property that the generalization error is bounded, which indicates that minimizing the training error will have a high probability of minimizing the testing error. This new stability notion makes it possible to measure the generalization performance between different NP approximations. For instance, for any two meta-datasets  $\mathcal{D}_{1:M}^{1}$  and  $\mathcal{D}_{1:M}^{2}$  from  $\mu_{N,\tau}$ , train NPs on  $\mathcal{D}_{1:M}^{1}$  and  $\mathcal{D}_{1:M}^{2}$  are  $\delta_{1}$ -stable and  $\delta_{2}$ -stable, respectively. Then  $R_{\mathcal{D}_{1:M}^{1}}(\theta)$  is more stable than  $R_{\mathcal{D}_{1:M}^{2}}(\theta)$  if  $\delta_{1} < \delta_{2}$ . This implies that  $R_{\mathcal{D}_{1:M}^{1}}(\theta)$  is close to  $R_{\tau}(\theta)$  with higher probability than  $R_{\mathcal{D}_{1:M}^{2}}(\theta)$ , i.e. minimizing  $R_{\mathcal{D}_{1:M}^{1}}(\theta)$  will lead to solutions that are of high probabilities with better generalization performance than minimizing  $R_{\mathcal{D}_{1:M}^{2}}(\theta)$ .
Based on the above analysis, we can see that the reliability of data points is crucial to the success of NPs and frail NPs are susceptible to noise.
Stability vs. Generalization Error The sparsity of the data, incomplete and noisy introduces challenges to the algorithm stability. NP models are biased to the quality of context data and target data, so small changes in the training data (noisy) may significantly change the models. In this case, unstable solutions will introduce high training error variance, and minimizing the training error may not guarantee consistent error reduction on the testing dataset, i.e., low generalization performance. In other words, the algorithm stability has a direct impact on generalization performance, and an unstable NP solution has low generalization performance. We take NPs with 1D regression task as an example [9] to invest
Figure 1: Stability vs. generalization error with different numbers of replaced noisy datasets.
tigate the relationship between generalization performance and stability of NPs. The total number of training and testing datasets is 200 and 100. We trained the NPs model with curves generated from the Gaussian process with RBF kernels by replacing the normal data dataset with a noisy dataset, i.e. the number of replaced datasets is turned in  $\{1, 10, 20, 50, 80, 100\}$ . We quantify stability changes of NPs with the generalization error when the number of replaced datasets increases from 1 to 100. We compute the difference between training error and test error to measure generalization error. We define the difference between test error and training error as  $R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta)$ , and compute  $P(|R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta)| \leq \epsilon)$  with 100 different runs to measure stability. Here we choose  $\epsilon$  in Definition 4.1 as 0.0015 to cover all error differences when the number of replaced datasets is 1. As shown in Figure 1, the generalization error increases when the number of replaced points increases since the testing error becomes lower. On the contrary, the stability of NP decreases with the number of replaced points increases. This indicates that stability decreases with generalization error increases. This study demonstrates that existing NPs suffer from lower generalization performance due to low algorithmic stability. Therefore, it is important to develop a stable solution for NPs that offers good generalization performance.
In this section, inspired by the previous work [19], we present a stable solution for NPs with stability and high generalization. Algorithmic stability provides an intuitive way to measure the changes in the outputs of a learning algorithm when the input is changed. Various ways have been introduced to measure algorithmic stability. Following the definition of uniform stability [2], given a stable NP, the approximation results remain stable if the change of the datasets. For instance, we can remove a subset of easily predictable data points from  $\mathcal{D}_{1:M}$  to obtain  $\mathcal{D}_{1:M}^{\prime}$ . It is desirable that the solution of minimizing both  $\mathcal{D}_{1:M}$  and  $\mathcal{D}_{1:M}^{\prime}$  together will be more stable than the solution of minimizing  $\mathcal{D}_{1:M}$  only. The following Theorem formally proves the statement.
Theorem 5.1. Let  $\mathcal{D}_{1:M}$  ( $M \geq 2$ ) be a sampled meta-dataset of measure  $\mu_{N,\tau}$ . Let  $\mathcal{D}_s \in \mathcal{D}_{1:M}$  be a subset of the meta-dataset, which satisfy that  $\forall (\mathbf{x}_i, y_i) \in \mathcal{D}_s, -\log P_\theta(y_i | \mathbf{x}_i, \mathcal{D}^C) \leq R_{\mathcal{D}_{1:M}}(\theta)$ . Let  $\mathcal{D}_{1:M}' = \mathcal{D}_{1:M} - \mathcal{D}_s$ , then for any  $\epsilon > 0$  and  $1 > w_0 > 0$ ,  $1 > w_1 > 0$  ( $w_0 + w_1 = 1$ ),  $w_0 R_{\mathcal{D}_{1:M}}(\theta) + w_1 R_{\mathcal{D}_{1:M}', \mathcal{D}}'(\theta)$  and  $R_{\mathcal{D}_{1:M}}(\theta)$  are  $\delta_1$ -stable and  $\delta_2$ -stable, respectively, then  $\delta_1 \leq \delta_2$ .
Proof. Let's assume that  $R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta) \in [-a_1, a_1]$  and  $R_{\tau}(\theta) - (w_0 R_{\mathcal{D}_{1:M}}(\theta) + w_1 R_{\mathcal{D}_{1:M}'}(\theta)) \in [-a_2, a_2]$  are two random variables with zero mean, where  $a_1 = \sup \{R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta)\}$  and  $a_2 =$
$\sup \{R_{\tau}(\theta) - (w_0R_{\mathcal{D}_{1:M}}(\theta) + w_1R_{\mathcal{D}_{1:M}'}(\theta))\}$ . Based on Markov's inequality<sup>1</sup>, for any  $t > 0$ , we have
$$
P \left(R _ {\tau} (\theta) - R _ {\mathcal {D} _ {1: M}} (\theta) \geq \epsilon\right) \leq \frac {\mathbb {E} \left[ e ^ {t \left(R _ {\tau} (\theta) - R _ {\mathcal {D} _ {1 : M}} (\theta)\right)} \right]}{e ^ {t \epsilon}}. \tag {4}
$$
Based on Hoeffding's lemma², we have  $\mathbb{E}[e^{t(R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta))}] \leq e^{\frac{1}{2} t^2 a_1^2}$ , i.e.  $P(R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta) \geq \epsilon) \leq \frac{e^{\frac{1}{2} t^2 a_1^2}}{e^{t\epsilon}}$ . Similarly, we have  $P(R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta) \leq -\epsilon) \leq \frac{e^{\frac{1}{2} t^2 a_1^2}}{e^{t\epsilon}}$ . Combining those two inequalities, we have  $P(|R_{\tau}(\theta) - R_{\mathcal{D}_{1:M}}(\theta)| \geq \epsilon) \leq \frac{2e^{\frac{1}{2} t^2 a_1^2}}{e^{t\epsilon}}$ , i.e.
$$
P \left(\left| R _ {\tau} (\theta) - R _ {\mathcal {D} _ {1: M}} (\theta) \right| \leq \epsilon\right) \geq 1 - \frac {2 e ^ {\frac {1}{2} t ^ {2} a _ {1} ^ {2}}}{e ^ {t \epsilon}}. \tag {5}
$$
$$
P \left(\left| R _ {\tau} (\theta) - \left(w _ {0} R _ {\mathcal {D} _ {1: M}} (\theta) + w _ {1} R _ {\mathcal {D} _ {1: M} ^ {\prime}} (\theta)\right) \right| \leq \epsilon\right) \geq 1 - \frac {2 e ^ {\frac {1}{2} t ^ {2} a _ {2} ^ {2}}}{e ^ {t \epsilon}}. \tag {6}
$$
In this case, the relationship between  $a_1$  and  $a_2$  is
$$
\begin{array}{l} a _ {2} = \sup  \left\{R _ {\tau} (\theta) - R _ {\mathcal {D} _ {1: M}} (\theta) + w _ {1} \left(R _ {\mathcal {D} _ {1: M}} (\theta) - R _ {\mathcal {D} _ {1: M} ^ {\prime}} (\theta)\right) \right\} \\ = \sup  \left\{R _ {\tau} (\theta) - R _ {\mathcal {D} _ {1: M}} (\theta) \right\} + w _ {1} \sup  \left\{R _ {\mathcal {D} _ {1: M}} (\theta) - R _ {\mathcal {D} _ {1: M} ^ {\prime}} (\theta) \right\} \tag {7} \\ = a _ {1} + \lambda_ {1} \sup  \left\{R _ {\mathcal {D} _ {1: M}} (\theta) - R _ {\mathcal {D} _ {1: M} ^ {\prime}} (\theta) \right\} \\ \end{array}
$$
Since  $\forall (\mathbf{x}_i,y_i)\in \mathcal{D}_s, - \log P_\theta (y_i|\mathbf{x}_i,\mathcal{D}_s^{\mathcal{C}})\leq R_{\mathcal{D}_{1:M}}(\theta)$ , we have  $-(1 / N)\sum_{i = 1}^{N}\log P_{\theta}(y_i|\mathbf{x}_i,\mathcal{D}_s^{\mathcal{C}})\leq$ $R_{\mathcal{D}_{1:M}}(\theta)$ . Then, since  $\mathcal{D}_{1:M} = \mathcal{D}_s\cup \mathcal{D}_{1:M}^\prime$ . This means that  $\sup \{R_{\mathcal{D}_{1:M}}(\theta) - R_{\mathcal{D}_{1:M}^\prime}(\theta)\} \leq 0$ . Thus, we have  $a_2\leq a_1$ . This turns out that  $\frac{2e^{\frac{1}{2}t^2a_2^2}}{e^{t\epsilon}}\leq \frac{2e^{\frac{1}{2}t^2a_1^2}}{e^{t\epsilon}}$ , i.e.  $\delta_1\leq \delta_2$ .