SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jingwen Fu $^{1*}$ , Zhizheng Zhang $^{2\dagger}$ , Dacheng Yin $^{3*}$ , Yan Lu $^{2}$ , Nanning Zheng $^{1\dagger}$
{zhizzzhang,yanlu}  $@$  microsoft.com
ydc@mail.ustc.edu.cn
National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University,
$^{2}$ Microsoft Research Asia,  $^{3}$ University of Science and Technology of China
This paper explores the connection between learning trajectories of Deep Neural Networks (DNNs) and their generalization capabilities when optimized using (stochastic) gradient descent algorithms. Instead of concentrating solely on the generalization error of the DNN post-training, we present a novel perspective for analyzing generalization error by investigating the contribution of each update step to the change in generalization error. This perspective enable a more direct comprehension of how the learning trajectory influences generalization error. Building upon this analysis, we propose a new generalization bound that incorporates more extensive trajectory information. Our proposed generalization bound depends on the complexity of learning trajectory and the ratio between the bias and diversity of training set. Experimental observations reveal that our method effectively captures the generalization error throughout the training process. Furthermore, our approach can also track changes in generalization error when adjustments are made to learning rates and label noise levels. These results demonstrate that learning trajectory information is a valuable indicator of a model's generalization capabilities.
The generalizability of a Deep Neural Network (DNN) is a crucial research topic in the field of machine learning. Deep neural networks are commonly trained with a limited number of training samples while being tested on unseen samples. Despite the commonly used independent and identically distributed (i.i.d.) assumption between the training and testing sets, there often exists a varying degree of discrepancy between them in real-world applications. Generalization theories study the generalization of DNNs by modeling the gap between the empirical risk [37] and the popular risk [37]. Classical uniform convergence based methods [20] adopt the complexity of the function space to analyze this generalization error. These theories discover that more complex function space results in a larger generalization error [38]. However, they are not well applicable for DNNs [33, 22]. In deep learning, the double descent phenomenon [6] exists, which tells that larger complexity of function space may lead to smaller generalization error. This violates the aforementioned property in uniform convergence methods and imposes demands in studying the generalization of DNNs.
Although the function space of DNNs is vast, not all functions within that space can be discovered by learning algorithms. Therefore, some representative works bound the generalization of DNNs based
on the properties of the learning algorithm, e.g., stability of algorithm [11], information-theoretic analysis [40]. These works rely on the relation between the input (i.e., training data) and output (weights of the model after training) of the learning algorithm to infer the generalization ability of the learned model. Here, the relation refers to how the change of one sample in the training data impacts the final weights of model in the stability of algorithms while referring to the mutual information between the weights and the training data in the information-theoretic analysis. Although some works [24, 11] leverage some information from training process to understand the properties of learning algorithm, there is limited trajectory information conveyed.
The purpose of this article is to enhance our theoretical comprehension of the relation between learning trajectory and generalization. While some recent experiments [9, 13, 29] have shown a strong correlation between the information contained in learning trajectory and generalization, the theoretical understanding behind this is still underexplored. By investigating the contribution of each update step to the change in generalization error, we give a new generalization bound with rich trajectory related information. Our work can serve as a starting point to understand those experimental discoveries.
Our contributions can be summarized below:
- We demonstrate that learning trajectory information serves as a valuable indicator of generalization abilities. With this motivation, we present a novel perspective for analyzing generalization error by investigating the contribution of each update step to the change in generalization error.
- Utilizing the aforementioned modeling technique, we introduce a novel generalization bound for deep neural networks (DNNs). Our proposed bound provides a greater depth of trajectory-related insights than existing methods.
- Our method effectively captures the generalization error throughout the training process. And the assumption corresponding to this method is also confirmed by experiments. Furthermore, our approach can also track changes in generalization error when adjustments are made to learning rates and label noise levels.
Generalization Theories Existing works on studying the generalization of DNNs can be divided into three categories: the methods based on the complexity of function space, the methods based on learning algorithms, and the methods based on PAC Bayes. The first category considers the generalization of DNNs from the perspective of the complexity of the function space. Many methods for measuring the complexity of the function space have been proposed, e.g., VC dimension [39], Rademacher Complexity [4] and covering number [33]. These works fail in being applied to DNN models since the complexity of the function space of a DNN model is too large to deliver a trivial result [41]. This thus motivates recent works to rethink the generalization of DNNs based on the accessible information in different learning algorithms such as stability of algorithm [11], information-theoretic analysis [40]. Among them, the stability of algorithm [7] measures how one sample change of training data impacts the model weights finally learned, and the information theory [30, 31, 40] based generalization bounds rely on the mutual information of the input (training data) and output (weights after training) of the learning algorithm. Another line is PAC Bayes [19] based method, which bounds the expectation of the error rates of a classifier chosen from a posterior distribution in terms of the KL divergence from a given prior distribution. Our research modifies the conventional Rademacher Complexity to calculate the complexity of the space explored by a learning algorithm, which in turn helps derive the generalization bound. Our approach resembles the first category, as we also rely on the complexity of the function space. However, our method differs as we focus on the function space explored by the learning trajectory, rather than the entire function space. The novelty of our technique lies in addressing the issue of dependence between training data and the function space explored by the learning trajectory, a dependence that is not permitted by the original Rademacher Complexity Theory.
Generalization Analysis for SGD The optimization plays an nonnegligible role in the success of DNN. Therefore, there are many prior works studying the generalization of DNNs by exploring property of SGD, which could be summarized into two categories: stability of SGD and information-theoretic analysis. The most popular way of the former category is to analyze the stability of the weights updating. Hardt et al. [11] is the first work to analyze the stability of SGD with the requirements of smooth and Lipschitz assumptions. Its follow-up works try to discard the smooth [5], or Lipschitz [25] assumptions towards getting a more general bound. Information-theoretic methods leverage the chain rule of KL-divergence to calculate the mutual information between the learned model weights and the data. This kind of works is mainly applied for Stochastic Gradient Langevin Dynamics(SGLD), i.e., SGD with noise injected in each step of parameters updating [28]. Negrea et al. [23], Haghifam et al. [10] improve the results using data-dependent priors. Neu et al. [24] construct an auxiliary iterative noisy process to adapt this method to the SGD scenario. In contrast to these studies, our approach utilizes more information related to learning trajectories. A more detailed comparison can be found in Table 2 and Appendix B.
3 Generalization Bound
Let us consider a supervised learning problem with an instance space  $\mathcal{Z}$  and a parameter space  $\mathcal{W}$ . The loss function can be defined as  $f: \mathcal{W} \times \mathcal{Z} \to \mathbb{R}_+$ . We denote the distribution of the instance space  $\mathcal{Z}$  as  $\mu$ . The  $n$  i.i.d samples draw from  $\mu$  are denoted as  $S = \{z_1, \dots, z_n\} \sim \mu^n$ . Given parameters  $\mathbf{w}$ , the empirical risk and popular risk are denoted as  $F_S(\mathbf{w}) \triangleq \frac{1}{n} \sum_{i}^{n} f(\mathbf{w}, z_i)$ , and  $F_{\mu}(\mathbf{w}) \triangleq \mathbb{E}_{z \sim \mu}[f(\mathbf{w}, z)]$  respectively. Our work studies the generalization error of the learned model, i.e.  $F_{\mu}(\mathbf{w}) - F_S(\mathbf{w})$ . For an optimization process, the learning trajectory is represented as a function  $\mathbf{J}: \mathbb{N} \to \mathcal{W}$ . We use  $\mathbf{J_t}$  to denote the weights of model after  $t$  times updating, where  $\mathbf{J_t} = \mathbf{J}(t)$ . The learning algorithm is defined as  $\mathcal{A}: \mu^n \times \mathbb{R} \to \mathbf{J}$ , where the second input  $\mathbb{R}$  denotes all randomness in the algorithm  $\mathcal{A}$ , including the randomness in initialization, batch sampling et al.. We simply use  $\mathcal{A}(S)$  to represent a random choice for the second input term. Given two functions  $U, V, \int_t U(t) \mathrm{d}V(t) \triangleq \sum_t U(t)(V(t + 1) - V(t))$  and we use  $\|\cdot\|$  to denote  $L2$  norm. If  $S$  is a set, then  $|S|$  denotes the number of elements in  $S$ .  $\mathbb{E}_t$  denotes taking the expectation conditioned on  $\{\mathbf{J_i} | i \leq t\}$ .
Let mini-batch  $B$  be a random subset sampled from dataset  $S$ , and we have  $|B| = b$ . The averaged function value of mini-batch  $B$  is denoted as  $F_{B}(\mathbf{w}) \triangleq \frac{1}{b}\sum_{z\in B}f(\mathbf{w},z)$ . The parameters updated with gradient descent can be formulated as:
$$
\mathbf {J} _ {\mathbf {t} + \mathbf {1}} = \mathbf {J} _ {\mathbf {t}} - \eta_ {t} \nabla F _ {S} (\mathbf {J} _ {\mathbf {t}}). \tag {1}
$$
where  $\eta_{t}$  is the learning rate for the  $t$ -th update. The parameter updating with stochastic gradient descent is:
$$
\mathbf {J} _ {\mathbf {t} + \mathbf {1}} = \mathbf {J} _ {\mathbf {t}} - \eta_ {t} \nabla F _ {B} (\mathbf {J} _ {\mathbf {t}}). \tag {2}
$$
Let  $\epsilon (\mathbf{w})\triangleq \nabla F_S(\mathbf{w}) - \nabla F_B(\mathbf{w})$  be the gradient noise in mini-batch updating, where  $\mathbf{w}$  is the weights of a DNN. Then we can transform Equation (2) into:
$$
\mathbf {J} _ {\mathbf {t} + \mathbf {1}} = \mathbf {J} _ {\mathbf {t}} - \eta_ {t} \nabla F _ {S} \left(\mathbf {J} _ {\mathbf {t}}\right) + \eta_ {t} \epsilon \left(\mathbf {J} _ {\mathbf {t}}\right). \tag {3}
$$
The covariance of the gradients over the entire dataset  $S$  can be calculated as:
$$
\Sigma (\mathbf {w}) \triangleq \frac {1}{n} \sum_ {i = 1} ^ {n} \nabla f (\mathbf {w}, z _ {i}) \nabla f (\mathbf {w}, z _ {i}) ^ {\mathrm {T}} - \nabla F _ {S} (\mathbf {w}) \nabla F _ {S} (\mathbf {w}) ^ {\mathrm {T}}. \tag {4}
$$
Therefore, the covariance of the gradient noise  $\epsilon (\mathbf{w})$  is:
$$
C (\mathbf {w}) \triangleq \frac {n - b}{b (n - 1)} \Sigma (\mathbf {w}). \tag {5}
$$
Since for any  $w$  we have  $\mathbb{E}(\epsilon(\mathbf{w})) = 0$ , we can represent  $\epsilon(\mathbf{w})$  as  $C(\mathbf{w})^{\frac{1}{2}}\epsilon'$ , where  $\epsilon'$  is a random distribution whose mean is zero and covariance matrix is an identity matrix. Here,  $\epsilon'$  can be any distributions, including Gaussian distribution [12] and  $S\alpha S$  distribution [35].
The primary objective of our work is to suggest a new generalization bound that incorporates more comprehensive trajectory-related information. The key aspects of this information are: 1) It should be adaptive and change according to different learning trajectories. 2) It should not rely on the extra information from data distribution  $\mu$  except from the training data  $S$ .
3.1 Investigating generalization alone learning trajectory
As annotated before, the learning trajectory is represented by a function  $\mathbf{J}:\mathbb{N}\to \mathcal{W}$ , which defines the relationship between the model weights and the training timesteps  $t$ .  $\mathbf{J}_t$  denotes the model weights after  $t$  times updating. Note that  $\mathbf{J}$  depends on  $S$ , because it comes from the equation  $\mathbf{J} = \mathcal{A}(S)$ . We simply use  $f(\mathbf{J}_{\mathbf{t}}):\mathcal{Z}\rightarrow \mathbb{R}_{+}$  to represent the function after  $t$ -times update. Our goal is to analyze the generalization error, i.e.,  $F_{\mu}(\mathbf{J}_{\mathbf{T}}) - F_{S}(\mathbf{J}_{\mathbf{T}})$ , where  $T$  represents the total training steps.
We reformulate the function corresponding to the finally obtained model as:
$$
f \left(\mathbf {J} _ {\mathbf {T}}\right) = f \left(\mathbf {J} _ {\mathbf {0}}\right) + \sum_ {t = 1} ^ {T} \left(f \left(\mathbf {J} _ {\mathbf {t}}\right) - f \left(\mathbf {J} _ {\mathbf {t} - \mathbf {1}}\right)\right). \tag {6}
$$
Therefore, the generalization error can be rewritten as:
$$
F _ {\mu} (\mathbf {J} _ {\mathbf {T}}) - F _ {S} (\mathbf {J} _ {\mathbf {T}}) = \underbrace {F _ {\mu} (\mathbf {J} _ {\mathbf {0}}) - F _ {S} (\mathbf {J} _ {\mathbf {0}})} _ {(i)} + \sum_ {t = 1} ^ {T} \underbrace {\left[ \left(F _ {\mu} (\mathbf {J} _ {\mathbf {t}}) - F _ {\mu} (\mathbf {J} _ {\mathbf {t} - \mathbf {1}})\right) - \left(F _ {S} (\mathbf {J} _ {\mathbf {t}}) - F _ {S} (\mathbf {J} _ {\mathbf {t} - \mathbf {1}})\right) \right]} _ {(i i) _ {t}}. \tag {7}
$$
In this form, we divide the generalization error into two parts.  $(i)$  is the generalization error before the training.  $(ii)_t$  is the generalization error caused by  $t$ -step update.
Typically, there is independence between  $\mathbf{J}_0$  and the data  $S$ . Therefore, we have  $\mathbb{E}(i) = 0$ . Combining with this, we have:
$$
\mathbb {E} \left[ F _ {\mu} \left(\mathbf {J} _ {\mathbf {T}}\right) - F _ {S} \left(\mathbf {J} _ {\mathbf {T}}\right) \right] = \mathbb {E} \sum_ {t = 1} ^ {T} (i i) _ {t}. \tag {8}
$$
Analyzing the generalization error after training can be transformed into analyzing the increase of generalization error for each update. This is a straightforward and quite different way to extract the information from learning trajectory compared with previous work. Here, we list two techniques that most used by previous works to extract the information from learning trajectory.
- (T1). This method leverages the chaining rule of mutual information to calculate a upper bound of the mutual information between  $\mathbf{J}_{\mathbf{T}}$  and the training data  $S$ , i.e.  $I(S; \mathbf{J}_{\mathbf{T}}) \leq I(S; \mathbf{J}_{\mathbf{t} \leq \mathbf{T}}) \leq \sum_{t=0}^{T} I(S; \mathbf{J}_{\mathbf{t}} | \mathbf{J}_{\mathbf{i} < t})$ .  $I(S; \mathbf{J}_{\mathbf{T}})$  is the value of concerning for their theory.
- (T2). This method assumes we have another data  $S'$ , which is obtained by replacing one sample in data  $S$  with another sample drawing from distribution  $\mu$ .  $\mathbf{J}'$  is the learning trajectory trained from data  $S'$  with same randomness value as  $\mathbf{J}$ . Denote  $\Delta_k \triangleq \| \mathbf{J}_k - \mathbf{J}_k' \|$  and assume  $\Delta_0 = 0$ . Then, the value of concerning is  $\Delta_T$ . The upper bound of  $\Delta_T$  is calculated by iteratively applying the formula  $\Delta_k \leq c_{k-1} \Delta_{k-1} + e_{k-1}$ .
(T1) is commonly utilized in analyzing Stochastic Gradient Langevin Dynamics(SGLD) [18, 2, 28], while (T2) is frequently employed in stability-based works for analyzing SGD [11, 15, 5]. Our method offers several benefits, including: 1) We directly focus on the change in generalization error, rather than intermediate values such as  $\Delta_{k}$  and  $I(S; \mathbf{J}_{\mathbf{t}} | \mathbf{J}_{\mathbf{i} < \mathbf{t}})$ , 2) The generalization error is equivalent to the sum of  $(ii)_t$ , while (T1) and (T2) takes the upper bound value of  $I(S; \mathbf{J}_{\mathbf{T}})$  and  $\Delta_T$ , and 3) From this perspective, We can extract more in-depth trajectory-related information. For (T1), the computation of  $I(S; \mathbf{J}_{\mathbf{t}} | \mathbf{J}_{\mathbf{i} < \mathbf{t}})$  primarily involves the information of  $\nabla F_{\mu}(\mathbf{J}_{\mathbf{t}})$ , which is inaccessible to us (Detail in Appendix D and Neu et al. [24]). (T2) faces the challenge that only the upper bounds of  $c_k$  and  $e_k$  can be calculated. The upper bounds remain unchanged across various learning trajectories. Consequently, both (T1) and (T2) have difficulty conveying meaningful trajectory information.
3.2 A New Generalization Bound
In this section, we introduce the generalization bound based on our aforementioned modeling. Let us start with the definition of commonly used assumptions.
Definition 3.1. The function  $f$  is  $L$ -Lipschitz, if for all  $\mathbf{w}_1, \mathbf{w}_2 \in \mathcal{W}$  and for all  $z \in \mathcal{Z}$ , wherein we have  $\| f(\mathbf{w}_1, z) - f(\mathbf{w}_2, z) \| \leq L \| \mathbf{w}_1 - \mathbf{w}_2 \|$ .
Definition 3.2. The function  $f$  is  $\beta$ -smooth, if for all  $\mathbf{w}_1, \mathbf{w}_2 \in \mathcal{W}$  and for all  $z \in \mathcal{Z}$ , wherein we have  $\| \nabla f(\mathbf{w}_1, z) - \nabla f(\mathbf{w}_2, z) \| \leq \beta \| \mathbf{w}_1 - \mathbf{w}_2 \|$ .
Definition 3.3. The function  $f$  is convex, if for all  $\mathbf{w}_1, \mathbf{w}_2 \in \mathcal{W}$  and for all  $z \in \mathcal{Z}$ , wherein we have  $f(\mathbf{w}_1, z) \geq f(\mathbf{w}_2, z) + (\mathbf{w}_1 - \mathbf{w}_2)^{\mathrm{T}} \nabla f(\mathbf{w}_2, z)$ .
Here,  $L$ -lipschitz assumption implies that the  $\| \nabla f(\mathbf{w},z)\| \leq L$  holds.  $\beta$ -smooth assumption indicates the largest eigenvalue of  $\nabla^2 f(\mathbf{w},z)$  is smaller than  $\beta$ . The convexity indicates the smallest eigenvalue of  $\nabla^2 f(\mathbf{w},z)$  are positive. These assumptions tell us the constraints of gradients and Hessian matrices of the training data and the unseen samples in the test set. Since the values of gradients and Hessian matrices in the training set are accessible, the key role of these assumptions is to deliver knowledge about the unseen samples in the test set.
In the following, we introduce a new generalization bound. We give the assumption required by our new generalization bound in the following.
Assumption 3.4. There is a value  $\gamma$ , so that for all  $\mathbf{w} \in \{\mathbf{J}_{\mathbf{t}} | t \in \mathbb{N}\}$ , we have  $\|\nabla F_{\mu}(\mathbf{w})\| \leq \gamma \| \nabla F_{S}(\mathbf{w}) \|$ .
Remark 3.5. Assumption 3.4 gives a restriction with the norm of popular gradient  $\nabla F_{\mu}(\mathbf{w})$ . This assumption is easily satisfied when  $n$  is a large number, because we have  $\lim_{n\to \infty}\| \nabla F_S(\mathbf{w})\| = \| \nabla F_{\mu}(\mathbf{w})\|$ . When the  $n$  is not large enough, the assumption will hold before SGD enters the neighbourhood of convergent point. Under the case that SGD enters the neighbourhood of convergent point, we give a relaxed assumption and its corresponding generalization bound in Appendix B. According to paper [42], this case will unusually happen in real situation. Section 4 gives experiments to explore the assumption.
Theorem 3.6. Under Assumption 3.4, given  $S \sim \mu^n$ , let  $\mathbf{J} = \mathcal{A}(S)$ , where  $\mathcal{A}$  denoted the SGD or GD algorithm training with  $T$  steps, we have:
$$
\mathbb {E} \left[ F _ {\mu} \left(\mathbf {J} _ {\mathbf {T}}\right) - F _ {S} \left(\mathbf {J} _ {\mathbf {T}}\right) \right] \leq - 2 \gamma^ {\prime} \mathbb {V} _ {m} \mathbb {E} \int_ {t} \frac {d F _ {S} \left(\mathbf {J} _ {\mathbf {t}}\right)}{\sqrt {n}} \sqrt {1 + \frac {\operatorname {T r} \left(\Sigma \left(\mathbf {J} _ {\mathbf {t}}\right)\right)}{\| \nabla F _ {S} \left(\mathbf {J} _ {\mathbf {t}}\right) \| _ {2} ^ {2}}} + \mathcal {O} (\eta_ {m}) \tag {9}
$$
where  $\mathbb{V}(\mathbf{w}) = \frac{\|\nabla F_S(\mathbf{w})\|}{\mathbb{E}_{U\subset S}\|\frac{|U|}{n}\nabla F_U(\mathbf{w}) - \frac{n - |U|}{n}\nabla F_{S / U}(\mathbf{w})\|},\quad \mathbb{V}_m = \max_t\mathbb{V}(\mathbf{J}_t),\quad \gamma' =$ $\max \{1,\max_{U\subset S;t}\frac{|U||\nabla F_U(\mathbf{J}_t)||}{n\|\nabla F_S(\mathbf{J}_t)\|}\} \gamma and\eta_m\triangleq \max_t\eta_t.$
Remark 3.7. Our generalization bound mainly relies on the information from gradients.  $\mathbb{V}(\mathbf{w})$  is related to the variance of the gradient. When the variance of the gradients across different samples in the training set  $S$  is large, then the value of  $\mathbb{V}(\mathbf{w})$  is small, and vice versa. Note that we have  $|U| < n$  due to  $U\subset S$ . Our bound will become trivial if  $\mathbb{E}_{U\subset S}\| \frac{|U|}{n}\nabla F_U(\mathbf{w}) - \frac{n - |U|}{n}\nabla F_{S / U}(\mathbf{w})\| = 0$ . This rarely happens in real case, because it requires that for all  $U\subset S$ , we have  $|U|\nabla F_U(\mathbf{w}) = (n - |U|)\nabla F_{S / U}(\mathbf{w})$ . We also give a relaxed assumption version of this theorem in Appendix B. The generalization bound provides a clear insight into how the reduction of training loss leads to a increase in generalization error.
Proof Sketch The proof of this theorem is placed in Appendix A. Here, we give the sketch for this proof.
Step 1 Beginning with Equation (8), we decompose the  $F_{\mu}(\mathbf{J}_{\mathbf{T}}) - F_{S}(\mathbf{J}_{\mathbf{T}})$  into a linear part  $(\mathrm{gen}^{lin}(\mathbf{J}_{\mathbf{T}}))$  and nonlinear part  $(\mathrm{gen}^{nl}(\mathbf{J}_{\mathbf{T}}))$ . We have  $\mathrm{gen}^{lin}(\mathbf{J}_{\mathbf{T}}) = \sum_{t=1}^{T}(ii)_{t}^{lin}$ , where  $(ii)_{t}^{lin} \triangleq (\mathbf{J}_{\mathbf{t}} - \mathbf{J}_{\mathbf{t}-1})^{\mathrm{T}}(\nabla F_{\mu}(\mathbf{J}_{\mathbf{t}-1}) - \nabla F_{S}(\mathbf{J}_{\mathbf{t}-1}))$ . The nonlinear part is  $\mathrm{gen}^{nl}(\mathbf{J}_{\mathbf{T}}) = F_{\mu}(\mathbf{J}_{\mathbf{T}}) - F_{S}(\mathbf{J}_{\mathbf{T}}) - \mathrm{gen}^{lin}(\mathbf{J}_{\mathbf{T}})$ . We take these two parts differently. Here, we focus on analyzing  $\mathrm{gen}^{lin}(\mathbf{J}_{\mathbf{T}})$  because it dominates under small learning rate. Detail discussion of  $\mathrm{gen}^{nl}(\mathbf{J}_{\mathbf{T}})$  is given in Appendix (Proposition A.1 and Subsection C.3)
Step 2 We construct the additive linear space  $\mathcal{L}_{\mathbf{J}|S} \triangleq \{\sum_{t=0}^{T-1} \mathbf{w}_t^{\mathrm{T}} \nabla f(\mathbf{J}_t) \mid \| \mathbf{w}_t \| \leq \Delta_t\}$ , where  $\Delta_t \triangleq \| \eta_t \nabla F_S(\mathbf{J}_t) \|$ . Then  $\mathbb{E}[\mathrm{gen}^{lin}(\mathbf{J}_T)] \leq 2\gamma' \mathbb{V}_m \mathbb{E}R_S(\mathcal{L}_{\mathbf{J}|S})$ , where  $R_S(\mathcal{L}_{\mathbf{J}|S}) \triangleq \mathbb{E}_{\sigma} \sup_{h \in \mathcal{L}_{\mathbf{J}|S}} \left( \frac{1}{n} \sum_{i=1}^{n} \sigma_i h(z_i) \right)$ .