SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiayuan Ye†, Zhenyu Zhu‡, Fanghui Liu‡* Reza Shokri†, Volkan Cevher‡
† National University of Singapore ‡ EPFL, Switzerland † University of Warwick †{jiayuan,reza}@comp.nus.edu.sg ‡{zhenyu.zhu, volkan.cevher}@epfl.ch †fanghui.liu@warwick.ac.uk
We analytically investigate how over-parameterization of models in randomized machine learning algorithms impacts the information leakage about their training data. Specifically, we prove a privacy bound for the KL divergence between model distributions on worst-case neighboring datasets, and explore its dependence on the initialization, width, and depth of fully connected neural networks. We find that this KL privacy bound is largely determined by the expected squared gradient norm relative to model parameters during training. Notably, for the special setting of linearized network, our analysis indicates that the squared gradient norm (and therefore the escalation of privacy loss) is tied directly to the per-layer variance of the initialization distribution. By using this analysis, we demonstrate that privacy bound improves with increasing depth under certain initializations (LeCun and Xavier), while degrades with increasing depth under other initializations (He and NTK). Our work reveals a complex interplay between privacy and depth that depends on the chosen initialization distribution. We further prove excess empirical risk bounds under a fixed KL privacy budget, and show that the interplay between privacy utility trade-off and depth is similarly affected by the initialization.
Deep neural networks (DNNs) in the over-parameterized regime (i.e., more parameters than data) perform well in practice but the model predictions can easily leak private information about the training data under inference attacks such as membership inference attacks [44] and reconstruction attacks [17, 7, 29]. This leakage can be mathematically measured by the extent to which the algorithm's output distribution changes if DNNs are trained on a neighboring dataset (differing only in a one record), following the differential privacy (DP) framework [23].
To train differential private model, a typical way is to randomly perturb each gradient update in the training process, such as stochastic gradient descent (SGD), which leads to the most widely applied DP training algorithm in the literature: DP-SGD [2]. To be specific, in each step, DP-SGD employs gradient clipping, adds calibrated Gaussian noise, and yields differential privacy guarantee that scales with the noise multiplier (i.e., per-dimensional Gaussian noise standard deviation divided by the clipping threshold) and number of training epochs. However, this privacy bound [2] is overly general as its gradient clipping artificially neglects the network properties (e.g., width and depth) and training schemes (e.g., initializations). Accordingly, a natural question arises in the community:
How does the over-parameterization of neural networks (under different initializations) affect the privacy bound of the training algorithm over worst-case datasets?
*Work was done while Fanghui was at LIONS, EPFL.
Table 1: Our privacy utility trade-off bounds for training linearized network (3) via Langevin diffusion, under different hidden-layer width  $m$ , depth  $L$  and initializations. The per-layer widths  $m_0 = d$ ,  $m_1, \dots, m_{L-1} = m$  and  $m_L = o$  where  $d$  is the data dimension and  $o$  is number of classes. For KL privacy bounds, we assume Assumption 2.2 holds and  $L \geq 2$  for simplicity. For the excess risk bounds, we assume  $o = 1$ ,  $d, m = \tilde{\Omega}(n)$  are large, and Assumption 2.2. Under LeCun and Xavier, we prove privacy utility trade-offs that improve with over-parameterization (increasing depth).
Initialization	Variance βlfor layer l	Gradient norm constant B (7)	Approximate lazy training distance R (9)	Excess Empirical risk under ε-KL privacy (Corollary 6.4)
LeCun [34]	1/ml-1	om(L-1+d/m)/2L-1	O(n/m·1/L-1+d/m)	O(1/n2+√1/2Lε)
He [30]	2/ml-1	om(L-1+d/m)	O(n/m·1/L-1+d/m)	O(1/n2+√1/ε)
NTK [3]	2/ml, l< L1/o, l=L	dm(L-1/2+o/m)	O(n/m·1/L-1/2+1/m)	O(1/n2+√d/ε)
Xavier [27]	2/ml-1+ml	od(L-1+d+o/2m)/2L-3(1+d/m)(1+o/m)	O(n/d·(1+d/m)(1+1/m)/L-1+1/2m)	O(1/n2+√1/2Lε)
To answer this question, we circumvent the difficulties of analyzing gradient clipping, and instead algorithmically focus on analyzing privacy for the Langevin diffusion algorithm without gradient clipping nor Lipschitz assumption on loss function. It avoids an artificial setting in DP-SGD [2] where a constant sensitivity constraint is enforced for each gradient update and thus makes the privacy bound insensitive to the network over-parameterization. Theoretically, we prove that the KL privacy loss for Langevin diffusion scales with the expected gradient difference between the training on any two worst-case neighboring datasets (Theorem 3.1). By proving precise upper bounds on the expected  $\ell_2$ -norm of this gradient difference, we thus obtain KL privacy bounds for fully connected neural network (Lemma 3.2) and its linearized variant (Corollary 4.2) that changes with the network width, depth and per-layer variance for the initialization distribution. We summarized the details of our KL privacy bounds in Table 1, and highlight our key observations below.
- Width always worsen privacy, under all the considered initialization schemes. Meanwhile, the interplay between network depth and privacy is much more complex and crucially depends on which initialization scheme is used and how long the training time is.
- Regarding the specific initialization schemes, under small per-layer variance in initialization (e.g. in LeCun and Xavier), if the depth is large enough, our KL privacy bound for training fully connected network (with a small amount of time) as well as linearized network (with finite time) decays exponentially with increasing depth. To the best of our knowledge, this is the first time that an improvement of privacy bound under over-parameterization is observed.
We further perform numerical experiments (Section 5) on deep neural network trained via noisy gradient descent to validate our privacy analyses. Finally, we analyze the privacy utility trade-off for training linearized network, and prove that the excess empirical risk bound (given any fixed KL privacy budget) scales with a lazy training distance bound  $R$  (i.e., how close is the initialization to a minimizer of the empirical risk) and a gradient norm constant  $B$  throughout training (Corollary 6.4). By analyzing these two terms precisely, we prove that under certain initialization distributions (such as LeCun and Xavier), the privacy utility trade-off strictly improves with increasing depth for linearized network (Table 1). To our best knowledge, this is the first time that such a gain in privacy-utility trade-off due to over-parameterization (increasing depth) is shown. Meanwhile, prior results only prove (nearly) dimension-independent privacy utility trade-off for such linear models in the literature [45, 32, 37]. Our improvement demonstrates the unique benefits of our algorithmic framework and privacy-utility analysis in understanding the effect of over-parameterization.
2A key difference between this paper and existing privacy utility analysis of Langevin diffusion [26] is that we analyze in the absence of gradient clipping or Lipschitz assumption on loss function. Our results also readily extend to discretized noisy GD with constant step-size (as discussed in Appendix E).
<sup>3</sup>We focus on KL privacy loss because it is a more relaxed distinguishability notion than standard  $(\varepsilon, \delta)$ -DP, and therefore could be upper bounded even without gradient clipping. Moreover, KL divergence enables upper bound for the advantage (relative success) of various inference attacks, as studied in recent works [39, 28].
over-parameterization in DNNs and NTK. Theoretical demonstration on the benefit of overparameterization in DNNs occurs in global convergence [3, 21] and generalization [4, 16]. Under proper initialization, the training dynamics of over-parameterized DNNs can be described by a kernel function, termed as neural tangent kernel (NTK) [31], which stimulates a series of analysis in DNNs. Accordingly, over-parameterization has been demonstrated to be beneficial/harmful to several topics in deep learning, e.g., robustness [15, 54], covariate shift [50]. However, the relationship between over-parameterization and privacy (based on the differential privacy framework) remains largely an unsolved problem, as the training dynamics typically change [14] after adding new components in the privacy-preserving learning algorithm (such as DP-SGD [2]) to enforce privacy constraints.
Membership inference privacy risk under over-parameterization. A recent line of works [47, 48] investigates how over-parameterization affects the theoretical and empirical privacy in terms of membership inference advantage, and proves novel trade-off between privacy and generalization error. These literature are closet to our objective of investigating the interplay between privacy and overparameterization. However, Tan et al. [47, 48] focus on proving upper bounds for an average-case privacy risk defined by the advantage (relative success) of membership inference attack on models trained from randomly sampled training dataset from a population distribution. By contrast, our KL privacy bound is heavily based on the strongest adversary model in the differential privacy definition, and holds under an arbitrary worst-case pair of neighboring datasets, differing only in one record. Our model setting (e.g., fully connected neural networks) is also quite different from that of Tan et al. [47, 48]. The employed analysis tools are accordingly different.
Differentially private learning in high dimension. Standard results for private empirical risk minimization [9, 46] and private stochastic convex optimization [11, 12, 5] prove that there is an unavoidable factor  $d$  in the empirical risk and population risk that depends on the model dimension. However, for unconstrained optimization, it is possible to seek for the dimension-dependency in proving risk bounds for certain class of problems (such as generalized linear model [45]). Recently, there is a growing line of works that proves dimension-independent excess risk bounds for differentially private learning, by utilizing the low-rank structure of data features [45] or gradient matrices [32, 37] during training. Several follow-up works [33, 13] further explore techniques to enforce the low-rank property (via random projection) and boost privacy utility trade-off. However, all the works focus on investigating a general high-dimensional problem for private learning, rather than separating the study for different network choices such as width, depth and initialization. Instead, our study focuses on the fully connected neural network and its linearized variant, which enables us to prove more precise privacy utility trade-off bounds for these particular networks under over-parameterization.
2 Problem and Methodology
We consider the following standard multi-class supervised learning setting. Let  $\mathcal{D} = (z_1, \dots, z_n)$  be an input dataset of size  $n$ , where each data record  $z_i = (x_i, y_i)$  contains a  $d$ -dimensional feature vector  $x_i \in \mathbb{R}^d$  and a label vector  $y_i \in \mathcal{Y} = \{-1, 1\}^o$  on  $o$  classes. We aim to learn a neural network output function  $f_W(\cdot): \mathcal{X} \to \mathcal{Y}$  parameterized by  $\mathbf{W}$  via empirical risk minimization (ERM)
$$
\min  _ {\boldsymbol {W}} \mathcal {L} (\boldsymbol {W}; \mathcal {D}) := \frac {1}{n} \sum_ {i = 1} ^ {n} \ell \left(\boldsymbol {f} _ {\boldsymbol {W}} \left(\boldsymbol {x} _ {i}\right); \boldsymbol {y} _ {i}\right), \tag {1}
$$
where  $\ell(f_{W}(\boldsymbol{x}_i); \boldsymbol{y}_i)$  is a loss function that reflects the approximation quality of model prediction  $f_{W}(\boldsymbol{x}_i)$  compared to the ground truth label  $\boldsymbol{y}_i$ . For simplicity, throughout our analysis, we employ the cross-entropy loss  $\ell(f_{W}(\boldsymbol{x}); \boldsymbol{y}) = -\langle \boldsymbol{y}, \log \text{softmax}(f_{W}(\boldsymbol{x})) \rangle$  for multi-class network with  $o \geq 2$ , and  $\ell(f_{W}(\boldsymbol{x}); \boldsymbol{y}) = \log (1 + \exp(-\boldsymbol{y} f_{W}(\boldsymbol{x}))$  for single-output network with  $o = 1$ .
Fully Connected Neural Networks. We consider the  $L$ -layer, multi-output, fully connected, deep neural network (DNN) with ReLU activation. Denote the width of hidden layer  $l$  as  $m_{l}$  for  $l = 1,\dots ,L - 1$ . For consistency, we also denote  $m_0 = d$  and  $m_L = o$ . The network output  $f_{W}(\pmb {x})\coloneqq h_{L}(\pmb {x})$  is defined recursively as follows.
$$
\boldsymbol {h} _ {0} (\boldsymbol {x}) = \boldsymbol {x}; \quad \boldsymbol {h} _ {l} (\boldsymbol {x}) = \phi \left(\boldsymbol {W} _ {l} \boldsymbol {x}\right) \text {f o r} l = 1, \dots , L - 1; \quad \boldsymbol {h} _ {L} (\boldsymbol {x}) = \boldsymbol {W} _ {L} \boldsymbol {h} _ {L - 1} (\boldsymbol {x}), \tag {2}
$$
where  $h_l(\pmb{x})$  denotes the post-activation output at  $l$ -th layer, and  $\{\pmb{W}_l \in \mathbb{R}^{m_l \times m_{l-1}} : l = 1, \dots, L\}$  denotes the set of per-layer weight matrices of DNN. For brevity, we denote the vector  $\pmb{W} \coloneqq (\mathrm{Vec}(\pmb{W}_1), \dots, \mathrm{Vec}(\pmb{W}_L)) \in \mathbb{R}^{m_1 \cdot d + m_2 \cdot m_1 + \dots + o \cdot m_{L-1}}$ , i.e., the concatenation of vectorizations for weight matrices of all layers, as the model parameter.
Linearized Network. We also analyze the following linearized network, which is used in prior works [35, 3, 41] as an important tool to (approximately and qualitatively) analyze the training dynamics of DNNs. Formally, the linearized network  $f_{W}^{lin,0}(x)$  is a first-order Taylor expansion of the fully connected ReLU network at initialization parameter  $W_0^{lin}$ , as follows.
$$
\boldsymbol {f} _ {\boldsymbol {W}} ^ {\text {l i n}, 0} (\boldsymbol {x}) \equiv \boldsymbol {f} _ {\boldsymbol {W} _ {0} ^ {\text {l i n}}} (\boldsymbol {x}) + \left. \frac {\partial \boldsymbol {f} _ {\boldsymbol {W}} (\boldsymbol {x})}{\partial \boldsymbol {W}} \right| _ {\boldsymbol {W} = \boldsymbol {W} _ {0} ^ {\text {l i n}}} \left(\boldsymbol {W} - \boldsymbol {W} _ {0} ^ {\text {l i n}}\right), \tag {3}
$$
where  $f_{W_0^{lin}}(\pmb{x})$  is the output function of the fully connected ReLU network (2) at initialization  $W_0^{lin}$ . We denote  $\mathcal{L}_0^{lin}(\pmb{W};\mathcal{D}) = \frac{1}{n}\sum_{i=1}^n\ell\left(\pmb{f}_{\pmb{W}_0^{lin}}(\pmb{x}_i) + \frac{\partial\pmb{f}_{\pmb{W}}(\pmb{x})}{\partial\pmb{W}}|_{\pmb{W}=\pmb{W}_0^{lin}}(\pmb{W}-\pmb{W}_0^{lin});\pmb{y}_i\right)$  as the empirical loss function for training linearized network, by plugging (3) into (1).
Langevin Diffusion. Regarding the optimization algorithm, we focus on the Langevin diffusion algorithm [36] with per-dimensional noise variance  $\sigma^2$ . Note that we aim to avoid gradient clipping while still proving KL privacy bounds. After initializing the model parameters  $W_0$  at time zero, the model parameters  $W_{t}$  at subsequent time  $t$  evolves as the following stochastic differential equation.
$$
\mathrm {d} \boldsymbol {W} _ {t} = - \nabla \mathcal {L} (\boldsymbol {W} _ {t}; \mathcal {D}) \mathrm {d} t + \sqrt {2 \sigma^ {2}} \mathrm {d} \boldsymbol {B} _ {t}. \tag {4}
$$
Initialization Distribution. The initialization of parameters  $W_0$  crucially affects the convergence of Langevin diffusion, as observed in prior literatures [52, 25, 24]. In this work, we investigate the following general class of Gaussian initialization distributions with different (possibly depth-dependent) variances for the parameters in each layer. For any layer  $l = 1, \dots, L$ , we have
$$
\left[ \boldsymbol {W} ^ {l} \right] _ {i j} \sim \mathcal {N} (0, \beta_ {l}), \text {f o r} (i, j) \in [ m _ {l} ] \times [ m _ {l - 1} ], \tag {5}
$$
where  $\beta_{1},\dots ,\beta_{L} > 0$  are the per-layer variance for Gaussian initialization. By choosing different variances, we recover many common initialization schemes in the literature, as summarized in Table 1.
2.1 Our objective and methodology
We aim to understand the relation between privacy, utility and over-parameterization (depth and width) for the Langevin diffusion algorithm (under different initialization distributions). For privacy analysis, we prove a KL privacy bound for running Langevin diffusion on any two worst-case neighboring datasets. Below we first give the definition for neighboring datasets.
Definition 2.1. We denote  $\mathcal{D}$  and  $\mathcal{D}'$  as neighboring datasets if they are of same size and only differ in one record. For brevity, we also denote the differing records as  $(\pmb{x}, \pmb{y}) \in \mathcal{D}$  and  $(\pmb{x}', \pmb{y}') \in \mathcal{D}'$ .
Assumption 2.2 (Bounded Data). For simplicity, we assume bounded data, i.e.,  $\| x\| _2\leq \sqrt{d}$
We now give the definition for KL privacy, which is a more relaxed, yet closely connected privacy notion to the standard  $(\varepsilon, \delta)$  differential privacy [22], see Appendix A.2 for more discussions. KL privacy and its relaxed variants are commonly used in previous literature [8, 10, 53].
Definition 2.3 (KL privacy). A randomized algorithm  $\mathcal{A}$  satisfies  $\varepsilon$ -KL privacy if for any neighboring datasets  $\mathcal{D}$  and  $\mathcal{D}'$ , we have that the KL divergence  $\mathrm{KL}(\mathcal{A}(\mathcal{D})\| \mathcal{A}(\mathcal{D}')\leq \varepsilon$ , where  $\mathcal{A}(\mathcal{D})$  denotes the algorithm's output distribution on dataset  $\mathcal{D}$ .
In this paper, we prove KL privacy upper bound for  $\max_{\mathcal{D},\mathcal{D}'}\mathrm{KL}(\pmb{W}_{[0:T]}\| \pmb{W}_{[0:T]}')$  when running Langevin diffusion on any worst-case neighboring datasets. For brevity, here (and in the remaining paper), we abuse the notations and denote  $\pmb{W}_{[0:T]}$  and  $\pmb{W}_{[0:T]}^{\prime}$  as the distributions of model parameters trajectory during Langevin diffusion processes Eq. (4) with time  $T$  on  $\mathcal{D}$  and  $\mathcal{D}'$  respectively.
For utility analysis, we prove the upper bound for the excess empirical risk given any fixed KL divergence privacy budget for a single-output neural network under the following additional assumption (it is only required for utility analysis and not needed for our privacy bound).
Assumption 2.4 ([40, 20, 21]). The training data  $\boldsymbol{x}_1, \dots, \boldsymbol{x}_n$  are i.i.d. samples from a distribution  $P_x$  that satisfies  $\mathbb{E}[\boldsymbol{x}] = 0$ ,  $\| \boldsymbol{x} \|_2 = \sqrt{d}$  for  $\boldsymbol{x} \sim P_x$ , and with probability one for any  $i \neq j$ ,  $\boldsymbol{x}_i \nVdash \boldsymbol{x}_j$ .
Our ultimate goal is to precisely understand how the excess empirical risk bounds (given fixed KL privacy budget) are affected by increasing width and depth under different initialization distributions.
3 KL Privacy for Training Fully Connected ReLU Neural Networks
In this section, we perform the composition-based KL privacy analysis for Langevin Diffusion given random Gaussian initialization distribution under Eq. (5) for fully connected ReLU network. More specifically, we prove upper bound for the KL divergence between distribution of output model parameters when running Langevin diffusion on an arbitrary pair of neighboring datasets  $\mathcal{D}$  and  $\mathcal{D}'$ .
Our first insight is that by a Bayes rule decomposition for density function, KL privacy under a relaxed gradient sensitivity condition can be proved (that could hold without gradient clipping).
Theorem 3.1 (KL composition under possibly unbounded gradient difference). The KL divergence between running Langevin diffusion (4) for DNN (2) on neighboring datasets  $\mathcal{D}$  and  $\mathcal{D}'$  satisfies
$$
\operatorname {K L} \left(\boldsymbol {W} _ {[ 0: T ]} \| \boldsymbol {W} _ {[ 0: T ]} ^ {\prime}\right) = \frac {1}{2 \sigma^ {2}} \int_ {0} ^ {T} \mathbb {E} \left[ \| \nabla \mathcal {L} \left(\boldsymbol {W} _ {t}; \mathcal {D}\right) - \nabla \mathcal {L} \left(\boldsymbol {W} _ {t}; \mathcal {D} ^ {\prime}\right) \| _ {2} ^ {2} \right] \mathrm {d} t. \tag {6}
$$
Proof sketch. We compute the partial derivative of KL divergence with regard to time  $t$ , and then integrate it over  $t \in [0, T]$  to compute the KL divergence during training with time  $T$ . For computing the limit of differentiation, we use Girsanov's theorem to compute the KL divergence between the trajectory of Langevin diffusion processes on  $\mathcal{D}$  and  $\mathcal{D}'$ . The complete proof is in Appendix B.1.
Theorem 3.1 is an extension of the standard additivity [51] of KL divergence (also known as chain rule [1]) for a finite sequence of distributions to continuous time processes with (possibly) unbounded drift difference. The key extension is that Theorem 3.1 does not require bounded sensitivity between the drifts of Langevin Diffusion on neighboring datasets. Instead, it only requires finite second-order moment of drift difference (in the  $\ell_2$ -norm sense) between neighboring datasets  $\mathcal{D}$ ,  $\mathcal{D}'$ , which can be proved by the following Lemma. We prove that this expectation of squared gradient difference incurs closed-form upper bound under deep neural network (under mild assumptions), for running Langevin diffusion (without gradient clipping) on any neighboring dataset  $\mathcal{D}$  and  $\mathcal{D}'$ .
Lemma 3.2 (Drift Difference in Noisy Training). Let  $M_T$  be the subspace spanned by gradients  $\{\nabla \ell(\pmb{f}_{\pmb{W}_t}(\pmb{x}_i; \pmb{y}_i) : (\pmb{x}_i, \pmb{y}_i) \in \mathcal{D}, t \in [0, T]\}_{i=1}^n$  throughout Langevin diffusion  $(\pmb{W}_t)_{t \in [0, T]}$ . Denote  $\|\cdot\|_{M_T}$  as the  $\ell_2$  norm of the projected input vector onto  $M_T$ . Suppose that there exists constants  $c, \beta > 0$  such that for any  $\pmb{W}$ ,  $\pmb{W}'$  and  $(\pmb{x}, \pmb{y})$ , we have  $\|\nabla \ell(f_{\pmb{W}}(\pmb{x}); \pmb{y})) - \nabla \ell(f_{\pmb{W}'}(x); \pmb{y})\|_2 < \max\{c, \beta \| \pmb{W} - \pmb{W}'\|_{M_T}\}$ . Then running Langevin diffusion Eq. (4) with Gaussian initialization distribution (5) satisfies  $\varepsilon$ -KL privacy with  $\varepsilon = \frac{\max_{\mathcal{D}, \mathcal{D}'} \int_0^T \mathbb{E}\left[\|\nabla \mathcal{L}(\pmb{W}_t; \mathcal{D}) - \nabla \mathcal{L}(\pmb{W}_t; \mathcal{D}')\|_2^2\right] dt}{2\sigma^2}$  where
$$
\begin{array}{l} \int_ {0} ^ {T} \mathbb {E} \left[ \| \nabla \mathcal {L} (\boldsymbol {W} _ {t}; \mathcal {D}) - \nabla \mathcal {L} (\boldsymbol {W} _ {t}; \mathcal {D} ^ {\prime}) \| _ {2} ^ {2} \right] \mathrm {d} t \leq 2 T \cdot \underbrace {\mathbb {E} \left[ \| \nabla \mathcal {L} (\boldsymbol {W} _ {0} ; \mathcal {D}) - \nabla \mathcal {L} (\boldsymbol {W} _ {0} ; \mathcal {D} ^ {\prime}) \| _ {2} ^ {2} \right]} _ {\text {g r a d i e n t d i f f e r e n c e a t i n i t i a l i z a t i o n}} \\ + \underbrace {\frac {2 \beta^ {2}}{n ^ {2} (2 + \beta^ {2})} \left(\frac {e ^ {(2 + \beta^ {2}) T} - 1}{2 + \beta^ {2}} - T\right) \cdot \left(\mathbb {E} \left[ \| \nabla \mathcal {L} (\boldsymbol {W} _ {0} ; \mathcal {D}) \| _ {2} ^ {2} \right] + 2 \sigma^ {2} r a n k (M _ {T}) + c ^ {2}\right)} _ {\text {g r a d i e n t d i f f e r e n c e f l u c t u a t i o n d u r i n g t r a n i n g}} + \underbrace {\frac {2 c ^ {2} T}{n ^ {2}}} _ {\text {n o n - s m o o t h n e s s}}. \\ \end{array}
$$
Proof sketch. The key is to reduce the problem of upper bounding the gradient difference at any training time  $T$ , to analyzing its two subcomponents:  $\| \nabla \ell (f_{\mathbf{W}_t}(\pmb {x});\pmb {y})) - \nabla \ell (f_{\mathbf{W}_t}(\pmb{x}');\pmb {y}')\| _2^2\leq$  2||Vf(w0(x);y)-Vf(w0(x');y')||2+22 +2c², where (x,y) and
$(\pmb{x}^{\prime},\pmb{y}^{\prime})$  are the differing data between neighboring datasets  $\mathcal{D}$  and  $\mathcal{D}'$ . This inequality is by the Cauchy-Schwartz inequality. In this way, the second term in Lemma 3.2 uses the change of parameters