SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {1} _ {\mathcal {E}} \cdot \max  _ {k \in [ I ]} \| \boldsymbol {y} _ {r, k} ^ {i} - \bar {\boldsymbol {x}} _ {r} \| \leq 2 (K + 1) I (2 \sigma \eta + \gamma), \quad f o r a n y i \in [ N ]. \tag {5}
$$
The proof of Lemma 1 is deferred to Appendix A.3, which relies on a jump start analysis. If  $\| G_r\| >\gamma /\eta$ , then (4) trivially holds due to the clipping operation. When  $\| G_r\| \leq \gamma /\eta$ , a recursive argument shows that the discrepancy  $\| \pmb{x}_{r,k}^i -\bar{\pmb{x}}_r\|$  depends on the magnitude of the increment at the starting point, that is the lag error  $\| \mathbb{E}[G_r^i ] - \nabla f_i(\bar{x}_r)\|$  and  $\| G_r\|$  after removing noise. According to the construction of  $G_{r}^{i}$ , we know  $\mathbb{E}[G_r^i ] = \frac{1}{I}\sum_{k = 1}^I\nabla f_i(\pmb{y}_{r,k}^i) = \frac{1}{I}\sum_{k = 1}^I\nabla f_i(\pmb{x}_{q_r^i,k}^i)$ . Therefore, under the event  $\mathcal{E}$ , the lag error at the  $r$ -th round can be bounded by the discrepancies of previous  $K + 1$  rounds at most, which is the bound (5). In summary, the discrepancy at any round can be controlled recursively if the discrepancy at the initial round is small. This insight motivates the initialization  $G_0^i = \nabla F_i(\bar{x}_0,\tilde{\xi}_i)$  in Algorithm 1, which enjoys zero initial lag error.
Finally, Lemma 1 shows that the condition of (3) can be satisfied by choosing  $\eta, \gamma, I$ , which establishes descent of the global objective from  $\bar{\boldsymbol{x}}_r$  to  $\bar{\boldsymbol{x}}_{r + 1}$ . Summing from  $r = 0,\dots ,R - 1$  and applying concentration bounds over martingale difference sequences to yield high probability bounds for error terms coming from stochastic gradient noise and client subsampling yields
$$
f \left(\bar {\boldsymbol {x}} _ {R}\right) - f \left(\bar {\boldsymbol {x}} _ {0}\right) \leq \sum_ {r = 0} ^ {R - 1} \left[ \mathbb {1} _ {\bar {A} _ {r}} U \left(\bar {\boldsymbol {x}} _ {r}\right) + \mathbb {1} _ {\mathcal {A} _ {r}} V \left(\bar {\boldsymbol {x}} _ {r}\right) \right] + \left(1 2 \sigma^ {2} + \Gamma_ {2} I\right) \eta \log \frac {1}{\delta}, \tag {6}
$$
where  $\mathcal{A}_r = \{\| G_r\| \leq \gamma /\eta \}$  denotes the clipping indicator, and  $U(x),V(x)$  are defined in the proof of Theorem 1. According to the choice of  $\eta ,\gamma$  , the dominant term of  $U(\bar{\pmb{x}}_r)$  is  $-\gamma I\| \nabla f(\bar{\pmb{x}}_r)\|$  and that of  $V(\bar{\pmb{x}}_r)$  is  $-\eta I\| \nabla f(\bar{\pmb{x}}_r)\|^2$  . Plugging into (6) and rearranging proves Theorem 1.
5 Lower Bound for Clipped Minibatch SGD
Clipped minibatch SGD is a natural extension of minibatch SGD [40, 45] to the relaxed smooth setting (see pseudocode in Algorithm 2, Appendix B). Clipped minibatch SGD is nearly identical to minibatch SGD: with the addition of gradient clipping to each round's update. In the similar spirit of [45], we are interested in this algorithm because it has the same computation and communication structure as EPISODE++ and it is important to understand whether EPISODE++ has any advantage over clipped minibatch SGD. In fact, we will show that clipped minibatch SGD is significantly hindered by the combination of relaxed smoothness, client heterogeneity, and client subsampling.
Assumption 2. There exists  $M > 0$  such that  $\| \nabla f(\pmb {x})\| \leq M$  for all  $\pmb{x}$  with  $f(\pmb {x})\leq f(\pmb {x}_0)$
A line of work on relaxed smoothness in the single-machine setting [52, 8] has shown that the number of iterations required to find an  $\epsilon$ -stationary point by gradient descent (GD) under relaxed smoothness is  $\tilde{\Omega}\left(\frac{\Delta L_1M}{\epsilon^2}\right)$ , while that of GD with gradient clipping is  $O\left(\frac{\Delta L_0}{\epsilon^2}\right)$ . In this way, gradient clipping can remove the dependence on  $M$ , which can be large, and significantly speed up optimization.
Theorem 2 shows a surprising result: under some conditions on the participation ratio, number of clients, and heterogeneity parameter  $\rho$ , clipped minibatch SGD requires  $\tilde{\Omega}\left(\frac{\Delta L_1M}{\epsilon^2}\right)$  rounds, showing that applying gradient clipping to minibatch SGD does not eliminate dependence on  $M$ .
Theorem 2. Fix  $\epsilon >0,0 < \delta < 1,L_0 > 0,L_1 > 0,\kappa >0,\rho >\frac{2 + \log(2 - \delta)}{\log(2 - \delta)},M > \max (\frac{L_0}{L_1},\epsilon),N\geq$ $\frac{(\rho + 1)(1 + \log(2 - \delta))}{(\rho - 1)\log(2 - \delta) - 2}$ . Define  $Q = \left\lfloor \frac{\kappa + (\rho - 1)M}{\kappa + (\rho + 1)M} N\right\rfloor$ . Let  $\mathcal{F}(L_0,L_1,M,\kappa ,\rho ,N)$  denote the set of problem instances  $\{f_i\}_{i = 1}^N$  satisfying Assumptions I(ii)-(iv) and 2 with  $\sigma = 0$ . For any fixed choice of parameters  $\gamma ,\eta$  based on the knowledge of above constants, there exists  $\{f_i\}_{i = 1}^N\in \mathcal{F}(L_0,L_1,M,\kappa ,\rho ,N)$  such that clipped minibatch SGD initialized at  $\pmb{x}_0$  with  $1\le S\le \frac{\log(2 - \delta)(Q + 1)}{N - Q + \log(2 - \delta)}$  will satisfy  $\mathbb{P}\left(\| \nabla f(\pmb {x}_r)\| < \epsilon \text{ for some } 0\leq r\leq R - 1\right) > 1 - \delta$  only if
$$
R \geq \frac {L _ {1} M \left(f (\boldsymbol {x} _ {0}) - f ^ {*} - \frac {1 5 \epsilon^ {2}}{1 6 L _ {0}}\right)}{2 \epsilon^ {2} \left(1 + \log \frac {L _ {1} M}{L _ {0}}\right)}.
$$
The proof is included in Appendix B. This result shows that clipped minibatch SGD in our setting suffers the same problem as GD in the single-machine setting: divergence can only be avoided with a very small step size, leading to slow convergence. In contrast, the convergence rate of EPISODE++ in the same setting is independent of  $M$ .
The proof of Theorem 2 analyzes clipped minibatch SGD for three different problem instances. The first contains linear local objectives with high heterogeneity: if the clipping threshold is sufficiently small  $(\frac{\gamma}{\eta} \leq M)$ , then clipped minibatch SGD will never converge with probability  $\delta$ . The second instance contains homogeneous, exponential local objectives: the learning rate must be sufficiently small  $\eta < O\left(\frac{1}{L_1M}\right)$  to avoid divergence due to the exponentially increasing gradient magnitude. However, with a large clipping threshold and small learning rate, the convergence of clipped minibatch SGD will depend on  $M$  for the third problem instance, which has homogeneous linear objectives. Note that our lower bound is different from previous lower bounds in [52, 51] which are in the settings of single machine [52] or almost sure bounded noise [51], since our lower bound is considering noise from client subsampling and client data heterogeneity which is not almost surely bounded.
To validate our theory, we evaluate EPISODE++ and baselines in the training of RNNs for two text classification tasks. We compare EPISODE++ to CELGC [32], clipped minibatch SGD, and NaiveParallelClip [32], which is a naive parallel implementation of SGD with gradient clipping that requires communication at every iteration. As an ablation study, we also evaluate two algorithms closely related to EPISODE++. The first is a naive extension of EPISODE [9] for client subsampling, where each participating client's control variate  $G_r^i$  is resampled at the beginning of each round, and  $G_r = \frac{1}{S} \sum_{i \in S_r} G_r^i$ . The second is an algorithm which we refer to as SCAFFOLDClip [9], which applies gradient clipping to each local step of SCAFFOLD [24]. We evaluate these six
(a) Training loss and testing accuracy for SNLI dataset.
(b) Training loss and testing accuracy for Sentiment140 dataset.
Figure 1: Final training loss and testing accuracy for all algorithms, as participation ratio and data similarity varies. (a) and (b) show results for SNLI and Sentiment140, respectively.
algorithms on natural language inference with the SNLI dataset [4] and sentiment classification with the Sentiment140 dataset [5]. More experimental results can be found in Appendix C.
All experiments use uniform client sampling, a batch size of 64 (on each client) and the multi-class hinge loss. See Appendix C.1 for full details on hyperparameters. All experiments were implemented in PyTorch and ran on eight NVIDIA V100 GPUs.
SNLI SNLI [4] is a 3-way text classification task, in which the logical relationship of a pair of sentences must be classified as either entailment, neutral, or contradictory. The dataset contains 570k pairs of sentences. Because SNLI is a centralized dataset, we follow the heterogeneity protocol in [24] to divide the dataset into clients according to a similarity parameter  $s$  between  $0\%$  and  $100\%$ . According to this protocol,  $s\%$  of each client's local dataset is allocated from a randomly shuffled set of examples, while the remaining  $(100 - s)\%$  is allocated from a set of examples which is sorted by label. In this way, the similarity of the label distributions of client datasets grows with  $s$ . The network consists of a one-layer bidirectional RNN encoder followed by a three-layer fully connected classifier. We train for  $R = 5375$  communication rounds with  $I = 4$  for all algorithms except NaiveParallelClip, which uses  $R = 21500$  and  $I = 1$ , so that every algorithm runs the same number of training steps.
Sentiment140 Sentiment140 [5] is a sentiment prediction task designed for FL. The dataset is comprised of tweets, each labeled as either positive or negative. We follow the data processing steps of [28] to discard users with small datasets and to split into training and testing sets. In order to control the data heterogeneity between clients, we follow a similar protocol as described for SNLI to form client datasets by combining the datasets of original users in the Sentiment140 dataset. The process is nearly identical to that of SNLI, but here we allocate users to each local dataset instead of examples.  $s\%$  of each client's local dataset is allocated from a randomly shuffled set of users, while the remaining  $(100 - s)\%$  is allocated from a set of users sorted by the proportion of positive samples. We train for  $R = 2000$  communication rounds with  $I = 4$  for all algorithms except NaiveParallelClip, which uses  $R = 8000$  and  $I = 1$ . We use the same network architecture as SNLI.
Figure 2: Learning curves for SNLI and Sentiment140 under the setting  $S = 4$ ,  $s = 30\%$  (SNLI) and  $S = 4$ ,  $s = 10\%$  (Sentiment140). For NaiveParallelClip, we show the first 5375 (SNLI) and 2000 (Sentiment140) rounds to compare all algorithms with a fixed number of communication rounds.
Method	SNLI	Sent140
Train Loss	Test Accuracy	Train Loss	Test Accuracy
EPISODE++	0.317 ± 0.001	83.89 ± 0.07	0.258 ± 0.004	77.97 ± 0.16
CELGC	0.529 ± 0.005	77.45 ± 0.37	0.493 ± 0.005	76.36 ± 0.18
NaiveParallelClip	0.377 ± 0.001	81.74 ± 0.09	0.365 ± 0.004	77.73 ± 0.07
Clipped MinibatchSGD	0.642 ± 0.008	72.74 ± 0.20	0.549 ± 0.004	75.10 ± 0.27
SCAFFOLDClip	0.424 ± 0.002	81.11 ± 0.06	0.431 ± 0.001	77.38 ± 0.16
EPISODE	0.455 ± 0.005	80.24 ± 0.13	0.466 ± 0.003	76.89 ± 0.17
Table 2: Average results for three trials under  $S = 4$ ,  $s = 30\%$  (SNLI) or  $S = 4$ ,  $s = 10\%$  (Sent140). The error is the distance from the average to the max/min across three runs.
We study the effect of client subsampling and data heterogeneity by varying each of these values in a controlled way. We first fix the heterogeneity  $s = 30\%$  and vary  $S \in \{2,4,6,8\}$ , then we fix the number of participating clients  $S = 4$  and vary  $s \in \{10\%, 30\%, 50\% \}$ . It should be noted that in the experiments with varying  $S$ , we always train for a fixed number of iterations  $RI$ . This means that separate training runs with different  $S$  will have the same per-client computation cost (number of gradient computations), but the total computation cost of a training run scales with  $S$ . For this setting, we use  $N = 8$  clients.
To simulate large-scale federated learning, we also include results with a larger number of clients  $N = 128$ . For this setting, we use  $S = 16$  participating clients in each round, and data heterogeneity of  $s = 30\%$  (SNLI) and  $s = 10\%$  (Sentiment140).
Figure 1 contains the final training loss and testing accuracy for the variety of settings of client participation and data heterogeneity. For a single setting of participation and heterogeneity, Figure 2 and Table 2 show learning curves and average results over three trials, respectively. More learning curves are given in Appendix C.2. Learning curves for large-scale experiments are shown in Figure 3. EPISODE++ achieves the minimum training loss and maximum testing accuracy of all algorithms in nearly every setting. Only with full participation  $S = 8$  does NaiveParallelClip achieve a lower training loss than EPISODE++, but EPISODE++ maintains a higher testing accuracy. Also, NaiveParallelClip requires a much larger communication cost than EPISODE++ to perform the same number of training iterations: when the number of communication rounds is fixed, EPISODE++ significantly outperforms NaiveParallelClip.
Effect of Subsampling The first two plots of Figures 1(a) and 1(b) show the performance of each algorithm as the number of participating clients  $S$  varies over  $\{2,4,6,8\}$  with fixed data similarity  $s = 30\%$ . Clipped minibatch SGD, NaiveParallelClip, and EPISODE all exhibit degraded performance as  $S$  decreases, whereas EPISODE++, SCAFFOLDClip, and CELGC maintain relatively constant performance as  $S$  decreases. Despite the constant performance of CELGC and SCAFFOLDClip under client sampling, both algorithms are significantly outperformed by EPISODE++.
Figure 3: Learning curves for large-scale training with  $N = 128$ ,  $S = 16$ , and  $s = 30\%$  (SNLI) or  $s = 10\%$  (Sentiment140). We compare all algorithms with a fixed number of communication rounds.
Effect of Heterogeneity The last two plots of each row in Figure 1 show each algorithm's performance as the data similarity  $s$  varies over  $\{50\%, 30\%, 10\% \}$  for SNLI and  $\{20\%, 10\%, 0\% \}$  for Sentiment140, with fixed  $S = 4$ . For both datasets, decreasing data similarity negatively impacts the performance of clipped minibatch SGD, CELGC, and NaiveParallelClip. EPISODE++, SCAFFOLDClip, and EPISODE are able to maintain performance as data similarity decreases, though EPISODE++ maintains a significantly better performance than SCAFFOLDClip and EPISODE.
Large-Scale Experiments With a larger number of clients ( $N = 128$ ,  $S = 16$ ), the relative performance of each algorithm is similar to the  $N = 8$  setting, as shown in Figure 3. Here, the proportion of participating clients  $S / N = 1 / 8$  is smaller than that of the  $N = 8$  experiments, where  $S / N \geq 1 / 4$ . This suggests that the effect of partial client participation may be stronger in the large-scale experiments. EPISODE++ still outperforms all other algorithms in the large-scale setting, while maintaining about the same test accuracy as the  $N = 8$  setting, demonstrating the effectiveness of EPISODE++ for large-scale federated learning.
Comparison with Ablations By comparing EPISODE++ against the closely related EPISODE and SCAFFOLDClip, we can see that the use of information from previous rounds and episodic gradient clipping are both critical for the superior performance of EPISODE++. EPISODE only utilizes information from the currently participating clients, ignoring information from clients that participated in previous rounds. As a result, the performance of EPISODE degrades as  $S$  decreases. On the other hand, SCAFFOLDClip determines whether to perform clipping individually for each local step, as opposed to the episodic gradient clipping of EPISODE++. and EPISODE. Although SCAFFOLDClip maintains performance under changes in the participation ratio and data similarity, the level it maintains is significantly lower than that of EPISODE++.
Communication Cost NaiveParallelClip suffers a large communication cost for the same number of training iterations compared with other algorithms, due to the cost of synchronizing clients at every iteration. As shown in Figure 2 and Figure 3, EPISODE++ outperforms all other algorithms by a wide margin when the number of communication rounds is fixed. Also, EPISODE requires twice the number of communication operations per training round, which doubles the time required for communication per round compared to all other algorithms.
We have presented EPISODE++, the first algorithm for FL with heterogeneous data and client subsampling under relaxed smoothness. We proved that EPISODE++ finds an  $\epsilon$ -stationary point with high probability, and its convergence rate satisfies linear speedup and resilience to heterogeneity while enjoying reduced communication. We also presented a lower bound showing that the convergence rate of a special case of clipped minibatch SGD in our setting suffers a dependence on  $M$  (the maximum gradient norm of the objective in a sublevel set), implying that applying gradient clipping to minibatch SGD does not alleviate the problem of exploding gradients. Our experimental results for RNN training on text classification tasks demonstrate the superior performance of EPISODE++ compared to baselines. One limitation of our current work is that our lower bound assumes  $\sigma = 0$ , and we plan to get a better lower bound in the future for  $\sigma > 0$ .
We would like to thank the anonymous reviewers for their helpful comments. Michael Crawshaw is supported by the Institute for Digital Innovation fellowship from George Mason University. Michael Crawshaw and Mingrui Liu are both supported by a grant from George Mason University. Computations were run on ARGO, a research computing cluster provided by the Office of Research Computing at George Mason University (URL: https://orc.gmu.edu). The work of Yajie Bao was done when he was virtually visiting Mingrui Liu's research group in the Department of Computer Science at George Mason University.
[1] Yossi Arjevani, Yair Carmon, John C Duchi, Dylan J Foster, Nathan Srebro, and Blake Woodworth. Lower bounds for non-convex stochastic optimization. Mathematical Programming, 199 (1-2):165-214, 2023.
[2] Yajie Bao, Michael Crawshaw, Shan Luo, and Mingrui Liu. Fast composite optimization and statistical recovery in federated learning. In International Conference on Machine Learning, pages 1508-1536. PMLR, 2022.
[3] Rémi Bardenet and Odalric-Ambrym Maillard. Concentration inequalities for sampling without replacement. Bernoulli, 21(3):1361 - 1385, 2015. doi: 10.3150/14-BEJ605. URL https://doi.org/10.3150/14-BEJ605.
[4] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2015.
[5] Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018.
[6] Wenlin Chen, Samuel Horvath, and Peter Richtarik. Optimal client sampling for federated learning. arXiv preprint arXiv:2010.13723, 2020.
[7] Yae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, and Tong Zhang. On the convergence of federated averaging with cyclic client participation. arXiv preprint arXiv:2302.03109, 2023.
[8] Michael Crawshaw, Mingrui Liu, Francesco Orabona, Wei Zhang, and Zhenxun Zhuang. Robustness to unbounded smoothness of generalized signsgd. Advances in neural information processing systems, 2022.
[9] Michael Crawshaw, Yajie Bao, and Mingrui Liu. Episode: Episodic gradient clipping with periodic resampled corrections for federated learning with heterogeneous data. In The Eleventh International Conference on Learning Representations, 2023.
[10] Ashok Cutkosky and Harsh Mehta. High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems, 34:4883-4895, 2021.
[11] Aymeric Dieuleveut and Kumar Kshitij Patel. Communication trade-offs for local-sgd with large step size. Advances in Neural Information Processing Systems, 32:13601-13612, 2019.
[12] Matthew Faw, Litu Rout, Constantine Caramanis, and Sanjay Shakkottai. Beyond uniform smoothness: A stopped analysis of adaptive sgd. arXiv preprint arXiv:2302.06570, 2023.
[13] Yann Fraboni, Richard Vidal, Laetitia Kameni, and Marco Lorenzi. Clustered sampling: Low-variance and improved representativity for clients selection in federated learning. In International Conference on Machine Learning, pages 3407-3416. PMLR, 2021.
[49] Honglin Yuan and Tengyu Ma. Federated accelerated stochastic gradient descent. Advances in Neural Information Processing Systems, 33:5332-5344, 2020.
[50] Honglin Yuan, Manzil Zaheer, and Sashank Reddi. Federated composite optimization. In International Conference on Machine Learning, pages 12253-12266. PMLR, 2021.
[51] Bohang Zhang, Jikai Jin, Cong Fang, and Liwei Wang. Improved analysis of clipping algorithms for non-convex optimization. Advances in Neural Information Processing Systems, 33:15511–15521, 2020.
[52] Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. In International Conference on Learning Representations, 2020.
[53] Xinwei Zhang, Mingyi Hong, Sairaj Dhople, Wotao Yin, and Yang Liu. Fedpd: A federated learning framework with optimal rates and adaptivity to non-iid data. arXiv preprint arXiv:2005.11418, 2020.
A Deferred Proofs of the Upper Bound
A.1 Notation and Preliminaries
Let  $q_0^i = 0$ ,  $\pmb{y}_{0,k}^{i} = \bar{\pmb{x}}_{0}$ , and define
$$
q _ {r} ^ {i} := \left\{ \begin{array}{l l} r - 1 & i \in \mathcal {S} _ {r - 1} \\ q _ {r - 1} ^ {i} & \text {o t h e r w i s e} \end{array} \right. \tag {7}
$$
$$
\boldsymbol {y} _ {r, k} ^ {i} := \left\{ \begin{array}{l l} \boldsymbol {x} _ {r - 1, k} ^ {i} & i \in \mathcal {S} _ {r - 1} \\ \boldsymbol {y} _ {r - 1, k} ^ {i} & \text {o t h e r w i s e} \end{array} \right. \tag {8}
$$
The intermediate variable  $q_{r}^{i}$  is the most recent round before (not including) round  $r$  in which client  $i$  participated. Similarly,  $\pmb{y}_{r,k}^{i}$  is the local model at step  $k$  during the most recent round before (not including) round  $r$  in which client  $i$  participated. Then we know that  $\pmb{y}_{r,k}^{i} = \pmb{x}_{q_{r}^{i},k}^{i}$ . Therefore  $\pmb{G}_{r}^{i} = \frac{1}{I}\sum_{k=0}^{I-1}\nabla F_{i}(\pmb{y}_{r,k}^{i};\xi_{q_{r}^{i},k}^{i})$ . Also, we will initialize  $\pmb{G}_{0}^{i} = \nabla F_{i}(\bar{\pmb{x}}_{0}^{i};\tilde{\xi}_{i}^{i})$  for all clients.
Denote by  $\mathcal{A}_r = \{\| G_r\| \leq \frac{\gamma}{\eta}\}$ . Our analysis is simplified by taking the following conditions throughout the proof:
$$
(K + 1) \eta I \left(A L _ {0} + B L _ {1} \kappa + 4 B L _ {1} \rho \left(2 \sigma + \frac {\gamma}{\eta}\right)\right) \leq \frac {1}{6 0} \tag {9}
$$
$$
2 (K + 1) I (2 \sigma \eta + \gamma) \leq \frac {C}{L _ {1}},
$$
where  $C > 0$  is a constant,  $A$  and  $B$  are defined in terms of  $C$  (see Lemma 2), and  $K$  is defined in Lemma 8. Usage of these conditions will be explicitly stated when used.
Through out this section, we denote the filtration
$$
\mathcal {F} _ {r} = \sigma \left(\{\tilde {\xi} _ {i} \} _ {i \in [ N ]}, \{\xi_ {q, k} ^ {i}: q \leq r, k \in [ I ] \} _ {i \in [ N ]}\right).
$$
We use  $\mathbb{E}_r[\cdot ] = \mathbb{E}[\cdot \mid \mathcal{F}_r]$  and  $\mathbb{P}_r(\cdot)$  to denote the conditional expectation and probability given the filtration  $\mathcal{F}_r$ . Notice that, given  $\mathcal{F}_r$ , the global weight  $\bar{x}_r$  is fixed but the subsampling set  $\mathcal{S}_r$  is still random and is independent of  $\mathcal{F}_r$ .
A.2 Auxiliary Lemmas
Lemma 2 (Corollary A.4 in [51]). Let  $f: \mathbb{R}^d \to \mathbb{R}$  be  $(L_0, L_1)$ -smooth and  $C > 0$ . For any  $\pmb{x}, \pmb{y} \in \mathbb{R}^d$  with  $\| \pmb{x} - \pmb{y} \| \leq \frac{C}{L_1}$ ,