SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lin Wang $^{1,2}$ , Yongxin Guo $^{1,2}$ , Tao Lin $^{4,5}$ , Xiaoying Tang $^{1,2,3*}$
$^{1}$ School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)
2The Shenzhen Institute of Artificial Intelligence and Robotics for Society
<sup>3</sup>The Guangdong Provincial Key Laboratory of Future Networks of Intelligence
$^{4}$ Research Center for Industries of the Future, Westlake University
$^{5}$ School of Engineering, Westlake University.
Partial client participation has been widely adopted in Federated Learning (FL) to reduce the communication burden efficiently. However, an inadequate client sampling scheme can lead to the selection of unrepresentative subsets, resulting in significant variance in model updates and slowed convergence. Existing sampling methods are either biased or can be further optimized for faster convergence. In this paper, we present DELTA, an unbiased sampling scheme designed to alleviate these issues. DELTA characterizes the effects of client diversity and local variance, and samples representative clients with valuable information for global model updates. In addition, DELTA is a proven optimal unbiased sampling scheme that minimizes variance caused by partial client participation and outperforms other unbiased sampling schemes in terms of convergence. Furthermore, to address full-client gradient dependence, we provide a practical version of DELTA depending on the available clients' information, and also analyze its convergence. Our results are validated through experiments on both synthetic and real-world datasets.
Federated Learning (FL) is a distributed learning paradigm that allows a group of clients to collaborate with a central server to train a model. Edge clients can perform local updates without sharing their data, which helps to protect their privacy. However, communication can be bottleneck in FL, as edge devices often have limited bandwidth and connection availability [58]. To reduce the communication burden, only a subset of clients are typically selected for training. However, an improper client sampling strategy, such as uniform client sampling used in FedAvg [38], can worsen the effects of data heterogeneity in FL. This is because the randomly selected unrepresentative subsets can increase the variance introduced by client sampling and slow down convergence.
Existing sampling strategies can be broadly classified into two categories: biased and unbiased. Unbiased sampling is important because it can preserve the optimization objective. However, only a few unbiased sampling strategies have been proposed in FL, such as multinomial distribution (MD) sampling and cluster sampling. Specifically, cluster sampling can include clustering based on sample size and clustering based on similarity. Unfortunately, these sampling methods often suffer from slow convergence, large variance, and computation overhead issues [2, 13].
To accelerate the convergence of FL with partial client participation, Importance Sampling (IS), an unbiased sampling strategy, has been proposed in recent literature [5, 49]. IS selects clients with a large gradient norm, as shown in Figure 1. Another sampling method shown in Figure 1 is
Figure 1: Client selection illustration of different methods. IS (left) selects high-gradient clients but faces redundant sampling issues. Cluster-based IS (mid) addresses redundancy, but using small gradients for updating continuously can slow down convergence. In contrast, DELTA (right) selects diverse clients with significant gradients without clustering operations.
Figure 2: Comparison of the convergence performance for different sampling methods. In this example, we use a logistic regression model on non-iid MNIST data and sample 10 out of 200 clients. We run 500 communication rounds and report the average of the best 10 accuracies at 100, 300, and 500 rounds. This shows the accuracy performance from the initial training state to convergence.
cluster-based IS, which first clusters clients according to the gradient norm and then uses IS to select clients with a large gradient norm within each cluster.
Though IS and cluster-based IS have their advantages, 1) IS could be inefficient because it can result in the transfer of excessive similar updates from the clients to the server. This problem has been pointed out in recent works [52, 63], and efforts are being made to address it. One approach is to use cluster-based IS, which groups similar clients together. This can help, but 2) cluster-based IS has its drawbacks in terms of convergence speed and clustering effect. Figure 2 illustrates that both of these sampling methods can perform poorly at times. Specifically, compared with cluster-based IS, IS cannot fully utilize the diversity of gradients, leading to redundant sampling and a lack of substantial improvement in accuracy [52, 2]. While the inclusion of clients from small gradient groups in cluster-based IS leads to slow convergence as it approaches convergence, as shown by experimental results in Figure 6 and 7 in Appendix B.2. Furthermore, the clustering algorithm's performance tends to vary when applied to different client sets with varying parameter configurations, such as different numbers of clusters, as observed in prior works [52, 51, 56].
To address the limitations of IS and cluster-based IS, namely excessive similar updates and poor convergence performance, we propose a novel sampling method for Federated Learning termed DivErse cLienT sAmpling (DELTA). Compared to IS and cluster-based IS methods, DELTA tends to select clients with diverse gradients, as shown in Figure 1. This allows DELTA to utilize the advantages of a large gradient norm for convergence acceleration while also overcoming the issue of gradient similarity.
Additionally, we propose practical algorithms for DELTA and IS that rely on accessible information from partial clients, addressing the limitations of existing analysis based on full client gradients [35, 5]. We also provide convergence rates for these algorithms. We replace uniform client sampling with DELTA in FedAvg, referred to as FedDELTA, and replace uniform client sampling with IS in FedAvg, referred to as FedIS. Their practical versions are denoted as FedPracDELTA and FedPracIS.
Toy Example and Motivation. We present a toy example to illustrate our motivation, where each client has a regression model. The detailed settings of each model and the calculation of each sampling algorithm's gradient are provided in Appendix B.1. Figure 3 shows that IS deviates from the ideal global model when aggregating gradients from clients with large norms. This motivates us to consider the correlation between local and global gradients in addition to gradient norms when sampling clients. Compared to IS, DELTA selects clients with large gradient diversities, which exploits the clients' information of both gradient norms and directions, resulting in a closer alignment to the ideal global model.
Our contributions. In this paper, we propose an efficient unbiased sampling scheme in the sense that (i) It effectively addresses the issue of excessive similar gradients without the need for additional clustering, while taking advantage of the accelerated convergence of gradient-norm-based IS and (ii) it is provable better than uniform sampling or gradient norm-based sampling. The sampling scheme is versatile and can be easily integrated with other optimization techniques, such as momentum, to improve convergence further.
As our key contributions,
Figure 3: Model update comparison: The closer to the ideal global update (black arrow), the better the sampling algorithm is. The small window shows the projection of 3 clients' functions  $F_{1}, F_{2}, F_{3}$  in the X-Y plane, where  $\nabla F_{1} = (2,2)$ ,  $\nabla F_{2} = (4,1)$ ,  $\nabla F_{3} = (6, -3)$  at (1, 1). The enlarged image shows the aggregated gradients of FedAvg, IS, DELTA and ideal global gradient. Each algorithm samples two out of three clients: FedIS tends to select Client 2 and 3 with largest gradient norms, DELTA tends to select Client 1 and 3 with the largest gradient diversity and FedAvg is more likely to select Client 1 and 2 compared to IS and DELTA. The complete gradient illustration with clients' gradient is shown in Figure 5 in Appendix.
- We present DELTA, an unbiased FL sampling scheme based on gradient diversity and local variance. Our refined analysis shows that FedDELTA surpasses the state-of-the-art FedAvg in convergence rate by eliminating the  $\mathcal{O}(1 / T^{2 / 3})$  term and a  $\sigma_G^2$ -related term of  $\mathcal{O}(1 / T^{1 / 2})$ .
- We present a novel theoretical analysis of nonconvex FedIS, which yields a superior convergence rate compared to existing works while relying on a more lenient assumption. Moreover, our analysis eliminates the  $\mathcal{O}(1 / T^{2 / 3})$  term of the convergence rate, in contrast to FedAvg.
- We present a practical algorithm for DELTA in partial participation settings, utilizing available information to mitigate the reliance on full gradients. We prove that the convergence rates of these practical algorithms can attain the same order as the theoretical optimal sampling probabilities for DELTA and IS.
Client sampling in federated learning (FL) can be categorized into unbiased and biased methods [14]. Unbiased methods, including multinomial sampling and importance sampling [30, 5, 49], ensure that the expected client aggregation is equivalent to the deterministic global aggregation when all clients participate. Unlike unbiased sampling, which has received comparatively little attention, biased sampling has been extensively examined in the context of federated learning, such as selecting clients with higher loss [7] or larger updates [48]. Recently, cluster-based client selection, which involves grouping clients into clusters and sampling from these clusters, has been proposed to sample diverse clients and reduce variance [41, 12, 52]. Nevertheless, the clustering will require extra communication and computational resources. The proposed DELTA algorithm can be seen as a muted version of a diverse client clustering algorithm without clustering operation.
While recent works [57, 28] have achieved comparable convergence rates to ours using variance reduction techniques, it is worth noting that these techniques are orthogonal to ours and can be easily integrated with our approach. Although [60] achieved the same convergence rate as ours, but their method requires dependent sampling and mixing participation conditions, which can lead to security problems and exceed the communication capacity of the server. In contrast, our method avoids these issues by not relying on such conditions.
A more comprehensive discussion of the related work can be found in Appendix A.
3 Theoretical Analysis and An Improved FL Sampling Strategy
This section presents FL preliminaries and analyzes sampling algorithms, including the convergence rate of nonconvex FedIS in Section 3.2, improved convergence analysis for FL sampling in Section 3.3, and proposal and convergence rate of the DELTA sampling algorithm in Section 3.4.
In FL, the objective of the global model is a sum-structured optimization problem:
$$
f ^ {*} = \min  _ {x \in \mathbb {R} ^ {d}} [ f (x) := \sum_ {i = 1} ^ {m} w _ {i} F _ {i} (x) ], \tag {1}
$$
where  $F_{i}(x) = \mathbb{E}_{\xi_{i} \sim D_{i}}[F_{i}(x, \xi_{i})]$  represents the local objective function of client  $i$  over data distribution  $D_{i}$ , and  $\xi_{i}$  means the sampled data of client  $i$ .  $m$  is the total number of clients and  $w_{i}$  represents the weight of client  $i$ . With partial client participation, FedAvg randomly selects  $|S_{t}| = n$  clients  $(n \leq m)$  to communicate and update model. Then the loss function of actual participating users in each round can be expressed as:
$$
f _ {S _ {t}} \left(x _ {t}\right) = \frac {1}{n} \sum_ {i \in S _ {t}} F _ {i} \left(x _ {t}\right). \tag {2}
$$
Table 1: Comparison of convergence rate for different sampling algorithms: Number of communication rounds required to reach  $\epsilon$  or  $\epsilon +\varphi$  ( $\epsilon$  for unbiased sampling and  $\epsilon +\varphi$  for biased sampling, where  $\varphi$  is a nonconvergent constant term) accuracy for FL.  $\sigma_L$  is local variance bound, and  $G$  bound is  $E\| \nabla F_i(x) - \nabla f(x)\|^2 \leq G^2$ .  $\Gamma$  is the distance of global optimum and the average of local optimum (Heterogeneity bound),  $\mu$  corresponds to  $\mu$  strongly convex. and  $\zeta_G$  is the gradient diversity.
Algorithm	Convexity	Partial Worker	Unbiasedness	Convergence rate	Assumption
SGD	S/N	✓	✓	σL2μmKε+(1/μ)/σL2mKε2+1/ε	σL bound
FedDELTA	N	✓	✓	σL2nKε2+M2/Kε	Assumption 3
FedPracDELTA	N	✓	✓	U2σ2nKε+U2M2/Kε	Assumption 3 and Assumption 4
FedIS (ours)	N	✓	✓	σL2+Kσ2nKε2+M2/Kε	Assumption 3
FedIS (others) [5]	N	✓	✓	M2nKε2+A2+1/ε+σG/ε3/2	Assumption 3 and ρ Assumption
FedIS (others) [36]	S	✓	✓	σL2+4nK2G2+6nΓ/μ2nKε+K2G2/ε+||w0-w*||2/μKε	G bound
FedPracIS (ours)	N	✓	✓	σL2+KU2σ2nKε2+M2/Kε	Assumption 3 and Assumption 4
FedAvg [65]	N	✓	✓	σL2nKε2+4Kσ2nKε2+M2/Kε+K1/3M2/ε	G bound
FedAvg [21]	N	✓	✓	M2nKε2+A2+1/ε+σG/ε3/2	Assumption 3
DivFL [2]	S	✓	×	1/ε+1/ε	Heterogeneity Gap
Power-of-Choice [7]	S	✓	×	σL2+G2/ε+Γ/ε	Heterogeneity Gap
FedAvg [65]	N	×	✓	σL2mKε2+σL/(4K)+σG/ε	σG bound
Arbitrary Sampling[60]	N	Mix	✓	ζG+(1+σL)nρ/nKε2+M2/Kε	Assumption 3
$M^2 = \sigma_L^2 + 4K\sigma_G^2, \hat{M}^2 = \sigma_L^2 + K(1 - n / m)\sigma_G^2, \hat{M}^2 = \sigma_L^2 + 6K\sigma_G^2, \hat{M}^2 = \sigma_L^2 + 4K\zeta_G^2, \hat{M}^2 = K\zeta_G^2 + K\sigma_L^2.$
Convexity: S and N are abbreviations for strong convex and nonconvex, respectively.  $\rho$  assumption: Bound of the similarity among local gradients.
Mix participation: the number of participating clients is random, from none to full participation.
For ease of theoretical analysis, we make the following commonly used assumptions:
Assumption 1 (L-Smooth). There exists a constant  $L > 0$ , such that  $\| \nabla F_i(x) - \nabla F_i(y) \| \leq L \| x - y \|, \forall x, y \in \mathbb{R}^d$ , and  $i = 1, 2, \ldots, m$ .
Assumption 2 (Unbiased Local Gradient Estimator and Local Variance). Let  $\xi_t^i$  be a random local data sample in the round  $t$  at client  $i$ :  $\mathbb{E}\left[\nabla F_i(x_t, \xi_t^i)\right] = \nabla F_i(x_t), \forall i \in [m]$ . The function  $F_i(x_t, \xi_t^i)$  has a bounded local variance of  $\sigma_{L,i} > 0$ , satisfying  $\mathbb{E}\left[\left\| \nabla F_i(x_t, \xi_t^i) - \nabla F_i(x_t) \right\|^2\right] = \sigma_{L,i}^2 \leq \sigma_L^2$ .
Assumption 3 (Bound Dissimilarity). There exists constants  $\sigma_G \geq 0$  and  $A \geq 0$  such that  $\mathbb{E}\|\nabla F_i(x)\|^2 \leq (A^2 + 1)\|\nabla f(x)\|^2 + \sigma_G^2$ . When all local loss functions are identical,  $A^2 = 0$  and  $\sigma_G^2 = 0$ .
The above assumptions are commonly used in both non-convex optimization and FL literature, see e.g. [21, 27, 60].
We notice that Assumption 3 can be further relaxed by Assumption 2 of [24]. We also provide Proposition C.4 in Appendix C to show all our convergence analysis, including Theorem 3.1,3.5 and Corollary 4.1,4.2 can be easily extended to the relaxed assumption while keeping the order of convergence rate unchanged.
3.2 Convergence Analysis of FedIS
As discussed in the introduction, IS faces an excessive gradient similarity problem, necessitating the development of a novel diversity sampling method. Prior to delving into the specifics of our new sampling strategy, we first present the convergence rate of FL under standard IS analysis in this section; this analysis itself is not well explored, particularly in the nonconvex setting. The complete FedIS algorithm is provided in Algorithm 2 of Appendix D, which differs from DELTA only in sampling probability (line 2) by using  $p_i \propto \left\| \sum_{k=0}^{K-1} g_{t,k}^i \right\|$ .
Theorem 3.1 (Convergence rate of FedIS). Let constant local and global learning rates  $\eta_L$  and  $\eta$  be chosen as such that  $\eta_L < \min(1/(8LK), C)$ , where  $C$  is obtained from the condition that  $\frac{1}{2} - 10L^2 K^2 (A^2 + 1) \eta_L^2 - \frac{L^2 \eta K (A^2 + 1)}{2n} \eta_L > 0$ , and  $\eta \leq 1 / (\eta_L L)$ . In particular, suppose  $\eta_L = \mathcal{O}\left(\frac{1}{\sqrt{T}KL}\right)$  and  $\eta = \mathcal{O}\left(\sqrt{Kn}\right)$ , under Assumptions 1-3, the expected gradient norm of FedIS algorithm 2 will be bounded as follows:
$$
\min  _ {t \in [ T ]} \mathbb {E} \| \nabla f (x _ {t}) \| ^ {2} \leq \mathcal {O} \left(\frac {f ^ {0} - f ^ {*}}{\sqrt {n K T}}\right) + \underbrace {\mathcal {O} \left(\frac {\sigma_ {L} ^ {2} + K \sigma_ {G} ^ {2}}{\sqrt {n K T}}\right) + \mathcal {O} \left(\frac {M ^ {2}}{T}\right)} _ {\text {o r d e r o f} \Phi}. \tag {3}
$$
where  $T$  is the total communication round,  $K$  is the total local epoch times,  $f^0 = f(x_0)$ ,  $f^* = f(x_*)$ ,  $M = \sigma_L^2 + 4K\sigma_G^2$  and the expectation is over the local dataset samples among clients.
The FedIS sampling probability  $p_i^t$  is determined by minimizing the variance of convergence with respect to  $p_i^t$ . The variance term  $\Phi$  is:
$$
\Phi = \frac {5 \eta_ {L} ^ {2} K L ^ {2}}{2} M ^ {2} + \frac {\eta \eta_ {L} L}{2 m} \sigma_ {L} ^ {2} + \frac {L \eta \eta_ {L}}{2 n K} \operatorname {V a r} \left(\frac {1}{m p _ {i} ^ {t}} \hat {g} _ {i} ^ {t}\right), \tag {4}
$$
where  $\mathrm{Var}\left(\frac{1}{(mp_i^t)\hat{g}_i^t}\right)$  is called update variance. By optimizing the update variance, we get the sampling probability FedIS:
$$
p _ {i} ^ {t} = \frac {\left\| \hat {g} _ {i} ^ {t} \right\|}{\sum_ {j = 1} ^ {m} \left\| \hat {g} _ {j} ^ {t} \right\|}, \tag {5}
$$
where  $\hat{g}_i^t = \sum_{k=0}^{K-1} \nabla F_i(x_{k,t}^i, \xi_{k,t}^i)$  is the sum of the gradient updates of multiple local updates. The proof details of Theorem 3.1 and derivation of sampling probability FedIS are detailed in Appendix D and Appendix F.1.
Remark 3.2 (Explanation for the convergence rate). It is worth mentioning that although a few works provide the convergence upper bound of  $FL$  with gradient-based sampling, several limitations exist in these analyses and results:
1) [49, 35] analyzed FL with IS using a strongly convex condition, whereas we extended the analysis
2) Our analysis results, compared to the very recent non-convex analysis of FedIS [5] and FedAvg, remove the term  $\mathcal{O}(T^{-\frac{2}{3}})$ , although all these works choose a learning rate of  $\mathcal{O}(T^{-\frac{1}{2}})$ . Thus, our result achieves a tighter convergence rate when we use  $\mathcal{O}(1 / T + 1 / T^{2 / 3})$  (provided by [43]) as our lower bound of convergence (see Table 1).
The comparison results in Table 1 reveal that even when  $\sigma_G$  is large and becomes a dependency term for convergence rate, FedIS (ours) is still better than FedAvg and FedIS (others) since our result reduces the coefficient of  $\sigma_G$  in the dominant term  $\mathcal{O}(T^{-\frac{1}{2}})$ .
Remark 3.3 (Novelty of our FedIS analysis). Despite the existence of existing convergence analysis of partial participant FL [65, 47], including FedIS that builds on this analysis [35, 4], none of them take full advantage of the nature of unbiased sampling, and thus yield an imprecise upper bound on convergence. To tighten the FedIS upper bound, we first derive a tighter convergence upper bound for unbiased sampling FL. By adopting uniform sampling for unbiased probability, we achieve a tighter FedAvg convergence rate. Leveraging this derived bound, we optimize convergence variance using IS.
Compared with existing unbiased sampling FL works, including FedAvg and FedIS (others), our analysis on FedIS entails: (1) A tighter Local Update Bound Lemma: We establish Lemma C.3 using Assumption 3, diverging from the stronger assumption  $\| \nabla F_i(x_t)) - \nabla f(x_t)\| ^2\leq \sigma_G^2$  (used in [65, 47]), and the derived Lemma C.3 achieves a tighter upper bound than other works (Lemma 4 in [47], Lemma 2 in [65]). (2) A tighter upper bound on aggregated model updates  $E\| \Delta_{t}\|^{2}$ : By fully utilizing the nature of unbiased sampling, we convert the bound analysis of  $A_{2} = E\| \Delta_{t}\|^{2}$  equally to a bound analysis of participant variance  $V\left(\frac{1}{mp_i^t}\hat{g}_i^t\right)$  and aggregated model update with full user participation. In contrast, instead of exploring the property the unbiased sampling, [47] repeats to use Lemma 4 and [65] uses Lemma 2 for bound  $A_{2}$ . This inequality transform imposes a loose upper bound for  $A_{2}$ , resulting in a convergence variance term determined by  $\eta_L^3$ , which reacts to the rate order being  $\mathcal{O}(T^{-\frac{2}{3}})$ . (3) Relying on a more lenient assumption: Beyond the aforementioned analytical improvement, our IS analysis obviates the necessity for unusual assumptions in other FedIS analysis such as Mix Participation [35] and  $\rho$ -Assumption [4].
Remark 3.4 (Extending FedIS to practical algorithm). The existing analysis of IS algorithms [35, 5] relies on information from full clients, which is not available in partial participation FL. We propose a practical algorithm for FedIS that only uses information from available clients and provide its convergence rate in Corollary 4.1 in Section 4.
Algorithm 1 FedDELTA and FedPracDELTA : Federated learning with unbiased diverse sampling
Require: initial weights  $x_0$ , global learning rate  $\eta$ , local learning rate  $\eta_l$ , number of local epochs  $K$ , number of training rounds  $T$
Ensure: trained weights  $x_{T}$
1: for round  $t = 1,\dots ,T$  do  
2: Sampling clients using DELTA (13)  
3: Sampling clients using Practical DELTA (16)  
4: for each worker  $i\in S_t$  in parallel do  
5:  $\begin{array}{l}x_{t,0}^{i} = x_{t}\\ \text{for} k = 0,\dots ,K - 1\text{do}\\ \text{compute} g_{t,k}^{i} = \nabla F_{i}(x_{t,k}^{i},\xi_{t,k}^{i})\\ \text{Local update}:x_{t,k + 1}^{i} = x_{t,k}^{i} - \eta_{L}g_{t,k}^{i}\\ \text{Let}\Delta_{t}^{i} = x_{t,K}^{i} - x_{t,0}^{i} = -\eta_{L}\sum_{k = 0}^{K - 1}g_{t,k}^{i}\\ \end{array}$   
5:  
6:  
7:  
8:  
9:  
10: At Server:  
11:Receive  $\Delta_t^i,i\in S_t$   
12: let  $\Delta_t = \frac{1}{|S_t|}\sum_{i\in S_t}\frac{n_i}{np_i^t}\Delta_t^i$   
13: Server update:  $x_{t + 1} = x_{t} + \eta \Delta_{t}$   
14: Broadcast  $x_{t + 1}$  to clients
Despite its success in reducing the variance term in the convergence rate, FedIS is far from optimal due to issues with high gradient similarity and the potential for further minimizing the variance term (i.e., the global variance  $\sigma_G$  and local variance  $\sigma_L$  in  $\Phi$ ). In the next section, we will discuss how to address this challenging variance term.
3.3 An Improved Convergence Analysis for FedDELTA
FedIS and FedDELTA have different approaches to analyzing objectives, with FedIS analyzing the global objective and FedDELTA analyzing a surrogate objective  $\tilde{f}(x)$  (cf. (7)). This leads to different convergence variance and sampling probabilities between the two methods. A flowchart (Figure 8 in Appendix E) has been included to illustrate the differences between FedIS and FedDELTA.
The limitations of FedIS. As shown in Figure 1, IS may have excessive similar gradient selection. The variance  $\Phi$  in (4) reveals that the standard IS strategy can only control the update variance  $\mathrm{Var}(^{1} / (mp_{i}^{t})\hat{g}_{i}^{t}$ , leaving other terms in  $\Phi$ , namely  $\sigma_{L}$  and  $\sigma_{G}$ , untouched. Therefore, the standard IS is ineffective at addressing the excessive similar gradient selection problem, motivating the need for a new sampling strategy to address the issue of  $\sigma_{L}$  and  $\sigma_{G}$ .
The decomposition of the global objective. As inspired by the proof of Theorem 3.1 as well as the corresponding Lemma C.1 (stated in Appendix) proposed for unbiased sampling, the gradient of global objective can be decomposed into the gradient of surrogate objective  $\tilde{f} (x_{t})$  and update gap,
$$
\mathbb {E} \| \nabla f (x _ {t}) \| ^ {2} = \mathbb {E} \left\| \nabla \tilde {f} _ {S _ {t}} (x _ {t}) \right\| ^ {2} + \chi_ {t} ^ {2}, \tag {6}
$$
where  $\chi_t = \mathbb{E}\left\| \nabla \tilde{f}_{S_t}(x_t) - \nabla f(x_t)\right\|$  is the update gap.