SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where in line (a) we used the fact the success probability for a batch is selected uniformly at random from  $\Theta$ . In line (b) we used the fact that  $L_{t}^{p}(\hat{q}_{t}) - L_{t}^{p}(1 / 2 - \delta)\geq 0$  since  $\hat{q}_t\in \mathcal{D} = [1 / 2 - \delta ,1 / 2 + \delta ]$ . Similarly term involving  $L_{t}^{q}$  is also handled. In line (c) we applied  $(E_{\mathbb{P}}[L_t^p (\hat{q}_t)|\hat{q}_t\geq 1 / 2] - L_t^p (1 / 2-$ $\delta))\geq \delta$  since  $E_{\mathbb{P}}[L_t^p (\hat{q}_t)|\hat{q}_t\geq 1 / 2]\geq 1 / 2$  and  $L_{t}^{p}(1 / 2 - \delta) = 1 / 2 - \delta$ . Similar bounding is done for the term involving  $E_{\mathbb{Q}}$  as well. In line (d) we used Eq.(6).
Algorithm 5 LPA: a black-box reduction to produce a low-switching online regression algorithm
input Online regression oracle ALG, failure probability  $\delta$  maximum standard deviation  $\sigma$  (see Definition 1). 1: Initialize prev  $= 0\in \mathbb{R}^{K}$ $b = 1$  2: Get estimate  $\tilde{\theta}_t$  from ALG  $(z_{1:t - 1})$  3: Output  $\hat{\theta}_t =$  prev 4: Receive an observation  $z_{t}$  // test to detect non-staionarity 5: if  $\sum_{j = b + 1}^{t}\| \mathrm{prev} - \tilde{\theta}_{j}\|_{2}^{2} > 5K\sigma^{2}\log (2T / \delta)$  then 6: Set  $b = t + 1$  , prev  $= z_{t}$  7: Restart ALG 8: else if  $t - b + 1$  is a power of 2 then 9: Set prev  $= \sum_{j = b}^{t}z_{j} / t - b + 1$  10: end if 11: Update ALG with  $z_{t}$
Thus we get the total expected regret within batch 1 as
$$
\sum_ {t = 1} ^ {\Delta} E [ L _ {t} (\hat {q} _ {t}) ] - L _ {t} (q _ {t}) \geq \frac {\delta \Delta}{8} \exp (- \beta)
$$
The total regret within any batch  $i \in [M]$  can be lower bounded using exactly the same arguments as above. Hence summing the total regret across all batches yields
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} E \left[ L _ {t} \left(\hat {q} _ {t}\right) \right] - L _ {t} \left(q _ {t}\right) \geq \frac {T}{\Delta} \cdot \frac {\delta \Delta}{8} \exp (- \beta) \\ = \frac {V _ {T} \Delta}{1 6} \cdot \exp (- \beta) \\ = T ^ {2 / 3} V _ {T} ^ {1 / 3} \exp (- \beta) / 1 6. \\ \end{array}
$$
The  $\Omega (\sqrt{T})$  part of the lowerbound follows directly from Theorem 3.2.1 in Hazan [37] by choosing  $\mathcal{D}$  with diameter bounded by  $\Omega (1)$ .
D Design of low switching online regression algorithms
Even though Algorithm 4 has attractive performance guarantees, it requires retraining with weighted ERM at every round. This is not satisfactory since the retraining can be computationally expensive. In this section, we aim to design a version of Algorithm 4 with few retraining steps while not sacrificing the statistical efficiency (up to constants). To better understand why this goal is attainable, consider a time window  $[1,n]\subseteq [T]$  where the true label margins remain constant or drift very slowly. Due to the slow drift, one reasonable strategy is to re-train the model (with weighted ERM) using the past data only at time points within  $[1,n]$  that are powers of 2 (i.e via a doubling epoch schedule). For rounds  $t\in [1,n]$  that are not powers of 2, we make predictions with a previous model  $h_{\mathrm{prev}}$  computed at  $t_{\mathrm{prev}}\coloneqq 2^{\lfloor \log_2t\rfloor}$  which is trained using data seen upto the time  $t_{\mathrm{prev}}$ . Observe that this constitutes at least half of the data seen until round  $t$ . This observation when combined with the slow drift of label margins implies that the performance of the model  $h_{\mathrm{prev}}$  at round  $t$  will be comparable to the performance of a model obtained by retraining using entire data collected until round  $t$ .
To formalize this idea, we need an efficient online change-point-detection strategy that can detect intervals where the TV of the true label marginals is low and retrain only (modulo at most  $\log T$  times within a low TV window) when there is enough evidence for sufficient change in the TV of the true marginals. We address this problem via a two-step approach. In the first step, we construct a generic black-box reduction that takes an online regression oracle as input and converts it into
another algorithm with the property that the number of switches in its predictions is controlled without sacrificing the statistical performance. Recall that the purpose of the online regression oracles is to track the true label margins. The output of our low-switching online algorithm remains the same as long as the TV of the true label margins (TV computed from the time point of the last switch) is sufficiently small. Then we use this low-switching online regression algorithm to re-train the classifier when a switch is detected.
We next provide the Low switching through Phased Averaging (LPA) (Algorithm 5), our black-box reduction to produce low switching regression oracles. We remark that this algorithm is applicable to the much broader context of online regression or change point detection and can be of independent interest.
We now describe the intuition behind Algorithm 5. The purpose of Algorithm 5 is to denoise the observations  $z_{t}$  and track the underlying ground truth  $\theta_{t}$  in a statistically efficient manner while incurring low switching cost. Hence it is applicable to the broader context of online non-parametric regression [3, 56, 7] and offline non-parametric regression [66, 72].
Algorithm 5 operates by adaptively detecting low TV intervals. Within each time window it performs a phased averaging in a doubling epoch schedule. i.e consider a low TV window  $[b, n]$ . For a round  $t \in [b, n]$  let  $t_{\mathrm{prev}} := 2^{\lfloor \log_2(t - b + 1) \rfloor}$ . In round  $t$ , the algorithm plays the average of the observations  $z_b : t_{\mathrm{prev}}$ . So we see that in any low TV window, the algorithm changes its output only at-most  $O(\log T)$  times.
For the above scheme to not sacrifice statistical efficiency, it is important to efficiently detect windows with low TV of the true label marginals. Observe that the quantity prev computes the average of at-least half of the observations within a time window that start at time  $b$ . So when the TV of the ground truth within a time window  $[b, t]$  is small, we can expect the average to be a good enough representation of the entire ground truth sequence within that time window. Consider the quantity  $R_{t} := \sum_{j = b + 1}^{t} \| \text{prev} - \theta_{j} \|_{2}^{2}$  which is the total squared error (TSE) incurred by the fixed decision prev within the current time window. Whenever the TV of the ground truth sequence  $\theta_{b:t}$  is large, there will be a large bias introduced by prev due to averaging. Hence in such a scenario the TSE will also be large indicating non-stationarity. However, we can't compute  $R_{t}$  due to the unavailability of  $\theta_{j}$ . So we approximate  $R_{t}$  by replacing  $\theta_{j}$  with the estimates  $\bar{\theta}_{j}$  coming from the input online regression algorithm that is not constrained by switching cost restrictions. This is the rationale behind the non-stationarity detection test at Step 5. Whenever a non-staionarity is detected we restart the input online regression algorithm as well as the start position for computing averages (in Step 6).
We have the following guarantee for Algorithm 5.
Theorem 18. Suppose the input black box ALG given to Algorithm 5 is adaptively minimax optimal (see Definition 1). Then the number of times Algorithm 5 switches its decision is at most  $\tilde{O}(T^{1/3}V_T^{2/3})$  with probability at least  $1 - \delta$ . Further, Algorithm 5 satisfies  $\sum_{t=1}^{T}\|\hat{\theta}_t - \theta_t\|_2^2 = \tilde{O}(T^{1/3}V_T^{2/3})$  with probability at least  $1 - \delta$ , where  $V_T = \sum_{t=2}^{T}\|\theta_t - \theta_{t-1}\|_1$ .
Remark 19. Since Algorithm 5 is a black-box reduction, there are a number of possible candidates for the input policy ALG that are adaptively minimax. Examples include FLH with online averages as base learners [38] or Aligator algorithm [7].
Armed with a low switching online regression oracle LPA, one can now tweak Algorithm 4 to have sparse number of retraining steps while not sacrificing the statistical efficiency (up to multiplicative constants). The resulting procedure is described in Algorithm 6 (in App. E) which enjoys similar rates as in Theorem 6 (see Theorem 22).
E Omitted proofs from Section 4
First we recall a result from Baby et al. [7].
Proposition 20 (Theorem 5 of Baby et al. [7]). Consider the online regression protocol defined in Definition 1. Let  $\hat{\theta}_t$  be the estimate of the ground truth produced by the Aligator algorithm from Baby et al. [7]. Then with probability at-least  $1 - \delta$ , the total squared error (TSE) of Aligator satisfies
$$
\sum_ {t = 1} ^ {T} \left\| \theta_ {t} - \hat {\theta} _ {t} \right\| _ {2} ^ {2} = \tilde {O} \left(T ^ {1 / 3} V _ {T} ^ {2 / 3} + 1\right),
$$
where  $V_{T} = \sum_{t=2}^{T} \| \theta_{t} - \theta_{t-1} \|_{1}$ . This bound is attained without any prior knowledge of the variation  $V_{T}$ .
The high probability guarantee also implies that
$$
\sum_ {t = 1} ^ {T} E \left[ \| \theta_ {t} - \hat {\theta} _ {t} \| _ {2} ^ {2} \right] = \tilde {O} \left(T ^ {1 / 3} V _ {T} ^ {2 / 3} + 1\right),
$$
where the expectation is taken with respect to randomness in the observations.
Next, we verify that the noise condition in Definition 1 is satisfied for the empirical label marginals computed at Step 5 of Algorithm 4.
Lemma 21. Let  $s_t$  be as in Step 5 of Algorithm 4. Then it holds that  $s_t = q_t + \epsilon_t$  with  $\epsilon_t$  being independent across  $t$  and  $\mathrm{Var}(\epsilon_t) \leq 1 / N$ .
Proof. Since  $s_t$  is simply the empirical label proportions, it holds that  $E[s_t] = q_t$ . Further  $\operatorname{Var}(s_t) \leq 1$  as the indicator function is bounded by  $1/N$ . This concludes the proof.
Theorem 6. Suppose the true label marginal satisfies  $\min_{t,k} q_t(k) \geq \mu > 0$ . Choose the online regression oracle in Algorithm 4 as FLH-FTL (App. F) or Algator from Baby et al. [7] with its predictions clipped such that  $\hat{q}_t[k] \geq \mu$ . Then with probability at least  $1 - \delta$ , Algorithm 4 produces hypotheses with  $R_{dynamic}^{\mathcal{H}} = \tilde{O}\left(T^{2/3} V_T^{1/3} + \sqrt{T \log(|\mathcal{H}| / \delta)}\right)$ , where  $V_T = \sum_{t=2}^{T} \| q_t - q_{t-1} \|_1$ . Further, this result is attained without any prior knowledge of the variation budget  $V_T$ .
Proof. In the proof we first proceed to bound the instantaneous regret at round  $t$ . Re-write the population loss as:
$$
L _ {t} (h) = \frac {1}{N (t - 1)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} E \left[ \frac {q _ {t} (y _ {i j})}{q _ {i} (y _ {i j})} \ell (h (x _ {i j}), y _ {i j}) \right],
$$
where the expectation is taken with respect to randomness in the samples.
We define the following quantities:
$$
L _ {t} ^ {\operatorname {e m p}} (h) := \frac {1}{N (t - 1)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \frac {q _ {t} \left(y _ {i j}\right)}{q _ {i} \left(y _ {i j}\right)} \ell \left(h \left(x _ {i j}\right), y _ {i j}\right), \tag {7}
$$
$$
\tilde {L} _ {t} (h) := \frac {1}{N (t - 1)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} E \left[ \frac {\hat {q} _ {t} \left(y _ {i j}\right)}{\hat {q} _ {i} \left(y _ {i j}\right)} \ell \left(h \left(x _ {i j}\right), y _ {i j}\right) \right], \tag {8}
$$
$$
\tilde {L} _ {t} ^ {\mathrm {e m p}} (h) := \frac {1}{N (t - 1)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \frac {\hat {q} _ {t} (y _ {i j})}{\hat {q} _ {i} (y _ {i j})} \ell (h (x _ {i j}), y _ {i j}).
$$
We decompose the regret at round  $t$  as
$$
\begin{array}{l} L _ {t} (h _ {t}) - L _ {t} (h _ {t} ^ {*}) = L _ {t} (h _ {t}) - \tilde {L} _ {t} (h _ {t}) + \tilde {L} _ {t} (h _ {t}) - \tilde {L} _ {t} ^ {\mathrm {e m p}} (h _ {t}) + L _ {t} ^ {\mathrm {e m p}} (h _ {t} ^ {*}) - L _ {t} (h _ {t} ^ {*}) + \tilde {L} _ {t} ^ {\mathrm {e m p}} (h _ {t}) - L _ {t} ^ {\mathrm {e m p}} (h _ {t} ^ {*}) \\ \leq \underbrace {L _ {t} (h _ {t}) - \tilde {L} _ {t} (h _ {t})} _ {\mathrm {T 1}} + \underbrace {\tilde {L} _ {t} (h _ {t}) - \tilde {L} _ {t} ^ {\mathrm {e m p}} (h _ {t})} _ {\mathrm {T 2}} + \underbrace {L _ {t} ^ {\mathrm {e m p}} (h _ {t} ^ {*}) - L _ {t} (h _ {t} ^ {*})} _ {\mathrm {T 3}} + \underbrace {\tilde {L} _ {t} ^ {\mathrm {e m p}} (h _ {t} ^ {*}) - L _ {t} ^ {\mathrm {e m p}} (h _ {t} ^ {*})} _ {\mathrm {T 4}}, \\ \end{array}
$$
where in the last line we used Eq.(2). Now we proceed to bound each terms as note above.
Note that for any label  $m$
$$
\begin{array}{l} \left| \frac {q _ {t} (m)}{q _ {i} (m)} - \frac {\hat {q} _ {t} (m)}{\hat {q} _ {i} (m)} \right| \leq \left| \frac {q _ {t} (m)}{q _ {i} (m)} - \frac {q _ {t} (m)}{\hat {q} _ {i} (m)} \right| + \left| \frac {q _ {t} (m)}{\hat {q} _ {i} (m)} - \frac {\hat {q} _ {t} (m)}{\hat {q} _ {i} (m)} \right| \\ \leq \frac {1}{\mu^ {2}} \left(| q _ {i} (m) - \hat {q} _ {i} (m) | + | q _ {t} (m) - \hat {q} _ {t} (m) |\right), \tag {9} \\ \end{array}
$$
where in the last line, we used the assumption that the minimum label marginals (and hence of the online estimates via clipping) is bounded from below by  $\mu$ . So by applying triangle inequality and using the fact that the losses are bounded by  $B$  in magnitude, we get
$$
\begin{array}{l} T 1 \leq \frac {B}{N (t - 1) \mu^ {2}} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} E \left[ \| \hat {q} _ {i} - q _ {i} \| _ {1} + \| \hat {q} _ {t} - q _ {t} \| _ {1} \right] \\ \leq \frac {B \sqrt {K}}{(t - 1) \mu^ {2}} \sum_ {i = 1} ^ {t - 1} E \left[ \| \hat {q} _ {i} - q _ {i} \| _ {2} + \| \hat {q} _ {t} - q _ {t} \| _ {2} \right] \\ \le_ {(a)} \frac {B \sqrt {K}}{\mu^ {2}} \left(E [ \| \hat {q} _ {t} - q _ {t} \| _ {2} ] + \sqrt {\frac {\sum_ {i = 1} ^ {t - 1} E [ \| q _ {i} - \hat {q} _ {i} \| _ {2} ^ {2} ]}{t - 1}}\right) \\ \leq_ {(b)} \frac {B \sqrt {K}}{\mu^ {2}} \left(E [ \| \hat {q} _ {t} - q _ {t} \| _ {2} ] + \phi \cdot \frac {V _ {T} ^ {1 / 3}}{(t - 1) ^ {1 / 3}}\right), \tag {10} \\ \end{array}
$$
where line (a) is a consequence of Jensen's inequality. In line (b) we used the following fact: by Lemma 21 and Proposition 16, the expected cumulative error of the online oracle at any step is bounded by  $\phi t^{1/3} V_t^{2/3}$  for some multiplier  $\phi$  which can contain poly-logarithmic factors of the horizon (see Proposition 20).
Proceeding in a similar fashion, the term  $T4$  can be bounded by Eq.(10).
Next, we proceed to handle T3. Let  $h \in \mathcal{H}$  be any fixed hypothesis. Then each summand in Eq.(7) is an independent random variable assuming values in  $[0, B / \mu]$  (recall that the losses lie within  $[0, B]$ ). Hence by Hoeffding's inequality we have that
$$
\begin{array}{l} L _ {t} ^ {\mathrm {e m p}} (h) - L _ {t} (h) \leq \frac {B}{\mu} \sqrt {\frac {\log (3 T | \mathcal {H} | / \delta)}{N (t - 1)}}, \\ \leq \frac {B}{\mu} \sqrt {\frac {\log (3 T | \mathcal {H} | / \delta)}{(t - 1)}}, \tag {11} \\ \end{array}
$$
with probability at-least  $1 - \delta /(3T|\mathcal{H}|)$ . Now taking union bound across all hypotheses in  $\mathcal{H}$ , we obtain that:
$$
T 3 \leq \frac {B}{\mu} \sqrt {\frac {\log (3 | \mathcal {H} | / \delta)}{(t - 1)}}, \tag {12}
$$
with probability at-least  $1 - \delta /(3T)$
To bound T2, we notice that it is not possible to directly apply Hoeffding's inequality because the summands in Eq.(8) are correlated through the estimates of the online algorithm. So in the following, we propose a trick to decorrelate them. For any hypothesis  $h \in \mathcal{H}$ , we have that
$$
\begin{array}{l} \frac {\hat {q} _ {t} \left(y _ {i j}\right)}{\hat {q} _ {i} \left(y _ {i j}\right)} \ell \left(h \left(x _ {i j}, y _ {i j}\right)\right) - E \left[ \frac {\hat {q} _ {t} \left(y _ {i j}\right)}{\hat {q} _ {i} \left(y _ {i j}\right)} \ell \left(h \left(x _ {i j}, y _ {i j}\right)\right) \right] \\ = \underbrace {\left(\frac {\hat {q} _ {t} \left(y _ {i j}\right)}{\hat {q} _ {i} \left(y _ {i j}\right)} - \frac {q _ {t} \left(y _ {i j}\right)}{q _ {i} \left(y _ {i j}\right)}\right) \ell \left(h \left(x _ {i j} , y _ {i j}\right)\right)} _ {U _ {i j}} - \\ \underbrace {E \left[ \left(\frac {\hat {q} _ {t} (y _ {i j})}{\hat {q} _ {i} (y _ {i j})} - \frac {q _ {t} (y _ {i j})}{q _ {i} (y _ {i j})}\right) \ell \left(h \left(x _ {i j} , y _ {i j}\right)\right) \right]} _ {V _ {i j}} + \\ \underbrace {\frac {q _ {t} (y _ {i j})}{q _ {i} (y _ {i j})} \ell (h (x _ {i j} , y _ {i j})) - E \left[ \frac {q _ {t} (y _ {i j})}{q _ {i} (y _ {i j})} \ell (h (x _ {i j} , y _ {i j})) \right]} _ {W _ {i j}}. \\ \end{array}
$$
Now using Eq.(9) and proceeding similar to the bounding steps of Eq.(10), we obtain
$$
\begin{array}{l} \frac {1}{N (t - 1)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} U _ {i j} \leq \frac {B}{N (t - 1) \mu^ {2}} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \| \hat {q} _ {i} - q _ {i} \| _ {1} + \| \hat {q} _ {t} - q _ {t} \| _ {1} \\ \leq \frac {B \sqrt {K}}{\mu^ {2} (t - 1)} \sum_ {i = 1} ^ {t - 1} \| \hat {q} _ {i} - q _ {i} \| _ {2} + \| \hat {q} _ {t} - q _ {t} \| _ {2} \\ \leq_ {(a)} \frac {B \sqrt {K}}{\mu^ {2}} \left(\| \hat {q} _ {t} - q _ {t} \| _ {2} + \sqrt {\frac {\sum_ {i = 1} ^ {t - 1} \| q _ {i} - \hat {q} _ {i} \| _ {2} ^ {2}}{t - 1}}\right) \\ \le_ {(b)} \frac {B \sqrt {K}}{\mu^ {2}} \left(\| \hat {q} _ {t} - q _ {t} \| _ {2} + \phi \cdot \frac {V _ {T} ^ {1 / 3}}{(t - 1) ^ {1 / 3}}\right), \\ \end{array}
$$
with probability at-least  $1 - \delta /3$ . In line (a) we used Jensen's inequality and in the last line we used the fact the online oracle attains a high probability bound on the total squared error (TSE) (see Proposition 20).
$\frac{1}{N(t - 1)}\sum_{i = 1}^{t - 1}\sum_{j = 1}^{N}V_{ij}$  can be bounded using the same expression as above using similar logic.
To bound  $\frac{1}{N(t - 1)}\sum_{i = 1}^{t - 1}\sum_{j = 1}^{N}W_{ij}$ , we note that it is the sum of independent random variables. Hence using the same arguments used to obtain Eq.(11), we have that
$$
\frac {1}{N (t - 1)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} W _ {i j} \leq \frac {B}{\mu} \sqrt {\frac {\log (3 T | \mathcal {H} | / \delta)}{(t - 1)}},
$$
with probability at-least  $1 - \delta /(3T|\mathcal{H}|)$ . Hence taking a union bound across all hypothesis classes and across the high probability event of low TSE for the online algorithm yields that
$$
T 2 \leq \frac {2 B \sqrt {K}}{\mu^ {2}} \left(\| \hat {q} _ {t} - q _ {t} \| _ {2} + \phi \cdot \frac {V _ {T} ^ {1 / 3}}{(t - 1) ^ {1 / 3}}\right) + \frac {B}{\mu} \sqrt {\frac {\log (3 T | \mathcal {H} | / \delta)}{(t - 1)}},
$$
with probability at-least  $1 - 2\delta /(3T)$
Combining the bounds developed for T1,T2,T3 and T4 and by taking a union bound across the event that resulted in Eq.(12), we obtain the following bound on instantaneous regret.
$$
L _ {t} \left(h _ {t}\right) - L _ {t} \left(h _ {t} ^ {*}\right) \leq \frac {2 B \sqrt {K}}{\mu^ {2}} \left(\| \hat {q} _ {t} - q _ {t} \| _ {2} + E \left[ \left| \hat {q} _ {t} - q _ {t} \right| \right] + \phi \cdot \frac {V _ {T} ^ {1 / 3}}{(t - 1) ^ {1 / 3}} + \sqrt {\frac {\log (3 T | \mathcal {H} | / \delta)}{(t - 1)}}\right), \tag {13}
$$
with probability at-least  $1 - \delta /T$
Note that via Jensen's inequality:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} E [ \| q _ {t} - \hat {q} _ {t} \| _ {2} ] \leq \sqrt {T \sum_ {t = 1} ^ {T} E [ \| q _ {t} - \hat {q} _ {t} \| _ {2} ^ {2} ]} \\ \leq \phi T ^ {2 / 3} V _ {T} ^ {1 / 3}, \\ \end{array}
$$
where in the last line we used Proposition 20.
Similarly it can be shown that
$$
\sum_ {t = 1} ^ {T} \| q _ {t} - \hat {q} _ {t} \| _ {2} \leq \phi T ^ {2 / 3} V _ {T} ^ {1 / 3},
$$
under the event that resulted in Eq.(13).
$$
\sum_ {t = 1} ^ {T} \frac {V _ {T} ^ {1 / 3}}{t ^ {1 / 3}} \leq 2 T ^ {2 / 3} V _ {T} ^ {1 / 3}.
$$
$$
\sum_ {t = 1} ^ {T} \frac {1}{\sqrt {t}} \leq 2 \sqrt {T}.
$$
Hence combining the above bounds and adding Eq.(13) across all time steps, followed by a union bound across all rounds, we obtain that
$$
\sum_ {t = 1} ^ {T} L _ {t} \left(h _ {t}\right) - L _ {t} \left(h _ {t} ^ {*}\right) \leq \frac {4 B \sqrt {K}}{\mu^ {2}} \left(3 \phi T ^ {2 / 3} V _ {T} ^ {1 / 3} + \sqrt {T \log (3 T | \mathcal {H} | / \delta)}\right),
$$
with probability at-least  $1 - \delta$
Next, we prove Theorem 18.
Proof. First we proceed to bound the number of switches. Observe that between two time points where condition in Line 5 of Algorithm 5 evaluates true, we can have at-most  $\log T$  switches due to the doubling epoch schedule in Line 8.
We first bound the number of times, condition in Line 5 is satisfied. Suppose for some some time  $t$ , we have that  $\sum_{j = b + 1}^{t}\| \mathrm{prev} - \tilde{\theta}_{j}\|_{2}^{2} > 4K\sigma^{2}\log (T / \delta)$ . Suppose throughout the run of the algorithm, this is  $i^{th}$  time the previous condition is satisfied. Let  $n_i\coloneqq t - b + 1$  and let  $C_i = \operatorname {TV}[b\to t]$
where  $\mathrm{TV}[p\to q] = \sum_{t = p + 1}^{q}\| \theta_{t} - \theta_{t - 1}\|_{1}$ . Due to the doubling epoch schedule, we have that that  $\mathrm{prev} = \frac{1}{\ell}\sum_{j = b}^{\ell}y_{j}$  and  $E[\mathrm{prev}] = \frac{1}{\ell}\sum_{j = b}^{\ell}\theta_{j}$  for some  $n_i\geq \ell \geq (t - b + 1) / 2 = n_i / 2$
$$
\begin{array}{l} \sum_ {j = b + 1} ^ {t} \| \operatorname {p r e v} - \tilde {\theta} _ {j} \| _ {2} ^ {2} \leq \sum_ {j = b + 1} ^ {t} 2 \| \operatorname {p r e v} - \theta_ {j} \| _ {2} ^ {2} + 2 \| \tilde {\theta} _ {j} - \theta_ {j} \| _ {2} ^ {2} \\ \leq \sum_ {j = b + 1} ^ {t} 2 \| E [ \text {p r e v} ] - \theta_ {j} \| _ {2} ^ {2} + 2 \| \text {p r e v} - E [ \text {p r e v} ] \| _ {2} ^ {2} + 2 \| \tilde {\theta} _ {j} - \theta_ {j} \| _ {2} ^ {2} \\ \leq_ {(a)} 2 (\ell C _ {i} ^ {2} + 2 \sigma^ {2} K \log (2 T / \delta)) + 2 \phi n _ {i} ^ {1 / 3} C _ {i} ^ {2 / 3} \\ \leq 4 \max  \left\{n _ {i} C _ {i} ^ {2}, \phi n _ {i} ^ {1 / 3} C _ {i} ^ {2 / 3} \right\} + 4 \sigma^ {2} K \log (2 T / \delta)), \tag {14} \\ \end{array}
$$
with probability at-least  $1 - \delta /(T)$  . In line (a) we used the following facts: i) Due to Hoeffding's inequality,  $\| \mathrm{prev} - E[\mathrm{prev}]\| _2^2\leq \sigma^2 K\log (4T / \delta)) / \ell \leq 2\sigma^2 K\log (2T / \delta)) / n_i$  with probability at-least  $1 - \delta /(2T)$  ; ii)  $\| E[\mathrm{prev}] - \theta_j\| _2 = \| \frac{1}{\ell}\sum_{i = b}^{\ell}\theta_i - \theta_j\| _2\leq \frac{1}{\ell}\sum_{i = b}^{\ell}\| \theta_i - \theta_j\| _2\leq C_i$  ; iii)  $\| \tilde{\theta}_j - \theta_j\| _2^2\leq$ $\phi n_i^{1 / 3}C_i^{2 / 3}$  with probability at-least  $1 - \delta /(2T)$  due to condition in Theorem 18; iv) Union bound over the events in (i) and (iii).
Since the condition in Line 5 is satisfied at round  $t$ , Eq.(14) will imply that  $5K\sigma^2\log (2T / \delta)\leq 4\max \{n_iC_i^2,\phi n_i^{1 / 3}C_i^{2 / 3}\} +4\sigma^2 K\log (2T / \delta))$ . Rearranging the above, we find that
$$
C _ {i} \gtrsim K / \sqrt {n _ {i}},
$$
where we suppress the dependence on constants and  $\log T$ .
Let the condition in Line 5 be satisfied  $M$  number of times. By union bound, we have that with probability at-least  $1 - \delta$
$$
\begin{array}{l} V _ {T} \geq \sum_ {i = 1} ^ {M} C _ {i} \\ \gtrsim \sum_ {i = 1} ^ {M} K / \sqrt {n _ {i}} \\ \stackrel {>} {_ {\sim_ {(a)}}} K M \frac {1}{\sqrt {(1 / M) \sum_ {i = 1} ^ {M} n _ {i}}} \\ \gtrsim K M ^ {3 / 2} / \sqrt {T}, \\ \end{array}
$$
where in Line (a) we used Jensen's inequality. Rearranging we get that
$$
M = \tilde {O} \left(T ^ {1 / 3} V _ {T} ^ {2 / 3} K ^ {- 2 / 3}\right), \tag {15}
$$
Now we proceed to bound the total squared error (TSE) incurred by Algorithm 5. Let  $\hat{\theta}_j$  be the output of Algorithm 5 at round  $j$ . Suppose at times  $b - 1$  and  $c + 1$ , the condition in Line (5) is satisfied. Observe that the condition in Line 5 is not satisfied for any times in  $[b, c]$ . Then we can conclude that within the interval  $[b, c]$  we have that  $\sum_{j = b}^{c}\| \hat{\theta}_j - \tilde{\theta}_j\|_2^2 \leq 5K\sigma^2\log (4T / \delta)\log (T)$ , since there are only at-most  $\log T$  times within  $[b, c]$  where condition in Line 9 is satisfied. So we have that