SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where KL is the Kullback-Leibler divergence (we refer to the classic definition of those distances as in [Tsybakov, 2008]). Let  $\tau'$  and  $\tau''$  be two strings in  $\{-1,1\}^\nu$  that only differ in one coordinate. Let  $\operatorname{Ber}(p)$  be a Bernoulli distribution with parameter  $p \in [0,1]$ . We have that:
$$
\begin{array}{l} \mathrm {K L} (Q ^ {(\tau^ {\prime})}, Q ^ {(\tau^ {\prime \prime})}) = \sum_ {t = 1} ^ {T} \mathrm {K L} (Q _ {t} ^ {(\tau^ {\prime})}, Q _ {t} ^ {(\tau^ {\prime \prime})}) \\ = \sum_ {t = T - \tilde {r} + 1} ^ {T} \mathrm {K L} \left(Q _ {t} ^ {\left(\tau^ {\prime}\right)}, Q _ {t} ^ {\left(\tau^ {\prime \prime}\right)}\right) \\ = \frac {1}{\nu} \sum_ {t = 1} ^ {\tilde {r}} \mathrm {K L} \left(\operatorname {B e r} \left(\frac {1}{2} + \frac {1}{1 6 \sqrt {6}} \left(\sqrt {\frac {\nu}{\tilde {r}}} + \Delta_ {t}\right)\right), \operatorname {B e r} \left(\frac {1}{2} - \frac {1}{1 6 \sqrt {6}} \left(\sqrt {\frac {\nu}{\tilde {r}}} + \Delta_ {t}\right)\right)\right) \tag {21} \\ \end{array}
$$
where the first equality is due to the factorization property of the KL-divergence, the second and third equality are due to the definition of the family  $\mathcal{Q}$ . For any  $\epsilon < 1/4$ , it holds that
$$
\operatorname {K L} \left(\operatorname {B e r} \left(\frac {1}{2} - \epsilon\right), \operatorname {B e r} \left(\frac {1}{2} + \epsilon\right)\right) \leq 1 2 \epsilon^ {2}.
$$
By plugging the above inequality in (21), and using the fact that  $(x + y)^2 \leq 2x^2 + 2y^2$ , we obtain that
$$
\begin{array}{l} \mathrm {K L} (Q ^ {(\tau^ {\prime})}, Q ^ {(\tau^ {\prime \prime})}) \leq \frac {2 4}{1 6 ^ {2} \cdot 6 \nu} \left[ \sum_ {t = 1} ^ {\tilde {r}} \frac {\nu}{\tilde {r}} + \sum_ {t = 1} ^ {\tilde {r}} \Delta_ {\tilde {r}} ^ {2} \right] \\ \leq \frac {2 4}{1 6 ^ {2} \cdot 6 \nu} \left[ \nu + \tilde {r} \Delta_ {\tilde {r}} ^ {2} \right] \\ \end{array}
$$
By using the definition of  $\tilde{r}$ , it holds that  $\Delta_{\tilde{r}}^{2}\tilde{r} \leq \nu$ . Thus, we have that  $\mathrm{KL}(Q^{(\tau')}, Q^{(\tau'')} \leq 1/32$ . If we use this inequality with (20), we have
$$
\min_{\substack{\tau^{\prime},\tau^{\prime \prime}\\ \operatorname {hd}(\tau^{\prime},\tau^{\prime \prime}) = 1}}\| Q^{(\tau^{\prime})}\wedge Q^{(\tau^{\prime \prime})}\|_{1}\geq 7 / 8    .
$$
Hence, we can conclude that there exists  $\tau \in \{-1, 1\}^{\nu}$  such that
$$
\mathbb {E} \left[ Q _ {T} ^ {(\tau)} \left(L _ {h _ {\mathcal {A}}}\right) - Q _ {T} ^ {(\tau)} \left(L _ {h ^ {*}} ^ {\tau}\right) \right] \geq \frac {7 \sqrt {6} \Phi (\tilde {r})}{7 6 8}.
$$
By using this Lemma, we can easily prove Theorem 8.
Proof of Theorem 8. . By Lemma 12, there exists  $\tau$  such that if  $P_{t} = Q_{t}^{(\tau)}$  for all  $1\leq t\leq T$ , then the algorithm  $\mathcal{A}$  with input  $Z_{1},\ldots ,Z_{T}$  satisfies
$$
\mathbb {E} \left[ P _ {T} \left(L _ {h _ {\mathcal {A}}}\right) - P _ {T} \left(L _ {h ^ {*}}\right) \right] \geq 7 \sqrt {6} \Phi^ {*} / 7 6 8. \tag {22}
$$
Due to (18), we observe that the distributions  $P_{1},\ldots ,P_{T}$  satisfy the assumption on the drift.
Let  $E = P_{T}(L_{h_{A}}) - P_{T}(L_{h^{*}})$ . Observe that due to the construction of  $Q^{(\tau)}$  and (19), we have that  $E \leq \Phi(\tilde{r}) / (8\sqrt{6}) \leq 3\Phi^{*} / (8\sqrt{6})$ , where the last inequality is due to Proposition 11. Let  $\alpha > 0$  be a later defined value. We have that:
$$
\begin{array}{l} \mathbb {E} [ E ] = \mathbb {E} [ E | E > \alpha \Phi^ {*} ] \Pr (E \geq \alpha \Phi^ {*}) + \mathbb {E} [ E | E <   \alpha \Phi^ {*} ] \Pr (E \leq \alpha \Phi^ {*}) \\ \leq \left(3 \Phi^ {*} / (8 \sqrt {6})\right) \Pr (E \geq \alpha \Phi^ {*}) + \alpha \Phi^ {*} (1 - P r (E \leq \alpha \Phi^ {*})) \\ = \Pr (E \geq \alpha \Phi^ {*}) [ 3 \Phi^ {*} / 2 0 - \alpha \Phi^ {*} ] + \alpha \Phi^ {*}. \\ \end{array}
$$
By using the above inequality together with (22), we finally obtain that:
$$
\Pr (E \geq \alpha \Phi^ {*}) \geq \frac {7 \sqrt {6} \Phi^ {*} / 7 6 8 - \alpha \Phi^ {*}}{3 \Phi^ {*} / (8 \sqrt {6}) - \alpha \Phi^ {*}} = \frac {7 \sqrt {6} / 7 6 8 - \alpha}{3 / (8 \sqrt {6}) - \alpha}
$$
We set  $\alpha = 1 / (112\sqrt{6})$  and we finally obtain that
$$
\Pr \left(E \geq \frac {\Phi^ {*}}{1 1 2 \sqrt {6}}\right) \geq 1 / 8
$$
B Relaxing Assumption 2
Given a family  $\mathcal{F}$ , it is possible that the exact computation of  $\| \mathbb{P}_T^r -\mathbb{P}_T^{2r}\|_{\mathcal{F}}$  is computationally hard. In this section, we show an example on how to relax Assumption 2 to allow an approximation of this quantity.
Let  $\alpha \geq 1$ . We say that an algorithm  $A$  in an  $\alpha$ -approximation procedure if given  $Z_{t - 2r + 1},\ldots ,Z_T$ , it computes an estimate  $A(Z_{T - 2r + 1},\dots,Z_T)$  such that
$$
1 \leq \frac {\| \mathbb {P} _ {T} ^ {r} - \mathbb {P} _ {T} ^ {2 r} \| _ {\mathcal {F}}}{A (Z _ {T - 2 r + 1} , \dots , Z _ {T})} \leq \alpha
$$
for any  $r \leq T / 2$ . That is, the algorithm  $A$  does not compute the value of the supremum of the norm  $\| \cdot \|_{\mathcal{F}}$  exactly, but it guarantees a constant factor approximation  $\alpha$ . In this case, we can modify the algorithm as follows.
Algorithm 2: Adaptive Learning Algorithm under Drift with Approximation Procedure
$i\gets 0$    
while  $r_i\leq T / 2$  do if  $A(Z_{t - 2r + 1},\dots ,Z_T)\leq 4\cdot S(r_i,\delta)$  then  $\begin{array}{l}\mid i\leftarrow i + 1;\\ \mathrm{end}\\ \mathrm{else}\\ \mid \mathrm{return}~r_i;\\ \mathrm{end}\\ \mathrm{end}\\ \mathrm{return}~r_i \end{array}$
Theorem 13. Let  $\delta \in (0,1)$ . Let Assumptions 1 hold, and assume that there exists an  $\alpha$ -approximation procedure for estimating  $\| \mathbb{P}_T^r - \mathbb{P}_T^{2r} \|_{\mathcal{F}}$  for any  $r \leq T/2$ . Given  $Z_1, \ldots, Z_T$ , there exists an algorithm that outputs a value  $\hat{r} \leq T$  such that with high-probability, it holds that
$$
\| P _ {T} - \mathbb {P} _ {T} ^ {\hat {r}} \| _ {\mathcal {F}} = \alpha \cdot O \left(\min  _ {r \leq T} \left[ \frac {\alpha C _ {\mathcal {F} , 1}}{\sqrt {r}} + \max  _ {t <   r} \| P _ {T} - P _ {T - t} \| _ {\mathcal {F}} + \alpha C _ {\mathcal {F}, 2} \sqrt {\frac {\log (\log (r + 1) / \delta)}{r}} \right]\right)
$$
Proof. The proof follows the same strategy as the one of the main theorem with slight modifications, and we discuss those changes. We replace the definition of  $U(r,\delta)$  with
$$
U (r, \delta) = (1 8 \alpha + 9) S (r, \delta) + \| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}}
$$
in order to take into account the additional error due to the approximation procedure. We can observe that
$$
A \left(Z _ {T - r _ {i + 1} - 1}, \dots , Z _ {T}\right) \leq 4 S (r _ {i}, \delta)
$$
implies
$$
\| \mathbb {P} _ {T} ^ {r _ {i}} - \mathbb {P} _ {T} ^ {r _ {i + 1}} \| _ {\mathcal {F}} \leq 4 \alpha S (r _ {i}, \delta)
$$
since  $A$  is an  $\alpha$ -approximation procedure. We can follow the steps of Proposition 4 with those different constants to show an equivalent statement of this Proposition. On the other hand, we have that if
$$
A \left(Z _ {t - 2 r + 1}, \dots , Z _ {T}\right) > 4 S \left(r _ {i}, \delta\right)
$$
$$
\| \mathbb {P} _ {T} ^ {r} - \mathbb {P} _ {T} ^ {2 r} \| _ {\mathcal {F}} > 4 S (r _ {i}, \delta) ,
$$
and Proposition 5 applies. Therefore, we can use the same proof strategy of Theorem 1 to prove Theorem 13