SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let  $1/5$ -Err denote the random variable that counts the number of time steps in which at least  $1/5$ th of the experts make an error. As in the proof of Theorem 5.3,  $1/5$ -Err upper bounds both the number of mistakes made by  $\mathsf{POP}_{[u,w]}$ , which we denote by OurError, as well as the number of times in which algorithm ChallengeAT returns an "above threshold" answer, which we denote by NumTop. By Theorem 4.2, we know that (w.h.p.) NumTop  $\geq r$ . Also let WorstExpert denote the largest number of mistakes made by a single expert.
Consider the time  $i^*$  at which  $\mathsf{POP}_{[u,w]}$  halts. If it halts because  $u \leq v \leq w$  mistakes have been made, then
$$
k \cdot \operatorname {W o r s t E x p e r t} \geq 1 / 5 - \operatorname {E r r} \geq \operatorname {O u r E r r o r} \geq u = \Omega (k d \ln (T)).
$$
Alternatively, if  $\mathsf{POP}_{[u,w]}$  halts after  $r$  "above threshold" answer, then
$$
k \cdot \operatorname {W o r s t E x p e r t} \geq 1 / 5 - \operatorname {E r r} \geq \operatorname {N u m T o p} \geq r = \Omega (k d \ln (T)).
$$
At any case, when  $\mathsf{POP}_{[u,w]}$  halts it holds that at least one expert made at least  $\Omega$  ( $d\ln(T)$ ) mistakes. Therefore, by Theorem F.1, we have that  $\mathrm{OPT}_{i^*} \geq d\ln(T)$ .
Theorem F.4. Let  $H$  be a hypothesis class with  $\operatorname{Ldim}(H) = d$ . There exists an  $(\varepsilon, \delta)$ -Challenge-DP online learning algorithm providing the following guarantee. When executed on an adaptively
chosen sequence of inputs  $(x_{1},y_{1}),\ldots ,(x_{T},y_{T})$ , then the algorithm makes at most  $\tilde{O}\left(\frac{d\cdot\mathrm{OPT}}{\varepsilon^2} +\frac{d^2}{\varepsilon^2}\right)$  mistakes (w.h.p.), where
$$
\mathrm {O P T} \triangleq \min  _ {h \in H} \sum_ {i = 1} ^ {T} | h (x _ {i}) - y _ {i} |.
$$
Proof sketch. This is obtained by repeatedly re-running  $\mathsf{POP}_{[u,w]}$ , with the parameter setting specified in Lemma F.3. We refer to the time span of every single execution of  $\mathsf{POP}_{[u,w]}$  as a phase.
By construction, in every phase,  $\mathsf{POP}_{[u,w]}$  makes at most  $w = \tilde{\Theta} (kd)$  mistakes. By Lemma F.3 every hypothesis in  $H$  makes at least  $d\cdot \ln (T)$  mistakes in this phase. Therefore, there could be at most  $\tilde{O}\left(\max \left\{1,\frac{\mathrm{OPT}}{d}\right\}\right)$  phases, during which we incur a total of at most  $\tilde{O}\left(\frac{d\cdot\mathrm{OPT}}{\varepsilon^2} +\frac{d^2}{\varepsilon^2}\right)$  mistakes.