SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let  $p(x) = \operatorname*{Pr}_{y\sim (Y|X)}[y\neq h^{*}(x)]$  denote the probability of error if we query  $x$ , so
$$
\underset {x \sim \mathcal {D} _ {X}} {\mathbb {E}} [ p (x) ] \leq \eta .
$$
Suppose we query a point  $x$  and do not get an error. Then the hypotheses that disagree with  $h^*$  are downweighted by an  $e^{-\alpha}$  factor, so
$$
\frac {\lambda_ {i , S} (h ^ {*})}{\lambda_ {i + 1 , S} (h ^ {*})} = \underset {h \sim \lambda_ {i, S}} {\mathbb {E}} [ 1 + (e ^ {- \alpha} - 1) 1 _ {h (x) \neq h ^ {*} (x)} ] = 1 - (1 - e ^ {- \alpha}) \widetilde {r} (x).
$$
On the other hand, if we do get an error then the disagreeing hypotheses are effectively upweighted by  $e^{\alpha}$ :
$$
\frac {\lambda_ {i , S} \left(h ^ {*}\right)}{\lambda_ {i + 1 , S} \left(h ^ {*}\right)} = 1 + \left(e ^ {\alpha} - 1\right) \widetilde {r} (x).
$$
$$
\begin{array}{l} \underset {y | x} {\mathbb {E}} \left[ \log \frac {\lambda_ {i + 1 , S} \left(h ^ {*}\right)}{\lambda_ {i , S} \left(h ^ {*}\right)} \middle | \mathcal {F} _ {i} \right] \\ = - (1 - p (x)) \log \left(1 - \left(1 - e ^ {- \alpha}\right) \widetilde {r} (x)\right) - p (x) \log \left(1 + \left(e ^ {\alpha} - 1\right) \widetilde {r} (x)\right) \tag {6} \\ \geq (1 - p (x)) \left(1 - e ^ {- \alpha}\right) \widetilde {r} (x) - p (x) \left(e ^ {\alpha} - 1\right) \widetilde {r} (x) \\ = (1 - e ^ {- \alpha}) \widetilde {r} (x) - p (x) \widetilde {r} (x) \left(e ^ {\alpha} - e ^ {- \alpha}\right). \\ \end{array}
$$
Using that  $\widetilde{r}(x) \leq 1$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \log \frac {\lambda_ {i + 1 , S} \left(h ^ {*}\right)}{\lambda_ {i , S} \left(h ^ {*}\right)} \mid \mathcal {F} _ {i} \right] \geq (1 - e ^ {- \alpha}) \mathbb {E} _ {x \sim q} [ \widetilde {r} (x) ] - (e ^ {\alpha} - e ^ {- \alpha}) \mathbb {E} _ {x \sim q} [ p (x) ] \\ \geq 0. 9 \alpha \underset {x \sim q} {\mathbb {E}} [ \widetilde {r} (x) - 2. 3 p (x) ], \\ \end{array}
$$
where the last step uses  $\alpha \leq 0.2$ . Finally,
$$
\underset {x \sim q} {\mathbb {E}} [ p (x) ] = \underset {x \sim \mathcal {D} _ {X}} {\mathbb {E}} \left[ p (x) \frac {q (x)}{\mathcal {D} _ {X} (x)} \right] \leq \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}.
$$
This proves the first desired result. For the second, note that if we query  $x$ , then conditioned on  $\mathcal{F}_i$
$$
\max  \left\{0, \log \frac {\lambda_ {i + 1 , S} (h ^ {*})}{\lambda_ {i , S} (h ^ {*})} \right\} = \left\{ \begin{array}{l l} 0 & \text {w i t h p r o b a b i l i t y} p (x), \\ \log (1 + (1 - e ^ {- \alpha}) \widetilde {r} (x)) & \text {o t h e r w i s e}. \end{array} \right.
$$
Since  $\log (1 + (1 - e^{-\alpha})\widetilde{r} (x))\leq (1 - e^{-\alpha})\widetilde{r} (x)\leq \alpha \widetilde{r} (x)$ , taking the expectation over  $x$  gives the result.
The above lemma, combined with Lemma 2.1, proves the potential will grow at desired rate at each iteration. But remember that Lemma 2.1 requires the condition that no ball has probability greater than  $80\%$ , so we need to check this condition is satisfied. The following lemma shows that if we cap the set  $S_{i}$ , then the probability is not concentrated on any small balls.
Lemma A.2. In Algorithm 1, for every iteration  $i$ ,  $S_{i}$  is such that no radius  $c_{4}\eta + c_{5}\varepsilon$  ball has more than  $80\%$  probability under  $\overline{\lambda}_{i,S_i}$ .
Proof. If  $S_{i} = S_{i - 1}$ , then by the construction of  $S_{i}$ , there are no radius  $c_{4}\eta +c_{5}\varepsilon$  balls have probability greater than  $80\%$  under  $\overline{\lambda}_{i,S_{i - 1}} = \overline{\lambda}_{i,S_i}$ . Otherwise, we have  $S_{i - 1}\neq S_{i}$  and a ball  $B(\mu ,3c_4\eta +3c_5\varepsilon)$  is added to  $S_{i}$  in this iteration. We first prove a useful claim below.
Claim A.3. If a ball  $B' = (\mu, 3c_4\eta + 3c_5\varepsilon)$  is added to  $S_i$  at some iteration  $i$ ,  $\lambda_i(B(\mu, c_4\eta + c_5\varepsilon)) \geq 0.6$ .
Proof. If  $B'$  is added to  $S_{i}$  at the iteration  $i$ , then there exists some ball  $D$  with radius  $c_{4}\eta + c_{5}\varepsilon$  such that  $\bar{\lambda}_{i,S_{i-1}}(D) \geq 0.8$ . If a set of hypotheses gains probability after capping, the gained probability comes from the reduced probability of other hypotheses not in this set. Therefore, the gained probability of any set is upper bounded by half of the probability of the complement of that set before capping. This means  $\lambda_{i}(D) \geq 0.6$  because otherwise after capping  $\bar{\lambda}_{i,S_{i-1}}(D) < 0.8$ , which is a contradiction. As a result,  $\lambda_{i}(B(\mu, c_{4}\eta + c_{5}\varepsilon)) \geq \lambda_{i}(D) \geq 0.6$ .
By Claim A.3, the probability of  $B(\mu, c_4\eta + c_5\varepsilon)$  is at least 0.6 over the uncapped distribution  $\lambda_i$ . So any ball not intersecting  $B(\mu, c_4\eta + c_5\varepsilon)$  has probability at most 0.4 before capping. After capping these balls will have probability no more than 0.7. At the same time, any ball intersects  $B(\mu, c_4\eta + c_5\varepsilon)$  would be completely inside  $B(\mu, 3c_4\eta + 3c_5\varepsilon)$  so its probability would be at most 0.5 after capping.
Now we are ready to apply Lemma A.1 and Lemma 2.1 except one caution. Remember that in the beginning of the algorithm, we compute a  $2\eta$ -packing  $H' \subseteq H$  of the instance. From the well-known relationship between packing and covering (for example, see Vershynin [2018, Lemma 4.2.8]), we have  $|H'| \leq N(H, \eta)$ . Every hypothesis in  $H$  is within  $2\eta$  to some hypothesis in  $H'$ , so there exists a hypothesis in  $H'$  with error less than  $3\eta$ . This means that the best hypothesis  $h^* \in H'$  has error  $3\eta$  instead of  $\eta$ . The following lemma serves as the cornerstone of the proof of the query complexity upper bound, which states that the potential grows at rate  $\Omega\left(\frac{1}{m^*}\right)$  in each iteration.
Lemma A.4. Given  $c_{4} \geq 300$  and  $\mathrm{err}(h^{*}) \leq 3\eta$ , there exists a sampling distribution  $q$  such that
$$
\mathbb {E} [ \Delta_ {i} | \mathcal {F} _ {i} ] \geq \mathbb {E} \left[ \Delta_ {i} | \mathcal {F} _ {i} \right] - 2 \alpha \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \gtrsim \frac {\alpha}{m ^ {*} \left(H , \mathcal {D} _ {X} , c _ {4} \eta , c _ {5} \varepsilon - 2 \eta , \frac {9 9}{1 0 0}\right)} i f h ^ {*} \notin S _ {i},
$$
as well as  $|\Delta_i| \leq \alpha$  always and  $\operatorname{Var}[\Delta_i|\mathcal{F}_i] \leq \alpha \mathbb{E}[|\Delta_i||\mathcal{F}_i|] \lesssim \alpha \mathbb{E}[\Delta_i|\mathcal{F}_i]$ .
Proof. For the sake of bookkeeping, we let  $m^{*} = m^{*}\left(H,\mathcal{D}_{X},c_{4}\eta ,c_{5}\varepsilon -2\eta ,\frac{99}{100}\right)$  in this proof and the following text. We first bound the expectation. By Lemma A.1 applied to  $S\in \{H,H\setminus S_i\}$  with  $3\eta$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \Delta_ {i} \mid \mathcal {F} _ {i} \right] - 2 \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \geq 0. 9 \alpha \left(\mathbb {E} _ {x \sim q} \left[ r _ {i, H, h *} (x) + r _ {i, H \backslash S _ {i}, h *} (x) \right] - 1 3. 8 \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}\right) \\ - 2 \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}, \\ \end{array}
$$
where  $q$  is the query distribution of the algorithm at iteration  $i$ . Now, by the definition of
$$
\overline {{\lambda}} _ {i, S} = \frac {1}{2} \lambda_ {i} + \frac {1}{2} \lambda_ {i, H \backslash S},
$$
we have for any  $x$  that
$$
\bar {r} _ {i, S _ {i}, h ^ {*}} (x) = \frac {1}{2} \left(r _ {i, h ^ {*}} (x) + r _ {i, H \backslash S _ {i}, h ^ {*}} (x)\right)
$$
$$
\begin{array}{l} \mathbb {E} \left[ \Delta_ {i} \mid \mathcal {F} _ {i} \right] - 2 \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \\ \geq 1. 8 \alpha \left(\underset {x \sim q} {\mathbb {E}} \left[ \bar {r} _ {i, S _ {i}, h ^ {*}} (x) \right] - 6. 9 \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}\right) - 2 \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \tag {7} \\ \geq 1.8 \alpha \left(\underset {x \sim q} {\mathbb {E}} \left[ \overline {{r}} _ {i, S _ {i}} (x) \right] - 8. 1 \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}\right). \\ \end{array}
$$
Algorithm 1 chooses the sampling distribution  $q$  to maximize  $\mathbb{E}_{x\sim q}[\overline{r}_{i,S_i}(x)] - \frac{c_4}{20}\eta \max_x\frac{q(x)}{\mathcal{D}_X(x)}\leq$ $\mathbb{E}_{x\sim q}[\overline{r}_{i,S_i}(x)] - 15\eta \max_x\frac{q(x)}{\mathcal{D}_X(x)}$  because  $c_{4}\geq 300$ . By Lemma A.2,  $\overline{\lambda}_{i,S_i}$  over  $H^{\prime}$  has no radius- $(c_{4}\eta +c_{5}\varepsilon)$  ball with probability larger than  $80\%$ , so by Lemma 2.1  $q$  satisfies
$$
\underset {x \sim q} {\mathbb {E}} [ \overline {{r}} _ {i, S _ {i}} (x) ] - 1 5 \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \geq \underset {x \sim q} {\mathbb {E}} [ \overline {{r}} _ {i, S _ {i}} (x) ] - \frac {c _ {4}}{2 0} \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \gtrsim \frac {1}{m ^ {*} \left(H ^ {\prime} , \mathcal {D} _ {X} , c _ {4} \eta , c _ {5} \varepsilon , \frac {9 9}{1 0 0}\right)}.
$$
Because  $H' \subseteq H$  is a maximal  $2\eta$ -packing, every hypothesis in  $H$  is within  $2\eta$  of some hypothesis in  $H'$ . The problem  $\left(H, \mathcal{D}_X, c_4\eta, c_5\varepsilon - 2\eta, \frac{99}{100}\right)$  is harder than the problem  $\left(H', \mathcal{D}_X, c_4\eta, c_5\varepsilon, \frac{99}{100}\right)$  because we can reduce the latter to the former by simply adding more hypotheses and solve
it then map the solution back by returning the closest hypothesis in  $H'$ . Hence,  $m^* \geq m^* \left(H', \mathcal{D}_X, c_4\eta, c_5\varepsilon, \frac{99}{100}\right)$ . Therefore,
$$
\mathbb {E} \left[ \Delta_ {i} | \mathcal {F} _ {i} \right] - 2 \alpha \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \geq 1. 8 \alpha \left(\mathbb {E} _ {x \sim q} [ \bar {r} _ {i, S _ {i}} (x) ] - 8. 1 \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}\right) \gtrsim \frac {\alpha}{m ^ {*}}.
$$
We now bound the variance. The value of  $\Delta_{i}$  may be positive or negative, but it is bounded by  $|\Delta_i| \leq \alpha$ . Thus
$$
\operatorname {V a r} \left[ \Delta_ {i} \mid \mathcal {F} _ {i} \right] \leq \mathbb {E} \left[ \Delta_ {i} ^ {2} \mid \mathcal {F} _ {i} \right] \leq \alpha \mathbb {E} \left[ \left| \Delta_ {i} \right| \mid \mathcal {F} _ {i} \right].
$$
By Lemma A.1 and (7) we have
$$
\begin{array}{l} \mathbb {E} [ | \Delta_ {i} | | \mathcal {F} _ {i} ] = \mathbb {E} [ 2 \max  \{\Delta_ {i}, 0 \} - \Delta_ {i} | \mathcal {F} _ {i} ] \\ \leq 4 \alpha \underset {x \sim q} {\mathbb {E}} [ \bar {r} _ {i, S _ {i}} (x) ] - 1. 8 \alpha \left(\underset {x \sim q} {\mathbb {E}} [ \bar {r} _ {i, S _ {i}} (x) ] - 8. 1 \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}\right) \\ \leq 2. 2 \alpha \left(\underset {x \sim q} {\mathbb {E}} [ \bar {r} _ {i, S, h ^ {*}} (x) ] + 6. 7 \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}\right) \\ \leq \frac {2 . 2 \alpha}{1 . 8 \alpha} \mathbb {E} \left[ \Delta_ {i} | \mathcal {F} _ {i} \right] + 2. 2 \alpha \cdot 6. 9 \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} + 2. 2 \alpha \cdot 6. 7 \eta \max _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \\ \leq 1. 3 \mathbb {E} [ \Delta_ {i} | \mathcal {F} _ {i} ] + 3 0 \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}. \\ \end{array}
$$
Since  $\mathbb{E}_{x\sim q}[\Delta_i|\mathcal{F}_i] - 2\alpha \eta \max_x\frac{q(x)}{\mathcal{D}_X(x)}\gtrsim \frac{1}{m^*}\geq 0$  we have
$$
\eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \leq \frac {1}{2 \alpha} \underset {x \sim q} {\mathbb {E}} [ \Delta_ {i} | \mathcal {F} _ {i} ],
$$
$$
\operatorname {V a r} [ \Delta_ {i} | \mathcal {F} _ {i} ] \leq \alpha \mathbb {E} [ | \Delta_ {i} | | \mathcal {F} _ {i} ] \lesssim \alpha \mathbb {E} [ \Delta_ {i} | \mathcal {F} _ {i} ].
$$
A.3 Concentration of potential
We have showed that the potential will grow at  $\Omega\left(\frac{1}{m^{*}}\right)$  per iteration, but only in expectation, while our goal is to obtain a high probability bound. Let  $\mu_{k} \coloneqq \sum_{i < k} \mathbb{E}[\Delta_{i}|\mathcal{F}_{i-1}] \gtrsim k/m^{*}$ , then
$$
\begin{array}{l} \mathbb {E} \left[ \left(\psi_ {k} - \mu_ {k}\right) - \left(\psi_ {k - 1} - \mu_ {k - 1}\right) | \mathcal {F} _ {k - 1} \right] = \mathbb {E} \left[ \psi_ {k} - \psi_ {k - 1} | \mathcal {F} _ {k - 1} \right] - \left(\mu_ {k} - \mu_ {k - 1}\right) \\ = \mathbb {E} \left[ \Delta_ {i} | \mathcal {F} _ {i} \right] - \mathbb {E} \left[ \Delta_ {i} | \mathcal {F} _ {i} \right] \geq 0. \\ \end{array}
$$
Apparently  $|\psi_k - \mu_k|$  is upper bounded, so  $\psi_k - \mu_k$  is a supermartingale. To show a high probability bound, we will use Freedman's inequality. A version is stated in Tropp [2011]. We slighted modify it so it can be applied to supermartingale as the following. (XXX Not sure if the following is correct. I can't find a version of supermartingale.)
Theorem A.5 (Freedman's Inequality). Consider a real-valued supermartingale  $\{Y_k : k = 0,1,2,\dots\}$  that is adapted to the filtration  $\mathcal{F}_0 \subseteq \mathcal{F}_1 \subseteq \mathcal{F}_2 \subseteq \dots \subseteq \mathcal{F}$  with difference sequence  $\{X_k : k = 1,2,3,\dots\}$ . Assume that the difference sequence is uniformly bounded:
$$
X _ {k} \leq R \text {a l m o s t s u r e l y f o r} k = 1, 2, 3, \dots
$$
Define the predictable quadratic variation process of the supermartingale:
$$
W _ {k} := \sum_ {j = 1} ^ {k} \mathbb {E} \left[ X _ {j} ^ {2} | \mathcal {F} _ {j - 1} \right] f o r k = 1, 2, 3, \dots
$$
Then, for all  $t \geq 0$  and  $\sigma^2 > 0$ ,
$$
\Pr \left(\exists k \geq 0: Y _ {k} \leq - t a n d W _ {k} \leq \sigma^ {2}\right) \leq \exp \left(- \frac {t ^ {2} / 2}{\sigma^ {2} + R t / 3}\right).
$$
Then we can prove a high probability bound as the following.
Lemma A.6. With probability  $1 - \delta$ ,  $\phi_{i} = 0$  for some  $i = O\left(m^{*}\log \frac{|H|}{\delta}\right)$  so  $h^* \in S_i$ .
Proof. Remember we have that
$$
\phi_ {k} = \phi_ {0} + \psi_ {k} + \sum_ {i <   k} \log \frac {\lambda_ {i , H \backslash S _ {i + 1}} (h ^ {*})}{\lambda_ {i , H \backslash S _ {i}} (h ^ {*})}.
$$
Since  $S_{i + 1}\supseteq S_i$  for all  $i$ ,  $\lambda_{i,H\backslash S_{i + 1}}(h^{*})\geq \lambda_{i,H\backslash S_{i}}(h^{*})$  if  $h^*\notin S_{i + 1}$ , we have
$$
\phi_ {k} \geq \phi_ {0} + \psi_ {k} \quad \text {i f} h ^ {*} \notin S _ {k}.
$$
Let  $K = O\left(m^{*}\log \frac{|H|}{\delta}\right)$ . Let's assume by contradiction that  $\phi_K < 0$  for for, then  $h^* \notin S_i$  for  $i \leq K$ . We know by Lemma A.4 that
$$
\mu_ {k} := \sum_ {i <   k} \mathbb {E} \left[ \Delta_ {i} \mid \mathcal {F} _ {i - 1} \right] \gtrsim \frac {k}{m ^ {*}}
$$
and that  $\sum_{i < k} \operatorname{Var}\left[\Delta_i\right] \leq \frac{1}{4} \mu_k$  by picking  $\alpha$  small enough. Moreover,  $|\Delta_i| \leq \alpha$  always. To use Freedman's inequality, let's set the RHS
$$
\exp \left(- \frac {t ^ {2} / 2}{\sigma^ {2} + R t / 3}\right) \leq \delta .
$$
Solving the above quadratic equation, one solution is that  $t \geq \frac{R}{3} \log \frac{1}{\delta} + \sqrt{\frac{R^2}{9} \log^2 \frac{1}{\delta} + 2 \sigma^2 \log \frac{1}{\delta}}$ . Let's substitute in  $R = \alpha$  and  $\sigma^2 = \sum_{i < k} \operatorname{Var}_{i-1}(\Delta_i)$ , with  $1 - \delta$  probability we have for any  $k > O(m^* \log \frac{1}{\delta})$  that
$$
\begin{array}{l} \psi_ {k} \geq \mu_ {k} - \sqrt {\frac {\alpha^ {2}}{9} \log^ {2} \frac {1}{\delta} + 2 \sum_ {i <   k} \operatorname {V a r} (\Delta_ {i}) \log \frac {1}{\delta}} - \frac {\alpha}{3} \log \frac {1}{\delta} \\ \geq \mu_ {k} - \sqrt {\frac {\alpha^ {2}}{9} \log^ {2} \frac {1}{\delta} + \frac {1}{2} \mu_ {k} \log \frac {1}{\delta}} - \frac {\alpha}{3} \log \frac {1}{\delta} \\ \geq \mu_ {k} - \max  \left\{\frac {\sqrt {2} \alpha}{3} \log \frac {1}{\delta}, \sqrt {\mu_ {k} \log \frac {1}{\delta}} \right\} - \frac {\alpha}{3} \log \frac {1}{\delta} \\ \geq \frac {1}{2} \mu_ {k} \\ \gtrsim \frac {k}{m ^ {*}}. \\ \end{array}
$$
The second last inequality is because the first term outscales all of the rest. Since  $K = O\left(m^{*}\log \frac{|H|}{\delta}\right)$ , we have
$$
\psi_ {K} \geq 2 \log | H |
$$
with  $1 - \delta$  probability. Then  $\phi_K \geq \phi_0 + \psi_k$  because  $\phi_0 \geq \log \frac{1}{2|H|} \geq -2\log |H|$  and this contradicts  $h^* \notin S_K$ . Therefore, with probability at least  $1 - \delta$ ,  $h^* \in S_K$  and by definition,  $\phi_i = 0$  for some  $i \leq K$  as desired.
A.4 Bounding the Size of  $|C|$
So far we've shown that after  $O\left(m^{*}\log \frac{|H|}{\delta}\right)$  iterations,  $h^*$  will be included in the set  $S_{i}$ . The last thing we need to prove Theorem 1.1 is that with high probability,  $C$  is small, which is equivalent to show that not many balls will be added to  $S_{i}$  after  $O\left(m^{*}\log \frac{|H|}{\delta}\right)$  iterations. To show this, we first need to relate the number of balls added to  $S_{i}$  to  $\psi_{i}$ . Let  $\mathcal{E}_i$  denote the number of errors  $h^*$  made up to iteration  $i$  (and set  $\mathcal{E}_i = \mathcal{E}_{i-1}$  if  $h^* \in S_i$ ) and  $\mathcal{N}_i$  denote the number of balls added to  $S_i$  up to iteration  $i$  (again set  $\mathcal{N}_i = \mathcal{N}_{i-1}$  if  $h^* \in S_i$ ).
Lemma A.7. The following inequality holds for every  $i$ :
$$
\mathcal {N} _ {i} \leq 5 (\psi_ {i} + 2 \alpha \mathcal {E} _ {i}) + 1.
$$
Proof. We divide the  $i$  iterations into phases. A new phase begins and an old phase ends if at this iteration a new ball is added to the set  $S_{i}$ . We use  $p_1, \ldots, p_k$  for  $k \leq i$  to denote phases and  $i_1, \ldots, i_k$  to denote the starting iteration of the phases. We analyse how the potential changes from the phase  $p_j$  to the phase  $p_{j+1}$ . Let's say the ball  $B_2 = (\mu_2, 3c_4\eta + 3c_5\varepsilon)$  is added at the beginning of  $p_{j+1}$  and  $B_1 = (\mu_1, 3c_4\eta + 3c_5\varepsilon)$  is the ball added at the beginning of  $p_j$ . Then the ball  $B_2' = (\mu_2, c_4\eta + c_5\varepsilon)$  and the ball  $B_1' = (\mu_1, c_4\eta + c_5\varepsilon)$  are disjoint. Otherwise,  $B_2' \subseteq B_1$  so  $B_2$  would not have been added by the algorithm. At the beginning of  $p_j$ ,  $B_1'$  has probability no less than 0.6 by Claim A.3. Therefore,  $B_2'$  has probability no more than 0.4. Similarly, at the beginning of  $p_{j+1}$ ,  $B_2'$  has probability at least 0.6 by Claim A.3. Since during one iteration the weight of a hypothesis cannot change too much, at iteration  $i_{j+1} - 1$ ,  $B_2'$  has weight at least 0.5 by picking  $\alpha$  small enough. Therefore, we have  $\log \lambda_{i_{j+1}-1}(B_2') - \log \lambda_{i_j}(B_2') \geq \log \frac{0.5}{0.4} \geq \frac{1}{5}$ . Moreover, note that  $S_i$  does not change from iteration  $i_j$  to iteration  $i_{j+1} - 1$  by the definition of phases. Now we compute
$$
\begin{array}{l} \sum_ {l = i _ {j}} ^ {i _ {j + 1} - 1} \Delta_ {l} = \log \frac {\lambda_ {i _ {j + 1} - 1} (h ^ {*})}{\lambda_ {i _ {j}} (h ^ {*})} + \log \frac {\lambda_ {i _ {j + 1} - 1 , H \setminus S _ {i _ {j}}} (h ^ {*})}{\lambda_ {i _ {j} , H \setminus S _ {i _ {j}}} (h ^ {*})}, \\ = \log \frac {w _ {i _ {j + 1} - 1} (h ^ {*})}{w _ {i _ {1}} (h ^ {*})} \frac {\sum_ {h \in H} w _ {i _ {1}} (h)}{\sum_ {h \in H} w _ {i _ {j + 1} - 1} (h)} + \log \frac {w _ {i _ {j + 1} - 1} (h ^ {*})}{w _ {i _ {j}} (h ^ {*})} \frac {w _ {i _ {j}} (H \setminus S _ {i _ {j}})}{w _ {i _ {j + 1} - 1} (H \setminus S _ {i _ {j}})}. \\ \end{array}
$$
The change of the weight of  $h^*$  is
$$
\frac {w _ {i _ {j + 1}} (h ^ {*})}{w _ {i _ {j}} (h ^ {*})} = e ^ {- \alpha \mathcal {E} _ {p _ {j}}},
$$
where  $\mathcal{E}_{p_j}$  is the number of errors  $h^*$  made in  $p_j$ . Consequently,
$$
\begin{array}{l} \sum_ {l = i _ {j}} ^ {i _ {j + 1} - 1} \Delta_ {l} = - 2 \alpha \mathcal {E} _ {p _ {j}} + \log \frac {\sum_ {h \in H} w _ {i _ {j}} (h)}{\sum_ {h \in H} w _ {i _ {j + 1} - 1} (h)} + \log \frac {w _ {i _ {j}} (H \setminus S _ {i _ {j}})}{w _ {i _ {j + 1} - 1} (H \setminus S _ {i _ {j}})} \\ \geq - 2 \alpha \mathcal {E} _ {p _ {j}} + \frac {1}{5}. \\ \end{array}
$$
The last step above comes from
$$
\log \frac {\sum_ {h \in H} w _ {i _ {j}} (h)}{\sum_ {h \in H} w _ {i _ {j + 1} - 1} (h)} \geq \log \frac {\sum_ {h \in B _ {2} ^ {\prime}} w _ {i _ {j + 1} - 1} (h)}{\sum_ {h \in B _ {2} ^ {\prime}} w _ {i _ {j}} (h)} \frac {\sum_ {h \in H} w _ {i _ {j}} (h)}{\sum_ {h \in H} w _ {i _ {j + 1} - 1} (h)} = \log \frac {\lambda_ {i _ {j + 1} - 1} \left(B _ {2} ^ {\prime}\right)}{\lambda_ {i _ {j}} \left(B _ {2} ^ {\prime}\right)} \geq \frac {1}{5},
$$
$$
\log \frac {w _ {i _ {j}} (H \setminus S _ {i _ {j}})}{w _ {i _ {j + 1} - 1} (H \setminus S _ {i _ {j}})} \geq 0
$$
because the weight  $w(h)$  only decreases. Summing over all phases  $j$  and we get
$$
\psi_ {i} \geq - 2 \alpha \mathcal {E} _ {i} + \frac {1}{5} \left(\mathcal {N} _ {i} - 1\right).
$$
Since  $i$  may not exactly be the end of a phase, the last phase may end early so we have  $\mathcal{N}_i - 1$  instead of  $\mathcal{N}_i$ . Rearrange and the proof finishes.
We have already bounded  $\psi_{i}$ , so we just need to bound  $\mathcal{E}_i$  in order to bound  $\mathcal{N}_i$  by the following lemma.
Lemma A.8. For every  $k$ , with probability at least  $1 - \delta$
$$
\mathcal {E} _ {k} \leq \frac {1}{\alpha} \left(\psi_ {k} + \sqrt {2} \log \frac {1}{\delta}\right).
$$
Proof. Let  $q$  be the query distribution at iteration  $i - 1$  and  $p(x)$  be the probability that  $x$  is corrupted by the adversary. Then the conditional expectation of  $\mathcal{E}_i - \mathcal{E}_{i-1}$  is
$$
\mathbb {E} \left[ \mathcal {E} _ {i} - \mathcal {E} _ {i - 1} | \mathcal {F} _ {i} \right] = \Pr_ {x \sim q} \left[ h ^ {*} (x) \text {i s w r o n g} \right] = \mathbb {E} _ {x \sim q} \left[ p (x) \right] = \mathbb {E} _ {x \sim \mathcal {D}} \left[ p (x) \frac {q (x)}{\mathcal {D} (x)} \right] \leq \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}.
$$
Then if  $h^* \notin S$ , from Lemma A.4
$$
\mathbb {E} \left[ \Delta_ {i} - 2 \alpha \left(\mathcal {E} _ {i} - \mathcal {E} _ {i - 1}\right) \mid \mathcal {F} _ {i} \right] \geq \mathbb {E} \left[ \Delta_ {i} \mid \mathcal {F} _ {i} \right] - 2 \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \gtrsim \frac {1}{m ^ {*}}.
$$
Therefore,  $\mathbb{E}[\alpha (\mathcal{E}_i - \mathcal{E}_{i - 1})|\mathcal{F}_i]\leq \frac{1}{2}\mathbb{E}[\Delta_i|\mathcal{F}_i]$  and  $\mathbb{E}[\Delta_i - \alpha (\mathcal{E}_i - \mathcal{E}_{i - 1})|\mathcal{F}_i]\geq \frac{1}{2}\mathbb{E}[\Delta_i|\mathcal{F}_i]$ . This means that  $\psi_{k} - \alpha \mathcal{E}_{k} - \frac{1}{2}\mu_{k}$  is a supermartingale. We then bound Var  $[\Delta_i - \alpha (\mathcal{E}_i - \mathcal{E}_{i - 1})|\mathcal{F}_i]$ . Note that  $|\Delta_{i} - \alpha (\mathcal{E}_{i} - \mathcal{E}_{i - 1})|\leq 2\alpha$ , so
$$
\operatorname {V a r} \left[ \Delta_ {i} - \alpha \left(\mathcal {E} _ {i} - \mathcal {E} _ {i - 1}\right) \mid \mathcal {F} _ {i} \right] \leq \mathbb {E} \left[ \left(\Delta_ {i} - \alpha \left(\mathcal {E} _ {i} - \mathcal {E} _ {i - 1}\right)\right) ^ {2} \mid \mathcal {F} _ {i} \right] \leq 2 \alpha \mathbb {E} \left[ \left| \Delta_ {i} - \alpha \left(\mathcal {E} _ {i} - \mathcal {E} _ {i - 1}\right) \right| \mid \mathcal {F} _ {i} \right].
$$
$$
\mathbb {E} \left[ | \Delta_ {i} - \alpha (\mathcal {E} _ {i} - \mathcal {E} _ {i - 1}) | | \mathcal {F} _ {i} \right] \leq \mathbb {E} \left[ | \Delta_ {i} | | \mathcal {F} _ {i} \right] + \alpha \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}.
$$