SlowGuess/NeurIPS_2023_OCR huggingface dataset:
for  $h^* \notin S_i$ , and  $\phi_i = 0$  for  $h^* \in S_i$ . We show that  $\phi_i$  grows by  $\Omega\left(\frac{1}{m^*}\right)$  in expectation in each iteration. Thus, as in the example of Figure 1, either  $\lambda(h^*)$  grows as a fraction of the whole distribution, or as a fraction of the "low-density" region.
If at any iteration we find that  $\overline{\lambda}$  has some heavy ball  $B(\mu, 2\eta + \varepsilon)$  so Lemma 2.1 would not apply, we add  $B(\mu', 6\eta + 3\varepsilon)$  to  $S$ , where  $B(\mu', 2\eta + \varepsilon)$  is the heaviest ball before capping. We show that this ensures that no small heavy ball exists in the capped distribution  $\overline{\lambda}$ . Expanding  $S$  only increases the potential function, and then the lack of heavy ball implies the potential will continue to grow.
Thus the potential (2) starts at  $-2\log |H|$ , and grows by  $\Omega \left(\frac{1}{m^{*}}\right)$  in each iteration. After  $O(m^{*}\log H)$  iterations, we will have  $\phi_{i}\geq 0$  in expectation (and with high probability by Freedman's inequality). This is only possible if  $h^{\ast}\in S$ , which means that one of the centers  $\mu$  of the balls added to  $S$  is a valid answer.
In fact, with some careful analysis we can show that with  $1 - \delta$  probability that one of the first  $O(\log \frac{H}{\delta})$  balls added to  $S$  is a valid answer. The algorithm can then check all the centers of these balls, using the following active agnostic learning algorithm:
Theorem 2.2. Active agnostic learning can be solved for  $\varepsilon = 3\eta$  with  $O\left(|H| \log \frac{|H|}{\delta}\right)$  samples.
Proof. The algorithm is the following. Take any pair  $h, h'$  with  $\| h - h' \| \geq 3\eta$ . Sample  $O\left(\log \frac{|H|}{\delta}\right)$  observations randomly from  $(x \sim \mathcal{D}_x \mid h(x) \neq h'(x))$ . One of  $h, h'$  is wrong on at least half the queries; remove it from  $H$  and repeat. At the end, return any remaining  $h$ .
To analyze this, let  $h^* \in H$  be the hypothesis with error  $\eta$ . If  $h^*$  is chosen in a round, the other  $h'$  must have error at least  $2\eta$ . Therefore the chance we remove  $h^*$  is at most  $\delta / |H|$ . In each round we remove a hypothesis, so there are at most  $|H|$  rounds and at most  $\delta$  probability of ever crossing off  $h^*$ . If we never cross off  $h^*$ , at the end we output some  $h$  with  $\| h - h^* \| \leq 3\eta$ , which gives  $\varepsilon = 3\eta$ .
The linear dependence on  $|H|$  makes the Theorem 2.2 algorithm quite bad in most circumstances, but the dependence only on  $|H|$  makes it perfect for our second stage (where we have reduced to  $O(\log |H|)$  candidate hypotheses).
Overall, this argument gives an  $O\left(m^{*}\log \frac{|H|}{\delta} +\log \frac{|H|}{\delta}\log \frac{\log|H|}{\delta}\right)$  sample algorithm for agnostic active learning. One can simplify this bound by observing that the set of centers  $C$  added by our algorithm form a packing, and must therefore all be distinguishable by the optimal algorithm, so  $m^{*}\geq \log C$ . This gives a bound of
$$
O \left(\left(m ^ {*} + \log \frac {1}{\delta}\right) \log \frac {| H |}{\delta}\right).
$$
By starting with an  $\eta$ -net of size  $N$ , we can reduce  $|H|$  to  $N$  with a constant factor increase in  $\eta$ .
With some properly chosen constants  $c_4$  and  $c_5$ , the entire algorithm is formally described in Algorithm 1.
Remark 1: As stated, the algorithm requires knowing  $m^*$  to set the target sample complexity / number of rounds  $k$ . This restriction could be removed with the following idea.  $m^*$  only enters the analysis through the fact that  $O\left(\frac{1}{m^*}\right)$  is a lower bound on the expected increase of the potential
function in each iteration. However, the algorithm knows a bound on its expected increase in each round  $i$ ; it is the value
$$
\tau_ {i} = \max  _ {q} \underset {x \sim q} {\mathbb {E}} [ \bar {r} _ {i, S _ {i}} (x) ] - \frac {c _ {4}}{2 0} \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)}.
$$
optimized in the algorithm. Therefore, we could use an adaptive termination criterion that stops at iteration  $k$  if  $\sum_{i=1}^{k} \tau_i \geq O(\log \frac{|H|}{\delta})$ . This will guarantee that when terminating, the potential will be above 0 with high probability so our analysis holds.
Remark 2: The algorithm's running time is polynomial in  $|H|$ . This is in general not avoidable, since the input is a truth table for  $H$ . The bottleneck of the computation is the step where the algorithm checks if the heaviest ball has mass greater than  $80\%$ . This step could be accelerated by randomly sampling hypothesis and points to estimate and find heavy balls; this would improve the dependence to nearly linear in  $|H|$ . If the hypothesis class has some structure, like the binary search example, the algorithm can be implemented more efficiently.
Algorithm 1 Competitive Algorithm for Active Agnostic Learning
Compute a \(2\eta\) maximal packing \(H^{\prime}\)   
Let \(w_{0} = 1\) for every \(h\in H^{\prime}\)   
\(S_0\gets \emptyset\)   
\(C\gets \emptyset\)   
for \(i = 1,\ldots ,k = O\left(m^{*}\log \frac{|H^{\prime}|}{\delta}\right)\) do Compute \(\lambda_i(h) = \frac{w_{i - 1}(h)}{\sum_{h\in H}w_{i - 1}(h)}\) for every \(h\in H\) if there exists \(c_{4}\eta +c_{5}\varepsilon\) ball with probability \(>80\%\) over \(\overline{\lambda}_{i,S_{i - 1}}\) then \(S_{i}\gets S_{i}\cup B(\mu^{\prime},3c_{4}\eta +3c_{5}\varepsilon)\) where \(B(\mu^{\prime},c_{4}\eta +c_{5}\varepsilon)\) is the heaviest radius \(c_{4}\eta +c_{5}\varepsilon\) ball over \(\lambda_{i}\) \(C\gets C\cup \{\mu '\}\) else \(\begin{array}{rl} & {\mathrm{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textit{~\textbf{~\textit{~\textbf{~\textit{~\textit{~\textbf{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textit{~\textiti}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
Generalization for Better Bounds. To get a better dependence for 1d threshold functions, we separate out the Lemma 2.1 bound on (1) from the analysis of the algorithm given a bound on (1). Then for particular instances like 1d threshold functions, we get a better bound on the algorithm by giving a larger bound on (1).
Theorem 2.3. Suppose that  $\mathcal{D}_x$  and  $H$  are such that, for any distribution  $\lambda$  over  $H$  such that no radius- $(c_4\eta + c_5\varepsilon)$  ball has probability more than  $80\%$ , there exists a distribution  $q$  over  $X$  such that
$$
\underset {x \sim q} {\mathbb {E}} [ r (x) ] - \frac {c _ {4}}{2 0} \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {x} (x)} \geq \beta
$$
for some  $\beta > 0$ . Then for  $\varepsilon \geq c_1 \eta$ ,  $c_4 \geq 300$ ,  $c_5 = \frac{1}{10}$  and  $c_1 \geq 90 c_4$ , let  $N = N(H, \mathcal{D}_x, \eta)$  be the size of an  $\eta$ -cover of  $H$ . Algorithm 1 solves  $(\eta, \varepsilon, \delta)$  active agnostic learning with  $O\left(\frac{1}{\beta} \log \frac{N}{\delta} + \log \frac{N}{\delta} \log \frac{\log N}{\delta}\right)$  samples.
Corollary 2.4. There exists a constant  $c_{1} > 1$  such that, for 1d threshold functions and  $\varepsilon > c_{1}\eta$ , Algorithm 1 solves  $(\eta, \varepsilon, \delta)$  active agnostic learning with  $O\left(\log \frac{1}{\varepsilon\delta}\log \frac{\log\frac{1}{\varepsilon}}{\delta}\right)$  samples.
Proof. Because the problem is only harder if  $\eta$  is larger, we can raise  $\eta$  to be  $\eta = \varepsilon / C$ , where  $C > 1$  is a sufficiently large constant that Theorem 2.3 applies. Then  $1d$  threshold functions have an  $\eta$ -cover of size  $N = O(1 / \varepsilon)$ . To get the result by Theorem 2.3, it suffices to show  $\beta = \Theta(1)$ .
Each hypothesis is of the form  $h(x) = 1_{x \geq \tau}$ , and corresponds to a threshold  $\tau$ . So we can consider  $\lambda$  to be a distribution over  $\tau$ .
Let  $\lambda$  be any distribution for which no radius-  $R$  with probability greater than  $80\%$  ball exists, for  $R = c_4\eta +c_5\varepsilon$ . For any percent  $p$  between 0 and 100, let  $\tau_{p}$  denote the pth percentile of  $\tau$  under  $\lambda$  (i.e., the smallest  $t$  such that  $\operatorname*{Pr}[\tau \leq t] \geq p / 100$ ). By the ball assumption,  $\tau_{10}$  and  $\tau_{90}$  do not lie in the same radius-  $R$  ball. Hence  $\| h_{\tau_{10}} - h_{\tau_{90}}\| > R$ , or
$$
\operatorname * {P r} _ {x} [ \tau_ {1 0} \leq x <   \tau_ {9 0} ] > R.
$$
We let  $q$  denote  $(\mathcal{D}_x \mid \tau_{10} \leq x < \tau_{90})$ . Then for all  $x \in \operatorname{supp}(q)$  we have  $r(x) \geq 0.1$  and
$$
\frac {q (x)}{D _ {x} (x)} = \frac {1}{\operatorname * {P r} _ {x \sim D _ {x}} [ x \in \operatorname {s u p p} (q) ]} <   \frac {1}{R}.
$$
Therefore we can set
$$
\beta = \mathop{\mathbb{E}}_{x\sim q}\left[ r (x)\right] - \frac{c_{4}}{20}\eta \max_{x}\frac{q(x)}{D_{x}(x)}\geq 0.1 - \frac{c_{4}\eta}{20(c_{4}\eta + c_{5}\varepsilon)}\gtrsim 1,
$$
as needed.
3 Proof of Lemma 2.1
Proof. WLOG, we assume that  $\operatorname{Pr}_{h\sim \lambda}[h(x) = 0] \geq \operatorname{Pr}_{h\sim \lambda}[h(x) = 1]$  for every  $x \in \mathcal{X}$ . This means  $r(x) = \mathbb{E}_{h\sim \lambda}[h(x)]$ . This can be achieved by flipping all  $h(x)$  and observations  $y$  for all  $x$  not satisfying this property; such an operation doesn't affect the lemma statement.
We will consider an adversary defined by a function  $g: X \to [0,1]$ . The adversary takes a hypothesis  $h \in H$  and outputs a distribution over  $y \in \{0,1\}^X$  such that  $0 \leq y(x) \leq h(x)$  always, and  $\operatorname{err}(h) = \mathbb{E}_{x,y}[h(x) - y(x)] \leq \eta$  always. For a hypothesis  $h$ , the adversary sets  $y(x) = 0$  for all  $x$  with  $h(x) = 0$ , and  $y(x) = 0$  independently with probability  $g(x)$  if  $h(x) = 1$  —unless  $\mathbb{E}_x[h(x)g(x)] > \eta$ , in which case the adversary instead simply outputs  $y = h$  to ensure the expected error is at most  $\eta$  always.
We consider the agnostic learning instance where  $x \sim \mathcal{D}_x$ ,  $h \sim \lambda$ , and  $y$  is given by this adversary. Let  $\mathcal{A}$  be an  $(\eta, \varepsilon)$  algorithm which uses  $m$  measurements and succeeds with  $99\%$  probability. Then it must also succeed with  $99\%$  probability over this distribution. For the algorithm to succeed on a sample  $h$ , its output  $\widehat{h}$  must have  $\| h - \widehat{h} \| \leq 2\eta + \varepsilon$ . By the bounded ball assumption, for any choice of adversary, no fixed output succeeds with more than  $80\%$  probability over  $h \sim \lambda$ .
Now, let  $\mathcal{A}_0$  be the behavior of  $\mathcal{A}$  if it observes  $y = 0$  for all its queries, rather than the truth;  $\mathcal{A}_0$  is independent of the input.  $\mathcal{A}_0$  has some distribution over  $m$  queries, outputs some distribution of answers  $\widehat{h}$ . Let  $q(x) = \frac{1}{m} \operatorname*{Pr}[\mathcal{A}_0 \text{ queries } x]$ , so  $q$  is a distribution over  $\mathcal{X}$ . Since  $\mathcal{A}_0$  outputs a fixed distribution, by the bounded ball assumption, for  $h \sim \lambda$  and arbitrary adversary function  $g$ ,
$$
\Pr_{h\sim \lambda}[\mathcal{A}_{0}\text{succeeds} ]\leq 80\% .
$$
But  $\mathcal{A}$  behaves identically to  $\mathcal{A}_0$  until it sees its first nonzero  $y$ . Thus,
$$
99 \% \leq \Pr [ A \text {succeeds} ] \leq \Pr [ A _ {0} \text {succeeds} ] + \Pr [ A \text {sees a non-zero } y ]
$$
$$
\Pr [ \mathcal{A} \text {sees a non-zero } y ] \geq 19 \%
$$
Since  $\mathcal{A}$  behaves like  $\mathcal{A}_0$  until the first nonzero, we have
$$
\begin{array}{l} 19 \% \leq \Pr [ \mathcal {A} \text {sees a non - zero } y ] \\ = \Pr \left[ \mathcal {A} _ {0} \text {m a k e s a q u e y} x \text {w i t h} y (x) = 1 \right] \\ \leq \mathbb {E} [ \text {N u m b e r q u e i e r s} x \text {b y} \mathcal {A} _ {0} \text {w i t h} y (x) = 1 ] \\ = m \underset {h \sim \lambda} {\mathbb {E}} \underset {y} {\mathbb {E}} \underset {x \sim q} {\mathbb {E}} [ y (x) ]. \\ \end{array}
$$
As an initial note, observe that  $\mathbb{E}_{h,y}[y(x)]\leq \mathbb{E}_h[h(x)] = r(x)$  so
$$
\underset {x \sim q} {\mathbb {E}} [ r (x) ] \geq \frac {0 . 1 9}{m}.
$$
Thus the lemma statement holds for  $\eta = 0$ .
Handling  $\eta > 0$ . Consider the behavior when the adversary's function  $g: X \to [0,1]$  satisfies  $\mathbb{E}_{x \sim \mathcal{D}_x}[g(x)r(x)] \leq \eta / 10$ . We denote the class of all adversary satisfying this condition as  $G$ . We have that
$$
\underset {h \sim \lambda} {\mathbb {E}} \left[ \underset {x \sim \mathcal {D} _ {x}} {\mathbb {E}} [ g (x) h (x) ] \right] = \underset {x \sim \mathcal {D} _ {x}} {\mathbb {E}} [ g (x) r (x) ] \leq \eta / 1 0.
$$
Let  $E_{h}$  denote the event that  $\mathbb{E}_{x\sim \mathcal{D}_x}[g(x)h(x)]\leq \eta$ , so  $\operatorname*{Pr}[\overline{E}_h]\leq 10\%$ . Furthermore, the adversary is designed such that under  $E_{h}$ ,  $\mathbb{E}_y[y(x)] = h(x)(1 - g(x))$  for every  $x$ . Therefore:
$$
\begin{array}{l} 0. 1 9 \leq \Pr [ \mathcal {A} _ {0} \text {m a k e s a q u e y} x \text {w i t h} y (x) = 1 ] \\ \leq \Pr [ \overline {{E}} _ {h} ] + \Pr [ \mathcal {A} _ {0} \text {m a k e s a q u e y} x \text {w i t h} y (x) = 1 \cap E _ {h} ] \\ \leq 0. 1 + \mathbb {E} [ \text {N u m b e r q u e i r e s} x \text {b y} \mathcal {A} _ {0} \text {w i t h} y (x) = 1 \text {a n d} E _ {h} ] \\ = 0. 1 + m \underset {h} {\mathbb {E}} \left[ \mathbb {1} _ {E _ {h}} \underset {x \sim q} {\mathbb {E}} \left[ \mathbb {E} y (x) \right] \right] \\ = 0. 1 + m \underset {h} {\mathbb {E}} \left[ \mathbb {1} _ {E _ {h}} \underset {x \sim q} {\mathbb {E}} [ h (x) (1 - g (x)) ] \right] \\ \leq 0. 1 + m \underset {x \sim q} {\mathbb {E}} [ \mathbb {E} _ {h} [ h (x) ] (1 - g (x)) ] \\ = 0. 1 + m \underset {x \sim q} {\mathbb {E}} [ r (x) (1 - g (x)) ]. \\ \end{array}
$$
$$
\max  _ {q} \min  _ {g \in G} \mathbb {E} _ {x \sim q} [ r (x) (1 - g (x)) ] \geq \frac {9}{1 0 0 m} \tag {4}
$$
over all distributions  $q$  and functions  $g:X\to [0,1]$  satisfying  $\mathbb{E}_{x\sim \mathcal{D}_x}[g(x)r(x)]\leq \eta /10$ . We now try to understand the structure of the  $q,g$  optimizing the LHS of (4).
Let  $g^{*}$  denote an optimizer of the objective. First, we show that the constraint is tight, i.e.,  $\mathbb{E}_{x\sim \mathcal{D}_x}[g^* (x)r(x)] = \eta /10$ . Since increasing  $g$  improves the constraint, the only way this could not happen is if the maximum possible function,  $g(x) = 1$  for all  $x$ , lies in  $G$ . But for this function, the LHS of (4) would be 0, which is a contradiction; hence we know increasing  $g$  to improve the objective at some point hits the constraint, and hence  $\mathbb{E}_{x\sim \mathcal{D}_x}[g^* (x)r(x)] = \eta /10$ .
For any  $q$ , define  $\tau_q \geq 0$  to be the minimum threshold such that
$$
\underset {x \sim \mathcal {D} _ {x}} {\mathbb {E}} \left[ r (x) \cdot 1 _ {\frac {q (x)}{\mathcal {D} _ {X} (x)} > \tau_ {q}} \right] <   \eta / 1 0.
$$
and define  $g_{q}$  by
$$
g _ {q} (x) := \left\{ \begin{array}{l l} 1 & \frac {q (x)}{\mathcal {D} _ {X} (x)} > \tau_ {q} \\ \alpha & \frac {q (x)}{\mathcal {D} _ {X} (x)} = \tau_ {q} \\ 0 & \frac {q (x)}{\mathcal {D} _ {X} (x)} <   \tau_ {q} \end{array} \right.
$$
where  $\alpha \in [0,1]$  is chosen such that  $\mathbb{E}_{x\sim \mathcal{D}_x}[r(x)g_q(x)] = \eta /10$ ; such an  $\alpha$  always exists by the choice of  $\tau_{q}$ .
For any  $q$ , we claim that the optimal  $g^{*}$  in the LHS of (4) is  $g_{q}$ . It needs to maximize
$$
\underset {x \sim \mathcal {D} _ {X}} {\mathbb {E}} \left[ \frac {q (x)}{\mathcal {D} _ {X} (x)} r (x) g (x) \right]
$$
subject to a constraint on  $\mathbb{E}_{x\sim \mathcal{D}_X}[r(x)g(x)]$ ; therefore moving mass to points of larger  $\frac{q(x)}{\mathcal{D}_X(x)}$  is always an improvement, and  $g_{q}$  is optimal.
We now claim that the  $q$  maximizing (4) has  $\max \frac{q(x)}{\mathcal{D}_X(x)} = \tau_q$ . If not, some  $x'$  has  $\frac{q(x')}{\mathcal{D}_X(x')} > \tau_q$ . Then  $g_q(x') = 1$ , so the  $x'$  entry contributes nothing to  $\mathbb{E}_{x \sim q}[r(x)(1 - g_q(x))]$ ; thus decreasing  $q(x)$  halfway towards  $\tau_q$  (which wouldn't change  $g_q$ ), and adding the savings uniformly across all  $q(x)$  (which also doesn't change  $g_q$ ) would increase the objective.
So there exists a  $q$  satisfying (4) for which  $\operatorname*{Pr}\left[\frac{q(x)}{\mathcal{D}_X(x)} >\tau_q\right] = 0$ , and therefore the set  $T = \left\{x\mid \frac{q(x)}{\mathcal{D}_X(x)} = \tau_q\right\}$  satisfies  $\mathbb{E}_{\mathcal{D}_X}[r(x)\mathbb{1}_{x\in T}]\geq \eta /10$  and a  $g_{q}$  minimizing (4) is
$$
g _ {q} (x) = \frac {\eta}{1 0} \frac {\mathbb {1} _ {x \in T}}{\mathbb {E} _ {\mathcal {D} _ {X}} [ r (x) \mathbb {1} _ {x \in T} ]}.
$$
$$
\begin{array}{l} \underset {x \sim q} {\mathbb {E}} \left[ r (x) g _ {q} (x) \right] = \underset {x \sim \mathcal {D} _ {X}} {\mathbb {E}} \left[ \frac {q (x)}{\mathcal {D} _ {X} (x)} r (x) \frac {\eta}{1 0} \frac {\mathbb {1} _ {x \in T}}{\mathbb {E} _ {\mathcal {D} _ {X}} [ r (x) \mathbb {1} _ {x \in T} ]} \right] \\ \leq \frac {\eta}{1 0} \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \\ \end{array}
$$
and so by (4),
$$
\underset {x \sim q} {\mathbb {E}} [ r (x) ] - \frac {\eta}{1 0} \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \geq \frac {9}{1 0 0 m}
$$
as desired.
4 Conclusion
We have given an algorithm that solves agnostic active learning with (for constant  $\delta$ ) at most an  $O(\log |H|)$  factor more queries than the optimal algorithm. It is NP-hard to improve upon this  $O(\log |H|)$  factor in general, but for specific cases it can be avoided. We have shown that 1d threshold functions, i.e. binary search with adversarial noise, is one such example where our algorithm matches the performance of disagreement coefficient-based methods and is within a log  $\log \frac{1}{\varepsilon}$  factor of optimal.
5 Acknowledgments
Yihan Zhou and Eric Price were supported by NSF awards CCF-2008868, CCF-1751040 (CAREER), and the NSF AI Institute for Foundations of Machine Learning (IFML).
Mohammad Azad, Igor Chikalov, Shahid Hussain, Mikhail Moshkov, and Beata Zielosko. Decision Trees with Hypotheses. Springer International Publishing, 2022. doi: 10.1007/978-3-031-08585-7. URL https://doi.org/10.1007/978-3-031-08585-7.
Maria-Florina Balcan, Alina Beygelzimer, and John Langford. Agnostic active learning. In Proceedings of the 23rd international conference on Machine learning, pages 65-72, 2006.
Michael Ben-Or and Avinatan Hassidim. The bayesian learner is optimal for noisy binary search (and pretty good for quantum as well). In 2008 49th Annual IEEE Symposium on Foundations of Computer Science, pages 221-230, 2008. doi: 10.1109/FOCS.2008.58.
David D Lewis and Jason Catlett. Heterogeneous uncertainty sampling for supervised learning. In Machine learning proceedings 1994, pages 148-156. Elsevier, 1994.
Robert Nowak. Generalized binary search. In 2008 46th Annual Allerton Conference on Communication, Control, and Computing, pages 568-574. IEEE, 2008.
Robert D Nowak. The geometry of generalized binary search. IEEE Transactions on Information Theory, 57(12):7893-7906, 2011.
Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009.
Joel Tropp. Freedman's inequality for matrix martingales. 2011.
Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
A Query Complexity Upper Bound
In this section we present the whole proof of the query complexity upper bound of Algorithm 1, as stated in Theorem 1.1.
A.1 Notation
We remind the readers about some definitions first. Remember that  $w_{i}(h)$  denote the weight of hypothesis  $h$  in iteration  $i$  and  $\lambda_{i,S}(h) = \frac{w_i(h)}{\sum_{h' \in S} w_i(h')}$  for some  $S \subseteq H$  denote the proportion of  $h$  in  $S$ . We view  $\lambda_{i,S}$  as a distribution of hypotheses in  $S$  so for  $h \notin S$ ,  $\lambda_{i,S}(h) = 0$ . For a set  $S \subseteq H$  of hypotheses, we define  $w_{i}(S) := \sum_{h \in S} w(h)$  and  $\lambda_{i}(h) = \lambda_{i,H}(h)$ .
Define  $r_{\lambda,h^*}(x) \coloneqq \operatorname{Pr}_{h \sim \lambda}[h(x) \neq h^*(x)]$ , and  $r_\lambda(x) = \min_{y \in \{0,1\}} \operatorname{Pr}_{h \sim \lambda}[h(x) \neq y]$ , so  $r_\lambda(x) = \min(r_{\lambda,h^*}(x), 1 - r_{\lambda,h^*}(x))$ .
$$
\bar {\lambda} _ {i, S} (h) := \frac {1}{2} \lambda_ {i} (h) + \frac {1}{2} \lambda_ {i, H \backslash S} (h) = \left\{ \begin{array}{l l} \frac {1}{2} \lambda_ {i} (h) & h \in S \\ \lambda_ {i} (h) \cdot \frac {1 - \frac {1}{2} \Pr_ {h \sim \lambda_ {i}} [ h \in S ]}{1 - \Pr_ {h \sim \lambda_ {i}} [ h \in S ]} & h \notin S \end{array} \right. \tag {5}
$$
as the "capped" distribution in iteration  $i$ .
Finally, for notational convenience define  $r_{i,S} \coloneqq r_{\lambda_{i,S}}$ ,  $r_{i,S,h} \coloneqq r_{\lambda_{i,S},h}$  and  $\bar{r}_{i,S} \coloneqq r_{\overline{\lambda}_{i,S}}$ .
The main focus of our proof would be analyzing the potential function
$$
\phi_ {i} (h ^ {*}) = \left\{ \begin{array}{l l} \log \lambda_ {i} (h ^ {*}) + \log \lambda_ {i, H \setminus S _ {i}} (h ^ {*}) & h ^ {*} \notin S _ {i} \\ 0 & h ^ {*} \in S _ {i}, \end{array} \right.
$$
where  $h^*$  is the best hypothesis in  $H$ . We would like to show that  $\phi_{i+1}(h^*) - \phi_i(h^*)$  is growing at a proper rate in each iteration. We pick  $S_i$  to be an expanding series of sets, i.e.,  $S_i \subseteq S_{i+1}$  for any  $i \geq 1$ . However, the change of the "capped" set  $S_i$  makes this task challenging. Therefore, we instead analyze the following quantity defined as
$$
\Delta_ {i} (h ^ {*}) := \left\{ \begin{array}{l l} \log \frac {\lambda_ {i + 1} (h ^ {*})}{\lambda_ {i} (h ^ {*})} + \log \frac {\lambda_ {i + 1 , H \setminus S _ {i}} (h ^ {*})}{\lambda_ {i , H \setminus S _ {i}} (h ^ {*})} & h ^ {*} \notin S _ {i} \\ 0 & h ^ {*} \in S _ {i}, \end{array} \right.
$$
and  $\phi_{i + 1}(h^{*}) - \phi_{i}(h^{*}) = \Delta_{i}(h^{*}) + \log \frac{\lambda_{i + 1,H\setminus S_{i + 1}}(h^{*})}{\lambda_{i + 1,H\setminus S_{i}}(h^{*})}$  if  $h^*\notin S_{i + 1}$ . Further, we define  $\psi_k(h^*)\coloneqq \sum_{i <   k}\Delta_i(h^*)$  so by definition  $\phi_k(h^*) = \phi_0(h^*) + \psi_k(h^*) + \sum_{i <   k}\log \frac{\lambda_{i + 1,H\setminus S_{i + 1}}(h^*)}{\lambda_{i + 1,H\setminus S_{i}}(h^*)}$  if  $h^*\notin S_{i + 1}$ . In the following text, we will drop the parameter  $h^*$  when the context is clear and just use  $\phi_i,\Delta_i$  and  $\psi_{i}$  instead.
A.2 Potential Growth
We will lower bound the conditional per iteration potential increase by first introducing a lemma that relates the potential change to the optimization problem (3).
Lemma A.1. Assume that  $\mathrm{err}(h^{*})\leq \eta$  , then for any set  $S$  of hypotheses containing  $h^*$  and query distribution  $q$  , we have
$$
\mathbb {E} \left[ \log \frac {\lambda_ {i + 1 , S} (h ^ {*})}{\lambda_ {i , S} (h ^ {*})} \Bigg | \mathcal {F} _ {i} \right] \geq 0. 9 \alpha \left(\underset {x \sim q} {\mathbb {E}} [ r _ {i, S, h} (x) ] - 2. 3 \eta \max  _ {x} \frac {q (x)}{D _ {X} (x)}\right)
$$
for  $\alpha \leq 0.2$ . Moreover,
$$
\mathbb {E} \left[ \max  \left\{0, \log \frac {\lambda_ {i + 1 , S} \left(h ^ {*}\right)}{\lambda_ {i , S} \left(h ^ {*}\right)} \right\} \mid \mathcal {F} _ {i} \right] \leq \alpha \underset {x \sim q} {\mathbb {E}} [ r _ {i, S, h ^ {*}} (x) ].
$$
Proof. For notational convenience, define  $\widetilde{r}(x) \coloneqq r_{i,S,h^*}(x)$ .
Observe that
$$
\frac {\lambda_ {i , S} (h ^ {*})}{\lambda_ {i + 1 , S} (h ^ {*})} = \frac {w _ {i} (h ^ {*})}{w _ {i + 1} (h ^ {*})} \frac {\sum_ {h \in S} w _ {i + 1 , S} (h)}{\sum_ {h \in S} w _ {i , S} (h)} = \frac {w _ {i} (h ^ {*})}{w _ {i + 1} (h ^ {*})} \underset {h \sim \lambda_ {i, S}} {\mathbb {E}} \left[ \frac {w _ {i + 1 , S} (h)}{w _ {i , S} (h)} \right].
$$