SlowGuess/NeurIPS_2023_OCR huggingface dataset:
in [DK14] give a nearly-linear time algorithm that given an upper bound on  $\mathrm{OPT} \leq \Gamma$ , returns  $\hat{H}$  such that  $\| \hat{H} - P \|_{\mathrm{TV}} \leq \alpha \cdot \Gamma + \epsilon$  for some large constant  $\alpha = 512$ . Acharya et al. in [AJOS14, AFJ+18] provide a  $O(n \log n)$  time algorithm that finds a hypothesis with accuracy guarantee of  $\alpha = 9$ . For proper learners, Bousquet et al. in [BKM19] showed that the best  $\alpha$  in Equation 1 that one can hope for is  $\alpha = 3$  as long as the sample complexity does not grow with the domain size  $|\mathcal{X}|$ . Aamand et al. in [AAC+23] also study statistical computational tradeoffs (time-data) for the hypotheses over discrete domains. Unlike our work, their setting allows pre-processing of the class  $\mathcal{H}$  in polynomial time.
There is a long history of research focusing on a special case of this problem where  $\mathcal{H}$  is a specific structured class of distributions such as mixtures of Gaussians [KMV12, DK14, SOAJ14, KSS18, ABM18,  $\mathrm{ABH^{+}20}$ ], histograms [Pea95, CDSS14, CDKL22], and polynomials [ADLS17]. The abstract hypothesis selection we study here is commonly used as a subroutine in solving these problems (usually in conjunction with some sort of a cover method). For a survey of results, see [Dia16].
Lastly, there is another field of study that tackles a question similar to ours: the problem of sorting items with noisy comparisons. One can view hypothesis selection as the task of finding the minimum item. With the exception of  $\left[\mathrm{AFJ}^{+}18\right]$  that we have discussed above, these probabilistic noise models do not capture the geometric structure presenting in our problem. Therefore, we did not find any of these result yielding an immediate solution to our problem.
Let  $[n]$  denote the set  $\{1,2,\ldots ,n\}$ . Suppose that we have a probability distribution  $P$  over a domain  $\mathcal{X}$ . For element  $x$  in  $\mathcal{X}$ ,  $P(x)$  denotes the probability of  $x$ . We assume  $\mathcal{X}$  is the domain of all the probability distributions in this article. For any subset of the domain  $S$ ,  $P(S)$  denotes the sum of the probabilities of all the elements in  $S$ . We denote the total variation distance between two distributions  $P_{1}$  and  $P_{2}$  by  $\| P_1 - P_2\|_{\mathrm{TV}}$ , and it is defined as  $\max_{S\subset \mathcal{X}}|P_1(S) - P_2(S)|$  where  $S$  is any measurable subset of the domain. We say  $P$  is  $\epsilon$ -close to  $Q$  iff  $\| \bar{P} -Q\|_{\mathrm{TV}}$  is at most  $\epsilon$ . Also, we say  $P$  is  $\epsilon$ -far from  $Q$  iff  $\| P - Q\|_{\mathrm{TV}}$  is greater than  $\epsilon$ . We denote the Scheffé set of two distributions as follows:  $S(H_{1},H_{2})\coloneqq \{x\in \mathcal{X}\mid H_{1}(x) < H_{2}(x)\}$ .
2.2 Basic tools
In this section, we focus on our primary tools we have used throughout this article. First, we start with a comparisons algorithm that allows us to compare two hypotheses. Next, we use the fact that one can estimate the probabilities of the  $O(n^2)$  many Scheffé sets up to error  $c \cdot \epsilon$  (for some small constant  $c < 1$ ) with probability at least  $1 - \delta$  using  $O(\log (n / \delta) / \epsilon^2)$  samples from  $P$ .
2.2.1 Comparing two hypotheses
Our algorithms works based on a basic operation that allows us to compare two hypotheses  $H_{1}$  and  $H_{2}$  based on the probabilities of their Scheffé sets. We pick the hypothesis that appears to be closer to  $P$  and declare it the winner (and the other hypothesis the loser). The challenging part is that these comparisons are not ideal. As in, even when our estimations of the probabilities of the Scheffé sets are accurate enough, we may pick a hypothesis that is further. However, one can guarantee that if the distance of the further hypothesis among  $H_{1}$  and  $H_{2}$  is much worse than the distance of the closer one, the comparison procedure would never select it as the winner. The algorithm is given three parameters: 1)  $\Gamma$ : an upper bound for OPT; 2)  $\epsilon$ : indicating the estimation error for the Scheffé set probabilities is  $\Theta(\epsilon)$ ; and 3) the confidence parameter  $\delta$ . Upon invoking this algorithm, it determines the result of the comparison with the guarantees formalized in the following lemma:
Lemma 2.1. Upon receiving three parameters:  $\Gamma$ ,  $\epsilon$ , and  $\delta$ , Algorithm 1 uses  $O(\log(1/\delta)/\epsilon^2)$  samples and satisfies the following guarantees with probability at least  $1 - \delta$ :
- If  $H_{1}$  is  $\Gamma$ -close to  $P$ , then the algorithm returns  $H_{1}$ .
- If  $H_{1}$  is  $(\max(\Gamma, \|H_{2} - P\|_{TV}) + 2 \|H_{2} - P\|_{TV} + \epsilon)$ -far from  $P$ , then the algorithm returns  $H_{2}$ .
Algorithm 1 Choosing between two hypotheses
1: procedure COMPARE  $(H_{1}, H_{2}, \Gamma, \epsilon, \delta$ , sample access to  $P$   
2: Draw  $m = \Theta(\log(1/\delta)/\epsilon^{2})$  samples from  $P$   
3:  $\hat{q} \gets$  fraction of samples from  $P$  in  $S(H_{1}, H_{2})$  using the PDF-comparator.  
4:  $p_{1} \gets H_{1}$  ( $S(H_{1}, H_{2})$ ).  
5:  $p_{2} \gets H_{2}$  ( $S(H_{1}, H_{2})$ ).  
6:  $\epsilon' \gets \epsilon/2$   
7: if  $|p_{1} - \hat{q}| \leq \Gamma + \epsilon'$  or  $|p_{1} - \hat{q}| \leq |p_{1} - \hat{q}|$  then  
8: Output  $H_{1}$ .  
9: else  
10: Output  $H_{2}$ .
Remark 3. While we have assumed that we have access to the probabilities of the Scheffé sets in the above algorithm, this assumption is not necessary. In fact, one can obtain similar guarantees to Lemma 2.1 as long as we have estimates of these probabilities up to accuracy  $c \cdot \epsilon$  for a sufficiently small constant  $c < 1$ . Hence, our result can be adjusted to the setting that we have sample access to  $H_{i}$ 's instead.
Our terminology based on comparisons: We say a distribution  $H$  is excellent iff  $\| H - P\|_{\mathrm{TV}}$  is  $\Gamma$ . We usually use  $H^{*}$  to denote an excellent hypothesis. We say a distribution  $H$  is decent iff  $\| H - P\|_{\mathrm{TV}}$  is greater than  $\Gamma$ , but smaller than  $3\Gamma +\epsilon$ . It implies that a decent hypothesis may win an excellent  $H^{*}\in \mathcal{H}$ . We say a distribution  $H$  is acceptable iff it is excellent or decent. And, we say a distribution  $H$  is unacceptable iff it is not acceptable. Note that it is impossible for an unacceptable hypothesis to win an excellent hypothesis with probability greater than  $\delta$ . Throughout this article, when we say valid comparisons, we refer to an event in which the Scheffé sets are estimated accurately, so the conditions in Lemma 2.1 hold. Since we have only  $O(n^2)$  many Scheffé sets, using  $O(\log (n / \delta) / \epsilon^2)$  samples is enough to assume all the comparisons are valid with a probability of at least  $1 - \delta$ .
2.2.2 Space efficient sample summaries for comparisons within a batch of hypotheses
In order to obtain sample efficiency, we need sufficient information about samples that enables us to reuse the same set of samples for multiple comparisons. Below, we describe two approaches to summarize the sample set.
Scheffé counts. Observe that to compare two hypotheses  $H_{1}$  and  $H_{2}$ , we only use the count of the number of samples in their Scheffé set  $S(H_{1},H_{2})$ . Hence, instead of storing samples directly, for every pair of hypotheses in  $\mathcal{H}$ , we store the number of samples in their Scheffé set. We refer to this number as the Scheffé counts. If we have  $m$  samples, we can store all the  $O(n^{2})$  Scheffé counts using  $O(n^{2}\log m)$  bits.
Storing each sample via a sorted list. In this approach, we store a succinct summary of each sample that allows us to infer its membership in each Scheffé set. This information suffices to perform the comparisons we need later. To store a sample  $x$ , we save an ordered list of hypotheses that is sorted according to the probability of  $x$  (the PDF at point  $x$ ). That is, we store a list of indices  $i_1, i_2, \ldots, i_n$  such that  $H_{i_1}(x) \leq H_{i_2}(x) \leq \dots \leq H_{i_n}(x)$  (use hypothesis indices to break the tie). The summary of a single sample can be computed in  $O(n \log n)$  time and takes  $O(n \log n)$  bits of space to store. Now, to compare the PDF of two hypotheses  $H_i$  and  $H_j$ , we simply can find  $i$  and  $j$  in the list and check whether  $i$  precedes  $j$  in the list. Alternatively if we want to compare PDF's faster, we can store the indices in another list called  $d_x$  that maps indices to list positions. We set  $d_x[i_\ell] = \ell$  for every  $\ell \in [n]$ . In this case, to compare the PDF's, we can simple check if  $d_x[i] < d_x[j]$ .
2.2.3 All-go-against-all tournament
There are standard techniques in the literature to solve hypothesis selection problem when the estimates of all the Scheffé sets are available. See Chapter 6 in [DL01] and [MS08]. These approaches
The proof of this lemma is provided in Section D.
It is worth noting that we may not answer the PDF-comparison correctly when  $H_{i}(x) = H_{j}(x)$ . However, since  $x$  does not contribute to the discrepancy between  $H_{i}$  and  $H_{j}$ , its inclusion (or exclusion) to the Scheffé set is inconsequential.
Algorithm 2 Random ladder tournament
1: procedure RANDOM LADDER TOURNAMENT  $(\mathcal{H},p_0,\Gamma ,\epsilon$  sample access to  $P$  and  $\mathcal{D}$    
2:  $k\gets [2000 / p_0]$    
3:  $Q\gets$  an empty list.   
4:  $W_{0}\leftarrow \emptyset$ $\triangleright$  Assume  $W_{0}$  is a hypothesis that always loses.   
5: for  $i = 1,2,\ldots ,k$  do   
6:  $R_{i}\gets$  a random hypothesis drawn from  $\mathcal{D}$    
7:  $W_{i}\gets$  COMPARE  $(W_{i - 1},R_i,\Gamma ,\epsilon ,1 / (100k))$    
8: Add  $W_{i}$  to  $Q$  with probability  $p_0$    
9: Add  $W_{k}$  to  $Q$    
10: Output a random hypothesis in  $Q$
suggest that we only need accurate estimation of the probability that  $P$  assigns to  $O(|\mathcal{H}|^2) = O(n^2)$  sets, therefore, the sample complexity can be proportionate to  $O(\log n)$  instead of  $O(|\mathcal{X}|)$ . For a formal statement, see Fact D.1.
3 Random ladder tournament
In this section, we focus on the Random ladder tournament which is a proper learner (with promise) of  $P$  in a finite class of  $n$  hypotheses,  $\mathcal{H}$ . The algorithm is given a parameter  $\Gamma$  as an upper bound for  $\mathrm{OPT}(\mathcal{H}, X)$ . It outputs  $\hat{H} \in \mathcal{H}$  such that, with high probability we have:  $\| \hat{H} - P \|_{\mathrm{TV}} \leq 3\Gamma + \epsilon$ . Bousquet et al. in [BKM19] have shown that no proper learner can achieve an error better than  $3\mathrm{OPT} + \epsilon$  unless the sample complexity grows with the domain size  $|\mathcal{X}|$ . Their lower bound holds even when the algorithm is provided with  $\Gamma = \mathrm{OPT}$ . Hence, the factor  $\alpha = 3$  above is optimal.
For clarity in our presentation, in Section 3.1, we present the random ladder tournament assuming that: 1) We set the confidence parameter to be a small constant ( $\delta = 0.1$ ). 2) the algorithm can call the COMPARE subroutine without specifying how this subroutine is implemented. In Section A.1, we show one can modify this algorithm to work for arbitrary small confidence parameter  $\delta$ . Also, in Section 3.2, we discuss how one can implement this algorithm with memory constraints. We use the sample summary approaches described in Section 2.2.2, and we obtain two (sub-optimal) memory-data tradeoffs for this algorithm.
3.1 Random ladder tournament with no memory constraints
In this section, we present the random ladder tournament that solves the hypothesis selection problem while a parameter  $\Gamma$  is given to the algorithm as an upper bound for OPT. For this result, we assume there is a meta distribution over  $\mathcal{H}$ , denoted by  $\mathcal{D}$ , with a non-negligible probability  $p_0$  to draw a hypothesis that is OPT-close. The algorithm considers hypotheses one at a time where at step  $i$ , we draw  $R_i$  from the meta distribution. While later we use this algorithm with other meta distributions, it may help the reader to view the meta distribution as a uniform distribution over a set of  $n$  hypotheses,  $\mathcal{H}$ . In this case, if we draw a random hypothesis from  $\mathcal{H}$ , it is guaranteed to be  $\Gamma$ -close to  $P$  with probability  $p_0 \geq 1/n$ . Our algorithm finds a sufficiently close hypothesis using  $\Theta(1/p_0)$  comparisons.
At a high-level, our algorithm goes through a list of randomly drawn hypotheses from  $\mathcal{D}$ , compares them, and keeps track of the current winner hypothesis. We denote the current winner hypothesis at step  $i$  by  $W_{i}$ . Initially, we start with  $W_{0}$  being equal to a fake hypothesis that loses to any other hypothesis. Then at every step, we take a randomly drawn hypothesis  $R_{i}$  and compare it with  $W_{i-1}$ . We set the current winner of step  $i$ ,  $W_{i}$ , be the winner of a comparison between  $W_{i-1}$  and  $R_{i}$ . We show that after  $k = \Theta(1/p_{0})$  steps either the final winner,  $W_{k}$ , is an  $\Gamma$ -close hypotheses, or many of  $W_{i}$ 's that we have encountered are close to  $P$ . That is, either  $W_{k}$  is an excellent hypothesis or a random  $W_{i}$  is an acceptable choice. To exploit this fact at every step, we add each  $W_{i}$  to a list, namely  $Q$ , with some small probability. At the end, we also add  $W_{k}$  to  $Q$ . We prove a random hypothesis in  $Q$  will be close to  $P$  with high probability. Algorithm 2 shows the formal description of our approach, and we prove its performance in Theorem 4. Figure 1 illustrates a visual representation of the algorithm.
Theorem 4. Suppose we can draw i.i.d. random hypotheses from an arbitrary meta distribution  $\mathcal{D}$  over  $\mathcal{H}$ . And, we have a hypothesis  $P$  that we aim to learn properly in  $\mathcal{H}$ . Assume we are given parameters  $p_0$  and  $\Gamma$  such that the probability that a random hypothesis is  $\Gamma$ -close to  $P$  is at least  $p_0$ . For any  $\epsilon \in (0,1)$ , Algorithm 2 is  $(\alpha = 3,\epsilon,\delta = 0.1)$ -proper learner (with promise) for the class  $\mathcal{H}$ .
Proof. First, note that each comparison, invoked in Line 7, will satisfy the properties of Lemma 2.1 with probability at least  $1 - 1 / (100k)$ . Hence, using Lemma 2.1 and the union bound, one can assume with probability 0.99 for all the comparisons we have:
- If  $W_{i}$  is  $\Gamma$ -close to  $P$ , it will not lose; it remains as the current winner for the rest of the steps:  $W_{i} = W_{i + 1} = \dots = W_{k}$ .
- If  $W_{i - 1}$  is  $(3\Gamma + \epsilon)$ -far from  $P$ , and  $R_i$  is  $\Gamma$ -close to  $P$ , then  $W_i$  is equal to  $R_i$ .
For the rest of this proof, we fix the response of comparisons for which the above conditions hold. The randomness used in the rest of this proof only depends on the randomness in the meta distribution over hypotheses and the internal coin tosses of the algorithm.
Our goal is to show that most of the hypotheses in the list,  $Q$ , are acceptable, so a randomly selected hypothesis in  $Q$  will be acceptable as well. Our first step is to show that the expected number of acceptable hypotheses in  $Q$  is high compared to the size of  $Q$ . For each  $i$  in  $[k]$ , we define an indicator random variable  $I_{i}$  that is one if we add an acceptable hypothesis to the  $Q$  at step  $i$  in Line 8, and zero otherwise. Also, we define another indicator variable  $I_{k+1}$  corresponding to the event that the final  $W_{k}$ , which we added to  $Q$  in Line 9, is an acceptable hypothesis. More formally, we have:
$$
I _ {i} := \mathbb {1} _ {W _ {i} \text {i s a c c e p t a b l e , a n d w e a d d W _ {i} t o Q i n L i n e 8 .}} \quad \forall i \in [ k ],
$$
$$
I _ {k + 1} := \mathbb {1} _ {W _ {k} \text {i s a c c e p t a b l e}}.
$$
Clearly, the sum of  $I_{i}$ 's indicates the number of acceptable hypotheses in  $Q$ , so we focus on finding a lower bound for the expected value of this quantity. Let  $\alpha_{i}$  be the probability of  $W_{i}$  being acceptable. Without loss of generality, we set  $\alpha_{0}$  equal to zero. It is not hard to see that:  $\mathbf{E}[I_i] = p_0\cdot \alpha_i$  for  $i\in [k]$ . Moreover for the last indicator variable, we have:
$$
\begin{array}{l} \mathbf {E} \left[ I _ {k + 1} \right] = \Pr \left[ W _ {k} \text {b e i n g a c c o p t a b l e} \right] \geq \Pr \left[ \| W _ {k} - P \| _ {\mathrm {T V}} \leq \Gamma \right] \\ = \sum_ {i = 1} ^ {k} \Pr \left[ W _ {i - 1} \text {l o s e s t o} R _ {i}, \text {a n d} \| R _ {i} - P \| _ {\mathrm {T V}} \leq \Gamma , \text {a n d} R _ {i} \text {d o e s n o t l o s e t o} R _ {i + 1}, R _ {i + 2}, \dots , R _ {k} \right]. \\ \end{array}
$$
Assuming the pairwise comparisons are done perfectly,  $R_{i}$  being  $\Gamma$ -close to  $P$ , automatically implies that  $R_{i}$  does not lose to any of the hypotheses  $R_{i+1}, \ldots, R_{k}$ . Thus, we have:
$$
\begin{array}{l} \mathbf {E} \left[ I _ {k + 1} \right] \geq \sum_ {i = 1} \Pr \left[ W _ {i - 1} \text {l o s e s t o} R _ {i}, \text {a n d} \| R _ {i} - P \| _ {\mathrm {T V}} \leq \Gamma \right] \\ = \sum_ {i = 1} ^ {k} \mathbf {P r} \left[ W _ {i - 1} \text {l o s e s t o} R _ {i} \mid \| R _ {i} - P \| _ {\mathrm {T V}} \leq \Gamma \right] \cdot \mathbf {P r} \left[ \| R _ {i} - P \| _ {\mathrm {T V}} \leq \Gamma \right]. \\ \end{array}
$$
Note that the probability of  $\| R_i - P \|_{\mathrm{TV}} \leq \Gamma$  is at least  $p_0$ . Now given that  $R_i$  is  $\Gamma$ -close to  $P$ , it will certainly win any hypotheses that is  $(3\Gamma + \epsilon)$ -far from  $P$ . Thus, the event that  $W_i$  is  $(3\Gamma + \epsilon)$ -far must have a lower probability than the event that  $W_{i-1}$  loses to  $R_i$ . Therefore, we obtain the following lower bound:
$$
\begin{array}{l} \mathbf {E} \left[ I _ {k + 1} \right] \geq \sum_ {i = 1} ^ {k} \Pr \left[ \| W _ {i - 1} - P \| _ {\mathrm {T V}} > (3 \Gamma + \epsilon) \mid \| R _ {i} - P \| _ {\mathrm {T V}} \leq \Gamma \right] \cdot p _ {0} \\ = \sum_ {i = 1} ^ {k} \Pr \left[ W _ {i - 1} \text {b e i n g u n a c c e p t a b l e} \mid \| R _ {i} - P \| _ {\mathrm {T V}} \leq \Gamma \right] \cdot p _ {0}. \\ \end{array}
$$
Recall that we pick  $R_{i}$  independently from all the previous hypothesis,  $R_{1}, R_{2}, \ldots, R_{i-1}$ , so one can say  $W_{i-1}$  is independent of  $R_{i}$ . This implies:
$$
\mathbf {E} \left[ I _ {k + 1} \right] \geq \sum_ {i = 1} ^ {k} \Pr \left[ W _ {i - 1} \text {b e i n g u n a c c e p t a b l e} \right] \cdot p _ {0} = \sum_ {i = 1} ^ {k} \left(1 - \alpha_ {i - 1}\right) \cdot p _ {0}.
$$
Putting it all together, the expected value of the sum of  $I_{i}$ 's is:
$$
\mathbf {E} \left[ \sum_ {i = 1} ^ {k + 1} I _ {i} \right] \geq \sum_ {i = 1} ^ {k} p _ {0} \cdot \alpha_ {i} + (1 - \alpha_ {i - 1}) \cdot p _ {0} = k \cdot p _ {0} + p _ {0} \cdot (\alpha_ {k} - \alpha_ {0}) \geq k \cdot p _ {0}.
$$
The last inequality above is due to the fact that we set  $\alpha_0$  to zero. Note that the above equation states that the expected number of acceptable hypotheses in  $Q$  is at least  $k\cdot p_0$ . On the other hand, the expected number of hypotheses in  $Q$  is  $k\cdot p_0 + 1$ . Thus, the expected number of unacceptable hypotheses in  $Q$  is at most one. Now, by Markov's inequality, the probability that we have more than 50 unacceptable hypotheses in  $Q$  is at most 0.02. And, by the Chernoff bound, we know that the probability of having less than 1000 many hypothesis in  $Q$  is at most:
$$
\Pr [ \# \text {h y p o t h e s e s i n} Q <   1 0 0 0 ] = \Pr \left[ \operatorname {B i n} (k, p _ {0}) <   \frac {k \cdot p _ {0}}{2} \right] \leq \exp \left(- \frac {k \cdot p _ {0}}{8}\right) \leq 1 0 ^ {- 1 0 0}.
$$
It is not hard to see that the ratio of unacceptable hypotheses to the total number of hypotheses in  $Q$  is at most  $50 / 1000 = 0.05$ . Therefore, a random hypothesis in  $Q$  is acceptable with probability at least 0.95. Hence, by the union bound the total error probability is bounded by:
$\mathbf{Pr}[\text{Outputting an unacceptable hypothesis}] \leq \mathbf{Pr}[\text{Outputting an unacceptable hypothesis | valid comparisons}]$
$$
+ \Pr [ \text {a t l e a s t o n e i n v i d a l c o m p a r i s o n s} ] \leq 0. 0 5 + 0. 0 1 <   0. 1
$$
Therefore, the proof is complete.
3.2 Memory-data tradeoffs of random ladder tournament
As we have discussed, one of the advantages of Algorithm 2 is its flexibility in the usage of memory and data. In the following, we describe a tradeoff that we can obtain for this algorithm using the sample summaries presented in Section 2.2.2. At a high-level, the following is how we achieve the tradeoff: suppose we have  $b$  bits of memory. We choose the largest integer  $t$  so that we can store the sample summary needed to compare  $t$  hypotheses. Then, we run the random ladder tournament while we draw new samples and refresh the sample summary at every  $t$  step. Our two described sample summaries Scheff'e counts and the sorted list lead to the following memory-data tradeoffs. Although these tradeoffs are not as tight as our main result (by factors of  $\log n$  and  $1 / \epsilon$ ), they have better accuracy guarantees ( $\alpha = 3$  instead of  $\alpha = 9$  in our main result).
Lemma 3.1. Suppose we have  $n$  hypotheses in  $\mathcal{H}$ . For every  $p_0 \geq 1/n$ ,  $\epsilon$ , and an integer  $t$  between two and  $k = \Theta(1/p_0)$ , one can implement Algorithm 2 in such away that it uses:
$$
s = O \left(\frac {1}{t p _ {0}} \cdot \frac {\log p _ {0} ^ {- 1}}{\epsilon^ {2}}\right) s a m p l e s, a n d \quad b = O \left(t ^ {2} \log \left(\frac {\log p _ {0} ^ {- 1}}{\epsilon}\right) + t \log n\right) b i t s o f m e m o r y.
$$
Lemma 3.2. Suppose we have  $n$  hypotheses in  $\mathcal{H}$ . For every  $p_0 \geq 1/n$ ,  $\epsilon$ , and an integer  $t$  between two and  $k = \Theta(1/p_0)$ , one can implement Algorithm 2 in such away that it uses:
$$
s = O \left(\frac {1}{t p _ {0}} \cdot \frac {\log p _ {0} ^ {- 1}}{\epsilon^ {2}}\right) s a m p l e s, a n d \quad b = O \left(t \cdot \frac {\log \left(p _ {0} ^ {- 1}\right) \cdot \log n}{\epsilon^ {2}} + t \log n\right) b i t s o f m e m o r y.
$$
For the proofs of the above lemmas, see Section A.2.
MA was supported by NSF awards CNS-2120667, CNS-2120603, CCF-1934846, and BU's Hariri Institute for Computing. This work was predominantly done while MA was affiliated with Boston University and Northeastern University. MB was supported by NSF awards CCF-1947889 and CNS-2046425, and a Sloan Research Fellowship. AS was supported in part by NSF awards CCF-1763786 and CNS-2120667 as well as Faculty Awards from Google and Apple.