Title: A Competitive Algorithm for Agnostic Active Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: active learning

binary classification

competitive ratio
Abstract: For some hypothesis classes and input distributions, \emph{active}
  agnostic learning needs exponentially fewer samples than passive
  learning; for other classes and distributions, it offers little to
  no improvement.  The most popular algorithms for agnostic active
  learning express their performance in terms of a parameter called
  the disagreement coefficient, but it is known that these algorithms
  are inefficient on some inputs.

  We take a different approach to agnostic active learning, getting an
  algorithm that is \emph{competitive} with the optimal algorithm for
  any binary hypothesis class $H$ and distribution $\mathcal{D}_X$ over $X$.
  In particular, if any algorithm can use $m^*$ queries to get
  $O(\eta)$ error, then our algorithm uses $O(m^* \log H)$ queries to
  get $O(\eta)$ error.  Our algorithm lies in the vein of the
  splitting-based approach of Dasgupta [2004], which gets a similar
  result for the realizable ($\eta = 0$) setting.

  We also show that it is NP-hard to do better than our algorithm's
  $O(\log H)$ overhead in general.

A Competitive Algorithm for Agnostic Active Learning

Abstract

For some hypothesis classes and input distributions, active agnostic learning needs exponentially fewer samples than passive learning; for other classes and distributions, it offers little to no improvement. The most popular algorithms for agnostic active learning express their performance in terms of a parameter called the disagreement coefficient, but it is known that these algorithms are inefficient on some inputs.

We take a different approach to agnostic active learning, getting an algorithm that is competitive with the optimal algorithm for any binary hypothesis class H and distribution D X over X . In particular, if any algorithm can use m ∗ queries to get O ( η ) error, then our algorithm uses O ( m ∗ log | H | ) queries to get O ( η ) error. Our algorithm lies in the vein of the splitting-based approach of Dasgupta [2004], which gets a similar result for the realizable ( η = 0 ) setting.

We also show that it is NP-hard to do better than our algorithm's O (log | H | ) overhead in general.

1 Introduction

Active learning is motivated by settings where unlabeled data is cheap but labeling it is expensive. By carefully choosing which points to label, one can often achieve significant reductions in label complexity [Cohn et al., 1994]. A canonical example with exponential improvement is one-dimensional threshold functions h τ ( x ) := 1 x ≥ τ : in the noiseless setting, an active learner can use binary search to find an ε -approximation solution in O ( log 1 ε ) queries, while a passive learner needs Θ ( 1 ε ) samples [Cohn et al., 1994, Dasgupta, 2005, Nowak, 2011].

In this paper we are concerned with agnostic binary classification. We are given a hypothesis class H of binary hypotheses h : X → { 0 , 1 } such that some h ∗ ∈ H has err( h ∗ ) ≤ η , where the error

̸

is measured with respect to an unknown distribution D over X ×{ 0 , 1 } . In our active setting, we also know the marginal distribution D X of x , and can query any point x of our choosing to receive a sample y ∼ ( Y | X = x ) for ( X,Y ) ∼ D . The goal is to output some ̂ h with err( ̂ h ) ≤ η + ε , using as few queries as possible.

The first interesting results for agnostic active learning were shown by Balcan et al. [2006], who gave an algorithm called Agnostic Active (A 2 ) that gets logarithmic dependence on ε in some natural settings: it needs ˜ O ( log 1 ε ) samples for the 1d linear threshold setting (binary search), as long as as ε > 16 η , and ˜ O ( d 2 log 1 ε ) samples for d -dimensional linear thresholds when D X is the uniform sphere and ε > √ dη . This stands in contrast to the polynomial dependence on ε necessary in the passive setting. The bound's requirement that ε ≳ η is quite natural given a lower bound of Ω ( d η 2 ε 2 )

due to [Kääriäinen, 2006, Beygelzimer et al., 2009], where d is the VC dimension. Subsequent works have given new algorithms [Dasgupta et al., 2007, Beygelzimer et al., 2010] and new analyses [Hanneke, 2007a] to get bounds for more general problems, parameterized by the 'disagreement coefficient' of the problem. But while these can give better bounds in specific cases, they do not give a good competitive ratio to the optimum algorithm: see (Hanneke [2014], Section 8.2.5) for a realizable example where O ( log 1 ε ) queries are possible, but disagreement-coefficient based bounds lead to Ω ( 1 ε ) queries.

By contrast, in the realizable, identifiable setting ( η = ε = 0 ), a simple greedy algorithm is competitive with the optimal algorithm. In particular, Dasgupta [2004] shows that if any algorithm can identify the true hypothesis in m queries, then the greedy algorithm that repeatedly queries the point that splits the most hypotheses will identify the true hypothesis in O ( m log | H | ) queries. This extra factor of log | H | is computationally necessary: as we will show in Theorem 1.2, avoiding it is NP-hard in general. This approach can extend [Dasgupta, 2005] to the PAC setting (so ε > 0 , but still η = 0 ), showing that if any algorithm gets error ε in m ∗ queries, then this algorithm gets error 8 ε in roughly ˜ O ( m ∗ · log | H | ) queries (but see the discussion after Theorem 8.2 of Hanneke [2014], which points out that one of the logarithmic factors is in an uncontrolled parameter τ , and states that 'Resolving the issue of this extra factor of log 1 τ remains an important open problem in the theory of active learning.').

The natural question is: can we find an agnostic active learning algorithm that is competitive with the optimal one in the agnostic setting?

Our Results. Our main result is just such a competitive bound. We say an active agnostic learning algorithm A solves an instance ( H, D X , η, ε, δ ) with m measurements if, for every distribution D with marginal D X and for which some h ∗ ∈ H has err( h ∗ ) ≤ η , with probability 1 -δ , A uses at most m queries and outputs ̂ h ∈ H with err ( ̂ h ) ≤ η + ε . Let m ∗ ( H, D X , η, ε, δ ) be the optimal number of queries for this problem, i.e., the smallest m for which any A can solve ( H, D X , η, ε, δ ) .

̸

Define N ( H, D X , α ) to be the size of the smallest α -cover over H , i.e., the smallest set S ⊆ H such that for every h ∈ H there exists h ′ ∈ S with Pr x ∼D X [ h ( x ) = h ′ ( x )] ≤ α . When the context is clear, we drop the parameters and simply use N . Of course, N is at most | H | .

Theorem 1.1 (Competitive Bound) . There exist some constants c 1 , c 2 and c 3 such that for any instance ( H, D X , η, ε, δ ) with ε ≥ c 1 η , Algorithm 1 solves the instance with sample complexity

and polynomial time.

Even the case of η = 0 is interesting, given the discussion in [Hanneke, 2014] of the gap in [Dasgupta, 2005]'s bound, but the main contribution is the ability to handle the agnostic setting of η > 0 . The requirement that ε ≥ O ( η ) is in line with prior work [Balcan et al., 2006, Dasgupta, 2005]. Up to constants in η and ε , Theorem 1.1 shows that our algorithm is within a log N ≤ log | H | factor of the optimal query complexity.

We show that it NP-hard to avoid this log N factor, even in the realizable ( η = ε = δ = 0) case: Theorem 1.2 (Lower Bound) . It is NP-hard to find a query strategy for every agnostic active learning instance within an c log | H | for some constant c > 0 factor of the optimal sample complexity.

This is a relatively simple reduction from the hardness of approximating SETCOVER [Dinur and Steurer, 2014]. The lower bound instance has η = ε = δ = 0 , although these can be relaxed to being small polynomials (e.g., ε = η = 1 3 | X | and δ = 1 3 | H | ).

Extension. We give an improved bound for our algorithm in the case of noisy binary search (i.e., H consists of 1d threshold functions). When η = Θ( ε ) , N ( H, D X , ε ) = Θ( 1 ε ) and m ∗ ( η, ε, . 99) = O (log 1 ε ) . Thus Theorem 1.1 immediately gives a bound of O (log 2 1 εδ ) , which is nontrivial but not ideal. (For η ≪ ε , the same bound holds since the problem is strictly easier when η is smaller.) However, the bound in Theorem 1.1 is quite loose in this setting, and we can instead give a bound of

for the same algorithm, Algorithm 1. This matches the bound given by disagreement coefficient based algorithms for constant δ . The proof of this improved dependence comes from bounding a new parameter measuring the complexity of an H, D x pair; this parameter is always at least Ω( 1 m ∗ ) but may be much larger (and is constant for 1d threshold functions). See Theorem 2.3 for details.

1.1 Related Work

Active learning is a widely studied topic, taking many forms beyond the directly related work on agnostic active learning discussed above [Settles, 2009]. Our algorithm can be viewed as similar to 'uncertainty sampling' [Lewis, 1995, Lewis and Catlett, 1994], a popular empirical approach to active learning, though we need some modifications to tolerate adversarial noise.

One problem related to the one studied in this paper is noisy binary search, which corresponds to active learning of 1d thresholds. This has been extensively studied in the setting of i.i.d. noise [Burnashev and Zigangirov, 1974, Ben-Or and Hassidim, 2008, Dereniowski et al., 2021] as well as monotonic queries [Karp and Kleinberg, 2007]. Some work in this vein has extended beyond binary search to (essentially) active binary classification [Nowak, 2008, 2011]. These algorithms are all fairly similar to ours, in that they do multiplicative weights/Bayesian updates, but they query the single maximally informative point. This is fine in the i.i.d. noise setting, but in an agnostic setting the adversary can corrupt that query. For this reason, our algorithm needs to find a set of high-information points to query.

Another related problems is decision tree learning. The realizable, noiseless case η = ε = 0 of our problem can be reduced to learning a binary decision tree with minimal depth. Heged˝ us [1995] studied this problem and gave basically the same upper and lower bound as in Dasgupta [2005]. Kosaraju et al. [2002] studied a split tree problem, which is a generalization of binary decision tree learning, and also gave similar bounds. Azad et al. [2022] is a monograph focusing on decision tree learning, in which many variations are studied, including learning with noise. However, this line of work usually allows different forms of queries so their results are not directly comparable from results in the active learning literature.

For much more work on the agnostic active binary classification problem, see Hanneke [2014] and references therein. Many of these papers give bounds in terms of the disagreement coefficient, but sometimes in terms of other parameters. For example, Katz-Samuels et al. [2021] has a query bound that is always competitive with the disagreement coefficient-based methods, and sometimes much better; still, it is not competitive with the optimum in all cases.

In terms of the lower bound, it is shown in Laurent and Rivest [1976] that the problem is NP-complete, in the realizable and noiseless setting. To the best of our knowledge, our Theorem 1.2 showing hardness of approximation to within a O (log | H | ) factor is new.

Minimax sample complexity bounds. Hanneke and Yang [2015] and Hanneke [2007b] have also given 'minimax' sample complexity bounds for their algorithms, also getting a sample complexity within O (log | H | ) of optimal. However, these results are optimal with respect to the sample complexity for the worst-case distribution over y and x . But the unlabeled data x is given as input. So one should hope for a bound with respect to optimal for the actual x and only worst-case over y ; this is our bound.

We give the following example to illustrate that our bound, and indeed our algorithm, can be much better.

Example 1.3. Define a hypothesis class of N hypotheses h 1 , · · · , h N , and log N + N data points x 1 , · · · , x log N + N . For each hypothesis h j , the labels of the first N points express j in unary and the labels of the last log N points express j in binary. We set η = ε = 0 and consider the realizable case.

In the above example, the binary region is far more informative than the unary region, but disagreement coefficient-based algorithms just note that every point has disagreement. Our algorithm will query the binary encoding region and take O (log N ) queries. Disagreement coefficient based algorithms, including those in Hanneke and Yang [2015] and Hanneke [2007b], will rely on essentially uniform sampling for the first Ω( N/ log N ) queries. These algorithms are 'minimax' over x , in the sense that if you didn't see any x from the binary region , you would need almost as many samples as they use. But you do see x from the binary region, so the algorithm should make use of it to get exponential improvement.

Future Work. Our upper bound assumes full knowledge of D X and the ability to query arbitrary points x . Often in active learning, the algorithm receives a large but not infinite set of unlabeled sample points x , and can only query the labels of those points. How well our results adapt to this setting we leave as an open question.

Similarly, our bound is polynomial in the number of hypotheses and the domain size. This is hard to avoid in full generality-if you don't evaluate most hypotheses on most data points, you might be missing the most informative points-but perhaps it can be avoided in structured examples.

2 Algorithm Overview

Our algorithm is based on a Bayesian/multiplicative weights type approach to the problem, and is along the lines of the splitting-based approach of Dasgupta [2004].

We maintain a set of weights w ( h ) for each h ∈ H , starting at 1 ; these induce a distribution λ ( h ) := w ( h ) ∑ h w ( h ) which we can think of as our posterior over the 'true' h ∗ .

Realizable setting. As initial intuition, consider the realizable case of η = ε = 0 where we want to find the true h ∗ . If h ∗ really were drawn from our prior λ , and we query a point x , we will see a 1 with probability E h ∼ λ h ( x ) . Then the most informative point to query is the one we are least confident in, i.e., the point x ∗ maximizing

Suppose an algorithm queries x 1 , . . . , x m and receives the majority label under h ∼ λ each time. Then the fraction of h ∼ λ that agree with all the queries is at least 1 -∑ m i =1 r ( x i ) ≥ 1 -mr ( x ∗ ) . This suggests that, if r ( x ∗ ) ≪ 1 m , it will be hard to uniquely identify h ∗ . It is not hard to formalize this, showing that: if no single hypothesis has 75% probability under λ , and any algorithm exists with sample complexity m and 90% success probability at finding h ∗ , we must have r ( x ∗ ) ≥ 1 10 m .

This immediately gives an algorithm for the η = ε = 0 setting: query the point x maximizing r ( x ) , set w ( h ) = 0 for all hypotheses h that disagree, and repeat. As long as at least two hypotheses remain, the maximum probability will be 50% < 90% and each iteration will remove an Ω( 1 m ) fraction of the remaining hypotheses; thus after O ( m log H ) rounds, only h ∗ will remain. This is the basis for Dasgupta [2004].

Handling noise: initial attempt. There are two obvious problems with the above algorithm in the agnostic setting, where a (possibly adversarial) η fraction of locations x will not match h ∗ . First, a single error will cause the algorithm to forever reject the true hypothesis; and second, the algorithm makes deterministic queries, which means adversarial noise could be placed precisely on the locations queried to make the algorithm learn nothing.

To fix the first problem, we can adjust the algorithm to perform multiplicative weights: if in round i we query a point x i and see y i , we set

̸

for a small constant α = 1 5 . To fix the second problem, we don't query the single x ∗ of maximum r ( x ∗ ) , but instead choose x according to distribution q over many points x with large r ( x ) .

To understand this algorithm, consider how log λ i ( h ∗ ) evolves in expectation in each step. This increases if the query is correct, and decreases if it has an error. A correct query increases λ i in proportion to the fraction of λ placed on hypotheses that get the query wrong, which is at least r ( x ) ; and the probability of an error is at most η max x q ( x ) D x ( x ) . If at iteration i the algorithm uses query distribution q , some calculation gives that

The algorithm can choose q to maximize this bound on the potential gain. There's a tradeoff between concentrating the samples over the x of largest r ( x ) , and spreading out the samples so the adversary can't raise the error probability too high. We show that if learning is possible by any algorithm (for a constant factor larger η ), then there exists a q for which this potential gain is significant.

̸

Lemma 2.1 (Connection to OPT) . Define ∥ h -h ′ ∥ = Pr x ∼D x [ h ( x ) = h ′ ( x )] . Let λ be a distribution over H such that no radius(2 η + ε ) ball B centered on h ∈ H has probability at least 80% . Let m ∗ = m ∗ ( H, D X , η, ε, 99 100 ) . Then there exists a query distribution q over X with

At a very high level, the proof is: imagine h ∗ ∼ λ . If the algorithm only sees the majority label y on every query it performs, then its output ̂ h is independent of h ∗ and cannot be valid for more than 80% of inputs by the ball assumption; hence a 99% successful algorithm must have a 19% chance of seeing a minority label. But for m ∗ queries x drawn with marginal distribution q , without noise the expected number of minority labels seen is m ∗ E [ r ( x )] , so E [ r ( x )] ≳ 1 /m ∗ . With noise, the adversary can corrupt the minority labels in h ∗ back toward the majority, leading to the given bound.

The query distribution optimizing (1) has a simple structure: take a threshold τ for r ( x ) , sample from D x conditioned on r ( x ) > τ , and possibly sample x with r ( x ) = τ at a lower rate. This means the algorithm can efficiently find the optimal q .

Except for the caveat about λ not already concentrating in a small ball, applying Lemma 2.1 combined with (1) shows that log λ ( h ∗ ) grows by Ω( 1 m ∗ ) in expectation for each query. It starts out at log λ ( h ∗ ) = -log H , so after O ( m ∗ log H ) queries we would have λ ( h ∗ ) being a large constant in expectation (and with high probability, by Freedman's inequality for concentration of martingales). Of course λ ( h ∗ ) can't grow past 1 , which features in this argument in that once λ ( h ∗ ) > 80% , a small ball will have large probability and Lemma 2.1 no longer applies, but at that point we can just output any hypothesis in the heavy ball.

Handling noise: the challenge. There is one omission in the above argument that is surprisingly challenging to fix, and ends up requiring significant changes to the algorithm: if at an intermediate step λ i concentrates in the wrong small ball, the algorithm will not necessarily make progress. It is entirely possible that λ i concentrates in a small ball, even in the first iteration-perhaps 99% of the hypotheses in H are close to each other. And if that happens, then we will have r ( x ) ≤ 0 . 01 for most x , which could make the RHS of (1) negative for all q .

In fact, it seems like a reasonable Bayesian-inspired algorithm really must allow λ ( h ∗ ) to decrease in some situations. Consider the setting of Figure 1. We have three hypotheses, h 1 , h 2 , and h 3 , and a prior λ = (0 . 9 , 0 . 099999 , 10 -6 ) . Because λ ( h 3 ) is so tiny, the algorithm presumably should ignore h 3 and query essentially uniformly from the locations where h 1 and h 2 disagree. In this example, h 3 agrees with h 1 on all but an η mass in those locations, so even if h ∗ = h 3 , the query distribution can match h 1 perfectly and not h 3 . Then w ( h 1 ) stays constant while w ( h 3 ) shrinks. w ( h 2 ) shrinks much faster, of course, but since the denominator is dominated by w ( h 1 ) , λ ( h 3 ) will still shrink. However, despite λ ( h 3 ) shrinking, the algorithm is still making progress in this example: λ ( h 2 ) is shrinking fast, and once it becomes small relative to λ ( h 3 ) then the algorithm will start querying points to distinguish h 3 from h 1 , at which point λ ( h 3 ) will start an inexorable rise.

Our solution is to 'cap' the large density balls in λ , dividing their probability by two, when applying Lemma 2.1. Our algorithm maintains a set S ⊆ H of the 'high-density region,' such that the capped

distribution:

has no large ball. Then Lemma 2.1 applies to λ , giving the existence of a query distribution q so that the corresponding r ( x ) is large. We then define the potential function

for h ∗ / ∈ S i , and ϕ i = 0 for h ∗ ∈ S i . We show that ϕ i grows by Ω( 1 m ∗ ) in expectation in each iteration. Thus, as in the example of Figure 1, either λ ( h ∗ ) grows as a fraction of the whole distribution, or as a fraction of the 'low-density' region.

If at any iteration we find that λ has some heavy ball B ( µ, 2 η + ε ) so Lemma 2.1 would not apply, we add B ( µ ′ , 6 η +3 ε ) to S , where B ( µ ′ , 2 η + ε ) is the heaviest ball before capping. We show that this ensures that no small heavy ball exists in the capped distribution λ . Expanding S only increases the potential function, and then the lack of heavy ball implies the potential will continue to grow.

Thus the potential (2) starts at -2 log | H | , and grows by Ω( 1 m ∗ ) in each iteration. After O ( m ∗ log H ) iterations, we will have ϕ i ≥ 0 in expectation (and with high probability by Freedman's inequality). This is only possible if h ∗ ∈ S , which means that one of the centers µ of the balls added to S is a valid answer.

In fact, with some careful analysis we can show that with 1 -δ probability that one of the first O (log H δ ) balls added to S is a valid answer. The algorithm can then check all the centers of these balls, using the following active agnostic learning algorithm:

Theorem 2.2. Active agnostic learning can be solved for ε = 3 η with O ( | H | log | H | δ ) samples.

̸

Proof. The algorithm is the following. Take any pair h, h ′ with ∥ h -h ′ ∥ ≥ 3 η . Sample O ( log | H | δ ) observations randomly from ( x ∼ D x | h ( x ) = h ′ ( x )) . One of h, h ′ is wrong on at least half the queries; remove it from H and repeat. At the end, return any remaining h .

To analyze this, let h ∗ ∈ H be the hypothesis with error η . If h ∗ is chosen in a round, the other h ′ must have error at least 2 η . Therefore the chance we remove h ∗ is at most δ/ | H | . In each round we remove a hypothesis, so there are at most | H | rounds and at most δ probability of ever crossing off h ∗ . If we never cross off h ∗ , at the end we output some h with ∥ h -h ∗ ∥ ≤ 3 η , which gives ε = 3 η .

The linear dependence on | H | makes the Theorem 2.2 algorithm quite bad in most circumstances, but the dependence only on | H | makes it perfect for our second stage (where we have reduced to O (log | H | ) candidate hypotheses).

Overall, this argument gives an O ( m ∗ log | H | δ +log | H | δ log log | H | δ ) sample algorithm for agnostic active learning. One can simplify this bound by observing that the set of centers C added by our algorithm form a packing, and must therefore all be distinguishable by the optimal algorithm, so m ∗ ≥ log C . This gives a bound of

By starting with an η -net of size N , we can reduce | H | to N with a constant factor increase in η .

With some properly chosen constants c 4 and c 5 , the entire algorithm is formally described in Algorithm 1.

Remark 1: As stated, the algorithm requires knowing m ∗ to set the target sample complexity / number of rounds k . This restriction could be removed with the following idea. m ∗ only enters the analysis through the fact that O ( 1 m ∗ ) is a lower bound on the expected increase of the potential

function in each iteration. However, the algorithm knows a bound on its expected increase in each round i ; it is the value

optimized in the algorithm. Therefore, we could use an adaptive termination criterion that stops at iteration k if ∑ k i =1 τ i ≥ O (log | H | δ ) . This will guarantee that when terminating, the potential will be above 0 with high probability so our analysis holds.

Remark 2: The algorithm's running time is polynomial in | H | . This is in general not avoidable, since the input is a truth table for H . The bottleneck of the computation is the step where the algorithm checks if the heaviest ball has mass greater than 80%. This step could be accelerated by randomly sampling hypothesis and points to estimate and find heavy balls; this would improve the dependence to nearly linear in | H | . If the hypothesis class has some structure, like the binary search example, the algorithm can be implemented more efficiently.

Algorithm 1 Competitive Algorithm for Active Agnostic Learning

̸

Compute a 2 η maximal packing H ′ Let w 0 = 1 for every h ∈ H ′ . S 0 ←∅ C ←∅ for i = 1 , . . . , k = O ( m ∗ log | H ′ | δ ) do Compute λ i ( h ) = w i -1 ( h ) ∑ h ∈ H w i -1 ( h ) for every h ∈ H if there exists c 4 η + c 5 ε ball with probability > 80% over λ i,S i -1 then S i ← S i ∪ B ( µ ′ , 3 c 4 η +3 c 5 ε ) where B ( µ ′ , c 4 η + c 5 ε ) is the heaviest radius c 4 η + c 5 ε ball over λ i C ← C ∪ { µ ′ } else S i ← S i -1 Compute λ i,S i = { 1 2 λ i ( h ) h ∈ S i λ i ( h ) · 1 -1 2 Pr h ∼ λ i [ h ∈ S i ] 1 -Pr h ∼ λ i [ h ∈ S i ] h / ∈ S i Compute r i,S i ( x ) = min { E h ∼ λ i,S i [ h ( x )] , 1 -E h ∼ λ i,S i [ h ( x )] } for every x ∈ X Find a query distribution by solving q ∗ = max q E x ∼ q [ r i,S i ( x )] -c 4 20 η max x q ( x ) D X ( x ) subject to E x ∼D X [ q ( x )] ≤ 3 η (3) Query x ∼ q ∗ , getting label y Set w i ( h ) = { w i -1 ( h ) if h ( x ) = y e -α w i -1 ( h ) if h ( x ) = y for every h ∈ H ′ Find the best hypothesis ˆ h in C using the stage two algorithm in Theorem 2.2 return ˆ h

Generalization for Better Bounds. To get a better dependence for 1d threshold functions, we separate out the Lemma 2.1 bound on (1) from the analysis of the algorithm given a bound on (1). Then for particular instances like 1d threshold functions, we get a better bound on the algorithm by giving a larger bound on (1).

Theorem 2.3. Suppose that D x and H are such that, for any distribution λ over H such that no radius-( c 4 η + c 5 ε ) ball has probability more than 80% , there exists a distribution q over X such that

for some β > 0 . Then for ε ≥ c 1 η , c 4 ≥ 300 , c 5 = 1 10 and c 1 ≥ 90 c 4 , let N = N ( H, D x , η ) be the size of an η -cover of H . Algorithm 1 solves ( η, ε, δ ) active agnostic learning with O ( 1 β log N δ +log N δ log log N δ ) samples.

Corollary 2.4. There exists a constant c 1 > 1 such that, for 1 d threshold functions and ε > c 1 η , Algorithm 1 solves ( η, ε, δ ) active agnostic learning with O ( log 1 εδ log log 1 ε δ ) samples.

Proof. Because the problem is only harder if η is larger, we can raise η to be η = ε/C , where C > 1 is a sufficiently large constant that Theorem 2.3 applies. Then 1 d threshold functions have an η -cover of size N = O (1 /ε ) . To get the result by Theorem 2.3, it suffices to show β = Θ(1) .

Each hypothesis is of the form h ( x ) = 1 x ≥ τ , and corresponds to a threshold τ . So we can consider λ to be a distribution over τ .

Let λ be any distribution for which no radiusR with probability greater than 80% ball exists, for R = c 4 η + c 5 ε . For any percent p between 0 and 100 , let τ p denote the pth percentile of τ under λ (i.e., the smallest t such that Pr[ τ ≤ t ] ≥ p/ 100 ). By the ball assumption, τ 10 and τ 90 do not lie in the same radiusR ball. Hence ∥ h τ 10 -h τ 90 ∥ > R , or

We let q denote ( D x | τ 10 ≤ x < τ 90 ) . Then for all x ∈ supp( q ) we have r ( x ) ≥ 0 . 1 and

Therefore we can set

as needed.

3 Proof of Lemma 2.1

Lemma 2.1 (Connection to OPT) . Define ∥ h -h ′ ∥ = Pr x ∼D x [ h ( x ) = h ′ ( x )] . Let λ be a distribution over H such that no radius(2 η + ε ) ball B centered on h ∈ H has probability at least 80% . Let m ∗ = m ∗ ( H, D X , η, ε, 99 100 ) . Then there exists a query distribution q over X with

̸

Proof. WLOG, we assume that Pr h ∼ λ [ h ( x ) = 0] ≥ Pr h ∼ λ [ h ( x ) = 1] for every x ∈ X . This means r ( x ) = E h ∼ λ [ h ( x )] . This can be achieved by flipping all h ( x ) and observations y for all x not satisfying this property; such an operation doesn't affect the lemma statement.

We will consider an adversary defined by a function g : X → [0 , 1] . The adversary takes a hypothesis h ∈ H and outputs a distribution over y ∈ { 0 , 1 } X such that 0 ≤ y ( x ) ≤ h ( x ) always, and err( h ) = E x,y [ h ( x ) -y ( x )] ≤ η always. For a hypothesis h , the adversary sets y ( x ) = 0 for all x with h ( x ) = 0 , and y ( x ) = 0 independently with probability g ( x ) if h ( x ) = 1 -unless E x [ h ( x ) g ( x )] > η , in which case the adversary instead simply outputs y = h to ensure the expected error is at most η always.

We consider the agnostic learning instance where x ∼ D x , h ∼ λ , and y is given by this adversary. Let A be an ( η, ε ) algorithm which uses m measurements and succeeds with 99% probability. Then it must also succeed with 99% probability over this distribution. For the algorithm to succeed on a sample h , its output ̂ h must have ∥ h -̂ h ∥ ≤ 2 η + ε . By the bounded ball assumption, for any choice of adversary, no fixed output succeeds with more than 80% probability over h ∼ λ .

Now, let A 0 be the behavior of A if it observes y = 0 for all its queries, rather than the truth; A 0 is independent of the input. A 0 has some distribution over m queries, outputs some distribution of answers ̂ h . Let q ( x ) = 1 m Pr[ A 0 queries x ] , so q is a distribution over X . Since A 0 outputs a fixed distribution, by the bounded ball assumption, for h ∼ λ and arbitrary adversary function g ,

But A behaves identically to A 0 until it sees its first nonzero y . Thus,

and so

A

Pr[

sees a non-zero

y

]

19%

.

Since A behaves like A 0 until the first nonzero, we have

As an initial note, observe that E h,y [ y ( x )] ≤ E h [ h ( x )] = r ( x ) so

Thus the lemma statement holds for η = 0 .

Handling η > 0 . Consider the behavior when the adversary's function g : X → [0 , 1] satisfies E x ∼D x [ g ( x ) r ( x )] ≤ η/ 10 . We denote the class of all adversary satisfying this condition as G . We have that

Let E h denote the event that E x ∼D x [ g ( x ) h ( x )] ≤ η , so Pr[ E h ] ≤ 10% . Furthermore, the adversary is designed such that under E h , E y [ y ( x )] = h ( x )(1 -g ( x )) for every x . Therefore:

Thus

over all distributions q and functions g : X → [0 , 1] satisfying E x ∼D x [ g ( x ) r ( x )] ≤ η/ 10 . We now try to understand the structure of the q, g optimizing the LHS of (4).

Let g ∗ denote an optimizer of the objective. First, we show that the constraint is tight, i.e., E x ∼D x [ g ∗ ( x ) r ( x )] = η/ 10 . Since increasing g improves the constraint, the only way this could not happen is if the maximum possible function, g ( x ) = 1 for all x , lies in G . But for this function, the LHS of (4) would be 0 , which is a contradiction; hence we know increasing g to improve the objective at some point hits the constraint, and hence E x ∼D x [ g ∗ ( x ) r ( x )] = η/ 10 .

For any q , define τ q ≥ 0 to be the minimum threshold such that and define g q by

where α ∈ [0 , 1] is chosen such that E x ∼D x [ r ( x ) g q ( x )] = η/ 10 ; such an α always exists by the choice of τ q .

For any q , we claim that the optimal g ∗ in the LHS of (4) is g q . It needs to maximize

subject to a constraint on E x ∼D X [ r ( x ) g ( x )] ; therefore moving mass to points of larger q ( x ) D X ( x ) is always an improvement, and g q is optimal.

We now claim that the q maximizing (4) has max q ( x ) D X ( x ) = τ q . If not, some x ′ has q ( x ′ ) D X ( x ′ ) > τ q . Then g q ( x ′ ) = 1 , so the x ′ entry contributes nothing to E x ∼ q [ r ( x )(1 -g q ( x ))] ; thus decreasing q ( x ) halfway towards τ q (which wouldn't change g q ), and adding the savings uniformly across all q ( x ) (which also doesn't change g q ) would increase the objective.

So there exists a q satisfying (4) for which Pr [ q ( x ) D X ( x ) > τ q ] = 0 , and therefore the set T = { x | q ( x ) D X ( x ) = τ q } satisfies E D X [ r ( x ) x ∈ T ] ≥ η/ 10 and a g q minimizing (4) is

Therefore and so by (4),

as desired.

4 Conclusion

We have given an algorithm that solves agnostic active learning with (for constant δ ) at most an O (log | H | ) factor more queries than the optimal algorithm. It is NP-hard to improve upon this O (log | H | ) factor in general, but for specific cases it can be avoided. We have shown that 1d threshold functions, i.e. binary search with adversarial noise, is one such example where our algorithm matches the performance of disagreement coefficient-based methods and is within a log log 1 ε factor of optimal.

5 Acknowledgments

Yihan Zhou and Eric Price were supported by NSF awards CCF-2008868, CCF-1751040 (CAREER), and the NSF AI Institute for Foundations of Machine Learning (IFML).

References

Mohammad Azad, Igor Chikalov, Shahid Hussain, Mikhail Moshkov, and Beata Zielosko. Decision Trees with Hypotheses . Springer International Publishing, 2022. doi: 10.1007/978-3-031-08585-7. URL https://doi.org/10.1007/978-3-031-08585-7 .

Maria-Florina Balcan, Alina Beygelzimer, and John Langford. Agnostic active learning. In Proceedings of the 23rd international conference on Machine learning , pages 65-72, 2006.

Michael Ben-Or and Avinatan Hassidim. The bayesian learner is optimal for noisy binary search (and pretty good for quantum as well). In 2008 49th Annual IEEE Symposium on Foundations of Computer Science , pages 221-230, 2008. doi: 10.1109/FOCS.2008.58.

Alina Beygelzimer, Sanjoy Dasgupta, and John Langford. Importance weighted active learning. In Proceedings of the 26th annual international conference on machine learning , pages 49-56, 2009.

Alina Beygelzimer, Daniel J Hsu, John Langford, and Tong Zhang. Agnostic active learning without constraints. Advances in neural information processing systems , 23, 2010.

Marat Valievich Burnashev and Kamil'Shamil'evich Zigangirov. An interval estimation problem for controlled observations. Problemy Peredachi Informatsii , 10(3):51-61, 1974.

David Cohn, Les Atlas, and Richard Ladner. Improving generalization with active learning. Machine learning , 15:201-221, 1994.

Sanjoy Dasgupta. Analysis of a greedy active learning strategy. Advances in neural information processing systems , 17, 2004.

Sanjoy Dasgupta. Coarse sample complexity bounds for active learning. Advances in neural information processing systems , 18, 2005.

Sanjoy Dasgupta, Daniel J Hsu, and Claire Monteleoni. A general agnostic active learning algorithm. Advances in neural information processing systems , 20, 2007.

Dariusz Dereniowski, Aleksander Lukasiewicz, and Przemyslaw Uznanski. Noisy searching: simple, fast and correct. CoRR , abs/2107.05753, 2021. URL https://arxiv.org/abs/2107.05753 .

Irit Dinur and David Steurer. Analytical approach to parallel repetition. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing , pages 624-633, 2014.

Steve Hanneke. A bound on the label complexity of agnostic active learning. In Proceedings of the 24th international conference on Machine learning , pages 353-360, 2007a.

Steve Hanneke. Teaching dimension and the complexity of active learning. In International conference on computational learning theory , pages 66-81. Springer, 2007b.

Steve Hanneke. Theory of disagreement-based active learning. Foundations and Trends® in Machine Learning , 7(2-3):131-309, 2014.

Steve Hanneke and Liu Yang. Minimax analysis of active learning. J. Mach. Learn. Res. , 16(1): 3487-3602, 2015.

Tibor Heged˝ us. Generalized teaching dimensions and the query complexity of learning. In Proceedings of the eighth annual conference on Computational learning theory , pages 108-117, 1995.

Matti Kääriäinen. Active learning in the non-realizable case. In Algorithmic Learning Theory: 17th International Conference, ALT 2006, Barcelona, Spain, October 7-10, 2006. Proceedings 17 , pages 63-77. Springer, 2006.

Richard M. Karp and Robert Kleinberg. Noisy binary search and its applications. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms , SODA '07, page 881-890, USA, 2007. Society for Industrial and Applied Mathematics. ISBN 9780898716245.

Julian Katz-Samuels, Jifan Zhang, Lalit Jain, and Kevin Jamieson. Improved algorithms for agnostic pool-based active classification. In International Conference on Machine Learning , pages 53345344. PMLR, 2021.

S Rao Kosaraju, Teresa M Przytycka, and Ryan Borgstrom. On an optimal split tree problem. In Algorithms and Data Structures: 6th International Workshop, WADS'99 Vancouver, Canada, August 11-14, 1999 Proceedings , pages 157-168. Springer, 2002.

Hyafil Laurent and Ronald L Rivest. Constructing optimal binary decision trees is np-complete. Information processing letters , 5(1):15-17, 1976.

David D Lewis. A sequential algorithm for training text classifiers: Corrigendum and additional data. In Acm Sigir Forum , volume 29, pages 13-19. ACM New York, NY, USA, 1995.

David D Lewis and Jason Catlett. Heterogeneous uncertainty sampling for supervised learning. In Machine learning proceedings 1994 , pages 148-156. Elsevier, 1994.

Robert Nowak. Generalized binary search. In 2008 46th Annual Allerton Conference on Communication, Control, and Computing , pages 568-574. IEEE, 2008.

Robert D Nowak. The geometry of generalized binary search. IEEE Transactions on Information Theory , 57(12):7893-7906, 2011.

Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009.

Joel Tropp. Freedman's inequality for matrix martingales. 2011.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018.

A Query Complexity Upper Bound

In this section we present the whole proof of the query complexity upper bound of Algorithm 1, as stated in Theorem 1.1.

A.1 Notation

We remind the readers about some definitions first. Remember that w i ( h ) denote the weight of hypothesis h in iteration i and λ i,S ( h ) = w i ( h ) ∑ h ′ ∈ S w i ( h ′ ) for some S ⊆ H denote the proportion of h in S . We view λ i,S as a distribution of hypotheses in S so for h / ∈ S , λ i,S ( h ) = 0 . For a set S ⊆ H of hypotheses, we define w i ( S ) := ∑ h ∈ S w ( h ) and λ i ( h ) = λ i,H ( h ) .

̸

Define r λ,h ∗ ( x ) := Pr h ∼ λ [ h ( x ) = h ∗ ( x )] , and r λ ( x ) = min y ∈{ 0 , 1 } Pr h ∼ λ [ h ( x ) = y ] , so r λ ( x ) = min( r λ,h ∗ ( x ) , 1 -r λ,h ∗ ( x )) .

Define

̸

as the "capped" distribution in iteration i .

Finally, for notational convenience define r i,S := r λ i,S , r i,S,h := r λ i,S ,h and r i,S := r λ i,S .

The main focus of our proof would be analyzing the potential function

where h ∗ is the best hypothesis in H . We would like to show that ϕ i +1 ( h ∗ ) -ϕ i ( h ∗ ) is growing at a proper rate in each iteration. We pick S i to be an expanding series of sets, i.e., S i ⊆ S i +1 for any i ≥ 1 . However, the change of the "capped" set S i makes this task challenging. Therefore, we instead analyze the following quantity defined as

and ϕ i +1 ( h ∗ ) -ϕ i ( h ∗ ) = ∆ i ( h ∗ ) + log λ i +1 ,H \ S i +1 ( h ∗ ) λ i +1 ,H \ S i ( h ∗ ) if h ∗ / ∈ S i +1 . Further, we define ψ k ( h ∗ ) :=

∑ i<k ∆ i ( h ∗ ) so by definition ϕ k ( h ∗ ) = ϕ 0 ( h ∗ )+ ψ k ( h ∗ )+ ∑ i<k log λ i +1 ,H \ S i +1 ( h ∗ ) λ i +1 ,H \ S i ( h ∗ ) if h ∗ / ∈ S i +1 . In the following text, we will drop the parameter h ∗ when the context is clear and just use ϕ i , ∆ i and ψ i instead.

A.2 Potential Growth

We will lower bound the conditional per iteration potential increase by first introducing a lemma that relates the potential change to the optimization problem (3).

Lemma A.1. Assume that err( h ∗ ) ≤ η , then for any set S of hypotheses containing h ∗ and query distribution q , we have

for α ≤ 0 . 2 . Moreover,

Proof. For notational convenience, define ˜ r ( x ) := r i,S,h ∗ ( x ) .

Observe that

Therefore

Using that ˜ r ( x ) ≤ 1 , we have

where the last step uses α ≤ 0 . 2 . Finally,

This proves the first desired result. For the second, note that if we query x , then conditioned on F i

Since log(1 + (1 -e -α ) ˜ r ( x )) ≤ (1 -e -α ) ˜ r ( x ) ≤ α ˜ r ( x ) , taking the expectation over x gives the result.

The above lemma, combined with Lemma 2.1, proves the potential will grow at desired rate at each iteration. But remember that Lemma 2 . 1 requires the condition that no ball has probability greater than 80% , so we need to check this condition is satisfied. The following lemma shows that if we cap the set S i , then the probability is not concentrated on any small balls.

Lemma A.2. In Algorithm 1, for every iteration i , S i is such that no radius c 4 η + c 5 ε ball has more than 80% probability under λ i,S i .

̸

Proof. If S i = S i -1 , then by the construction of S i , there are no radius c 4 η + c 5 ε balls have probability greater than 80% under λ i,S i -1 = λ i,S i . Otherwise, we have S i -1 = S i and a ball B ( µ, 3 c 4 η +3 c 5 ε ) is added to S i in this iteration. We first prove a useful claim below.

Claim A.3. If a ball B ′ = ( µ, 3 c 4 η +3 c 5 ε ) is added to S i at some iteration i , λ i ( B ( µ, c 4 η + c 5 ε )) ≥ 0 . 6 .

Proof. If B ′ is added to S i at the iteration i , then there exists some ball D with radius c 4 η + c 5 ε such that ¯ λ i,S i -1 ( D ) ≥ 0 . 8 . If a set of hypotheses gains probability after capping, the gained probability comes from the reduced probability of other hypotheses not in this set. Therefore, the gained probability of any set is upper bounded by half of the probability of the complement of that set before capping. This means λ i ( D ) ≥ 0 . 6 because otherwise after capping ¯ λ i,S i -1 ( D ) < 0 . 8 , which is a contradiction. As a result, λ i ( B ( µ, c 4 η + c 5 ε )) ≥ λ i ( D ) ≥ 0 . 6 .

̸

Let p ( x ) = Pr y ∼ ( Y | X ) [ y = h ∗ ( x )] denote the probability of error if we query x , so

Suppose we query a point x and do not get an error. Then the hypotheses that disagree with h ∗ are downweighted by an e -α factor, so

̸

On the other hand, if we do get an error then the disagreeing hypotheses are effectively upweighted by e α :

By Claim A.3, the probability of B ( µ, c 4 η + c 5 ε ) is at least 0 . 6 over the uncapped distribution λ i . So any ball not intersecting B ( µ, c 4 η + c 5 ε ) has probability at most 0 . 4 before capping. After capping these balls will have probability no more than 0 . 7 . At the same time, any ball intersects B ( µ, c 4 η + c 5 ε ) would be completely inside B ( µ, 3 c 4 η +3 c 5 ε ) so its probability would be at most 0 . 5 after capping.

Now we are ready to apply Lemma A.1 and Lemma 2.1 except one caution. Remember that in the beginning of the algorithm, we compute a 2 η -packing H ′ ⊆ H of the instance. From the well-known relationship between packing and covering (for example, see Vershynin [2018, Lemma 4.2.8]), we have | H ′ | ≤ N ( H,η ) . Every hypothesis in H is within 2 η to some hypothesis in H ′ , so there exists a hypothesis in H ′ with error less than 3 η . This means that the best hypothesis h ∗ ∈ H ′ has error 3 η instead of η . The following lemma serves as the cornerstone of the proof of the query complexity upper bound, which states that the potential grows at rate Ω ( 1 m ∗ ) in each iteration.

Lemma A.4. Given c 4 ≥ 300 and err( h ∗ ) ≤ 3 η , there exists a sampling distribution q such that

as well as | ∆ i | ≤ α always and Var[∆ i |F i ] ≤ α E [ | ∆ i ||F i ] ≲ α E [∆ i |F i ] .

Proof. For the sake of bookkeeping, we let m ∗ = m ∗ ( H, D X , c 4 η, c 5 ε -2 η, 99 100 ) in this proof and the following text. We first bound the expectation. By Lemma A.1 applied to S ∈ { H,H \ S i } with 3 η , we have

where q is the query distribution of the algorithm at iteration i . Now, by the definition of

we have for any x that and thus

Algorithm 1 chooses the sampling distribution q to maximize E x ∼ q [ r i,S i ( x )] -c 4 20 η max x q ( x ) D X ( x ) ≤ E x ∼ q [ r i,S i ( x )] -15 η max x q ( x ) D X ( x ) because c 4 ≥ 300 . By Lemma A.2, λ i,S i over H ′ has no radius-( c 4 η + c 5 ε ) ball with probability larger than 80%, so by Lemma 2.1 q satisfies

Because H ′ ⊆ H is a maximal 2 η -packing, every hypothesis in H is within 2 η of some hypothesis in H ′ . The problem ( H, D X , c 4 η, c 5 ε -2 η, 99 100 ) is harder than the problem ( H ′ , D X , c 4 η, c 5 ε, 99 100 ) because we can reduce the latter to the former by simply adding more hypotheses and solve

it then map the solution back by returning the closest hypothesis in H ′ . Hence, m ∗ ≥ m ∗ ( H ′ , D X , c 4 η, c 5 ε, 99 100 ) . Therefore,

We now bound the variance. The value of ∆ i may be positive or negative, but it is bounded by | ∆ i | ≤ α . Thus

Since E x ∼ q [∆ i |F i ] -2 αη max x q ( x ) D X ( x ) ≳ 1 m ∗ ≥ 0 , we have

A.3 Concentration of potential

We have showed that the potential will grow at Ω ( 1 m ∗ ) per iteration, but only in expectation, while our goal is to obtain a high probability bound. Let µ k := ∑ i<k E [∆ i |F i -1 ] ≳ k/m ∗ , then

Apparently | ψ k -µ k | is upper bounded, so ψ k -µ k is a supermartingale. To show a high probability bound, we will use Freedman's inequality. A version is stated in Tropp [2011]. We slighted modify it so it can be applied to supermartingale as the following. (XXX Not sure if the following is correct. I can't find a version of supermartingale.)

Theorem A.5 (Freedman's Inequality) . Consider a real-valued supermartingale { Y k : k = 0 , 1 , 2 , · · · } that is adapted to the filtration F 0 ⊆ F 1 ⊆ F 2 ⊆ ··· ⊆ F with difference sequence { X k : k = 1 , 2 , 3 , · · · } . Assume that the difference sequence is uniformly bounded:

Define the predictable quadratic variation process of the supermartingale:

Then, for all t ≥ 0 and σ 2 > 0 ,

and thus

Then we can prove a high probability bound as the following.

Lemma A.6. With probability 1 -δ , ϕ i = 0 for some i = O ( m ∗ log | H | δ ) so h ∗ ∈ S i .

Proof. Remember we have that

Since S i +1 ⊇ S i for all i , λ i,H \ S i +1 ( h ∗ ) ≥ λ i,H \ S i ( h ∗ ) if h ∗ / ∈ S i +1 , we have

Let K = O ( m ∗ log | H | δ ) . Let's assume by contradiction that ϕ K < 0 for for, then h ∗ / ∈ S i for i ≤ K . We know by Lemma A.4 that

and that ∑ i<k Var[∆ i ] ≤ 1 4 µ k by picking α small enough. Moreover, | ∆ i | ≤ α always. To use Freedman's inequality, let's set the RHS

Solving the above quadratic equation, one solution is that t ≥ R 3 log 1 δ + √ R 2 9 log 2 1 δ +2 σ 2 log 1 δ . Let's substitute in R = α and σ 2 = ∑ i<k Var i -1 (∆ i ) , with 1 -δ probability we have for any k > O ( m ∗ log 1 δ ) that

The second last inequality is because the first term outscales all of the rest. Since K = O ( m ∗ log | H | δ ) , we have

ψ

K

2 log

|

H

|

with 1 -δ probability. Then ϕ K ≥ ϕ 0 + ψ k because ϕ 0 ≥ log 1 2 | H | ≥ -2 log | H | and this contradicts h ∗ / ∈ S K . Therefore, with probability at least 1 -δ , h ∗ ∈ S K and by definition, ϕ i = 0 for some i ≤ K as desired.

A.4 Bounding the Size of | C |

So far we've shown that after O ( m ∗ log | H | δ ) iterations, h ∗ will be included in the set S i . The last thing we need to prove Theorem 1.1 is that with high probability, C is small, which is equivalent to show that not many balls will be added to S i after O ( m ∗ log | H | δ ) iterations. To show this, we first need to relate the number of balls added to S i to ψ i . Let E i denote the number of errors h ∗ made up to iteration i (and set E i = E i -1 if h ∗ ∈ S i ) and N i denote the number of balls added to S i up to iteration i (again set N i = N i -1 if h ∗ ∈ S i ).

Lemma A.7. The following inequality holds for every i :

Proof. We divide the i iterations into phases. A new phase begins and an old phase ends if at this iteration a new ball is added to the set S i . We use p 1 , . . . , p k for k ≤ i to denote phases and i 1 , . . . , i k to denote the starting iteration of the phases. We analyse how the potential changes from the phase p j to the phase p j +1 . Let's say the ball B 2 = ( µ 2 , 3 c 4 η +3 c 5 ε ) is added at the beginning of p j +1 and B 1 = ( µ 1 , 3 c 4 η +3 c 5 ε ) is the ball added at the beginning of p j . Then the ball B ′ 2 = ( µ 2 , c 4 η + c 5 ε ) and the ball B ′ 1 = ( µ 1 , c 4 η + c 5 ε ) are disjoint. Otherwise, B ′ 2 ⊆ B 1 so B 2 would not have been added by the algorithm. At the beginning of p j , B ′ 1 has probability no less than 0 . 6 by Claim A.3. Therefore, B ′ 2 has probability no more than 0 . 4 . Similarly, at the beginning of p j +1 , B ′ 2 has probability at least 0 . 6 by Claim A.3. Since during one iteration the weight of a hypothesis cannot change too much, at iteration i j +1 -1 , B ′ 2 has weight at least 0 . 5 by picking α small enough. Therefore, we have log λ i j +1 -1 ( B ′ 2 ) -log λ i j ( B ′ 2 ) ≥ log 0 . 5 0 . 4 ≥ 1 5 . Moreover, note that S i does not change from iteration i j to iteration i j +1 -1 by the definition of phases. Now we compute

The change of the weight of h ∗ is

where E p j is the number of errors h ∗ made in p j . Consequently,

The last step above comes from

and

because the weight w ( h ) only decreases. Summing over all phases j and we get

Since i may not exactly be the end of a phase, the last phase may end early so we have N i -1 instead of N i . Rearrange and the proof finishes.

We have already bounded ψ i , so we just need to bound E i in order to bound N i by the following lemma.

Lemma A.8. For every k , with probability at least 1 -δ ,

Proof. Let q be the query distribution at iteration i -1 and p ( x ) be the probability that x is corrupted by the adversary. Then the conditional expectation of E i -E i -1 is

Then if h ∗ / ∈ S , from Lemma A.4

Therefore, E [ α ( E i -E i -1 ) |F i ] ≤ 1 2 E [∆ i |F i ] and E [∆ i -α ( E i -E i -1 ) |F i ] ≥ 1 2 E [∆ i |F i ] . This means that ψ k -α E k -1 2 µ k is a supermartingale. We then bound Var[∆ i -α ( E i -E i -1 ) |F i ] . Note that | ∆ i -α ( E i -E i -1 ) | ≤ 2 α , so

As a result,

By picking α small enough, ∑ i<k Var[∆ i -α ( E i -E i -1 ) |F i ] ≤ 1 8 µ k . Moreover, | ∆ i -α ( E i -E i -1 ) | ≤ 2 α always. Therefore by Freedman's inequality, with 1 -δ probability we have for any k that

Rearrange and we proved the lemma.

Combining Lemma A.7 and Lemma A.8, we can show C is small with high probability as the lemma follows.

Lemma A.9. For k = O ( m ∗ log | H | δ ) , with probability at least 1 -2 δ , h ∗ ∈ S k and | C | ≤ O ( log | H | δ ) at iteration k .

Proof. By union bound, with probability at least 1 -2 δ , Lemma A.6 and A.8 will hold at the same time. This means h ∗ is added to S k . By definition, 0 ≥ ϕ k ≥ ϕ 0 + ψ k , so ψ k ≤ 2 log | H | . Therefore, by Lemma A.7 and A.8, the number of balls added | C | is O ( log | H | +log 1 δ ) = O ( log | H | δ ) .

A.5 Putting Everything Together

We proved that after O ( m ∗ log | H | δ ) iterations, h ∗ ∈ S i and C is small with high probability. Hence, running the stage two algorithm to return a desired hypothesis will not take much more queries. We are ready to put everything together and finally prove Theorem 1.1.

Theorem 1.1 (Competitive Bound) . There exist some constants c 1 , c 2 and c 3 such that for any instance ( H, D X , η, ε, δ ) with ε ≥ c 1 η , Algorithm 1 solves the instance with sample complexity

and polynomial time.

Proof. Let's pick c 1 , c 4 , c 5 as in Theorem 2.3 and pick the confidence parameter to be δ 3 . Then by Lemma A.9, with probability 1 -2 δ 3 , the first O ( log | H | δ ) ball added to S i will contain h ∗ . Since each ball added to C has radius 3 c 4 η +3 c 5 ε , the best hypothesis in C has error (3 + 3 c 4 ) η +3 c 5 ε . By Theorem 2.2, with probability 1 -δ 3 , the algorithm will return a hypothesis with error (9 + 9 c 4 ) η +9 c 5 ε ≤ η + ε . Therefore, by union bound, the algorithm will return a desired hypothesis with probability 1 -δ . This proves the correctness of the algorithm.

The stage one algorithm makes

queries. The stage two algorithm makes O ( | C | log | C | δ ) queries by Theorem 2.2. Note that C is a c 4 η + c 5 ε -packing because the center of added balls are at least c 4 η + c 5 ε away, so m ∗ ( H, D X , c 4 2 η, c 5 2 ε, 99 100 ) ≥ log | C | . Since | C | ≤ log | H | δ by Lemma A.9, stage two algorithm takes O ( ( m ∗ ( H, D X , c 4 2 η, c 5 2 ε, 99 100 ) +log 1 δ ) log | H | δ ) queries. Picking c 2 = c 4 , c 3 = c 5 2 , we get the desired sample complexity bound.

To compute the packing at the beginning of the algorithm, we need to compute the distance of every pair of hypotheses, which takes O ( | H | 2 | X | ) time. Computing r in each round takes O ( | H || X | ) time and solving the optimization problem takes O ( | X | ) time. Therefore, the remaining steps in stage one takes O ( m ∗ | H || X | log | H | δ ) time. Stage two takes O ( log | H | δ log log | H | δ δ ) time. Therefore, the overall running time is polynomial of the size of the problem.

Similarly, we can prove Theorem 2.3, which is a stronger and more specific version of Theorem 1.1. Theorem 2.3. Suppose that D x and H are such that, for any distribution λ over H such that no radius-( c 4 η + c 5 ε ) ball has probability more than 80% , there exists a distribution q over X such that

for some β > 0 . Then for ε ≥ c 1 η , c 4 ≥ 300 , c 5 = 1 10 and c 1 ≥ 90 c 4 , let N = N ( H, D x , η ) be the size of an η -cover of H . Algorithm 1 solves ( η, ε, δ ) active agnostic learning with O ( 1 β log N δ +log N δ log log N δ ) samples.

Proof. By Lemma A.9 (with m ∗ replaced by 1 β and setting confidence parameter to δ 3 ) after O ( 1 β log N δ ) queries, with probability at least 1 -2 δ 3 , a hypothesis in C will be within c 4 η + c 5 ε to h ∗ and | C | = O ( log N δ ) . From Theorem 2.2, with probability at least 1 -δ 3 , stage two algorithm then outputs a hypothesis ˆ h that is 9 c 4 η + 9 c 5 ε from h ′ so err ( ˆ h ) ≤ 9 c 4 η + 9 c 5 ε ≤ η + ε by the choice of the constants. The stage two algorithm makes O ( log N δ log log N δ δ ) queries. Overall, the algorithm makes O ( 1 β log N δ +log N δ log log N δ δ ) queries and succeeds with probability at least 1 -δ .

B Query Complexity Lower Bound

In this section we derive a lower bound for the agnostic binary classification problem, which we denote by AGNOSTICLEARNING. The lower bound is obtained from a reduction from minimum set cover, which we denote by SETCOVER. The problem SETCOVER consists a pair ( U, S ) , where U is a ground set and S is a collection of subsets of U . The goal is to find a set cover C ⊆ S such that ⋃ S ∈ C S = U of minimal size | C | . We use K to denote the cardinality of the minimum set cover.

Lemma B.1 (Dinur and Steurer [2014], Corollary 1.5) . There exists hard instances SETCOVERHARD with the property K ≥ log | U | such that for every γ > 0 , it is NP-hard to approximate SETCOVERHARD to within (1 -γ ) ln | U | .

Proof. This lemma directly follows from Dinur and Steurer [2014, Corollary 1.5]. In their proof, they constructed a hard instance of SETCOVER from LABELCOVER. The size of the minimum cover K ≥ | V 1 | = Dn 1 and log | U | = ( D + 1) ln n 1 ≤ K . So the instance in their proof satisfies the desired property.

Then we prove the following lemma by giving a ratio-preserving reduction from SETCOVER to AGNOSTICLEARNING.

Lemma B.2. If there exists a deterministic α -approximation algorithm for AGNOSTICLEARNING ( H, D x , 1 3 | X | , 1 3 | X | , 1 4 | H | ) , there exists a deterministic 2 α -approximation algorithm for SETCOVERHARD .

Proof. Given an instance of SETCOVERHARD, for each s ∈ S , number the elements u ∈ s in an arbitrary order; let f ( s, u ) denote the index of u in s 's list (and padding 0 to the left with the extra bit). We construct an instance of AGNOSTICLEARNING as the following:

Let the domain X have three pieces: U , V := { ( s, j ) | s ∈ S , j ∈ [1 + log | s | ] } , and D = { 1 , . . . , log | U |} , an extra set of log | U | more coordinates.

On this domain, we define the following set of hypotheses:

(a) For u ∈ U , define h u which only evaluates 1 on u ∈ U and on ( s, j ) ∈ V if u ∈ s and the j 'th bit of (2 f ( s, u ) + 1) is 1.

(b) For d ∈ D , define h d which only evaluates 1 on d .

(c) Define h 0 which evaluates everything to 0 .

Let D X be uniform distribution over X and set η = 1 3 |X| and ε = 1 3 |X| . Set δ = 1 4 | H | .

Any two hypotheses satisfy ∥ h 1 -h 2 ∥ ≥ 1 |X| > ε = η , so err( h ∗ ) = 0 . First we show that m ∗ ( H, D x , 1 3 | X | , 1 3 | X | , 1 4 | H | ) ≤ K +log | U | . Indeed there exists a deterministic algorithm using K +log | U | queries to identify any hypothesis with probability 1. Given a smallest set cover C , the algorithm first queries all ( s, 0) ∈ V for s ∈ C . If h ∗ = h u for some u , then for the s ∈ S that covers u , ( s, 0) will evaluate to true. The identity of u can then be read out by querying ( s, j ) for all j . The other possibilitiesh d for some d or 0 -can be identified by evaluating on all of D with log U queries. The total number of queries is then at most K +log | U | in all cases, so m ∗ ≤ K +log | U | ≤ 2 K .

We now show how to reconstruct a good approximation to set cover from a good approximate query algorithm. We feed the query algorithm y = 0 on every query it makes, and let C be the set of all s for which it queries ( s, j ) for some j . Also, every time the algorithm queries some u ∈ U , we add an arbitrary set containing u to C . Then the size of C is at most the number of queries. We claim that C is a set cover: if C does not cover some element u , then h u is zero on all queries made by the algorithm, so h u is indistinguishable from h 0 and the algortihm would fail on either input h 0 or h u . Thus if A is a deterministic α -approximation algorithm for AGNOSTICLEARNING, we will recover a set cover of size at most αm ∗ ≤ α ( K +log | U | ) ≤ 2 αK , so this gives a deterministic 2 α -approximation algorithm for SETCOVERHARD.

Similar results also holds for randomized algorithms, we just need to be slightly careful about probabilities.

Lemma B.3. If there exists a randomized algorithm for AGNOSTICLEARNING ( H, D x , 1 3 | X | , 1 3 | X | , 1 4 | H | ) , there exists a randomized 2 α -approximation algorithm for SETCOVERHARD with success probability at least 2 3 .

Proof. We use the same reduction as in Lemma B.2. Let A be an algorithm solves AGNOSTICLEARNING ( H, D x , 1 3 | X | , 1 3 | X | , 1 4 | H | ) . To obtain a set cover using A , we keeping giving A label 0 and construct the set C as before. Let q C be a distribution over the reconstructed set C . Assume that by contradiction with probability at least 1 3 , C is not a set cover. Then, with probability at least 1 / 3 , there is some element v such that both h v and h 0 are consistent on all queries the algorithm made; call such a query set 'ambiguous'.

Then what is the probability that the agnostic learning algorithm fails on the input distribution that chooses h ∗ uniformly from H ? Any given ambiguous query set is equally likely to come from any of the consistent hypotheses, so the algorithm's success probability on ambiguous query sets is at most 1 / 2 . The chance the query set is ambiguous is at least 2 3 | H | : a 1 3 H chance that the true h ∗ is h 0 and the query set is ambiguous, and at least as much from the other hypotheses making it ambiguous. Thus the algorithm's fails to learn the true hypothesis with at least 1 3 | H | probability, contradicting the assumed 1 4 | H | failure probability.

Therefore, a set cover of size at most 2 αK can be recovered with probability at least 1 3 using the agnostic learning approximation algorithm.

The following theorem will then follow.

Theorem 1.2 (Lower Bound) . It is NP-hard to find a query strategy for every agnostic active learning instance within an c log | H | for some constant c > 0 factor of the optimal sample complexity.

Proof. Let's consider the instance of set cover constructed in Lemma B.2. Let c = 0 . 1 and note that 0 . 1 log | H | ≤ 0 . 49 log | H | 2 . If there exists a polynomial time 0 . 49 log | H | 2 approximation algorithm for the instance, then there exists a polynomial time 0 . 98 log | H | 2 ≤ 0 . 98 log | U | approximation algorithm for SETCOVERHARD, which is a contradiction to Lemma B.1.

Peer reviews:
comments: SUMMARY:
This paper studies agnostic active learning with a finite hypothesis space. The goal is to achieve a competitive query complexity with an optimal algorithm.

STRENGTHS:
Interesting and important problem.

The approximation hardness based on Set-Cover is a nice addition.

--- rebuttal comment ---
After reviewers clarified concerns and added more results, I raised my score from 3 to 7.

WEAKNESSES:
The authors seem to be unaware of "Minimax analysis of active learning" by Steve Hanneke and Liu Yang [JMLR 2015]. Hanneke and Yang introduce the star number (a combinatorial dimension), which characterizes various active learning settings (realizable, different noise settings, including the one here) leading to near-tight lower and upper bounds. This seems to make the analyses here a special case of the achieved results. E.g., on a quick look Theorem 8 seems to have a similar competitive ratio as the proposed algorithms: roughly $d\log (1/\varepsilon)$ (with $d = VC$) instead of $\log |H| \geq d$.

Similarly, the discussed bounds $\mathcal{O}(m\log H)$ have been devised by Hegedűs, Tibor. "Generalized teaching dimensions and the query complexity of learning." [COLT 1995] long before Dasgupta and Nowak.

Claiming polynomial runtime is misleading, as one typically would mean polynomial in the instance space $X$ (or the sample, like in standard PAC) and not the hypothesis space $\mathcal{H}$, as is here the case.

QUESTIONS:
The authors write $|H|$ multiple times implying that the hypothesis space is finite but also use packings / coverings of $H$. Do the results generalize to hypothesis spaces with infinite size?

LIMITATIONS:
See weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper applied a multiplicative weight update /  generalized binary search style algorithm to solve agnostic active learning. It proposes a novel \"capping\" approach to the weight (over hypotheses) to ensure the potential function always grows by some amount, and it proves this amount is lower bounded by $\\Omega(1/m^*)$ where $m^*$ is the theoretical optimal instance-dependent label complexity. In the end, it shows that the proposed algorithm is competitive in the sense that its label complexity is at most, roughly speaking, $\\tilde{O}(m^* \\log |H|)$. And it also shows that removing the extra \"log(|H|)\" factor is NP-hard.\n\n--\n\nI've read the author's response. It addressed my major concern about related work so I increased my score to 7. I would encourage the authors to provide a comprehensive comparison with related works, including directly comparing bounds in different settings. I'm still not very convinced about the argument around computational efficiency, but that should not block this paper from being accepted.\n\nSTRENGTHS:\n- The proposed algorithm is a novel and nontrivial modification of the classical multiplicative weight update algorithm. The analysis from a competitive perspective also looks new to me in the setting of agnostic active learning.\n\n- I briefly checked the proof and they look sound to me.\n\n- Understanding the theoretical lower and upper bounds of instance dependent label complexity bounds for agnostic active learning is an interesting problem, and this paper makes a solid contribution by providing an algorithm whose label complexity is close to the optimal with an additional log(|H|) factor.\n\nWEAKNESSES:\n1. The additional factor of log(|H|) makes the label complexity of the proposed algorithm look not very strong to me: it can be worse than existing algorithms in many scenarios. The argument that getting rid of this factor is NP-hard does not look very relevant to me, since the main contribution of this paper is more on the information-theoretic side (the proposed algorithm is already by no means computationally efficient in the sense that it is polynomial in the number of hypotheses (or its epsilon cover at best) and the size of example space X), and it is known that even learning linear classifier without too much assumptions on noise is NP-hard.\n\n2. It would clearer if there were more discussion about how good the label complexity is by comparing it with known upper and lower bounds in different settings. Especially, there are a few papers (e.g. Hanneke, Steve, and Liu Yang. \"Minimax analysis of active learning.\" J. Mach. Learn. Res. 16.1 (2015): 3487-3602.) that give minimax analysis of active learning. I would like to see how your results compared with theirs. It would also be interesting if you could mention how the proposed algorithm works with noise assumptions (bounded, Tsybakov, etc.).\n\n3. This paper focuses on the case where $\\epsilon \\gtrapprox \\eta$, but it looks to me the case where $\\epsilon$ is smaller can also be interesting (though of course the label complexity won't be exponentially better w.r.t. 1/\\epsilon). It would be interesting if the theoretical results could cover that as well.\n\n4. This paper is overall clear, but some wordings are a bit confusing to me unless I check the detailed proofs. E.g., the proof sketch in line 149 - 154 (\"algorithm\", \"majority label\", \"inputs\", \"independent\").\n\nQUESTIONS:\nI would like the authors to comment on the weaknesses 1~3 I mentioned above.\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-28"}

comments: SUMMARY:
The paper studies agnostic active learning by proposing a competitive algorithm that achieves at most a $\log H$ multiplicative factor on top of the optimal query lower bound of $m^*$. While similar result was known for the realizable setting, this paper makes a step toward understanding the agnostic setting. As a complementary result, it also shows that it is NP-hard to improve the $O(\log H)$ overhead both in realizable and agnostic settings.

STRENGTHS:
The paper proposed a strategy that actively learns the target function in the agnostic setting with competitive sample complexity. Given that similar strategies only handle the realizable setting, it makes a significant contribution to the active learning problems in general settings. The proofs seem sound. The paper is well written.

WEAKNESSES:
The paper is clear in the description of the algorithm and the analysis of how it queries. However, the computational cost of the algorithm is vague. In Theorem 1.1, it claims that the algorithm runs in polynomial time, but it seems not so at least in the dimension parameter d.

QUESTIONS:
Can the author comment on the exact running time of the algorithm on each parameter, for example, $d, m, |H|, |X|, \frac{1}{\epsilon}$, (or any other important parameters)?

LIMITATIONS:
No concerns.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-07

comments: SUMMARY:
The authors provide an algorithm for learning in the presence of agnostic noise. The algorithm finds a hypothesis that gets error $O(\eta)$ where $\eta$ is the noise parameter. The algorithm requires querying specific points, so it uses a stronger oracle than the standard active learning. The algorithm is possibly inefficient for many classes as it is polynomial in the number of hypotheses (or to the size of the cover). Their algorithm is optimal up to log factors, and they provide a lower bound showing that it is NP-Hard to improve this.

STRENGTHS:
This work provides a query-efficient algorithm with tight results (upper and lower bound) for the agnostic setting.

WEAKNESSES:
The setting in their analysis is not active learning but learning with queries which is a stronger oracle. So the title of the paper/abstract is misleading. In my opinion, active learning is a completely different model than the one the authors use. Also for examples the 41-46 lines, the papers there are for active learning and not for the model the authors analyze. I think the authors should be more clear about this. About the NP-Hardness result. This result makes sense when the $|H|$ is polynomial over $\epsilon$ and the dimension. But almost always the $|H|$ is exponentially large, i.e., $d$-dimensional functions.

QUESTIONS:
N/A

LIMITATIONS:
No liomitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-10