Title: Sample-Efficient Agnostic Boosting
Venue: NeurIPS 2024
Accepted: yes
Keywords: boosting; sample complexity; learning theory; reinforcement learning
Abstract: The theory of boosting provides a computational framework for aggregating approximate weak learning algorithms, which perform marginally better than a random predictor, into an accurate strong learner. In the realizable case, the success of the boosting approach is underscored by a remarkable fact that the resultant sample complexity matches that of a computationally demanding alternative, namely Empirical Risk Minimization (ERM). This in particular implies that the realizable boosting methodology has the potential to offer computational relief without compromising on sample efficiency.

Despite recent progress, in agnostic boosting, where assumptions on the conditional distribution of labels given feature descriptions are absent, ERM outstrips the agnostic boosting methodology in being quadratically more sample efficient than all known agnostic boosting algorithms. In this paper, we make progress on closing this gap, and give a substantially more sample efficient agnostic boosting algorithm than those known, without compromising on the computational (or oracle) complexity. A key feature of our algorithm is that it leverages the ability to reuse samples across multiple rounds of boosting, while guaranteeing a generalization error strictly better than those obtained by blackbox applications of uniform convergence arguments. We also apply our approach to other previously studied learning problems, including boosting for reinforcement learning, and demonstrate improved results.

Sample-Efficient Agnostic Boosting

Abstract

The theory of boosting provides a computational framework for aggregating approximate weak learning algorithms, which perform marginally better than a random predictor, into an accurate strong learner. In the realizable case, the success of the boosting approach is underscored by a remarkable fact that the resultant sample complexity matches that of a computationally demanding alternative, namely Empirical Risk Minimization (ERM). This in particular implies that the realizable boosting methodology has the potential to offer computational relief without compromising on sample efficiency.

Despite recent progress, in agnostic boosting, where assumptions on the conditional distribution of labels given feature descriptions are absent, ERM outstrips the agnostic boosting methodology in being quadratically more sample efficient than all known agnostic boosting algorithms. In this paper, we make progress on closing this gap, and give a substantially more sample efficient agnostic boosting algorithm than those known, without compromising on the computational (or oracle) complexity. Akey feature of our algorithm is that it leverages the ability to reuse samples across multiple rounds of boosting, while guaranteeing a generalization error strictly better than those obtained by blackbox applications of uniform convergence arguments. Wealso apply our approach to other previously studied learning problems, including boosting for reinforcement learning, and demonstrate improved results.

1 Introduction

A striking observation in statistical learning is that given a small number of samples it is possible to learn the best classifier from an almost exponentially large class of predictors. In fact, it is possible to do using a conceptually straightforward procedure - Empirical Risk Minimization (ERM) - that finds a classifier that is maximally consistent with the collected samples. Substantiating this observation, the fundamental theorem of statistical learning (e.g., [SSBD14]) states that with high probability ERM can guarantee ε -excess population error with respect to the best classifier from a finite, but large hypothesis class H given merely m agnostic ≈ (log |H| ) /ε 2 identically distributed and independent (IID) samples of pairs of features and labels from the population distribution. Under an additional assumption - that of realizability - guaranteeing that there exists a perfect classifier in the hypothesis class achieving zero error, a yet quadratically smaller number m realizable ≈ (log |H| ) /ε of samples suffice. In the absence of such assumption, the learning problem is said to take place in the agnostic setting, i.e., under of a lack of belief in the ability of any hypothesis to perfectly fit the observed data.

This ability to generalize to the population distribution and successfully (PAC) learn from an almost exponentially large, and hence expressive, hypothesis class given limited number of examples suggests that the primary bottleneck for efficient learning is computational. Indeed, even with modest sample requirements, finding a maximally consistent hypothesis within an almost exponentially large class via, say, enumeration or global search, is generally computationally intractable.

It is against this backdrop that the theory of boosting offers a compelling alternative. The starting point is the realization that often, both in practice and in theory, it is easy to construct simple, yet inaccurate rules-of-thumbs (or weak learners ) that perform ever so slightly better than a random classifier. A natural question then arises (paraphrased from [Sch90]'s abstract): can one convert such mediocre learning rules into one that performs extremely well? Boosting algorithms offer a positive resolution to this question by providing convenient and computationally efficient reductions that aggregate such weak learners into a proficient learner with an arbitrarily high accuracy.

Realizable Boosting. Consider the celebrated Adaboost algorithm [FS97] which operates in the (noiseless) realizable binary classification setting. On any distribution consistent with a fixed labeling function (or concept ), a weak learner here promises an accuracy strictly better than half, since guessing labels randomly gets any example correct with probability half. Given access to such a weak learner and m realizable boosting ≈ (log |H| ) /ε samples 1 , Adaboost makes ≈ log 1 /ε calls to the weak learner to produce a classifier with (absolute) error at most ε on any distribution consistent with the same labeling function. Thus, not only is Adaboost computationally efficient provided, of course, such weak learners can be found, but also its sample complexity is no worse than that of ERM. This underscores the fact that the realizable boosting methodology has the potential to offer computational relief, without compromising on sample efficiency.

Agnostic Boosting. In practice, realizability is an onerous assumption; it is too limiting for the observed feature values alone to determine the label completely and deterministically, and that such a relation can be perfectly captured by an in-class hypothesis. Agnostic learning forgoes such assumptions. In their absence, bounds on absolute error are unachievable, e.g., when labels are uniformly random irrespective of features, no classifier can achieve accuracy better than half. Instead, in agnostic learning, the goal of the learner is to output a hypothesis with small excess error with respect to the best in-class classifier. If an in-class hypothesis is perfect on a given distribution, this relative error translates to an absolute error bound, thus generalizing the realizable case perfectly. Indeed, such model agnosticism has come to be a lasting hallmark of modern machine learning.

Early attempts at realizing the promise of boosting in the agnostic setting were met with limited success: while they did boost the weak learner's accuracy, the final hypothesis produced was not competitive with the best in-class hypothesis. We survey some of these in the related work section. A later result, and the work most related to ours, is due to [KK09]. A weak learner in this setting returns a classifier with a correlation against the true labels that is γ (say 0 . 1 ) times that of the best in-class

hypothesis. Random guesses of the labels produce a correlation of zero; hence, a weak classifier interpolates the performance of the best in-class hypothesis with that of a random one. Given access to such a weak learner, the boosting algorithm of [KK09] makes ≈ 1 /ε 2 calls to the weak learner and draws m agnostic boosting ≈ (log |H| ) /ε 4 samples to produce a learning rule (not necessarily in the hypotheses class, hence improper ) with ε -excess error. The dependency of the sample complexity in the target accuracy is thus quadratically worse than that of ERM. This gap persists untarnished for other known agnostic boosting algorithms too. In this work, we seek to diminish this fundamental gap and construct a more sample-efficient agnostic boosting algorithm.

Our main result is an efficient boosting algorithm that upon receiving ≈ (log |H| ) /ε 3 samples produces an improper learning rule with ε -excess error on the population loss. This is accomplished by the careful reuse of samples across rounds of boosting. We also extend these guarantees to infinite function classes, and give applications of our main result in reinforcement learning, and in agnostically learning halfspaces, in each case improving known results. We detail key contributions and technical challenges in achieving them next.

1.1 Contributions and technical innovations

Contribution 1: Sample-efficient Agnostic Booster. We provide a new potential-based agnostic boosting algorithm (Algorithm 1) that achieves ε -excess error when given a γ -weak learner operating with a base class B . In Theorem 4, we prove that the sample complexity of this new algorithm scales as (log |B| ) /γ 3 ε 3 , improving upon all known approaches to agnostic boosting. See Table 1.

A key innovation in our algorithm design and the source of our sample efficiency is the careful recursive reuse of samples between rounds of boosting, via a second-order estimator of the potential in Line 5.II. In contrast, [KK09] draws fresh samples for every round of boosting.

A second coupled innovation, this time in our analysis, is to circumvent a uniform convergence argument on the boosted hypothesis class , which would otherwise result in a sample complexity that scales as ≈ 1 /ε 4 . Indeed, the algorithm in [BCHM20] reuses the entire training dataset across all rounds of boosting. This approach succeeds in boosting the accuracy on the empirical distribution; however, success on the population distribution now relies on a uniform convergence (or sample compression) argument on the boosted class, the complexity of which grows linearly with the number of rounds of boosting since boosting algorithms are inevitably improper (i.e., output a final hypothesis by aggregating weak learners, hence outside the base class). Instead, we use a martingale argument on the smaller base hypothesis class to show that the empirical distributions constructed by our data reuse scheme and fed to the weak learner track the performance of any base hypothesis on the population distribution. This is encapsulated in Lemma 6.

Finally, while we follow the potential-based framework laid in [KK09], we find it necessary to alter the branching logic dictating what gets added to the ensemble at every step. At each step, the algorithm makes progress via including the weak hypothesis or making a step 'backward' via adding in a negation of the sign of the current ensemble. We note that there is a subtle error in [KK09] (see Appendix A), that although for their purposes is rectifiable without a change in the claimed sample complexity, leads to 1 /ε 4 sample complexity here in spite of the above modifications. At the leisure of 1 /ε 4 sample complexity, the fix is to test which of these alternatives fares better by drawing fresh samples every round. However, given a smaller budget, the error of the negation of the sign of the ensemble, which lies outside the base class, is not efficiently estimable. Instead, in Line 9 we give a different branching criteria that can be evaluated using the performance of the weak hypothesis on past data alone.

Contribution 2: Trading off Sample and Oracle Complexity. Although Theorem 4 offers an unconditional improvement on the sample complexity, it makes more calls to the weak learners than previous works. To rectify this, we give a second guarantee on the performance of Algorithm 1 in Theorem 9 (in Appendix B), with the oracle complexity matching that of known results. The resultant sample complexity improves known results for all practical (i.e., sub-constant) regimes of ε . This is made possible by a less well known variant of Freedman's inequality [Pin94, Pin20] that applies to random variables bounded with high probability.

Contribution 3: Extension to Infinite Classes. Although in our algorithm the samples fed to the weak leaner are independent conditioned on past sources of randomness, our relabeling and data reuse across rounds introduces complicated inter-dependencies between samples. For example, a sample drawn in the past can simultaneously be used as is in the present round (i.e., in Line 5.I), in addition to implicitly being used to modify the label of a different sample via the weak hypothesis it induced in the past (i.e., in Line 5.II). Thus, the textbook machinery of extending finite hypothesis results to infinite class applicable to IID samples via symmetrization and Rademacher complexity (see, e.g., [SSBD14, MRT18, BBL05]) is unavailable to us. Instead, we first derive L 1 covering number based bounds in Theorem 19 (in Appendix F). Through a result in empirical process theory [VDVW97], we translate these to a VC ( B ) /γ 3 ε 3 sample complexity bound, where VC ( B ) is the VC dimension of class B , in Theorem 5.

Contribution 4: Applications in Reinforcement Learning and Agnostic Learning of Halfspaces. Building on earlier reductions from reinforcement learning to supervised learning [KL02], [BHS22] initiated the study of function-approximation compatible reinforcement learning, given access to a weak learner over the policy class. By applying our agnostic booster to this setting, we improve their sample complexity by poly ( ε, γ ) factors for binary-action MDPs, as detailed in Table 2. Also, following [KK09], we apply our agnostic boosting algorithm to the problem of learning halfspaces over the Boolean hypercube and exhibit improved boosting-based results in Theorem 8.

Contribution 5: Experiments. In preliminary experiments in Section 7, we demonstrate that the sample complexity improvements of our approach do indeed manifest in the form of improved empirical performance over previous agnostic boosting approaches on commonly used datasets.

2 Related work

The possibility of boosting was first posed in [KV94], and was resolved positively in a remarkable result due to [Sch90] for the realizable case. The Adaboost algorithm [FS97] paved the way for its practical applications (notably in [VJ01]). We refer the reader to [SF13] for a comprehensive text that surveys the many facets of boosting, including its connections to game theory and online learning. See also [HLR23, GLR22, AGHM21] for recent developments.

The fact that Adaboost and its natural variants are brittle in presence of label noise and lack of realizability [LS08] prompted the search for boosting algorithms in the realizable plus label noise [DIK + 21, KS03] and agnostic learning models [BDLM01, MM02, KMV08, Kal07, CBC16]. In general, these boosting models are incomparable: although agnostic learning implies success in the random noise model, agnostic weak learning also constitutes a stronger assumption. Early agnostic boosting results could not boost the learner's accuracy to match that of the best in-class hypothesis; this limitation was tied to their notion of agnostic weak learning. Our work is most closely related to [KK09, Fel10]; we use the same notion of agnostic weak learning.

Boosting has also been extended to the online setting. [BKL15, CLL12, JGT17] study boosting in the mistake bound (realizable) model, while [BCHM20, RT22, HS21] focus on regret minimization. Our scheme of data reuse is inspired by variance reduction techniques [SLRB17, JZ13, FLLZ18, ABS23] in convex optimization, although there considerations of uniform convergence and generalization are absent, and our algorithm does not admit a natural gradient descent interpretation.

3 Problem setting

Let D ∈ ∆( X×{± 1 } ) be the joint population distribution over features, chosen from X , and (signed) binary labels with respect to which a classifier's h : X → {± 1 } performance may be assessed. The

2

-

z

(

z

+2)

e

-

z

performance criterion we consider is the 0-1 loss over the true labels and the classifier's predictions.

̸

̸

Relatedly, one may measure the correlation between the classifier's predictions and the true labels.

Note that for signed binary labels, we have that l D ( h ) = 1 2 (1 -corr D ( h )) . Therefore, these notions are equivalent in that a classifier that maximizes the correlation with true labels also minimizes the 0-1 loss, and vice versa, even in a relative error sense.

Definition 1 (Agnostic Weak Learner) . A learning algorithm is called a γ -agnostic weak learner with sample complexity m : R + × R + → N ∪ { + ∞} with respect to a hypothesis class H and a base hypothesis class B if, for any ε 0 , δ 0 > 0 , upon being given m ( ε 0 , δ 0 ) independently and identically distributed samples from any distribution D ′ ∈ ∆( X × {± 1 } ) , it can output a base 2 hypothesis W ∈ B such that with probability 1 -δ 0 we have

As remarked in [KK09], typically m ( ε, δ ) = O ((log |B| /δ ) /ε 2 ) , and we use this fact in compiling Table 1. However, following [KK09, BCHM20], we state our main result for fixed ε 0 , δ 0 in Theorem 4, where a necessary and irreducible ε 0 term shows up in our final accuracy.

Although not explicitly mentioned in our weak learning definition, our algorithm falls within the distribution-specific boosting framework [KK09, Fel10]. In particular, like previous work on agnostic boosting, Algorithm 1 can be implemented by relabeling examples , instead of adaptively reweighing them. Thus, the overall marginal distribution of any D ′ fed to the learner on the feature space X is the same as that induced by the population distribution D on X . Under such promise on inputs, distribution-specific weak learners may be easier to find.

4 The algorithm and main results

Notations. Given two functions f, g : X → R and generic scalars α, β ∈ R , we use af + bg to denote a function such that ( αf + βg )( x ) = αf ( x ) + βg ( x ) for all x ∈ X . Given a function f : X → R , we take sign( f ) to be a function such that for all x ∈ X , sign( f )( x ) = 1 ( f ( x ) ≥ 0) -1 ( f ( x ) < 0) . Define a filtration sequence {F t : t ∈ N ≥ 0 } , where F t capture all source of randomness the algorithm is subject to in the first t iterations. For brevity, we define E t [ · ] = E [ ·|F t ] . For any feature-label dataset ̂ D , we use E ̂ D and corr ̂ D to denote the empirical average and empirical correlation over ̂ D .

Potential function. Define the potential function ϕ : R → R as

Algorithm 1 Agnostic Boosting via Sample Reuse

1: Inputs: Sampling oracle for D supported on X × {± 1 } , γ -agnostic weak learning oracle W , step-size η , mixing parameter σ , number of iterations T , per-iteration sample size S , resampling parameter m , branching tolerance τ , post-selection sample size S 0 , potential ϕ : R → R .

2: Initialize a zero hypothesis H 1 = 0 .

3: for t = 1 to T do

4: Sample S IID examples from the distribution D to create a dataset ̂ D t .

5: Construct a sampling distribution D t that samples ( x, y ) uniformly from ̂ D t if t = 1 , and for t > 1 produces IID samples ( x, ˆ y ) as follows:

I With probability 1 -σ , return a sample ( x, ˆ y ) from D t -1 .

II With remaining probability σ , draw η ′ ∼ Unif [0 , η ] , pick ( x, y ) uniformly from ̂ D t , construct a pseudo label ˆ y = { +1 with probability p t ( x, y, η ′ ) , -1 with probability 1 -p t ( x, y, η ′ ) , and return ( x, ˆ y ) , where

6: Sample m samples from D t to create another dataset ̂ D ′ t .

7: Call the weak learning oracle on ̂ D ′ t to get W t = W ( ̂ D ′ t ) .

8: Measure the empirical correlation of W t on ̂ D ′ t as corr ̂ D ′ t ( W t ) = ∑ ( x, ̂ y ) ∈ ̂ D ′ t ̂ yW t ( x ) .

9: Set h t = W t /γ if corr ̂ D ′ t ( W t ) > τ else h t = -sign( H t ) .

10: Update H t +1 = H t + ηh t .

11: end for

12: Sample S 0 IID examples from the distribution D to create a dataset ̂ D 0 .

13: Output the hypothesis h = arg max h ∈{ sign( H t ): t ∈ [ T ] } ∑ ( x,y ) ∈ ̂ D 0 yh ( x ) .

We can use this to assign a population potential to any real-valued hypothesis H : X → R as

To maximize correlation between true labels and the hypothesis H 's outputs, one wants H ( x ) > 0 whenever y = +1 for most samples drawn from the underlying distribution. Since ϕ is a monotonically decreasing function, equivalently, higher classifier accuracy typically corresponds to lower values of population potential. However, there are limits to the utility of this argument: a low value of the potential alone does not translate to successful learning. In agnostic learning, one is concerned not with the error of the learned classifier per se , but with its excess error over the best in-class hypothesis. We will provide a precise relation between the potential and the excess error in Lemma 3.

We will use the following properties of ϕ (proved in Appendix C). Going forward, these will be the sole characteristics of ϕ we will appeal to. The potential we use is similar to the one used in [KK09, Dom00], but has been modified to remove a jump discontinuity in the second derivative at z = 0 as our approach requires a twice continuously differentiable potential.

Lemma 2. We make the following elementary observations about ϕ :

ϕ is convex and in C 2 , i.e., it is two-times continuously differentiable everywhere.

ϕ is non-negative on R and ϕ (0) = 2 .

III. For all z ∈ R , ϕ ′ ( z ) ∈ [ -1 , 0] . Further, for any z < 0 , ϕ ′ ( z ) = -1 .

IV. ϕ is 1 -smooth, i.e., ∀ z ∈ R , ϕ ′′ ( z ) ≤ 1 .

For any real-valued hypotheses H , h and g on X , to ease analysis, we introduce

Equivalently, Φ ′ D ( H,h ) can be characterized as the D -induced semi inner product between the functional derivative ∂ Φ D ( H ) /∂H and h . But for ease of presentation, we forgo this formal interpretation in favor of the literal one stated above.

A key property of the above potential is stated in the next lemma (proved in Appendix C). It gives us a strategy to control the relative error of the learned hypothesis, the quantity on the right, by individually minimizing both terms on the left, as we discuss next.

Lemma 3. For any distribution D ∈ ∆( X ×{± 1 } ) , real-valued hypothesis H : X → R , and binary hypothesis h ∗ : X → {± 1 } , we have

Description of the algorithm. Each round of Algorithm 1 adds some multiple of either the weak hypothesis W t or -sign ( H t ) to the current ensemble H t ; this choice is dictated by the empirical correlation of the weak hypothesis on the dataset it was fed. The construction of the relabeled distribution D t via our data reuse scheme ensures that if any hypothesis in the base class B produces sufficient correlation on it, its addition to the ensemble must decrease the potential Φ associated with the ensemble. Concretely, as we prove in Lemma 6, for all h ∈ B , corr D t ( h ) closely tracks -Φ ′ ( H t , h ) . The key to our improved sample complexity is the fact that this invariant can be maintained while sampling only S ≈ 1 /γε fresh samples each round, by repurposing samples from earlier rounds to construct the dataset fed to the weak learner. The mixing parameter σ controls the proportion of these two sources of samples used to construct D t .

However, this explanation is opaque when it comes to motivating the need for -sign ( H t ) . Let's rectify that: let h ∗ ∈ arg min h ∈H corr D ( h ) be the best in-class hypothesis. If -Φ ′ D ( H t , h ∗ ) is sufficiently large, so is corr D t ( h ∗ ) by Lemma 6, which assures us a non-negligible weak learning edge. As such -Φ ′ D ( H t , W t ) is large and the potential drops. If the weak hypothesis fails to make sufficient progress, the algorithms adds -sign( H t ) to the ensemble, which, again by Lemma 6 corresponds to decreasing the potential value by some function of -Φ ′ D ( H t , -sign( H t )) = Φ ′ D ( H t , sign( H t )) , using linearity of Φ ′ D in its second argument. Thus, when run for sufficiently many iterations, because the potential is bounded above at initialization both the terms on the left side of Lemma 3 must become small. This then implies a bounded correlation gap between the best in-class hypothesis, and the majority vote of the ensembles considered in some iteration; the last line picks the best of these.

4.1 Main result for finite hypotheses classes

The key feature of our algorithm is that it is designed to reuse samples across successive rounds of boosting. The soundness of this scheme, which Lemma 6 substantiates, is based on the observation that in each round H t changes by a small amount, thereby inducing an incremental change in the distribution fed to the weak learner. Although our algorithm needs a total number of iterations comparable to [KK09], this data reuse lowers the number of fresh samples needed every iteration to just 1 /γε , instead of 1 /γ 2 ε 2 , resulting in improved sample complexity, as we show next.

Theorem 4 (Main result for finite hypotheses class) . Choose any ε, δ > 0 . There exists a choice of η, σ, T, τ, S 0 , S, m satisfying 3 T = O ((log |B| ) /γ 2 ε 2 ) , η = O ( γ 2 ε/ log |B| ) , σ = η/γ, τ = O ( γε ) , S = O (1 /γε ) , S 0 = O (1 /ε 2 ) , m = m ( ε 0 , δ 0 ) + O (1 /γ 2 ε 2 ) such that for any γ -agnostic weak learning oracle (as defined in Definition 1) with fixed tolerance ε 0 and failure probability δ 0 , Algorithm 1 when run with the potential defined in (1) produces a hypothesis h such that with probability 1 -10 δ 0 T -10 δT ,

while making T = O ((log |B| ) /γ 2 ε 2 ) calls to the weak learning oracle, and sampling TS + S 0 = O ((log |B| ) /γ 3 ε 3 ) labeled examples from D .

In Appendix B, we provide a different result (Theorem 9), also using Algorithm 1, where the learner makes O (1 /γ 2 ε 2 ) call to weak learner, exactly matching the oracle complexity of existing results, while drawing O ((log |B| ) /γ 3 ε 3 +(log |B| ) 3 /γ 2 ε 2 ) samples.

4.2 Extensions to infinite classes

As mentioned in Section 1.1, the reuse of samples prohibits us from appealing to symmetrization and Rademacher complexity based arguments. Instead, our generalization to infinite classes is based

on L 1 covering numbers. Using empirical process theory [VDVW97], we upper bound L 1 covering number by a suitable function of VC dimension, yielding the following result (proved in Appendix F).

Theorem 5 (Main result for VC Dimension) . There exists a setting of parameters such that for any for any γ -agnostic weak learning oracle with fixed tolerance ε 0 and failure probability δ 0 , Algorithm 1 produces a hypothesis h such that with probability 1 -10 δ 0 T -10 δT ,

while making T = O ( VC ( B ) /γ 2 ε 2 ) calls to the weak learning oracle, and sampling TS + S 0 = O ( VC ( B ) /γ 3 ε 3 ) labeled examples from D .

5 Sketch of the analysis

In this section, we provide a brief sketch of the analysis outlined in the proof of Theorem 4. Our intent is to convey the plausibility of a 1 /ε 3 sample complexity result, up to the exclusion of other factors. Hence, ≈ and ≲ inequalities below only hold up to constants and polynomial factors in other paramters, e.g., in γ, log |B| . Formal proofs are reserved for Appendix D.

Bounding the correlation gap (Theorem 4). A central tool in bounding the correlation gap is Lemma 3. We want to ensure for some t , since our algorithm at the end picks the best one, that

On the other hand, using 1 -smoothness of ϕ , we can upper bound Φ D on successive iterates as Φ D ( H t +1 ) ≤ Φ D ( H t ) + η Φ ′ D ( H t , h t ) + η 2 / 2 γ 2 . Rearranging this to telescope the sum produces

Hence, by setting a η ≈ 1 / √ T , we know that there exists a t where -Φ ′ D ( H t , h t ) ≲ 1 / √ T .

In Lemma 6, we establish that the core guarantee our data resue scheme provides: that for all h in the base class B and for h ∗ , the correlation on the resampled distribution D t constructed by the algorithm tracks the previously stated quantity of interest -Φ ′ D ( H t , · ) .

Lemma 6. There exists a C > 0 such that with probability 1 -δ , for all t ∈ [ T ] and h ∈ B ∪ { h ∗ } , we have

Using the definition of weak learner, we know that corr D t ( h ∗ ) ≲ corr D t ( W t ) /γ . Now, using Lemma 6 twice and the linearity of Φ ′ D ( H, · ) , we get

Now, if at each step we could choose h t ∈ {-sign( H t ) , W t /γ } , which ever maximized -Φ ′ D ( H t , · ) , we would have for some t that

Alas, -sign( H t ) is not in B , thus corr D t ( -sign( H t )) can be really far from -Φ ′ D ( H t , -sign( H t )) , i.e., Lemma 6 does not apply to -sign ( H t ) . To circumvent this, instead of choosing the maximizer, in the algorithm and in the actual proof, we use a relaxed criteria for choosing between W t /γ and -sign ( H t ) that depends on the correlation of W t on D t alone, and hence can be efficiently evaluated. The spirit of this modification is to adopt -sign ( H t ) only if W t by itself fails to make enough progress, the threshold for which can be stated in the terms of target accuracy.

Generalization over B via sample reuse (Lemma 6). Here, we sketch a proof of Lemma 6 that ties the previous proof sketch together, and forms the basis of our sample reuse scheme. Our starting point is the following claim (Claim 10) that uses the fact that ϕ is second-order differentiably continuous to arrive at the fact that for any t and h : X → R , we have

Simultaneously, using E t -1 to condition on the randomness in the first t -1 rounds, by construction, our data reuse and relabeling scheme gives us

Thus, suitably scaling and adding the two together, we get the identity that

where ∆ t = Φ ′ D ( H t , h ) + ( 1 + η σ ) corr D t ( h ) . Thus, ∆ t forms a martingale-like sequence; the sign of ∆ t is indeterminate. To establish concentration, we apply Freedman's inequality, noting that the conditional variance of the associated martingale difference sequence scales as 1 / √ S . A union bound over B ∪ { h ∗ } yields the claim. Relatedly, to reach a better oracle complexity in Theorem 9, we show a uniform high-probability on the martingale difference sequence, and then apply a variant of Freedman's inequality [Pin94, Pin20] that can adapt to martingale difference sequences that are bounded with high probability instead of almost surely.

6 Applications

In this section, we detail the implications of our results for previously studied learning problems.

6.1 Boosting for reinforcement learning

[BHS22] initiated the approach of boosting weak learners to construct a near-optimal policy for reinforcement learning. Plugging Algorithm 1 into their meta-algorithm yields the following result for binary-action MDPs, improving upon the sample complexity in [BHS22]. Here, V π is the expected discounted reward of a policy π , V ∗ is its maximum. β is the discount factor of the underlying MDP, and C ∞ , D ∞ and E , E ν are distribution mismatch and policy completeness terms (related to the inherent Bellman error). In the episodic model , the learner interacts with the MDP in episodes. In the ν -reset model , the learner can seed the initial state with a fixed well dispersed distribution ν as a means to exploration. See Appendix G for a complete statement of results and details of the setting.

Theorem 7 (Informal; stated formally in Theorem 22) . Let W be a γ -weak learner for the policy class Π operating with a base class B , with sample complexity m ( ε 0 , δ 0 ) = (log |B| /δ 0 ) /ε 2 0 . Fix tolerance ε and failure probability δ . In the episodic access model, there is an algorithm using that uses the weak learner W to produce a policy π such that with probability 1 -δ , we have V ∗ -V π ≤ ( C ∞ E ) / (1 -β )+ ε, while sampling O ((log |B| ) /γ 3 ε 4 ) episodes of length O ((1 -β ) -1 ) . In the ν -reset access model, there is a setting of parameters such that Algorithm 2 when given access to W produces a policy π such that with probability 1 -δ , we have V ∗ -V π ≤ ( D ∞ E ν ) / (1 -β ) 2 + ε , while sampling O ((log |B| ) /γ 3 ε 5 ) episodes of length O ((1 -β ) -1 ) .

6.2 Agnostically learning halfspaces

We apply our algorithm in a black-box manner to agnostically learn halfspaces over the n -dimensional boolean hypercube when the data distribution has uniform marginals on features. The aim is this section is not to obtain the best known bounds, but rather to provide an example illustrating that agnostic boosting is both a viable and flexible approach to construct agnostic learners, and where our improvements carry over. Following [KK09], we use ERM over the parities of degree at most d , for d ≈ 1 /ε 4 , as our weak learners; the the weak learner's edge here is γ = n -d . An application of our boosting algorithm (proved in Appendix I) to this problem improves the sample complexity of O ( ε -8 n 80 ε -4 ) indicated in [KK09].

Theorem 8. Let D be any distribution over {± 1 } n ×{± 1 } with uniform distribution over features. By H = { sign( w ⊤ x -θ ) : ( w,θ ) ∈ R n +1 } , denote the class of halfspaces. There exists some d such that running Algorithm 1 with ERM over parities of degree at most d produces a classifier h such that l D ( ¯ h ) ≤ min h ∈H l D ( h ) + ε , while using O ( ε -7 n 60 ε -4 ) samples in n poly (1 /ε ) time.

Note that ERM over the class of halfspaces directly, although considerably more sample efficient, takes (1 /ε ) poly ( n ) time, i.e., it is exponentially slower for moderate values of ε . There are known statistical query lower bounds [DKZ20] requiring n poly ( ε -1 ) queries for agnostic learning of halfspaces with Gaussian marginals, suggesting that a broad class of algorithms, regardless of the underlying parametrization, might not fare any better. For completeness, we note that a better sample complexity is attainable by direct L 1 -approximations of halfspaces via low-degree polynomials [DGJ + 10], instead of the approach taken in [KK09, KOS04] and mirrored here which first constructs an L 2 -approximation, but such structural improvements apply equally to presented and compared results.

7 Experiments

In Table 3, we report the results of preliminary experiments with Algorithm 1 against the agnostic boosting algorithms in [KK09] and [BCHM20] as baselines on UCI classification datasets [SWHB89, HRFS99, SED + 88], using decision stumps [PVG + 11] as weak learners. We also introduce classification noise of 5%, 10% and 20% during training to measure the robustness of the algorithms to label noise. Accuracy is estimated using 30 -fold cross validation with a grid search over the mixing weight σ and the number of boosters T . The algorithm in [KK09] does not reuse samples between rounds, while [BCHM20] uses the same set of samples across all rounds. In contrast, Algorithm 1 blends fresh and old samples every round, with σ controlling the proportion of each. See Appendix J for additional details. We note that the Ionsphere dataset includes 351 samples, while Diabetes contains 768, and Spambase contains 4601. The benefits of sample reuse are less stark in a data-rich regime. This could explain some of the under-performance on Spambase, disregarding which the proposed algorithm substantially outperforms the alternatives.

8 Conclusion

We give an agnostic boosting algorithm with a substantially lower sample requirement than ones known, enabled by efficient recency-aware data reuse between boosting iterations. Improving our oracle complexity or proving its optimality, and closing the sample complexity gap to ERM are interesting directions for future work.

References

[ABS23] Naman Agarwal, Brian Bullins, and Karan Singh. Variance-reduced conservative policy iteration. In Shipra Agrawal and Francesco Orabona, editors, Proceedings of The 34th International Conference on Algorithmic Learning Theory , volume 201 of Proceedings of Machine Learning Research , pages 3-33. PMLR, 20 Feb-23 Feb 2023.

[AGHM21] Noga Alon, Alon Gonen, Elad Hazan, and Shay Moran. Boosting simple learners. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing , pages 481-489, 2021.

[BBL05] Stéphane Boucheron, Olivier Bousquet, and Gábor Lugosi. Theory of classification: A survey of some recent advances. ESAIM: probability and statistics , 9:323-375, 2005.

[BCHM20] Nataly Brukhim, Xinyi Chen, Elad Hazan, and Shay Moran. Online agnostic boosting via regret minimization. Advances in Neural Information Processing Systems , 33:644654, 2020.

[BDLM01] Shai Ben-David, Philip M Long, and Yishay Mansour. Agnostic boosting. In Computational Learning Theory: 14th Annual Conference on Computational Learning Theory, COLT 2001 and 5th European Conference on Computational Learning Theory, EuroCOLT 2001 Amsterdam, The Netherlands, July 16-19, 2001 Proceedings 14 , pages 507-516. Springer, 2001.

[BHS22] Nataly Brukhim, Elad Hazan, and Karan Singh. A boosting approach to reinforcement learning. Advances in Neural Information Processing Systems , 35:33806-33817, 2022.

[BKL15] Alina Beygelzimer, Satyen Kale, and Haipeng Luo. Optimal and adaptive algorithms for online boosting. In International Conference on Machine Learning , pages 2323-2331. PMLR, 2015.

[CBC16] Shang-Tse Chen, Maria-Florina Balcan, and Duen Horng Chau. Communication efficient distributed agnostic boosting. In Artificial Intelligence and Statistics , pages 1299-1307. PMLR, 2016.

[CLL12] Shang-Tse Chen, Hsuan-Tien Lin, and Chi-Jen Lu. An online boosting algorithm with theoretical justifications. In Proceedings of the 29th International Coference on International Conference on Machine Learning , pages 1873-1880, 2012.

[DGJ + 10] Ilias Diakonikolas, Parikshit Gopalan, Ragesh Jaiswal, Rocco A Servedio, and Emanuele Viola. Bounded independence fools halfspaces. SIAM Journal on Computing , 39(8):3441-3462, 2010.

[DIK + 21] Ilias Diakonikolas, Russell Impagliazzo, Daniel M Kane, Rex Lei, Jessica Sorrell, and Christos Tzamos. Boosting in the presence of massart noise. In Conference on Learning Theory , pages 1585-1644. PMLR, 2021.

[DKZ20] Ilias Diakonikolas, Daniel Kane, and Nikos Zarifis. Near-optimal sq lower bounds for agnostically learning halfspaces and relus under gaussian marginals. Advances in Neural Information Processing Systems , 33:13586-13596, 2020.

[Dom00] C Domingo. Madaboost: a modification of adaboost. In Proc. of the 13th Conference on Computational Learning Theory, COLT'00 , 2000.

[Dud74] Richard M Dudley. Metric entropy of some classes of sets with differentiable boundaries. Journal of Approximation Theory , 10(3):227-236, 1974.

[Fel10] Vitaly Feldman. Distribution-specific agnostic boosting. Innovations in Theoretical Computer Science (ITCS) , pages 241-250, 2010.

[FLLZ18] Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator. Advances in neural information processing systems , 31, 2018.

[Fre75] David A Freedman. On tail probabilities for martingales. the Annals of Probability , pages 100-118, 1975.

[FS97] Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences , 55(1):119-139, 1997.

[GLR22] Kasper Green Larsen and Martin Ritzert. Optimal weak to strong learning. Advances in Neural Information Processing Systems , 35:32830-32841, 2022.

[Hau95] David Haussler. Sphere packing numbers for subsets of the boolean n-cube with bounded vapnik-chervonenkis dimension. Journal of Combinatorial Theory, Series A , 69(2):217-232, 1995.

[HLR23] Mikael Møller Høgsgaard, Kasper Green Larsen, and Martin Ritzert. AdaBoost is not an optimal weak to strong learner. In Proceedings of the 40th International Conference on Machine Learning , Proceedings of Machine Learning Research, pages 13118-13140. PMLR, 2023.

[HRFS99] Mark Hopkins, Erik Reeber, George Forman, and Jaap Suermondt. Spambase. UCI Machine Learning Repository, 1999. DOI: https://doi.org/10.24432/C53G6X.

[HS21] Elad Hazan and Karan Singh. Boosting for online convex optimization. In International Conference on Machine Learning , pages 4140-4149. PMLR, 2021.

[JGT17] Young Hun Jung, Jack Goetz, and Ambuj Tewari. Online multiclass boosting. Advances in neural information processing systems , 30, 2017.

[JZ13] Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. Advances in neural information processing systems , 26, 2013.

[Kal07] Satyen Kale. Boosting and hard-core set constructions: a simplified approach. In Electronic Colloquium on Computational Complexity (ECCC) , volume 14. Citeseer, 2007.

[KK09] Varun Kanade and Adam Kalai. Potential-based agnostic boosting. Advances in neural information processing systems , 22, 2009.

[KL02] Sham Kakade and John Langford. Approximately optimal approximate reinforcement learning. In Proceedings of the Nineteenth International Conference on Machine Learning , pages 267-274, 2002.

[KMV08] Adam Tauman Kalai, Yishay Mansour, and Elad Verbin. On agnostic boosting and parity learning. In Proceedings of the fortieth annual ACM symposium on Theory of computing , pages 629-638, 2008.

[KOS04] Adam R Klivans, Ryan O'Donnell, and Rocco A Servedio. Learning intersections and thresholds of halfspaces. Journal of Computer and System Sciences , 68(4):808-840, 2004.

[KS03] Adam Kalai and Rocco A Servedio. Boosting in the presence of noise. In Proceedings of the thirty-fifth annual ACM symposium on Theory of computing , pages 195-205, 2003.

[KV94] Michael Kearns and Leslie Valiant. Cryptographic limitations on learning boolean formulae and finite automata. Journal of the ACM (JACM) , 41(1):67-95, 1994.

[LS08] Philip M Long and Rocco A Servedio. Random classification noise defeats all convex potential boosters. In Proceedings of the 25th international conference on Machine learning , pages 608-615, 2008.

[MM02] Yishay Mansour and David McAllester. Boosting using branching programs. Journal of Computer and System Sciences , 64(1):103-112, 2002.

[MRT18] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning . MIT press, 2018.

[Pin94] Iosif Pinelis. Optimum bounds for the distributions of martingales in banach spaces. The Annals of Probability , pages 1679-1706, 1994.

[Pin20] Iosif Pinelis. Extension of bernstein's inequality when the random variable is bounded with large probability. MathOverflow, 2020. URL:https://mathoverflow.net/q/371436 (version: 2020-09-11).

[Put14] Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming . John Wiley & Sons, 2014.

[PVG + 11] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011.

[RT22] Vinod Raman and Ambuj Tewari. Online agnostic multiclass boosting. Advances in Neural Information Processing Systems , 35:25908-25920, 2022.

[Sch90] Robert E Schapire. The strength of weak learnability. Machine learning , 5:197-227, 1990.

[SED + 88] Jack W Smith, James E Everhart, WC Dickson, William C Knowler, and Robert Scott Johannes. Using the adap learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the annual symposium on computer application in medical care , page 261. American Medical Informatics Association, 1988.

[SF13] Robert E Schapire and Yoav Freund. Boosting: Foundations and algorithms. Kybernetes , 42(1):164-166, 2013.

[SLRB17] Mark Schmidt, Nicolas Le Roux, and Francis Bach. Minimizing finite sums with the stochastic average gradient. Mathematical Programming , 162:83-112, 2017.

[SSBD14] Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms . Cambridge university press, 2014.

[SWHB89] V. Sigillito, S. Wing, L. Hutton, and K. Baker. Ionosphere. UCI Machine Learning Repository, 1989. DOI: https://doi.org/10.24432/C5W01B.

[VDVW97] Aad W Van Der Vaart and Jon A Wellner. Weak convergence and empirical processes: with applications to statistics . Springer New York, 1997.

[VJ01] Paul Viola and Michael Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001 , volume 1, pages I-I. Ieee, 2001.

Appendix

Limitations. The primary contribution of this work is theoretical. Extensively demonstrating the empirical efficacy of the proposed approach and fully characterizing when it fares best is left to future work. Further, in comparison to realizable boosting (with or without label noise), the existence of an agnostic weak leaner here is a stronger assumption. Nevertheless, we believe, especially given the recent interest in agnostic boosting, the current work presents a substantial and concrete improvement over the state of the art, and is a first step in making agnostic boosting practical.

Organization of the appendix. In Appendix A, we point out an error in the branching criteria in [KK09]. Next, in Appendix B, we give a second guarantee on our algorithm that improves the oracle complexity at a vanishing (in ε ) cost to the sample complexity. In Appendices C and D, we prove the main result and the lemmas leading up to it. Appendix E provides a proof of the improved result in Appendix B. Appendix F furnishes a proof for the claims concerning extensions to infinite classes. In Appendices G and H, we define the reinforcement learning setup, formally state the RL boosting result along with accompanying algorithms, and prove it. Appendix I substantiates our improved sample complexity bound for learning halfspaces. Finally, in Appendix J, we provide additional experimental details, and Appendix K provides a practical guide for adapting the proposed algorithm.

A Branching criteria in [KK09]

At a high level, the boosting algorithm in [KK09] is an iterative one, that adds either the weak hypothesis or the negation of the sign of the present ensemble to the ensemble mixture in each of the T ≈ 1 /ε 2 rounds. The algorithm makes use of samples to fulfill two objectives: (a) provide m ≈ 1 /ε 2 samples to the weak learner to obtain a weak hypothesis that generalizes, (b) use s ≈ 1 /ε 2 samples to decide whether to add the weak hypothesis or the voting classifier to the current mixture, by comparing the empirical performance on both on said samples. The algorithm uses fresh samples for part (a) every round; this is sound and contributes Tm ≈ 1 /ε 4 to the net sample complexity.

However, as described in [KK09], the algorithm reuses the same s samples for part (b) across all rounds. This means these s samples determine which of the two choices gets added to the mixture at the end of the first step, and hence H 1 . In the next step, however, because H 1 through relabelling of new samples determines the weak hypothesis, these samples are no longer IID with respect to the weak hypothesis or -sign( H 1 ) , since they have already played a demonstrable part in determining it. This effects occur and compound at all time steps, not just the first. In the analysis, on top of page 7, the analysis in [KK09] uses a Chernoff-Hoeffding bound to say that the performance of the weak hypothesis and -sign( H 1 ) on these s samples transfers approximately to the population. However, this inequality may only be applied to IID random variables.

In the case of [KK09], there is a simple and satisfactory fix: resample these s examples from the population distribution every round. The sample complexity, now T ( m + s ) instead of Tm + s , remains O (1 /ε 4 ) , and no change to the performance gurantee needs to be made.

In our adaptation, this highlights an additional challenge. Even if one were to sate the weak learner with a total fewer number of samples, i.e., perform part (a) with fewer samples, through a uniform convergence result on the base hypothesis (crucially, not the boosted hypothesis , whose complexity grows with number of iterations), part (b) requires s fresh samples, and hence 1 /ε 4 samples in total, in determining which of the weak hypothesis or -sign of the ensemble provides a greater magnitude of descent. Here, note that -sign( H t ) lies outside the base class. To circumvent this, we give a branching criteria to decide which component to add to the present mixture, based on the performance of the weak hypothesis, the only component whose performance is estimable with bounded generalization error, alone on the reused data. Concretely, we introduce a threshold τ to choose the weak hypothesis, if it makes at least τ progress, and otherwise, choose -sign( H t ) even if in truth it is worse than the weak hypothesis. This deviation requires careful handling in the proof.

B Improved oracle complexity

Here, we provide a different result where Algorithm 1 makes O (1 /γ 2 ε 2 ) call to weak learner, matching exatcly the oracle complexity of existing results, while drawing O ((log |B| ) /γ 3 ε 3 +

(log |B| ) 3 /γ 2 ε 2 ) samples. Notice that the second term in the sample complexity has a smaller order, whenever ε is sub-constant. The proof may be found in Appendix E.

Theorem 9 (Improved oracle complexity for finite hypotheses class) . Choose any ε, δ > 0 . There exists a choice of η, σ, T, τ, S 0 , S, m satisfying T = O (1 /γ 2 ε 2 ) , η = O ( γ 2 ε ) , σ = η/γ, τ = O ( γε ) , S = O ((log |B| ) /γε +(log |B| ) 3 ) , S 0 = O (1 /ε 2 ) , m = m ( ε 0 , δ 0 ) + O (1 /γ 2 ε 2 ) such that for any γ -agnostic weak learning oracle (as defined in Definition 1) with fixed tolerance ε 0 and failure probability δ 0 , Algorithm 1 when run with the potential defined in (1) produces a hypothesis h such that with probability 1 -10 δ 0 T -10 δT ,

while making T = O (1 /γ 2 ε 2 ) calls to the weak learning oracle, and sampling TS + S 0 = O ((log |B| ) / ( γ 3 ε 3 ) + (log |B| ) 3 / ( γ 2 ε 2 )) labeled examples from D .

C Proof of auxiliary lemmas

Proof of Lemma 2. Part 2 is evident from the definition. Taking care that right and left first (and second) derivatives exist and are equal at z = 0 , by explicit computation, we have

From this, one may immediately verify Part 1, since ϕ ′′ is non-negative. Non-negativity of ϕ ′′ also implies that ϕ ′ is non-decreasing, and hence, being clearly non-positive, is between [ -1 , 0] ; this is Part 3. By elementary calculus, ϕ ′′ is maximize at z = 1 where it equals 1 /e implying the conclusion in Part 4.

Proof of Lemma 3. By the definition of Φ ′ D , we have

Note that whenever yH ( x ) < 0 , ϕ ′ ( yH ( x )) = -1 , which Lemma 2.III attests to. On the other hand, if yH ( x ) ≥ 0 , since h ∗ is a binary classifier, y sign ( H )( x ) = 1 ≥ yh ∗ ( x ) , and since, again by Lemma 2.III, ϕ ′ ( yH ( x )) ≥ -1 , we have in this case

Plugging these into the previous derivation, we arrive at

finishing the proof of the claim.

D Proofs for the main result

Proof of Theorem 4. Let h ∗ ∈ arg min h ∈H corr D ( h ) . Using the update rule for H t +1 and the fact that ϕ is 1 -smooth (Lemma 2.IV), we arrive at

using the definition of Φ ′ D , and that h t is either a binary classifier, or a 1 /γ -scaled version of it. Rearranging this to telescope the sum produces

where we use the fact that Φ D ( 0 ) = 2 , and that it is non-negative (Lemma 2.II).

Hence, ∃ t ∈ [ T ] , such that -Φ ′ D ( H t , h t ) is small. Our proof strategy going forward is to use this fact to imply that both the terms on left side of inequality in Lemma 3, namely, -Φ ′ D ( -sign ( H t )) and -Φ ′ D ( H t , h ∗ ) , are small for some t , implying a small correlation gap on the population distribution.

Note that if corr D ′ t ( W t ) ≥ τ , the algorithm sets h t = W t /γ , else it chooses h t = -sign ( H t ) . We analyze these cases separately. For both, Lemma 6 which relates the empirical correlation on D t with Φ ′ D will prove indispensable. Going forward, we will define ε Gen as indicated above to capture the generalization error over the base hypothesis class.

For brevity of notation, we condition the analysis going forward on three events. Let E A be that event that for all t , | corr D t ( W t ) -corr ̂ D ′ t ( W t ) | ≤ ε ′ / 10 . Since ̂ D ′ t is constructed from IID samples from D t , setting m ≥ 100 /ε ′ 2 √ log T/δ , Pr( E A ) ≥ 1 -δ , by an application of Hoeffding's inequality and union bound over t . Here, note that unlike S setting a higher value of m doesn't increase the number of points sampled from D , since D ′ t is resampled from already collected data. Similarly, denote by E B the event that for all t , | corr D ( sign ( H t )) -corr ̂ D 0 ( sign ( H t )) | ≤ ε ′′ / 10 . Again, by Hoeffding's inequality and union bound over t , choosing S 0 = 100 /ε ′′ 2 √ log T/δ , we have Pr( E B ) ≥ 1 -δ , since the samples in ̂ D 0 were chosen independently of those used to compute H t 's. Finally, we will take the success of Lemma 6 (call this E C ) for granted in the analysis below, for brevity of notation, conditioning our analysis on all three events.

Case A: When h t = W t /γ . When this happens, corr D t ( W t ) ≥ corr D ′ t ( W t ) -ε ′ / 10 ≥ τ -ε ′ / 10 . Applying Lemma 6 and noting that Φ ′ D ( H t , · ) is linear in its argument, we have

Rearranging this

Case B: When h t = -sign ( H t ) . Here corr D t ( W t ) ≤ corr D ′ t ( W t ) + ε ′ / 10 ≤ τ + ε ′ / 10 . Applying Lemma 6, and using the weak learning condition (Definition 1), we have that

′

Combining the two. In either case, combining Equations (3) and (4), we have

or using the identity -max( -f ( x )) = min f ( x ) ,

Now, set

Now, either there exists some t such

or the minimum operator in the last expression always accepts the first clause, in which case for all t ,

which contradicts Equation (2). Henceforth let t ∗ be the iteration for which Equation (5) holds. Finally, given the event E B , we have

Compiling this with the inequality in Equation (5), we get

Setting ε ′ = ε/ 5 γ , ε ′′ = ε/ 5 and plugging in the proposed hyper-parameters with appropriate constants yields the claimed result.

Proof of Lemma 6. Fix any hypothesis h ∈ B . For any η ′ ≥ 0 , define H η ′ t = H t + η ′ h t , and

First, we derive recursive expansions of Φ ′ D ( H t , h ) and D t , the two quantities we wish to relate.

Claim 10. For any t and h : X → R , we have

Claim 11. For any t ,

Adding (1 + η/σ ) times the last expression to the expansion of Φ ′ D ( H t , h ) , we have

Using this, we conclude that ∆ ′ t = (1 -σ ) -t ∆ t forms a martingale sequence with respect to the {F t : t ∈ N ≥ 0 } filtration sequence, as ∆ ′ t = E t -1 [(1 -σ ) -t ∆ t ] = (1 -σ ) t -1 ∆ t -1 = ∆ ′ t -1 . The associated martingale difference sequence δ ′ t = ∆ ′ t -∆ ′ t -1 can be bounded both in worst-case and second-moment terms as we show next.

Now, we are ready to apply Freedman's inequality for martingales.

Theorem 13 (Freedman's inequality [Fre75]) . Consider a real-valued martingale { Y k : k ∈ Z ≥ 0 } with respect to some filtration sequence { Σ k : k ∈ Z ≥ 0 } , and let { X k : k ∈ Z > 0 } be the associated difference sequence. Assume that the difference sequence is uniformly bounded: | X k | ≤ R almost surely for k ∈ Z > 0 . Define the predictable quadratic variation process: W k := ∑ k j =1 E j -1 ( X 2 j ) for k ∈ Z > 0 , where E j -1 ( · ) := E ( · | Σ j -1 ) . Then, for all t ≥ 0 and σ 2 > 0 ,

Applying Freedman's inequality to ∆ ′ t , since the total conditional variance of our martingale difference sequence is bounded almost surely as shown before, we get for any

that

Hence, using ∆ ′ t = (1 -σ ) -t ∆ t , with probability 1 -δ , we have

Taking a union bound over the choice of h from B ∪ { h ∗ } and over t , along with repeating the argument on the martingale -∆ t to furnish the promised two-sided bound, concludes the claim.

Proof of Claim 10. Using the fundamental theorem of calculus and that ϕ ∈ C 2 , which Lemma 2.I certifies, we get

where use the fact that for binary labels y ∈ {-1 , 1 } , y 2 = 1 , and the definitions of Φ ′ D and Φ ′′ D .

Proof of Claim 11. First, we establish a recursive structure on the random random distribution D t without any conditional expectation in place.

Claim 14. For any t , we have

Using the fact that ̂ D t identically samples data points from D , we arrive at

Proof of Claim 14. By the definition of D t , we have

where in the first equality we use the fact that for any binary random variable Y supported on {-1 , 1 } with Pr( Y = 1) = p , we have E [ Y ] = 2 p -1 .

Proof of Claim 12. Using the recursive expansion of Φ ′ D (Claim 10) and D t (Claim 14), we have

Using Lemma 2, ϕ ′ , ϕ ′′ are uniformly bounded in magnitude by one, Φ ′ D ( H, · ) and Φ ′′ D ( H, · , g ) are uniformly bounded in magnitude by one for any H : X → R and 1 /γ -uniformly bounded function g . Hence, | δ ′ t | ≤ (1 -σ ) -t 2( σ + η/γ ) . To bound the conditional variance, we use the identity

( a + b ) 2 ≤ 2 a 2 +2 b 2 in the first line below to show

where we use the fact that the S samples consisting ̂ D t are independent and identically sampled from D conditioned on F t -1 in the second equality, and that the expectation of any function over ̂ D t is the equivalent sample average. Finally, using an identity on geometric sums, we get

where we appeal to the inequality 1 -(1 -σ ) 2 = 2 σ -σ 2 ≥ σ for any σ ∈ [0 , 1] .

E Proofs for improved oracle complexity

Proof of Theorem 9. The overall proof is similar to that of Theorem 4, with the exception of the following bound on ε Gen, which we state now (and prove next) in a new lemma relating the empirical correlation on D t with Φ ′ D . The principal difference between Lemma 6 and Lemma 15 is the presence of a 1 / √ S in both terms on the right in Lemma 15, however this comes at the cost of a higher polynomial dependence in log |B| in the second term on the right side of Lemma 15.

Lemma 15. There exists a C > 0 such that with probability 1 -δ , for all t ∈ [ T ] and h ∈ B ∪ { h ∗ } , we have

From here, with the new definition of ε Gen as defined above, identically following the steps in the proof of Theorem 4, which we skip to avoid repetition, we arrive at

Setting ε ′ = ε/ 5 γ , ε ′′ = ε/ 5 and plugging in the proposed hyper-parameters with appropriate constants yields the claimed result.

Proof of Lemma 15. The proof of the present lemma is largely similar to that of Lemma 6, with two exceptions: (a) we use a variant of Freedman's inequality that applies when the martingale difference sequences (for us, δ ′ t are bounded with high probability, instead of admitting an almost sure absolute bound; (b) to apply this result, we establish a high probability upper bound on δ ′ t that scales as 1 / √ S .

Fix any hypothesis h ∈ B . For any η ′ ≥ 0 , define H η ′ t = H t + η ′ h t , and

As before, adding (1 + η/σ ) times the expression in Claim 11 to the the expansion of Φ ′ D ( H t , h ) in Claim 10, we observe that E t -1 [∆ t ] = (1 -σ )∆ t -1 , and hence conclude that ∆ ′ t = (1 -σ ) -t ∆ t forms a martingale sequence with respect to the {F t : t ∈ N ≥ 0 } filtration sequence. As shown in Claim 12, the associated martingale difference sequence δ ′ t = ∆ ′ t -∆ ′ t -1 admits a total variance bound of ∑ t s =1 E s -1 [ δ ′ s 2 ] ≤ 8( σ 2 + η 2 /γ 2 ) σ (1 -σ ) 2 t S .

Now, we state a variant of Freedman's inequality that applies when martingale difference sequences are bounded with high probability.

Theorem 16 (Freedman's inequality with high probability bounds [Pin94, Pin20]) . Fix a positive integer n . Consider a real-valued martingale { Y k : k ∈ Z ≥ 0 } with respect to some filtration sequence { Σ k : k ∈ Z ≥ 0 } , and let { X k : k ∈ Z > 0 } be the associated difference sequence. Assume that the difference sequence is uniformly bounded with high probability for some R,δ ′ :

Define the predictable quadratic variation process: W n := ∑ n j =1 E j -1 ( X 2 j ) , where E j -1 ( · ) := E ( · | Σ j -1 ) . Then, for all t ≥ 0 and σ 2 > 0 ,

To apply this variant of Freedman's inequality, we establish a high probability bound on ∆ ′ t , that is, ignoring logarithmic terms, substantially better than the almost sure bound in Claim 12.

Claim 17. There exists a universal constant C , such that for any t , we have

Combining this with the almost sure bound on the total conditional variance of our martingale difference sequence, we get for any

that

Hence, using ∆ ′ t = (1 -σ ) -t ∆ t , with probability 1 -2 δ , we have

Taking a union bound over the choice of h from B ∪ { h ∗ } and over t , along with repeating the argument on the martingale -∆ t to furnish the promised two-sided bound, concludes the claim.

Proof of Claim 17. The proof begins similarly to the one for Claim 12. Using the recursive expansion of Φ ′ D (Claim 10) and D t (Claim 14), we have

However, this time, we note that, conditioned on F t -1 , A 1 and A 2 are averages of S zero-mean IID random variables, where each constituent is absolutely bounded by one in magnitude (Lemma 2). Hence, by Hoeffding's inequality, with 1 -2 δ , we have that

Note that since this statement holds true for all realizations in F t -1 - in words, it is a statement about the randomness in the present round alone - it remains true when not subject to such filtration's, i.e., by marginalizing over F t -1 . A union bound over t now yields the claim

F Proofs for extensions to infinite classes

Definition 18 (Covering number) . Given a set F in linear space V , a semi norm ∥·∥ on V , N ( ε, F , ∥·∥ ) is the size of the smallest set G such that for all f ∈ F , there exists a g ∈ G such that ∥ f -g ∥ ≤ ε .

Proof of Theorem 5. First, we state (and subsequently prove) the following sample complexity result using L 1 covering numbers. We note that it may be possible to improve this result for specific classes of functions, i.e., monotonic functions, by applying chaining techniques [Dud74] to L 2 distances.

Theorem 19 (Main result for Covering Number) . Define L 1 , D X ( f, g ) = E x ∼D X [ | f ( x ) -g ( x ) | ] , for any two functions f, g : X → R , where D X is the marginal distribution of D on the features set X . There exists a setting of parameters such that for any for any γ -agnostic weak learning oracle with fixed tolerance ε 0 and failure probability δ 0 , Algorithm 1 produces a hypothesis h such that with probability 1 -10 δ 0 T -10 δT ,

while making T = O ((log N ( ε/ 10 γ, B , L 1 , D X )) /γ 2 ε 2 ) calls to the weak learning oracle, and sampling TS + S 0 = O ((log N ( ε/ 10 γ, B , L 1 , D X )) /γ 3 ε 3 ) labeled examples from D .

The claim follows immediately by the application of the following result from [VDVW97] to place an upper bound of the L 1 covering in terms of the VC dimension in Theorem 19. This result originally due to [Hau95] was established for distances defined by n -point empirical measures, for some finite n . The version in [VDVW97] works on arbitrary distributions, by first proving that by a result proven for empirical-type measures transfers without loss to any distribution.

Theorem 20 (Theorem 2.6.4 in [VDVW97]) . There exists a universal constant C such that for any VC class B , any probability measure D X , and 0 ≤ ε ≤ 1 ,

.

Proof of Theorem 19. We first invoke Lemma 6 but on a minimal ε Cov-cover (say B Cov) of B with respect to the L 1 , D X semi norm to get that there exists a C > 0 such that with probability 1 -δ , for all t ∈ [ T ] and h ∈ B Cov ∪ { h ∗ } , we have

Fix any g ∈ B . Now, by virtue of the ε Cov-cover, there exists a h ∈ B such that L 1 , D X = E x ∼D X [ | f ( x ) -g ( x ) | ] ≤ ε Cov. Now, using the definition of Φ ′ D and a uniform (absolute) upper bound on ϕ ′ (Lemma 2.III), we get

Similarly, using the fact that the marginal distribution of D t on X is the same as D X , we get

Combining these, we have that with probability 1 -δ , for all t ∈ [ T ] and h ∈ B Cov ∪ { h ∗ } , we have

From here, proceeding identically as the steps in the proof of Theorem 4 yields

Setting ε ′ = ε/ 5 γ , ε ′′ = ε/ 5 , ε Cov = ε/ 10 γ and plugging in the proposed hyper-parameters with appropriate constants yields the claimed result.

G Boosting for reinforcement learning

MDP. In this section, we consider a Markov Decision Process M = ( S , A , r, P, β, µ 0 ) , where S is a set of states, A = {± 1 } is a binary set of actions, r : S × A → [0 , 1] determines the (expected) reward at any state-action pair, P : S × A → S captures the transition dynamics of the MDP, i.e., P ( s ′ | s, a ) is the probability of moving to state s ′ upon taking action a at state s , β ∈ [0 , 1) is the discount factor, and µ 0 is the initial state distribution. Without any loss, one may restrict their consideration to Markovian policies [Put14] of the form π : S → ∆( A ) , where an agent at each point in time chooses action a at state s independently with probability π ( a | s ) .

For any state s ∈ S , action a ∈ A , and distribution µ ∼ ∆( S ) over states, define state-action and state-value functions as

Algorithm 2 RL Boosting adapted from [BHS22]

1: Input : iteration budget T , state distribution µ , step sizes η t , post-selection sample size P

2: Initialize a policy π 0 ∈ Π arbitrarily.

3: for t = 1 to T do

4: Run Algorithm 1 to get π ′ t , while using Algorithm 3 to produce a distribution over state-actions (ignore ̂ Q ) by executing the current policy π t -1 starting from the initial state distribution µ .

5: Update π t = (1 -η t ) π t -1 + η t π ′ t .

6: end for

7: Run each policy π t for P rollouts to compute an empirical estimate ̂ V π t of the expected return.

8: return π = π t ′ where t ′ = arg max t ̂ V π t .

We will abbreviate V π µ 0 = V π , since it captures the value of any policy starting from the canonical starting state distribution. Finally, the occupancy measure µ π µ ′ induced by a policy π starting from an initial state distribution µ ′ is stated below. We will take µ π = µ π µ 0 as a matter of convention.

Accessing the MDP. Following [BHS22], we consider two models of accessing the MDP; we will furnish a different result for each. In the episodic model , the learner interacts with the MDP in a limited number of episodes of reasonable length (i.e., ≈ (1 -β ) -1 ), and the starting state of MDP is always drawn from µ 0 . In the second, termed rollouts with ν -resets , the learner's interaction is still limited to a small number of episodes, however, the MDP now samples its starting state from ν . It is important to stress that in both cases, the learner's objective is the same, to maximize V π starting from µ 0 . However, ν could be more spread out over the state space than µ 0 , and provide an implicit source of explanation, and the learner's guarantee as shown next benefits from its dependence on a milder notion of distribution mismatch in this case.

Weak Leaner. For convenience, we restate our weak learning definition, this time using π to denote policies, instead of h . Note that our definition is equivalent to that used by [BHS22], because for binary actions a random policy induces an accuracy of half regardless of the distribution over features and labels. One may use the identity corr D ( π ) = 1 -2 l D ( π ) to observe this. In fact, our assumption of weak learning might ostensibly seem weaker, since it operates with 0 / 1 losses (equivalently, correlations), whereas the losses in previous work are assumed general linear. However, for binary actions, this difference is insubstantial and purely stylistic.

Definition 21 (Agnostic Weak Learner) . A learning algorithm is called a γ -agnostic weak learner with sample complexity m : R + × R + → N ∪ { + ∞} with respect to a policy class Π and a base policy class B if, for any ε 0 , δ 0 > 0 , upon being given m ( ε 0 , δ 0 ) independently and identically distributed samples from any distribution D ′ ∈ ∆( X × {± 1 } ) , it can output a base policy W ∈ B such that with probability 1 -δ 0 we have

Policy Completeness and Distribution Mismatch. π ∗ ∈ arg max π V π be a reward maximizing policy, and V ∗ be its value. Let Π be the convex hull of the boosted policy class, i.e., the outputs of the boosting algorithm. For any state distribution µ ′ , define the policy completeness E µ ′ term as

In words, this term captures how well the greedy policy improvement operator is approximated by Π in an state-averaged sense over the distribution induces by any policy in Π . Finally, we define distribution mismatch coefficients below.

Theorem 22. Let W be a γ -weak learner for the policy class Π operating with a base class B , with sample complexity m ( ε 0 , δ 0 ) = (log |B| /δ 0 ) /ε 2 0 . Fix tolerance ε and failure probability δ . In the episodic access model, there is a setting of parameters such that Algorithm 2 when given access to W produces a policy π such that with probability 1 -δ , we have

Algorithm 3 Trajectory Sampler adapted from [BHS22]

1: Sample state s 0 ∼ µ and action a ′ ∼ Unif ( A ) .

3: Take action a ′ at state s h , then continue to execute π , and use a termination probability of 1 -β . Upon termination, set R ( s h , a ′ ) as the sum of rewards from time h onwards.

2: Sample s ∼ µ π as follows: at every step h , with probability β , execute π ; else, accept s h .

4: Define the vector ̂ Q , such that for all a ∈ A , ̂ Q ( a ) = 2 R ( s h , a ′ ) · I a = a ′ .

5: With probability C ̂ Q ( a ′ ) , set y = a ′ else set y ∈ A - { a ′ } , where C = (1 -β ) / 2 .

6: return ( s h , ̂ Q,y ) .

while sampling while sampling

episodes of length O ((1 -β ) -1 ) .

H Proofs for boosting for reinforcement learning

Proof of Theorem 22. The proof here closely follows that of Theorem 7 in [BHS22], and we only indicate the necessary departures. Since we utilize the outer algorithm from previous work, the associated guarantees naturally carry over. The departure comes from our substitution of the internal boosting procedure in Algorithm 2 with Algorithm 1; in fact, in its place [BHS22] use a result of [HS21] which can be seen as a generalization of [KK09], e.g., it uses fresh samples for every round of boosting, to other non-binary action sets preserving its ≈ 1 /ε 4 sample complexity. In this view, the improvement in sample complexity by using the present paper's apprach seems natural.

For the episodic model, applying the second part of Theorem 9 in [BHS22], while noting the smoothness of V π , and combining the result with Lemma 18 and Lemma 11 in [BHS22], we have with probability 1 -Tδ

where π ′ t is the output of the internal boosting algorithm in Line 4 of Algorithm 2, and D t is the distribution produced by Algorithm 3 with π t -1 selected as the policy for execution. Here, by explicit calculation, noting a ′ ∼ Unif ( A ) , one may verify for any t that

Recall that A = {± 1 } , and hence π : S → ∆( {± 1 } ) . Hence, for any policy pair π 1 , π 2 we have

Therefore, all we need to ensure is that output of Algorithm 1 as instantiated in Algorithm 2 every round has an excess correlation gap over the best policy Π no more that (1 -β ) 2 ε/C ∞ , which Algorithm 1 assures us can be accomplished with O ( C 3 ∞ log |B| (1 -β ) 6 γ 3 ε 3 ) samples. The total number of samples is T = O ( C 2 ∞ 3 ) times greater.

(1 -β ) ε

episodes of length O ((1 -β ) -1 ) . In the ν -reset access model, there is a setting of parameters such that Algorithm 2 when given access to W produces a policy π such that with probability 1 -δ , we have

Similarly, for the ν -reset model, applying the first part of Theorem 9 in [BHS22], and combining the result with Lemma 19 and Lemma 11 in [BHS22], we have

Again, we need to ensure is that output of Algorithm 1 as instantiated in Algorithm 2 every round has an excess correlation gap over the best policy Π no more that (1 -β ) 3 ε/D ∞ , which Algorithm 1 assures us can be accomplished with O ( D 3 ∞ log |B| (1 -β ) 9 γ 3 ε 3 ) samples. The total number of samples is

I Proofs for agnostic learning of halfspaces

Proof of Theorem 8. Using an approximation result of [KOS04], we observe that ERM on the Fourier basis χ S ( x ) = ∏ i ∈ S x i , namely parities on subsets S , can be used to produce a weak learner. This result guarantees that an n -dimensional halfspace can be approximated with uniform weighting on the hypercube to ε 2 ℓ 2 -error using degree-limited B n,d = {± χ S : | S | ≤ d } as a basis, where d = 20 ε -4 . As a result, at least one h ∈ B n,d must have high correlation.

Lemma 23 (Lemma 5 in [KMV08]) . Let D be any data distribution over {± 1 } n ×{-1 , 1 } with marginal distribution Unif ( {± 1 } n ) on the features. For any fixed ε and d = 20 ε -4 , there exists some h ∈ B n,d such

The result follows directly from the preceding lemma, which provides a weak learner for the task, and Theorem 4. We note that |B n,d | < n d and γ = n -d , so

J Additional experimental details

For all algorithms, we perform a grid search on the number of boosting rounds with T ∈ { 25 , 50 , 100 } . For Algorithm 1 we search over σ ∈ { 0 . 1 , 0 . 25 , 0 . 5 } as well. Rather than using a fixed η , our implementation uses an adaptive step-size scheme proportional to the empirical correlation on the current relabeled distribution. Our experiments were performed using the fractional relabeling scheme stated in [KK09], intended to reduce the stochasticity the algorithm is subject to. In particular, rather than sampling labels, we provide both ( x, y ) and ( x, -y ) in our dataset with weights 1+ w t ( x,y ) 2 and 1 -w t ( x,y ) 2 respectively. For the baselines, w t ( x, y ) = -ϕ ′ mad ( H t y ) = min(1 , exp( -H t ( x ) y )) , where ϕ mad is the Madaboost potential [Dom00]. For the implementation of the proposed algorithm, for greater reproducibility, we use the weighting function w t ( x, y ) = (1 -σ ) ϕ ′ ( H t -1 ( x ) y ) -ϕ ′ ( H t ( x ) y ) , which is analytically equivalent to computing the expectation of p t ( x, y, η ′ ) in Algorithm 1 over η ′ ∼ Unif [0 , η ] . The runtime used for all experiments was a Google Cloud Engine VM instance with 2 vCPUs (Intel Xeon 64-bit @ 2.20 GHz) and 12 GB RAM.

K Guide for practical adaptation of Algorithm 1

For many hyperparamters, our theory provides strong clues (the link between η and σ ). Briefly, in practice, given a fixed dataset with m total data points, there are two parameters we think a practitioner should concern herself with: the mixing parameter σ and the number of rounds of boosting T , while the rest can be determined, or at least guessed well, given these. The first choice σ dictates the relative

weight ascribed to reused samples across rounds of boosting, while T , apart from determining the running time and generalization error, implicitly limits the algorithm to using m total /T fresh samples each round. For η , one may use the adaptive step-size schedule suggested in the previous section, which also obviates the difficulty of selecting γ . Similarly, our branching criteria, whose necessity is explained in Appendix A, although theoretically both sound and necesssary, is overly conservative. In practice, we believe choosing the better of -sign ( H t ) and W t whichever produces the greatest empirical distribution on the relabeled distribution will perform best.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: All claims are backed by theoretical results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: All assumptions are stated explicitly. Further, the appendix contains a separate section on limitations of the present work.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Please see the paper.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please see the additional materials.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Please see the additional materials.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please see the additional materials and the experiments section for details.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Please see the table in the main paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please see the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: It does.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The present work falls within the realm of foundational research, and is in its basic nature theoretical.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Not applicable.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All datasets used are cited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets are introduced in the present work.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the agnostic boosting problem. That is the learner is given access to a weak learner that given $m_0$ samples form any distribution $D'$ returns a hypothesis $h$ from some baseset $\\mathcal{B}$ such that it is $corr_{D'}(h)\\geq \\gamma corr_{D'}(h^*)-\\varepsilon_0$ with probabilty $\\delta_0$, where $h^*$ is the hypothesis in a hypothesis set $\\mathcal{H}$ with the largest $corr_{D'}$. Furthermore the learner is given access to samples from a unknown target distribution $D$. The goal of the learner is now to create a hypothesis $\\bar{h}$ (not necessary in $\\mathcal{B}$ or $\\mathcal{H}$) such that  $corr_{D}(\\bar{h})\\geq \\max_{h\\in\\mathcal{H}}corr_{D}(h)-\\varepsilon$ with probabiilty $1-\\delta$, with as few samples from $D$ and calls to the weak learner. \nThe paper present a new  learning algorithm for solving this problem, which uses (with log factors not included) $O(\\log(|B|)/(\\gamma^3\\varepsilon^3))$ many samples and $O(\\log(|B|)/(\\gamma^2\\varepsilon^2))$ many calls to the weak learner. This is an improvement in the number of samples need from $D$ of a factor at least $O(\\gamma\\varepsilon)$ over previous know results, this improvment cost a factor of $\\log(|B|)$ more calls to the weak learner compared to the best know which uses $O(1/(\\gamma^2\\varepsilon^2))$. However they also have a second algorithm that uses $O(\\log(|B|)/(\\gamma^3\\varepsilon^3)+\\log^3(|B|)/(\\gamma^2\\varepsilon^2))$ many samples from $D$ and $O(1/(\\gamma^2\\varepsilon^2))$ many calls to the weak learner. The highlevel explanation of this better bound on the sample complexity is that the best previous known algorithm was running $O(1/\\gamma^2\\varepsilon^2)$ rounds and in each round using $O(\\log(|B|)/(\\gamma^2\\varepsilon^2)$ new samples from $D$. However by reusing samples from previous rounds an agumentening the samples they are able to only use $O(1/(\\gamma\\varepsilon))$ samples per round and in $O(\\log(|B|)/(\\gamma^2\\varepsilon^2))$ rounds. They also extend there results to infinite classes $\\mathcal{B}$. \nFurthermore the paper show how there new algorithms also have implications for Agnostic learning halfspaces and Boosting for reinformencment learning. They also evaluate their first algorithm against the previous best known algorithm and claim that they demonstrate a improvement empirically.\n\nSTRENGTHS:\nOriginality: They claim the paper has three innovations, reusing samples from previous rounds by argumenting them smartly, using a martingal argument to not use a uniform convergence results that they claim would result in a $O(1/(\\varepsilon^4))$ sample bound, and using  a different branching idea compared to previous work when. Given the discrebtion of the previous work they the authors provide the ideas seems novel and different from previous approaches. \n\nQuality: The main contains no full proofs so I have not checked the technical soundness of the paper this is also my reason for my confidence score. I have included questions below on the proof sketch. In the introduction it is said that \"Finally, in preliminary experiments in Section 7, we demonstrate that\n     the sample complexity improvements of our approach do indeed manifest in the form of improved\n     empirical performance over previous agnostic boosting approaches on commonly used datasets.\" do you compare to more than one previous approach? Is it exactly the algorithm 1 you run? As the experiments are preliminary I would also suggest maybe using a word like \"suggest\" instead of \"demonstrate\" in the sentence above. As also mentioned in the limitations the theoretical work is the primary contribution of the paper.\n\nSignificance: The theoretical results seems significant and a nice improvment over previous work in the sample complexity and a interresting step towards \"matching\" the samples complexity of agnostic learning which has dependency  $O(1/\\varepsilon^2)$ instead of now $O(1/\\varepsilon^3)$ (if it is possible to achieve a sample complexity $O(1/(\\varepsilon^2 \\gamma^2))$ in the agnostic boosting case). As they state the experiments are preliminary and I also personally think that the theoretical results are more than enough to make the paper significant in it self.\n\nWEAKNESSES:\nThe only small weakness I can come up with is the $log(|B|)$ factor in the oracle sample complexity of the algorithm 1 (and they mention themself)  which is taken care of in algorithm 2 for most parameters of $log(|B|),\\varepsilon,\\gamma$ (not considering hidden $log$ factors in the notation).\n\nQUESTIONS:\nThe comment given below Definition 1, do [KK09] and [BCHM20] also have the additive $\\varepsilon_0$ term so they have to set $\\varepsilon_0 = \\varepsilon$? (So my question is: is the comment \"necessary\" also meant for [KK09] and [BCHM20] as well?).\n\n Line 208 is it $-  \\phi ' ( H_t,  h^{*}) $ or what is $H$ in $-\\phi '(H,h_t^{*})$?\n\n\nLine 8 algorithm 1, what is  $D^{'}_t$ - I guess nevermind the line 9 says $\\hat{D}^{'}_{t}$ is that also the case for Line 8? And on line 8 to me the \"with as\" sounds a bit off (disclaimer: I'm not a native english speaker so please take this into considerations for this suggestion).\n\n\n Theorem 4:  Is there no dependence on $\\delta$ or $\\delta_{0}$ in the sample complexity? Or are they suppressed in the notation as the footnote suggests?(if this is the case why is $\\log|B|$ not suppressed in the notation as well?) \n\n\nProof sketch of Theorem 4 questions start\n\n\n Line 238 why $\\eta \\approx 1/\\sqrt{T}$? (So $\\eta = O(\\sqrt{1/T})=O(\\gamma\\varepsilon/\\sqrt{\\log|B|})$ is in the statement of Theorem 4?)\n    ´\n   Line 239.5 is $\\varepsilon_{gen} \\approx \\varepsilon$  ?\n\nLine 241 is it$\\phi'_{D}(H_{t},W_t/\\gamma)$ in the last equation?\n\n Line 244 is it $-\\phi '_{D}(H_{t},-sign(H_t))$ in the last equation?\n  \nLine 245-246 why is this relaxed criteria sufficient for the proof of Theorem 4? And could that be explained in stead in line 240-244 instead of the thing which is not atractable?\n Proof sketch of Theorem 4 questions end.\n\nLine 281-282 \"there is an algorithm using that uses the weak learner $\\mathcal{W}$\" sounds off to me (disclaimer: I'm not a native english speaker so please take this into considerations for this suggestion).\n\nWhat is the reason for not comparing to [BCHM20] in the experiments?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a new agnostic boosting algorithm whose sample complexity is shown to be polynomially better than that of previous methods.\nCompared to existing techniques, the advantage of the proposed algorithm comes mainly from the novel concept of reusing samples across multiple rounds of boosting.\nThrough their analysis, the authors offer other theoretical insights that may be of independent interest.\n\nSTRENGTHS:\n1. Good writing style. Well above average, with few typos and overall high signal-to-noise ratio.\n2. Noteworthy rigour in the theoretical analysis. Catching the subtle stochastic dependencies missed by [KK09] is good sign of diligence. (I did not check details exhaustively, though. Specially those in the appendices.)\n3. Substantial contribution. Up to some caveats, this work improves the sample complexity of agnostic boosting by factor of $εγ$, which is particularly valuable considering that it leaves the SOTA only another $εγ$ factor away from ERM's complexity.\n4. Some empirical results are provided. This is a good practice, even if the main focus is theoretical.\n5. *(Bonus)* I tend to be sceptical towards citing web links, but [Pin20] was a nice one.\n\nWEAKNESSES:\nI will start by the experiments.\nNote that I can appreciate the work for its theoretical contributions alone and see the experiments as a welcome extra.\nWith that being said:\n1. Information on the computer resources used for the experiments seems to be missing. This is in contrast with Question 8 of the paper checklist (at 840):\n    > For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?\n\n    To that, the authors reply \"Yes. Please see the appendix.\" (at 843 and 844), but I could not find this information in the appendix, the main text, or the supplementary material.\n2. Without information about the computational cost of the experiments, it is hard to understand the limitation to only three datasets.\nThe experiments in [KK09] include 5 more datasets and, even though I can see the experiments brought by the authors here are larger (e.g., 30-fold rather than 10-fold cross-validation), hardware has got much better since [KK09].\n3. From the description of the experiments, it is clear that they generated much more data than what is present in the text. More of this data should be available, even if only in the supplementary material.\n4. At line 296, \"less stark\" is a bit too generous. The baseline performed consistently and significantly better than the proposed method in the presence of the noise. While I praise the authors for being upfront about this, the phenomenon should be sufficient to motivate further experimentation and a deeper discussion.\n\nMoving to the theoretical side, I believe the main issues lie in the presentation of the work.\nTo be clear, this does not concern the writing style, which I appreciated, but rather the clarity and contextualization relative to prior works.\n\n5. While the intro is clear at almost all points, clarity seems to progressively decrease up to (the crucial!) section 5. I understand that much of this comes from the complexity of the subject itself. I understand the big challenge, but given the absolute importance of section 5, I believe the authors underperformed there. A similar note applies to the paragraph starting at 205: I am confident there is some significant \"curse of knowledge\" at play there.\nStill, a criticism such as this is too costly to substantiate, so it will not significantly affect my evaluation (directly).\n6. The contextualization should be better. A paper not only **ought** to make it *possible* for the reader to infer precisely what parts of the contributions are original, it **should** make the delineation *easy*. Specially for a paper that demands heavier background, it is crucial to be very clear about the extent of your contributions, not only by stating them explicitly, with absolute clarity but also by explicitly indicating which ideas come from previous works.\nBelow, I list the reasons why I believe what the authors offer (most notably the explicit enumeration of contributions in Section 1.1) is insufficient in this regard.\n    1. I believe that the implications for agnostic learning for halfspaces should be presented as a case for the relevance of the problem, rather than as a contribution. The translation of results in the setting of the paper to that of agnostically learning halfspaces is already present in [KK09] and seems to require minimal extra effort. Still, the authors promote it as a contribution quite strongly, with multiple occurrences of the claim in the text and its highlight as Contribution 4 (at 131).\n    2. A similar note may apply to the application to boosting for reinforcement learning (Contribution 5 at 134). I preferred to ask the authors directly about this.\n    3. Similarly, I found it necessary to ask the authors about the exact nature of Contribution 3. See question below.\n    4. The point mentioned in Contribution 2 (113) should be further discussed. If I understand correctly, when taking the oracle complexity into account, the bound from Theorem 8 remains incomparable to the one from [KK09] (see, e.g., Table 1) given the cubic dependence on $\\log \\lvert \\mathscr{B} \\rvert$. The authors should elaborate on this matter. For example, on one hand, understanding this logarithmic factor to be negligible (as the authors suggest in the caption of Table 1) weakens the case for Contribution 2, since it only improves the oracle complexity by this exact factor. On the other hand, taking it as important jeopardizes the contribution of Theorem 8 as the factor shows up (twice) in the final sample complexity bound.\n    5. The authors should highlight more the main technical challenges arising from modifying the algorithm by [KK09]. In my opinion, the contribution would be fair even if the challenges were minor so, if authors thought this was the case (and I am not saying it is), they could be explicit about it and still make a good case for the paper.\n    6. [BCHM20] looks a bit off within in Table 1 since its bound is strictly worse than that of [KK09]. This is not a serious issue, but it is worth a brief comment.\n7. The need for $\\varepsilon_0$ and $\\delta_0$ is worth more discussion than what is offered around 166.\n\n>NOTE: I apologize if my attempt to remain objective ended up making me sound harsh.\n\nOverall, I tend towards acceptance, but with reservations. I find it quite feasible that I misunderstood some point, and that the authors can clarify important points in the rebuttal, so there is a fair chance I end up raising my initial score.\n\n\n### Minor issues and suggestions\nWhile reading the paper, I took some notes that the authors may find useful. The authors are free to ignore those.\n\n- You may want to have a look at the (very recent) works on parallel boosting [1] and [2]. I'm not sure the connection is deep, but they also employ some notion of \"sample reuse\".\n- [23] missing \"it\"\n- \"excess of $ε$\" is better than \"$ε$-excess\" in many ways, if you think about it\n- [Table 1] Put \"inefficient\" within parentheses\n- [Table 1] Rephrase the sentence starting with \"$\\mathscr{B}$ is the base...\". It's quite confusing.\n- [All tables] Use `booktabs` for better tables. Also, the tips in its documentation can be very useful\n- [41 (and many other places)] In American usage, colons are usually followed by a capital letter\n- [66] \"these\" is ambiguous\n- [67] \"A weak learner\" $\\to$ \"A $\\gamma$-weak learner\" (in particular, solves undefined usage shortly after)\n- [73] missing hyphen\n- [85] remove extra comma\n- Floating objects (figures, tables, etc.) ended up inconveniently far from their first reference in the text\n- [100] double \"of\"\n- [103] \"making a step “backward” via adding in a negation of the sign of the current ensemble\" is a bit too heavy to get this early in the text\n- [124] remove extra \"to\"\n- Having hyperlinks to algorithm lines would be nice\n- Number theorems and lemmas by section (e.g., \"Theorem 4.2\" rather than \"Theorem 5\")\n- [138] \"MDP\" is still undefined\n- [151] fix \"boostng\"\n- [161] A pedantic reader may need a quick definition of $\\Delta(\\;\\cdot\\;)$\n- [everywhere] use `\\colon` rather than `:` to define functions (it gives the right spacing)\n- Consider TikZ for Figure 1 to have better control over the appearance (compared to GeoGebra) and vector graphics by default. Perhaps this extra control to can save some space.\nHere is a quick draft as an example:\n```latex\n\\begin{tikzpicture}\n    \\draw[color=gray, thin, dotted] (-3.1, -0.6) grid (4.1, 4.1);\n\n    \\draw[->] (-3.2,0) -- (4.2,0);\n    \\draw[->] (0,-0.7) -- (0,4.2);\n\n    \\draw[color=red, domain=-2.075:2.6] plot (\\x, {2 - \\x}) node[right] {$2 - z$};\n    \\draw[color=blue, domain=-2.075:4, smooth] plot (\\x, {(\\x + 2) * exp(-\\x)}) node[above] {$(z + 2) e^{-z}$};\n\\end{tikzpicture}\n```\n- Many mathematical equations are left without proper punctuation. Those are integral part of the text and should be treated as such.\n- [Definition 1] Consider using $\\sup$ rather than $\\max$ for robustness\n- [176] \"a functions\" $\\to$ \"a function\"\n- [Algorithm 1] \"Agonstic\" $\\to$ \"Agnostic\"\n- [Algorithm 1] Number all lines\n- [Algorithm 1, first line of line 5] probably missing a \"uniform\"\n- [198] Use `\\coloneqq`\n- [208] no need for the apostrophe to pluralize\n- **This is a bit more important.** $h^*$ only gets defined at 482 but its used many times earlier.\n- [237] double period\n- [239] The use of $\\pm$ in the next equation is undefined and confusing\n- [240] \"linearity\" $\\to$ \"the linearity\"\n- [Table 3] omit the 0s to save space. E.g., \"0.12\" $\\to$ \".12\". This should solve the overfull hbox.\n- [483] dangling \"]\" below\n- [577] \"semi norm\" $\\to$ \"seminorm\"\n- [578] \"size\" is ambiguous in this context\n- [Theorem 19] Missing `\\operatorname` for \"corr\"\n- [582 and 583] Inconsistent parenthetization\n- [598] Missing `\\operatorname` for \"Cov\"\n- [Random] In my humble opinion, this work would be better presented by first focusing on the method by [KK09] and building up from there, gradually discussing the modifications and their implications. Of course, this is also a matter of style, and the change could be quite costly at this point.\n\n[1] Karbasi, A. and Larsen, K.G., 2024, March. The impossibility of parallelizing boosting. In International Conference on Algorithmic Learning Theory (pp. 635-653). PMLR.\\\n[2] Lyu, X., Wu, H. and Yang, J., 2024. The Cost of Parallelizing Boosting. In Proceedings of the 2024 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) (pp. 3140-3155). Society for Industrial and Applied Mathematics.\n\nQUESTIONS:\n1. What hardware was used for experiments? How long did them take to run?\n2. Why include $T$ in the grid search of the experiments? Wouldn't it suffice to fix the highest value? The fact that the proposed method, in some sense, \"rolls back to the best model\" (see last line of Algorithm 1) makes that seem even more reasonable.\n3. At 250, what do you mean by \"the sign of $\\Delta_t$ is indeterminate\"?\n4. How original is Contribution 5? How much of it is present in previous works? How much effort is needed to fit your results into existing frameworks to obtain Theorem 7? (I already had a look at appendices G and H, but I am still not totally sure.)\n5. Similarly, how original is Contribution 3? Are the authors aware of any previous works applying similar strategies ($L_1$ covering number based bounds $\\to$ Theorem 20)? (To be clear, I do value the nice connection. I am simply trying to size precisely its originality.)\n6. At 116, what do you mean by \"The resultant sample complexity improves known results for all practical (i.e., sub-constant) regimes of ε\"? I would imagine that applications are actually prone to employ larger values of ε; above, say, the machine precision.\n\nLIMITATIONS:\nI think my criticisms regarding the experiments (1-4) are pertinent here.\nMoreover, I got the impression that the authors are particularly capable of tackling the presentation issues. They demonstrated to have a good grasp of the subject and a good writing skills. I suspect that, with some effort, they can make this and future papers excel in clarity and appeal.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method of classifier boosting that improves the sample complexity bound in agnostic settings. To achieve it, they propose an algorithm carefully reusing the sample across iteration for updating pseudo labels. They also demonstrate the usefulness of the method with the agnostic halfspace learning and the reinforcement learning, as well as experimental results.\n\nSTRENGTHS:\n- Well motivated.\n- Having significant technical novelties.\n\nWEAKNESSES:\n- It lacks discussions on the comparison with prominent competitors in terms of the trade off between the sample- and time- complexities: SGD-like implementations of ERM.\n- Hard to follow the technical part and understand which portion of the algorithm is new and important for the improvement in the sample complexity. In particular, one could use the intuitive explanation on the role of each nontrivial step in the algorithm such as \n  - stochastic branching in Line 5,\n  - sampling $\\eta'$ in Line 5.II,\n  - stochastic relabelling in Line 5.II,\n  - branching condition in Line 9, and so on.\n- Also, the proof sketch (Section 5) could be more streamlined, unifying it with the aforementioned intuitive explanations.\n\nQUESTIONS:\n- On the note right after Theorem 6, the time complexity of ERM: Can we just employ an SGD-like algorithm + hinge loss to achieve the same order of the sample complexity as ERM with a manageable time complexity, like $O(n /\\epsilon^2)$?\n- Section 6.2: How does it compare to the non-boosting results in the RL literature?\n- Section 7: The baseline accuracy in Table 3 barely degrades as the label noise increases. Even far from that, it goes **up** with the Spambase data. How do you explain this phenomenon?\n- Also in Section 7:\n  - What is the meaning of $\\pm$ in the table? standard deviation?\n  - Did you separate the data for the accuracy estimation and the grid search?\n\nLIMITATIONS:\nThe limitations are adequately addressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper continues the study of agnostic boosting for binary classification tasks. They provide a new agnostic boosting algorithm that, by re-using samples across multiple rounds of boosting, is more sample-efficient than existing algorithms. As corollaries,  they derive improved sample complexities for agnostically learning half-spaces and boosting for RL. Finally, the authors run experiments and show that their agnostic boosting algorithm outperforms existing algorithms on commonly used datasets.\n\nSTRENGTHS:\n- The technical contributions are solid, the algorithmic ideas are interesting, and the proof techniques are non-trivial. In particular, I liked the idea of sample reuse across boosting steps. \n-  Overall, I feel that this a nice contribution to the literature on agnostic boosting\n\nWEAKNESSES:\nMy main issue with this paper is its presentation. \n\n-  Lemma 9 is referenced/linked a lot in the main text, however it is in the Appendix. I think it would be a good idea to put at least an informal version of Lemma 9 in the main text if it is that important. \n- In my opinion, Section 5 (sketch of analysis) is very dense. I feel that the sketch was a bit too detailed for it to be a sketch but also not  detailed enough for it to be a full proof. I think it would be better to move Lemma 2 to the Appendix, expand more on the description of the algorithm (lines 205 - 216), make Section 5 more high-level and pointing out the exact techniques used to prove Theorem 4, and expand more on the implications of your results for half spaces and RL. \n- Since the proposed agnostic boosting algorithm follows the potential-based framework of [KK09], I think it would be nice to give the readers how these algorithms work in general. For example, it would be nice to elaborate in the main text what the \"potential-based framework\" for agnostic boosting mentioned in Line 102 actually is (providing an algorithmic template would be even better). \n- As is, Section 5 doesn't provide me much intuition about: (1) where/how sample reuse is saving me and (2) why the author's chose the potential function they chose. Overall, I think it would be better to make Section 5 less technical, but provide the reader with more intuition about the algorithmic choices.\n\nQUESTIONS:\n-  It would also be nice if you empirically compared your algorithm to the agnostic booster from [BCHM20].\n\nLIMITATIONS:\nNo Limitations", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new agnostic boosting algorithm that improves the sample complexity of a previous algorithm by a factor of $\\gamma^{-1}\\epsilon^{-1}$ at the expense of an increase in the computational (oracle) complexity by a factor of logarithmic size or VC dimension of the base learning class. Here, $\\epsilon$ is the desired additive closeness to a risk minimiser and $\\gamma$ the multiplicative approximation ratio to a risk minimiser provided by the weak learner. Alternatively, the oracle complexity can be reduced to that of the previous algorithm at the expense of an increased sample complexity that is not directly comparable to that of previous algorithms.\n\nSTRENGTHS:\nThe theoretical contribution of a rate improvement in sample complexity for agnostic boosting is significant. This is achieved by a sophisticated procedure and corresponding analysis that draws from a range of theoretical insights, especially the extension to infinite base learning classes. The inherent complexity of this contribution is managed by a very rigorous and mature presentation.\n\nWEAKNESSES:\nThe proposed procedure has a very large number of parameters. I believe the proofs provide concrete valid settings for those parameters as a function of the desired PAC guarantees. However, it seems that the practical accessibility / usability of the procedure would benefit from a a more explicit treatment of those choices and/or the elimination of some degrees of freedom at least in the main text version of the algorithm.\n\nGenerally, despite the rigorous and detailed presentation, the authors should be aware that their paper is still a very challenging read and I have some worry that the boosting literature becomes increasingly in-penetrable for the wider machine learning community. Any improvement of the general state of affairs that the authors can provide here would be strongly appreciated.\n\nMinor points\n- The references in the first sentence should be revised. It does not make sense that the 1990 boosting article resolves a question first posed in work from 1994.\n- In line 97, it seems to me you mean \"inevitably\" instead of \"invariable\".\n- Typo in line 151: \"boostng\"\n\nQUESTIONS:\n- Why do the main results in the table focus on the finite base learning class when the extension to the infinite case via VC dimensions are available?\n- What exactly is the ERM baseline? The identification of an empirical risk minimiser in $\\mathcal{B}$? If so, is it correct to give its complexity as $\\infty$ in the table (at least for the case of finite $|\\mathcal{B}|$)\n- To what degree is the algorithm distribution-specific? The definition of the weak learner asks the sample complexity to be valid for all distributions.\n\nLIMITATIONS:\nThe limitations of the proposed procedure that go beyond those that are usually present in this branch of literature are the trade-off between sample and oracle complexity as well as a larger number of parameters. The first is discussed very explicitly while the second seems to be mostly a presentation issue as mentioned above.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-17"}