Title: Information-theoretic Limits of Online Classification with Noisy Labels
Venue: NeurIPS 2024
Accepted: yes
Keywords: Online classification

noisy label

pairwise testing

Hellinger divergence

Le Cam-Birge testing
Abstract: We study online classification with general hypothesis classes where the true labels are determined by some function within the class, but are corrupted by *unknown* stochastic noise, and the features are generated adversarially. Predictions are made using observed *noisy* labels and noiseless features, while the performance is  measured via minimax risk when comparing against *true* labels. The noisy mechanism is modeled via a general noisy kernel that specifies, for any individual data point, a set of distributions from which the actual noisy label distribution is chosen. We show that minimax risk is *tightly* characterized (up to a logarithmic factor of the hypothesis class size) by the *Hellinger gap* of the noisy label distributions induced by the kernel, *independent* of other properties such as the means and variances of the noise. Our main technique is based on a novel reduction to an online comparison scheme of two hypotheses, along with a new *conditional* version of Le Cam-Birgé testing suitable for online settings. Our work provides the first comprehensive characterization of noisy online classification with guarantees that apply to the *ground truth* while addressing *general* noisy observations.

Information-theoretic Limits of Online Classification with Noisy Labels

Abstract

We study online classification with general hypothesis classes where the true labels are determined by some function within the class, but are corrupted by unknown stochastic noise, and the features are generated adversarially. Predictions are made using observed noisy labels and noiseless features, while the performance is measured via minimax risk when comparing against true labels. The noisy mechanism is modeled via a general noisy kernel that specifies, for any individual data point, a set of distributions from which the actual noisy label distribution is chosen. We show that minimax risk is tightly characterized (up to a logarithmic factor of the hypothesis class size) by the Hellinger gap of the noisy label distributions induced by the kernel, independent of other properties such as the means and variances of the noise. Our main technique is based on a novel reduction to an online comparison scheme of two-hypotheses, along with a new conditional version of Le Cam-Birgé testing suitable for online settings. Our work provides the first comprehensive characterization for noisy online classification with guarantees that apply to the ground truth while addressing general noisy observations.

1 Introduction

Learning from noisy data is a fundamental problem in many machine learning applications. Noise can originate from various sources, including low-precision measurements of physical quantities, communication errors, or noise intentionally injected by methods such as differential privacy. In such cases, one typically learns by training on noisy (or observed) data while aiming to build a model that performs well on the true (or latent) data. This paper focuses on online learning [20] from noisy labels, where one receives noiseless, adversarially generated features and corresponding noisy labels sequentially, and predicts the true labels as the data arrive.

Online learning has been primarily studied in the agnostic setting [1, 19, 7], where one receives the labels in their plain (noise-free) form and the prediction risk is evaluated on the observed labels. It is typically assumed that both the features and observed labels are generated adversarially, and prediction quality is measured via the notion of regret , which compares the actual cumulative risk incurred by the predictor with the minimal cumulative risk incurred by the best expert in a hypothesis class. While this approach is mathematically appealing, it does not adequately characterize online learning scenarios when our goal is to achieve good performance with respect to grand truth data that may be different from the observed (noisy) ones.

This paper considers an online learning scenario that differs from classical agnostic online learning in two aspects: (i) we assume that the noisy labels are derived from a (semi-) stochastic mechanism rather than from pure adversarial selections; (ii) our prediction risk is evaluated on the true labels, not noisy observations. To better motivate the study of such a scenario, we consider the following example first introduced by Ben-David et al. [1]:

Example 1. Let H ⊂ { 0 , 1 } X be a finite hypothesis class. Consider the following online learning game between Nature/Adversary and Learner that is played over a time horizon T . At the start, Nature fixes a ground truth classifier h ∈ H . At each time step t ≤ T , Nature adversarially selects feature x t ∈ X and reveals it to the learner. The learner makes a prediction ˆ y t based on prior features x t = { x 1 , · · · , x t } and noisy labels ˜ y t -1 = { ˜ y 1 , · · · , ˜ y t -1 } . Nature then selects a (unknown) noise parameter η t ∈ [0 , η ] for some given η (known to learner), and generates 1 :

where ⊕ denotes binary addition and y t = h ( x t ) is the true label. It was shown in [1, Thm 15] that there exists a predictor ˆ y T such that:

̸

Note that the risk in (1) is significant, as the error introduced by noise to the true labels increases linearly as ηT , yet the risk remains independent of the time horizon T . This mirrors the fast rates known in the PAC learning literature when benign noise is present. Despite its foundational nature, the understanding of this phenomenon beyond simple Massart's noise has been largely unexplored.

This paper introduces a novel online learning framework for modeling general noisy mechanisms. In particular, it encompasses (1) as a very specific instance and provides a clear and comprehensive characterization of the underlying paradigm. Formally, let Y be the set of true (latent) labels and ˜ Y be the set of noisy (observed) labels, which we assume are finite and of size N,M , respectively. Let X be the feature space. We model the noisy mechanism by a noisy kernel :

where D ( ˜ Y ) is the set of all distributions over ˜ Y . That is, the kernel K maps each pair ( x , y ) to a subset Q x y := K ( x , y ) ⊂ D ( ˜ Y ) of distributions over ˜ Y . Observe that the noisy kernel provides a compact way of modeling noisy label distributions without explicitly referring to the noise . This is more convenient for our discussion, as ultimately the statistical information is solely determined by the noisy label distributions.

For any given H ⊂ Y X and kernel K , we consider the following robust (noisy) online classification scenario: Nature first selects h ∈ H ; at each time step t , Nature chooses (adversarially) x t ∈ X and reveals it to the learner; the learner then makes a prediction ˆ y t , based on the features x t and noisy labels ˜ y t -1 ; an adversary then selects a distribution ˜ p t ∈ Q x t h ( x t ) , samples ˜ y t ∼ ˜ p t and reveals ˜ y t to the learner. Let Φ and Ψ be the strategies of the learner and Nature/adversary, respectively. The goal of the learner is to minimize the following expected minimax risk :

̸

where ˆ y t = Φ( x t , ˜ y t -1 ) . Note that the adversarial selection of distribution ˜ p t from the kernel set Q x t h ( x t ) provides more flexibility for modeling scenarios when the noisy label distribution changes even with the same true label, such as the Massart's noise in Example 1. We refer to Section 2 for a more complete specification of our setting.

1.1 Main Contributions

̸

Our main contributions in this paper establish the fundamental limits of minimax risk in (3) by providing nearly matching lower and upper bounds across a wide range of hypothesis classes H and noisy kernels K . Observe that, to allow for non-trivial prediction rules, the induced noisy label distributions must be statistically distinguishable for distinct true labels. To formalize this intuition, we define, for any noisy kernel K and feature x ∈ X , the Hellinger gap as γ H ( x ) = inf y = y ′ ∈Y inf p ∈Q x y ,q ∈Q x y ′ { H 2 ( p, q ) } , where H 2 ( p, q ) = ∑ M m =1 ( √ p [ m ] -√ q [ m ]) 2 is the squared Hellinger distance. That is, γ H ( x ) measures the minimal squared Hellinger distance of the induced noisy label distributions over all distinct true labels.

Our main result (see also Theorem 2) can be summarized as follows:

Theorem 1. Let H ⊂ Y X be any finite class, and K be any noisy kernel such that inf x ∈X γ H ( x ) ≥ γ H for some γ H > 0 , and Q x y ⊂ D ( ˜ Y ) is closed and convex for all x , y . Then:

Moreover, for any K ∈ N and any kernel K with at least log K features x ∈ X for which γ H ( x ) ≤ γ H , there exists a class H of size K that satisfies: ˜ r T ( H , K ) ≥ Ω ( log |H| γ H ) .

Theorem 1 shows that the Hellinger gap is the right characterization for the minimax risk upto at most a logarithmic factor. Moreover, the risk bound depends solely on the gap parameter γ H and log |H| , independent of time horizon T , the size of Y and ˜ Y , and the properties of noise such as means and variances. For the bounded Bernoulli noise in Example 1, the set Q x y corresponds to Bernoulli distributions with parameters in [0 , η ] if y = 0 and in [1 -η, 1] if y = 1 , leading to the Hellinger gap γ H = 1 -2 √ η (1 -η ) . This matches the dependency on η in Example 1 2 . However, our result holds for any noisy kernel. For instance, if we shift Q x 0 to Bernoulli distribution with parameter 0 and Q x 1 with parameters in [1 -2 η, 1] , then γ H = 1 - √ 2 η = Θ(1 -2 η ) . This is tighter than the dependency on η in Example 1 (for η → 1 2 ), since 1 -2 √ η (1 -η ) = Θ((1 -2 η ) 2 ) .

Our main proof technique for establishing Theorem 1 is based on a novel (black box) reduction to an online comparison scheme of two-hypotheses in H , as demonstrated in Theorem 3. This allows us to reduce the noisy online classification problem to a hypothesis testing problem, which effectively decouples the adversarial property of the features from the stochastic property of the noisy labels. However, due to the adversarial selection of the noisy label distributions, the classical hypothesis testing techniques does not apply. To resolve this issue, we establish in Theorem 4, a generalization of the Le Cam-Birgé Test with varying conditional marginals for handling pairwise testing via the Hellinger gap, which is a result of independent interest.

Tight dependency on log |H| . Although the lower and upper bounds in Theorem 1 differ by a log |H| factor, this is compensated by the fact that we are dealing with the most general classes and kernels. This can be tightened for various special cases. Indeed, for a class H with binary true labels and arbitrary noisy labels, we demonstrate in Theorem 5 that the minimax risk is upper bounded by 16 log |H| γ L , where γ L is the L 2 -gap that substitutes the Hellinger distance with L 2 -distance in Theorem 1. This is proved via a novel reduction to online conditional distribution estimation under L 2 -distance. Moreover, we demonstrate in Appendix G (Theorem 6) that the (optimal) O ( log |H| γ H ) upper bound holds if |Q x y | = 1 for all x , y , i.e., the noisy label distribution is determined by data.

1.2 Related Work

Online learning with noisy data was discussed in [6], which specifically considers generalized linear functions with zero-mean and bounded variance noises. Our work differs in that we focus on classification instead of regression. Moreover, our noisy model does not require that the noise be zero-mean. To the best of our knowledge, [1] is the only work that has specifically considered the classification task, but this was limited to bounded Bernoulli noise. From a technical standpoint, analogous ideas of pairwise comparison have been considered in differential privacy literature, such as in [11], but only in batch settings. The reduction to online conditional probability estimation was also explored in [10] within the context of online decision making . However, a distinguishing feature of our work is that our conditional probability estimation problem is necessarily misspecified , as our noisy label distributions are selected adversarially and are unknown a priori to the learner. Our problem setup is further related to differentially private conditional distribution learning, as in [26], and robust hypothesis testing , discussed in [17, Chapter 16]. Online conditional probability estimation has been widely studied, see [18, 3, 2, 4, 25, 24]. Conditional density estimation in the batch setting has also been extensively studied, see [12] for KL-divergence with misspecification and [9] for L 2 loss. Learning from noisy labels in the batch case was discussed in [16] (see also the references therein) by leveraging suitably defined proxy losses. There has been a long line of research on online prediction with adversarial observable labels in the agnostic formulation, see [5, 1, 19, 7].

2 Notation and Preliminaries

Let X be a set of features (or instances), Y be a set of labels, and ˜ Y be a set of noisy observations. We assume throughout the paper that |Y| = N and | ˜ Y| = M for some integers N,M ≥ 2 . We denote D ( ˜ Y ) as the set of all probability distributions over ˜ Y .

Let H ⊂ Y X be a hypotheses class and K be a noisy kernel in (2). We consider the following robust online classification scenario: (1) Nature first selects some h ∈ H ; (2) At time t , Nature adversarially selects x t ∈ X ; (3) Learner predicts ˆ y t ∈ Y , based on (noisy) history observed thus far (i.e., x t , ˜ y t -1 ); (4) An adversary then selects ˜ p t ∈ Q x t h ( x t ) , and generates a noisy sample ˜ y t ∼ ˜ p t .

̸

The goal of the learner is to minimize the cumulative error : ∑ T t =1 1 { h ( x t ) = ˆ y t } .

Note that the cumulative error is a random variable that depends on all the randomness associated with the game. To remove the dependency on such randomness and to assess the fundamental limits of the prediction quality, we consider the following two measures 3 :

Definition 1. Let H ⊂ Y X be a set of hypotheses and K : X × Y → 2 D ( ˜ Y ) be a noisy kernel. We denote by Φ the (possibly randomized) strategies of the learner . The expected minimax risk is:

̸

By skolemization [19], the operator Q T K is equivalent to sup ˜ p E ˜ y T ∼ ˜ p , where ˜ p runs over all (joint) distributions over ˜ Y T such that ∀ t ∈ [ T ] , ∀ ˜ y t -1 ∈ ˜ Y t -1 the conditional marginal ˜ p ˜ y t | ˜ y t -1 ∈ Q x t h ( x t ) .

Definition 2. Let H , K , and Φ be as in Definition 1. For any δ > 0 , the high probability minimax risk at confidence δ is the minimum quantity B δ ( H , K ) ≥ 0 such that there exists a predictor Φ satisfying:

̸

where ˜ p is selected as in the discussion above and ˆ y t ∼ Φ( x t , ˜ y t -1 ) .

Note that the kernel map K is generally known to the learner when constructing the predictor Φ . However, the induced kernel sets Q x t h ( x t ) are not, since they depend on the unknown ground truth classifier h and adversarially generated features x T . In certain cases, such as Theorem 3 and Example 4, the kernel map K is also not required to be known.

For any x ∈ X and y ∈ Y , we denote by Q x y the set induced by a kernel. We can assume, w.l.o.g., that the Q x y s are convex and closed sets, since the adversary can select an arbitrary distribution from Q x y s at each time step, including randomized strategies that effectively sample from a mixture (i.e., convex combination) of distributions in Q x y s.

One must introduce some constraints on the kernel K in order to obtain meaningful results. To do so, we introduce the following well-separation condition:

̸

Definition 3. Let L : D ( ˜ Y ) 2 → R ≥ 0 be any divergence, we say a kernel K is well-separated w.r.t. L at scale γ > 0 , if ∀ x ∈ X , ∀ y, y ′ ∈ Y with y = y ′ we have L ( Q x y , Q x y ′ ) def = inf p ∈Q x y ,q ∈Q x y ′ L ( p, q ) ≥ γ.

̸

Example 2. For any y ∈ Y , we specify a canonical distribution p y ∈ D ( ˜ Y ) . A natural noisy kernel would be to define Q x y = { p ∈ D ( ˜ Y ) : TV ( p, p y ) ≤ ϵ } , where TV denotes total variation. In this case, the kernel is well-separated with the gap γ under total variation if min y = y ′ ∈Y TV ( p y , p y ′ ) ≥ γ +2 ϵ. In particular, this subsumes Example 1 if, for y ∈ { 0 , 1 } , we define p y as the distribution that assigns probability 1 to y , and take ϵ = η , where the TV-gap equals γ = 1 -2 η .

3 Main Results

We begin by stating our main result of this paper.

Theorem 2. Let H ⊂ Y X be a finite class of size K , and K be a kernel that is well-separated at scale γ H w.r.t. squared Hellinger divergence (Definition 3). Then, the high probability minimax risk (Definition 2) with confidence δ > 0 is upper bounded by:

Moreover, for any kernel K such that there exist at least L distinct features x ∈ X 4 for which inf y = y ′ ∈Y H 2 ( Q x y , Q x y ′ ) ≤ γ H , one can find a class H of size K such that:

Observe that, the upper bound holds with high probability and the risk is independent of the time horizon (i.e., the so-called fast rates known in the PAC-learning literature). Moreover, the bound is independent of the size of Y and ˜ Y . Asimple integration argument yields the expected risk upper bound ˜ r ( H , K ) ≤ O ( log 2 K γ H ) , which matches the lower bound upto only a log K factor. This demonstrates that, the Hellinger gap of the induced noisy label distributions is the right characterization for the minimax risk. Moreover, the Hellinger distance can be transformed from other f -divergences (such total variation) without depending on the size of ˜ Y [17, Chapter 7.6].

̸

Example 3. Let K be the kernel in Example 2. Let λ = min y = y ′ ∈Y TV ( p y , p y ′ ) . Hence, the kernel is well-separated with TV-gap λ -2 ϵ . Since H 2 ( p, q ) ≥ TV ( p, q ) 2 [17, Eq. 7.22], the Hellinger gap is lower bounded by ( λ -2 ϵ ) 2 . Invoking Theorem 2, we have for any hypothesis class H , the following risk upper bound holds: B δ ( H , K ) ≤ O ( log |H| log( |H| /δ ) ( λ -2 ϵ ) 2 ) .

The rest of this section is devoted to establishing Theorem 2. Our main proof technique is based on a novel reduction to pairwise testing of two hypotheses as developed in Section 3.1, along with explicit testing rules in Section 3.2 based on a novel conditional version of Le Cam-Birgé testing.

3.1 Reduction to Pairwise Comparison: a Generic Approach

We first introduce the following key technical concept. Recall that our robust online classification problem is completely determined by the tuple ( H , K ) .

Definition 4. A problem ( H , K ) is said to be pairwise testable with confidence δ > 0 and error bound C ( δ ) ≥ 0 if, for any pair h i , h j ∈ H , the sub-problem ( { h i , h j } , K ) admits a predictor (i.e., pairwise tester) Φ i,j that achieves cumulative risk ≤ C ( δ ) w.p. ≥ 1 -δ (see Definition 2).

Clearly, any prediction rule for ( H , K ) serves as a pairwise testing rule for all the sub-problems ( { h i , h j } , K ) with h i , h j ∈ H . Perhaps surprisingly, we will show in this section that any pairwise testing rules for the sub-problems can also be converted into a prediction rule for ( H , K ) , incurring only an additional logarithmic factor on the risk bounds.

To this end, suppose that the tuple ( H , K ) is pairwise testable and the class H = { h 1 , · · · , h K } is finite with size K . Let Φ i,j be the testing rule (will be constructed in Section 3.2) for h i , h j with error bound C ( δ ) and confidence δ > 0 . Let x T , ˜ y T be any realization of problem ( H , K ) . We define, for any h i ∈ H and t ∈ [ T ] , a surrogate loss vector:

̸

That is, the loss v i t [ j ] = 1 if and only if the test Φ i,j ( x t , ˜ y t -1 ) differs from h i ( x t ) . Given access to testers Φ i,j s, our prediction rule for ( H , K ) is then presented in Algorithm 1.

̸

Theorem 3. Let H ⊂ Y X be any hypothesis class of size K and K be any noisy kernel. If ( H , K ) is pairwise testable with error bound C ( δ ) as in Definition 4, then for any δ > 0 , the predictor in Algorithm 1 with C = C ( δ/ (2 K )) achieves the high probability minimax risk (Definition 2):

̸

Sketch of Proof. At a high level, our goal is to identify the ground truth classifier h k ∗ using the testing results of Φ i,j s. Note that pairwise testability implies, w.p. ≥ 1 -δ , the errors made by tester Φ k,k ∗ on h k ∗ is upper bounded by C ( δ/ 2 K ) for all k ∈ [ K ] simultaneously. However, for any other pair i, j = k ∗ , the tester Φ i,j does not provide any guarantees, since the samples used to test h i , h j originate from h k ∗ and is not realizable for Φ i,j . The key technical challenge is to extract the testing results for Φ k,k ∗ from the other irrelevant tests (i.e., Φ i,j with k ∗ ̸∈ { i, j } ), even when the k ∗ is unknown . This is resolved by our definition of l i t in Algorithm 1, which computes for each i the maximum testing loss over all of its competitors. This ensures that, for the ground truth k ∗ , the loss l k ∗ t ≤ C ( δ/ 2 K ) . While for any other i = k ∗ , we have l i t ≥ ∑ t r =1 v i r [ k ∗ ] ≥ ∑ t r =1 1 { h i ( x r ) = h k ∗ ( x r ) } -C ( δ/ 2 K ) . Therefore, any hypothesis h i for which l i t > C ( δ/ 2 K ) cannot be the ground truth. Algorithm 1 then maintains an index set S t that eliminates all h i for which l i t > C ( δ/ 2 K ) , and makes prediction ˆ y t = h ˆ k t ( x t ) with ˆ k t sampling uniformly from S t .

̸

̸

̸

To derive the risk bound, we use a potential -based analysis that relates the size of S t s with the prediction error 1 { h k ∗ ( x t ) = ˆ y t } . The intuition behind the analysis is that if E [1 { h k ∗ ( x t ) = ˆ y t } ] is large, then there will be many elements i ∈ S t for which h i ( x t ) = h k ∗ ( x t ) , and thus the loss l i t will (potentially) increase. Since Algorithm 1 constructs S t +1 by eliminating all i ∈ S t for which l i t > C ( δ/ 2 K ) , one can therefore bound the prediction error by the change in the size of S t s. The key technical challenge here is to control the hypotheses that differ from k ∗ but for which the tester Φ k,k ∗ errs, which is resolved by carefully defining a potential function. The claimed upper bound then follows by a similar argument as [14, Thm 2]. See Appendix B for complete proof.

̸

Note that, the reduction of Theorem 3 is general and does not rely on specific properties of the kernel K (such as the well-separation condition). It provides a black box reduction that converts any pairwise testing rule for two-hypotheses to a general online classification rule that introduces only a logarithmic factor on the risk bounds. This effectively decouples the adversarial property of features from the stochastic property of the noisy labels.

To understand how Theorem 3 operates, we consider the following example:

̸

Example 4. Let H ⊂ { 0 , 1 } X , and K be the bounded Bernoulli noise kernel with parameter η in Example 1. For any h i , h j ∈ H , we construct the following testing rule. We may assume, w.l.o.g., that h i ( x ) = h j ( x ) for all x ∈ X , since any x for which h i ( x ) = h j ( x ) do not affect our testing. Moreover, by relabeling, we can assume that h i ( x ) = 0 and h j ( x ) = 1 for all x ∈ X . At time step t , after observing the noisy labels ˜ y t -1 , we compute ˆ µ t = 1 t -1 ∑ t -1 r =1 ˜ y r . If ˆ µ t ≥ 1 2 , the tester predicts ˆ y t = 1 ; else, it predicts ˆ y t = 0 . By Azuma's inequality, the probability of making an error at step t is upper bounded by e -(1 -2 η ) 2 ( t -1) / 2 . Thus, for any n ≤ T , the probability of making any errors after

̸

step n is upper bounded by ∑ ∞ t = n e -(1 -2 η ) 2 ( t -1) / 2 ≤ e -(1 -2 η ) 2 n/ 2 (1 -2 η ) 2 . Taking n = 2 log(1 /δ (1 -2 η ) 2 ) (1 -2 η ) 2 one can upper bound the probability by δ . Therefore, the tuple ( H , K ) is pairwise testable with C ( δ ) ≤ 2 log(1 /δ (1 -2 η ) 2 ) (1 -2 η ) 2 . Invoking Theorem 3, we have:

Note that the risk bound in (9) recovers the risk in Example 1 up to a logarithmic factor, though it employs a completely different approach (cf. [1]). Moreover, Example 4 provides the key advantage that the risk holds with high probability and at a fast rate , which is known to be non-trivial for cumulative errors (see, e.g., [22, 21]). To our knowledge, it remains unclear whether the approach proposed in [1] admits a high probability guarantee.

3.2 Proof of Theorem 2: the conditional Le Cam-Birgé Testing

As demonstrated in Section 3.1, the risk of noisy online classification can be reduced to the pairwise testing of two hypotheses. However, we still need to construct the explicit pairwise testing rules. This section is devoted to providing a generic testing rule for general kernels.

̸

Let h 0 and h 1 be any two hypotheses. We may assume, w.l.o.g., that h 0 ( x ) = h 1 ( x ) for all x ∈ X , since the features for which h 0 and h 1 agree do not affect the testing. We now provide a more compact characterization of the kernel without explicitly referring to true labels. Let x T be any realization of features. For any i ∈ { 0 , 1 } , t ∈ [ T ] , and kernel K , we write Q x t i := K ( x t , h i ( x t )) .

We define Q J 0 and Q J 1 as the sets of all (joint) distributions over ˜ Y J induced by the kernel upto time step J for h 0 , h 1 , respectively. Equivalently, for i ∈ { 0 , 1 } , we have p ∈ Q J i if and only if for all t ∈ [ J ] and ˜ y t -1 ∈ ˜ Y t -1 , the conditional marginal p ˜ y t | ˜ y t -1 ∈ Q x t i .

The pairwise testing of h 0 , h 1 at time step J +1 is then equivalent to the (robust) hypothesis testing w.r.t. sets Q J 0 and Q J 1 . This is typically resolved using Le Cam-Birgé testing [17, Chapter 32.2] if the distributions are of product form. However, this does not hold for our purpose, since the distributions in Q J i can have highly correlated marginals. Our main result for addressing this issue is a conditional version of Le Cam-Birgé testing, as stated in Theorem 4 below. To the best of our knowledge, this conditional version is novel.

Theorem 4 ( conditional Le Cam-Birgé Testing) . Let Q J 0 and Q J 1 be the classes induced by a kernel upto time J as defined above. For any t ≤ J , we denote γ t = H 2 ( Q x t 0 , Q x t 1 ) and assume that Q x t i is convex for all i ∈ { 0 , 1 } . Then, there exists a testing rule ψ : ˜ Y J →{ 0 , 1 } such that

̸

̸

Sketch of Proof. The proof requires a suitable application of the minimax theorem by expressing the testing error as a linear function and arguing that the Q J i s are convex. The error bound is then controlled by a careful application of the chain-rule of Rényi divergence. See Appendix C.

Theorem 4 immediately implies the following cumulative risk bound:

Proposition 1. Let h 0 , h 1 be any hypotheses, x T be any realization of features and Q T i , Q x t i be defined as above with γ t = H 2 ( Q x t 0 , Q x t 1 ) . Then, there exists a tester ˆ y T such that for all δ > 0 , i ∈ { 0 , 1 } and ˜ p ∈ Q T i , w.p. ≥ 1 -δ over ˜ y T ∼ ˜ p , we have:

̸

Proof. Let n ∗ be the minimal number satisfying the RHS. If t ≤ n ∗ (this can be checked at each time step t using only x t and K ), we predict arbitrarily. If t ≥ n ∗ +1 , we use the tester ψ in Theorem 4 with J = n ∗ to produce an index ˆ i ∈ { 0 , 1 } and make the prediction h ˆ i ( x t ) for all following time steps. That is, we only use the tester at step n ∗ +1 and reuse the same testing result for all following

time steps. By Theorem 4, the probability of making errors after step n ∗ +1 is upper bounded by δ . Therefore, the cumulative risk is upper bounded by n ∗ with probability ≥ 1 -δ .

̸

Proof of Theorem 2. Let h 0 , h 1 ∈ H be any two-hypotheses. For any time step t such that h 0 ( x t ) = h 1 ( x t ) , we have, by the well-separation condition, that the gap γ t ≥ γ H in Proposition 1. Consider the following testing rule: for any time step t such that h 1 ( x t ) = h 2 ( x t ) , we predict the agreed label; else, we predict the same way as in Proposition 1. Clearly, we only make errors for the second case. Invoking Proposition 1 with γ t = γ H for all t ∈ [ T ] , we have n ∗ ≤ 2 log(2 /δ ) γ H . Therefore, the tuple ( H , K ) is pairwise testable with C ( δ ) = 2 log(2 /δ ) γ H . The upper bound on classification risk then follows by Theorem 3. The lower bound follows by Le Cam's two point method and constructing a hard hypothesis class using an epoch approach. We refer to Appendix D for the complete details.

Remark 1. Note that our techniques can be easily extended to infinite classes using the covering techniques from [1, 22]. Moreover, by applying Proposition 1, our results can be extended to scenarios where the gap parameters γ t are not uniformly bounded, such as in the case of Tsybakovtype noise [8], which would lead to risk bounds that scale sublinearly with T , in contrast to the constant risk in Theorem 2. We leave the details and extensions for a longer manuscript [23].

4 Tighter Bounds for Binary Labels via L 2 Gap

We have demonstrated in Theorem 2 that the minimax risk is tightly characterized by the Hellinger gap induced by the kernel. However, the dependency on log |H| remains sub-optimal. We show in this section a tight dependency on log |H| for classes with binary true labels via the L 2 gap.

Theorem 5. Let H ⊂ { 0 , 1 } X be any finite binary valued class, K be any noisy kernel that is well-separated at scale γ L w.r.t. the L 2 -distance 5 (Definition 3). Then, the expected minimax risk, as in Definition 1, is upper bounded by: ˜ r T ( H , K ) ≤ 16 log |H| γ L .

We begin with the following simple geometry fact that is crucial to our proof.

Lemma 1. Let Q ⊂ D ( ˜ Y ) be a convex and closed set, p be a point outside of Q with γ def = inf q ∈Q L 2 ( p, q ) . Denote by q ∗ ∈ Q the (unique) point that attains L 2 ( p, q ∗ ) = γ . Then for any q ∈ Q , we have L 2 ( q, p ) -L 2 ( q, q ∗ ) ≥ L 2 ( p, q ∗ ) = γ.

Proof. By the hyperplane separation theorem , the hyperplane perpendicular to line segment p -q ∗ at q ∗ separates Q and p . Therefore, the degree θ of angle formed by p -q ∗ -q is greater than π/ 2 . By the law of cosines, L 2 ( q, p ) ≥ L 2 ( q, q ∗ ) + L 2 ( q ∗ , p ) = L 2 ( q, q ∗ ) + γ .

Our key idea of proving Theorem 5 is to reduce the robust (noisy) online classification problem to a suitable conditional distribution estimation problem, as discussed next.

Online conditional distribution estimation. Let F ⊂ D ( ˜ Y ) X be a class of functions mapping X to distributions in D ( ˜ Y ) . Online conditional distribution estimation is a game between Nature and an estimator that follows the following protocol: (1) at each times step t , Nature selects some x t ∈ X and reveals it to the estimator; (2) the estimator then makes an estimation ˆ p t ∈ D ( ˜ Y ) , based on x t , ˜ y t -1 ; (3) Nature then selects some ˜ p t ∈ D ( ˜ Y ) , samples ˜ y t ∼ ˜ p t and reveals ˜ y t to the estimator. The goal is to find a (deterministic) estimator Φ that minimizes the regret :

where ˆ p t = Φ( x t , ˜ y t -1 ) , Q T is the operator specified in Definition 1 by setting Q x y := D ( ˜ Y ) for all x , y , and L is any divergence. We emphasize that the distributions ˜ p T are not necessarily realizable by f and are selected completely arbitrarily. This contrasts with the well-specified cases employed in [10, 4], and is the key that enables us to handle the unknown noisy label distributions.

We now establish the following key technical lemma, see Appendix E for proof.

Lemma2. Let F ⊂ D ( ˜ Y ) X be a finite distribution-valued function class. Then, for the L 2 divergence, there exists an estimator Φ , i.e., the Exponential Weight Average (EWA) algorithm, such that

Moreover, estimation ˆ p t is a convex combination of { f ( x t ) : f ∈ F} .

Proof Sketch of Theorem 5. We provide the high level ideas and refer to Appendix F for complete details. We define the following distribution-valued function class F using hypothesis class H and noisy kernel K . For any x ∈ X , we denote by Q x 0 and Q x 1 the sets of noisy label distributions corresponding to labels 0 and 1 , respectively. Since the kernel K is well-separated at scale γ L under L 2 divergence, we have, by the hyperplane separation theorem , that there must be q x 0 ∈ Q x 0 and q x 1 ∈ Q x 1 such that L 2 ( q x 0 , q x 1 ) = L 2 ( Q x 0 , Q x 1 ) ≥ γ L . We now define for any h ∈ H the function f h such that ∀ x ∈ X , f h ( x ) = q x h ( x ) . Let F = { f h : h ∈ H} and Φ be the estimator from Lemma 2 with class F and L 2 divergence (using x T , ˜ y T from the original noisy classification game). Our classification rule is defined as ˆ y t = arg min y { L 2 ( q x t y , ˆ p t ) : y ∈ { 0 , 1 }} . That is, we predict the label y so that q x t y is closer to ˆ p t under L 2 divergence, where ˆ p t = Φ( x t , ˜ y t -1 ) .

Let h ∗ ∈ H be the underlying true classification function. We have by Lemma 2 that

̸

where Q T K is the operator in Definition 1. Now, our key technical goal is to show that L 2 (˜ p t , ˆ p t ) -L 2 (˜ p t , f h ∗ ( x t )) ≥ L 2 (ˆ p t , f h ∗ ( x t )) ≥ γ L 4 1 { ˆ y t = h ∗ ( x t ) } via Lemma 1 and a geometric argument, as illustrated in the figure below:

Q

x

t

h

∗

(

x

t

)

Q

x

t

1

-

h

∗

(

x

t

)

ˆ

p

t

q

x

t

1

-

h

∗

(

x

t

)

f

h

∗

(

x

t

)

˜

p

t

γ

L

/

2

̸

The expected minimax risk bound ∑ T t =1 1 { ˆ y t = h ∗ ( x t ) } ≤ 16 log |H| γ L then follows from (11).

Although both our proofs and those provided in [1] are based on the EWA algorithm, the analysis and resulting algorithms are fundamentally different. For instance, in [1], the learning rate of EWA depends on the parameter η , while we set it to 1 / 4 (see Appendix E). More importantly, our proof applies to any noisy kernel that satisfies the well-separation condition (including cases where | ˜ Y| > 2 ), which benefits from our geometric interpretation of the kernels. Interestingly, for the specific setting investigated in [1] (i.e., Example 1), our result yields the same order up to a constant factor, since 1 -2 √ η (1 -η ) = Θ((1 -2 η ) 2 ) for η ∈ [0 , 1 2 ) . In general, we have 4 γ L ≤ γ H ≤ √ Mγ L .

5 Discussion

In this paper, we provide nearly matching lower and upper bounds for online classification with noisy labels via the Hellinger gap of the induced noisy label distributions. Our approach works for a wide range of hypothesis classes and noisy mechanisms. We expect our results to have a wide range of applications, such as online learning under (local) differential privacy constraints and online denoising tasks involving data derived from (noisy) physical measurements (such as learning from quantum data [15]). The main open problem remaining is to close the logarithmic gap in Theorem 2 for general kernels. While our work primarily focuses on the information-theoretically achievable minimax risks, we believe that finding computationally efficient predictors (including oracle-efficient methods as in [14]) would also be of significant interest.

References

Shai Ben-David, Dávid Pál, and Shai Shalev-Shwartz. Agnostic online learning. In Conference on Learning Theory , volume 3, 2009.

Alankrita Bhatt and Young-Han Kim. Sequential prediction under log-loss with side information. In Algorithmic Learning Theory , pages 340-344. PMLR, 2021.

Blair Bilodeau, Dylan Foster, and Daniel Roy. Tight bounds on minimax regret under logarithmic loss via self-concordance. In International Conference on Machine Learning , pages 919-929. PMLR, 2020.

Blair Bilodeau, Dylan J Foster, and Daniel M Roy. Minimax rates for conditional density estimation via empirical entropy. arXiv preprint arXiv:2109.10461 , 2021.

N. Cesa-Bianchi and G. Lugosi. Prediction, Learning and Games . Cambridge University Press, 2006.

Nicolo Cesa-Bianchi, Shai Shalev-Shwartz, and Ohad Shamir. Online learning of noisy data. IEEE Transactions on Information Theory , 57(12):7907-7931, 2011.

Amit Daniely, Sivan Sabato, Shai Ben-David, and Shai Shalev-Shwartz. Multiclass learnability and the erm principle. J. Mach. Learn. Res. , 16(1):2377-2404, 2015.

Ilias Diakonikolas, Daniel M Kane, Vasilis Kontonis, Christos Tzamos, and Nikos Zarifis. Efficiently learning halfspaces with tsybakov noise. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing , pages 88-101, 2021.

Sam Efromovich. Conditional density estimation in a regression setting. The Annals of Statistics , 35:2504-2535, 2007.

Dylan J Foster, Sham M Kakade, Jian Qian, and Alexander Rakhlin. The statistical complexity of interactive decision making. arXiv preprint arXiv:2112.13487 , 2021.

Sivakanth Gopi, Gautam Kamath, Janardhan Kulkarni, Aleksandar Nikolov, Zhiwei Steven Wu, and Huanyu Zhang. Locally private hypothesis selection. In Conference on Learning Theory , pages 1785-1816. PMLR, 2020.

Peter D Grünwald and Nishant A Mehta. Fast rates for general unbounded loss functions: from erm to generalized bayes. The Journal of Machine Learning Research , 21(1):2040-2119, 2020.

Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends® in Optimization , 2(3-4):157-325, 2016.

Sham Kakade and Adam T Kalai. From batch to transductive online learning. Advances in Neural Information Processing Systems , 18, 2005.

Abram Magner and Arun Padakandla. Fat shattering, joint measurability, and pac learnability of povm hypothesis classes. arXiv preprint arXiv:2308.12304 , 2023.

Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. Advances in neural information processing systems , 26, 2013.

Yury Polyanskiy and Yihong Wu. Information Theory: From Coding to Learning . Cambridge University Press, 2022.

Alexander Rakhlin and Karthik Sridharan. Sequential probability assignment with binary alphabets and large classes of experts. arXiv preprint arXiv:1501.07340 , 2015.

Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Random averages, combinatorial parameters, and learnability. In Advances in Neural Information Processing Systems , 2010.

Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms . Cambridge university press, 2014.

Dirk van der Hoeven, Nikita Zhivotovskiy, and Nicolò Cesa-Bianchi. High-probability risk bounds via sequential predictors. arXiv preprint arXiv:2308.07588 , 2023.

Changlong Wu, Ananth Grama, and Wojciech Szpankowski. Online learning in dynamically changing environments. In Conference on Learning Theory , pages 325-358. PMLR 195, 2023.

Changlong Wu, Ananth Grama, and Wojciech Szpankowski. Robust online classification: From estimation to denoising. arXiv preprint arXiv:2309.01698 , 2023.

Changlong Wu, Mohsen Heidari, Ananth Grama, and Wojciech Szpankowski. Expected worst case regret via stochastic sequential covering. arXiv preprint arXiv:2209.04417 , 2022.

Changlong Wu, Mohsen Heidari, Ananth Grama, and Wojciech Szpankowski. Precise regret bounds for log-loss via a truncated bayesian algorithm. In Advances in Neural Information Processing Systems , volume 35, pages 26903-26914, 2022.

Changlong Wu, Yifan Wang, Ananth Grama, and Wojciech Szpankowski. Learning functional distributions with private labels. In International Conference on Machine Learning (ICML) , volume 202 of PMLR , pages 37728-37744. PMLR, 23-29 Jul 2023.

Tong Zhang. Mathematical analysis of machine learning algorithms . Cambridge University Press, 2023.

A Martingale Concentration Inequalities

In this appendix, we present some standard concentration results for martingales, which will be useful for deriving high probability guarantees. We refer to [27, Chapter 13.1] for the proofs.

Lemma 3 (Azuma's Inequality) . Let X 1 , · · · , X T be an arbitrary random process adaptive to some filtration {F t } t ≤ T such that | X t | ≤ M for all t ≤ T . Let Y t = E [ X t | F t -1 ] be the conditional expected random variable of X t . Then for all δ > 0 , we have

and

The following lemma provides a tighter concentration when X t ≥ 0 , which can be viewed as an Martingale version of the multiplicative Chernoff bound.

Lemma 4 ([27, Theorem 13.5]) . Let X 1 , · · · , X T be an arbitrary random process adaptive to some filtration {F t } t ≤ T such that 0 ≤ X t ≤ M for all t ≤ T . Let Y t = E [ X t | F t -1 ] be the conditional expected random variable of X t . Then for all δ > 0 we have

and

Proof. Applying [27, Thm 13.5] with ξ t = X t /M and λ = 1 in the theorem.

B Proof of Theorem 3

Let h k ∗ ∈ H be the underlying true classification function and x T be the realization of features. We take C = C ( δ/ 2 K ) in Algorithm 1. By definition of pairwise testability and union bound, we have w.p. ≥ 1 -δ/ 2 over the randomness of ˜ y T and the internal randomness of Φ k,k ∗ s that for all k ∈ [ K ] ,

̸

Note that for any other { i, j } ̸∋ k ∗ , equation (12) may not hold for predictor Φ i,j . However, our following argument relies only on the guarantees for predictors Φ k,k ∗ , which effectively makes our pairwise testing realizable .

We now condition on the event defined in (12). Let v k t with k ∈ [ K ] and t ∈ [ T ] be the surrogate loss vector, as defined in (7). We observe the following key properties

We have for all t ∈ [ T ] that

̸

For any k = k ∗ , we have for all t ∈ [ T ] :

̸

The first property is straightforward by the definition of v k t and (12). The second property holds since the lower bound is attained when j = k ∗ .

We now analyze the performance of Algorithm 1. By property (13), we know that k ∗ ∈ S t for all t ∈ [ T ] , i.e., | S t | ≥ 1 . Let N t = | S t | . We define for all t ∈ [ T ] the potential :

̸

̸

Clearly, we have E t ≤ 2 C ( δ/ (2 K )) N t . Let D t = |{ k ∈ S t : h k ( x t ) = h k ∗ ( x t ) }| . We have:

̸

since for any k ∈ S t such that h k ( x t ) = h k ∗ ( x t ) , either k is removed from S t +1 (which contributes at most N t -N t +1 ) or its contribution to E t +1 is decreased by 1 when compared to E t (this is because by our construction of Algorithm 1 and property (14) once the contributions of k to E t equals 0 it must be excluded from S t +1 ). We have, by definition of ˆ y t , that:

̸

By a standard argument [14, Thm 2], we have:

Moreover, we observe that

where ( a ) and ( b ) follow by E t ≤ 2 C ( δ/ (2 K )) N t and N t ≥ N t +1 ; ( c ) follows by repeating the same argument for another T -1 steps.

Therefore, we conclude

̸

where the randomness is on the selection of ˆ k t ∼ S t . Since our selection of ˆ k t are independent (conditioning on S t ) for different t , and the indicator is bounded by 1 and non-negative, we can invoke Lemma 4 (second part) to obtain a high probability guarantee of confidence δ/ 2 by introducing an extra log(2 /δ ) additive term. The theorem now follows by a union bound with the event (12).

C Proof of Theorem 4

We start with an application of the minimax theorem to hypothesis testing 6 .

Lemma 5. Let P 0 and P 1 be two sets of distributions over a finite domain Ω . If P 0 and P 1 are convex under L 1 distance (i.e., total variation), then

Moreover, if ϕ ∗ is the function attains minimal, then the tester ψ ∗ ( ω ) = 1 { ϕ ∗ ( ω ) < 0 . 5 } achieves

̸

̸

Proof. Observe that the function ϕ can be viewed as a vector in [0 , 1] Ω . Moreover, the distributions over Ω can be viewed as vectors in [0 , 1] Ω as well. Therefore, we have E ω ∼ p 0 [1 -ϕ ( ω )] + E ω ∼ p 1 [ ϕ ( ω )] = ⟨ p 0 , 1 -ϕ ⟩ + ⟨ p 1 , ϕ ⟩ , which is a linear function w.r.t. both ( p 0 , p 1 ) and ϕ . Since the both P 0 ×P 1 and [0 , 1] Ω are convex and [0 , 1] Ω is compact, we can invoke the minimax theorem [5, Thm 7.1] to obtain

̸

where the last equality follows by Le Cam's two point lemma [17, Theorem 7.7]. Let ϕ ∗ be the function attains minimal and ψ ∗ ( ω ) = 1 { ϕ ∗ ( ω ) < 0 . 5 } . We have 1 { ψ ∗ ( ω ) = i } ≤ 2(1 -i -ϕ ∗ ( ω )) for all i ∈ { 0 , 1 } . To see this, for i = 0 , we have ψ ∗ ( ω ) = 0 only if ϕ ∗ ( ω ) < 0 . 5 , thus 1 -ϕ ∗ ( ω ) ≥ 0 . 5 (the case for i = 1 follows similarly). Therefore, we have for all p 0 ∈ P 0 , p 1 ∈ P 1

̸

̸

This completes the proof.

We now establish the following key property, which demonstrates that the distribution classes constructed in Theorem 4 satisfy the condition of Lemma 5.

Lemma 6. Let Q J 0 and Q J 1 be the sets in Theorem 4. Then Q J 0 and Q J 1 are convex under L 1 distance.

Proof. Let p 1 , p 2 ∈ Q J i for i ∈ { 0 , 1 } and λ ∈ [0 , 1] . We need to show that p = λp 1 +(1 -λ ) p 2 ∈ Q J i as well. For any given t ∈ [ T ] , we have

where the last inclusion follows by convexity of Q x t i as assumed in Theorem 4. Therefore, we have p ∈ Q J i by definition of Q J i .

Now, our main technical part is to bound the total variation TV ( Q J 0 , Q J 1 ) . The primary challenge comes from controlling the dependencies of conditional marginals of the distributions. To this end, we introduce the concept of Renyi divergence . Let p 1 , p 2 be two distributions over the same finite domain Ω , the α -Renyi divergence is defined as

If p, q are distributions over domain Ω 1 × Ω 2 and r is a distribution over Ω 1 , then the conditional α -Renyi divergence is defined as

The following property about Renyi divergence is well known [17, Chapter 7.12]:

̸

Lemma 7. Let p, q be two distributions over Ω 1 × Ω 2 and p (1) and q (1) be the restrictions of p, q on Ω 1 , respectively. Then the following chain rule holds

where r ( ω 1 ) = p (1) ( ω 1 ) α q (1) ( ω 1 ) 1 -α e -( α -1) D α ( p (1) ,q (1) ) is a distribution over Ω 1 .

The following key result bounds the Renyi divergence between Q J 0 and Q J 1 :

Proposition 2. Let Q J 0 and Q J 1 be the sets in Theorem 4. If inf p ∈Q x t 0 ,q ∈Q x t 1 D α ( p, q ) ≥ η t holds for all t ≤ J . Then

Proof. We prove by induction on J . The base case for J = 1 is trivial, since Q 1 0 = Q x 1 0 and Q 1 1 = Q x 1 1 . We now prove the induction step with J ≥ 2 . For any pair p ∈ Q J 0 and q ∈ Q J 1 , we have by Lemma 7 that D α ( p, q ) = D α ( p (1) , q (1) ) + D α ( p, q | r ) , where p (1) , q (1) are restrictions of p , q on ˜ y J -1 and r is a distribution over ˜ Y J -1 . By definition of α -Renyi divergence, we have

where ( a ) follows since p ˜ y J | ˜ y J -1 ∈ Q x J 0 and q ˜ y J | ˜ y J -1 ∈ Q x J 1 by the definition of Q J 0 and Q J 1 ; ( b ) follows by assumption. The result then follows by induction hypothesis D α ( p (1) , q (1) ) ≥ ∑ J -1 t =1 η t , since p (1) ∈ Q J -1 0 and q (1) ∈ Q J -1 1 .

The following result converts the Renyi divergence based bounds to that with Hellinger divergence. Proposition 3. Let Q J 0 and Q J 1 be the sets in Theorem 4. If H 2 ( Q x t 0 , Q x t 1 ) ≥ γ t ≥ 0 holds for all t ∈ [ J ] . Then

Proof. Observe that, for any distributions p, q we have

Specifically, for give p ∈ Q J 0 and q ∈ Q J 1 , we have

where η t s are the constants in Proposition 2 and the last inequality follows by e -1 2 η t ≤ 1 -γ t / 2 due to (17) again. This completes the proof.

Proof of Theorem 4. We have, by Lemma 5, that the testing error is upper bounded by 2(1 -inf p ∈Q J 0 ,q ∈Q J 1 || p -q || TV ) . Fix any such p, q , we have by [17, Equation 7.22] that 1 -|| p -q || TV ≤ 1 -1 2 H 2 ( p, q ) . The result then follows by Proposition 3.

D Proof of Theorem 2 (Lower Bound)

We denote L ≤ log K with K = |H| , and x 1 , · · · , x L be L distinct elements in X satisfies the condition of the theorem. We define for any b ∈ { 0 , 1 } L a function h b such that for all i ∈ [ L ] , h b ( x i ) = y i if b [ i ] = 0 and h b ( x i ) = y ′ i otherwise, where y i = y ′ i ∈ Y are the elements that satisfy inf p ∈Q x i y i ,q ∈Q x i y ′ i { H 2 ( p, q ) } ≤ γ H . Let H be the class consisting of all such h b . Let q i ∈ Q x i y i

̸

and q ′ i ∈ Q x i y ′ i be the elements satisfying H 2 ( q i , q ′ i ) ≤ γ H . We now partition the features x T into L epochs, each of length T/L , such that each epoch i has constant feature x i . Let h be a random function selected uniformly from H . We claim that for any prediction rule ˆ y t and any epoch i we have

̸

where ˜ y t ∼ q i if h ( x i ) = y i and ˜ y t ∼ q ′ i otherwise. The proposition now follows by counting the errors for all L epochs.

We now establish (18) using the Le Cam's two point method. Clearly, for each epoch i , the prediction performance depends only on the label y i = h ( x i ) , which is uniform over { y i , y ′ i } and independent for different epochs by construction. For any time step j during the i th epoch, we denote by ˜ y j -1 and ˜ y ′ j -1 the samples generated from q i and q ′ i , respectively. By the Le Cam's two point method [17, Theorem 7.7] the expected error at step j is lower bounded by

where the inequality follows from [17, Equation 7.22]. Note that the RHS of (19) is monotone decreasing w.r.t. H 2 (˜ y j -1 , ˜ y ′ j -1 ) , since H 2 ( p, q ) ≤ 2 for all p, q .

By the tensorization of Hellinger divergence [17, Equation 7.23], we have

where the last inequality is implied by H 2 ( q i , q ′ i ) ≤ γ H . Using the fact log(1 -x ) ≥ -x 1 -x , we have if γ H ≤ 1 and j -1 ≤ 1 γ H then 2 -2(1 -γ H / 2) j -1 ≤ 2(1 -e -1 ) < 2 . Therefore, the RHS of (19) is lower bounded by an absolute positive constant for all j -1 ≤ 1 γ H , and hence the expected cumulative error will be lower bounded by Ω(1 /γ H ) during epoch i . This completes the proof.

E Proof of Lemma 2

Before presenting a formal proof, we first develop some technical concepts. Let ˜ Y be the noisy label set and D ( ˜ Y ) be the class of distributions over ˜ Y . We say a function ℓ : ˜ Y × D ( ˜ Y ) → R + is α -exp-concave if for any ˜ y ∈ ˜ Y , the function e -αℓ (˜ y,p ) is concave w.r.t. p for some α ∈ R ≥ 0 .

Proposition 4. The function ℓ (˜ y, p ) = || e ˜ y -p || 2 2 is 1 / 4 -Exp-concave, where e ˜ y denotes distribution assigning probability 1 on ˜ y .

Proof. We have by [13, Lemma 4.2] that a function f is α -Exp-concave if and only if

For any q ∈ D ( ˜ Y ) , we denote f ( p ) = || p -q || 2 2 . We have ∇ f ( p ) = 2( p -q ) and ∇ 2 f ( p ) = 2 I , where I is the identity matrix. Taking any u ∈ R J , we have 1 4 ⟨ u, 2( p -q ) ⟩ 2 ≤ || u || 2 2 || p -q || 2 2 ≤ 2 || u || 2 2 = 2 u T Iu , where the first inequality follows by Cauchy-Schwarz inequality and the second inequality follows by:

since p, q ∈ D ( ˜ Y ) . This completes the proof.

Wenowintroduce the Exponential Weighted Average (EWA) algorithm and its regret analysis under the Exp-concave losses, which is mostly standard [5, Chapter 3.3] and we include it here for completeness. Let F = { f 1 , · · · , f K } ⊂ D ( ˜ Y ) X be a D ( ˜ Y ) -valued function class and ℓ : ˜ Y × D ( ˜ Y ) → R ≥ 0 be α -Exp-concave. The EWA algorithm is presented in Algorithm 2.

Algorithm 2: Exponential Weighted Average (EWA) estimator

: Class

Input

F

=

{

f

Set

w

1

=

{

1

,

· · ·

,

1

1

,

· · ·

} ∈

for t = 1 , · · · , T do

Receive x t ; Make prediction:

Receive noisy label ˜ y t ;

for k ∈ [ K ] do

Set

w

t

+1

[

k

] =

w

t

[

, f

K

K

;

k

]

e

}

-

αℓ

(˜

y

t

and

α

-Exp-concave loss

ℓ

,f

k

(

x

t

))

;

Algorithm 2 provides the following regret bound:

Proposition 5 ([5, Proposition 3.1]) . Let F ⊂ D ( ˜ Y ) X be any finite class and ℓ be an α -Exp-concave loss. If ˆ p t is the estimator in Algorithm 2, then for any x T ∈ X T and ˜ y T ∈ ˜ Y T we have

Proof of Lemma 2. Let Φ be the EWA estimator as in Algorithm 2 with input class F , loss ℓ (˜ y, p ) def = L 2 ( e ˜ y , p ) and α = 1 / 4 . Let ˜ y T be any realization of the noisy labels. We denote e t as the standard base of R M with value 1 at position ˜ y t and zeros otherwise. By 1 / 4 -Exp-concavity of loss ℓ (Proposition 4) and the regret bound from Proposition 5, we have:

Note that, this bound holds point-wise w.r.t. any individual x T and ˜ y T .

Fix any x T and (joint) distribution ˜ p over ˜ Y T . We denote E t as the conditional expectation on ˜ y t over the randomness of ˜ y T ∼ ˜ p conditioning on ˜ y t -1 and denote ˜ p t as the conditional marginal. By the elementary identity E [ L 2 ( X,p ) -L 2 ( X,q )] = L 2 ( E [ X ] , p ) -L 2 ( E [ X ] , q ) for any random variable X over D ( ˜ Y ) , we have for all t ∈ [ T ] that:

since E t [ e t ] = ˜ p t for ˜ y t ∼ ˜ p t and ˆ p t depends only on ˜ y t -1 . We now take E ˜ y T on both sides of (20). By sup E ≤ E sup and the law of total probability (i.e., E ˜ y T [ X 1 + · · · + X T ] = E ˜ y T [ E 1 [ X 1 ] + · · · + E T [ X T ]] for any random variables X T ), we have:

where ˜ p runs over all (joint) distributions over ˜ Y T . The lemma then follows by the equivalence between operators Q T K ≡ sup ˜ p E ˜ y T when taking the kernel set Q x y := D ( ˜ Y ) for all x , y (see the discussion following Definition 1). The last part follows by the fact that the EWA estimator automatically ensures ˆ p t is a convex combination of { f ( x t ) : f ∈ F} for all t ∈ [ T ] .

R

F Proof of Theorem 5

We define the following distribution valued function class F using hypothesis class H and noisy kernel K . For any x ∈ X , we denote by Q x 0 and Q x 1 the sets of noisy label distributions corresponding to labels 0 and 1 , respectively. Since the kernel K is well-separated at scale γ L under L 2 divergence, we have, by the hyperplane separation theorem , that there must be q x 0 ∈ Q x 0 and q x 1 ∈ Q x 1 such that L 2 ( q x 0 , q x 1 ) = L 2 ( Q x 0 , Q x 1 ) ≥ γ L . We now define for any h ∈ H the function f h such that ∀ x ∈ X , f h ( x ) = q x h ( x ) . Let F = { f h : h ∈ H} and Φ be the estimator from Lemma 2 with class F and L 2 divergence (using x T , ˜ y T from the original noisy classification game). Our classification predictor is as follows:

That is, we predict the label y so that q x t y is closer to ˆ p t under L 2 divergence, where ˆ p t = Φ( x t , ˜ y t -1 ) .

Let h ∗ ∈ H be the underlying true classification function and x T be the realization of features. We have by Lemma 2 and 1 / 4 -Exp-concavity of L 2 divergence that 7

where Q T K is the operator in Definition 1.

For any time step t , we denote by y t = h ∗ ( x t ) the true label. Since q x t y ∈ Q x t y are the elements satisfying L 2 ( q x t 0 , q x t 1 ) = L 2 ( Q x 1 0 , Q x t 1 ) ≥ γ L and ˆ q t is a convex combination of q x t 0 and q x t 1 (Lemma 2), we have q x t y t is the closest element in Q x t y t to ˆ p t under L 2 divergence. Note that, we also have ˜ p t ∈ Q x t y t . Invoking Lemma 1, we find

Denote a t = L 2 (˜ p t , ˆ p t ) -L 2 (˜ p t , f h ∗ ( x t )) . We have, by (23) and f h ∗ ( x t ) = q x t y t that a t ≥ L 2 (ˆ p t , f h ∗ ( x t )) . Therefore:

For all t ∈ [ T ] , a t ≥ 0 , since ∀ p, q, L 2 ( p, q ) ≥ 0 ;

̸

If ˆ y t = y t , then a t ≥ γ L / 4 . This is because the event { ˆ y t = y t } implies that L 2 (ˆ p t , q x t y t ) ≥ L 2 (ˆ p t , q x t 1 -y t ) . Hence, L 2 (ˆ p t , f h ∗ ( x t )) = L 2 (ˆ p t , q x t y t ) ≥ γ L / 4 . Here, we used the following geometric fact:

̸

̸

This implies that ∀ t ∈ [ T ] , a t ≥ γ L 4 1 { ˆ y t = y t } , therefore:

̸

The expected minimax risk now follows from (22).

G Tight Bounds for Kernel Sets of Size One

In this appendix, we establish an upper bound for the case when the kernel set size |Q x y | = 1 for all x , y . This includes, for instance, the case when the parameter η t is known in Example 1.

Theorem 6. Let H ⊂ Y X be any finite class and K be any noisy kernel that is well-separated at scale γ H w.r.t. squared Hellinger distance such that |Q x y | = 1 for all x , y . Then the high probability minimax risk at confidence δ > 0 is upper bounded by

Proof. Our proof follows a similar path as in the proof of Theorem 5, but replacing the L 2 loss with log-loss. Specifically, for any h ∈ H , we define f h ( x ) = q x h ( x ) , where q x h ( x ) is the unique element in Q x h ( x ) . Denote F = { f h : h ∈ H} . We run the EWA algorithm (Algorithm 2) over F with α = 1 and ℓ being the log-loss [10], and produce an estimator ˆ p T . The classifier is then given by

Now, our key observation is that the noisy label distribution ˜ p t = f h ∗ ( x t ) is well-specified (since |Q x y | = 1 , the only choice for ˜ p t is f h ∗ ( x t ) ), where h ∗ is the ground truth classifier. Therefore, invoking [10, Lemma A.14], we find

̸

We claim that 1 { ˆ y t = h ∗ ( x t ) } ≤ 4 γ H H 2 (˜ p t , ˆ p t ) . Clearly, this automatically satisfies if ˆ y t = h ∗ ( x t ) . For ˆ y t = h ∗ ( x t ) , we have H 2 ( q x t ˆ y t , ˆ p t ) ≤ H 2 ( q x t h ∗ ( x t ) , ˆ p t ) = H 2 (˜ p t , ˆ p t ) by definition of ˆ y t . This implies that

where the first inequality follows by triangle inequality of Hellinger distance (the factor 1 4 comes from the conversion form squared Hellinger distance to Hellinger distance), and the second inequality follows by definition of γ H . Therefore, we have w.p. ≥ 1 -δ that

̸

This completes the proof since |H| ≥ |F| .

Observe that the key ingredient in the proof of Theorem 6 is the realizability of ˜ p t by f h ∗ (i.e., well-specified) due to the property |Q x y | = 1 , which does not hold for general kernels.

H Adaptive v.s. Oblivious Adversaries

In this appendix, we explain how the guarantees for oblivious adversaries can be extended to adaptive adversaries. This primarily follows from [5, Lemma 4.1], but needs careful adaptation to fit our needs. We consider the following abstract treatment: we assume that the adversary performs any operation Q t at time step t and produces an action z t . For any randomized prediction rule ˆ y T , the adaptive risk can be expressed as

Assume now that the randomness of ˆ y t 's is independent and that ˆ y t depends only on z t . We claim that

We prove the case for T = 2 to demonstrate the ideas; the general case follows by induction. Observe that

̸

where ( a ) follows since ℓ ( z 1 , ˆ y 1 ) is independent of Q 2 E ˆ y 2 ; ( b ) follows by the linearity of expectation; ( c ) follows by the independence of ˆ y 1 and ˆ y 2 , since the term Q 2 E ˆ y 2 ℓ ( z 2 , ˆ y 2 ) has nothing to do with the realization of ˆ y 1 ; ( d ) follows since E ˆ y 1 [ ℓ ( z 1 , ˆ y 1 )] is independent of z 2 ; ( e ) follows by the linearity of expectation.

Observe that, all the predictors constructed in this paper have independent internal randomness (in fact, the only place where randomness is introduced is in Algorithm 1); thus, our derived risk bounds hold for adaptive adversaries as well.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction is accurate to reflect the paper's contributions.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitation is discussed in the Discussion section

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The assumptions are accurate and the proofs are complete.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We confirm the paper meets the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a pure theory paper, we do not see any direct societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of online classification where the true labels are corrupted by unknown stochastic noise and the features are adversarially generated. The main contributions of this paper include: 1. Establishing fundamental limits of minimax risk in online classification with noisy labels by nearly matching lower and upper bounds across various hypothesis classes and noisy kernels; 2. Introducing a reduction to an online comparison scheme of two hypotheses and a new conditional version of Le Cam-Birgé testing suitable for online settings; 3. Characterizing the minimax risk using the Hellinger gap of noisy label distributions.\n\nSTRENGTHS:\n+ The theoretical analysis is rigorous, providing clear bounds and demonstrating their tightness through detailed proofs. The paper systematically addresses various aspects of the problem, ensuring a comprehensive exploration.\n+ The paper is well-structured, with clear definitions and thorough explanations of the concepts and results. Theorems and proofs are presented in a logical sequence.\n\nWEAKNESSES:\n- While the theoretical contributions are good, the paper could benefit from discussing practical implementations and real-world applications of the proposed methods.\n- Some assumptions, such as the nature of the noise and the hypothesis class, might be restrictive. A more detailed examination of these assumptions' impact on the results' generality would strengthen the paper.\n- The paper lacks experimental validation of the theoretical results. Providing empirical evidence, even in simulated settings, would enhance the credibility and applicability of the findings.\n\nQUESTIONS:\n1. How do the assumptions regarding the noise model and hypothesis class impact the generality of the results? Can the methods be adapted to other types of noise or broader hypothesis classes?\n2. Have you considered any practical applications or scenarios where your theoretical findings could be applied? If so, what are the potential challenges in these applications?\n\nLIMITATIONS:\n- Limitations: In the discussion section, the discussion on the limitations is not sufficient. It's difficult for readers to find this part.\n- Societal Impact: As this is a pure theory paper, there are no direct societal impacts. However, the authors could briefly mention potential positive impacts, such as improved robustness in machine learning models dealing with noisy data, and any negative implications if misapplied, such as in adversarial settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies online classification with stochastic label noise, where a hypothesis $h^*$ is drawn from some hypothesis class $H$ and the learner wants to learn $h^*$ by observing the noisy labels of the online data.\nThe standard mistake bound used in online learning counts the number of predictions different from the true labels. In this paper, instead, the mistake bound counts the number of predictions that disagree with the target $h^*$.\nThis paper proposes Noisy Kernel, a new framework to model the label noise, which generalizes the bounded noise setting studied by prior work. The main result of this paper is to show that the Hellinger gap of the noisy kernel gives a good characterization of the min-max mistake bound of learning a finite hypothesis class under their model. Specifically, they design a learner with mistake bound $\\log^2(|H|)/\\gamma_H$, where $\\gamma_H$ is the Hellinger gap of the noise kernel. On the other hand, for any fixed noisy kernel, they design a hypothesis class $H$ such that every learner has a mistake bound at least $\\log(|H|)/\\gamma_H$ to learn $H$. For the binary classification problem, the author further designs a learner with a mistake bound $\\log(|H|)/\\gamma_L$, where $\\gamma_L$ is the $L^2$ gap of the noisy kernel, improving the dependence on $\\log(|H|)$.\n\nSTRENGTHS:\n1. This paper proposes a novel framework that models the label noise for the online classification problem. This generalizes the previous model for bounded label noise.\n2. To prove the upper bound for the mistake bound, the paper introduces a novel reduction from the classification problem to the pairwise hypothesis testing problem, which could potentially be useful for designing algorithms for other learning problems.\n3. A matching lower bound on the dependence on the Hellinger gap of the noisy kernel is proved.\n\nWEAKNESSES:\n1. Though the framework proposed by the paper models a large class of noise models, the learner designed by the paper only works for learning finite hypothesis classes.\nPrior work, Agnostic Online Learning by Ben-David et al. proves a mistake bound that depends on the Littlestone dimension of the hypothesis class. There is a big gap between hypothesis classes with finite Littlestone dimension and finite hypothesis classes. For example, the class of singleton functions is infinite but has Littlestone dimension 1. Such a simple class is not learnable using the learner designed by the paper because it is impossible to do pairwise hypothesis testing for an unbounded set of hypotheses.\nFrom this point of view, I do not think the paper fully answers which class is learnable under their noisy model. \nPerhaps, this might be because the noise model studied by the paper is too general and is thus hard for one to design algorithms that make use of the structure of the hypothesis class.\n\n2. Though the paper gives a tight bound on the dependence on the Hellinger gap of the noise kernel, the upper bound and the lower bound differ by a $\\log(|H|)$ factor, which is sometimes linear in the dimension of the input and thus cannot be ignored in general. Though the paper shows that for binary classification, they can close this gap, a new gap on the dependence of the divergence gap is created.\n\nQUESTIONS:\n1. The model of the noisy kernel is not as intuitive as the prior noisy models such as Massart noise. In section 5, the authors mention that the noisy kernel model has many applications such as learning with privacy constraints or learning from quantum data. What would be the noisy kernels for these types of problems?\n\n2. To run the learner designed by the paper, one has to know the error bound of the pairwise hypothesis testing problem. How can one know such an error bound before running the online learning algorithm? If we do not know such a bound, would it be possible to get any data-driven algorithm?\n\n3. I would like the authors to comment on the weakness pointed out above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies online classification where the features are chosen adversarially, but the labels are corrupted by some unknown stochastic noise. The learner makes predictions using the true features and noisy labels, but is evaluated against the true labels (noiseless) in the minimax sense. The authors consider a very general noise generation mechanism specified by a kernel, which for every (feature, true label) pair, outputs a subset of distributions over the noisy label space. The adversary generates the noisy label by picking a distribution output by the kernel and sampling from it. The authors study the minimax risk under this setting and derive upper and lower bounds on the minimax risk in terms of the Hellinger gap of the noisy distributions output by the kernel and the cardinality of the hypothesis class. The upper and lower bounds differ only by a log factor in the cardinality of the hypothesis class, showing that (up to log factors), the Hellinger gap is the correct complexity measure of the kernel that characterizes the minimax rates. The authors prove these results by developing a novel reduction from noise online classification to hypothesis testing. Along the way, they provide guarantees for a new conditional version of Le Cam-Birge testing in the online setting.\n\nSTRENGTHS:\n- The paper is well-written and easy to follow\n- The problem of noisy online classification is well-motivated (but also see weakness below)\n- The technical contributions are interesting and the proof techniques are novel. In particular, I found the reduction to pairwise hypothesis testing to be nice\n- Overall, I think this paper makes a solid contribution to the literature on online classification\n\nWEAKNESSES:\n- More clarification is needed about the problem setup beyond just stating the minimax value. From what I can get, the adversary is oblivious and must pick the entire sequence of features, true labeling hypothesis, and noisy labels before the game begins. However, there seems to be some subtlety in the choice of how the adversary can pick the noisy labels which I feel requires explanation in the main text. In particular, the adversary can select noisy label distributions based on the realized values of the previous nosy labels. This contrasts from a weaker adversary which has to pick the entire sequence of noisy label distributions up front, and then samples the entire sequence of noisy labels. \n\n- Lack of motivation for generalization. While it is nice to provide results for a generalized noise mechanism, ideally there is some sort of justification for the generalization beyond theoretical curiosity (i.e why should I care). It would be nice if you can provide concrete example of real-world situations (i.e. specifying the instance space, label, space, noisy label space, kernel etc.) which are captured by your generalization, but not by previous noise models. As far as I can tell, applications to privacy, online denoting, and physical measurements are mentioned, but not expanded upon. Overall, I think it would be nice to provide more motivation behind why I should care about generalizing the noise mechanism. \n\n- Lack of universal lower bounds. While the upper bounds are stated for every (finite) hypothesis class, the lower bounds are of the form \"there exists a hypothesis class such that the minimax value is at least...\". It would nice to have universal lower bounds of the form \"for every hypothesis class, the minimax value is at least...\". \n\n- Finite Hypothesis Class. This paper currently only studies the minimax rates for finite hypothesis classes. There is a remark indicating that the results extend to infinite hypothesis classes via covering techniques from [1, 21], however neither the proof of this statement, nor the resulting minimax rates are provided. I am not super convinced that these existing covering techniques can give you bounds on the minimax value that are independent of the sizes of the true and noisy label spaces (like your existing bounds are). For example, for the simply hypothesis class $H = \\{ x \\mapsto a: a \\in [N] \\}$, to cover a single point $x \\in X$, you need the entire hypothesis class $H$.  It would be nice if you can give the upper and lower bounds on the minimax value for infinite hypothesis classes that your results imply along with a proof sketch. \n\nPlease see questions below regarding each of these weakness.\n\nQUESTIONS:\n- Is my interpretation of the minimax value correct? If so, I think it would be nice to make this explicit in text. \n- Can you give me some concrete learning problems for which your results provide new insights (i.e can you justify the generalization of the noise mechanism)?\n- Can you comment about universal lower bounds in your setting? \n- Can you comment on the generalization of your results to infinite hypothesis classes? Is it true that using your results with existing covering arguments would result in factors of N and M (the cardinalities of true and noisy label spaces) appearing in your bounds?\n- Right now, your bounds are written in terms of two separate complexity measures - one on the hypothesis class (i.e its cardinality) and one on the kernel (i.e. well-separated). Could there be a single complexity measure, a function of both the hypothesis class and kernel, that better captures the minimax value? If so, do you have any intuition on what this might be or what difficulties arise when trying to do so? \n- Your existing results assume that the kernel outputs sets of distributions that are convex and closed. It would be nice if you can make explicit where exactly this assumption is being used in the main text. In addition, what can be said if you are guaranteed that the kernel always outputs a finite set of distributions (say of size at most G)? Can you comment on how would the minimax rates would scale with G?\n\nLIMITATIONS:\nNo limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper generalized an agnostic online learning setting from [1]: Nature sequentially produces arbitrary feature vectors, and a noiseless label chosen from an arbitrary hypothesis within a given class. The learner observes the feature vector (exactly), and a noisy label chosen from a distribution chosen from a class of distributions, where the identity of the class depends on the noiseless label. The goal of the learner is to minimize the expected cumulative risk. The paper's main result (Theorem 2, also presented in a simplified form in Theorem 1) is characterization of the high probability (upper bound) and expected risk (lower bound) in terms of the Hellinger gap. The result follows from a proposed learning algorithm, which is based on pairwise testing. In addition, assuming a positive $L_2$ gap, tighter upper bounds are obtained w.r.t. the cardinality size of the hypothesis class (Theorem 5).\n\nSTRENGTHS:\n1. Theorem 2 captures the exact dependence on the Hellinger distance $\\gamma_H$, by providing upper and lower bounds whose dependence on that parameter is $\\Theta(1/\\gamma_H)$.\n\n2. An additional result based on $L_2$ distance, which is tight also in the log-cardinality of the hypothesis class. \n\n3. The introduction of the surrogate pairwise loss function and evaluating the loss by its maximum over all competing hypothesis, as well as the resulting reduction of online learning to binary hypothesis testing is interesting.\n\nWEAKNESSES:\nMy main issue with the paper is that given the separation condition and finite hypothesis class, the resulting learning task is “easy”:\n\n1. The assumption of a strictly positive Hellinger distance significantly eases the learning task. The immediate consequence of this is that the cumulative risk is bounded in $T$, and also hints the ease of the learner's task. As the algorithm also suggests, the problem is reduced to a composite binary hypothesis testing using $T$ samples, followed by static play of that decision. \n\n2. Given that the problem is interesting, it is worth to already address the infinite class setting, and noises that are not uniformly bounded. While the technical details *might* be routine, the dependency of the risk bounds should be interesting, and, in relation to the previous comment, might also lead to elaborated risk bounds, with interesting dependency on the function class (parametric/a-parametric) and the noise conditions.\n\nQUESTIONS:\n1. What is the conclusion from Example 1? From the paragraph that precedes it, it appears that it meant to justify either the semi-stochastic setting or the evaluation of the risk on the true labels, but it is not discussed how. From reading the rest of the paper, it appears that the analysis in this paper is a generalization of the result in Example 1, but this is not an intrinsic motivation. \n\n2. Proof of Theorem 2: To the best of my knowledge, such problem is called *composite* hypothesis testing (rather than robust), and this was extensively studied in the information-theoretic literature. In this regard, the error in hypothesis testing is related to the KL divergence, for which chain rule is much more suitable to distributions that are not in product form via the standard chain rule. My question is why can't the analysis be made in terms of the KL divergence ? \n\nFinally, regarding line 41, what is differential piracy ? Only attacking some ships ? :-)\n\nLIMITATIONS:\nThe paper properly discusses limitations throughout.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}