Title: Non-Uniform Multiclass Learning with Bandit Feedback
Venue: NeurIPS 2025
Accepted: yes
Keywords: Non-Uniform Learning

Bandit Feedback

Multiclass Learning
Abstract: We study the problem of multiclass learning with bandit feedback in both the i.i.d. batch and adversarial online models. In the *uniform* learning framework, it is well known that no hypothesis class $\mathcal{H}$ is learnable in either model when the effective number of labels is unbounded. In contrast, within the *universal* learning framework, recent works by (Hanneke et al., 2025b) and (Hanneke et al., 2025a) have established surprising exact equivalences between learnability under bandit feedback and full supervision in both the i.i.d. batch and adversarial online models, respectively. This raises a natural question: What happens in the *non-uniform* learning framework, which lies between the uniform and universal learning frameworks? Our contributions are twofold: (1) We provide a combinatorial characterization of learnable hypothesis classes in both models, in the realizable and agnostic settings, within the non-uniform learning framework. Notably, this includes elementary and natural hypothesis classes, such as a countably infinite collection of constant functions over some domain that is learnable in both models. (2) We construct a hypothesis class that is non-uniformly learnable under full supervision in the adversarial online model (and thus also in the i.i.d. batch model), but not non-uniformly learnable under bandit feedback in the i.i.d. batch model (and thus also not in the adversarial online model). This serves as our main novel technical contribution that reveals a fundamental distinction between the non-uniform and universal learning frameworks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nStudies the non-uniform model for multiclass classification with unbounded number of labels under bandit feedback. Considers both PAC and online learning settings. Provides a characterization of learnability in realizable/agnostic versions of PAC and online settings. Further unlike universal learning, there is a separation between bandit feedback and full feedback in the non-uniform setting. The non-uniform setting is in between uniform and universal learning settings. In uniform setting, sample complexity/regret should only depend on the hypothesis class. In non-uniform, it can depend on hypothesis as well. In universal learning, it can depend on hypothesis and distribution/individual sequence as well.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nConsider a very natural question that was left unaddressed by previous work. Last few years have seen a lot of work on unbounded label spaces and bandit feedback. Makes the paper timely.\n\nWeakness:\nNovelty is very limited. The characterization is exactly what one would expect. In standard online or PAC learning, we know that non-uniformly learnable classes are exactly those that are countable unions of uniformly learnable classes. This is exactly what the authors show here too: countable unions of classes that are learnable uniformly. While the paper fills a gap technically speaking, the answer and the methodology used to provide the answer does not provide sufficiently novel insights into learnability in general. As a result, the paper is not appropriate for a broad audience conference like NeurIPS. It might be more suitable for a theory focused conference. This is especially so since the focus on unbounded label spaces and the subtle differences between uniform and non-uniform learnability, while fully justified from a theoretical point of view, are of limited relevance to the broad ML community.\n\nQUESTIONS:\nWhat is truly novel in your algorithms and proofs?\nWhy should the broader ML community be interested in these result as opposed to a more specialized theory-focused conference?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider multiclass classification in the non-uniform learning framework. They establish structural results showing that in contrast to the uniform and universal learning framework, the non-uniform learning framework exhibits a separation between learning under full supervision and learning with bandit feedback; that is, when the learner only observes whether or not the prediction is correct. In more detail, the authors characterize the hypothesis classes which are non-uniformly learnable under bandit feedback in both the online and PAC settings by presenting them as countable unions of classes with bounded dimensions which are known to characterize learnability in the corresponding uniform framework. The authors also exhibit a strict separation between bandit feedback and full supervision in the non-uniform framework by exhibiting a specific hypothesis class which is learnable under full supervision (even in the uniform learning framework) but not learnable with bandit feedback in the non-uniform framework.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* This work proves several open questions in the literature of learnability under bandit feedback, of which previous works considered only the uniform and the universal learning frameworks. As the non-uniform learning framework is well-studied in the learning theory literature, studying the role of bandit feedback in this framework seems natural.\n* The results are presented clearly and are easy to understand.\n* The fact that in the non-uniform learning framework there is a strict separation between full supervision and bandit feedback is surprising given that such a separation does not exist in the uniform and the universal learning frameworks.\n\nWeaknesses:\n\n* From a technical perspective, the results established in this work are obtained in a fairly straightforward manner from existing results in the learning theory literature. For example, showing the equivalence of non-uniform learnability under bandit feedback with the structural property of being a countable union of classes with finite BL-dim seems quite analogous to the corresponding result from binary classification - where a hypothesis class is non-uniformly learnable if and only if it is a countable union of classes with finite VC dimension. The structural properties presented in this paper seem to follow very similar concepts.\n\nQUESTIONS:\nI would appreciate it if the authors could elaborate on the technical novelties of their results, specifically, what additional techniques are needed which are not present in more classical results relating uniform and non-uniform learnability. As it stands, it seems to me like a quite general statement could be proven: A class $H$ is non-uniformly learnable in the X setting if and only if $H$ is a countable union of classes which are uniformly learnable in the X setting (in which case there is most likely a known dimension characterizing such classes). Is my intuition correct? Do the authors believe such a general statement could be proven formally?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies learnability under uniform, non-uniform and universal learning frameworks, as recently investigated by Hanneke et al 2025a,b. Among the contributions: (i) the paper gives a combinatorial characterization in the non-uniform learning framework, and (ii) the authors are able to construct a hypothesis class that is non-uniformly learnable under full supervision in the adversarial online model but not non-uniformly learnable under bandit feedback, revealing a separation between the non-uniform and universal learning settings (in the universal setting we have an exact equivalences between multiclass learnability under bandit feedback and full feedback).\n\nSTRENGTHS AND WEAKNESSES:\nOn the positive side, I praise the authors' efforts to try and explain the frameworks and the scope of their results.\n\nOn the negative side, I'm not sure how compelling or widely interesting such results are (in general), and in particular for the Neurips audience. In this respect, I am not buying the authors' motivations in L. 271 of the paper: \"in multiclass settings, it is preferable for guarantees to remain independent of the number of labels, even when finite. [...] mathematical frameworks involving infinity mostly provide clearer insights.\" I don't see practical motivations behind such a line of research nor do the authors attempt to provide any.\n\nPresentation-wise, the paper leaves a lot to be desired. It is a bit awkward that, up until page 6, there is not a clear explanation of the specifics of the learning frameworks the authors are alluding to. Though I am very much aware of the basics, I am not familiar with the difference between non-uniform and universal, and I don't see any clear explanation of that in the paper, despite the lengthy (and quite redundant) introductory sections. Moreover, as I go over Sect. 4 (e.g., Lemma 4.1) I stumble into undefined concepts (like ``effective label space\"), that the authors seem to take for granted.\n\nMinor points:\n-  l. 58, \"addressing\" --> \"address\" ?\n- L. 106 \"unless\": is it \"until\" rather than \"unless\" ?\n- L. 150 \"bandit feedback setting's feedback\": how about just \"bandit feedback\" ? \n- L. 159-160: upper bound and lower bound: which upper and lower bound, exactly ?\n- L. 164 \"bounded littlestone dimension\": this comes out of the blue, frankly, at this point ...\n- L. 199: what is C ?\n- L. 203 \"finite effective label space\": same issue, it comes out of the blue in the flow of the presentation\n- L. 307: I assume the history also contains $X_t$ ?\n- L. 341 \"we need the following a result\" (typo). Btw, what is ``the following result\" you are alluding to here ? Is it Thm B.1 from Daniely et al. 2011 ? If so, you cannot write things this way... you should recall the content of the theorem on the spot.\n\nQUESTIONS:\n- What is the universal learning setting ?\n- What is the effective label space ?\n- Please articulate the applicability of your setting to real-world problems. E.g., why is it more appealing to rely on an algorithm that also works on infinite label spaces when the label space is finite but large, if we only have bandit feedback available.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies when multiclass hypothesis classes remain learnable from bandit-only feedback in the “non-uniform” setting that sits between classic PAC learning and universal learning. Learnability in the i.i.d. (PAC) regime occurs iff the class can be decomposed into countably many subclasses, each with finite effective label space and finite Natarajan dimension. In the adversarial online regime, learnability occurs iff the class is a countable union of subclasses with finite bandit Littlestone dimension. To match these information-theoretic conditions, the authors present a generic Bandit-SOA algorithm as well as separation examples, showing the non-uniform and universal frameworks fundamentally diverge.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The paper proves tight if-and-only-if characterizations for non-uniform multiclass learning under bandit feedback in both PAC and online settings (Theorems 1.1 - 1.3). Proofs are mostly self-contained and appear to build correctly on previous lines of work.\n2. Resolves an open question left by Hanneke et al. (2025). The negative separation example (Prop 1.4) shows a fundamental distinction between bandit learning and full supervised learning in the non-uniform framework.\n3. Extends non-uniform learnability from binary to multiclass and bandit feedback, which (to the best of my knowledge) has not been done; the combinatorial conditions seem new.\n\nWeakness:\n1. Practical relevance is indirect: all algorithms rely on full knowledge of countable decompositions and invoke BSOA sub-routines with exponential overhead. The paper does not discuss computational complexity or whether efficient learners exist even for finite-label subclasses.\n2. It seems the generic “countable union + mistake/sample budget allocator” technique is adapted from Benedek and Itai (1988); novelty mainly lies in multiclass extension. Would like to see more clarity on what new proof ideas are required beyond existing reductions.\n\nQUESTIONS:\nAre the non-uniform learners implementable in polynomial time for finite-label classes with small BLdim/Natarajan dim? If not, is there any hope of an efficient surrogate?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies multiclass prediction when only bandit feedback is available, and the number of possible labels may be infinite. Under the uniform framework no class is learnable once labels are unbounded, and prior work show that under the universal framework bandit feedback is as powerful as full supervision. This work studies the non-uniform setting, and relate to learnability to Natarajan dimension in the PAC setting and bandit Littlestone dimension in the online learning setting.\n\nSTRENGTHS AND WEAKNESSES:\nAs someone not closely following this line of work, the result in this paper seemed interesting and substantial to me. In particular, they give a nice exact characterization of the learnability in the non-uniform setting. The theoretical results seem very clean. \n\nWhile the results are new, the proofs are relatively simple, and does not seem to lead to new techniques or insights. Hence, I am not quite sure how surprising the results should be considered. To me, the results feel more or less direct analogs to some of the classical results in non-uniform learnability, for example, Chapter 7 in the book [1]. As such, I think some of the classical results in non-uniform learning should be mentioned somewhere, and a comparison should be made. \n\nThe paper is structured oddly in places. For example, it seems weird that the definition for various complexity measures only appears in the appendix. This makes reading the paper somewhat difficult. \n\nIn the introduction, a table summarizing and juxtaposing the three frameworks and learnability criteria, and highlighting some key examples, could be helpful. A more concrete illustration and examples around the key definitions and theorems would also be helpful. \n\n[1] Understanding machine learning, Shalev-Shwartz and Ben-David. \n\nOther comments:\n- Apperances of section X should be capitalized (Section X).\n- Section 5 have just one subsection and nothing else. If only one subsection, just present as a section.\n\nQUESTIONS:\nThe algorithm based on BSOA is used to learn non-uniformly under bandit feedback in the online setting. How does the algorithm perform in terms of computational complexity? \n\nCan the results be generalized to other forms of feedback?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}