Title: Tradeoffs between Mistakes and ERM Oracle Calls in Online and Transductive Online Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online Learning

Transductive Online Learning

Oracles
Abstract: We study online and transductive online learning in settings where the learner can interact with the concept class only via Empirical Risk Minimization (ERM) or weak consistency oracles on arbitrary subsets of the instance domain. This contrasts with standard online models, where the learner has full knowledge of the concept class. The ERM oracle returns a hypothesis that minimizes the loss on a given subset, while the weak consistency oracle returns only a binary signal indicating whether the subset is realizable by a concept in the class. The learner's performance is measured by the number of mistakes and oracle calls.

In the standard online setting with ERM access, we establish tight lower bounds in both the realizable and agnostic cases: $\Omega(2^{d_\mathrm{LD}})$ mistakes and $\Omega(\sqrt{T 2^{d_\mathrm{LD}}})$ regret, respectively, where $T$ is the number of timesteps and $d_\mathrm{LD}$ is the Littlestone dimension of the class. We further show how existing results for online learning with ERM access translate to the setting with a weak consistency oracle, at the cost of increasing the number of oracle calls by $O(T)$.

We then consider the transductive online model, where the instance sequence is known in advance but labels are revealed sequentially. For general Littlestone classes, we show that the optimal mistake bound in the realizable case and in the agnostic case can be achieved using $O(T^{d_\mathrm{VC}+1})$ weak consistency oracle calls, where $d_\mathrm{VC}$ is the VC dimension of the class. 
On the negative side, we show that $\Omega(T)$ weak consistency queries are necessary for transductive online learnability, and that $\Omega(T)$ ERM queries are necessary to avoid exponential dependence on the Littlestone dimension.
%
Finally, for special families of concept classes, we demonstrate how to reduce the number of oracle calls using randomized algorithms while maintaining similar mistake bounds. In particular, for Thresholds on an unknown ordering, $O(\log T)$ ERM queries suffice, and for $k$-Intervals, $O(T^3 2^{2k})$ weak consistency queries suffice.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper continues the study of online learning (and transductive online learning) in restricted access models. While the problem of online is very well understood, most results are based on algorithms that require the knowledge of the hypothesis class and perform some intractable computations. This paper considers restricted setting where the algorithm has either access to an ERM oracle, a restricted ERM oracle that can only query the ERM on points seen so far, and a weak consistency oracle that only identifies if the sample is realizable or not.\n\nThe authors prove several interesting results in these settings. One of the results shows a strong exponential in terms of the Littleston dimesnion lower bound on the mistake of any learner that uses ERM oracles, which matches the known upper bound. Another interesting result is the connection between weak consistency oracles and restricted ERMs. They show that any algorithm that uses restricted ERMs can be simulated by an algorithm that uses weak consistency oracle with a multiplicative O(T) factor in the query complexity.\n\nIn the transductive setting, a significant result is showing that with $O(T^{d_{VC}}$ many oracle calls, it is possible to obtain the optimal mistake bound even in the weak consistency oracle setting. It is further showed that $O(T)$ oracle calls is not sufficient even for the ERM oracle. Moreover, mistake and oracle bounds are derived for specific classes of interest.\n\nSTRENGTHS AND WEAKNESSES:\nI guess the question of what is achievable in models where the computation is more restricted is an important question in learning theory and the authors seem to make significant steps in the online learning direction. I also believe showing that learning in such restricted models with respect to partial concepts is impossible is an interesting result, which makes point to the challenge of learning with respect to such complicated models that are hoped to capture to some extent the data assumptions. \n\nI think one important part, which is also mentioned in the paper, is that it is not clear how powerful are randomized algorithms and what mistake bounds is attainable if we are allowed polynomially (in $T$) many oracle calls, independent of VC dim. The only negative result in the paper shows insufficiency of linear number of oracle calls for the weak consistency oracle. If I am not mistaken, it is not even clear what happens to the randomized learners with ERM oracles for polynomially many oracle calls. I think this can be an important future direction for this\n\nQUESTIONS:\nIs it clear whether the doubly exponential calls for ERM oracle is necessary and is the oracle calls for randomized algorithms of the same order?\n\nIs it clear if the multiplicative $O(T)$ factor in oracle calls for reducing weak consistency model to restricted ERM model necessary? Moreover, it makes sense from the lower bound in the transductive setting that a lower bound for restricted ERM will imply a lower bound for the weak consistency model too. But in the online learning, there are only lower bounds for the restricted ERM model. I think that this should also imply some lower bounds for consistency model but it is not completely clear to me. The reason is that the weak consistency oracle can query the realizability of samples that are not already observed too.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies online and transductive learning in settings where the learner can only interact with the hypothesis class through ERM or optimization oracles of varying power. Statistically optimal algorithms in the online setting (such as SOA) and the transductive setting typically require direct knowledge of and access to the hypothesis class. This paper asks whether optimal regret/mistake bounds are achievable when the learner can only access the concept class using ERM or other types of optimization and consistency oracles. Foreshadowing that the optimal regret is too much to hope for, the paper aims to understand the tradeoff between the number of oracle calls and the regret/mistake bound broadly.\n\nFor the online setting, the paper shows that the mistake bound is lower bounded by $\\Omega(2^{LD})$ or $\\Omega(3^{LD})$ for randomized and deterministic algorithms, respectively, regardless of the number of oracle calls. In the agnostic case, the regret is lower bounded by $\\Omega(\\sqrt{T 2^{LD}})$. The authors also provide upper bounds that demonstrate exponential dependence on the Littlestone dimension (LD). For the transductive setting, they show that achieving optimal regret/mistake bounds is possible in $O(T^{VC})$ even using a weaker oracle model. While some bounds improve in this setting, it appears that even in the transductive setting, a no-regret algorithm still requires exponential dependence on LD unless a linear number of oracle calls is allowed.\nI find the paper overall interesting. The study of oracle efficiency is of long-standing interest to the COLT community --- which has a good presence within NeurIPS --- and is motivated by real concerns about the practical efficiency of learning algorithms. I find this to be a solid contribution for the theoretically minded NeurIPS audience and I lean towards acceptance.\n\nSTRENGTHS AND WEAKNESSES:\nThat being said, I would have felt much more inclined to recommending acceptance more strongly if the paper went the additional mile of 1) connecting with the broader literature a but better, 2) justifying and motivating the study of their work in such generality under such strong adversarial assumptions. Detailed comments below:\n\nI found it difficult to fully connect and compare the findings of this paper to the broader literature on oracle efficiency in online learning, e.g.:\n1. Hazan and Koren, The computational power of optimization in online learning, STOC 2016\n2. Kalai and Vempala, Efficient algorithms for online decision problems, JCSS 2005\n3. Dudík et al., Oracle-efficient Online Learning and Auction Design, FOCS 2017\n\nFor example, [1] provides upper and lower bounds on the number of oracle calls required for achieving vanishing regret, though the dependence on $\\mathrm{LD}$ or $\\mathrm{VC}$ isn't explicitly considered in their work. Are those results subsumed by yours, or do they highlight tradeoffs that are qualitatively different? The techniques shown in [2–3] do not apply to general VC or LD classes (as [1] already establishes), but given the relevance of your work to the oracle-efficient online learning literature, I would have expected a discussion of all of these works.\n\nThe paper would also benefit from a clearer motivation for studying this particular model at the generality of VC or LD classes. I appreciated the improved bounds for certain classes in the transductive setting, but my concern is not a lack of results, but rather a lack of context for paper broadly. There are some minimal attempts at this (e.g., lines 56 on computability issues) but I find the discussion severely lacking.\n\nSince this paper is being submitted to NeurIPS and not COLT, the authors should embrace the additional responsibility of communicating to a broader audience beyond the TCS community.\n\nThis is especially important given that in both theory and practice, a number of works have demonstrated that oracles are quite powerful in many practical online learning settings. For example, [2,3] demonstrate that oracle efficiency and no-regret guarantees can be achieved in settings that are broadly of interest to the community. Moreover, for general VC classes, oracles have also been shown to be quite powerful outside brittle counter examples. For instance:\n\n4. Haghtalab et al., Oracle-Efficient Online Learning for Beyond Worst-Case Adversaries, NeurIPS 2022\n5. Block et al., On the Performance of Empirical Risk Minimization with Smoothed Data, COLT 2024\n\nIn particular, [5] shows that repeated “restricted ERM” suffices for achieving no-regret in realizable online learning if the marginal distribution is smooth. Similarly, [4] studies the agnostic setting under smoothness and also shows that ERM oracle calls suffice for online learning of VC classes. This smoothness condition has come to be widely accepted by the community as a useful test for whether impossibility results against adversaries hold beyond brittle counter examples and demonstrate practical concerns.  Looking at your lower bounds, the constructions also demonstrate quite a lot of brittleness because they are based on requiring exact identification of randomly chosen real values which cannot be done with finite number of oracle calls.\n\nIn light of these results—and the broader NeurIPS audience—I would have liked to see a more rigorous discussion of why the fully adversarial setting in your model is worth studying and why should it be interesting to the broader NeurIPS community. What should be the takeaways from reading your paper in terms of practical tradeoffs in online environments?\n\n\nMinor:\n•\tLine 190: “a a concept” → “a concept”\n\nQUESTIONS:\nSee above two points on explicit comparison to Hazan and Koren and motivation given the broader context.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper investigates online and transductive online learning scenarios where the learner interacts with the concept class solely through ERM or weak consistency oracles. The authors establish tight lower bounds for the number of mistakes and oracle calls in both realizable and agnostic settings, demonstrating an exponential dependence on the Littlestone dimension. They also explore the transductive setting, where instance sequences are known in advance, and show that polynomial oracle calls can achieve optimal mistake bounds. Special cases like thresholds and k-intervals are analyzed, revealing improved performance with randomized algorithms.

STRENGTHS AND WEAKNESSES:
Strengths: 
1. The paper provides rigorous lower and upper bounds, clearly delineating the trade-offs between mistakes and oracle calls. The proofs are well-structured and detailed, with key insights highlighted.
2. The exploration of weak consistency oracles in online learning is novel, bridging gaps between PAC learning and online settings. The results for specific concept classes (e.g., thresholds, k-intervals) offer insights.

Weaknesses:
No specific one.

QUESTIONS:
The transductive setup is a good bypass for the hardness of online learning. Out of curiosity, have you considered other routes to bypass, e.g., i.i.d. assumptions on the covariates, margin assumptions for linear separators, and smoothed online learning?

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the number of mistakes and regret achievable when a learner can only interact with the concept class via Empirical Risk Minimization Oracle or weak consistency oracles. \nUnlike standard online models where the learner has full knowledge of the concept class, as here, the learner interacts with the concept class solely through various oracles. The main motivation comes from the computational intractability of implementing algorithms like the Standard Optimal Algorithm (SOA), which, despite being optimal, involves computing Littlestone dimension multiple times even though it is APX-hard. As a result they work with an ERM oracle which given a labeled dataset returns a concept class which minimizes the error on the dataset.  This oracle is known to be sufficient in PAC learning settings. They also consider a weak-consistancy oracle which only returns a binary signal whether the dataset is realizable by some concept in the class. \n\nTheir results are as follows:\nA $\\Omega(2^{d_L})$ mistake lower bound in the realizable setting. They also show a $\\Omega\\sqrt{T 2^{d_L}})$ on the regret in teh agnostic setting. Their result establishes that an exponential dependence on the little-stone dimension is essential.  The proof  for this lower bound involves constructing a concept class of random threshold functions over $T = 2^{d_L}$ points embedded in a high-dimensional space. The adversary carefully defines nested hyper-rectangles using random values that are unknown to the learner. Since the (agnostic) ERM oracle does not know the future nested regions  the learner is forced to make predictions with only 1/2 probability of being correct at each step. \n\nThey also show that any deterministic online learning algorithm that uses the restricted ERM oracle can be simulated using the weaker consistency oracle. The cost is an increase in the number of oracle calls by a factor of O(T), while maintaining the same mistake bounds. This implies that existing ERM-based algorithms can be adapted for the weak consistency setting.\n\nThey also show a number of results for transductive learning, which effectively shows that the negative results in the online setting can be circumvented here. They also show several new results for specific concept classes (all of which are  summarized in Table 2). Unfortunately, I am not as familiar with this part of the literature and am not sure I can judge their novelty.\n\nSTRENGTHS AND WEAKNESSES:\nI feel the authors establish an important lowerbound for ERM oracles in online learning. This is in line with recent works that have tried to understand better algorithms and oracles that circumvent known hardness results. I found the paper to be well written and the results clear. The proofs are mostly in the appendix but they seemed correct.\n\nI am not an expert in the area and have little background knowledge. Nonetheless, I think the paper would be of interest to the NeurIPS community.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-17"}