Title: Agnostic Continuous-Time Online Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: continuous-time online learning

query efficiency

adaptive adversaries

importance weighting

Littlestone dimension
Abstract: We study agnostic online learning from continuous-time data streams, a setting that naturally arises in applications such as environmental monitoring, personalized recommendation, and high-frequency trading. Unlike classical discrete-time models, learners in this setting must interact with a continually evolving data stream while making queries and updating models only at sparse, strategically selected times. We develop a general theoretical framework for learning from both *oblivious* and *adaptive* data streams, which may be noisy and non-stationary. For oblivious streams, we present a black-box reduction to classical online learning that yields a regret bound of $T \cdot R(S)/S$ for any class with discrete-time regret $R(S)$, where $T$ is the time horizon and $S$ is the *query budget*. For adaptive streams, which can evolve in response to learner actions, we design a dynamic query strategy in conjunction with a novel importance weighting scheme that enables unbiased loss estimation. In particular, for hypothesis class $\mathcal{H}$ with a finite Littlestone dimension, we establish a tight regret bound of $\tilde{\Theta}(T \cdot \sqrt{\mathsf{Ldim}(\mathcal{H})/S})$ that holds in both settings. Our results provide the first *quantitative* characterization of agnostic learning in continuous-time online environments with limited interaction.

Agnostic Continuous-Time Online Learning

Abstract

We study agnostic online learning from continuous-time data streams, a setting that naturally arises in applications such as environmental monitoring, personalized recommendation, and high-frequency trading. Unlike classical discrete-time models, learners in this setting must interact with a continually evolving data stream while making queries and updating models only at sparse, strategically selected times. We develop a general theoretical framework for learning from both oblivious and adaptive data streams, which may be noisy and non-stationary. For oblivious streams, we present a black-box reduction to classical online learning that yields a regret bound of T · R ( S ) /S for any class with discrete-time regret R ( S ) , where T is the time horizon and S is the query budget . For adaptive streams, which can evolve in response to learner actions, we design a dynamic query strategy in conjunction with a novel importance weighting scheme that enables unbiased loss estimation. In particular, for hypothesis class H with a finite Littlestone dimension, we establish a tight regret bound of ˜ Θ( T · √ Ldim ( H ) /S ) that holds in both settings. Our results provide the first quantitative characterization of agnostic learning in continuous-time online environments with limited interaction.

1 Introduction

Online learning from continuous data streams is encountered in many real-world applications, such as environmental sensor networks that sample air quality, recommendation engines that adapt as users click in real time, and high-frequency trading platforms that react rapidly to market transients. Unlike batch or discrete-time online settings, the underlying system state evolves continually through time. Consequently, learning algorithms must query the system to construct models of the data streams, which may be noisy and non-stationary. Moreover, due to the high cost of retraining online models, both the query and update budgets are often limited.

Given the widespread applications of continuous-time online learning, our objective is to establish its foundations from a learning-theoretic perspective. We are interested in questions such as: How frequently must a learner sample from a continuous stream to maintain accurate predictions? What happens when the stream itself adapts in response to past queries: Can effective learning still take place? And how can we provide strong performance guarantees when no perfect predictor exists?

Our approach to this problem is related to the update-and-deploy framework developed by Devulapalli and Hanneke [7] who recently introduced the setting of continuous-time online learning. Here, the query strategy and the learning algorithm are treated as distinct entities: the query strategy selects continuous-time query points without access to labels, while the learning algorithm updates its parameters only at these discrete query times. During each interval between queries, the predictor remains fixed and processes all incoming data. At each query instant, the learning algorithm observes

the label, updates the predictor, and redeploys it for the next interval. We note that the theory of continuous-time online learning developed by Devulapalli and Hanneke [7] primarily addresses the realizable setting, in which the data stream is assumed to be generated by a function in a given hypothesis class H . However, this assumption may not hold in real-world scenarios, where data is often noisy or misspecified.

In this paper, we tackle the more challenging agnostic setting [1], where the data stream may be selected arbitrarily and may not align with any function in the hypothesis class. Without assuming the existence of a perfectly consistent hypothesis, our primary objective becomes one of minimizing an appropriate notion of regret . Regret measures how much more loss the learner incurs compared to the best fixed predictor from a given hypothesis in hindsight. However, we should emphasize that it is not clear a priori how regret should be defined in the continuous-time setting. We introduce two notions of regret, depending on how the data stream is generated, namely, oblivious and adaptive regrets.

Oblivious Data Streams. An oblivious data stream in our context is a continuous stream that is arbitrarily chosen in advance before the learning process begins and does not depend on the learner's actions or query times. In this model, regret can be simply defined as the gap between the expected loss incurred by the algorithm when processing the data stream and the minimal expected loss incurred by the best fixed predictor in the hypothesis class on the same data stream.

Adaptive Data Streams. A more challenging setting arises when the data stream is adaptive ; that is, it may change in response to the learner's past queries and deployed predictors. This can occur, for example, when a query made by the learner perturbs the environment. One can model an adaptive data stream as being controlled by an adversary: initially, the adversary commits to a realization of a continuous data stream, but after each query, it may select a new process based on the learner's query times and predictor history. In this case, we define regret as the difference between the expected loss of the algorithm and the expected loss of the best fixed predictor in the hypothesis class, evaluated on the same data stream generated jointly by the adversary and the learning algorithm. This notion is analogous to regret formulations used in the bandit learning literature [4].

1.1 Main Contributions

We develop generic algorithmic frameworks for agnostic continuous-time online learning under both oblivious and adaptive data streams with optimal regret guarantees. We establish fundamental upper and lower bounds on regret that reveal precise trade-offs between query budget S , time horizon T , and complexity of the hypothesis class H . Together, these results establish a comprehensive theoretical foundation for agnostic continuous-time online learning.

Our main results can be summarized as follows:

We show that the regret under oblivious data stream can be reduced to classical discrete-time online regret via a black-box reduction. This is achieved by partitioning the time horizon into S epochs, each of length ∆ := T/S , and sampling one query uniformly within each epoch. Our main technical insight is that the loss incurred by any predictor on the queried example forms an unbiased estimate of the continuous-time risk over the entire epoch. We prove that for any hypothesis class admitting a classical discrete-time regret bound R ( S ) , the corresponding continuous-time regret scales as T · R ( S ) /S ; see Theorem 1.

For adaptive processes, we show in Proposition 1 that a simple uniform query strategy with fixed epochs does not admit sublinear regret. This was resolved by employing a query strategy that samples from dynamic (random) epochs as in [7]. However, such a strategy no longer yields an unbiased estimate due to the complex randomness of epoch boundaries. Our main technical innovation is a novel weighting scheme that transforms the queried loss into an unbiased estimate, even in the presence of random epochs-an idea we believe is of independent interest. More importantly, our weighting scheme is realized algorithmically (cf. Algorithm 3), not just in the analysis. Equipped with these techniques, we show in Theorem 2 that adaptive continuous-time regret for any class H of finite Littlestone dimension scales as ˜ O ( T · √ Ldim ( H ) /S ) .

Finally, we provide lower bounds that match our upper bounds. For finite Littlestone dimensional class H , we show in Theorem 3 that a lower bound of Ω( T · √ Ldim ( H ) /S ) is necessary for any algorithm, under both oblivious and adaptive data streams. This matches our upper bounds in all relevant parameters: T , S , and Ldim ( H ) . Notably, it demonstrates that even though the optimal

query strategies differ between oblivious and adaptive settings, their regrets are of the same order. Our proof follows from novel constructions of hard data processes; see Appendix D.

1.2 Related Work

Online learning in discrete time has been extensively studied in both the realizable and agnostic settings [12, 4, 16]. A foundational characterization of agnostic learnability for binary-valued hypothesis classes was established by Ben-David et al. [1] via the Littlestone dimension and was subsequently extended to multiclass and real-valued prediction settings [14, 5, 8]. The study of queryefficient and selective sampling models-where the learner decides when to observe labels -has provided deep insights into label complexity and active learning trade-offs in both online and batch frameworks [2, 6, 9]. Devulapalli and Hanneke [7] recently initiated the study of continuoustime online learning where they designed the update-and-deploy framework which is a protocol governing how a learning algorithm interacts with a continuous data stream. This setting departs from classical models by requiring the learner to query both features and labels in a continuous-time environment. Although their work establishes a theory for the realizable setting, the agnostic case, particularly its fundamental limits-remains largely unexplored. We address this gap by providing the first quantitative characterization of regret in the continuous-time agnostic setting. Our approach introduces several technical innovations, including importance-weighted unbiased loss estimators and the novel applications of Khintchine-type inequalities to derive sharp lower bounds in both the oblivious and adaptive settings. Additionally, our setting is related to problems in nonparametric filtering and online learning with delayed or partial feedback [13, 4].

2 Problem Setup

Let X be an instance space, Y be a label space, and H ⊂ Y X be a hypothesis class. A continuous-time data process over X × Y is defined as a collection of joint random variables Z := { ( X t , Y t ) } t ∈ R , where Z t := ( X t , Y t ) ∈ X × Y denotes the random sample at time t ∈ R .

We consider the update-and-deploy framework from Devulapalli and Hanneke [7] as described in Protocol 1. The learner begins with an initial predictor ˆ h 0 ∈ Y X and selects the first query time t 1 ∈ R + . The predictor ˆ h 0 is used to make predictions on instances X t for all t < t 1 , without revealing the corresponding sample ( X t , Y t ) to the learner. At query time t 1 , the environment (or Nature) reveals the labeled example ( X t 1 , Y t 1 ) , after which the learner updates the predictor to ˆ h 1 and selects the next query time t 2 . This process repeats until a fixed query budget S ∈ N + is exhausted.

Protocol 1 Continuous-Time Online Learning Protocol

1: Select initial predictor ˆ h 0 ∈ Y X and initial query time t 1 ∈ R +

2: for s = 1 , . . . , S do

3: Use ˆ h s -1 to make predictions on X t for t ∈ [ t s -1 , t s )

4: Query and observe Z t s := ( X t s , Y t s ) at time t s

5: Choose predictor ˆ h s ∈ Y X and the next query time t s +1 based on history { Z t 1 , . . . , Z t s }

6: end for

Note that the selection of ˆ h s and t s can be randomized 2 , and that the predictor ˆ h s need not belong to H (i.e., is improper). For clarity of exposition, we also assume that the learning process occurs within a finite time horizon T ∈ R + . In this case, we also assume t 0 = 0 and t S = T by convention.

Let A be any continuous-time online learning algorithm, and let Z := { ( X t , Y t ) } t ∈ R denote any data process that may depend on A . We define the risk of the algorithm as:

where ℓ : Y × Y → [0 , 1] is a bounded loss function, and the expectation is taken over the randomness in Z and the internal randomness of A . For simplicity, we assume that the function

f ( t ) := ℓ ( ˆ h s -1 ( X t ) , Y t ) is measurable for any sample path. Similarly, for any (static) hypothesis h ∈ H , we define the risk T ( h, Z ) by replacing the ˆ h s -1 with h in (1).

For any algorithm A , hypothesis class H and data process Z , regret is defined as:

Note that our definition of regret in (2) depends on the data process. To remove this dependency and to quantify the impact of the structural properties of H on regret, we introduce the following two notions of minimax regret for the worst-case data processes.

Oblivious Minimax Regret. In this setting, we assume that the data process Z is selected arbitrarily but independent (i.e., oblivious) of the internal randomness of A . The oblivious minimax regret for a hypothesis class H is then defined as:

where the infimum is taken over all learning algorithms A that make at most S queries and the supremum is taken over all data-generating processes Z that may depend on the algorithm's structure but not on its internal random choices (such as the realized ˆ h s 's and t s 's).

Adaptive Minimax Regret. A more challenging setting arises when the data process Z can adapt to the learner's behavior. In this case, Nature may select an initial process Z (1) and, after observing the s 'th query at time t s , switch to a new process Z ( s +1) based on the full interaction history up to that point (i.e., the realized { ( t i , ˆ h i ) } i ≤ s but not the next query time t s +1 ).

Let A be a learning algorithm and Φ be an adaptive strategy for Nature, we define the adaptive risk

where the expectation is over the joint distribution of { ˆ h s -1 } s ∈ [ S ] , { t s } s ∈ [ S ] and { Z ( s ) } s ∈ [ S ] induced by A and Φ , and we denote Z ( s ) := { ( X ( s ) t , Y ( s ) t ) } t ∈ R . We can also define the adaptive risk for any (static) hypothesis h ∈ H by replacing the ˆ h s -1 with h in (4), denoted by: risk a T ( h, A , Φ) .

Then, the adaptive minimax regret is defined as:

where A runs over all algorithms with ≤ S queries and Φ runs over all adaptive strategies for Nature.

3 Main Results

We start with a simple query strategy shown in Algorithm 1 for oblivious data processes. The core idea is to partition the time horizon into S epochs, each of length ∆ := T/S . Within each epoch, we select a query time uniformly at random and observe a sample. These queried samples are then fed into a classical online learning algorithm, which produces the predictor ˆ h s used for the next epoch. 3 .

The rationale behind this approach is that since the sample is uniformly queried from an interval, the loss incurred by ˆ h s -1 on the queried sample is an unbiased estimate of the actual (continuous) risk for that interval. Therefore, any regret guarantee achieved by the expert algorithm B on the queried samples can be translated to the regret of our continuous-time online algorithm (Algorithm 1) in a black-box fashion. This observation is formally stated in the following lemma:

Algorithm 1 Uniform Random Query with Fixed Epochs

1: Partition the time horizon into blocks of length ∆ := T/S and select ˆ h 0 arbitrarily

2: for s = 1 , 2 , . . . , S do

3: Deploy predictor ˆ h s -1 for the entire epoch [( s -1)∆ , s ∆)

4: Sample t s uniformly from [( s -1)∆ , s ∆) .

5: Query and observe Z t s := ( X t s , Y t s ) at step t s .

6: Use any expert algorithm B to produce ˆ h s with data { Z t 1 , · · · , Z t s }

7: end for

Lemma 1. Let Z be any (oblivious) data process, and { ( t s , ˆ h s -1 , Z t s ) } s ∈ [ S ] be the (random) selections generated by Algorithm 1. Then, for any s ∈ [ S ] and any (random) predictor h s -1 ∈ Y X that depends only on { ( t i , ˆ h i -1 , Z t i ) } i ∈ [ s -1] , we have:

where the expectation is over all randomness involved.

Proof. By the law of total probability, we have

where we used the fact that conditioning on { ( t i , ˆ h i -1 ) } i ∈ [ s -1] and Z , the index t s is independent of h s -1 and is uniform over [( s -1)∆ , s ∆) (we then express the expectation on t s as an integral).

Note that Lemma 1 holds for any predictor h s -1 that satisfies the stated condition (including a static predictor), not just for the produced predictor ˆ h s -1 . We now introduce the classic regret as:

where the expectation is over the internal randomness of B that produces ˆ h s -1 .

Theorem 1. Let H be any hypothesis class that admits an expert algorithm B with ˜ regret S ( B , H ) ≤ R ( S ) for some function R : N → R . Then, the oblivious minimax regret of H satisfies

Moreover, this upper bound is achieved by Algorithm 1 using the expert algorithm B .

Proof. Let Z be any fixed data process, and { t s , ˆ h s -1 , Z t s } s ∈ [ S ] be the selections made by Algorithm 1 using expert algorithm B . Invoking Lemma 1 and the linearity of expectation, we have

where the expectation is over all randomness involved. Similarly, for any (static) h ∈ H , we have risk T ( h, Z ) = ∆ · E [ ∑ S s =1 ℓ ( h ( X t s ) , Y t s ) ] . Therefore,

where the first inequality follows by inf E ≥ E inf and linearity of expectation. This completes the proof, and the final identity follows by ∆ := T/S .

Instantiating Theorem 1 to specific hypothesis classes, we have:

Corollary 1. Let Y = [0 , 1] and H ⊂ { 0 , 1 } X be a binary-valued class of Littlestone dimension Ldim ( H ) . Then, for absolute loss ℓ ( y, y ′ ) := | y -y ′ | , the oblivious minimax regret of H satisfies

Proof. By [1], class H admits an expert algorithm B with classic regret upper bounded by R ( S ) ≤ O ( √ S log S · Ldim ( H )) under the expected misclassification loss. The result then follows from Theorem 1 by plugging in R ( S ) and noting that the expected misclassification loss can be interpreted as the absolute loss [4, Chapter 8].

Observe that Corollary 1 provides the precise trade-off between the query budget S , time horizon T and the resulting regret. For instance, it shows that if the query budget grows linearly as S = Ω( T ) , then the regret is upper bounded by ˜ O ( √ T · Ldim ( H )) . On the other hand, if the budget is sublinear as T α for some α < 1 , then the regret grows as ˜ O ( T 1 -α 2 · √ Ldim ( H )) . We will show in Section 3.2 that this regret is tight w.r.t. all parameters T , S and Ldim ( H ) .

Remark 1. Note that Theorem 1 provides a black-box reduction from any achievable classical discrete-time online regret to the (oblivious) continuous-time regret. For instance, for real-valued class H ⊂ [0 , 1] X with sequential-fat-shattering dimension of order α -p , we have R ( S ) ≤ ˜ O ( S p -1 p ) under absolute loss [14]. This leads to the continuous-time risk: regret o T ( H , S ) ≤ ˜ O ( T · S -1 /p ) . Furthermore, Algorithm 1 is computationally efficient, provided that the expert algorithm B is efficient (such as the oracle-efficient algorithms in [11, 3, 17]).

3.1 Adaptive Minimax Regret

One might observe that the key ingredient in the proof of Lemma 1 is the independence of the query time t s from the data process Z . This independence allows us to obtain an unbiased estimate of the continuous-time risk over the epoch [( s -1)∆ , s ∆) , even if only a single sample is queried and the process is selected in a completely arbitrary manner.

Unfortunately, this property does not carry over to an adaptively selected data process that may depend on the query times. This is demonstrated formally in the following proposition.

̸

Proposition 1. For any hypothesis class H ⊂ { 0 , 1 } X that contains two h 1 , h 2 ∈ H and x 1 , x 2 ∈ X such that h 1 ( x 1 ) = h 2 ( x 1 ) but h 1 ( x 2 ) = h 2 ( x 2 ) , we can construct an adaptive process, such that Algorithm 1 with any B and S incurs regret lower bounded by Ω( T ) under absolute loss.

Proof. Let B be uniformly sampled from { 0 , 1 } . At each epoch s , we set X t := x 1 , Y t := h 1 ( x 1 ) for t ∈ [( s -1)∆ , t s ] , and switch to X t := x 2 , Y t := B for t ∈ ( t s , s ∆) . Note that this data process depends on the (random) query times t s 's and is realizable w.r.t. H ; that is, either h 1 or h 2 incurs 0 risk. Since the queried example is always ( x 1 , h 1 ( x 1 )) , which is independent of B . Therefore, the expected risk is lower bounded by the risk of ˆ h s -1 's on the intervals ( t s , s ∆) as:

̸

̸

where the penultimate identity follows by E [( s ∆ -t s )] = ∆ 2 . Therefore, there must exist some B ∈ { 0 , 1 } such that Algorithm 1 incurs T/ 4 regret for our adaptive process constructed above.

Proposition 1 implies that for any class H ⊂ { 0 , 1 } X with Littlestone dimension at least 2 (which satisfies the stated condition), Algorithm 1 incurs adaptive regret ≥ T 4 . This effectively rules out almost all hypothesis classes of interest. Note that the main reason our adaptive process breaks the arguments in the proof of Theorem 1 is that the loss at the query example t s is no longer an unbiased estimate of the continuous-time risk over the entire epoch [( s -1)∆ , s ∆) .

To address this issue, we employ a query strategy adapted from [7], by sampling the query time t s uniformly from the dynamic epoch [ t s -1 , t s -1 +∆) , as described in Algorithm 2. However, since the data process Z ( s +1) after the query time t s may change under an adaptive adversary, an analog of Lemma 1 no longer holds. Our main idea is a novel weighting scheme on the queried losses (see Algorithm 3), which serves as a substitute unbiased estimator for the continuous-time risks. This is formalized in Lemma 2 below and constitutes the main technical innovation of this section.

Algorithm 2 Uniform Random Query with Dynamic Epochs

1: Select the initial predictor ˆ h 0 ∈ Y X arbitrarily and sample t 1 uniform over [0 , ∆) .

2: for s = 1 , 2 , . . . , S do

3: Use ˆ h s -1 to make predictions on X t for t ∈ [ t s -1 , t s )

4: Query and observe Z ( s ) t s := ( X ( s ) t s , Y ( s ) t s ) at time t s

5: Use Weighted-EWA (Algorithm 3) to produce ˆ h s with data { ( t i , Z ( i ) t i ) } i ≤ s

6: Sample t s +1 uniformly from the dynamic epoch [ t s , t s +∆)

7: end for

Algorithm 3 Weighted-EWA

1: Let H := { h 1 , · · · , h k } be finite of size K

2: Set initial weight w 0 = (1 , · · · , 1) ∈ [0 , 1] K and the learning rate η = √ log K S

3: for s = 1 , 2 , . . . , S do

4: Retrieve data ( t s , Z ( s ) t s )

5: For all k ∈ [ K ] , update weight

6: Produce predictor

7: end for

Lemma 2. Let Φ be any adaptive strategy and { ˆ h s -1 , t s , Z ( s ) } s ∈ [ S ] be the realizations generated by interactions between Algorithm 2 and Φ (see Section 2). Then, for any s ∈ [ S ] and any (random) predictor h s -1 ∈ Y X that depends only on { ˆ h i -1 , t i , Z ( i ) } i ≤ s -1 , we have:

where the expectation is over all randomness involved.

Proof. Conditioning on any realizations of { ˆ h i -1 , t i , Z ( i ) } i ≤ s -1 , h s -1 and Z ( s ) , the conditional distribution of t s remains uniform over [ t s -1 , t s -1 +∆) . Denote f ( t ) := ℓ ( h s -1 ( X ( s ) t ) , Y ( s ) t ) and τ = t s -t s -1 for notation convenience. We have

where the key identity ( ⋆ ) follows by exchanging the order of integrations, and ( ⋆⋆ ) follows by setting t := t s since t s is distributed uniformly over [ t s -1 , t s -1 +∆) . The lemma then follows by taking the expectation over the remaining randomness and by the law of total probability.

Intuitively, Lemma 2 shows that the continuous-time risk of predictor h s -1 over the (random) interval [ t s -1 , t s ) can be estimated by a weighted loss of h s -1 at the queried sample Z ( s ) t s . In particular, the weighting factor depends on the query times and discounts those that are far from the previous query time. It is also instructive to observe that the expected loss at the query sample without weighting always overestimates the continuous-time risk.

The following lemma provides a useful bound on the discrete-time (weighted) regret incurred by Algorithm 3 at the query samples. The proof follows from the same regret analysis of the classical EWA algorithm [4, Theorem 2.2] and is deferred to Appendix B for readability.

Lemma 3. Let H ⊂ [0 , 1] X be any finite class and ℓ : [0 , 1] × [0 , 1] → [0 , 1] be any loss that is convex in the first argument. Then, Algorithm 3 yields the following regret:

where α s := ( 1 -t s -t s -1 ∆ ) ∈ [0 , 1] and { ( t s , Z ( s ) t s ) } s ∈ [ S ] run over all possible selections.

Note that the regret guarantee of Lemma 3 holds for any selection of the data { ( t s , Z ( s ) t s ) } s ∈ [ S ] , including adaptive strategies. It remains valid even when the α s 's take different forms.

We are now ready to state our main technical result for this section.

Lemma 4. Let H ⊂ [0 , 1] X be any finite class and ℓ : [0 , 1] × [0 , 1] → [0 , 1] be any loss that is convex in the first argument. Then, the adaptive minimax regret of H satisfies: regret a T ( H , S ) ≤ O ( T · √ (log |H| ) /S ) . Moreover, this is achieved by Algorithm 2 with ∆ := 4 T S .

Proof. The proof follows the same argument as Theorem 1, by replacing Lemma 1 with Lemma 2, and invoking Lemma 3 to bound the weighted regret on the queried samples. The only remaining step is to select an appropriate ∆ that ensures t S ≥ T , since the epochs [ t s -1 , t s ) are no longer fixed. To this end, by the multiplicative Chernoff bound [18, Corollary 2.18], we have:

Setting ∆ := 4 T/S , the error probability becomes exp( -S/ 16) . Therefore, the expected risk contributed by the 'bad' event { t S < T } is at most T · exp( -S/ 16) , which is negligible.

Using a covering argument similar to that in [1, Lemma 12], we extend Lemma 4 to hypothesis classes with finite Littlestone dimension below. The full proof is deferred to Appendix C.

Theorem 2. Let H ⊂ { 0 , 1 } X be a binary-valued class of finite Littlestone dimension and ℓ be the absolute loss ℓ ( y, y ′ ) = | y -y ′ | . Then, the adaptive minimax regret for H satisfies:

We remark that the use of the EWA algorithm in the proof of Lemma 4 is not essential. In fact, any algorithm that achieves low weighted regret, as in Lemma 3, enables a black-box reduction. This includes algorithms from the online convex optimization literature [10], such as online gradient descent and the Follow-the-Regularized-Leader (FTRL) family of methods.

3.2 Lower Bounds

In the preceding sections, we presented algorithms for both oblivious and adaptive data streams that achieve sublinear regret rates. We now show that no algorithm with a bounded query budget can attain a better rate. Specifically, we establish matching lower bounds in both settings, thereby demonstrating that our regret guarantees are optimal .

Theorem 3. Let Y = [0 , 1] , H ⊂ { 0 , 1 } X be any class with Littlestone dimension Ldim ( H ) and ℓ be the absolute loss ℓ ( y, y ′ ) = | y -y ′ | . Then, for any algorithm A with query budget S , there exists an oblivious data process Z such that:

Note that the lower bound in Theorem 3 holds for oblivious data processes, which automatically applies to adaptive data processes. This implies that both upper bounds in Corollary 1 and Theorem 2 are tight (up to a log S factor) w.r.t. all parameters: T , S , and Ldim ( H ) .

To proceed, we introduce the following key inequality (see [4, Lemma A.9]).

We present here a simplified proof under adaptive data processes and assume that the query strategy is data-independent . The stronger result for oblivious data processes and general query strategies follows from a more intricate construction; the complete proof is deferred to Appendix D.

Proof of Theorem 3 (weaker version). We first consider the case where H := { h 0 , h 1 } with h 0 ( x ) = 0 and h 1 ( x ) = 1 for some fixed x . Let { b 1 , · · · , b S } ∼ Unif ( { 0 , 1 } S ) , we construct an adaptive data process Φ ( b ) as follows. We set the initial data process Z (1) t := ( x , b 1 ) for all t ∈ R . After the s query, we switch to another process with Z ( s +1) t := ( x , b s +1 ) for all t ∈ R . Denote T s := t s -t s -1 as the (random) epoch length between the s and s -1 queries. We have

where we used the independence between b s and { ˆ h s -1 , t s } . For comparator loss, we have:

where ( a ) follows by changing of variables ϵ s := 1 -b s 2 and setting a s := E [ T s ] ; ( b ) follows by Lemma 5 (lower bound); ( c ) follows by Cauchy-Schwarz and the fact that a 1 + · · · + a S = T . By the probabilistic method, there must exist some Φ ( b ) that attains an expected regret of Ω( T/ √ S ) .

For general class H with Littlestone dimension d := Ldim ( H ) , we follow an approach similar to that of [1, Lemma 14]. We divide the query budget into d phases , each with S/d queries. The adversarial

strategy proceeds as follows: let τ be a Littlestone tree of H of depth d . We begin with x 1 as the root of τ and assign the labels using the construction for the simplified case above. Once the learner uses all S/d queries, we choose the next input x 2 as the child of x 1 in τ whose outgoing edge label minimizes the comparator loss in (7). This process continues until all query budget is exhausted. Let T r be the length of the time-interval for the r 'th phase, the total regret is lower bounded by:

Here, we used the fact that there exists h ∈ H whose predictions attain (7) for all phases.

Remark 2. Note that step (b) in Eq. (7) implicitly uses the data-independence assumption of the queries (i.e., the a s is independent of ϵ s 's). This assumption is removed in our full proof (Appendix D) by constructing an oblivious data process. Furthermore, although our simplified proof here is reminiscent of the classic arguments in [1], its adaptation to the continuous-time setting and the phase-based scheduling constitutes a substantive theoretical advancement.

4 Discussion

Our work focuses primarily on the worst-case setting, where the data process is arbitrary and no distributional assumptions are made. In this adversarial regime, we show that our algorithms achieve optimal regret rates, tightly matching lower bounds in both the oblivious and adaptive settings. These results provide a precise characterization of the inherent difficulty of learning from agnostic continuous data streams with limited interaction. However, it is important to note that for specific data processes, improved strategies may be possible by leveraging structural or statistical assumptions. For instance, consider the case where the data process is i.i.d. and is realizable w.r.t. a finite hypothesis class. A strategy that queries at uniformly spaced intervals of length ∆ := T/S and updates the predictor using empirical risk minimization (ERM) at each query point incurs total risk of order:

since the risk of ERM with s i.i.d. samples is O (1 /s ) [15]. In contrast, a naive strategy that queries S examples at the very beginning and freezes the predictor for the remainder of the time horizon incurs a total risk of order T/S . Therefore, designing optimal query strategies tailored to specific data-generating processes remains an intriguing direction for future research.

Another modeling choice that may raise concern is our use of integration over time to define risk, which may appear idealized in real-world settings, where predictions are inherently discrete. We emphasize that the integral formulation is primarily a mathematical abstraction used to simplify exposition and analysis. An analogous discrete-time version, where risk is computed as the sum of per-step losses, leads to the same bounds under mild assumptions. In that case, care must be taken to ensure that the number of queries does not exceed the total number of time steps, but all of our arguments and regret guarantees extend naturally with minor modifications.

Acknowledgments. This work is partially supported by the NSF Center for Science of Information (CSoI) Grant CCF-0939370, and also by NSF Grants CCF-2006440 and and CCF-2211423.

References

Shai Ben-David, Dávid Pál, and Shai Shalev-Shwartz. Agnostic online learning. In Conference on Learning Theory , volume 3, 2009.

Alina Beygelzimer, Sanjoy Dasgupta, and John Langford. Importance weighted active learning. In Proceedings of the 26th annual international conference on machine learning , pages 49-56, 2009.

Adam Block, Yuval Dagan, Noah Golowich, and Alexander Rakhlin. Smoothed online learning is as easy as statistical learning. arXiv preprint arXiv:2202.04690 , 2022.

N. Cesa-Bianchi and G. Lugosi. Prediction, Learning and Games . Cambridge University Press, 2006.

Amit Daniely, Sivan Sabato, Shai Ben-David, and Shai Shalev-Shwartz. Multiclass learnability and the erm principle. J. Mach. Learn. Res. , 16(1):2377-2404, 2015.

Ofer Dekel, Claudio Gentile, and Karthik Sridharan. Selective sampling and active learning from single and multiple teachers. The Journal of Machine Learning Research , 13(1):2655-2697, 2012.

Pramith Devulapalli and Steve Hanneke. Learning from snapshots of discrete and continuous data streams. arXiv preprint arXiv:2412.06079 , 2024.

Steve Hanneke, Shay Moran, Vinod Raman, Unique Subedi, and Ambuj Tewari. Multiclass online learning and uniform convergence. In Gergely Neu and Lorenzo Rosasco, editors, Proceedings of the Thirty-Sixth Conference on Learning Theory , volume 195 of Proceedings of Machine Learning Research , pages 5682-5696. PMLR, 12-15 Jul 2023.

Steve Hanneke and Liu Yang. Minimax analysis of active learning. J. Mach. Learn. Res. , 16(1):3487-3602, 2015.

Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends® in Optimization , 2(3-4):157-325, 2016.

Sham Kakade and Adam T Kalai. From batch to transductive online learning. Advances in Neural Information Processing Systems , 18, 2005.

Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine learning , 2:285-318, 1988.

Shie Mannor and Nahum Shimkin. On-line learning with imperfect monitoring. In Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003. Proceedings , pages 552-566. Springer, 2003.

Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Random averages, combinatorial parameters, and learnability. In Advances in Neural Information Processing Systems , 2010.

Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms . Cambridge university press, 2014.

Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and Trends® in Machine Learning , 4(2):107-194, 2012.

Changlong Wu, Jin Sima, and Wojciech Szpankowski. Oracle-efficient hybrid online learning with unknown distribution. In Shipra Agrawal and Aaron Roth, editors, Proceedings of the 37th Conference on Learning Theory , volume 247 of Proceedings of Machine Learning Research , pages 4992-5018. PMLR, 30 Jun-03 Jul 2024.

Tong Zhang. Mathematical analysis of machine learning algorithms . Cambridge University Press, 2023.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We confirm that the abstract and introduction accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations are discussed in Appendix 4.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We confirm that the assumptions and proofs are complete.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: This is a pure theory paper with no experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We confirm our research aligns fully with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a pure theory paper, we do not see any immediate societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This is a pure theory paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or nonstandard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: We used LLMs solely for writing assistance and editing purposes. No part of the core methodology, theoretical results, or scientific contributions relies on or was derived using LLMs.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Empirical Validation

We complement our theoretical results with controlled simulations to illustrate how the proposed update rules behave under various data processes. We implement three algorithms corresponding to our theoretical constructions:

Uniform Epoch: updates are performed at fixed, evenly spaced intervals.

Dynamic Epoch: update times are chosen adaptively based on past intervals.

Dynamic + Reweighting: same as Dynamic, but each update is weighted by

to correct post-epoch bias.

All algorithms use the Exponential Weights Algorithm (EWA) with absolute loss over a class of threshold experts,

We instantiate K = 201 uniformly spaced thresholds τ j = j K +1 , and simulate over N = 30 , 000 discrete time steps (corresponding to a continuous horizon of T = 1 ) to approximate the integration in our theoretical formulation. The number of model updates is varied as

Each configuration is averaged over five independent random seeds to reduce stochastic variance.

Data processes. In all environments, the features are generated independently as x t ∼ Unif[0 , 1] . The environments differ only in how the labels y t are generated as (possibly adversarial) functions of x t and the query times { t s } . We consider one stochastic and three adversarial constructions:

Oblivious Drift: Let t ↦→ θ ( t ) = 0 . 5 + 0 . 3 sin(4 πt/T ) denote a drifting threshold completing two full cycles over the horizon [0 , T ] . At each discrete tick t , the label is assigned as

followed by 5% independent label flips ( y t ← 1 -y t with probability 0 . 05 ). This produces a smooth periodic drift that is entirely oblivious to the learner's updates.

Adversarial A1 (Bit-after-Query): Labels alternate over time between two regimes, with switches exactly at the learner's query times. One regime uses the informative threshold y t = 1 { x t ≥ 0 . 5 } ; the other fixes labels to a constant random bit B s ∼ Bernoulli(1 / 2) .

Adversarial A2 (Post-Query Shift): Labels alternate over time between two threshold regimes, with switches exactly at the learner's query times. One regime follows the balanced rule y t = 1 { x t ≥ 0 . 5 } ; the other adopts a skewed rule y t = 1 { x t ≥ 0 . 8 } , creating abrupt post-query shifts that make the data imbalanced and harder to predict.

Adversarial A3 (Refresh-Random-Bits): After each query, the label switches to a freshly sampled random bit B s ∼ Bernoulli(1 / 2) , which is held fixed until the next query.

Evaluation. We measure the mean regret

and report mean ± std across five seeds. For visual reference, we also fit the curve C/ √ S in log-space.

Results. Figure 1 plots the average regret versus S (log-log scale), together with a fitted C/ √ S reference line. Across all settings, the regret decays approximately as O (1 / √ S ) , consistent with our theoretical prediction. Moreover, the deviations around the mean are of small order, indicating that the behavior holds not only in expectation but is also robust across runs.

In the oblivious case, all three algorithms achieve the expected scaling, differing only by constants.

(a)

Oblivious drift

(b)

Adversarial A1: bit-after-query

RegretscalingvsS-oblivious_drift

uniform

uniform

10-1

dynamic

dynamic

dynamic_weighted

dynamic_weighted

fit: C / sqrt(S)

fit: C/sqrt(S)

10-1

Regret

Regret

10-2

102

103

102

103

S (number of updates)

S (number of updates)

Regret scaling vs S—adv_A2_post_query_shift

RegretscalingvsS-adv_A3_refresh_random_bits

2 × 10-1

uniform

uniform

dynamic

dynamic

dynamic_weighted

dynamic_weighted

fit: C / sqrt(S)

10-1

fit: C / sqrt(S)

10-1

Regret

Regret

6 ×10-2

10-2

4 × 10-2

3 ×10-2

10-3

102

103

102

103

S (number of updates)

S (number of updates)

(c)

(d)

Under A1 and A2 , the dynamic schedule outperforms uniform updates, confirming the benefit of adaptive query placement.

Under A3 , the dynamic+ reweighting strategy outperforms the unweighted dynamic variant, demonstrating the effectiveness of reweighting. Nevertheless, the uniform schedule also attains small regret, which we attribute to the simplicity of our threshold-expert class.

B Proof of Lemma 3

Denote ℓ s ( · , · ) := α s · ℓ ( · , · ) and K := |H| for notation convenience. Let W s = ∑ K k =1 w s [ k ] be the potential. We have:

where the first inequality follows by Hoeffding's lemma [4, Lemma A.1] and the fact that α s ∈ [0 , 1] , the second inequality follows by Jensen's inequality and convexity of ℓ s , and the last identity follows

by the definition of ˆ h s -1 (see Algorithm 3). Summing from s = 1 to S , we have:

The lemma follows from the fact that:

and setting η = √ (log K ) /S .

Remark 3. Note that the main difference compared to classical regret analysis of the EWA algorithm [4, Theorem 2.2] is that the losses ℓ s at different time steps s may vary.

C Proof of Theorem 2

By [1, Lemma 12], the class H admits a finite expert class G ⊂ X ∗ →{ 0 , 1 } such that for any h ∈ H and data { Z ( s ) s := ( X ( s ) t s , Y ( s ) t s ) } s ∈ [ S ] , there exists g ∈ G satisfying:

This implies that, when running Algorithm 3 over G , the incurred (weighted) regret against H is always upper bounded by the regret against G . Moreover, by [1], we have:

Invoking Lemma 3 on expert class G , the (weighted) regret for H on the queried examples grows as:

The result then follows from the same arguments as in Lemma 4.

D Proof of Theorem 3

This appendix presents the complete proof of Theorem 3. Before showing the formal proof, we distinguish between the following two settings for query strategies :

Oblivious Query Strategy : A query strategy that is independent of the data;

Adaptive Query Strategy : A query strategy that depends on the data.

Similarly, we distinguish between the following two settings for Nature :

Oblivious Nature : An adversarial environment where the data-generating process is chosen independently of the learner's actions;

Adaptive Nature : An adversarial environment where the data-generating process is chosen dependently on the learner's actions.

This classification yields four lower-bound regimes-(O-O), (O-A), (A-O), and (A-A)-where the first letter specifies the query strategy and the second the behavior of Nature. Formally, a label (X-Y) asserts that for every query strategy of type X, there exists a data-generation process of type Y that achieves the stated lower bound.

The following proposition is straightforward:

Proposition 2. The following implications hold among the different types of lower bounds:

Proof. The proof follows from the facts that (A-X) ⇒ (O-X) and (X-O) ⇒ (X-A) for all X ∈ { O , A } . The first implication holds because a lower bound that applies to all Adaptive query strategies also applies to all Oblivious strategies under the same type X of data process. The second implication holds because an Oblivious Nature is a special case of an Adaptive Nature.

Proposition 2 essentially states that if a lower bound holds for any adaptive query strategy with an oblivious data process, then it is the strongest , as it also holds in the other three settings. Observe that the lower bounds we established in the proof of Section 3.2 can be classified as type (O-A), which is the weakest type among our classification above.

We now present below the complete proof of Theorem 3 with the strongest type (A-O). For the reader's convenience, we restate the theorem below:

Theorem 4. Let H ⊂ { 0 , 1 } X be a binary-valued class of finite Littlestone dimension. Then, for any learning algorithm A with adaptive query strategy and a query budget S , there exists an oblivious data process Z such that:

The proof of Theorem 4 relies on several technical lemmas, as follows.

Lemma 6. Let B 1 , . . . , B m i.i.d. ∼ Bernoulli ( 1 2 ) . Choose indices I 1 , . . . , I k independently and uniformly from { 1 , . . . , m } (with replacement) and set Y t := B I t for t = 1 , . . . , k . Let J be uniformly sampled from { 1 , . . . , m } , independently of all previous choices. Let S ⊆ { I 1 , . . . , I k } be the set of all distinct sampled indices and denote ˜ k = | S | . Define

where for each j ∈ S , t ( j ) is any index with I t ( j ) = j . Then

Proof. Condition on ( Y 1 , . . . , Y k ) . For each j ∈ S , B j is revealed exactly as Y t ( j ) , hence E [ B j | Y 1 , . . . , Y k ] = Y t ( j ) . For j / ∈ S , B j remains independent Bernoulli( 1 2 ) , so E [ B j | Y 1 , . . . , Y k ] = 1 2 .

Averaging over the uniform J ,

Simplifying, as claimed.

Lemma 7. Let Φ : { 0 , 1 } k → [0 , 1] be any (possibly random) predictor. Then, with the notation as in Lemma 6, we have:

Proof. Condition on Y 1 , · · · , Y k , we denote p := E [ B J | Y 1 , · · · , Y k ] . Then, for any given b ∈ [0 , 1] , we have

where the second identity follows from Lemma 6. The lemma follows by taking expectation on both sides over the internal randomness of Φ .

Lemma 8. With the notations of Lemma 6, let S ⊆ { I 1 , . . . , I k } , ˜ k = | S | , ¯ Y distinct := 1 ˜ k ∑ j ∈ S B j . Then, conditioning on the indices I 1 , . . . , I k (equivalently on S ),

Consequently,

Proof. Condition on S (hence on ˜ k ). Write X j := B j -1 2 , so the { X j } j ∈ S are independent, mean 0 , and take values in {± 1 2 } . Then

By the Khintchine's inequality (upper bound in Lemma 5),

Dividing by m yields the conditional bound E [∣ ∣ ∣ ˜ k m ( ¯ Y distinct -1 2 ) ∣ ∣ ∣ ∣ ∣ ∣ S ] ≤ √ ˜ k 2 m . The second bound follows from the concavity of the square root: E [ √ ˜ k ] ≤ √ E [ ˜ k ] . Finally, for sampling k times with replacement from m bins, E [ ˜ k ] = m ( 1 -( 1 -1 m ) k ) , giving the stated closed form.

Proof of Theorem 4. We start by considering the simplified case where H := { h 0 , h 1 } with h 0 ( x ) = 0 and h 1 ( x ) = 1 for some fixed x .

We partition the time horizon into T · S (small) blocks each of length 1 S . We will assign the same label within each block, and assign the feature x for all time steps. Let

for some m to be determined. For each block t , we assign its label Y t in the following manner: we first sample I t ∼ Unif( { 1 , · · · , m } ) independent of all other randomness and set Y t := B I t .

Let t 1 , · · · , t S be the blocks that are queried by any learning algorithm, possibly dependent on the data. We may assume, w.l.o.g., that all the queries are in different blocks (since the same queries within one block reveal no additional information as the labels are the same). Now, consider any block t between t s and t s +1 that is not queried. By the law of total probability, we have:

where we have used the fact that the distribution of Y t conditioned on the queried observations Y t 1 , · · · , Y t S is exactly the same as B J in Lemma 6 for k = S . Since there are ( T · S -S ) blocks that are not queried and each has size 1 S , the total risk is lower bounded by (for S ≤ m ) 4 :

We now analyze the comparator loss. Denote by T j the total length of all blocks that are assigned with variable B j for j ∈ [ m ] , where ∑ m j =1 T j = T . Conditioned on any realization of T j 's, the comparator risk can be expressed as:

Using a similar argument for proving (7), we conclude that:

Putting everything together and taking m := 5 S , we arrive at the expected regret lower bound:

The lower bound for the simplified case now follows by choosing the oblivious data process to be the (deterministic) realization of the Y t 's that attains the expected regret lower bound above, which must exist by the probabilistic method.

For general class H with Littlestone dimension d := Ldim ( H ) , we construct the following oblivious data process. We partition the time horizon T into d phases, each of length T/d . Let τ be a Littlestone tree of H of depth d (see [1, Lemma 14]), and let x 1 be the root of τ . We assign the feature x 1 during the first phase and choose labels as in the simplified case above 5 , for some value m to be determined. Based on the realization of labels in phase one, we set the feature x 2 for phase two to be the child of x 1 in τ whose outgoing edge label minimizes the comparator loss, and generate labels in the same way using fresh randomness. We repeat this process for all d phases.

By the definition of Littlestone tree, there must exist some h ∈ H that attains the minimum comparator loss in each of the d phases. Invoking (9), the total comparator loss is upper bounded by

We now lower bound the risk of the learning algorithm. To avoid overly technical complications, we assume that the learner assigns a fixed (but arbitrary) query budget S r in phase r ∈ [ d ] , although the actual query times can be chosen completely arbitrarily 6 . Note that in this case, the learner may assign many queries in a single phase, so S r may exceed m for some r . Fortunately, the upper bound in Lemma 8 holds even if k ≥ m . Setting k := S r for all r ∈ [ d ] and invoking a similar argument as in (8), we obtain the following lower bound on the learner's total risk:

where we used the fact that ∑ d r =1 S r ≤ S , and the final inequality follows from Cauchy-Schwarz. Therefore, by setting m := 20 S/d , the total expected regret is lower bounded by

T 2 √ 2 m -T · S 2 dm √ m -T √ dS 2 dm -O (1) ≥ 0 . 048 · T √ S/d -O (1) ≥ Ω ( T · √ d √ S ) . This completes the proof of Theorem 4.

Remark 4. Note that a key technical innovation in our proof-compared to classical lower bound constructions such as in [1]-is that the expected risk incurred by the learning algorithm is no longer T/ 2 . Instead, in our construction, the learner's risk and that of the comparator are subtly linked through the choice of m . The final regret bound leverages the crucial fact that, as m increases, the learner's risk grows faster than the comparator's loss, enabling a non-trivial lower bound on regret via appropriate tuning of m .

To illustrate the necessity of our technical argument in the proof of Theorem 4, we explain why a naive approach-similar to the one used in [1]-fails to yield the optimal lower bound. Suppose we partition the time horizon T into blocks of length ∆ , and assign the label uniformly at random from { 0 , 1 } within each block. In this setting, the learner's expected risk is not T/ 2 . Instead, the learner can query at the beginning of each block and repeat the observed label for the entire block, incurring zero risk on any queried block. Therefore, the total expected risk becomes T -S · ∆ 2 . On the other hand, a standard argument shows that the comparator loss (for the class { h 0 , h 1 } ) is upper bounded by T 2 -O ( √ ∆ · T ) . To obtain non-trivial regret, one must have

This implies ∆ ≤ T S 2 , and the resulting regret lower bound is Ω ( T S ) , which is significantly weaker than the optimal Ω ( T √ S ) regret.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates agnostic continuous-time online learning. In contrast to the classical formulation of online learning, in which the input is a discrete sequence, the input here is a continuous data stream. While previous research has addressed the realizable setting, the more realistic and challenging agnostic setting has remained largely unexplored.  The authors demonstrate that, given a hypothesis class with Littlestone dimension $d$, the regret of online learning this class is exactly $\\tilde{\\Theta}(T\\sqrt{d / S})$ for both oblivious and adaptive adversarial settings, where $T$ represents the time-span and $S$ denotes the number of queries the algorithm can make.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\nThe authors establish a nearly tight $\\tilde{\\Theta}(T\\sqrt{d / S})$ regret bound for the fundamental problem of agnostic continuous-time online learning. For an oblivious adversary, they propose a simple black-box reduction to the discrete case. In the case of an adaptive adversary, they present a novel re-weighting scheme attaining the same bound. Additionally, they provide a matching lower bound that holds even for the most challenging scenario (an adaptive query strategy against an oblivious nature).\n\nFurthermore, the organization and presentation of the paper are quite commendable, making it easy to read and understand.\n\n**Weakness:**\n\nI did not identify any significant weakness. However, I have concerns regarding the correctness of Lemma 6. Please refer to \"Questions\" for further details.\n\nIn the preliminaries, the authors define the protocol (Protocol 1) in a way that it chooses a predictor $\\hat{h}_s\\in\\mathcal{Y}^{\\mathcal{X}}$. However, it appears that sometimes the output is indeed a distribution over predictors (particularly in the adaptive setting, where this is necessary), and at any time step the actual predictor is drawn from this distribution (rather than keep outputting $\\hat{h}_s$ for a period). I recommend the authors clarify this part to make the definition consistent.\n\nQUESTIONS:\n1. I am concerned that Lemma 6 may not hold as $I_1,\\dots,I_k$ could overlap. For example, let $m = k = 2$. Given that $Y_1 = Y_2 = 0$, it holds that $E[B_J \\mid Y_1=Y_2=0]>0$ since it is possible (with positive probability) that $J\\neq I_1 = I_2$ and $B_J = 1$. However, the Lemma gives $1/2 + k/m(\\bar{Y}  - 1/2) = 0$.  As this lemma serves as a key inequality in the proof of the lower bound, can you provide a more detailed explanation regarding its validity?\n2. It seems the lower bound only holds when $S$ is not excessively large: in the formula before line 413, the inequality requires $T / \\sqrt{S} = \\Omega(1)$ (i.e., $S = O(T^2)$). In contrast, the upper bound seems to hold even as $S \\to \\infty$. Therefore, the bounds do not match in the extreme regime. Is my understanding correct?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers a continuous online learning framework in which a learner selects discrete query times at which the data stream is observed (to evaluate the loss) and the learning strategy is updated. This framework was recently studied in [7], but only in the realizable setting—where the data is assumed to be generated by a function within the hypothesis class. In contrast, the present work focuses on the more challenging agnostic setting. The authors generalize the notion of expected regret to this continuous-time framework by integrating the loss over time, rather than summing it at discrete points.\n\nTwo settings are considered: the oblivious data stream, where the data is generated arbitrarily in advance and independently of the learner’s randomness; and the adaptive data stream, where the data stream may depend on any past information. For both settings, the authors propose a black-box reduction to an expert algorithm, using the observed losses at the random query times as unbiased estimators of the integrated continuous loss. The regret bound then follows by plugging these estimators into the standard regret analysis of the expert algorithm.\n\nIn addition to upper bounds, the authors provide a nearly matching lower bound (up to logarithmic factors) in the oblivious setting, showing that the regret guarantees are essentially optimal.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\n- The paper is well-written and easy to follow.\n- The problem considered is interesting and the authors provide a clean and complete analysis with tight upper and lower bounds. \n\nWeaknesses:\n- The results are restricted to finite hypothesis classes. Is it not possible to extend the black-box reduction to continuous hypothesis classes, and more generally to reduce to any discrete online convex optimization algorithm, rather than limiting to expert algorithms?\n- The paper would benefit from synthetic numerical experiments to illustrate the practical implications of the continuous-time framework on regret. In particular, when setting $T = S$, we would expect to recover similar performance—does this hold in practice? Additionally, how stable are the results, given that they rely on an estimation technique?\n- The proofs appear to be fairly standard and rely on combining known results: the agnostic lower bound follows techniques from [1]; the sampling process for query points is borrowed from [7]; and the use of unbiased loss estimators within an expert algorithm mirrors approaches commonly used in adversarial bandits and online learning with partial feedback.\n\nQUESTIONS:\n- Is it possible to derive high-probability guarantees, rather than bounds only in expectation? Alternatively, can one obtain upper bounds on the actual expected regret (i.e., using the infimum over $h$ in Equation (5)) instead of pseudo-regret bounds (i.e., the expectation as in Equation (4))?\n\n- The results are presented differently for the oblivious and adaptive settings. Is it not possible to unify the presentation, perhaps by giving a generic analogue of the black-box reduction in Theorem 1 for the adaptive setting—similar to what is done for the oblivious case—rather than relying on the more case-specific Lemma 4 and Theorem 2? The motivation for diverging in presentation is not entirely clear.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies agnostic online learning from continueous-time data streams. learners have to interact with continually envolving data stream while making queries and updating model at strategically selected times. This paper develop theoretical framework for learning from both oblivious and adaptive data streams.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \nThis paper develops generic algorithmic frameworks for agnostic continuous-time online learning under both oblivious and adaptive data streams with optimal regret guarantees, giving both upper and lower bounds.\n\nWeakness:\nNo experimental results on either synthetic or real world data are conducted in the paper, it remains unclear whether the algorithms proposed can be used in practice.\n\nQUESTIONS:\n1. The paper builds on and extends the update-and-deploy framework of Devulapalli and Hanneke [7], which focused on the realizable setting. Could the authors clearly delineate the conceptual and technical differences and challenges introduced by the agnostic setting?\n2. The paper proposes a novel importance weighting scheme (Algorithm 3) to obtain unbiased loss estimates under adaptive query times with dynamic epochs. Could the authors clarify how sensitive is the performance and regret bound to the exact choice of weighting function?\n3. Practical data streams can be complex and non-stationary beyond Littlestone dimension and scale characterization. Could the authors discuss or exemplify scenarios or hypothesis classes where these bounds translate to meaningful learning guarantees in real-world continuous data and elaborate on how these theoretical guarantees relate to computational or sample complexity in applications like high-frequency trading or personalized recommendations?\n\nLIMITATIONS:\nYes, the authors have acknowledged and discussed the limitations in Appendix A,", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends te setting of (realizable) continuous-time online learning from Devulapalli and Hanneke to the agnostic case. In this framework, there is a stream in continuous time of queries $X_t$, and the algorithm needs to make online predictions on the stream  by picking a hypothesis to use with a few discrete observation of the true labels $Y_t$ to the stream. The original assumed that the labels in the stream were generated by some hypothesis in the hypothesis class. This paper drops this assumption, and show algorithms (and some matching lower bounds) for both oblivious and adaptive adversaries. In the oblivious case the algorithm splits the time-horizon in equal length epochs and samples a uniform time for each epoch to see the label and use as a black-box an OL algorithm. In the adaptive case this may not work anymore, and they show how to use dynamic epochs, also used by Devulapalli and Hanneke, and dynamic step sizes on EWA to circumvent this issue.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper studies a very natural (and more challenging) extension of the continuous-time online learning framework proposed by Devulapalli and Hanneke, which by itself it already a somewhat natural problem. Many of the techniques in the algorithms seem to drawn upon the ideas of DH24, but are certainly not trivial extensions in any way. The algorithm (or, better saying, reduction) for the oblivious case is clean and serves as a nice way to \"warm-up\" to the adaptive case, although the oblivious case is interesting in itself. Also, the slight modification of EWA  leads to a (relatively) elegant analysis. Unfortunately I did not have the time to check the proofs as I would like to have done, but the few derivations that I had time to check were relatively clearly exposed and seemed to be correct. \n\nOverall, although there does not seem to be major novel ideas in the framework nor in the algorithms, this is a quite solid contribution, and shows optimal rates for both the oblivious and adaptive cases, which makes this a even more complete and solid work. While I have some questions, none of them seem to be crucial for clarity/significance/originality/quality of the paper.\n\nQUESTIONS:\nSmall technical question, but DH24 assume the process $Z_t$  to be countable. I don't think this matters too much except on some of the technical conditions required later on (such as measurability of the sample paths). Do the authors think there are some technical kinks that might need to be ironed out because of this?\n\nLIMITATIONS:\nThe limitations of the setup seem to be well discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}