Title: Online Prediction with Limited Selectivity
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online prediction

selective prediction
Abstract: Selective prediction [Dru13, QV19] models the scenario where a forecaster freely decides on the prediction window that their forecast spans. Many data statistics can be predicted to a non-trivial error rate *without any* distributional assumptions or expert advice, yet these results rely on that the forecaster may predict at any time. We introduce a model of Prediction with Limited Selectivity (PLS) where the forecaster can start the prediction only on a subset of the time horizon. We study the optimal prediction error both on an instance-by-instance basis and via an average-case analysis. We introduce a complexity measure that gives instance-dependent bounds on the optimal error. For a randomly-generated PLS instance, these bounds match with high probability.

Online Prediction with Limited Selectivity

Abstract

Selective prediction [Dru13, QV19] models the scenario where a forecaster freely decides on the prediction window that their forecast spans. Many data statistics can be predicted to a non-trivial error rate without any distributional assumptions or expert advice, yet these results rely on that the forecaster may predict at any time. We introduce a model of Prediction with Limited Selectivity (PLS) where the forecaster can start the prediction only on a subset of the time horizon. We study the optimal prediction error both on an instance-by-instance basis and via an average-case analysis. We introduce a complexity measure that gives instancedependent bounds on the optimal error. For a randomly-generated PLS instance, these bounds match with high probability.

1 Introduction

In selective prediction [Dru13, QV19], a forecaster observes n numbers in [0 , 1] one by one. At any time t , having observed the first t numbers, the forecaster may predict the average of the next w ≤ n -t unseen numbers. Both the stopping time t and the window length w are freely chosen by the forecaster, and only one such prediction needs to be made. The goal is to minimize the expected prediction error-the expected squared difference between the forecast and the actual average. How small can this error be?

Surprisingly, Drucker [Dru13] showed that the forecaster can guarantee an error that vanishes as n → + ∞ , even though the sequence might be arbitrarily and adversarially chosen. Specifically, this result holds without any distributional assumption on the sequence (other than boundedness), and is thus robust to any misspecification, non-stationarity, or adversarial corruption in the data. Moreover, it directly addresses the prediction error, rather than a regret with respect to a class of experts.

In this paper, we study a variant of the selective prediction model where the forecaster only has limited selectivity . Concretely, the forecaster is given a subset of the time horizon, which specifies the timesteps on which they are allowed to make a prediction. This model captures many natural scenarios where predictions are either infeasible or unnecessary during certain time periods. For example, people tend to care about weather forecasts primarily when they have plans for outdoor activities. An investor may be restricted from trading specific commodities during particular seasons, making market predictions less relevant at those times. Similarly, epidemic forecasts are most critical before and during pandemics.

The main contributions of this work are summarized as follows:

We introduce a theoretical model of Prediction with Limited Selectivity (PLS), which generalizes the selective prediction models of [Dru13, QV19].

We define a complexity measure termed 'approximate uniformity', which gives instance-dependent bounds on the optimal error that a forecasting algorithm can achieve on a PLS instance.

For PLS instances that are randomly generated according to a k -monotone sequence (Definition 4), we show that the instance-dependent bounds match up to a constant factor with high probability.

1.1 Problem Formulation

Definition 1 (Prediction with limited selectivity) . The forecaster is given n and a stopping time set T ⊆ { 0 , 1 , 2 , . . . , n -1 } , and the nature secretly chooses a sequence x ∈ [0 , 1] n . At each timestep t = 1 , 2 , . . . , n , the forecaster observes x t . At any timestep t ∈ T , after seeing x 1 , . . . , x t , the forecaster may optionally choose a window length w ∈ { 1 , 2 , . . . , n -t } and make a prediction ˆ µ on the average µ = 1 w ∑ w i =1 x t + i . Once a prediction is made, the game ends and the forecaster incurs an error of (ˆ µ -µ ) 2 . The forecaster must make one prediction before all n numbers are revealed.

An instance of PLS consists of a sequence length n and a stopping time set T . When the forecaster is fully-selective (namely, T = { 0 , 1 , 2 , . . . , n -1 } ), PLS recovers the selective mean prediction setup of [Dru13, QV19]. For simplicity, we focus on the special case of mean prediction, which already captures most of the interesting aspects of PLS. Nevertheless, the problem setup and our results can be easily extended to the more general prediction settings in [QV19]; see Section 5 for more details.

Following prior work on selective prediction, we measure the performance of a forecaster using its worst-case error over all possible choices of the sequence x ∈ [0 , 1] n .

Definition 2. The worst-case error of algorithm A is error worst ( A ) := sup x ∈ [0 , 1] n error( A , x ) , where error( A , x ) denotes the expected squared error that A incurs on sequence x ∈ [0 , 1] n .

We will frequently use the following equivalent yet more convenient representation for a PLS instance.

Definition 3. The block representation of a PLS instance with sequence length n and stopping time set T is

where m = |T | and t 1 < t 2 < · · · < t m are the elements of T in ascending order.

In the rest of the paper, we will use the stopping time set T and the block representation L interchangeably to represent a PLS instance. Intuitively, each block length l i corresponds to l i consecutive timesteps t i + 1 , t i + 2 , . . . , t i + l i = t i +1 between adjacent stopping times in T . During these timesteps, the forecaster observes new data but cannot make predictions.

The fully-selective setup of [Dru13, QV19] corresponds to the block representation L that is an all-one sequence. When L consists of block lengths of various magnitudes, we naturally expect that the PLS instance becomes harder in the sense that the optimal forecaster has a higher worst-case error. We will formalize this intuition and derive instance-dependent error bounds for every PLS instance L in terms of a simple and combinatorial complexity measure of L .

In addition to the instance-dependent analysis, we will also study settings where the stopping time set T is randomly generated. Concretely, we consider a setup where each timestep between 0 and n -1 gets included in the stopping time set T independently, possibly with different probabilities.

Definition 4 (Random stopping time set) . For integer n ≥ 1 and ( p ⋆ 0 , p ⋆ 1 , . . . , p ⋆ n -1 ) ∈ [0 , 1] n , a p ⋆ -random stopping time set T is a random subset of { 0 , 1 , . . . , n -1 } obtained by independently including each element t with probability p ⋆ t .

1.2 Our Results

We prove both upper and lower bounds on the optimal worst-case error in PLS, both on an instanceby-instance basis and via an average-case analysis.

Approximate uniformity. We introduce a complexity measure, termed approximate uniformity , that captures the hardness of a PLS instance.

Definition 5. The approximate uniformity of PLS instance L = ( l 1 , l 2 , . . . , l m ) is

For the fully-selective case that L = (1 , 1 , . . . , 1) , we have ˜ U ( L ) = m . More generally, ˜ U ( L ) captures the 'effective horizon length' in instance L . As we show in the following, the approximate uniformity roughly characterizes the lowest worst-case error that a forecasting algorithm can achieve.

Instance-dependent error bounds. Our main algorithmic contribution is a forecasting algorithm with a worst-case error upper bounded in terms of ˜ U ( L ) . This generalizes the O (1 / log n ) error bound of [Dru13] for the fully-selective case.

Theorem 1. For every PLS instance L , there is a forecasting algorithm with a worst-case error of O (1 / log ˜ U ( L )) .

We prove Theorem 1 in two steps. First, we establish an O (1 / log m ) upper bound for the special case that the m block lengths are approximately uniform in the sense of being within a constant factor. Then, we reduce a general PLS instance L to the special case by merging the blocks into longer blocks with approximately uniform lengths. We show that at least Ω( ˜ U ( L )) longer blocks can be obtained in this way, so the result for the special case implies the O (1 / log ˜ U ( L )) upper bound.

Complementary to Theorem 1, we give two lower bounds on the worst-case error.

Theorem 2. For every PLS instance L = ( l 1 , l 2 , . . . , l m ) , every forecasting algorithm has a worstcase error of Ω(max { 1 / [ ˜ U ( L )] 2 , 1 / log m } ) .

While the first lower bound of Ω(1 / ˜ U 2 ) does not match the upper bound in Theorem 1, it already has some interesting applications. The following corollary (proved in Appendix A) presents concrete examples where both the sequence length n and the number of blocks m tend to infinity, yet the worst-case error remains lower bounded by a constant. In the first example, the block lengths are geometrically increasing, so m is at most logarithmic in the sequence length n . The second example, inspired by the Cantor set, shows that an Ω(1) error is still unavoidable even if m is polynomial in n . Corollary 3. For every m ≥ 1 , on the PLS instance L m = (2 0 , 2 1 , 2 2 , . . . , 2 m -1 ) with sequence length n = 2 m -1 and m blocks, every forecasting algorithm has an Ω(1) worst-case error. Furthermore, for every k ≥ 1 , there is a PLS instance L ′ k with sequence length n = 3 k and m = 2 k +1 -1 blocks, on which every forecasting algorithm has an Ω(1) worst-case error.

The second lower bound of Ω(1 / log m ) shows that an m -block PLS instance is the easiest when all blocks have the same length: An O (1 / log m ) worst-case error can be achieved when l 1 = l 2 = · · · = l m , while no algorithm can achieve a worst-case error ≪ 1 / log m on any instance with m blocks. While this result might sound intuitive, our proof of the Ω(1 / log m ) bound is non-trivial. The proof involves a novel hierarchical decomposition of the m blocks into a ternary tree and the design of a random process on the resulting tree. This extends a construction of [QV19] for m blocks of equal lengths, which is based on representing the m blocks as a full binary tree of depth log m .

An average-case analysis. While the instance-dependent upper and lower bounds do not match on every PLS instance, they do match with high probability when the instance is randomly generated. A sequence is k -monotone if it can be partitioned into at most k contiguous monotone subsequences. Our next result states that, if T is a p ⋆ -random stopping time set (Definition 4) for a k -monotone sequence p ⋆ , both |T | and ˜ U ( T ) can be bounded in terms of ∥ p ⋆ ∥ 1 with high probability.

Theorem 4. Suppose that p ⋆ ∈ [0 , 1] n is k -monotone and T is a p ⋆ -random stopping time set. Let m 0 := ∑ n -1 t =0 p ⋆ t . The following two hold simultaneously with probability 1 -e -m 0 / 3 -1 /n over the randomness in T : (1) |T | = O ( m 0 ) ; (2) ˜ U ( T ) = Ω( m 0 / ( k log 2 n )) .

As a direct corollary, our bounds on the optimal worst-case error are tight up to a constant factor with high probability. We prove this corollary in Appendix A.

Corollary 5. If k -monotone sequence p ⋆ ∈ [0 , 1] n satisfies ∑ n -1 t =0 p ⋆ t ≥ Ω(( k log 2 n ) 1+ c ) for some constant c > 0 , with high probability over the randomness in the p ⋆ -random stopping time set T , the forecasting algorithm from Theorem 1 has a worst-case error that is optimal up to a constant factor.

Concretely, if k = O (1) is a constant, Corollary 5 applies whenever the expected number of stopping times, ∑ n -1 t =0 p ⋆ t , is at least polylog( n ) . If k = O ( n α ) is polynomial in n for some α ∈ (0 , 1) , we have nearly-tight bounds as long as ∑ n -1 t =0 p ⋆ t = Ω( n β ) for some β ∈ ( α, 1] .

1.3 Related Work

Most closely related to our study is the prior work on selective prediction. Drucker [Dru13] introduced the problem of selective mean prediction under the name of 'density prediction game' and proved an

O (1 / log n ) upper bound on the expected error. Qiao and Valiant [QV19] proved a matching lower bound, and extended the positive result of [Dru13] to the setting of predicting more general functions.

Chen, Valiant, and Valiant [CVV20] introduced a more general framework that encompasses selective prediction as well as other data-collection procedures including importance sampling and snowball sampling. Brown-Cohen [BC21] subsequently obtained a faster algorithm under this framework. Qiao and Valiant [QV21] studied a 'learning' variant of selective prediction, in which the learner observes multiple sequences and aims to identify the sequence with the highest average inside a prediction window of their choice.

All the positive results above are based on the Ramsey-theoretic observation that a sufficiently long, bounded sequence must be 'predictable' or 'repetitive' at some scale. This allows a selective forecaster to randomly select a timescale and achieve a vanishing error as the sequence length goes to infinity. Similar observations have been made in different contexts [Fei15, FKT17, MHO25].

More broadly, our setting is related to the recent work on online prediction with abstention, where the forecasting algorithm is allowed to occasionally abstain from making predictions at an additional cost [ZC16, CDG + 18, NZ20, GKCS21, GHMS23, PRT + 24].

2 Instance-Dependent Upper Bounds

2.1 Special Case: Approximately Uniform Block Lengths

Recall from Definition 3 that a PLS instance can be represented by a list of block lengths L = ( l 1 , l 2 , . . . , l m ) . Towards proving Theorem 1, we start with the case that the block lengths do not vary drastically and prove an O (1 / log m ) upper bound on the worst-case error. Our algorithm is defined in Algorithm 2. It calls RandomSelect (Algorithm 1) to obtain a randomized prediction position i and length j . The algorithm then reads the first i -1 blocks of the sequence, and uses the mean of the last j blocks (namely, blocks i -j, i -j +1 , . . . , i -1 ) to predict the mean of the next j blocks ( i through i + j -1 ).

Algorithm 1 ( RandomSelect ( s, k ) ) is a recursive procedure that prescribes a prediction position and a window length within 2 k consecutive blocks (with indices s to s +2 k -1 ). With probability 1 /k , it outputs ( s +2 k -1 , 2 k -1 ) , i.e., predicting the average of the last 2 k -1 blocks using that of the first 2 k -1 . Otherwise, it computes p , the proportion of the total length of the first 2 k -1 blocks within the 2 k blocks. It then recurses on one of the two halves with probability p and 1 -p , respectively.

Algorithm 1: RandomSelect ( s, k ) Input: Integers s, k ≥ 1 . Output: A pair ( i, j ) such that s ≤ i -j and i + j ≤ s +2 k . 1 With probability 1 /k , return ( s +2 k -1 , 2 k -1 ) ; 2 p ← ( ∑ 2 k -1 -1 i =0 l s + i ) / ( ∑ 2 k -1 i =0 l s + i ) ;

3 return RandomSelect ( s, k -1) with probability p , and RandomSelect ( s +2 k -1 , k -1) with the remaining probability 1 -p ;

Proposition 6. On PLS instance L = ( l 1 , l 2 , . . . , l m ) that satisfies max { l 1 ,l 2 ,...,l m } min { l 1 ,l 2 ,...,l m } ≤ C , Algorithm 2 has a worst-case error of O ( C/ log m ) .

This extends the result of [Dru13] for C = 1 , i.e., all blocks have the same length. The proof is similar to those in prior work and deferred to Appendix B. We provide a brief proof sketch below.

Proof sketch. For k ≥ 1 and µ ∈ [0 , 1] , let L ( k, µ ) be the maximum squared error that Algorithm 2 incurs on a sequence of 2 k blocks with average µ . We prove by induction that L ( k, µ ) ≤ O ( C/k ) · µ (1 -µ ) . The proposition then follows from C/k = O ( C/ log m ) and µ (1 -µ ) = O (1) .

Algorithm 2: Prediction with Limited Selectivity on Approximately Uniform Blocks

Input: Instance L = ( l 1 , l 2 , . . . , l m ) . Sequential access to sequence x ∈ [0 , 1] n of length

n

=

l

1

+

l

2

+

· · ·

+

l

m

1 k ←⌊ log 2 m ⌋ ;

2 ( i, j ) ← RandomSelect (1 , k ) ;

3 t ← l 1 + l 2 + · · · + l i -1 ;

4 Read x 1 , x 2 , . . . , x t ;

5 w 0 ← l i -j + l i -j +1 + · · · + l i -1 ;

6 ˆ µ ← 1 w 0 ( x t + x t -1 + · · · + x t -w 0 +1 ) ;

7 w ← l i + l i +1 + · · · + l i + j -1 ;

8 Predict the mean of x t +1 , . . . , x t + w as ˆ µ ;

2.2 The General Case

To prove Theorem 1, we merge the m blocks of possibly different lengths into ≈ ˜ U ( L ) blocks with lengths within a constant factor, so that Proposition 6 can be applied to obtain an O (1 / log ˜ U ( L )) error bound. Formally, we say that a PLS instance L ′ is a merge of another instance L , if L ′ can be obtained from L by merging consecutive blocks and taking a contiguous subsequence.

For instance, (5 , 9) is a merge of (1 , 2 , 3 , 4 , 5 , 6) witnessed by ( i 1 , i 2 , i 3 ) = (2 , 4 , 6) : we merge consecutive elements to obtain (1 , 5 , 9 , 6) and take the contiguous subsequence (5 , 9) . Naturally, if the merge of a PLS instance can be solved with a low worst-case error, so can the original instance. We prove the following lemma in Appendix B.

Lemma 7. If L ′ is a merge of L , the minimum worst-case error that can be obtained on L is smaller than or equal to that on L ′ .

The next lemma states that any instance L has a merge of length Ω( ˜ U ( L )) that consists of elements within a constant factor.

Proof. By definition of ˜ U , there exist indices 1 ≤ i 0 ≤ j 0 ≤ m such that l i 0 + l i 0 +1 + ··· + l j 0 max { l i 0 ,l i 0 +1 ,...,l j 0 } = ˜ U ( L ) . Using the shorthands L := l i 0 + l i 0 +1 + · · · + l j 0 and M := max { l i 0 , l i 0 +1 , . . . , l j 0 } , we have L = ˜ U ( L ) · M . Then, we construct L ′ by merging the block lengths l i 0 , l i 0 +1 , . . . , l j 0 . We set T := M/ ( C -1) and greedily form longer blocks of length ≈ T . Formally, we run the following procedure:

Start with i 1 = i 0 and counter cnt = 1 .

Check whether l i cnt + l i cnt +1 + · · · + l j 0 < T . If so, set m ′ = cnt -1 , L ′ = ( l ′ 1 , l ′ 2 , . . . , l ′ m ′ ) , and end the procedure.

Otherwise, find the smallest k ∈ { i cnt , i cnt +1 , . . . , j 0 } such that l i cnt + l i cnt +1 + · · · + l k ≥ T .

Set l ′ cnt = l i cnt + l i cnt +1 + · · · + l k and i cnt+1 = k +1 . Increment cnt by 1 and return to Step 2.

Clearly, the resulting L ′ = ( l ′ 1 , l ′ 2 , . . . , l ′ m ′ ) is a merge of L witnessed by indices i 1 , i 2 , . . . , i m ′ +1 . The construction ensures that l ′ j ∈ [ T, T + M ) for every j ∈ [ m ′ ] . Therefore,

.

Theorem 1 directly follows from Proposition 6, Lemma 7, and Lemma 8.

Proof of Theorem 1. Applying Lemma 8 with C = 2 shows that L has a merge L ′ = ( l ′ 1 , l ′ 2 , . . . , l ′ m ′ ) such that m ′ ≥ ⌊ ˜ U ( L ) / 2 ⌋ and max { l ′ 1 , l ′ 2 , . . . , l ′ m ′ } / min { l ′ 1 , l ′ 2 , . . . , l ′ m ′ } ≤ 2 . By Proposition 6, there is a forecasting algorithm for L ′ with a worst-case error of O (1 / log m ′ ) = O (1 / log ˜ U ( L )) . Then, Lemma 7 gives a forecasting algorithm for L with the same error bound.

3 Instance-Dependent Lower Bounds

3.1 Lower Bound in Terms of Approximate Uniformity

Theorem 9 (First part of Theorem 2) . For every PLS instance L , every forecasting algorithm A has a worst-case error of error worst ( A ) ≥ 1 16[ ˜ U ( L )] 2 .

Recall from Definition 3 that the block representation L = ( l 1 , l 2 , . . . , l m ) corresponds to a PLS instance with sequence length n = l 1 + l 2 + · · · + l m . 1 The n timesteps are naturally divided into m blocks, where each block i ∈ [ m ] consists of timesteps B i := { l 1 + l 2 + · · · + l i -1 + j : j ∈ [ l i ] } . We prove Theorem 9 using the following observation: Regardless of the prediction window [ t +1 , t + w ] chosen by the forecasting algorithm, an Ω(1 / ˜ U ( L )) fraction of the timesteps within the window must come from the same unseen block.

Lemma 10. Let L = ( l 1 , l 2 , . . . , l m ) be a PLS instance with sequence length n and stopping time set T . Then, for every i 0 ∈ [ m ] , t = l 1 + l 2 + · · · + l i 0 -1 ∈ T and w ∈ [ n -t ] , there exists i ∈ { i 0 , i 0 +1 , . . . , m } such that | B i ∩ [ t +1 , t + w ] | ≥ w 2 ˜ U ( L ) .

The proof is deferred to Appendix C. Next, we show how Theorem 9 follows from Lemma 10.

Proof of Theorem 9 assuming Lemma 10. Consider the random sequence x ∈ { 0 , 1 } n constructed by setting all entries within each block to the same bit chosen independently and uniformly at random. Formally, we draw µ 1 , µ 2 , . . . , µ m ∼ Bernoulli (1 / 2) independently. Then, for each i ∈ [ m ] and j ∈ B i , we set x j = µ i .

Fix a forecasting algorithm A . For t ∈ T and w ∈ [ n -t ] , let E t,w denote the event that A makes a prediction at time t on X t,w := 1 w ∑ w i =1 x t + i . We will show that, conditioning on event E t,w and any observation x 1: t = ( x 1 , x 2 , . . . , x t ) , the conditional variance of X t,w is at least Ω(1 / [ ˜ U ( L )] 2 ) . This would imply that the conditional expectation of the squared error incurred by A is lower bounded by Ω(1 / [ ˜ U ( L )] 2 ) , and the theorem would then follow from the law of total expectation.

Recall that t ∈ T must be of form l 1 + l 2 + · · · + l i 0 -1 for some i 0 ∈ [ m ] . Then, X t,w can be equivalently written as X t,w = ∑ m i = i 0 α i · µ i , where α i := 1 w | B i ∩ [ t +1 , t + w ] | denotes the fraction of timesteps in [ t +1 , t + w ] that fall into block B i . Since we sample µ 1 , . . . , µ m independently, conditioning on event E t,w and the observations x 1: t -both of which are solely determined by the randomness in µ 1 , µ 2 , . . . , µ i 0 -1 and A -each of µ i 0 , µ i 0 +1 , . . . , µ m still follows Bernoulli (1 / 2) independently and has a variance of 1 / 4 . Therefore, the conditional variance of X t,w is given by

By Lemma 10, there exists i ∈ { i 0 , i 0 +1 , . . . , m } such that α i ≥ 1 2 ˜ U ( L ) , so Var [ X t,w | E t,w , x 1: t ] is at least 1 4 · ( 1 2 ˜ U ( L ) ) 2 = 1 16[ ˜ U ( L )] 2 .

3.2 Hard Instance via Tree Construction

Theorem 11 (Second part of Theorem 2) . For every PLS instance L = ( l 1 , l 2 , . . . , l m ) , every forecasting algorithm A has a worst-case error of error worst ( A ) ≥ Ω(1 / log m ) .

Similar to the proof of Theorem 9, we randomly generate a sequence x by first choosing µ ∈ [0 , 1] m , and setting every entry within the i -th block to µ i . The key difference is that, instead of drawing each µ i independently, we carefully design the correlation between different entries of µ . This correlation structure is specified by a tree construction similar to [QV19]: We build a tree whose leaves correspond to the m entries µ 1 , . . . , µ m . We assign a noise to each edge of the tree, and the value of a leaf is set to the sum of noises on the root-to-leaf path. Again, we will argue that for every possible prediction window [ t +1 , t + w ] , even after seeing the first t entries, the conditional variance in the average of x t +1 , . . . , x t + w is still sufficiently high.

A key difference between our construction and that of [QV19] is that they focused on the special case where l 1 = l 2 = · · · = l m = 1 , so that a full binary tree of depth log 2 m suffices. In contrast, to handle the general case, our construction involves a ternary tree in which every internal node has either two or three children, depending on whether the current subtree contains a long block whose length dominates the total block length.

We formally introduce our tree construction as follows.

Definition 7 (Tree construction) . Given a PLS instance ( l 1 , l 2 , . . . , l m ) , we construct a tree using the following recursive procedure:

If m = 1 , return a tree with a single leaf node that corresponds to l 1 .

Let S := l 1 + l 2 + · · · + l m . If there exists i ⋆ ∈ [ m ] such that l i ⋆ > S/ 2 , such index i ⋆ must be unique, and we construct a ternary tree where: (1) The left subtree of the root node is the tree construction for ( l 1 , . . . , l i ⋆ -1 ) ; (2) The middle subtree is a single leaf node corresponding to the i ⋆ -th block; (3) The right subtree is the tree construction for ( l i ⋆ +1 , . . . , l m ) .

Otherwise, every l i is at most S/ 2 . We choose the cutoff i ⋆ ∈ [ m ] as the smallest index such that l 1 + l 2 + · · · + l i ⋆ ≥ S/ 4 . Note that we must have l 1 + l 2 + · · · + l i ⋆ = ( l 1 + l 2 + · · · + l i ⋆ -1 )+ l i ⋆ ≤ S/ 4 + S/ 2 = 3 S/ 4 . We recursively build two trees for ( l 1 , l 2 , . . . , l i ⋆ ) and ( l i ⋆ +1 , l i ⋆ +2 , . . . , l m ) , and return the tree obtained from joining these two subtrees.

By construction, the tree has m leaf nodes, each of which corresponds to one of the m blocks. For each node v in the tree, let I ( v ) denote the set of block indices that correspond to one of the leaves in the subtree rooted at v . It is clear that every I ( v ) is of form { i, i +1 , . . . , j } for some 1 ≤ i ≤ j ≤ m . We write size( v ) := |I ( v ) | as the number of leaves in the subtree rooted at v .

Next, we assign a noise magnitude to each node in the tree.

The noise magnitude is always in [0 , 1] : it takes value 0 at the root node and takes value 1 at every leaf node. Then, we assign random values in [0 , 1] to the nodes in the tree construction as follows.

Definition 9 (Node value) . The root node r is assigned value µ r = 1 / 2 deterministically. Then, for every edge ( u, v ) in the tree, after µ u is determined, we independently draw µ v such that:

Note that the above is well-defined: By Definition 8, it always holds that σ ( u ) < σ ( v ) . So, regardless of whether µ u equals 1 -σ ( u ) 2 or 1+ σ ( u ) 2 , we always have µ u ∈ [ 1 -σ ( v ) 2 , 1+ σ ( v ) 2 ] . Therefore, there exists a unique distribution over { 1 -σ ( v ) 2 , 1+ σ ( v ) 2 } with an expectation of µ u .

Finally, we construct the hard instance by setting the value of each block to the corresponding node value in the tree construction.

Definition 10 (Hard instance) . Given a PLS instance L = ( l 1 , l 2 , . . . , l m ) , we construct a tree following Definition 7 and assign values to its nodes following Definitions 8 and 9. For each i ∈ [ m ] ,

let µ i denote the value of the leaf node that corresponds to the i -th block. Finally, the sequence x consists of l 1 copies of µ 1 , l 2 copies of µ 2 , . . . , l m copies of µ m in order.

3.3 Structural Properties

Towards proving Theorem 11 using the hard instance from Definition 10, we make some observations on the tree structure as well as the node values. We first note that, for each edge ( u, v ) in the tree, the conditional variance of µ v given µ u takes a simple form. Indeed, this is the main motivation behind the choices of the noise magnitudes and node values.

Lemma 12. For every edge ( u, v ) in the tree and conditioning on any realization of µ u , it holds that Var [ µ v | µ u ] = ln(size( u )) -ln(size( v )) 4 ln m .

Proof. Since adding a constant does not change the variance, Var [ µ v | µ u ] = Var [ µ v -1 / 2 | µ u ] = E [ ( µ v -1 / 2) 2 | µ u ] -[ E [ µ v -1 / 2 | µ u ]] 2 . By Definition 9, we have E [ µ v | µ u ] = µ u , so the second term [ E [ µ v -1 / 2 | µ u ]] 2 reduces to ( µ u -1 / 2) 2 . Then, we note that | µ u -1 / 2 | = σ ( u ) / 2 and | µ v -1 / 2 | = σ ( v ) / 2 always hold, which further simplifies Var [ µ v | µ u ] into [ σ ( v )] 2 / 4 -[ σ ( u )] 2 / 4 . Finally, the lemma follows from the choices of of σ ( u ) and σ ( v ) in Definition 8.

The next technical lemma (which we prove in Appendix C) states that, for any 1 ≤ i ≤ j ≤ m , there exists an edge ( u, v ) in the tree such that: (1) Observing the first i -1 blocks does not reveal the value of µ v ; (2) The remaining variance of µ v has a significant contribution to the average of blocks i, i +1 , . . . , j . To state the lemma succinctly, we define the 'total length' of a set S . We will mostly use this notation for S = I ( v ) (where v is a node in the tree) or S = [ i, j ] (where 1 ≤ i ≤ j ≤ m ).

Definition 11. The total length of set S is totlen( S ) := ∑ m i =1 l i · [ i ∈ S ] .

✶ Lemma 13. For any 1 ≤ i ≤ j ≤ m , there exists an edge ( u, v ) in the tree such that: (1) I ( v ) ∩ { 1 , 2 , . . . , i -1 } = ∅ ; (2) totlen( I ( v )) ≥ Ω(1) · totlen([ i, j ]) ; (3) size( v ) ≤ size( u ) / 2 .

3.4 Lower Bound in Terms of Number of Blocks

We prove Theorem 11 by putting together Lemmas 12 and 13. As in the proof of Theorem 9, we can decompose any prediction window [ t +1 , t + w ] into several complete blocks i 0 , i 0 +1 , . . . , j 0 -1 and a possibly incomplete block j 0 . If the complete blocks constitute at least half of the window, we apply Lemma 13 to identify an edge ( u, v ) in the tree, such that the noise in µ v | µ u contributes an Ω(1 / log m ) variance to the average to be predicted. Otherwise, we lower bound the variance directly by considering the edge above the leaf that corresponds to block j 0 .

Proof of Theorem 11. We consider the random sequence x ∈ { 0 , 1 } n constructed in Definition 10 and fix a forecasting algorithm A . For t ∈ T and w ∈ [ n -t ] , let E t,w denote the event that A makes a prediction at time t on X t,w := 1 w ∑ w i =1 x t + i . We will show that, conditioning on any event E t,w as well as the observations x 1: t = ( x 1 , x 2 , . . . , x t ) , the conditional variance of X t,w is at least Ω(1 / log m ) . This would lower bound the conditional expectation of the squared error incurred by A by Ω(1 / log m ) , and the theorem would then follow from the law of total expectation.

Recall that t ∈ T must be of form l 1 + l 2 + · · · + l i 0 -1 for some i 0 ∈ [ m ] . Let j 0 ∈ [ m ] be the smallest number such that l i 0 + l i 0 +1 + · · · + l j 0 ≥ w . Let δ := w -( l i 0 + l i 0 +1 + · · · + l j 0 -1 ) . Consider the following two cases, depending on whether δ exceeds half of the window length w :

Case 1: δ ≥ w/ 2 . Let v be the leaf that corresponds to the j 0 -th block in the tree construction, and u be the parent of v . Note that size( v ) = 1 and size( u ) ≥ 2 . By Lemma 12, Var [ µ v | µ u ] is given by ln(size( u )) -ln(size( v )) 4 ln m ≥ ln 2 4 ln m = Ω ( 1 log m ) . Furthermore, since event E t,w and x 1: t only depend on the realization of µ 1 , µ 2 , . . . , µ i 0 -1 , the value of µ j 0 (namely, µ v ) still has an Ω(1 / log m ) variance conditioning on E t,w and x 1: t . Then, since δ ≥ w/ 2 entries among x t +1 , x t +2 , . . . , x t + w are set to µ j 0 , Var [ X t,w | E t,w , x 1: t ] is at least 1 4 · Var [ µ j 0 | E t,w , x 1: t ] ≥ Ω(1 / log m ) .

Case 2: δ < w/ 2 . In this case, we have totlen([ i 0 , j 0 -1]) = l i 0 + l i 0 +1 + · · · + l j 0 -1 = w -δ ≥ w/ 2 . Applying Lemma 13 with i = i 0 and j = j 0 -1 gives an edge ( u, v ) such

that: (1) I ( v ) ∩ { 1 , 2 , . . . , i 0 -1 } = ∅ ; (2) totlen( I ( v )) ≥ Ω(1) · totlen([ i 0 , j 0 -1]) ; (3) size( v ) ≤ size( u ) / 2 .

By Lemma 12, Var [ µ v | µ u ] is equal to ln(size( u )) -ln(size( v )) 4 ln m ≥ ln 2 4 ln m = Ω(1 / log m ) . Since event E t,w and the observation of x 1: t only depend on the realization of µ 1 , µ 2 , . . . , µ i 0 -1 , and none of these i 0 -1 leaves is inside the subtree rooted at v , µ v still has an Ω(1 / log m ) conditional variance. Furthermore, within the lengthw prediction window, the number of entries that are affected by µ v is exactly totlen( I ( v )) ≥ Ω(1) · totlen([ i 0 , j 0 -1]) ≥ Ω(1) · w . Therefore, the conditional variance of X t,w is at least Ω(1) · Var [ µ v | E t,w , x 1: t ] ≥ Ω(1 / log m ) .

4 An Average-Case Analysis

As a warm-up towards proving Theorem 4, we analyze the special case that p ⋆ consists of n identical entries. The full proof is presented in Appendix D.

Proposition 14. Let T be a p ⋆ -random stopping time set for p ⋆ = ( p, p, . . . , p ) ∈ [0 , 1] n . With probability at least 1 -e -np/ 3 -1 /n over the randomness in T : (1) |T | ≤ 2 np = O ( np ) ; (2) ˜ U ( T ) ≥ n/ ⌈ (2 ln n ) /p ⌉ -1 = Ω( np/ log n ) .

Proof. We first note that |T | follows Binomial ( n, p ) , so a multiplicative Chernoff bound gives Pr [ |T | > 2 np ] ≤ e -np/ 3 . Towards lower bounding ˜ U ( T ) , let L = ( l 1 , l 2 , . . . , l m ) be the block representation of T and let L 0 := ⌈ (2 ln n ) /p ⌉ . By definition, ˜ U ( T ) is at least l 1 + l 2 + ··· + l m max { l 1 ,l 2 ,...,l m } = n -min T max { l 1 ,l 2 ,...,l m } . If we additionally have max { l 1 , l 2 , . . . , l m } ≤ L 0 and min T ≤ L 0 , we would have ˜ U ( T ) ≥ n -L 0 L 0 = n/ ⌈ (2 ln n ) /p ⌉ -1 . Thus, it remains to show that, with probability at least 1 -1 /n , both max { l 1 , l 2 , . . . , l m } ≤ L 0 and min T ≤ L 0 hold.

Consider the complementary event: for either max { l 1 , l 2 , . . . , l m } > L 0 or min T > L 0 to hold, there must be some t ∈ { 0 , 1 , . . . , n -L 0 -1 } such that T ∩ [ t +1 , t + L 0 ] = ∅ . For each fixed t , t +1 , t +2 , . . . , t + L 0 get included in T independently with probability p . Thus, T ∩ [ t +1 , t + L 0 ] = ∅ holds with probability (1 -p ) L 0 . By the union bound, Pr [max { l 1 , l 2 , . . . , l m } > L 0 ∨ min T > L 0 ] is at most ( n -L 0 ) · (1 -p ) L 0 , which is further upper bounded by ne -pL 0 ≤ ne -2 ln n = 1 /n using 1 -p ≤ e -p and L 0 ≥ (2 ln n ) /p . This completes the proof.

Our proof of Proposition 14 implies that, when p ⋆ consists of n copies of the same value p ∈ [0 , 1] , the resulting value of ˜ U is at least ≈ np/ log n with high probability. Furthermore, the proof of this lower bound still holds if each entry of p ⋆ is lower bounded by p instead of exactly equal to p . To prove Theorem 4, the addition step is then to show that, within each k -monotone sequence p ⋆ , we can always find a consecutive subsequence of length n ′ such that each entry is at least p ′ , and n ′ p ′ is at least ≈ ∑ n t =1 p ⋆ t / ( k log n ) . This is done by identifying a monotone subsequence in p ⋆ that contributes at least a (1 /k ) -fraction of the sum, and then finding an appropriate prefix or suffix of that subsequence.

5 Discussion

An obvious open problem is to tighten the instance-dependent error bounds (Theorems 1 and 2). Since the optimal error is Θ(1 / log n ) for the full-selectivity case [Dru13, QV19], one might hope to strengthen the 1 / [ ˜ U ( L )] 2 lower bound to 1 / log ˜ U ( L ) , or at least 1 / polylog( ˜ U ( L )) . Unfortunately, as we show in Proposition 16 (Appendix E), this is not possible: there exists a family of instances ( L k ) + ∞ k =1 such that ˜ U ( L k ) → + ∞ as k → + ∞ , but a worst-case error of O (1 / ˜ U ( L k )) can be achieved on each L k .

Therefore, to obtain tighter instance-dependent bounds, we need to identify a complexity measure that characterizes the hardness of PLS more exactly. A concrete starting point is to examine the instance family from Proposition 16, which is based on a construction that resembles the Cantor set. Roughly speaking, these instances suggest that a sharper complexity measure should account for

the number of approximately uniform blocks that can be obtained via not only merging consecutive blocks, but also 'skipping' some shorter blocks at the cost of an additional term in the prediction error.

While we focus on predicting the average of a number sequence, our results can be easily extended to the setting of predicting more general functions (including smooth and concatenation-concave functions studied by [QV19]). In particular, they imply an O (1 / log 1 / 2 ˜ U ( L )) upper bound (on the worst-case absolute error) for predicting smooth functions and an O (1 / log ˜ U ( L )) bound (on the squared error) for concatenation-concave functions. Since the average function is both smooth and concatenation-concave, the lower bounds in Theorem 2 also apply to these broader function classes.

Yet another natural direction is to revisit other classic online learning settings (such as the experts problem) from the perspective of limited selectivity, i.e., when the learner is only allowed to change its prediction or action on some given timesteps. The approximate uniformity measure as well as some of our proof techniques would be natural first steps towards understanding these models.

Acknowledgments and Disclosure of Funding

We thank the anonymous reviewers, whose suggestions have helped improved this paper.

References

[BC21] Jonah Brown-Cohen. Faster algorithms and constant lower bounds for the worst-case expected error. In Advances in Neural Information Processing Systems (NeurIPS) , pages 27709-27719, 2021.

[CDG + 18] Corinna Cortes, Giulia DeSalvo, Claudio Gentile, Mehryar Mohri, and Scott Yang. Online learning with abstention. In International Conference on Machine Learning (ICML) , pages 1059-1067, 2018.

[CVV20] Justin Chen, Gregory Valiant, and Paul Valiant. Worst-case analysis for randomly collected data. In Advances in Neural Information Processing Systems (NeurIPS) , pages 18183-18193, 2020.

[Dru13] Andrew Drucker. High-confidence predictions under adversarial uncertainty. Transactions on Computation Theory , 5(3):12, 2013.

[Fei15] Uriel Feige. Why are images smooth? In Innovations in Theoretical Computer Science (ITCS) , pages 229-236, 2015.

[FKT17] Uriel Feige, Tomer Koren, and Moshe Tennenholtz. Chasing ghosts: competing with stateful policies. SIAM Journal on Computing , 46(1):190-223, 2017.

[GHMS23] Surbhi Goel, Steve Hanneke, Shay Moran, and Abhishek Shetty. Adversarial resilience in sequential prediction via abstention. In Advances in Neural Information Processing Systems (NeurIPS) , pages 8027-8047, 2023.

[GKCS21] Aditya Gangrade, Anil Kag, Ashok Cutkosky, and Venkatesh Saligrama. Online selective classification with limited feedback. In Advances in Neural Information Processing Systems (NeurIPS) , pages 14529-14541, 2021.

[MHO25] Theodore MacMillan, James P. Hilditch, and Nicholas T. Ouellette. Expected correlation in time-series analysis. Physical Review E , 111(2):024121, 2025.

[NZ20] Gergely Neu and Nikita Zhivotovskiy. Fast rates for online prediction with abstention. In Conference on Learning Theory (COLT) , pages 3030-3048, 2020.

[PRT + 24] Stephen Pasteris, Alberto Rumi, Maximilian Thiessen, Shota Saito, Atsushi Miyauchi, Fabio Vitale, and Mark Herbster. Bandits with abstention under expert advice. In Advances in Neural Information Processing Systems (NeurIPS) , pages 126059-126087, 2024.

[QV19] Mingda Qiao and Gregory Valiant. A theory of selective prediction. In Conference on Learning Theory (COLT) , pages 2580-2594, 2019.

[QV21] Mingda Qiao and Gregory Valiant. Exponential weights algorithms for selective learning. In Conference on Learning Theory (COLT) , pages 3833-3858, 2021.

[ZC16] Chicheng Zhang and Kamalika Chaudhuri. The extended littlestone's dimension for learning with mistakes and abstentions. In Conference on Learning Theory (COLT) , pages 1584-1616, 2016.

A Proofs of Corollaries

A.1 Proof of Corollary 3

Corollary 3. For every m ≥ 1 , on the PLS instance L m = (2 0 , 2 1 , 2 2 , . . . , 2 m -1 ) with sequence length n = 2 m -1 and m blocks, every forecasting algorithm has an Ω(1) worst-case error. Furthermore, for every k ≥ 1 , there is a PLS instance L ′ k with sequence length n = 3 k and m = 2 k +1 -1 blocks, on which every forecasting algorithm has an Ω(1) worst-case error.

Proof. In light of the Ω(1 / [ ˜ U ( L )] 2 ) lower bound in Theorem 2, it suffices to show that each of these instances has an O (1) approximate uniformity.

The first family. Fix m ≥ 1 and consider L m = (2 0 , 2 1 , . . . , 2 m -1 ) . For every 1 ≤ i ≤ j ≤ m , we have

It follows that ˜ U ( L m ) ≤ 2 = O (1) .

The second family. We construct the instances L ′ 1 , L ′ 2 , . . . recursively: L ′ 1 is set to (1 , 1 , 1) . For each k ≥ 2 , L ′ k is defined as L ′ k -1 ◦ (3 k -1 ) ◦ L ′ k -1 , where ◦ denotes sequence concatenation. Then, we prove by induction on k that: (1) each L ′ k corresponds to a PLS instance with n k = 3 k and m k = 2 k +1 -1 ; (2) ˜ U ( L ′ k ) ≤ 3 .

For the base case that k = 1 , we indeed have n 1 = 3 = 3 k , m 1 = 3 = 2 k +1 -1 and ˜ U ( L ′ 1 ) = 3 . For k ≥ 2 , assuming that the statements hold for L ′ k -1 , we have n k = n k -1 +3 k -1 + n k -1 = 3 k and m k = m k -1 +1+ m k -1 = (2 k -1) + 1 + (2 k -1) = 2 k +1 -1 . To upper bound ˜ U ( L ′ k ) , consider an arbitrary contiguous subsequence ( l i , l i +1 , . . . , l j ) in L ′ k . If this subsequence contains the entry 3 k -1 in the middle, we have

If the subsequence does not contain the middle entry, it must be a contiguous subsequence of L ′ k -1 , so the upper bound

would follow from the induction hypothesis. This completes the proof.

A.2 Proof of Corollary 5

Corollary 5 follows from Theorems 1, 2 and 4.

Corollary 5. If k -monotone sequence p ⋆ ∈ [0 , 1] n satisfies ∑ n -1 t =0 p ⋆ t ≥ Ω(( k log 2 n ) 1+ c ) for some constant c > 0 , with high probability over the randomness in the p ⋆ -random stopping time set T , the forecasting algorithm from Theorem 1 has a worst-case error that is optimal up to a constant factor.

Proof. Let m 0 := ∑ n -1 t =0 p ⋆ t . Assuming that m 0 ≥ Ω(( k log 2 n ) 1+ c ) , we have m 0 = ω (log n ) and k log 2 n = O ( m 1 / (1+ c ) 0 ) . By Theorem 4, it holds with probability 1 -e -m 0 / 3 -1 /n = 1 -O (1 /n ) that |T | ≤ O ( m 0 ) and

By Theorem 1, there is a forecasting algorithm with a worst-case error of at most

on the PLS instance T . By Theorem 2, the upper bound |T | ≤ O ( m 0 ) implies that every forecasting algorithm must have an Ω(1 / log m 0 ) worst-case error on instance T . This completes the proof.

B Proofs for Section 2

B.1 Proof of Proposition 6

Proposition 6. On PLS instance L = ( l 1 , l 2 , . . . , l m ) that satisfies max { l 1 ,l 2 ,...,l m } min { l 1 ,l 2 ,...,l m } ≤ C , Algorithm 2 has a worst-case error of O ( C/ log m ) .

Proof. For integer k ≥ 1 and µ ∈ [0 , 1] , let L ( k, µ ) denote the maximum expected squared loss that Algorithm 2 incurs on a sequence of 2 k blocks with average µ . Let α := ( C +1) 2 C and ϕ ( x ) := x (1 -x ) . We prove by induction that L ( k, µ ) ≤ α k · ϕ ( µ ) . The proposition would then follow from the above together with the observations that α = O ( C ) , 1 /k = O (1 / log m ) , and ϕ ( µ ) = O (1) .

The base case. When k = 1 , Algorithm 2 calls RandomSelect (1 , 1) , which always returns ( i, j ) = (2 , 1) . Then, Algorithm 2 sets

and predict that the average of x l 1 +1 through x l 1 + l 2 is equal to ˆ µ . Then, the resulting error is given by ( µ 1 -µ 2 ) 2 , where

are the averages of the two blocks. Let c := l 2 /l 1 . Note that we have

Dividing both sides by l 1 gives µ 1 + cµ 2 = ( c +1) µ .

Then, the squared error can be upper bounded as follows:

For any µ 1 , µ 2 ∈ [0 , 1] , c > 0 and µ = 1 1+ c µ 1 + c 1+ c µ 2 , it holds that

Rearranging the above gives

where the second step holds since ϕ ( µ 1 ) and ϕ ( µ 2 ) are non-negative for any µ 1 , µ 2 ∈ [0 , 1] .

Noting that c ∈ [1 /C, C ] and that the function x ↦→ ( x +1) 2 x is increasing on [1 , + ∞ ) and decreasing on (0 , 1) , we obtain the base case

Inductive step. Now, consider the case that k ≥ 2 , assuming that the induction hypothesis holds for L ( k -1 , µ ) . Shorthand N := 2 k -1 for brevity. When Algorithm 2 calls RandomSelect (1 , k ) , with probability 1 /k , the pair ( N +1 , N ) is returned, causing Algorithm 2 to predict that the average of blocks N +1 through 2 N (denoted by µ 2 ) is the same as the average of the first N blocks (denoted by µ 1 ). In this case, the squared loss is given by ( µ 1 -µ 2 ) 2 .

With the remaining probability k -1 k , RandomSelect recurses on either the first N blocks or the last N blocks. Note that the resulting behavior of Algorithm 2 is exactly identical to when the algorithm runs on either the instance L 1 = ( l 1 , l 2 , . . . , l N ) or the instance L 2 = ( l N +1 , l N +2 , . . . , l 2 N ) , which can be controlled using L ( k -1 , · ) . Let c := ∑ N i =1 l N + i ∑ N i =1 l i . Note that we have c ∈ [1 /C, C ] , since the assumption that max { l 1 ,l 2 ,...,l m } min { l 1 ,l 2 ,...,l m } ≤ C implies

Dividing the above by ∑ N i =1 l i gives 1 /C ≤ c ≤ C . The probability of recursing on the first half is given by (1 -1 k ) · 1 1+ c , and that of recursing on the second half is (1 -1 k ) · c 1+ c . Then, by the induction hypothesis, the conditional expectation of the squared loss in this case is at most

Then, we get

(Equation (1))

where the second step applies α = ( C +1) 2 C ≥ ( c +1) 2 c , which follows from c ∈ [1 /C, C ] . This completes the inductive step and thus the proof.

B.2 Proof of Lemma 7

Lemma 7. If L ′ is a merge of L , the minimum worst-case error that can be obtained on L is smaller than or equal to that on L ′ .

Proof. Let L ′ = ( l ′ 1 , l ′ 2 , . . . , l ′ m ′ ) be a merge of L = ( l 1 , l 2 , . . . , l m ) witnessed by indices i 1 , i 2 , . . . , i m ′ +1 . Let n := l 1 + l 2 + · · · + l m and n ′ := l ′ 1 + l ′ 2 + · · · + l ′ m ′ denote the sequence lengths in L and L ′ , respectively.

Suppose that a forecasting algorithm A ′ has a worst-case error of ϵ on L ′ . Then, the following is a forecasting algorithm for L , which we denote by A :

Let t 0 := l 1 + l 2 + · · · + l i 1 -1 . Read the first t 0 elements in the sequence and disregard them.

Simulate A ′ . Whenever A ′ tries to read the next element in the sequence, read the next element and forward it to A ′ .

When A ′ predicts ˆ µ as the average of the next w elements, make the same prediction.

Clearly, whenever A runs on sequence x ∈ [0 , 1] n , it has the same behavior as A ′ running on the lengthn ′ sequence x ′ = ( x t 0 +1 , x t 0 +2 , . . . , x t 0 + n ′ ) . Therefore, the expected error of A on x is exactly the expected error of A ′ on x ′ , which is further upper bounded by ϵ .

C Proofs for Section 3

C.1 Proof of Lemma 10

Lemma 10. Let L = ( l 1 , l 2 , . . . , l m ) be a PLS instance with sequence length n and stopping time set T . Then, for every i 0 ∈ [ m ] , t = l 1 + l 2 + · · · + l i 0 -1 ∈ T and w ∈ [ n -t ] , there exists i ∈ { i 0 , i 0 +1 , . . . , m } such that | B i ∩ [ t +1 , t + w ] | ≥ w 2 ˜ U ( L ) .

Proof. Fix i 0 ∈ [ m ] , t = l 1 + l 2 + · · · + l i 0 -1 and w ∈ [ n -t ] . Recall that the i -th block is defined as B i := { l 1 + l 2 + · · · + l i -1 + j : j ∈ [ l i ] } . Let j 0 be the index of the last block that intersects [ t +1 , t + w ] . Formally, j 0 is the smallest number in { i 0 , i 0 +1 , . . . , m } such that l i 0 + l i 0 +1 + · · · + l j 0 ≥ w .

Let δ := w -( l i 0 + l i 0 +1 + · · · + l j 0 -1 ) . Note that the prediction window [ t +1 , t + w ] consists of j 0 -i 0 complete blocks ( B i 0 through B j 0 -1 ) along with the first δ timesteps in block B j 0 . We consider the following two cases, depending on whether δ exceeds half of the window length w :

Case 1: δ ≥ w/ 2 . In this case, block j 0 would satisfy the lemma. This is because j 0 ∈ { i 0 , i 0 +1 , . . . , m } and

where the last step holds since ˜ U ( L ) ≥ 1 for every L .

Case 2: δ < w/ 2 . In this case, we have l i 0 + l i 0 +1 + · · · + l j 0 -1 = w -δ ≥ w/ 2 . Furthermore, by definition of ˜ U ( L ) ,

It follows that

In particular, there exists i ∈ { i 0 , i 0 +1 , . . . , j 0 -1 } such that

C.2 Proof of Lemma 13

Lemma 13. For any 1 ≤ i ≤ j ≤ m , there exists an edge ( u, v ) in the tree such that: (1) I ( v ) ∩ { 1 , 2 , . . . , i -1 } = ∅ ; (2) totlen( I ( v )) ≥ Ω(1) · totlen([ i, j ]) ; (3) size( v ) ≤ size( u ) / 2 .

Proof. Let u 1 be the deepest node in the tree such that { i, i +1 , . . . , j } ⊆ I ( u 1 ) . Note that such a node must exist, since the root node r satisfies I ( r ) = { 1 , 2 , . . . , m } ⊇ { i, i +1 , . . . , j } . Then, we consider two cases, depending on whether I ( u 1 ) contains a long block that constitutes more than half of the total length.

Case 1: I ( u 1 ) contains a long block. Let i ⋆ ∈ I ( u 1 ) be the unique index such that l i ⋆ > totlen( I ( u 1 )) / 2 . Then, by construction of the tree (Definition 7), u 1 has a child u 2 that is a leaf node corresponding to the i ⋆ -th block. We claim that i ⋆ must be in [ i, j ] . Otherwise, by Definition 7, u 1 has two other children: one corresponding to blocks I ( u 1 ) ∩ [1 , i ⋆ -1] and the other corresponding to I ( u 1 ) ∩ [ i ⋆ + 1 , m ] . Then, one of these two children must contain the set { i, i + 1 , . . . , j } , contradicting our choice of u 1 .

Then, ( u 1 , u 2 ) would be the desired edge. For the first condition, since I ( u 2 ) = { i ⋆ } and i ⋆ ≥ i , we have I ( u 2 ) ∩ { 1 , 2 , . . . , i -1 } = ∅ . For the second, we have totlen( I ( u 2 )) = l i ⋆ > totlen( I ( u 1 )) / 2 ≥ totlen([ i, j ]) / 2 . Finally, since size( u 1 ) ≥ 2 and size( u 2 ) = 1 , it holds that size( u 2 ) ≤ size( u 1 ) / 2 .

Case 2: I ( u 1 ) has no long blocks. By Definition 7, u 1 has two children u 2 and u 3 such that I ( u 1 ) can be partitioned into I ( u 2 ) and I ( u 3 ) . It follows that { i, i +1 , . . . , j } is partitioned into I ( u 2 ) ∩ [ i, j ] and I ( u 3 ) ∩ [ i, j ] . In particular, we must have

Without loss of generality, we assume that totlen( I ( u 3 ) ∩ [ i, j ]) is at least totlen([ i, j ]) / 2 in the following; the other case can be handled in a symmetric way.

Let i ′ be the smallest number in I ( u 3 ) . Then, we have I ( u 3 ) ∩ [ i, j ] = { i ′ , i ′ +1 , . . . , j } . Let v 1 denote the deepest node in the subtree rooted at u 3 such that { i ′ , i ′ + 1 , . . . , j } ⊆ I ( v 1 ) . 2 Then, we further consider the following two subcases, depending on whether I ( v 1 ) contains a long block (compared to totlen( I ( v 1 )) ):

Case 2a: There exists i ⋆ ∈ I ( v 1 ) such that l i ⋆ > totlen( I ( v 1 )) / 2 . By Definition 7, v 1 has a child v 2 that is a leaf node corresponding to the i ⋆ -th block. Furthermore, we claim that i ⋆ must be in [ i ′ , j ] ; otherwise, we have i ⋆ ≥ j +1 , and v 1 has another child with an index set that contains { i ′ , i ′ +1 , . . . , i ⋆ -1 } ⊇ { i ′ , i ′ +1 , . . . , j } , which contradicts our choice of v 1 .

Then, ( v 1 , v 2 ) would be the desired edge: I ( v 2 ) = { i ⋆ } and i ⋆ ≥ i ′ ≥ i together give the first condition I ( v 2 ) ∩ { 1 , 2 , . . . , i -1 } = ∅ . For the second condition, we note that totlen( I ( v 2 )) = l i ⋆ > totlen( I ( v 1 )) / 2 ≥ totlen([ i ′ , j ]) / 2 ≥ totlen([ i, j ]) / 4 . Finally, the third condition follows from size( v 2 ) = 1 ≤ size( v 1 ) / 2 .

Case 2b: Every block in I ( v 1 ) has a length of at most totlen( I ( v 1 )) / 2 . By Definition 7, v 1 has a left child v 2 that satisfies

Since I ( v 1 ) contains { i ′ , i ′ +1 , . . . , j } , we have

Combining the above gives totlen( I ( v 2 )) ≥ totlen([ i, j ]) / 8 .

At this point, the edge ( v 1 , v 2 ) already satisfies the first two conditions: For the first, we note that I ( v 2 ) ⊆ { i ′ , i ′ + 1 , . . . , j } and is thus disjoint from { 1 , 2 , . . . , i -1 } . For the second, we have already shown that totlen( I ( v 2 )) ≥ totlen([ i, j ]) / 8 . However, the last condition size( v 2 ) ≤ size( v 1 ) / 2 might not hold in general.

Fortunately, this issue can be resolved via yet another case analysis. If v 2 is a leaf node, we immediately have the third condition, as size( v 2 ) = 1 ≤ size( v 1 ) / 2 . If there exists i ⋆ ∈ I ( v 2 ) such that l i ⋆ > totlen( I ( v 2 )) / 2 , v 2 would have a child v 3 that is a leaf corresponding to the i ⋆ -th block. In this case, ( v 2 , v 3 ) would be the desired edge, since totlen( I ( v 3 )) > totlen( I ( v 2 )) / 2 ≥ totlen([ i, j ]) / 16 and size( v 3 ) = 1 ≤ size( v 2 ) / 2 . Finally, if I ( v 2 ) does not contain a long block, v 2 must have two children v 3 and v 4 such that

Without loss of generality, assume that size( v 3 ) ≤ size( v 2 ) / 2 . Then, ( v 2 , v 3 ) gives the desired edge, since totlen( I ( v 3 )) ≥ totlen( I ( v 2 )) / 4 ≥ totlen([ i, j ]) / 32 .

D Proof of Theorem 4

We prove Theorem 4 in this appendix. Compared to the proof for constant probability sequences (Proposition 14), our analysis essentially reduces the k -monotone case to the constant p ⋆ case by showing that every k -monotone p ⋆ contains a sufficiently long subsequence, such that the length of the subsequence times the minimum value almost matches ∑ n -1 t =0 p ⋆ t .

Lemma 15. For any k -monotone sequence p ⋆ = ( p ⋆ 0 , p ⋆ 1 , . . . , p ⋆ n -1 ) ∈ [0 , 1] n , there exists a contiguous subsequence ( p ⋆ i , p ⋆ i +1 , . . . , p ⋆ j ) such that

Proof. By definition of k -monotone sequences, p ⋆ can be partitioned into at most k contiguous subsequences, each of which is monotone. Therefore, there exist 0 ≤ i 0 ≤ j 0 ≤ n -1 such that: (1) ( p ⋆ i 0 , p ⋆ i 0 +1 , . . . , p ⋆ j 0 ) is monotone; (2) S := ∑ j 0 t = i 0 p ⋆ t ≥ 1 k ∑ n -1 t =0 p ⋆ t .

Without loss of generality, we assume that ( p ⋆ i 0 , p ⋆ i 0 +1 , . . . , p ⋆ j 0 ) is non-decreasing; the non-increasing case can be handled symmetrically. We claim that there exists i ∈ { i 0 , i 0 +1 , . . . , j 0 } such that

where H n := 1 1 + 1 2 + · · · + 1 n = O (log n ) . Assuming this claim, ( p ⋆ i , p ⋆ i +1 , . . . , p ⋆ j 0 ) gives the desired subsequence, as it is non-decreasing and satisfies

To prove this claim, suppose towards a contradiction that, for every i ∈ { i 0 , i 0 +1 , . . . , j 0 } , it holds that ( j 0 -i +1) · p ⋆ i < S H n . It then follows that

a contradiction.

Finally, we prove Theorem 4 using Lemma 15 and an argument similar to that of Proposition 14.

Proof of Theorem 4. We start with the high-probability upper bound on |T | . Let m 0 := ∑ n -1 t =0 p ⋆ t . Since |T | is the sum of n independent Bernoulli random variables with means p ⋆ 0 , p ⋆ 1 , . . . , p ⋆ n -1 , we have E [ |T | ] = m 0 , and a multiplicative Chernoff bound gives

To lower bound ˜ U ( T ) , we apply Lemma 15 to obtain a contiguous subsequence ( p ⋆ i 0 , p ⋆ i 0 +1 , . . . , p ⋆ j 0 ) of p ⋆ such that

where p min := min { p ⋆ i 0 , p ⋆ i 0 +1 , . . . , p ⋆ j 0 } denotes the minimum entry in the subsequence. Let L 0 := ⌈ 2 ln n p min ⌉ = O ( k log 2 n ) m 0 · ( j 0 -i 0 +1) . Note that we may assume j 0 -i 0 +1 ≥ 2 L 0 without loss of generality; otherwise, we would have m 0 = O ( k log 2 n ) , in which case the Ω( m 0 / ( k log 2 n )) lower bound on ˜ U ( T ) would trivially follow from ˜ U ( T ) ≥ 1 .

̸

Good event E . Next, we define a 'good event' which will be shown to imply a lower bound on ˜ U ( T ) and happen with high probability. For each t ∈ { i 0 , i 0 +1 , . . . , j 0 -L 0 +1 } , let E t denote the event that T ∩ [ t, t + L 0 -1] = ∅ . Let E := ⋂ j 0 -L 0 +1 t = i 0 E t be the intersection of these events. In other words, E is the event that { i 0 , i 0 +1 , . . . , j 0 } contains no L 0 consecutive elements such that none of them is included in T .

Event E implies lower bound on ˜ U . When event E happens, both

are non-empty. Thus, if we list the elements in T ∩ [ i 0 , j 0 ] in increasing order:

we must have t 0 ≤ i 0 + L 0 -1 and t m ′ ≥ j 0 -L 0 +1 . Furthermore, for every neighboring entries t i -1 and t i ( i ∈ [ m ′ ] ), we must have t i -t i -1 ≤ L 0 ; otherwise, t i -1 +1 , t i -1 +2 , . . . , t i -1 + L 0 ∈ ( t t -1 , t i ) would give L 0 consecutive elements that are outside T , contradicting event E .

Let L = ( l 1 , l 2 , . . . , l m ) be the block representation of instance T . Note that L contains a contiguous subsequence

that corresponds to the m ′ +1 stopping times ( t 0 , t 1 , . . . , t m ′ ) . Then, event E implies that every entry t i -t i -1 is at most L 0 . Furthermore, the sum of these m ′ entries is given by

Therefore, by definition of ˜ U , E implies that

Finally, since L 0 = O ( k log 2 n ) m 0 · ( j 0 -i 0 +1) , the above gives a lower bound of Ω ( m 0 k log 2 n ) .

Event E happens with high probability. It remains to show that Pr [ E ] ≥ 1 -1 /n . Fix t ∈ { i 0 , i 0 + 1 , . . . , j 0 -L 0 + 1 } . For each i ∈ { 0 , 1 , . . . , L 0 -1 } , we have t + i ∈ [ i 0 , j 0 ] , which implies p ⋆ t + i ≥ min { p ⋆ i 0 , p ⋆ i 0 +1 , . . . , p ⋆ j 0 } = p min. It follows that

By the union bound, we have

This completes the proof.

E Instance with O (1 / ˜ U ( L )) Worst-Case Error

Proposition 16. For every k ≥ 2 , there is a PLS instance L such that: (1) ˜ U ( L ) = 2 k ; (2) There is a forecasting algorithm with an O (1 /k ) worst-case error on L .

Proof. Fix k ≥ 2 . We construct a sequence of instances L 1 , L 2 , . . . as follows:

L 1 = (1 , 1 , . . . , 1) is the all-one sequence of length 2 k .

For every h ≥ 2 , we set

where ◦ denotes sequence concatenation, and ( k -1) L h -1 denotes the sequence obtained from L j -1 by multiplying every entry by k -1 .

By a simple induction on h , the sum of entries in each L h is given by n h = (2 k ) h : When h = 1 , we indeed have n 1 = 2 k = (2 k ) 1 . For each h ≥ 2 , assuming n h -1 = (2 k ) h -1 , we have

Therefore, for each h ≥ 2 , the middle entry in L h , 2 · (2 k ) h -1 , constitutes a (1 /k ) -fraction of the entire sequence length n h . Before and after this middle entry are two copies of L h -1 scaled up by a factor of k -1 .

In the rest of the proof, we show that ˜ U ( L h ) = 2 k for every h ≥ 1 . Furthermore, for all sufficiently large h , L h admits a forecaster with an O (1 /k ) worst-case error. These two claims would then prove the proposition.

Analyze the approximate uniformity. We start with the direction that ˜ U ( L h ) ≥ 2 k . By construction, the first 2 k entries of L h , ( l 1 , l 2 , . . . , l 2 k ) , are exactly given by ( k -1) h -1 times L 1 . Thus, the definition of ˜ U gives

For the other direction, we show that ˜ U ( L h ) ≤ 2 k by induction on h . When h = 1 , we clearly have ˜ U ( L 1 ) = 2 k . Assuming ˜ U ( L h -1 ) ≤ 2 k , we analyze L h . Consider an arbitrary contiguous subsequence ( l i , l i +1 , . . . , l j ) in L h . If the subsequence contains the middle entry 2 · (2 k ) h -1 , we would have

Otherwise, ( l i , l i +1 , . . . , l j ) must be a contiguous subsequence of L h -1 scaled up by a factor of k -1 . It then follows from the induction hypothesis that

Therefore, ˜ U ( L h ) = 2 k holds for every h ≥ 1 .

Upper bound the worst-case error. Next, we give a forecasting algorithm for L h , which is similar to both the algorithms of [Dru13, QV19] as well as our Algorithm 2:

If h = 1 , there are 2 k blocks of equal length. Predict that the average of the last k blocks is the same as that of the first k blocks.

If h > 1 , L h consists of three parts: the left half, the middle entry 2 · (2 k ) h -1 , and the right half. With probability 1 /h , predict that the average of the right half is the same as that of the left half. With the remaining probability 1 -1 /h , run the same algorithm recursively on either the left or the right half, with equal probability.

For each h ≥ 1 and µ ∈ [0 , 1] , let L ( h, µ ) denote the highest possible squared error that the algorithm above incurs on instance L h when the sequence has an average of µ . We will prove by induction that

where ϕ ( x ) = x (1 -x ) . It then follows that, for every h ≥ k , there is a forecasting algorithm for L h with a worst-case error of O (1 /k ) = O (1 / ˜ U ( L h )) .

Base case. For the base case that h = 1 , let µ 1 and µ 2 be the averages of the two halves, respectively. Clearly, µ 1 + µ 2 = 2 µ and the algorithm incurs a squared error of ( µ 1 -µ 2 ) 2 . It follows that

We note the identity

which is a special case of Equation (1) when c = 1 . Therefore, for any µ 1 , µ 2 ∈ [0 , 1] that satisfy µ 1 + µ 2 = 2 µ , we have

Thus, we verified the base case that

Inductive step. Consider h ≥ 2 and assume that the induction hypothesis holds for L ( h -1 , µ ) . Recall that L h consists of a left half, a middle block that constitutes a (1 /k ) -fraction of the total length, and a right half. Moreover, the two halves are scaled copies of L h -1 . Let µ 1 and µ 2 denote the averages of the two halves, respectively. Let µ 0 denote the average of the middle block. Then, we have

It follows that

Therefore, we have

Plugging the induction hypothesis L ( h -1 , µ ) ≤ 4 h -1 · ϕ ( µ ) + 4 k into the above gives

Applying Equation (2) to the supremum above shows that

Finally, since ϕ ( x ) = x (1 -x ) is 1 -Lipschitz on [0 , 1] , the constraint | µ 1 + µ 2 -2 µ | ≤ 2 /k implies that ϕ ( µ 1 + µ 2 2 ) is (1 /k ) -close to ϕ ( µ ) . Therefore, we conclude that

This completes the inductive step and thus finishes the proof.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims made in the abstract and introduction are formally stated in Section 1.2, and they apply to the problem setting formally introduced in Section 1.1. They accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]

Justification: The discussion in Section 5 discusses the remaining gap in the theoretical results, and suggests directions for future work.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The assumptions are clearly stated either as part of the problem formulation (Section 1.1) or in the theorem statements (Section 1.2). All the theoretical results have proofs, in either the main paper or the appendices.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: The paper does not include experiments requiring code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work is purely theoretical and there is no immediate societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Prediction with Limited Selectivity (PLS), a variant of selective prediction where forecasters can only make predictions on a predetermined subset of timesteps (rather than any time). The authors develop a complexity measure called \"approximate uniformity\" and establish instance-dependent upper and lower bounds on the optimal error. They also provide an average-case analysis on k-monotone sequences.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1.  The proposed PLS model addresses realistic constraints where forecasters have limited selectivity, extending selective prediction to practical scenarios. The provided examples (weather forecasting, trading restrictions) are compelling and demonstrate clear real-world relevance.\n\n2. The author provides a clean theoretical framework. The introduced \"approximate uniformity\" naturally extends the fully-selective to the limited-selective scenarios and captures the hardness of a PLS instance. Using this notation, the paper establishes matching upper and lower bounds.\n\n\n### Weaknesses\n\n1. The paper is entirely theoretical with lack of empirical evidence. The motivating examples (weather, trading restrictions) are compelling, but there is no simulation demonstrating the algorithm's behavior on synthetic or real sequences. Even small-scale experiments on synthetic or real sequences would help readers gauge convergence speed and constant factors.\n\n2. There is a still large gap between upper ($O(1 / \\log (\\tilde{U}))$) and lower bounds ($\\Omega (1 / \\tilde{U}^2)$) on individual instances, which might limit the understanding on worst-case's precision. A short empirical illustration (toy instances displaying the gap) or discussion could strengthen intuition.\n\nQUESTIONS:\nSee Weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new theoretical model called Prediction with Limited Selectivity (PLS), which is a generalization of the existing selective prediction framework. In this model, a forecaster predicts the average of future values in a sequence but can only initiate a prediction at specific, predetermined times. The authors introduce a complexity measure called \"approximate uniformity\" to provide instance-dependent upper and lower bounds on the worst-case prediction error. The paper shows that for randomly generated instances, these bounds match with high probability, making them nearly tight.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThis is a well-written and theoretically sound paper that addresses a novel and practical extension of the selective prediction problem. The key strengths are:\n1. Novel Problem Formulation: The PLS model is a natural and well-motivated generalization of selective prediction. The examples provided, such as weather forecasting for outdoor activities or market predictions during specific trading seasons, effectively illustrate the real-world relevance of limited prediction opportunities.\n2. Significant Theoretical Contributions: The introduction of the \"approximate uniformity\" measure, Ũ(L), is a key contribution that provides an intuitive way to quantify the difficulty of a given PLS instance. The main results (Theorems 1 and 2) provide both upper and lower bounds on the worst-case error in terms of this measure, which is a strong theoretical result.\n3. Rigorous Analysis: The proofs are detailed and build upon prior work in a non-trivial way. For instance, the proof of the Ω(1/log m) lower bound involves a novel hierarchical decomposition using a ternary tree, extending the binary tree construction used in previous work.\n4. Average-Case Analysis: The paper doesn't stop at worst-case instance analysis. It provides an average-case analysis for randomly generated stopping time sets, showing that the derived bounds are tight with high probability under these conditions. This significantly strengthens the results.\n\nWeaknesses and Areas for Improvement:\nWhile the paper is strong, there are a few areas where it could be improved:\n\n1. Clarity on \"Hard\" Instances: The paper proves an Ω(1) worst-case error for specific PLS instances, such as those with geometrically increasing block lengths or a Cantor set-like structure. While this is a powerful result, it would be beneficial to provide more intuition as to why these structures are inherently difficult to predict. A brief discussion on how the lack of \"uniformity\" prevents the forecaster from finding a predictable scale would be helpful.\n2. Gap Between Bounds: The authors correctly point out the gap between the upper bound of O(1/log Ũ(L)) and the lower bound of Ω(1/[Ũ(L)]²). While they prove this gap is unavoidable for their complexity measure, it remains the central open question. The discussion in Section 5 is good, but it could be slightly expanded to speculate on what a more exact complexity measure might look like, perhaps drawing more from the properties of the Cantor set-based instances mentioned.\n\nMinor Notational Clarifications: The paper is generally very clear, but some notations could be briefly reiterated. For example, when discussing the proof of Theorem 11, a quick reminder of what µi (the value of the i-th block) represents would aid readability as the reader navigates complex proofs.\n\nQUESTIONS:\nQuestions for Authors:\n1. In the proof of Lemma 8, the procedure for merging blocks seems central to the main result. Could you provide a small, concrete example of this greedy procedure to help solidify the reader's understanding?\n2. The ternary tree construction in Definition 7 is a key innovation for the lower bound proof. Is there an intuitive reason why a ternary, as opposed to a binary, structure is necessary to handle blocks of varying lengths? Does it relate to the three cases a sub-problem can fall into (left, right, or dominated by a single large block)?\n3. The average-case analysis focuses on k-monotone sequences for the inclusion probabilities p*. What is the motivation behind this specific choice? Does this structure arise in practical scenarios, or is it chosen primarily for its analytical tractability?\n\nLIMITATIONS:\nNo serious limitation", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of prediction with limited selectivity (PLS), where a forecaster observes a sequence and must choose when and over what window to predict the future mean, using a restricted set of stopping times. The paper introduces approximate uniformity and show that the worst-case prediction error can be bounded in terms of it. The paper provides matching upper and lower bounds up to logarithmic factors, and show that these bounds are tight with high probability when stopping times are randomly generated from a k-monotone distribution.\n\nSTRENGTHS AND WEAKNESSES:\nI am not an expert on online learning with prediction, so I may not have assessed the strength and weakness appropriately. From what I can tell, the results and the analysis techniques are new. The techniques developed in this paper has potentials to be applied elsewhere. The high level idea of connecting locally random structure to approximate uniformity may also be useful in other online learning problems.\n\nThe main issues are 1. the algorithm seems to assume access to the block partitioning of the input (i.e., the sequence of block lengths), this is often unrealisitic in applications. 2. Theorem 1 seems to be established under some merge. But it is unclear to me how this merge might be found.\n\nQUESTIONS:\n1. The algorithm predicts the mean of future blocks by using the mean of past blocks. Is there any assumptions on stationarity of data that supports this? Why should we expect the past data be indicative of the future data?\n2. Lemma 8: The lemma only says that there exist a merge that satisfy this, but how could this merge be found? \n3. Proof of Lemma 8: T appears to be an tuning parameter to control the max/min ratio of merged block lengths to C. Why is T = M/(C-1) the right choice? \n4. Section 4 mentions that this is a warm up, but it would help to highlight what challenges arise when $p^\\ast$ is not uniform, and how these uniform-case techniques will generalize.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper addresses the following problem. In an on-line fashion, $n$ numbers from $[0,1]$ arrive one after the other. The learning agent chooses a moment $t$ to stop and a time horizon $w$ and then tries to guess the average of the next $w$ numbers. No assumption is made about the numbers, so the framework is adversarial.

It is known from the earlier work that this problem has a solution with $o(1)$ loss as $n\to\infty$ (which is quite surprising). This paper deals with the situation when there is a restricted set of possible stopping moments. The result is an upper and lower bound in terms of a combinatorial characteristic of the sequence of stopping moments. If the sequence is exponentially sparse, the worst-case error is $\Omega(1)$.

There is also a study of a probabilistic setup, where every moment can be a stopping moment with some probability. Matching upper and lower bounds are obtained for this case.

STRENGTHS AND WEAKNESSES:
I believe the paper neatly resolves a nice and clearly defined problem.


The problem is not very popular in the community and the literature review is, correspondingly, quite short. The problem, to my knowledge, has no immediate applications. Still I believe the result is nice and warrants publication.

QUESTIONS:
None

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-03