Precise Regret Bounds for Log-loss via a Truncated Bayesian Algorithm
Abstract
We study sequential general online regression, known also as sequential probability assignments, under logarithmic loss when compared against a broad class of experts.
We obtain tight, often matching, lower and upper bounds for sequential minimax regret, which is defined as the excess loss incurred by the predictor over the best expert in the class.
After proving a general upper bound we consider some specific classes of experts from Lipschitz class to bounded Hessian class and derive matching lower and upper bounds with provably optimal constants.
Our bounds work for a wide range of values of the data dimension and the number of rounds.
To derive lower bounds, we use tools from information theory (e.g., Shtarkov sum), and for upper bounds we resort to new “smooth truncated covering” of the class of experts.
This allows us to find constructive proofs by applying a simple and novel truncated Bayesian algorithm.
Our proofs are substantially simpler than the existing ones and yet provide tighter (and often optimal) bounds.
Introduction
In online learning and sequential probability assignments arising in information theory, portfolio optimization, and machine learning, the training algorithm consumes d dimensional data in rounds t ∈ { 1 , 2 , . . .
T } and predicts the label ˆ y t based on data received and labels observed so far.
After prediction, the true label y t is revealed and the loss ℓ ( y t , ˆ y t ) is incurred.
The (pointwise) regret is defined as the (excess) loss incurred by the algorithm over a class of experts, also called the hypothesis class.
More precisely, in each round t ≥ 1 the learner obtains a d dimensional input/ feature vector x t ∈ R d .
In addition to x t the learner may use the past observations ( x r , y r ) , r < t to make a prediction ˆ y t of true label.
Therefore, the prediction can be written as ˆ y t = ϕ t
( y t − 1 , x t ) , where y t − 1 represents the labels in the past t − 1 rounds, x t represents the input vectors in t rounds, and ϕ t represents the strategy of the learner to obtain its prediction based on the past and current observations.
Once a prediction is made, nature reveals the true label y t and the learner incurs loss ℓ : ˆ Y × Y → R , where ˆ Y and Y are the prediction and true label domains respectively.
Hereafter, we assume throughout ˆ
Y = [0 , 1] , Y = { 0 , 1 } with logarithmic loss
[Equation]
In regret analysis, we are interested in comparing the accumulated loss of the learner with that of the best strategy within a predefined class of predictors (experts) denoted by H .
More precisely, H is a collection of predicting functions h : R d 7→ ˆ Y with input being x t at time t .
Therefore, given a learner ϕ t , t > 0 and ( y t , x t ) Tt =1 after T rounds the pointwise regret is defined as
[Equation]
[Equation]
where ˆ y t = ϕ t ( y t − 1 , x t ) .
Observe that the first term above represents the accumulated loss incurred by the learning algorithm, while the second summation deals with the best prediction within H .
We highlight two useful perspectives on analyzing the regret next.
Fixed Design: This point of view studies the minimal regret for the worst realization of the label with the feature vector x T known in advance.
Suppose that the learner has a fixed strategy ϕ t , t > 0 .
Then, the fixed design minimax regret for a given x T is defined as
[Equation]
Further, the fixed design maximal minimax regret is given by:
[Equation]
Sequential Design: In this paper we mostly focus on the sequential or agnostic regret in which the optimization on regret is performed at each time t without knowing in advance x T or y T .
Then the sequential (maximal) minimax regret is given by [24]:
[Equation]
In [37] it is shown that r aT ( H ) ≥ r ∗ T ( H ) for all H .
We will use r ∗ T ( H ) as our tool to derive lower bounds for r aT ( H ) .
Our main goal is to gain insights into the growth of sequential regret r aT ( H ) for various classes H and to show how the structure of H , as well as the relationship between d and T impact the precise growth of the regret.
To see this more clearly, we briefly review regret in universal source coding.
Regrets in Information Theory.
In universal compression, the dependence between regret and the reference class was intensively studied [10, 21, 27, 28, 32, 38, 39].
Here, there is no feature vector x t , and the dimension d = 1 .
A sequence y T is generated by a source P that belongs to a class of sources S , which can be viewed as the reference class H in online learning.
The minimax regret for the logarithmic loss is given by [9, 31, 10]:
[Equation]
where Q is the universal probability assignment approximating the unknown P .
The main question is how the structure of S impacts the growth of the minimax regret.
Let m denote the alphabet size (in online learning, we only consider m = 2 ).
It is known [10, 21, 27, 28, 32, 38, 39] that for Markov sources of order r , regret grows as m r ( m − 1) / 2 log T for fixed m
[27, 21, 28, 33], while in [33] minimax regret was analyzed for all ranges of m and T .
For non-Markovian sources, the growth is super logarithmic.
For example, for renewal sources of order r the regret is Θ( T r/ ( r +1) )
[7] and the precise constant in front of the leading term is know for r = 1 [11].
However, it should be pointed out that [6, 1] studied the general classes of densities smoothly parameterized by a d -dimensional data to obtain general results for minimax regret that can be phrased as an online regret.
Main Contributions.
Our main results are summarized in Table 1.
One of the main contributions of this paper is the concept of a global sequential covering used to prove constructively general upper bounds on regret (Theorem 1).
We establish Theorem 1 via a novel smooth truncation approach enabling us to find tight upper bounds that subsume the state-of-the-art results (e.g., [23, 3]) obtained non-algorithmically.
In fact, Algorithm 2 developed in this paper achieves these bounds.
Moreover,
Theorem 1 provides optimal constants that are crucial to derive the best bounds in special cases discussed next.
For general Lipschitz parametric class H , in Theorem 2, we derive the upper bound d log( T/d ) + O ( d ) for T > d .
In Theorem 3, we show that the leading constant 1 (in front of d log( T/d ) ) is optimal for T ≫ d log( T ) .
Furthermore, we obtain the best constant for the leading term d 2 log( T/d ) when the Hessian of log f is bounded for any function f ∈ H (see Theorem 4).
Then, we show in Theorem 5 that the constant 12 in our bound is optimal for functions of the form f ( ⟨ w , x ⟩ ) , where w ∈ R d is the parameter of the function, ⟨ w , x ⟩ is the inner product in R d , w and x are in a general ℓ s -norm unite ball, and T ≫ d ( s +2)
/s .
This result recovers all the lower bounds in [29] obtained for logistic regression (however, the technique of [29] works for other functions with bounded second derivatives, like the probit function).
Lastly, when d ≥ T , for a linear function of the form |⟨ w , x ⟩| we show that the growth is at least Ω( T s/ ( s +1) ) under ℓ s ball and
at most ˜ O
( T 2 / 3 ) for ℓ 2 ball (see Theorem 6 and Example 2).
The main technique used in our paper (smooth truncation) is novel with other potential applications (e.g., average minimax regret).
Instead of the conventional approach for truncating only the values close to { 0 , 1 } , we truncate all values in [0 , 1] in a smooth way (see Lemma 4).
This allows us to obtain an upper bound via a simple truncated Bayesian algorithm.
Our proofs are substantially simpler (cf.
[3]) yet provide tighter and often optimal bounds.
In summary, our main contributions are: (i) constructive proofs through a new smooth truncated
Bayesian algorithm; (ii) the novel application of global sequential covering in the context of logarithmic loss; (iii) lower and upper bounds with optimal leading constants; and (iv) novel informationtheoretic techniques for the lower bounds.
[Table]
f
Any r aT ( H f ) ≤ d log (cid:0)
Td +(cid:1) + O ( d )
Theoremunder ℓ s ball
T ≫ d log T r aT ( H f ) ≥ d log (cid:0)
Td (cid:1) − O
( d log log T ) Theorem
Bounded Hessian
Any r aT ( H f ) ≤ d 2 log (cid:18)
T d +(cid:19) + O ( d )
Theorem 4 of log f under ℓ 2 ball f ( ⟨ w , x ⟩ ) with f ′ (0) ̸
{+= 
 T ≫ d (+} s +2)
/s r aT ( H f ) ≥ d 2 log
[Equation]
T d
[Equation]
O ( d )
Theorem 5 under ℓ s ball |⟨ w , x ⟩| under ℓ s ball
[Equation]
Theorem|⟨ w , x ⟩| under ℓ 2 ball
[Equation]
Example
Related Work
In this paper we study sequential minimax regret for general online regression with logarithmic loss using tools of information theory, in particular universal source coding (lower bounds)
[1, 10, 18, 21, 26, 27, 28, 38] and sequential covering (upper bounds).
Most of the existing works in online regression deals with logistic regression.
We first mention the work of [13], who studied pointwise regret of logistic regression for the proper setting.
Unlike improper learning, studied in this paper, where feature x t at time t is also available to the learner, [13] showed that pointwise regret is Θ( T 1 / 3 ) for d = 1 and O ( √ T ) for d > 1 .
Furthermore, [17] demonstrates that regret for logistic regression grows as O ( d log T/d ) .
This was further generalized in [12].
These results were strengthened in [29], which also provides matching lower bounds.
Precise asymptotics for the fixed design minimax regret were recently presented in [14, 15].
Regret bounds under logarithmic loss for general expert class H was first investigated by Vovk under the framework of mixable losses [16, 34].
In particular, Vovk showed that for finite class H , the regret growth is log |H| via the aggregating algorithm (i.e., the Bayesian algorithm that we will discuss below).
We refer the reader to [5, Chapter 3.5, 3.6] and the references therein for more results on this topic.
Cesa-Bianchi and Lugosi [5] were the first to investigate log-loss under general (infinite) expert class H [5, Chapter 9.10, 9.11], where they derived a general upper bound using the concept of covering number and a two-stage prediction scheme.
In particular, Cesa-Bianchi and Lugosi showed that for Lipschitz parametric classes with values bounded away from { 0 , 1 } , one can achieve a regret bound of the form d/ 2 log( T/d ) .
When the values are close to { 0 , 1 } , they used a hard truncation approach, which gives a sub-optimal bound of the form 3 / 2 d log( T/d ) (i.e., this bound is not explicitly shown in [5] but can be derived using their approach).
Moreover, the approach of [5] only works for fixed design regret (or simulatable in their context).
In [23], the authors extended the result of [5, Chapter 9.10] to the sequential case via the machinery of sequential covering that was established in [22].
However, [23] also used the same hard truncation as in [5] resulting in suboptimal upper bounds.
In [3], the authors obtained an upper bound similar to the upper bound presented in
Theorem 1 via the observation that the log function is self-concordant.
In particular, this allows them to resolve the tight bounds for non-parametric Lipschitz functions that map [0 , 1] s → [0 , 1] .
However, their bounds are proved non-constructively , i.e., the proof does not provide an algorithm that achieves such bounds.
In [4], the authors used a similar idea of smoothing for controlling the unboundedness of log-loss, however, they are assumed that the features x T are presented i.i.d. .
More importantly, the results in [4] only holds for the average case regret.
Problem Formulation and Preliminaries
We denote X as the input feature space and H as the concept class, which is a set of functions mapping
X → [0 , 1] .
We use an auxiliary set W to index H .
We say that a function g is sequential if it maps
X ∗ → [0 , 1] , where X ∗ is set of all finite sequences with elements in X .
We denote G as a class of sequential functions.
If T is a time horizon, then for any t ∈ [ T ] , we write x t = { x 1 , · · · , x t } and y t = { y 1 , · · · , y t } .
We use standard asymptotic notation f ( t ) =
O ( g ( t )) if there exists a constant C such that f ( t ) ≤ Cg ( t ) for sufficient large t ≥ 0 , and f ( t ) ≪ g ( t ) if lim sup t →∞ f ( t ) /g ( t ) = 0 .
We assume the log function log( x ) is the nature logarithm to the base of e .
The main objective of this paper is to study the growth of the sequential minimax regret r aT ( H ) for a large class of experts H .
We accomplish this goal using two different techniques.
For the lower bound, we precisely estimate the fixed design minimax regret r ∗ T ( H| x T ) using the Shtarkov sum [31], discussed next.
For the upper bound, we construct a global cover set G of H and design a new (truncated)
Bayesian algorithm to find precise bounds with constants that are provably optimal.
Lower Bounds.
We investigate the lower bound of adversarial regret r aT ( H ) by considering its corresponding fixed design minimax regret r ∗ T ( H| x T ) and r ∗ T ( H ) =
max x T r ∗ T ( H| x T ) .
We are able to do this due to the recent result [37], which we quote next.
Lemma 1 (Wu et al., 2022) .
Let H be any general hypothesis class and ℓ be any loss function.
Then
[Equation]
, and the inequality is strict for certain H , and loss function ℓ .
We establish precise growth of r ∗ T ( H ) by estimating the Shtarkov sum that was intensively analyzed in information theory [31, 10] and recently applied in online learning [30, 14].
For the logarithmic loss, the Shtarkov sum (conditioned on x T ) is defined as follows
[Equation]
where P h ( y T | x T ) =
Q
Tt =1 h ( x t )
y t (1 − h ( x t )) 1 − y t
and we interpret h ( x t ) =
P ( y t = 1 | x t ) .
The regret can be expressed in terms of the Shtarkov sum (see [14, Equation (6)] or [5, Theorem 9.1]) as
[Equation]
It is known that the leading term in the Shtarkov sum for parametric classes H is often independent of x T [29, 12, 14, 15].
Therefore, the Shtarkov sum often gives the leading growth of r ∗ T ( H| x T ) independent of x T , which also suggests the leading growth of the agnostic regret r aT ( H ) .
Upper Bounds.
We now discuss our constructive approach to upper bounds.
In the next section, we present our Smooth truncated Algorithm (Algorithm 2) that provides a constructive and often achievable upper bound.
Here we focus on some, mostly known, preliminaries.
Let G ⊂ [0 , 1] X ∗ be any reference class.
Let W be an index set of G and µ be an arbitrary finite measure over W .
The standard Bayesian predictor with prior µ is presented in Algorithm 1.
Based on this algorithm, we have the following two lemmas that are used to establish most of the upper bounds in this paper.
See e.g., [19, Lemma 3] or [5, Chapter 3.3] for proofs.
[Equation]
Lemma 2.
Let G be a collection of functions
g w : X ∗ → [0 , 1] , w ∈ W .
Let ˆ y t be the Bayesian prediction rule as in Step 4 of Algorithm 1 with prior µ .
Then, for any x T and y T we have
[Equation]
R
[Equation]
1d µ
[Equation]
where p w ( y T | x T ) =
e −
P
Tt =1 ℓ ( g w ( x t ) ,y t ) and ℓ is the log-loss as in equation (1).
The following lemma bounds the regret under log-loss of finite classes, which is well known.
Lemma 3.
For any finite class of experts G , we have r aT ( G ) ≤ log |G| .
Main Results
We start with a concept of covering set called the global sequential cover that was used implicitly in [24, Section 6.1] for the Lipschitz losses and dated back to the ideas in [2].
Definition 1 (Global sequential covering) .
For any H ⊂ [0 , 1] X , we say that class of sequential functions G ⊂ [0 , 1] X ∗ is a global sequential α -covering of H at scale T if for any x T ∈ X T and h ∈ H , there exists g ∈ G
such that ∀ t ∈
[ T ] ,
[Equation]
Throughout we assume that 0 ≤ α ≤ 1 .
Note that the global sequential covering is different from the (local) sequential covering used in [3] (originally from [24]), since our covering function does not depend on the underlying trees as in [24] 2 .
This is crucial to apply our covering set directly in an algorithmic way (see Algorithm 2).
Particularly, it enables us to establish our lower and upper bounds for Lipschitz classes of functions with the optimal constants on the leading term.
We further improve these results for Lipschitz class with bounded Hessian.
Finally, we study cases when the data dimension d grows faster than T by bounding the covering size through the sequential fat-shattering number.
In particular, we prove matching (up to poly log T factor) upper and lower bounds for the generalized linear functions.
General Results.
We are now in the position to state our first main general finding.
Theorem 1.
If for any α > 0 there exists a global sequential α -covering set G α of H , then
[Equation]
and this bound is achievable by Algorithm 2.
We should point out that Theorem 1 also improves the results of [3] by obtaining better constants in front of both αT and log |G
α | (i.e., from (4 , 4) to (2 , 1) ).
The proof is based on the following key lemma that is established in Appendix A.
Input : Reference class G with index set W and prior µ over W , and truncation parameter α 1: Let p w ( y 0 | x 0 ) =
1 for all w ∈ W 2: for t = 1 , · · · , T do 3:
Receive feature x t 4:
For all w ∈ W , set
[Equation]
Make prediction
[Equation]
Receive label y t 7:
For all w ∈ W , update: p w ( y t | x t ) =
[Equation] [Equation]
( x
[Equation]
y
[Equation]
p
[Equation]
[Equation]
x
[Equation]
8: end for
Lemma 4.
Suppose H has a global sequential α -covering set G for some α ∈ [0 , 1] .
Then, there exists a truncated set ˜ G of G with | ˜
G| = |G| such that for all x T , y T and h ∈ H there exists a ˜ g ∈ ˜
G satisfying
[Equation]
[Equation]
where p h ( y T | x T ) =
T
[Equation]
and p ˜
[Equation]
T | x T ) =
[Equation]
x
[Equation]
Proof of Theorem 1.
We show that for any 0 ≤ α ≤ 1 if an α -covering set G α exists, then one can achieve the claimed bound for such an α .
To do so, we run the Smooth truncated Bayesian Algorithm (Algorithm 2) on G α with uniform prior and truncation parameter α .
We denote by ˜
G
α the truncated class of G α as in Lemma 4 (same as the step 4 of Algorithm 2).
We now fix x T , y T .
By Lemma(with G being ˜ G α ), we have
[Equation]
the last equality follows from |G α | = | ˜
G α | .
Since
P
Tt =1 ℓ ( f ( x t ) , y t ) =
− log p f
[Equation]
y T | x T ) for any function f , then by Lemma 4 we conclude that
[Equation]
The result follows by combining the inequalities and noticing that log(1+ x ) ≤ x for all x ≥ − 1 .
We further note that for any constant c 1 , c 2 for which the bound r aT ( H ) ≤ c 1 αT + c 2 log |G
α | holds universally we must have c 1 ≥ 2 and c 2 ≥ 1 .
Therefore, our bounds are optimal with respect to the constants 3 .
To see this, we let X =
[ T ] and define g to be the function that maps every t ∈ [ T ] to 12 .
Let H be the class of functions that maps to [1 / 2 − α, 1 / [-
-]2 + α ] .
Clearly, H is α -covered by g .
By noting that the maximum probability is (1 / 2 + α ) T = (1 + 2 α ) T (1 / 2) T , we compute the Shtarkov sum (5) to get:
[Equation]
where ∼ holds when α is sufficiently small.
This implies that we must have c 1 ≥ 2 .
The fact that c 2 ≥ 1 is due to the fact that mixability constant of log-loss is 1 , which also follows from Theorembelow.
Lipschitz Parametric Class.
We now consider a Lipschitz parametric function class.
Given a function f : W × X → [0 , 1] , define the following class
[Equation]
where w ∈ W is often a vector in R d .
We will assume that f ( w , x ) is L -Lipschitz on w for every x , where L ∈ R + .
More formally, ∀ w 1 , w 2 ∈ W and x ∈ X we have
[Equation]
where || · || is some norm on W .
For example, if we take W ⊂ R d then the norm can be ℓ 1 , ℓ 2 or ℓ ∞ norm.
For any specific norm || · || , we write B ( R ) for the ball under such norm with radius R in W .
In particular, we denote by B ds ( R ) the ball in R d of radius R under ℓ s norm centered at the origin.
Theorem 2.
Let f : B ds ( R ) × R d → [0 , 1] be a L -Lipschitz function under ℓ s norm.
Then
[Equation]
Proof.
By L -Lipschitz condition, to find an α -covering in the sense of Definition 1, we only need to find a covering of B ds ( R ) with radius α/L .
By standard result (see e.g. Lemma 5.7 and Example 5.
of [35]) we know that the covering size is upper bounded by
[Equation]
By Theorem 1, we find
[Equation]
Taking α = d/T , we conclude
[Equation]
This completes the proof for T ≥ d .
The upper bound T is achieved by predicting 12 every time.
Example 1.
For logistic function f ( w , x ) =
(1 + e −⟨ w , x ⟩ ) − 1 , and w ∈ B d 2 ( R ) with x ∈ B d 2 (1) our result recovers those of [12], but with a better leading constant (the bound in [12] has a constant 5 ).
Note that, the result in [3] also provides a sub-optimal constant c ∼ 4 .
Moreover, our bounds have a logarithmic dependency on Lipschitz constant L .
The question arises whether the factor in front of log T can be improved to d/ 2 instead of d as discussed in some recent papers [29, 14, 15].
In Theorem 3 below, we show that, in general, it cannot unless we further strengthen our assumption (see Theorem 4).
For the ease of presentation, we only consider the parameters restricted to ℓ 2 norm.
The proof can be found in Appendix B.
Theorem 3.
For any d, T, R, L such that T ≫ d log( RLT ) , there exists L -Lipschitz function f : B d 2 ( R ) × R d → [0 , 1] such that
[Equation]
Lipschitz Class with Bounded Hessian.
As we have demonstrated in Theorem 3 the leading constant 1 of the regret for Lipschitz parametric classes can not be improved in general.
We now show that for some special function f one can improve the constant to 12 , as already noticed in [29, 14, 15].
For any function f : R d × R d → [0 , 1] , we say the Hessian of log f is uniformly bounded on X ⊂ R d , if there exists constant C such that for any w ∈ R d and x ∈ X and y ∈ { 0 , 1 } we have
[Equation]
where ∇ 2 w is the Hessian at w .
The proof of the next theorem can be found in Appendix C.
Theorem 4.
Let f : R d × R d → [0 , 1] be a function such that the Hessian of log f is uniformly bounded by C on X .
Let
[Equation]
be such a class of f restricted to some compact set W ⊂ R d .
Then
[Equation]
where W ∗ = { w + u | w ∈ W , u ∈ B d 2 ( p d/CT ) } , Vol ( · ) is volume under Lebesgue measure.
In particular, for W = B
d 2 ( R ) , we have
[Equation]
Note that, Theorem 4 subsumes the results of [17, 29] 4 , where the authors considered functions of form f ( ⟨ w , x ⟩ ) and requires that the second derivative of log f is bounded, see also [5, Chapter 11.10].
However, the KL-divergence-based argument of [17] can not be used directly in the setup of
Theorem 4 since we do not assume the function f has a linear structure.
Our main proof technique of
Theorem 4 is a direct application of Lemma 2 and an estimation of the integrals via Taylor expansion; see Appendix C for more details on the proof.
Finally, we complete this part with the following lower bound for generalized linear functions under unit ℓ s balls.
See Appendix D for proof.
Theorem 5.
Let f : R → [0 , 1] be an arbitrary function such that there exists c 1 , c 2 ∈ (0 , 1) and for all r > 0 we have [ c 1 − c 2 d − r ,
c
1 + c 2 d − r ] ⊂ f
([ − d − r , d − r ]) for sufficiently large d .
Let
[Equation]
where s > 0 .
Then
[Equation]
where O hides some absolute constant that is independent of d, T .
Note that for the logistic function f ( x ) =
(1 + e − x ) − 1 Theorem 5 holds with c 1 = 12 and c 2 = 15 .
Therefore,
If s = 1 , then
[Equation]
If s = 2 , then
[Equation]
If s = ∞ , then
[Equation]
This recovers all the lower bounds from [29].
We note that a simple sufficient condition for Theoremto hold is to require f ′ (0) ̸
= 0 if f ( x ) is differentiable.
Large Growth.
We now present some results for large d growing even faster than T .
We will show that the size of global sequential covering (Definition 1) of a class H can be bounded by the sequential fat-shattering number of H in a similar fashion as in [24].
We first introduce the notion of sequential fat-shattering number as in [24].
We denote { 0 , 1 } d ∗ to be the set of all binary sequences of length less than or equal to d .
A binary tree of depth d with labels in X is defined to be a map τ : { 0 , 1 } d ∗ → X .
For any function class
H ⊂ [0 , 1] X , we say H
α -fat shatters tree τ if there exists [0 , 1] -value tree s : { 0 , 1 } d ∗ → [0 , 1] such that for any binary sequence ϵ d 1 ∈ { 0 , 1 } d ∗ there exist h ∈ H such that for all t ∈ [ d ] :
If ϵ t = 0 , then h ( τ ( ϵ t − 1 1 )) ≤ s ( ϵ t − 1 1 ) − α ;
If ϵ t = 1 , then h ( τ ( ϵ t − 1 1 )) ≥ s ( ϵ t − 1 1 ) + α.
Definition 2.
The sequential α -fat shattering number of H is defined to be the maximum number d ( α ) such that H α -fat shatters a tree τ of depth d :
= d ( α ) .
In the lemma below, we present an upper bound for the cardinality of the global covering with algorithmically constructed cover set G α , see e.g., [24, Section 6.1].
We provide a proof in Appendix E for completeness.
Lemma 5.
Let H be any class map X
→
[0 , 1] and d ( α ) be the sequential α -fat shattering number of H .
Then there exists a global sequential α -covering set G α of H as in Definition 1 such that
[Equation]
Example 2.
By [24] we know that the sequential α -fat shattering number of linear functions f ( w , x ) = |⟨
w , x ⟩| with w , x ∈ B d 2 (1) is of order ˜ O
( α − 2 ) where in ˜
O we hide a polylog factor.
Lemma 5 implies that the global sequential α -covering number is upper bounded by (cid:24)
[Equation]
By Theorem 1, we have
[Equation]
by taking α = T − 1 / 3 .
This bound is independent of the data dimension d .
Remark 1.
Observe that for any class H with sequential fat-shattering number of order α − s one can achieve a regret upper bound of order ˜ O
( T s/s +1 ) by Theorem 1.
We refer to [24, 25] for the estimations of sequential fat-shattering number of a variety of classes.
Finally, we present the following general lower bound.
See Appendix F for proof.
Theorem 6.
For any s ≥ 1 , we define
[Equation]
We can view the vectors in D s as functions mapping [ T ] → [0 , 1] .
Then
[Equation]
To see why Theorem 6 implies a lower bound for f ( w , x ) = |⟨ w , x ⟩| with d ≥ T , as in Example 2, we take w , x ∈ B T 2 (1) (i.e., with d = T ) and define x t = e t with e t being the standard base of R T that takes value 1 at position t and zeros otherwise.
Note that the functions of H f with f ( w , x ) = |⟨ w , x ⟩| restricted on x T is exactly D 2 .
Then
[Equation]
and this is a matching lower bound of Example 2.
Note that, it is proved in [23] that for function f ( w , x ) = ⟨ w , x ⟩ +1, one can achieve the regret of form ˜ O ( √ T ) 5 .
Example 2 implies that the generalized linear functions of form f ( ⟨ w , x ⟩ ) can have different regrets with polynomial gap even with a simple shift on the value (though they have the same covering number).
It is therefore an interesting open problem to investigate a tighter complexity measure (instead of a covering number as in Definition 1) that captures this phenomenon.
Conclusion
In this paper, we presented best known lower and upper bounds on sequential online regret for a large class of experts.
We accomplish this by designing a new smooth truncated Bayesian algorithm, together with the concept of global sequential covering, that achieves these upper bounds.
For the lower bound, we use a novel information-theoretic approach based on the Shtarkov sum.
We expect that these techniques can be generalized to a broader set of problems, e.g., when the features x T is present stochastically.
We leave these to the future investigations.
Acknowledgments
This work was partially supported by the NSF Center for Science of Information (CSoI)
Grant
CCF-0939370, by NSF Grants CCF-2006440, CCF-2007238, and CCF- 2211423, and in addition by
Google Research Grant and by Rolls Royce.