Title: Smoothed Analysis of Sequential Probability Assignment
Venue: NeurIPS 2023
Accepted: yes
Keywords: Online learning

Log loss

Information theory

Smoothed Analysis

Beyond worst case analysis

Oracle Efficient Online Learning
Abstract: We initiate the study of smoothed analysis for the sequential probability assignment problem with contexts. We study information-theoretically optimal minmax rates as well as a framework for algorithmic reduction involving the maximum likelihood estimator oracle. Our approach establishes a general-purpose reduction from minimax rates for sequential probability assignment for smoothed adversaries to minimax rates for transductive learning. This leads to optimal (logarithmic) fast rates for parametric classes and classes with finite VC dimension. On the algorithmic front, we develop an algorithm that efficiently taps into the MLE oracle, for general classes of functions. We show that under general conditions this algorithmic approach yields sublinear regret.

Smoothed Analysis of Sequential Probability Assignment

Abstract

We initiate the study of smoothed analysis for the sequential probability assignment problem with contexts. We study information-theoretically optimal minmax rates as well as a framework for algorithmic reduction involving the maximum likelihood estimator oracle. Our approach establishes a general-purpose reduction from minimax rates for sequential probability assignment for smoothed adversaries to minimax rates for transductive learning. This leads to optimal (logarithmic) fast rates for parametric classes and classes with finite VC dimension. On the algorithmic front, we develop an algorithm that efficiently taps into the MLE oracle, for general classes of functions. We show that under general conditions this algorithmic approach yields sublinear regret.

1 Introduction

Sequential probability assignment - also known as online learning under the logarithmic loss - is a fundamental problem with far-reaching impact on information theory, statistics, finance, optimization, and sequential decision making [Rissanen, 1983, 1984, Cover, 1991, Feder et al., 1992, Xie and Barron, 1997, Merhav and Feder, 1998, Xie and Barron, 2000, Yang and Barron, 1999, Jiao et al., 2013, Orabona and P´ al, 2016, Foster et al., 2018]. In recent years, methods for incorporating contexts or side information into sequential probability assignment have gained much attention [Rakhlin and Sridharan, 2015, Fogel and Feder, 2017, 2018, Foster et al., 2018, Bhatt and Kim, 2021, Bilodeau et al., 2021, Wu et al., 2022a], in part due to their newly forged connection to sequential decision making applications, the contextual bandit problem, and learning in Markov Decision Processes (MDPs) (see e.g. [Foster and Krishnamurthy, 2021] and [Foster et al., 2021a]). In this setting, a forecaster who has access to historical data x 1: t -1 , y 1: t -1 consisting of contexts x τ (e.g., day τ 's meteorological information) and the outcomes y τ ∈ { 0 , 1 } (e.g., whether τ was a rainy day) wishes to predict y t given a new context x t . The forecaster uses a probability assignment p t to estimate the probability of y t = 1 outcome and incurs the logarithmic loss, i.e., -log p t ( y t ) , which rewards the forecasters for having assigned high probability to the realized outcome. The goal of the forecaster is to suffer low regret against a chosen reference class of predictors.

Alarge body of prior work on sequential probability assignment with contexts has focused on settings where contexts are presented i.i.d. from an unknown distribution (see [Fogel and Feder, 2017, Bhatt and Kim, 2021, Bilodeau et al., 2021, Wu et al., 2022a] and the references within); this problem is also referred to as conditional density estimation. In these cases, sequential probability assignment is known to enjoy small regret for several reference classes such as Vapnik-Chervonenkis (VC) classes. On the other hand, attempts to consider context distributions that evolve unpredictably and

adversarially have faced strong impossibility results even for simple reference classes of predictors. For example, for the reference class of simple one-dimensional thresholds assigning p t = θ 0 1 { x t ≤ a } + θ 1 1 { x t > a } for a ∈ [0 , 1] , regret is bounded by O (log T ) in the i.i.d. case [Fogel and Feder, 2017] but is lower bounded by Ω( T ) when the sequence of contexts is chosen adversarially (folklore e.g. Littlestone [1988]). In the face of the increasing need to adapt to evolving contexts in modern applications, these impossibility results indicate that new models of adversarial behavior must be considered for obtaining rigorous guarantees that guide the design of sequential probability assignment in practical applications.

In recent years, smoothed analysis of adaptive adversaries [Haghtalab et al., 2021, Rakhlin et al., 2011] has emerged as a framework for going beyond the worst-case adversaries while making minimal assumptions about the adaptive process that generates a sequence. In this setting, contexts are chosen from an evolving sequence of so-called σ -smooth distributions, whose density is bounded above by 1 /σ times that of a base measure (such as the uniform distribution). Remarkably, these methods, established by Haghtalab et al. [2021] for 0 -1 loss and extended to regression by Haghtalab et al. [2022], Block et al. [2022], have established performance guarantees for the sequential prediction problem that matches the optimal performance in the i.i.d setting. This raises the question as to whether the sequential probability assignment problem may similarly enjoy improved minmax regret bounds for smoothed adaptive sequences.

Beyond minmax rates, an important feature of probability assignment and, its analogue, density estimation is the availability of fundamental and natural estimation techniques such as maximum likelihood estimation (MLE). For i.i.d. sequences, under general conditions, MLE is known to be optimal asymptotically and often serves as a starting point for designing more sophisticated estimators. Going beyond i.i.d. sequences, we ask whether MLE can be made to achieve good statistical behavior on adaptive sequences. More generally, algorithmic perspective is increasingly important for the sequential probability assignment problem and its applications to contextual bandits and reinforcement learning (where algorithm design is as fundamental a consideration as minmax rates [Agarwal et al., 2014, Simchi-Levi and Xu, 2022, Foster and Rakhlin, 2020, Langford et al., 2007, Foster et al., 2021b]). In this space, oracle-efficient sequential decision making algorithms that repurpose existing offline algorithmic routines have received special interest [Kalai and Vempala, 2005, Dud´ ık et al., 2020, Wang et al., 2022, Kakade et al., 2007, Simchi-Levi and Xu, 2022]. Here again, recent progress on smoothed analysis for sequential prediction with 0 -1 loss and regression loss [Haghtalab et al., 2021, Block et al., 2022] has shown promise in bridging the computational and information-theoretical gaps between what is obtainable in the i.i.d. case and for smoothed adaptive sequences.

In this paper, we initiate the study of smoothed analysis for sequential probability assignment and seek to understand fundamental information-theoretic limits on the minmax regret as well as design natural and oracle-efficient algorithms for this problem. Additionally, we investigate whether in the smoothed analysis setting, maximum likelihood estimation can efficiently address sequential probability assignment while achieving small regret. To the best of our knowledge, our work is the first to consider oracle-efficient algorithms (and particularly the MLE) for the sequential probability assignment problem.

1.1 Main results

Reduction to transductive learning. Our first main result is a reduction from regret bounds against a smoothed adversary to regret bounds against (a generalized version of) a transductive adversary. That is, we show that the minimax regret in the smoothed analysis setting is upper bounded by the minimax regret in the setting where a set of contexts is provided to the learner and the adversary is constrained to picking the contexts from this set. For F , a class of hypotheses mapping contexts to [0 , 1] , let us define the minmax regret in the transductive case over T times steps when a context set of size M is provided to the learner to be R M T ( F ) . We establish in Theorem 3.1 that for all σ -smooth sequences the minmax regret R T ( F , σ ) satisfies, for any k > 1 ,

Furthermore, in Theorem 3.4 we upper bound R kT T ( F ) by connecting the worst case adversarial regret in this setting to the scale-sensitive VC dimension of F which is a prototypical offline complexity of the class. Our results obtain a logarithmic dependence on 1 /σ . In particular, in

Corollary 3.4.1, we show that for VC classes (and parametric classes) the regret is bounded by R T ( F , σ ) ≤ O ( d log ( T σ ) ) , where d is the VC dimension of class F .

Efficient Reduction from Sequential Probability Assignment to MLE. Our second contribution is initiating the study of oracle-efficient algorithm design for sequential probability assignment. In particular, for small alphabet size, we design a natural algorithm (Algorithm 1) that efficiently uses an MLE oracle and achieves sublinear regret in the smoothed setting. Our Theorem 4.1 gives a general regret bound in terms of the statistical complexity of the class F and the smoothness parameter σ .

For VC classes, this achieves regret rate of T 4 / 5 √ d σ . To the best of our knowledge, this is the first oracle-efficient algorithm and analysis of the follow-the-perturbed-leader style algorithms for the logarithmic loss.

Probability assignment for VC classes. For VC classes F , we explicitly construct sequential probability assignments and establish their regret guarantees in the smoothed setting. That is, we construct a probability assignment based on a Bayesian mixture over F that satisfies R T ( F , σ ) ≤ Cd log ( T σ ) where d is the VC dimension of class F . While this approach is not oracle-efficient, it indeed achieves regret bound with optimal dependence on T and σ . This motivates a natural direction for future work as to whether such mixture-based methods can be implemented oracle efficiently or if there is a tradeoff between the regret and the computational complexity of sequential probability assignment.

2 Preliminaries

Let X be a set of contexts and Y = { 0 , 1 } . Then, the problem being studied entails a sequential game where at each timestep t , based on the history of contexts x 1: t := ( x 1 , . . . , x t ) where x i ∈ X and associated bits y 1: t -1 ∈ { 0 , 1 } t -1 , the player must assign a probability q ( ·| x 1: t , y 1: t -1 ) to what the upcoming bit y t will be. Once the bit y t is revealed (possibly in an adversarial fashion) the player incurs loss -log q ( y t | x 1: t , y 1: t -1 ) and the game proceeds to the next step. For a hypothesis class F ⊂ {X → [0 , 1] } , the associated regret for a probability assignment strategy Q = { q ( ·| x 1: t , y 1: t -1 ) } n t =1 for a fixed x 1: T , y 1: T is

where p f (1 | x t ) = f ( x t ) ; i.e. the function f assigns probability Bern( f ( x t )) to the upcoming bit given the context x t .

Our statistical results apply to a general loss function ℓ and general actions of the learner a t . For a set of inputs { ( x i , y i ) } T i =1 specified by the adversary and a set of actions { a i } T i =1 of the learner, regret is defined by

where for log-loss a t = q ( ·| x 1: t , y 1: t -1 ) ; the action is a probability mass function (pmf) over { 0 , 1 } .

The regret in (1) is often studied under various adversary models; i.e. various different probabilistic assumptions (or lack thereof) on the model generating x t and y t . In this work, we consider worst-case y t (in contrast to the realizable setting where Y t ∼ Bern( f ∗ ( x t )) for a fixed unknown f ∗ ∈ F ) and X t ∼ D t where D t s form an adaptive sequence of smooth distributions.

Definition 2.1 (Smooth distribution and adversary [Haghtalab et al., 2020]) . Consider a fixed and known base distribution µ on X (such as the uniform distribution if X supports it). A distribution D on X is said to be σ -smooth if for all measurable sets A ⊆ X , D ( A ) ≤ µ ( A ) σ . We denote the set of all σ -smooth distributions by ∆ σ ( µ ) . An adversary, characterized by a joint distribution D with X t ∼ D t (where D t may possibly depend on the history) is said to be a σ -smooth adaptive adversary if D t ∈ ∆ σ ( µ ) for all t ∈ { 1 , . . . , T } .

The minmax regret for σ -smooth adaptive adversaries is then given by

where Q is the set of all probability assignment strategies.

We are particularly interested in how geometric properties of the function class F affect R T ( F , σ ) . There are several notions of covering numbers and combinatorial dimensions that quantify the 'richness' and complexity of a class, but the scale-sensitive VC dimension will be of particular interest to us and is invoked in our results.

Definition 2.2 (Scale-sensitive VC dimension) . Let F be a function class. For any α > 0 and points x 1 , . . . , x m ∈ X , we say that F shatters the set x 1 , . . . , x m at scale α if there exist s 1 . . . s m ∈ R such that for each ϵ ∈ {-1 , 1 } n there exists a function f ∈ F such that ϵ i ( f ( x i ) -s i ) ≥ α 2 . The scale sensitive VC dimension at scale α of F , denoted by VC( F , α ) is defined as the largest m such that there is a set of m points x 1 , . . . , x m ∈ X such that F shatters the set at scale α . The (traditional) VC dimension of a binary class F is defined as VC( F ) = lim α → 0 + VC( F , α ) .

Throughout, we use the following result of Haghtalab et al. [2021] about σ -smooth distributions. This results aids us in reduction from smoothed learning to transductive learning.

Theorem 2.1 (Coupling Lemma of Haghtalab et al. [2021]) . Let D σ be an adaptive sequence of t σ -smooth distributions on X . There is a coupling Π such that

X 1 , . . . , X t is distributed according D σ ,

For every j ≤ t , { Z i,k } i ≥ j,k ∈ [ K ] are uniformly and independently distributed on X , conditioned on X 1 , . . . , X j -1 .

For any t , with probability at least 1 -(1 -σ ) K , X t ∈ { Z t,k } k =1: K .

3 General reduction to transductive learning

In this section, we will consider the minimax regret for the smoothed online learning game with respect to the loss function 1 ℓ against a general class of functions F . In Section 3.1, we will show that the minimax regret can be reduced to the minimax regret for a version of transductive learning with respect to the same loss function and class of functions. In Section 3.2, we give general upper bounds for the transductive setting. There is a subtle but important difference between reduction that directly involve regret compared to recent efforts (such as Haghtalab et al. [2021], Block et al. [2022], Haghtalab et al. [2022]) using reductions between proxies of regret, such as covering numbers and sequential complexities. This is particularly important for log loss since its complexity is not captured by covering numbers. We discuss this point further in Section 3.2.2.

3.1 Regret-to-regret Reduction

We work with a general loss function ℓ and general actions of the learner a t . We note that, we can write the minmax value of the smoothed setting in extensive form as

In order to bound this, we consider a generalization of the notion of online learning that is referred to as transductive learning. In this setting, at the start of the interaction the adversary chooses a set of contexts X = { X i } M i =1 for some M ≥ T and provides this to the player. The game proceeds as

before with the adversary picking ( x t , y t ) at time t and the learner picking an action a t and suffering a loss ℓ ( a t , ( x t , y t )) . However, the adversary is now constrained to pick x t ∈ X at all times t . We can then define the minmax regret indexed by X as

Furthermore, define the worst-case transductive learning regret for sets of size M as R M T ( F ) = max X ⊆X , | X | = M R T ( F , X ) . In the following theorem, we show that the regret against σ -smoothed adversaries is bounded by the regret in the transductive learning setting when the set of contexts is drawn from the base distribution µ .

Theorem 3.1. Let F be any class of functions from X to R and let σ ∈ (0 , 1] . Then, for any T and k , we have

Theorem 3.1 shows that we can reduce the problem of evaluating the minimax regret for smoothed adversaries to evaluating the minimax regret for transductive learning. Note that the second term (1 -σ ) k ≤ e -kσ and thus, in order to get bounds that are sublinear one needs to consider k = c log T/σ for an appropriate absolute constant c . As we will see in the next section, this leads to logarithmic dependence on σ -1 . Moreover, by Theorem 3.1 and since E X ∼ µ T R T ( F , X ) ≤ R T ( F , σ ) , we can see that the transductive learning regret exactly captures the smoothed regret up to polylog ( T σ ) factors.

3.2 Bounds for Transductive Learning

In this section, we discuss ways to upper bound transductive learning regret R M T ( F ) so as to achieve bounds on R T ( F , σ ) via Theorem 3.1.

3.2.1 Using Covering Numbers

One of the approaches common in online learning is to characterize the regret in terms of geometric properties (such as covering numbers) of the function class F . The notion of covering required varies depending on the loss function and the stochastic properties of the data-typically completely adversarial problems require stronger notions of sequential coverings [Rakhlin et al., 2015a,b] while for stochastic problems usually weaker offline coverings suffice. In our smoothed case, we show that the offline complexity notion of scale-sensitive VC dimension as defined in Definition 2.2 is adequate. Similar ideas were considered for the case of regression and convex Lipshitz losses in Haghtalab et al. [2022], Block et al. [2022].

Let us first define the notion of approximation according to which a cover will be constructed; we will consider a pointwise approximation. This notion is similar to the notion of global sequential covering in Wu et al. [2022b].

Definition 3.1. Let F be a function class. A set of functions ˜ F is said to be a ϵ -covering of F if for any f ∈ F there exists g ∈ ˜ F such that sup x ∈X | f ( x ) -g ( x ) | ≤ ϵ . We will use N ( F , ϵ ) to denote the size of the minimal ϵ -covering of F .

Note that while the metric in Definition 3.1 is quite stringent, using this cover in the transductive learning case requires us to only consider function classes with bounded domain size . We capture this using the following theorem.

Theorem 3.2 (Upper bound on transductive learning) . Let F be a function class and ϵ > 0 . Then,

where F| Z is the projection of hypothesis class F on the set Z .

The proof of this theorem follows from relating transductive learning to the worst sequential prediction on a finite set of points using formalism presented in Wu et al. [2022b]. The proof of this theorem is deferred to the Appendix C.

Next, we recall that the covering number N ( F , ϵ ) is bounded as a function of the scale sensitive VC dimension of the class F and the number of points in the domain.

Theorem 3.3 (Rudelson and Vershynin [2006]) . There exist universal constants c, C such that for all α > 0 , any function class F defined on a finite set X , and ϵ > 0 , we have

Finally, putting together Theorem 3.1, Theorem 3.2 and Theorem 3.3 we get the following.

Theorem 3.4 (Minimax smoothed regret and scale-sensitive VC dimension) .

We can instantiate the bound in Theorem 3.4 in terms of T and σ for two particularly interesting cases: when VC( F , ϵ ) scales as d log ( 1 /ϵ ) (often referred to as parametric classes) and when VC( F , ϵ ) scales as ϵ -p (often referred to as nonparametric classes). A canonical example of the former are VC classes ; for a class with VC dimension d , VC( F VC , ϵ ) = Cd log ( 1 ϵ ) (see for example [Vershynin, 2018, Theorem 8.3.18]). A canonical example of the latter are functions of bounded variation, F BV which have VC( F BV , ϵ ) = C ϵ (see for example [Musayeva, 2020, Bartlett et al., 1997]). This class is known to have unbounded sequential covering numbers [Rakhlin et al., 2010] and therefore is not learnable with a worst-case adversary-this can be seen as a simple consequence of the fact that F BV contains all one-dimensional thresholds.

Corollary 3.4.1 (Rates for parametric and nonparametric classes) . If VC( F , ϵ ) = d log ( 1 /ϵ ) , then for a large enough T

In particular, note that Corollary 3.4.1 shows that for VC classes R T ( F VC , σ ) = ˜ O ( d · log ( T σ ) ) (tight, see also concurrent work [Wu et al., 2023] for a similar bound) and for functions of bounded variation R T ( F BV , σ ) = ˜ O ( √ T log ( 1 σ ) ) ; note that the minmax rates for the worst-case adversary scale as Ω( T ) for both these cases. We note that the above bound may be loose for general nonparametric classes, but should be improvable using a multiscale (chaining) version of Theorem 3.4 but we do not focus on this here.

Though the above results give satisfactory bounds in the minimax sense for many classes F of interest, it is useful to consider explicit constructions of probability assignment rules. For the case of finite VC dimension, we give an explicit probability assignment rule by considering a discretization of the class and using a mixture probability assignment rule. In particular, this strategy (denoted by Q VC )

yields optimal regret R T ( F VC , σ, Q VC ) ≤ O ( d log ( T σ ) ) . For the formal statements, proofs and detailed discussion see Appendix D.

3.2.2 Examples without Covering numbers

This reduction approach to characterizing minmax regret in the log-loss is interesting since all previous approaches have used covering numbers of some kind-either sequential covering numbers

If VC( F , ϵ ) = ϵ -p , then

or stronger notions of global covering. However, in stark contrast to the 0/1 loss and several other loss functions [Rakhlin et al., 2015b], covering numbers cannot capture the minmax regret for the log-loss, at least in the adversarial case. Consider the following class of functions on context set

this class, Rakhlin and Sridharan [2015] construct a follow-the-regularized leader (FTRL) based algorithm achieving regret O ( √ T ) . However, Bilodeau et al. [2020] show an upper bound on the regret in terms of sequential covering numbers which is not improvable in general-this shows that sequential covering numbers are not adequate to capture the minmax regret rates for the log-loss. Wu et al. [2022b] further consolidate this by considering the following class, closely resembling F Lin ,

minmax regret for F AbsLin is ˜ Θ( T 2 / 3 ) , demonstrating the surprising fact that by a simple linear transformation of the hypothesis class (which does not change its covering number) one can obtain minmax rates that differ by a polynomial factor! On the other hand, our reduction-based approach bypasses the need for using any covering based arguments and therefore would lead to tight (at least up to poly log( T/σ ) ) rates.

We remark that exact characterizations of the minmax regret with log-loss (often referred to as the minmax redundancy in the information theory literature) in the no-context (adversarial) case is most often calculated by studying the so-called stochastic complexity of the class F [Rissanen, 1996]. This can be extended to (worst-case) transductive learning with contexts x 1 , . . . , x T ; in this case the minmax optimal regret for a fixed horizon is achieved by the normalized maximum likelihood (NML) probability assignment [Shtar'kov, 1987], and can be expressed as R T T ( F ) = max x 1: T log ( ∑ y 1: T ∈{ 0 , 1 } T max f ∈F ∏ T t =1 p f ( y t | x t ) ) . This expression has been evaluated previously for online logistic regression [Jacquet et al., 2021] and more general hypothesis classes [Wu et al., 2022b]. It is an intriguing question to understand what properties of F the stochastic complexity depends on, given that the above examples illustrates that covering numbers do not capture it. Our reduction provides a technique to use such fine-grained understanding of the regret to directly lift the bounds to the more general smoothed adversary setting.

4 Oracle-Efficient Smoothed Sequential Probability Assignment

In the previous section, we consider a purely statistical perspective on the minimax value of the sequential probability assignment problem for smoothed adversaries. In this section, we will focus on an algorithmic perspective and design an algorithm that is efficiently implemented using calls to an MLE oracle. We will focus on the setting when the base measure is the uniform measure on the input space X and the label space Y = { 0 , 1 } . In this setting, we are given access to an oracle OPT which given a data set S = { x i , y i } m i =1 outputs a hypothesis that minimizes the loss on S . That is,

In the context of the logarithmic loss, this corresponds to maximum likelihood estimation. Most of the analysis holds for a general loss function ℓ (with regret scaling appropriate bounds on the values and derivatives), but for clarity one can think of ℓ as the log-loss. In particular, Algorithm 1 is written for the log-loss.

The main framework we work in is the follow-the-perturbed-leader (FTPL) framework. Here, our algorithm uses the oracle on a data set consisting of the historical samples and a set of hallucinated samples. The hallucinated samples are intended to 'stabilize' the predictions of the algorithm. This gives us a probability assignment Q FTPL = { q FTPL ( ·| x 1: t , y 1: t -1 ) } T t =1 .

In order to state the regret bound, we need the following notions. For any class F , define the truncated class F α as F α = { f α : f α ( x ) = f ( x )+ α 1+2 α where f ∈ F } . We will also need the notion of Rademacher complexity Rad( F , T ) = sup X ⊂X , | X | = T E ϵ [ sup f ∈F 1 T ∑ x ∈ X ϵ x f ( x ) ] .

Algorithm 1: Probability assignment Q FTPL

Input: time horizon T , smoothness parameter σ , VC dimension d , Samples per step Parameter n , Truncation parameter α

1 for t ← 1 to T do 2 Generate N ( t ) ∼ Poi ( n ) fresh hallucinated samples ( ˜ x ( t ) 1 , ˜ y ( t ) 1 ) , · · · , ( ˜ x ( t ) N , ˜ y ( t ) N ) , which are i.i.d. conditioned on N with ˜ x ( t ) i ∼ U ( X ) and ˜ y ( t ) i ∼ U ( { 0 , 1 } ) Call the oracle to compute ˜ h t ← OPT ( { ( ˜ x ( t ) i , ˜ y ( t ) i ) } i ∈ [ N ( t ) ] ∪ { x τ , y τ } τ ∈ [ t -1] ) 3 Observe x t 4 Assign probability q FTPL (1 | x 1: t , y 1: t -1 ) = ˜ h t ( x t )+ α 1+2 α 5 Receive y t 6 end

Theorem 4.1 (Main Regret Bound) . For any hypothesis class F and parameters n, α , we have that the regret of Algorithm 1 for σ -smoothed adversaries is bounded as

We will instantiate this bound for case when the class F has bounded VC dimension. For such classes, it is known that the Rademacher complexity is bounded. We state this in the following corollary.

Note that, in particular, for VC classes we have R T ( F VC , σ ) scales as T 4 / 5 . Improving this to achieve the minimax rate discussed in Corollary 3.4.1 is an interesting open question.

Remark. A slightly improved regret scaling as T 3 / 4 can be achieved by assuming access to an oracle that can optimize a mixed objective function involving the log-loss and signed sum of the functions in class. However, these oracles do not have the natural interpretation in terms of maximum likliehood estimation.

4.1 Analysis

The main challenge in designing algorithms in the follow-the-perturbed-leader framework is designing the distribution of the hallucinated samples so as to balance the tradeoff between the 'stability' of the algorithm i.e. how little the algorithm changes its prediction from time step to time step, and the 'perturbation' i.e. how much the addition of the hallucinated samples changes the prediction of the algorithm from the outputting the best hypothesis on the historical samples. This is captured by the following lemma.

Lemma 4.2 (Follow the Perturbed Leader bound) . Let ℓ be a convex loss function, and let F be a hypothesis class. Let D t denote the distribution of the adversary at time t and let Q t denote the distribution of the hypothesis h t output by Algorithm 1. Then, we have that the regret of Algorithm 1 (where we use ˜ s t = ( ˜ x t , ˜ y t ) to denote a hallucinated data point) 2 is bounded by

We provide a proof in Appendix E for completeness. Given this decomposition of the regret, we need to handle both terms carefully. Just to appreciate the tradeoff, note that as we increase the number of hallucinated examples, the perturbation term generally increases, but the stability term generally decreases. First, let us focus on the stability term which is harder to deal with. The main approach we will use is a generalization of the decomposition of the stability term introduced in Haghtalab et al. [2022] even when the losses are unbounded, as is the case with the log-loss. The main idea is to decompose the stability term in terms of the distance between the distribution of the average prediction at the next time step and the distribution of the current time step, as captured by the χ 2 distance and a term that captures how different the predictions of the algorithm are when the sample is resampled from the same distribution. The proof can be found in Appendix F. 3

Lemma 4.3 ( χ 2 + Generalization ⇒ Stability) . Let Q t denote learner's distribution over F in at round t , D t be adversary's distribution at time t (given the history s 1 , · · · , s t -1 ), s t ∼ D t be the realized adversarial instance at time t , and s ′ t be an independent copy s ′ t ∼ D t . Let R ( t +1) refers to the randomness used by the algorithm in round t +1 . Then,

Given this lemma, we move on to bounding the χ 2 divergence between the distribution of the average prediction at the next time step and the distribution of the current time step. This is done using the Ingster method to bound the divergence of mixtures. We include a proof in Appendix G for completeness.

Next, we move on second term in Lemma 4.3. Note that this term involves the difference between the loss of the hypothesis output at time t +1 evaluated on two independent points s t and s ′ t drawn from D t . The main idea to bound the term is to use a stronger version of the coupling lemma Theorem 2.1 which allows us to extract subsequences of points sampled according to smooth distributions from iid samples from the base measure. This allows us to relate the required generalization bound to the Rademacher complexity of the class composed with the loss. Using the truncation and the contraction principle, we get the desired bound. The proof can be found in Appendix H.

Lemma 4.5 (Generalization) . Let h t +1 denote the hypothesis output by the Algorithm 1 at time t +1 . Then, for any m ≤ n , we have

The final term that we want to bound is the perturbation term. In order to bound this note that we set our truncation parameter α such that the loss our the prediction made by our algorithm is bounded and consequently the perturbation term is bounded by n log 1 α , see Appendix I. Theorem 4.1 follows by combining the above results.

5 Conclusions and Open Problems

In this paper, we initiated the study of sequential probability assignment with smoothed adversaries. We characterize the minimax regret in terms of the minimax regret for transductive learning and use this to provide tight regret bounds, e.g., for VC classes. Furthermore, we initiate the study of oracle efficiency in this setting and show that sublinear regret can be achieve for general classes.

Our work motivates several directions for future work. An interesting direction is whether the optimal O (log( T )) regret is achievable for some classes, such as VC classes, using oracle-efficient algorithms. More generally, are there computational barriers to obtaining fast rates in prediction with log-loss in this setting?

Acknowledgments and Disclosure of Funding

This work was supported in part by the National Science Foundation under grant CCF-2145898, a C3.AI Digital Transformation Institute grant, and Berkeley AI Research Commons grants. This work was partially done while authors were visitors at the Simons Institute for the Theory of Computing.

References

Jorma Rissanen. A universal data compression system. IEEE Transactions on information theory , 29 (5):656-664, 1983.

Jorma Rissanen. Universal coding, information, prediction, and estimation. IEEE Transactions on Information theory , 30(4):629-636, 1984.

Thomas M Cover. Universal portfolios. Mathematical finance , 1(1):1-29, 1991.

Meir Feder, Neri Merhav, and Michael Gutman. Universal prediction of individual sequences. IEEE transactions on Information Theory , 38(4):1258-1270, 1992.

Qun Xie and Andrew R Barron. Minimax redundancy for the class of memoryless sources. IEEE Transactions on Information Theory , 43(2):646-657, 1997.

Neri Merhav and Meir Feder. Universal prediction. IEEE Transactions on Information Theory , 44(6): 2124-2147, 1998.

Qun Xie and Andrew R Barron. Asymptotic minimax regret for data compression, gambling, and prediction. IEEE Transactions on Information Theory , 46(2):431-445, 2000.

Yuhong Yang and Andrew Barron. Information-theoretic determination of minimax rates of convergence. Annals of Statistics , pages 1564-1599, 1999.

Jiantao Jiao, Haim H Permuter, Lei Zhao, Young-Han Kim, and Tsachy Weissman. Universal estimation of directed information. IEEE Transactions on Information Theory , 59(10):6220-6242, 2013.

Francesco Orabona and D´ avid P´ al. Coin betting and parameter-free online learning. Advances in Neural Information Processing Systems , 29, 2016.

Dylan J Foster, Satyen Kale, Haipeng Luo, Mehryar Mohri, and Karthik Sridharan. Logistic regression: The importance of being improper. In Conference On Learning Theory , pages 167-208. PMLR, 2018.

Alexander Rakhlin and Karthik Sridharan. Sequential probability assignment with binary alphabets and large classes of experts. arXiv preprint arXiv:1501.07340 , 2015.

Yaniv Fogel and Meir Feder. On the problem of on-line learning with log-loss. In 2017 IEEE International Symposium on Information Theory (ISIT) , pages 2995-2999. IEEE, 2017.

Yaniv Fogel and Meir Feder. Universal batch learning with log-loss. In 2018 IEEE International Symposium on Information Theory (ISIT) , pages 21-25. IEEE, 2018.

Alankrita Bhatt and Young-Han Kim. Sequential prediction under log-loss with side information. In Algorithmic Learning Theory , pages 340-344. PMLR, 2021.

Blair Bilodeau, Dylan J Foster, and Daniel M Roy. Minimax rates for conditional density estimation via empirical entropy. arXiv preprint arXiv:2109.10461 , 2021.

Changlong Wu, Mohsen Heidari, Ananth Grama, and Wojciech Szpankowski. Expected worst case regret via stochastic sequential covering. arXiv preprint arXiv:2209.04417 , 2022a.

Dylan J Foster and Akshay Krishnamurthy. Efficient first-order contextual bandits: Prediction, allocation, and triangular discrimination. Advances in Neural Information Processing Systems , 34: 18907-18919, 2021.

Dylan J Foster, Sham M Kakade, Jian Qian, and Alexander Rakhlin. The statistical complexity of interactive decision making. arXiv preprint arXiv:2112.13487 , 2021a.

Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine learning , 2:285-318, 1988.

Nika Haghtalab, Tim Roughgarden, and Abhishek Shetty. Smoothed analysis with adaptive adversaries. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS) , pages 942-953. IEEE, 2021.

Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Stochastic, constrained, and smoothed adversaries. In Advances in neural information processing systems , pages 17641772, 2011.

Nika Haghtalab, Yanjun Han, Abhishek Shetty, and Kunhe Yang. Oracle-efficient online learning for beyond worst-case adversaries. Advances in Neural Information Processing Systems , 2022.

Adam Block, Yuval Dagan, Noah Golowich, and Alexander Rakhlin. Smoothed online learning is as easy as statistical learning. In Conference on Learning Theory , pages 1716-1786. PMLR, 2022.

Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert Schapire. Taming the monster: A fast and simple algorithm for contextual bandits. In International Conference on Machine Learning , pages 1638-1646. PMLR, 2014.

David Simchi-Levi and Yunzong Xu. Bypassing the monster: A faster and simpler optimal algorithm for contextual bandits under realizability. Mathematics of Operations Research , 47(3):1904-1931, 2022.

Dylan Foster and Alexander Rakhlin. Beyond ucb: Optimal and efficient contextual bandits with regression oracles. In International Conference on Machine Learning , pages 3199-3210. PMLR, 2020.

John Langford, Lihong Li, and Alexander Strehl. Vowpal wabbit open source project. 2007.

Dylan J Foster, Alexander Rakhlin, David Simchi-Levi, and Yunzong Xu. Instance-dependent complexity of contextual bandits and reinforcement learning: A disagreement-based perspective. 2021b.

Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences , 71(3):291-307, 2005.

Miroslav Dud´ ık, Nika Haghtalab, Haipeng Luo, Robert E Schapire, Vasilis Syrgkanis, and Jennifer Wortman Vaughan. Oracle-efficient online learning and auction design. Journal of the ACM (JACM) , 67(5):1-57, 2020.

Guanghui Wang, Zihao Hu, Vidya Muthukumar, and Jacob Abernethy. Adaptive oracle-efficient online learning. Advances in Neural Information Processing Systems , 2022.

Sham M Kakade, Adam Tauman Kalai, and Katrina Ligett. Playing games with approximation algorithms. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing , pages 546-555, 2007.

Nika Haghtalab, Tim Roughgarden, and Abhishek Shetty. Smoothed analysis of online and differentially private learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 9203-9215. Curran Associates, Inc., 2020.

Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Sequential complexities and uniform martingale laws of large numbers. Probability Theory and Related Fields , 161(1-2):111-153, 2015a.

Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning via sequential complexities. J. Mach. Learn. Res. , 16(1):155-186, 2015b.

Changlong Wu, Mohsen Heidari, Ananth Grama, and Wojciech Szpankowski. Precise regret bounds for log-loss via a truncated bayesian algorithm. Advances in Neural Information Processing Systems , 2022b.

Mark Rudelson and Roman Vershynin. Combinatorics of random processes and sections of convex bodies. Annals of Mathematics , pages 603-648, 2006.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science . Cambridge university press, 2018.

Khadija Musayeva. Sample complexity result for multi-category classifiers of bounded variation. arXiv preprint arXiv:2003.09176 , 2020.

Peter L Bartlett, Sanjeev R Kulkarni, and S Eli Posner. Covering numbers for real-valued function classes. IEEE transactions on information theory , 43(5):1721-1724, 1997.

Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Random averages, combinatorial parameters, and learnability. Advances in Neural Information Processing Systems , 23, 2010.

Changlong Wu, Ananth Grama, and Wojciech Szpankowski. Online learning in dynamically changing environments. arXiv preprint arXiv:2302.00103 , 2023.

Blair Bilodeau, Dylan Foster, and Daniel Roy. Tight bounds on minimax regret under logarithmic loss via self-concordance. In International Conference on Machine Learning , pages 919-929. PMLR, 2020.

Jorma J Rissanen. Fisher information and stochastic complexity. IEEE transactions on information theory , 42(1):40-47, 1996.

Yurii Mikhailovich Shtar'kov. Universal sequential coding of single messages. Problemy Peredachi Informatsii , 23(3):3-17, 1987.

Philippe Jacquet, Gil Shamir, and Wojciech Szpankowski. Precise minimax regret for logistic regression with categorical feature values. In Algorithmic Learning Theory , pages 755-771. PMLR, 2021.

Raphail Krichevsky and Victor Trofimov. The performance of universal encoding. IEEE Transactions on Information Theory , 27(2):199-207, 1981.

Sudeep Kamath, Alon Orlitsky, Dheeraj Pichapati, and Ananda Theertha Suresh. On learning distributions from their samples. In Conference on Learning Theory , pages 1066-1100. PMLR, 2015.

Daniel A Spielman and Shang-Hua Teng. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. Journal of the ACM (JACM) , 51(3):385-463, 2004.

Adam Block and Max Simchowitz. Efficient and near-optimal smoothed online learning for generalized linear functions. Advances in neural information processing systems , 2022.

Adam Tauman Kalai and Santosh Vempala. Efficient algorithms for universal portfolios. Journal of Machine Learning Research , pages 423-440, 2002.

Haipeng Luo, Chen-Yu Wei, and Kai Zheng. Efficient online portfolio with logarithmic regret. Advances in neural information processing systems , 31, 2018.

Julian Zimmert, Naman Agarwal, and Satyen Kale. Pushing the efficiency-regret pareto frontier for online learning of portfolios and quantum states. In Conference on Learning Theory , pages 182-226. PMLR, 2022.

R´ emi J´ ez´ equel, Dmitrii M Ostrovskii, and Pierre Gaillard. Efficient and near-optimal online portfolio selection. arXiv preprint arXiv:2209.13932 , 2022.

Tim Van Erven, Dirk Van der Hoeven, Wojciech Kotlowski, and Wouter M Koolen. Open problem: Fast and optimal online portfolio selection. In Conference on Learning Theory , pages 3864-3869. PMLR, 2020.

Thomas M Cover and Erik Ordentlich. Universal portfolios with side information. IEEE Transactions on Information Theory , 42(2):348-363, 1996.

Jason E Cross and Andrew R Barron. Efficient universal portfolios for past-dependent target classes. Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics , 13(2):245-276, 2003.

L´ aszl´ o Gy¨ orfi, G´ abor Lugosi, and Frederic Udina. Nonparametric kernel-based sequential investment strategies. Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics , 16(2):337-357, 2006.

Alankrita Bhatt, J Jon Ryu, and Young-Han Kim. On universal portfolios with continuous side information. In International Conference on Artificial Intelligence and Statistics . PMLR, 2023.

Nicolo Cesa-Bianchi and G´ abor Lugosi. Prediction, learning, and games . Cambridge university press, 2006.

A Related work

Sequential probability assignment is a classic topic in information theory with extensive literature, see the survey by Merhav and Feder [1998] and the references within. In particular, the idea of probability assignments that are Bayesian mixtures over the reference class of distributions [Krichevsky and Trofimov, 1981] is of central importance-such mixture probability assignments arise as the optimal solution to several operational information theoretic and statistical problems [Kamath et al., 2015]. It is also known that the Bayesian mixture approach often outperforms the 'plug-in' approach of estimating a predictor from the reference class and then playing it [Merhav and Feder, 1998]. A similar Bayesian mixture probability assignment in the contextual probability assignment problem was used by Bhatt and Kim [2021], where the covering over the VC function class was obtained in a data-dependent manner . This idea of using a mixture over an empirical covering along with a so-called 'addβ ' probability assignment was then used by Bilodeau et al. [2021]. Combining this with the key idea of discretizing the class of functions as per the Hellinger divergence induced metric, they obtained matching rates for several interesting classes in the realizable case (i.e. y t ∣ ∣ x t generated according to a fixed unknown distribution in the reference class); see also Yang and Barron [1999] for more intuition behind usage of Hellinger coverings for stochastic data. Recent work of Wu et al. [2022a,b] has also employed an empirical covering with an addβ probability assignment for both stochastic and adversarial adversaries.

A complementary approach, more common in the online learning literature is to study fundamental limits of sequential decision making problems non-constructively (i.e. providing bounds on the minmax regret without providing a probability assignment that achieves said regret). This sequential complexities based approach of Rakhlin et al. [2015b,a] has been employed for the log-loss by Rakhlin and Sridharan [2015] and Bilodeau et al. [2020]; however the latter suggests that sequential complexities might not fully capture the log-loss problem.

Smoothed analysis, initiated by Spielman and Teng [2004] for the study of efficiency of algorithms such as the simplex method, has recently shown to be effective in circumventing both statistical and computatonal lower bounds in online learning for classification and regression Haghtalab et al. [2021], Rakhlin et al. [2011], Block et al. [2022], Haghtalab et al. [2020], Block and Simchowitz [2022]. This line of work establishes that smoothed analysis is a viable line of attack to construct statistically and computationally efficient algorithms for sequential decision making problems and has developed tools such as the coupling lemma to understand smoothed sequential decision-making. But handling the unbounded and non-Lipschitz log-loss and MLE oracle is the main conceptual novelty introduced in this work.

Due to the fundamental nature of the problem, the notion of computational efficiency for sequential probability assignment and the closely related problem of portfolio selection has been considered in the literature. Kalai and Vempala [2002] presents an efficient implementation of Cover's universal portfolio algorithm using techniques from Markov chain Monte Carlo. Recently, there has been a flurry of interest in using follow the regularized leader (FTRL) type techniques to achieve low regret and low complexity simultaneously [Luo et al., 2018, Zimmert et al., 2022, J´ ez´ equel et al., 2022], see also Van Erven et al. [2020] and the references within. However, none of these methods consider the contextual version of the problem and are considerably different from the oracle-efficient approach. On the other hand, work studying portfolio selection with contexts [Cover and Ordentlich, 1996, Cross and Barron, 2003, Gy¨ orfi et al., 2006, Bhatt et al., 2023] does not take oracle-efficiency into account.

Concurrent Work: Wu et al. [2023] also study the problem of sequential probability assignment (and general mixable losses) and for VC classes achieve the optimal regret of O ( d log( T/σ )) . In addition to the smooth adversaries, they also studied general models capturing the setting where the base measures are not known. They work primarily in the information theoretical setting and do not present any results regarding efficient algorithms. A major contribution of our work is the study of oracle-efficient online learning for the log-loss, which has not been considered by Wu et al. [2023] Indeed, to the best of knowledge, oracle efficiency has not been considered at all for losses such as the log loss even for worst-case settings. Considering oracle-efficiency for log-loss is an important and natural line of research as the ERM oracle corresponds to the maximum likelihood estimator (MLE) that is commonly used in practice. Our paper presents sublinear regret oracle efficient algorithms for the log loss (which does not have an analog in Wu et al. [2023]).

B Deferred Proof from Section 3

In order to obtain an upper bound on R T ( F , σ ) in terms of R kT T ( F ) for some k , we will consider (2) and proceed inductively. The main idea is to note that since D i is σ -smoothed, conditioned on the history thus far, we can invoke the coupling lemma given in Theorem 2.1.

For the sake of illustration, first consider the simple case of T = 1 . Let X 1 , Z 1 . . . Z k denote the coupling alluded to in Theorem 2.1. Recall that X 1 ∼ D 1 and Z 1: k ∼ µ k . Defining the event E 1 := { X 1 ∈ Z 1: k } , we have

where (3) uses that inf a 1 sup y 1 R 1 ( F , X 1 , y 1 , a 1 ) ≤ 1 4 , (4) follows by the coupling lemma and (5) follows from the definition of transductive learning regret. The next step is to generalize this to arbitrary T . The key aspect that makes this possible is that for all t ≤ T , we have D t ∈ ∆ σ ( µ ) , even conditioned on the past, allowing us to apply the coupling lemma. Furthermore, we need that R T ≤ T for arbitrary sequences which is indeed guaranteed for reasonable losses such as the log-loss as noted above.

We now move to general case. We will prove this inductively. Assume that we have used the coupling lemma till time t -1 and replaced the samples from the smooth distributions with samples from the uniform. That is assume the induction hypothesis, for time t as

Using the coupling lemma, we have that there exists a coupling Π t such that X t , Z t, 1 . . . Z t,k ∼ Π t and an event E t = { X t ∈ { Z t, 1 . . . Z t,k } } that occurs with probability 1 -(1 -σ ) k . Using Z t := { Z t, 1 . . . Z t,k } we have

Combining with the induction hypothesis, gives us the induction hypothesis for the next t as required. The desired result follows by upper bounding the average with the supremum over all subsets of size kT .

C Proof of Theorem 3.2

First, recall the notion of the global sequential covering for a class Wu et al. [2022b].

Definition C.1 (Global Sequential Covering Wu et al. [2022b]) . For any class F , we say that F ′ α ⊂ X ∗ → [0 , 1] is a global sequential α -covering of F at scale T if for any sequence x 1: T and h ∈ F , there is a h ′ ∈ F ′ such that for all i ,

Theorem C.1 (Wu et al. [2022b]) . If F ′ α is a global sequential α -covering of F at scale T , then

To finish the proof note that a ϵ -cover in the sense of Definition 3.1 gives a global sequential cover in the sense of Definition C.1.

D VC Classes

In this section, we construct a probability assignment for the case when F ⊂ {X → [0 , 1] } is a VC class . To motivate this probability assignment, consider the no-context case, which is a classic problem in information theory, where the (asymptotically) optimal probability assignment is known to be the Krichevsky and Trofimov [1981] (KT) probability assignment which is a Bayesian mixture of the form

for a particular prior w ( θ ) . This can be written sequentially as q KT (1 | y 1: t -1 ) = ∑ t -1 i =1 y i +1 / 2 t -1+1 leading to it sometimes being called the add-1/2 probability assignment; by choosing w ( θ ) to be Beta( β, β ) prior one can achieve a corresponding addβ probability assignment. We extend the mixture idea to the contextual case. In particular, for functions f 1 , . . . , f m ∈ F , one can choose a mixture probability assignment as 5

This is the approach employed presently with a carefully chosen f 1 , . . . , f m . We remark that for VC classes this mixture approach may be extended to any mixable [Cesa-Bianchi and Lugosi, 2006, Chapter 3] loss.

First consider VC classes more carefully: i.e. each f ∈ F VC is characterized by three things: a set A ⊆ X , where A ∈ A ⊂ 2 X with the VC dimension of the collection A being d < ∞ ; as well as two numbers θ 0 , θ 1 ∈ [0 , 1] . Then, we have

The following equivalent representation of this hypothesis class is more convenient to use. We consider each f to be characterized by a tuple f = ( g, θ 0 , θ 1 ) where

In other words, g belongs to a class G of binary functions-this is simply the class of functions { x ↦→ 1 { x / ∈ A } ∣ ∣ A ∈ A} in the original notation; so that clearly VCdim ( G ) = d . Then, we have p f ( ·| x ) = p g,θ 0 ,θ 1 ( ·| x ) = Bernoulli( θ 0 ) if g ( x ) = 0 ; and p g,θ 0 ,θ 1 ( ·| x ) = Bernoulli( θ 1 ) otherwise.

Recalling the definition of regret against a particular f = ( g, θ 0 , θ 1 ) for a sequential probability assignment strategy Q = { q ( ·| x 1: t , y 1: t -1 ) } T t =1

where q ( y 1: T ∥ x 1: T ) := ∏ T t =1 q ( y t | x 1: t , y 1: t -1 ) .

In the smoothed analysis case, we have X t ∼ D t where D t for all t is σ -smoothed. Recall that in this case, we are concerned with the regret

D.1 Proposed probability assignment

Let µ be the dominating measure for the σ -smoothed distribution of X 1: T . Let g 1 , . . . , g m ϵ ∈ G be an ϵ -cover of the function class G under the metric δ µ ( g 1 , g 2 ) = Pr X ∼ µ ( g 1 ( X ) = g 2 ( X )) . The following lemma bounds m ϵ .

̸

Lemma D.1 (Covering number of VC classes under the metric δ , Vershynin [2018]) .

for an absolute constant c .

Following the idea of using a mixture probability assignment, we take a uniform mixture over g 1 , . . . , g m ϵ and θ 0 , θ 1 so that

and consequently the sequential probability assignment (or equivalently, the probability assigned to 1) is

One can observe that q (0 | x 1: t , y 1: t -1 ) , q (1 | x 1: t , y 1: t -1 ) > 0 and q (0 | x 1: t , y 1: t -1 ) + q (1 | x 1: t , y 1: t -1 ) = 1 so that q is a legitimate probability assignment. Let the strategy induced by this uniform mixture be called Q VC .

D.2 Analysis of Q VC for smoothed adversaries

We note from (6) that for the Q VC as defined in the last section, we have

where g i ∗ ∈ { g 1 , . . . , g m ϵ } is the function i ∗ ∈ [ m ] that minimizes the Hamming distance between the binary strings ( g i ∗ ( x 1 ) , . . . , g i ∗ ( x T )) and ( g ∗ ( x 1 ) , . . . , g ∗ ( x T )) .

We now take a closer look at the second term of (7). Firstly, note that for any ( g, θ 0 , θ 1 ) we have p g,θ 0 ,θ 1 ( y 1: T | x 1: T ) =

where for j ∈ { 0 , 1 }

Next, we note that for any g ∈ G

where (8) follows from properties of the Laplace probability assignment (or that of the Beta/Gamma functions), captured by Lemma D.2.

where Γ( · ) represents the Gamma function.

Putting this back into (7) (and rearranging), we have

where (9) follows since for any natural numbers k ≤ n and θ ∈ [0 , 1] we have ( n k ) θ k (1 -θ ) n -k ≤ 1 . Now, note that

If | k -k ′ | , | n -n ′ | ≤ δ , and max { n, n ′ } ≤ N then by for example [Bhatt and Kim, 2021, Proposition 6] we have that

We now wish to use this bound in (9). For this, we will recall the definitions of n 0 ( g ; x 1: T ) and k 0 ( g ; x 1: T , y 1: T ) for a particular function g and observe that for two functions g, g ′ we have that both | n 0 ( g ; x 1: T ) -n 0 ( g ′ ; x 1: T ) | , | k 0 ( g ; x 1: T , y 1: T ) -k 0 ( g ′ ; x 1: T , y 1: T ) | ≤ d H ( g ( x 1: T ) , g ′ ( x 1: T )) where d H ( · , · ) denotes the Hamming distance and g ( x 1: T ) := ( g ( x 1 ) , . . . , g ( x T )) ∈ { 0 , 1 } T . Thus, by using (10) in (9) with δ = d H ( g ∗ ( x 1: T ) , g i ∗ ( x 1: T )) , N = T , we get

Note that (11) has effectively removed any dependence on y, θ ∗ 0 , θ ∗ 1 . We then have for some absolute constant C , (recalling the definition of i ∗ and F from earlier)

Finally, we can control the last term in (12) by the following result, which follows from the coupling lemma and variance sensitive upper bounds on suprema over VC classes.

Lemma D.3 (Lemma 3.3 of Haghtalab et al. [2021]) .

Plugging the above into (12) and taking ϵ = σ T 2 gives us

E Proof of Lemma 4.2

Proof. Note that this proof holds for general loss functions. Let R T denote the regret.

Let us focus on the second term.

where h ∗ = inf h ∈F α ∑ T i =1 ℓ ( h, s t ) . (13) follows by comparing the optimal of the truncated class with the whole class, see [Cesa-Bianchi and Lugosi, 2006, Lemma 9.5]. (14) follows from the Be-the-leader lemma Cesa-Bianchi and Lugosi [2006].

F Proof of Lemma 4.3

Denote by R ( t ) = ( N ( t ) , { ˜ s i } i ∈ N ( t ) ) the fresh randomness generated at the beginning of time t , which is independent of { s τ } τ<t generated by the adversary. Let Q t be the distribution of the learner's action h t ∈ H in Algorithm 1, Formally,

Let P t be the distribution of r t . The reason why we introduce this notion is that h t in Algorithm 1 only depends on the vector r t -1 .

The main step in the proof is to introduce an independent sample from the distribution D t in order to decouple the dependence of the distribution Q t +1 on the test point s t .

where we get (15) by adding and subtracting the middle term corresponding to evaluating the loss on an independent sample s ′ t and (16) by observing that s t and s ′ t are equally distributed. Since the second term is the same in the required equation, we can focus on the first term.

Here we use the notation ˜ Q t +1 = E s t ∼D t [ Q t +1 ] for the mixture distribution. In order to bound this, we look a variational interpretation of the χ 2 distance between two distributions P and Q .

Lemma F.1 (Hammersley-Chapman-Robbins bound) . For any pair of measures P and Q and any measurable function h : X → R , we have

Applying this to (17), we get

as required. As noted before, for the particular use in our analysis a simpler version of the lemma similar to Haghtalab et al. [2022] suffices but we include the general version since we believe such a version is useful in providing improved regret bounds for the problem.

G Upper Bounding χ 2 Distance: Proof of Lemma 4.4

In this section, we will focus on bounding the χ 2 distance between the distribution of actions at time steps. The reasoning in this section closely follows Haghtalab et al. [2022]. We reproduce it here for completeness.

We assume that X is discrete. Define

As each ˜ x i is uniformly distributed on X and ˜ y i ∼ U ( { 0 , 1 } ) , by the subsampling property of the Poisson distribution, the 2 |X| random variables { n 0 ( x ) , n 1 ( x ) } x ∈X are i.i.d. distributed as Poi ( n/ 2 |X| ) .

Since the historic data is only a translation, it suffices to consider the distributions at time t = 0 and t = 1 . Let n 1 0 ( x ) = n 0 ( x ) + 1 ( x 1 = x, y 1 = 0) with n 1 1 definied similarly. Let P and Q be the probability distributions of { n 0 ( x ) , n 1 ( x ) } x ∈X and { n 1 0 ( x ) , n 1 1 ( x ) } x ∈X , respectively. Note that the output of the oracle depends only on this vector and thus by the data processing inequality it suffices to bound χ 2 ( P, Q ) .

Note that the distribution P is a product Poisson distribution:

As for the distribution Q , it could be obtained from P in the following way: the smooth adversary draws x ⋆ ∼ D , independent of { n 0 ( x ) , n 1 ( x ) } x ∈X ∼ P , for some σ -smooth distribution D ∈ ∆ σ ( X ) . He then chooses a label y ⋆ = y ( x ⋆ ) ∈ { 0 , 1 } as a function of x ⋆ , and sets

̸

Consequently, given a σ -smooth distribution D and a labeling function y : X → { 0 , 1 } used by the adversary, the distribution Q is a mixture distribution Q = E x ⋆ ∼D X [ Q x ⋆ ] , with

̸

We will use the Ingster method to control the χ 2 between the mixture distribution Q and the base distribution P .

Lemma G.1 (Ingster's χ 2 method) . For a mixture distribution E θ ∼ π [ Q θ ] and a generic distribution P , the following identity holds:

where θ ′ is an independent copy of θ .

Let x ⋆ 1 , x ⋆ 2 be an arbitrary pair of instance. Using the closed-form expressions of distributions P and Q x ⋆ , it holds that

Using the fact that { n 0 ( x ) , n 1 ( x ) } x ∈X are i.i.d. distributed as Poi ( n/ 2 |X| ) under P , we have

We will use the fact that the probability of collision between two independent draws x ⋆ 1 , x ⋆ 2 ∼ D is small. That is using the Lemma G.1, we have

where the last inequality follows from the definition of a σ -smooth distribution.

H Upper Bounding Generalization Error: Proof of Lemma 4.5

The proof of the theorem is similar to the [Haghtalab et al., 2022, Section 4.2.2]. In our setting, we need to deal with general losses. We shall need the following property of smooth distributions which is a slightly strengthened version of the coupling lemma in Theorem 2.1 shown in Haghtalab et al. [2022].

Lemma H.1. Let X 1 , · · · , X m ∼ Q and P be another distribution with a bounded likelihood ratio: dP/dQ ≤ 1 /σ . Then using external randomness R , there exists an index I = I ( X 1 , · · · , X m , R ) ∈ [ m ] and a success event E = E ( X 1 , · · · , X m , R ) such that Pr[ E c ] ≤ (1 -σ ) m , and

Fix any realization of the Poissonized sample size N ∼ Poi ( n ) . Choose m in Lemma H.1. Since for any σ -smooth D t , it holds that

the premise of Lemma H.1 holds with parameter σ/ 2 for P = D t , Q = U ( X×{ 0 , 1 } ) . Consequently, dividing the self-generated samples ˜ s 1 , · · · , ˜ s N into N/m groups each of size m , and running the procedure in Lemma H.1, we arrive at N/m independent events E 1 , · · · , E N/m , each with probability at least 1 -(1 -σ/ 2) m ≥ 1 -T -2 . Moreover, conditioned on each E j , we can pick an element u j ∈ { ˜ s ( j -1) m +1 , · · · , ˜ s jm } such that

For notational simplicity we denote the set of unpicked samples { ˜ s ( j -1) m +1 , · · · , ˜ s jm }\{ u j } by v j . As a result, thanks to the mutual independence of different groups and s t ∼ D t conditioned on s 1: t -1 (note that we draw fresh randomness at every round), for E ≜ ∩ j ∈ [ N/m ] E j we have

Let us denote h t +1 = O t +1 ( ˜ s 1 , · · · , ˜ s N , s 1: t -1 , s t ) the output of the algorithm at time t when ˜ s 1 , · · · , ˜ s N denotes the hallucinated data points and s 1: t -1 , s t denotes the observed data points. We will use the fact that O t +1 is a permutation invariant function. Consequently, for each j ∈ [ N/m ] we have

where (18) follows from the conditional iid (and thus exchangeable) property of ( u 1 , · · · , u N/m , s t ) after the conditioning, and (19) is due to the invariance of the O t +1 after any permutation of the inputs. On the other hand, if s ′ t , u ′ 1 , · · · , u ′ N/m are independent copies of s t ∼ D t , by independence it is clear that

Consequently, using the shorthand u 0 = s t , u ′ 0 = s ′ t , we have

The last inequality uses the fact that the algorithm always outputs a function in F α . Further, we have used the Ledoux-Talagrand contraction inequality.

Theorem H.2 (Ledoux-Talagrand Contraction) . Let g : R → R be a L -Lipschitz function. For a function class F , denote by g ◦ F the compositions of function in F with g . Then, for all n ,

Last inequality follows from the fact that the derivative of the log loss is bounded by 1 /α when truncated at level α . Note that the union bound gives

Thus,the law of total expectation gives

The last equation follows from the fact that the output of the algorithm has loss always bounded by log ( 1 /α ) .

We get the desired result by taking the expectation of N ∼ Poi ( n ) , and using Pr[ N > n/ 2] ≥ 1 -e -n/ 8 in the above inequality completes the proof.

I Bound on the Perturbation Term

Lemma I.1 (Perturbation) .

Proof. Note from the truncation step in Algorithm 1, we have that L ( ˆ h, ˜ s t ) ≤ log ( α ) . We get the desired bound by taking expectations.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the contextual sequential probability assignments, and specifically examines cases where the contexts $x_{1:T}$ are generated by $\\sigma$-smooth adversaries as introduced in [Haghtalab et al. 2021], and where the labels $y_{1:T}$ are adversarially generated. The primary findings pertain to the minimax regrets under the logarithmic loss, as compared to a generic set of experts $\\mathcal{F}$. More specifically, the paper demonstrates the following:\n\n1. If the class $\\mathcal{F}$ possesses a scale-sensitive VC-dimension of order $\\epsilon^{-p}$, then the information-theoretical minimax regret grows as $T^{p/(p+1)}\\cdot \\text{poly}\\log(T/\\sigma)$, which is tight up to polylogarithmic factors.\n\n2. An efficient algorithm exists, given an MLE oracle, that attains regret of order $T^{2/(2+w)}\\sqrt{1/\\sigma}$, provided the Rademacher complexity of $\\mathcal{F}$ is of order $T^{-w}$. For the VC-class, specifically where $w=1/2$, this algorithm yields a regret bound of order $T^{4/5}$.\n\n3. For the VC-class, it demonstrates that the information-theoretical regret is of the order $d\\log(T/\\sigma)$ up to a constant factor (the same result was also found in concurrent work [Wu et al. 2023]).\n\nThe principal contribution of this paper that distinguishes it from previous work on contextual sequential probability assignments is the introduction of an oracle-efficient algorithm that achieves sublinear regret. Although the primary techniques are rooted in the FTPL (Follow The Perturbed Leader) framework as outlined in [Haghtalab et al. 2022], this unique contribution sets a new precedent in this line of research.\n\nSTRENGTHS:\nThe primary strength of this work lies in the oracle-efficient algorithm, which presents the first known instance of computationally efficient sublinear regrets under log-loss for general non-parametric classes, albeit using an MLE oracle. This is likely to inspire further research in this field.\n\nWEAKNESSES:\nThe main weakness is the novelty of the techniques used in this paper. From a purely technical standpoint, Algorithm 1 and its analysis seem to merely mimic those used in [Haghtalab et al. 2022]. In fact, there are many places in the proof that appear to be verbatim copied!\n\nThere are also some presentation and technical issues, which I outline below:\n- There are many places where the index of summation is $i$ but the summand is written as $s_t$, for instance, in Lemma 4.2. I suggest the authors conduct a thorough proofreading, including the appendix, to correct issues like this;\n- I recommend that the authors include a proof for Corollary 4.1.1. It's unclear to me how Theorem 4.1 implies this bound for certain values of $n$, $\\alpha$, and $m$;\n- The authors claim that a $T^{3/4}$ bound could be achieved for an alternative oracle. This is not clear to me at all. Are you refering to the Theorem 7 of [Block et al., 2022]? If so, it should be properly cited and clearly explained by what you mean \"mixed objective function\";\n-  In line 636 (appendix), it appears that Lemma 3.3 from [Haghtalab et al., 2021] requires $\\epsilon> \\frac{\\sigma}{T\\log T}$? The choice of $\\epsilon=\\frac{\\sigma}{T^2}$ does not align with the condition stated in that Lemma. I believe a stronger concentration result, such as Lemma 21 from [Wu et al., 2023], would be required in this context;\n- In the proof of Lemma 4.4, presented in Appendix G, you assert that the distributions $P, Q$ are solely dependent on the ensembles $\\\\{n_0(x),n_1(x)\\\\}$. However, I contend that this is not a self-evident fact. It is essential, at the very least, to posit that your OPT oracle is permutation invariant with respect to the input sample sequence. This assumption, it should be noted, does not necessarily hold for all optimization methods; stochastic gradient descent, for example, may not exhibit this property.\n\nThese are the primary reasons I give a \"weak accept\".\n\nQUESTIONS:\nI have the following questions for the authors:\n\n1. It seems that your FTPL algorithm relies on an exact ERM oracle. I'm curious, how would the performance or operation of the algorithm change if the oracle was only approximately optimal?\n2. Is there any computational lower bounds as in [Haghtalab et al. 2022] that can be proved for the log-loss?\n3. Your bound in Theorem 4.1 seems to rely solely on the Rademacher complexity. Does this suggest that your analysis isn't capable of achieving a $\\log(T)$ dependency for the VC-class? For instance, consider the linear function $|\\langle w,x\\rangle|$ with $w,x$ in a unit ball (of a Hilbert space). This class admits a $1/\\sqrt{T}$ Rademacher complexity, but it's known that the information theoretical lower bound is $T^{2/3}$.\n4. Is it possible to achieve better dependency on $T$ with your FTPL algorithm if the contexts are generated from an i.i.d. process (i.e., with $\\sigma=1$)?\n\nLIMITATIONS:\nNo issue with negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the sequential probability assignment problem with a smoothed adversary. The learner sequentially assigns a probability given contexts, which the adversary generates from a distribution that can be far from a known base distribution by a factor of $1/\\sigma$. For the problem with the i.i.d. setting, there is an $O(\\log T)$ regret method, while we must incur $\\Omega(T)$ regret in the pure adversarial setting. Thus, it is well motivated to study what regret is possible in the smoothed setting with respect to $T$ and $1/\\sigma$. \n\nThe authors provide a general regret bound via reduction to transductive learning. E.g., if the class of functions that map contexts to a probability has the VC dimension of $d$, the regret is $O(d \\log (T/\\sigma))$, achieving the logarithmic dependence on $T$ and $1/\\sigma$. The authors also present an FTPL-style algorithm that is efficient in terms of calls to MLE oracles. It is shown that the algorithm attains $T^{4/5}\\sqrt{d/\\sigma}$ regret for the VC class.\n\nSTRENGTHS:\n1. The paper studies an interesting online learning problem under the smoothed setting, which is well motivated by the known results in stochastic and adversarial settings. \n2. The results imply some interesting consequences. Although I'm not an expert, I found Section 3.2.2 to be interesting, where the authors discuss the benefit of their covering-argument-free idea.\n\nWEAKNESSES:\n1. Theorem 3.1, the reduction to transductive learning, seems to be relying heavily on the coupling lemma by Haghtalab et al. [2021]. \n2. The analysis of the oracle-efficient algorithm in Section 4.1 also appears to be relying on Haghtalab et al. [2022].\n\nQUESTIONS:\n1. I would appreciate it if the authors could describe intuitive technical differences from Haghtalab et al. [2021, 2022]. \n2. The scale-sensitive VC dimension appears to be related to the fat-shattering dimension. Is there some connection between them?\n\n#### Minor comments\n- In Algorithm 1's input, capitalization is inconsistent. \n- In Algorithm 1, line 2, a period or a line break is missing before \"Call the oracle...\"\n- I prefer references arranged in alphabetical order. \n- Hyperlinks do not work.\n\nLIMITATIONS:\nAlthough limitations are not explicitly discussed, the range to which their results apply seems clear from the description.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the sequential probability assignment problem, which is the following:\n\nThe algorithm (forecaster), based on past context and outcomes, must assign probabilities to 0-1 values for the next outcome given the latest context. The forecaster competes against a reference class of predictor functions (experts) and aims to suffer low regret compared to the best of these functions. For a logarithmic loss incurred for incorrect predictions, regret is defined as the difference in the total log-loss of the forecaster and the best of the experts.\n\nWhen the contexts are drawn independently from an unknown distribution, and the class of predictor functions is special (say, has a bounded VC dimension), the sequential probability assignment has low (i.e. sublinear) regret. However, if the contexts are adversarial, there are impossibility results even for simple classes of predictors.\n\nTherefore, following Haghtalab et al. (2021), the authors study a smoothed version of the problem where contexts can only be chosen adaptively from the set of sigma-smoothed distributions. They study how the properties of the class of the predictor functions like bounded “scale-sensitive” VC dimensions affect the regret. \n\nThe main tool used is a powerful coupling lemma from Haghtalab et al. (2021) that converts the problem with an adaptive sequence of t contexts to a problem with t*K uniformly and independently distributed contexts (the length of the sequence increases but they are no more adaptive). Thus, results for i.i.d. samples may be applied where the regret is essentially logarithmic in T (number of time steps), which is the first result of the paper.\n\nThe second question studied is an algorithmic approach to solving the problem using calls to MLE. A follow the perturbed leader type algorithm using ideas generalized from Haghtalab et al. (2022) is used. Here, however, the regret is not logarithmic but T^0.8. This is the second main result in this paper.\n\nSTRENGTHS:\nThe paper is well-written and the considered problem is natural. The results are interesting.\n\nWEAKNESSES:\nIt seems the first main result of this paper is already presented in a paper that has been accepted at COLT 2023 (Wu et al.) Even though this seems to be independent work of a different group of authors, I would still find it strange if the same result was now also accepted at NeurIPS, in particular because the proofs are quite similar. Both rely essentially on the coupling arguments from the work of Haghtalab et al. I also have to say that my impression is that the known coupling argument is really the key ingredient and the proof in this submission is \"only\" an adaption of the arguments by Haghtalab to the log-loss setting. Maybe the authors could point out more clearly if there were significant challenges in this adaption. \n\nThe second main result is not contained in the COLT paper. However, it unfortunately only shows a regret of T^0.8 and no logarithmic one and it also follows known algorithmic approaches from the literature like follow the perturbed leader.\n\nQUESTIONS:\n--\n\nLIMITATIONS:\nn.a.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper discusses smoothed analysis of probability assignments in an online setting. The paper shows how the problem can be reduced to a transductive setting and obtains an upper bound on the regret using covering numbers which is further bounded by the scale sensitive VC dimension. The results are instantiated for various classes of functions, including VC function class and the non-parametric class where the scale sensitive VC dimension grows as \epsilon^{-p} at scale \epsilon. 

On the algorithmic side, the authors propose an oracle efficient FTPL style algorithm to predict the probability assignments and show weaker regret bounds for the same (by bounding the Rademacher complexity)

STRENGTHS:
- The paper is very well written and is very clear and easy to follow.
- The paper is the first to consider Oracle efficient algorithms for the sequential prob. assignment problem
- The paper creatively combines earlier ideas (Coupling Lemma) to reduce the smoothed adversary setting to a transductive setting.

WEAKNESSES:
Please see questions below

QUESTIONS:
- The authors can elaborate the results of Wu.et al [2023] and highlight the differences and similarities (both in terms of results and techniques). 
-  Is Algorithm 1 actually implementable in practice - especially without the knowledge of \sigma, the smoothness parameter? 
- How do the results change in case the adversary is realizable w.r.t to the class F that the regret is measured against?
- How critical is the assumption on uniform distribution over X. For instance if X does not support a uniform distribution, how does Algorithm 1 and it's guarantees change?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of sequential probability assignment in the smoothed setting. In particular, a learner receives labelled examples sequentially, where the contexts are drawn from some smooth distribution which can otherwise be chosen adversarially (in each step) and the labels can be chosen adversarially. The goal of the learner is to minimize the regret (minimax rate of excess error with respect to the optimum hypothesis on the subsequence seen so far). \n\nThe first main result provided is that the optimum regret for the problem is characterized by the optimum regret of a different problem where the adversary chooses a priori the set of contexts that they will have to choose from throughout the interaction with the learner. Using the demonstrated relationship between the two problems, the authors provide an upper bound on the regret in the smoothed setting, which involves the scale sensitive VC-dimension of the hypothesis class and a number of free parameters that can be chosen accordingly to adapt the bound to different regimes for the VC-dimension. For VC-classes the provided adapted bound is essentially tight.\n\nThe second main result provided is an oracle-efficient algorithm achieving sublinear regret on VC-classes (and more generally on classes with polynomially decaying Rademacher complexity), provided access to an ERM oracle for the hypothesis class considered. The algorithm follows the approach of Follow-the-Perturbed-Leader, which takes advantage of the trade-off between the stability of the algorithm's intermediate states and the excess error due to perturbing the current set of samples (by adding a number of hallucinated uniformly random data points).\n\nSTRENGTHS:\nThe paper demonstrates that the smoothed analysis framework is relevant to providing provable guarantees for the problem of sequential probability assignment, hence initiating (and motivating) the (further) study of the problem through the lens of smoothed analysis. The results provided include essentially tight bounds on the (smoothed) regret for VC-classes and two main conceptual contributions: First, characterizing the minimax regret for the problem considered in terms of the minimax regret of another relevant problem (rather than some combinatorial notion of dimension, which might even be hopeless in general). Second, repurposing ERM as an oracle to be exploited by algorithms with sublinear regret.\n\nOverall, the results are presented with clarity and sufficient detail, and provide bounds for the considered problem that are fairly general and adaptable to hypothesis classes with different properties.\n\nWEAKNESSES:\nThe first weakness of the paper is that the oracle-efficient algorithm proposed is not necessarily efficient, due to the potentially high complexity of the ERM oracle, especially since the labels are chosen adversarially. In other words, it is not clear whether assuming oracle access to an ERM is reasonable. A discussion on existing or simple positive results regarding the oracle's implementation or even a pointer to empirical results that demonstrate its success on a relevant setting would be appreciated.\n\nFurthermore, many of the technical contributions of the paper are not discussed in the main text. For example, the only hint provided for the proof of Theorem 3.1 in the main text is that it uses Theorem 2.1. In particular, it is not clear what (if any) technical obstacles arise when one tries to apply Thm 2.1 to this setting, or if adapting some approach from prior work is sufficient. Similarly, a proof sketch for Theorem 3.2 would be helpful for the reader.\n\nOverall, while the problem considered is well-motivated and the results are concrete, it is not clear whether there are strong conceptual and technical contributions relatively to prior work (smoothed analysis has already been applied to online learning and most of the tools used to demonstrate the results existed in the literature, at least in some less general form -- e.g., Theorem 2.1 and the idea of decomposing the stability term in Haghtalab et al. [2022]).\n\n-- Most of my main concerns where addressed in the rebuttal and for this reason I increased my score from 6 to 7.\n\nQUESTIONS:\nMy main questions are related to the two main points raised in the weaknesses section. Given the concreteness of the provided results, I would be happy to increase my score if the issues I pointed to are sufficiently addressed by the authors' answer.\n\n1. Are there some simple function classes for which the ERM oracle can be efficiently implemented under the marginal assumptions that correspond to the considered setting? Alternatively, is there a(n empirical, conceptual or theoretical) reason for which one might expect to heuristically obtain an approximate ERM oracle?\n\n2. What are the main technical hurdles arising when one tries to instantiate prior techniques in the considered setting? In which ways does the technical work differ from prior work?\n\nI have also found a small number of typos:\n- line 294: likliehood $\\to$ likelihood\n- lines 306-307: i=1 $\\to$ t=1 (3 times, one for each summation)\n- line 318: at in $\\to$ in\n- line 321: refers $\\to$ refer\n- line 344: achieve $\\to$ achieved\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-25"}