Original: Oracle-Efﬁcient Online Learning for Smoothed Adversaries(with Appendices)
Revised: Oracle-Efﬁcient Online Learning for Smoothed Adversaries

Original: Abstract
Revised: Abstract

Original: In this paper, we study oracle-efﬁcient algorithms for smoothed analysis of onlinelearning.
Revised: We study the design of computationally efﬁcient online learning algorithms under smoothed analysis.

Original: In this setting,  an adversary is constrained to generating samples fromdistributions whose density is upper bounded by 1 /  times the uniform density.
Revised: In this setting, at every step an adversary generates a sample from an adaptively chosen distribution whose density is upper bounded by 1 /  times the uniform density.

Original: Given access to an ofﬂine optimization  oracle, we give the ﬁrst computationallyefﬁcient online algorithms whose sublinear regret depends only on the pseudo/VCdimension d of the class and the smoothness parameter  .
Revised: Given access to an ofﬂine optimization (ERM) oracle, we give the ﬁrst computationally efﬁcient online algorithms whose sublinear regret depends only on the pseudo/VC dimension d of the class and the smoothness parameter  .

Original: In particular, weachieve oracle-efﬁcient regret bounds of O ( p Td   1 ) for learning real-valuedfunctions and O ( p
Revised: In particular, we achieve oracle-efﬁcient regret bounds of O ( p Td   1 ) for learning real-valued functions and O ( p

Original: Td   12 ) for learning binary-valued functions.
Revised: Td   12 ) for learning binary-valued functions.

Revised: Our results establish that online learning is computationally as easy as ofﬂine learning, under the smoothed analysis framework.

Original: This contraststhe computational separation between online learning with worst-case adversariesand ofﬂine learning established by [HK16].
Revised: This contrasts the computational separation between online learning with worst-case adversaries and ofﬂine learning established by [HK16].

Original: In the binary setting, our algorithmsalso achieve improved bounds for worst-case setting with small domains.
Revised: Our algorithms also achieve improved bounds for some settings with binary-valued functions and worst-case adversaries.

Original: Inparticular, we give an oracle-efﬁcient algorithm withregret of O ( p
Revised: These include an oracle-efﬁcient algorithm with O ( p

Original: T ( d |X| ) 1 / 2 ) ,which is a reﬁnement of the earlier O ( p 
Revised: T ( d |X| ) 1 / 2 ) regret that reﬁnes the earlier O ( p T |X| )

Original:  T |X| ) bound by [DS16].
Revised: bound of [DS16] for ﬁnite domains, and an oracle-efﬁcient algorithm with O ( T 3 / 4 d 1 / 2 ) regret for the transductive setting.

Original:  Introduction
Revised:  Introduction

Original: Adversarial online learning is a cornerstone of modern machine learning and has led to signiﬁcantadvances in computer science broadly.
Revised: Adversarial online learning is a cornerstone of modern machine learning and has led to signiﬁcant advances in computer science broadly.

Original: A recent line of work on “beyond the worst-case analysis” ofonline learning has brought into light the overly pessimistic nature of standard characterizations ofonline learnability [RST11, GR17, HRS20, HRS21].
Revised: A recent line of work on “beyond the worst-case analysis” of online learning has brought into light the overly pessimistic nature of standard characterizations of online learnability [RST11, GR17, HRS20, HRS22].

Original: This is exempliﬁed by the results of [HRS21]showing that adversarial online learnability is statistically as easy as PAC learnability, in presenceof noise.
Revised: This is exempliﬁed by the results of [HRS22] showing that adversarial online learnability is statistically as easy as PAC learnability, in presence of noise.

Original: That is, under smoothed analysis , online and ofﬂine learnability are both characterized bythe ﬁniteness of the VC dimension of a hypothesis class as opposed to the much larger Littlestonedimension that characterizes online learnability in the worst-case [BDPSS09].
Revised: That is, under smoothed analysis , online and ofﬂine learnability are both characterized by the ﬁniteness of the VC dimension of a hypothesis class as opposed to the much larger Littlestone dimension that characterizes online learnability in the worst-case [BDPSS09].

Original: However, to fullydeliver on the promise revealed by these statistical insights, there needs to be an algorithmic frameworkfor realizing this connection between online and ofﬂine learnability.
Revised: However, to fully deliver on the promise revealed by these statistical insights, there needs to be an algorithmic framework for realizing this connection between online and ofﬂine learnability.

Original: In this paper, we ask Do efﬁcient ofﬂine learning algorithms lead to efﬁcient online learning algorithmswith comparable regret guarantees, underthe smoothed analysis framework?
Revised: In this paper, we ask whether efﬁcient ofﬂine learning algorithms lead to efﬁcient online learning algorithms with comparable regret guarantees, under smoothed analysis?

Original: In more detail, smoothed analysis is a perspective on algorithm design, introduced by [ST04] andformalized for online learning by [RSS12, HRS20], in which the adversary is restricted to generatingan instance at every round from a distribution that is not overly concentrated, i.e. a distributionwhose density is upper bounded by 1 /  times that of the uniform distribution 1 .
Revised: In more detail, smoothed analysis is a perspective on algorithm design, introduced by [ST04] and formalized for online learning by [RST11, HRS20], in which the adversary is restricted to generating an instance at every round from a distribution that is not overly concentrated, i.e., a distribution whose [Table] density is upper bounded by 1 /  times that of the uniform distribution 1 .

Original: The smoothnessof the adversary’s actions captures the noise and imprecision inherent in the real world and as a
Revised: The smoothness of the adversary’s actions captures the noise and imprecision inherent in the real world.

Original: [Table]
Revised: As 1 /  decreases, so does the uncertainty of the learner about the distribution of future instances.

Original: model gracefully captures the expressivity of worst-case instances while circumventing the overlypessimistic nature of the worst-case analysis.
Revised: This gracefully captures the expressivity of worst-case instances while circumventing the overly pessimistic nature of the worst-case analysis.

Original: The question of whether ofﬂine learning algorithms can lead to online learning algorithms is naturallycaptured by the oracle-efﬁciency framework (e.g., [DHL + 17, KV05, HK16]).
Revised: The question of whether ofﬂine learning algorithms can lead to online learning algorithms is naturally captured by the oracle-efﬁciency framework (e.g., [DHL + 20, KV05, HK16]).

Original: In this setting, wehave access to an ofﬂine learning algorithm or equivalently an empirical risk minimization (ERM)oracle which can compute an optimal hypothesis given any history of the actions of the adversary.
Revised: In this setting, we have access to an ofﬂine learning algorithm or equivalently an empirical risk minimization (ERM) oracle which can compute an optimal hypothesis given any history of the actions of the adversary, using

Original: Our computationally efﬁcient online algorithms can be design to utilize polynomial number of callsto the ERM oracle.
Revised: O (1) computation. Efﬁcient algorithms must then be designed to tap into the existing ERM oracle, using polynomial number of calls and computation.

Original: Our main goal is to design oracle-efﬁcient online algorithms whose regret is characterized by ofﬂinestatistical complexity measures, such as the VC dimension or pseudo dimension of a hypothesis class.
Revised: We aim to design oracle-efﬁcient online algorithms whose regret resemble the statistically optimal regret as much as possible. In particular, under smoothed analysis, these bounds must be characterized by ofﬂine statistical complexity measures, such as the VC dimension or pseudo dimension of a hypothesis class.

Original: Interestingly,in the worst-case instances (in absence of smoothed analysis) [HK16] showed that suchalgorithms cannot exist.
Revised: Interestingly, [HK16] showed that such computationally algorithms cannot exist for fully worst-case adversaries.

Original: Thus, positively resolving this question will show that smoothed analysissimultaneously overcomes both statistical and computational impossibilities.
Revised: Therefore, designing oracle-efﬁcient online algorithms for adversaries who are not fully worst-case, must simultaneously overcome both statistical and computational impossibilities.

Original: This isindeed what weshow by the main contributions of our work.
Revised: Designing algorithms that achieve this is the main contribution of this paper.

Original: 1.1 Main Results
Revised: 1.1 Main Results

Original: We  give the ﬁrst oracle-efﬁcient online learning algorithms whose regret is characterized by thestatistical ofﬂine complexity measures, under smoothed analysis.
Revised: We consider online learning under smoothed analysis and give the ﬁrst oracle-efﬁcient online learning algorithms whose regret is characterized by the statistical ofﬂine complexity measures.

Original: In particular, we show that thereare efﬁcient algorithms, given access to an ERM oracle, that achieve sublinear regret that dependsonly on the pseudo- (or VC) dimension of a class of hypothesis.
Revised: In particular, we show that there are efﬁcient algorithms, given access to an ERM oracle, that achieve sublinear regret that depends only on the pseudo- (or VC) dimension of a class of hypotheses, as well as a parameter that captures the power of the adversary, i.e.,  .

Revised: We study both the real-valued and binary valued losses and achieve nearly tight upper and lower bound for these settings.

Original: We summarize our main results in Table 1.
Revised: We summarize our main results in Table 1.

Original: We study both the real-valued and binary valued losses. For the  real-valued case, we show a regretbound of O ( p Td   1 ) .
Revised: Upper bounds and Algorithms. For the general real-valued case, we design an algorithm based on the Relax-and-Randomize principle of [RSS12] that achieves a regret bound of O ( p Td   1 ) for smoothed online learning, where d is the pseudo-dimension of the hypothesis class. This algorithm uses 2 oracle calls per round.

Original: We improve upon this bound for the binary classiﬁcation setting achievingregret of O ( p Td   1 / 2 ) .
Revised: We improve these regret bounds for the binary classiﬁcation setting under smoothed analysis and achieve regret of O ( p Td   1 / 2 ) .

Revised: The algorithm that achieves these improved bounds is a variant of FTPL that uses Poisson random variables in the design of its perturbations.

Revised: This algorithm uses 1 oracle call per round.

Revised: The improved regret in the binary setting crucially leverages the fact that smoothness over the instance domain X also implies smoothness over the instance-label pairs X ⇥ Y , when the label set is binary (or constant in size).

Original: While these bounds demonstrate sublinear regret that only depends on d and 1 , their dependence on  1 does not match the (non-efﬁcient) statistical regret bound of [HRS21],which gave a p
Revised: While these bounds demonstrate sublinear regret that only depends on d and  , their dependence on  does not match the (non-efﬁcient) statistically optimal regret bound of e

Original:  Td log(  1 ) dependence.
Revised: O ( p Td ln(1 /  ) .

Revised: For the case of binary classiﬁcation under smoothed analysis, the statistically optimal regret bound is due to [HRS22].

Revised: For the other settings, we provide the optimal statistical rates in Appendix F.

Original:  We further discuss this computational-statistical gapbetween online learning algorithms by presenting lower bounds for efﬁcient algorithms in presenceof beyond worst-case adversaries and highlight important directions for future research.
Revised: Lower bounds. We further investigate the gap between the computational and statistical regret upper bounds. We present an algorithm-independent lower bound for the setting of smoothed online learning that shows that any algorithm with runtime o ( p d/  ) will incur a ⌦ ( p

Revised: T ( d/  ) 1 / 2 ) regret.

Revised: We note that this lower bound demonstrates the same dependence on 1 /  as our upper bound (for the binary classiﬁcation setting under smoothed analysis).

Original:  In addition to the application of our algorithms  in the smoothed setting, with appropriate settings ofparameters, our algorithms improve upon the regret dependence interms of the domain size |X| inthe worst case binary classiﬁcation with  regret O ( p
Revised: We also provide algorithm-dependent regret lower bounds that demonstrate improved dependence on parameter d , and apply to all relax-andrandomize and FTPL-style algorithms. Improved Bounds for the Classical Settings. In addition to constrained adversaries, our algorithms and analysis also imply improved regret bounds for two classical settings, namely, online learning in bounded domains with worst-case adversaries and traditional online transductive learning . For worst-case adversaries in binary classiﬁcation with domain size |X| , we achieve a regret of

Original:  T ( d |X| ) 1 / 2 ) , improving upon the O ( p
Revised: O ( p T ( d |X| ) 1 / 2 ) , improving upon the O ( p

Original: T |X| )
Revised: T |X| )

Original: 61 bound of [DS16] and the regret dependence in terms ofthe VC dimension d in transductive binaryclassiﬁcation to O
Revised: bound of [DS16]. For online transductive binary classiﬁcation we achieve a regret bound of O

Original: ( T 3 / 4 d 1 / 4 ) , improving upon the O
Revised: ( T 3 / 4 d 1 / 4 ) , improving upon the O

Original: ( T 3 / 4 d 1 / 2 ) bound in [KK06].
Revised: ( T 3 / 4 d 1 / 2 ) bound in [KK05].

Revised: Both improvements are enabled by our novel Poissonized FTPL analysis and techniques which achieve stronger regret bounds in the binary classiﬁcation setting.

Original: 1.2 Technical Overview
Revised: 1.2 Technical Overview

Original: Random Playout for Beyond Worst-Case Adversaries.
Revised: Random Playout for Beyond Worst-Case Adversaries.

Original: Our algorithms are based on the randomplayout design principles, including the admissible relaxation framework of [RSS12] and the Follow- the-Perturbed-Leader framework of [KV05].
Revised: Our algorithms are based on the random playout design principles, including the admissible relaxation framework of [RSS12] and the Followthe-Perturbed-Leader framework of [KV05].

Original: We show that this framework is useful for analyzingonline learning algorithms in the beyond worst-case setting, especially in smoothed analysis.
Revised: We show that this framework is useful for analyzing online learning algorithms in the beyond worst-case setting, especially in smoothed analysis.

Original: Inthis setting, smoothness captures a level of predictability about the future.
Revised: In this setting, smoothness captures a level of predictability about the future.

Original: This is made formalby a technique from [HRS21] that shows that any sequence of T instances generated by adaptivesmoothed adversaries can be seen as a subset of T/  uniformly random instances from X with highprobability.
Revised: This is made formal by a technique from [HRS22] that shows that any sequence of T instances generated by adaptive smoothed adversaries can be seen as a subset of T/  uniformly random instances from X with high probability.

Original: We implement this algorithmically by self-generating random instances and labels as astand-in for the future.
Revised: We implement this algorithmically by self-generating random instances and labels as a stand-in for the future.

Original: While the self-generated samples may not include adversary’s next choicewith some probability, these frameworks can be used to account for the uncertainty in each step.
Revised: While the self-generated samples may not include adversary’s next choice with some probability, these frameworks can be used to account for the uncertainty in each step.

Original: Furthermore, we show that the inclusion of additional self-generated samples has a small impacton the achievable regret by proving that the regularized Rademacher complexity (which acts as anadmissible relaxation) is monotone in the set of generated samples (Lemma 4.1).
Revised: Furthermore, we show that the inclusion of additional self-generated samples has a small impact on the achievable regret by proving that the regularized Rademacher complexity (which acts as an admissible relaxation) is monotone in the set of generated samples (Lemma 4.1).

Original: This monotonicityproperty leads to regret bounds that gracefully degrade as a small function of 1 /  .
Revised: This monotonicity property leads to regret bounds that gracefully degrade as a small function of 1 /  .

Original: Stability, Poissonization and Generalization.
Revised: Stability, Poissonization and Generalization.

Original: To obtain even stronger regret bounds for the binarysetting, our analysis builds on the notion of stability , i.e., how little the distribution of learner’s actionschanges across time steps.
Revised: To obtain even stronger regret bounds for the binary setting, our analysis builds on the notion of stability , i.e., how little the distribution of learner’s actions changes across time steps.

Original: A crucial ingredient in controlling stability is our novel Poissonizationtechnique that randomly sets the number of samples to be self-generated from an appropriatelychosen Poisson distribution.
Revised: A crucial ingredient in controlling stability is our novel Poissonization technique that randomly sets the number of samples, which are to be self-generated, from an appropriately chosen Poisson distribution.

Original: This allows us an additional degree of independence that is essential forcontrolling the loss from one step to the next using information theoretic techniques.
Revised: This allows us an additional degree of independence that is essential for controlling the loss from one step to the next using information theoretic techniques.

Original: The stability analysis of the algorithm also depends crucially on a modiﬁed generalization error of the
Revised: The stability analysis of the algorithm also depends crucially on a modiﬁed generalization error of the

Original: ERM, when it is trained on uniformly generated training samples and tested on smoothly distributedfresh instances.
Revised: ERM, when it is trained on uniformly generated training samples and tested on smoothly distributed fresh instances.

Original: To bound this, we show a strong conditional independence property satisﬁed by thecoupling from [HRS21].
Revised: To bound this, we show a strong conditional independence property satisﬁed by the coupling from [HRS22].

Original: This is instrumental for bounding the generalization error by allowing us toextract smooth variables from a set of uniform variables, which can then be used for symmetrization.
Revised: This is instrumental for bounding the generalization error by allowing us to extract smooth variables from a set of uniform variables, which can then be used for symmetrization.

Original: We expect that this approach will be of independent interest for future work.
Revised: We expect that this approach will be of independent interest for future work.

Original: 1.3 Related works
Revised: 1.3 Related works

Original: Our work relates to several paradigms and approaches to online learnability.
Revised: Our work relates to several paradigms and approaches to online learnability.

Original: Oracle-Efﬁcient Online Learning.
Revised: Oracle-Efﬁcient Online Learning.

Original: Since the seminal work of [KV05], inspired by applicationdomains such as game theory, there has been a long line of work elucidating the computationalaspects of online learning.
Revised: Since the seminal work of [KV05] there has been a long line of work elucidating the computational aspects of online learning.

Original: [KV05] proposed the inﬂuential follow-the-perturbed-leader algorithm.[KKL07] study notions of regret when the learner is given access to an approximate optimizationoracle.
Revised: [KV05] proposed the inﬂuential follow-the-perturbed-leader algorithm. [KKL07] consider linear functions and study notions of regret when the learner is given access to an approximate optimization oracle.

Original: [KK06] study the transductive learning setting  and give an efﬁcient algorithm that convertsofﬂine learnablity to online learnability.
Revised: [KK05] study the transductive learning setting where instances (but not labels) are known in advance and give an oracleefﬁcient online algorithm.

Original: [RSS12] propose a generaladmissible relaxation frameworkto develop efﬁcient algorithms based on the upper bound of the value of the game.
Revised: [RSS12] propose a general relaxation framework to develop efﬁcient algorithms based on the upper bound of the value of the game.

Original: [DHL + 17] presentthe computationally efﬁcient Generalized-FTPL algorithm  and provide conditions under which itachieves vanishing regret.
Revised: [DHL + 20] present the oracle-efﬁcient Generalized-FTPL algorithm that is no-regret for a wide range of combinatorial and economics settings.

Original: On the ﬂip size, [HK16] show that an ⌦ ( p N ) lower bound is unavoidablein general in order to obtain nontrivial regret where the N is the number of actions of the learnersuggesting that one needs to look beyond the worst-case in order to get truly efﬁcient  algorithms.
Revised: On the ﬂip size, [HK16] show that an ⌦ ( p N ) lower bound is unavoidable in general in order to obtain nontrivial regret where the N is the number of actions of the learner. This suggests that one needs to look beyond the worst-case in order to get truly efﬁcient online algorithms.

Original: Beyond Worst-case Approaches to Online Learning.
Revised: Beyond Worst-case Approaches to Online Learning.

Original: Various notion of beyond worst-casebehavior of online learning has been studied in the literature [RST11, HRS20, RS13b, DFHJ17,
Revised: Various notion of beyond worst-case behavior of online learning has been studied in the literature [RST11, HRS20, RS13b, DHJ + 17,

Original: BCKP20].
Revised: BCKP20].

Original: Most closely related are [RST11, HRS20, HRS21].
Revised: Most closely related are [RST11, HRS20, HRS22].

Original: [RST11] studied smoothed analysisof online learning but only gave explicit regret bounds for simple classes such as thresholds.
Revised: [RST11] studied smoothed analysis of online learning but only gave explicit regret bounds for simple classes such as thresholds.

Original: [HRS20,
Revised: [HRS20,

Original: HRS21] both study the notion of smoothed analysis with adaptive adversary and show that statisticallythe regret is bounded by O ( p
Revised: HRS22] both study the notion of smoothed analysis with adaptive adversary and show that statistically the regret is bounded by O ( p

Original: Td log(1 /  )) but do not provide efﬁcient algorithms.
Revised: Td log(1 /  )) but do not provide efﬁcient algorithms.

Original: Concurrent Work.
Revised: Concurrent Work.

Original: In a concurrent and independent work, [BDGR22] also gives oracle-efﬁcientalgorithms for smoothed online learning.
Revised: In a concurrent and independent work, [BDGR22] also gives oracle-efﬁcient algorithms for smoothed online learning.

Original: In the binary classiﬁcation setting, [BDGR22] obtains a regret bound of e
Revised: In the binary classiﬁcation setting, [BDGR22] obtains a regret bound of e

Original: O ( p Td   1 ) using an FTPL-based algorithm.
Revised: O ( p Td   1 ) using an FTPL-based algorithm.

Original: In comparison, our result (Theorem 3.2)demonstrates a regret bound of e
Revised: In comparison, our result (Theorem 3.2) demonstrates a regret bound of e

Original: O ( p
Revised: O ( p

Original: Td   12 ) with strictly better dependence on  .
Revised: Td   12 ) with strictly better dependence on  .

Original: Our regretbound’s improved dependence on parameter  can be attributed to our novel technical innovations,including the introduction and careful analysis of modiﬁed generalization error and stability viaa new coupling-based argument, and a Poissonization approach for self-generating samples thatcan leverage information theoretic arguments.
Revised: Our regret bound’s improved dependence on parameter  can be attributed to our novel technical innovations, including the introduction and careful analysis of modiﬁed generalization error and stability via a new coupling-based argument, and a Poissonization approach for self-generating samples that can leverage information theoretic arguments.

Original: For the case of real-valued functions with pseudo dimension d , [BDGR22] achieve regret e
Revised: For the case of real-valued functions with pseudo-dimension d , [BDGR22] achieve regret e

Original: O (   1 p Td ) with e
Revised: O (   1 p Td ) with e

Original: O ( p T ) calls to the oracle per round.
Revised: O ( p T ) calls to the oracle per round.

Original: Inour paper, we obtain better regret of order e
Revised: In our paper, we obtain better regret of order e

Original: O ( p Td   1 ) using only 2 oracle calls per round 2 .
Revised: O ( p Td   1 ) using only 2 oracle calls per round 2 .

Original: Ourstronger regret bounds are due to the fact that their algorithm is constrained to self-generating T -longsequences as a stand-in for the future, while we are generating substantially longer sequences thatallow us to leverage the monotonicity of Rademacher complexity.
Revised: Our stronger regret bounds are due to the fact that their algorithm is constrained to self-generating T -long sequences as a stand-in for the future, while we are generating substantially longer sequences that allow us to leverage the monotonicity of Rademacher complexity.

Revised: Importantly, these novel techniques for achieving improved dependence on  also enable us to improve on the small-domain result of [DS16] and the transductive learning result of [KK05].

Original:  Preliminaries 
Revised:  Preliminaries 2.1 Smoothed Online Learning

Original: 2.1 Smoothed Online Learning

Original: Let X be the space of instances, Y =
Revised: Let X be the space of instances, Y =

Original: [  1 , 1] be the space of labels, and H : X !
Revised: [  1 , 1] be the space of labels, and H : X !

Original: Y be thehypothesis class with pseudo dimension d (See deﬁnition B.1 or [AB99] for the deﬁnition of pseudodimension).
Revised: Y be the hypothesis class with pseudo dimension d (See deﬁnition B.1 or [AB99] for the deﬁnition of pseudo dimension).

Original: Let l : Y ⇥ Y !
Revised: Let l : Y ⇥ Y !

Original: [0 , 1] be a convex loss function with Lipschitz constant G in its ﬁrstcomponent.
Revised: [0 , 1] be a convex loss function with Lipschitz constant G in its ﬁrst component.

Original: We also consider the special case where
Revised: We also consider the special case where

Original: Y =
Revised: Y =

Original: {  1 , +1 } is binary and the hypothesis128 class
Revised: {  1 , +1 } is binary and the hypothesis class

Original: H has VC dimension d .
Revised: H has VC dimension d .

Original: In this case, we consider the classiﬁcation loss l ( b y, y )
Revised: In this case, we consider the classiﬁcation loss l ( b y, y )

Original: = 1 { b y 6 = y } .
Revised: = 1 { b y 6 = y } .

Original: We work with the smoothed adaptive online adversarial setting from [HRS21].
Revised: We work with the smoothed adaptive online adversarial setting from [HRS22].

Original: We will consider -smooth adversaries, where a distribution is  -smooth if its density is upper bounded by 1 /  timesthe density of the uniform distribution over the same domain.
Revised: We will consider  -smooth adversaries, where a distribution is  -smooth if its density is upper bounded by 1 /  times the density of the uniform distribution over the same domain.

Original: We remark that all of our resultsgeneralize to arbitrary known base distributions as well.
Revised: We remark that all of our results generalize to arbitrary known base distributions as well.

Original: Deﬁnition 2.1 (  -smoothness) .
Revised: Deﬁnition 2.1 (  -smoothness) .

Original: Let X be a domain that supports a uniform distribution U .
Revised: Let X be a domain that supports a uniform distribution U .

Original: A measureµ on X is  -smooth if for all measurable subsets A ✓ X , µ ( A )  U ( A )  .
Revised: A measure µ on X is  -smooth if for all measurable subsets A ✓ X , µ ( A )  U ( A )  .

Original: The set of all  -smoothdistributions on domain X is denoted by   ( X ) .
Revised: The set of all  -smooth distributions on domain X is denoted by   ( X ) .

Original: In online learning with adaptive smoothed adversaries, the learner and the adversary plays a repeatedgame for T time steps.
Revised: In online learning with adaptive smoothed adversaries, the learner and the adversary plays a repeated game for T time steps.

Original: At each time step t 2 [ T ] , the adversary chooses a  -smooth distribution
Revised: At each time step t 2 [ T ] , the adversary chooses a  -smooth distribution

Original: D X t 2  ( X ) .
Revised: D X t 2  ( X ) .

Original: A random instance x t ⇠ D X t is then drawn and presented to the learner.
Revised: A random instance x t ⇠ D X t is then drawn and presented to the learner.

Original: After receiving139 x t , the learner predicts its label to be b y t 2 Y , while the adversary simultaneously chooses y t 2 Y 140 as its true label.
Revised: After receiving x t , the learner predicts its label to be [Equation] , while the adversary simultaneously chooses y [Equation]

Revised: as its true label.

Original: The learner then suffers loss l ( b y t , y t ) .
Revised: The learner then suffers loss l ( b y t , y t ) .

Original: The above protocol is equivalent to a settingwhere the adversary chooses a distribution D t 2  ( X ⇥ Y ) over labeled instances s t 
Revised: The above protocol is equivalent to a setting where the adversary chooses a distribution D t 2  ( X ⇥ Y ) over labeled instances s t =

Original: = ( x t , y t )142 whose marginal on
Revised: ( x t , y t ) whose marginal on

Original: X is  -smooth, and the learner simultaneously chooses a classiﬁer h t 2 Y X .
Revised: X is  -smooth, and the learner simultaneously chooses a classiﬁer h t 2 Y X .

Original: It is143 not hard to see that this equivalence holds for both randomized and deterministic interactions, see[RS14, Section 7.6] for a more detailed discussion.
Revised: It is not hard to see that this equivalence holds for both randomized and deterministic interactions, see [RS14, Section 7.6] for a more detailed discussion.

Original: We will abbreviate D X t to D t when it is clearfrom the context.
Revised: We will abbreviate D X t to D t when it is clear from the context.

Original: We allow the adversary to be adaptive, i.e., the choice of D t can depend on the realization of previousinstances { ( x i , y i ) }
Revised: We allow the adversary to be adaptive, i.e., the choice of D t can depend on the realization of previous instances { ( x i , y i ) }

Original: t  1 i =1 as well as the learner’s previous predictions.
Revised: t  1 i =1 as well as the learner’s previous predictions.

Original: We denote with D  the adaptivesequence of  -smooth distributions D 1 , · · · , D T on the instances.
Revised: We denote with D  the adaptive sequence of  -smooth distributions D 1 , · · · , D T on the instances.

Original: Accordingly, let Q t 2  ( Y )149 denote the learner’s prediction rule on instance x t , and let Q denote the adaptive sequence ofdistributions Q 1 , · · · , Q T .
Revised: Accordingly, let Q t 2  ( Y ) denote the learner’s prediction rule on instance x t , and let Q denote the adaptive sequence of distributions Q 1 , · · · , Q T .

Original: We denote the expected regret of a learner with prediction rules Q on the151 adaptive sequence D  by
Revised: We denote the expected regret of a learner with prediction rules Q on the adaptive sequence D  by

Original: [Equation]
Revised: [Equation]

Original: We remove D  and Q from this notation when they are clear from the context.
Revised: We remove D  and Q from this notation when they are clear from the context.

Original: An important property of smoothness is that it implies  coupling between uniform and adaptivesmooth processes.
Revised: An important property of smoothness is that it implies a probability coupling between uniform and adaptive smooth processes as ﬁrst observed by [HRS22].

Revised: We will describe these in more details in

Revised: Section 3 and 4.3.

Original: We will consider the original result from [HRS21] in Lemma B.1 and  a slightlystrengthened statement in Lemma C.6.
Revised: We use both the original result of [HRS22] (stated in Lemma B.1) and introduce a slightly strengthened version (in Lemma C.6).

Original: 2.2 Ofﬂine Optimization Oracle
Revised: 2.2 Ofﬂine Optimization Oracle

Original: We consider computationally efﬁcient algorithms given access to an ofﬂine optimization oracle.
Revised: We consider computationally efﬁcient algorithms given access to an ofﬂine optimization oracle.

Original: Forthe case of binary classiﬁcation, the oracle outputs the solution of empirical risk minimization on theinput data.
Revised: For the case of binary classiﬁcation, the oracle outputs the solution of empirical risk minimization on the input data.

Original: Deﬁnition 2.2 (ERM Oracle) .
Revised: Deﬁnition 2.2 (ERM Oracle) .

Original: For a hypothesis class H and a loss function l , the oracle OPT ( opt )takes a set 3 of inputs S
Revised: For a hypothesis class H and a loss function l , the oracle OPT ( opt ) takes a set 3 of inputs S

Original: =
Revised: =

Original: ( x i , y i ) } i 

Revised: ( x i , y i ) } i

Original: [ I ] where ( x i , y i ) 2 X ⇥ Y for all
Revised: [ I ] where ( x i , y i ) 2 X ⇥ Y for all

Original: i 2 [ I ] and returns
Revised: i 2 [ I ] and returns

Original: OPT H ,l ( S ) = min h 2 H
Revised: OPT H ,l ( S ) = min h 2 H

Original: I
Revised: I

Original: X i =
Revised: X i =

Original: l ( h ( x i ) , y i ) and opt H ,l ( S ) 2 arg min h 2 H
Revised: l ( h ( x i ) , y i ) and opt H ,l ( S ) 2 arg min h 2 H

Original: I
Revised: I

Original: X i =
Revised: X i =

Original: l ( h ( x i ) , y i ) .
Revised: l ( h ( x i ) , y i ) .

Original: For the case of real-valued functions, we consider an oracle that can minimize a mixture of binaryand real-valued loss values deﬁned below.
Revised: For the case of real-valued functions, we consider an oracle that can minimize a mixture of binary and real-valued loss values deﬁned below.

Original: Deﬁnition 2.3 (Real-valued optimization oracle) .
Revised: Deﬁnition 2.3 (Real-valued optimization oracle) .

Original: For a hypothesis class H and two loss functions l r and l b , the oracle OPT takes two sets of inputs S and S 0 over X ⇥ Y and returns
Revised: For a hypothesis class H and two loss functions l r and l b , the oracle OPT takes two sets of inputs S and S 0 over X ⇥ Y and returns

Original: [Equation]
Revised: [Equation]

Original: We remark that these oracles are used in most previous works, including [RSS12].
Revised: We remark that these oracles are used in most previous works, including [RSS12].

Original: They constitutea special form of regularized loss minimization oracles, where the regularization is given directlyby a random process.
Revised: They constitute a special form of regularized loss minimization oracles, where the regularization is given directly by a random process.

Original: For the binary setting where Y = {± 1 } and l r = l b = 1 { b y [Equation] } , the above167 optimization oracle is equivalent to ERM oracles.
Revised: For the binary setting where Y = {± 1 } and l r = l b = 1 { b y 6 = y } , the above optimization oracle is equivalent to ERM oracles.

Original: We consider each call to the ofﬂine optimization oracle as having unit cost plus the additional runtimeneeded for creating and inputting the set of inputs that is linear in the length of the said histories.
Revised: We consider each call to the ofﬂine optimization oracle as having unit cost plus the additional runtime needed for creating and inputting the set of inputs that is linear in the length of the said histories.

Original: We note that our approach and results directly extend to using ERM oracles with (arbitrarily small)additive approximation error, such as those guaranteed by FPTAS optimization algorithms, usingstandard techniques presented by [DHL + 17, Section 6]. 
Revised: We note that our approach and results directly extend to using ERM oracles with (arbitrarily small) additive approximation error, such as those guaranteed by FPTAS optimization algorithms, using standard techniques presented by [DHL + 20, Section 6]. Remark 1.

Original: Oracle-Efﬁcient Online Learning
Revised: Though the oracles as deﬁned above are required to work on arbitrary inputs, both Algorithms 1 and 2 from our work only call the optimization oracle on instances from the smoothed distributions or the uniform distribution. Thus, it sufﬁces to have oracles that work for average-case instances. This makes the design of such oracles easier both from theoretical and practical points of view.

Original: 3.1 Learning with Real-Valued Functions
Revised: 3.1 Learning with Real-Valued Functions

Original: In this section, we propose an oracle-efﬁcient algorithm for real-valued functions with regrete
Revised: In this section, we propose an oracle-efﬁcient algorithm for real-valued functions with regret e

Original: O ( p dT/  ) .
Revised: O ( p dT/  ) .

Original: We consider the optimization oracle deﬁned in Deﬁnition 2.3 with the loss functionsspeciﬁed by l r (ˆ y, y )
Revised: We consider the optimization oracle deﬁned in Deﬁnition 2.3 with the loss functions speciﬁed by l r (ˆ y, y )

Original: =
Revised: =

Original: 12 G l (ˆ y, y ) and l b (ˆ y, y ) =
Revised: 12 G l (ˆ y, y ) and l b (ˆ y, y ) =

Original: [Equation]
Revised: [Equation]

Original:  12 .

Original: We begin by describing our algorithm.
Revised: We begin by describing our algorithm.

Original: At each time step t 2 [ T ] , the algorithm draws e
Revised: At each time step t 2 [ T ] , the algorithm draws e

Original: O ⇣
Revised: O ⇣

Original: T  t  ⌘ fresh new instances from the uniform distribution, denoted with V ( t ) , together with their randomlabels E ( t ) that are drawn i.i.d.
Revised: [Equation]  t  ⌘ fresh new instances from the uniform distribution, denoted with V ( t ) , together with their random labels E ( t ) that are drawn i.i.d.

Original: from the Rademacher distribution, and treat them as hints for thefuture.
Revised: from the Rademacher distribution, and treat them as hints for the future.

Original: Let S ( t ) denote the set of labeled instances ( V ( t ) , E ( t ) ) .
Revised: Let S ( t ) denote the set of labeled instances ( V ( t ) , E ( t ) ) .

Original: Our algorithm then applies the ofﬂineoptimization oracle to two input sequences: one where the real history s 1: t  1 is mixed with twocopies 4 of S ( t ) and the current instance is labeled +1 , and another, where the current label is labeled 1 .
Revised: Our algorithm then applies the ofﬂine optimization oracle to two input sequences: one where the real history s 1: t  1 is mixed with two copies 4 of S ( t ) and the current instance is labeled +1 , and another, where the current label is labeled  1 .

Original: Formally, we consider
Revised: Formally, we consider

Original: [Equation]
Revised: [Equation]

Original: Since the two input sequences to the optimization oracle only disagree on one label, the difference inthe optimal errors is alwaysbounded within [  1 , +1] , thus guarantees b y t 2 Y .
Revised: Since the two input sequences to the optimization oracle only disagree on one label, the difference in the optimal errors is always within [  1 , +1] , thus guarantees b y t 2 Y .

Original: Intuitively, the reasonˆ y t includes the gap between the error of these two optimal classiﬁers is to make the algorithm hedgeits bets against which instances will be generated by the adversary next.
Revised: Intuitively, ˆ y t accounts for the gap between the errors of these two optimal classiﬁers so as to hedge the algorithm’s bets against which instances will be played next.

Original: A formal description of thealgorithm is given in Algorithm 1.
Revised: A formal description of the algorithm is given in Algorithm 1.

Original: The main motivation for the algorithm is the coupling lemma from [HRS21].
Revised: The main motivation for the algorithm is the coupling lemma of [HRS22].

Original: It states that a samplefrom any  -smooth distribution can be thought of as generated by ﬁrst sampling O    samplesfrom the uniform distribution and then selecting one of them as the sample.
Revised: It states that a sample from any  -smooth distribution can be thought of as generated by ﬁrst sampling O    samples from the uniform distribution and then selecting one of them.

Original: The algorithm thus canbe thought of as generating samples from the uniform distribution to account for the uncertainty inthe choice of the adversary.
Revised: The algorithm thus can be thought of as generating samples from the uniform distribution to account for the uncertainty in the adversary’s choice.

Original: We will discuss this intuition and sketch a proof of the following theoremin Section 4.3.
Revised: We will discuss this intuition and the proof of the following theorem in Section 4.3.

Original: Theorem 3.1 (Regret Upper Bound) .
Revised: Theorem 3.1 (Regret Upper Bound) .

Original: For any  -smooth adversary D  , Algorithm 1 has expectedregret upper bounded by e
Revised: For any  -smooth adversary D  , Algorithm 1 has expected regret upper bounded by e

Original: O ( G p
Revised: O ( G p

Original: Td/  ) , where e
Revised: Td/  ) , where e

Original: O hides factors that are polynomial in log( T ) andlog(1 /  ) .
Revised: O hides factors that are polynomial in log( T ) and log(1 /  ) .

Original: Here G is the Lipschitz constant of the loss and d is the pseudodimension of the class.
Revised: Here G is the Lipschitz constant of the loss and d is the pseudo-dimension of the class.

Original: Furthermore, the algorithm is oracle-efﬁcient: at every round t , this algorithm uses two oracle callswith histories of length e
Revised: Furthermore, the algorithm is oracle-efﬁcient: at every round t , this algorithm uses two oracle calls with histories of length e

Original: O ( T/  ) .
Revised: O ( T/  ) .

Original: Input: T,  1 K 100 log T/  . 2 for t 1 to T do
Revised: Input

Original: Receive x t . for i = t +1 , · · · , T ; k = 1 , · · · , K do Draw new v ( t ) i,k
Revised: T o

Original: ⇠ U ( X ) .
Revised: ⇣ s t  ( x

Original: Draw new ✏ ( t ) i,k ⇠ U ( {  1 , +1 } ) . end S ( t ) n ( v ( t ) i,k , ✏ ( t ) i,k ) o i = t +1:
Revised: ⇣

Original: T k =1: K . b y t OPT ⇣ s 1: t  1 ; S ( t ) [ S (t ) [ { ( x t ,  1) } ⌘ 
Revised: [ S (

Original: OPT ⇣ s 1: t  1 ; S ( t ) [ S ( t ) [ { ( x t , +1) } ⌘. Receive y t , suffer loss l ( b y t , y t ) . 11 end
Revised: ⌘

Original: 3.2 Improved Bounds for Binary Classiﬁcation
Revised: 3.2 Improved Bounds for Binary Classiﬁcation

Original: In this section, we focus on the important special case where the labels are binary and the loss functionis the classiﬁcation loss 1 { b y 

Revised: In this section, we focus on the important special case where the labels are binary and the loss function is the classiﬁcation loss 1 { b y

Original: = y } .
Revised: = y } .

Original: We present Algorithm 2 that achieves regret e
Revised: We present Algorithm 2 that achieves regret e

Original: O ( p Td   1 / 2 ) withbetter dependence on the smoothness parameter  compared to Algorithm 1.
Revised: O ( p Td   1 / 2 ) with better dependence on the smoothness parameter  compared to Algorithm 1.

Original: Theorem 3.2 (Regret Bound for Efﬁcient Smoothed Online Learning) .
Revised: Theorem 3.2 (Regret Bound for Efﬁcient Smoothed Online Learning) .

Original: In the setting of binaryclassiﬁcation with  -smoothed adversaries, Algorithm 2 has expected regret that is at moste O min ⇢p Td
Revised: In the setting of binary classiﬁcation with  -smoothed adversaries, Algorithm 2 has expected regret that is at most

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: , q [Equation] ! . Furthermore, Algorithm 2 is a proper learning oracle-efﬁcient algorithm: at every round t , thisalgorithm uses a single ERM oracle call on a history that is of length t + O ( T/ p  ) with highprobability.
Revised: Furthermore, Algorithm 2 is a proper learning oracle-efﬁcient algorithm: at every round t , this algorithm uses a single ERM oracle call on a history that is of length t + O ( T/ p  ) with high probability.

Original: Unlike the algorithm for the real-valued case, the improved algorithm uses the FTPL framework.
Revised: Unlike the algorithm for the real-valued case, the improved algorithm uses the FTPL framework.

Original: Thealgorithm itself is easy to describe: the algorithm generates a random number N from the Poissondistribution with an appropriately chosen parameter, generates N uniformly random points alongwith random labels and then predicts using the hypothesis that has the lowest error on the past dataappended with the newly sampled data.
Revised: The algorithm itself is easy to describe: the algorithm generates a random number N from the Poisson distribution with an appropriately chosen parameter, generates N uniformly random points along with random labels and then predicts using the hypothesis that has the lowest error on the past data appended with the newly sampled data.

Original: On the surface, this algorithm seems similar to Algorithm 1.
Revised: On the surface, this algorithm seems similar to Algorithm 1.

Original: But there are two key differences: unlike Algorithm 1 which uses the difference in value of twooptimizations, Algorithm 2 follows the prediction of the hypothesis with the lowest error.
Revised: But there are two key differences: unlike Algorithm 1 which uses the difference in value of two optimizations, Algorithm 2 follows the prediction of the hypothesis with the lowest error.

Original: Thismakes Algorithm 2 a proper online learning algorithm.
Revised: This makes Algorithm 2 a proper online learning algorithm.

Original: Secondly, unlike Algorithm 1 which uses adecreasing number of random examples over time, Algorithm 2 has the number of samples distributedaccording to a Poisson (with the same parameter in each step).
Revised: Secondly, unlike Algorithm 1 which uses a decreasing number of random examples over time, Algorithm 2 has the number of samples distributed according to a Poisson (with the same parameter in each step).

Original: The fact that the number of hints isdrawn from the Poisson distribution is crucial for our analysis of the stability of the algorithm.
Revised: The fact that the number of hints is drawn from the Poisson distribution is crucial for our analysis of the stability of the algorithm.

Original: Wesketch the proof of the regret bounds in Section 4.4.
Revised: We sketch the proof of the regret bounds in Section 4.4.

Original: Algorithm 2: Smoothed Online Learning based on Poisson Number of Hints
Revised: Algorithm 2: Smoothed Online Learning based on Poisson Number of Hints

Original: Input: time horizon T , smoothness parameter  , VC dimension d 1 n min  , T  hallucinated  are .d
Revised: Input: time horizon T , smoothness parameter  , VC dimension d 1 n min { T/ p  , T p |X| /d } ; 2 for t 1 to T do generate N ( t ) ⇠ Poi ( n ) fresh hallucinated samples ( e x ( t ) 1 , e y ( t ) 1 ) , · · · , ( e x ( t ) N , e y ( t ) N ) , which are i.i.d.

Original:  N call 6 end
Revised: conditioned on N with e x ( t ) i ⇠ U ( X ) and e y

Original: While our main interest is on beyond the worst-case adversaries, our results improve upon existingresults for worst-case analysis of online learning as well.
Revised: While our main interest is on beyond the worst-case adversaries, our results improve upon existing results for worst-case analysis of online learning as well.

Original: For ﬁnite domain and binary-valued losssettings where worst-case adversaries are vacuously  -smooth for 1 /  = |X| , Theorem 3.2 alsoachieves an oracle-efﬁcient regret bound of O ( p
Revised: For ﬁnite domain and binary-valued loss settings where worst-case adversaries are vacuously  -smooth for 1 /  = |X| , Theorem 3.2 also achieves an oracle-efﬁcient regret bound of O ( p

Original: T ( d |X| ) 1 / 2 ) , which is a reﬁnement of O ( p
Revised: T ( d |X| ) 1 / 2 ) , which is a reﬁnement of O ( p

Original: T |X| )
Revised: T |X| )

Original: 226 bound of [DS16], because VC dimension d is at most X , and is usually much smaller.
Revised: bound of [DS16], because VC dimension d is at most X , and is usually much smaller.

Original: Similarly,our bound can be instantiated in the setting of transductive learning with |X| =
Revised: Similarly, our bound can be instantiated in the setting of transductive learning with |X| =

Original: T , which improves
Revised: T , which improves

Original: O ( T 3 / 4 p d ) bound of [KK06] to O
Revised: O ( T 3 / 4 p d ) bound of [KK05] to O

Original: ( T 3 / 4 d 1 / 4 ) .
Revised: ( T 3 / 4 d 1 / 4 ) .

Original: Corollary 3.3 (Regret for Small Domain) .
Revised: Corollary 3.3 (Regret for Small Domain and Transductive Learning) .

Original: There is an oracle-efﬁcient algorithm for online learningwith binary labels (in the worst-case) that achieves an expected regret of O ( p
Revised: There is an oracle-efﬁcient algorithm for online learning with binary labels (in the worst-case) that achieves an expected regret of O ( p

Original: T ( d |X| ) 1 / 2 ) for any231 hypothesis class with VC dimension d on domain X .
Revised: T ( d |X| ) 1 / 2 ) for any hypothesis class with VC dimension d on domain X .

Original: For transductive learning with binary labels,there is an oracle efﬁcient algorithm, with expected regret O ⇣
Revised: For transductive learning with binary labels, there is an oracle-efﬁcient algorithm, with expected regret O ⇣

Original: T 3 / 4 d 1 /⌘ .
Revised: T 3 / 4 d 1 /⌘ .

Revised: Remark 2.

Original: Proof Sketches for Main Regret Bounds
Revised: Both Algorithms 1 and 2 can be adapted to deal with unknown  . In particular, the regret bounds hold for any approximation e  that is a lower bound of the real  up to constant multiplicative factors. This corresponds to settings where the world is more smooth than we give it credit. Even when we have extremely poor upper and lower bounds, we can use hedging to still get non-trivial regret with only a minor blow up in computation.

Revised: We will provide more details in Appendix D about working with knowledge of approximate e  .

Original: Before discussing the proof sketches for the main regret bounds, we will ﬁrst introduce two frameworks for designing efﬁcient algorithms for online learning.
Revised: Before discussing the proof sketches for the main regret bounds, we will ﬁrst introduce two frameworks for designing efﬁcient algorithms for online learning.

Original: 4.1 Relaxations and Admissibility
Revised: 4.1 Relaxations and Admissibility

Original: The proof of Theorem 3.1 relies on the admissible relaxation framework proposed in [RSS12].
Revised: The proof of Theorem 3.1 relies on the admissible relaxation framework proposed in [RSS12].

Original: Arelaxation Rel T is a sequence of functions Rel T ( H| s 1: t ) for each t 2 [ T ] , which map the history of239 the play to real values that upper bounds the conditional value of the game.
Revised: A relaxation Rel T is a sequence of functions Rel T ( H| s 1: t ) for each t 2 [ T ] , which map the history of the play to real values that upper bounds the conditional value of the game.

Original: We will make use of animportant algorithmic aspect of the relaxation framework, which states that whenever an algorithm isadmissible with respect to some relaxation, its expected regret can be upper bounded in terms of thevalue of the relaxation at the beginning of the game.
Revised: We will make use of an important algorithmic aspect of the relaxation framework, which states that whenever an algorithm is admissible with respect to some relaxation, its expected regret can be upper bounded in terms of the value of the relaxation at the beginning of the game.

Original: Deﬁnition 4.1 (Admissibility) .
Revised: Deﬁnition 4.1 (Admissibility) .

Original: In the smoothed online learning setting, let Q be an algorithm thatgives rise to a sequence of distributions Q 1 , · · · , Q T on the predicted labels.
Revised: In the smoothed online learning setting, let Q be an algorithm that gives rise to a sequence of distributions Q 1 , · · · , Q T on the predicted labels.

Original: We say Q is admissible245 with respect to a relaxation{ Rel T
Revised: We say Q is admissible with respect to a relaxation

Original: ( H | s 1: t ) } Tt =0 , if for any sequence of instances s 1: T ,
Revised: [Equation]

Original: For all t 2 [ T ] ,
Revised: For all t 2 [ T ] ,

Original: [Equation]
Revised: [Equation]

Original: where   ( X ) is the set of  -smooth distributions on X;
Revised: where   ( X ) is the set of  -smooth distributions on X

Original: The ﬁnal value satisﬁesRel T ( H | s 1: T )   inf h 2 H L ( h, s 1: T ) .
Revised: The ﬁnal value satisﬁes

Original:  The following proposition is the analog of the results of [RSS12] when the adversary is smooth.
Revised: [Equation] The following proposition is the analog of the results of [RSS12] when the adversary is smooth.

Original: Thefull proof is presented in Appendix F.
Revised: The full proof is presented in Appendix G.

Original: Proposition 4.1 (Regret Bound via Admissibility) .
Revised: Proposition 4.1 (Regret Bound via Admissibility) .

Original: In the smoothed online learning setting, let
Revised: In the smoothed online learning setting, let

Original: Q =
Revised: Q =

Original: ( Q 1 , · · · , Q T ) be an algorithm that is admissible with respect to relaxations
Revised: ( Q 1 , · · · , Q T ) be an algorithm that is admissible with respect to relaxations

Original: Rel T
Revised: Rel T

Original: ( H ) , then253 the following bound on the expected regret holds regardless of the strategies D  of the adversary:
Revised: ( H ) , then the following bound on the expected regret holds regardless of the strategies D  of the adversary:

Original: [Equation]
Revised: [Equation]

Original: 4.2 Follow the Perturbed Leader
Revised: 4.2 Follow the Perturbed Leader

Original: When the labels are binary, Algorithm 2 achieves an improved regret using the Follow the Perturbed
Revised: When the labels are binary, Algorithm 2 achieves an improved regret using the Follow the Perturbed

Original: Leader (FTPL) principle [KV05].
Revised: Leader (FTPL) principle [KV05].

Original: An FTPL algorithm makes predictions by applying ERM oracle tothe perturbed histories of the play.
Revised: An FTPL algorithm makes predictions by applying ERM oracle to the perturbed histories of the play.

Original: At every time step t 2 [ T ] , the algorithm chooses a distributionover labeled instances, from which it draws N random instances ( e [Equation] ( t ) 1 ) , · · · , ( e x ( t ) N , e y ( t ) N ) .
Revised: At every time step t 2 [ T ] , the algorithm chooses a distribution over labeled instances, from which it draws N random instances ( e x ( t ) 1 , e y ( t ) 1 ) , · · · , ( e x ( t ) N , e y ( t ) N ) .

Original: Thepredicted label is then given byb y t = h t ( x t ) , whereh t opt H ,l ⇣
Revised: The predicted label is then given by

Original: [Equation]
Revised: [Equation]

Original: 1: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: { ( e x( t )i , e y ( t ) i )
Revised: x t ) , where

Original: [Equation]
Revised: [Equation]

Original: N ] ⌘ .

Original: The standard analysis of FTPL bounds the expected regret as follows:
Revised: The standard analysis of FTPL bounds the expected regret as follows:

Original: [Equation]
Revised: [Equation]

Original: Stability

Original: [Equation]

Original: | {z }

Original: Perturbation

Original: [Equation]

Original: where h ⇤ = arg inf h 2 H
Revised: where h ⇤ = arg inf h 2 H

Original: P
Revised: P

Original: Tt =1 l ( h ( x t ) , y t ) .
Revised: Tt =1 l ( h ( x t ) , y t ) .

Original: Note that the perturbation term is already well-understood from statistical learning theory since it isessentially the Rademacher complexity of H for sample size N .
Revised: Note that the perturbation term is already well-understood from statistical learning theory since it is essentially the Rademacher complexity of H for sample size N .

Original: Therefore, we will focus on boundingthe stability term by designing perturbations that can leverage the anti-concentration property ofsmoothed adversaries.
Revised: Therefore, we will focus on bounding the stability term by designing perturbations that can leverage the anti-concentration property of smoothed adversaries.

Original: 4.3 Proof Sketch of Theorem 3.1 
Revised: 4.3 Proof Sketch of Theorem 3.

Original: Elaborating on the intuition laid out in Section 3, we will use the coupling technique introducedby [HRS21] (see Lemma B.1 for a complete description) to replace the sequence of T randominputs { x 1 , · · · , x T } generated by the adaptive adversary with TK inputs { z t,k } t 2 [ T ] ,k [-
-] [ K ] that are270 generated i.i.d.
Revised: Elaborating on the intuition laid out in Section 3, we will use the coupling technique introduced by [HRS22] (see Lemma B.1 for a complete description) to replace the sequence of T random inputs { x 1 , · · · , x T } generated by the adaptive adversary with TK inputs { z t,k } t 2 [ T ] ,k [-
-]2 [ K ] that are generated i.i.d.

Original: from the uniform distribution over
Revised: from the uniform distribution over

Original: X , such that with high probability { x 1 , · · · , x T } ✓{ z t,k } t 2 [ T ] ,k 2 [ K ] .
Revised: X , such that with high probability { x 1 , · · · , x T } ✓ { z t,k } t 2 [ T ] ,k 2 [ K ] .

Original: This implies that, up to a small probability of failure, it is sufﬁcient to consider a272 simpler setting where the adversary is promised to pick future instances from a larger set of uniformlydistributed samples (which we call the set of hints ).
Revised: This implies that, up to a small probability of failure, it is sufﬁcient to consider a simpler setting where the adversary is promised to pick future instances from a larger set of uniformly distributed samples (which we call the set of hints ).

Original: This setting differs from the standard transductivelearning setting [KK06, CbS11] in two signiﬁcant ways: 1) The set of hints is not revealed to thelearner beforehand; 2) the hint set is larger, by a multiplicative factor of K ⇡ 1 /  , than the set ofrealized instances.
Revised: This setting differs from the standard transductive learning setting [KK05, CbS11] in two signiﬁcant ways: 1) The set of hints is not revealed to the learner beforehand; 2) the hint set is larger, by a multiplicative factor of K ⇡ 1 /  , than the set of realized instances.

Original: It turns out that both issues can be handled elegantly in the admissible relaxations framework of [RSS12].
Revised: It turns out that both issues can be handled elegantly in the admissible relaxations framework of [RSS12].

Original: For the ﬁrst issue, note that the Algorithm 1 can be seen as self-generating hints and although they do not necessarily correspond to the adversary’s sample, the relaxation-based argument guarantees that matching the randomness of hints at a distribution level sufﬁces to bound the regret of the algorithm (see Appendix B.7 for more details).
Revised: For the ﬁrst issue, note that the Algorithm 1 can be seen as self-generating hints and although they do not necessarily correspond to the adversary’s sample, the relaxation-based argument guarantees that matching the randomness of hints at a distribution level sufﬁces to bound the regret of the algorithm (see Appendix B.7 for more details).

Original: For the second issue, our relaxation will be based on a characterization of the uncertainty in the future that is monotone in the set of hints, which we call regularized Rademacher complexity .
Revised: For the second issue, our relaxation will be based on a characterization of the uncertainty in the future that is monotone in the set of hints, which we call regularized Rademacher complexity .

Original: Formally, for a set of unlabeled instances Z = { z i } Ii =1 and a function  : H !
Revised: Formally, for a set of unlabeled instances Z = { z i } Ii =1 and a function  : H !

Original: R , the Rademacher complexity for set Z regularized by  is deﬁned as
Revised: R , the Rademacher complexity for set Z regularized by  is deﬁned as

Original: [Equation]
Revised: [Equation]

Original:  We show that regularized Rademacher complexity is monotone as a function of the dataset.
Revised: oi . We show that regularized Rademacher complexity is monotone as a function of the dataset.

Original: See
Revised: See

Original: Appendix B.2 for a proof of Lemma 4.1.
Revised: Appendix B.2 for a proof of Lemma 4.1.

Original: Lemma 4.1 (Monotonicity of Regularized Rademacher Complexity) .
Revised: Lemma 4.1 (Monotonicity of Regularized Rademacher Complexity) .

Original: For any dataset z 1: m 2 X m280 and any additional data point x 2 X , we have R (  , z 1: m )  R (  , z 1: m [ { x } ) .
Revised: For any dataset z 1: m 2 X m and any additional data point x [Equation] , we have [Equation]

Original: This monotonicity ensures that using the hint set, which is a superset of possible instances, will stilllead to a no-regret algorithm.
Revised: This monotonicity ensures that using the hint set, which is a superset of possible instances, will still lead to a no-regret algorithm.

Original: Finally, the relaxation we use to analyze Algorithm 1 is the expected Rademacher complexity of theunion of future hints, regularized by the past total loss, i.e., 
Revised: Finally, the relaxation we use to analyze Algorithm 1 is the expected Rademacher complexity of the union of future hints, regularized by the past total loss, i.e., [Equation] i

Original: [Equation]
Revised: [Equation]

Original: where L r ( h, s 1: t ) =
Revised: where L r ( h, s 1: t ) =

Original: P ti =1 l r ( h ( x i ) , y i ) for h 2 H .
Revised: P ti =1 l r ( h ( x i ) , y i ) for h 2 H .

Original: Here  = 10 TK (1   ) K 2 o (1 /T ) representsthe penalty caused by the failure of coupling.
Revised: Here  = 10 TK (1   ) K 2 o (1 /T ) represents the penalty caused by the failure of coupling.

Original: Once admissibility is established, we will obtain aregret bound using Proposition 4.1.
Revised: Once admissibility is established, we will obtain a regret bound using Proposition 4.1.

Original: See Appendix B for a complete proof of the theorem.
Revised: See Appendix B for a complete proof of the theorem.

Original: We remarkthat the ﬁnal regret bound can also be stated in terms of the Rademacher complexity which is datadependent and adapts readily to benign structures of the instance domain.
Revised: We remark that the ﬁnal regret bound can also be stated in terms of the Rademacher complexity which is data dependent and adapts readily to benign structures of the instance domain.

Original: 4.4 Proof Sketch of Theorem 3. 
Revised: 4.4 Proof Sketch of Theorem 3.

Original: Let Q t be the distribution of the learner’s action h t 2 H in Algorithm 2 and D t denote the distribution292 of the adversary at time t .
Revised: Let Q t be the distribution of the learner’s action h t 2 H in Algorithm 2 and D t denote the distribution of the adversary at time t .

Original: The main quantity we will use to analyze the algorithm is the stabilityStability
Revised: The main quantity we will use to analyze the algorithm is the stability

Original: [Equation]
Revised: [Equation]

Original: We analyze this expression by breaking it down into a sum of two quantities:
Revised: We analyze this expression by breaking it down into a sum of two quantities:

Original: [Equation]
Revised: [Equation]

Original: | {z }

Original: Modiﬁed

Original: [Table] error . Here, R ( t ) is the fresh randomness generated by the algorithm at the beginning of time t .
Revised: Here, R ( t ) is the fresh randomness generated by the algorithm at the beginning of time t .

Original: In order to see where this expression comes from, note that the ﬁrst term itself would be an upperbound on the stability, if neither of Q t and Q t +1 depend on the new observation s t = ( x t , y t ) at time297 t .
Revised: In order to see where this expression comes from, note that the ﬁrst term itself would be an upper bound on the stability, if neither of Q t and Q t +1 depend on the new observation s t = ( x t , y t ) at time t .

Original: However, while
Revised: However, while

Original: Q t is independent of s t , Q t +1 does depend on s t because h t +1 is trained on s t .
Revised: Q t is independent of s t , Q t +1 does depend on s t because h t +1 is trained on s t .

Original: To298 overcome this dependence, we introduce a ghost sample st that allows us to decouple h t +1 ⇠ Q t +1299 and the new observation.
Revised: To overcome this dependence, we introduce a ghost sample s 0 t that allows us to decouple h t + ⇠ Q t +1 and the new observation.

Original: This gives rise to the second term which we call modiﬁed generalizationerror .
Revised: This gives rise to the second term which we call modiﬁed generalization error .

Original: We formally discuss this decomposition in Appendix C.2.
Revised: We formally discuss this decomposition in Appendix C.2.

Original: The ﬁrst term is the total variation (TV) distance between Q t and the mixture distribution
Revised: The ﬁrst term is the total variation (TV) distance between Q t and the mixture distribution

Original: E s t ⇠ D t [ Q t +1 ] .
Revised: E s t ⇠ D t [ Q t +1 ] .

Original: In order to bound this term, we closely use the independence properties of the303 Poisson distribution which allows us to write an explicit expression for the total variation distancewhich we can then bound using the Ingster–Suslina method.
Revised: In order to bound this term, we closely use the independence properties of the Poisson distribution which allows us to write an explicit expression for the total variation distance which we can then bound using the Ingster–Suslina method.

Original: We formally prove this in Lemma C.4.
Revised: We formally prove this in Lemma C.4.

Original: The intuitive idea behind bounding the second term is as follows.
Revised: The intuitive idea behind bounding the second term is as follows.

Original: Consider the simpler setting oft = 1 (i.e. no history) and D t =
Revised: Consider the simpler setting of t = 1 (i.e. no history) and D t =

Original: U ( X ⇥ {± 1 } ) (i.e. the new observation s t follows the same307 distribution as the self-generated samples).
Revised: U ( X ⇥ {± 1 } ) (i.e. the new observation s t follows the same distribution as the self-generated samples).

Original: In this case, the generalization error is precisely thedifference between the test error and the training error with N + 1 iid training data, and classical
Revised: In this case, the generalization error is precisely the difference between the test error and the training error with N + 1 iid training data, and classical

Original: Rademacher complexity gives an upper bound O ( p d/N ) .
Revised: Rademacher complexity gives an upper bound O ( p d/N ) .

Original: For general  -smooth D t , we establish astrong conditional independence property of the coupling argument from [HRS21].
Revised: For general  -smooth D t , we establish a strong conditional independence property of the coupling argument from [HRS22].

Original: This states thatthere exists a coupling between uniform and adaptive smooth processes, such that when the inclusionproperty is satisﬁed, the distribution of the realized uniform variables conditional on the unrealizeduniform variables is also identical to the smooth distributions given by the adversary.
Revised: This states that there exists a coupling between uniform and adaptive smooth processes, such that when the inclusion property is satisﬁed, the distribution of the realized uniform variables conditional on the unrealized uniform variables is also identical to the smooth distributions given by the adversary.

Original: This will beinstrumental for bounding the generalization error by allowing us to extract smooth variables from aset of uniform variables, which can then be used to for the purpose of symmetrization.
Revised: This will be instrumental for bounding the generalization error by allowing us to extract smooth variables from a set of uniform variables, which can then be used to for the purpose of symmetrization.

Original: We formallyprove the upper bound on the modiﬁed generalization in Lemma C.5.
Revised: We formally prove the upper bound on the modiﬁed generalization in Lemma C.5.

Original: Using this bound on stability, we can get a bound on the regret of the algorithm using analysistechniques for FTPL.
Revised: Using this bound on stability, we can get a bound on the regret of the algorithm using analysis techniques for FTPL.

Original: For a full proof, see Appendix C.
Revised: For a full proof, see Appendix C.

Original: Remark 1.
Revised: Remark 3.

Original: The proof for the modiﬁed generalization needs the smoothness of both the covariates xand labels y .
Revised: The proof for the modiﬁed generalization needs the smoothness of both the covariates x and labels y .

Original: This can be ensured with a loss of a constant in the binary (and generally the ﬁnite labelspace) setting.
Revised: This can be ensured with a loss of a constant in the binary (and generally the ﬁnite label space) setting.

Original: This is the main reason that this proof does not generalize to the real valued setting.
Revised: This is the main reason that this proof does not generalize to the real valued setting.

Original:  Discussion, Additional Results, and Open Problems
Revised:  Discussion, Additional Results, and Open Problems

Original: Computational Lower Bounds.
Revised: Computational Lower Bounds.

Original: Our main contribution is oracle-efﬁcient online learning algorithms in the smoothed setting that achieve an O ( p dT   1 ) regret upper bound in the real-valuedcase, and an O ( p dT   1 / 2 ) upper bound for binary classiﬁcation.
Revised: Our main contribution is oracle-efﬁcient online learning algorithms in the smoothed setting that achieve an O ( p dT   1 ) regret upper bound in the real-valued case, and an O ( p dT   1 / 2 ) upper bound for binary classiﬁcation.

Original: However, neither of these upperbounds is statistically optimal; the statistically optimal regret here ise ⇥ ( p dT log(1 /  ))
Revised: However, neither of these upper bounds is statistically optimal; the statistically optimal regret here is

Original: [HRS21].
Revised: [Equation]

Original: We ask the following question: is the above discrepancy an artifact of our regret analysis, or anintrinsic limitation of our or all oracle-efﬁcient algorithms.
Revised: We ask the following question: is the above discrepancy an artifact of our regret analysis, or an intrinsic limitation of our or all oracle-efﬁcient algorithms.

Original: We show in Theorem D.1 that the regret of our current family of algorithms cannot be improved bytuning parameters.
Revised: We show in Theorem E.1 that the regret of our current family of algorithms cannot be improved by tuning parameters.

Original: In addition, we also show the following general lower bound for computationallyefﬁcient algorithms.
Revised: In addition, we also show the following general lower bound for computationally efﬁcient algorithms.

Original: See Appendix D for the proofs and more details.
Revised: See Appendix E for the proofs and more details.

Original: Theorem 5.1 (Computational Lower Bound for Smoothed Online Learning) .
Revised: Theorem 5.1 (Computational Lower Bound for Smoothed Online Learning) .

Original: For 1 /  d , any proper algorithm which only has access to the ERM oracle and achieves a regreto (min { T, p
Revised: For 1 /   d , any proper algorithm which only has access to the ERM oracle and achieves a regret o (min { T, p

Original: T ( d/  ) 1 / 2 } ) for any  -smoothed online learning problem must have an !
Revised: T ( d/  ) 1 / 2 } ) for any  -smoothed online learning problem must have an !

Original: ( p d/  )total running time.
Revised: ( p d/  ) total running time.

Original: Here, the total running time refers to the total number of operations performed byour algorithm, in which each oracle call takes unit time, and maintaining each element in the input tothe oracle also takes unit time.
Revised: Here, the total running time refers to the total number of operations performed by our algorithm, in which each oracle call takes unit time, and maintaining each element in the input to the oracle also takes unit time.

Original: Theorem 5.1 implies an exponential statistical-computational gap in smoothed online learning: forexponentially small  , achieving the statistical regret e
Revised: Theorem 5.1 implies an exponential statistical-computational gap in smoothed online learning: for exponentially small  , achieving the statistical regret e

Original: O ( p
Revised: O ( p

Original: Td log(1 /  )) requires an exponentialrunning time.
Revised: Td log(1 /  )) requires an exponential running time.

Original: However, Theorem 5.1 still exhibits gaps to our computational upper bounds.
Revised: However, Theorem 5.1 still exhibits gaps to our computational upper bounds.

Original: Wediscuss this further in the appendix and present the following open problem.
Revised: We discuss this further in the appendix and present the following open problem.

Original: Open Question.
Revised: Open Question.

Original: For d/   T 2 in the smoothed setting, does any algorithm achieving o ( T ) regretrequire ⌦ ( poly ( T, 2 d , 1 /  ))
Revised: For d/   T 2 in the smoothed setting, does any algorithm achieving o ( T ) regret require ⌦ ( poly ( T, 2 d , 1 /  ))

Original: computational time given access to the ERM oracle?
Revised: computational time given access to the ERM oracle?

Original: Check List
Revised: Other Constrained Adversary Models Online learning in presence of constrained adversaries is an active research area with several parallel proposed models. While the main focus of our work is on the smoothed analysis framework, our techniques extend to several other frameworks and establish a connection between existing lines of work. In particular, our application of the probability coupling technique to hallucinate a superset of future instances relates to an extension of the framework of transductive learning (and the literature on hints) to a setting where the learner has access to K hints per time step in the future. We call this the K -hints transductive framework and formally address it in

Original: For all authors...
Revised: Appendix H. More broadly, smoothed analysis also related to a line of work on predictable sequences as it enforces a notion of similarity between future instances that holds at the stochastic level. We believe exploring the connections between these various models of constrained and predictable adversaries in online learning is an important research direction for the future.  Check List

Original:  (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A] (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
Revised: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [N/A] (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?[N/A]
Revised: [N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original:  (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A] 
Revised: [N/A] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?[N/A]
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original:  (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?[N/A]
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original:  (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]