Original: Oracle-Efﬁcient Online Learning for Smoothed Adversaries (with Appendices)
Revised: Oracle-Efﬁcient Online Learning for Smoothed Adversaries Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: In this paper, we study oracle-efﬁcient algorithms for smoothed analysis of onlinelearning.
Revised: In this paper, we study oracle-efﬁcient algorithms for smoothed analysis of onlinelearning.

Original: In this setting, an adversary is constrained to generating samples fromdistributions whose density is upper bounded by 1 /  times the uniform density.
Revised: In this setting, an adversary is constrained to generating samples fromdistributions whose density is upper bounded by 1 /  times the uniform density.

Original: Given access to an ofﬂine optimization oracle, we give the ﬁrst computationallyefﬁcient online algorithms whose sublinear regret depends only on the pseudo/VCdimension d of the class and the smoothness parameter  .
Revised: Given access to an ofﬂine optimization oracle, we give the ﬁrst computationallyefﬁcient online algorithms whose sublinear regret depends only on the pseudo/VCdimension d of the class and the smoothness parameter  .

Original: In particular, weachieve oracle-efﬁcient regret bounds of O ( p Td   1 ) for learning real-valuedfunctions and O ( p
Revised: In particular, weachieve oracle-efﬁcient regret bounds of O ( p Td   1 ) for learning real-valuedfunctions and O ( p

Original: Td   12 ) for learning binary-valued functions.
Revised: Td   12 ) for learning binary-valued functions.

Original: This contraststhe computational separation between online learning with worst-case adversariesand ofﬂine learning established by [HK16].
Revised: This contraststhe computational separation between online learning with worst-case adversariesand ofﬂine learning established by [HK16].

Original: In the binary setting, our algorithmsalso achieve improved bounds for worst-case setting with small domains.
Revised: In the binary setting, our algorithmsalso achieve improved bounds for worst-case setting with small domains.

Original: Inparticular, we give an oracle-efﬁcient algorithm with regret of O ( p
Revised: Inparticular, we give an oracle-efﬁcient algorithm with regret of O ( p

Original: T ( d |X| ) 1 / 2 ) ,which is a reﬁnement of the earlier O ( p
Revised: T ( d |X| ) 1 / 2 ) ,which is a reﬁnement of the earlier O ( p

Original: T |X| ) bound by [DS16].
Revised: T |X| ) bound by [DS16].

Original:  Introduction
Revised:  Introduction

Original: Adversarial online learning is a cornerstone of modern machine learning and has led to signiﬁcantadvances in computer science broadly.
Revised: Adversarial online learning is a cornerstone of modern machine learning and has led to signiﬁcantadvances in computer science broadly.

Original: A recent line of work on “beyond the worst-case analysis” ofonline learning has brought into light the overly pessimistic nature of standard characterizations ofonline learnability [RST11, GR17, HRS20, HRS21].
Revised: A recent line of work on “beyond the worst-case analysis” ofonline learning has brought into light the overly pessimistic nature of standard characterizations ofonline learnability [RST11, GR17, HRS20, HRS21].

Original: This is exempliﬁed by the results of [HRS21]showing that adversarial online learnability is statistically as easy as PAC learnability, in presenceof noise.
Revised: This is exempliﬁed by the results of [HRS21]showing that adversarial online learnability is statistically as easy as PAC learnability, in presenceof noise.

Original: That is, under smoothed analysis , online and ofﬂine learnability are both characterized bythe ﬁniteness of the VC dimension of a hypothesis class as opposed to the much larger Littlestonedimension that characterizes online learnability in the worst-case [BDPSS09].
Revised: That is, under smoothed analysis , online and ofﬂine learnability are both characterized bythe ﬁniteness of the VC dimension of a hypothesis class as opposed to the much larger Littlestonedimension that characterizes online learnability in the worst-case [BDPSS09].

Original: However, to fullydeliver on the promise revealed by these statistical insights, there needs to be an algorithmic frameworkfor realizing this connection between online and ofﬂine learnability.
Revised: However, to fullydeliver on the promise revealed by these statistical insights, there needs to be an algorithmic frameworkfor realizing this connection between online and ofﬂine learnability.

Original: In this paper, we ask
Revised: In this paper, we ask

Original: Do efﬁcient ofﬂine learning algorithms lead to efﬁcient online learning algorithmswith comparable regret guarantees, under the smoothed analysis framework?
Revised: Do efﬁcient ofﬂine learning algorithms lead to efﬁcient online learning algorithmswith comparable regret guarantees, under the smoothed analysis framework?

Original: In more detail, smoothed analysis is a perspective on algorithm design, introduced by [ST04] andformalized for online learning by [RSS12, HRS20], in which the adversary is restricted to generatingan instance at every round from a distribution that is not overly concentrated, i.e. a distributionwhose density is upper bounded by 1 /  times that of the uniform distribution 1 .
Revised: In more detail, smoothed analysis is a perspective on algorithm design, introduced by [ST04] andformalized for online learning by [RSS12, HRS20], in which the adversary is restricted to generatingan instance at every round from a distribution that is not overly concentrated, i.e. a distributionwhose density is upper bounded by 1 /  times that of the uniform distribution 1 .

Original: The smoothnessof the adversary’s actions captures the noise and imprecision inherent in the real world and as a
Revised: The smoothnessof the adversary’s actions captures the noise and imprecision inherent in the real world and as a

Original: [Table]
Revised: [Table]

Original: model gracefully captures the expressivity of worst-case instances while circumventing the overlypessimistic nature of the worst-case analysis.
Revised: model gracefully captures the expressivity of worst-case instances while circumventing the overlypessimistic nature of the worst-case analysis.

Original: The question of whether ofﬂine learning algorithms can lead to online learning algorithms is naturallycaptured by the oracle-efﬁciency framework (e.g., [DHL + 17, KV05, HK16]).
Revised: The question of whether ofﬂine learning algorithms can lead to online learning algorithms is naturallycaptured by the oracle-efﬁciency framework (e.g., [DHL + 17, KV05, HK16]).

Original: In this setting, wehave access to an ofﬂine learning algorithm or equivalently an empirical risk minimization (ERM)oracle which can compute an optimal hypothesis given any history of the actions of the adversary.
Revised: In this setting, wehave access to an ofﬂine learning algorithm or equivalently an empirical risk minimization (ERM)oracle which can compute an optimal hypothesis given any history of the actions of the adversary.

Original: Our computationally efﬁcient online algorithms can be design to utilize polynomial number of callsto the ERM oracle.
Revised: Our computationally efﬁcient online algorithms can be design to utilize polynomial number of callsto the ERM oracle.

Original: Our main goal is to design oracle-efﬁcient online algorithms whose regret is characterized by ofﬂinestatistical complexity measures, such as the VC dimension or pseudo dimension of a hypothesis class.
Revised: Our main goal is to design oracle-efﬁcient online algorithms whose regret is characterized by ofﬂinestatistical complexity measures, such as the VC dimension or pseudo dimension of a hypothesis class.

Original: Interestingly, in the worst-case instances (in absence of smoothed analysis) [HK16] showed that suchalgorithms cannot exist.
Revised: Interestingly, in the worst-case instances (in absence of smoothed analysis) [HK16] showed that suchalgorithms cannot exist.

Original: Thus, positively resolving this question will show that smoothed analysissimultaneously overcomes both statistical and computational impossibilities.
Revised: Thus, positively resolving this question will show that smoothed analysissimultaneously overcomes both statistical and computational impossibilities.

Original: This is indeed what weshow by the main contributions of our work.
Revised: This is indeed what weshow by the main contributions of our work.

Original: 1.1 Main Results
Revised: 1.1 Main Results

Original: We give the ﬁrst oracle-efﬁcient online learning algorithms whose regret is characterized by thestatistical ofﬂine complexity measures, under smoothed analysis.
Revised: We give the ﬁrst oracle-efﬁcient online learning algorithms whose regret is characterized by thestatistical ofﬂine complexity measures, under smoothed analysis.

Original: In particular, we show that thereare efﬁcient algorithms, given access to an ERM oracle, that achieve sublinear regret that dependsonly on the pseudo- (or VC) dimension of a class of hypothesis.
Revised: In particular, we show that thereare efﬁcient algorithms, given access to an ERM oracle, that achieve sublinear regret that dependsonly on the pseudo- (or VC) dimension of a class of hypothesis.

Original: We summarize our main results in
Revised: We summarize our main results in

Original: Table 1.
Revised: Table 1.

Original: We study both the real-valued and binary valued losses.
Revised: We study both the real-valued and binary valued losses.

Original: For the real-valued case, we show a regretbound of O ( p Td   1 ) .
Revised: For the real-valued case, we show a regretbound of O ( p Td   1 ) .

Original: We improve upon this bound for the binary classiﬁcation setting achievingregret of O ( p Td   1 / 2 ) .
Revised: We improve upon this bound for the binary classiﬁcation setting achievingregret of O ( p Td   1 / 2 ) .

Original: While these bounds demonstrate sublinear regret that only depends on d and  1 , their dependence on   1 does not match the (non-efﬁcient) statistical regret bound of [HRS21],which gave a p
Revised: While these bounds demonstrate sublinear regret that only depends on d and  1 , their dependence on   1 does not match the (non-efﬁcient) statistical regret bound of [HRS21],which gave a p

Original: Td log(   1 ) dependence.
Revised: Td log(   1 ) dependence.

Original: We further discuss this computational-statistical gapbetween online learning algorithms by presenting lower bounds for efﬁcient algorithms in presenceof beyond worst-case adversaries and highlight important directions for future research.
Revised: We further discuss this computational-statistical gapbetween online learning algorithms by presenting lower bounds for efﬁcient algorithms in presenceof beyond worst-case adversaries and highlight important directions for future research.

Original: In addition to the application of our algorithms in the smoothed setting, with appropriate settings ofparameters, our algorithms improve upon the regret dependence in terms of the domain size |X| inthe worst case binary classiﬁcation with regret O ( p
Revised: In addition to the application of our algorithms in the smoothed setting, with appropriate settings ofparameters, our algorithms improve upon the regret dependence in terms of the domain size |X| inthe worst case binary classiﬁcation with regret O ( p

Original: T ( d |X| ) 1 / 2 ) , improving upon the O ( p
Revised: T ( d |X| ) 1 / 2 ) , improving upon the O ( p

Original: T |X| )
Revised: T |X| )

Original: 61 bound of [DS16] and the regret dependence in terms of the VC dimension d in transductive binaryclassiﬁcation to O
Revised: 61 bound of [DS16] and the regret dependence in terms of the VC dimension d in transductive binaryclassiﬁcation to O

Original: ( T 3 / 4 d 1 / 4 ) , improving upon the O
Revised: ( T 3 / 4 d 1 / 4 ) , improving upon the O

Original: ( T 3 / 4 d 1 / 2 ) bound in [KK06].
Revised: ( T 3 / 4 d 1 / 2 ) bound in [KK06].

Original: 1.2 Technical Overview
Revised: 1.2 Technical Overview

Original: Random Playout for Beyond Worst-Case Adversaries.
Revised: Random Playout for Beyond Worst-Case Adversaries.

Original: Our algorithms are based on the randomplayout design principles, including the admissible relaxation framework of [RSS12] and the Follow- the-Perturbed-Leader framework of [KV05].
Revised: Our algorithms are based on the randomplayout design principles, including the admissible relaxation framework of [RSS12] and the Follow- the-Perturbed-Leader framework of [KV05].

Original: We show that this framework is useful for analyzingonline learning algorithms in the beyond worst-case setting, especially in smoothed analysis.
Revised: We show that this framework is useful for analyzingonline learning algorithms in the beyond worst-case setting, especially in smoothed analysis.

Original: Inthis setting, smoothness captures a level of predictability about the future.
Revised: Inthis setting, smoothness captures a level of predictability about the future.

Original: This is made formalby a technique from [HRS21] that shows that any sequence of T instances generated by adaptivesmoothed adversaries can be seen as a subset of T/  uniformly random instances from X with highprobability.
Revised: This is made formalby a technique from [HRS21] that shows that any sequence of T instances generated by adaptivesmoothed adversaries can be seen as a subset of T/  uniformly random instances from X with highprobability.

Original: We implement this algorithmically by self-generating random instances and labels as astand-in for the future.
Revised: We implement this algorithmically by self-generating random instances and labels as astand-in for the future.

Original: While the self-generated samples may not include adversary’s next choicewith some probability, these frameworks can be used to account for the uncertainty in each step.
Revised: While the self-generated samples may not include adversary’s next choicewith some probability, these frameworks can be used to account for the uncertainty in each step.

Original: Furthermore, we show that the inclusion of additional self-generated samples has a small impacton the achievable regret by proving that the regularized Rademacher complexity (which acts as anadmissible relaxation) is monotone in the set of generated samples (Lemma 4.1).
Revised: Furthermore, we show that the inclusion of additional self-generated samples has a small impacton the achievable regret by proving that the regularized Rademacher complexity (which acts as anadmissible relaxation) is monotone in the set of generated samples (Lemma 4.1).

Original: This monotonicityproperty leads to regret bounds that gracefully degrade as a small function of 1 /  .
Revised: This monotonicityproperty leads to regret bounds that gracefully degrade as a small function of 1 /  .

Original: Stability, Poissonization and Generalization.
Revised: Stability, Poissonization and Generalization.

Original: To obtain even stronger regret bounds for the binarysetting, our analysis builds on the notion of stability , i.e., how little the distribution of learner’s actionschanges across time steps.
Revised: To obtain even stronger regret bounds for the binarysetting, our analysis builds on the notion of stability , i.e., how little the distribution of learner’s actionschanges across time steps.

Original: A crucial ingredient in controlling stability is our novel Poissonizationtechnique that randomly sets the number of samples to be self-generated from an appropriatelychosen Poisson distribution.
Revised: A crucial ingredient in controlling stability is our novel Poissonizationtechnique that randomly sets the number of samples to be self-generated from an appropriatelychosen Poisson distribution.

Original: This allows us an additional degree of independence that is essential forcontrolling the loss from one step to the next using information theoretic techniques.
Revised: This allows us an additional degree of independence that is essential forcontrolling the loss from one step to the next using information theoretic techniques.

Original: The stability analysis of the algorithm also depends crucially on a modiﬁed generalization error of the
Revised: The stability analysis of the algorithm also depends crucially on a modiﬁed generalization error of the

Original: ERM, when it is trained on uniformly generated training samples and tested on smoothly distributedfresh instances.
Revised: ERM, when it is trained on uniformly generated training samples and tested on smoothly distributedfresh instances.

Original: To bound this, we show a strong conditional independence property satisﬁed by thecoupling from [HRS21].
Revised: To bound this, we show a strong conditional independence property satisﬁed by thecoupling from [HRS21].

Original: This is instrumental for bounding the generalization error by allowing us toextract smooth variables from a set of uniform variables, which can then be used for symmetrization.
Revised: This is instrumental for bounding the generalization error by allowing us toextract smooth variables from a set of uniform variables, which can then be used for symmetrization.

Original: We expect that this approach will be of independent interest for future work.
Revised: We expect that this approach will be of independent interest for future work.

Original: 1.3 Related works
Revised: 1.3 Related works

Original: Our work relates to several paradigms and approaches to online learnability.
Revised: Our work relates to several paradigms and approaches to online learnability.

Original: Oracle-Efﬁcient Online Learning.
Revised: Oracle-Efﬁcient Online Learning.

Original: Since the seminal work of [KV05], inspired by applicationdomains such as game theory, there has been a long line of work elucidating the computationalaspects of online learning.
Revised: Since the seminal work of [KV05], inspired by applicationdomains such as game theory, there has been a long line of work elucidating the computationalaspects of online learning.

Original: [KV05] proposed the inﬂuential follow-the-perturbed-leader algorithm.[KKL07] study notions of regret when the learner is given access to an approximate optimizationoracle.
Revised: [KV05] proposed the inﬂuential follow-the-perturbed-leader algorithm.[KKL07] study notions of regret when the learner is given access to an approximate optimizationoracle.

Original: [KK06] study the transductive learning setting and give an efﬁcient algorithm that convertsofﬂine learnablity to online learnability.
Revised: [KK06] study the transductive learning setting and give an efﬁcient algorithm that convertsofﬂine learnablity to online learnability.

Original: [RSS12] propose a general admissible relaxation frameworkto develop efﬁcient algorithms based on the upper bound of the value of the game.
Revised: [RSS12] propose a general admissible relaxation frameworkto develop efﬁcient algorithms based on the upper bound of the value of the game.

Original: [DHL + 17] presentthe computationally efﬁcient Generalized-FTPL algorithm and provide conditions under which itachieves vanishing regret.
Revised: [DHL + 17] presentthe computationally efﬁcient Generalized-FTPL algorithm and provide conditions under which itachieves vanishing regret.

Original: On the ﬂip size, [HK16] show that an ⌦ ( p N ) lower bound is unavoidablein general in order to obtain nontrivial regret where the N is the number of actions of the learnersuggesting that one needs to look beyond the worst-case in order to get truly efﬁcient algorithms.
Revised: On the ﬂip size, [HK16] show that an ⌦ ( p N ) lower bound is unaviodablein general in order to obtain nontrivial regret where the N is the number of actions of the learnersuggesting that one needs to look beyond the worst-case in order to get truly efﬁcient algorithms.

Original: Beyond Worst-case Approaches to Online Learning.
Revised: Beyond Worst-case Approaches to Online Learning.

Original: Various notion of beyond worst-casebehavior of online learning has been studied in the literature [RST11, HRS20, RS13b, DFHJ17,
Revised: Various notion of beyond worst-casebehavior of online learning has been studied in the literature [RST11, HRS20, RS13b, DFHJ17,

Original: BCKP20].
Revised: BCKP20].

Original: Most closely related are [RST11, HRS20, HRS21].
Revised: Most closely related are [RST11, HRS20, HRS21].

Original: [RST11] studied smoothed analysisof online learning but only gave explicit regret bounds for simple classes such as thresholds.
Revised: [RST11] studied smoothed analysisof online learning but only gave explicit regret bounds for simple classes such as thresholds.

Original: [HRS20,
Revised: [HRS20,

Original: HRS21] both study the notion of smoothed analysis with adaptive adversary and show that statisticallythe regret is bounded by O ( p
Revised: HRS21] both study the notion of smoothed analysis with adaptive adversary and show that statisticallythe regret is bounded by O ( p

Original: Td log(1 /  )) but do not provide efﬁcient algorithms.
Revised: Td log(1 /  )) but do not provide efﬁcient algorithms.

Original: Concurrent Work.
Revised: Concurrent Work.

Original: In a concurrent and independent work, [BDGR22] also gives oracle-efﬁcientalgorithms for smoothed online learning.
Revised: In a concurrent and independent work, [BDGR22] also gives oracle-efﬁcientalgorithms for smoothed online learning.

Original: In the binary classiﬁcation setting, [BDGR22] obtains a regret bound of e
Revised: In the binary classiﬁcation setting, [BDGR22] obtains a regret bound of e

Original: O ( p Td   1 ) using an FTPL-based algorithm.
Revised: O ( p Td   1 ) using an FTPL-based algorithm.

Original: In comparison, our result (Theorem 3.2)demonstrates a regret bound of e
Revised: In comparison, our result (Theorem 3.2)demonstrates a regret bound of e

Original: O ( p
Revised: O ( p

Original: Td   12 ) with strictly better dependence on  .
Revised: Td   12 ) with strictly better dependence on  .

Original: Our regretbound’s improved dependence on parameter  can be attributed to our novel technical innovations,including the introduction and careful analysis of modiﬁed generalization error and stability viaa new coupling-based argument, and a Poissonization approach for self-generating samples thatcan leverage information theoretic arguments.
Revised: Our regretbound’s improved dependence on parameter  can be attributed to our novel technical innovations,including the introduction and careful analysis of modiﬁed generalization error and stability viaa new coupling-based argument, and a Poissonization approach for self-generating samples thatcan leverage information theoretic arguments.

Original: For the case of real-valued functions with pseudo dimension d , [BDGR22] achieve regret e
Revised: For the case of real-valued functions with pseudo dimension d , [BDGR22] achieve regret e

Original: O (   1 p Td ) with e
Revised: O (   1 p Td ) with e

Original: O ( p T ) calls to the oracle per round.
Revised: O ( p T ) calls to the oracle per round.

Original: Inour paper, we obtain better regret of order e
Revised: Inour paper, we obtain better regret of order e

Original: O ( p Td   1 ) using only 2 oracle calls per round 2 .
Revised: O ( p Td   1 ) using only 2 oracle calls per round 2 .

Original: Ourstronger regret bounds are due to the fact that their algorithm is constrained to self-generating T -longsequences as a stand-in for the future, while we are generating substantially longer sequences thatallow us to leverage the monotonicity of Rademacher complexity.
Revised: Ourstronger regret bounds are due to the fact that their algorithm is constrained to self-generating T -longsequences as a stand-in for the future, while we are generating substantially longer sequences thatallow us to leverage the monotonicity of Rademacher complexity.

Original:  Preliminaries
Revised:  Preliminaries

Original: 2.1 Smoothed Online Learning
Revised: 2.1 Smoothed Online Learning

Original: Let X be the space of instances, Y = [  1 , 1] be the space of labels, and H : X !
Revised: Let X be the space of instances, Y 2 [  1 , 1] be the space of labels, and H : X !

Original: Y be thehypothesis class with pseudo dimension d (See deﬁnition B.1 or [AB99] for the deﬁnition of pseudodimension).
Revised: Y be the hypothesisclass with pseudo dimension d (See deﬁnition B.1 or [AB99] for the deﬁnition of pseudo dimension).

Original: Let l : Y ⇥ Y !
Revised: Let l : Y ⇥ Y !

Original: [0 , 1] be a convex loss function with Lipschitz constant G in its ﬁrstcomponent.
Revised: [0 , 1] be a convex loss function with Lipschitz constant G in its ﬁrst component.

Original: We also consider the special case where
Revised: We also consider the special case where

Original: Y =
Revised: Y =

Original: {  1 , +1 } is binary and the hypothesis128 class H has VC  dimension d .
Revised: {  1 , +1 } is binary and the hypothesis class H has VC 128 dimension d .

Original: In this case, we consider the classiﬁcation loss l ( b y, y )

Original: = 1 { b y 6 = y } .

Original: We work with the smoothed adaptive online adversarial setting from [HRS21].
Revised: We work with the smoothed adaptive online adversarial setting from [HRS21].

Original: We will consider -smooth adversaries, where a distribution is  -smooth if its density is upper bounded by 1 /  timesthe density of the uniform distribution over the same domain.
Revised: We will consider -smooth adversaries, where a distribution is  -smooth if its density is upper bounded by 1 /  timesthe density of the uniform distribution over the same domain.

Original: We remark that all of our resultsgeneralize to arbitrary known base distributions as well.
Revised: We remark that all of our resultsgeneralize to arbitrary known base distributions as well.

Original: Deﬁnition 2.1 (  -smoothness) .
Revised: Deﬁnition 2.1 (  -smoothness) .

Original: Let X be a domain that supports a uniform distribution U .
Revised: Let X be a domain that supports a uniform distribution U .

Original: A measureµ on X is  -smooth if for all measurable subsets A ✓ X , µ ( A )  U ( A )  .
Revised: A measureµ on X is  -smooth if for all measurable subsets A ✓ X , µ ( A )  U ( A )  .

Original: The set of all  -smoothdistributions on domain X is denoted by   ( X ) .
Revised: The set of all  -smoothdistributions on domain X is denoted by   ( X ) .

Original: In online learning with adaptive smoothed adversaries, the learner and the adversary plays a repeatedgame for T time steps.
Revised: In online learning with adaptive smoothed adversaries, the learner and the adversary plays a repeatedgame for T time steps.

Original: At each time step t 2 [ T ] , the adversary chooses a  -smooth distribution
Revised: At each time step t 2 [ T ] , the adversary chooses a  -smooth distribution

Original: D X t 2  ( X ) .
Revised: D X t 2  ( X ) .

Original: A random instance x t ⇠ D X t is then drawn and presented to the learner.
Revised: A random instance x t ⇠ D X t is then drawn and presented to the learner.

Original: After receiving 139 x t , the learner predicts its label to be b y t 2 Y , while the adversary simultaneously chooses y t 2 Y 140 as its true label.
Revised: After receiving 139 x t , the learner predicts its label to be b y t 2 Y , while the adversary simultaneously chooses y t 2 Y 140 as its true label.

Original: The learner then suffers loss l ( b y t , y t ) .
Revised: The learner then suffers loss l ( b y t , y t ) .

Original: The above protocol is equivalent to a settingwhere the adversary chooses a distribution D t 2  ( X ⇥ Y ) over labeled instances s t
Revised: The above protocol is equivalent to a settingwhere the adversary chooses a distribution D t 2  ( X ⇥ Y ) over labeled instances s t

Original: = ( x t , y t ) 142 whose marginal on
Revised: = ( x t , y t ) 142 whose marginal on

Original: X is  -smooth, and the learner simultaneously chooses a classiﬁer h t 2 Y X .
Revised: X is  -smooth, and the learner simultaneously chooses a classiﬁer h t 2 Y X .

Original: It is 143 not hard to see that this equivalence holds for both randomized and deterministic interactions, see[RS14, Section 7.6] for a more detailed discussion.
Revised: We 143 will abbreviate

Original: We will abbreviate D X t to D t when it is clearfrom the context.
Revised: D X t to D t when it is clear from the context.

Original: We allow the adversary to be adaptive, i.e., the choice of D t can depend on the realization of previousinstances { ( x i , y i ) }
Revised: We allow the adversary to be adaptive, i.e., the choice of D t can depend on the realization of previousinstances { ( x i , y i ) }

Original: t  1 i =1 as well as the learner’s previous predictions.
Revised: t  1 i =1 as well as the learner’s previous predictions.

Original: We denote with D  the adaptivesequence of  -smooth distributions D 1 , · · · , D T on the instances.
Revised: We denote with D  the adaptivesequence of  -smooth distributions D 1 , · · · , D T on the instances.

Original: Accordingly, let Q t 2  ( Y ) 149 denote the learner’s prediction rule on instance x t , and let Q denote the adaptive sequence ofdistributions Q 1 , · · · , Q T .
Revised: Accordingly, let Q t 2  ( Y ) 147 denote the learner’s prediction rule on instance x t , and let Q denote the adaptive sequence ofdistributions Q 1 , · · · , Q T .

Original: We denote the expected regret of a learner with prediction rules Q on the 151 adaptive sequence D  by
Revised: We denote the expected regret of a learner with prediction rules Q on the 149 adaptive sequence D  by

Original: [Equation]
Revised: [Equation]

Original: We remove D  and Q from this notation when they are clear from the context.
Revised: We remove D  and Q from this notation when they are clear from the context.

Original: An important property of smoothness is that it implies coupling between uniform and adaptivesmooth processes.
Revised: An important property of smoothness is that it implies coupling between uniform and adaptivesmooth processes.

Original: We will consider the original result from [HRS21] in Lemma B.1 and a slightlystrengthened statement in Lemma C.6.
Revised: We will consider the original result from [HRS21] in Lemma B.1 and a slightlystrengthened statement in Lemma C.5.

Original: 2.2 Ofﬂine Optimization Oracle
Revised: 2.2 Ofﬂine Optimization Oracle

Original: We consider computationally efﬁcient algorithms given access to an ofﬂine optimization oracle.
Revised: We consider computationally efﬁcient algorithms given access to an ofﬂine optimization oracle.

Original: Forthe case of binary classiﬁcation, the oracle outputs the solution of empirical risk minimization on theinput data.
Revised: Forthe case of binary classiﬁcation, the oracle outputs the solution of empirical risk minimization on theinput data.

Original: Deﬁnition 2.2 (ERM Oracle) .
Revised: Deﬁnition 2.2 (ERM Oracle) .

Original: For a hypothesis class H and a loss function l , the oracle OPT ( opt )takes a set 3 of inputs S
Revised: For a hypothesis class H and a loss function l , the oracle OPT ( opt )takes a set 3 of inputs S

Original: =
Revised: =

Original: ( x i , y i ) } i 

Revised: ( x i , y i ) } i

Original: [ I ] where ( x i , y i ) 2 X ⇥ Y for all
Revised: [ I ] where ( x i , y i ) 2 X ⇥ Y for all

Original: i 2 [ I ] and returns
Revised: i 2 [ I ] and returns

Original: OPT H ,l ( S ) = min h 2 H
Revised: OPT H ,l ( S ) = inf h 2 H

Original: I
Revised: I

Original: X i =
Revised: X i =

Original: l ( h ( x i ) , y i ) and opt H ,l ( S ) 2 arg min h 2 H
Revised: l ( h ( x i ) , y i ) and opt H ,l ( S ) 2 arg inf h 2 H

Original: I
Revised: I

Original: X i =
Revised: X i =

Original: l ( h ( x i ) , y i ) .
Revised: l ( h ( x i ) , y i ) .

Original: For the case of real-valued functions, we consider an oracle that can minimize a mixture of binaryand real-valued loss values deﬁned below.
Revised: For the case of real-valued functions, we consider an oracle that can minimize a mixture of binaryand real-valued loss values deﬁned below.

Original: Deﬁnition 2.3 (Real-valued optimization oracle) .
Revised: Deﬁnition 2.3 (Real-valued optimization oracle) .

Original: For a hypothesis class H and two loss functions l r and l b , the oracle OPT takes two sets of inputs S and S 0 over X ⇥ Y and returns
Revised: For a hypothesis class H and two loss functions l r and l b , the oracle OPT takes two sets of inputs S and S 0 over X ⇥ Y and returns

Original: [Equation]
Revised: [Equation]

Original: We remark that these oracles are used in most previous works, including [RSS12].
Revised: We remark that these oracles are used in most previous works, including [RSS12].

Original: They constitutea special form of regularized loss minimization oracles, where the regularization is given directlyby a random process.
Revised: They constitutea special form of regularized loss minimization oracles, where the regularization is given directlyby a random process.

Original: For the binary setting where Y = {± 1 } and l r = l b = 1 { b y [Equation] } , the above 167 optimization oracle is equivalent to ERM oracles.
Revised: For the binary setting where Y = {± 1 } and l r = l b = 1 { b y 6 = y } , the above 165 optimization oracle is equivalent to ERM oracles.

Original: We consider each call to the ofﬂine optimization oracle as having unit cost plus the additional runtimeneeded for creating and inputting the set of inputs that is linear in the length of the said histories.
Revised: We consider each call to the ofﬂine optimization oracle as having unit cost plus the additional runtimeneeded for creating and inputting the set of inputs that is linear in the length of the said histories.

Original: We note that our approach and results directly extend to using ERM oracles with (arbitrarily small)additive approximation error, such as those guaranteed by FPTAS optimization algorithms, usingstandard techniques presented by [DHL + 17, Section 6].
Revised: We note that our approach and results directly extend to using ERM oracles with (arbitrarily small)additive approximation error, such as those guaranteed by FPTAS optimization algorithms, usingstandard techniques presented by [DHL + 17, Section 6].

Original:  Oracle-Efﬁcient Online Learning
Revised:  Oracle-Efﬁcient Online Learning

Original: 3.1 Learning with Real-Valued Functions
Revised: 3.1 Learning with Real-Valued Functions

Original: In this section, we propose an oracle-efﬁcient algorithm for real-valued functions with regrete
Revised: In this section, we propose an oracle-efﬁcient algorithm for real-valued functions with regrete

Original: O ( p dT/  ) .
Revised: O ( p dT/  ) .

Original: We consider the optimization oracle deﬁned in Deﬁnition 2.3 with the loss functionsspeciﬁed by l r (ˆ y, y )
Revised: We consider the optimization oracle deﬁned in Deﬁnition 2.3 with the loss functionsspeciﬁed by l r (ˆ y, y )

Original: =
Revised: =

Original: 12 G l (ˆ y, y ) and l b (ˆ y, y ) =
Revised: 12 G l (ˆ y, y ) and l b (ˆ y, y ) =

Original: [Equation]
Revised: [Equation]

Original:  12 .

Original: We begin by describing our algorithm.
Revised: We begin by describing our algorithm.

Original: At each time step t 2 [ T ] , the algorithm draws e
Revised: At each time step t 2 [ T ] , the algorithm draws e

Original: O ⇣
Revised: O ⇣

Original: T  t  ⌘ fresh new instances from the uniform distribution, denoted with V ( t ) , together with their randomlabels E ( t ) that are drawn i.i.d.
Revised: T  t  ⌘ freshnew instances from the uniform distribution, denoted with V ( t ) , together with their random labels

Original: from the Rademacher distribution, and treat them as hints for thefuture.
Revised: E ( t ) , and treat them as hints for the future.

Original: Let S ( t ) denote the set of labeled instances ( V ( t ) , E ( t ) ) .
Revised: Let S ( t ) denote the set of labeled instances ( V ( t ) , E ( t ) ) .

Original: Our algorithm then applies the ofﬂineoptimization oracle to two input sequences: one where the real history s 1: t  1 is mixed with twocopies 4 of S ( t ) and the current instance is labeled +1 , and another, where the current label is labeled 1 .
Revised: Our algorithm then applies the ofﬂine optimization oracle to two input sequences: one where the realhistory s 1: t  1 is mixed with two copies 4 of S ( t ) and the current instance is labled +1 , and another,where the current label is labeled  1 .

Original: Formally, we consider
Revised: Formally, we consider

Original: [Equation]
Revised: [Equation]

Original: Since the two input sequences to the optimization oracle only disagree on one label, the difference inthe optimal errors is always bounded within [  1 , +1] , thus guarantees b y t 2 Y .
Revised: Since the two input sequences to the optimization oracle only disagree on one label, the difference inthe optimal errors is always bounded within [  1 , +1] , thus guarantees b y t 2 Y .

Original: Intuitively, the reasonˆ y t includes the gap between the error of these two optimal classiﬁers is to make the algorithm hedgeits bets against which instances will be generated by the adversary next.
Revised: Intuitively, the reasonˆ y t includes the gap between the error of these two optimal classiﬁers is to make the algorithm hedgeits bets against which instances will be generated by the adversary next.

Original: A formal description of thealgorithm is given in Algorithm 1.
Revised: A formal description of thealgorithm is given in Algorithm 1.

Original: The main motivation for the algorithm is the coupling lemma from [HRS21].
Revised: The main motivation for the algorithm is the coupling lemma from [HRS21].

Original: It states that a samplefrom any  -smooth distribution can be thought of as generated by ﬁrst sampling O    samplesfrom the uniform distribution and then selecting one of them as the sample.
Revised: It states that a samplefrom any  -smooth distribution can be thought of as generated by ﬁrst sampling O    samplesfrom the uniform distribution and then selecting one of them as the sample.

Original: The algorithm thus canbe thought of as generating samples from the uniform distribution to account for the uncertainty inthe choice of the adversary.
Revised: The algorithm thus canbe thought of as generating samples from the uniform distribution to account for the uncertainty inthe choice of the adversary.

Original: We will discuss this intuition and sketch a proof of the following theoremin Section 4.3.
Revised: We will discuss this intuition and sketch a proof of the following theorem in Section 4.3.

Original: Theorem 3.1 (Regret Upper Bound) .
Revised: Theorem 3.1 (Regret Upper Bound) .

Original: For any  -smooth adversary D  , Algorithm 1 has expectedregret upper bounded by e
Revised: For any  -smooth adversary D  , Algorithm 1 has expectedregret upper bounded by e

Original: O ( G p
Revised: O ( G p

Original: Td/  ) , where e
Revised: Td/  ) , where e

Original: O hides factors that are polynomial in log( T ) andlog(1 /  ) .
Revised: O hide factors that are polynomial in log( T ) andlog(1 /  ) .

Original: Here G is the Lipschitz constant of the loss and d is the pseudodimension of the class.
Revised: Here G is the Lipschitz constant of the loss and d is the pseudodimension of the class.

Original: Furthermore, the algorithm is oracle-efﬁcient: at every round t , this algorithm uses two oracle callswith histories of length e
Revised: Furthermore, the algorithm is oracle-efﬁcient: at every round t , this algorithm uses two oracle callswith histories of length e

Original: O ( T/  ) .
Revised: O ( T/  ) .

Original: Input: T,  1 K 100 log T/  .
Revised: Input: T,  1 K 100 log T/  .

Original: 2 for t 1 to T do
Revised: 2 for t 1 to T do

Original: Receive x t .
Revised: Receive x t .

Original: for i =
Revised: for i =

Original: t+1 , · · · , T ; k = 1 , · · · , K do
Revised: t , · · · , T ; k = 1 , · · · , K do

Original: Draw new v ( t ) i,k
Revised: Draw new v ( t ) i,k

Original: ⇠ U( X ) .
Revised: ⇠ U

Original: Draw new ✏ ( t ) i,k ⇠ U( {  1 , +1 }) . end
Revised: Draw new ✏ ( t ) i,k ⇠ U {  1 , +1 } end

Original: S ( t ) n ( v( t ) i,k , ✏( t ) i,k ) o i =
Revised: S ( t ) n ( v t ) i,k , ✏ t ) ,k ) o i =

Original: t +1:
Revised: t +1:

Original: T k =1: K .
Revised: T k =1: K .

Original: b y t
Revised: b y t

Original: OPT ⇣ s 1: t  1 ; S ( t )
Revised: OPT ⇣ s 1: t  1 ; S ( t )

Original: [ S ( t ) [ { ( x t ,  1) } ⌘ 
Revised: [ S ( t ) [ { ( x t ,  1) } ⌘

Original: OPT ⇣ s 1: t  1 ; S ( t )
Revised: OPT ⇣ s 1: t  1 ; S ( t )

Original: [ S ( t ) [ { ( x t , +1) } ⌘ .
Revised: [ S ( t ) [ { ( x t , +1) } ⌘ .

Original: Receive y t , suffer loss l ( b y t , y t ) .
Revised: Receive y t , suffer loss l ( b y t , y t ) .

Original: 11 end
Revised: 11 end

Original: 3.2 Improved Bounds for Binary Classiﬁcation
Revised: 3.2 Improved Bounds for Binary Classiﬁcation

Original: In this section, we focus on the important special case where the labels are binary and the loss functionis the classiﬁcation loss 1 { b y 

Revised: In this section, we focus on the important special case where the labels are binary and the loss functionis the classiﬁcation loss 1 { b y

Original: = y } .
Revised: = y } .

Original: We present Algorithm 2 that achieves regret e
Revised: We present Algorithm 2 that achieves regret e

Original: O ( p Td   1 / 2 ) withbetter dependence on the smoothness parameter  compared to Algorithm 1.
Revised: O ( p Td   1 / 2 ) withbetter dependence on the smoothness parameter  compared to Algorithm 1.

Original: Theorem 3.2 (Regret Bound for Efﬁcient Smoothed Online Learning) .
Revised: Theorem 3.2 (Regret Bound for Efﬁcient Smoothed Online Learning) .

Original: In the setting of binaryclassiﬁcation with  -smoothed adversaries, Algorithm 2 hasexpected regret that is at moste
Revised: In the setting of binaryclassiﬁcation with  -smoothed adversaries, Algorithm 2 has regret that is at moste

Original: O min ⇢p
Revised: O min ⇢p

Original: Td

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: , q
Revised: q

Original: [Equation]
Revised: [Equation]

Original: ! . Furthermore, Algorithm 2 is a proper learning oracle-efﬁcient algorithm: at every round t , thisalgorithm uses a single ERM oracle call on a history that is of length t + O ( T/ p  ) with highprobability.
Revised: Furthermore, Algorithm 2 is a proper learning oracle-efﬁcient algorithm: at every round t , thisalgorithm uses a single ERM oracle call on a history that is of length t + O ( T/ p  ) with highprobability.

Original: Unlike the algorithm for the real-valued case, the improved algorithm uses the FTPL framework.
Revised: Unlike the algorithm for the real-valued case, the improved algorithm uses the FTPL framework.

Original: Thealgorithm itself is easy to describe: the algorithm generates a random number N from the Poissondistribution with an appropriately chosen parameter, generates N uniformly random points alongwith random labels and then predicts using the hypothesis that has the lowest error on the past dataappended with the newly sampled data.
Revised: Thealgorithm itself is easy to describe: the algorithm generates a random number N from the Poissondistribution with an appropriately chosen parameter, generates N uniformly random points alongwith random labels and then predicts using the hypothesis that has the lowest error on the past dataappended with the newly sampled data.

Original: On the surface, this algorithm seems similar to Algorithm 1.
Revised: On the surface, this algorithm seems similar to Algorithm 1.

Original: But there are two key differences: unlike Algorithm 1 which uses the difference in value of twooptimizations, Algorithm 2 follows the prediction of the hypothesis with the lowest error.
Revised: But there are two key differences: unlike Algorithm 1 which uses the difference in value of twooptimizations, Algorithm 2 follows the prediction of the hypothesis with the lowest error.

Original: Thismakes Algorithm 2 a proper online learning algorithm.
Revised: Thismakes Algorithm 2 a proper online learning algorithm.

Original: Secondly, unlike Algorithm 1 which uses adecreasing number of random examples over time, Algorithm 2 has the number of samples distributedaccording to a Poisson (with the same parameter in each step).
Revised: Secondly, unlike Algorithm 1 which uses adecreasing number of random examples over time, Algorithm 2 has the number of samples distributedaccording to a Poisson (with the same parameter in each step).

Original: The fact that the number of hints isdrawn from the Poisson distribution is crucial for our analysis of the stability of the algorithm.
Revised: The fact that the number of hints isdrawn from the Poisson distribution is crucial for our analysis of the stability of the algorithm.

Original: Wesketch the proof of the regret bounds in Section 4.4.
Revised: Wesketch the proof of the regret bounds in Section 4.4.

Original: Algorithm 2: Smoothed Online Learning based on Poisson Number of Hints
Revised: Algorithm 2: Smoothed Online Learning based on Poisson Number of Hints

Original: Input: time horizon T , smoothness parameter  , VC dimension d 1 n min , T hallucinated are .d N
Revised: Input: time horizon T , smoothness parameter  , VC dimension d 1 n end

Original: call 6 end While our main interest is on beyond the worst-case adversaries, our results improve upon existingresults for worst-case analysis of online learning as well.
Revised: While our main interest is on beyond the worst-case adversaries, our results improve upon existingresults for worst-case analysis of online learning as well.

Original: For ﬁnite domain and binary-valued losssettings where worst-case adversaries are vacuously  -smooth for 1 /  = |X| , Theorem 3.2 alsoachieves an oracle-efﬁcient regret bound of O ( p
Revised: For ﬁnite domain and binary-valued losssettings where worst-case adversaries are vacuously  -smooth for 1 /  = |X| , Theorem 3.2 alsoachieves an oracle-efﬁcient regret bound of O ( p

Original: T ( d |X| ) 1 / 2 ) , which is a reﬁnement of O ( p
Revised: T ( d |X| ) 1 / 2 ) , which is a reﬁnement of O ( p

Original: T |X| )
Revised: T |X| )

Original: 226 bound of [DS16], because VC dimension d is at most X , and is usually much smaller.
Revised: 223 bound of [DS16], because VC dimension d is at most X , and is usually much smaller.

Original: Similarly,our bound can be instantiated in the setting of transductive learning with |X| =
Revised: Similarly,our bound can be instantiated in the setting of transductive learning with |X| =

Original: T , which improves
Revised: T , which improves

Original: O ( T 3 / 4 p d ) bound of [KK06] to O
Revised: O ( T 3 / 4 p d ) bound of [KK06] to O

Original: ( T 3 / 4 d 1 / 4 ) .
Revised: ( T 3 / 4 d 1 / 4 ) .

Original: Corollary 3.3 (Regret for Small Domain) .
Revised: Corollary 3.3 (Regret for Small Domain) .

Original: There is an oracle-efﬁcient algorithm for online learningwith binary labels (in the worst-case) that achieves an expected regret of O ( p
Revised: There is an oracle-efﬁcient algorithm for online learningwith binary labels (in the worst-case) that achieves a regret of O ( p

Original: T ( d |X| ) 1 / 2 ) for any231 hypothesis  class with VC dimension d on domain
Revised: T ( d |X| ) 1 / 2 ) for any hypothesis 228 class with VC dimension d on domain

Original: X .
Revised: X .

Original: For transductive learning with binary labels,there is an oracle efﬁcient algorithm, withexpected regret O ⇣
Revised: For transductive learning with binary labels, there is anoracle efﬁcient algorithm, with regret O ⇣

Original: T 3 / 4 d 1 /⌘ .
Revised: T 3 / 4 d 1 /⌘ .

Original:  Proof Sketches for Main Regret Bounds
Revised:  Proof Sketches for Main Regret Bounds

Original: Before discussing the proof sketches for the main regret bounds, we will ﬁrst introduce two frameworks for designing efﬁcient algorithms for online learning.
Revised: Before discussing the proof sketches for the main regret bounds, we will ﬁrst introduce two frameworks for designing efﬁcient algorithms for online learning.

Original: 4.1 Relaxations and Admissibility
Revised: 4.1 Relaxations and Admissibility

Original: The proof of Theorem 3.1 relies on the admissible relaxation framework proposed in [RSS12].
Revised: The proof of Theorem 3.1 relies on the admissible relaxation framework proposed in [RSS12].

Original: Arelaxation Rel T is a sequence of functions Rel T ( H| s 1: t ) for each t 2 [ T ] , which map the history of 239 the play to real values that upper bounds the conditional value of the game.
Revised: Arelaxation Rel T is a sequence of functions Rel T ( H| s 1: t ) for each t 2 [ T ] , which map the history of 236 the play to real values that upper bounds the conditional value of the game.

Original: We will make use of animportant algorithmic aspect of the relaxation framework, which states that whenever an algorithm isadmissible with respect to some relaxation, its expected regret can be upper bounded in terms of thevalue of the relaxation at the beginning of the game.
Revised: We will make use of animportant algorithmic aspect of the relaxation framework, which states that whenever an algorithm isadmissible with respect to some relaxation, its expected regret can be upper bounded in terms of thevalue of the relaxation at the beginning of the game.

Original: Deﬁnition 4.1 (Admissibility) .
Revised: Deﬁnition 4.1 (Admissibility) .

Original: In the smoothed online learning setting, let Q be an algorithm thatgives rise to a sequence of distributions Q 1 , · · · , Q T on the predicted labels.
Revised: In the smoothed online learning setting, let Q be an algorithm thatgives rise to a sequence of distributions Q 1 , · · · , Q T on the predicted labels.

Original: We say Q is admissible 245 with respect to a relaxation { Rel T
Revised: We say Q is admissible 242 with respect to a relaxation { Rel T

Original: ( H | s 1: t ) }
Revised: ( H | s 1: t ) }

Original: Tt =0 , if for any sequence of instances s 1: T ,
Revised: Tt =0 , if for any sequence of instances s 1: T ,

Original: For all t2 [ T] ,
Revised: For all t T

Original: [Equation]
Revised: [Equation]

Original: where   ( X ) is the set of  -smooth distributions on X ;
Revised: where   ( X ) is the set of  -smooth distributions on X ;

Original: The ﬁnal value satisﬁes Rel T ( H | s 1: T )   inf h 2 H L ( h, s 1: T ) .
Revised: The ﬁnal value satisﬁes Rel T ( H | s 1: T )   inf h 2 H L ( h, s 1: T ) .

Original: The following proposition is the analog of the results of [RSS12] when the adversary is smooth.
Revised: The following proposition is the analog of the results of [RSS12] when the adversary is smooth.

Original: Thefull proof is presented in Appendix F.
Revised: Thefull proof is presented in Appendix F.

Original: Proposition 4.1 (Regret Bound via Admissibility) .
Revised: Proposition 4.1 (Regret Bound via Admissibility) .

Original: In the smoothed online learning setting, let
Revised: In the smoothed online learning setting, let

Original: Q =
Revised: Q =

Original: ( Q 1 , · · · , Q T ) be an algorithm that is admissible with respect to relaxations
Revised: ( Q 1 , · · · , Q T ) be an algorithm that is admissible with respect to relaxations

Original: Rel T
Revised: Rel T

Original: ( H ) , then 253 the following bound on the expected regret holds regardless of the strategies D  of the adversary:
Revised: ( H ) , then 250 the following bound on the expected regret holds regardless of the strategies D  of the adversary:

Original: [Equation]
Revised: [Equation]

Original: 4.2 Follow the Perturbed Leader
Revised: 4.2 Follow the Perturbed Leader

Original: When the labels are binary, Algorithm 2 achieves an improved regret using the Follow the Perturbed
Revised: When the labels are binary, Algorithm 2 achieves an improved regret using the Follow the Perturbed

Original: Leader (FTPL) principle [KV05].
Revised: Leader (FTPL) principle [KV05].

Original: An FTPL algorithm makes predictions by applying ERM oracle tothe perturbed histories of the play.
Revised: An FTPL algorithm makes predictions by applying ERM oracle tothe perturbed histories of the play.

Original: At every time step t 2 [ T ] , the algorithm chooses a distributionover labeled instances, from which it draws N random instances (e
Revised: At every time step t 2 [ T ] , the algorithm chooses a distributionover labeled instances, from which it draws N random instances (

Original: [Equation]
Revised: [Equation]

Original: ( t ) 1 ) , · · · , ( e x ( t ) N , e y ( t ) N ) .
Revised: ) 1 ) , · · · , ( e x ( t ) N , e y ( t ) N ) .

Original: Thepredicted label is then given by b y t = h t ( x t ) , whereh t opt H ,l ⇣ [Equation] 1: t  
Revised: Thepredicted label is then given by b y t = h t ( x t ) , whereh t opt H ,l ⇣ s 1: t  1 [ { ( e x ( t ) i , e y ( t ) i )

Original: [Equation]
Revised: [Equation]

Original: { ( e x ( t ) i , e y ( t ) i )

Original: [Equation] N ] ⌘ .
Revised: 2 [ N ] ⌘ .

Original: The standard analysis of FTPL bounds the expected regret as follows:
Revised: The standard analysis of FTPL bounds the expected regret as follows:

Original: [Equation]
Revised: [Equation]

Original: Stability

Original: [Equation]

Original: | {z }
Revised: | {z }

Original: Perturbation
Revised: Perturbation

Original: [Equation]
Revised: [Equation]

Original: where h ⇤ = arg inf h 2 H
Revised: where h ⇤ = arg inf h 2 H

Original: P
Revised: P

Original: Tt =1 l ( h ( x t ) , y t ) .
Revised: Tt =1 l ( h ( x t ) , y t ) .

Original: Note that the perturbation term is already well-understood from statistical learning theory since it isessentially the Rademacher complexity of H for sample size N .
Revised: Note that the perturbation term is already well-understood from statistical learning theory since it isessentially the Rademacher complexity of H for sample size N .

Original: Therefore, we will focus on boundingthe stability term by designing perturbations that can leverage the anti-concentration property ofsmoothed adversaries.
Revised: Therefore, we will focus on boundingthe stability term by designing perturbations that can leverage the anti-concentration property ofsmoothed adversaries.

Original: 4.3 Proof Sketch of Theorem 3.1
Revised: 4.3 Proof Sketch of Theorem 3.1

Original: Elaborating on the intuition laid out in Section 3, we will use the coupling technique introducedby [HRS21] (see Lemma B.1 for a complete description) to replace the sequence of T randominputs { x 1 , · · · , x T } generated by the adaptive adversary with TK inputs { z t,k } t 2 [ T ] ,k 

Revised: Elaborating on the intuition laid out in Section 3, we will use the coupling technique introducedby [HRS21] (see Lemma B.1 for a complete description) to replace the sequence of T randominputs { x 1 , · · · , x T } generated by the adaptive adversary with TK inputs { z t,k } t 2 [ T ] ,k

Original: [ K ] that are 270 generated i.i.d.
Revised: [ K ] that are 267 generated i.i.d.

Original: from the uniform distribution over
Revised: from the uniform distribution over

Original: X , such that with high probability { x 1 , · · · , x T } ✓{ z t,k } t 2 [ T ] ,k 2 [ K ] .
Revised: X , such that with high probability { x 1 , · · · , x T } ✓{ z t,k } t 2 [ T ] ,k 2 [ K ] .

Original: This implies that, up to a small probability of failure, it is sufﬁcient to consider a 272 simpler setting where the adversary is promised to pick future instances from a larger set of uniformlydistributed samples (which we call the set of hints ).
Revised: This implies that, up to a small probability of failure, it is sufﬁcient to consider a 269 simpler setting where the adversary is promised to pick future instances from a larger set of uniformlydistributed samples (which we call the set of hints ).

Original: This setting differs from the standard transductivelearning setting[KK06, CbS11] in two signiﬁcant ways: 1) The set of hints is not revealed to thelearner beforehand; 2) the hint set is larger, by a multiplicative factor of K ⇡ 1 /  , than the set ofrealized instances.
Revised: This setting differs from the standard transductivelearning setting in two signiﬁcant ways: 1) The set of hints is not revealed to the learner beforehand;2) the hint set is larger, by a multiplicative factor of K ⇡ 1 /  , than the set of realized instances.

Original: It turns out that both issues can be handled elegantly in the admissible relaxations framework of [RSS12].
Revised: It turns out that both issues can be handled elegantly in the admissible relaxations framework of [RSS12].

Original: For the ﬁrst issue, note that the Algorithm 1 can be seen as self-generating hints and although they do not necessarily correspond to the adversary’s sample, the relaxation-based argument guarantees that matching the randomness of hints at a distribution level sufﬁces to bound the regret of the algorithm (see Appendix B.7 for more details).
Revised: For the ﬁrst issue, note that the Algorithm 1 can be seen as self-generating hints and although they do not necessarily correspond to the adversary’s sample, the relaxation-based argument guarantees that matching the randomness of hints at a distribution level sufﬁces to bound the regret of the algorithm (see Appendix B.7 for more details).

Original: For the second issue, our relaxation will be based on a characterization of the uncertainty in the future that is monotone in the set of hints, which we call regularized Rademacher complexity .
Revised: For the second issue, our relaxation will be based on a characterization of the uncertainty in the future that is monotone in the set of hints, which we call regularized Rademacher complexity .

Original: Formally, for a set of unlabeled instances Z = { z i } Ii =1 and a function  : H !
Revised: Formally, for a set of unlabeled instances Z = { z i } Ii =1 and a function  : H !

Original: R , the Rademacher complexity for set Z regularized by  is deﬁned as
Revised: R , the Rademacher complexity for set Z regularized by  is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: We show that regularized Rademacher complexity is monotone as a function of the dataset.
Revised: We show that regularized Rademacher complexity is monotone as a function of the dataset.

Original: See
Revised: See

Original: Appendix B.2 for a proof of Lemma 4.1.
Revised: Appendix B.2 for a proof of Lemma 4.1.

Original: Lemma 4.1 (Monotonicity of Regularized Rademacher Complexity) .
Revised: Lemma 4.1 (Monotonicity of Regularized Rademacher Complexity) .

Original: For any dataset z 1: m 2 X m 280 and any additional data point x 2 X , we have R (  , z 1: m )  R (  , z 1: m [ { x } ) .
Revised: For any dataset z 1: m 2 X m 276 and any additional data point x 2 X , we have R (  , z 1: m )  R (  , z 1: m [ { x } ) .

Original: This monotonicity ensures that using the hint set, which is a superset of possible instances, will stilllead to a no-regret algorithm.
Revised: This monotonicity ensures that using the hint set, which is a superset of possible instances, will stilllead to a no-regret algorithm.

Original: Finally, the relaxation we use to analyze Algorithm 1 is the expected Rademacher complexity of theunion of future hints, regularized by the past total loss, i.e.,
Revised: Finally, the relaxation we use to analyze Algorithm 1 is the expected Rademacher complexity of theunion of future hints, regularized by the past total loss, i.e.,

Original: [Equation]
Revised: [Equation]

Original: where L r ( h, s 1: t ) =
Revised: where L r ( h, s 1: t ) =

Original: P ti =1 l r ( h ( x i ) , y i ) for h 2 H .
Revised: P ti =1 l r ( h ( x i ) , y i ) for h 2 H .

Original: Here  = 10 TK (1   ) K 2 o (1 /T ) representsthe penalty caused by the failure of coupling.
Revised: Here  = 10 TK (1   ) K represents the penaltycaused by the failure of coupling.

Original: Once admissibility is established, we will obtain aregret bound using Proposition 4.1.
Revised: Once admissibility is established, we will obtain a regret boundusing Proposition 4.1.

Original: See Appendix B for a complete proof of the theorem.
Revised: See Appendix B for a complete proof of the theorem.

Original: We remarkthat the ﬁnal regret bound can also be stated in terms of the Rademacher complexity which is datadependent and adapts readily to benign structures of the instance domain.

Original: 4.4 Proof Sketch of Theorem 3. 
Revised: 4.4 Proof Sketch of Theorem 3.

Original: Let Q t be the distribution of the learner’s action h t 2 H in Algorithm 2 and D t denote the distribution 292 of the adversary at time t .
Revised: Let Q t be the distribution of the learner’s action h t 2 H in Algorithm 2 and D t denote the distribution 286 of the adversary at time t .

Original: The main quantity we will use to analyze the algorithm is the stability
Revised: The main quantity we will use to analyze the algorithm is the stability

Original: Stability 
Revised: Stability =

Original: [Equation]
Revised: [Equation]

Original: We analyze this expression by breaking it down into a sum of two quantities:
Revised: We analyze this expression by breaking it down into a sum of two quantities:

Original: [Equation]
Revised: [Equation]

Original: | {z } Modiﬁed 
Revised: Modiﬁed generalization error

Original: [Table]
Revised: [Equation]

Original: error . Here, R ( t ) is the fresh randomness generated by the algorithm at the beginning of time t .
Revised: Here, R ( t ) is the fresh randomness generated by the algorithm at the beginning of time t .

Original: In order to see where this expression comes from, note that the ﬁrst term itself would be an upperbound on the stability, if neither of Q t and Q t +1 depend on the new observation s t = ( x t , y t ) at time 297 t .
Revised: In order to see where this expression comes from, note that the ﬁrst term itself would be an upperbound on the stability, if neither of Q t and Q t +1 depend on the new observation s t = ( x t , y t ) at time 291 t .

Original: However, while
Revised: However, while

Original: Q t is independent of s t , Q t +1 does depend on s t because h t +1 is trained on s t .
Revised: Q t is independent of s t , Q t +1 does depend on s t because h t +1 is trained on s t .

Original: To 298 overcome this dependence, we introduce a ghost sample st that allows us to decouple h t +1 ⇠ Q t +1 299 and the new observation.
Revised: To 292 overcome this dependence, we introduce a ghost sample st that allows us to decouple h t +1 ⇠ Q t +1 293 and the new observation.

Original: This gives rise to the second term which we call modiﬁed generalizationerror .
Revised: This gives rise to the second term which we call modiﬁed generalizationerror .

Original: We formally discuss this decomposition in Appendix C.2.
Revised: We formally discuss this decomposition in Appendix C.2.

Original: The ﬁrst term is the total variation (TV) distance between Q t and the mixture distribution
Revised: The ﬁrst term is the total variation (TV) distance between Q t and the mixture distribution

Original: E s t ⇠ D t [ Q t +1 ] .
Revised: E s t ⇠ D t [ Q t +1 ] .

Original: In order to bound this term, we closely use the independence properties of the 303 Poisson distribution which allows us to write an explicit expression for the total variation distancewhich we can then bound using the Ingster–Suslina method.
Revised: In order to bound this term, we closely use the independence properties of the 297 Poisson distribution which allows us to write an explicit expression for the total variation distancewhich we can then bound using the Ingster–Suslina method.

Original: We formally prove this in Lemma C.4.
Revised: We formally prove this in Lemma C.3.

Original: The intuitive idea behind bounding the second term is as follows.
Revised: The intuitive idea behind bounding the second term is as follows.

Original: Consider the simpler setting oft
Revised: Consider the simpler setting oft

Original: = 1 (i.e. no history) and D t =
Revised: = 1 (i.e. no history) and D t =

Original: U ( X ⇥ {± 1 } ) (i.e. the new observation s t follows the same 307 distribution as the self-generated samples).
Revised: U ( X ⇥ {± 1 } ) (i.e. the new observation s t follows the same 301 distribution as the self-generated samples).

Original: In this case, the generalization error is precisely thedifference between the test error and the training error with N + 1 iid training data, and classical
Revised: In this case, the generalization error is precisely thedifference between the test error and the training error with N + 1 iid training data, and classical

Original: Rademacher complexity gives an upper bound O ( p d/N ) .
Revised: Rademacher complexity gives an upper bound O ( p d/N ) .

Original: For general  -smooth D t , we establish astrong conditional independence property of the coupling argument from [HRS21].
Revised: For general  -smooth D t , we establish astrong conditional independence property of the coupling argument from [HRS21].

Original: This states thatthere exists a coupling between uniform and adaptive smooth processes, such that when the inclusionproperty is satisﬁed, the distribution of the realized uniform variables conditional on the unrealizeduniform variables is also identical to the smooth distributions given by the adversary.
Revised: This states thatthere exists a coupling between uniform and adaptive smooth processes, such that when the inclusionproperty is satisﬁed, the distribution of the realized uniform variables conditional on the unrealizeduniform variables is also identical to the smooth distributions given by the adversary.

Original: This will beinstrumental for bounding the generalization error by allowing us to extract smooth variables from aset of uniform variables, which can then be used to for the purpose of symmetrization.
Revised: This will beinstrumental for bounding the generalization error by allowing us to extract smooth variables from aset of uniform variables, which can then be used to for the purpose of symmetrization.

Original: We formallyprove the upper bound on the modiﬁed generalization in Lemma C.5.
Revised: We formallyprove the upper bound on the modiﬁed generalization in Lemma C.4.

Original: Using this bound on stability, we can get a bound on the regret of the algorithm using analysistechniques for FTPL.
Revised: Using this bound on stability, we can get a bound on the regret of the algorithm using analysistechniques for FTPL.

Original: For a full proof, see Appendix C.
Revised: For a full proof, see Appendix C.

Original: Remark 1.
Revised: Remark 1.

Original: The proof for the modiﬁed generalization needs the smoothness of both the covariates xand labels y .
Revised: The proof for the modiﬁed generalization needs the smoothness of both the covariates xand labels y .

Original: This can be ensured with a loss of a constant in the binary (and generally the ﬁnite labelspace) setting.
Revised: This can be ensured with a loss of a constant in the binary (and generally the ﬁnite labelspace) setting.

Original: This is the main reason that this proof does not generalize to the real valued setting.
Revised: This is the main reason that this proof does not generalize to the real valued setting.

Original:  Discussion, Additional Results, and Open Problems
Revised:  Discussion, Additional Results, and Open Problems

Original: Computational Lower Bounds.
Revised: Computational Lower Bounds.

Original: Our main contribution is oracle-efﬁcient online learning algorithms in the smoothed setting that achieve an O ( p dT   1 ) regret upper bound in the real-valuedcase, and an O ( p dT   1 / 2 ) upper bound for binary classiﬁcation.
Revised: Our main contribution is oracle-efﬁcient online learning algorithms in the smoothed setting that achieve an O ( p dT   1 ) regret upper bound in the real-valuedcase, and an O ( p dT   1 / 2 ) upper bound for binary classiﬁcation.

Original: However, neither of these upperbounds is statistically optimal; the statistically optimal regret here is e ⇥ ( p dT log(1 /  ))
Revised: However, neither of these upperbounds is statistically optimal; the statistically optimal regret here is e ⇥ ( p dT log(1 /  ))

Original: [HRS21].
Revised: [HRS21].

Original: We ask the following question: is the above discrepancy an artifact of our regret analysis, or anintrinsic limitation of our or all oracle-efﬁcient algorithms.
Revised: We ask the following question: is the above discrepancy an artifact of our regret analysis, or anintrinsic limitation of our or all oracle-efﬁcient algorithms.

Original: We show in Theorem D.1 that the regret of our current family of algorithms cannot be improved bytuning parameters.
Revised: We show in Theorem D.1 that the regret of our current family of algorithms cannot be improved bytuning parameters.

Original: In addition, we also show the following general lower bound for computationallyefﬁcient algorithms.
Revised: In addition, we also show the following general lower bound for computationallyefﬁcient algorithms.

Original: See Appendix D for the proofs and more details.
Revised: See Appendix D for the proofs and more details.

Original: Theorem 5.1 (Computational Lower Bound for Smoothed Online Learning) .
Revised: Theorem 5.1 (Computational Lower Bound for Smoothed Online Learning) .

Original: For 1 /  d , any proper algorithm which only has access to the ERM oracle and achieves a regreto (min { T, p
Revised: For 1 /  d , any proper algorithm which only has access to the ERM oracle and achieves a regreto (min { T, p

Original: T ( d/  ) 1 / 2 } ) for any  -smoothed online learning problem must have an !
Revised: T ( d/  ) 1 / 2 } ) for any  -smoothed online learning problem must have an !

Original: ( p d/  )total running time.
Revised: ( p d/  )total running time.

Original: Here, the total running time refers to the total number of operations performed byour algorithm, in which each oracle call takes unit time, and maintaining each element in the input tothe oracle also takes unit time.

Original: Theorem 5.1 implies an exponential statistical-computational gap in smoothed online learning: forexponentially small  , achieving the statistical regret e
Revised: Theorem 5.1 implies an exponential statistical-computational gap in smoothed online learning: forexponentially small  , achieving the statistical regret e

Original: O ( p
Revised: O ( p

Original: Td log(1 /  )) requires an exponentialrunning time.
Revised: Td log(1 /  )) requires an exponentialrunning time.

Original: However, Theorem 5.1 still exhibits gaps to our computational upper bounds.
Revised: However, Theorem 5.1 still exhibits gaps to our computational upper bounds.

Original: Wediscuss this further in the appendix and present the following open problem.
Revised: Wediscuss this further in the appendix and present the following open problem.

Original: Open Question.
Revised: Open Question.

Original: For d/   T 2 in the smoothed setting, does any algorithm achieving o ( T ) regretrequire ⌦ ( poly ( T, 2 d , 1 /  ))
Revised: For d/   T 2 in the smoothed setting, does any algorithm achieving o ( T ) regretrequire ⌦ ( poly ( T, 2 d , 1 /  ))

Original: computational time given access to the ERM oracle?
Revised: computational time given access to the ERM oracle?

Original:  Check List
Revised:  Check List

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes](b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?
Revised: g., data splits, hyperparameters, how they were chosen)?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?
Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [N/A]
Revised: [N/A]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]