Smoothed Online Convex Optimization Based on Discounted-Normal-Predictor
Abstract
In this paper, we investigate an online prediction strategy named as DiscountedNormal-Predictor [Kapralov and Panigrahy, 2010] for smoothed online convex optimization (SOCO), in which the learner needs to minimize not only the hitting cost but also the switching cost.
In the setting of learning with expert advice,
Daniely and Mansour [2019] demonstrate that Discounted-Normal-Predictor can be utilized to yield nearly optimal regret bounds over any interval, even in the presence of switching costs.
Inspired by their results, we develop a simple algorithm for SOCO:
Combining online gradient descent (OGD) with different step sizes sequentially by Discounted-Normal-Predictor .
Despite its simplicity, we prove that it is able to minimize the adaptive regret with switching cost, i.e., attaining nearly optimal regret with switching cost on every interval.
By exploiting the theoretical guarantee of OGD for dynamic regret, we further show that the proposed algorithm can minimize the dynamic regret with switching cost in every interval.
Introduction
Recently, a variant of online convex optimization (OCO), named as smoothed OCO (SOCO) has received lots of attention in the machine learning community [Goel et al., 2019, Li and Li, 2020].
In each round t , the learner chooses an action w t from a convex domain W and a convex loss function f t ( · ) :
W (cid:55)→ R is selected by an adversary.
While in the traditional OCO, the learner only suffers a hitting cost f t ( w t ) , in SOCO, it further pays a switching cost m ( w t , w t − 1 ) , which penalizes the learner for changing its actions between rounds.
SOCO has found wide applications in real-world problems where the change of states brings additional costs [Lin et al., 2012, Kim and Giannakis, 2014, Goel et al., 2017], such as the wear-and-tear cost of switching servers [Lin et al., 2011].
For general convex functions, a natural choice of the switching cost is the distance between successive actions, i.e., m ( w t , w t − 1 )
= (cid:107)
[Equation]
t
[Equation]
t
[Equation]
1 (cid:107) .
Then, the total loss in the t -th round becomes
[Equation]
where λ ≥ 0 is the trade-off parameter, and G is the upper bound of the norm of gradients.
Here,
G is introduced to ensure that the two terms weighted by λ are on the same order.
Following the convention of online learning [Cesa-Bianchi and Lugosi, 2006], we choose regret to measure the performance, and meanwhile take into account the switching cost.
Let T be the total number of iterations.
The standard metric would be regret with switching cost :
[Equation]
deﬁned as the difference between the cumulative loss of the learner and that of the best ﬁxed decision in hindsight.
It is easy to verify that online gradient descent (OGD)
[Zinkevich, 2003] attains an
O ( (cid:112) (1 + λ ) T ) bound for regret with switching cost (c.f. Theorem 6), which can be proved to be optimal [Gradu et al., 2020, Theorem 4].
However, regret is not suitable for changing environments in which the best decision may vary over time.
To address this limitation, new performance measures, including adaptive regret and dynamic regret have been proposed [Zhang, 2020, Cesa-Bianchi and Orabona, 2021].
Adaptive regret enforces the algorithm to have a small regret over every interval [Hazan and Seshadhri, 2007, Daniely et al., 2015], which essentially measures the performance w.r.t.
a changing comparator.
By extending adaptive regret to SOCO, we obtain adaptive regret with switching cost :
[Equation]
where τ is the length of the interval, and
[Equation]
is the regret with switching cost over interval [ r, s ] .
To cope with changing environments, dynamic regret directly compares the learner against a sequence of comparators u 1 , . .
, u T ∈ W [Zinkevich, 2003].
Similarly, we can incorporate the switching cost into dynamic regret, and obtain dynamic regret with switching cost :
[Equation]
In the literature, there are only two works that investigate adaptive regret with switching cost.
However, one of them relies on strong convexity [Gradu et al., 2020], and the other one makes use of sophisticated techniques [Zhang et al., 2022].
There also exist preliminary studies on dynamic regret with switching cost, but are limited to the special case that λ = 1 /G
[Zhang et al., 2021a] or follow different settings [Li et al., 2018, Goel et al., 2019].
Besides, all of previous works only target one performance measure.
Notice that for OCO, we do have algorithms that are able to minimize adaptive regret and dynamic regret simultaneously [Zhang et al., 2020, Cutkosky, 2020].
In this paper, we develop a simple algorithm for SOCO based on Discounted-Normal-Predictor [Kapralov and Panigrahy, 2010, 2011], which is designed for online bit prediction and can be used to combine two experts.
In a recent study, Daniely and Mansour [2019] reveal that DiscountedNormal-Predictor automatically controls the switching cost, and with the help of projection, can be extended to support adaptive regret.
Motivated by their observations, we ﬁrst analyze a variant of
Discounted-Normal-Predictor with conservative updating (abbr.
DNP-cu), and prove that it suffers a small loss on every interval, even in the presence of switching costs.
To ensure adaptivity, we choose conservative updating instead of projection because the former one can be analyzed more easily.
Then, we create multiple OGD with different step sizes, and combine them sequentially by
DNP-cu.
Theoretical analysis shows that the proposed method achieves an O ( (cid:112) (1 + λ ) τ log T ) bound for adaptive regret with switching cost.
Furthermore, based on the fact that OGD is also equipped with dynamic regret bounds, we establish nearly optimal guarantees for dynamic regret with switching cost in every interval.
Speciﬁcally, for any interval [ r, s ] with length τ , our method attains an O ( (cid:112) (1 + λ ) τ (1 + P r,s ) log T ) bound for dynamic regret with switching cost, where
[Equation]
(5) is the path-length of an arbitrary comparator sequence u r , . .
, u s ∈ W .
Compared with state-of-the-art results [Zhang et al., 2021a, 2022], this paper has the following advantages.
Our O ( (cid:112) (1 + λ ) τ log T ) adaptive regret bound is on the same order as that of Zhang et al.
[2022], but our method is more simple.
In contrast, their algorithm pieces together several techniques, including online coin betting, geometric covering intervals
and the reduction from unconstrained online learning to constrained online learning.
Although Zhang et al.
[2021a] establish an O ( (cid:112) T (1 + P 1 ,T ))
dynamic regret bound with switching cost, they only consider the case
that λ = 1 /G and the whole interval [1 , T ] .
By comparison, our dynamic regret bound holds for any λ ≥ 0 and any interval.
To the best of our knowledge, this is the ﬁrst effort to minimize both adaptive regret and dynamic regret, under the setting of SOCO.
Finally, we would like to emphasize the strength of Discounted-Normal-Predictor, which, in our opinion, was not getting enough attention.
The results of Daniely and Mansour [2019] and this paper demonstrate that Discounted-Normal-Predictor provides an elegant way (and a very different methodology) to minimize both adaptive regret and dynamic regret, with or without switching cost, under the setting of learning with expert advice (LEA) or OCO.
Related Work
We brieﬂy review the related work on OCO and SOCO, as well as Discounted-Normal-Predictor.
2.1 Online Convex Optimization (OCO)
During the past decades, OCO has been extensively studied, and various algorithms have been proposed to minimize the regret, i.e., (2) with λ =
[Shalev-Shwartz, 2011, Hazan, 2016].
It is well-known that OGD [Zinkevich, 2003] achieves an O ( √ T ) bound which is minimax optimal
[Abernethy et al., 2008].
We can obtain tighter regret bounds if the loss function satisﬁes speciﬁcal curvature properties, such as strong convexity [Shalev-Shwartz et al., 2007], exponential concavity [Hazan et al., 2007, Orabona et al., 2012], and smoothness [Srebro et al., 2010, Chiang et al., 2012].
Adaptive regret has been explored under the setting of LEA [Littlestone and Warmuth, 1994, Freund et al., 1997, Adamskiy et al., 2012, György et al., 2012, Luo and Schapire, 2015] and OCO [Hazan and Seshadhri, 2007, 2009, Jun et al., 2017b, Zhang et al., 2019, 2021b, Wan et al., 2022].
We follow the deﬁnition of Daniely et al.
[2015], which is given by (3) with λ = 0 .
For general convex functions,
Jun et al.
[2017a] establish an O ( √ τ log T ) bound, which holds for any interval length τ .
Dynamic regret is proposed by Zinkevich [2003], and its deﬁnition can be found by setting λ = 0 in (4).
Existing works have analyzed two types of dynamic regret: (i) the general version where the comparator sequence is arbitrary [Hall and Willett, 2013, Zhang et al., 2018b, Zhao et al., 2020a,
Baby and Wang, 2021], and (ii) the worst-case version where u t is chosen as the minimizer of f t ( · ) [Jadbabaie et al., 2015, Besbes et al., 2015, Yang et al., 2016, Mokhtari et al., 2016, Zhang et al., 2017,
Zhao and Zhang, 2021].
For general convex functions, Ader attains an optimal O ( (cid:112) T (1 + P 1 ,T )) bound, where P 1 ,T is the path-length of u 1 , . . .
, u T [Zhang et al., 2018b].
Both adaptive regret and dynamic regret are designed for changing environments, but our understanding of their relationship is quite limited.
Currently, we know that it is possible to bound the worst-case dynamic regret by adaptive regret [Zhang et al., 2018c], and minimize adaptive regret and the general dynamic regret simultaneously [Zhang et al., 2020, Cutkosky, 2020].
2.2 Smoothed Online Convex Optimization (SOCO)
The research of SOCO is motivated by real-world applications where the switching cost plays a crucial role
[Lin et al., 2011, 2012].
Besides regret, competitive ratio is another popular metric for
SOCO, and a large number of algorithms have been proposed to yield dimension-free competitive ratio.
However, all of them are limited to the lookahead setting where the learner can observe the hitting cost f t ( · ) before deciding its action w t , and rely on strong conditions [Bansal et al., 2015,
Antoniadis and Schewior, 2018, Chen et al., 2018, Goel et al., 2019, Goel and Wierman, 2019, Argue et al., 2020a].
We note that SOCO is closely related to convex body chasing (CBC)
[Friedman and
Linial, 1993, Antoniadis et al., 2016, Bansal et al., 2018, Argue et al., 2019, Bubeck et al., 2019, 2020, Argue et al., 2020b, Sellke, 2020].
For more details, please refer to Lin et al.
In the study of online control, Gradu et al.
[2020] and Zhang et al.
[2022] have developed adaptive algorithms for OCO with memory, which can be applied to bounding adaptive regret with switching cost.
However, Gradu et al.
[2020] need to assume the hitting cost is strongly convex.
Although Zhang et al.
[2022] successfully deliver an O ( (cid:112) (1 + λ ) τ log T ) bound, their method is rather complex.
Require: Two parameters: n and Z 1: Set x 1 = 0 , and ρ = 1 − 1 /n 2: for t = 1 , . . .
, T do 3:
Predict g ( x t ) 4:
Receive b t 5:
Set x t +1 =
ρx t + b t 6: end for
Speciﬁcally, they make use of online coin betting [Orabona and Pál, 2016], geometric covering intervals [Daniely et al., 2015], and the reduction from unconstrained online linear optimization to constrained online linear optimization [Cutkosky and Orabona, 2018].
For dynamic regret with switching cost, Zhang et al.
[2021a] extend Ader to support switching costs, and prove an optimal O ( (cid:112) T (1 + P 1 ,T )) bound, but they only consider the case that λ
= 1 /G .
Other works on dynamic regret with switching cost are incomparable to our paper, because they either rely on strong convexity [Li et al., 2018, Li and Li, 2020], set the switching cost as the squared distance [Goel et al., 2019], or assume an upper bound of the total switching cost is given in advance [Chen et al., 2018, Zhao et al., 2020b].
2.3 Discounted-Normal-Predictor
Following the terminology of Kapralov and Panigrahy [2010], we introduce Discounted-NormalPredictor in the context of the bit prediction problem.
Let b 1 , . . .
, b T be an adversarial sequence of bits, where b t ∈ [ − 1 , 1] can take real values.
In each round t , the algorithm is required to output a conﬁdence level c t ∈ [ − 1 , 1] , then observes the value of b t , and ﬁnally gets a payoff c t b t .
The goal is to maximize the cumulative payoff of the algorithm (cid:80)
Tt =1 c t b t .
Let n > 0 be a parameter for the interval length, and deﬁne the discount factor ρ = 1 − 1 /n .
As the name suggests, Discounted-Normal-Predictor maintains a discounted deviation x t = (cid:80) t − 1 j =
ρ t − 1 − j b j at each round t , and the prediction is determined by g ( x t ) for a conﬁdence function g ( · ) deﬁned as
[Equation] [Equation]
where Z > 0 is a parameter, and erf( x ) = 2 √ π (cid:82) x 0 e − s 2 ds is the error function.
The complete procedure is summarized in Algorithm 1.
For any Z ≤ 1 /e , Kapralov and Panigrahy [2010, Theorem 14] have proved that Discounted-Normal-Predictor satisﬁes
[Equation]
where we set n = T in Algorithm 1.
By choosing Z = o (1 /T ) , we observe that it has O ( √ T log T ) regret against the strategy that predicts the majority bit (whose payoff is | (cid:80) j b j | ), as well as a subconstant o (1) loss.
To address the problem of learning with two experts, we can deﬁne b t as the difference between the losses of experts and restrict c t ∈ [0 , 1] .
It follows that the algorithm suffers
O ( √ T log T ) and o (1) regret w.r.t.
the two experts, respectively [Kapralov and Panigrahy, 2010,
Lemma 15].
Discounted-Normal-Predictor can be applied to the general setting of N experts, by aggregating experts one by one.
We note that the problem of trading off regret to the best expert for regret to the “special” expert stems from the study of Even-Dar et al.
[2007], and is later investigated by Sani et al.
Kapralov and Panigrahy [2010, Theorem 5] also investigate a variant of the adaptive regret, which uses an inﬁnite window with geometrically decreasing weighting.
To this end, they propose a conservative updating rule to control the value of the deviation x t .
Let U ( n ) =
O ( (cid:112) n log(1 /Z )) be a constant such that | g ( x ) | = 1 for | x | ≥ U ( n ) .
The current bit b t is utilized to update x t only when the conﬁdence of the algorithm is low i.e., | x t | < U ( n ) or when the algorithm predicts incorrectly i.e., g ( x t ) b t < 0 .
Otherwise, they will ignore b t and set x t +1 = ρx t .
However, it remains open whether their conservative updating works with the standard adaptive regret [Daniely et al., 2015], which is answered afﬁrmatively by our Theorem 1.
Daniely and Mansour [2019] extend Discounted-Normal-Predictor to support the switching cost and the adaptive regret.
The conﬁdence function is modiﬁed slightly as
[Equation]
where
[Equation]
and Π [0 , 1] [ · ] denotes the projection operation onto the set [0 , 1] .
First, by a more careful analysis, they demonstrate that Discounted-Normal-Predictor has similar regret bounds even in the presence of switching costs.
Second, they introduce a projection operation to prevent x t from being too large, and then derive tight bounds for the standard adaptive regret.
Speciﬁcally, the updating rule for x t becomes
[Equation]
where
[Equation]
Our work is inspired by Daniely and Mansour [2019], but with the following differences.
• While Daniely and Mansour [2019] consider the setting of LEA, we investigate OCO.
• To bound the adaptive regret, Daniely and Mansour [2019] introduce the projection operation, as shown in (9).
In contrast, we make use of the conservative updating [Kapralov and Panigrahy, 2010], and provide a more simple analysis.
• We study not only the adaptive regret, but also the dynamic regret.
Our algorithm is equipped with nearly optimal bounds for both metrics, in the presence of switching costs.
Main Results
We take Discounted-Normal-Predictor with conservative updating (DNP-cu) as our meta-algorithm, and present its performance by taking the switching cost into consideration.
Then, we use DNP-cu to combine multiple OGD sequentially, and show its adaptive regret and dynamic regret with switching cost.
Due to space limitations, all the proofs are deferred to Appendix A.
3.1 The Meta-algorithm
To consist with previous studies, we describe DNP-cu from the perspective of bit prediction, and require the prediction to lie in [0 , 1] so that it can be used later as a meta-algorithm to combine experts.
The detailed procedure is summarized in Algorithm 2.
Compared with the original algorithm [Kapralov and Panigrahy, 2010], we make two modiﬁcations.
We choose the conﬁdence function g ( · ) in (7), whose property has been revealed by Daniely and Mansour [2019, Lemmas 18 and 19] more formally.
We adapt the updating rule to the fact that g ( · ) belongs to [0 , 1] instead of [ − 1 , 1] .
Speciﬁcally, we perform the standard updating in Line 6, when the conﬁdence is low i.e., x t ∈ [0 , U ( n )] , or when the algorithm predicts incorrectly
i
e., x t < 0& b t > 0 or x t > U ( n )& b t < 0 .
Otherwise, we follow Line 8, which only shrinks x t and ignores b t .
To analyze the performance of DNP-cu, we consider both the payoff and the switching cost, referred to as reward below.
We have the following theorem which exploits the possibility that the magnitude of the bit sequence may be smaller than 1 .
Set
7: else 8: for
Theorem 1 Suppose Z ≤ 1 e and n ≥ max { 8 e, 16 log 1 Z } .
For any bit sequence b 1 , . . .
, b T such that | b t | ≤ µ ≤ 1 , the cumulative reward of Algorithm 2 over any interval [ r, s ] with length τ satisﬁes
[Equation]
where U ( n ) is deﬁned in (10).
Furthermore, for intervals starting from 1 , we have s
[Equation]
And the change of successive predictions satisﬁes
[Equation]
Remark:
First, the above theorem reveals that DNP-cu controls the switching cost automatically, and the trade-off between the payoff and the switching cost is determined by the magnitude of the bit sequence.
Second, thanks to the conservative updating, we are able to bound the cumulative reward over any interval, which can be exploited to support adaptive regret.
Recall that in the total loss (1), the trade-off parameter λ could be arbitrary.
As a result, to utilize
DNP-cu for SOCO, we also need a ﬂexible way to balance the payoff and the switching cost.
To this end, we introduce λ and deﬁne the reward in the t -round as g ( x t ) b t − λ | g ( x t ) − g ( x t − 1 ) | .
Then, our goal is to maximize
[Equation]
for every interval [ r, s ] .
Based on Theorem 1, a straightforward way is to multiply the bit sequence by 1 /λ , and then pass it to Algorithm 2.
But in this way, the lower bound will scale linearly with λ .
To improve the dependence on λ , we will multiply the bit sequence by 1 / √ λ [Daniely and Mansour, 2019], and have the following corollary based on (11) and (13) of Theorem 1.
Corollary 2 Suppose Z ≤ 1 e , n ≥ max { 8 e, 16 log 1 Z } , and b 1 , . .
, b T is a bit sequence such that | b t | ≤ 1 .
Running Algorithm 2 over the scaled bit sequence
[Equation]
for any interval [ r, s ] with length τ , we have
[Equation]
And the change of successive predictions satisﬁes
[Equation]
Remark:
From the deﬁnition of U ( n ) in (10), we have s (cid:88)
[Equation]
for any interval [ r, s ] .
Notice that the bound in (17) is independent of the interval length τ , so it holds even when τ is larger than n .
On the other hand, for any interval [ r, s ] whose length is no larger than n , i
[Equation]
e., τ/n ≤ 1 , we also have the following regret bound of Algorithm 2 w.r.t.
the baseline strategy which always outputs 1 :
[Equation]
As elaborated in the analysis (c.f. Lemma 3), when using Algorithm 2 to combine multiple algorithms, there are two issues that need to be addressed.
We do not destroy the theoretical guarantee of early algorithms, which is ensured by (17).
We can inherit the theoretical guarantee of the current algorithm, which is archived by (18).
3.2 Smoothed OGD
We introduce the following common assumptions for OCO [Shalev-Shwartz, 2011].
Assumption 1 All the functions f t ’s are convex over the domain W .
Assumption 2 The gradients of all functions are bounded by G , i.e.,
[Equation] [Equation]
Assumption 3 The diameter of the domain W is bounded by D , i.e
[Equation] [Equation]
Without loss of generality, we assume
[Equation]
since we can always redeﬁne f t ( w ) as f t ( w ) − min w ∈W f t ( w ) which belongs to [0 , GD ] according to (19) and (20).
First, we explain how to use DNP-cu to combine predictions of two algorithms designed for OCO.
Let A 1 and A 2 be two online learners, and denote their predictions in the t -th round by w 1 t and w 2 t , respectively.
Let A be a meta-algorithm which outputs a convex combination of w 1 t and w 2 t , i.e.,
[Equation]
(22) where the weight w t ∈ [0 , 1] .
We have the following lemma regarding
Algorithm
Require: Three parameters: M , n and Z
Require: Two algorithms: A 1 and A1:
Let D be an instance of DNP-cu, i.e., Algorithm 2, with parameter n and Z 2: Receive w 11 and w 21 from A 1 and A 2 respectively 3:
Receive the prediction w 1 from D 4: for t = 1 , . .
, T do 5:
Predict w t according to (22) 6:
Send the loss function f t ( · ) to A 1 and A7:
Receive w 1 t +1 and w 2 t +1 from A 1 and A 2 respectively 8:
Send the bit (cid:96) t max( √ λ, 1) to D , where (cid:96) t is deﬁned in (28) 9:
Receive the prediction w t +1 from D 10: end for
Lemma 1 Assume the outputs of A 1 and A 2 move slowly such that
[Equation] [Equation]
where M ≥ 0 is some constant.
Under Assumptions 1, 2 and 3, the meta-regret of A w.r.t.
A 1 over any interval [ r, s ] satisﬁes s
[Equation]
(24) and the meta-regret of A w.r.t.
A 2 satisﬁes s (cid:88) t = r
[Equation]
25) where
[Equation] [Equation]
Remark:
By mapping w t and (cid:96) t in Lemma 1 to g ( x t ) and b t in Corollary 2, we immediately see that the weight w t in (22) can be determined by invoking DNP-cu, i.e., Algorithm 2 to process the scaled bit sequence
[Equation]
Then, we can utilize lower bounds in Corollary 2 to establish upper bounds for the regret in Lemma 1.
We name the strategy of aggregating two algorithms by DNP-cu as Combiner, and summarize its procedure in Algorithm 3.
In the following, we will use online gradient descent (OGD) with constant step size [Zinkevich, 2003] as our expert-algorithm.
OGD performs gradient descent to update the current solution w t :
[Equation]
Require: Three parameters: K , M and Z 1: for i = 1 , . . .
, K do 2:
Set n ( i ) =
T 2 1 − i 3:
Let A i be an instance of OGD with step size η ( i ) deﬁned in (29) 4: if i = 1 then 5:
Set B 1 = A6: else 7:
Let B i be an instance of Combiner, i.e., Algorithm 3 which combines B i − 1 and A i with parameters M , n ( i ) and Z 8: end if 9: end for 10: for t = 1 , . . .
, T do 11:
Run B 1 , . . .
, B K sequentially for one step 12:
Predict the output of B K , denoted by w t 13: end for where η > 0 is the step size, and Π W [ · ] denotes the projection onto W .
Notice that it is important to choose a constant step size, which makes it easy to analyze the regret, as well as the dynamic regret, over any interval [ r, s ] .
First, we create K instances of OGD, denoted by A 1 , . . .
, A K , where the value of K will be determined later.
The step size of A i is set to be
[Equation]
where
[Equation]
Then, we use Combiner, i.e., Algorithm 3 to aggregate them sequentially.
We will create a sequence of algorithms B 1 , . . .
, B K , where B i is obtained by combining B i − 1 with A i , and B 1 =
A 1 .
The parameter n in Algorithm 3 is set to be n ( i ) when forming B i .
In each iteration t , we invoke
B 1 , . . .
, B K sequentially for one step, and return the output of B K as the prediction w t .
The completed procedure is named as smoothed OGD, and summarized in Algorithm 4.
Remark: Our method has a similar structure with those of Cutkosky [2020] and Zhang et al.
[2022], in the sense that we all combine multiple experts sequentially.
In contrast, other approaches for adaptive regret use a two-level framework, where a meta-algorithm aggregates multiple experts (which is allowed to sleep) simultaneously [Hazan and Seshadhri, 2007, Daniely et al., 2015, Jun et al., 2017a,b, Zhang et al., 2019, 2021b].
On the other hand, all the previous works, including
Cutkosky [2020] and Zhang et al.
[2022], need to construct a set of sub-intervals, and maintain a sub-routine for each one.
In this way, they can use a small number of sub-intervals to cover any possible interval, and attain a small regret on that interval.
By comparison, our method does not rely on any special construction of sub-intervals, making it more elegant.
3.3 Theoretical Guarantees
Next, we provide the theoretical guarantee of smoothed OGD.
We ﬁrst characterize its regret with switching cost over any interval.
Theorem 3 Assume
[Equation]
31) and set
[Equation]
M = 2 and Z = 1 /T in Algorithm 4.
Under Assumptions 1, 2 and 3, we have
[Equation]
for any interval [ r, s ] with length τ , and any w ∈ W .
Remark: Our O ( (cid:112) (1 + λ ) τ log T ) bound is on the same order as that of Zhang et al.
When λ = 0 , we get the O ( √ τ log T ) adaptive regret for general convex functions [Jun et al., 2017a].
Our proposed method is also equipped with nearly optimal dynamic regret with switching cost over any interval, as stated below.
Theorem 4 Under the condition of Theorem 3, we have
[Equation]
where τ is the interval length, and P r,s , deﬁned in (5), is the path-length of an arbitrary comparator sequence u r , . .
, u s ∈ W .
Remark:
The above theorem shows that our method can minimize the dynamic regret with switching cost over any interval.
According to the Ω( (cid:112) T (1 + P 1 ,T )) lower bound of dynamic regret [Zhang et al., 2018b, Theorem 2], we know that our upper bound is optimal, up to a logarithmic factor.
Theorem 4 is very general and can be simpliﬁed in different ways.
If we choose a ﬁxed comparator such that P r,s = 0 , Theorem 4 reduces to Theorem 3 and matches that of Zhang et al.
If we ignore the switching cost and set λ = 0 , we obtain an O ( (cid:112) τ (1 + P r,s ) log T ) bound for dynamic regret over any interval, which recovers the results of Zhang et al.
[2020, Theorem 4] and Cutkosky [2020, Theorem 7].
When both P r,s = 0 and λ = 0 , we obtain the O ( √ τ log T ) adaptive regret of Jun et al.
[2017a].
Conclusion and Future Work
Based on a variant of Discounted-Normal-Predictor (DNP-cu), we design a novel algorithm, named as smoothed OGD for SOCO.
Our algorithm combines multiple instances of OGD sequentially by
DNP-cu, and thus is very simple.
Theoretical analysis shows that it attains nearly optimal bounds for both adaptive regret and dynamic regret over any interval, in the presence of switching costs.
We also conduct preliminary experiments to verify our theories, and present the results in Appendix B.
For adaptive regret and dynamic regret, we can obtain tighter bounds when the hitting cost exhibits additional curvature properties such as exponential concavity [Hazan and Seshadhri, 2007, Baby and
Wang, 2021] and smoothness [Zhang et al., 2019, Zhao et al., 2020a].
It remains unclear whether
DNP-cu can exploit such information to further improve the performance.
For SOCO, it is common to consider the lookahead setting, and the problem is still nontrivial due to the coupling created by the switching cost [Chen et al., 2018].
For dynamic regret with switching cost, Zhang et al.
[2021a] have demonstrated that Assumption 2 is unnecessary in the lookahead setting.
It would be interesting to develop a lookahead version of DNP-cu, and investigate whether we can drop Assumption 2 as well.
Acknowledgments and Disclosure of Funding
This work was partially supported by NSFC (62122037, 61976112), and the Major Key Project of
PCL (PCL2021A12).
[-Then, we must have x k +
-]
[-Then, we must have x k +
-]
[-Lemma 
-]
[-Lemma 
-]
[-+
-]
[-+
-]
[-+
-]