Title: An Equivalence Between Static and Dynamic Regret Minimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: online learning

dynamic regret

non-stationary

parameter-free
Abstract: We study the problem of dynamic regret minimization in online convex optimization, in which the objective is to minimize the difference between the cumulative loss of an algorithm and that of an arbitrary sequence of comparators. While the literature on this topic is very rich, a unifying framework for the analysis and design of these algorithms is still missing. In this paper we show that *for linear losses, dynamic regret minimization is equivalent to static regret minimization in an extended decision space*. Using this simple observation, we show that there is a frontier of lower bounds trading off penalties due to the variance of the losses and penalties due to variability of the comparator sequence, and provide a framework for achieving any of the guarantees along this frontier. As a result, we also prove for the first time that adapting to the squared path-length of an arbitrary sequence of comparators to achieve regret $R_{T}(u_{1},\dots,u_{T})\le O(\sqrt{T\sum_{t} \\|u_{t}-u_{t+1}\\|^{2}})$ is impossible. However, using our framework we introduce an alternative notion of variability based on a locally-smoothed comparator sequence $\bar u_{1}, \dots, \bar u_{T}$, and provide an algorithm guaranteeing dynamic regret of the form $R_{T}(u_{1},\dots,u_{T})\le \tilde O(\sqrt{T\sum_{i}\\|\bar u_{i}-\bar u_{i+1}\\|^{2}})$, while still matching in the worst case the usual path-length dependencies up to polylogarithmic terms.

An Equivalence Between Static and Dynamic Regret Minimization

Abstract

We study the problem of dynamic regret minimization in online convex optimization, in which the objective is to minimize the difference between the cumulative loss of an algorithm and that of an arbitrary sequence of comparators. While the literature on this topic is very rich, a unifying framework for the analysis and design of these algorithms is still missing. In this paper we show that for linear losses, dynamic regret minimization is equivalent to static regret minimization in an extended decision space . Using this simple observation, we show that there is a frontier of lower bounds trading off penalties due to the variance of the losses and penalties due to variability of the comparator sequence, and provide a framework for achieving any of the guarantees along this frontier. As a result, we also prove for the first time that adapting to the squared path-length of an arbitrary sequence of comparators to achieve regret R T ( u 1 , . . . , u T ) ≤ O ( √ T ∑ t ∥ u t -u t +1 ∥ 2 ) is impossible. However, using our framework we introduce an alternative notion of variability based on a locallysmoothed comparator sequence ¯ u 1 , . . . , ¯ u T , and provide an algorithm guaranteeing dynamic regret of the form R T ( u 1 , . . . , u T ) ≤ ˜ O ( √ T ∑ i ∥ ¯ u i -¯ u i +1 ∥ 2 ) , while still matching in the worst case the usual path-length dependencies up to polylogarithmic terms.

1 Introduction

This paper introduces new techniques for Online Convex Optimization (OCO), a framework for designing and analyzing algorithms which learn on-the-fly from a stream of data [14, 51, 5, 31, 6]. Formally, consider T rounds of interaction between the learner and their environment. In each round, the learner chooses w t ∈ W from a convex feasible set W ⊆ R d , the environment reveals a G -Lipschitz convex loss function ℓ t : W → R , and the learner incurs a loss of ℓ t ( w t ) . The classic objective in this setting is to minimize the learner's regret relative to any fixed benchmark u ∈ W :

In this paper, we study the more general problem of minimizing the learner's regret relative to any sequence of benchmarks u 1 , . . . , u T ∈ W [17, 18]:

Francesco Orabona

This objective is typically referred to as dynamic regret, to distinguish it from the special case where the comparator sequence is fixed u 1 = · · · = u T (referred to as static regret). We focus in particular on the special case of Online Linear Optimization (OLO), in which ℓ t ( w ) = ⟨ g t , w ⟩ for some g t ∈ R d . Note that OCO problems can always be reduced to OLO via the well-known inequality ℓ t ( w t ) -ℓ t ( u ) ≤ ⟨ g t , w t -u ⟩ for g t ∈ ∂ℓ t ( w t ) , where ∂ℓ t ( w t ) is the subdifferential set of ℓ t at w t [see, e.g. , 36], so throughout this paper we will focus on the OLO setting.

Intuitively, if the sequence of comparators u 1 , . . . , u T varies too much, it should be impossible to achieve low dynamic regret. On the other hand, we know it is possible to achieve sublinear regret if the sequence of comparators is constant, i.e. , u 1 = · · · = u T , because this is simply the static case. Hence, we need a way to quantify the complexity, or variability , of the comparator sequence. The most commonly used notion of complexity in this regard is the path-length of the comparator sequence [17, 18], defined as

It is possible to show that Online Gradient Descent has a dynamic regret of O (( D + P T ) G T in bounded domains, where D is an upper bound on the diameter of the feasible set and G is the Lipschitz constant of the losses [51]. This bound was improved to O ( √ DP ∥·∥ T G √ T ) and shown to be minimax optimal by Zhang et al. [46].

∥·∥ √ )

Notice that the path-length bounds scale with a rather pessimistic constant of D = sup w,w ′ ∈W ∥ w -w ′ ∥ . A better bound would instead scale with the squared path-length:

which can be significantly smaller 2 than the penalty in the bound above: P ∥·∥ 2 T ≤ DP ∥·∥ T . However, guarantees scaling with P ∥·∥ 2 T are not well understood in general compared with the more common P ∥·∥ T bounds, and have only been obtained by restricting the comparator sequence to u t = argmin w ∈W ℓ t ( w ) or under additional assumptions such as strong-convexity [44, 45, 7].

In this paper, we focus on the challenging case that the domain is unbounded , where recent works have achieved the dynamic regret ˜ O ( √ max t,t ′ ∥ u t -u t ′ ∥ P ∥·∥ T T ) in the worst case [20, 25, 21, 47]. Of particular interest, Jacobsen and Cutkosky [20], Zhang et al. [47] achieve bounds of the form

which avoids the pessimistic multiplicative penalty of max t,t ′ ∥ u t -u t ′ ∥ , but results in a coupling between the gradient and variability penalties. It is unclear if it is possible to obtain a guarantee which cleanly separates the variability and variance penalties, to achieve dynamic regret scaling as

R T ( u 1 , . . . , u T ) ≤ O (√ P ∥·∥ 2 T ∑ T t =1 ∥ g t ∥ 2 ) . In fact, it is not clear in general how to reason about potential trade-offs that may result from adapting to different measures of variability.

Contributions. In this paper, we show how to reformulate the dynamic regret miniminization problem as an equivalent static regret problem (Section 2). This equivalence allows us to use results for the static regret setting to prove both upper and lower bounds for dynamic regret.

In our first application of this equivalence, we show that the ideal guarantee scaling the with squared path-length O ( √ P ∥·∥ 2 T ∑ T t =1 ∥ g t ∥ 2 ) is not possible in general (Section 3). We do this by proving

a novel lower bound showing that there is a fundamental trade-off between the penalties incurred due to comparator variability and penalties incurred due to loss variance, leading to a new frontier of dynamic regret lower bounds.

Our second application is to provide a framework for achieving any of the variance/variability tradeoffs along the lower bound frontier, up to polylogarithmic terms (Section 4). Our framework allows us to develop dynamic regret algorithms by simply choosing suitable dual-norm pairs ( ∥·∥ , ∥·∥ ∗ ) in the static regret problem. Along with our matching lower bound, this framework provides a concrete way to reason about different measures of comparator variability and the trade-offs they entail, and to design algorithms achieving those trade-offs.

While our lower bound demonstrates that the ideal squared path-length guarantee cannot be achieved, using our framework we show that it is possible to achieve an alternative guarantee that scales with

where ¯ u ( τ ) i is a local average of the comparator sequence at a timescale of τ (see Section 4.1). Similar to P ∥·∥ 2 T , this variability measure maintains the property that it matches the worst-case guarantees based on path-length up to polylogarithmic terms, i.e. , ¯ P ∥·∥ 2 T ≤ ˜ O (max t,t ′ ∥ u t -u t ′ ∥ P ∥·∥ T ) . These are the first guarantees for general OCO that fully decouple the variance and variability penalties for dynamic regret without explicitly incurring pessimistic max t,t ′ ∥ u t -u t ′ ∥ penalties.

Related Work. Our approach is inspired by the Haar OLR algorithm of Zhang et al. [47]. In that work, dynamic regret is approached by interpreting the comparator sequence as a high-dimensional 'signal' which is decomposed into a frequency domain representation using a dictionary of features. Then, for each feature vector in the dictionary, a 1-dimensional parameter-free [32, 28] algorithm is used to learn how well that feature correlates with the losses. This allows one to compete with an arbitrary comparator sequence, so long as it can be represented in terms of the chosen dictionary of features. We take a similar but slightly more general approach. Our framework also represents the comparator sequence as a high-dimensional signal, but we instead use this signal to define an equivalent static regret problem , a perspective that allows us design algorithms for dynamic regret by simply choosing suitable dual-norm pairs.

Other prior works in the general OCO setting have also studied various alternative forms of variability such as the temporal variability ∑ T -1 t =1 sup w ∈W | ℓ t ( w ) -ℓ t +1 ( w ) | [3, 22, 4] or deviation of the comparator from a given dynamical model ∑ T -1 t =1 ∥ u t -Φ t ( u t -1 ) ∥ [16]. Alternative variance penalties have also been studied in the dynamic setting, such as the small-loss penalties ∑ T t =1 ℓ t ( u t ) or gradient variation penalties ∑ T t =1 sup w ∈W ∥∇ ℓ t ( w ) -∇ ℓ t +1 ( w ) ∥ [15, 49, 21, 50]. It is also possible to achieve a smaller regret with stronger assumptions on the losses [2]. It is important to note however that almost all prior works, with the exception of Jacobsen and Cutkosky [20], Luo et al. [25] and Zhang et al. [47], study dynamic regret only in the easier bounded domain setting.

There is also an often ignored connection between measures of comparator variability and the function classes studied in non-parametric regression. In particular, considering the case that the losses are ℓ t ( x ) = ( x -u t ) 2 , then the sequence of comparators u 1 , . . . , u T with bounded path length C T and bounded squared path length ( C ′ T ) 2 corresponds to the sequence with discrete total variation bounded by C T and the discrete Sobolev class with bound C ′ T , respectively. In this setting, the minimax rates are known [24, 35] and Koolen et al. [24] obtain the minimax regret for the Sobolev classes, while Baby and Wang [1] for both classes with slightly stronger assumptions. However, these results are not directly related to this paper because we consider linear losses.

Notations. We will use the following definitions and notations. The elements of a matrix A ∈ R n × m are denoted by A ij for i = 1 , . . . , n and j = 1 , . . . , m . Similarly, the elements of a vector u ∈ R d are u i for i = 1 , . . . , d . The Kronecker product of matrices A ∈ R m × n and B ∈ R p × q is the block matrix defined by

We let e t denote the t th standard basis vector of R T and I d is the d × d identity matrix. For a square matrix A , Diag ( A ) is the diagonal matrix that contains the elements of the diagonal of A . For a positive definite matrix M , we define the weighted norm ∥ x ∥ M := √ ⟨ x , Mx ⟩ . For a matrix A ∈ R m × n , we denote its Frobenius norm by ∥ A ∥ F := √ ∑ m i =1 ∑ n j =1 A 2 i,j . The vec operator is the mapping defined by stacking the columns of a matrix A in a vector. We will denote by ∥ A ∥ p,p the entry-wise p -norm of A , i.e. , ∥ A ∥ p,p := ∥ vec ( A ) ∥ p .

2 A dynamic-to-static reduction

In this section, we present a general reduction from dynamic regret to static regret. The key idea is to embed the comparator sequence in a high dimensional space W T , where T is the number of rounds, so that competing with a fixed comparator ˜ u ∈ W T in this high-dimensional space is equivalent to competing with a sequence of comparators in the original space W . In this way, we can reduce the problem of minimizing the dynamic regret to the one of minimizing the static regret.

Our reduction is shown in Algorithm 1. We simply embed the linear losses g t in a high-dimensional space by setting

where e t ∈ R T is the t th standard basis vector of R T and 0 d ∈ R d denotes the vector of zeros. We pass these losses to the online learning algorithm A , which predicts with a vector ˜ w t ∈ W T . Finally, we set w t ∈ R d equal to the t th 'component' of ˜ w t , and play w t .

We show that the dynamic regret of the resulting algorithm will be equal to the static regret of the algorithm A . In particular, for any sequence ⃗ u = ( u 1 , . . . , u T ) in W we will denote the concatenation of ⃗ u into a single vector in W T as

Then, the following proposition shows that the dynamic regret of Algorithm 1 w.r.t any sequence ⃗ u = ( u 1 , . . . , u T ) is equal to the static regret of A w.r.t ˜ u .

Proposition 1. Let W ⊆ R d and let A be an online learning algorithm with domain W T . Then, for any sequence ⃗ u = ( u 1 , . . . , u T ) ∈ W T , Algorithm 1 guarantees

Proof. The proof is immediate from Equations (2) and (3). In fact, observe that the cumulative loss of the comparator sequence is precisely

We get a similar relationship for the algorithm's cumulative loss. Hence, we have R T ( ⃗ u ) = ∑ T t =1 ⟨ g t , w t -u t ⟩ = ∑ T t =1 ⟨ ˜ g t , ˜ w t -˜ u ⟩ = R Seq T ( ˜ u ) .

Remark 1. It is important to note that the regret equivalence holds in the context of linear losses . However, our reduction can still be leveraged for arbitrary convex losses by first applying the standard reduction to OLO: ∑ T t =1 ℓ t ( w t ) -ℓ t ( u t ) ≤ ∑ T t =1 ⟨ g t , w t -u t ⟩ = R Seq T ( ˜ u ) for g t ∈ ∂ℓ t ( w t ) .

While our reduction is exceptionally simple, its utility should not be understated. Proposition 1 is a regret equivalence -welose nothing by taking this perspective, yet it allows us to immediately apply all the usual techniques and approaches from the static regret setting. For instance, given any dual norm pair ( ∥·∥ , ∥·∥ ∗ ) , it is well-understood how to develop algorithms which adapt simultaneously to the comparator norm ∥ ˜ u ∥ and to the gradient variance ∑ T t =1 ∥ ˜ g t ∥ 2 ∗ to guarantee

Such algorithms are commonly referred to as 'parameter-free', or 'comparator adaptive', because they achieve this adaptation by completely removing the parameter that depends on the unknown comparator ˜ u [ e.g. , 26, 27, 32, 9, 12, 20, 8, 21, 48]. In this way, we have effectively reduced the problem of minimizing dynamic regret to the problem of selecting a dual-norm pair ( ∥·∥ , ∥·∥ ∗ ) that meaningfully measures the 'difficulty' of the sequence in ˜ u and the losses ˜ g t . In particular, ( ∥·∥ , ∥·∥ ∗ ) should be chosen with the following considerations in mind:

∥ ˜ u ∥ should produce a meaningful measure of variability of the comparator sequence u 1 , . . . , u T . For instance, we will show in Proposition 2 that the squared path-length arises from a particular weighted norm applied to ˜ u .

∥ ˜ g t ∥ ∗ should not 'blow up'. Ideally ∥ ˜ g t ∥ ∗ should match the magnitude of the true losses g t up to polylog factors.

( ∥·∥ , ∥·∥ ∗ ) should be chosen with computational considerations in mind. For instance, to apply an FTRL-based algorithm to the losses ˜ g t ∈ R dT , efficient implementation will typically require ∥·∥ ∗ to have sparse subgradients. In general, an ideal dual-norm pair should facilitate updating only O (log T ) variables at a time, so as to match the O ( d log T ) per-step computation enjoyed by existing dynamic regret algorithms. We will see one such example in Section 4.1.

In the next section, we show that there is in fact a fundamental trade-off between the penalties induced by the dual-norm pair ( ∥·∥ , ∥·∥ ∗ ) , creating a tension between the first two considerations.

3 Lower bounds for unconstrained dynamic regret

In the static regret setting, there is a well-known trade-off between the way in which we measure the complexity of the comparator u and the way in which we measure the complexity of the linear losses g t . For example, in Online Mirror Descent [29, 42] one can get a regret guarantee that depends on the maximum diameter of the feasible set with respect to a norm ∥ · ∥ , while the linear losses are measured using the dual norm ∥ · ∥ ∗ . The equivalence in Proposition 1 suggests that a similar tension exists for the dynamic regret.

Given the structure of our reduction, it makes sense to focus on the weighted norms ∥·∥ M and ∥·∥ M -1 , where M is a symmetric positive definite matrix. In particular, the next theorem shows that there is a fundamental trade-off between a variability penalty ∥ ˜ u ∥ M and a variance penalty G 2 Tr( M -1 ) related to the losses. The proof is provided in Appendix A.1 and it is based on a lower bound to the tail of Rademacher chaos of order 2.

Theorem 1. Let the number of rounds T ≥ T 0 , where T 0 is a universal constant. Let A be an online learning algorithm, and suppose A guarantees R T (0) ≤ Gϵ T for any sequence of linear losses g 1 , . . . , g T ∈ R satisfying | g t | ≤ G . Let M -1 ∈ R T × T be any symmetric positive definite matrix, denote ˜ M -1 := M -1 -Diag ( M -1 ) and V T := Tr( M -1 ) + ∥ ˜ M -1 ∥ F . Suppose that ∥ ˜ M -1 ∥ 2 F ≥ T 2 max i ∑ j ( ˜ M -1 ij ) 2 . Then, for any P satisfying T 0 ≤ log 2 √ PV T 2 ϵ T ≤ T , there is a sequence of losses g 1 , . . . , g T ∈ R , and ˜ u = ( u 1 , . . . , u T ) ⊤ ∈ R T satisfying ∥ ˜ u ∥ M = √ P such that we have

Let us first briefly discuss the conditions on M . First, note that the restriction that M be positive definite and symmetric simply specifies that ∥·∥ M defines a valid norm. The condition on ∥ ˜ M -1 ∥ F = ∥ M -1 -Diag ( M -1 ) ∥ F is less straight forward to interpret, but it essentially states that the total 'variance' of ˜ M -1 is at least as much as that of any of its columns. on a technical level this assumption leads to the restriction that P satisfies log 2 ( √ PV T / 2 ϵ T ) ≤ T . This is a natural restriction which encodes the fact that if P is too large relative to T ( i.e. , when log 2 ( √ PV/ 2 ϵ T ) ≥ T ), one can ensure 'low' regret by simply playing w t = 0 on every round:

and hence the only lower bounds in such settings are trivial ones and it suffices to consider only P satisfying log ( √ PV T / 2 ϵ ) ≤ T . We will see in Proposition 2 that the matrix that produces the squared path-length satisfies this condition, and it can be seen that symmetric matrices with equal column sums (as is the case in Proposition 3) satisfy this condition as well.

The result of Theorem 1 shows that there is a frontier of lower bounds which trade off penalties related to variability of the comparator sequence and penalties related to the variance of the subgradients. That is, one can not guarantee a small variability penalty in all situations without also accepting a large subgradient variance penalty. The next proposition shows that i) the squared path-length can be represented by a particular choice of the weighted norm ∥ ˜ u ∥ M , and ii) the fundamental tension between ∥ ˜ u ∥ M and its corresponding variance penalty Tr( M -1 ) prevents any algorithm from attaining the ideal variability dependence of ∥ ˜ u ∥ M = (√ ∑ T -1 t =1 ∥ u t -u t +1 ∥ 2 ) . In fact, the corresponding variance penalty is G 2 Tr( M -1 ) = O ( G 2 T 2 ) , resulting in a vacuous guarantee. Proof of the proposition can be found in Appendix A.2.

Proposition 2. (Adapting to Squared Path-length Requires Superlinear Regret) Define the finitedifference operator Σ ∈ R T as the matrix with entries

Let S = Σ ⊤ Σ and M = S ⊗ I d . Then, M satisfies the assumptions of Theorem 1 and

Proposition 2 shows that adapting to the squared path-length of an arbitrary comparator sequence necessarily requires incurring a linear penalty, so adapting to the squared path-length is impossible without facing a vacuous guarantee. However, we will show in Section 4.1 that it is possible to adapt to a measure of variability which is similar in spirit to the squared path-length, yet only incurs a Tr( M -1 ) = O (log T ) variance penalty.

Remark 2. The matrix M in Proposition 2 uniquely exposes the the squared path-length up to the bias term ∥ u T ∥ 2 . Such a bias term must appear because in the static regret setting, wherein u 1 = . . . = u T = u , the variability measure ∥·∥ M must still reduce to a dependence on ∥ u ∥ , otherwise

the guarantee would violate existing ˜ Ω( ∥ u ∥ √ T ) lower bounds for static regret. More generally, we show in Appendix A.3 that any other choice of bias would similarly lead to Tr ( M -1 ) ≥ Ω( T 2 ) , so Proposition 2 along with our lower bound in Theorem 1 are sufficient to show that adapting to squared path-length requires accepting a vacuous guarantee.

4 Dynamic regret for unconstrained OLO via weighted norms

So far, we've seen that there exists a frontier of lower bounds trading off a variability penalty, measured by ∥ ˜ u ∥ M , and a loss variance penalty, measured by Tr( M -1 ) , and that the tension between these two quantities makes it impossible to adapt to the squared path-length of the comparator sequence without accepting a vacuous regret guarantee. A natural next question is whether there are choices of M which lead to a more favorable trade-off of these two quantities. In this section, we provide a simple framework for achieving lower bounds along the frontier described by Theorem 1, and an instance which successfully achieves an improved variance/variability trade-off. The guarantees on the lower bound frontier can be achieved using any parameter-free algorithm along with the 1-dimensional reduction of Cutkosky and Orabona [9] to extend the algorithm to dual-norm pair ( ∥·∥ M , ∥·∥ M -1 ) . The generic procedure is summarized in Algorithm 2 for convenience.

Theorem 2. Let S ∈ R T × T be a symmetric positive definite matrix, M = S ⊗ I d , and ϵ > 0 . There is an algorithm A such that for any g 1 , . . . , g T ∈ R d satisfying ∥ g t ∥ 2 ≤ G for all t and any sequence ⃗ u = ( u 1 , . . . , u T ) ∈ R dT , the dynamic regret is bounded as

For the proof, we will need the following technical lemma.

Lemma 1. Let S ∈ R T × T be a symmetric positive definite matrix and let M = S ⊗ I d . For t = 1 , . . . , T , let g t ∈ R d and let ˜ g t = e t ⊗ g t . Then, we have ∥ ˜ g t ∥ 2 M = ∥ g t ∥ 2 2 S tt .

Proof. Using the mixed-product property ( A ⊗ B )( C ⊗ D ) = AC ⊗ BD and the transpose property ( A ⊗ B ) ⊤ = A ⊤ ⊗ B ⊤ of the Kronecker product, we have that

Proof of Theorem 2. Applying Proposition 1, we have R T ( ⃗ u ) = ∑ T t =1 ⟨ ˜ g t , ˜ w t -˜ u ⟩ = R Seq T ( ˜ u ) . Since M is symmetric and positive definite, ( ∥·∥ M , ∥·∥ M -1 ) is a valid dual-norm pair. By Lemma 1,

we have ∥ ˜ g t ∥ 2 M -1 = ∥ g t ∥ 2 2 S -1 tt ≤ G 2 ∥ ∥ S -1 ∥ ∥ ∞ , ∞ := G 2 . Hence, let A be any algorithm which guarantees a parameter-free regret w.r.t. ( ∥·∥ , ∥·∥ ∗ ) on losses satisfying ∥ ˜ g t ∥ M -1 ≤ G . Note that any parameter-free algorithm can be extended to handle arbitrary dual-norm pairs by leveraging the one-dimensional reduction of Cutkosky and Orabona [9, Section 3], that reduces the OLO problem to a unconstrained 1d problem plus an OLO problem in the unitary ball defined by the primal norm. For instance, applying Jacobsen and Cutkosky [20, Algorithm 1] with the one-dimensional reduction one can easily show (see details in Appendix B.1)

Note in particular that by Lemma 1, we have ∑ T t =1 ∥ ˜ g t ∥ 2 M -1 = ∑ T t =1 S -1 tt ∥ g t ∥ 2 ≤ G ∑ T t =1 S -1 tt = G Tr( S -1 ) , so this bound matches the lower bound from Section 3, up to polylogarithmic terms. 3 Thus, any valid choice of M will be on the lower bound frontier of Section 3.

4.1 Trading-off Variance and Variability

Leveraging the algorithm characterized by Theorem 2, we now show that it is indeed possible to choose M such that ∑ T t =1 ∥ ˜ g t ∥ 2 M -1 is only O (log ( T ) ∑ T t =1 ∥ g t ∥ 2 ) , in exchange for a variability penalty which is still similar in spirit to the squared path-length.

Inspired by the Haar OLR algorithm of [47], we apply Theorem 2 using S = H n H ⊤ n , where H n is the unnormalized Haar basis matrix of order n = ⌈ log 2 T ⌉ . The Haar wavelet transform and its basis matrix are common tools in the signal processing literature; we recall the basic definitions and facts for convenience in Appendix B.2. With this choice, we have the following bounds on ∥ ˜ u ∥ M and ∥ ˜ g t ∥ 2 M -1 . The proof can be found in Appendix B.3.

Proposition 3. Let n = log 2 T and H n be the unnormalized Haar basis matrix of order n . For any τ ∈ { 2 i : i = 0 , . . . , log 2 T } , let N τ = T/τ and let I ( τ ) 1 , . . . , I ( τ ) N τ be a partition of [ T ] into intervals of length τ . Define the average comparator in interval I ( τ ) i to be ¯ u ( τ ) i = 1 τ ∑ t ∈I ( τ ) i u t , and define the squared path-length at time-scale τ < T to be

and ¯ P ( ⃗ u , T ) = ∥ ∥ ∥ ¯ u ( T ) 1 ∥ ∥ ∥ 2 2 = ∥ ¯ u ∥ 2 2 . Then, setting S = [ H n H ⊤ n ] -1 and M = S ⊗ I d , we have

Summarizing, by applying Algorithm 1 with S = [ H n H ⊤ n ] -1 we ensure regret

This is the first fully decoupled guarantee for general dynamic regret which incurs no pessimistic multiplicative penalties of the form max t,t ′ ∥ u t -u t ′ ∥ . That is, the terms depending on the comparators and the terms depending on the gradients appear in separate sums. Moreover, observe that

this measure of variability can immediately be related to the more standard (first-order/non-squared) path-length using the local averaging lemma of Zhang et al. [47] (Lemma D.7). We have

where ¯ D = max τ,i ∥ ∥ ∥ ¯ u ( τ ) i -¯ u ( τ ) i +1 ∥ ∥ ∥ ≤ max i,j ∥ u i -u j ∥ . Thus, applying Algorithm 1 with dualnorm pair ( ∥·∥ H -⊤ n H ⊤ n , ∥·∥ H n H ⊤ n ) still guarantees worst-case regret

which matches the guarantees of prior works, up to polylogarithmic terms.

̸

Importantly, with M = H -⊤ n H -1 n ⊗ I d the dual-norm pair ( ∥·∥ M , ∥·∥ M -1 ) leads to updates that can be implemented efficiently, in requiring only O (log T ) variables to be updated on each round. This is because the Haar basis matrices are locally supported -the columns of H n = ( h (1) . . . h ( T ) ) ∈ R T × T form an orthogonal basis with the property that for any t , [ h ( i ) ] t = 0 for only 1 + log 2 T indices i (see Proposition 5). Hence, ( H ⊤ ⊗ I d ) ˜ g t = ( H ⊤ ⊗ I d )( e t ⊗ g t ) = ( H ⊤ e t ) ⊗ g t , is a block vector with only 1 + log 2 T active blocks, requiring that we update only O ( d log T ) indices to maintain each of the variables needed to implement Algorithm 2. We provide the full details of this computation in Appendix B.4, which we summarize below in Proposition 4.

Proposition 4. The algorithm characterized by applying Theorem 2 with S = [ H n H ⊤ n ] -1 can be implemented with O ( d log T ) per-round computation.

5 Recovering Variance-Variability Coupling Guarantees

Our main focus throughout the paper has been on designing algorithms that achieve a regret bounds of the form R T ( ⃗ u ) ≤ O ( √ f ( u 1 , . . . , u T ) V ( g 1 , . . . , g T ) ) for some functions f and V , which cleanly separates the penalties associated with difficult loss sequences from the penalties associated with difficult comparator sequences. However, the first works to achieve unconstrained dynamic regret guarantees uncovered guarantees of a slightly different form, containing a gradient-comparator correlation penalty:

for some reference point ¯ u [20, 47]. Guarantees of this form allow some degree of coupling between the variability and variance penalties. This can be appealing in certain situations. For instance, guarantees of the form above have the appealing property that the variance penalty completely disappears on any rounds where the comparator u t matches the reference point ¯ u . This can be a very powerful property when one has a priori access to a benchmark model (represented by ¯ u ) which can be expected to predict well on average , so that we accumulate the variance penalties only when facing atypical/unexpected conditions.

The prior works achieving a coupling guarantee do so using rather mysterious means. For instance, the guarantee of Jacobsen and Cutkosky [20] achieves the coupling guarantee almost by coincidence, as it appears in response to a composite regularizer they add to the update to cancel out certain unstable terms in the analysis, and the analysis of Zhang et al. [47] recovers a guarantee of a similar form using a rather difficult analysis of the frequency-domain representation of ˜ u after projecting onto the Haar basis vectors. So far there is no unifying explanation of the principles leading to these sorts of guarantees.

Our equivalence in Proposition 1 instead shows that guarantees of the form Equation (4) can instead be understood through the lens of reward-regret duality, a standard tool used to design algorithms in the static regret setting. The reward-regret duality states that in order to guarantee regret of the form R T ( u ) ≤ f ( u ) for all u ∈ W , it suffices to design an algorithm that guarantees -∑ T t =1 ⟨ g t , w t ⟩ ≥ f ∗ ( -∑ T t =1 g t ) for any g 1 , . . . , g T . Using Proposition 1, we immediately have the following analogous design principle for dynamic regret. Proof is deferred to Appendix C.1.

Theorem 3. Let Wealth T := -∑ T t =1 ⟨ ˜ g t , ˜ w t ⟩ denote the 'wealth' of an algorithm A and let ( f, f ∗ ) be a Fenchel conjugate pair. Then A guarantees Wealth T ≥ f ∗ T ( -∑ T t =1 ˜ g t ) for any sequence ˜ g 1 , . . . , ˜ g T if and only if R T ( ⃗ u ) ≤ f T ( ˜ u ) for any sequence ⃗ u = ( u 1 , . . . , u T ) in W , where ˜ u = ( u ⊤ 1 , . . . , u ⊤ T ) ⊤ is the concatenation of the sequence ⃗ u into a vector.

So, suppose we would like to design an algorithm that guarantees for any sequence ⃗ u = ( u 1 , . . . , u T ) and any ⃗ g = ( g 1 , . . . , g T ) regret of the form

for some f T ( ˜ u ) and V T ( ˜ u ) = V T ( ˜ u ; ⃗ g ) . Then, since √ ab = min η ≥ 0 a 2 η + η 2 b , any such algorithm must have R T ( ⃗ u ) ≤ f T ( ˜ u ) 2 η + η 2 V T ( ˜ u ) for every η ≥ 0 . So, via Proposition 1 and the the reward-regret duality of Theorem 3, we have that the desired guarantee is equivalent to guaranteeing for all η ≥ 0 a wealth lower bound of

where f ∗ T and V ∗ T are the Fenchel conjugates of f T and V T respectively, and ( f 1 □ f 2 ) denotes the infimal convolution [34, 19] of f 1 and f 2 :

Thus, the variance/variability coupling guarantees observed in Equation (4) can be interpreted as achieving wealth lower-bounds for potential functions involving infimal convolution.

The above discussion provides a general characterization of variance/variability coupling guarantees, though it is admittedly less clear how difficult it is to design algorithms from this perspective due to the rather complicated potential function that appears. Nonetheless, we believe that this provides a valuable perspective and insight that could be of general interest. An important direction for future work is to develop useful tools for working with potential functions of this form.

6 Conclusion

In this paper, we have shown a way to reduce the problem of dynamic regret minimization to the static one. We proved a novel frontier of lower bounds showing a fundamental trade-off between penalties on the comparators and penalties on the variance of the gradients. In particular, we have shown that it is not possible to achieve a guarantee that scales with √ ∑ T -1 t =1 ∥ u t -u t +1 ∥ 2 without incurring a variance penalty of O ( GT ) . We developed a simple framework for achieving guarantees along the lower bound frontier, and used it to develop the first algorithm making a non-trivial variance/variability decoupling guarantee against arbitrary comparator sequences. Our framework is simple but powerful, allowing one to fully utilize the rich literature of static regret algorithms for online learning.

We conclude by noting some directions for future work. There is a lot of exciting potential to explore different measures of variability induced by different choices of the matrix M , as well as going beyond weighted norms. As mentioned in Section 5, developing a useful toolset for potential functions involving infimal convolution is an important next-step for developing and understanding guarantees with a coupled variance/variability penalty, such as Equation (4). Also, our lower bound in Section 3 illustrates the variance-variability trade-off, but achieving the correct logarithmic dependencies proved to be very challenging - many of the standard tools for proving lower bounds in unconstrained settings revolve around anti-concentration results that do not readily extend to arbitrary weighted norms and higher-dimensions. We look forward to exciting development in these future directions.

Acknowledgments

We thank Yu-Xiang Wang for the discussion on the function classes studied in non-parametric regression theory.

References

Dheeraj Baby and Yu-Xiang Wang. Online forecasting of total-variation-bounded sequences. In Advances in Neural Information Processing Systems , volume 32, 2019.

Dheeraj Baby and Yu-Xiang Wang. Optimal dynamic regret in exp-concave online learning. In Conference on Learning Theory , pages 359-409. PMLR, 2021.

Omar Besbes, Yonatan Gur, and Assaf Zeevi. Non-stationary stochastic optimization. Operations Research , 63(5):1227-1244, 2015. doi: 10.1287/opre.2015.1408.

Nicolò Campolongo and Francesco Orabona. A closer look at temporal variability in dynamic online learning, 2021.

Nicolò Cesa-Bianchi and Gabor Lugosi. Prediction, learning, and games . Cambridge University Press, 2006.

Nicolò Cesa-Bianchi and Francesco Orabona. Online learning algorithms. Annual Review of Statistics and Its Application , 8:165-190, 2021.

Ting-Jui Chang and Shahin Shahrampour. On online optimization: Dynamic regret analysis of strongly convex and smooth problems. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 6966-6973, 2021.

Keyi Chen, Ashok Cutkosky, and Francesco Orabona. Implicit parameter-free online learning with truncated linear models. In International Conference on Algorithmic Learning Theory , pages 148-175. PMLR, 2022.

Ashok Cutkosky and Francesco Orabona. Black-box reductions for parameter-free online learning in banach spaces. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors, Proceedings of the 31st Conference On Learning Theory , volume 75 of Proceedings of Machine Learning Research , pages 1493-1529. PMLR, 06-09 Jul 2018.

Irit Dinur, Ehud Friedgut, Guy Kindler, and Ryan O'Donnell. On the Fourier tails of bounded functions over the discrete cube. In Proceedings of the thirty-eighth annual ACM symposium on Theory of computing , pages 437-446, 2006.

Bogdan J Falkowski. Generalized haar spectral representations and their applications. Nanyang Technological University. Singapore , 1998.

Dylan J. Foster, Alexander Rakhlin, and Karthik Sridharan. Online learning: Sufficient statistics and the burkholder method. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors, Proceedings of the 31st Conference On Learning Theory , volume 75 of Proceedings of Machine Learning Research , pages 3028-3064. PMLR, 06-09 Jul 2018.

Gene H Golub and Charles F Van Loan. Matrix computations . JHU press, 2013.

Geoffrey J. Gordon. Regret bounds for prediction problems. In Proc. of the twelfth annual conference on Computational learning theory (COLT) , pages 29-40, 1999.

Andras Gyorgy and Csaba Szepesvari. Shifting regret, mirror descent, and matrices. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning , volume 48 of Proceedings of Machine Learning Research , pages 2943-2951, New York, New York, USA, 20-22 Jun 2016. PMLR.

Eric C. Hall and Rebecca M. Willett. Online optimization in dynamic environments, 2016.

Mark Herbster and Manfred K Warmuth. Tracking the best regressor. In Proceedings of the eleventh annual conference on Computational learning theory , pages 24-31, 1998.

Mark Herbster and Manfred K Warmuth. Tracking the best linear predictor. Journal of Machine Learning Research , 1(281-309):10-1162, 2001.

Jean-Baptiste Hiriart-Urruty and Claude Lemaréchal. Fundamentals of convex analysis . Springer Science & Business Media, 2004.

Andrew Jacobsen and Ashok Cutkosky. Parameter-free mirror descent. In Po-Ling Loh and Maxim Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory , volume 178 of Proceedings of Machine Learning Research , pages 4160-4211. PMLR, 02-05 Jul 2022.

Andrew Jacobsen and Ashok Cutkosky. Unconstrained online learning with unbounded losses. In International Conference on Machine Learning (ICML) . PMLR, 2023.

Ali Jadbabaie, Alexander Rakhlin, Shahin Shahrampour, and Karthik Sridharan. Online Optimization : Competing with Dynamic Comparators. In Guy Lebanon and S. V. N. Vishwanathan, editors, Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics , volume 38 of Proceedings of Machine Learning Research , pages 398-406, San Diego, California, USA, 09-12 May 2015. PMLR.

Charles Royal Johnson. Positive definite matrices. The American Mathematical Monthly , 77(3): 259-264, 1970.

Wouter M Koolen, Alan Malek, Peter L Bartlett, and Yasin Abbasi-Yadkori. Minimax time series prediction. In Advances in Neural Information Processing Systems , volume 28, 2015.

Haipeng Luo, Mengxiao Zhang, Peng Zhao, and Zhi-Hua Zhou. Corralling a larger band of bandits: A case study on switching regret for linear bandits. In Po-Ling Loh and Maxim Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory , volume 178 of Proceedings of Machine Learning Research , pages 3635-3684. PMLR, 02-05 Jul 2022.

Brendan Mcmahan and Matthew Streeter. No-regret algorithms for unconstrained online convex optimization. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems , volume 25. Curran Associates, Inc., 2012.

H. Brendan McMahan and Francesco Orabona. Unconstrained online linear learning in Hilbert spaces: Minimax algorithms and normal approximations. In Maria Florina Balcan, Vitaly Feldman, and Csaba Szepesvári, editors, Proceedings of The 27th Conference on Learning Theory , volume 35 of Proceedings of Machine Learning Research , pages 1020-1039, Barcelona, Spain, 13-15 Jun 2014. PMLR.

Zakaria Mhammedi and Wouter M. Koolen. Lipschitz and comparator-norm adaptivity in online learning. In Conference on Learning Theory , pages 2858-2887. PMLR, 2020.

Arkadij Semenoviˇ c Nemirovskij and David Borisovich Yudin. Problem complexity and method efficiency in optimization . Wiley, New York, NY, USA, 1983.

Ryan O'Donnell and Yu Zhao. Polynomial bounds for decoupling, with applications. arXiv preprint arXiv:1512.01603 , 2015.

Francesco Orabona. A modern introduction to online learning. arXiv preprint arXiv:1912.13213 , 2019. Version 6.

Francesco Orabona and Dávid Pál. Coin betting and parameter-free online learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems , NIPS'16, page 577-585, Red Hook, NY, USA, 2016. Curran Associates Inc.

Francesco Orabona and Dávid Pál. Scale-free online learning. Theoretical Computer Science , 716:50 - 69, 2018. ISSN 0304-3975. doi: https://doi.org/10.1016/j.tcs.2017.11.021. Special Issue on ALT 2015.

R. Tyrrell Rockafellar. Convex Analysis . Princeton University Press, 1970.

Veeranjaneyulu Sadhanala, Yu-Xiang Wang, and Ryan J Tibshirani. Total variation classes beyond 1d: Minimax rates, and the limitations of linear smoothers. In Advances in Neural Information Processing Systems , volume 29, 2016.

Shai Shalev-Shwartz. Online learning and online convex optimization. Foundations and Trends in Machine Learning , 4(2), 2011.

Radomir S. Stankovi´ c and Bogdan J. Falkowski. The Haar wavelet transform: its status and achievements. Computers & Electrical Engineering , 29(1):25-44, 2003. ISSN 0045-7906.

Willi-Hans Steeb and Tan Kiat Shi. Matrix calculus and Kronecker product with applications and C++ programs . World Scientific, 1997.

Josef Stoer, Roland Bulirsch, R Bartels, Walter Gautschi, and Christoph Witzgall. Introduction to numerical analysis , volume 2. Springer, 1980.

Matthew Streeter and H Brendan McMahan. Less regret via online conditioning. arXiv preprint arXiv:1002.4862 , 2010.

David F Walnut. An introduction to wavelet analysis . Springer Science & Business Media, 2013.

Manfred K Warmuth and Arun K Jagota. Continuous and discrete-time nonlinear gradient descent: Relative loss bounds and convergence. In Electronic proceedings of the 5th International Symposium on Artificial Intelligence and Mathematics , 1997.

Henry Wolkowicz and George PH Styan. Bounds for eigenvalues using traces. Linear algebra and its applications , 29:471-506, 1980.

Tianbao Yang, Lijun Zhang, Rong Jin, and Jinfeng Yi. Tracking slowly moving clairvoyant: Optimal dynamic regret of online learning with true and noisy gradient. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning , volume 48 of Proceedings of Machine Learning Research , pages 449-457, New York, New York, USA, 2016. PMLR.

Lijun Zhang, Tianbao Yang, Jinfeng Yi, Rong Jin, and Zhi-Hua Zhou. Improved dynamic regret for non-degenerate functions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017.

Lijun Zhang, Shiyin Lu, and Zhi-Hua Zhou. Adaptive online learning in dynamic environments. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages 1330-1340, 2018.

Zhiyu Zhang, Ashok Cutkosky, and Yannis Paschalidis. Unconstrained dynamic regret via sparse coding. Advances in Neural Information Processing Systems , 36, 2024.

Zhiyu Zhang, Heng Yang, Ashok Cutkosky, and Ioannis C Paschalidis. Improving adaptive online learning using refined discretization. In International Conference on Algorithmic Learning Theory , pages 1208-1233. PMLR, 2024.

Peng Zhao, Yan-Feng Xie, Lijun Zhang, and Zhi-Hua Zhou. Efficient methods for non-stationary online learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 11573-11585. Curran Associates, Inc., 2022.

Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. Adaptivity and non-stationarity: Problem-dependent dynamic regret for online convex optimization. Journal of Machine Learning Research , 25(98):1-52, 2024.

Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03) , pages 928-936, 2003.

A Proofs for Section 3 (Lower bounds for unconstrained dynamic regret)

In this section, we provide proof of our main lower bound result from Section 3. We first introduce a technical tool from the literature on decoupling theory and a key lemma (Lemma 2). Proof of our main result is in Appendix A.1.

Consider a function f : [ -1 , 1] d → R , defined as

Define A the matrix with elements A i,j . In this section we will use the following notations for quantities related to a polynomial induced by the quadratic form x ↦→ ⟨ x , Ax ⟩ (see page 6 of O'Donnell and Zhao [30])

One of the key difficulties in deriving the lower bound is that squared weighted norms x ↦→⟨ x , Ax ⟩ introduce dependencies between the coordinates of x , which breaks the usual lower bound arguments which rely on anti-concentration of independent Rademacher random variables. Instead, we must leverage an anti-concentration result that holds for polynomials of random variables.

Theorem 4 (Theorem 3 of Dinur et al. [10]) . There is a universal constant C such that the following holds. Suppose G : {± 1 } d → R is a polynomial of degree at most 2 and assume Var[ g ] = 1 . Let t ≥ 1 and suppose that Inf i [ g ] ≤ C -2 t -2 for all i ∈ [ d ] . Then

Using this anti-concentration result, the following key lemma provides a general lower bound on the wealth obtainable by any algorithm, subject to the weighting imposed by a matrix A .

Lemma 2. Let A be an online learning algorithm, and suppose A guarantees R T (0) ≤ Gϵ T for any sequence of linear losses g 1 , . . . , g T ∈ R satisfying | g t | ≤ G . Let A ∈ R T × T be any symmetric positive definite matrix, and let B = A -Diag ( A ) . Then, there is a universal constant C > 0 such that for any 1 ≤ q ≤ ∥ B ∥ F C √ 2 max i ∑ T j =1 B 2 ij , there is a sequence of losses g 1 , . . . , g T ∈ R such that

Proof. Let Y 1 , . . . , Y T be independent Rademacher random variables and set g t = GY t , so that E [ R T (0)] = E [ ∑ T t =1 g t w t ] = 0 . Then, using the regret equivalence of Proposition 1 and conditioning on any event E with P {E} > 0 , we have

where the last line uses the fact that A guarantees R T (0) ≤ Gϵ T for any g 1 , . . . , g T satisfying | g t | ≤ G for all t . Re-arranging, we have

and

Next, let ˜ g t = e t ⊗ g t for all t and consider the event

for some q > 0 . We proceed by lower bounding the probability of this event.

Observe that

̸

Denote B = A -Diag ( A ) and note that f ( Y 1 , . . . , Y T ) = ∑ i,j Y i Y j B ij is a polynomial of degree at most 2 and variance Var[ f ] = ∑ i,j B 2 ij = ∥ A -Diag ( A ) ∥ 2 F = ∥ B ∥ 2 F . Moreover, since A is symmetric we have Inf i [ f ] = ∑ T j =1 B 2 ij + B 2 ji = 2 ∑ T j =1 B 2 ij for any i . It follows that if we let g ( Y ) = f ( Y ) √ ∥ B ∥ 2 F = f ( Y ) ∥ B ∥ F , then g is a polynomial of degree at most 2 , Var[ g ] = 1 , and for any i ∈ [ T ] we have Inf i [ g ] = 2 ∑ T j =1 B 2 ij ∥ B ∥ 2 F . Hence by Theorem 4, there is a universal constant C such that for any 1 ≤ q ≤ ∥ B ∥ F C √ 2 max i ∑ T j =1 B 2 ij it holds that

Now observe that P {E} = P { f ( Y ) ≥ q ∥ B ∥ F } by construction, so Equation (5) can be bound as

which implies the existence of a sequence g 1 , . . . , g T ∈ R such that R T (0) ≥ Gϵ T [ 1 -2 4 C 2 q 2 ] and

A.1 Proof of Theorem 1

In this section we prove our main lower bound.

Theorem 1. Let the number of rounds T ≥ T 0 , where T 0 is a universal constant. Let A be an online learning algorithm, and suppose A guarantees R T (0) ≤ Gϵ T for any sequence of linear losses g 1 , . . . , g T ∈ R satisfying | g t | ≤ G . Let M -1 ∈ R T × T be any symmetric positive definite matrix, denote ˜ M -1 := M -1 -Diag ( M -1 ) and V T := Tr( M -1 ) + ∥ ˜ M -1 ∥ F . Suppose that ∥ ˜ M -1 ∥ 2 F ≥ T 2 max i ∑ j ( ˜ M -1 ij ) 2 . Then, for any P satisfying T 0 ≤ log 2 √ PV T 2 ϵ T ≤ T , there is a sequence of losses g 1 , . . . , g T ∈ R , and ˜ u = ( u 1 , . . . , u T ) ⊤ ∈ R T satisfying ∥ ˜ u ∥ M = √ P such that we have

Proof. Denote A = M -1 and B = A -Diag ( A ) . By Lemma 2, there is a universal constant C and a sequence g 1 , . . . , g T ∈ R such that for any 1 ≤ q ≤ ∥ B ∥ F C √ 2 max i ∑ T j =1 B 2 ij , it holds that

and

Hence, choosing comparator sequence u 1 , . . . , u T ∈ R to satisfy and ˜ u = ( u 1 , . . . , u T ) ⊤ = -√ P A ∑ T t =1 ˜ g t ∥ ∑ T t =1 ˜ g t ∥ A ∈ R T , we have ∥ ˜ u ∥ A -1 = ∥ ˜ u ∥ M = √ P and

Now, for

Indeed, observe that this choice satisfies 1 ≤ q ≤ ∥ B ∥ F C √ 2 max i ∑ T j =1 B 2 ij as required:

where the final inequality uses the assumption ∥ B ∥ 2 F / 2 max i ∑ ij B 2 ij ≥ T 4 . Hence, we have that

so that the overall the regret can be lower-bounded as

A.2 Proof of Proposition 2

Proposition 2. (Adapting to Squared Path-length Requires Superlinear Regret) Define the finitedifference operator Σ ∈ R T as the matrix with entries

Let S = Σ ⊤ Σ and M = S ⊗ I d . Then, M satisfies the assumptions of Theorem 1 and

Proof. We first show the properties that ∥ ˜ u ∥ 2 F = ∥ u T ∥ 2 2 + ∑ T -1 t =1 ∥ u t -u t +1 ∥ 2 2 and Tr( Σ -1 Σ -⊤ ) = ∑ T t =1 [ Σ -1 Σ -⊤ ] tt = ∑ T t =1 T -t +1 = T ( T +1) 2 , and show that M satisfies the conditions of Theorem 1 at the end.

Observe that

and since ( Σ ⊤ ⊗ I d )( Σ ⊗ I d ) = ( Σ ⊤ Σ ) ⊗ I d = M , we have

Using the inverse property of the Kronecker product, we also have

and by Lemma 8 we have that Σ -1 is the upper-triangular matrix of all 1 's, that is, the matrix with entries

and likewise, Σ -⊤ is a lower-triangular matrix of 1 ′ s . In other words, for any t we have

So, summing over t we have

̸

Now we show that M satisfies the conditions of Theorem 1. M = S ⊗ I d = [ Σ ⊤ Σ ] ⊗ I d is clearly symmetric since it is the Kronecker product of two symmetric matrices. Observe that for any x = 0 ∈ R T we have Σ x = 0 by positive definiteness of Σ (Lemma 8) and thus ⟨ x , S x ⟩ = ⟨ Σ x , Σ x ⟩ > 0 . Thus, M = S ⊗ I d is the Kronecker product of symmetric positive definite matrices, so M is symmetric positive definite [see, e.g. , 38, Chapter 2].

̸

Lastly, let B = Σ -1 Σ -⊤ -Diag ( Σ -1 Σ -⊤ ) . We are to show that ∥ B ∥ F ≥ T 2 ∑ j B 2 ij for any i . First observe that calculation of [ Σ -1 Σ -⊤ ] tt is generalized to

̸

for any i, j , and likewise B ij = T -max { j, i } +1 for i = j and 0 otherwise, from which it is easily seen that max i ∑ j B 2 ij = ∑ j B 2 1 j , so for any i we have

On the other hand,

for any i , where the last line applies the inequality in the previous display.

A.3 Sufficiency of Proposition 2

The choice of M in Proposition 2 uniquely exposes the squared path-length up to the constant offset term ∥ u T ∥ 2 . In this section we demonstrate that the choice of offset term in Proposition 2 does not make any significant difference for the claim that adapting to the squared path-length requires incurring a Tr( M -1 ) ≥ Ω( T 2 ) penalty, and hence that Proposition 2 is sufficient to demonstrate that adapting to the squared path-length is not possible without incurring vacuous regret.

To expedite the discussion, we first introduce two technical lemmas, proven in Appendices A.3.1 and A.3.2 respectively.

Lemma 3. Let v ∈ R T be an arbitrary non-zero vector and let B ∈ R T × T be a symmetric matrix with eigenvalues 0 = λ 1 ( B ) < λ 2 ( B ) ≤ . . . ≤ λ T ( B ) . Then

Lemma 4. Let Σ ∈ R T × T denote the finite-difference matrix defined in Proposition 2 and let M = Σ ⊤ Σ . Then, for any T > 1 , we have

where λ max ( M -1 ) is the maximal eigenvalue of M -1 .

Now, Consider the 1-dimensional setting and note that for any positive definite M we can find a unique Σ such that M = Σ ⊤ Σ . Hence,

so without loss of generality we can focus on Σ for which

where v = 0 ∈ R T . 4 Note such a constant offset term is unavoidable: it is what captures the static regret guarantee in the case where u 1 = . . . = u T = u . Proposition 2 considers v = (0 , . . . , 0 , 1) to get ∥ ˜ u ∥ 2 M = ∥ u T ∥ 2 + ∑ T t =2 ∥ u t -u t -1 ∥ 2 , though below we will show that any vector v would still lead to Tr( M -1 ) = Ω( T 2 ) .

̸

It is clear that the only way to construct expressions of the form above is via matrices Σ satisfying

where c ∈ {-1 , 1 } and the order of the rows indices of the vector can be permuted without loss of generality. In particular, the only matrices that can produce these expressions (again noting that the rows can be permuted without loss of generality) are of the form

so M = Σ ⊤ Σ = ∆ ⊤ ∆ + vv ⊤ . Moreover, ∆ ⊤ ∆ is a symmetric matrix with a unique zero eigenvalue (corresponding to vectors in the span of 1 = (1 , . . . , 1) ∈ R T ), so applying Lemma 3,

Now, define v 0 = (0 , . . . , 0 , 1) ∈ R T and observe that M 0 := ∆ ⊤ ∆ + v 0 v ⊤ 0 is precisely the matrix studied in Proposition 2. We have via the interlacing property of rank-1 updates to symmetric matrices that λ t ( ∆ ⊤ ∆ ) ≤ λ t ( M 0 ) [13, Theorem 8.1.8], so overall we have

where the last inequality applies Lemma 4 to bound λ max ( M -1 0 ) and recalls Tr( M -1 0 ) = T ( T +1) 2 from Proposition 2.

Hence, the variance penalty will still be Ω( T 2 ) regardless of the choice of bias ⟨ v , ˜ u ⟩ 2 in the variability measure. Combined with our lower bound in Theorem 1, it follows that adapting to the squared path-length necessarily implies a variance penalty of Tr( M -1 ) ≥ Ω( T 2 ) , leading to a vacuous regret upper bound.

A.3.1 Proof of Lemma 3

Lemma 3. Let v ∈ R T be an arbitrary non-zero vector and let B ∈ R T × T be a symmetric matrix with eigenvalues 0 = λ 1 ( B ) < λ 2 ( B ) ≤ . . . ≤ λ T ( B ) . Then

Proof. Let A := B + vv ⊤ . Since B is symmetric, we have via the interlacing property that there is an a 1 , . . . , a T ≥ 0 such that ∑ T t =1 a t = ∥ v ∥ 2 and λ t ( A ) = λ t ( B ) + a i [see, e.g. , Theorem 8.1.8 in 13]. Hence,

To analyze the constrained optimization in the last line, let α 1 , . . . , α T ≥ 0 , β ∈ R , and define the Lagrangian

For any t , we have

Plugging this into the dual D ( α 1 , . . . , α T , β ) = min a 1 ,...,a T L ( a 1 , . . . , a T , α 1 , . . . , α T , β ) we have

The derivatives of the dual w.r.t α t are

Observe that for λ 1 ( B ) = 0 , we have ∂D ∂α t = -1 √ β -α t ≤ 0 , so D is decreasing in α 1 , so the dual is maximized when α 1 = 0 . Using the relation a 1 = 1 √ β -α 1 -λ 1 ( B ) above we have a 1 = 1 √ β -α 1 -λ 1 ( B ) = 1 √ β . Equating the other derivatives for t > 1 to zero we have

where we used the relationship a t = 1 √ β -α t -λ t ( B ) from above. Finally, the optimal β is such that ∑ T t =1 a t = 1 √ β = ∥ v ∥ 2 , so overall we have

A.3.2 Proof of Lemma 4

Lemma 4. Let Σ ∈ R T × T denote the finite-difference matrix defined in Proposition 2 and let M = Σ ⊤ Σ . Then, for any T > 1 , we have

where λ max ( M -1 ) is the maximal eigenvalue of M -1 .

Proof. The matrix M -1 = Σ -1 Σ -⊤ is symmetric and positive definite, hence has real eigenvalues. The eigenvalues of M -1 can be bound in terms of its trace as follows (see, e.g. , Theorem 2.1 Wolkowicz and Styan [43], provided for convenience in Theorem 5):

Next, observe that by Lemma 8, matrix Σ -1 is an upper-triangular matrix of all 1 's, so that

Hence,

Moreover,

Thus,

Overall, λ max ( M -1 ) is bounded by

where the last line observes that 1 -1 √ 3 ≤ 1 2 ≤ T 4 for T ≥ 2 and the fact that 1 √ 3 + 1 4 ≈ 0 . 83 ≤ 9 10 .

B Proofs for Section 4 (Dynamic regret for unconstrained OLO via weighted norms)

B.1 Details on the 1-Dimensional Reduction

In this section, for completeness we provide the details of the 1-dimensional reduction of Cutkosky and Orabona [9], specialized to dual weighted-norm pairs ( ∥·∥ M , ∥·∥ M -1 ) as well as its regret guarantee.

For concreteness, we choose adaptive FTRL with AdaGrad-norm stepsizes [40] as the direction learner. For simplicity we use the scale-free version of [33], so that the direction learner's update is slightly simpler, not requiring prior knowledge of the Lipschitz constant G ≥ ∥ ˜ g t ∥ M -1 .

Using Cutkosky and Orabona [9, Theorem 2], we have that the regret of Algorithm 2 is equal to

where R A T is the regret of A over a sequence of G -Lipschitz linear losses and R direction T is the regret of (scale-free) adaptive FTRL with a feasible set equal to the unitary ball defined by ∥ · ∥ M .

Choosing the algorithm A to be [20, Algorithm 1], we have

Focusing now on the regret of the direction learner, define the distance generating function ψ ( ˜ x ) = 1 2 ∥ ˜ x 2 ∥ M . Using [31, Theorem 4.3], we have that ψ is 1-strongly convex w.r.t ∥ · ∥ M . Hence, using the regret guarantee of Scale-free FTRL, i.e. , Theorem 1 of Orabona and Pál [33], for any ˜ v ∈ R dT such that ∥ ˜ v ∥ M ≤ 1 the regret of the direction learner is

Applying this with ˜ v = ˜ u ∥ ˜ u ∥ M and combining with the previous two displays leads to the bound stated in the proof of Theorem 2.

B.2 The Haar Matrices and their Properties

In this section we provide some useful supporting lemmas related to the Haar matrices H n . We first introduce the Haar basis vectors , which make up the columns of the matrix H n . Throughout this section we assume for simplicity that T is a power of 2.

Definition 1. For any τ ∈ { 2 i : i = 1 : log 2 ( T ) } and i ∈ [ T/τ ] , the Haar basis vector at timescale τ and location i is the vector in R T with entries

The Haar basis vectors are often arranged into the columns of a matrix as follows:

where h 0 = (1 , 1 , . . . , 1) ⊤ ∈ R T . This matrix is referred to as the (unnormalized) Haar basis matrix of order n = log 2 ( T ) . It is well-known that H n has the following equivalent recursive form [38, 11, 37]:

So, for instance, we have

and so on. For our purposes, we will primaly work in terms of the matrices H n rather than the basis vectors h ( τ ) i . The main utility of defining the basis vectors h ( τ ) i is that their definition easily implies the following useful result, which states that the Haar basis vectors are sparsely supported w.r.t time.

̸

Proposition 5. Let n = log 2 T and let H n ∈ R T × T be the unnormalized Haar basis matrix of order n . Then for any t ∈ [ T ] , there at most 1 + log T indices i for which [ H n ] t,i = 0 .

The proof follows immediately from Definition 1 ( i.e. , any t can fall into only one of the intervals covered at each of the log 2 ( T ) time-scales) and accounting for the additional column h 0 of all 1 's.

In what follows, we will also use the following well-known relationship between the vec operator and the Kronecker product (see, e.g. , Steeb and Shi [38, Chapter 2.11]).

Proposition 6. Let A , B , and C be matrices of appropriate dimensions such that the product ABC exists. Then, vec ( ABC ) = ( C ⊤ ⊗ A ) vec ( B ) .

The following three lemmas will be used to prove the guarantees of the algorithm characterized in Section 4.1 (Propositions 3 and 4).

Lemma 5. Let n = log 2 ( T ) , v = ( v 1 , . . . , v T ) ⊤ ∈ R T , and let H n be the unnormalized Haar basis matrix of order n . Then

Proof. From Equation (7), we have that

Moreover, leveraging Proposition 6 we have

Lemma 6. Let H n be the unnormalized Haar basis matrix of order n . Then, H n H ⊤ n satisfies

where 1 2 n -1 and 0 2 n -1 are 2 n -1 × 2 n -1 matrices of 1's and 0's respectively.

where

Proof. For brevity, let us denote B n = H n H ⊤ n . The first equality follows from elementary properties of block matrices and the Kronecker product: using the recursive form of H n , we have

To get the second expression, let us proceed by induction. We have B 0 = (1) and

Next, let us assume that B n satisfies

Then, applying the recursive form Equation (8) for B n +1 , we have

where the last line observes that 1 2 n -1 ⊗ ( 1 1 1 1 ) = 1 2 n and that after adding the two block matrices, the top left and bottom right blocks are both

via Equation (8). Hence, the stated result follows by induction.

Now using this, we have the following bound on the norm of the high-dimensional surrogate losses. Lemma 7. Let n = log 2 ( T ) , e t be the t th standard basis vector of R T , and for g t ∈ R d let ˜ g t = e t ⊗ g t ∈ R dT . Let H n be a Haar matrix of order n and let B = H n ⊗ I d be it's block extension to sequence in R d . Then, we have

Proof. Using Lemma 1, we have that

Moreover, using Equation (9) it can easily be seen that the diagonal entries of H n H ⊤ n are log 2 T +1 , so we have

B.3 Proof of Proposition 3

Proposition 3. Let n = log 2 T and H n be the unnormalized Haar basis matrix of order n . For any τ ∈ { 2 i : i = 0 , . . . , log 2 T } , let N τ = T/τ and let I ( τ ) 1 , . . . , I ( τ ) N τ be a partition of [ T ] into intervals of length τ . Define the average comparator in interval I ( τ ) i to be ¯ u ( τ ) i = 1 τ ∑ t ∈I ( τ ) i u t , and define the squared path-length at time-scale τ < T to be

and ¯ P ( ⃗ u , T ) = ∥ ∥ ∥ ¯ u ( T ) 1 ∥ ∥ ∥ 2 2 = ∥ ¯ u ∥ 2 2 . Then, setting S = [ H n H ⊤ n ] -1 and M = S ⊗ I d , we have

Proof. The proof of the claim ∥ ˜ g t ∥ 2 M -1 = ∥ ˜ g t ∥ 2 HH ⊤ = ∥ g t ∥ 2 2 [log 2 ( T ) + 1] is provided in Lemma 7.

To see the form of ∥ ˜ u ∥ 2 M , let us first write

The result then follows by showing that

so that

where for brevity we have dropped the argument ⃗ u on ¯ P ( ⃗ u , τ ) .

Equation (10) is best shown via example; the general case is mostly a tedius exercise which we provide at the end. Assume T = 4 , then the Haar matrix of order n = log 2 ( T ) = 2 is

It is well-known that for any T the columns of H log 2 ( T ) form an orthogonal basis of R T [41, Chapter 6.1.1], which implies that ˜ H 2 is orthonormal. So, ˜ H -1 2 = ˜ H ⊤ 2 and

which leads to Equation (10) after applying the Kronecker product:

More generally, start with d = 1 begin again by factoring

where now ˜ H n is the normalized Haar basis matrix of order n = log 2 ( T ) and

The result is then attained by unrolling the recursion for H ⊤ n ˜ u given by Lemma 5 and factoring in the normalization factors D -2 n . The result for d > 1 is then immediately implied by observing that the block matrix H -1 n ⊗ I d will act upon the vector components of ˜ u ∈ R dT in an identical way to how H -1 n acts upon a vector of scalars.

B.4 Proof of Proposition 4

Proposition 4. The algorithm characterized by applying Theorem 2 with S = [ H n H ⊤ n ] -1 can be implemented with O ( d log T ) per-round computation.

Proof. Note that the losses passed to the 1-dimensional parameter-free algorithm are ⟨ ˜ v t , ˜ g t ⟩ = ⟨ ˜ v t , e t ⊗ g t ⟩ , and since e t ⊗ g t has only d active indices we can compute the 1-dimensional learner's losses in O ( d ) . As such, the 1-dimensional learner can be implemented in O ( d ) per-round computation.

For the direction learner, we are to show that each of the relevant variables can be maintained using only O ( d log T ) per-round computation.

Using Proposition 3, we immediately have V t +1 = V t + ∥ ˜ g t ∥ 2 M -1 = V t +(log T +1) ∥ g t ∥ 2 , so V t +1 can be maintained using only O ( d ) per-round computation ( i.e. , to compute ∥ g t ∥ 2 ).

For the scaling factor ∥ ∥ ∥ ˜ θ t +1 ∥ ∥ ∥ M -1 , observe that

Hence, we again have O ( d ) per-round computation to compute ∥ ˜ g t ∥ 2 M -1 , and letting h t = H ⊤ n e t we can decompose the last term as

From Proposition 5, for any t the vector h t = H ⊤ n e t has only log T +1 active non-zero elements by construction of the Haar basis, so given Λ t , the product Λ t h t takes a linear combination of log T +1 vectors in R d , for O ( d log T ) operations. Note that the variable Λ t can also be maintained with O ( d log T ) operations since each term is g t h ⊤ t , which involves updating O (log T ) columns of Λ t -1 ∈ R d × T . Hence overall we can maintain ∥ ∥ ∥ ˜ θ t +1 ∥ ∥ ∥ M -1 using O ( d log T ) per-round computation.

Lastly, consider the variable ˜ θ t +1 . Observe that we can maintain a variable ̂ θ t +1 = -( H ⊤ n ⊗ I d ) ∑ t s =1 ˜ g s using O ( d log T ) computation:

since h t ⊗ g t is a block vector containing log ( T ) + 1 non-zeros blocks of length d . Hence,

and again via the construction of the Haar basis, each row of H n ( i.e. , each column of H ⊤ n ) has only log T +1 non-zero entries, we can compute each d × 1 block of ˜ θ t +1 using O ( d log T ) computation. Finally, observe that in order to implement the direction learner, we need only compute the t th d × 1 block of ˜ θ t . Indeed, since for each t , the vector ˜ g t = e t ⊗ g t has only d non-zero indices, it suffices to retrieve the corresponding indices of ˜ v t to implement direction learner.

We note that the memory overhead of maintaining each of these variables can also likely be reduced by more careful bookkeeping, and acknowledging the fact that the algorithm only really needs to retrieve the t th block of ˜ w t , since the losses are ˜ g t = e t ⊗ g t . We omit these considerations here for brevity.

C Proofs for Section 5 (Recovering Variance-Variability Coupling Guarantees)

C.1 Proof of Theorem 3

Theorem 3. Let Wealth T := -∑ T t =1 ⟨ ˜ g t , ˜ w t ⟩ denote the 'wealth' of an algorithm A and let ( f, f ∗ ) be a Fenchel conjugate pair. Then A guarantees Wealth T ≥ f ∗ T ( -∑ T t =1 ˜ g t ) for any sequence ˜ g 1 , . . . , ˜ g T if and only if R T ( ⃗ u ) ≤ f T ( ˜ u ) for any sequence ⃗ u = ( u 1 , . . . , u T ) in W , where ˜ u = ( u ⊤ 1 , . . . , u ⊤ T ) ⊤ is the concatenation of the sequence ⃗ u into a vector.

Proof. Thanks to Proposition 1, the proof is essentially the same as the usual one. We provide the argument here for completeness.

From Proposition 1, R T ( ⃗ u ) = R Seq T ( ˜ u ) = ∑ T t =1 ⟨ ˜ g t , ˜ w t -˜ u ⟩ for ˜ g t = e t ⊗ g t and ˜ u = ∑ T t =1 e t ⊗ u t . Hence, recalling the definition of the Fenchel conjugate, we have

Similarly, for the other direction, suppose we have R T ( ⃗ u ) = R Seq T ( ˜ u ) ≤ f T ( ˜ u ) for any ˜ u . Then re-arranging, we have Wealth T ≥ 〈 -∑ T t =1 ˜ g t , ˜ u 〉 -f T ( ˜ u ) , and since this holds for any ˜ u , we can choose the one that tightens the bound to get Wealth T ≥ sup ˜ u 〈 -∑ T t =1 ˜ g t , ˜ u 〉 -f T ( ˜ u ) = f ∗ T ( -∑ T t =1 ˜ g t ) .

D Supporting Lemmas

Lemma 8. Let Σ ∈ R T × T be the finite-difference operator, having entries

The inverse of Σ the upper-triangular matrix of 1 's:

The eigenvalues of Σ and Σ -1 are λ i = 1 for all i ∈ [ T ] .

x ↦→ x ⊤ Σ x is positive definite.

Moreover, the analogous properties hold for the block matrix Σ ⊗ I d ∈ R dT × dT .

Proof. The inverse of Σ is the upper-triangular matrix ∆ characterized by entries

To see why, observe that we have Σ T,T ∆ T,T = 1 and for i < T we have

Then,

and likewise for [ ∆Σ ] ij . Hence Σ∆ = ∆Σ = I and M -1 = ∆ .

Next, since Σ and Σ -1 are upper-triangular, their eigenvalues are equal to their diagonal entries, and hence both have eigenvalues λ i = 1 for all i .

To see that the asymmetric matrix Σ is positive definite, it suffices to show that the symmetric part of Σ , i.e. , the matrix Σ S = ( Σ + Σ ⊤ ) / 2 , is positive definite [23]. Luckily, Σ S is also a well-known variation of the discrete difference operator and is known to be positive definite [see, e.g. , Theorem 7.4.7 in 39].

For the block matrix B = Σ ⊗ I d , the inverse is given immediately by the inverse property of the Kronecker product: B -1 = ( Σ ⊗ I d ) -1 = Σ -1 ⊗ I d . We also have that B = Σ ⊗ I d and B -1 have eigenvalues λ i = 1 for all i ∈ [ dT ] , since both are again upper-triangular with 1 's on their main diagonal. Finally, we have positive definiteness of B using the fact that the symmetric part of B = Σ ⊗ I d is 1 2 ( B + B ⊤ ) = 1 2 ( Σ ⊗ I d + Σ ⊤ ⊗ I d ) = 1 2 ( Σ + Σ ⊤ ) ⊗ I d by the distributive property, hence B is the Kronecker product of two symmetric positive definite matrices, so B is positive definite [38, Chapter 2].

We borrow the following eigenvalue bound from [43].

Theorem 5. (Wolkowicz and Styan [43, Theorem 2.1]) Let A be a symmetric n × n matrix with eigenvalues λ 1 ( A ) ≤ . . . ≤ λ n ( A ) . Then

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our abstract and introduction clear state the main claims of our paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We clearly state the problem setting and the assumptions therein. We clearly state which results are optimal up to logarithmic factors. We discuss the computational complexity of our newly proposed algorithm at the end of Section 4.1. Attaining the tightest matching logarithmic dependencies in our lower bound is still an open problem, as mentioned in the conclusion.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All the theorems provide the full set of assumptions. All of our main results are proven explicitly either in the main text or in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: This is a theoretical paper that studies theoretical guarantees for online learning algorithms.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: This is a theoretical paper that studies theoretical guarantees for online learning algorithms.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: This is a theoretical paper that studies theoretical guarantees for online learning algorithms.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: This is a theoretical paper that studies theoretical guarantees for online learning algorithms.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: This is a theoretical paper that studies theoretical guarantees for online learning algorithms.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Given the theoretical nature of this work, there are no ethical concerns to be addressed.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a theoretical paper that studies theoretical guarantees for online learning algorithms. As stated in the guidelines, this is foundational research and it is not tied to particular applications, let alone deployments.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No models nor data is associated to this paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: No assets were used in this paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper demonstrates that dynamic regret minimization can be reduced to static regret minimization by embedding the comparator sequence into a higher-dimensional space, allowing the use of static regret algorithms for dynamic settings. It establishes a trade-off between the penalties associated with loss variance and comparator sequence variability, proving that achieving regret bounds purely based on the squared path-length without additional penalties is infeasible. Additionally, the paper introduces a new measure of variability based on the locally-smoothed squared path-length, providing a more practical approach to managing dynamic regret.\n\nSTRENGTHS:\n- The paper establishes a novel equivalence between static and dynamic regret minimization. This theoretical contribution provides a unified framework for analyzing and designing algorithms for dynamic regret minimization, leveraging the well-established techniques for static regret.\n\n- By establishing the trade-off between penalties due to the variance of the losses and the variability of the comparator sequence, the paper provides deep insights into the inherent limitations and trade-offs in dynamic regret minimization. This understanding is crucial for developing more effective algorithms and sets a foundation for future research in the area.\n\n- The introduction of the locally-smoothed squared path-length reduces variance penalties, balances adaptability by smoothing out noise, and facilitates the decoupling of regret terms, leading to more practical and robust dynamic regret bounds.\n\nWEAKNESSES:\n- My major concern is the main titile could be somewhat overclaimed, as the proposed reduciton is restricted in OLO (or reduce OCO to OLO). Though I admit that analysis in non-convex problems (in bandit feedback) could be extremely hard, dynamic regret and static regret is still discussed in these problems. So may be ''DR is equal to SR in OCO'' may be more suitable.\n\n- The approach involves embedding the comparator sequence into a higher-dimensional space, which could significantly increase computational complexity. Related concerns should be discussed, particularly in terms of implementing the proposed methods in large-scale or real-time applications. \n\n- The paper does not provide intuitive examples or case studies to illustrate how to obtain dynamic regret in practical problems. I believe a simple example of existing problems (for example, non-stationary OCO) allows us to compare dynamic regret obtained by this reduction with existing studies, making it easier to understand.\n\n- I'm not fully convinced by purely theoretical results. May be adding some simple evaluations to demonstrate the dynamic regret is indeed same as static regret by the proposed reduction can provide empirical evidences.\n\n- The reasons for choosing this locally-smoothed square path-length is not clear. If I get it right, path-lengthes sharing similar local-smoothness would all fullfill the requirements. Meanwhile, this locally-smoothed square path-length is too detailed such that it may loose empirical intuitions or physical meanings to some extent.\n\nQUESTIONS:\n1. Does the high-dimensional embedding reduction also hold for adpative regret (to static regret)?\n2. Can we directly design a unified algorithm (best-of-both-worlds) without knowing the types of regret based on this reduction?\n\nLIMITATIONS:\n\\", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-10"}

comments: SUMMARY:
The paper addresses the problem of minimizing dynamic regret. The main goal is to provide a unified perspective on the problems of minimizing both dynamic and static regret. The first key contribution, presented in Proposition 1, is an interesting and straightforward observation demonstrating a general reduction from dynamic to static regret, showing the equivalence between these two notions.

In Theorem 1, the paper establishes a lower bound, indicating that squared path-length incurs a linear penalty, making adaptation to squared path-length impossible.

Furthermore, using a parameter-free algorithm in Algorithm 2, the paper achieves an upper bound that matches the previously derived lower bound for any weighted norm 
$\|\cdot\|_{M}$. 

In Section 4.1, the authors discuss the suitable choice for $M$. By setting $M=(HH^{\top}) ^{−1}$, where 
$H$ is the unnormalized Haar basis matrix, they derive in Proposition 3 an upper bound for the regret that fully decouples comparators and gradient terms.

STRENGTHS:
The message of the paper is clear, and the theoretical results are solid, consisting of a collection of small steps and observations that lead to important and surprising conclusions. The paper is extremely well-written and a pleasure to read.

WEAKNESSES:
I do not see any particular weaknesses in the paper.

Minor point: There are typos in lines 164 to 167. $V$ should be $V_T$ and $\epsilon$ should be $\epsilon_T$.

QUESTIONS:
I would like to know which measures of variability the authors expect to obtain with different choices of $M$ that yield reasonable and significant regret bounds.

LIMITATIONS:
The authors addressed the limitations of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
This paper studied dynamic regret minimisation in convex optimisation. First, they proved some interesting equivalence between dynamic regret and static regret on extended decision space. Based on this observation, they showed a lower bound which implying that the hypothesis that optimal dynamic regret scales with square path length is impossible. The authors then provided an algorithm that incur regret scales with new notion of path length, which match the novel lower bound.

STRENGTHS:
The main contribution of this paper is demonstrating the equivalence between dynamic regret and static regret in extended space, which is quite interesting. This allows the authors to present a new lower bound and prove a new upper bound for dynamic regret. Although I am not very familiar with the literature on dynamic regret, I believe the result is significant if it is correct, especially in comparison to previous results stated in this paper.

WEAKNESSES:
I found the writing of this paper quite difficult to follow, especially in the main sections (3 and 4). The comparison between the lower bound and upper bound could be better explained.

My question: The lower bound is $\tilde \Omega(\max_M G\sqrt{||\tilde{u}||_M \mathrm{Tr}(M) })$, and the upper bound essentially is $\tilde O(\min_M G\sqrt{||\tilde{u}||_M \mathrm{Tr}(M) })$. However, without strong duality on regret, there still seems to be a gap between the lower bound and upper bound. Correct me if I misunderstood, but I cannot find the reason why the lower bound and upper bound are tight as claimed in the paper.

QUESTIONS:
See weaknesses

LIMITATIONS:
Authors did discuss the limitation and potential future works.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a reduction from the dynamic regret minimization for Online Convex Optimization (OCO) to a static regret minimization problem over an extended domain. Using this reduction, the authors establish a lower bound that highlights the trade-off between the variation of the comparator and the variance of the gradient norm. This general lower bound indicates that a squared path-length bound is impossible. Furthermore, the authors provide a general upper bound for dynamic regret based on techniques from parameter-free online learning. By specifying $M$, the paper demonstrates that a new type of squared dynamic regret bound is achievable.\n\nSTRENGTHS:\n- This paper provides a simple yet effective reduction from the dynamic regret minimization problem to the static regret minimization problem. Subsequently, the techniques developed for comparator-adaptive methods can be applied to the dynamic regret minimization problem.\n- The paper offers a new analysis of the lower bound for dynamic regret minimization, revealing the intrinsic trade-off between the variability of the comparator sequence and the gradient norm.\n- A new type of squared dynamic regret bound is introduced in this paper, based on this novel perspective.\n\nWEAKNESSES:\n- One of my main concerns about the paper is that the results appear to be somewhat overstated. The abstract claims that \"we show that dynamic regret minimization is equivalent to static regret minimization in an extended decision space.\" This claim is somewhat misleading to me, as the proposed reduction only applies to the online linear optimization problem (or OCO with linearized surrogate loss). It is unclear if the proposed reduction holds for a broader range of dynamic regret minimization problems, such as those involving exp-concave or strongly convex loss functions. If not, it would be more appropriate to explicitly mention this limitation in the title and abstract and to provide a more detailed discussion on these limitations in the main paper.\n\n- The high-level idea of this paper is quite similar to that of Zhang et al., 2023. Both approaches convert the dynamic regret minimization problem into a static regret minimization problem in another domain, using comparator-adaptive online learning methods to minimize extended static regret. While the authors discuss this in the related work section, it might be beneficial to explicitly state that previous work has already somewhat demonstrated equivalence between the dynamic regret minimization and static regret minimization problems.\n\n-  As mentioned earlier, a similar reduction was presented in Section 2.2 of Zhang et al., 2023. It would be helpful if the authors provided a more detailed comparison of the two reductions. In the related work section, the paper states, \"We take a similar but slightly more general approach.\" A more detailed explanation of how Proposition 1 generalizes previous work would be beneficial. Specifically, it would be useful to clarify whether the proposition can be derived by selecting a specific dictionary in Zhang et al., 2023. A more detailed discussion in Section 2 of this paper would be advantageous.\n\nOverall, I find the paper interesting, but I am concerned about the overstated contributions and the insufficient comparison with previous work. I would be happy to raise my score if these issues are properly addressed.\n\nReference: Zhiyu Zhang, Ashok Cutkosky, and Ioannis Ch. Paschalidis. Unconstrained dynamic regret via sparse coding, 2023.\n\n===post-rebuttal===\nMy primary concern was the significance of the proposed framework compared with Zhang et al. (2023) and the overstatement regarding the generality of the framework. After several discussions with the authors, I am convinced that the paper offers a more flexible and general framework for handling the dynamic regret minimization problem beyond the work of Zhang et al. (2023). To reflect this, I have updated my score to 6. Nevertheless, I still encourage the authors to clarify the limitations and provide a more detailed discussion on the contributions of the proposed framework in the revision.\n\nQUESTIONS:\nPlease refer to the weakness part.\n\nLIMITATIONS:\nThis paper is theoretical in nature, and I do not identify any potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe main contribution of the paper is showing that, by `lifting' an online linear optimization (OLO) problem to a higher dimensional space, the dynamic regret in the original setting is equal to the $\\textit{static}$ regret achieved in the higher dimensional setting. While simple, this clean reduction allows easier and more elegant reasoning about how to develop and analyze dynamic regret algorithms since static regret algorithms and analyses are so well studied and understood. \n\nThey then employ this reduction to prove the existence of a trade-off between comparator variability penalty in terms of a $M$ matrix norm and $G^2 Tr(M^{-1})$, where $G$ is the Lipschitz parameter of the losses. By choosing $M$ in such a way that the comparator variability term becomes the squared path-length, they are able to show that adapting a dynamic regret rate to the squared path-length implies a supra-linear (i.e., vacuous) guarantee.\n\nThey complement the above general trade-off by providing an algorithm that is able to match the lower bounds along the frontier implied by different specifications of $M$. They instantiate this with a particular matrix setting and attain a corresponding positive result which says it is possible to instead adapt to a modified definition of squared path-length (more specifically, a locally smoothed counterpart).\n\nSTRENGTHS:\nWhile simple, their reduction is quite elegant and insightful. This conceptual contribution is also nicely complemented by the authors employing it to achieve a new lower-bound (that can be matched via existing methods). The presentation is clear and intuitive.\n\nWEAKNESSES:\nSome weaknesses of the paper are that the focus on matrix norms in Section 3 may be a bit limited in scope and that the statement of Theorem 1 is quite convoluted. More broadly, having additional implications of the equivalence would be desirable (e.g., having a more thorough and completed version of Section 5 and/or a more comprehensive list of observations in a similar vein).\n\nQUESTIONS:\n- Do you believe there are any implications of your reduction that are particularly useful from a practical/deployment perspective, or do you see the contribution as being a strictly conceptual one?\n- Do you have any additional intuition for the locally-smoothed path-length quantity? For example, how smoothed would one generally expect this quantity to be?\n\nLIMITATIONS:\nThe authors have addressed limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the problem of dynamic regret minimization in unbounded domains. The authors propose a novel lossless reduction from dynamic regret minimization to static regret minimization by treating the changing comparators in dynamic regret as a fixed one in another decision space with higher dimensions. The key in the reduced static regret minimization is to choose an appropriate matrix norm which is used to transform the static regret bound in lifted domains into a meaningful dynamic regret in the original decision space. Denoting by $M$ the norm-related matrix, this work proposes a novel lower bound regarding the variability penalty $\\\\|\\tilde{u}\\\\|_M$ and a variance penalty $G^2 \\text{Tr} (M^{-1})$. Consequently, the authors proved that a dynamic regret depending on the squared path length is impossible. By contrast, by choosing a specific matrix norm defined by Haar basis matrices, it suffices to achieve a dynamic regret bound with the squared path length defined on some averaged comparators.\n\nSTRENGTHS:\nThe dynamic-to-static regret is pretty impressive. Although it is a little conceptual and less practical, it provides us with a novel way for dynamic regret minimization. \n\nThe lower bound in Theorem 1 is also meaningful by depicting the influences of the variability penalty $\\\\|\\tilde{u}\\\\|_M$ and the variance penalty $G^2 \\text{Tr} (M^{-1})$, providing guidance for consequent algorithm design.\n\nWEAKNESSES:\nI have some major questions for the results regarding the squared path length and I hope that the authors can answer the questions to solve my concerns.\n\n1. I believe that the lower bound in Theorem 1 is correct, which lower-bounds by the dynamic regret with $\\\\|\\tilde{u}\\\\|_M$ and $G^2 \\text{Tr} (M^{-1})$. However, I am doubtful about Proposition 2 when translating this lower bound to a dynamic regret measured by the squared path length. Proposition 2 only shows that choosing a specific $M$ does not suffice to achieve a valid squared-path-length dynamic regret. However, it does not rule out the possibility of other choices of $M$. Or, is there some one-to-one mapping between the choice of $M$ and the squared path length? In my point of view, discussions of this issue are missing and I hope that the authors could revise the paper according to it in the revised version.\n2. As for upper bounds, the obtained bound (in Section 4) is less favorable than I originally expected. The squared path length is defined on some averaged comparators, which are averaged on the intervals induced by the Harr OLR algorithm. The relationship between the current averaged squared length and the desired squared length is not discussed. On the top of Page 8, the authors show that their result can recover the previous dynamic regret defined with $P_T$ (non-squared path length). However, on Page 2, the authors stated that $P_T$ based dynamic regret is not favorable enough with an overly pessimistic diameter quantity. As a result, implying $P_T$ based dynamic regret is not convincing enough, at least for me, to validate the significance of the obtained result. \n3. The obtained regret also suffers from additional $\\log T$ terms, which is not favorable enough in the full information setup. Do previous works with similar setups also suffer from the same issue?\n4. The dynamic-to-static reduction is novel. However, when designing algorithms, the computational complexity is not discussed at all, which is pretty important because the dimension is related to time horizon $T$ in the reduced problem. I suggest that the authors could add discussions about computational complexity in the revised version.\n\nOverall, my main concern is that despite the novelty of the dynamic-to-static reduction, to what extent this reduction is significant is not fully discussed. I believe that this reduction is of interest to the community. However, the currently obtained results do not suffice to validate its significance.\n\nQUESTIONS:\nA minor type maybe: In Line 24, a missing inverse operator in the matrix $S$? (i.e., $[H_n H_n^\\top]^{-1}$ but not $H_n H_n^\\top$?)\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and, if applicable, potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}