SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[AABR09] Jacob Abernethy, Alekh Agarwal, Peter L Bartlett, and Alexander Rakhlin. A stochastic view of optimal regret through minimax duality. In Conference on Learning Theory, 2009.
[ABRT08] Jacob Abernethy, Peter L Bartlett, Alexander Rakhlin, and Ambuj Tewari. Optimal strategies and minimax lower bounds for online convex games. In Conference on Learning Theory, pages 415-423, 2008.
[AHMS13] Oren Anava, Elad Hazan, Shie Mannor, and Ohad Shamir. Online learning for time series prediction. In Conference on learning theory, pages 172-184. PMLR, 2013.
[AHZ15] Oren Anava, Elad Hazan, and Assaf Zeevi. Online time series prediction with missing data. In International conference on machine learning, pages 2191-2199. PMLR, 2015.
[AM16] Oren Anava and Shie Mannor. Heteroscedastic sequences: beyond gaussianity. In International Conference on Machine Learning, pages 755-763. PMLR, 2016.
[AW01] Katy S Azoury and Manfred K Warmuth. Relative loss bounds for on-line density estimation with the exponential family of distributions. Machine Learning, 43(3):211-246, 2001.
[BGZ15] Omar Besbes, Yonatan Gur, and Assaf Zeevi. Non-stationary stochastic optimization. Operations research, 63(5):1227-1244, 2015.
[BW19] Dheeraj Baby and Yu-Xiang Wang. Online forecasting of total-variation-bounded sequences. Advances in Neural Information Processing Systems, 32, 2019.
[BW20] Dheeraj Baby and Yu-Xiang Wang. Adaptive online estimation of piecewise polynomial trends. Advances in Neural Information Processing Systems, 33:20462-20472, 2020.
[BW21] Dheeraj Baby and Yu-Xiang Wang. Optimal dynamic regret in exp-concave online learning. In Conference on Learning Theory, pages 359–409. PMLR, 2021.
[BW22] Dheeraj Baby and Yu-Xiang Wang. Optimal dynamic regret in proper online learning with strongly convex losses and beyond. In International Conference on Artificial Intelligence and Statistics, pages 1805–1845. PMLR, 2022.
[BZW21] Dheeraj Baby, Xuandong Zhao, and Yu-Xiang Wang. An optimal reduction of TV-denoising to adaptive online learning. In International Conference on Artificial Intelligence and Statistics, pages 2899-2907. PMLR, 2021.
[CDV93] Albert Cohen, Ingrid Daubechies, and Pierre Vial. Wavelets on the interval and fast wavelet transforms. Applied and computational harmonic analysis, 1993.
[CLW21] Liyu Chen, Haipeng Luo, and Chen-Yu Wei. Impossible tuning made possible: A new expert algorithm and its applications. In Conference on Learning Theory, pages 1216-1259. PMLR, 2021.
[CO18] Ashok Cutkosky and Francesco Orabona. Black-box reductions for parameter-free online learning in banach spaces. In Conference On Learning Theory, pages 1493-1529. PMLR, 2018.
[CRT06] Emmanuel J Candès, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory, 52(2):489-509, 2006.
[Cut19] Ashok Cutkosky. Combining online learning guarantees. In Conference on Learning Theory, pages 895-913. PMLR, 2019.
[Cut20] Ashok Cutkosky. Parameter-free, dynamic, and strongly-adaptive online learning. In International Conference on Machine Learning, pages 2250-2259. PMLR, 2020.
[ZCP22] Zhiyu Zhang, Ashok Cutkosky, and Ioannis Paschalidis. PDE-based optimal strategy for unconstrained online learning. In International Conference on Machine Learning, pages 26085-26115. PMLR, 2022.
[Zin03] Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning, pages 928-936, 2003.
[ZLZ18] Lijun Zhang, Shiyin Lu, and Zhi-Hua Zhou. Adaptive online learning in dynamic environments. Advances in neural information processing systems, 31, 2018.
$\left[\mathrm{ZYY}^{+}17\right]$  Lijun Zhang, Tianbao Yang, Jinfeng Yi, Rong Jin, and Zhi-Hua Zhou. Improved dynamic regret for non-degenerate functions. Advances in Neural Information Processing Systems, 30, 2017.
$\left[\mathrm{XYZ}^{+}18\right]$  Lijun Zhang, Tianbao Yang, Zhi-Hua Zhou, et al. Dynamic regret of strongly adaptive methods. In International conference on machine learning, pages 5882-5891. PMLR, 2018.
[ZZZZ20] Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. Dynamic regret of convex and smooth functions. Advances in Neural Information Processing Systems, 33:12510-12520, 2020.
[ZZZZ21] Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. Adaptivity and nonstationarity: Problem-dependent dynamic regret for online convex optimization. arXiv preprint arXiv:2112.14368, 2021.
Organization Appendix A summarizes a list of comparator statistics involved in our theoretical analysis. Appendix B surveys additional related works. Appendix C, D and E respectively present details of our general sparse coding framework, its special version with wavelet dictionaries, and applications in time series forecasting. Finally, Appendix F presents additional discussion of our technical results.
A List of comparator statistics
A major task in comparator adaptive online learning is finding suitable statistics to quantify the regularity of a comparator sequence. Several such statistics are defined throughout this paper, which are summarized in Table 2. Note that for the definition of sparsity on the last line, we assume the dictionary  $\mathcal{H}$  is orthogonal, and the sequence  $u_{1:T}$  is contained in its span. Then,  $z^{(n)}$  is defined as the projection of  $u_{1:T}$  onto a feature vector  $h_{1:T,n}$ , i.e.,
$$
z ^ {(n)} = \left\langle h _ {1: T, n}, u _ {1: T} \right\rangle \frac {h _ {1 : T , n}}{\| h _ {1 : T , n} \| _ {2} ^ {2}}.
$$
This is well defined due to our assumption  $\| h_{1:T,n}\| _2\geq 1$  from Section 2.1.
Name	Notation	Definition
Maximum range	M	maxt||ut||
Comparator average	u	1/T∑t=1Tut
Path length	P	∑t=1T-1||ut+1-ut||2
Norm sum	S	∑t=1T||ut||2
First order variability	S̅	∑t=1T||ut-ubar||2
Energy	E	∑t=1T||ut||2
Second order variability	E̅	∑t=1T||ut-ubar||2
Number of switches	K	∑t=1T-11[ut+1≠ut]
Sparsity on a dictionary H	SparsityH	(∑n=1N||z(n)||2/n∑n=1N||z(n)||2
Table 2: A list of comparator statistics.
Next, let us discuss their relations, in order to interpret our quantitative contribution more clearly (Table 1). It is clear that  $S \leq MT$ , therefore the fine baseline Eq.(4) from [JC22] improves the coarse one Eq.(3); comparing their associated switching regret bounds follows the same reasoning.
To compare our results to the baselines, observe that
$$
\| \bar {u} \| _ {2} \sqrt {T} = \left\| \frac {1}{\sqrt {T}} \sum_ {t = 1} ^ {T} u _ {t} \right\| _ {2} \leq \sum_ {t = 1} ^ {T} \frac {1}{\sqrt {T}} \| u _ {t} \| _ {2} \leq \sqrt {E} \leq \sqrt {M S},
$$
$$
\bar {S} = \sum_ {t = 1} ^ {T} \| u _ {t} - \bar {u} \| _ {2} \leq \sum_ {t = 1} ^ {T} \| u _ {t} \| _ {2} + T \| \bar {u} \| _ {2} = \sum_ {t = 1} ^ {T} \| u _ {t} \| _ {2} + \left\| \sum_ {t = 1} ^ {T} u _ {t} \right\| _ {2} \leq 2 S,
$$
$$
\bar {E} = \sum_ {t = 1} ^ {T} \| u _ {t} - \bar {u} \| _ {2} ^ {2} \leq 2 M \bar {S} \leq 4 M S.
$$
$$
\left\| \bar {u} \right\| _ {2} \sqrt {T} + \sqrt {P \bar {S}} \leq \sqrt {M S} + 2 \sqrt {2 P S} = O \left(\sqrt {(M + P) S}\right),
$$
$$
\left\| \bar {u} \right\| _ {2} \sqrt {T} + \sqrt {K E} \leq \sqrt {M S} + 2 \sqrt {K M S} = O \left(\sqrt {(1 + K) M S}\right).
$$
That is, both our path-length-dependent bound and the switching regret bound are at least as good as the results from [JC22]. Concrete benefits are demonstrated in Example 1 and 2.
Finally, as a sanity check, our path-length-dependent bound does not violate the lower bound  $\Omega(P)$ : even when  $\bar{u} = 0$ ,
$$
P = \sum_ {t = 1} ^ {T - 1} \| u _ {t + 1} - u _ {t} \| _ {2} \leq \sum_ {t = 1} ^ {T - 1} \| u _ {t + 1} - \bar {u} \| _ {2} + \sum_ {t = 1} ^ {T - 1} \| u _ {t} - \bar {u} \| _ {2} \leq 2 \bar {S},
$$
therefore our bound is never better than  $\tilde{O}(P)$ .
B More on related work
Online regression Our sparse coding framework converts unconstrained dynamic OCO to a special form of online regression. The standard setting of the latter [RS14a] considers a repeated game as well: in each round, we observe a covariate  $x_{t} \in \mathbb{R}^{d}$ , make a prediction  $\hat{y}_{t} \in \mathbb{R}$  (which depends on  $x_{t}$ ), and then observe a label  $y_{t} \in \mathbb{R}$ . The performance metric is the minimax regret under the square loss
$$
\operatorname {R e g r e t} _ {T} (\mathcal {F}) = \sum_ {t = 1} ^ {T} \left(\hat {y} _ {t} - y _ {t}\right) ^ {2} - \inf _ {f \in \mathcal {F}} \sum_ {t = 1} ^ {T} \left(f (x _ {t}) - y _ {t}\right) ^ {2}.
$$
Roughly, the problem is of a nonparametric type if the complexity of the function class  $\mathcal{F}$  is not fixed a priori, but grows with  $T$  (i.e., the amount of data).
Overall, such an online regression problem is highly general, as static OCO is recovered if  $x_{t}$  is time-invariant. The setting we utilize is a variant with (i) vector output; (ii) general convex losses; (iii)  $x_{t}$  specified by the dictionary, possibly being sparse itself (e.g., wavelets); and (iv) the function class  $\mathcal{F}$  being linear, but unbounded. As discussed in Footnote 11, our setting deviates from the conventional definition of regression, as a general convex loss function does not necessarily have minimizers. We adopt the terminology of "regression" for streamlined exposition.
Existing works on online nonparametric regression [RS14a, GG15] have established the relation of this problem to certain path length characterizations of dynamic regret. However, the generality of this setting makes the analysis challenging, and especially, algorithms can be computationally expensive. With a bounded domain assumption (on predictions  $\hat{y}_t$ ), a recent breakthrough [BW21] simultaneously achieved several notions of optimality for path-length-dependent bounds, with efficient computation. Readers are referred to [BW21, Appendix A] for a thorough discussion of this line of works.
For the special case of Online Linear Regression (OLR) with square losses, the celebrated VAW forecaster [AW01, Vov01] guarantees  $O(N \log T)$  regret against any unbounded coefficient vector  $\hat{u} \in \mathbb{R}^N$ , where  $N$  is the dimension of the feature space. Such a fast rate becomes vacuous in the nonparametric regime (when  $N > T$ ) [GY14], therefore [Ger13] proposed a sparsity regret bound  $\tilde{O}(\|\hat{u}\|_0)$  and an accompanying inefficient algorithm as its high dimensional generalization. Efficient computation was addressed by [GW18], but the obtained result only applies to bounded  $\hat{u}$ . In a rough sense, such sparsity regret bounds are the square loss and feature-based analogue of the  $L_1$ -norm parameter-free bounds in OLO [Ora19, Chapter 9]. They are also closely related to sparsity oracle inequalities in statistics, as reviewed by [Ger13].
Parametric time series models For time series forecasting, most prior works are devoted to parametric strategies with strong inductive bias, such as the ARMA model, state space models, and more recent deep learning models. Online learning has been applied to such models as well [AHMS13, AHZ15, AM16, KM16,  $\mathrm{HLS}^{+}18$ ], leading to forecasting guarantees under mild statistical assumptions. When convexity is present, some of these problems could be reframed as special cases of our OLR problem, with a constant-size dictionary that does not grow with  $T$ ; for example, learning the autoregressive model corresponds to defining the features as the fixed-length observation history. Also, Appendix E shows that given a parametric time series forecaster (possibly without performance guarantees), our algorithm can be applied on top of it, in order to provably correct its nonstationary bias.
Other sparsity topics in OL Finally, we review other sparsity-related topics in online learning, which do not fit into the scope of this paper. [LLZ09, Xia09, DSSST10, SST11] considered using
online learning to solve batch  $L_{1}$  regularized problems. The goal is to achieve sparse predictions instead of sparsity adaptive regret bounds. [Kal14, FKK16, KKLP17] studied online sparse regression, where only a subset of features are available in each round. The challenge is to handle bandit feedback in OLR.
C Detail on the general framework
This section presents details on our general sparse coding framework. Appendix C.1 introduces the static subroutine we adopt from [MK20]. Appendix C.2 proves our main results, but with additional gradient adaptivity compared to the main paper.
C.1 Unconstrained static subroutine
The following static OCO algorithm and its guarantee are due to [MK20, Section 3.1]. We assume that  $\| \hat{g}_t\| _2\leq \hat{G}$ , and  $\hat{G} >0$
Algorithm 3 FREEGRAD [MK20, Definition 4]: scale-free and gradient adaptive unconstrained static OLO.
Require: A hyperparameter  $\varepsilon > 0$ ; dimension  $d$ ; Lipschitz constant  $\hat{G}$ .
1: Initialize a gradient sum counter  $s = 0 \in \mathbb{R}^d$  and a variance counter  $v = \hat{G}^2$ .
2: for  $t = 1, 2, \ldots$  do
3: Predict
$$
\hat {x} _ {t} = - \varepsilon s \cdot \frac {(2 v + \hat {G} \| s \| _ {2}) \hat {G} ^ {2}}{2 (v + \hat {G} \| s \| _ {2}) ^ {2} \sqrt {v}} \cdot \exp \left(\frac {\| s \| _ {2} ^ {2}}{2 v + 2 \hat {G} \| s \| _ {2}}\right).
$$
4: Observe the loss gradient  $\hat{g}_t$ .
5: Update  $s \gets s + \hat{g}_t$ , and  $v \gets v + \hat{g}_t^2$ .
6: end for
Lemma C.1 (Theorem 20 of [MK20]). With any hyperparameter  $\varepsilon > 0$ , for all  $T \in \mathbb{N}_+$  and  $\hat{u} \in \mathbb{R}$ , Algorithm 3 guarantees
$$
\sum_ {t = 1} ^ {T} \langle \hat {g} _ {t}, \hat {x} _ {t} - \hat {u} \rangle \leq \varepsilon \hat {G} + \left[ 2 \| \hat {u} \| _ {2} \sqrt {V _ {T} \log_ {+} \left(\frac {2 \| \hat {u} \| _ {2} V _ {T}}{\varepsilon \hat {G} ^ {2}}\right)} \right] \vee \left[ 4 \| \hat {u} \| _ {2} \hat {G} \log \left(\frac {4 \| \hat {u} \| _ {2} \sqrt {V _ {T}}}{\varepsilon \hat {G}}\right) \right],
$$
$$
V _ {T} = \hat {G} ^ {2} + \sum_ {t = 1} ^ {T} \| \hat {g} _ {t} \| _ {2} ^ {2}.
$$
C.2 Proof of the main result
We now present the analysis of our general sparse coding framework. The following lemma is a slightly more general version of Lemma 2.1 in the main paper, which characterizes the performance of our single direction learner (Algorithm 1). Recall that  $g_{t} \in \partial l_{t}(x_{t})$  from the OCO-OLO reduction.
Lemma C.2 (Lemma 2.1, full). Let  $\varepsilon >0$  be an arbitrary hyperparameter for Algorithm 3. Applying its 1D version as the static subroutine, for all  $T\in \mathbb{N}_{+}$  and  $u_{1:T}\in \operatorname {span}(h_{1:T})$ , against any adversary  $\mathcal{E}$ , Algorithm 1 guarantees
$$
\sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) - \sum_ {t = 1} ^ {T} l _ {t} \left(u _ {t}\right) \leq \varepsilon G + \left(G \frac {\left\| u _ {1 : T} \right\| _ {2}}{\left\| h _ {1 : T} \right\| _ {2}} + \sqrt {\sum_ {t = 1} ^ {T} \left\langle g _ {t} , u _ {t} \right\rangle^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(\max  _ {t} \left\| u _ {t} \right\| _ {2}, T, \varepsilon^ {- 1}\right).
$$
The simplified form (Lemma 2.1) is recovered by using  $\| h_{1:T}\| _2\geq 1$  and  $\| g_t\| _2\leq G$
Proof of Lemma 2.1. Subsuming poly-logarithmic factors, the static regret bound of our static subroutine (Algorithm 3) can be written as
$$
\sum_ {t = 1} ^ {T} \hat {g} _ {t} (\hat {x} _ {t} - \hat {u}) \leq \varepsilon \hat {G} + | \hat {u} | \left(\hat {G} + \sqrt {\sum_ {t = 1} ^ {T} \hat {g} _ {t} ^ {2}}\right) \cdot \mathrm {p o l y l o g} \left(| \hat {u} |, T, \varepsilon^ {- 1}\right),
$$
where  $\hat{u}$  is any 1D static comparator that the subroutine handles.
Now, for any single-directional comparator  $u_{1:T} \in \operatorname{span}(h_{1:T})$  considered in this lemma, there exists  $\hat{u} \in \mathbb{R}$  such that  $u_{1:T} = \hat{u} h_{1:T}$ . The dynamic regret can be rewritten as
$$
\sum_ {t = 1} ^ {T} l _ {t} (x _ {t}) - \sum_ {t = 1} ^ {T} l _ {t} (u _ {t}) \leq \sum_ {t = 1} ^ {T} \langle g _ {t}, x _ {t} - u _ {t} \rangle = \sum_ {t = 1} ^ {T} \langle g _ {t}, h _ {t} \hat {x} _ {t} - h _ {t} \hat {u} \rangle = \sum_ {t = 1} ^ {T} \hat {g} _ {t} (\hat {x} _ {t} - \hat {u}),
$$
and the RHS can be bounded using the static regret bound above. Note that  $|\hat{g}_t| = |\langle g_t, h_t \rangle| \leq G$ , therefore the surrogate Lipschitz constant  $\hat{G}$  from the static regret bound can be assigned to  $G$ .
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) - \sum_ {t = 1} ^ {T} l _ {t} \left(u _ {t}\right) \leq \varepsilon G + | \hat {u} | \left(G + \sqrt {\sum_ {t = 1} ^ {T} \left\langle g _ {t} , h _ {t} \right\rangle^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(| \hat {u} |, T, \varepsilon^ {- 1}\right) \\ = \varepsilon G + \left(G \frac {\| u _ {1 : T} \| _ {2}}{\| h _ {1 : T} \| _ {2}} + \sqrt {\sum_ {t = 1} ^ {T} \langle g _ {t} , u _ {t} \rangle^ {2}}\right) \cdot \text {p o l y l o g} \left(\frac {\| u _ {1 : T} \| _ {2}}{\| h _ {1 : T} \| _ {2}}, T, \varepsilon^ {- 1}\right) \\ \leq \varepsilon G + \left(G \frac {\| u _ {1 : T} \| _ {2}}{\| h _ {1 : T} \| _ {2}} + \sqrt {\sum_ {t = 1} ^ {T} \langle g _ {t} , u _ {t} \rangle^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(\max  _ {t} \| u _ {t} \| _ {2}, T, \varepsilon^ {- 1}\right), \\ \end{array}
$$
where the last line is due to our assumption that  $\| h_{1:T}\| _2\geq 1$
Next, we prove the unconstrained dynamic regret bound with general dictionaries (Theorem 1).
Theorem 4 (Theorem 1, full). Consider any collection of signals  $z^{(n)} \in \operatorname{span}(h_{1:T,n})$ ,  $\forall n$ . We define its reconstruction error (for the comparator  $u_{1:T}$ ) as  $z^{(0)} = u_{1:T} - \sum_{n=1}^{N} z^{(n)} \in \mathbb{R}^{dT}$ . Then, for all  $T \in \mathbb{N}_+$  and  $u_{1:T} \in \mathbb{R}^{dT}$ , against any adversary  $\mathcal{E}$ , Algorithm 2 guarantees
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) - \sum_ {t = 1} ^ {T} l _ {t} \left(u _ {t}\right) \leq \varepsilon G - \sum_ {t = 1} ^ {T} \left\langle g _ {t}, z _ {t} ^ {(0)} \right\rangle \\ + \left(G \sum_ {n = 1} ^ {N} \frac {\left\| z ^ {(n)} \right\| _ {2}}{\| h _ {1 : T , n} \| _ {2}} + \sum_ {n = 1} ^ {N} \sqrt {\sum_ {t = 1} ^ {T} \left\langle g _ {t} , z _ {t} ^ {(n)} \right\rangle^ {2}}\right) \cdot \mathrm {p o l y l o g} \left(\max _ {t, n} \left\| z _ {t} ^ {(n)} \right\| _ {2}, T, N, \varepsilon^ {- 1}\right), \\ \end{array}
$$
Proof of Theorem 4. The idea of this theorem is a dynamic analogue of [Cut19] to aggregate the regret bound of single direction learners. For all decomposition  $u_{1:T} = \sum_{n=0}^{N} z^{(n)}$  such that  $z^{(n)} \in \operatorname{span}(h_{1:T,n})$  for all  $n \in [1:T]$ , we have
$$
\sum_ {t = 1} ^ {T} l _ {t} (x _ {t}) - \sum_ {t = 1} ^ {T} l _ {t} (u _ {t}) \leq \left\langle g _ {1: T}, x _ {1: T} - u _ {1: T} \right\rangle = \left\langle - g _ {1: T}, z ^ {(0)} \right\rangle + \sum_ {n = 1} ^ {N} \left\langle g _ {1: T}, w _ {1: T, n} - z ^ {(n)} \right\rangle .
$$
For the first term on the RHS,  $\left\langle -g_{1:T},z^{(0)}\right\rangle = -\sum_{t = 1}^{T}\left\langle g_t,z_t^{(n)}\right\rangle$  . As for the rest, we plug in Lemma C.2, with hyperparameter  $\varepsilon /N$
$$
\begin{array}{l} \sum_ {n = 1} ^ {N} \left\langle g _ {1: T}, w _ {1: T, n} - z ^ {(n)} \right\rangle \\ \leq \sum_ {n = 1} ^ {N} \left\{\varepsilon N ^ {- 1} G + \left(G \frac {\left\| z ^ {(n)} \right\| _ {2}}{\left\| h _ {1 : T , n} \right\| _ {2}} + \sqrt {\sum_ {t = 1} ^ {T} \left\langle g _ {t} , z _ {t} ^ {(n)} \right\rangle^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(\max  _ {t} \left\| z _ {t} ^ {(n)} \right\| _ {2}, T, N, \varepsilon^ {- 1}\right) \right\} \\ \leq \varepsilon G + \left(G \sum_ {n = 1} ^ {N} \frac {\left\| z ^ {(n)} \right\| _ {2}}{\left\| h _ {1 : T , n} \right\| _ {2}} + \sum_ {n = 1} ^ {N} \sqrt {\sum_ {t = 1} ^ {T} \left\langle g _ {t} , z _ {t} ^ {(n)} \right\rangle^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(\max  _ {t, n} \left\| z _ {t} ^ {(n)} \right\| _ {2}, T, N, \varepsilon^ {- 1}\right). \\ \end{array}
$$
Next, we show how this dynamic regret bound recovers the static regret bound in  $\mathbb{R}^d$ . As discussed in Section 2.2, the static setting amounts to picking  $N = d$  and  $\mathcal{H}_t = I_d$ , and the decomposed signals  $z^{(n)}$  are determined by orthogonal projection of the static comparator sequence  $u_{1:T} = [u,\dots ,u]$ .
Specifically,  $z_{t}^{(n)}$  is a  $d$ -dimensional vector which is zero except the  $n$ -th entry; its  $n$ -th entry equals the  $n$ -th entry of the static comparator  $u$ . If we index the gradient as  $g_{t} = [g_{t,1},\dots ,g_{t,d}] \in \mathbb{R}^{d}$  and the static comparator as  $u = [u_{1},\ldots ,u_{d}] \in \mathbb{R}^{d}$ , then  $\left\langle g_t,z_t^{(n)}\right\rangle = g_{t,n}u_n$ . Applying Theorem 4, against static  $u_{1:T}$ ,
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) - \sum_ {t = 1} ^ {T} l _ {t} \left(u _ {t}\right) \\ \leq \varepsilon G + \left(G \sum_ {n = 1} ^ {N} \frac {\left\| z ^ {(n)} \right\| _ {2}}{\left\| h _ {1 : T , n} \right\| _ {2}} + \sum_ {n = 1} ^ {N} \sqrt {\sum_ {t = 1} ^ {T} \left\langle g _ {t} , z _ {t} ^ {(n)} \right\rangle^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(\max  _ {t, n} \left\| z _ {t} ^ {(n)} \right\| _ {2}, T, N, \varepsilon^ {- 1}\right) \\ \leq \varepsilon G + \left(G \sum_ {i = 1} ^ {d} \frac {\left| u _ {i} \right| \sqrt {T}}{\sqrt {T}} + \sum_ {i = 1} ^ {d} \left| u _ {i} \right| \sqrt {\sum_ {t = 1} ^ {T} g _ {t , i} ^ {2}}\right) \cdot \operatorname {p o l y l o g} \left(\left\| u \right\| _ {\infty}, T, N, \varepsilon^ {- 1}\right) \\ \leq \varepsilon G + \left(G \| u \| _ {1} + \| u \| _ {2} \sqrt {\sum_ {t = 1} ^ {T} \| g _ {t} \| _ {2} ^ {2}}\right) \cdot \text {p o l y l o g} \left(\| u \| _ {\infty}, T, N, \varepsilon^ {- 1}\right). \quad \text {(C a u c h y - S c h w a r z)} \\ \end{array}
$$
In the asymptotic regime with large  $T$ ,  $\mathrm{Regret}_T(u_{1:T}) = \tilde{O}(\|u\|_2 \sqrt{T})$ .
D Detail on the wavelet algorithm
This section presents details of our wavelet algorithm. The pseudocode is presented in Appendix D.1. Appendix D.2 introduces the wavelet-specific notations for our analysis. Appendix D.3 presents a generic sparsity based bound for our algorithm. Appendix D.4 and D.5 prove our main results. Auxiliary lemmas are contained in Appendix D.6. Finally, Appendix D.7 works out the details of the two examples from the main paper.
D.1 Pseudocode
For all  $m \in \mathbb{N}_+$ , let  $T = 2^m$ , and let  $\mathrm{Haar}_m$  be the  $T \times T$  Haar dictionary matrix defined in Section 3.1, for  $d = 1$ . We apply the following variant (Algorithm 4) of our sparse coding framework, in order to remove all  $d$  dependence from the final regret bound. It adopts the  $d$  dimensional version of the static subroutine (FREEGRAD), instead of the 1D version in Section 2. The pseudocode mirrors the combination of Algorithm 1 and 2.
It is equivalent to view Algorithm 4 as operating on a  $dT \times dT$  "master" dictionary matrix  $\mathcal{H}$ , defined block-wise as the following: for all  $(i,j) \in [1:T]^2$ , the  $(i,j)$ -th block of  $\mathcal{H}$  is the product of the