SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\sum_ {m = 1} ^ {m ^ {*}} \sum_ {t = 2 ^ {m} - 1} ^ {2 ^ {m + 1} - 2} \| u _ {t} - \bar {u} _ {m} \| _ {2} \leq \sum_ {m = 1} ^ {m ^ {*}} \sum_ {t = 2 ^ {m} - 1} ^ {2 ^ {m + 1} - 2} \left(\| u _ {t} - \bar {u} \| _ {2} + \| \bar {u} _ {m} - \bar {u} \| _ {2}\right) = \bar {S} + \sum_ {m = 1} ^ {m ^ {*}} 2 ^ {m} \| \bar {u} _ {m} - \bar {u} \| _ {2}.
$$
The last sum on the RHS is the first order variability of a locally averaged version of  $u_{1:T}$ . Due to Lemma D.7,
$$
\sum_ {m = 1} ^ {m ^ {*}} 2 ^ {m} \left\| \bar {u} _ {m} - \bar {u} \right\| _ {2} \leq \bar {S}.
$$
Combining everything above,
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\| \bar {u} \| _ {2} \sqrt {T} + \sqrt {\bar {E}} + \sqrt {P \bar {S}}\right).
$$
It remains to show that  $\sqrt{\overline{E}} \leq \sqrt{P\overline{S}}$ , thus the former can be absorbed into the latter. Plugging in the definitions, this is equivalent to showing
$$
\sum_ {t = 1} ^ {T} \| u _ {t} - \bar {u} \| _ {2} ^ {2} \leq \sum_ {t = 1} ^ {T} P \| u _ {t} - \bar {u} \| _ {2},
$$
and it suffices to prove  $\| u_t - \bar{u} \| \leq P$  for all  $t \in [1:T]$ . This is completed in Lemma D.8. Till this point, we have shown the desirable result in the situation of "exact dyadic partitioning".
To complete the proof, we turn to the general situation where  $T$  cannot be partitioned into dyadic blocks. This follows from a similar "padding" construction from the proof of Theorem 2. Let  $T^{*} = 2^{\lceil \log_{2}T\rceil}$ , and by definition,  $T^{*}\leq 2T$ . Let us consider a hypothetical length  $T^{*}$  game with the rounds  $t > T$  constructed as follows: the loss gradient  $g_{t} = 0\in \mathbb{R}^{d}$ , and  $u_{t} = \bar{u}$ . Then, the regret of any algorithm on the length  $T^{*}$  hypothetical game equals its regret on the actual length  $T$  game, and the regret bound for the former applies to the latter as well: if we write  $P^{*}$  and  $\bar{S}^{*}$  as the statistics of the extended length  $T^{*}$  comparator, then
$$
\operatorname {R e g r e t} _ {T} (u _ {1: T}) \leq \tilde {O} \left(\| \bar {u} \| _ {2} \sqrt {T ^ {*}} + \sqrt {P ^ {*} \bar {S} ^ {*}}\right).
$$
Clearly,  $\bar{S}^{*} = \bar{S}$  and  $T^{*}\leq 2T$ . As for the path length,  $P^{*} = P + \| u_{T} - \bar{u}\|_{2}$ , and due to Lemma D.8,  $\| u_T - \bar{u}\| _2\leq P$ . Plugging it back completes the proof.
D.6 Useful lemma
Our analysis uses two auxiliary lemmas. First, we show that local averaging makes a signal "more regular". Consider any signal  $u_{1:T} \in \mathbb{R}^{dT}$ , with the  $t$ -th round component  $u_t \in \mathbb{R}^d$ . Local averaging refers to replacing any  $k$  consecutive components of  $u_{1:T}$  by their average, i.e., setting
$$
u _ {\tau + 1}, \dots , u _ {\tau + k} = k ^ {- 1} \sum_ {i = 1} ^ {k} u _ {\tau + i},
$$
for some  $\tau \in [0:T - k]$
Lemma D.7. Let a signal  $w_{1:T} \in \mathbb{R}^{dT}$  be the result of  $u_{1:T}$  after local averaging, and  $\bar{w} = T^{-1}\sum_{t=1}^{T} w_t \in \mathbb{R}^d$ . Then, the path length, the norm sum and the energy of  $w_{1:T}$ , including their centered versions, are all dominated by those of  $u_{1:T}$ . That is,
1.  $\sum_{t=1}^{T-1}\|w_{t+1}-w_t\|_2 \leq \sum_{t=1}^{T-1}\|u_{t+1}-u_t\|_2;$
2.  $\begin{array}{r}\sum_{t = 1}^{T}\| w_t - \bar{w}\| _2\leq \sum_{t = 1}^{T}\| u_t - \bar{u}\| _2; \end{array}$
3.  $\begin{array}{r}\sum_{t = 1}^{T}\| w_{t} - \bar{w}\|_{2}^{2}\leq \sum_{t = 1}^{T}\| u_{t} - \bar{u}\|_{2}^{2}. \end{array}$
4.  $\begin{array}{r}\sum_{t = 1}^{T}\| w_{t}\|_{2}\leq \sum_{t = 1}^{T}\| u_{t}\|_{2}, \end{array}$  and  $\begin{array}{r}\sum_{t = 1}^{T}\| w_{t}\|_{2}^{2}\leq \sum_{t = 1}^{T}\| u_{t}\|_{2}^{2}. \end{array}$
Proof of Lemma D.7. Starting from the first part of the lemma, we prove for the general case of  $0 < \tau < T - k$ . The boundary cases ( $\tau = 0$  and  $\tau = T - k$ ) are analogous.
Local averaging only affects the path length caused by the averaged entries  $u_{\tau + 1}, \ldots, u_{\tau + k}$ , and the two entries  $u_{\tau}$  and  $u_{\tau + k + 1}$  right besides averaging boundary; this original path length quantity in
$u_{1:T}$  is  $\sum_{i=0}^{k} \| u_{\tau+i+1} - u_{\tau+i} \|_2$ . After averaging, the path length among these entries becomes
$$
\begin{array}{l} \left\| u _ {\tau} - k ^ {- 1} \sum_ {i = 1} ^ {k} u _ {\tau + i} \right\| _ {2} + \left\| k ^ {- 1} \sum_ {i = 1} ^ {k} u _ {\tau + i} - u _ {\tau + k + 1} \right\| _ {2} \\ = k ^ {- 1} \left\| \sum_ {i = 1} ^ {k} \left(u _ {\tau} - u _ {\tau + i}\right) \right\| _ {2} + k ^ {- 1} \left\| \sum_ {i = 1} ^ {k} \left(u _ {\tau + i} - u _ {\tau + k + 1}\right) \right\| _ {2} \\ \leq k ^ {- 1} \sum_ {i = 1} ^ {k} \left(\left\| u _ {\tau} - u _ {\tau + i} \right\| _ {2} + \left\| u _ {\tau + i} - u _ {\tau + k + 1} \right\| _ {2}\right) \\ \leq k ^ {- 1} \sum_ {i = 1} ^ {k} \left(\sum_ {j = 0} ^ {k} \| u _ {\tau + j + 1} - u _ {\tau + j} \| _ {2}\right) \\ = \sum_ {i = 0} ^ {k} \| u _ {\tau + i + 1} - u _ {\tau + i} \| _ {2}. \\ \end{array}
$$
Now consider the second part of the lemma. After local averaging,  $\bar{w} = \bar{u}$ . The affected part of the signal contributes to the following first order variability
$$
\sum_ {t = 1} ^ {k} \| w _ {\tau + i} - \bar {w} \| _ {2} = k \left\| k ^ {- 1} \sum_ {i = 1} ^ {k} u _ {\tau + i} - \bar {u} \right\| _ {2} = \left\| \sum_ {i = 1} ^ {k} u _ {\tau + i} - k \bar {u} \right\| _ {2} \leq \sum_ {t = 1} ^ {k} \| u _ {\tau + i} - \bar {u} \| _ {2}.
$$
As for the third part of the lemma,
$$
\sum_ {t = 1} ^ {k} \| w _ {\tau + i} - \bar {w} \| _ {2} ^ {2} = k \left\| k ^ {- 1} \sum_ {i = 1} ^ {k} u _ {\tau + i} - \bar {u} \right\| _ {2} ^ {2} \leq k ^ {- 1} \left(\sum_ {i = 1} ^ {k} \| u _ {\tau + i} - \bar {u} \| _ {2}\right) ^ {2} \leq \sum_ {t = 1} ^ {k} \| u _ {\tau + i} - \bar {u} \| _ {2} ^ {2},
$$
where the last inequality is due to AM-QM inequality.
The final part of the proof is the uncentered version of Part 2 and 3, which follows the same steps. In fact, any fixed reference point (for the variability) works, i.e., for all  $v \in \mathbb{R}^d$ ,
$$
\sum_ {t = 1} ^ {T} \left\| w _ {t} - v \right\| _ {2} \leq \sum_ {t = 1} ^ {T} \left\| u _ {t} - v \right\| _ {2},
$$
$$
\sum_ {t = 1} ^ {T} \left\| w _ {t} - v \right\| _ {2} ^ {2} \leq \sum_ {t = 1} ^ {T} \left\| u _ {t} - v \right\| _ {2} ^ {2}.
$$
We also use another simple lemma.
Lemma D.8. Consider any comparator sequence  $u_{1:T}$ . For all  $t$ , we have  $\| u_t - \bar{u} \|_2 \leq P$ .
Proof of Lemma D.8. Starting from the definition,
$$
\left\| u _ {t} - \bar {u} \right\| _ {2} = \left\| u _ {t} - \sum_ {i = 1} ^ {T} T ^ {- 1} u _ {i} \right\| _ {2} \leq T ^ {- 1} \sum_ {i = 1} ^ {T} \left\| u _ {t} - u _ {i} \right\| _ {2},
$$
and for all  $i, t \in [1:T]$ ,  $\| u_t - u_i\|_2 \leq P$  due to triangle inequality.
D.7 Quantitative example
This subsection presents details of our two quantitative examples, Example 1 and 2.
Tracking outliers We first calculate the relevant statistics of the comparator  $u_{1:T}$ . Note that we assume  $k \leq \sqrt{T}$ , and in this way, there is only a small amount of  $u_t$  with large magnitude, which can then be called outliers.
$$
\bar {u} = \frac {1}{T} \left(k \sqrt {T} + T - k\right),
$$
$\left|\bar{u}\right| = \Theta (1),M = \sqrt{T},K = 1$  or 2,  $P = \Theta (\sqrt{T}),\bar{E}\leq E = kT + T - k = \Theta (kT).$  As for  $S$  and
$$
S = k \sqrt {T} + T - k = \Theta (T),
$$
$$
\begin{array}{l} \bar {S} = k (\sqrt {T} - \bar {u}) + (T - k) (\bar {u} - 1) \\ = 2 k T ^ {- 1} (T - k) (\sqrt {T} - 1) \\ \leq O (k \sqrt {T}). \\ \end{array}
$$
Intuitively, we have  $|\bar{u}| = \Theta(1)$  while  $M = \sqrt{T}$ ;  $\bar{S} = O(k\sqrt{T})$  while  $S = \Theta(T)$ . This explains the improvements detailed next. For each algorithm considered in Table 1, we evaluate both its switching regret bound and its path-length-dependent bound.
- The minimax algorithm ADER [ZLZ18] is not applicable, as  $M$  grows with  $T$  and can be larger than any fixed diameter  $D$ .
- The  $P$ -dependent bound of the coarse baseline [JC22, Algorithm 6], c.f., Eq.(3), is
$$
\tilde {O} \left(\sqrt {(M + P) M T}\right) = \tilde {O} (T).
$$
With  $P = O(KM)$ , the resulting  $K$ -dependent bound is
$$
\tilde {O} \left(M \sqrt {(1 + K) T}\right) = \tilde {O} (T).
$$
- The  $P$ -dependent bound of the fine baseline [JC22, Algorithm 2], c.f., Eq.(4), is
$$
\tilde {O} \left(\sqrt {(M + P) S}\right) = \tilde {O} \left(T ^ {3 / 4}\right).
$$
$$
\tilde {O} \left(\sqrt {(1 + K) M S}\right) = \tilde {O} (T ^ {3 / 4}).
$$
- Our path length bound is
$$
\tilde {O} \left(\left| \bar {u} \right| \sqrt {T} + \sqrt {P \bar {S}}\right) = \tilde {O} \left(\sqrt {k T}\right).
$$
Same for our switching regret bound,
$$
\tilde {O} \left(\left| \bar {u} \right| \sqrt {T} + \sqrt {K \bar {E}}\right) = \tilde {O} \left(\sqrt {k T}\right).
$$
Persistent oscillation Again, we calculate the statistics of the comparator  $u_{1:T}$ .  $\bar{u} = 1$ ,  $M \leq 2$ ,  $K = k$ ,  $P = \Theta (k / \sqrt{T})$ . Crucially,  $\bar{S} = \sqrt{T}$  and  $\bar{E} = \Theta (1)$ , while  $S = \Theta (T)$  and  $E = \Theta (T)$ . Here the  $K$ -dependent bounds of the baselines are loose compared to their corresponding  $P$ -dependent bounds, due to using the relation  $P = O(KM)$ . Therefore we will only evaluate their  $P$ -dependent bounds.
- Suppose one knows that  $M \leq 2$  beforehand, then ADER can be applied with  $D = 2$ . The regret bound is
$$
\tilde {O} \left(\sqrt {(D + P) D T}\right) = \tilde {O} \left(\sqrt {T} + k ^ {1 / 2} T ^ {1 / 4}\right).
$$
- One could check that the  $P$ -dependent bounds of the coarse and the fine baselines are also
$$
\tilde {O} \left(\sqrt {(M + P) M T}\right) = \tilde {O} \left(\sqrt {T} + k ^ {1 / 2} T ^ {1 / 4}\right),
$$
$$
\tilde {O} \left(\sqrt {(M + P) S}\right) = \tilde {O} \left(\sqrt {T} + k ^ {1 / 2} T ^ {1 / 4}\right).
$$
- For our algorithm, the  $P$ -dependent bound is
$$
\tilde {O} \left(\left| \bar {u} \right| \sqrt {T} + \sqrt {P \bar {S}}\right) = \tilde {O} \left(\sqrt {T}\right).
$$
The  $K$  -dependent bound is
$$
\tilde {O} \left(\left| \bar {u} \right| \sqrt {T} + \sqrt {K \bar {E}}\right) = \tilde {O} \left(\sqrt {T}\right).
$$
E Application: Time series forecasting
This section presents an application of our framework in time series forecasting. $^{15}$  Roughly speaking, we aim to address the following question:
Given a black box forecaster, can we make it provably robust against (structured) nonstationarity?
Along the way, our objective is to show that
- Simultaneously handling unconstrained domains and dynamic comparators in online learning brings downstream benefits in time series forecasting.
- Our sparse coding framework can enhance empirically developed forecasting strategies.
Setting Let us consider the following forecasting problem, which resembles the online learning game introduced at the beginning of this paper. The difference is that, here, we further assume access to a black box forecaster  $\mathcal{A}$ . In each (the  $t$ -th) round,
1. The black box forecaster  $\mathcal{A}$  produces a prediction  $a_{t} \in \mathbb{R}^{d}$  based on the observed history  $(z_{1:t-1}$  and  $l_{1:t-1})$ .
2. After observing  $a_{t}$ , we make a prediction  $x_{t} \in \mathbb{R}^{d}$ .
3. The environment reveals a true value  $z_{t} \in \mathbb{R}^{d}$  and a convex loss function  $l_{t}: \mathbb{R}^{d} \to \mathbb{R}$ .  $l_{t}$  is  $G$ -Lipschitz with respect to  $\| \cdot \|_2$ , and  $z_{t}$  is one of its minimizer satisfying  $l_{t}(z_{t}) = 0$ .
Our goal is to achieve low total loss  $\sum_{t=1}^{T} l_t(x_t)$ . Since trivially picking  $x_t = a_t$  already achieves a total loss of  $\sum_{t=1}^{T} l_t(a_t)$ , our goal is to improve it in certain situations, by designing a more sophisticated prediction rule based on  $a_t$ .
Intuition In the above setting,  $\mathcal{A}$  can be any algorithm that predicts  $z_{1:T}$  in a reasonable, but non-robust manner. Taking the weather forecasting for example, there are a few notable cases.
-  $\mathcal{A}$  is a simulator of the governing meteorological equations, which uses the online observations  $z_{1:t - 1}$  as boundary conditions.
-  $\mathcal{A}$  is an autoregressive model, which predicts a linear combination of the past observations. The coefficients are determined by statistical modeling.
-  $\mathcal{A}$  is a large deep learning model trained on offline datasets (e.g., the weather history at geographically similar locations).
Even though such forecasters typically lack performance guarantees, their predictions can be used to construct time-varying Bayesian priors (see our discussion in the Introduction): given  $a_{t}$ , we will apply a fine-tuning adjustment  $\delta_{t}$  to predict  $x_{t} = a_{t} + \delta_{t}$ . Intuitively, the total loss is low if  $a_{t}$  is close to the true value  $z_{t}$ , i.e., when the prior is good.
Reduction to unconstrained dynamic regret Concretely, if  $x_{t} = a_{t} + \delta_{t}$ , then due to convexity, for all subgradients  $g_{t}\in \partial l_{t}(x_{t})$  we have  $l_{t}(x_{t}) - l_{t}(z_{t})\leq \langle g_{t},\delta_{t}\rangle -\langle g_{t},z_{t} - a_{t}\rangle$ . The RHS is the instantaneous regret of  $\delta_t$  in an OLO problem with loss gradient  $g_{t}$  and comparator  $z_{t} - a_{t}$ . Applying our unconstrained dynamic OLO algorithm, the total loss in forecasting can be bounded as
$$
\sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) \leq \operatorname {R e g r e t} _ {T} \left(z _ {1: T} - a _ {1: T}\right).
$$
That is, the total loss bound adapts to the complexity of the error sequence  $z_{1:T} - a_{1:T}$  (of the given black box forecaster). This contains  $a_{1:T} = 0$  as a special case, where no side information is assumed.
Let us compare this bound to the baseline  $\sum_{t=1}^{T} l_t(a_t)$ , which corresponds to trivially picking  $x_t = a_t$ .
- If  $z_{1:T} = a_{1:T}$ , i.e., the black box  $\mathcal{A}$  is perfect, then the baseline loss is  $\sum_{t=1}^{T} l_t(a_t) = 0$ . In this case, due to Theorem 4, our general sparse coding framework guarantees  $\sum_{t=1}^{T} l_t(x_t) \leq \varepsilon G$ , where  $\varepsilon > 0$  is an arbitrary hyperparameter. That is, our algorithm is worse than the baseline by at most a constant.
$^{15}$ Code is available at https://github.com/zhiyuzz/NeurIPS2023-Sparse-Coding.
- If  $z_{1:T} \neq a_{1:T}$ , then in general, the baseline loss  $\sum_{t=1}^{T} l_t(a_t)$  is linear in  $T$ . In contrast, our algorithm could guarantee a sublinear  $\mathrm{Regret}_T(z_{1:T} - a_{1:T})$ , thus also a sublinear total loss, when the error sequence  $z_{1:T} - a_{1:T}$  is structurally simple (e.g., sparse under a transform, or low path length) with respect to our prior knowledge.
In summary, the idea is that by sacrificing at most a constant loss when  $\mathcal{A}$  is perfect ( $z_{1:T} = a_{1:T}$ ), we could robustify  $\mathcal{A}$  against certain structured unseen environments, improving the linear total loss to a sublinear rate.
Importance of unconstrained domain The above application critically relies on the ability of our algorithm to handle unconstrained domains. To demonstrate this, suppose we instead use the bounded domain algorithm from [ZLZ18] to pick the fine-tuning adjustment  $\delta_t$ . Then, the above analysis only holds if an upper bound  $D$  of the maximum error  $\max_t\| z_t - a_t\|_2$  is known a priori – this is a stringent requirement in practice. Furthermore, when  $z_{1:T} = a_{1:T}$ , such an alternative approach only guarantees  $\sum_{t=1}^{T}l_t(x_t)\leq\tilde{O}(D\sqrt{T})$ , which is considerably worse than the baseline 0. In other words, the alternative fine-tuning strategy could ruin the black box forecaster  $\mathcal{A}$ , when the latter performs well.
In the rest of this section, we present experiments for this time series application. Appendix E.1 demonstrates the power law phenomenon, which shows that both the time series  $z_{1:T}$  and the error sequence  $z_{1:T} - a_{1:T}$  could exhibit exploitable structures. This implies good performance guarantees using our theoretical framework. Appendix E.2 goes one step further by actually testing the fine-tuning performance of our algorithm.
E.1 Power law phenomenon
This subsection further verifies the power law phenomenon discussed in Section 2.2, with both wavelet and Fourier dictionaries. The goal is to present concrete examples where signal structures can be exploited by our framework, generating more interpretable, sublinear regret bounds.
Wavelet dictionary We first verify the power law on the Haar wavelet dictionary. Intuitively it is suitable when the dynamics of the environment exhibits switching behavior. To this end, consider the following stochastic time series model
$$
z _ {t} = z _ {t - 1} \beta_ {t} + \zeta_ {t}, \tag {17}
$$
where  $\{\beta_t\}$  and  $\{\zeta_t\}$  are iid random variables satisfying  $\zeta_t \sim \mathrm{Uniform}(-q, q)$  and
$$
\beta_ {t} = \left\{ \begin{array}{l l} - 1, & \text {w . p .} \quad p, \\ 1, & \text {w . p .} \quad 1 - p. \end{array} \right.
$$
Picking  $T = 2^{15} = 32768$ ,  $p = 0.0005$  and  $q = 0.005$ , we generate four sample paths of  $z_{1:T}$  using four arbitrary random seeds (2020, 2021, 2022 and 2023), and the obtained time domain signals are plotted in the first row of Figure 2. As the switching probability  $p$  is chosen to be low enough, all the sample paths exhibit a small amount of sharp switches, corrupted by the noise term  $\zeta_t$ . According to our intuition from signal processing, the Haar wavelet transform of these signals is sparse.
Now let us verify this intuition. We take the Haar wavelet transform of these signals, sort the transform domain coefficients and plot the results on log-log scales – these are shown as the solid blue lines in the second row of Figure 2. Using the largest 100 transform domain coefficients on each plot, we fit a liner model using least square, which is shown as the dashed orange line. The slope of each line is  $-\alpha$ , where  $\alpha$  is displayed in the legend. It can be seen that for all four sample paths, the fitted  $\alpha$  is within (0.5, 1), thus justifying the power law phenomenon [Pri21]. Given  $\alpha$ , the regret of our Haar wavelet algorithm is  $\tilde{O}(T^{1.5 - \alpha})$ , as shown in Section 2.2.
As for the implication in time series forecasting, let us consider forecasting  $z_{1:T}$  with  $a_{1:T} = 0$ , i.e., without the external forecaster  $\mathcal{A}$ . Given the power law, the total forecasting loss of our fine-tuning approach is  $\sum_{t=1}^{T} l_t(x_t) \leq \tilde{O}(T^{1.5 - \alpha})$ .
We also remark that although only four sample paths are demonstrated, we observe the power law phenomenon on all random seeds we tried in the experiment.
(a) Seed= 2020.
(b) Seed= 2021.
(c) Seed= 2022.
Figure 2: Verifying the power law on the Haar Wavelet dictionary. First row: time domain signals. Second row: sorted transform domain coefficients on a log-log plot. The dashed orange line is the best linear fit on the log-log plot, using the largest 100 transform domain coefficients. From left to right: four arbitrary random seeds.
(d) Seed= 2023.
Fourier dictionary Next, we verify the power law on the Fourier dictionary. Here we use the Jena weather forecasting dataset, $^{16}$  which records the weather data at a German city, Jena, every 10 minutes. We take the data from Jan 1st, 2010 till July 1st, 2022, consisting of  $T = 656956$  time steps. Two different modalities, namely the temperature and the humidity, are considered. The actual temperature and humidity sequences are plotted in Figure 3.
Figure 3: Time domain behavior of the weather data.
For the sequence of temperature  $z_{1:T}$ , we perform its Discrete Fourier Transform (DFT), which returns  $T$  complex number as the frequency domain coefficients. We discard the second half of the coefficients due to symmetry, since the input of the transform is real. For the remaining coefficients, we take their absolute values, sort them and plot the result on a log-log plot. Similar to the wavelet experiment, we also fit a linear model using the largest 100 transform domain coefficients. These are shown as Figure 4 (Left), which exhibit the power law phenomenon.
Furthermore, we perform the same procedure on the temperature difference sequence  $\{z_{t + 1} - z_t\}$ , where the  $t$ -th entry is the change of temperature from the  $t$ -th time step to the  $t + 1$ -th time step. The result is shown as Figure 4 (Right). Although the tail is heavier, we can still observe similar power-law phenomenon for large transform domain coefficients.
Now, let us discuss again the implication of the observed power law in time series forecasting. First, consider forecasting  $z_{1:T}$  without  $\mathcal{A}$ . Given the power law of  $z_{1:T}$  itself, the Fourier version of our forecaster guarantees sublinear total loss. Next, consider forecasting  $z_{1:T}$  with  $\mathcal{A}$  being the zeroth-order hold forecaster, i.e.,  $a_t = z_{t-1}$ . The power law of the difference sequence  $\{z_{t+1} - z_t\}$  implies good forecasting performance of our framework in this context.
16Available at https://www.bgc-jena.mpg.de/wetter/.
Figure 4: Verifying the power law on the Fourier dictionary. Left: the DFT of the temperature sequence. Right: the DFT of the temperature difference sequence.
Parallel results on the humidity sequence are reported in Figure 5, with a similar qualitative behavior. It illustrates the prevalence of the power law across different types of the data.
Figure 5: Verifying the power law on the Fourier dictionary. Left: the DFT of the humidity sequence. Right: the DFT of the humidity difference sequence.
E.2 Fine-tuning forecaster
Finally, we test the performance of our forecasting framework on the synthetic switching data and the actual temperature sequence. For the first case, our framework is equipped with the Haar wavelet dictionary. The Fourier dictionary is adopted in the second case. In both cases, we compare our algorithm against the baseline from [JC22]. More specifically, we take our online learning algorithm (Algorithm 2) and the algorithm from [JC22], plug them both into the time series forecasting workflow introduced at the beginning of this section, and then compare their total forecasting loss.
Concretely, let us start from the wavelet dictionary.
Wavelet dictionary In this case, consider the setting without the external forecaster  $\mathcal{A}$ . We run both online learning algorithms (our Algorithm 2 and the baseline [JC22, Algorithm 2]), and use their outputs directly as the time series predictions. Our Algorithm 2 is equipped with the Haar wavelet dictionary defined in Section 3.1. The configurations of the time series model are the same as the previous subsection, with  $T = 2^{15} = 32768$ ,  $p = 0.0005$  and  $q = 0.005$ . The loss functions  $l_{t}$  are the absolute loss.
Both algorithms require a confidence hyperparameter  $\varepsilon$ , and we set it to 1. Since the time series data Eq.(17) is random, we run both algorithms on 10 random seeds, and calculate their total loss. Our algorithm achieves a total loss of 44048, which is considerably lower than the baseline's total loss 62465. This is consistent with the theoretical results developed so far.
Fourier dictionary Next, we turn to the task of temperature forecasting. The data is reported in the previous subsection. We take its first  $T = 50000$  entries, and assign it to the true time series  $z_{1:T}$ ; the loss functions  $l_t$  are the absolute loss. The black box forecaster  $\mathcal{A}$  is assigned to the zeroth-order hold forecaster, i.e.,  $a_t = z_{t-1}$ .
For our framework, we need to specify the dictionary. Although using the entire DFT matrix could lead to low regret guarantees (as demonstrated by the power law), this is computationally challenging. Instead, we exploit the fact that the weather is naturally periodic, with the period of one day. Picking the base frequency  $\omega$  accordingly, we define features indexed by  $k$  (the harmonic order) as
$$
h _ {t, 2 k - 1} = \cos (k \omega t),
$$
$$
h _ {t, 2 k} = \sin (k \omega t).
$$