SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 4 Haar OLR with known time horizon.
Require: A time horizon  $T = 2^{m}$ ; the  $T \times T$  Haar dictionary matrix  $\mathrm{Haar}_m$ ; and a hyperparameter  $\varepsilon > 0$  (default is 1).  
1: Let  $N = T$ . For all  $n \in [1:N]$ , initialize a copy of the  $d$  dimensional version of Algorithm 3 (FREEGRAD) as  $\mathcal{A}_n$ , with hyperparameter  $\varepsilon / N$ .  
2: for  $t = 1,2,\ldots, do$   
3: Receive the  $t$ -th row of  $\mathrm{Haar}_m$ , and index it as  $[h_{t,1},\dots,h_{t,N}]$ ; note that  $h_{t,n} \in \mathbb{R}$ .  
4: for  $n = 1,2,\dots,N$  do  
5: If  $h_{t,n} \neq 0$ , query  $\mathcal{A}_n$  for its output, and assign it to  $\hat{x}_{t,n} \in \mathbb{R}^d$ ; otherwise,  $\hat{x}_{t,n}$  is arbitrary.  
6: Define  $w_{t,n} = h_{t,n}\hat{x}_{t,n} \in \mathbb{R}^d$ .  
7: end for  
8: Predict  $x_t = \sum_{n=1}^N w_{t,n} \in \mathbb{R}^d$ , receive loss gradient  $g_t \in \mathbb{R}^d$ .  
9: for  $n = 1,2,\dots,N$  do  
10: If  $h_{t,n} \neq 0$ , compute  $\hat{g}_{t,n} = h_{t,n} g_t$  and send it to  $\mathcal{A}_n$  as its surrogate loss gradient.  
11: end for  
12: end for
Algorithm 5 Anytime Haar OLR (Algorithm 4 with doubling trick).
1: for  $m = 1,2,\ldots ,$  do  
2: Run Algorithm 4 for  $2^{m}$  rounds, which uses the matrix Haar  $m$ . The hyperparameter is set to 1.  
3: end for
$(i,j)$ -th entry of  $\mathrm{Haar}_m$  (which is a scalar) and the  $d$ -dimensional identity matrix  $I_d$ . That is,  $\mathcal{H}$  is a block matrix; each block is a diagonal matrix with equal diagonal entries determined by  $\mathrm{Haar}_m$ . Roughly, the algorithm measures distances in  $\mathbb{R}^d$  by the  $L_2$  norm, while measuring  $\mathbb{R}^T$  by the  $L_1$  norm.
Algorithm 4 alone is not sufficient for our purpose: it must take an integer  $m$  and run for a fixed  $T = 2^m$  rounds. We apply a meta algorithm (Algorithm 5), which simply restarts the known  $T$  algorithm using the classical doubling trick, c.f., [SS11, Section 2.3.1].
D.2 More background
Although the analysis of our framework is simpler than [JC22], a challenge is carefully indexing all the quantities to account for the vectorized setting. It is thus important to introduce a few notations to streamline the presentation. Haar $_m$  is the  $T \times T$  Haar dictionary matrix defined in Section 3.1, with  $T = 2^m$ . Recall the statistics of the comparator sequence, summarized in Appendix A.
Local interval Given any scale-location pair  $(j,l)$ , let the support  $I^{(j,l)}$  be the time interval where the feature  $h^{(j,l)}$  is nonzero. That is,
$$
I ^ {(j, l)} := \left[ 2 ^ {j} (l - 1) + 1: 2 ^ {j} l \right].
$$
Moreover, let  $I_{+}^{(j,l)}$  denote the first half of this interval, and  $I_{-}^{(j,l)}$  for the second half.  $h^{(j,l)}$  is 1 on  $I_{+}^{(j,l)}$ , and -1 on  $I_{-}^{(j,l)}$ .
Normalization Let  $\tilde{\mathrm{Haar}}_m$  be the orthonormal matrix obtained by scaling the columns of  $\mathrm{Haar}_m$ . The normalized feature vectors are also denoted by tilde, i.e., instead of  $h^*$  and  $h^{(j,l)}$ , the normalized features are  $\tilde{h}^*$  and  $\tilde{h}^{(j,l)}$ . They are vectors in  $\mathbb{R}^T$ , with the  $t$ -th component denoted by  $\tilde{h}_t^*$  and  $\tilde{h}_t^{(j,l)}$ , in  $\mathbb{R}$ .
Coordinate sequence Consider any comparator sequence  $u_{1:T} \in \mathbb{R}^{dT}$ . For all coordinate  $i \in [1:d]$ , we define its  $i$ -th coordinate sequence as  $u_{1:T}^{(i)} \in \mathbb{R}^T$ : the  $t$ -th entry of this coordinate sequence  $u_{1:T}^{(i)}$ , denoted by  $u_t^{(i)}$ , is the  $i$ -th coordinate of  $u_t$ .
Transform domain coefficient We will also use the transform domain coefficients of  $u_{1:T}$ , under the Haar wavelet transform. Recall that in the single-feature, generic setting (Section 2.2), we denoted a single transform domain coefficient by  $\hat{u} \in \mathbb{R}$ . With wavelets, the transform domain encodes  $dT$ -dimensional vectors. According to our convention so far, we will denote them by scale-location pairs  $(j,l)$ : given a  $(j,l)$  pair, the "coefficient"  $\hat{u}^{(j,l)}$  is a  $d$ -dimensional vector. There are  $T - 1$  pairs of  $(j,l)$  in total; complementing the representation, we use another  $\hat{u}^* \in \mathbb{R}^d$  to represent the "coefficient" for the all-one feature.
Given any scale parameter  $j \in [1: \log_2 T]$  and location parameter  $l \in [1: 2^{-j}T]$ , let
$$
\hat {u} ^ {(j, l)} := \left[ \left\langle \tilde {h} ^ {(j, l)}, u _ {1: T} ^ {(1)} \right\rangle , \dots , \left\langle \tilde {h} ^ {(j, l)}, u _ {1: T} ^ {(d)} \right\rangle \right],
$$
and for the all-one feature,
$$
\hat {u} ^ {*} := \left[ \left\langle \tilde {h} ^ {*}, u _ {1: T} ^ {(1)} \right\rangle , \dots , \left\langle \tilde {h} ^ {*}, u _ {1: T} ^ {(d)} \right\rangle \right].
$$
That is, each entry is the inner product between the normalized feature and a coordinate sequence from  $u_{1:T}$ .
Due to the orthonormality of the applied transform (specified by the normalized features  $\tilde{h}^*$  and  $\tilde{h}^{(j,l)}$ ), the energy is preserved between the time domain and the transform domain, i.e.,
$$
E = \| u _ {1: T} \| _ {2} ^ {2} = \| \hat {u} ^ {*} \| _ {2} ^ {2} + \sum_ {j, l} \left\| \hat {u} ^ {(j, l)} \right\| _ {2} ^ {2},
$$
and also the second order variability (the energy of the centered dynamic component within  $u_{1:T}$ ),
$$
\bar {E} = \sum_ {t = 1} ^ {T} \| u _ {t} - \bar {u} \| _ {2} ^ {2} = \sum_ {j, l} \left\| \hat {u} ^ {(j, l)} \right\| _ {2} ^ {2}. \tag {11}
$$
Moreover, since  $\bar{h}^*$  equals  $1 / \sqrt{T}$  times the all-one vector,
$$
\left\| \hat {u} ^ {*} \right\| _ {2} ^ {2} = \sum_ {i = 1} ^ {d} \left\langle \tilde {h} ^ {*}, u _ {1: T} ^ {(i)} \right\rangle^ {2} = \sum_ {i = 1} ^ {d} \left(\frac {1}{\sqrt {T}} \sum_ {t} u _ {1: T} ^ {(i)}\right) ^ {2} = T \sum_ {i = 1} ^ {d} \left(\frac {1}{T} \sum_ {t} u _ {1: T} ^ {(i)}\right) ^ {2} = \| \bar {u} \| _ {2} ^ {2} T. \tag {12}
$$
Detail reconstruction Given the transform domain coefficients, we can reconstruct details of the comparator  $u_{1:T}$  on the time domain. Similar to our notation in the generic framework (Section 2.2), we keep the letter  $z$ , but replace the index  $n$  by  $(j,l)$ , which is more suitable for indexing wavelets.
Let  $z^{(j,l)} \in \mathbb{R}^{dT}$  be the detail of  $u_{1:T}$  along the  $(j,l)$ -th feature. It is the concatenation of  $T$  vectors in  $\mathbb{R}^d$ , and for all  $t$ , the  $t$ -th of these vectors is defined by
$$
z _ {t} ^ {(j, l)} := \hat {u} ^ {(j, l)} \tilde {h} _ {t} ^ {(j, l)} \in \mathbb {R} ^ {d}.
$$
Similarly, we can define the detail  $z^{*}$  along the feature  $\tilde{h}^*$ . Its  $t$ -th component is
$$
z _ {t} ^ {*} := \hat {u} ^ {*} \tilde {h} _ {t} ^ {*},
$$
and clearly, the RHS does not depend on  $t$  since  $\tilde{h}^*$  is the normalization of the all-one feature  $h^*$ .
Let us also sum the details across different locations. Given a scale  $j$ , let
$$
z ^ {(j)} := \sum_ {l} z ^ {(j, l)} \in \mathbb {R} ^ {d T}.
$$
Note that the summands are sequences that do not overlap: at each entry, only one of the summand sequence is nonzero. The full reconstruction is obtained by summing all the details,
$$
u _ {1: T} := z ^ {*} + \sum_ {j = 1} ^ {\log_ {2} T} z ^ {(j)}.
$$
Statistics of the detail sequence We can define statistics of the detail sequences just like the statistics of the comparator  $u_{1:T}$ . Specifically, define the first order variability of the  $(j,l)$ -th detail as
$$
\bar {S} ^ {(j, l)} := \sum_ {t = 1} ^ {T} \left\| z _ {t} ^ {(j, l)} \right\| _ {2}.
$$
Note that since the  $z_{t}^{(j,l)}$  sequence is centered (with average being equal to 0), its first order variability equals its norm sum, c.f., Appendix A. Summing over the locations, the first order variability at the  $j$ -th scale is
$$
\bar {S} ^ {(j)} := \sum_ {t = 1} ^ {T} \left\| z _ {t} ^ {(j)} \right\| _ {2},
$$
which equals  $\sum_l\bar{S}^{(j,l)}$
Similarly, we can define the path length of the detail sequences. A caveat is that we only count the path length within the support  $I^{(j,l)}$  of the feature  $h^{(j,l)}$ ,
$$
P ^ {(j, l)} := \sum_ {t = 2 ^ {j} (l - 1) + 1} ^ {2 ^ {j} l - 1} \left\| z _ {t + 1} ^ {(j, l)} - z _ {t} ^ {(j, l)} \right\| _ {2}.
$$
The comparator's movement when the support changes does not count. Summing over the locations,
$$
P ^ {(j)} := \sum_ {l} P ^ {(j, l)}.
$$
D.3 Generic sparsity adaptive bound
With the notation from the previous subsection, we now present a generic sparsity adaptive regret bound for Algorithm 4 (fixed  $T$  Haar OLR). Since the latter is a variant of our main sparse coding framework (Section 2), the result can be analogously derived, although the notations need to be treated carefully.
Lemma D.1. For any  $m$ ,  $T = 2^m$  and  $u_{1:T} \in \mathbb{R}^{dT}$ , with any hyperparameter  $\varepsilon > 0$ , Algorithm 4 guarantees
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \varepsilon G + G \left(\left\| z ^ {*} \right\| _ {2} + \sum_ {j = 1} ^ {\log_ {2} T} \sum_ {l = 1} ^ {2 ^ {- j} T} \left\| z ^ {(j, l)} \right\| _ {2}\right) \cdot \operatorname {p o l y l o g} \left(M, T, \varepsilon^ {- 1}\right).
$$
The proof sums the regret bound of the  $d$ -dimensional version of the static subroutine (Lemma C.1), across  $T$  different copies. It is very similar to Theorem 1, therefore omitted.
It might be more convenient to use the transform domain coefficients  $\hat{u}^{(j,l)}$  in the bound, rather than the reconstructed details  $z^{(j,l)}$ . In this case, we have
$$
\left\| z ^ {(j, l)} \right\| _ {2} ^ {2} = \sum_ {t} \left\| z _ {t} ^ {(j, l)} \right\| _ {2} ^ {2} = \sum_ {t} \left[ \left\| \hat {u} ^ {(j, l)} \right\| _ {2} ^ {2} \left| \tilde {h} _ {t} ^ {(j, l)} \right| ^ {2} \right] = \left\| \hat {u} ^ {(j, l)} \right\| _ {2} ^ {2} \sum_ {t} \left| \tilde {h} _ {t} ^ {(j, l)} \right| ^ {2} = \left\| \hat {u} ^ {(j, l)} \right\| _ {2} ^ {2}.
$$
$$
\left\| z ^ {*} \right\| _ {2} ^ {2} = \left\| \hat {u} ^ {*} \right\| _ {2} ^ {2}.
$$
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \varepsilon G + G \left(\left\| \hat {u} ^ {*} \right\| _ {2} + \sum_ {j = 1} ^ {\log_ {2} T} \sum_ {l = 1} ^ {2 ^ {- j} T} \left\| \hat {u} ^ {(j, l)} \right\| _ {2}\right) \cdot \operatorname {p o l y l o g} (M, T, \varepsilon^ {- 1}). \tag {13}
$$
D.4 Unconstrained switching regret
In the  $K$ -switching regret, the complexity of the comparator is characterized by its amount of switches. The idea is that, if the comparator  $u_{1:T}$  is static on a support  $I^{(j,l)}$  for some  $(j,l)$ , then the corresponding transform domain coefficient  $\hat{u}^{(j,l)} = 0 \in \mathbb{R}^d$ . We have the following bound for the fixed  $T$  algorithm (Algorithm 4).
Lemma D.2. For any  $m$ ,  $T = 2^m$  and  $u_{1:T} \in \mathbb{R}^{dT}$ , Algorithm 4 with the hyperparameter  $\varepsilon = 1$  guarantees
$$
\operatorname {R e g r e t} _ {T} (u _ {1: T}) = \tilde {O} \left(\| \bar {u} \| _ {2} \sqrt {T} + \sqrt {K \bar {E}}\right).
$$
Proof of Lemma D.2. Consider any scale  $j$ . Since the supports  $\{I^{(j,l)}\}_{l}$  do not overlap, if  $u_{1:T}$  shifts  $K$  times, then there are at most  $K$  choices of location  $l$  such that the transform domain coefficient  $\hat{u}^{(j,l)}$  is nonzero. Furthermore, since there are  $\log_2 T$  scales in total, there are at most  $K \log_2 T$  pairs of  $(i,l)$  such that  $\hat{u}^{(j,l)}$  is nonzero. Therefore, using Cauchy-Schwarz and Eq.(11),
$$
\sum_ {j, l} \left\| \hat {u} ^ {(j, l)} \right\| _ {2} \leq \sqrt {K \log_ {2} T} \sqrt {\sum_ {j , l} \left\| \hat {u} ^ {(j , l)} \right\| _ {2} ^ {2}} = \sqrt {K \bar {E} \log_ {2} T}.
$$
Plugging this into Eq.(13), and further using Eq.(12) for  $\| \hat{u}^* \|_2$  complete the proof.
The anytime bound in general follows from the classical doubling trick. A twist is that the analysis is slightly more involved than the standard one, e.g., [SS11, Section 2.3.1], as we also need to relate the comparator statistics on each block to those for the entire signal  $u_{1:T}$ .
Theorem 2 (Switching regret). For all  $T \in \mathbb{N}_+$  and  $u_{1:T} \in \mathbb{R}^{dT}$ , Algorithm 5 guarantees
Proof of Theorem 2. First, assume  $T$  can be exactly decomposed into  $m^*$  segments with dyadic lengths  $2^1, \ldots, 2^{m^*}$ . We use  $\bar{u}_m$ ,  $K_m$  and  $\bar{E}_m$  to represent the statistics of the comparator sequence on the length  $2^m$  block, and let  $I^m$  denote the time interval that this block operates on.  $\bar{u}$ ,  $K$  and  $S$  denote the statistics of the entire signal  $u_{1:T}$ , c.f., Appendix A. From Lemma D.2,
$$
\begin{array}{l} \operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \sum_ {m = 1} ^ {m ^ {*}} \tilde {O} \left(\left\| \bar {u} _ {m} \right\| _ {2} \sqrt {2 ^ {m}} + \sqrt {K _ {m} \bar {E} _ {m}}\right) \\ \leq \tilde {O} \left[ \| \bar {u} \| _ {2} \left(\sum_ {m = 1} ^ {m ^ {*}} \sqrt {2 ^ {m}}\right) + \sum_ {m = 1} ^ {m ^ {*}} \| \bar {u} _ {m} - \bar {u} \| _ {2} \sqrt {2 ^ {m}} + \sum_ {m = 1} ^ {m ^ {*}} \sqrt {K _ {m} \bar {E} _ {m}} \right]. \tag {14} \\ \end{array}
$$
The first term follows from the standard doubling trick analysis [SS11, Section 2.3.1],
$$
\sum_ {m = 1} ^ {m ^ {*}} \sqrt {2 ^ {m}} \leq \frac {\sqrt {2}}{\sqrt {2} - 1} \sqrt {2 ^ {m ^ {*}}} = O (\sqrt {T}). \tag {15}
$$
As for the second term in Eq.(14), using Cauchy-Schwarz,
$$
\sum_ {m = 1} ^ {m ^ {*}} \left\| \bar {u} _ {m} - \bar {u} \right\| _ {2} \sqrt {2 ^ {m}} \leq \sqrt {m ^ {*} \left(\sum_ {m = 1} ^ {m ^ {*}} 2 ^ {m} \left\| \bar {u} _ {m} - \bar {u} \right\| _ {2} ^ {2}\right)}.
$$
$m^{*} = O(\log T)$ , and also observe that the sum (in the parenthesis) on the RHS equals the second order variability of the following signal: for any time  $t$  in the  $m$ -th block, the signal's component is  $\bar{u}_m \in \mathbb{R}^d$ . This signal is a locally averaged version of the original comparator  $u_{1:T}$ , and the key idea is that local averaging decreases the variability. Formally, due to Lemma D.7, we have
$$
\sum_ {m = 1} ^ {m ^ {*}} \left\| \bar {u} _ {m} - \bar {u} \right\| _ {2} \sqrt {2 ^ {m}} \leq \tilde {O} (\sqrt {\bar {E}}). \tag {16}
$$
For the third term in Eq.(14), using Cauchy-Schwarz again,
$$
\sum_ {m = 1} ^ {m ^ {*}} \sqrt {K _ {m} \bar {E} _ {m}} \leq \sqrt {\left(\sum_ {m = 1} ^ {m ^ {*}} K _ {m}\right) \left(\sum_ {m = 1} ^ {m ^ {*}} \bar {E} _ {m}\right)} \leq \sqrt {K \bar {E}}.
$$
The sum of  $K_{m}$  is straightforward. The inequality for the sum of  $\bar{E}_{m}$  follows from the observation that on the  $m$ -th block,  $\bar{u}_{m}$  minimizes  $\sum_{t\in I^{m}}\| u_{t} - x\|_{2}^{2}$  with respect to  $x\in \mathbb{R}^d$ .
Also, notice that the second term in Eq.(14) is dominated by the third term. If  $K = 0$ , then both  $\sqrt{E}$  and  $\sqrt{KE}$  equal 0. If  $K \geq 1$ , then  $\sqrt{E} \leq \sqrt{KE}$ . Therefore, Eq.(14) can be written as
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\left\| \bar {u} \right\| _ {2} \sqrt {T} + \sqrt {K \bar {E}}\right).
$$
As for the general setting where  $T$  cannot be exactly decomposed into dyadic blocks: consider the smallest  $T^{*} > T$  such that  $T^{*}$  can be decomposed. Due to doubling intervals,  $T^{*} \leq 2T$ . Let us consider a hypothetical length  $T^{*}$  game with the rounds  $t > T$  constructed as follows: the loss gradient  $g_{t} = 0 \in \mathbb{R}^{d}$ , and  $u_{t} = \bar{u}$ . In this case, with  $K$  and  $\bar{E}$  still representing the statistics of the length  $T$  sequence  $u_{1:T}$ , the number of switches on the entire time interval  $[1 : T^{*}]$  is at most  $K + 1$ , and the second order variability on  $[1 : T^{*}]$  is  $\bar{E}$ ; furthermore, it is clear that  $(K + 1)\bar{E} \leq 2K\bar{E}$ . The regret of any algorithm on this hypothetical length  $T^{*}$  game is the same as the length  $T$  game, therefore bounding the latter follows from bounding the former.
D.5 Path-length-based bound
Next, we turn to bounds that depend on the path length  $P$  of the comparator  $u_{1:T}$ . Similar to the switching regret analysis, we will first consider the setting with fixed dyadic  $T$  (Algorithm 4), and then extend its guarantee through a doubling trick.
D.5.1 Fixed dyadic horizon
In the following, we consider Algorithm 4; assume  $T = 2^m$  for some  $m$ . The static component (i.e.,  $z^*$ ) and the dynamic component (i.e.,  $u_{1:T} - z^*$ ) of  $u_{1:T}$  are analyzed separately; the former is fairly standard, while the latter is more challenging. We will first consider the dynamic component, and proceed in three steps.
Step 1 Considering any scale  $j$ , we aim to show  $\sum_{l} \left\| \hat{u}^{(j,l)} \right\|_{2} \leq \sqrt{P^{(j)} \bar{S}^{(j)}}$ , which relates the transform domain coefficients to the regularity of the reconstructed signals.
Lemma D.3. For all  $(j,l)$  pair,
$$
\left\| \hat {u} ^ {(j, l)} \right\| _ {2} = 2 ^ {- 1 / 2} \sqrt {P ^ {(j , l)} \bar {S} ^ {(j , l)}},
$$
$$
\sum_ {l} \left\| \hat {u} ^ {(j, l)} \right\| _ {2} \leq 2 ^ {- 1 / 2} \sqrt {P ^ {(j)} \bar {S} ^ {(j)}}.
$$
Proof of Lemma D.3. Let us start from the first part of this lemma, and express the detail sequence  $z^{(j,l)}$ , and equivalently  $z^{(j)}$ , more explicitly on its support  $I^{(j,l)}$ .
$$
z _ {t} ^ {(j)} = \left\{ \begin{array}{l l} 2 ^ {- j / 2} \hat {u} ^ {(j, l)}, & t \in I _ {+} ^ {(j, l)}; \\ - 2 ^ {- j / 2} \hat {u} ^ {(j, l)}, & t \in I _ {-} ^ {(j, l)}. \end{array} \right.
$$
Rewriting  $P^{(j,l)}$  and  $\bar{S}^{(j,l)}$ ,
$$
P ^ {(j, l)} = \sum_ {t = 2 ^ {j} (l - 1) + 1} ^ {2 ^ {j} l - 1} \left\| z _ {t + 1} ^ {(j)} - z _ {t} ^ {(j)} \right\| _ {2} = 2 ^ {1 - j / 2} \left\| \hat {u} ^ {(j, l)} \right\| _ {2}.
$$
$$
\bar {S} ^ {(j, l)} = \sum_ {t \in I ^ {(j, l)}} \left\| z _ {t} ^ {(j)} \right\| _ {2} = 2 ^ {- j / 2} \left\| \hat {u} ^ {(j, l)} \right\| _ {2} \cdot 2 ^ {j} = 2 ^ {j / 2} \left\| \hat {u} ^ {(j, l)} \right\| _ {2},
$$
which yields the equality in the lemma. The second part follows from Cauchy-Schwarz.
Step 2 Showing that  $P^{(j)} \leq P$  and  $\bar{S}^{(j)} \leq \bar{S}$ . That is, the reconstructed signals are easier than the original comparator  $u_{1:T}$ . Here,  $P$  and  $\bar{S}$  should be considered separately.
Lemma D.4. For any  $u_{1:T}$  and any scale parameter  $j^{*}$ ,  $P^{(j^{*})} \leq P$ .
Proof of Lemma D.4. From the definition of  $P$  and the reconstruction of  $u_{1:T}$  from detail sequences,
$$
P = \sum_ {t = 1} ^ {T - 1} \| u _ {t + 1} - u _ {t} \| _ {2} = \sum_ {t = 1} ^ {T - 1} \left\| z _ {t + 1} ^ {*} - z _ {t} ^ {*} + \sum_ {j} \left(z _ {t + 1} ^ {(j)} - z _ {t} ^ {(j)}\right) \right\| _ {2} = \sum_ {t = 1} ^ {T - 1} \left\| \sum_ {j} \left(z _ {t + 1} ^ {(j)} - z _ {t} ^ {(j)}\right) \right\| _ {2},
$$
where the last equality is due to  $z^{*}$  being a constant sequence.
Consider removing "shorter" scales with  $1 \leq j < j^{*}$ , which is equivalent to local averaging, c.f., Appendix D.6. Due to Lemma D.7, the path length does not increase, i.e,
$$
\sum_ {t = 1} ^ {T - 1} \left\| \sum_ {j} \left(z _ {t + 1} ^ {(j)} - z _ {t} ^ {(j)}\right) \right\| _ {2} \geq \sum_ {t = 1} ^ {T - 1} \left\| \sum_ {j \geq j ^ {*}} \left(z _ {t + 1} ^ {(j)} - z _ {t} ^ {(j)}\right) \right\| _ {2}.
$$
Then, we can further remove the rounds where the path length is not counted in  $P^{(j^*)}$ , i.e., when a time  $t \in I^{(j^*, l)}$  but  $t + 1 \in I^{(j^*, l + 1)}$ .
$$
\mathrm {R H S} \geq \sum_ {l} \sum_ {t = 2 ^ {j ^ {*}} (l - 1) + 1} ^ {2 ^ {j ^ {*}} l - 1} \left\| \sum_ {j \geq j ^ {*}} \left(z _ {t + 1} ^ {(j)} - z _ {t} ^ {(j)}\right) \right\| _ {2}.
$$
Now, consider any location  $l$ , which determines the time interval  $I^{(j^*,l)} = [2^{j^*}(l - 1) + 1:2^{j^*}l]$ . Any detail sequence  $z^{(j)}$  with scale  $j > j^*$  is constant on this time interval, thus removing it does not change the path length at all. Therefore,
$$
P \geq \sum_ {l} \sum_ {t = 2 ^ {j ^ {*}} (l - 1) + 1} ^ {2 ^ {j ^ {*}} l - 1} \left\| z _ {t + 1} ^ {(j ^ {*})} - z _ {t} ^ {(j ^ {*})} \right\| _ {2} = P ^ {(j ^ {*})}.
$$
As for the first order variability,
Lemma D.5. For any  $u_{1:T}$  and any scale parameter  $j^{*}$ ,  $\bar{S}^{(j^{*})} \leq \bar{S}$ .
Proof of Lemma D.5. From the definition, noticing that  $\bar{u}$  is entirely captured by the all-one feature,
$$
\bar {S} = \sum_ {t = 1} ^ {T} \| u _ {t} - \bar {u} \| _ {2} = \sum_ {t = 1} ^ {T} \left\| \sum_ {j = 1} ^ {\log_ {2} T} z _ {t} ^ {(j)} \right\| _ {2}.
$$
Due to Lemma D.7, removing short scales amounts to local averaging, which decreases the variability.
$$
\bar {S} \geq \sum_ {t = 1} ^ {T} \left\| \sum_ {j \geq j ^ {*}} z _ {t} ^ {(j)} \right\| _ {2} = \sum_ {l} \sum_ {t \in I ^ {(j ^ {*}, l)}} \left\| z _ {t} ^ {(j ^ {*})} + \sum_ {j > j ^ {*}} z _ {t} ^ {(j)} \right\| _ {2}.
$$
For any  $l$ , consider the support of the  $(j^{*}, l)$ -th feature,  $I^{(j^{*}, l)}$ . Observe that  $\sum_{j > j^{*}} z_{t}^{(j)}$  is time invariant throughout  $I^{(j^{*}, l)}$ , let us denote it as  $v \in \mathbb{R}^d$ . Meanwhile, for some  $w \in \mathbb{R}^d$ ,  $z_{t}^{(j^{*})}$  equals  $w$  on  $I_{+}^{(j^{*}, l)}$ , the first half of this interval, while being  $-w$  on the second half  $I_{-}^{(j^{*}, l)}$  of this interval. Therefore,
$$
\sum_ {t \in I ^ {(j ^ {*}, l)}} \left\| z _ {t} ^ {(j ^ {*})} + \sum_ {j > j ^ {*}} z _ {t} ^ {(j)} \right\| _ {2} = 2 ^ {j ^ {*} - 1} \left(\| v + w \| _ {2} + \| v - w \| _ {2}\right) \geq 2 ^ {j ^ {*}} \| w \| _ {2} = \sum_ {t \in I ^ {(j ^ {*}, l)}} \left\| z _ {t} ^ {(j ^ {*})} \right\| _ {2}.
$$
$$
\bar {S} \geq \sum_ {l} \sum_ {t \in I ^ {(j ^ {*}, l)}} \left\| z _ {t} ^ {(j ^ {*})} \right\| _ {2} = \bar {S} ^ {(j ^ {*})}.
$$
Step 3 Summarizing the above relations, and using the property that there are only  $\log_2 T$  scales.
Lemma D.6. For any  $m$ ,  $T = 2^m$  and  $u_{1:T} \in \mathbb{R}^{dT}$ , Algorithm 4 with the hyperparameter  $\varepsilon = 1$  guarantees
$$
\operatorname {R e g r e t} _ {T} (u _ {1: T}) = \tilde {O} \left(\| \bar {u} \| _ {2} \sqrt {T} + \sqrt {P \bar {S}}\right).
$$
Proof of Lemma D.6. Starting from the generic regret bound, Eq.(13) for Algorithm 4.
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \varepsilon G + G \left(\left\| \hat {u} ^ {*} \right\| _ {2} + \sum_ {j, l} \left\| \hat {u} ^ {(j, l)} \right\| _ {2}\right) \cdot \operatorname {p o l y l o g} \left(M, T, \varepsilon^ {- 1}\right).
$$
Due to Eq.(12),  $\| \hat{u}^* \|_2 = \| \bar{u} \|_2 \sqrt{T}$ . Then, combining Lemma D.3, D.4 and D.5,
$$
\sum_ {j, l} \left\| \hat {u} ^ {(j, l)} \right\| _ {2} \leq O \left(\sqrt {P \bar {S}} \log_ {2} T\right).
$$
Plugging it into the generic bound completes the proof.
D.5.2 Anytime bound
Now we are ready to prove an anytime unconstrained dynamic regret bound that depends on the path length.
Proof of Theorem 3. Similar to the analysis of the switching regret (Theorem 2), we first consider the situation where the time horizon  $T$  can be exactly decomposed into  $m^{*}$  segments with dyadic lengths  $2^{1},\ldots ,2^{m^{*}}$ . In this situation, we have
$$
\begin{array}{l} \operatorname {R e g r e t} _ {T} (u _ {1: T}) \leq \tilde {O} \left[ \sum_ {m = 1} ^ {m ^ {*}} \| \bar {u} _ {m} \| _ {2} \sqrt {2 ^ {m}} + \sum_ {m = 1} ^ {m ^ {*}} \sqrt {P _ {m} \bar {S} _ {m}} \right] \\ \leq \tilde {O} \left[ \| \bar {u} \| _ {2} \sqrt {T} + \sqrt {\bar {E}} + \sum_ {m = 1} ^ {m ^ {*}} \sqrt {P _ {m} \bar {S} _ {m}} \right], \\ \end{array}
$$
where the second line follows from the proof of Theorem 2, specifically Eq.(15) and Eq.(16).
Now let us consider the remaining sum on the RHS. Using Cauchy-Schwarz,
$$
\sum_ {m = 1} ^ {m ^ {*}} \sqrt {P _ {m} \bar {S} _ {m}} \leq \sqrt {\left(\sum_ {m = 1} ^ {m ^ {*}} P _ {m}\right) \left(\sum_ {m = 1} ^ {m ^ {*}} \bar {S} _ {m}\right)} \leq \sqrt {P \left(\sum_ {m = 1} ^ {m ^ {*}} \sum_ {t = 2 ^ {m} - 1} ^ {2 ^ {m + 1} - 2} \| u _ {t} - \bar {u} _ {m} \| _ {2}\right)},
$$