SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} R _ {T} ^ {\text {s t a t i c}} \left(u ^ {*}\right) = \sum_ {t = 1} ^ {T} f _ {t} \left(u _ {t}\right) - f _ {t} \left(u ^ {*}\right) \leq \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t}, \left(u _ {t} - u ^ {*}\right) \right\rangle \\ \leq \sum_ {t = 1} ^ {T} \frac {\left\| u _ {t} - u ^ {*} \right\| ^ {2} - \left\| u _ {t + 1} - u ^ {*} \right\| ^ {2}}{2 \gamma_ {t}} + \sum_ {t = 1} ^ {T} \frac {\gamma_ {t} \left\| \nabla f _ {t} \right\| ^ {2}}{2} \\ \leq \sum_ {t = 1} ^ {T} \| u _ {t} - u ^ {*} \| ^ {2} \left(\frac {1}{2 \gamma_ {t}} - \frac {1}{2 \gamma_ {t - 1}}\right) + \sum_ {t = 1} ^ {T} \frac {\gamma_ {t} \| \nabla f _ {t} \| ^ {2}}{2} \\ \leq (\hat {D} ^ {2} + \frac {b \hat {D} ^ {2} + \epsilon_ {\max}}{a}) \frac {1}{2 \gamma_ {T}} + (\frac {4 a b ^ {2} \hat {D} ^ {2}}{\tilde {\epsilon} _ {0}} + \frac {a ^ {2} (\epsilon_ {\max} + b \hat {D} ^ {2}) ^ {2}}{\tilde {\epsilon} _ {0} ^ {2}}) \sum_ {t = 1} ^ {T} \frac {\gamma_ {t} \kappa_ {t} ^ {2}}{2} \\ \leq \frac {3 \kappa_ {\operatorname* {m a x}}}{2 \tilde {\epsilon} _ {0}} \sqrt {T} \left(\sqrt {(\hat {D} ^ {2} + \frac {b \hat {D} ^ {2} + \epsilon_ {\operatorname* {m a x}}}{a}) (4 a b ^ {2} \tilde {\epsilon} _ {0} \hat {D} ^ {2} + a ^ {2} (\epsilon_ {\operatorname* {m a x}} + b \hat {D} ^ {2}) ^ {2})}\right). \\ \end{array}
$$
We have the learning rate schedule  $\gamma_{t} = \frac{\gamma}{\sqrt{t}}$  and  $1 / \gamma_0 = 0$ . The third inequality holds because of the aforementioned bounds of  $\| u_t - u^*\|^2$  and  $\|\nabla f_t\|$ . The last step is obtained by setting  $\gamma = \frac{\tilde{\epsilon}_0\sqrt{\hat{D}^2 + \frac{b\hat{D}^2 + \epsilon_{\max}}{a}}}{\kappa_{\max}\sqrt{4ab^2\tilde{\epsilon}_0\hat{D}^2 + a^2(\epsilon_{\max} + b\hat{D}^2)^2}}$  and  $\sum_{t=1}^{T}\frac{1}{\sqrt{t}} \leq 2\sqrt{T}$ , where  $\kappa_{\max} = \max\{\kappa_{1:T}\}$ ,  $\epsilon_{\max} = \max\{\epsilon_{1:T}\}$ .
According to the setting of  $\tilde{\epsilon}_0$ , we have two conditions.
1. If  $\epsilon_v \leq bV^2 + \epsilon$ ,  $\beta^* = \sqrt{(bV^2 + \epsilon + \tilde{\epsilon}_0) / a} \in \left[\sqrt{\tilde{\epsilon}_0 / a}, \sqrt{(b\hat{D}^2 + \epsilon + \tilde{\epsilon}_0) / a}\right]$  can be attained. Then, for the optimal comparator in hindsight  $u^* = (\beta^*, \phi^*)$ , we have the accumulated cost:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} f _ {t} \left(u ^ {*}\right) = \sum_ {t = 1} ^ {T} \kappa_ {t} \left(a \beta^ {*} + \frac {b \| \phi^ {*} - w _ {t} \| ^ {2} + \epsilon_ {t} + \epsilon_ {0}}{\beta^ {*}} + \Delta_ {t}\right) \\ = \left(2 \sqrt {a \left(b V ^ {2} + \epsilon + \epsilon_ {0}\right)}\right) \sum_ {t = 1} ^ {T} \kappa_ {t} + \sum_ {t = 1} ^ {T} \kappa_ {t} \Delta_ {t}. \\ \end{array}
$$
Finally, we obtain:
$$
\begin{array}{l} \mathrm {A E R} _ {\mathcal {A}} ^ {T} \leq \frac {1}{T} \sum_ {t = 1} ^ {T} f _ {t} (u _ {t}) \\ \leq \frac {1}{T} \left(\sum_ {t = 1} ^ {T} f _ {t} \left(u ^ {*}\right) + R _ {T} ^ {\text {s t a t i c}} \left(u ^ {*}\right)\right) \\ = \left(2 \sqrt {a \left(b V ^ {2} + \epsilon + \epsilon_ {0}\right)}\right) \sum_ {t = 1} ^ {T} \frac {\kappa_ {t}}{T} + \sum_ {t = 1} ^ {T} \frac {\kappa_ {t} \Delta_ {t}}{T} \\ + \frac {3 \kappa_ {\operatorname* {m a x}}}{2 \tilde {\epsilon} _ {0} \sqrt {T}} \left(\sqrt {(\hat {D} ^ {2} + \frac {b \hat {D} ^ {2} + \epsilon_ {\operatorname* {m a x}}}{a}) (4 a b ^ {2} \tilde {\epsilon} _ {0} \hat {D} ^ {2} + a ^ {2} (\epsilon_ {\operatorname* {m a x}} + b \hat {D} ^ {2}) ^ {2})}\right). \\ \end{array}
$$
2. If  $\epsilon_v > bV^2 + \epsilon$ ,  $\beta^* = \sqrt{(bV^2 + \epsilon + \tilde{\epsilon}_0) / a} < \sqrt{\tilde{\epsilon}_0 / a}$  cannot be attained. We can simply upper bound the cost function by replacing  $\epsilon_0$  with  $\tilde{\epsilon}_0$  and obtain a newly attainable optimal comparator in hindsight  $\tilde{u}^* = (\tilde{\beta}^*, \phi^*)$ , where  $\tilde{\beta}^* = \sqrt{(bV^2 + \epsilon + \tilde{\epsilon}_0) / a} \in \left[\sqrt{\tilde{\epsilon}_0 / a}, \sqrt{(b\hat{D}^2 + \epsilon + \tilde{\epsilon}_0) / a}\right]$ . Then, we have the accumulated cost:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} f _ {t} (\tilde {u} ^ {*}) = \sum_ {t = 1} ^ {T} \kappa_ {t} (a \tilde {\beta} ^ {*} + \frac {b \| \phi^ {*} - w _ {t} \| ^ {2} + \epsilon_ {t} + \tilde {\epsilon} _ {0}}{\tilde {\beta} ^ {*}} + \Delta_ {t}) \\ = \left(2 \sqrt {a \left(b V ^ {2} + \epsilon + \tilde {\epsilon} _ {0}\right)}\right) \sum_ {t = 1} ^ {T} \kappa_ {t} + \sum_ {t = 1} ^ {T} \kappa_ {t} \Delta_ {t}. \\ \end{array}
$$
$$
\begin{array}{l} \mathrm {A E R} _ {\mathcal {A}} ^ {T} \leq \frac {1}{T} \sum_ {t = 1} ^ {T} f _ {t} (u _ {t}) \\ \leq \frac {1}{T} \left(\sum_ {t = 1} ^ {T} f _ {t} \left(\tilde {u} ^ {*}\right) + R _ {T} ^ {\text {s t a t i c}} \left(\tilde {u} ^ {*}\right)\right) \\ = \left(2 \sqrt {a \left(b V ^ {2} + \epsilon + \tilde {\epsilon} _ {0}\right)}\right) \sum_ {t = 1} ^ {T} \frac {\kappa_ {t}}{T} + \sum_ {t = 1} ^ {T} \frac {\kappa_ {t} \Delta_ {t}}{T} \\ + \frac {3 \kappa_ {\max}}{2 \tilde {\epsilon} _ {0} \sqrt {T}} \left(\sqrt {\left(\hat {D} ^ {2} + \frac {b \hat {D} ^ {2} + \epsilon_ {\max}}{a}\right) \left(4 a b ^ {2} \tilde {\epsilon} _ {0} \hat {D} ^ {2} + a ^ {2} \left(\epsilon_ {\max} + b \hat {D} ^ {2}\right) ^ {2}\right)}\right). \\ \end{array}
$$
D.2 AER bound in possibly shifting environments (Proof of Theorem 5.1)
Theorem D.2. Consider both static and shifting environments. If the excess risk's upper bound of the base learner  $\mathcal{A}(u_t, S_t)$  can be formulated as a unified form in Eq.(1), then, the AER of DCML (in Algo. 1) is upper bounded by:
$$
A E R _ {\mathcal {A}} ^ {T} \leq \underbrace {\frac {2}{T} \sum_ {n = 1} ^ {N} (\sqrt {a (b V _ {n} ^ {2} + \epsilon_ {n} + \epsilon_ {0})} \kappa_ {n} + \frac {\Delta_ {n}}{2})} _ {\text {o p t i m a l t r a d e - o f f i n h i d s i g h t}} + \underbrace {\frac {3}{2 T} \sum_ {n = 1} ^ {N} \tilde {D} _ {n} G _ {n} \sqrt {M _ {n} - 1}} _ {\text {a v e r a g e r e g r e t o v e r s l o t s}} + \underbrace {\frac {\tilde {D} _ {m a x}}{T} \sqrt {2 P ^ {*} \sum_ {n = 1} ^ {N} G _ {n} ^ {2}}} _ {\text {r e g r e t w . r . t e n v i r o n m e n t s h i f t}},
$$
Here,  $\kappa_{n} = \sum_{k=1}^{M_{n}} \kappa_{n,k}$ ,  $\Delta_{n} = \sum_{k=1}^{M_{n}} \kappa_{n,k} \Delta_{n,k}$ , where the subscript  $n, k$  indicates  $k$ -th task in  $n$ -th slot. The optimal meta-parameter in hindsight of  $n$ -th slot is  $u_{n}^{*} = (\beta_{n}^{*}, \phi_{n}^{*})$  and the path length of  $N$  slots is  $P^{*} = \sum_{n=1}^{N-1} \| u_{n}^{*} - u_{n+1}^{*} \| + 1$ .  $\phi_{n}^{*} = \sum_{k=1}^{M_{n}} \frac{\kappa_{n,k}}{\kappa_{n}} w_{n,k}$  is the weighted average of the base learner outputs within the slot.  $\beta_{n}^{*} = \sqrt{(bV_{n}^{2} + \epsilon_{n} + \epsilon_{0}) / a}$ , where  $V_{n}^{2} = \sum_{k=1}^{M_{n}} \frac{\kappa_{n,k}}{\kappa_{n}} \| \phi_{n}^{*} - w_{n,k} \|_{2}^{2}$  is the variance in  $n$ -th slot and  $\epsilon_{n} = \sum_{k=1}^{M_{n}} \frac{\kappa_{n,k}}{\kappa_{n}} \epsilon_{n,k}$ . Moreover,  $G_{n}$  is the maximal gradient norm of the cost function,  $\tilde{D}_{n}$  is the diameter of meta-parameters in  $n$ -th slot, and  $\tilde{D}_{max} = \max \{\tilde{D}_{n}\}_{n=1}^{N}$ .
Proof. A possibly shifting environment can be considered slot-wise static or static, so we need to derive the dynamic regret bound. We use the subscript  $n, k$  to indicate the  $k$ -th task inside the  $n$ -th slot and the corresponding timestep is  $t = \sum_{i=1}^{n-1} M_i + k$ . To obtain such a bound, we first use the following result for the meta-parameter updates inside the  $n$ -th slot:
$$
\begin{array}{l} \left\| u _ {n, k + 1} - u _ {n} ^ {*} \right\| ^ {2} = \left\| \Pi_ {\mathcal {U}} \left(u _ {n, k} - \gamma_ {n, k} \nabla f _ {n, k} \left(u _ {n, k}\right)\right) - u _ {n} ^ {*} \right\| \leq \left\| u _ {n, k} - \gamma_ {n, k} \nabla f _ {n, k} - u _ {n} ^ {*} \right\| ^ {2} \\ \leq \left\| u _ {n, k} - u _ {n} ^ {*} \right\| ^ {2} + \gamma_ {n, k} ^ {2} \left\| \nabla f _ {n, k} \right\| ^ {2} - 2 \gamma_ {n, k} \langle \nabla f _ {n, k}, (u _ {n, k} - u _ {n} ^ {*}) \rangle . \\ \end{array}
$$
So we have  $\langle \nabla f_{n,k},(u_{n,k} - u_n^*)\rangle \leq \frac{\|u_{n,k} - u_n^*\|^2 - \|u_{n,k + 1} - u_n^*\|^2}{2\gamma_{n,k}} +\frac{\gamma_{n,k}\|\nabla f_{n,k}\|^2}{2}$ .
Let  $G_{n}$  be the upper bound of the gradient norm related to the  $n$ -th slot, i.e.,  $\max_k\{\| \nabla f_{n,k}\| \} \leq G_n$ . Applying the above result and the update rule in Algo. 1, we have the dynamic regret:
$$
\begin{array}{l} R _ {T} ^ {\text {d y n a m i c}} \left(u _ {1: N} ^ {*}\right) = \sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n}} f _ {n, k} \left(u _ {n, k}\right) - f _ {n, k} \left(u _ {n} ^ {*}\right) \\ \leq \sum_ {n = 1} ^ {N} \left(\sum_ {k = 1} ^ {M _ {n} - 1} \nabla f _ {n, k} ^ {\top} \left(u _ {n, k} - u _ {n} ^ {*}\right) + \nabla f _ {n, M _ {n}} ^ {\top} \left(u _ {n, M _ {n}} - u _ {n} ^ {*}\right)\right) \\ \leq \sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n} - 1} \left[ \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n , k + 1} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} + G _ {n} ^ {2} \frac {\gamma_ {n , k}}{2} \right] \\ + \sum_ {n = 1} ^ {N} \left(\frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n + 1 , 1} - u _ {n} ^ {*} \| ^ {2}}{2 \rho} + G _ {n} ^ {2} \frac {\rho}{2}\right). \\ \end{array}
$$
The meta-parameter updates that transition between slots adopt the hopping learning rate  $\rho$ , which reflects the environment change.
$$
T _ {1} = \sum_ {n = 1} ^ {N} \left(\sum_ {k = 1} ^ {M _ {n} - 1} \left[ \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n , k + 1} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} + G _ {n} ^ {2} \frac {\gamma_ {n , k}}{2} \right] + G _ {n} ^ {2} \frac {\rho}{2}\right)
$$
$$
T _ {2} = \sum_ {n = 1} ^ {N} \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \| ^ {2} + \| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \| ^ {2} - \| u _ {n + 1 , 1} - u _ {n} ^ {*} \| ^ {2}}{2 \rho}.
$$
First, let us tackle  $T_{2}$
$$
\begin{array}{l} T _ {2} = \sum_ {n = 1} ^ {N} \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \| ^ {2}}{2 \rho} + \sum_ {n = 1} ^ {N} \frac {\| u _ {n + 1} ^ {*} \| ^ {2} - \| u _ {n} ^ {*} \| ^ {2} + 2 u _ {n + 1 , 1} ^ {\top} (u _ {n} ^ {*} - u _ {n + 1} ^ {*})}{2 \rho} \\ = \sum_ {n = 1} ^ {N - 1} \frac {\left\| u _ {n , M _ {n}} - u _ {n} ^ {*} \right\| ^ {2} - \left\| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \right\| ^ {2}}{2 \rho} \\ + \sum_ {n = 1} ^ {N - 1} \frac {\| u _ {n + 1} ^ {*} \| ^ {2} - \| u _ {n} ^ {*} \| ^ {2} + 2 u _ {n + 1 , 1} ^ {\top} (u _ {n} ^ {*} - u _ {n + 1} ^ {*})}{2 \rho} \\ + \frac {\| u _ {N , M _ {N}} - u _ {N} ^ {*} \| ^ {2} - \| u _ {N + 1 , 1} - u _ {N + 1} ^ {*} \| ^ {2} + \| u _ {N + 1} ^ {*} \| ^ {2} - \| u _ {N} ^ {*} \| ^ {2} + 2 u _ {N + 1 , 1} ^ {\top} (u _ {N} ^ {*} - u _ {N + 1} ^ {*})}{2 \rho} \\ \leq \sum_ {n = 1} ^ {N - 1} \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \| ^ {2}}{2 \rho} + \frac {\| u _ {N} ^ {*} \| ^ {2} - \| u _ {1} ^ {*} \| ^ {2}}{2 \rho} + \frac {\tilde {D} _ {\max}}{\rho} \sum_ {n = 1} ^ {N - 1} \| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \| \\ + \frac {\left\| u _ {N , M _ {N}} - u _ {N} ^ {*} \right\| ^ {2} - \left\| u _ {N + 1 , 1} \right\| ^ {2} + 2 u _ {N + 1 , 1} ^ {\top} u _ {N} ^ {*} - \left\| u _ {N} ^ {*} \right\| ^ {2}}{2 \rho} \\ \leq \sum_ {n = 1} ^ {N - 1} \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2} - \| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \| ^ {2}}{2 \rho} + \frac {\| u _ {N} ^ {*} \| ^ {2} - \| u _ {1} ^ {*} \| ^ {2}}{2 \rho} + \frac {\tilde {D} _ {\max}}{\rho} \sum_ {n = 1} ^ {N - 1} \| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \| \\ + \frac {\left\| u _ {N , M _ {N}} - u _ {N} ^ {*} \right\| ^ {2}}{2 \rho}. \\ \end{array}
$$
The first inequality is obtained with
$$
u _ {n + 1, 1} ^ {\top} \left(u _ {n} ^ {*} - u _ {n + 1} ^ {*}\right) \leq \left\| u _ {n + 1, 1} \right\| \left\| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \right\| \leq \tilde {D} _ {\max } \left\| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \right\|,
$$
where  $\tilde{D}_{\mathrm{max}} = \max \{\tilde{D}_n\}_{n=1}^N$  is the maximal slot-wise diameter of the set of meta-parameters. Since we have assumed that  $\mathbf{0} \in \mathcal{W}$ , it gives  $\|u_{n+1,1}\| \leq \tilde{D}_{\mathrm{max}}, \forall n$ . The last inequality of the above proof holds because  $\frac{-\|u_{N+1,1}\|^2 + 2u_{N+1,1}^\top u_N^* - \|u_N^*\|^2}{2\rho} = \frac{-\|u_{N+1,1} - u_N^*\|^2}{2\rho} \leq 0$ .
Let  $\hat{D}_n$  be the diameter of the base learner outputs in  $n$ -th slot, and let  $\hat{D}_{\mathrm{max}} = \max \{\hat{D}_n\}_{n=1}^N$ ,  $\kappa_n^{\mathrm{max}} = \max \{\kappa_{n,k}\}_{k=1}^{M_n}$ ,  $\epsilon_n^{\mathrm{max}} = \max \{\epsilon_{n,k}\}_{k=1}^{M_n}$ ,  $\epsilon_{\mathrm{max}} = \max \{\epsilon_n^{\mathrm{max}}\}_{n=1}^N$ .
We constrain  $\beta_{n,k} \in \left[\sqrt{\tilde{\epsilon}_0 / a}, \sqrt{(b\hat{D}_n^2 + \epsilon_{\max} + \tilde{\epsilon}_0) / a}\right]$ , where  $\tilde{\epsilon}_0 = \epsilon_0 + \epsilon_v$  is related to the randomness of the base learner  $\epsilon_0$  and a factor  $\epsilon_v$  to control the range of  $\beta_t$ . We will see later that  $\epsilon_v$  can affect the upper bound of the gradient norm of the meta-parameter. We can further verify that  $\frac{2b\gamma_{n,k}\kappa_{n,k}}{\beta_{n,k}} \leq 1$  can always hold with  $\tilde{\epsilon}_0 < \frac{a^2(\epsilon_{\max} + b\hat{D}_n^2)}{4b^2}$ . Hence,  $\phi_{n,k}$  is always inside the smallest ball that contains  $\hat{\mathcal{W}}_n$ . Then, we can define  $\tilde{D}_n^2 \stackrel{\mathrm{def}}{=} \hat{D}_n^2 + \frac{b\hat{D}_n^2 + \epsilon_n^{\max} + \tilde{\epsilon}_0}{a}$ . Therefore, we have
$$
\tilde {D} _ {\max } ^ {2} = (1 + b / a) \hat {D} _ {\max } ^ {2} + \left(\epsilon_ {\max } + \tilde {\epsilon} _ {0}\right) / a.
$$
Now we tackle  $T_{1}$ . Since we scale the learning rate of the meta-parameter for shifting environments, we need to set  $\rho > \gamma_{n,k}$  for better tracking of the change. Then, we note that if we define  $\frac{1}{\gamma_{n,0}} = 0$ , we have
$$
\begin{array}{l} \sum_ {k = 1} ^ {M _ {n} - 1} \frac {\| u _ {n , k + 1} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} = \sum_ {k = 0} ^ {M _ {n} - 1} \frac {\| u _ {n , k + 1} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} = \sum_ {k = 0} ^ {M _ {n} - 2} \frac {\| u _ {n , k + 1} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} + \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , M _ {n} - 1}} \\ \geq \sum_ {k = 0} ^ {M _ {n} - 2} \frac {\| u _ {n , k + 1} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} + \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2}}{2 \rho} \\ = \sum_ {k = 1} ^ {M _ {n} - 1} \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k - 1}} + \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2}}{2 \rho}. \\ \end{array}
$$
In addition, we have  $\| u_{n,k} - u_n^*\|^2 = \| \phi_{n,k} - \hat{\phi}_n^*\|^2 + |\beta_{n,k} - \beta_n^*|^2 \leq \hat{D}_n^2 + \frac{b\hat{D}_n^2 + \epsilon_n^{\max}}{a} \leq \tilde{D}_n^2$ .
So, we can bound the dynamic regret as
$$
\begin{array}{l} R _ {T} ^ {\text {d y n a m i c}} \left(u _ {1: N} ^ {*}\right) \leq T _ {1} + T _ {2} \\ \leq \sum_ {n = 1} ^ {N} \left(\sum_ {k = 1} ^ {M _ {n} - 1} \left[ \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} - \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k - 1}} + G _ {n} ^ {2} \frac {\gamma_ {n , k}}{2} \right] - \frac {\| u _ {n , M _ {n}} - u _ {n} ^ {*} \| ^ {2}}{2 \rho} + G _ {n} ^ {2} \frac {\rho}{2}\right) \\ + \sum_ {n = 1} ^ {N - 1} \frac {\left\| u _ {n , M _ {n}} - u _ {n} ^ {*} \right\| ^ {2} - \left\| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \right\| ^ {2}}{2 \rho} \\ + \frac {\left\| u _ {N} ^ {*} \right\| ^ {2} - \left\| u _ {1} ^ {*} \right\| ^ {2}}{2 \rho} + \frac {\tilde {D} _ {\max}}{\rho} \sum_ {n = 1} ^ {N - 1} \left\| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \right\| + \frac {\left\| u _ {N , M _ {N}} - u _ {N} ^ {*} \right\| ^ {2}}{2 \rho} \\ = \sum_ {n = 1} ^ {N} \left(\sum_ {k = 1} ^ {M _ {n} - 1} \left[ \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k}} - \frac {\| u _ {n , k} - u _ {n} ^ {*} \| ^ {2}}{2 \gamma_ {n , k - 1}} + G _ {n} ^ {2} \frac {\gamma_ {n , k}}{2} \right] + G _ {n} ^ {2} \frac {\rho}{2}\right) \\ + \sum_ {n = 1} ^ {N} \frac {\left\| u _ {n , M _ {n}} - u _ {n} ^ {*} \right\| ^ {2}}{2 \rho} + \sum_ {n = 1} ^ {N - 1} \frac {- \left\| u _ {n + 1 , 1} - u _ {n + 1} ^ {*} \right\| ^ {2}}{2 \rho} - \sum_ {n = 1} ^ {N} \frac {\left\| u _ {n , M _ {n}} - u _ {n} ^ {*} \right\| ^ {2}}{2 \rho} \\ + \frac {\| u _ {N} ^ {*} \| ^ {2} - \| u _ {1} ^ {*} \| ^ {2}}{2 \rho} + \frac {\tilde {D} _ {\max}}{\rho} \sum_ {n = 1} ^ {N - 1} \| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \| \\ \leq \sum_ {n = 1} ^ {N} \left(\sum_ {k = 1} ^ {M _ {n} - 1} \tilde {D} _ {n} ^ {2} \left[ \frac {1}{2 \gamma_ {n , k}} - \frac {1}{2 \gamma_ {n , k - 1}} \right] + G _ {n} ^ {2} \frac {\gamma_ {n , k}}{2} + G _ {n} ^ {2} \frac {\rho}{2}\right) \\ + \frac {\tilde {D} _ {\max } ^ {2}}{\rho} + \frac {\tilde {D} _ {\max } ^ {2}}{\rho} \sum_ {n = 1} ^ {N - 1} \| u _ {n} ^ {*} - u _ {n + 1} ^ {*} \|. \\ \end{array}
$$
The last inequality is derived with  $\| u_N^*\| \leq \tilde{D}_{\mathrm{max}}$  and ignoring the negative terms.
Denote the path length of the reference sequence as  $P^{*} = \sum_{n=1}^{N-1} \| u_{n}^{*} - u_{n+1}^{*} \| + 1$ . By setting  $\gamma_{n,k} = \frac{\tilde{D}_{n}}{G_{n}\sqrt{k}}$  (satisfies  $1 / \gamma_{n,0} = 0$ ),  $\rho = \sqrt{\frac{2P^{*}\tilde{D}_{\max}^{2}}{\sum_{n=1}^{N} G_{n}^{2}}}$ , and using  $\sum_{k=1}^{M_{n}-1} \frac{1}{\sqrt{k}} \leq 2\sqrt{M_{n}-1}$ , we have
$$
\begin{array}{l} R _ {T} ^ {\text {d y n a m i c}} \left(u _ {1: N} ^ {*}\right) \leq \sum_ {n = 1} ^ {N} \left(\frac {3 \tilde {D} _ {n} G _ {n} \sqrt {M _ {n} - 1}}{2} + \frac {G _ {n} ^ {2}}{2} \rho\right) + \frac {\tilde {D} _ {\max } ^ {2} P ^ {*}}{\rho} \\ \leq \sum_ {n = 1} ^ {N} \frac {3 \tilde {D} _ {n} G _ {n} \sqrt {M _ {n} - 1}}{2} + \tilde {D} _ {\max } \sqrt {2 P ^ {*} \sum_ {n = 1} ^ {N} G _ {n} ^ {2}}. \\ \end{array}
$$
For each slot  $n$ , the optimal prior in hindsight is  $u_{n}^{*} = (\phi_{n}^{*}, \beta_{n}^{*})$ . By solving  $\min_u \sum_{k=1}^{M_n} f_{n,k}(u)$ , we can get:
$$
\phi_ {n} ^ {*} = \sum_ {k = 1} ^ {M _ {n}} \frac {\kappa_ {n , k}}{\sum_ {k = 1} ^ {M _ {n}} \kappa_ {n , k}} w _ {n, k}, \beta_ {n} ^ {*} = \sqrt {(b V _ {n} ^ {2} + \epsilon_ {n} + \epsilon_ {0}) / a},
$$
where  $V_{n}^{2} = \sum_{k = 1}^{M_{n}}\frac{\kappa_{n,k}}{\sum_{k = 1}^{M_{n}}\kappa_{n,k}}\| \phi_{n}^{*} - w_{n,k}\|^{2}$  and  $\epsilon_{n} = \frac{\sum_{k = 1}^{M_{n}}\kappa_{n,k}\epsilon_{n,k}}{\sum_{k = 1}^{M_{n}}\kappa_{n,k}}$
Thus, the accumulated cost for the slot-wise optimal priors in hindsight:
$$
\begin{array}{l} \sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n}} f _ {n, k} \left(u _ {n} ^ {*}\right) = \sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} \left(a \beta_ {n} ^ {*} + \frac {b \| \phi_ {n} ^ {*} - w _ {n , k} \| ^ {2} + \epsilon_ {n , k} + \epsilon_ {0}}{\beta_ {n} ^ {*}} + \Delta_ {n, k}\right) \\ = \sum_ {n = 1} ^ {N} \left(2 \sqrt {a \left(b V _ {n} ^ {2} + \epsilon_ {n} + \epsilon_ {0}\right)}\right) \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} + \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} \Delta_ {n, k} \\ \end{array}
$$
Finally, we obtain the bound in the main theorem:
$$
\begin{array}{l} \mathrm {A E R} _ {\mathcal {A}} ^ {T} \leq \frac {1}{T} \sum_ {t = 1} ^ {T} f _ {t} (u _ {t}) \\ \leq \frac {1}{T} \left(\sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n}} f _ {n, k} \left(u _ {n} ^ {*}\right) + R _ {T} ^ {\text {d y n a m i c}} \left(u _ {1: N} ^ {*}\right)\right) \\ = \frac {1}{T} \sum_ {n = 1} ^ {N} \left(2 \sqrt {a \left(b V _ {n} ^ {2} + \epsilon_ {n} + \epsilon_ {0}\right)}\right) \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} + \frac {1}{T} \sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} \Delta_ {n, k} \\ + \frac {1}{T} \sum_ {n = 1} ^ {N} \frac {3 \tilde {D} _ {n} G _ {n} \sqrt {M _ {n} - 1}}{2} + \frac {1}{T} \tilde {D} _ {\max } \sqrt {2 P ^ {*} \sum_ {n = 1} ^ {N} G _ {n} ^ {2}}. \\ \end{array}
$$
Moreover, by limiting the range of  $\beta_{n,k} \in \mathcal{B} \stackrel{\mathrm{def}}{=} \left[\sqrt{\tilde{\epsilon}_0 / a}, \sqrt{(b\hat{D}_n^2 + \epsilon_{\max} + \tilde{\epsilon}_0) / a}\right]$ , we have
$$
| \partial_ {\beta_ {n, k}} f _ {n, k} | = | \kappa_ {n, k} (a - \frac {b \| \phi_ {n , k} - w _ {n , k} \| ^ {2} + \epsilon_ {n , k} + \epsilon_ {0}}{\beta_ {n , k} ^ {2}}) | \leq \frac {a \kappa_ {n} ^ {\max} (b \hat {D} _ {n} ^ {2} + \epsilon_ {n} ^ {\max})}{\tilde {\epsilon} _ {0}},
$$
$$
| \partial_ {\phi_ {n, k}} f _ {n, k} | = \frac {2 b \kappa_ {n , k} \| \phi_ {n , k} - w _ {n , k} \|}{\beta_ {n , k}} \leq \frac {2 b \kappa_ {n} ^ {\max} \hat {D} _ {n}}{\sqrt {\tilde {\epsilon} _ {0} / a}}.
$$
Therefore,  $\| \nabla f_{n,k}\| ^2\leq \frac{a^2(\kappa_n^{\max})^2(b\hat{D}_n^2 + \epsilon_n^{\max})^2}{\tilde{\epsilon}_0^2} +\frac{4ab^2(\kappa_n^{\max})^2\hat{D}_n^2}{\tilde{\epsilon}_0},\forall k\in [M_n].$
Moreover, we let  $G_{n}^{2} = \frac{a^{2}(\kappa_{n}^{\max})^{2}(b\hat{D}_{n}^{2} + \epsilon_{n}^{\max})^{2}}{\tilde{\epsilon}_{0}^{2}} + \frac{4ab^{2}(\kappa_{n}^{\max})^{2}\hat{D}_{n}^{2}}{\tilde{\epsilon}_{0}}$ , replace  $\tilde{D}_n$  with the tighter  $\sqrt{\hat{D}_n^2 + \frac{b\hat{D}_n^2 + \epsilon_n^{\max}}{a}}$  and use  $\tilde{D}_{\mathrm{max}}^2 = (1 + b / a)\hat{D}_{\mathrm{max}}^2 + (\epsilon_{\mathrm{max}} + \tilde{\epsilon}_0) / a$ .
Analogous to the proof of static AER, if  $\epsilon_v \leq bV_n^2 + \epsilon_n$ ,  $\beta_n^* = \sqrt{(bV_n^2 + \epsilon_n + \epsilon_0) / a} \in \mathcal{B}$  can be attained. Then, the above AER can be alternately bounded as:
$$
\begin{array}{l} \mathrm {A E R} _ {\mathcal {A}} ^ {T} \leq \frac {1}{T} \sum_ {n = 1} ^ {N} \left(2 \sqrt {a \left(b V _ {n} ^ {2} + \epsilon_ {n} + \epsilon_ {0}\right)}\right) \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} + \frac {1}{T} \sum_ {n = 1} ^ {N} \sum_ {k = 1} ^ {M _ {n}} \kappa_ {n, k} \Delta_ {n, k} \\ + \frac {3}{2 \tilde {\epsilon} _ {0} T} \sum_ {n = 1} ^ {N} \kappa_ {n} ^ {\max} \left(\sqrt {(\hat {D} _ {n} ^ {2} + \frac {b \hat {D} _ {n} ^ {2} + \epsilon_ {n} ^ {\max}}{a}) (M _ {n} - 1)} \sqrt {a ^ {2} (b \hat {D} _ {n} ^ {2} + \epsilon_ {n} ^ {\max}) ^ {2} + 4 a b ^ {2} \tilde {\epsilon} _ {0} \hat {D} _ {n} ^ {2}}\right) \\ + \frac {1}{T} \sqrt {\frac {(a + b) \hat {D} _ {\max } ^ {2} + \epsilon_ {\max } + \tilde {\epsilon} _ {0}}{a}} \sqrt {2 P ^ {*} \left[ \sum_ {n = 1} ^ {N} \frac {(a ^ {2} (b \hat {D} _ {n} ^ {2} + \epsilon_ {n} ^ {\max }) ^ {2} + 4 a b ^ {2} \tilde {\epsilon} _ {0} \hat {D} _ {n} ^ {2}) (\kappa_ {n} ^ {\max }) ^ {2}}{\tilde {\epsilon} _ {0} ^ {2}} \right]}. \\ \end{array}
$$
Otherwise, if  $\epsilon_v > bV_n^2 +\epsilon_n$ ,  $\beta_{n}^{*} = \sqrt{(bV_{n}^{2} + \epsilon_{n} + \epsilon_{0}) / a} < \sqrt{\tilde{\epsilon}_{0} / a}$  cannot be attained by optimizing within  $\mathcal{B}$ . We can simply upper bound the cost function by replacing  $\epsilon_0$  with  $\tilde{\epsilon}_0$  and obtain newly attainable optimal comparators in hindsight  $\tilde{u}_n^* = (\tilde{\beta}_n^*,\phi_n^*)$ , where  $\beta_{n}^{*} = \sqrt{(bV_{n}^{2} + \epsilon_{n} + \tilde{\epsilon}_{0}) / a}\in \left[\sqrt{\tilde{\epsilon}_{0} / a},\sqrt{(b\hat{D}^{2} + \epsilon_{\max} + \tilde{\epsilon}_{0}) / a}\right]$ . Finally, we can obtain a bound for this case by replacing  $\epsilon_0$  in the first line of the above dynamic AER bound with  $\tilde{\epsilon}_0$ .
E Missing Proofs in Section 6
E.1 Cost function for Gibbs base learner