SlowGuess/NeurIPS_2023_OCR huggingface dataset:
However, this does not work in shifting environments. If we view three slots as one static environment (with large variance), the optimal overall model prior  $\phi^{*}$  will be close to the origin, and the diameter is  $\hat{D}$ , which is much larger than any one of the slot  $\hat{D}_n$ . A larger diameter ( $e.g., \|\phi^* - w_3\| \gg \|\phi_3^* - w_3\|)$  implies the need for more samples for each task to avoid forgetting, which is often not satisfied. A theoretical interpretation is provided in the discussion following Theorem 6.1. To address this, in the next session, we propose an algorithm that dynamically adapts the meta-parameter when environment change occurs so as to rapidly reconstruct meta-knowledge in each static slot. Moreover, it is not as catastrophic as directly adapting model parameters since the forgetting is w.r.t meta-knowledge, which is constrained in a core set of  $\phi_{1:T}^{*}$ , which has a diameter much smaller than  $\hat{D}$ .
4.2 Dynamic algorithm
As discussed in the last section, it would be problematic in a shifting environment if the metalearner adopts the same updating strategy as in the static setting. Specifically, if we aim to optimize the static regret, the learning rate of metaparameter decays with  $\mathcal{O}(1 / t)$  for Follow The Leader (FTL) [41] and  $\mathcal{O}(1 / \sqrt{t})$  for Online Gradient Descent (OGD). Clearly, such strategies hardly learn from new tasks in a shifting environment. On the other side, a constant learning rate of the meta-parameter is often adopted for optimizing the regular dynamic regret [40]. This strategy could perform poorly when the environment only occasionally changes with a huge step, whereas the theoretical optimal learning rate becomes large. A large learning rate causes the metalearner to easily forget the previous tasks, making convergence difficult inside each slot. We offer a theoretical comparison with this in the discussion of Theorem 6.2.
To balance the learning and forgetting tradeoff of meta-knowledge, we proposed the Dynamic Continual Meta-Learning (DCML) algorithm (see Algo. 1) that dynamically adjusts the learning rate of the meta-parameter. When a changing point is detected, the meta-learning
Algorithm 1 Dynamic Continual Meta-Learning (DCML)  
Input: Convex set  $\mathcal{W},\phi_1 = \mathbf{0}\in \mathcal{W},T,\beta_1 > 0$  , initial learning rate  $\{\gamma_0,\rho \}$    
if a pre-trained model  $\phi_0\in \mathcal{W}$  then  $\phi_{1} = \phi_{0}$    
end if  
 $n = 0,k = 0$    
for  $t = 1$  to  $T$  do Sample task distribution:  $\mu_t\sim \tau_t$  . Sample dataset  $S_{t}\sim \mu_{t}^{mt}$  . Learn base parameter  $w_{t} = \mathcal{A}(u_{t},S_{t})$  e.g.,  $\mathcal{A} = K$  step SGD or RLM  $w_{t} = \phi_{t} - \sum_{i = 1}^{K}\beta_{t}\nabla_{\phi_{t}^{i - 1}}\mathcal{L}_{S_{t}}(\phi_{t}^{i - 1}),\phi_{t}^{0} = \phi_{t},$ $\phi_t^i = \phi_t^{i - 1} - \beta_t\nabla_{\phi_t^{i - 1}}\mathcal{L}_{S_t}(\phi_t^{i - 1});$ $w_{t} = \mathrm{argmin}_{w\in \mathcal{W}}\mathcal{L}_{S_{t}}(w) + \frac{1}{\beta_{t}} ||w - \phi_{t}||^{2};$  Estimate excess risk upper bound  $f_{t}(u_{t})$  in Eq. (1); if  $t = 1$  or environment change detected:  $\tau_t\neq \tau_{t - 1}$  then  $M_n = k,n = n + 1,k = 1,\gamma_t = \rho$  else  $k = k + 1,\gamma_t = \gamma_0 / \sqrt{k};$  end if Update meta-parameter:  $\nabla f_t(u_t) = (\kappa_t(a - \frac{b\|\phi_t - w_t\|^2 + \epsilon_t + \epsilon_0}{\beta_t^2}),\frac{2b\kappa_t(\phi_t - w_t)}{\beta_t});$ $u_{t + 1} = \Pi_U(u_t - \gamma_t\nabla f_t(u_t)),i.e.$ $\phi_{t + 1} = (1 - \frac{2b\kappa_t\gamma_t}{\beta_t})\phi_t + \frac{2b\kappa_t\gamma_t}{\beta_t} w_t;$ $\beta_{t + 1} = \beta_{t} - \gamma_{t}(a\kappa_{t} - \frac{\kappa_{t}(b\|\phi_{t} - w_{t}\|^{2} + \epsilon_{t} + \epsilon_{0})}{\beta_{t}^{2}});$  end for
rate  $\gamma_{t}$  adapts to a large hopping learning rate  $\rho$ , then relaunches the decay, in  $\mathcal{O}(1 / \sqrt{k})$ , from  $\gamma_0$  in each slot. Besides, the meta-parameter itself is also adaptively updated. According to  $\phi_{t + 1} = (1 - \frac{2b\kappa_t\gamma_t}{\beta_t})\phi_t + \frac{2b\kappa_t\gamma_t}{\beta_t} w_t$  in Algo. 1, adjusting  $\beta_{t}$  and  $\gamma_{t}$  represent the explicit control on task-level and meta-level trade-off, respectively. A corresponding discussion is provided in Appendix G.1. For a better understanding of DCML, we present a detailed introduction to it using SGD as the base learner in Appendix. G.3
Finally, the changing point can be detected with a broad class of Out-Of-Distribution (OOD) detection methods, e.g., Bayes Online Changing-point Detection (BOCD) [42], loss threshold [13], or setting a fixed-length sliding window. Specifically, when the window size equals 1, a constant learning rate is set to the meta learner, which recovers optimizing the typical dynamic regret as in [40].
We present a general theorem for the proposed DCML framework, which could be deployed for various base learners with the same excess risk upper bound form as Eq. (1).
Theorem 5.1. Consider both static and shifting environments. If the excess risk's upper bound of the base learner  $\mathcal{A}(u_t, S_t)$  can be formulated as a unified form in Eq.(1), then, the AER of DCML (in Algo. 1) is upper bounded by:
$$
A E R _ {\mathcal {A}} ^ {T} \leq \underbrace {\frac {2}{T} \sum_ {n = 1} ^ {N} \sqrt {a (b V _ {n} ^ {2} + \epsilon_ {n} + \epsilon_ {0})} \kappa_ {n} + \frac {\Delta_ {n}}{2}} _ {\text {o p t i m a l t r a d e - o f f i n h i n d s i g h t}} + \underbrace {\frac {3}{2 T} \sum_ {n = 1} ^ {N} \tilde {D} _ {n} G _ {n} \sqrt {M _ {n} - 1}} _ {\text {a v e r a g e r e g r e t o v e r s l o t s}} + \underbrace {\frac {\tilde {D} _ {m a x}}{T} \sqrt {2 P ^ {*} \sum_ {n = 1} ^ {N} G _ {n} ^ {2}}} _ {\text {r e g r e t w . r . t e n v i r o n m e n t s h i f t}}
$$
Here,  $\kappa_{n} = \sum_{k=1}^{M_{n}} \kappa_{n,k}$ ,  $\Delta_{n} = \sum_{k=1}^{M_{n}} \kappa_{n,k} \Delta_{n,k}$ , where the subscript  $n,k$  indicates  $k$ -th task in  $n$ -th slot. The optimal meta-parameter in hindsight of  $n$ -th slot is  $u_{n}^{*} = (\beta_{n}^{*}, \phi_{n}^{*})$  and the path length of  $N$  slots is  $P^{*} = \sum_{n=1}^{N-1} \| u_{n}^{*} - u_{n+1}^{*} \| + 1$ .  $\phi_{n}^{*} = \sum_{k=1}^{M_{n}} \frac{\kappa_{n,k}}{\kappa_{n}} w_{n,k}$  is the weighted average of the base learner outputs within the slot.  $\beta_{n}^{*} = \sqrt{(bV_{n}^{2} + \epsilon_{n} + \epsilon_{0}) / a}$ , where  $V_{n}^{2} = \sum_{k=1}^{M_{n}} \frac{\kappa_{n,k}}{\kappa_{n}} \| \phi_{n}^{*} - w_{n,k} \|_{2}^{2}$  is the variance in  $n$ -th slot and  $\epsilon_{n} = \sum_{k=1}^{M_{n}} \frac{\kappa_{n,k}}{\kappa_{n}} \epsilon_{n,k}$ . Moreover,  $G_{n}$  is the maximal gradient norm of the cost function,  $\tilde{D}_{n}$  is the diameter of meta-parameters in  $n$ -th slot, and  $\tilde{D}_{max} = \max \{\tilde{D}_{n}\}_{n=1}^{N}$ .
The above theorem can be applied to static and shifting environments, where the proof is provided in Appendix D.2. For comparison, we also derive a general theorem from static regret in Appendix D.1.
Important factors for bi-level trade-off (a) The first term in Theorem 5.1 indicates the optimal trade-off can be achieved with the  $N$  optimal meta-parameters, which is affected by the task similarities within each environment (slot variance  $V_{n}^{2}$ ) and the environment changing points. (b) The second term is the average static regret over slots, which is related to the task similarities via the slot diameter  $\tilde{D}_n$  and represents how well the algorithm can balance the task-level trade-off. (c) The last term is the regret w.r.t the environmental shift, reflecting how well the algorithm can address the meta-level trade-off. It is affected by the path length  $P^*$ , determined by the environment similarities and the non-stationarity (how frequently the environment changes).
Remark 5.2. The above theorem has considered different sample numbers for each task. We can see the best initialization (bias) in hindsight  $(\phi_n^*)$  is related to the weighted average of the task models  $w_{n,k}$  in  $n$ -th slot through  $\kappa_{n,k}$ . E.g., for SGD,  $\kappa_{n,k} = \sqrt{1 / K\alpha + 2 / m_{n,k}\alpha} L$  implies that the task trained on more samples will have a smaller weight. Since it's easier to recover model performance for the tasks with more samples, less information on these tasks is conserved in the meta-knowledge.
6 Results on specific base learners
Theorem 5.1 provides general theoretical guarantees without specifying the base leaner. In this section, we discuss the theorem in depth with two typical base learners: SGD within the meta-initialization setting and Gibbs algorithm for the meta-regularization.
6.1 Gibbs algorithm
Theorem 6.1. Let  $\mathcal{W} \subset \mathbb{R}^d$ . Assume that the loss  $\ell(\cdot, z) \in [0,1]$  is  $L$ -Lipschitz  $\forall z \in \mathcal{Z}$  and that  $\mathcal{L}_{\mu_t}(\cdot)$  has  $\alpha$ -quadratic-growth for all  $t \in [T]$ . Let  $w_t = \mathcal{A}_{\text{Gibbs}}(\beta_t, \phi_t, S_t)$  be the output of the Gibbs algorithm (Definition B.2) on  $S_t$ , where  $\phi_t$  is the mean of a prior Gaussian  $\mathcal{N}(\phi_t, \sigma_t^2 \mathbf{1}_d)$  with  $\sigma_t = m_t^{-1/4} d^{-1/4} L^{-1/2}$  and  $\beta_t$  is the inverse temperature. Consider DCML (Algo. 1) and further assume that each slot has equal length  $M$  and each task uses the sample number  $m$ . Then we have
$$
A E R _ {G i b b s} ^ {T} \in \mathcal {O} \left(\left(1 + \frac {1}{N} \sum_ {n = 1} ^ {N} V _ {n} + \frac {\sqrt {M N} + \sqrt {P ^ {*}}}{M \sqrt {N}}\right) \frac {1}{m ^ {\frac {1}{4}}}\right).
$$
The detailed proof is provided in Appendix E.1 and E.3. We also proved in Appendix E.2 that the static AER bound is in  $\mathcal{O}\left(\left((V + 1) + 1 / \sqrt{T}\right)m^{-1 / 4}\right)$ . As we claimed before, when  $N = 1$  (which means the environment is static), the dynamic AER bound recovers the static AER bound.
Benefits of DCML Arbitrarily setting the prior mean  $\phi_t$ , the excess risk for single-task learning with Gibbs algorithm has an upper bound in  $\mathcal{O}\left((D + 1)m^{-1/4}\right)$ , where  $D$  is the diameter of the parameter space. While the static AER becomes  $\mathcal{O}((V + 1)m^{-1/4})$  with rate  $\mathcal{O}(1/\sqrt{T})$ . Since  $V^2$  is the variance of all the outputs of the base learner, we have  $V \leq \hat{D} \leq D$ , which illustrates the benefits of the proposed DCML algorithm. Moreover, in shifting environments, the bound is further improved in Theorem 6.1, where the complexity factor is decreased by  $\frac{1}{N}\sum_{n=1}^{N} V_n \leq V$  with rate  $\mathcal{O}(1/\sqrt{M})$ . This means that by considering environment change, DCML can achieve the same AER with a smaller  $M$  (fewer tasks), i.e., faster-constructing meta-knowledge in new environments.
6.2 Stochastic Gradient Descent (SGD)
Theorem 6.2. Let  $\mathcal{W} \subset \mathbb{R}^d$ . Consider that  $\ell(\cdot, z)$  is convex,  $\beta$ -smooth,  $L$ -Lipschitz  $\forall z \in \mathcal{Z}$ . Assume that  $\mathcal{L}_{\mu_t}(\cdot)$  has  $\alpha$ -quadratic-growth for all  $t \in [T]$ . Let SGD be the base learner for each task where it outputs  $w_t = \mathcal{A}_{SGD}(\eta_t, \phi_t, S_t)$  with learning rate  $\eta_t$  and initialization  $\phi_t$ . Consider DCML (Algo. 1) for the SGD cost function. Further, assume each slot has equal length  $M$ , and each task uses the sample number  $m$  and the same number of updating steps  $K$ . Then we have
$$
A E R _ {S G D} ^ {T} \in \mathcal {O} \left(\left(\frac {1}{N} \sum_ {n = 1} ^ {N} V _ {n} + \frac {\sqrt {M N} + \sqrt {P ^ {*}}}{M \sqrt {N}}\right) \sqrt {\frac {1}{K} + \frac {1}{m}}\right).
$$
The detailed proof is provided in Appendix E.4 and E.6. The CML setting uses offline batch-training for each task, so the related rate of the base learner is  $\mathcal{O}(\sqrt{1 / K + 1 / m})$ , which is determined by both the step number and the sample size. Let  $\bar{V} = \frac{1}{N}\sum_{n = 1}^{N}V_{n}$ . CML focuses on fast learning new tasks and quickly recovers the model performance on past tasks with few-shot examples ( $m$  is small). Hence, the optimal trade-off is dominated by the average deviation over slots  $\bar{V}$  and the path length  $P^*$ , where  $P^*$  reflects the environment similarities and the non-stationarity.
Static environment If the environment is static, i.e.,  $N = 1$ , we have  $P^{*} = 1$ ,  $M = T$ . The bound becomes in  $\mathcal{O}\left((V + \frac{1}{\sqrt{T}})\sqrt{\frac{1}{K} + \frac{1}{m}}\right)$ , which recovers the static AER bound in Appendix E.5.
Shifting environment When the environment occasionally changes with a large step, as discussed in Sec 4.2,  $N$  is small, and  $M$ ,  $P^{*}$  is large. The proposed bound has an improved rate  $\mathcal{O}(1 / \sqrt{M})$  on  $P^{*}$  compared to  $\mathcal{O}(\bar{V} + \frac{1}{\sqrt{M}} + \sqrt{\frac{P^{*}}{NM}})$  - an equivalent form of the task average regret bound in [31]. A more detailed comparison w.r.t [31] has been discussed in the proof for both static and dynamic AER. Similar to Theorem 6.1, the dynamic AER is improved w.r.t. the static AER with the improvement on complexity factor by  $\bar{V} \leq V$ . In addition, if the environments differ a lot and frequently change,  $N$ ,  $P^{*}$  is large, and  $M$  is small. It's impossible to obtain a small AER, as demonstrated in the experiment on the Synbols dataset in Tab. 1.
We first conduct analytic experiments on a synthetic dataset to offer an in-depth understanding of the theoretical counterpart. Then, we test the proposed algorithm on a recent CML benchmark - OSAKA [13], which empirically considers shifting environments on a large scale. The experimental results validate the proposed theory and illustrate the superiority of DCML in shifting environments.
7.1 Moving 2D Gaussian
We introduce a simple synthetic dataset, where the data is generated by the following strategy. Let the environment be a 2D Gaussian distribution with a moving mean. Initially,  $\tau_0 = \mathcal{N}(\tilde{\phi}_0^*,\mathbb{I}_2)$  with  $\tilde{\phi}_0^* = (-6,6)^T$ . Then at time point  $t$ ,  $\tau_{t} = \mathcal{N}(\tilde{\phi}_{t}^{*},\mathbb{I}_{2})$  is updated by:  $\tilde{\phi}_t^* = \tilde{\phi}_{t - 1}^* +(1,1)$  with probability  $p$  and  $\tilde{\phi}_t^* = \tilde{\phi}_{t - 1}^*$  with probability  $1 - p$ . Then we have  $N$  changing points  $\{t_n\}_{n = 1}^N$ . At each time step, we first sample  $w_{t}^{*}\sim \tau_{t}$  from the current environment  $\tau_{t}$  as the mean of the task distribution  $\mu_t = \mathcal{N}(w_t^*,0.1\mathbb{I}_2)$ , which is also a 2D Gaussian. For task  $t$ , we sample  $m$  examples  $S_{t}^{tr}\sim \mu_{t}^{m}$  as train data,  $m^{\prime}$  samples  $S_{t}^{te}\sim \mu_{t}^{m^{\prime}}$  as test data. Then the corresponding empirical risk is
Figure 3: Moving 2D Gaussian mean estimation: (a) tracking example, (b) visualization of test loss, static and dynamic AER, learning rates of base and meta learners.
Figure 4: Average test acc at final step  $t = 10000$  w.r.t environment change probability  $p$  on the OMF dataset.
$\mathcal{L}_{S_t^{tr}}(w) = \frac{1}{m}\sum_{i = 1}^m\| w - x_i\|^2$  . SGD uses  $\phi_t$  as initialization and a learning rate  $\eta_t$  to optimize this objective, and outputs  $w_{t} = \mathcal{A}_{\mathrm{SGD}}(\eta_{t},\phi_{t},S_{t}^{tr})$  . Since  $w_{t}$  is obtained from limited data, it can be far away from  $w_{t}^{*}$  . If the environment shifts to the  $n$  -th slot at  $t_n$  , then the predicted environment mean  $\hat{\phi}_n^* = \phi_{t_n}$  . We denote  $\tilde{\phi}_n^* = \tilde{\phi}_{t_n}^*$  the mean of the actual environment, and the hindsight environment mean of  $n$  -th slot as  $\phi_n^* = \frac{1}{t_{n + 1} - t_n}\sum_{t = t_n}^{t_{n + 1} - 1}w_t$
In Fig. 3 (a), we visualize an example of tracking the environment shifts, where the task mean estimation is  $w_{t}$ , the output of SGD. The other three terms are as described above. We further visualize the test loss, the dynamic, and the static AER bounds in Fig. 3 (b). The red dotted lines represent the actual environment-changing points. We observe that the static AER is much larger than the dynamic AER, consistent with the result in Theorem 6.2. In addition, the evolving trend of dynamic AER is the same as the average test loss, which does not hold for static AER. We also find that the learning rate of the initialization  $\phi_t$  (meta lr in Fig.3 (b)) is scheduled by the changing points, and the base learning rate is adjusted automatically by the algorithm. Additional experimental settings and results are provided in Appendix F.
7.2 OSAKA benchmark
Experimental set-up OSAKA considers a task-agnostic setting of unknown task boundaries and simultaneously switches the task and environment with low probabilities. We slightly modify the setting to introduce more non-stationarity by always switching to a new task and changing the environment with probability  $p$  at each timestep. We pre-train a meta-model as initialization from one environment. The new environment is selected with probability 0.5 as the pre-trained one and 0.25 for each of the two unseen environments, consisting of OOD tasks. We further test DCML on two representative datasets in OSAKA. For Synbols [43] dataset, the model is pre-trained to classify characters from an alphabet on randomized backgrounds. During CML time, the model is exposed to the pre-trained environment, a new alphabet for character classification, and a heterogeneous environment that conducts font classification, which induces significant concept shift. For Omniglot-MNIST-FashionMNIST(OMF), we pre-train the meta-model on the first 1000 classes of Omniglot [44]. Then, it is exposed to the full Omniglot dataset and two unseen datasets - MNIST [45] and FashionMNIST [46] at CML time. More details and baselines are deferred to Appendix F.1.1.
We tested 5-level of non-stationarity with  $p = \{0.2, 0.4, 0.6, 0.8, 1.0\}$  for OMF and for Symbols we set  $p = \{0.2, 0.4, 1.0\}$ . The average test accuracy of all encountered tasks is used as the performance metric, which reflects the AER. In addition, we denote DCML (oracle) for the algorithm as offering correct environment-changing points and DCML (window) as using a fixed-length sliding window.
Results The average test accuracies at the last timestep for all environments and every single environment are reported in Tab.1 for OMF with  $p = 0.4$  and Synbols with  $p = 0.2$ . Additional experimental results can be found in Appendix F.2.2. Clearly, DCML constantly outperforms all the baselines with improvements of  $1 \sim 2\%$  points on OMF data and  $0.7\%$  points on Synbols data compared to the best baseline method. A comparison w.r.t powerful baselines is presented in Fig. 5 for OMF data, where the average test accuracy over rounds in three environments is visualized separately.
Table 1: Average test accuracy (%) on OMF & Synbols datasets of OSAKA benchmark
OMF, p = 0.4	SYNBOLS, p = 0.2
MODEL	ALL ENV.	OMNIGLOT	MNIST	FMNIST	ALL ENV.	ALPHA.	NEW ALPHA.	FONT
FINE TUNING	10.8±1.5	11.2±2.2	10.4±0.8	10.5±0.8	25.3±0.7	25.4±0.9	25.1±0.4	25.2±0.5
METACOG [8]	40.9±0.9	54.4±1.2	25.6±0.9	28.9±0.4	25.3±0.8	25.7±0.9	25.0±0.8	25.1±1.0
METABGD [8]	54.5±7.1	70.5±8.5	40.2±5.5	36.8±5.8	29.9±6.9	31.8±9.6	28.1±4.8	27.5±3.6
ANIL [47]	83.2±0.3	99.1±0.1	73.9±0.1	60.2±0.1	60.5±0.4	77.6±0.6	53.6±0.2	32.7±0.4
MAML [4]	83.8±0.4	99.3±0.1	75.4±0.1	61.0±0.1	76.7±0.4	96.8±0.1	73.1±0.1	40.5±0.7
CMAML [13]	85.9±0.7	98.1±0.3	83.8±1.8	64.2±0.9	61.9±2.5	76.2±2.1	56.8±3.2	37.9±2.4
DCML(ORACLE)	86.5±0.7	97.4±0.6	85.7±2.6	65.9±1.4	77.2±0.5	96.0±0.3	73.4±0.1	42.4±0.6
DCML(WINDOW)	86.9±0.7	97.2±0.7	86.8±2.0	66.7±1.7	77.4±0.4	96.2±0.2	73.8±0.2	42.4±0.7
(b)
Figure 5: Average test accuracy on (a) Omniglot, the pre-trained environment, (b) FashionMNIST, and (c) MNIST, the two unseen environments where the environment shifts with probability  $p = 0.2$ .
Analysis of learning and forgetting In Fig. 5 for OMF data, MAML and ANIL do not suffer forgetting since they never update meta-parameters. The proposed algorithm forgets a bit more than CMAML but learns much faster in new environments, illustrating a better trade-off. The fine-grained adjustment for the meta-learning rate makes it possible to quickly reconstruct meta-knowledge w.r.t environment change and constantly learn in new environments. On the other hand, MetaBGD and MetaCOG perform poorly since using BGD hinders acquiring new knowledge. All the methods perform worse on Synbols data in Tab. 1, where each environment is more diverse than OMF, and the environments differ a lot, indicating a large  $\hat{D}_n$  and  $P^*$ . Moreover, we observe a performance drop of CMAML on Synbols. Since CMAML uses empirical loss for modulation, it is prone to overfitting and can cause difficulty of convergence inside each slot when  $\hat{D}_n$  is large, as discussed in Sec. 4.1.
Analysis of stability w.r.t shifting probability We conduct an ablation study on OMF w.r.t environment change probability in Fig. 4. The results demonstrate the stability of the proposed algorithm and the increase in performance w.r.t  $p$ . Since the environments have high similarities in OMF, the meta-model is better learned by seeing more tasks given the fixed time steps with a larger  $p$ .
Analysis of environment change detection Empirical results suggest that DCML (window) performs better than DCML (oracle). This is reasonable when slots of small length exist where adapting to new environments causes additional context-switch costs (justified in Appendix F.2.1). Given the changing probability  $p$ , the expected slot length is  $1 / p$ , used as the sliding window size. In practice, even though  $p$  is unknown, hyper-parameter searching for the one-dimensional window size is much easier. Because the similarity between any two environments can differ a lot, it's hard to find a single optimal hyper-parameter for change detection methods like BOCD or loss threshold.
This paper theoretically studies the stability and plasticity dilemma in CML. Based on the novel AER objective, it proposes a continual meta-learning framework in both static and shifting environments. The proposed DCML algorithm can quickly reconstruct meta-knowledge to alleviate forgetting and quickly adapt to new environments when change occurs. The corresponding theory provides tighter bounds and more flexible base learner selections. In addition, adaptively learning the meta-parameter can facilitate the training process in deep learning. Empirical evaluations on both synthetic and real datasets demonstrate the superiority of the proposed method.
We appreciate constructive feedback from anonymous reviewers and meta-reviewers. This work is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant, the Collaborative Research and Development Grant from SSQ Assurances and NSERC, and the China Scholarship Council.