SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathrm {A E R} _ {\mathrm {S G D}} ^ {T} \leq \left(\frac {1}{N} \sum_ {n = 1} ^ {N} \sqrt {2 (2 \alpha V _ {n} ^ {2} + 2 (\frac {1}{K \alpha} + \frac {2}{m \alpha}) L ^ {2} + \epsilon_ {0})}\right) \sqrt {\frac {1}{K \alpha} + \frac {2}{m \alpha}} L + 2 (\frac {1}{K \alpha} + \frac {2}{m \alpha}) L ^ {2} \\ + \frac {3}{2 \tilde {\epsilon} _ {0} M} \sqrt {\frac {1}{K \alpha} + \frac {2}{m \alpha}} L \sqrt {M - 1} \times \\ \frac {1}{N} \sum_ {n = 1} ^ {N} \left(\sqrt {(1 + 4 \alpha) \hat {D} _ {n} ^ {2} + (\frac {4}{K \alpha} + \frac {8}{m \alpha}) L ^ {2}} \sqrt {(\alpha \hat {D} _ {n} ^ {2} + (\frac {1}{K \alpha} + \frac {2}{m \alpha}) L ^ {2}) ^ {2} + 8 \alpha^ {2} \tilde {\epsilon} _ {0} \hat {D} _ {n} ^ {2}}\right) \\ + \frac {1}{M} \sqrt {(1 + 4 \alpha) \hat {D} _ {\max } ^ {2} + 4 L ^ {2} (\frac {1}{K \alpha} + \frac {2}{m \alpha}) + 2 \tilde {\epsilon} _ {0}} \times \sqrt {\frac {1}{K \alpha} + \frac {2}{m \alpha}} L \\ \times \sqrt {\frac {2 P ^ {*}}{N} \frac {1}{N} \sum_ {n = 1} ^ {N} \left[ \frac {(\alpha \hat {D} _ {n} ^ {2} + (\frac {1}{K \alpha} + \frac {2}{m \alpha}) L ^ {2}) ^ {2}}{\tilde {\epsilon} _ {0} ^ {2}} + \frac {8 \alpha^ {2} \hat {D} _ {n} ^ {2}}{\tilde {\epsilon} _ {0}} \right]}. \\ \end{array}
$$
which implies  $\tilde{\epsilon}_0 \geq a\kappa\epsilon / L'$ , where  $a\kappa\epsilon / L' \in \mathcal{O}(m^{-3/2} + K^{-3/2})$ . So it's sufficient to ensure  $\tilde{\epsilon}_0 \in \Omega(m^{-3/2} + K^{-3/2})$ .
For SGD, by the law of large numbers, we have  $\epsilon_0 = 2\alpha \| w_t - \mathbb{E}_{S_t,\mathcal{A}}W_t\|^2 \to 2\alpha \| w_t - \mathbb{E}_\mathcal{A}W_t\|^2$  as  $m \to \infty$ . In addition,  $2\alpha \| w_t - \mathbb{E}_\mathcal{A}W_t\|^2 \to 0$  as  $K \to \infty$ , so we have  $\epsilon_0 \in o_m(1) + o_K(1)$ .
Then, if we set  $\epsilon_v = bV_n^2 < bV_n^2 + \epsilon$  s.t.  $\tilde{\epsilon}_0 = \epsilon_0 + \epsilon_v$  satisfies the above condition, we have
$$
\mathrm {A E R} _ {\mathrm {S G D}} ^ {T} \leq \mathcal {O} \left(\left(\frac {1}{N} \sum_ {n = 1} ^ {N} V _ {n} + o _ {m} (1) + o _ {K} (1) + \frac {\frac {1}{\epsilon_ {0} + V ^ {2}}}{\sqrt {M}} + \frac {\frac {1}{\epsilon_ {0} + V ^ {2}} \sqrt {\frac {P ^ {*}}{N}}}{M}\right) \sqrt {\frac {1}{K} + \frac {1}{m}}\right).
$$
The above bound has additional  $o_m(1) + o_T(1)$  compared to Theorems in [31]. Since the variance  $V_n^2$  (task similarity) in this paper is calculated via the average SGD iterates. A similar term will be introduced using the same algorithm outputs for estimating  $V$  in [31]. Moreover, our bound is in terms of average task excess risk, while [31] upper bounds the average task regret for online base learners using a different meta-learning algorithm.
Considering a constant variance, we have
$$
\mathbf {A E R} _ {\mathrm {S G D}} ^ {T} \in \mathcal {O} \left(\left(\frac {1}{N} \sum_ {n = 1} ^ {N} V _ {n} + \frac {1}{\sqrt {M}} + \frac {\sqrt {P ^ {*}}}{M \sqrt {N}}\right) \sqrt {\frac {1}{K} + \frac {1}{m}}\right).
$$
When we have a large  $P^*$  with a small  $N$  (the environment occasionally changes with a large shift), the above bound has an improved rate of  $\mathcal{O}(1 / \sqrt{M})$  or  $\sqrt{P^*}$  on the path length  $P^*$  (reflecting
environment similarities and shifts) compared to  $\mathcal{O}\left(\frac{1}{N}\sum_{n = 1}^{N}V_n + \frac{1}{\sqrt{M}} +\min \{\sqrt{\frac{P^*}{MN}},\frac{P^*}{NM}\}\right)$  the equivalent form of Theorem 3.3 [31] in this case.
If we have  $V_{n} \to 0$ , we can set  $\epsilon_v = bV_n^2 + \epsilon + 1 / M^{1/4} > bV_n^2 + \epsilon$ , s.t.  $\tilde{\epsilon}_0 = \epsilon_0 + \epsilon_v$  satisfies the aforementioned condition. In this case, we have:
$$
\begin{array}{l} \mathcal {O} \left(\left(\frac {1}{N} \sum_ {n = 1} ^ {N} V _ {n} + o _ {m} (1) + o _ {K} (1) + \frac {1 + \frac {1}{\epsilon_ {0} + V ^ {2} + 1 / M ^ {1 / 4}}}{\sqrt {M}} + \frac {\frac {1}{\epsilon_ {0} + V ^ {2} + 1 / M ^ {1 / 4}} \sqrt {\frac {P ^ {*}}{N}}}{M}\right) \sqrt {\frac {1}{K} + \frac {1}{m}}\right) \\ = \mathcal {O} \left(\left(\frac {1}{\sqrt [ 4 ]{M}} \left(1 + \sqrt {\frac {P ^ {*}}{N M}}\right) + o _ {m} (1) + o _ {K} (1)\right) \sqrt {\frac {1}{K} + \frac {1}{m}}\right), \text {w h e n} V _ {n} \rightarrow 0. \\ \end{array}
$$
The above bound has an improvement of rate  $\mathcal{O}(1 / \sqrt[4]{M})$  on  $P^{*}$  as  $V_{n} \to 0$  compared to the equivalent term  $\mathcal{O}\left(\frac{1}{\sqrt[4]{M}} + \sqrt{\frac{P^{*}}{NM}} + o_{m}(1) + o_{K}(1)\right)$  of Theorem 3.3 in [31].
In this section, we provide experimental settings, some additional experimental results, and experimental details. The code is available in https://github.com/livreQ/DynamicCML.
F.1 Experimental settings
Figure 6: Summary of experimental settings
F.1.1 Detailed setting in OSAKA
Task data generating process Following OSAKA, we use a pre-trained meta-model as initialization. The datasets in OSAKA contain three environments. One is used for pre-training, and the other two are considered unseen, consisting of OOD tasks compared to the pre-trained environment. We run 10 CML episodes for each method, where each episode has a length of  $T = 10000$  time steps.
At each time step, we choose to change to another environment with probability  $p$ . We select the new environment with a probability of 0.5 for the pre-trained environment and 0.25 for one of the other two. Then, a task is sampled from the selected environment. The above generation process is slightly different from the original OSAKA, which mainly considers a task-agnostic setting (i.e., unknown task boundary). They used a single probability for switching the task and environment simultaneously. In contrast, we focus on shifting environments under the task-aware setting, where the task switches with probability 1, and the environment changes with the process described above. A visualization of the data-generating process is presented in Fig. 6.
Datasets We test DCML on two representative datasets in OSAKA.
- The Symbols [43] dataset uses characters from different alphabets on randomized backgrounds as the pre-training environment. A new alphabet and font classification tasks are considered unseen environments. This is a large and complex dataset including a heterogeneous environment (font) with extremely large concept shift w.r.t other environments. We conduct a 4-way 5-shot classification for this dataset.
- For Omniglot-MNIST-FashionMNIST(OMF), we pre-train the meta-model on the first 1000 classes of Omniglot [44]. Then, at CML time, the models are updated on a stream of tasks generated by the process discussed in the previous section exposed to the full Omniglot dataset and two OOD datasets - MNIST [45] and FashionMNIST [46]. In this study, we conduct 10-way 5-shot classification as the benchmark does.
Table 3: Comparison of computational complexities with baselines
Methods	Computational complexity
Fine-tuning	O(T*K)
MetaCOG	O(T*(K+M))
MetaBGD	O(T*(K+M))
ANIL	O(T*K)
MAML	O(T*K)
CMAML	O(T*(K+1))
DCML	O(T*(K+1))
Baselines We compared the proposed algorithm with different baselines. For a fair comparison, we further use the same meta-model pre-trained with MAML [4] for all the methods. During the continual learning phase, all the methods adapt with few-shot data and are then evaluated on separate test data for each task.
- ANIL is a variation of MAML that only adapts the network's head w.r.t new tasks. Following OSAKA, MAML, and ANIL do not update the meta-model during the CL phase. So they will not suffer from forgetting but lack the plasticity to learn from tasks in new environments. They are compared to show the problem with static representations in shifting environments.
- CMAML [13] includes an update modulation phase and a prolonged adaptation phase. The former uses a function  $g_{\lambda} : \mathbb{R} \to (0,1)$  to modulate the learning rate of the meta-model proportionally to the loss value. The latter updates the meta-model with buffered tasks when the environment changes or a task switch is detected. Otherwise, it keeps updating w.r.t the previous model. When  $\lambda = 0$ , CMAML is equivalent to MAML, which never updates the meta-model. When  $\lambda \to \infty$ , CMAML updates the meta-model with a constant learning rate.
MetaBGD and MetaCOG [8] perform CML based on MAML and Bayesian Gradient Descent (BGD), where Meta-COG introduced a per-parameter mask.
- Fine-tuning uses the meta-model as initialization and consistently updates this model w.r.t the tasks encountered, which can be considered a lower bound in the setting to illustrate catastrophic forgetting.
Remark F.1. Two modifications w.r.t original OSAKA benchmark: (1) We consider a task-aware setting, where the task constantly shifts at each timestep. (2) The cumulative performance in the original OSAKA is evaluated on the model before updating to current task data, which does not use the support query splits. Instead, we follow the CML setting and evaluate all the baseline algorithms on the model after updating to the current task.
Computational Complexity The computational complexities of all the methods are provided in Tab. 3, where the run time is measured in terms of the number of gradient computations. All the methods are assumed to use a  $K$ -step SGD as the inner algorithm.
- Fine-tuning uses the meta-model as initialization and consistently updates this model w.r.t the tasks encountered. MAML and ANIL do not update the meta-model during the learning phase. Therefore, these methods need to compute  $T * K$  gradients for the  $T$  encountered tasks.
MetaBGD and MetaCOG perform CML based on MAML and use Bayesian Gradient Descent (BGD) for meta-model adaptation, which requires  $M$  Monte Carlo samplings for the meta-gradient computation. Hence, the computation complexity is  $\mathcal{O}(T*(K + M))$ .
- DCML and CMAML need one meta-gradient and  $K$  inner gradients computations for each task, so the complexity is  $\mathcal{O}(T * (K + 1))$ .
(a) Average test loss w.r.t changing probability  $p$ .
(b) Bayes online changing-point detection
Figure 7: Additional experiments for moving 2D Gaussian
F.2 Additional results
F.2.1 Moving 2D Gaussian
Comparison of different environment shift detection methods In Fig. 7 (a), we plot the average test loss at  $t = 200$  w.r.t the environment changing probability  $p$  in log scale, where we set the window size to 10 for "window". We can see that the "window" performs similarly to the "oracle" when the environment changes occur not too frequently. "Static" shows the worst result, then the "detect ." We simply (without carefully adjusting the hyper-parameters) adapted the Bayes Online Changing-point Detection (BOCD) to the algorithm output, which, in fact, highly depends on the performance of the base learner. Fig. 7 (b) illustrates the detection results of BOCD, where the green lines are the detection results and the red lines are the actual changing points. The upper figure shows one of the dimensions of the algorithm output  $w_{t}$  with blue points. The lower figure illustrates the run-length posterior at each time step using a logarithmic color scale, where darker indicates a higher probability. We can see that with the current hyper-parameter setting, detection results are delayed compared to the actual changing points. BOCD is more likely to miss detecting some changing points rather than conducting false detection. The pseudo-code for BOCD is provided in Algo. 3.
Not always a necessity for the exact detection of changing points We observed a better performance of "window" than "oracle" for the OSAKA dataset in the main paper, which implies that the exact changing points detection may not be necessary.
This result is related to the algorithm, which does not necessarily hold for other methods that maintain multiple meta-models. By updating a single meta-model online, the context switch cost exists for reconstructing the meta-knowledge in each slot. Setting a fixed window size can ensure the meta-knowledge quality of each slot and yield good performance on average.
In addition, the phenomenon is related to the overlap between the consecutive environment distributions and the environment shift probability  $p$ . We empirically tested these factors, and the result is presented in Fig. 8. In Fig. 8, "oracle" is better than "window" when  $p < 0.1$ , and the gap becomes more evident when the distribution overlap is smaller.
Remark F.2. A larger overlap between distributions indicates that they are more similar, which can provide a better transfer performance (smaller test loss in Fig. 8 (a) than in (b)). In this case, the two distributions are close, so it's hard to determine precisely which distribution the encountered task belongs to. When two distributions are well-separated, it's easy to detect the shift correctly, but the transfer error is larger.
F.2.2 OSAKA benchmark
We present in Tab. 6 additional results for the Symbols dataset with the environment changing probability  $p = 0.4$  and  $p = 1.0$ . We can see DCML performs much better than baselines on new environments, especially the hardest one - Font.
(b)
Figure 8: Justification of "Oracle" and "Window" (window size  $= 1 / p$ ) methods on the Moving 2D Gaussian mean estimation: (a) Consecutive environment distributions with a large overlap, where  $\tau_{t} = \mathcal{N}(\tilde{\phi}_{t}^{*},\mathrm{I}_{2})$ , and  $\tilde{\phi}_t^* = \tilde{\phi}_{t - 1}^* +(1,1)$  with probability of  $p$ . (b) Consecutive environment distributions with a small overlap, where  $\tau_{t} = \mathcal{N}(\tilde{\phi}_{t}^{*},0.1\mathrm{I}_{2})$ , and  $\tilde{\phi}_t^* = \tilde{\phi}_{t - 1}^* +(2,2)$  with probability of  $p$ .
avg_test_acc_s_o_far
(a)
avg_test_acc(so_far_omniglot
(b)
avg_test_acc(so_far_fasion_mnist
(c)
Figure 9: Average test accuracy on (a) all environments (b) Omniglot, the pre-trained environment (c) Fashion MNIST, the unseen environment (d) MNIST, the unseen environment when the environment changes with probability  $p = 0.8$ .
avg_test_acc_s_o_far_mnist
(d)
In addition, we provide the experimental records in wandb for the OMF dataset when the environment changes with probability  $p = 0.8$ . The average test accuracies w.r.t timesteps in the CL phase are plotted in Fig. 9.
Table 4: Average test accuracy (%) at  $t = {10000}$  on Synbols datasets of OSAKA benchmark
SYNBOLS, p = 0.4	SYNBOLS, p = 1.0
MODEL	ALL ENV.	ALPHA.	NEW ALPHA.	FONT	TOTAL	ALPHA.	NEW ALPHA.	FONT
FINE TUNING	25.3±0.8	25.4±1.2	25.2±0.5	25.1±0.3	25.2±0.7	25.4±1.0	25.2±0.5	25.1±0.4
METACOG [8]	25.1±0.9	25.1±1.3	25.0±0.7	25.0±0.3	25.0±0.8	25.7±0.9	25.0±0.8	25.0±1.0
METABGD [8]	29.2±7.5	30.9±10.8	27.8±5.0	27.0±3.2	28.6±6.7	30.2±9.6	27.4±4.6	26.6±3.2
ANIL [47]	60.6±0.3	77.5±0.1	53.4±0.2	32.6±0.2	60.4±0.1	77.6±0.1	53.7±0.1	32.8±0.2
MAML [4]	77.0±0.3	96.8±0.1	72.9±0.2	40.3±0.3	76.8±0.1	96.8±0.2	73.0±0.2	40.2±0.4
CMAML [13]	64.9±1.5	79.0±3.8	60.2±0.6	40.3±1.4	63.1±1.1	77.0±3.1	58.6±0.8	40.1±1.6
DCML(ORACLE)	77.1±0.6	95.8±0.2	73.2±0.3	42.3±0.4	77.1±0.1	95.7±0.5	73.1±0.4	43.4±1.2
DCML(WINDOW)	77.2±0.5	96.0±0.3	73.4±0.4	42.2±0.8	77.2±0.2	95.6±0.6	73.2±0.5	44.2±1.6
F.3 Experimental details
F.3.1 Moving 2D Gaussian
Model As we described in the main paper, the hypothesis space of the moving 2D Gaussian is  $\mathbb{R}^2$ . And we adopt mean square loss  $\| w - x\|^2$ , which is 2-strongly convex, so it's 2-quadratic growth with  $\alpha = 2$ . Moreover, we limited the data to  $[-10, 10] \times [-10, 10]$ . Thus, for the given loss function, the Lipsshitz constant is  $L = 20\sqrt{2}$ , and we set  $L = 30$  in the code.
Training Details The hyper parameter settings and training details for Moving 2D Gaussian are presented in Table 5. We can see with the Algorithm 1 provided in the main paper. There is no need to set the hyper-parameters for the base learner. They are automatically adapted to the derived equations. Moreover, the initial learning rate of the meta learner is controlled by D_hat Guess. Then, it's also scheduled automatically with time and environment changing points.
Table 5: Moving 2D Gaussian experimental details
Hyper-parameters	Fig. 3(b)	Fig. 7(a)
task horizon T	200
loss	mean square loss
Lipschitz constant L	30
α	2
hopping lr ρ	0.8
D_hat_guess	5
sample numbers m	100	100
inner epoch K	1	2
changing prob p	0.05	[0.025, 0.063, 0.158, 0.398, 0.631, 1]
Computing Resource All experiments for Moving 2D Gaussian were tested on a Mac with an Intel Core i5 CPU and 8G memory.
F.3.2 OSAKA benchmark
Model for OMF dataset We used a CNN network architecture as the basic model. The four modules are identical: A  $3 \times 3$  2d convolution layer with 64 filters, stride 1, and padding 1. The following are a batch normalization layer, a Relu layer, and a max-pooling layer of stride 2. The image data passing the aforementioned modules form a  $64 \times 1 \times 1$  feature map, which was further taken into a fully connected layer. Then the fully connected layer outputs the logits for a 10-class classification. At last, the cross-entropy loss is calculated with the logits and the corresponding labels.
Model for Symbols dataset We used a CNN network architecture as the basic model. The four modules are identical: A  $3 \times 3$  2d convolution layer with 64 filters, stride 1, and padding 1. The following are a batch normalization layer, a Relu layer, and a max-pooling layer of stride 2. The image data passing the aforementioned modules form a  $64 \times 2 \times 2$  feature map, which was further flattened and taken into a fully connected layer. Then, the fully connected layer outputs the logits for a 4-class classification. At last, the cross-entropy loss is calculated with the logits and the corresponding labels.
Training Details The hyper-parameter settings and training details for OSAKA data sets are outlined in Table 6.
Computing Resource The experiments for OSAKA were run on a server node with 6 CPUs and 1 GPU of 32GB memory.
Table 6: OSAKA experimental details
Hyper-parameters	OMF	Symbols
task horizon T	10000
loss	cross entropy
α	4
D_hat_guess	100
image size	28*28	32*32
image channel	1	3
n_shots	5	5
n_shots_test	15	15
n_ways	10	4
prob_statio	0.0	0.0
sample number m=n_shots * n_ways	50	20
inner epoch K	8	16
prob_env_SWITCH p	[0.2, 0.4, 0.6, 0.8, 1.0]	[0.2, 0.4, 1.0]
Lipschitz constant L	200	100
hopping lr ρ	1.0	0.2
eta_0	6.0	20
epsilon_0_rate	1.0	4.0
Hyper-parameter Search Following OSAKA, the hyper-parameters were tuned by random search, and the same number of trials were allocated for each algorithm. For each trial, we sampled hyperparameter combinations uniformly from the search space presented in Tab. 7 for baselines and in Tab. 8 for DCML.
According to the experimental setting of OSAKA, we do not need a validation dataset for searching the hyper-parameters. New CML episodes (task sequences) generated with different random seeds can be considered held-out data. The results reported in the paper are tested on 10 new CML episodes using the learned hyper-parameters.
Table 7: Hyper-parameter search space for baselines
Hyper-parameters	Search space
meta-learning rate η	0.0001, 0.0005, 0.001, 0.005, 0.01
base learning rate	0.0005, 0.001, 0.005, 0.01, 0.05, 0.1, 0.5
inner steps	1, 2, 4, 8, 16
first-order	True, False
modulation λ	0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 2.0, 2.5, 3.0
parameter variance σ	0.0001, 0.01, 0.1
step-size for parameter mean β	0.5, 1.0, 10
MC samples	5
threshold for task switch γ	-2
F.4 Bayes Online Changing-point Detection
We modify the implementation of BOCD in [52] for our experiments. The detailed method is presented in [42]. Given a sequence of tasks, we use the base leaner outputs  $w_{1:t}$  as observations of BOCD to be divided into non-overlapping product partitions. Each partition can be considered as sampled from the same static environment slot. BOCD model the detection of changing point as estimating the posterior distribution over the current run-length  $r_t \in [0, 1, \dots, t]$ , i.e.,  $P(r_t | w_{1:t}) = \frac{P(r_t, w_{1:t})}{P(w_{1:t})}$ . Thus,  $r_t = 0$  means to start a new partition.
We treat the hypothesis as factorized Gaussian and apply BOCD on each dimension of the algorithm output  $w_{t}$ . For Moving 2D Gaussian, we detect the whole  $w_{t}$ , since it's just 2D. Here, we provide the pseudo-code in Algorithm 3 with Gaussian Prior for each dimension applied in our experiments.
Table 8: Hyper-parameter search space for DCML
Hyper-parameters	Search space
quadratic growth parameter α	0.05, 0.5, 1, 2, 4, 6, 8, 10
Lipschitz constant L	50, 100, 150, 200, 250, 300, 350, 400
number of inner steps K	1, 2, 4, 8, 16
slot diameter guessˆD	50, 100, 200, 400, 600
hopping learning rate ρ	0.2, 0.4, 0.6, 0.8, 1.0
initialization β1	2, 4, 6, 8, 10, 12, 14, 16, 18, 20
initialization of ε0	1, 2, 4, 6, 8, 10
Require:  $P(r_0 = 0) = 1$ , prior mean  $\mu_1^0 = 0$ , prior precision  $\delta_1^0 = 1 / 0.1$ , changing probability  $p$ , data precision  $\delta_w = 1$
Algorithm 3 Pseudo code for detecting environment change
for  $t\gets 1$  to  $T$  do
end
observe new algorithm output  $w_{t}$    
evaluate the predictive probability  $\pi_t^r = P(w_t|\mu_t^r,\delta_t^r)$    
calculate the growth probability  $P(r_{t} = r_{t - 1} + 1,w_{1:t}) = P(r_{t - 1},w_{1:t - 1})\pi_t^r (1 - p)$    
calculate the changepoint probability  $P(r_{t} = 0,w_{1:t}) = \sum_{r_{t - 1}}P(r_{t - 1},w_{1:t - 1})\pi_t^r p$    
calculate evidence  $P(w_{1:t}) = \sum_{r_t}P(r_t,w_{1:t})$    
update run length distribution  $P(r_{t}|w_{1:t}) = P(r_{t},w_{1:t}) / P(w_{1:t})$    
update statistics:   
 $\delta_{t + 1}^{r} = \delta_{t}^{r},\delta_{t + 1}^{r + 1} = \delta_{t}^{r} + \delta_{w}$ $\mu_{t + 1}^{r} = \mu_{t}^{r},\mu_{t + 1}^{r + 1} = 1 / \delta_{t + 1}^{r + 1}*\left(\delta_{t}^{r}\mu_{t}^{r} + w_{t}\delta_{w}\right)$    
if arg max  $kP(r_t = k|w_{1:t}) <   \arg \max_kP(r_{t - 1} = k|w_{1:t - 1})$  then   
| changing point detected   
end
G Additional Discussion
G.1 Learning-forgetting w.r.t meta-updates
In this section, we offer some intuitive interpretations for the adaptive updates of the meta-parameter of the base learner.
The updates of the meta-parameter  $(\phi$  and  $\beta)$  in Algo. 1 and 2 can be separately expressed as:
$$
\phi_ {t + 1} = (1 - \frac {2 b \kappa_ {t} \gamma_ {t}}{\beta_ {t}}) \phi_ {t} + \frac {2 b \kappa_ {t} \gamma_ {t}}{\beta_ {t}} w _ {t} \mathrm {a n d} \beta_ {t + 1} = \beta_ {t} - \gamma_ {t} (a \kappa_ {t} - \frac {\kappa_ {t} (b \| \phi_ {t} - w _ {t} \| ^ {2} + \epsilon_ {t} + \epsilon_ {0})}{\beta_ {t} ^ {2}}).
$$
Considering the case that all the tasks have the same sample number,  $\beta_{t}$  reflects the confidence of  $\mathcal{A}$  on the given initialization for learning the task  $t$ . For example,  $\beta_{t}$  is the learning rate (step size) for SGD. A large learning rate means that the learner relies more on the data, and a small one means believing the prior. From the update of  $\beta$ , we can see a large  $\beta_{t+1}$  implies that the current task output is far from its initialization ( $\| \phi_{t} - w_{t} \|$  is large), which indicates the current task  $t$  may be an outlier, exceeding the range of meta-model for recovering its performance. Hence, the environment may have changed. So in the next task, the base learner learns more from the task data instead of keeping close to the initialization ( $\beta_{t+1}$  is large).
The learning rate of the initialization (or bias) is determined by  $\frac{2b\kappa_t\gamma_t}{\beta_t}$ . As mentioned above, a large distance from the initialization indicates that the current task  $t$  may be an outlier. So from  $t + 1$  step, the tasks are treated with low confidence to update the common initialization  $\phi_{t + 1}$  with  $\frac{2b\kappa\gamma_t}{\beta_{t + 1}}$  being small ( $\beta_{t + 1}$  is large). Then until the distance w.r.t the common initialization holds a small value for several rounds,  $\beta$  also decays to a smaller value. The tasks start to contribute more to updating the initialization.
The above mechanism works well for static environments and can address forgetting in shifting environments. But it will lose tracking ability in the shifting environment, so we need slot-wise adjusting for  $\gamma_{t}$  to obtain a better trade-off.
G.2 Real-world examples for practicality and necessity of CML