SlowGuess/NeurIPS_2023_OCR huggingface dataset:
fluctuations in the accuracy, as marked by the larger standard deviations in their accuracy over the ST-NPCL and the NPCL. This validates the fact that the conditioning on the correct task-specific context remains crucial to the performance of the NPCL.
Method	Average accuracy over 10 runs (%)
S-CIFAR-10 (Msize = 200)	S-CIFAR-10 (Msize = 500)
ST-NPCL (w/ only per-task latent)	54.6 ± 2.14	65.22 ± 1.89
NPCL w/ noisy task-specific priors	59.41 ± 3.0	65.1 ± 2.31
NPCL	63.78 ± 1.7	71.34 ± 1.48
Table 12: The importance of correct context: Comparison of S-CIFAR-10 accuracy (over 10 runs) of Single Task NPCL (ST-NPCL), the NPCL with noisy/corrupted task-specific priors and the standard the NPCL. The presence of noisy priors degrades the performance of the NPCL up to the extent of falling behind the ST-NPCL (without a hierarchical structure) on the setting with larger memory size.
G.5 On out-of-the box novel data identification
Our novel data identification experiments use the S-CIFAR-10 and S-CIFAR-100 datasets interchangeably as  $\mathcal{D}_{\mathrm{ID}}$  and  $\mathcal{D}_{\mathrm{OOD}}$  given the high degree of similarity between a number of their classes [21].<sup>2</sup> Namely, while evaluating the NPCL trained on S-CIFAR-100, we consider the entire CIFAR-10 test set as  $\mathcal{D}_{\mathrm{OOD}}$  whereas the evaluation of the S-CIFAR-10 model treats the test set of first 10 class labels of CIFAR100 to be  $\mathcal{D}_{\mathrm{OOD}}$ . Further, for an incremental task  $t$ , the test sets for  $[0, t]$  tasks make up for the ID data  $\mathcal{D}_{\mathrm{ID}}$ .
As shown in Table 13, the variances computed using either of our proposed metrics on  $\mathcal{D}_{\mathrm{ID}}$  are up to a magnitude lower than those on  $\mathcal{D}_{\mathrm{OOD}}$ . This trend is evident across the incremental evaluation steps even if the differences in the variances between  $\mathcal{D}_{\mathrm{ID}}$  and  $\mathcal{D}_{\mathrm{OOD}}$  slump with the further arriving tasks. Moreover, for the model trained on the more challenging S-CIFAR-100 setting, we observe that the differences between the  $\mathcal{D}_{\mathrm{ID}}$  and  $\mathcal{D}_{\mathrm{OOD}}$  variances even grow during the course of incremental training. This implies the potential perks of enabling the inter-task knowledge sharing among the NPCL parameters in a CL setup.
Incremental step	CIFAR-100 on S-CIFAR-10 model	CIFAR-10 on S-CIFAR-100 model
ID(δ)	OOD(δ)	ID(H)	OOD(H)	ID(δ)	OOD(δ)	ID(H)	OOD(H)
1	1e-6	1e-5	9.3e-6	8.4e-5	1.5e-6	8.9e-6	1.5e-4	1e-3
2	2.6e-6	1.4e-5	6.3e-5	2.2e-4	1.9e-6	5.8e-6	5.3e-4	1.7e-3
3	2.3e-6	6.2e-6	6.7e-5	2.1e-4	1.2e-6	3.6e-6	4.4e-4	1.5e-3
4	8.1e-7	4.8e-6	4.6e-5	2.2e-4	1.1e-6	2.5e-6	3.5e-4	1.2e-3
5	7.1e-7	1.7e-6	4.6e-5	1.1e-4	8e-7	2e-6	4.4e-4	1.2e-3
6	-	-	-	-	6.8e-7	1.3e-6	4.1e-4	8.5e-4
7	-	-	-	-	4.e-7	1.3e-6	3.2e-4	8.3e-4
8	-	-	-	-	4.9e-7	1e-6	3.2e-4	6.7e-4
9	-	-	-	-	3e-7	6.9e-7	2.5e-4	4.7e-4
10	-	-	-	-	3.3e-7	5.1e-7	2.5e-4	3.5e-4
Table 13: Average variances over softmax  $(\delta)$  and entropy  $(H)$  scores of incremental models on in-distribution (ID) and out-of-distribution (OOD) test sets using  $N = 50$  samples
G.6 On instance-level model confidence evaluation
For each target instance  $x_{*}$ , the instance-level model confidence evaluation framework [17] uses the  $N$  predictions obtained from stochastic sampling to compute: (a) the prediction interval width (PIW) between the [2.5, 97.5] percentile range of the  $N$  predicted classes, (b) the paired two-sample  $t$ -test [10] to evaluate the significance of difference between the mean predicted probabilities for the top-2
The labels for first ten CIFAR-100 classes are the same as https://huggingface.co/datasets/cifar100 and that for CIFAR-10 classes are the same as https://huggingface.co/datasets/cifar10.
most predicted classes. As a prerequisite to the latter test, we first verify the normality assumption of the probability differences for the NPCL (Fig. 10).
Similar to Fan et al. [10], after computing the PIW per test instance, we split the instances into two groups by the correctness of the majority-vote predictions, obtain the PIW of the true class per instance, and compute the mean PIW of the true class within each group. For t-test evaluation, we compute the mean accuracy per group of the test instances split by their  $t$ -test rejection status.
Figure 10: Q-Q plots for the differences in probability between the most and the second most predicted class.
Class	Accuracy	PIW	Accuracy by t-test status
Correct	Incorrect	Rejected	Not Rejected
1	82.30	74.17	102.21	83.37	50.00
2	94.00	62.90	79.86	94.07	80.00
3	74.00	54.92	68.48	74.14	64.29
4	71.50	65.42	74.32	72.06	25.00
5	84.80	92.93	106.90	85.37	22.22
6	76.50	75.22	103.58	76.58	60.00
7	94.20	104.9	129.56	94.39	3.00
8	90.50	81.10	127.06	91.12	22.22
9	96.90	72.81	110.86	97.00	66.67
10	96.30	80.60	109.56	96.48	60.00
Table 14: PIW (multiplied by 100) and  $t$  -test results for classes inferred from their respective task heads after S-CIFAR-10 training.
H Incompetence of Dot-product attention
Top-15 context points attended per query
(a) Cross-attention visualization
(b) Self-attention visualization
Figure 11: Scaled dot-product attention visualization: (a) top-15 context (buffer) points attended for 4 randomly chosen queries (test set samples). The queries are made after training on S-CIFAR-10. The sizes of the points correspond to the attention values while the colors denote the tasks they belong to. (b) self-attention weights of context points when all feature values are arranged in ascending order (along x and y-axis) shows that the ANP [27] mostly attends to the lowest or the maximum values in the context dataset.