NPCL: Neural Processes for Uncertainty-Aware Continual Learning
Authors: Saurav Jha
Dong Gong
He Zhao
Lina Yao
Venue: NeurIPS 2023 poster (2023)
Keywords: Continual Learning
Neural Process
Uncertainty
Incremental Learning
TL;DR: The proposed Neural Process-based Continual Learning method (NPCL) encodes tasks as probabilistic distributions over functions with task-specific modules in a hierarchical model, providing uncertainty measurements and boosted performance.

Abstract: Continual learning (CL) aims to train deep neural networks efficiently on streaming data while limiting the forgetting caused by new tasks.  However, learning transferable knowledge with less interference between tasks is difficult, and real-world deployment of CL models is limited by their inability to measure predictive uncertainties. To address these issues, we propose handling CL tasks with neural processes (NPs), a class of meta-learners that encode different tasks into probabilistic distributions over functions all while providing reliable uncertainty estimates. Specifically, we propose an NP-based CL approach (NPCL) with task-specific modules arranged in a hierarchical latent variable model. We tailor regularizers on the learned latent distributions to alleviate forgetting. The uncertainty estimation capabilities of the NPCL can also be used to handle the task head/module inference challenge in CL. Our experiments show that the NPCL  outperforms previous CL approaches. We validate the effectiveness of uncertainty estimation in the NPCL for identifying novel data and evaluating instance-level model confidence. Code is available at https://github.com/srvCodes/NPCL.

NPCL: Neural Processes for Uncertainty-Aware Continual Learning
Saurav Jha
Dong Gong" UNSW Sydney dong wongg.gnggunssw edu. au
UNSW Sydney saurav. Suurav.jhaeeee jha@unsww edu. au
He Zhao CSIRO's Data61 he zhao@ieee.... org
Lina Yao
CSIRO's Dataó1, UNSW Sydney lina. yaoddaao6. csiro.au
Abstract
Continual learning (CI.) aims to train deep neural networks efficiently on streaming data while limiting the forgetting caused by new tasks. However, learning trans- ferable knowledge with less interference between tasks is difficult, and real-world deployment of CL models limited by their inability to measure predictive un- certainties. To address these issues, propose handling CL tasks with neural processes (NPs), class of meta-learners that encode different tasks into probabilis- tic distributions over functions all while providing reliable uncertainty estimates. Specifically, we propose an NP-based CL approach (NPCL) with task-specific modules arranged in hierarchical latent variable model. We tailor regularizers the learned latent distributions to alleviate forgetting. The uncertainty estimation capabilities of the NPCL can also be used handle the task head/module inference challenge CL. Our experiments show that the NPCL outperforms previous CL approaches. We validate the effectiveness uncertainty estimation in the NPCL for identifying novel data and evaluating instance-level model confidence. Code is available ltttttt://// /eetps://giie h/ggggttub..... com/ srvCodes/NPCI
Introduction
Continual learning (CL) aims to help deep neural networks (DNNs) learn from stream of non- stationary tasks by retaining the previously acquired knowledge [54 35 To achieve this, CL agents target alleviating the catastrophic forgeettnn issue with restricted computational and memory costs [42]. This requires balancing the plasticity for new knowledge with the stability for old [37].
To handle forgetting in CL, experience replay (ER) methods [30 6 are one effective way to train DNNs on a memory buffer with a subset of the past tasks' experiences. Other than the ER methods, many regularization-bu based approaches have been proposed to penalize the forgetting on the DNNs' parametric 30 or representation spaces [ However, these may still suffer from interference due to the regularization on the entire parameter space 77]. To address this, parameter isolation methods 53 33 define task-specific training components but are usually confined to task incremental CL setups requiring task ID during testing [47). It is thus challenging for CL agents to maintain transferable and shareable knowledge. Lastly, a hurdle to the real-world deployment of CL agents is their inability measure predictive uncertainties, which impacts the potential utilization of CL across various practical applications, particularly those with critical safety considerations [31]
D. Gong the corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

To tackle the above issues, we propose to explore CL models using neural processes (NPs) 13 14 class of meta-leamers that model tasks as data- guttrrenn functions from stochastic process. NPs learn prior over functions by marginalizing over a set of data points, or context, thus enabling rapid adaptation to new observations through inference on functions. Additionally. their probabilistic nature endows them with reliable uncertainty quantification capabilities [14 27 24 25]. Our motivations to explore NPs for CL are thus two-fold. First, NPs exploit Bayes' theorem, which naturally enables CL through sequential posterior construction. Namely, NPs perform inference over the function space by learning context-based priors, which are updated to posteriors upon observing (additional) targets. Second, NPs meta-leam input correlations through set of latent variables, which could be a key to meta-learn knowledge transfer across multiple correlated tasks. However, NPs face challenges in directly addressing CL tasks, given that (a) the reliance single global latent leads to suboptimal modeling of complex CL tasks where multiple correlated tasks could occur simultaneously. (b) NPs cannot directly handle the forgetring of past task correlations arising from the non-static data stream.
Task
Eploodo momery
TOSE
To address the above desiderata, we propose Neu- ral Processes for Continual Learning (NPCL), a hierarchical latent variable model with a global latent variable to capture inter-task correlation and task task-specific latent variables for finer knowl- edge. Fig. shows the NPCL exploiting func- tional correlation among current and past task training samples of ER. The drift of global and past task-specific distributions away from their original forms is the major cause of forgetting in the NPCL. We thus propose to regularize the latent variables to be similar to their old forms and show the merits of regularization over typical parameter-basee regularization. We then lever- age the uncertainty encoded by the NPCL for the aforesaid CL challenge of task head inference. To this end, we propose using entropy as an uncer- tainty quantification metric (UQM). The NPCL. outperforms previous probabilistic CL models and delivers better or comparable results than state-of- tftte-of-the-an thet-errtr deterministic CL methods, which usu- ally have an edge over their probabilistic counterparts in terms of accuracy. Moreover, our ablations show the enhanced efficacy offered by the NPCL continual learning settings requiring model calibration and few-shot replay. To study the further usages of the NPCL's uncertainty estimation, we show its out-of- the-bos readiness for novel data detection and instance- confidence evaluation [77. Lastly, we list the key limitations of the NPCL as an attempt to lay further solid directions for uncertainty- aware continual learning.
Contoct
Target
Camner
Targer
Contact
Target
Fariaa
ABISOW
regletoebin
Clubal
Figure 1: Neural Processes for Continual Learning (NPCL): each training step involves minimizing the distance D between the context- based prior and the target-based posterior, along- side regularizing the task-specific and global dis- tributions towards their old forms.
2 Related Work
Continual Learning (CL). Existing CL methods address catastrophic forgetting through three major approaches: (a) Regularizution-busee methods penalize changes in a model's i's important weights for previous tasks, such as Elastic Weight Consolidation (EWC) 30 Synaptic Intelligence (SI) [54] etc.
(b) Parameter Isolation-based methods partition the network's 's parameters to specialize on individual tasks, c.g., Douillard et al. 9 learning task-specific tokens for Transformers.
(c) Replay-based
methods an episodic memory to preserve fraction of the past tasks' experience for preventing forgetting while learning on new tasks; c.g., experience replay (ER) 6 storing past inputs, dark experience replay (DER) 4 storing past logits, and Yan et al. [2 using loss-aware memory for ER. Our method uses (a) via regularization of distributions, (b) via task-specific latent heads, and (c)
via replay of past task inputs and distributions.
Neural Processes (NPs). NPs were introduced to meta-learn a distribution of a family of functions modeling the data-generating process through their deterministic [13 and/or latent summaries [14] Attentive NPs (ANPs) 27 replaced the averaging operation in NPs with a dot-product attention
48 to enhance their expressivity. NPs/ANPs rely on global latent that limits their ability to

---

model observations from multiple functions. Some works address this through local latent variables that model fine-grained correlation among a subset of the observations [50]. Recently, multi-task processes (MTPs) [6]] have been studied to model multiple tasks with NPs, owing to the hierarchy of task- task-specific latent variables conditioned on a global latent. However, existing MTPs cannot directly handle CL problem because (a) MTPs are not designed to learn on sequential tasks and thus do not handle forgetting; (b) MTPs target the multi-task learning problem where the label for an input spans the exhaustive output space of available tasks unlike the CL setup where each input may belong to one specific and unknown task, out of multiple seen tasks.
Besides, the added complexity of variational inference has limited NP applications to mostly proof- of-concept focused regression tasks [23]. The potential of NPs for large-scale classification tasks thus remains largely under-explore except in some recent works. Wang et al. [49, for instance, leverage the predictive uncertainties of NPs to decide on pseudo labels for unlabeled data semi-supervised classification. In our work, we use NPs to handle CL. with classification tasks, reflecting the benefits of principled Bayesian learning, uncertainty estimation, and easily integrated existing ER.
3 Preliminaries: Neural Processes
Given the data [(x', = Y ~ D' of a task t. the goal is to learn the mapping F;; X Y' reflecting the gett--enerating process. NPs [4 meta-learn the distribution over the mapping functions from the given tasks. This is equivalent to meta-learning the distribution over the predictions p(y' let, C²) for the target output y belonging a target data set T given the corresponding target input of and a context set C' [1 23.. J. To reflect the meta-learning hehavior of NPs [14, the training samples are split into the context set |C*| = and target set [T = + n containing context set C and additional samples. NPs learn the Gaussian priors and posteriors using a neural network fiooo0 ss FL for the predictive distribution, where and 0 parameterize an encoder q and a decoder p.
respectively. This involves deriving global variable 20 to estimate the prior p(= Φ), and then maximizing the marginal likelihood p(Y P(Y?IC* 0):
P(X = f p(x P(X71x7, 2 (p(29())
(I)
where p x maintaining the memorization of the task prior p(=91''; (, Φ) can help NPs avoid forgetting the 1-th task. Our aim behind enabling NP for CL is to seek a trade-off to preserve such task priors while sharing the parameters among tasks.
p(u, =0 is the generative likelihood. In CL with streaming tasks,
4 Continual Learning with Neural Processes
CL. considers learning from a series of different tasks arriving sequentially, Le., DD 1 < L 7 1 Here, Dt can belong to classification tasks with different classes in class incremental CL. 8 Let 7 be the cross-entropy (CE) loss for classification, the CL objective for the task involves minimizing:
Cee B(Z.D 1( (Fioo (1),
on all [0,4] I] tasks seen sequentially. Achieving Eq. ② is challenging in real-world CL scenarios, where the previous datasets can unavailable due to constraints on privacy, storage, etc. Learning on the sequential data with varying distributions causes catastrophic forgetting. To alleviate the issue, experience replay (ER) is used in CL to store and periodically revisit some past experiences, e.g., samples (x*, aet,y") of task t, in small episodic memory M for replay in the future [8] 6). In this work, we develop our method with the classical reservoir sampling-base boundary-agnostic updating of M.
ER for a task
CL methods with ER solely still suffer from severe forgetting issues 6 52 : and jointly optimizing parameters on D' and M has several drawbacks [35 4] Considering that deterministic mapping F* limits capturing the randomness behind the wollw data in stream, to utilize the meta-learning ability of NPs, we next propose extending models with Eq. (2) and Eq. (1) to arrive at our NPCL. model. It allocates small subsets of parameters to learn robust per-task and global priors and uses stochastic factors to meet data-driven challenges such as deducing the right parameters for inference.
3

---

MLDS
Deterrsradi: Parh
saterdion
Pv
Mt
Diteal
CAu
duting
Agorngation
atsetioon
Samplnn
Pertaa
Inpu tmages
Figure 2: Overview of the NPCL architecture: the decoding mechanism differs during training and inference. Red, Cyan, and Orange denote three different tasks.
4.1 Neural Processes for Continual Learning
Given the task in stream, we model the CL task based on NPs formulated in Eq. ①T. In ER framework with a small memory buffer, where the context and target could be from tasks indexed by t, Eq. (T) can be extended to derive the joint posterior for NPs [4] as:
P(YP:)CCU x
P(Y X 2 (=))p(=(c
(3)
where 20 models the joint distribution F0:t of CL tasks and enabler of the knowledge transfer 35] between these (see App.[A.3]oo for ELBO). Eq. (3) poses two challenges. First, a labeled context all NPs, which is unprepared in CL setups by default. To overcome this, we use the memory M offered by the ER ER-based setups as context during inference. Second, jointly modeling 1f ignores the dynamies of per-task stochasticities and is still prone to the bottlenecks of Eq. (2). We address the issue by introducing hierarchical modeling and redefining Eq. (3) the following.
is needed for inferring predictions
4.2
NPs with Hierarchical Task-specific Priors for CL
To learn informative task priors while tackling the forgetting issue in CL, we propose a hicrarchical modeling of the NP model. We preserve the global latent 20 to induce the direct knowledge transfer and add the task-specific upon the global variable to enhance the capturing of task-specific ledge in CL. We thus extend Eq. (3) with task-specifie latent variables 2' (2 (=",...²') 2²). As result, posterior is a two-ster step hierarchical latent variable model (Fig. ( where the global and the per-task latent variables model the inter and intra-task correlations, respectively:
p(Yp:IX cD:t) /
2 c*) p
T-1 [II
(4)
where the entire context CO:t is first encoded into 20 and then conditioned G the task task-specific context Ot := (C .., C²) are encoded into their respective latent variables. We refer to Eq. (4) as NP for CL (NPCL).. The hierarchical modeling enables NPCL learn the shareable knowledge via G and the task-specific knowledge via z' in the meta-learning fashion of NPs. Task identity is used in training to specify the task-specific latent variables. Unlike MTP (26) making predictions of all tasks for all inputs, NPCL needs to specify the corresponding output space for each test sample. We further discuss the relationship between NPCL and NP-based meta-learning in App.
4.3 The NPCL Architecture
As standard NPs 14 27], the training samples are split into context C and a target set T containing C and additional samples. Given the inputs z; from C or T. first pass these to feature extractor

---

f. With slight abuse of notation, we denote the features as 2, 22 E RIl and let lf denote the dimension. 2, concatenated with the one-hot encoded labels, ie.. [z ys is fed to the NPCL encoder with a deterministic and a latent path, and then to the decoder (Fig. 2 All the NPCL. layers use multi-layer perceptrons (MLPs) projections, i.e., MLP(x)) R + I tol, where |o] is the output feature dimension as a hyperparameter. We denote normal distribution with a mean μ and variance o² by ((,, o²); the global and the task-specifi distributions are .N NuG, ot)) and (N,,
Latent Encoder. The latent path comprises of the projection D = MLP([ yal) followed by two attention operations [48]. First, per- per-task projections form the keys, values and queries to taskwise self-attention layers S.ffat that produce order-invarian encodings st over the samples of task t. (s::-)nttm Second, all encodings {s fi=1 n++mm serve as the keys, values and queries to cross-attention layers C that enrich their order-invariance from intra-task s to inter-task s²_ s and 8 are used to derive the N and M Monte Carlo samples of the global 20 and the task- sasc-specific latent variables 2", respectively (see App. for more details) using the reparamcterization trick [29]. We set M = 1 to enhance the inter-task stochasticity in posterior while retaining superior computational efficiency (see App.
For each input, we thus get x (t++ +++ 1) latent outputs.
Deterministic Encoder. The deterministic path is similar to that of the ANP 27 and outputs an order-invariant representation r. for target x, (see App. (
Decoder. Based on the task information, the decoder adopts separate mechanisms during training and inference. At train time, we use the available task labels to filter the N true latent variables {z {=()==>> combine them with r. and T.,, and decode the logits h-. We discuss the decoding operation in the testing phase without task ID in Sec. 4.5
4.4 Learning Objectives for the NPCL
The learning of the NPCL involves variational inference alongside additional regularizations.
Evidence Lower Bound (ELBO). The intractability of Eq. (4) leads us to the following ELBO:
log po (Y x CC 2 E (T)
pog.o 2*)1
(=
(5)
t=
D' (1o(=")=); (1 (0,,,,,,)]] -
DO (1ol= (T)l=(1))llea(=91)))
where Po(Y X47, 2*) is approximated by the CE loss. D' and DO denote the KL divergence (KLD) between the approximate posterior and prior for the task-specific and global distributions, respectively, We derive the ELBO in App. A.I We next propose two techniques to counter forgetting in the NPCL.
Henceforth, we use D to denote the Jenshen-Shanne
(JS) divergence 11 between two distributions.
Global Regularization (GR). The training data of CL task is dominated by the t-th task samples. For the NPCL, this drifts the global distribution (H og) of past tasks towards the new task (Fig. Db.
We thus regularize their global distribution using the one learned at step t - 1:
LGR = (W(NGs
Task-specific Regularization (TR). While GR helps preserve the joint distribution of the past tasks, the hierarchy in the NPCL leaves their task-specific distributions to be still prone to forgetting (Fig. 3(a)). This can further amplify the posterior collapse [46] for past task-specific latent variables during CL training (Fig. (b)). To alleviate these, we regularize the learning of previous task distributions as:
CTR DWN (U)N(,,,)))), o.),,,N (p),))))) of););
where j is the step at which the task / arrived. Given the reliance of Eq. (6) and Eq. 7 on past distributions, we maintain a separate buffer, which we refer to as the distribution memory MN. to store the global N N(H. og) and the task-specific distributions (H -1) MMr is updated after each incremental training step, where we run additional pass over the training data of task t alongside replaying M to record the batchwise averaged global and task-specific means and variances.
Integrated objective. Using a, B, 7, and 8 to denote the loss weights, our total loss can be written as: L
Σ
(CeE + aD + BDD + M
Σ
ZGR + &CR. (8)
D +
M
(at DDM'MM
5

---

SCIMAR-10
-25
5.0
Gobel
10
30
40
10
30
40
sposits
Taxt
1105: spedtre letents
spacitk
necmerta usiing time
(a) Drift of past task distributions
10
30 40
10
40
opochs
Train apochs
(b) log K L(glp) w/o TR
(c) log K L(glp) w/ TR
Figure 3: Analyses the need for distribution regularization: (a) shows the increasing distances between current distributions of past tasks and their original distributions (learned while the tasks were introduced). (b) and (c) show the effect of global (GR) and task regularization (TR) on the activation of the global and task-specific latent units. Low KLD corresponds to an inactive unit.
where CE, Dt, and D act on the current task data D' and on the buffer M while GR and TR act only M. By setting 0 < {ax 3, 7,63 < 1, we resort to using the (respective) cold posteriors (55
4.5 Inference with Uncertainty Awareness
NPCL's inference uses f to obtain the features u. for the target test images. Although the task identification information is used to train the task-specific module, task identification of test samples is usually unavailable in general real CL tasks (except the restricted inssiincremental setting (8).. Given z, from the encoder, this leaves with {=] Jim possible modules and the corresponding latent variables to use and infer for obtaining the prediction. A naive solution is to average over N (L + 1) logits. But the number of tasks grows, the noise from incorrect task priors would dominate the posterior. We thus propose using entropy as an uncertainty quantification metric (UQM) to filter the logits of the true task head '/
h4 arg min U (hwr), Uhb))= = Σ 8(1) log()(i)), iEN
JE[1.(]
where ö the softmax function and U is the total Shannon entropy over the N logits per head. As we use true head vi during training, o E vi produces low entropy for within distribution data. In light of Eq. (9), the NPCL. can be seen as a special case of the mixture-of-expert (MoE) modeling 36 51. where leverage uncertainty select the top-1 expert during inference.
Experiments
Datasets. We evaluate the NPCL class and domain incremental learning (IL) settings. For class-IL, we use three public datasets: sequential CIFAR1O (S-CIFAR-10) (35, sequential CIFAR100
(S-CIFAR- [54), and sequential Tiny ImageNet (S-Tiny- -iny--mageeNeu 6 For domain-II..
Permuted MNIST (P-MNIST) [30] and Rotated MNIST (R-MNIST) [35. S-CIFAR-10, S-CIFAR- 100, and S- images and I 1000, 100, and 50 test images per class, respectively. The number of sequential tasks for S-CIFAR-10 is (2 classes per task); for S-CIFAR-100 and SmageN- 10 (10 and 20 classes per task, respectively); for P/R-MNIST is 20. P-MNIST creates tasks out of MNIST 32 by randomly permuting the pixels, and R-MNIST does by rotating images randomly [0, T).
host 10, 100, and 200 classes each with 5000, 500, and 500 training
Architectures. For a fair comparison against other methods, we rely on the Mammoth CL benchmark 3]. Our backbone for class-IL experiments is a 18 20 without pretraining, while for domain-II, we rely on a fully connected (FC) network with two hidden layers [35]. The NPCL relies on Xavier initialized [5] FC layers with 256-d hidden layers for class-IL and one 32-d layer for domain-IL setups. For class-IL, each FC layer is followed by layer normalization [[] and ReLU.
Configuration and hyperparameters, We train all models using SGD optimizer. The number of training epochs per task for
is 100, for S-CIFAR-(10/100 is 50, and that for (P/R)-M MNIST is I. We detail further the configurations, hyperparameters, and their tuning in App. D

---

Methad
S-CIFAR-10
S-CIFAR-100
S-Iluy-mmageNe Clare-ll
P-MNIST
R-MNIST
Clare-II
Class-I..
Damain- II.
Domoii--I II.
Toing ResNet
92.2 20.35
70:44
59. 59.99=0.18
94. 0.117
96.6:20.04
Join
0.11
70.5:20.24
59.85--0.17
95,02400221
96.3740.00
Joens
91.26+0.10
70.77+0.22
60.14++00.17
95.39 +-0.1
95.85+00.00
Joint NPC1.
92. 02.74220112
71.46:0.22 20
60.182:0.22 22
95.97:2:0.14
96.1120000
Muttnask NPCL
09.15:00.09
23.6290.1
35.53-b0.13
07,60:20.10
09.21
OLWC
80
19.492-0.12
0.58#:0.10
75.79:42.00 225
77.35-55.77
SI [E
19:08 1:0.17
6,58 =0.31
63.86:11.51 1.57
71.1 4:5.83
19.01=0.00
8.46 =0.22
Maine
500
500
2000
200
500
300
500
79:1:
57.
77.0.00..27 27
22.
38.58
8.4920.18
9.091.
12.37:0..87
80.6-0.86
85.011..00 88.11:1.44
(CaR1.
40. 02 3
47.55
13.05
46.53
49.82
7.53:10779
6.38:21.55
R13
30.91 22.74
28.71 23.23
0.70:0.19
10.64:0.21
74.7700.83
83.18 80.58 0.7-2088833.88888
85.22 89672
KPC -3]
22.34
38.35
DER
105111167
36.6
51.89
11.87
20.78
17.7511
81.74111
87.2910:46
90.412.01
14
46.143.44
99.442.76
22.92
18.70
83220.62
10240033
700021.44
00.44
25.09:2.77 16.16.6666
ANP
46.67:1.23
58.7:+0.65
23.2
39.06
881
9.55:0090
73.55=0.66
85.70=1.39
89.21
STAPPCL only pas-task Intem)
54.64-2.11
65.2221..89
25.45
42.1
10.2+1.00
13:74-3355
76.4+1.62
N2.06---0922
89.64-2111
Naive NPCL (wlo task head inf.)
19.54
44
20.71433.09
18.27
18.90
7.1911.02
$.88::0.90
68.37::1558
73.34:0.81
81.13:2.91
83.09 0.991:222
NPCL. (ours)
6.778
71.4411448
37.43
46.71
02.44.0.50
15.29:
83.11=0.900 86.52 1. 92.07
Table I: Classification accuracy for standard CL benchmarks across 10 runs. The best results are in red. The second best results are in blue. All runs of NP variants in the CL settings rely on ER. A-CIAAR-00 results are from Boschini et al. B while the rest are taken from Buzzega et al. [
Baselines. We employ several CL methods to compare the NPCL with. Regularization-baee methods include oEWC [44] and SI [54]: ; knowledge distillation based methods include iCaRL [42] and LwF B4]; rehearsal-based methods are ER [43]. RPC [4], FDR (2). DER 44. Among neural processes, we use the NP [14, the ANP (7). and the Single Task (ST) NPCL (see App. A.2 with only per-task latent variables. We use five non-CL CL benchmarks upper bounds on the performances: Joint ResNet / /NP/ ANP NPCL perform joint training of all tasks using a single task head while the multitask NPCL infers task heads in joint training using Eq. (9). Finally, the naive NPCL inference averages the logits of all task heads.
5.1 Results
Table the average accuracy after training all tasks. Across all settings, the NPCL boosts the performance of the ER and achieves either comparable or better results against the state-of-the- art (SOTA), e.g., DER. Compared to the regularization-ben bbaed oEWC and SI, the NPCL obtains significant gain in performance. This is because the former methods calculate weight importance, which is liable changes with new tasks. Regularizing explicitly towards the global and per-task distributions of past tasks helps the NPCL overcome this. Further, on both class and domain IL, the NPCL stands out in the most challenging setting where the episodic memory size is the smallest. On domain-Il. where the shift occurs within the domain instead of the classes, the performance of a number of methods degrade they forget the relations among a task's classes. Preserving the tasks' distributions helps the NPCL maintain valuable information in this case. Analyzing the backward transfer (BWT) scores (39] shows that the NPCL's forgetting is competitive or lesser than the SOTA (see Table 10p. Lastly, we note that the ST-NPCL with no hierarchy lags in BWT and accuracy due to limited knowledge transfer between tasks.
5.2 Ablation Studies
Why uncertainty-aware inference works? For our uncertainty-aware task head inference mechanism to be effective, CL model must produce probabilities that align well with the ground truth labels of the test samples. We thus ablate the calibration errors for differ- ent CL baselines using the rell-established Expected Calibration Error (ECE) [16] and Adaptive Calibra- tion Error (ACE) 38 metrics. Table T shows that the NPCL has the least calibration error across S-CIFAR- 10 (Maize = 200) and S-CIFAR-100 (Maize 500).
Method
S-CIFAR-10
S-CIFAR-100
Metric
ECE
ACE
ECE
ACE
0.4553
0.8532
0.6459
0.9499
DER ANP 27
0,2991
0.8391
0.2484
0.9447
0.34
0.8495
0.5441
0.9477
NPCL (burs)
0,2103
0.8155
0.1995
0.9421
Table 2: Model calibration errors averaged
across 10 runs.
In general, the probabilistic nature of the ANP 27 and
7

---

6.000
0.2
613
Conbodt
0.4
Carboot stxc
22
$1.4
0.24
6033
XB3
60:0
S
(a) Accuracy
(b) Uncertainty
(a) Accuracy
(b) Uncertainty
Heatmaps depicting the taskwise aver- aged accuracy and uncertainty of test samples per task head on S-CIFAR-10.
Figure 4:
Figure 5: Effect of context set size (Masze {5, 50, 100, 200)) on the accuracy and uncer- tainty of the NPCL on S-CIFAR-10.
the NPCL benefits them in confidence calibration over the deterministic methods with comparable accuracies, L..., ER 43 and DER 4
Uncertainty-Accuragy trade-off. Fig. 4 ablates the average accuracies and uncertainties of each task head predictions over the test set of each task on S-CIFAR-10 (see App. G.2 for S-CIFAR-100), First, we observe that the accuracy of predictions made by true task heads are, in general, magnitude higher than the rest. For uncertainty, this trend is reversed. This verifies assumption that restricting latent heads to learn only their true label distribution makes them more confident in modeling the within-task samples. Second, for recently trained tasks, the uncertainty differences between the true task heads and the rest are greater than the earlier tasks. This, in general, suggests that the extent of forgetting goes beyond CL model's accuracy and to other aspects of its learning such its predictive confidence. To support the latter claim, we probe the BWT of uncertainty and see a strong correlation with the BWT of accuracy (see Fig. 7
Learning objectives. Table shows the impact of distri- bution regularization, with the baseline being the NPCL trained with no regularization. We observe that the base- line performs worse than the ER the NPCL layers forget more. Including TR in our objectives leads to the single-most gain over the baseline. We further study how these objectives guide the learning of the global and task- task-specific distributions with training (see App. G.I.. We observe that the NPCL w/ TR leads to better learning of the current task as well as preserving the past task distributions but at the cost of drifting the global distribution. The NPCL w/ GR restricts the drift of the global distribution but not for the per-task distributions. The NPCL w/ GR and TR strikes a balance in between.
S-
S. Tiny- ImageNet
Method
CIFAR-10
ER
44.79
8.49
Baseline (wlo GR or TR)
32.24
7.15
NPCL (w/ only GR)
50.68
8.61
NPCL only TR)
57.28
11.36
GR TR)
63.78
12.44
Table 3: Accuracy w/ learning objectives
Effect of Monte Carlo (MC) samples. We spot two combinations of the number of global N and task-specific M MC samples in favor of performance. Out of these, we choose the one with the superior computational efficiency (see App. Efro details).
Context size. We study the average accuracy (Fig. 5(a) and uncertainty (Fig. ((b) after training on S-CIFAR-10 with [M] = 200, and then varying the context sizes during inference. Similar to other NPs 449 12, we find a positive correlation between context size and performance, indicating that the NPCL utilizes useful information from diverse contexts, thereby reducing its task inference ambiguity.
Few-shot replay settings, A key strength of NPs remains their few-shot learning capability. To study how well the NPCL retains this trait against other CL baselines, we ablate their accuracy and ECE [16] on rehearsal memory sizes of 5 and 10 (see Table 4. We find that on both memory sizes, the NPCL outperforms ER (3) and DER (4. For Maise = on S-CIFAR- we observe that the ER outperforms the DER in terms of accuracy. However, the latter still offers more confident predictions (characterized by a lower ECE). This implies that on few-shot CL replay settings while regularizing the predicted logits towards their old forms as done by the DER - helps improve the predictive confidence over the ER, regularizing the task distributions towards their old forms - as done by the NPCL- remains the superior way to enhance the model's predictive confidence.
8

---

Method
S-CIFAR-10
S-CIFAR-I00
Miie
10
Metric
Acc.
ECE
Acc.
ECE
Acc.
ECE
Acc.
ECB
ER 43
22.11
0.7281
25.39
0.696
9.44
0.8003
9.69
0.8014
DER
21.05
0.5931
25.2
0.5107
8.96
0.4593
10.97
0.542
NPCL (ours)
22.98
0,4709
26.15
0,441
10.22
0.39
12.64
0.4717
Table 4: Few-shot replay results: Accuracy (Acc.) and ECE [16] of different methods with very small buffer sizes of 5 and 10 for S-CIFAR-10 and S-CIFAR-100 averaged across runs.
Storage efficiency. For each task, the NPCL stores two new vectors task-specific mean and variance, and replaces the global mean and variance with the current global ones. The NPCL storage thus scales constantly in the size [M] of the memory. This offers a strong edge on storage efficiency when compared to DER [4] scaling quadratically, i.e., [M]] X Ne where No is the total number of classes. For instance, on S- Tiny-ImageNee with [M] = 500, Nc = 200, the NPCL's cumulative storage amounts to a (flattened) vector of size 6132 (256 x 10 x 2 for 256-d means and variances of 10 tasks plus 256 x for 256-d global mean and variance plus 500 for 1-d task labels) while that of DER amounts to 100,000 (200 500 for logits of 500 memory samples), i.e., 93.868 % relative storage efficiency. We report the storage efficiency of the NPCL over DER across all settings in App. G.3
5.3 Applications of Uncertainty Quantification
The probabilistic nature of the NPCL offers it an edge in leveraging data-driven UQMs. To further
study the usage of predictive uncertainties, conduct two experiments with a trained NPCL model.
Novel data identification. Novel data identi-
Incremental step
Dio CIFAR 10, Doto = CIFAR 100
fication seeks to distinguish out-of-distribution data (Doon) from in-domain data (DID). For-
Di (8)
DOoD (6)
Dio (H)
Doov (H)
getting makes CL. models struggle further on the task [9]. The probabilistic sampling in the
le-
1e-B
9.3e 8
8.4e
-.
1.4e-b
6.3
2.3e-8
6.2e-6
6.7e-5
2.
NPCL opens the door for leveraging its predic- tive variances- - which are more reliable esti-
8.1e-T
4.8e
4.6e
2.2e
7.1e-T
1.7e-6
1.6e
1.1e
mates of aleatoric uncertainty than pointwise predictions [22]. For the N predicted logits, we thus compute the variances over their softmax scores, o² (h)h))) and over their uncertainty scores, a2( (U.)). eevalutss these met-
Table 5: Average variances over softmax (6)
and entropy (H) scores on in-domain and out-of- domain test sets using N 50 ancestral samples.
rics for ID (S-CIFAR-10) 10) and OOD (first 10 classes of S-CIFAR-100) data after each task. We observe that the variance scores of either metrics on PID are up to a magnitude lower than those on POOD. We further observe an overall decrease in the variances with the arrival of further incre- mental tasks. This could be attributed to the generalization of more low-level features in the novel We detail further novel data identification experiments in App. G.5
data as in-domain 18 16].
Instance-level model confidence evaluation. The confidence evaluation framework of Han et al. [17] provides finer granularity for assessing the predic-
Class Accuracy
PIW
Accuracy by E-test status
Not Rejected
Correct
Incorrect
Rejected
tive confidence of classification models (see App. G.6 for more details and normality test). Table
82.30
74.17
102.21
83.37
50.00
2
94,00
62.90
79.86
94.07
80.00
shows the results of one run of the framework after training on S-CIFAR-10. Here, we use the task identity to select the latent head per class. We observe the mean prediction interval width (PIW) of the true class label among the correct predic- tions to be narrower than that of the incorrect pre- dictions, implying that the NPCL's variations of predicted class labels are smaller when the predic- tions are correct. We also notice higher accuracy among the test instances rejected by the 1-test than those not rejected.
74.00
54.92
71.50
65.42
74.32
72,06
25.00
84.80
92.93
106.90
85.37
22.22
76.50
75.22
103.58
76.58
60.00
94.20
104.9
129.56
94.39
3.00
90.50
81.10
127.05
91.12
22.22
96.90
72.81
110.86
97.00
66.67
96.30
80.60
109.56
60.00
Table 6: PIW (multiplied by 100) and t-test results for the first three classes of S- C-FAA--11 inferred from their respective task heads.
9

---

6 Limitations
We list the key limitations of the NPCL to facilitate future research directions.
Incompetence of dot-product attention. Similar to the ANP 127 the NPCL employs the permutation-invann scaled-dot product attention [48] to weigh the relevant context and target embeddings. Visualizing the attention weights computed by the cross-attention layers of the determin- istic path shows us that the top attended context for the target queries often contain points belonging to other CL tasks (Fig. (I(a) This limits the performance sensitivity of the NPCL with respect to the increase in context thus resulting in lag of accuracy behind SOTA CL setups with larger episodic memory sizes (see Table T). To further verify the relevance of the attended context, we visualize the self-attention weights of all context points. Fig. [1 shows that the lowest the maximum values in the context dataset have larger weights. Such an observation is in line with existing works pointing that the scaled-dot product attention can derive irrelevant encodings of the context points and can thus lag at exploiting the context embeddings properly 28
Computational overhead. 7 the number of parameters of the NPCL. with ER / DER 3 where the latter rely solely on the ResNet-18 backbone as they do not exploit parameter isolation for task heads. Overall, the percentage increase in parameter number is 57.6% for S-CIFAR- 10, 46.57% for S-CIFAR-I00 100 and S- -mmyeeeee,, and 55.25% for P/R-MNIST.
Method Dataset
S-CIFAR-I0
S-CIFAR-I00
S-Tiny-ImageNet
P/R-MNIST
ER/ DER
11.173.962
11,220.132
11,220,132
89,610
NPCL
19,397,706
24,091,556
24,091,556
162,166
Table 7: Comparison of the total number of parameters for ER / DER against the NPCL.
Inference time complexity. The reliance on self-attention means that the inference time complexity of the NPCL. is O(n * m). where n is the number of context points (sampled from the episodic memory) and m is the number of target points (the number of test samples). Due to this, the runtime for in- ference scales polynomially with the number of context points (sampled from the buffer). Table 6 reports the runtime the NPCL on S-CIFAR-10 and S-CIFAR- 100 settings by varying the context sizes. For reference, the first row reports the runtime of ER/ / DER whose inference complexity is O(1) in the memory buffer size.
Method
S-CTFAR-10
ER DER
3.72s
NPCL, M = 200
19.38s
NPCL, [M] 500
31.25s
NPCL, [M 1000
47.99s
NPCL. M 2000
84.86s
Table 8: Inference time with vary- ing context sizes
Incompatibility with logits-based replay. The NPCL is incompatible with logits-based replay because of the stochasticity in the posterior induced by Monte Carlo sampling. Overcoming this could help boost the performance of the NPCL further over SOTA like DER 4 and DER++
7 Conclusion
In this paper, we propose Neural Processes for Continual Learning (NPCL.), a hierarchical latent variable setup designed to jointly model the task-agnostic and task-specific data-generating functions in continual learning. We study the potential forgetting aspects in the NPCL and propose to regularize the previously learned distributions at a global and per-task granularity. We demonstrate that using entropy an uncertainty quantification metric helps the NPCL infer correct task heads and boost the performance of baseline experience replay to even surpass state-of-the-ae deterministic eeperieeeeeeeeee
models on several CL settings. Our robust ablations show the efficacy of the NPCL for model calibration measurement and few-shot replay in CL. We further study out-of-the-box applications of the uncertainty estimation capabilities of the NPCL for novel data identification and instance-level confidence evaluation. We conclude our ablations by listing the key limitations of the NPCL., which we hope could lay solid directions for further research uncertainty- aware continual learning.
10

---

Acknowledgment
This work was partially supported by an ARC DECRA Fellowship DE230101591 awarded to Dong Gong. We acknowledge the reviewers for their valuable feedback.
References
[1] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. 2016.
[2] Ari Benjamin, David Rolnick, and Konrad Kording. Measuring and regularizing networks in function space. In International Conference on Learning Representations, 2018.
[3] Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Class-incremental continual learning into the extended der-verse. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[4] Pietro Buzzega. Matteo Boschini, Angelo Porrello, Davide Abati, and SIMONE CALDER- ARA. Dark experience for general continual learning: strong, simple baseline. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 15920-15930, Curran As- sociates, Inc., 2020. URL htttps://proceeding nwwrsss.eripp....... httpppss...cppppr///////eee 077448229788877777777766668--
pdf
[5] Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-GEM. In International Conference on Learning Representations, 2019. URL http://eerreeeeeeeeeeeeoorrraaaa
__5FFX
[6] Arslan Chaudhry, Mareus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint 190. 2019.
[7] Arslan Chaudhry, Nacemullah Khan, Puneet Dokania, and Philip Torr. Continual learning low-rank orthogonal subspaces. Advances in Neural Information Processing Systems, 33: 9900-9911, 2020.
[8] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. continual learning survey: Defying forgetting in classification tasks. IEEE transactions pattern analysis and machine intelligence, 44(7): 3366-3385, 2021.
[9] Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord. Dytox: Trans- formers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9285-9295, 2022.
[10] Xinjie Fan, Shujian Zhang. Korawat Tanwisuth, Xiaoning Qian, and Mingyuan Zhou. Contextual dropout: An efficient sample-dependent dropout module. In International Conference on Learning Representations, 2021. URL https://openrevii/ htt.://wwnee//ooumnddddeaa
[11] Bent Fuglede and Flemming Topsoe. Jensen-shannn divergence and hilbert space embedding. In International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings. page 31. IEEE, 2004.
[12] Ning Gao, Hanna Ziesche, Ngo Anh Vien, Michael Volpp, and Gerhard Neumann. What matters for meta-learning vision regression tasks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14776-14786, 2022.
[13] Marta Gamelo,, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International Conference Machine Learning, pages 1704-1713, PMLR, 2018.
[14] Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint arXiv:1807.01622 2018.

---

[15] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedfor- ward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 149-256. JMLR Workshop and Conference Proceedings, 2010.
[16] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321-1330. PMLR, 2017.
[17] Xizewen Han, Huangjie Zheng, and Mingyuan Zhou, CARD: Classification and regression diffusion models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https //openreview. net www.oooddddnl//...0..
[18] Jakob D Havtorn, Jes Frellsen, Soren Hauberg, and Lars Maaløe. Hierarchical vaes know what they don't know. In International Conference on Machine Learning, pages 4117-4128, PMLR, 2021.
[19] Jiangpeng He and Fengqing Zhu. Out-of-distributie detection in unsupervised continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pages 3850-3855, 2022.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pultern recognition, pages 770-778, 2016.
[21] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and examples in neural networks. In International Conference on Learning Representations, 2017. URL https::/
[22] Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110:457-506, 2021.
[23] Saurav Jha, Dong Gong, Xuesong Wang, Richard E Turner, and Lina Yao. The neural process family: Survey, applications and perspectives. arXiv preprint wwXiv:2209.005//2 2022.
[24] Myong Chol Jung, He Zhao, Joanna Dipnall, Belinda Gabbe, and Lan Du Uncertainty estimation for multi-view data: The power of seeing the whole picture, Advances in Neural Information P Processing Systems, 35:6517-6530, 2022.
[25] Myong Chol Jung, He Zhao, Joanna Dipnall, Belinda Gabbe, and Lan Du Multimodal neural processes for uncertainty estimation. arXiv preprint arXiv:2304.01518. 2023.
[26] Donggyun Kim. Seongwoong Cho, Wonkwang Lee, and Seunghoon Hong. Multi-task processes. In International Conference on Learning Representations, 2022. URL /ttpttttr///oeereeviee
[27] Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh, Attentive neural processes. In International Conference on Learning Representations, 2019. URL / www.:e/wwwweeooummmmmmaa dd-oorr6mm.ddss
[28] Mingyu Kim, Kyeong Ryeol Go, and Se- Young Yun. Neural processes with stochastic atten- tion: Paying more attention to the context dataset. In International Conference on Learning Representations, 2022. URL /tttps://openrevie:: wwwpeeewwwwwwiieeeeeeeeeee wwt.:neeotnfiiddeieoeo id=JPkQwEdYng
[29] Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparame- terization trick. Advances in neural information processing systems, 28, 2015.
[30] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114(13):351133526, 2017.
et al.
[31] Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62, 2022.
12

---

[32] Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http::/yaan..eent www.nn.lecu..com wtw.l/..co/eee//nniissssss 2, 2010.
[33] Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. International Conference Machine Learning, pages 3925-3934. PMLR, 2019.
[34] Zhizhong and Derek Hoiem. Learning without forgetting. IEEE transactions on pultern analysis and machine intelligence, 40(12):2935-2947, 2017.
[35] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
[36] Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-experss. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery data mining, pages 1930-1939, 2018.
[37] Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The Investigating the continuum from catastrophic forgetting to age-limited learning effects, 2013,
dilemma:
[38] Jeremy Nixon, Michael Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR workshops, volume 2, 2019.
[39] Guy Oren and Lior Wolf. In defense of the lcarning without forgetting for task incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2209-2218, 2021.
[40] Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. International conference on machine learning, pages 1310-1318. PMLR, 2013.
[41] Federico Pernici, Matteo Bruni, Claudio Baecchi, Francesco Turchini, and Alberto Del Bimbo. Class-incremental learning with pre-allocated fixed classifiers. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 6259-6266. IEEE, 2021.
[42] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In ProceedinI gs of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[43] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations, 2019. URL https //openreview. peet/forummiddBIISS..
[44] Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka Grabska Barwiikka Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for continual learning. In International Conference on Machine Learning, pages -5334.
PMLR, 2018
[45] Claude Elwood Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379 423, 1948.
[46] Casper Kaae Sønderby. Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. Ladder variational autoencoders. Advances neural information processing systems, 29, 2016.
[47] Gido M van de Ven, Tinne Tuytelaars, and Andreas S Tolias. Three types of incremental learning. Nature Machine Intelligence, pages 1-13, 2022.
[48] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
13

---

[49] Jianfeng Wang, Thomas Lukasiewicz, Daniela Massiceti, Xiaolin Hu, Vladimir Pavlovic, and Alexandros Neophytou. Np-match: When neural processes meet semi-supervised learning. In International Conference on Machine Learning, pages 29934,2 PMLR, 2022.
[50] Qi Wang and Herke Van Hoof. Doubly stochastic variational inference for neural processes with hierarchical latent variables. In International Conference on Machine Learning, pages 10088-10228. PMLR, 2020.
[51] Qi Wang and Herke van Hoof. Learning expressive metaareprresentations with mixture of expert neural processes. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://
[52] Qingsen Yan, Dong Gong, Yuhang Liu, Anton van den Hengel, and Javen Qinfeng Shi. Learning bayesian sparse networks with full experience replay for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 109-118, 2022.
[53] Jachong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In International Conference on Learning Representations, 2018. URL https:://openreviee hetp://ww.eeeeooooomddddd/////s
[54] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987-3995. PMLR, 2017.
[55] Guodong Zhang, Shengyang Sun, David Duvenaud, and Roger Grosse. Noisy natural gradient as variational inference. In International conference on machine learning, pages 5852-5861. PMLR, 2018
14

---

Appendix
A Theory
A.1 ELBO derivation for the NPCL
Borrowing the conventions from Sec. for incremental task < ST assume the context C. and targets T comprise of samples from all seen classes. Accordingly, we define these as C = (X&+, Y and T = (X Y respectively. To enforce the prior that both C' and T follow the same distribution, we assume C CT' and therefore, C C 'T In order to derive predictions Y on X& the NPCL. relies on the context C to build conditional priors pe p=(= and (= CC), where Po is the decoder. The decoder's objective thus boils down to maximizing the log-likelihood of the observations, i.e., the evidence log pe (Y X C). In the following, we derive the evidence lower bound (ELBO):
DYP 1x7 .C)
oLog-likelhhodd of evidence)
(10a)
(Vg 1x .c) pa(= 1x4 x y (c)ss
(: (T, = 1)
(10b)
= =/ y (YP" (c)ds' CC))ss
(Integrate log-likelihood)
(10c)
1 Dogpo |x .C]
(By defimition)
(10d)
loR Po() 11x4 C)
(Re-introduce @ by Chain rule)
(10c)
log PolY X , (p
(Chain rule of probability; C C T
(10))
Po(=917)
.,, log Po(Y (o. )p==) (00tt.))a(=911
B I
(Equivalent fraction)
(10g)
(7(6917)9((
1og 1x* (x*",,,,
Po log (C()
log (0(2
(=
Ver r)
Po(=
(Split the expectation)
(10h)
Daa (1T + Dxi (gole (
(By detinition divergence)
(10i)
B 800
(oRpo(" c, Dalasls ( (= (()))
( KL. divergence 2 0)
(10)
where the evidence is equal to the sum of the reconstruction likelihood E30(=917) Loggpo(Xe (Y X0: x,,t,,, :) of the decoder and the KL divergence between the true posterior Po (2 and the approximate posterior go (2 learned using the variational distribu- tion. minus the prior matching term DKL (T))TT)pp((2 (C)). In particular, the NPCL learns two approximate distributions (TT) and 22(2 TT), that seek to estimate the global posterior po(2G 'T) and the task-specific posterior po G (Tt). To realize the latter posterior, we introduce the hierarchy of task specifcc latent variables 2013 This allows us expand and derive a lower bound to the reconstruction likelihood as:
15

---

log O .x." c.
(Reconstruction term)
(aa
=BT "1 [log Pu(Y 1 c,
[Introduce one-level latent bicrarchy)
po 1X4 Y c.
(11b)
= =B7 00(00)20, (1, [108 pol 2' 1 20)
[Integrate over individual tasks)
po(="|y3 v7, 07.0 ())
(1l)
Lo Votel log
polyfIx,,Co C p'>'''' 05,
(Chain mile of prohahility; T)
po(="[T)
(118)
L/ Pagist log
='= , (= (=0=="=0,T7
(Equivalent fraction)
Po(31[T), (a (7,
(1e))
1/ " (oxpo(+H|xxx==))
Drooso'' (aaloo("l=".T))llooe"e"""" T')l ,et)) + (oa ( TD)
(By definition of KL divergence)
(1I6)
2%/ (1 1
Du (va(s')
(:' KL divergence 0),
(11g)
Plugging Eq. 11g into Eq.
10
we get the final ELBO:
log ( .c)
(12s)
2 F 1 I/o ( - Da. (
Dau (
(By substitution)
(126)
Boy ml re hosp:///ooocooom//arrott... Du (2 ( (399
Daa (2o(= (T)lal
(Final ELBO)
(12c)
where the decoder pv serves the conditional prior network and is replaced by the encoder le serving as the surrogate posterior network. as can be seen to be producing two intermediate bottleneck distributions: (a) (= '7) transforms inputs into a distribution over global latent variables, (b) conditioned on the global latent variables, (=*1= ",l]=,T'' TT*
gathers the t-th task inputs and learns another distribution over the task-specific latent variables. The task-specific latent variables and their corresponding input covariates X are then used by the deterministic decoder pa to decode their corresponding logit h.. It indeed this dependency of ps the task identifier & that makes inference a challenging task in real-world CL settings.
A.2 Single Task NPCL and its ELBO
Single-Task (ST) NPCL preserves all but the inter-task cross attention CA9:t and the global distri- bution encoder G. layers from the architecture of the NPCL (Sec. 4.3 The task-specific latent variables zº are thus derived as:
( ~ N vi (6)) NN o(),,
(13)
where of and a carry the same meaning as Eq. T6.. For a fair comparison in Table we M to be the same the total number of global ancestral samples N in the NPCL, The corresponding
16

---

ELBO amounts to:
log (
(14a)
/ B logno(Y zt) DxL (
(Dropping 20 from Eq. (12c)
(14b)
A.3 ELBO for the NP [14] and the ANP 127
The NP [4] and the ANP 27 employ single latent variable 20 to model the global correlation of all tasks. In particular, compared to Sec. 4.3] the task-specific self-attention layer S and the task-specific distribution encoder of is no longer required. While this enables knowledge sharing among tasks, NPs and ANPs are limited in modeling finer intra-task stochastic factors. The ELBO can be given as:
Po (Y7 (X,,C) R (0 log (Y7 [x]] DxL
(15a)
(Dropping Oct from Eq.
(1cc)
(15b)
where 20 is derived in way similar to Eq. (6)), and the inputs Xy and C belong to [0, tasks without relying on the task labels for being encoded.
B NPs for Meta-Learning (ML) vs Continual Learning (CL)
Resemblance. For both ML and CL settings, we have multiple tasks and would like to learn an NP with flexible conditioning that generates task-specific functions. Specifically, for each task t. we have Y = lo:o (X*; 2²) where the latent variable 2 is conditioned on the task-specific context C². As 2 task-specific, this, turn, makes fe task-specific.
Differences. There are two major differences between the conventional NP for ML and our proposed NP for CL (NPCL):
1. Architectural difference. In standard NPs, z² is conditioned only on the task-specific context C while in the NPCL, z is conditioned ; where 2 is the global latent derived from the global context. The added conditioning of global latent reflects the need for cross-task knowledge transfer in CL and thus endows a two-level hierarchy into our model.
2. Functional difference. The functional difference between the ML and CL (inference) settings call for another adaptation the NPCL. Namely, MI aims to learn NPs that learn F(a:0) that can generalize to new tasks. That is why NPs tested on new task t, given its context C. On the other hand, in CL, we wish to learn an F that can perform well on all the seen tasks there is no new task during inference. Given the absence of labeled context during inference, our test-time context samples are thus a subset of the training data from the tasks seen so far.
In a nutshell, CL calls for learning general function that task-specific but also performs well on all the seen tasks subject to limited rehearsal memory. To achieve this, leverage an NP's ability of generating task-specific functions with flexible conditioning. But instead of using the conventional NPs for ML, we propose a hierarchical model introduce more inter-task knowledge sharing, thus tailored for the CL problem.
C Further on the NPCL Architecture
In the following, denote multi-heas dot product self-attention 48 by SA(K, V, Q) where K, V, and Q are the keys, values and queries, respectively. The equivalent notation for cross-attention is CA(K, V, Q).
Latent Encoder. The latent path learns the functional prior and posterior from the context and the target sets, respectively. Each label-concatenated input is projected as 4lat MLP(([x; v(); then
17

---

subjected to two attention operations. First, per-task projections form the keys, values, and queries to taskwise self-attention layers S. T ( ($)
(b o + s that produce order-invariant encodings ii=1 serve as the keys, values, and queries to the cross-attention layers CAffi (s), sf, s.) s + sf that enrich their order-invariance from intra-task sf to inter-task 5 st and و are then used to derive the global n: and the task-specific latent variables 2².
of over the task t. Second, all encodings {s) (s0:2)
Such globally attended inputs are passed in parallel to two MLP layers constituting the global distribution encoder ei whose outputs together parameterize the global distribution . (PG. oz)) over pn + (μc, oz) Samples ) drawn from this distribution the input set, i.e., (s()) {s{ i==
proxies for the variables capturing the global correlation over all tasks in the input set. It is indeed
this sampling step that induces the stochasticity into the learned posteriors of the NPCL.
To model finer task-specific distribution for task /. conditioned on the global distribution, we retain the task-specific self-attended representations s and concatenate these with the global latent variables {z {= Jim1 to produce N distinct encodings per input point. These encodings are then passed through the t-rh task distribution encoder eba that again constitutes a mean and a variance MLP head and produces outputs that parameterize the t-th task distribution N(H,, of), i.e., 4 {= + + (pt, of). Samples {x]"),=,
Ji=j drawn from each such distribution thus capture the per-task stochastic factors. To limit the randomness in the learned prior/posterior, we use M 1. The latent encoder
thus outputs subtotal of N (t + 1) encodings per input point.
Put together, the global and task-specific latent variables can be derived as:
( Ji=i ~ ((86) {
(16)
( 29), o' (s),, = NNH,, V.,
where G/ and the are the global and per-task distribution encoders, respectively.
Deterministic Encoder. The deterministic path is similar to that of an ANP [27 where the context projections qdl = MLP((I;; vil) form the keys, queries and values for self-attention operation, SAde (Φ Φdet Tl(t) ( + rı. The resulting invariant context representations {r:]m= are fed as values to a subsequent target-to-context cross-attention operation C'A The keys x; and queries x. for CdA come from the context I; E Xx and target v. E Xy covariates, respectively, i.e, C SC, F. r. where r. is invariant the order of context.
Decoder. Different from other NP variants, the NPCL decoder adopts separate decoding mechanisms during training and inference. At train time, we use the available task identity to filter the true N out of N * (t+11) 1) latent path outputs to be processed by the decoder. After this, the decoder concatenates target input x². E Xf with its N true task-specific latent variables { f obtained from the latent path and its invarianva feature r. obtained from the deterministic path thus resulting in N distinct inputs. For N > 1 samples of 2,, we first make N copies of x,, and r. each, and then concatenate these with each z', po thus performs the projection po([s.;; ([r,; (=())))) x - h.
where x E f+++0 and h, are the logits of an MLP classifier for the target label ye. We detail the dnference-time decoding in Sec. 4.5
D Experiments and Reproducibility
Configuration. For a fair comparison with the benchmarks of Buzzega et al. [4], we fix the batch sizes for new task's 's samples and for replay samples to 32 each for the class-IL datasets and to 128 each for the domain-IL datasets. Both the context and target datasets use the same set of augmentations. For S-CIFAR- 10, S-CIFAR- 100, and S to both stream and buffer examples following Buzzega et al. 44] and Boschini et [. For each setting of memory size on each dataset, the NPCL adopts the same learning rate (LR) as reported in
we apply random crops and horizontal flips
Buzzega et al. [4] and Boschini et al. 3 However, the NPCL training additionally relies on linearly increasing the learning rate (LR) over period of 4000 iterations for class-II. and 40 iterations for domain-Il. II. settings. We further apply gradient clipping 40 on 1.2-norm of the NPCL, parameters with cap of 10000.
18

---

Hyperparameter tuning. We arrive at the best hyperparameter settings for each of our datasets through grid search over a validation set made 10% of the training set on each dataset. The search range for number of samples N from the global distribution N (PG og) ) is [2, 5, 10, 20, 50, 100]. Out of these, we found N 50 during training and N 10 during evaluation to perform better in general across all settings.
Similarly, we conducted grid search over the batch size of the context [1/16, 1/8, 1/11,11///// 1/2, 1.25] of the original (target) batch sizes for each of the dataset. In general, we found that fixing the context batch size 1/8 of the target batch size performed better across all datasets. Such context batches are sampled from a context dataset Di for each task t. Dig is itself created by randomly selecting a subset of the training samples for each class at the beginning each incremental training task. To decide on the size of the subset for each class, we ran grid search over the range (50, 100, 150, 200) samples per class and found that incorporating 100 random samples per class into 2 performed well across all datasets.
over the range
Finally, to decide on the loss weights a, 3, and for D DG. LUR, and LiR+ we ran gridsearch for each over possible values [0.0, 0.01, 0.05, 0.08, 0.1, 0.15, 0.2, 0.4]. We report the best settings across datasets in Table
S-CIFAR-10 S-CIFAR-100
S-Tiny-ImageNe
P-MNIST
R-MNIST
a
0.05
0.05
0.01
0.1
0.1
3
0.01
0.01
0.01
0.05
0.05
0.2
0.08
0.05
0.1
0.1
0.1
0.1
0.1
0.15
0.15
Table 9: Hyperparameters for loss contributions that were tuned on validation sets for each dataset.
E
On the number of Monte Carlo samples
Figure(6)howe shows the effect of the number of Monte Carlo (MC) samples for global N and task-specific
observe two favorable spots in M latent variables accuracy during inference. In particular, terms of accuracy, one centered around (M = N = 50) and the other around (M = 10, N 20). It is worth noting that the total number of inference time MC samples grow quadratically with the number of tasks f, i.e., ONN Mt), and that a higher number of samples leads to a larger computational overhead. For instance, based Eq. (9) in the main paper, the inference S-CIFAR-100, i.e., t 10, given the two favorable spots amounts to selecting the set task-specific module predictions with the least uncertainty from total of (a) 50 10 500 predictions using (M = 1, N 50), and (b) 10 x 20 x 10 = 2000 predictions using (M = 10. N 20). We, therefore, opt for the more efficient setting of (M 1, N 50) throughout experiments in the paper.
the 10-th task of
Average Accuracy
37.5
36.5
30.0
M
10
20
35.0
53
34.5
100
34.0
2
10
23
50
100
Figure 6: The effect of the number of Monte Carlo (MC) samples of global (N) and task-specific (M)
latent variables on S-CIFAR-100 (Msize 500) accuracy.
19

---

Method
S-CIFAR-10
P-MNIST Domain-L
R-MNIST
Class-IL
Domain-
oEWC
-91.64
-36.69
-24.59
-95.78
-27.91
-22.91
LwF
-96.69
Mize
200
500
200
500
200
500
ER
-61.24
-45.35
-22.44
-14.90
-8.24
-7.52
GEM
-82.61
-74.31
-29.38
-18.76
-11.51
-7.19
A-GEM
-95.73
-94.01
-31.69
-28.53
-19.32
-19.36
iCaRL
-28.72
-25.71
FDR
-86.40
-85.62
-2062
-2.80
-1..31
-6.70
GSS
-75.25
-62.88
-47.85
-23.68
20.119
-17.45
HAL
-69.11
-62.11
-15.24
-11.58
-11.71
-6.78
DER
-40.76
-26.74
-13.79
-8.04
-5.99
-3.41
ANP
-62.00
49.18
-28.79
-16.44
-12.08
-10,63
ST-NPCL
46.91
-32.50
-17.03
-12.40
-7.9
-8.11
NPCL (ours)
-39.11
-27.62
-12.81
-8.60
-5.70
-4.10
Table 10: Backward transfer scores for the experiments in Table Best results are in red. Second best results are blue. All runs of the ANP, the ST-NPCL and the NPCL in the CL settings rely on experience replay (ER).
F Results: Backward Transfer
Table TO reports the backward transfer for the accuracy scores mentioned in table T We further compute the backward transfer based on uncertainty scores to study the effect of forgetting on uncertainty. Fig. shows the correlation between backward transfer of accuracy and uncertainty for the domain- IL datasets P-MNIST and R-MNIST.
P-MNIST
R-MNIST
20
Parkwoard trander scor
Rar Barkabart Frandr ocorr
10
10
0-
-10
accuracy
accuracy uncertainty
-20
Task
Figure 7: Backward transfer scores of tasks based on accuracy and uncertainty on domain-IL datasets with [M] = 500: higher negative backward transfer on accuracy correlates with a higher positive backward transfer on uncertainty and vice-versa. For better visibility, the uncertainty- backward transfer scores have been scaled by factor of 100.
G
Ablations
G.1
On the effect of regularization on the learned distributions
We record the per epoch L1-norms of global and task-specific means and variances on the last incremental task (task 4) of S-CIFAR-10. As shown in Fig. [8a] and Fig. [(b) regularizing the global distribution (GR) alleviates forgetting by limiting the learning of the global and the current task's (task 4) means and variances. This is evident through larger L1-norm of means and smaller -o-m of variances when GR 0, i.e, +TR setting. On the other hand, excluding all the objectives, i.e. the Bascline NPCL as well as excluding TR from the learning objectives, i.e., +GR setting lead to
20

---

relatively unstable evolution of the past task means and variances, hence characterizing an increased forgetting. Including both GR and TR in the objective. i.e., the NPCL helps find balance between preserving the global and the past-task distributions while facilitating the learning of the current task distribution.
Harr
304
vnt
200
180
354
100
Effect the global and task task-specific mcans.
130
2337
Ko-de-1 cark 26t
312
1271
(b) Effect on the global and task-specific variances.
Figure 8: Effect of the proposed global (GR) and task-specific (TR) regularizations on the learning of global and task-specific means and variances during the last incremental training task (task 4) of S-CIFAR-10. The NPCL uses both GR and TR while the baseline NPCL uses neither of them.
G.2 How does forgetting effect uncertainty?
Fig. 9 ablates the average accuracies and uncertainties of each task head predictions over the test set of each task at the end of incremental training on S-CIFAR-100. Similar to S-CIFAR-10 (Fig.
4 we observe that the accuracy of predictions made by true task heads are higher than the rest. For predictive uncertainties, the trend the opposite. Also, more recently trained tasks show lesser forgetting both in terms of accuracy (higher values) and uncertainty (lower values). This generalizes
21

---

our conclusion on S-CIFAR-10 10 regarding the outreach of forgetting in CL going beyond accuracy and to other aspects of learning such as the model's 's predictive confidence.
1:
94
62
13
52
6.4
7.
63
1/
0.32
0:34
0:37
0:42
0.23
0.51
0:51
2:41
0.45
0.58
0.4
4.1
5.8
31
50
0.74
0.18
6:41
0:51
01:44
0.54
0.42
9.31
0.5
11:0
17
811
7.2
944
4.4
5.4
4.5
4.3
0.85
4.42
6:00
D:7
0.33
0:53
0.56
0:45
0.35
84
9.8
3.
5.3
-30
0.42
45
0.41
0.4
0.38
059
0.6
0.47
0.48
0.5
05
72
8.1
6.3
2/2
5.5
0.85
C.42
0:46
0.90
0.15
0.37
057
0.44
74
30
0.5
3.2
4.6
0.45
2:45
6.45
040
0:43
6.55
0.4
0.2
0358
0.54
-10
0.77
4.5
4.4
2.0
1.3
4,7
1.4
0.45
0:10
0:31
0-1
0:17
0.07
0
Nodule
(a) Accuracy heatmap
(b) Uncertainty heatmap
Figure 9: Heatmaps depicting the average accuracy and uncertainty of individual task test sets per task head on S-CIFAR-100 with [M] 500 over an individual run.
G.3 On the storage gain of the NPCL over DER
Table II compares the total episodic memory sizes of the NPCL (ours) and DER 4.. We report storage sizes as the dimension of a single 1-d vector constructed by flattening all the vectors that need be stored by each method in the episodic memory. Namely, the NPCL stores of fixed dimension Rooo where is the total number of tasks in dataset and o is the output size of the mean and variance heads. On the other hand, DER stores [M] number of logits of dimension RlNel where No denotes the total number of classes in CL dataset. As a result, the NPCL has significant storage gains on settings with either large number of classes or larger memory size. It is worth noting that both the NPCL and DER rely on storing original input images and therefore, our comparison does not take the inputs into account.
+ 2 vectors
Method
S-CIFAR-10
S-CTFAR-100
S-Tiny-ImageNet
P-MNIST
R-MNIST
Mas
200
500
500
2000
200
500
200
500
200
500
DER NPCL (ours)
2000
5000
50000
200000
40000
100000
2000
5000
2000
5000
3272
3572
6132
7632
5832
6132
1544
1844
1544
1844
Storage gain (%)
-63.6
28.56
87.746
96.184
85.42
93.868
22,8
63.12
22.8
63.12
Storage size comparison of the NPCL with DER across different experimental settings of Table Gains are marked in bold.
Table 11:
G.4 On the importance of correct context
To study the significance of correct task-specific context C' for the NPCL, we design a simple experiment. While training on an incremental task t > 0 onwards, after having derived the global latent samples { J==1+ we tinker with the flow of the task-specific context points C' to the different task-specific encoders of the NPCL. In specific, instead of directing C to the I-th encoder, we misdirect to the task encoder j 3 j * I where, j is chosen at random from the pool of all seen task ids. Note that the presence of such randomly allocated context points during training implies that we now have a noisy task-specific prior qo (2 C²) to match in the ELBO, i.e., a corrupted second term the right-hand side of eq. (5). We keep all our other training settings (including the loss coefficient values) unchanged.
Table G.4 compares the performance of the NPCL with the noisy task-specific priors on the two different memory sizes of S-CIFAR-10. While the performance gain of the NPCL over its noisy prior counterpart remains significant, we observe that in comparison with the ST-NPCL (which lacks hierarchy), the presence of noisy priors degrade the performance of the NPCL (which has hierarchy) further as the replay memory size increases from 200 to 500. This is because, with larger memory size, more context points from past tasks are diverted to the random task components during training. This leads to a noisier task- prior matching. Such noisy priors further lead to higher
22

---

fluctuations in the accuracy, as marked by the larger standard deviations in their accuracy over the ST-NPCL and the NPCL. This validates the fact that the conditioning on the correct task-specific context remains crucial to the performance of the NPCI..
Average accuracy over 10 (%)
Method
S S-CIFAR-10 (Maze 200)
S-CIFAR-10 10 (MMsie 500)
ST-NPCL (w/ only per-task latent)
54.6 + 2.14
65.22 + 1.89
NPCL w/ noisy lask-specific priors
59.41 + 3.0
65.1 + 2.31
NPCL
63.78 1 1.7
71.34 1.48
The importance of correct context: Comparison of S-CIFAR-I0 accuracy (over 10 runs) of Single Task NPCL (ST-NPCL), the NPCL with noisy/corrupted task-specific priors and the standard the NPCL. The presence of noisy priors degrades the performance of the NPCL up to the extent of falling behind the ST-NPCL (without hierarchical structure) on the setting with larger memory size.
Table 12:
G.5
On out-of-the box novel data identification
Our novel data identification experiments use the S-CIFAR-10 and S-CIFAR-100 datasets interchange- ably as DId and DOOD given the high degree of similarity between number of their classes (2 Namely, while evaluating the NPCL trained on S-CIFAR-106 we consider the entire CIFAR-10 10 test set as DOOD whereas the evaluation of the S S-CIFAR-II 10 model treats the test set first 10 class labels of CIFAR 100 to be DooD. Further, for an incremental task f, the test sets for (0, t] tasks make up for ID data DID-
As shown in Table 13 the variances computed using either of our proposed metrics on Dnd are up magnitude lower than those on DOOD. This trend is evident across the incremental evaluation steps even if the differences in the variances between DID and DOoD slump with the further arriving tasks. Moreover, for the model trained on the more challenging S-CIFAR-I00 setting, we observe that the differences between the DID and DOoD variances even grow during the course of incremental training. This implies the potential perks of enabling the inter-task knowledge sharing among the NPCL parameters in CL setup.
Incremental step
CIFAR 100 on S. CIFAR 10 model
CIFAR 10 on S. CIFAR 100 model
Dm (6)
Doon (6)
Din (H)
Doon (H)
Dio (8)
Dooe (6)
Din (HI
Doon (HI
8
9.3e
8.4e
1.5e
8.9c 8
1.5c
lc
1.4e
6.3e
6
1.7c 8
2.3c
6.2e
6.7e
2.1e
1.2e
3.Ge
4.4e
1.5e
8.1e
4.8e-6
4.6e-6
2.2c
1.1e
2.5e
3.5e
1.2e
7.1e
1.7e-
4.6c
1.1e-4
8e-
2e-8
4.4c
1.2c
6T
1.3e
4.1e-1
8.5e
4.e
1.3e e
8.3e
4.9e-7
1e-8
3.2e
6.7e
3e
6.9e
2.5e
4.7e-1
10
3.3e
5.16-7
2.5e
3.5e
Table 13: Average variances over softmax (d) and entropy (H) scores of incremental models on in-distribution (ID) and out-of-distribution (OOD) test sets using N 50 samples
G.6 On instance-level model confidence evaluation
For each target instance x., the instance-level model confidence evaluation framework 17 uses the N predictions obtained from stochastic sampling compute: (a) the prediction interval width (PIW) between the [2.5, 97.5) percentile range the N predicted classes, (b) the paired two-sample 1-test
10 to evaluate the significance of difference between the mean predicted probabilities for the top-2
2The labels for first ten CIFAR-100 classes are the same
cifar100 and that for CIFAR-10 10 classes are the same sttrpp://uuggggggeeoooo//tttttt
23

---

most predicted classes. As prerequisite to the latter test, we first verify the normality assumption of the probability differences for the NPCL (Fig. Top.
Similar to Fan et al. 10 after computing the PIW per test instance, we split the instances into groups by the correctness of the majority-vote predictions, obtain the PIW of the true class per instance, and compute the mean PIW of the true class within each group. For t-test evaluation, we compute the mean accuracy per group of the test instances split by their é-test rejection status.
Q-Q plots for the differences in probability between the most and the second most predicted class.
Figure 10:
Class Accuracy
PIW
Accuracy by t-test status
Not Rejected
Correct
Incorrect
Rejected
82.30
74.17
102.21
83.37
50.00
94.00
62.90
79.86
94.07
80.00
74.00
54.92
68.48
74.14
64.29
71.50
65.42
74.32
72.06
25.00
5
84.80
92.93
106.90
85.37
22.22
76.50
75.22
103.58
76.58
60.00
94.20
104.9
129.56
94.39
3,00
90.50
81.10
127.06
91.12
22.22
96.90
72.81
110.86
97.00
66.67
10
96.30
80.60
109.56
96.48
60.00
Table 14: PIW (multiplied by 100) and tesest results for classes inferred from their respective task heads after S- CIFAR-100 training.
24

---

H
Incompetence of Dot-product attention
Top-15 context points attended per query
QQuery
Tosc
(a) Cross-attention visualization
0.58
0.0
(b) Self-attention visualization
Figure 11: Scaled dot-product attention visualization: (a) top- 15 context (buffer) points attended for 4 randomly chosen queries (test set samples). The queries made after training S-CIFAR- 10. The sizes of the points correspond to the attention values while the colors denote the tasks they belong to. (b) self-attention weights of context points when all feature values arranged in ascending order (along x and y-axis) shows that the ANP [27] mostly attends to the lowest or the maximum values in the context dataset.
25


Reviews:
[{"review_id": "D6dg64igaX", "reviewer": "Reviewer_qdzJ", "summary": "This paper introduces an uncertainty-aware continual learning framework based on neural processes. The proposed method casts CL into a hierarchical latent model from the global variables to the task-specific variables, and the corresponding regularization terms for each to ensure minimum forgetting along the course of new task learning. The naturally reliable uncertain estimation of NP facilitates class incremental setting by allowing accurate task-specific latent selections. ", "questions": "Please see the Weakness section. \n\nHow is the uncertainty estimation of the proposed method quantitatively? Some further evaluation using metrics like ECE can be more insightful. ", "limitations": "The authors discussed the limitations of the proposed method very comprehensively in Section 6. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The presentation of this paper is mostly clear, with the equations properly delivering the ideas. \n\n- The idea of applying NP in CL, explicitly modeling task-specific latent variables, and uncertainty quantification for class-incremental CL are very intriguing.\n\n", "weaknesses": "**Writing and clarity**\nThe writing can be further improved. \n\nFor example, in my understanding, following Line 163, the two regularization terms are introduced to counter the forgetting instead of being 'two key aspects of forgetting in NPCL.' \n\nIn line 114, the authors mentioned 'uses generative factors', while the term 'generative factors' is not appropriately explained here or in the following sections. \n\n\n**Quantitative comparisons**\n\nWhile clear improvements over some baseline methods and other NP-based methods are reported, I believe the compared CL methods are all relatively classic but old. More up-to-date might be helpful to better position the proposed method. \n\n\nOther minor points:\n\n- Figure 2 is too small \n\n- The authors might consider refining the formats of the reference list, as currently the formats are very inconsistent. \n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful reading and remarks on our work. In regard to the questions raised, we provide our explanations below:\n\n- **Writing, figures and referencing formats**: Thank you for pointing these out. We will be incorporating these into our paper.\n- **CL Baselines**: Thank you for your suggestion. We agree that CL is a fast-moving field and many new works are emerging. The CL baselines opted by us have been frequently used in a range of other recent CL papers [1-4]. We thus use these clean and powerful baseline models to clearly validate the effectiveness of the proposed NP-based model in CL. We will expand the group of methods used in our comparison.\n- **Quantitative results for uncertainty**: Thank you for your suggestion. We agree with the idea of quantitative uncertainty comparisons, and following your suggestion, we report the calibration error of NPCL with other methods to compare how well their predicted probabilities reflect the true probabilities. In particular, we report the Expected Calibration Error (ECE) [5] and the more robust Adaptive Calibration Error (ACE) [6] of ER, DER, a baseline Attentive NP (ANP) [7], and NPCL (ours) on the S-CIFAR-10 and the S-CIFAR-100 settings with memory sizes 200 and 500, respectively (see *Figure 1* in the pdf attached with the *global* response).  ECE and ACE are widely-used metrics for uncertainty estimation. As shown in Figure 1 in the attached pdf, NPCL consistently produces the most well-calibrated output probabilities. Moreover, even the baseline ANP, whose accuracy scores are comparable with that of the ER (see Table 1 in the main paper), produces predictions with lower calibration errors than the ER method. \n\n\n**References**:\n\n[1] Kim, S., Noci, L., Orvieto, A., & Hofmann, T. (2023). Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning. *CVPR*.\n\n[2] Sun, Z., Mu, Y., & Hua, G. (2023). Regularizing Second-Order Influences for Continual Learning. *CVPR*.\n\n[3] Boschini, M., Bonicelli, L., Buzzega, P., Porrello, A., & Calderara, S. (2022). Class-Incremental Continual Learning Into the eXtended DER-Verse. *IEEE TPAMI*, 45, 5497-5512.\n\n[4] Gong, D., Yan, Q., Liu, Y., Hengel, A.V., & Shi, J. (2022). Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning. *CVPR*, 109-118.\n\n[5] Guo, C., Pleiss, G., Sun, Y., & Weinberger, K.Q. (2017). On Calibration of Modern Neural Networks. *ICML*.\n\n[6] Nixon, J., Dusenberry, M.W., Zhang, L., Jerfel, G., & Tran, D. (2019). Measuring Calibration in Deep Learning. *CVPR Workshops*.\n\n[7] Kim, H., Mnih, A., Schwarz, J., Garnelo, M., Eslami, S.M., Rosenbaum, D., Vinyals, O., & Teh, Y.W. (2019). Attentive Neural Processes. *ICLR*.\n\n\n\n"}, {"review_id": "ssmHdPlsO4", "reviewer": "Reviewer_fvgr", "summary": "This paper presents an uncertainty-aware continual learning framework that utilizes Neural Processes (NPs). The NP model employs a hierarchical latent variable model in conjunction with an experience replay buffer, where a global latent variable captures inter-task correlations and task-specific latents encode more detailed knowledge. To prevent catastrophic forgetting, the method regularizes a Jenson-Shannon divergence between current and past distributions of latent variables. The paper employs entropy for uncertainty quantification.", "questions": "- Why was Jensen-Shannon Divergence (JSD) chosen as a regularization metric? Have other distribution divergence metrics, such as Kullback-Leibler Divergence (KLD), been explored?\n- A key strength of the NP family is its few-shot learning capability. Given that the size of the replay buffer can be a limiting factor in some applications, would the proposed approach still be effective with a much smaller buffer size (e.g., 10)?\n- How do accuracy and uncertainty depend on the size of the Monte Carlo sampling (N, M)?", "limitations": "The authors have addressed the limitations.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The concept of regularizing latent distributions between preceding and current tasks is both simple and intuitive.\n- Experimental results demonstrate that the proposed approach is competitive with state-of-the-art continual learning methods, while also being capable of quantifying model uncertainty.\n- The proposed method only stores two vectors for each task, showcasing memory efficiency in terms of experience replay.\n- Generally, the paper is well-written and easy to understand.", "weaknesses": "- Given that the encoder includes self-attention and cross-attention layers, the proposed method exhibits quadratic complexity relative to the training data. This complexity could prove problematic when the training dataset is substantial (e.g., 10000).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful reading and remarks on our work. In regard to the questions raised, we provide our explanations below:\n\n- **Choice of regularization metrics**: Thank you for your suggestion. We indeed consider KL-divergence as a possible alternative for our regularization metric. The table below reports the average accuracy of NPCL on S-CIFAR-10 (memory size = 200) and S-CIFAR-100 (memory size = 500) using JS and KL divergences as the regularization metrics. Overall, JS-divergence offers consistent gains over its asymmetric KL counterpart in both settings. This could be because KL-divergence induces harder constraints on the distributions of the global and task-specific posteriors in matching their respective past task priors. However, in CL, the evolving data distribution means that our global and task-specific posteriors keep changing as the model’s parameters adapt to fit the newly arrived data. Therefore, the symmetric JS-divergence, which has a relatively relaxed mid-point anchor between the current posterior and its past task prior to match, offers a better trade-off between the plasticity and stability of the model. Our observation is in line with previous work on NPs that use JS divergence as a superior choice for the prior matching loss term [1].\n\n|     NPCL          | S-CIFAR-10 | S-CIFAR-100 |\n|---------------|---------------------|----------------------|\n| JS-divergence | 63.78 +/- 1.7       | 37.43                |\n| KL-divergence | 62.55 +/- 1.58      | 35.82                |\n\n- **Few-shot replay setting**: Thanks for your suggestion. To evaluate the performance of NPCL in a few-shot replay setting, we report the accuracy of NPCL (ours), ER [2] and DER [3] on S-CIFAR-10 and S-CIFAR-100 with memory sizes of 5 and 10, respectively (see Fig. 2 in the global response PDF). Further, as pointed out by reviewer **qdzJ**, we also compare the expected calibration errors (ECE) of these models to study how well their predicted probabilities reflect the true likelihood of the labels.\n\nAs shown in Figure 2 of the PDF in the global response, NPCL consistently outperforms ER and DER in both memory size settings in terms of accuracy (higher the better) and ECE (lower the better). Interestingly, on the memory size of 5 on S-CIFAR-100, we observe that while the ER outperforms the DER in terms of accuracy, the latter still offers more confident predictions characterized by a lower ECE than that of the ER. NPCL, on the other hand, still exhibits the lowest ECE in this setting. This observation depicts that on very small CL memory settings, while regularizing the predicted logits towards their old forms – as done by the DER – helps maintain better predictive confidence than the simple ER, regularizing the task distributions towards their old forms – as done by the NPCL – remains the superior way to enhance the model’s predictive confidence.\n\n- **Effect of Monte Carlo samples M and N**: Please refer to our global response alongside Figure 3(a) of the attached PDF.\n\n- **Quadratic complexity of NPCL relative to the training data**: This is correct. One possible solution for improving the quadratic complexity of the current implementation could be to replace the self-attention and cross-attention blocks as well as the global average pooling operations entirely with Multi-Head Attention Pooling (MAP) [4]. In particular, the inducing points variant of the MAP could be used to summarize the set of $n$ input points with $m$ inducing points where $m << n$. This way, the time complexity could be brought down from $\\mathcal{O}(n^2)$ to $\\mathcal{O}(nm)$ where $n$ is the size of the input context/target set and $m << n$ is the number of inducing points encoding the original $n$ points. \n\nWe have currently performed a preliminary experiment with MAP where we replace only the global average pooling operations with MAP (please refer to our response to reviewer **1ufQ**). Per the advice from reviewer **1ufQ**,  the aforesaid experiment was done from the viewpoint of improving accuracy. We thus plan to explore MAPs for improving the computational complexity of NPs in CL as a potential direction for future work.\n\n**References**:\n\n[1] Wang, J., Lukasiewicz, T., Massiceti, D., Hu, X., Pavlovic, V., & Neophytou, A. (2022). NP-Match: When Neural Processes meet Semi-Supervised Learning. *ICML*.\n\n[2] Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., & Tesauro, G. (2018). Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference. *ICLR*.\n\n[3] Buzzega, P., Boschini, M., Porrello, A., Abati, D., & Calderara, S. (2020). Dark Experience for General Continual Learning: a Strong, Simple Baseline. *NeurIPS*.\n\n[4] Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., & Teh, Y. W. (2019, May). Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning (pp. 3744-3753). *PMLR*.\n"}, {"review_id": "NjU5w5oYS7", "reviewer": "Reviewer_v3Yx", "summary": "This paper suggests tackling continual learning (CL) with a neural process model (NP).  The authors describe a hierarchical latent variable model, with one global latent, and t per-task latents.  As is common in NPs, the latent posteriors are a function of a context of datapoints with their corresponding labels, and then a decoder maps the latents to an output prediction.  \n\nDuring training time, the model is trained with access to task labels, and uses data from different tasks to train the corresponding task-specific latent posteriors. A replay buffer stores and reuses data from old tasks during training, and additional regularization terms are introduced, in order to minimize effects of drifting away from previous tasks ('forgetting').\n\nAt test time, there is no access to task labels, and therefore the model is run for all tasks, and the output with the least entropy is used. \n\nThe model is compared to various CL approaches and is shown to achieve and sometimes surpass the state of the art. The paper describes more ablations and analysis of the results.", "questions": "1. The paragraph in line 110 is not clear. What does it mean to optimize parameters of $\\mathcal{D}^t$? \n2. In equation 3, does the context define the task (as modeled in standard NPs)? If so why are there superscripts of all the tasks 0:t? \n3. In equation 4 why are $z^t$ conditioned on the contexts $\\mathcal{C}^t$? If the task id is given then the context doesn’t have any information.\n4. The number of samples described in line 151 is not clear. If every point is mapped to one task then only the corresponding latent needs to be sampled, making the number of samples M. Is this wrong?", "limitations": "I did not find any unaddressed limitations or potential negative impact in this work.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The main strengths of this paper are:\n1. An interesting model with non-trivial components put together:\n    - the hierarchical structure of the latents\n    - the difference treatment of training time and test time with respect to access to context latent\n2. The result seems to surpass previous approaches\n3. Extensive analysis of the results", "weaknesses": "The main weakness of this paper is that modeling continual learning as meta learning via neural processes is not exactly valid. \nSpecifically, NPs assume that the dataset contains many different functions or tasks, and one of the main challenges is that the task id is not accessible and has to be inferred. This is the reason a \"context\" of labeled points is used - to infer the task at hand. \nIn the setup described in the paper, it is assumed that the task id is accessible at training time, which allows designating different parts of the model to different tasks.\nAt test time, the model uses a context coming from the training data with little overlap to the task of the test data. This is a major difference from NPs.\nI think that the presented model is interesting and can have a significant contribution to the community but that the good performance exhibited is not connected to NPs as stated, but rather merely a result of the higher capacity of hierarchical latent variable models.\nSince the model is conditioned on training points during evaluation, perhaps a better way to present it is as a non-parametric method (e.g.[1]) rather than a neural process.\n\n[1] Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning. Jannik Kossen, Neil Band, Clare Lyle, Aidan N. Gomez, Tom Rainforth, Yarin Gal", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful reading and remarks on our work. In regard to the questions raised, we provide our explanations below:\n\n- **Validity of NPs for CL**: We believe that the remark against the validity of our setting stems from a misunderstanding between the standard meta-learning (ML) settings (that NPs were initially designed for) and the continual learning (CL) setting (ours). Namely, in ML testing, we are interested in how the trained model performs on new tasks, each of which is associated with a \n  dataset $\\mathcal{D}$ specified for one task at a time. $\\mathcal{D}$ thus has a support/context set and a query set. As also mentioned in the review, the applications of NPs to ML keep the task id inaccessible such that the model learns to infer it from the context. This is because what defines a new task in ML is just the context points.\n\nHowever, in CL with a replay memory, our training and testing set could both comprise samples from all the tasks. That is to say, the testing phase of CL does not have any new tasks, which is the fundamental difference from the ML problem setting. Therefore, we believe that it is valid that our training and test data use the same set of context points. Consequently, it is also valid for us to access the task IDs of the context set points to derive more informative task-specific priors. \n\nIn practice, going beyond the ML setting, there have been several methods in the NP literature that use task id for *designating different parts of an NP model to different tasks* for multi-task learning [1,2] and *using the training samples as context at test time* for large-scale classification tasks [3,4].\n\n**Therefore, we believe that our use of NP for CL is valid and sincerely hope the reviewer reconsiders their rating.**\n\n- **Effect of context on the performance of NPCL**: To verify that the access to task ids make the task-specific context $\\mathcal{C}^t$ more informative, we design a simple experiment. While training on an incremental task $t > 0$ onwards, after having derived the global distribution samples $z^G$,  we tinker with the flow of the task-specific context points $C^t$ to the different task-specific components of the NPCL. Namely, instead of directing $\\mathcal{C}^t$ to its *correct* $t-th$ task encoder, we redirect it to the $j-th$ task encoder where, $j \\neq t$ and $j$ are chosen at random from the pool of all seen task ids. Note that the presence of such randomly allocated context points during training implies that we now have a noisy task-specific prior, *i.e.,* $q_\\phi(z^t$|$z^G, \\mathcal{C}^j)$ to match in the second term of the right-hand side of our ELBO, *i.e.,* eq. 5 in the main paper.  We keep all our other training settings (including the loss coefficient values) unchanged.\n\n Figure 4 in the global response PDF compares the performance of NPCL with the noisy task-specific priors on the two different memory sizes of S-CIFAR-10. While the performance gain of the NPCL over its noisy prior counterpart remains significant, we observe that in comparison with the ST-NPCL (which *lacks* hierarchy), the presence of noisy priors degrade the performance of NPCL (which *has* hierarchy) further as the replay memory size increases from 200 to 500. This is because, with a larger memory size, more context points from past tasks are diverted to the random task components during training. This leads to a noisier task-specific prior matching. Such noisy priors further lead to higher fluctuations in the accuracy, as marked by the larger standard deviations in their accuracy over ST-NPCL and NPCL. *This validates the fact that the performance of NPCL is connected to the conditioning on correct task-specific context and not merely on the higher capacity of the hierarchical latent variable model.*\n\n- **Line 110**:  In line 110, we mention optimizing parameters *on* $\\mathcal{D}^t$ rather than *of* $\\mathcal{D}^t$. Here, $\\mathcal{D}^t$ refers to the training samples from task $t$ (building upon the notation from lines $93-94$ and $103$ in main paper). By joint optimization on $\\mathcal{D}^t$ and the memory $\\mathcal{M}$, we mean that if the entire parametric space of the model is allocated to learning $\\mathcal{D}^t$ and $\\mathcal{M}$, then interference from the future tasks is more likely.\n\n\n- **Superscripts in eq. 3**: As mentioned earlier, in a standard image classification setting like ours, the context points are not the only definition of a task given that the training and testing data in such settings can come from multiple tasks simultaneously. We thus use the superscripts $0:t$ to differentiate the task-specific context $C^{0:t}$ and target $X^{0:t}$. For the same reason, $z^t$ is conditioned on $C^t$.\n\n- **Line 151**: We are sorry for the confusion here. Conditioned on each $z \\in $ {$z^G$}$_1^N$, we rely on sampling one set of {$z^t$}$_1^M$ from each task-specific NPCL distribution. Thus, for $M=1$, we have $N$ samples from each of the $t+1$ task-specific encoder, leading to a subtotal of $N \\times (t + 1)$ latents.  Further, since $z^t$ for different tasks are sampled from distributions learned by different task encoders, this does not imply mapping every point to only one task. For a more clear explanation, we provide a rough sketch of our hierarchical sampling framework in Figure 3(b) of the attached PDF. Also, in Figure 3(a), we shed light on the effect of different $M$ and $N$ values on the accuracy of NPCL.\n\n**References**:\n\n[1] Kim, D., Cho, S., Lee, W., & Hong, S. (2022). Multi-Task Processes. *ICLR*.\n\n[2] Shen, J., Zhen, X., Worring, M., & Shao, L. (2021). Multi-Task Neural Processes. ArXiv, abs/2111.05820.\n\n[3] Wang, J., Lukasiewicz, T., Massiceti, D., Hu, X., Pavlovic, V., & Neophytou, A. (2022). NP-Match: When Neural Processes meet Semi-Supervised Learning. *ICML*.\n\n[4] Jung, M.C., Zhao, H., Dipnall, J.F., Gabbe, B.J., & Du, L. (2023). Multimodal Neural Processes for Uncertainty Estimation. ArXiv, abs/2304.01518."}, {"review_id": "FHHpqVRzNa", "reviewer": "Reviewer_1ufQ", "summary": "This paper proposes a new method for Continual Learning (CL) through Neural Processes (NPs) framework. Especially, they are inspired by MTP [1] which utilizes the global latent and task-specific latent for inter-task and intra-task knowledge representation. In this paper, the authors additionally regularize on parameter updates of MTP not too much be updated, and task-wise inference through an uncertainty quantification metric. Through this, they can achieve comparable performance with state-of-the-art prior works while maintaining smaller memory. \n\n[1] Buzzega, Pietro, et al. \"Dark experience for general continual learning: a strong, simple baseline.\" Advances in neural information processing systems 33 (2020): 15920-15930.", "questions": "- Can you additionally analyze why small $N$ is not working well with the additional experimental results for diverse $N$ values?\n- In your design, the sum pooling is used to get the global and task-specific latent. Could you replace it with Multi-head Attention Pooling (MAP) [1] (e.g., using the learnable additional token for the latent) and test it for a few tasks?\n\n[1] Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., & Teh, Y. W. (2019, May). Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning (pp. 3744-3753). PMLR.\n", "limitations": "The authors adequately discussed the limitations of their work. Thanks for their detailed analysis of their limitations.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- This paper proposes new regulations to utilize NPs for CL.\n- The prior works are well summarized and they discussed their model adequately.\n- They evaluated it on multiple benchmarks including class and domain incremental learning benchmarks with diverse prior works including NPs [1] and ANPs [2], and showed their model, the NPCL shows comparable performance to state-of-the-art methods such as DER [3].\n- They studied diverse ablations such as their regularization losses, latent types (e.g., when only task-wise latent is learned), task-wise inference, and context sizes. In this study, they showed their task-wise inference is important for their model's performance and the uncertainty on the logits of the task heads is a good measurement for task label estimation. Additionally, they also compared their model with DER for the storage overhead. For this aspect, NPCL shows more efficient storage utilization than DER.\n- They also showed the uncertainty on the task heads can be utilized to identify the novel data from out-of-distribution or not and uncertainty-based task-wise inference can show higher confidence for correct task labels.\n- Lastly, they clearly discussed the limitations of their model such as more computational overhead and inference time complexity caused by multi-head attention.\n\n[1] Garnelo, M., Schwarz, J., Rosenbaum, D., Viola, F., Rezende, D. J., Eslami, S. M., & Teh, Y. W. (2018). Neural processes. arXiv preprint arXiv:1807.01622.\n\n[2] Kim, H., Mnih, A., Schwarz, J., Garnelo, M., Eslami, A., Rosenbaum, D., ... & Teh, Y. W. (2019). Attentive neural processes. arXiv preprint arXiv:1901.05761.\n\n[3] Buzzega, P., Boschini, M., Porrello, A., Abati, D., & Calderara, S. (2020). Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33, 15920-15930.", "weaknesses": "- The motivation is not strong enough. Their motivation to utilize NPs for CL is to measure predictive uncertainties. However, it is not clearly discussed why it is important for CL.\n- Figures (Figures 1, 2, 4, and 6) are not recognizable. Especially their fonts are too small. For Figure 6, they missed labeling the axes.\n- In lines 38-39, they mentioned one of their motivations is NPs can meta-learn input correlations across correlated tasks. However, in the limitation section, they analyzed it as the incompetence of dot-product attention (lines 305-306). It is the opposite analysis of their motivation.\n- The equation (7) and relevant explanations are confusing. In the previous section for GR, task $t$ is dominated by the t-$th$ task samples, while it is referred to as $j$ in this section. As I understood, they tried to regularize the task $t$'s task latent on that at the step when the task $t$ arrived, but it is not easy to follow in their explanation.\n- The equation (8) is hard to understand. In their explanation about the memory management of their model, the sample $x$ is not stored, but it is not in this equation (without $x$, $L_{CE}$ cannot be calculated). \n- The comparison in Table 1 can cause confusion. At first glance, we can expect NPCL and DER to show comparable performance in the same memory usage, but their storage usages are quite different as shown in Appendix E.3. I can recommend showing as a plot in which the x-axis is real storage usage and the y-axis is accuracy.\n- (minor) typos\n\n    - In equation (8), the bracket for $L_{GR}$ and $L_{TR}$ is missed. \n    - In table 2, baseline is not w/o GR *or* TR, but GR *and* TR.\n    - In line 208, with. -> with\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful reading and remarks on our work. In regard to the questions raised, we provide our explanations below:\n\n- **Motivation**: Our motivation to utilize NPs for CL is two-fold. First, based on lines 20-26, we aim to design parameter isolation methods for CL that can better: (a) exploit task-specific and inter-task correlations, and (b) infer which task-specific component to utilize for test samples in the lack of a given task id.  Second, we are interested in CL models that are reliable in their predictive confidence. This latter goal is particularly useful for the real-world deployment of models where changing data distributions can interfere with a model's interpretability of its predictions. We thus wish to know when to rely on a model’s predictions and when to call for human/expert interventions. We will elaborate further on our motivation in the next version of our paper.\n\n- **Lines 38-39**: Yes, it is true that NPs can meta-learn input correlations across correlated tasks. However, the degree of the learned correlations can vary with the type of inductive bias used by the model [1]. For instance, the attentive NPs using dot-product attention capture better correlations than the vanilla NPs. Nevertheless, dot-product attention, like other optimization algorithms, suffers from its own limitations [2]. Rather than an opposite analysis of our motivation, we thus suggest that lines 305-306 be read as an implication of the no-free-lunch theorem for supervised learning [3].\n\n- **On Eq. (7)**:  We apologize for the confusion and provide further explanation on the notation here. In line 165, the task $t$ indeed refers to the current training task whose samples dominate the training set (given that the replay memory $\\mathcal{M}$ contains a much smaller number of samples per past task). On line 172, by task $t$, we actually refer to the past tasks for which we are regularizing the learned distribution (learned from $\\mathcal{M}$) towards the distribution that we had learned when the past task was first seen, *i.e.,* the step $0 <= j < t$. To retrieve the latter past task distribution, we rely on the distribution memory $\\mathcal{M}_\\mathcal{N}$ (line 173). \n\n- **On Eq. (8)**: We believe there has been a misunderstanding regarding how the replay memory $\\mathcal{M}$ works. $\\mathcal{M}$ does  store the sample $x$. We state this very clearly in lines 107-109, and also in appendix E.3 (lines 624-625).  It is in addition to this memory $\\mathcal{M}$ that NPCL stores the *separate* distribution memory (line 173). Given that $x$ is available from $\\mathcal{M}$, $\\mathcal{L}_{CE}$ can thus be calculated.\n\n- **Table 1**: Following the previous point, we reiterate the fact that both NPCL and DER store the original sample $x$ in the replay memory $\\mathcal{M}$ (lines 107-109; Appendix E.3 lines 624-625). The difference in memory usage is that while DER relies on storing the logits for the individual samples $x$, NPCL instead stores the parameters of the global and task-specific distributions as well as the scalar task id of $x$. Given that both NPCL and DER store sample $x$, we do not take these into account for our comparison of storage gain (lines 624-625). We nevertheless agree with the idea of a plot depicting the real storage usage and plan to incorporate this.\n\n- **Analysis of $N$ values**: Please refer to the global response and the Fig. 3(a) in the attached PDF for the analysis. Intuitively, the hierarchy in NCPL implies that the $M$ Monte Carlo (MC) samples of different task-specific distributions are conditioned on the same set of $N$ MC samples of the global distribution (see Fig. 3(b) for a rough sketch of this).  With smaller $N$, we restrict the diversity of the samples $z^G$  of the global distribution that the task-specific encoders should look at. This is equivalent to limiting the support of the global distribution. We know that different regions of the global distribution could capture different CL tasks. Smaller $N$ values are thus likely to condition a task $t$'s encoder on a global sample $z^G$ that represents a region corresponding to another task that is less correlated to $t$. Similarly, with smaller $N$, we are also more likely to miss out on conditioning the $t-th$ encoder on samples from regions of the global distribution that capture tasks having a high correlation with $t$.\n\n- **Multi-head Attention Pooling (MAP)**: Per your advice, we experiment with using Multi-Head Attention Pooling (MAP) [4] as a possible alternative to the global average pooling. The results of NPCL with MAP on S-CIFAR-10 with a memory size of 200 are shown below. In particular, we observe that while MAP does lead to a slight improvement of accuracy on the first task, the additional number of learnable parameters inadvertently causes higher forgetting on subsequent incremental tasks. We thus leave the exploration of MAPs for NPs in CL as a potential direction for future work.\n\n| Method        | Task 1 | Task 2 | Task 3 | Task 4 | Task 5 |\n|---------------|--------|--------|--------|--------|--------|\n| NPCL          | 98.3   | 86.12  | 74.85  | 69.48  | 63.78  |\n| NPCL w/ MAP   | 98.46  | 80.43  | 68.2   | 59.74  | 51.77  | \n\n**References**:\n\n[1] Jha, S., Gong, D., Wang, X., Turner, R.E., & Yao, L. (2022). The Neural Process Family: Survey, Applications and Perspectives. *ArXiv, abs/2209.00517*.\n\n[2] Kim, M., Go, K., & Yun, S. (2022). Neural Processes with Stochastic Attention: Paying more attention to the context dataset. *ICLR*.\n\n[3] Goldblum, M., Finzi, M., Rowan, K., & Wilson, A.G. (2023). The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning. *ArXiv, abs/2304.05366*.\n\n[4] Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., & Teh, Y. W. (2019, May). Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning (pp. 3744-3753). PMLR.\n\n"}]

Author rebuttal:
We thank the reviewers for their careful comments and suggestions. Here we provide a single-page PDF that contains the figures and tables that we want the reviewers to see while considering our responses. We look forward to a helpful discussion period.

- **Effect of Monte Carlo samples on performance**: As asked by reviewers **fvgr** and **1ufQ**, we discuss the effect of the number of Monte Carlo (MC) samples for global *N* and task-specific latent variables *M* on accuracy during inference (please refer to the heatmap in Figure 3(a) of the attached PDF). In particular, we observe two favorable spots in terms of accuracy, one centered around $(M=1, N=50)$ and the other around $(M=10, N=20)$. It is worth noting that the total number of inference time MC samples grow quadratically with the number of tasks ‘t’, i.e., $\mathcal{O}$($N \times M \times t$), and that a higher number of samples leads to a larger computational overhead. For instance, based on eq. (9) in the main paper, the inference on the 10th task of S-CIFAR-100, i.e., $t = 10$, given the two favorable $M$ and $N$ values amounts to selecting the set of task-specific module predictions with the least uncertainty from a total of (a) $1 \times 50 \times 10 = 500$ predictions using $(M=1, N=50)$, and (b) $10 \times 20 \times 10 = 2000$ predictions using $(M=10, N=20)$. We, therefore, opt for the more efficient setting of $(M=1, N=50)$ throughout our experiments for all settings.


Meta-review: The paper received mixed views. After considerable discussion between the authors and reviewers, most of the concerns of the reviewers have been addressed. The reviewer with the lowest score also mentioned that their concerns have been addressed by the rebuttal. In view of the consensus by the reviewers, it is recommended to accept the paper. 

However, it is recommended that the authors update the paper to include the clarification of the settings as this was the main concern. While this has been addressed in the rebuttal, it would be required for a reader to understand the use of NP in the continual learning setting clearly and the task and function equivalence in this setting.
Decision: Accept (poster)