Incremental Learning via Robust Parameter Posterior Fusion

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 04:29:11

Abstract: The posterior estimation of parameters based on Bayesian theory is a crucial technique in Incremental Learning (IL). The estimated posterior is typically utilized to impose loss regularization, which aligns the current training model parameters with the previously learned posterior to mitigate catastrophic forgetting, a major challenge in IL. However, this additional loss regularization can also impose detriment to the model learning, preventing it from reaching the true global optimum. To overcome this limitation, this paper introduces a novel Bayesian IL framework, Robust Parameter Posterior Fusion (RP$^2$F). Unlike traditional methods, RP$^2$F directly estimates the parameter posterior for new data without introducing extra loss regularization, which allows the model to accommodate new knowledge more sufficiently. It then fuses this new posterior with the existing ones based on the Maximum A Posteriori (MAP) principle, ensuring effective knowledge sharing across tasks. Furthermore, RP$^2$F incorporates a common parameter-robustness priori to facilitate a seamless integration during posterior fusion. Comprehensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets show that RP$^2$F not only effectively mitigates catastrophic forgetting but also achieves backward knowledge transfer.

Main paper: 
 related works 2.1 incremental learning: 
 Incremental learning focuses on developing algorithms that can continuously learn from new data without forgetting previously acquired knowledge [12, 33] . Approaches within this field are broadly categorized into regularization-based methods, rehearsal-based methods, and dynamic architecture methods. In the following, we will review these three class methods.Regularization-based methods integrate regularization loss terms to prevent forgetting. Some methods [1, 22, 29, 68] punish the changes in critical parameters containing old-task knowledge. Other knowledge-distillation-based methods hinge on the principle of transferring knowledge from an older model (teacher) to a newer model (student) during the IL process. A seminal work is Learning without Forgetting (LWF) proposed by Li and Hoiem [28] , which utilizes knowledge distillation to preserve previously learned information while accommodating new knowledge. Based on LWF, several other notable methods [14, 26, 27, 47, 56, 57, 69, 71] have been developed. Rehearsal-based IL methods operate by retaining a subset of the original training data [3, 5, 6, 8, 9, 30, 38, 42] or generating synthetic data [49, 50, 65] , which is then replayed with new data during subsequent training phases. Dark Experience Replay (DER) [7] combines experience replay with a distillation mechanism to maintain a balance between old and new knowledge. DER selectively stores a subset of the data from previous tasks and rehearses it alongside new data, thus preserving the model's performance on historical tasks. On the other hand, generative-replaybased methods leverage generative models, such as Generative Adversarial Networks (GANs) [16] , to synthesize data from previous tasks, thereby eliminating the need to store real data. Dynamic architecture methods typically involve modifying the architecture of neural networks dynamically in response to new tasks. Some methods are based on masks [10, 19, 23, 34] , such as PackNet [34] . PackNet prunes weights that are non-essential for learned tasks, thereby freeing up network capacity for new tasks, a process that strategically balances between retaining learned knowledge and adapting to new information. On the other hand, expansion-based methods [40, [53] [54] [55] 64] like Dynamically Expandable Networks (DEN) [66] selectively expand the network by adding neurons as needed.As a subset of regularization-based methods, bayesian-based IL methods [20, 22, 25, 37, 39, 46, 61] utilise Bayesian inference through the learning process. Elastic Weight Consolidation (EWC) [22] applies a Bayesian perspective to determine the importance of neural network parameters, followed by imposing a constraint on significant deviations from previous parameters. IMM [25] proposes to merge the neural network parameters from previous tasks with the current task, which shares a similar motivation with the posterior fusion of RP 2 F. Beyond IMM, RP 2 F further introduces parameter robustness priori and a more precise perturbation-based Hessian estimation method.In the field of robust model training, the perturbation-based method is a primary research direction within the machine learning community. A common practice involves modeling parameter robustness using Lipschitz continuity and enhancing robustness by constraining the upper bound of the Lipschitz constant [2, 11, 18, 58] . However, recent studies [43] have highlighted that the Lipschitz constant is sensitive to the scale of model parameters and input features, and propose to employ condition numbers as a more precise metric of robustness. In the context of IL, robust methodologies have been integrated into some exemplar-based methods. They primarily concentrate on model robustness to exemplar variations. For instance, LiDER [5] forces the model to become robust to the changes of exemplars, thereby mitigating the over-fitting issue in exemplarbased methods. DRO [63] continually evolves the exemplar buffer to ensure the model learns more robust features. In contrast to these methods, our proposed RP 2 F does not rely on exemplar samples. Instead, we focus on investigating robustness against parameter variations, with the goal of mitigating information losses in posterior fusion, which contributes a novel perspective to IL. 
preliminary 3.1 problem definition: 
 Incremental learning (IL) is a cognitive process characterized by the gradual acquisition and assimilation of knowledge over time. As a popular setting of IL, Task Incremental Learning (Task-IL) [59] focuses on the sequential acquisition and retention of knowledge across multiple related tasks, where each task may involve distinct but related learning objectives.Formally, let 𝐷 1 , 𝐷 2 , . . . , 𝐷 𝑇 denote datasets of 𝑇 classification tasks, where 𝐷 𝑡 = {(𝑥 𝑡 𝑘 , 𝑦 𝑡 𝑘 ∈ 𝑌 𝑡 )} 𝑛 𝑡 𝑘=1 is associated with task 𝑡 that contains 𝑛 𝑡 input-label pairs. These tasks feature disjoint classes, i.e., 𝑌 𝑡 1 ∩ 𝑌 𝑡 2 = ∅, ∀𝑡 1 ≠ 𝑡 2 . Task-IL sequentially presents these tasks to an IL model, with only the associated dataset 𝐷 𝑡 being available at time step 𝑡. The model must adapt to task 𝑡 while also retaining knowledge of the former tasks. During the testing phase, the model's overall performance across all encountered tasks will be evaluated. ParameterIncremental learning has been extensively explored within the Bayesian framework. The Bayesian IL framework leverages Bayes' rule to systematically update the posterior parameter distribution as new data becomes available. The fundamental concept lies in the Bayes' theorem, which provides an estimation of the parameter posterior as follows:EQUATIONwhere 𝑃 (𝜃 |𝐷) represents the parameter posterior after observing data 𝐷. This posterior is shaped by the parameter priori 𝑃 (𝜃 ) and the likelihood 𝑃 (𝐷 |𝜃 ).In the context of IL, the posterior estimation after observing 𝑡 tasks in Eq. 1 can be extended into multiple tasks [20] as follows:EQUATIONUsing the posterior of all the former tasks as a priori, the posterior can be computed cumulatively:EQUATIONAn integral part of Bayesian-based IL methods is the estimation of the posterior distributions 𝑃 (𝜃 |𝐷 1 , . . . , 𝐷 𝑡 -1 ), which is often computationally challenging for complex models. The Laplace approximation [31] offers a tractable solution by approximating the posterior distribution with a Gaussian centred at the maximum a posteriori (MAP) estimate. This approximation is particularly useful when the posterior is unimodal. The mathematical representation of this approximation is formalized as follows:EQUATIONwhere 𝜇 𝑀𝐴𝑃 is the mode of the posterior distribution (i.e., the MAP estimation), and 𝐻 -1 𝑀𝐴𝑃 is the inverse of the Hessian around 𝜇 𝑀𝐴𝑃 , providing a second-order approximation of the posterior curvature.Building upon this framework, bayesian-based IL methods [20, 22, 25] emerge to incorporate Bayesian principles into neural network training for each task. They typically mitigate catastrophic forgetting by introducing a regularization term to the loss function that penalizes changes to important parameters:EQUATIONwhere 𝐿 𝑐𝑒 represents the cross-entropy loss and 𝜆 stands for a pre-defined hyperparameter weighting the regularizer. Here, the posterior 𝑃 (𝜃 |𝐷 1 , . . . , 𝐷 𝑡 -1 ) is approximated by the Gaussian distribution N (𝜃 |𝜇 1:𝑡 -1 , Λ 1:𝑡 -1 ), where 𝜇 1:𝑡 -1 is typically assigned to the value of 𝜃 1:𝑡 -1 and Λ 1:𝑡 -1 signifies the importance matrix. The continuous update of Λ 1:𝑡 -1 incorporates the Hessian associated with the new task [22] :EQUATIONHowever, the computation of the Hessian matrix poses a significant challenge, due to its computational complexity (second-order partial derivatives) and the extensive storage requirements. In practice, a prevalent approach is to approximate the Hessian using the Fisher information matrix [20, 22, 39, 46] :EQUATION 
method 4.1 robust bayesian il framework: 
 Although Bayesian-based IL methods have demonstrated effectiveness in mitigating forgetting, they encounter significant challenges. One such challenge is the control of regularization strength. In Eq. ( 5), the cross-entropy loss and the regularization terms may not be scaled equivalently. This imbalance may lead 𝜃 parameters transition from the local optimum associated with previous tasks 𝜃 1:𝑡 -1 to a global optimum that encompasses new learning objectives 𝜃 1:𝑡 (as illustrated in Figure 1 (b) ). This forgetting becomes noteworthy when the performance is sensitive to parameter changes.To address the aforementioned challenges, we propose the Robust Parameter Posterior Fusion (RP 2 F) framework for IL. Specifically, RP 2 F reformulates Eq. ( 2) by choosing a specific parameter robust priori 𝑃 (𝜃 ) as follows:EQUATIONEquation ( 8) enables us to model task-specific posterior solely based on the data from each respective task. In other words, instead of modeling the posterior of all encountered tasks 𝑃 (𝜃 |𝐷 1 , . . . , 𝐷 𝑡 -1 ) in Eq. ( 3), we are able to yield the MAP estimation directly for each individual task:EQUATIONwhere 𝐿 robust_pri (.) represents a parameter robustness priori weighted by 𝜆. Notably, different from Eq. ( 5), Eq. ( 9) derives the MAP estimation only for the single task 𝑡.Based on 𝜃 * 𝑡 , we apply the Laplace approximation to estimate 𝑃 (𝜃 ) Taking into account the practical constraints of storage and computation, our approach aligns with prior works [22] that employ a diagonal matrix to estimate the Hessian matrix. The diagonal matrix, denoted by Λ * 𝑡 , is thus regarded as indicative of the importance of the parameters. Building upon the Laplace approximation, we are able to derive the MAP estimation 𝜃 * 1:𝑡 of the posterior distribution in Eq. ( 8):EQUATIONFor the proof of Eq. ( 10), please refer to the supplementary materials. Additionally, we present a task-accumulative version, designed to fulfil the requirements of IL settings:EQUATIONwhere Λ * 1:𝑡 = Λ * 1:𝑡 -1 + Λ * 𝑡 also update in an accumulative manner. Compared to the conventional Bayesian-based IL methods, RP 2 F has two specific designs. Firstly, RP 2 F determines the optimal parameter through the MAP estimate of the fusion parameter posterior for all encountered tasks in Eq (10) . Unlike regularizer-based methods that struggle to balance the regularizer in Eq. ( 5), the posterior fusion manner facilitates a harmonious knowledge accumulation, achieving an enhanced equilibrium between maintaining previously learned information (stability) and incorporating new knowledge (plasticity). Secondly, the incorporation of the parameter-robustness priori mitigates potential forgetting induced by parameter updating.To implement the RP 2 F framework, two key components remain to be devised: the efficient estimation of Hessian and the design of a parameters-robustness priori. Conventionally, the Hessian is estimated using the Fisher information matrix. Although the Fisher information matrix possesses many desirable properties [22] , as described in Eq. ( 7), the common computation of Fisher inherently contains only first-order gradient information. This limitation is especially highlighted as previous works [22, 46] consider only the diagonal elements of the Fisher information matrix. To address this limitation, this paper introduces a novel Hessian matrix approximation method based on parameter perturbation, which effectively approximates second-order derivative information with the same computational complexity as first-order derivatives. In terms of the robustness priori to parameters, our investigation reveals that the uniformity in the eigenvalues of the features extracted by the network is closely linked to the robustness of the parameters. This observation motivates us to average the eigenvalues of features during training.Next, Sections 4.2 and 4.3 elaborate on the Hessian approximation method and the parameter robustness priori, respectively.Previous works [22, 46] typically employ the diagonal Fisher information matrix to estimate the Hessian, focusing exclusively on the first-order derivatives as outlined in Eq. ( 7). In contrast, our research presents a novel Hessian estimation method based on parameter perturbation. The estimation is based on the gradient corresponding to infinitesimal parameter shifts, thereby providing a more precise and efficient approximation of Hessian.To be more specific, considering the MAP estimation for 𝑡-th task posterior 𝜃 * 𝑡 , we introduce a minor perturbation 𝛿 to 𝜃 * 𝑡 . The second-order Taylor expansion on the empirical risk function is formulated as follows:EQUATIONwhere 𝐻 * 𝑡 = ∇ 2 𝜃 𝐿 𝑐𝑒 (𝜃, 𝐷 𝑡 )| 𝜃 =𝜃 * 𝑡 denotes the Hessian around 𝜃 * 𝑡 . The expansion facilitates the analytical derivation of the model's sensitivity to parameter changes. Considering that 𝜃 * 𝑡 is sufficiently trained such that the first-order gradient approaches zero, by setting 𝛿 = 𝜂1 where 1 signifies an all-one vector and 𝜂 represents an infinitesimally small constant, we differentiate both sides of Eq. ( 12) with respect to the perturbation 𝛿 yields the sum rows of the Hessian matrix:EQUATION)By taking the assumption that the Hessian of a single-layer approaches diagonalization [44, 45] , we estimate all the diagonal entries of 𝐻 * 𝑡 with Eq. 13 layer by layer, ultimately forming the diagonal matrix Λ * 𝑡 . Estimating Hessian via perturbation significantly simplifies the complexity of Hessian computation. For a network with 𝑛 parameters, this method boasts a computational complexity of O (𝑛). In contrast, the direct computation of the Hessian, considering its need to evaluate second-order derivatives for every pair of parameters, generally exhibits a computational complexity of O (𝑛 2 ), or even O (𝑛 3 ) when matrix inversion is involved. The efficient Hessian estimation enables a scalable and high-performance implementation of RP 2 F.In Section 4.1, by maximizing the fusion posterior, we derive the optimal estimation 𝜃 * 1:𝑡 . Intuitively, we aspire for 𝜃 * 1:𝑡 to exhibit linear connectivity with the local optima for each task (e.g., 𝜃 * 𝑡 ), aiming to minimize the difference 𝑓 (𝑥; 𝜃 * 𝑡 ) -𝑓 (𝑥; 𝜃 * 1:𝑡 ) for any 𝑥 ∈ 𝐷 𝑡 . In other words, as depicted in Figure 1 , our objective is to ensure that 𝜃 * 𝑡 is robust to parameter changes, which motivates us to incorporate robustness priori into parameters during training.The robustness of parameters is commonly characterized by the Lipschitz constant. However, the Lipschitz constant is sensitive to the scale of both the data and model parameters. As an alternative, inspired by [43] , our work introduces a relative metric to assess the robustness of the model, which compares the rate of change in model outputs against the rate of parameter alterations:EQUATIONwhere 𝛿 represents a perturbation applied to 𝜃 . The numerator in ( 14) captures the relative change in model output due to parameter perturbation, while the denominator reflects the relative scale of the parameter perturbation. This formulation thus provides a measure of output sensitivity that is normalized by both the extent of the perturbation and the scale of the parameters, offering a more precise evaluation of the model's robustness. Before delving into the robustness of deep neural models, we begin our examination with a simpler linear model 𝑓 (𝑋 ; Θ) = Θ𝑋 , where 𝑋 denotes the input feature matrix with each column corresponding to an individual input sample 𝑥 ∈ 𝐷. For this linear model, the relative-robustness metric is bounded by:EQUATIONwhere 𝑋 † represents the pseudo-inverse of 𝑋 . When employing the Euclidean norm (also known as the ℓ 2 norm) for the metric, 𝑅(Θ) is bounded by the ratio between the maximum singular value of 𝑋 to its minimum singular value:EQUATION)Building upon the robustness analysis in the linear scenario, we extend our investigation to a deep neural network composed of 𝐿 layers [20] :EQUATIONwhere 𝑋 𝑙 , Θ 𝑙 , and 𝜎 𝑙 denote the input features, parameters, and activation units of the 𝑙-th layer, respectively. We suppose that the robustness of all activation units is bounded by a positive constant 𝑀, i.e., 𝐿 𝑙=1 𝑅(𝜎 𝑙 ) ≤ 𝑀. Equation (17) elucidates that the robustness of the network parameters is governed by the uniformity of the singular values of the input features at each layer (extracted by the preceding layer). Yujun et al. [48] have demonstrated that this uniformity can be enhanced by penalizing the Frobenius norm of the covariance matrix. Based on this insight, we introduce the following priori to enhance the robustness of network parameters:EQUATIONwhere X𝑙 represents the normalized feature matrix of the 𝑙-th layer. However, Eq. ( 18) presents a considerable challenge in terms of computational complexity, particularly for high-dimensional deep neural networks. Given that features in the shallow layers of neural networks tend to approach a full rank [15] , in practice, we strategically apply regularization only to the features extracted from the penultimate layer.This section elaborates on our IL algorithm, which leverages the robust parameter posterior fusion framework. This approach integrates the perturbation-based Hessian approximation and the parameter-robustness priori proposed in Section 4.2 and 4.3, respectively. We will describe the training procedure of our method on a multi-head model consisting of a feature extractor 𝜃 and 𝑇 task-specific classifier 𝜏 1 , . . . , 𝜏 𝑇 .Learning task 1. During the phase of Task 1, the feature extractor alongside the classifier undergoes joint training using dataset 𝐷 1 .The training leverages cross-entropy loss and parameter robustness priori to enhance performance and ensure robustness:EQUATIONwhere 𝜏 • 𝜃 denotes a unified network that the classifier 𝜏 operating on features extracted by the feature extractor 𝜃 . Upon the completion of training for task 1, we assign the parameters of an additional fused posterior extractor with 𝜃 * 1:1 ←-𝜃 * 1 , and approximate the Hessian via Λ * 1:1 ←-𝜕𝐿 𝑐𝑒 (𝜃 * 1 +𝛿,𝐷 𝑡 ) 𝜂𝜕𝛿 .Learning the following tasks (𝑡 for illustration). We initialize 𝜃 𝑡 with the last optimal parameters 𝜃 * 𝑡 -1 learned from the previous task. This strategy aims to keep the optimization process for all tasks within the same vicinity of a local optimum, thereby facilitating effective fusion of the posterior distribution [25, 52] . Subsequently, during each training epoch, the model undergoes a four-step updating process: 1) training the feature extractor 𝜃 𝑡 ; 2) estimating the Hessian Λ 𝑡 surrounding 𝜃 𝑡 ; 3) updating the fused posterior extractor 𝜃 1 : 𝑡; 4) training the task-specific classifier 𝜏 𝑡 .The formulations for these four steps are presented as follows:EQUATION)where 𝛼 denotes the learning rate. This iterative update is designed to ensure that the classifier 𝜏 𝑡 remains compatible with the optimal fused posterior extractor 𝜃 1:𝑡 . Upon the training convergence, we obtain the optimal values of 𝜃 * 𝑡 , Λ * 𝑡 , 𝜃 * 1:𝑡 , and 𝜏 * 𝑡 . Among them, Λ * 𝑡 is utilized to refine the variance of the fused posteriorΛ * 1:𝑡 ←-Λ * 1:𝑡 -1 + Λ * 𝑡 .Λ * 1:𝑡 , along with the others, is maintained for the subsequent task training.Inference. During the testing phase, given sample 𝑥 associated with task identification 𝑡, we select the corresponding classifier 𝜏 * 𝑡 along with fused extractor 𝜃 * 1:𝑇 to perform a prediction:EQUATIONwhere ŷ denotes the classification result. 
experiments 5.1 settings: 
 Datasets. To ensure a comprehensive evaluation, we select the following three datasets to conduct experiments:• 5-split CIFAR-10 [24] . The CIFAR-10 dataset consists of 60,000 color images of 32 × 32 pixels distributed across 10 different classes, with each class containing 6,000 images. In our experiments, CIFAR-10 is divided into 5 splits based on the class labels. • 10-split CIFAR-100 [24] . Similar to CIFAR-10, CIFAR-100 features 100 classes containing 600 images each. We partition CIFAR-100 into 10 splits, with each containing 10 classes. • 10-split Tiny-ImageNet [51] . Tiny-ImageNet is a scaleddown version of the ImageNet dataset, consisting of 200 classes, each with 600 64 × 64 color images. For our study, this dataset is segmented into 10 splits, each with 20 classes.Performance metrics. To thoroughly evaluate the performance of RP 2 F, we employ two metrics [7, 30] : classification accuracy (ACC) and backward knowledge transfer (BWT). ACC is calculated after the model has been sequentially trained on all tasks. This metric serves as a direct indicator of the model's ability to classify images correctly across all classes, reflecting its overall performance in IL. BWT is designed to quantify how learning new tasks affects the performance of previously learned tasks. A negative value of BWT indicates forgetting, while a positive value suggests that learning new tasks has beneficial effects on previous tasks' performance. The formula for calculating BWT is defined as follows:EQUATIONwhere 𝐴𝐶𝐶 𝑗,𝑖 represents the accuracy on task 𝑖 once the model finish learning task 𝑗. To ensure the reliability of results, we conduct all experiments five times with random seeds and report the average performance.Baselines. We compare our method with various latest and classic IL methods, including Learning without Forgetting (LwF) [28] , Synaptic Intelligence (SI) [68] , Gradient Episodic Memory (GEM) [30] , online Elastic Weight Consolidation (oEWC) [46] , Learning without Memorizing (LwM) [13] , Dark Experience Replay (DER and DER++) [7] , Efficient Feature Transformation (EFT) [60] , Pototype Augmentation and Self-Supervision (PASS) [70] , Gradient Projection Memory (GPM) [41] , Adam-NSCL [62] , Always Be Dreaming (ABD) [50] , Complementary Learning System (CLS-ER) [3] , Filter Atom Swapping (FAS) [36] , DCPOC [53] , PRAKA [47] , and MIND [4] . We also report the performance of a base model (referred to as Joint), which is trained jointly using data from all tasks. Clearly, Joint does adhere to the Task-IL setting, and its results are typically considered as the upper bound for IL methods. Additionally, for methods specifically designed for Class-IL, we employ a multi-head version of them to obtain the experimental results.Implementation details. The experiments are conducted following the requirements of the Task-IL setting, where only the corresponding dataset is available for training in each task. For exemplar-based methods, we provide them with an extra sample buffer that can store up to 500 samples. We optimize the model parameters using Stochastic Gradient Descent (SGD), with learning rates adjusted specifically for each dataset: 0.2 for CIFAR-10, 0.05 for CIFAR-100, and 0.3 for Tiny-ImageNet. The hyperparameter 𝜆 (weighting the parameter-robustness regularize in Eq. ( 9)) is set to 1e-6 for CIFAR-10 and 1e-5 for both CIFAR-100 and Tiny-ImageNet. Further details are available in our supplementary code.In this section, we present a comprehensive comparison of our proposed RP 2 F method against several state-of-the-art baselines as detailed in Section 5.1. Table 1 presents the average accuracy (%) over five runs on 5-split CIFAR-10, 10-split CIFAR-100, and 10-split Tiny-ImageNet.The Joint model, which serves as the theoretical upper bound performance, consistently shows the highest accuracy across all datasets. We also found that some replay-based methods, such as GEM, DER, DER++, and ABD, exhibit competitive performance on small-scaled CIFAR-10, but experience a decline in efficacy on the CIFAR-100 and Tiny-ImageNet datasets. This observation suggests that the replay buffers and generative models may be insufficient for modeling complex data distributions.Additionally, EFT achieves the highest accuracy on the CIFAR-10 dataset with 95.09%, surpassing RP 2 F. The observed discrepancy in the performance of RP 2 F may be attributed to the inherent structure of CIFAR-10's 5-split configuration, where each task only contains data of two classes. This limited class diversity may lead to overfitted parameters, which further affect the accurate estimation of posterior within RP 2 F. Nevertheless, RP 2 F outperforms all other baselines on the more challenging CIFAR-100 and Tiny-ImageNet datasets, suggesting its superior capability to handle more diverse and complex datasets. This section delves into the ablation study to analyse the effectiveness of the parameter-perturbation-based Hessian estimation method and the parameter-robustness priori. Specifically, we explore three Hessian estimation strategies: the identity matrix, the Fisher information matrix, and the proposed parameter-perturbationbased approach. For each strategy, we further investigate the impact of introducing the parameter-robustness priori. The results on CIFAR-100 and Tiny-ImageNet are presented in Table 2 .We begin with the identity matrix as the simplest form of Hessian approximation, which yields an accuracy of 79.42% on CIFAR-100 and 63.96% on Tiny-ImageNet. Equipping the parameter-robustness priori provides a slight enhancement, improving accuracies to 79.91% and 64.62%, respectively. Shifting to the widely used Fisher information matrix, we observe a notable improvement in performance. Without the regularizer, it achieves 82.29% accuracy on CIFAR-100 and 64.40% on Tiny-ImageNet. Integrating the parameterrobustness priori further boosts the performance to 82.85% and 64.79%, respectively. Our novel parameter-perturbation-based Hessian estimation method is more competitive than the Fisher information matrix. Without the regularizer, it attains an accuracy of 82.37% on CIFAR-100 and 64.78% on Tiny-ImageNet. The incorporation of the parameter-robustness priori pushes the performance to the highest accuracy: 83.06% on CIFAR-100 and 65.81% on Tiny-ImageNet.All the results underscore the efficacy of parameter-perturbationbased Hessian estimation in accurately reflecting the true landscape of the loss function. Furthermore, these findings emphasize the ability of the parameter-robustness priori to enhance the model's robustness and mitigate forgetting.This section analyzes the backward knowledge transfer of RP 2 F, a critical IL ability to retain or even enhance performance on previously learned tasks when new tasks are introduced. Our discussion is based on empirical evidence in Figure 2 and Table 3 . Figure 2 visually illustrates the performance trajectory of each task through IL. As can be seen, DER++ exhibits a declining trend in accuracy for earlier learned tasks, signifying the occurrence of forgetting. In contrast, GPM maintains a more stable performance throughout the learning phases, showing its effectiveness in mitigating forgetting. Remarkably, RP 2 F exhibits improved performance on older tasks with the introduction of new tasks (with the exception of the first task).Further quantitative evidence of BWT is presented in Table 3 . The BWT of several methods are calculated on CIFAR-100 and Tiny-ImageNet, including DER++, LwF, OWM, PASS, GPM, Adam-NSCL, and RP 2 F. It is evident that all baseline methods exhibit a negative BWT, highlighting the challenge of forgetting in IL. In contrast, RP 2 F records a positive BWT of 0.56% on CIFAR-100 and 0.92% on Tiny-ImageNet, surpassing all other methods. Both visual illustration and quantitative values of BWT underscore RP 2 F's superior ability to backward knowledge transfer.This section provides a sensitivity analysis of the hyper-parameter 𝜆, responsible for weighting the parameter-robustness priori in Eq. (9) . We conduct experiments with various values of 𝜆 and summarize the results in Figure 3 . As can be seen, setting 𝜆 to zero or an excessively high value results in suboptimal performance of RP 2 F, attributing to the ineffectiveness or overwhelming impact of the regularizer, which in turn perturbs the training dynamics. Nevertheless, a stable performance window for 𝜆 is observed within the range of [1𝑒 -7, 1𝑒 -5]. Based on these empirical findings, we recommend setting 𝜆 to 1𝑒 -5 on CIFAR-100 and Tiny-ImageNet to achieve the best overall performance.  
conclusion: 
 In this work, we propose the Robust Parameter Posterior Fusion (RP 2 F) framework for incremental learning. RP 2 F models and fuses the parameter posterior distributions from both new and existing tasks, facilitating a more equitable integration of knowledge across tasks. Furthermore, we incorporate robustness priori into RP 2 F to mitigate potential forgetting induced by posterior fusion. By employing MAP estimation on the fused posterior, RP 2 F achieves a harmonious knowledge integration.RP 2 F has certain limitations, such as relying on task boundaries to perform posterior fusion. In future work, we plan to address this issue by incorporating online learning techniques to extend the applicability of our method.  
: 
 To mitigate catastrophic forgetting, a fundamental strategy is to identify and preserve the parameters critical to past tasks. This strategy has directed the development of numerous IL methodologies [1, 9, 34, 41, 68] , including techniques grounded in Bayesian theory [20, 22, 32, 39, 46] . The Bayesian-based approaches utilize Laplace approximation to model the parameter posterior distribution of old tasks as a Gaussian distribution. This Gaussian posterior then serves as a priori to guide subsequent network training, enforcing regularization to prevent substantial deviations from this priori. In this context, the variance of the Gaussian distribution (i.e., the Hessian through Laplace approximation) acts as a parameter importance matrix to identify critical parameters.While the regularization has proven its effectiveness in IL, controlling its strength poses a considerable challenge for Bayesianbased methods. A particular issue is ensuring that parameter adjustments through regularization accurately reflect the Maximum A Posteriori (MAP) estimate of the joint posterior distribution for both new and old tasks. Particularly, achieving a harmonious integration of the regularization (representing the old-task posterior) with the learning loss for new tasks remains a challenge [21] , even when employing predefined weights. Furthermore, the necessity for accommodating new data within IL contexts inevitably compels the parameters to diverge from the old-task posterior, potentially leading to information loss.To address the above challenges, this paper introduces a novel Bayesian IL framework, referred to as Robust Parameter Posterior Fusion (RP 2 F). Unlike traditional methods that integrate the posterior of old tasks through regularization, RP 2 F constructs a parameter posterior for the new task and then directly fuses it with the posterior from previous tasks. This integrated posterior enables the derivation of a closed-form MAP solution for parameters, facilitating a more coherent balance across tasks. As the learning process evolves, parameter estimates continuously adjust from the old-task MAP with the introduction of each new task, which may trigger potential forgetting. To mitigate this, RP 2 F incorporates a parameter robustness priori within the training regimen, which bolsters the model's resilience against changes in parameters.To implement RP 2 F, two critical techniques are imperative for enhancing performance: the computation of the Hessian matrix in Laplace approximation and the design of the parameter robustness priori. Existing methods [22, 46] typically approximate the diagonal of the Hessian matrix with the Fisher information matrix due to the computational and storage constraints, where the Fisher information matrix only depends on the model's first-order derivative information. Instead, this paper develops a perturbation-based method for estimating the Hessian information, which leverages derivatives with respect to small perturbations applied to the parameters. Furthermore, our analysis reveals that a network's robustness is closely linked to the uniformity of the singular values of the extracted feature matrix. Drawing on this insight, we propose regularizing the features during training to balance the singular values, thus diminishing the potential forgetting issue caused by parameter changes.In summary, the core contributions of this research are listed as follows:• We propose a novel Robust Posterior Parameter Fusion (RP 2 F) framework for IL, which leverages the Bayesian theory to derive a posterior fusion method across tasks. • To facilitate Laplace posterior estimation, we develop a Hessian approximation method via parameter perturbation. This method captures second-order derivative information while maintaining the computational complexity of first-order gradients.  
 
 Appendix:  


Decision: The reviewers agree that the submission involves a novel approach. 6VCT calls the experimental approach “exhaustive” and the results “very promising”. ZFPT adds that the approach to incremental learning is “well-motivated” and HGCW additionally felt the background was “sufficient and clear”.

In terms of limitations 6VCT wanted further information about the parameter-robustness priori which was provided in the rebuttal. ZFPT felt a comparison of time/space efficiency was warranted. HGCW wanted to see tasks other than classification. 

Two of the reviewers felt that concerns were sufficiently addressed in the rebuttal.

All reviewers agreed that this paper is worthy of aceptance (1 weak, 2 borderline). Given the arguments that this work is novel, achieves good results, and is of interest to the MM community I recommend that it is accepted with updates as noted in the rebuttal. This recommendation is only tempered by the reviewer’s rating that this is “moderately suitable” work, therefore perhaps more appropriate as a poster.

Reviews: review: paper_summary: paper_summary: This paper presents a method for incremental learning called as Robust Parameter Posterior Fusion. In this approach, the posterior on the backbone of the feature extracted for each task is learned separately and then fused together based on MAP principle. It also includes a shared parameter priori-based regularization to ensure robustness of the feature extractor parameters. Through extensive empirical experiments the authors demonstrate the promising results of their method when compared against prominent baselines in incremental learning.
strength_weakness: strength_weakness: - The paper presents a very novel approach for incremental learning (such as the inclusion of Hessian approximation using parameter perturbation), and technically it seems very strong & practical based on the assumptions and the design choices made. I expect this method to be valuable to the practitioners of this field. 

- The experiments are exhaustive, and the metrics are also very relevant. The results are very promising overall, with the proposed method outperforming on overall accuracy when compared against all the baselines on CIFAR-100 & Tiny-ImageNet.

- The approach is well described, especially when one thinks from the perspective of implementation. Also, the authors have done a good job at describing the prior work in this area.
limitations: limitations: - An important piece of this method seems to be the parameter-robustness priori. However, based on the current presentation, it's very hard to wrap my head around the significance of this piece (especially from an intuition perspective). I'd encourage the authors to provide an intuition behind what this does, and why is this important.  Furthermore, it seems that we still need to tune an hyperparameter $\lambda$ as we did in eq (5). Could you please describe the difference with current method and eq(5) in more detail so that it's easy to understand? 

- Could you please describe some applications where this method would be applicable, and which would be of the interest to ACM MM audience? 

- A small suggestion: Having an algorithm block would be nice. 

- Line 474-475: "..to exhibit the linear connectivity with local optima for each task.." - why is this the case? can you please explain? 

- A high level question - how do we think about meta learning in the context of incremental learning? I understand that in meta learning, at test time we would have to adapt to a new task, but can we think about the meta parameters as the latent representation of the posterior over all tasks?
paper_summary: paper_summary: The paper presents the Robust Parameter Posterior Fusion (RP2F) framework, which directly estimates the parameter posterior for new data without introducing extra loss regularization. This method effectively mitigates catastrophic forgetting and achieves backward knowledge transfer.
strength_weakness: strength_weakness: (1) Well-motivated approach to improve Incremental Learning;

(2) The proposed approach achieves great results compared with sufficient methods.
limitations: limitations: (1) Eq. (13) uses the finite difference method to derivate the approximation of the Hessian under the assumption that the gradient of the trained parameters is 0. However, the result is incorrectly expressed as the gradient with respect to $\delta$, which should be modified to $\frac{\partial L_{ce}\left(\theta _{t}^{*}+\delta ,D_t \right)}{\delta}$.

(2) A comparison of computational efficiency and memory usage between the proposed method and existing methods should be provided.
paper_summary: paper_summary: This paper proposes a novel Bayesian incremental learning framework called Robust Parameter Posterior Fusion (RP2F), which effectively alleviates the catastrophic forgetting problem and realizes reverse knowledge transfer by directly estimating and fusing the parameter posteriors of new and old data.
strength_weakness: strength_weakness: 1. To my best knowledge, this approach is novel. The authors clearly derive the methodology in the paper, which I think is sound.

2. The comparison method is sufficient, and the proposed method performs well on key incremental learning tasks

3. The background information is sufficient and clear.
limitations: limitations: 1. I am interested in validating the proposed method on tasks other than classification.

2. In sec 4.2, Eq. (13), the derivation of the approximation of the Hessian may be wrong. If I understand correctly, this paper uses the finite difference method and assumes that the gradient of the trained parameters is 0, then the numerator in the formula should be changed to $\delta$. Eq. represents the gradient of $\delta$ and cannot approximate the Hessian.