BEEF: Bi-Compatible Class-Incremental Learning via Energy-Based Expansion and Fusion
Authors: Fu-Yun Wang
Da-Wei Zhou
Liu Liu
Han-Jia Ye
Yatao Bian
De-Chuan Zhan
Peilin Zhao
Venue: ICLR 2023 poster (2023)
Keywords: EBMs
Compatibility
Continual Learning

Abstract: Neural networks suffer from catastrophic forgetting when sequentially learning tasks phase-by-phase, making them inapplicable in dynamically updated systems. Class-incremental learning (CIL) aims to enable neural networks to learn different categories at multi-stages. Recently, dynamic-structure-based CIL methods achieve remarkable performance. However, these methods train all modules in a coupled manner and do not consider possible conflicts among modules, resulting in spoilage of eventual predictions. In this work, we propose a unifying energy-based theory and framework called Bi-Compatible Energy-Based Expansion and Fusion (BEEF) to analyze and achieve the goal of CIL. We demonstrate the possibility of training independent modules in a decoupled manner while achieving bi-directional compatibility among modules through two additionally allocated prototypes, and then integrating them into a unifying classifier with minimal cost. Furthermore, BEEF extends the exemplar-set to a more challenging setting, where exemplars are randomly selected and imbalanced, and maintains its performance when prior methods fail dramatically.
Extensive experiments on three widely used benchmarks: CIFAR-100, ImageNet-100, and ImageNet-1000 demonstrate that BEEF achieves state-of-the-art performance in both the ordinary and challenging  CIL settings. The Code is available at https://github.com/G-U-N/ICLR23-BEEF.

Published as a conference paper at ICLR 2023
BEEF: BI-COMPATIBLE CLASS-INCREMENTAL LEARN- ING VIA ENERGY-BASED EXPANSION AND FUSION
Fu- Yun Wang Da-Wei Zhou'
Liu Liu Han-Jia Yelt
Yatao Bian
De-Chuan Zhan' Peilin Zhao?
'State Key Laboratory for Novel Software Technology, Nanjing University Tencent AI Lab
wangfuyune sma.emmall.... ju. edu.
{zhoudw, yeh zhandc) Lamda nju.. edu,
(leonliuliu, yataobianJ@tencert com peilinzhao@@htmail com
ABSTRACT
Neural networks suffer from catastrophic forgetting when sequentially learning tasks phase-by-pil by-phase, making them inapplicable dynamically updated systems. Class-incremental learning (CIL) aims to enable neural networks to learn different categories at multi-stages. Recently, dynamic-struccure-ban
CIL methods achieve remarkable performance. However, these methods train all modules in a coupled manner and do not consider possible conflicts among modules, resulting in spoilage of eventual predictions. In this work, propose unifying masey-bbased theory and framework called Bi-Compatible Energy-Basee Expansion and Fusion (BEEF) analyze and achieve the goal of CIL. We demonstrate the possibility of train- ing independent modules a decoupled manner while achieving bi-directional compatibility among modules through two additionally allocated prototypes. and then integrating them into a unifying classifier with minimal cost. Furthermore, BEEF extends the exemplar-sed set more challenging setting, where exemplars are randomly selected and imbalanced, and maintains its performance when prior methods fail dramatically. Extensive experiments on three widely used benchmarks: CIFAR-100, ImageNet- 100, and ImageNet- 1000 demonstrate that BEEF achieves state-of-the-art performance both the ordinary and challenging CIL settings. The Code is available at https: //tt://gttii gighhub..... com/ AAGG--NNNICLLRE -BEEF.
INTRODUCTION
The ability to continuously acquire new knowledge is necessary in our ever-changing world and is considered crucial aspect of human intelligence. In the realm of applicable AI systems, it is expected that these systems can learn new concepts in a stream while retaining knowledge of previously learned concepts. However, deep neural network-based systems, which have achieved great success, face a well-known issue known as catastrophic forgetting (French, 1999; Golab & Özsu, 2003; Zhou al., 2023b), whereby they abruptly forget prior knowledge when directly fine-tuned on new tasks. To address this challenge. the inaremnanll lcarning (CIL) field aims to design learning paradigms that enable deep neural networks learn novel categories in multi multi-stages while maintaining discrimination abilities for previous ones (Rebuffi et al., 2017; Zhou et al., 2023a).
Numerous approaches have been proposed to achieve the goal of CIL, with typical methods falling into two groups: regularization-basel methods and dynamic-structuur-basee methods. Regularization- based methods (Kirkpatrick et al., 2017; Aljundi et al., 2018; Li & Hoiem, 2017; Rebuffi al., 2017) add constraints (e.g., parameter drift penalty) when updating, thus forcing the model to maintain crucial information for old categories. However, these methods often suffer from the Kubilitt-plasticit dilemma, lacking the capacity to handle all categories simultancously. Dynamic- structure- based methods (Yan et al., 2021; Li et al., 2021) expand new modules at each learning stage to enhance the model's capacity and learn the task-specific knowledge through the new module, achieving remarkable performance. Whereas, these methods have an intrinsic drawback. They
"This work is done when Fu- Yun Wang works as an intern in Tencent AI Lab. "Correspondence to: Han-Jia Ye (ychj lamda..ju.edu.cm), Liu Liu (Iconliuliu ienelliutteeeeee @twncett.cnnemm Yatao Bian (yataobian @tencent.com)

---

Published as a conference paper at ICLR 2023
Exemplar-Set Set
Task1
copy
Tea
Train
Module1 2
PC
Softmax 8
add
P/
Sampling
Outputi he1(x)
affine
hes
F2
abgg (x][0 +
Exemplar-Set
Pb
Task2 Train
copy
Train-Sef
Module2 2
FC
Softmax
Pr
Outputs h661.2y
Output2 bgz
hgz
Model Expansion
Model Fusion
The conceptual illustration of BEEF. The training consists of two phases: expansion and fusion. At the expansion phase, we independently train the new module for the current task, while classifying all the samples from prior tasks into ps and classifying all samples generated from the built-in energy-based model into py.
Figure 1:
fusion phase, the output pa is equally added the output of prior modules mitigate task-bias and form unifying classifier.
directly retain all learned modules without considering confficts among modules, thus corrupting the joint feature representation and misleading the ultimate predictions. For example, old modules not considering the future possible updates may mislead the final prediction for new classes. This defect limits their performance on long-term incremental tasks.
In this paper, we aim to reduce the possible conflicts in dyynamic-structure-bae innovatively propose achieve bi-directional compatibility, which consists of backward compatibility and forward compatibility. To be specific, backward compatibility is committed to making the discrimination ability of old modules unaffected by new ones. On the contrary. forward compatibility aims to reduce impact of old modules on the ultimate predictions when new categories emerge. By achieving bi-directional compatibility, given sample from a specific task, the module responsible for the corresponding task will dominate the ultimate predictions in the ideal case.
methods and
contrary.forward
Fig. displays the BEEF training framework, which made of two phases (Model Expansion and Model Fusion). At the expansion phase, we assume that different modules are independent and train them isolatedly. Then, at the fusion phase, all trained modules are combined to form unifying classifier. To achieve bi-directional compatibility. we introduce two additional prototypes: forward prototype py and backward prototype P The backward prototype Р6 is set to measure the confidence of old classes, while the forward prototype Py aims measure uncertainty in the open world. To be specific, when training a new module, we set Ps as the cluster prototype for all old classes and set p
learn task boundary between the current task and prior ones. In the meanwhile, as the cluster prototype for samples from the unseen distributions generated through energy-based sampling and thus use measure uncertainty and better capture the current distribution. We show that BEEF can be deduced from a unifying extendable energy-based theoretical framework, which allows us transform the open-worke world problem into normal classification problem and model the input distribution synchronously while learning the discrimination ability. Vast experiments on three widely-used benchmarks show that our method achieves state-of-the-ard performance. Besides, prevalent CII. methods all require well-selected balanced exemplar-set for rehearsal, which might be impractical due to data privacy issues (Delange et al., 2021; Ji et al., 2014) and computation cost when choosing exemplars. Our method pushes it to a harder setting. With only randomly sampled data from prior tasks, BEEF maintains its effectiveness while the performance of other methods declines drastically.
RELATED WORK
Incremental learning. Most recent studies on incremental learning are either busk- or class- based. The crucial difference hetween them whether task-id is known at the evaluation phase (Van de
2

---

Published as a conference paper at ICLR 2023
Ven & Tolias, 2019). Our work is a baass- method, which is typically called dlassinncremenal learning (CIL), Prevalent CIL methods can be categorized into two classes: regularization based. and dynamic-struuture-bee bastre Regularization based methods impose constraints when learning tasks. Kirkpatrick et al. (2017); Aljundi et al. (2018) 8) penalize the parameter drift. Li & Hoiem (2017), 7); Rehuffi et al. (2017); Wu et al. (2019); Zhao al. (2020) utilize knowledge distillation (Hinton et al., 2015) to constrain the model's output. Douillard et al. (2020) propose a novel spatial knowledge distillation. Zhou al. (2022) propose the concept of forward compatible (Gheorghioiu et al., 2003; Shen et al., 2020) and squeeze the space for known categories, thereby reserving feature space for future categories. Dynamic-structure based methods create new modules to enhance the capacity for learning new tasks. Yan et al. (2021); Li et al. (2021) combine all modules together to form a unifying classifier, but it leads to an increasing training cost. Douillard et al. (2021) applies transformer (Dosovitskiy et al., 2020; Touvron et al., 2021) to CII. and dynamically expands task tokens when learning new tasks. Wang et al. (2022a) proposes to dynamically expand and compress the model based on gradient boosting (Mason et al., 1999) to adaptively learn new tasks. Liu et al. (2021b; 2023) cleverly apply reinforcement learning in CIL to obtain univserally better memory management strategy or hyperparameters. However, prevalent CIL approaches usually require well-selected class-balanced exemplar-set for rehearsal (Rebuffi et al., 2017), which has an evident impact on their performance (Masana et al., 2020) as we verify experimentally. BEEF not only achieves tfute-ff-the-al performance but shows strong robustness to the choice of exemplar-sed
Energy-based learning. EBMs define probability distributions with density proportional to exp( (x)(-E). where E is the energy function (LeCun et al., 2006). So far, the theory and implementation of EBMs have been well studied. Xie et al. (2016) show that the generative random field model can be derived from the discriminative ConvNet. Xie et al. (2018s) 2022) well study the cooperative training of two generative models for image modeling and synthesis. Additionally, Nijkamp et al. (2019) propose to treat the non-convergent short-run MCMC as a learned generator model or a flow model and show that the model is capable of generating realistic samples. Xie et al. (2021c) 1c) propose to learn a VAE to initialize the finite-step MCMC for efficient amortized sampling of the EBM. Xiao et al. (2021) propose a symbiotic composition of a VAE and an EBM that can generate high-quality images while achieving fast traversal of the data manifold. Zhao et al. (2021) propose multistage coarse-to-fnne expanding and sampling strategy, which starts with learning a coarse-level EBM from images at low resolution and then gradually transits to learn finer-level EBM from images at higher resolution. Besides, EBMs have been successfully applied in many fields such as data generation (Zhai et al., 2016; Zhao et al., 2016; Deng et al., 2020; Du & Mordatch, 2019) with various data formats including graph (Liu et al., 202 10) video (Xie et al., 2017; 2019), 3D volumetric shape (Xie et al., 2018b; 2020), 3D unordered point cloud (Xie et al., a0) amage-to-image translation (Xie et al., 2021c;b), saliency map (Zhang et al., 2022), etc., -ut-of-distribution detection (OOD) (Hendrycks & Gimpel, 2016; Bai et al., 2021; Liu et al., 2020; Lee et al., 2020; Lin et al., 2021), and density estimation (Silverman, 2018; Zhao et al., 2016), etc. Wang et al. (2021) model the open world uncertainty as an extra dimension in the classifier, achieving better calibration in OOD datasets. Grathwohl et al. (2019) propose to model the joint distribution F(x, y) for the classification problem. Bian et al. (2022) apply snergy-based learning for cooperative games and derive new player valuation methods. Zheng et al. (2021) propose to represent the statistical distribution within single natural image through an EBM framework. Xu et al. (2022) apply EBM for inverse optimal control and autonomous driving. There have been several attempts to apply EBMs incremental learning. Li al. (2020) propose a novel energy-based classification loss and network structure for continual learning. Joseph et al. (2022) build an energy- based latent aligner that recovers the corrupted latent representation. Wang et al. (2022b) propose anchor-based energy self-normalization classifier in incremental learning.
3 METHOD
In this section, we give a description of BEEF and how we apply EBM to CIL. to learn a unifying classifier while achieving bi-directional compatibility. In Sec. 3.1, we first introduce some basic knowledge of CIL. In Sec. 3.2, we present the definition of energy and then deduce optimization objective at the expansion phase. Then, to avoid the intractability of normalizing constant, we prove a gradient equivalent objective and explain why it helps to achieve the bi-directional ability. After that, we propose an efficient yet effective fusion strategy in See. 3.3. Additionally, we extend BEEF to Stable-BEEF by allocating multiple forward prototypes and backward prototypes. The detailed illustration and proof are provided in Appendix B.
3

---

Published as a conference paper at ICLR 2023
3.1 PRELIMINARIES
CII aims to learn a unifying classifier from sequence of data divided into several incremental sessions with different class groups. At the tdh incremental session, the model receives the training dataset D, {xx, v() v()]]== where x E Ae is input sample and y E V/ is the corresponding label, which is not accessible at latter sessions. Only a small amount of exemplars of previous categories retained in bize-limited exemplar-set V S D The model is expected to train on D, U Va and be evaluated on the test set of all known categories. At the following discussions, we focus on the details of the 1th incremental session without loss of generality. Particularly, spaces of all known classes and novel classes as y% = D and ) = Vv. respectively. [3ir] = K and |D:o M, representing the number of new categories and that of old ones.
denote the label
3.2 ENERGY-BASESD MODEL. EXPANSION
Let ho :x= - A be the newly created module (typically as a single-skeleton CNN), where X , and A K+ is K 1-standard simplex (i.e.. K + 2 dimensional vectors with non- negative elements that sum up to 1). Therefore, we can further decompose he as S F o Q where Φ: - d is the non-linear feature extractor, F Id - I *2 is a linear classifier transforming the feature into the K + dimensional logits, and S denotes the non-linear activation function softmax which constrains the final output the K + 1- standard simplex. hn [A] represents the k + 11hh element of the final output. Ignoring the bias, F can be denote as a d x (K + 2) matrix
F= Ps F Fasee Pf J. where Fasee with shape x K is the base classifier for the current task, and
Ps Ps is an additional prototype that measures past confidence / future uncertainty, transforming the feature extracted from & into logits index 0/ K
First, given an input-label pair (x. v) E X' x (V. UY.), we define the energy (x y) as
Ex y) log ho (o)
y yEYn
log (x)[00/))
y EE
where a : Vn > 1,2,..., K is a bijection function mapping a given label to its corresponding class index. The energy E(x, y) measures the uncertainty of predicting x's label y. Hence, the definition of the energy is compatible with traditional classification definitions, since we typically use ho(x)(o which is the negative exponent of the energy. to indicate the confidence of predicting x's label as y. Moreover, we use ha (x)(0 to represent the overall confidence of x's label belonging module to distinguish between the old categories. Hence, F((x.) y)
to V% and do not expect the for any y E V% is represented as - log [0)[[] The denominator M makes the energy larger and indicates that the current module has larger uncertainty about old categories due the limited supervision for old categories from VL.
Since Pe(u|x) =
En exp(-Ex(x(,)))) Ex(x,y)) and Px(x) = Σn En "xp(-Eo(x',,)) w')) the conditional probabil-
exp( exp((=E(xx)) (x(x)))
Ey expl (x.,))))
ity density and marginal probability density can be formulated as
(v) [x]]
E Pnn
Po(x) = ER ho [K)
Po(v(x)
y EE
Ex Σ k= ho (x')
We define the energy function Fx(x) via (
Σx expl expplBBap -Bo(x then Flg(x) is formulated
expl - Ea (x))
Ex(x) = - Σ
k=0 he (x)
(3)
With energy functions defined above, we give proof of the derivation of our optimization objective when training new module and demonstrate how it works to achieve bi-directional compatibility. Instead of simply learning a discriminator Po (y) x). which usually causes overconfident predicts even when receiving samples from unseen distributions,
estimate the joint distribution Po v)
arg min B
Ep (x.eat(x.v) [- log Pe (xx,))
(4)
arg min Eryea((s) [- log Pe (x)] + Er. (r.eat [- log ()(())))
(5)
The estimation of the joint distribution not only encourages the model to learn to distinguish all known categories but also helps model the input distribution, thus making the model sensitive the
4

---

Published as a conference paper at ICLR 2023
Pr
Update Punameters
Energy Alignment
ModelTearming
Energy Aligument
Energy Manrifold Update
Figure 2: Learning energy manifold. The learning process of the energy manifold. The left part illustrates the concrete workflow of model training. The middle and right part illustrate the corresponding high-level abstract energy manifold learning process.
input distribution drift. Therefore, when unseen categories emerge, it helps those modules alleviate overconfident predictions and reduce their impact on the ultimate predictions. However, due to the intractability of the normalizing constant Ex Ey exp(x- Eg Exx,, y))), optimize the gradient equivalent objective of Eq. 4.
Theorem 3.1 (Marginal Distribution Maximum Likelihood Estimation). Defining Eg(x) - Log [ + 1] and its corresponding marginal distribution as (x),, the optimiza- tion of ((a)) [- log Po(x))
equivalent to that of Exrea(()) [ log Σk (x) (x) I- log (x(x)[ + 1]] when gradient descend applied, where Ng is the ratio of the nor- malizing constants determined by Egg (x) and Eg(x), and @ means that parameters of f is frozen (i.e.. instances sampled from IP (x) are detached).
Theorem 3.2 (Conditional Distribution Maximum Likelihood Estimation). With preliminar- ies from Thm. 3.1, the optimization of (xrea(Xx.)) [- log Po(y)))] is equivalent that of (p.ca(ssy) [- log (x))oo' + MgBr, (x) log ho (x) + 1]] when gradient descend is ap- [+] plied, where En [ o'vy) = (b()), E Vn 0,
E yo
Due to the space limit, detailed proofs for Thm. 3.1 and Thm. 3.2 are deferred to Appendix A. Combining Thm. 3.1 and Thm. 3.2, the ultimate optimization objective can formulated as
[ log Σ K
Ey (x))
ha [x)
+ Ar (x) |- log hi (x)[K + 1]]
k=0
(6)
Eprea( [- log ha (x) (x)]l + Eg (x) log (x) +
which is upper bounded by
2Ep 2(gr.ei(x.))
[- log ho(x) (x) (y))] + Ng Eg hx) + +1]] Ax [- log (x)[K 1]]
(Objective)
We take Eq. 7 as the eventual training objective. Here, we explain the roles of different compo- nents and the reason why better bi-directional compatibility achieved through optimizing this objective. Expeat [- log he (x) (v)]] prompts the new module to accurately discriminate all categories from current task as well as build explicit decision boundaries between current task and prior ones. By setting рş as the special prototype for all old categories, we can better exploit the shared structure of all old categories and reduce the risk of over-fitting, which typically results from inadequate training samples on old categories. Given a sample from prior tasks, the new module perceives this task boundary and reduces the confidence of its own task, so that the old achieve better backward compatibility for
module dominates the ultimate prediction. Hence, old categories than naively tuning the new module on all categories. ( | log ho (x) (x)[ 1] encourages the module to reserve certain degree of confidence for virtual class py, thus measuring the out-of-distribution uncertainty for given samples and mitigating overconfident predictions. As shown in Fig. 2, (x, [- log he (x) 1]] introduces adversarial learning process, where we iteratively generate samples believed to have lower energies from Py (x) and then update the energy manifold to increase the energy of generated samples and decrease that of real samples. This process
5

---

Published as a conference paper at ICLR 2023
effectively enhances the modeling of the known input distribution, making in-distribution samples have low energy and outto--distribution data have high energy. Therefore, for a sample from unseen distributions, the module will produce predictions with large uncertainty (hg (hg (x)[ 11) and low confidence due to the fact that the confidence must be lower than he (x) K + 1]. Then, modules created in the future to handle these unknown distributions will dominate the final prediction. Hence, we achieve the forward compatibility for the future unseen categories.
3.3
ENERGY-BASED MODEL FUSION
After training the new module, we aim to fuse it with the prior ones form a unifying classifier for all seen categories. Assuming that we have trained a unifying model ha, for all the old tasks and a, maps the label to the output index of he, vanilla approach to combining the he, and he is to redefine the energy function as
Po acts seft task-discrimiinatr
Cat
Passla
Po
autterlly tiiordloye
E1 03 (x, = {
log ho. (x)), log ho (x)
y E V E )
Modute1
Modnle22
(8)
Figure 3: Pa acts soft task discriminator.
Then we have
It alleviates the task- biis and determines the dominant module for the ultimate prediction.
hy (x)))_/]rr [x))
y E V.
P(o.0,) (v/x)
m=i hew (xc > ha (x)
fey (00)) [w))
y D
Σ ho, [x] | + ho (x)(
(9)
However, this might cause task bias Different modules may produce predictions with different entropies, the combined model has tendency modules with larger entropies. As shown in Fig. Simply combing the modules as Eq. 9 leads to misclassification due to the larger entropy in module2. Considering that pa measures the confidence for old categories, we redefine E(o,,0 as
log (ha, (x) + who (x)(())(0 B}, y V log (o)
(10)
yE V
Then we have
he [o)l
E Vo (11)
P P(o,.o) (v|x)
Σ
he, [m] +ahu( =1 ho (x)
ha(x)
y Dn
[00_00/mm)-aooo00000-------- No (x)
We finetune a, 33 to minimize the negative log-likelihood on tiny sub-dataset (exemplar-set), thus mitigating the task bias, namly
ax ,²,
arg minE, (ra |- log P ((0.0.
(12)
0.9
3.4 SUMMARY OF BEEF
To conclude, propose a two-stage training approach: expansion and fusion. The expansion phase is intrinsically similar to naive fine-tuning, while trough our novel energy definition and gradient-equivalent optimization simplification, a K + 2 classification model through two additional prototypes (e.g. backward prototype ps and forward prototype py) achieve the bi-directional compatibility. Specifically, Ps learns measure the confidence on old categories by acting as the cluster prototype for all old samples and thus achieves the backward compatibility. In contrast, P, learns measure the open world uncertainty by acting as the cluster prototype for all samples generated by the built-in energy-based model to achieve the forward compatibility. By means of our energy-based framework, our model is able to learn the discrimination ability for the current task while synchronously modeling the input distribution. At the fusion phase, the confidence of the old prototype is added to the outputs of all old modules after passing through learnable affine transformation. forming the ultimate prediction on all categories. The fusion strategy alleviates the possible task bias and effectively improves the performance compared with the naive fusion strategy in Eq. 9. Equipped with energy-based nature, BEEF allows for pleasant by- product: test-time alignment. That is, we apply SGLD (Welling & Teh, 201 1) to decrease energy of given test samples. Through test-time alignment, some useful
expand the original K classification model into

---

Published as a conference paper at ICLR 2023
characteristics shared by training samples are transferred into the test sample to reduce its energy, making it more explicit and producing more convincing predictions. We find that this process does improve the performance in many protocols though requiring additional computational resources in evaluation. One inevitable drawback of dynamic-based methods is that they suffer from a linearly growing number of parameters the number of tasks, which might violate the memory usage limitations in CIL. Therefore, for practical usage and fair comparison, we apply the compression strategy following FOSTER (Wang et al., 2022a) compress the expanded dual branch model into a single skeleton after each incremental learning session, which we call BEEF- Compress
4 EMPIRICAL STUDIES
4.1 EXPERIMENTAL SETTINGS
Datasets. We validate our methods on widely used benchmarks of class-incremental learning CIFAR-
100 (Krizhevsky et al., 2009) and ImageNet100/1000 (Deng et al., 2009). CIFAR-100: CIFAR-100
consists of 50,000 training images with 500 images per class, and 10,000 test images with 100
images per class. ImageNet-1000, ImageNet- 1000 is large scale dataset composed of about 1.28 million images for training and 50,000 for validation with 500 images per class. ImageNet-100:
ImageNet-100 is composed of 100 classes randomly chosen from the original ImageNet- 1000 dataset.
CIAFR-100 Protocol. For benehmark CIFAR-100, we evaluate two widely recognized protocols: CIFAR-100 BO: All the 100 classes are averagely divided into 5, 10, and 20 groups, respectively, i.e., we should train all the 100 classes gradually with 20, 10, 5 classes per incremental session. In addition, models are allowed to save an exemplar-stt to store more than 2000 exemplars throughout all sessions. CIFAR- R-FAR-100 100 B50: We first train half of 100 classes at the base learning stage. Then, the rest 50 classes averagely divided into 5, 10, and 25 groups, respectively, i.e., we should train the rest 50 classes gradually with 10, 5, and 2 classes per incremental session. Slightly different from the first protocol, models are allowed to store more than 20 exemplars for each class. Therefore, after training all the 100 classes, there are also no more than 2,000 exemplars.
ImageNet Protocol. For benchmarking ImageNet-1000 100, we evaluate the performance incremental tasks. In the first task, we split the 100 classes averagely into 10 sequential incremental sessions, and up 2,000 exemplars are allowed to be stored in the exemplar-set. In the second task, models are first trained on 50 base classes and then sequentially trained on the 10 classes at the following incremental sessions (i.e., incremental sessions totally). The same as the protocol CIFAR- 100 B50, models are allowed to store no more than 20 exemplars for each class. For benchmark ImageNet- we train all the 1000 classes with 100 classes per step (10 steps in total) with an exemplar-set storing no more than 20,000 exemplars.
different
Compared methods. Our method and all baselines are implemented with Pytorch (Paszke et al., 2017) in PyCIL (Zhou et al., a)). We compare BEEF to strong egularizationnbased methods: iCaRL (Rebuffi et al., 2017), BiC (Wu et al., 2019), WA (Zhao et al., 2020), PodNet (Douillard et al., 2020), and Coil (Zhou et al., 2021b). Beside, method RPSNet (Rajasegaran et al., 2019), DER (Yan et al., 2021), Dytox (Douillard et al., 202 1), FOSTER (Wang al., 2022a). Apart from the above methods, we also compare with RMM (Liu et al., 2021b), which adjusts the memory partition strategy for new and old data. Among the compared methods, Dytox applies stronger neural architecture (Convit (d' Ascoli al., 2021)) and additional data augmentation; FOSTER (Wang et al., 2022a) additionally uses the AutoAugmentation (Cubuk al., 2019) to enhance the sample efficiency and improve classification accuracy. RMM (Liu et al., 2021b) achieves a better memory management strategy. With the same training memory, RMM chooses some new samples to train and discards the rest allow restoring more old exemplars. Note that all these are orthogonal to the method itself, and therefore we combine the augmentation and memory management strategy with BEEF for fair comparison with these methods, which we regard as BEEF-Compress
compare to the synamic-structuree based
4.2 RESULTS AND ANALYSIS
Comparison with SOTAS. CIFAR-100: Table 2. Fig. 4, and Fig. 7 summarize the experimental results on CIFAR- 100 benchmark. We can observe that SELE/BEEF-Comppeee achieves -f art performance under both BO and B50 protocols. And more evident performance improvements are achieved under B50 protocol. Specifically, BEEF improves average accuracy by 3.12. 3.62, 5,45 compared to DER (prior sfate-of-the-art dynamic-hased method) under protocol B50 with 5, 10, 25
7

---

Published as a conference paper at ICLR 2023
ImageNet-100 100 10 steps
ImageNet- 1000
10 steps
ImageNet- 100 B50 5 stops
Methods
top-1
top-5
top-1
top-5
top-1
top-5
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Bound
81.50
95.10
89.27
79.89
81.20
81.50
95.10
Replay
59.21
41.00
81.67
68.44
55.73
43.38
79.17
71.08
iCaRL
67.11
50.98
84,08
71.52
38.4
22.7
63.7
44.0
62.56
53.69
81.75
73.58
BIC
65.13
42.40
84.04
64.14
84.0
73.2
66.36
49.9
83.59
70.42
WA
68.60
55.04
89.53
80.32
65.67
55.60
86.60
81.10
65.81
56.64
84.97
79.36
PodNet
64.03
45.
84.06
68.58
73.84
62.94
89.51
83.52
DER
77.08
66.84
92.49
88,64
66.87
58.83
88.01
81.59
77:57
71.10
93.37
91.3
DyTox
71.85
57.94
90.72
83.52
68.14
59.75
87.03
82.93
RPSNet
87,90
74.00
RMM
79.52
FOSTER
78.71
70.14
68.34
58.53
80.22
75.52
BEEF
77.62
68.78
93.66
89.32
67.09
58.67
86.21
81.73
77.27
70.98
93.71
91.76
BEEF-Compress
79.34
71.12
93.30
88.94
80.52
74.62
94.10
91.42
Table 1: Performance ImageNet. We report both average and last accuracy of top-1 and top-5.
CIFAR-100 BO
CIFAR- 100 B50
Methods
steps
10 steps
20 steps
5 steps
10 steps
25 steps
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Bound
80,40
80.41
81.49
79.89
79.91
80.37
Replay
60.63
43.08
59.38
41.01
58.20
38.69
52.70
41.26
43.43
36.16
41.09
37.50
iCaRL
67.60
54.23
64.64
49.52
63.51
45.12
61.79
52.04
$2.69
44.64
52.10
45.57
BIC
67.63
56.22
65.38
50.79
62.38
43.08
61.68
49.19
57.04
43.82
53.61
40.38
WA
69.11
57.97
67.15
52.30
64.65
48.46
64.65
55.85
53.87
46.72
52.51
44,90
Coil
68,26
65.48
62,98
DER
71.15
62.4
69.94
58.59
67.98
53.95
68.58
61.94
66.40
58.85
60.66
49.30
RPSNet
70.5
68.6
DyTox
71.50
57.76
68.86
51.47
RMM
68.86
67.61
66.21
FOSTER
72.54
64.55
72.81
62.54
70.65
56.28
70.
64.01
67.94
60.44
63.83
54.31
BEEF
72.31
62.58
71.94
60.98
89.84
56.71
71.70
65.24
70.71
63.51
66.11
54.36
BFFF-Compress
73.05
62.48
72.93
61.45
71.69
57.06
71.58
64.54
71.70
61.19
64.32
54.81
Table 2: Performance on CIFAR-100. Wc report both the top-1 average and last accuracy.
incremental sessions. ImageNet--00/1000: Tabel 1 and Fig. 8 summarize the experimental results on both ImageNet- 100 and ImageNet- 1000 benchmarks. We can observe that BEEF, especially BEEF-Compress achieves very competitive performance compared to prior methods. Specifically, BEEF improves average accuracy by 0.63, 0.3 under Image 100 protocols. that the performance improvement of BEEF on ImageNet is relatively less significant compared to CIFAR-100, which may be attributed to the larger and more complex nature of the ImageNet dataset, making it more challenging to find suitable cluster prototypes as forward/backwaat prototypes.
also worth noting
Comparison under imbalanced exemplar-set. To compare the performance of different methods with imbalanced exemplar-set, we propose three different exemplar selection strategies: exp, random, and half-half. Refer to Appendix C.I for details. Fig. displays the average accuracy changes of different methods after each incremental session CIFAR-10M B50 with 5 steps. Fig. 5(c) illustrates the performance when exemplars are randomly sampled from all the available old instances. Though the exemplar-set statistically balanced, prior methods encounters a performance drop and the gap between BEEF and prior methods is enlarged. Fig. 5(b) and Fig. 5(a) illustrate the performance changes under extreme class imbalance and sample from half classes missing, respectively. Although the performance of prior methods declines dramatically, BEEF maintains its effectiveness under these two imbalanced protocols. BEEF achieves more than 10% performance gain under these challenging settings. In addition, since some classes have no exemplars stored in the exemplar-set to calculate the class center, iCaRL. based on NCM-classifier (Mensink et al., 2013) fails in the base training phase.
4.3 ABLATION STUDIES
Ablations of key components in BEEF, To verify the effectiveness of the components in BEEF, we conduct ablation studies on CIFAR- 100 B50 with 5 incremental sessions. As shown in Table 3, the average and last accuracy gradually increase fusion strategy, the unifying model get a large increase. Besides, by learning energy manifolds with the forward prototype py and energy alignment, we can further improve the performance.
add more components. It is notable that after the
8

---

Published as a conference paper at ICLR 2023
100
106
100
Avcurary (16)
Accurasy (5o
1.15
Accuracy
-aapp +DDoR 00A
+-aillo +DD0R
+0221
160
28
40
100
28
Number of classes
Number of classes
Number of casses
CIFAR-100 BO 20 steps
CIFAR-100 BO steps
CIFAR-100 BO steps
Figure 4: Performance on standard CIFAR- 100 BO protocols.
Accuraty (Mg)
Accuracy (K)
11399
Accuacy
+xplo +08
Kiplo UFC
+iplp +ICK
#2301 Ro
Numbar
Number
Number
(a) exp: = 0.8
(b) half-half: =
(c) random
Figure 5: Performance imbalanced protocols.
Expansion
Fusion
backward compatible
forward compatible
task discriminator
energy alignment
Avg
Last
63.60
55.16
64.50
56.50
70.51
64.43
70.92
64.79
71.75
65.24
Table 3: Ablations of key components in BEEF. We report the average and last accuracy on CIFAR- 100 B50
with 5 incremental sessions.
Sensitive study of pereerpprraeters. There are three hyper-parameters in BEEF when modeling energy manifold: hidden layer where energy modeling. the trade-off coefficient X, and the number of forward prototypes F in Stable-BEER, As shown in Fig. 6(a) conduct experiments three different hidden layers for modeling the energy manifold, and the experimental results show BEEF the robustness the choice of different hidden layer. We also change the trade-off coefficients X
from (0, le 1, le 2, le 3} and the number of forward prototypes from {1, 5, 20, 50}, and the average accuracies CIFAR- 100 B50 with 5 incremental sessions shown in Fig. 6(b). We
can see that with the increase of F and X, the accuracy has an upward trend.
70.3
70.5
70.3
20.18
Accuracy (%6
80
Trads off Purmater
to-1-
71.42
-75
20
1e-2-
70.64
71.21
71.22
71.51
BEEP-112
RESR-111
1-e3
71.0
50
60
70
30
50
100
Number of classes
Prototypes
(a) Influence of hidden layers
(b) Impact of X, F
Figure 6: Sensitive studies of hyper-parameters.
CONCLUSION. In this work presented BEEF for achieving efficient class-incremental learning. Under this framework, we efficiently train a specific module for the current task while achieving bi- directional compatibility and fuse it with the prior model under minimal effort. BEEF is equipped with a theoretical analysis showing that its training process is inherently the modeling an energy-based model. Compression strategies can be applied address the issue of growing storage overhead.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENT. This work is partially supported by NSFC (61921006, 620061 12, 62250069), NSF of Jiangsu Province (BK20200313), Collaborative Innovation Center of Novel Software Technology and Industrialization.
10

---

Published as a conference paper at ICLR 2023
REFERENCES
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseeny, Mareus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, pp. 139-154, 2018.
Haoyue Bai, Rui Sun, Lanqing Hong. Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S-H Gary Chan, and Zhenguo Li. Decaug: Out-of- of-distributiin generalization via decomposed feature representation and semantic augmentation. In AAAI, volume 35, pp. 6705-6713, 2021.
Yatao Bian, Yu Rong, Tingyang Xu, Jiaxiang Wu, Andreas Krause, and Junzhou Huang. Energy- based learning for cooperative games, with applications to valuation problems in machine learning. In ICLR, 2022. URL https: httttttt//peeereeeeeee /ttp///pppeeeeeeee///e/eeee net.
ImCroImw.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In CVPR, pp. 113-123, 2019.
Matthias Delange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions Pattern Analysis and Machine Intelligence, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248-255. leee, 2009.
Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc' Aurelio Ranzato. Residual energy-based models for text generation. arXiv preprini arXiv:2004. 2020..
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. 1929, 2020.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In ECCV, pp. 86-102. Springer, 2020.
Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthicu Cord. Dytox: Transformers for continual learning with dynamic token expansion. arXiv preprint arXiv:2111.11326 2021.
Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. NeurIPS, 32, 2019.
Stéphane Ascoli, Hugo Touvron, Matthew Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. In ICML, pp. 2286-2296. PMLR, 2021.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, (4):128-135, 1999.
Ovidiu Gheorghioiu, Alexandru Salcianu, and Martin Rinard. Interprocedural compatibility analysis for static object preallocation. In POPL, pp. 273-284, 2003.
Lukasz Golab and M Tamer Özsu. Issues in data stream management. ACM Sigmod Record, 32(2): 5-14, 2003.
Will Grathwohl, Kuan-Chieh Wang. Jömn Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.0322 032622622.. 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pp. 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv: 1610.02/36 2016.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network. arXiv preprint arXiv: /503.02533333 2(7), 2015.
11

---

Published as a conference paper at ICLR 2023
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771-1800, 2002.
Zhanglong Ji, Zachary C Lipton, and Charles Elkan. Differential privacy and machine learning: survey and review, arXiv preprint arXiv: 1412.7584 2014.
KJ Joseph, Salman Khan, Fahad Shahbaz Khan, Rao Muhammad Anwer, and Vineeth Bala- subramanian. Energy-based latent aligner for incremental learning. In CVPR, pp. 7452-7461, 2022.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka catastrophic forgetting in neural networks. PNAS, 11((1))3551-333 3326,, 2017.
et al. Overcoming
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang. A tutorial energy-based learning. Predicting structured data, 1(0), 2006.
Kyungmin Lee, Hunmin Yang, and Se- Yoon Oh. Adversarial training on joint energy based model for robust classification and out-of-distribution detection. ICCAS, pp. 17-21. IEEE, 2020.
Shuang Li, Yilun Du, Gido M van de Ven, and Igor Mordatch. Energy-based models for continual learning. arXiv preprint arXiv:2011. 2020.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-247) 2017.
Zhuoyun Li, Changhong Zhong, Sijia Liu, Ruixuan Wang, and Wei-Shi Zheng. Preserving earlier knowledge in continual learning with the help of all previous feature extractors. arXiv preprint wrXiv:2104.13614 2021.
Ziqian Lin, Sreya Dutta Roy, and Yixuan Li. Mood: Multi-level out-of-distribution detection. In CVPR, pp. 15313-15323, 2021.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. Graphebm: Molecular graph generation with energy-based models. arXiv preprint arXiv:2102.005444 2021a.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy- out-of of-diitrriuutinn detection. NeurlPS, 33:21 33:21464-21175. 2020.
Yaoyao Liu, Bernt Schiele, and Qianru Sun. Rmm: Reinforced memory management for class- incremental learning. NeurlPs, 34:3478-3490, 2021b.
Yaoyao Liu, Yingying Li, Bernt Schiele, and Qianru Sun. Online hyperparameter optimization for inaremeacemmttlll learning. arXiv preprint arXiv:2301.05032 2023.
Marc Masana. Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer. Class-incremental learning: survey and performance evaluation on image classifica- tion. arXiv preprint arXiv:2010.152111 2020.
Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean. Boosting algorithms as gradient descent. NeurIPS, 12, 1999.
Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-based image classification: Generalizing to new classes at near-zero cost. IEEE transactions on pattern analysis and machine intelligence, 35(11):2624-26)
2013.
Erik Nijkamp, Mitch Hill, Song- Song-Chun Zhu, and Ying Nian Wu. Learning non-convergent non- persistent short-run meme toward energy- based model. NeurlPs, 32, 2019.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary Vito Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
12

---

Published as a conference paper at ICLR 2023
Jathushan Rajasegaran, Munawar Hayat, Salman Khan, Fahad Shahbaz Khan, Ling Shao, and Ming- Hsuan Yang. An adaptive random path selection approach for incremental learning. arXiv preprint 10Xx::1906.01120, 2019.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, pp. 2001-2010, 2017.
Yantao Shen, Yuanjun Xiong, Wei Xia, and Stefano Soatto. Towards backward-compatible represen- tation learning. In CVPR, pp. 6368-66660 60087677, 2020.
Bernard W Silverman. Density estimation for statistics and data analysis. Routledge, 2018.
Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers. In ICCV, pp. 32 42, 2021.
Gido M Van de Ven and Andreas S Tollias. Three scenarios for continual learning. arXiv preprint orXiv:1904.07734 2019.
Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for class-incremental learning, arXiv preprint arXiv:2204.04000 2022a.
Yabin Wang, Zhiheng Ma, Zhiwu Huang, Yaowei Wang, Zhou Su, and Xiaopeng Hong. Isolation and impartial aggregation: A paradigm of incremental learning without interference. arXiv preprint arXiv:2211.15969, 2022b.
Yezhen Wang, Bo Li, Tong Che, Kaiyang Zhou, Ziwei Liu, and Dongsheng Li. Energy-based open-world uncertainty modeling for confidence calibration. In ICCV, pp. 9302-9311, 2021.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. ICMI., pp. 681-688. Citeseer, 2011.
Yue Wu. Yinpeng Chen, Lijuan Wang, Yuancheng Ye. Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR, pp. 374-382, 2019.
Zhisheng Xiao, Karsten Kreis, Jan Kautz, and Arash Vahdat. VAEBM: A symbiosis between varia- tional autoencoders and energy-based models. In ICLR, 2021. URL net/ forum? 5t
Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In ICML, pp. 2635-2644. PMLR, 2016.
Jianwen Xie, Song-Chur Cong-Chun Zhu, and Ying Nian Wu. Synthesizing dynamic patterns by generative convnet. In CVPR, pp. 7093-7101, 2017.
Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, and Ying Nian Wu. Cooperative training of descriptor and generator networks. IEEE transactions paltern analysis and machine intelligence, 42(1):27-45, 2018a.
Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Chun Zhu, and Ying Nian Wu. Learning descriptor networks for 3d shape synthesis and analysis. In CVPR, pp. 2018b. 8b.
Jianwen Xie, Song-Chun Zhu, and Ying Nian Wu. Learning energy-based spatial-temporal generative convnets for dynamic patterns. IEEE transactions on pattern analysis and machine intelligence, 43(2):516-531, 2019.
Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, and Ying Nian Wu. Generative voxelnet: learning energy-based models for 3d shape synthesis and analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2468-244 2284, 2020.
Jianwen Xie, Yifei Xu, Zilong Zheng, Song- Song-Chun Zhu, and Ying Nian Wu. Generative pointnet: Deep energy-based learning on unordered point sets for generation, reconstruction and classification. In CVPR, pp. 14976-1499 14955, 2021a.
13

---

Published as a conference paper at ICLR 2023
Jianwen Xie, Zilong Zheng, Xiaolin Fang. Song- Chug Zhu, and Ying Nian Wu. Cooperative training of fast thinking initializer and slow thinking solver for conditional learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):3957-3973, 2021b.
Jianwen Xie, Zilong Zheng, and Ping Li. Learning energy-based model with variational auto-encoder as amortized sampler. In AAAI, volume 35, pp. 10441-10451, 2021c.
Jianwen Xie, Yaxuan Zhu, Jun Li, and Ping Li. A tale of two flows: Cooperative learning of langevin flow and normalizing flow toward energy-based model. In ICLR. OpenReview.net, 2022. URL
pettps://openevieet net forrmmidd-1ddsiiuuu
Yifei Xu, Jianwen Xie, Tianyang Zhao, Chris Baker, Yibiao Zhao, and Ying Nian Wu. Energy- -bagyy continuous inverse optimal control. IEEE transactions on neural networks and learning systems, 2022.
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning, In CVPR, pp. 3014-3023, 2021.
Shuangfei Zhai, Yu Cheng, Rogerio Feris, and Zhongfei Zhang. Generative adversarial networks as variational training of energy based models. arXiv preprint arXiv: /6/ 1.01799, 2016.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018.
Jing Zhang, Jianwen Xie, Zilong Zheng, and Nick Barnes. Energy-based generative cooperative saliency prediction. In AAAI, volume 36, pp. 3280-3290, 2022.
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairness in class incremental learning. In CVPR, pp. 13208-13217, 2020.
Junbo Zhao, Michael Mathicu, and Yann LeCun. Energy- based generative adversarial network. arXiv preprint arXiv: /609. 06096669.030 03/06.,2666 2016.
Yang Zhao, Jianwen Xie, and Ping Li. Learning energy-based generative models via coarse-to-fin expanding and sampling. In ICLR. OpenReview.net, 2021. URL https:///penreviee.
Zilong Zheng, Jianwen Xie, and Ping Li. Patchwise generative convnet: Training energy-based models from single natural image for internal learning. In CVPR, pp. 2961-2970, 2021.
Da-Wei Zhou, Fu- Yun Wang, Han-Jia Ye, and De-Chuan Zhan. Pycil: A python toolbox for class- incremental learning. arXiv preprint arXiv:21 12.3333,, 2021a.
Da-We Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Co-transport for class-incremental learning. ACM MM, pp. 1645-1654, 2021b.
Da- Wei Zhou, Fu- Yun Wang, Han-Jia Ye, Liang Ma, Shiliang Pu, and De-Chuan Zhan. Forward compatible few-shot class-incremental learning. In CVPR, pp. 9046-9056, 2022.
Da-Wei Zhou, Qi- Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Deep inasemnncreentall learning: survey. arXiv preprint arXiv:2302.03644 2023a.
Da-Wei Zhou, Qi- Wang, Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficient class-incremental learning. In ICLR, 2023b.
14

---

Published as a conference paper at ICLR 2023
Appendix of BEEF
CONTENTS
A Proof for Thm. 3.1 and Thm. 3.2
15
B Proof's for Stable-BEEF
17
More Experimental Settings
20
C.1 Exemplar selection.
20
C.2 Implementation details.
21
D More Discussions
21
D.1 Contributions BEEF
D.2 Difference and connections with prior methods
D.3 The robustness of BEEF to the imbalanced exemplar-set
22
D.4 The efficiency of BEEF
22
More experimental results
23
E.1 Detailed performance on standard CIFAR-100 B50 protocols.
23
E.2 Performance of DEEF-Compress with backbones in various sizes.
23
E.3 Performance of prior work with forward compatibility.
23
E,4 Detailed performance on standard ImageNet- 100 protocols.
24
E.5 More results imbalanced protocols.
24
A PROOF FOR THM. 3.1 AND THM. 3.2
Here we provide the detailed proofs for Thm. 3.1 and Thm. 3.2.
Theorem 3.1 (Marginal Distribution Maximum Likelihood Estimation). Defining Ey(x) - -log [a(x)(K + 1] and its corresponding marginal distribution as (x),, the optimiza- tion of (x)e)(()) |- logoo(o)))) Po(x)] is equivalent to that of Ep..a(x) [ log log hh (x) Xpxxp(()) (X) + 1]] when gradient descend is applied, where Ag is the ratio of the nor- malizing constants determined by (x) and Eo(x), and 0 means that parameters of 0 is frozen (i.c.. instances sampled from IP,, (x) are detached).
Proof. Since Pa(x) = Ex expl (x)) we have
(sp(-Ex(x))
Eprea(()) |- log Po(x)] = Ep Ep (x) [Eo(x)] + log
Σ
(13)
x"
15

---

Published as a conference paper at ICLR 2023
Take the gradient of the right part in Eq. 13, and we have
Vo log
Σ exp(-Fo((x))
(sp(-Eo(x))
Vo Fo
Σx (Sp((-o(x'))
Ex= Aa (x)
Vo Ek= ho (x)
Ex exp( - Ex(
Σx= No (x
Vo(1 - ho hx( + 1])
Ex exp( -x((x))
Σ Voho(x) +1]
Ex cxpl- -xx(x') x
(14)
exp(-E%(X
(xp(-E%(x)
Vho(x) +
exp( - (xx(x')))
Σx x exp( (x)())
exp( - (xp((EE((x))
x
cxp( (xE((x))
exp( Exg
Vha(x) K + 1]
exp( (1--0
Ex exp( (5
(x) + 1]
x
Zp
exp(-E/
Ve ho
Zg
Ex' (xp(-=%)(x0
Zi
(x)
Vo log ha (x) K + 1]]
Zo
Vo LE
( [log he (x) 1ll
Zg
where 0 represents that 0 is frozen (i.e., gradients are not computed), -x (xp(-Ex(x')) is the normalizing constant for energy Ey (x), and Ze is the normalizing constant for energy Eg(x). Hence, the objective Ep [- log (x)] is equivalent to
K
Ep Exrea((x))
log
he [x)
+ Ag
|- log hx( + 1]]
(15)
k=
Zzz Zg
where \ø
Theorem 3.2 (Conditional Distribution Maximum Likelihood Estimation). With preliminar- ies from Thm. 3.1, the optimization of Ep (x.) [- log Po(y))) is equivalent to that of (x)))((x))) log (o + Hg |- log he (x) + 1]] when gradient descend is ap- hy [x][ plied, where μa Eh kg(x)[k] o'y))
(v)), y Vn
0,
E V.
Proof. Considering the definition of Po (y/x) in Eq. 2, we have
Ep Eprret(X.n)
log Po(y xx]
=Epret((xN)
Ex(x, -log Σ
exp(- exp(-Eo(xxx'))
hx
(16)
-Epm.e((xw)reee)
-log M
E E log (x) +
K
Ep Eprea(())
log ho [x)
k=0
16

---

Published as a conference paper at ICLR 2023
The gradient of Ex [- log 1 M
with respect to
is equal to that of
(yye |- log he (x) Therefore, define of o'(v) fa(v), components in Eq. 16 can be written in unifying form, that is Ep (x.))yyee log he (x) [x))]] For the last component in Eq. 16, we have
E yi
and the first two
(0,
y 3
(17)
Vo log
Σ ho [R] k=0
V (1 ho (x)) IK + 1I)
k= ho [x][
(x)o(x)) KK +1
E [x)
(18)
ho(x)][ 1]
Voho (x) K + +]
E ho (x)xx)
he (xx [K + +1]
ha(x)[K + 1]
Vo log ha (x) 1]
Σx An- hx()){Ai
K
I + 1]
log ho (x) [ + 1]
Σ ha [A:]
Therefore, the Objective Ee (x. |- log Po x)] is also equivalent to
Ep (x..) [- log he (x) + UgEp (x)) |- log ha (x()) + 1]]
(19)
hg(x)[K+]] where μa ΣK (x())]]]
L
B PROOFS FOR STABLE-BEEF
As we have claimed in Sec. 3 and Sec, 4, we expand the original BEEF to have multiple backward and forward prototypes (dubbed as Stable-BEEF), which stabilizes the training process and improves the performance. Here we give formal illustration of the expandable form and provide proof of it.
First, at the t incremental session, instead of creating one backward prototype ps measure the confidence for all the old tasks, we create backward prototype for each old task. This expansion prompts the new module learn a discriminator for each old task, thereby improving the performance and stability of training especially when there are clear domain shifts among old tasks. Therefore, there are t backward prototypes for measuring the confidence of old tasks. Furthermore, in Stable- BEEF, we create multiple forward prototypes during the training. Concretely, for samples generated through the marginal distribution defined by the energy function, we assign the most-likely pseudo labels to them. We set the number of forward prototypes to a constant number F. After introducing these multiple prototypes, ie.,, P&S and pys, we should redefine he X - K
Given an labut-label pair (x, y) E , & Uf Uf_ V. we define the energy Eo y) as { - log ho (v)),, Eo xx y) { log (he (W)D//DD
y E y.
(20)
i= t-1
where - 3. - K F (t 1) maps labels to their K class class-oorrsspondig prototypes and maps old labels to their
task cark-corrssponding backward prototypes.
Therefore, similar to Eq. 2, the conditional probability density and marginal probability density for Stable-BEie BEEF can be re-formulated as
Po(y)x)
Σ ho
(x)
E 3%
1 Po(x) = Σx
K+1-2 hx(x)
he (x)(())
V,
Ex + hx(xx)) [k) (21)
17

---

Published as a conference paper at ICLR 2023
And then we can induce that the energy function in Po(x) is formulated
K K
Ex(x) = - log
ha(x)[A].
(22)
Like Eq. 4, Stablc-BEEF also estimates the joint distribution Pe(x, y), that is
arg min (x)aa(() [- log Pa(x)] + Ex,.. |- log (o(v]x)]
(23)
And same to the original BEEF, Stable-BEEF also aims to find its gradient equivalent optimization objective to avoid the intractable normalizing constant in the joint distribution defined by the energy (x(x))) v).
Theorem B.1 (Marginal Distribution Maximum Likelihood Estimation for Stable-BEEF). Defining (x) log Ex=t +
+t= (x)) and its corresponding marginal distribution as 12) the optimization of (x))a(()) [- log Po (x)) is equivalent that of Errea(x) [-log log - (x)))
(000 [llog - ho (x) when gradient descend is applied, where >g is the ratio of the normalizing constants determined by (x) and Eo(x), and 0 means that parameters of is frozen (i.e., instances sampled from (x) are detached).
Proof. Since Px(x) = Ex expl (x)) we have
(x)(-Ex(x))
(x) |- log Po [o(x)] Ep (e)
[Eo
+
log Σ (xp(-))())).
(24)
Take the gradient of the right part Eq. 24, and we have
V log
exp( (-Eo(')
exp( Ex))
exp( Ex(()) VoEo(x)
W++-2 ho [x]
V Σ K++- (x)
===
k=0
Ex' exp( En (x'))
Σ K+t-
he [x]
Vo(1 Σ - 2+F (x)(
K+t-
exp( -x((x')
VO K+1-2+F ho [x)
Ex expl- (xx(')
exp( - -E6(''))
exp(-
K+1-2+F
(x
(25)
2K=K++1
x cxp(- -Eo(x'))
exp( exp(-Ex(x'))
exp( (x))))))))
E x exp( Sap(--E'le (E)()))
exp(-E8(x))
Dk=k+t-1 ho (x)
K+1-2+F
x Exp(-Eo
Ex' exp( (xp(-E%(x'))
Σ he (x)
%
exp(- - F% (x))
K+1-2+F
Σ
V Volog
Σ
[x)
Zu
Σx exp(-EX E,
K+111
2%
K++ K+t-2+F 2+
=
Zo
Bxg(())
Vo log
Σ
(x)
k=K++1
Z%
K K-1-2+F
}
-o
(xx(())
log
Σ
ho (x)
Zi
k=K+1-1
18

---

Published as a conference paper at ICLR 2023
where 8 represents that f is frozen (f.e., gradients are not computed), Z Σx exp( Ex)x')) is the normalizing constant for energy Eg (x), and Zp is the normalizing constant for energy Eg Ex)x.
Hence, the objective Er (prear(x)) lolog [x)] is equivalent to
k+t-2
K+1-++F Σ K44-1
Ep (x))
log Σ [k]
Ax
log
ho(x) + +11
(26)
Zz4 Zg
where Ag
Theorem B.2 (Conditional Distribution Maximum Likelihood Estimation for Stable-BEEF), With preliminaries from Thm. B.1, the optimization of (x [- log Po(y)))] equivalent to that of
Eprca((ss)) [- log he (x) [a())]] + 1a
log Σ K K+1-2+F
Lk=K+t-1 ho [x) when gradient descend
-K4e-2tF (x+
is applied, where μ4a Di [x(]][
Proof. Considering the definition of (y(y)) in Eq. 2, we have
log Poly xx]
-Bpree((.v)
Eg(x. y) +og
Σ
exp(-EE(xx,
N'EU!
1-1
log [-
he (x)
(27)
(P
13%
K41-2
E Exeea( (x) log
Σ
he (x)
The gradient of (x [- log ha (x)))) with respect to O is equal to that of Dil
Ep (x |- log ho (x)( (i 1,2, 2...... 1) no matter whatever [}]] is. Therefore,
the first components in Eq. 27 can be written in a unifying form, that is
E (x [- log he (x)((())))
(28)
where For the last component in Eq. 16, have
K++-2
V log
Σ ho (x)
k=0
V (1
K+1-2+F
he (x)
Σ K+ ho (x)
V Σ K=K+1-1 he [x]
K+t-2+F 2+
Σ K (x)
(29)
Am K4+-1 1 ho hx(x)
K++-2++
k= K4t-1 [x(
K++-++F
ΣK* K+-22 [x)
2+F
[x)
Rim K K+t-
K+1-2+2
K++-1 ho (x)
Vp log (x) +
KK--2
ho (x) (k) [k]
K=O
K+1- 2+F
1==+1-1 ho (x)
K+1-2-F
7
llog
Σ
ho (x [k]
K+t-2 he [k:]
k=
k= K+111
Therefore, the Objective Eer (x..)) Po x)] also equivalent to
K K+1-2+F
Epreat(x.g)
Mg
MglEe
-log log
Σ
(x)
(30)
k++-11
19

---

Published as a conference paper at ICLR 2023
- [M]
where My
Σk=0
[x]]]]]
Combining Thm. 26 and Thm. 30, get the final optimization objective
k+t-2
K+ 2+F (x
Ep (x))
-log log
Σ (x(
+ Xx
log
+
k=K+t-1
(31)
K+t-244 2+F
Bp (preat(X.s))
(- log ho (x) + +Mall x
llog
Σ
(x)
k=
Note that (x) 2 0 for all 1,2,...................... K + 2+ and then we get
K+t-2
(x)
-oog
Σ
[x)
$ E (x. (ogg
(32)
km=0
K+1-2+F
E
log
Σ
(x)
E
log
max
ho (x)
(33)
kE 1, 1.... 2+
k=K+t-1
Hence, Eq. 31 upper bounded by
2B Prpat (x.w) (- log ho) + A (x,x))
-
logg
max
ho (x)
+
JE (K
+
(34)
Hg Ep
log
max kE(K+t- K+
haa (x)
Taking Eq. 34 as the optimization objective, we expand the expansion phase in original BEEF into a more stable form with multiple backward and forward prototypes, namely the expansion phase in Stable-BEFF.
Here further discuss how achieve the expanded fusion phase in Stable-BEEF We also assume that we have trained a unifying model he, for all the prior tasks and there is a a, maps labels to output index of he.- Considering that those / - 1 backward prototypes measures the confidence for t prior tasks, similar to Eq. 10, we define
oy (x, {
log (he, (x)( acho(x)(o(()) + VE V., 1,, .- -
log hes (x))(
y Vi
(35)
Then we have
E Vo y E Vn
P P(o, (u|x)
ha(x)(oiy) Σ ha, [m] +ahn(x}[0] - - (x]
(36)
hh, (= (x)
Og and Br are obtained through the minimization of the negative log-likelihood on the exemplar-set.
C MORE EXPERIMENTAL SETTINGS
C.I EXEMPLAR SELECTION.
As we claimed, all the prior methods assume that all the old samples are available when selecting exemplars. Typically, they apply the exemplar selection strategy proposed in Rebuffi et al. (2017). where exemplars are carefully selected by greedily minimizing the derivation the feature center between the selected exemplars and all the old samples. Besides, they assume all the old categories are equal and store the same number of exemplars for each old class. This violates the truth in many application scenarios, where available exemplars for old categories become unavailable at the following sessions. Therefore, the robustness to the imbalance lack of some categories is important for CII. methods. Assuming there are k old classes
usually imbalanced and even some instances
20

---

Published as a conference paper at ICLR 2023
and we want to store exemplars for each old class. We design three protocols for the imbalance of exemplar-set: (1) half-halff: in this protocol, half of l classes are allowed to store more than m exemplars and the other half of & classes are only allowed to store less than exemplars. Specifically, defining the balance factor y E [0. 1), half of classes store (1 + ) exemplars while the other store (1 - 7) exemplars. (2) exp: in this protocol, use negative exponential sequence of length k as the weight for each category. Specifically, the 7th class has the weight exp(->i) and its number
of exemplars is defined by Σソ=ı exp(-1a)
exp(-71)
km (3) random: we uniformly sample km exemplars
from all available old instances. This protocol is the most similar to the original setting since the expectation of the number of exemplars for each category is m statistically.
C.2 IMPLEMENTATION DETAILS.
For ImageNet, we adopt the standard R8 (He al., 2016) as our feature extractor and set the batch size as 256. The learning rate starts from 0.1 and gradually decays at milestones (170 epochs in total). For CIFAR-IO0 100, we use ResNet-32 (He al., 2016) as our feature extractor and set the batch size to 128. The learning rate also starts from 0.1 and gradually decays at milestones (170 epochs in total). For both ImageNet and CIFAR-10O, we use SGD with the momentum of 0.9 and the weight decay of 5e-4 at the expansion phase. At the fusion phase, we use SGD with a momentum of 0.9 and set the weight decay to 0, and train the fused model on the exemplar-set for 60 epochs. We apply the data augmenation following Rebuffi et al. (2017) and Wang et al. (2022a). To generating samples from (x) when learning the energy manifold, we use representations of real instances as the starting point of SGLD (Welling & Tch, 2011) following Contrastive Divergence (Hinton, 2002), which can be seen as a disturbance to the original representation of the real samples. In order to further accelerate the sampling process, strategies, including mixup (Zhang et al., 2018), rotation, etc., on the feature representations of samples. Therefore, we are able generate new samples rather efficiently when learning the energy manifold. To further stabilize the training process and improve the performance, we expand the original BEEF into Stable-BEEF with multiple backward and forward prototypes. The number of backward prototypes is set to be the number of old tasks, and the number of forward prototypes is set to a constant number F. The detailed illustration and proof are provided in Appendix B. Besides, to simplify the training procedure and avoid the intractability of normalizing constants, we use trade-off coefficient X to replace Xg and (á in our implementation.
employed additional representation perturbation
D MORE DISCUSSIONS
D.1 CONTRIBUTIONS OF BEEF
Our contribution is three-fold: 1) We propose novel training paradigm for CIL: We efficiently train specific module for the current task while achieving bi-directional compatibility and then fuse it with the prior model under minimal effort. 2) We provide a theoretical proof showing that the training cost is inherently the modeling of an energy-basec model 3) We achieve ofee-----heertt performance with lower training cost and maintain the performance when only randomly sampled old data is available while other methods fail dramatically.
D.2 DIFFERENCE AND CONNECTIONS WITH PRIOR METHODS
Here we discuss the crucial difference and connections between our method and prior works. Though Wang et al. (2021) has proposed to use an energy-based extra dimension to model the open world uncertainty in OOD detection, it is not suitable for CIL where old categories should be considered to alleviate forgetting. Besides, we take the joint distribution P(x. y) = (x))))(()( into consideration, which forces the model to learn discrimination (P(y)x)) and be sensitive the input distribution shift (P(x)) and thus modules that do not match the input distribution will capture the distribution shift and weaken their influence on the ultimate predictions. Similar to Joseph et al. (2022), we also utilize the energy-based model to achieve energy alignment at the evaluation phase. While our energy-based model is inherently built at the training phase, Joseph et al. (2022) need to train an extra energy aligner after the training phase, introducing additional training costs. Apart from that, despite they assume that the task-ids of given samples are known in their implementation, we do not require that. Zhou et al. (2022) cleverly utilize multiple virtual prototypes to reserve feature space for future
21

---

Published as a conference paper at ICLR 2023
unseen categories, thus achieving forward compatibility. Our method can be seen as bi-directional compatible method, with one prototype Pa measuring the confidence for old categories and the other prototype Py modeling the out-of-distribution probability.
D.3 THE ROBUSTNESS OF BEEF TO THE IMBALANCED EXEMPLAR-SED SET
Here we give an explanation about why BEEF demonstrates strong robustness the imbalance of exemplar-set. Previous methods, whether based on dyyaaiccsttructur or regularization, rely on unifying feature representation and classifier. When the exemplar-se is unbalanced, that is, when the number of samples in some categories is very small or does not exist, the feature representation and classifier of the class are damaged. Although our method still needs to fuse all modules into a unified classifier, each module is decoupled from each other and is responsible for different tasks. Thus, when training new tasks, even if samples of some old classes are not available, the old modules responsible for these classes are not affected. In addition, when training new modules, we classify all old classes into a special backward prototype. This special backward prototype aims to learn the characteristics shared by categories in old tasks, which is insensitive to the imbalance of old samples. In the model fusion phase, the confidence of backward prototype is uniformly added to the output of old modules, thus acting as soft task discriminator without affecting the predictions of old modules on old tasks.
D.4 THE EFFICIENCY OF BEEF
Training Cost Analysis. Previous methods, whether based on knowledge distillation or dynamic structure, require the forward propagation of old modules when learning new tasks, to achieve a unifying classifier. Besides, with the increasing number of modules retained, dynamic-structuu sraeeeuur--beedd methods suffer from increasing training costs. The training of BEEF consists two phases, expansion and fusion. At the expansion phase, we only need to train the newly created module independently without the involvement of prior modules, so that the training cost at the expansion phase is equivalent to tuning single module and will not increase at incremental stages. Though we follow Stochastic Gradient Langevin Dynamics (SGLD) (Welling & Teh, 2011) to generate samples from (x) when learning the energy manifold. However, we have made many strategies to simplify and accelerate the process, including Contrastive Divergence, mixup, rotation, etc. At the fusion phase, we only need to learn two factors including scale factor a and bias factor 3 through training on the randomly selected exemplar-set, and therefore the training cost is minimal compared to the expansion phase.
Particularly, we compare the training cost of BEEF with the methods including the classical dynamic- structure-based method DER (Yan et al., 2021) 1) and regularization-basse based methods like iCaRL (Rebuffi et al., 2017) and WA (Wu et al., 2019). Without loss of generality, we take the 4 incremental stage as example. Although all the old modules are frozen in DER, the training process still requires the forward propagation of all 1 old modules and the forward propagation of the new module update the new module. Apart from that, DER also needs to be
and then applies backpropagation the final classifier with a balanced reserved dataset. Therefore, the training process of DER at tth incremental stage is: l forward propagation, x backpropagation, 1 x finetune classifier. Similarly, due to the requirements of knowledge distillation in regularization-based methods like iCaRL, the training cost is: 2 x forward propagation, 1 x backpropagation. In contrast, BEEF, we first train the new module in a decoupled manner, which only requires: 1 x forward propagation, x backward propagation. Then, the fusion phase only needs to tune two parameters with a small subset, whose cost is minimal and is even more efficient than the process of simply tuning the final classifier in DER. Therefore, the training cost of BEEF consists of: x forward propagation, backward propagation, x finetune and B.
Specifically, we compare the training time of BiC (regularization based), DER (dynamic-structurr based) and BEEF each incremental session CIFAR- 100 BO 10 steps protocol, the results are shown in Table, 4. We can observe that the average training time of BEEF is lower than both the regularization-ban based method BiC and the dynamic-structur--ban
method DER.
Memory usage analysis. Here, we report the peak memory for storing the exemplars and the learnable & frozen network parameters during the model training through all phases BO 5 steps protocol of Benchmark CIFAR- 100 and ImageNet- 100 in Table. 5. For the model parameters, we use the type float32 to save all of them. That is, each parameter in the model takes 4 bytes of memory.
22

---

Published as a conference paper at ICLR 2023
Training time cost in each session (min)
Methods
5
6
7
8
9
Average
BiC
25
42
43
44
42
43
45
43
46
48
42.1
DER
25
37
43
49
53
58
66
74
78
91
57.4
BEEF
30
31
33
40
38
41
43
42
45
46
38.9
Table 4: Training time cost comparisons.
For images with width W and height H, since each image has three channels and each pixel (with value 0-255) takes byte, therefore each image takes 3 W H bytes of memory. Assuming that we have N exemplars and the number of model parameters is M, the memory usage is calculated as 3 W X + 4 M. Note that the memory usage RMM varies with different choice of configs. For CIFAR- 100, the memory usage of it varies between 9.66 MB and 24.2 MB. For Im0geNet-000, the memory usage of it varies between 378 MB to 1949 MB. We report the mean value of them in the above table.
Peak memory usage of different methods (MB)
Benchmarks
Replay
iCaRL
BiC
PodNet
FOSTER
DyTox
DER
RMM
BEEF
CIFAR-100
7.64
7.64
7.64
7.64
7.64
50.62
16.74
16.93
16.74
ImageNet-100
330
330
330
330
330
333
554
1164
554
Table 5: Peak memory usage comparisons.
E
MORE EXPERIMENTAL RESULTS
E.1 DETAILED PERFORMANCE oN STANDARD CIFAR- 100 B50 PROTOCOLS.
We report the detailed accuracy on standard CIFAR-100 B50 protocols with 5, 10, incremental sessions in Fig. 7. We can observe that BEEF
Accuracy (96)
512
Accuracy
RRplyy DUR +3888
Replay ERER
Repla DHR f
PePoiNi BEEF
+BC
++9K" BE11
BBC +
9W
+99
*Wh
Number of classes
Number of classes
Number of c'cllsses
(a) CIFAR-100 B50 5 steps
(h) CIFAR-100 B50 steps
(c) CIFAR-100 B50 25 steps
Figure 7: Performance standard CIFAR-100 B50 protocols.
E.2 PERFORMANCE OF BEEF-COMPRESS WITH BACKBONES IN VARIOUS SIZES.
We report the detailed accuracy of BEEF-Compress with different backbones, including ResNet 20, ResNet 32, ResNet 44, ResNet 56, and ResNet 110 in Table 6. We can see that the training strategy of BEEF is consistently effective backbones of various sizes. The accuracy of each incremental session has upward trend the model becomes larger.
E.3 PERFORMANCE OF PRIOR WORK WITH FORWARD COMPATIBILITY.
Our overall training strategy is difficult integrate with other existing CIL methods, but find that our proposed forward compatibility implementation can be integrated into many methods. We experimented on the baseline eegularization-basel method WA and found that by implementing
23

---

Published as a conference paper at ICLR 2023
Accuracy in each session (%)
Backbones
Average
ResNet 20
77.18
74.15
70.96
63.45
61.28
58.22
67.87
ResNet
80.9
77.73
73.69
68.22
64.61
61.76
71.15
ResNet 44
81.54
77.87
74.10
69.42
65.67
62.06
71.78
ResNet 56
82.44
78.45
75.20
70.05
65.16
62.77
72.34
ResNet 110
83.6
79.72
75.47
70.44
65.89
63.52
73.11
Table 6: Performance of BEEF with backbones various sizes.
forward compatibility, that is, by classifying the perturbed features sampled by the built-in energy model into forward prototypes, we improve the performance by an evident margin. This might indicate that through achieving forward compatibility, our model is more likely learn representation with stronger generalization and transferability. We validate on CIFAR- 100 BO 10 steps protocols with ResNet-18, and here we report the detailed accuracy at each incremental session Table. 7, where WA+ means WA with forward compatibility.
Accuracy in each session (%)
Methods
2
10
Average
WA
89.5
78.85
75.73
71.4
68.22
64.05
62.2
57.62
55.3
53.53
67.64
WA+
90.8
80.65
76.9
71.97
69.22
65.12
63.39
58.74
56.74
55.24
68.87
Table 7: Performance of WA w/o the forward compatibility.
E.4
DETAILED PERFORMANCE ON STANDARD IMAGENET- 100 PROTOCOLS.
We report the detailed accuracy on standard ImageNet- 100 Fig. 8. We can observe that BEEF achieves competitive performance under different ImageNet- protocols.
100
100
90
90
Accuracy (%)
Accuracy (%)
80
0.55
-0.30
70
60
Raplay
60
Replay
HCaRL
+ PodNet
HCaRL
PodNet
50
BIC
BEEF
50
BiC
BEEF
WA
WA
20
40
60
80
100
0
20
40
60
80
100
Number of classes
Number of classes
(a) ImageNet- 100 BO 10 steps
(b) ImageNet-100 B50 steps
Figure 8: Performance on ImageNet-100.
E.5 MORE RESULTS ON IMBALANCED PROTOCOLS.
We report more results imbalanced protocols in Fig. 9. From the figure, we can observe that even under the configuration of more balanced exemplar-set compared to that Fig. 5, other methods still suffer from significant performance degradation, while BEEF achieves much higher average accuracy than the other methods. Specifically, in the case of half-half and 0.5, BEEF outperforms the best method by 4.25%, and in the case of exp and 7 0.9, BEEF outperforms the best method by 9.80%. These results further demonstrate the robustness our method BEEF in terms of exemplar-set selection strategy.
24

---

Published as a conference paper at ICLR 2023
100
Accuracy (%)
Accuracy (%)
80
4.25
9.80
60
60
Replay DER
Keplay +DDE
PodNc
PodNet
BIC
BEEF
+ BIC
BEEF
+A
20
20
40
100
60
100
Number of classes
Number of classes
(a) half-half: 7 0.5
(b) exp: 7 0.9
Figure 9: Performance on imbalanced protocols.
25


Reviews:
[{"review_id": "vtdnOhfOvev", "reviewer": "Reviewer_so2y", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studied the problem in dynamic-structure-based CIL methods, namely that coupled training of modules in different incremental steps leads to additional training costs and spoilage of eventual predictions. The authors proposed a unifying energy-based theory and framework called 3EF to train independent modules in a decoupled manner at each incremental step, and then fused the modules into a unifying classifier. Experimental results suggest that the proposed approach outperforms on the CIFAR100, ImageNet100, and ImageNet1000 benchmarks.", "summary_of_the_review": "The proposed method is somewhat novel and applying the energy-based model to CIL community is not new. On the other hand, I have doubts about the experimental results. My initial evaluation of this paper is “weak reject”.", "strength_and_weaknesses": "Strength:\nThis paper proposed to train a specific module for new samples and then fuse it with the prior model. Based on theoretical inspiration from the energy-based model, the model performance is further improved.\n\nWeaknesses:\n1. This paper proposes a method so-called ‘Efficient’, where training new models do not require the forward propagation of old models for knowledge distillation. However, SGLD in EBM indeed increased the computation. Different modules are required in different steps, resulting in more parameters amount. Some quantitative analysis may be more helpful to understand. (eg. parameters amount)\n2. Comparison with recent SOTA methods including RMM [1], and Foster [2] are missing. I suspect that maybe the paper cannot achieve true state-of-the-art performance. Eg. in Foster CIFAR100 B0 10 steps: 72.90. But this paper achieved 71.94.\n3. Some detailed accuracy plots at each incremental step on magnet benchmarks should be presented in this paper to show the effectiveness of the method.\n4. This paper needs to be highly polished. Eg. section C.4 ‘. Our method can is a bi-directional compatible method,’\n\n\n[1] RMM: Reinforced Memory Management for Class-Incremental Learning\n[2] FOSTER: Feature Boosting and Compression for Class-Incremental Learning", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "nakyRAVn5K", "reviewer": "Reviewer_2BsG", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to learn dynamic-structure-based CIL models in a decoupled manner. It learns independent models for different tasks and then fuses them at a low cost to make phase-wise predictions. The idea is simple and seems effective compared to simple baseline methods such as ICARL and PODNet, but can not achieve the top performance compared to the recent work FOSTER (the citation is missing from this ICLR submission)---which is a more effective version of DER.", "summary_of_the_review": "My main concerns are two-fold. 1) The unfair comparison between this paper and related works of using a single model for CIL. 2) The poor performance compared to an unmentioned sota method FOSTER.", "strength_and_weaknesses": "Strength******\n\n1. The paper is clearly written and easy to follow. Though there are many formulations of entropies, the interpretation of each formula is quite clear and straightforward. It is easy to follow.\n2. Implementation details are well-presented in the paper. There is a compressive analysis of the model's sensitivity to hyperparameters.\n3. The model fusion and alignment modules are interesting and novel.\n\nWeaknesses*******\n\nThe main idea of this paper, \"using independent models for different learning phases\", is often taken as a baseline. See the \"independent\" setting in table 3 of this paper [Gradient Episodic Memory for Continual Learning, NIPS 2017]. It is straightforward and not fair to the models using a single model.\n\nThough this submission designs a novel module of model fusion and alignment based on entropy, its poor performance is still a concern, given it has used so many copies of backbone networks (i.e., a much higher number of network parameters compared to the related methods using a single model). For example, its performances on ImageNet-based datasets (all settings in ImageNet-100 and -1000) are consistently lower than those of FOSTER [1] -- where FOSTER is a more efficient model than DER, in terms of computational speed and parameter quantity.\n\n[1] FOSTER: Feature Boosting and Compression for Class-Incremental Learning. ECCV 2022.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "j6IV27w5DW", "reviewer": "Reviewer_hrbR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a new CIL method, which trains independent modules in a decoupled manner and integrates the modules into a unifying classifier. They provide an energy-based explanation for the proposed method with theoretical analyses. Extensive experiment results on CIFAR-100, ImageNet-100, and ImageNet-1000 are provided. ", "summary_of_the_review": "Overall, this is an interesting paper. However, there is no open-source code. Besides, the analyses of memory usage and training cost are not provided. As these two points are very essential to the final contributions, the overall score is borderline reject.  ", "strength_and_weaknesses": "### Strengths\n\n- Extensive experiment results are provided.\n- This paper is well-organized and easy to follow.\n- The proposed method is technically sound\n\n### Weaknesses\n\n- The memory usage of the model parameters might violate the class-incremental learning benchmark protocol. The authors propose to train independent modules and then integrate them. However, each module requires a memory budget to save them. We can also use the same memory budget to save more exemplars to improve performance. Based on this, the authors should provide ablation results showing saving the independent modules is more efficient compared to saving the exemplars. \n\n- There is no analysis of memory usage. As the independent modules also need to use the memory, it is important to provide the memory usage in Tables 1 and 2.\n\n- The authors claimed that their method aims to address the “training cost” issue in CIL. However, there is no analysis of the training cost. The authors need to provide a comparison of the training cost to show their method performs better than other baselines. \n\n- There is no open-source code. The authors didn’t provide the implementation in the submission. It is important to provide the following researchers with enough materials to reproduce the results. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "X2njUKaGhR", "reviewer": "Reviewer_6aJg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work authors propose a novel dynamic-structure-based method for class incremental learning from the perspective of energy-based models. The proposed Efficient Energy-based Expanasion and Fusion (3EF) framework introduces the concept of bi-directional compatibility and decouples the training of different modules with a expansion and fusion two-stage training strategy. In addition, authors also show less requirement on the selection of  exemplar-set for rehearsal. Extensive experiments on three popular benchmarks show that the proposed method performs favorately against state-of-the-art methods.", "summary_of_the_review": "Overall, I think this work is well prepared and would inspire and advance the development of increamental learning field.", "strength_and_weaknesses": "Pros:\n1. This work is well-motivated by the training efficiency of dynamic-structure-based methods and possible conflict among modules in addressing novel classes.\n2. A novel method from the perspective of energy-based methods is proposed. The proposed energy-based framework consists of two stages, i.e., expansion phase where new modules are trained to satisfy both backward and forward compatibility and fusion phase where a unified classifier for all seen categories is obtained.\n3. The proposed method also improves robustness to the imbalance or lack of some categories in exemplar-set. \n4. This work is well-written and the structure of the paper is well-organized. The symbols, equations and their descriptions are clear and easy to follow.\n5. The experiment setting is clearly explained. Extensive experiments, including both comparison with SOTAs and ablation studies, demonstrate the effectiveness of the proposed method.\n\n\nCons:\n1. In this work only one dynamic-structure-based method is compared with. More strong dynamic-structure-based methods with rehearsal are suggested to compare with.\n2. Performance of applying the proposed training strategy to more existing models.\n3. 'For benchmark ImageNet-100 ...' in Sec. 4.1 are suggested to put in another paragraph for clarity.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "4vys8zDn9y", "reviewer": "Reviewer_zPVs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper the authors propose to use energy-based model to improve the performance of Class-incremental learning. They try to make their model to be bi-directional compatible, i.e. forward compatible, which aims to enable old modules to sensitively capture the input distribution shift and backward compatible, which aims to enable the discrimination ability of old modules unaffected by new ones.Given data from new classes, a new energy-based model is first trained with a designed gradient equivalent objective function to avoid the intractable normalizing constant during the energy-based model expansion phase. And after that the new energy model is fused with the previous models with a designed fusing function. Through experimental results on CIFAR-100 and ImageNet-100, the authors demonstrate that their model work better than the baselines. ", "summary_of_the_review": "In summary, I like the idea proposed in this paper that applies the energy-based model theory to the Class-Incremental Learning task. And the results seems to support their claims, while on the other hand, I also have some concerns that I may want authors to further clarify. ", "strength_and_weaknesses": "Strength:\n1. I think the idea that applies the energy based model (EBM) to class incremental learning (CIL) is interesting;\n2. The designs of the energy expansion objective and energy fusion function is meaningful;\n3. The experimental results seem to support their claim that the new proposed framework works better.\n\nPossible concerns:\n1. In the training objective (Eq,7), the constant $\\mu_{\\theta}$ and $\\lambda_{\\theta}$ are actually still intractable and related to the normalizing constant of EBM. And the way the authors solve this problem is simple treating it as a hyper-parameter to tune. Theoretically this may be incorrect. And in practice, introducing a new hyperparameter may mean that the performance will depends on the setting/tuning of this new hyperparameter. If we look at Figure 6 b, we can see that the setting of $\\lambda$ do influence the performance.\n\n2. Also there are other parameters like $\\alpha$ and $\\beta$ that need to be tuned, too. The authors tune these parameters on a tiny  sub-dataset. I would like to know how sensitive the performance is to these two parameters settings. Also, whether tuning on a tiny sub-dataset is feasible in practice and can give accurate enough results.\n\n3. For the energy-based model alignment part, the authors mention that they use Adam to accumulate gradient. Then I'm wondering whether they still include the noise term $\\omega_i$ in eq 13 or just do gradient desent or ascent. And how would the aligned images looks like comparing to the original images.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster