Out-of-Distribution Detection in Class Incremental Learning

## Abstract

Abstract Class incremental learning (CIL) aims to learn a model that can not only incrementally accommodate new classes, but also maintain the learned knowledge of old classes.
Out-of-distribution (OOD) detection in CIL is to retain this incremental learning ability, while being able to reject unknown samples that are drawn from different distributions of the learned classes.
This capability is crucial to the safety of deploying CIL models in open worlds.
However, despite remarkable advancements in the respective CIL and OOD detection, there lacks a systematic and large-scale benchmark to assess the capability of advanced CIL models in detecting OOD samples.
To fill this gap, in this study we design a comprehensive empirical study to establish such a benchmark, named OpenCIL .
To this end, we propose two principled frameworks for enabling four representative CIL models with 15 diverse OOD detection methods, resulting in 60 baseline models for OOD detection in CIL.
The empirical evaluation is performed on two popular CIL datasets with six commonly-used OOD datasets.
One key observation we find through our comprehensive evaluation is that the CIL models can be severely biased towards the OOD samples and newly added classes when they are exposed to open environments.
Motivated by this, we further propose a new baseline for OOD detection in CIL, namely Bi-directional Energy Regularization ( BER ), which is specially designed to mitigate these two biases in different CIL models by having energy regularization on both old and new classes. Its superior performance is justified in our experiments.
All codes and datasets are open-source at https://github.com/mala-lab/OpenCIL .

## 1 Introduction

Training of deep neural networks (DNNs) heavily rely on large-scale collected data of a fixed set of classes [, ], but data in real-world applications is constantly changing, leading to continuous new classes in training data.
Continual learning (CL) enables DNNs to continuously learn a sequence of tasks, with each task consisting of a set of unique classes.
The samples for the learned/old tasks are assumed to be not accessible in such dynamic environments. Class-incremental learning (CIL) is one type of CL where task identifiers are not known at testing time.
Compared to another type of CL, task-incremental learning, CIL is often considered a more practical setting in real applications [, ].
Thus, we focus on CIL in this study.

Figure: (a)
Refer to caption: x1.png

Many CIL methods have been introduced over the years to overcome Catastrophic Forgetting (CF) of the learned knowledge on old classes, i.e., degraded classification accuracy on old classes due to model updating on new classes [].
These include regularization-based methods [, , ], parameter-isolation-based methods [, , ], and
replay-based methods [, , , ].
They have shown remarkable performance on the in-distribution (ID) classes in incremental tasks, but lack the capability to recognize and reject out-of-distribution (OOD) samples that are unseen during incremental learning [, ] (i.e., no class overlapping between ID and OOD samples).
Such a capability is crucial to the safety of deploying CIL models in open environments in real-world applications such as autonomous driving [] and medical diagnosis [].
On the other hand, numerous OOD detection methods have been proposed, which can be generally categorized into post hoc-based methods [, , ] and fine-tuning-based methods [, , , , , , , , ].
However, these OOD methods are focused on static environments, where the samples of all tasks are accessible during training, making them ineffective in dynamic CIL environments, as shown in Fig. 1(a).

There are benchmarking studies on the respective CIL and OOD detection areas, e.g., OpenOOD [, ] for OOD detection and FACIL [] for CIL.
However, there lacks a systematic and large-scale benchmarking study to assess the synergy of existing state-of-the-art (SotA) CIL models and OOD detection methods.
To bridge this gap, we design a comprehensive performance benchmark for OOD detection in CIL, called OpenCIL, offering a unified evaluation protocol for different CIL models with diverse OOD detectors.
In particular, OpenCIL accommodates four representative CIL models with 15 diverse OOD detection methods, resulting in 60 baseline models in the comparison study.
To achieve this, we introduce two principled frameworks for incorporating post-hoc and fine-tuning-based OOD detection methods into three types of the aforementioned CIL models.
It provides a module-based evaluation pipeline that enables easy examination of not only the OOD detection capability for different CIL models but also the ability of different OOD detectors in the presence of catastrophic forgetting.
In terms of datasets, OpenCIL uses two popular CIL datasets CIFAR100 [] and large-scale ImageNet1K [], together with six commonly-used OOD datasets containing both near OOD datasets [] and far OOD datasets [], thereby building a systematic benchmark from diverse CIL and OOD detection perspectives.

One key observation we find in our experiments is that the CIL models have increasing biases towards OOD samples and newly added classes with the growth of incremental learning steps.
The underlying reasons are two-folds. One main reason is that due to the CF problem, the CIL models often have lower prediction confidence for samples of old ID classes (i.e., classes seen in the old tasks), compared to new ID classes, as illustrated in Fig. 1(b).
This leads to a difficulty in differentiate between old ID class samples and OOD samples.
Furthermore, as illustrated in Fig. 1(c), the DNN-based CIL models typically exhibit over-confident prediction on OOD samples, causing the misclassification of the OOD samples into not only the old ID classes but also the new ID classes.
Increasing the incremental learning steps results in larger classification semantic space and more severe CF, which continually amplifies the biases towards the OOD samples and newly added classes.
To mitigate this issue, we propose a new baseline for OOD detection in CIL, namely Bi-directional Energy Regularization (BER), which jointly optimizes two energy regularization terms to modulate the energy prediction of OOD samples w.r.t. samples of old and new class samples respectively.
This helps effectively reduce these two biases.
In summary, our main contributions are as follows:

- •
Benchmark. We introduce the OpenCIL benchmark, the first benchmark for comprehensively evaluating the capability of CIL models in rejecting diverse OOD samples. In OpenCIL, we design two principled baseline frameworks and an evaluation protocol for evaluating 60 baselines on two popular CIL datasets and six near/far OOD datasets. This enables a systematic assessment of the synergy of SotA CIL and OOD detection methods.
- •
Important observations. We design and analyze a series of experiments in OpenCIL that leads to a number of important observations, offering crucial insights into the design of CIL models for open-world applications.
- •
New baseline. Beyond the 60 baselines that are synthesized from existing CIL and OOD detection methods,
we also introduce a new baseline, namely Bi-directional Energy Regularization (BER), which provides a new framework for mitigating increasing biases towards OOD samples and newly added classes when incorporating fine-tuning OOD detection methods into CIL models.
- •
Code and datasets. The code of all our baselines and the pre-processed datasets under our OpenCIL evaluation protocol are made publicly available at an open-source repository [https://github.com/mala-lab/OpenCIL](https://github.com/mala-lab/OpenCIL), from which users can easily reproduce our results and evaluate their own datasets/models with minimal effort.

## 2 Related Work

Out-of-distribution (OOD) Detection.
The objective of this task is to determine whether a given input sample belongs to the learned classes (in-distribution) or unknown classes (out-of-distribution).
In recent years, OOD detection has been extensively developed, including post hoc-based methods [, , ] and fine-tuning-based methods [, , , , , , , , ].
The post hoc methods focus on devising new OOD scoring functions in the inference stage.
The fine-tuning-based methods focus on separating OOD samples from ID samples by training a strong classifier as OOD detector.
However, all these methods are applied to non-CIL models. There lacks exploration of their capability on CIL models, resulting in poor performance when there is catastrophic forgetting.
Our BER is a fine-tuning method that can be applied to different CIL models for improving their OOD detection performance.

Class Incremental Learning (CIL).
CIL performs the learning procedure in an incremental manner with growing data samples. It focuses on alleviating the catastrophic forgetting problem, in which the CIL models are required to remember the knowledge of the learned classes from old tasks while learning the discriminative information for the newly coming classes.
There are three main lines of work in this area [, ].
Regularization-based methods focus on applying discrepancy (between old and new models) as penalization terms in their objective functions [, , ].
Parameter-isolation-based methods aim to increase the model parameters in each new incremental step to prevent knowledge forgetting caused by parameter overwritten when learning new tasks [, , ].
Replay-based algorithms assume there is a memory budget allowing a handful of old class examples in the memory. These memory examples can be used to re-train/fine-tune the CIL model in each new incremental step [, , , ].
However, all these methods focus on tackling the CIL problem in a closed world, failing to take into account of distinguishing ID data from unknown samples (e.g., OOD data),
Our BER baselines can be applied to different pre-trained CIL models, in which all new class data and old class memory are used for fine-tuned a new OOD detector for the CIL models. It effectively equips the CIL models with significantly improved capability for OOD detection.

## 3 Baselines and Evaluation Protocol

### 3.1 Problem Statement

Our goal is to equip CIL models with a capability of rejecting OOD samples. Under this setting, a model is required to learn a sequence of tasks during training.
At testing time, the model is used to classify samples into old/new classes, or recognize the samples as unknowns at each incremental step, without having access to the task identifiers.
The model is evaluated based on its effectiveness in preventing the CF problem and distinguishing samples of old/new classes from OOD data.

Formally, CIL models are learned from a sequence of $c$ tasks ID data $T=\{T_{1},T_{2},...,T_{c}\}$.
For each $t$-th $(1\leq t\leq c)$ task, we have $T_{t}=(X_{t}^{train},X_{t}^{test},Y_{t})$, where $X_{t}^{train}$ denotes the training ID data, $X_{t}^{test}$ denotes the testing ID data, and $Y_{t}$ denotes the classification semantic space consisting of a set of unique classes, i.e., the label spaces between any two incremental tasks have no class overlapping: when $i\neq j$, $Y_{i}\cap Y_{j}=\emptyset$.
Thus, the set of all seen classes at task $t$ can be denoted as ${Q}_{t}=\cup_{i=1}^{t}Y_{i}$.
CIL assumes that data samples of the old classes are not accessible.
For many CIL methods, such as the replay-based methods, they assume the availability of a memory buffer, in which a memory block is assigned to each task to store a very small set of samples for the task, i.e., $B_{i}\subseteq(X_{i}^{train},Y_{i})$ is a small subset of training ID data sampled from task $T_{i}$. Thus, we have the memory $M_{t}=\cup_{i=1}^{t-1}B_{i}$ that includes replay data from all $t$ tasks. Let $\theta_{t}{(\cdot)}$ be the CIL model at the incremental learning step $t$,
then the memory and the training data of task $t$ form the training ID data: $T_{t}^{train+}=(X_{t}^{train},Y_{t})\cup M_{t}$ for training $\theta_{t}{(\cdot)}$ at the learning step $t$.
For replay-free CIL methods, $\theta_{t}{(\cdot)}$ is trained with $(X_{t}^{train},Y_{t})$ only.
During testing time at task $t$, let $T_{t}^{test}=X_{1}^{test}\cup X_{2}^{test}\cup...\cup X_{t}^{test}$ be the testing ID data from all $t$ tasks, $\theta_{t}{(\cdot)}$ is used to classify samples in $T_{t}^{test}$ into one of the classes in ${Q}_{t}$. This is the setting for the conventional CIL problem.

For OOD detection in CIL, in addition to $T_{t}^{test}$, the CIL model at task $t$, $\theta_{t}{(\cdot)}$, is also presented with an OOD dataset $X_{t}^{ood}$, which is a set of samples of unknown classes drawn from a different distribution as the ID data in $T_{t}$, i.e., the test data is $T_{t}^{test}\cup X_{t}^{ood}$.
Then given $x\in T_{t}^{test}\cup X_{t}^{ood}$, the goal of the CIL model $\theta_{t}{(\cdot)}$ is to either classify $x$ into the correct ID class from old/new tasks, or detect it as OOD data.

Figure: Figure 2: Two principled frameworks used in OpenCIL to incorporate OOD detection methods into the CIL models. Both frameworks are performed on pre-trained CIL models which are kept frozen when incorporating the OOD detection methods, and thus, their CIL performance is not affected.
Refer to caption: x4.png

### 3.2 Baselines for OOD Detection in CIL

#### Adapting State-of-the-art OOD Detectors to CIL.

There are two types of OOD detection methods: post-hoc and fine-tuning-based approaches, which exerts in the inference and training stages respectively.
Post-hoc methods use a statistic based on features or prediction logits derived from the pre-trained model as OOD score, which can be used for different pre-trained classification models. To adapt them to CIL models, we directly apply this kind of method to each incremental step of the CIL models for performing OOD detection, without having any effect on the performance of the CIL models.
Fine-tuning methods require the fine-tuning of part or all layers of the pre-trained CIL classifiers in each incremental step, which can intensify the catastrophic forgetting problem.
To avoid this issue, we freeze the feature extractor in the CIL models and utilize the fine-tuning OOD detection methods to fine-tune an extra classifier for OOD detection while maintaining the incremental learning accuracy.
These two types of methods are illustrated in Fig. 2 and summarized as follows:

(1) Framework for post-hoc-based methods.
A standard CIL model is first applied to learn the stream of ID data in an incremental manner to obtain the CIL model $\theta_{t}{(\cdot)}$, and then a post-hoc OOD scoring method is performed on the features/logits from the well-trained CIL model $\theta_{t}{(\cdot)}$ at each incremental step, as shown in Fig. 2 Left.

(2) Framework for fine-tuning-based methods.
As shown in Fig. 2 Right, the fine-tuning-based methods require the training of a standard CIL model $\theta_{t}{(\cdot)}$ in the first place, which contains a feature extractor $\phi_{t}{(\cdot)}$ and a classifier $h_{t}(\cdot)$, i.e., $\theta_{t}{(\cdot)}$ is a composition of $\phi_{t}{(\cdot)}$ and $h_{t}(\cdot)$.
Then we freeze both $\phi_{t}{(\cdot)}$ and $h_{t}(\cdot)$, and fine-tune an additional classifier $f_{t}(\cdot)$ only on top of $\phi_{t}{(\cdot)}$.
This means that the fine-tuned CIL model contains the same feature extractor $\phi_{t}{(\cdot)}$ and classifier $h_{t}(\cdot)$ as the standard pre-trained CIL model for incremental ID classification, but it also has the additional fine-tuned classifier $f_{t}(\cdot)$.
Lastly, we use a post-hoc OOD scoring function on the classifier $f_{t}(\cdot)$ to calculate the OOD score.
Notably, the fine-tuning of $f_{t}(\cdot)$ is applied to each incremental step using only the training ID data $T_{t}^{train}$ at task $t$. In doing so, we perform these fine-tuning-based OOD detection methods to obtain better OOD scoring in each incremental step, which does not affect any of the learning procedure of the CIL model $\theta_{t}{(\cdot)}$ (i.e., $\phi_{t}{(\cdot)}$ and $h_{t}(\cdot)$).
Therefore, the CIL model $\theta_{t+1}{(\cdot)}$ is trained based on the $t$-th CIL classifier $h_{t}{(\cdot)}$, rather than the fine-tuned classifier $f_{t}{(\cdot)}$.

With these two principled frameworks, different OOD detection methods can be easily incorporated into three different types of CIL models without impairing the incremental learning accuracy at all.

#### The Proposed Baseline: BER.

When establishing our OpenCIL benchmark, we found the increasing biases of the CIL models towards OOD samples and and newly added classes with the growth of incremental steps.
Therefore, we introduce the novel BER approach, a fine-tuning-based OOD detection approach, to tackle these bias issues, i.e., avoiding the classification of the old and new class samples as OOD samples.
BER consists of two components, namely New Task Energy Regularization (NTER) and Old Task Energy Regularization (OTER).
NTER is designed to distinguish OOD data from samples of new task classes, while OTER is designed to distinguish OOD data from samples of old task classes.
Below we introduce each component in detail.

New Task Energy Regularization (NTER).
Due to the overwhelming presence of new class samples, CIL models typically demonstrate a strong biased prediction on the OOD samples towards the new classes, i.e., they have high prediction confidence on classifying OOD samples into the classes in the new task, as illustrated in Fig. 1(c).
To address this issue, NTER synthesizes the pseudo OOD samples that are distributed on the decision boundary of different new classes, and further utilizes them to enlarge the decision boundary margin.
Specifically, NTER first randomly mixes up samples of different new classes as the pseudo-OOD samples. Formally, let $\mathbf{x}_{t}=\{x^{1}_{t},x^{2}_{t},...,x^{b}_{t},\},\mathbf{x}_{t}\in X^{
train}_{t}$ be one training batch of ID data from classes at task $t$, where task $t$ is the new task and $b$ is the batch size, $\mathbf{y}_{t}=\{y^{1}_{t},y^{2}_{t},...,y^{b}_{t},\}$ and $\mathbf{y}_{t}\in Y_{t}$ be the corresponding class label of $\mathbf{x}_{t}$,
then a pseudo-OOD sample $\bar{x}_{t}$ is synthesized as follows:

$$ $\bar{x}_{t}=\beta x^{i}_{t}+(1-\beta)x^{j}_{t},y^{i}_{t}\neq y^{j}_{t},$ (1) $$

where $\beta\in[0,1]$ is sampled from Beta distribution.
Motivated by the success of energy-based methods [], we further utilize these pseudo-OOD samples to regularize the classification of new class samples via the follwoing energy loss function:

$$ $\mathcal{L}_{n}=\mathbb{E}_{x_{t}\sim X^{train}_{t}}[(max(0,p_{in}-E(x_{t})))^ {2}]+\mathbb{E}_{(x_{t}^{i},x_{t}^{j})\sim X^{train}_{t}}[(max(0,E(\bar{x}_{t} )-p_{out}))^{2}],$ (2) $$

where $E(x;f)=-\tau\cdot log(\sum_{j=1}^{Q_{t}}e^{f_{j}(x)/\tau})$,
with $\tau$ be a temperature scaling hyperparameter.
Note that we do not combine all possible pairs of $(x_{t}^{i},x_{t}^{j})$ in the full training dataset $X^{train}_{t}$ to form $\bar{x}_{t}$, which are produced within the mini-batches for efficiency consideration [, ].
As a result, the time complexity is of the same magnitude as vanilla training, having minimal computational overhead.

Old Task Energy Regularization (OTER).
Due to the catastrophic forgetting problem, as illustrated in Fig. 1(b), the CIL models exhibit significantly lower prediction confidence on samples of old classes than new classes.
As a result, the CIL models often misclassify the old class samples as the OOD samples.
To address this issue, OTER performs a different mixup operation from NTER to generate more samples of old classes. In particular, it randomly mixes up new class samples with old class samples to synthesize old class samples.
This mixup operation not only increases the diversity of the small-sized old class samples but also transfers information from new class samples to old class samples. Then using these mixup samples in the fine-tuning stage largely enhances the prediction confidence of the old class samples and expand their decision boundary as well.
Formally, let $M_{t}=\{m^{1}_{t},m^{2}_{t},...,m^{s}_{t}\}$ be the samples of old classes in the memory bank for task $t$, and $s$ is the memory size,
then the synthesized old class samples $\bar{m_{t}}$ are generated via:

$$ $\bar{m_{t}}=\lambda x_{t}+(1-\lambda)m_{t},$ (3) $$

where $\lambda$ is a mixup hyperparameter, similar to $\beta$ in Eq. 1.
We further leverage these augmented ID samples to boost the prediction confidence of old class samples via the following energy loss function:

$$ $\mathcal{L}_{o}=\mathbb{E}_{(x_{t},m_{t})\sim(X^{train}_{t},M_{t})}[(max(0,E( \bar{m}_{t})-p_{in}))^{2}],$ (4) $$

where $E(x;f)=-\tau\cdot log(\sum_{j=1}^{Q_{t}}e^{f_{j}(x)/\tau})$.
Note that we only apply energy regularization to the ID data in OTER, without the energy regularization term on the pseudo-OOD samples in Eq. 2.
The old class samples cannot be used for synthesizing the pseudo-OOD samples via, e.g., mixup between old and new class samples, or mixup between samples of different old classes.
This is because, as the samples stored in the data replay memory, these old task samples are the most representative samples of the old classes, which are typically located near the class center in their respective belonging classes in the feature space.
Therefore, generating pseudo-OOD samples using these old class samples will severely compress the decision boundary of old task classes, resulting in a significant adverse impact on the OOD detection performance.

Lastly, we utilize the cross-entropy loss, together with the two energy regularization losses, to fine-tune the extra classifier $f_{t}(\cdot)$ for in the CIL models following the fine-tuning OOD detection framework.
Thus, the overall optimization objective of our BER approach at each task $t$ is as follows:

$$ $\mathcal{L}=\mathbb{E}_{(x,y)\sim(T^{train}_{t},Q_{t})}[\ell(f(x),y]+\alpha( \mathcal{L}_{n}+\mathcal{L}_{o}),$ (5) $$

where $\ell$ is a cross-entropy loss, $\alpha$ is the hyperparameter, $\mathcal{L}_{n}$ is as defined in Eq. 2, and $\mathcal{L}_{o}$ is as defined in Eq. 4.
During inference, BER utilizes the Energy Score defined in [] as OOD score.

### 3.3 Evaluation Protocol

#### CIL Datasets.

Following [, , , , ], we use two popular CIL datasets as the ID data in our benchmark: 1) CIFAR100 [], which consists of $50$K training images and $10$K testing images with $100$ classes, and 2) large-scale ImageNet1k [], which consists of $1.28$M training images and $50$K testing images with $1,000$ classes. Besides, these two datasets are also widely used as ID datasets in the area of OOD detection.

Following [, , ], the splits of the two ID datasets are as follows.
1) For CIFAR100, we train the CIL model gradually with $k$ classes per incremental step with a fixed memory size of 2,000 exemplars. We respectively evaluate the performance with the step size $k\in\{5,10,20\}$.
2) For ImageNet1K, since it is a much larger dataset, we train the CIL model gradually with a larger number of $k$ classes per step with a fixed memory size of 20,000 exemplars. $k\in\{50,100,200\}$ is used for ImageNet1K.

#### OOD Datasets.

Following the recent large-scale solely OOD detection benchmark OpenOOD [], we select six datasets as OOD data for each ID dataset respectively. 1) For the ID dataset CIFAR100, the OOD data includes two near OOD datasets – CIFAR10 [] and Tiny-ImageNet (TIN) [] – and four far OOD datasets: MNIST [], Texture [], SVHN [] and Places365 [].
2) For ImageNet1k, the OOD data includes four near OOD datasets – ImageNet_O [], iNaturalist [], OpenImage_O [] and Species [] – and two far OOD datasets: MNIST [] and Texture [].
Near OOD datasets means that their OOD samples have small semantic shift compared with the ID samples, while far OOD datasets means that their OOD samples are very different from the ID samples, typically containing obvious covariate (domain) shift [].

It is notable that, although these datasets have been widely used in the CIL and OOD detection community respectively, there is still no readily accessible and unified protocol for combining them into one experimental setting.
Our OpenCIL benchmark offers one way to unify them into a systematic evaluation setting, facilitating the application of diverse OOD detectors under different CIL models.

#### Performance Metrics.

To fairly compare the OOD detection performance among different incremental steps, we keep the ratio of testing OOD data $X_{t}^{ood}$ to testing ID data $T_{t}^{test}$ fixed at each incremental step $t$.
Specifically, we control the test OOD data fed into the CIL model in an incremental manner during the inference stage, increasing the number of testing OOD data $X_{t}^{ood}$ with increasing number of tasks proportionally.
Formally, the number of OOD samples for each OOD dataset at step $t$ is defined as:

$$ $|X_{t}^{ood}|=|X^{ood}|\times\frac{t}{|T|},$ (6) $$

where $|X^{ood}|$ and $|X_{t}^{ood}|$ denote the number of OOD samples in the full OOD dataset and in the testing OOD data used at step $t$ respectively, and $T$ is the total number of CIL tasks/steps.
In this way, the number of testing ID and OOD data grows at the same speed, keeping the ratio of them identical across all incremental steps.

Following standard CIL evaluation [], the measure of average incremental accuracy that reports the average classification accuracy over all incremental steps is used to evaluate the performance of CIL. At the same time, we also report the average results of three widely-used OOD detection performance metrics (AUC, FPR, AP) over all incremental steps.

More details about the datasets used and the performance metrics are provided in Appendix A.

## 4 Results and Analysis

We perform large-scale experiments that include the composition of $15$ OOD detection methods and four CIL models using three different incremental step sizes. Due to space limitation, we report the averaged results across all incremental steps at the step size of $k=10$ for CIFAR 100 and at the step size of $k=100$ for ImangeNet1K here. More fine-grained results at this step size and other two step sizes are presented in Appendix C.

### 4.1 Implementation Details

The 15 OOD detection methods include nine post hoc methods – MSP [], ODIN [], Energy [], MaxLogit [], GEN [], ReAct [], KLM [], Relation [], and NNGuide [] and six fine-tuning-based methods – LogitNorm [], T2FNorm [], AUGMIX[], REGMIX[], VOS[] and NPOS[]. The four CIL models include two regularization-based methods – iCaRL [] and WA [], one replay-based method BiC[], and one parameter-isolation-based method FOSTER[].

To ensure a fair comparison across methods originating from different areas, we use unified settings with common hyperparameters and architecture choices.
For example, the backbone ResNet32 is used when CIFAR100 is used the ID dataset, and ResNet18 is used whenever ImageNet1K is the ID dataset. These two backbones are the most commonly used architecture for the respective ID dataset in the CIL community.
For the CIL pre-training, we employ default hyperparameters of these CIL models as stated in their original papers.
For OOD fine-tuning, following [, ], we use the common setting with SGD optimizer, using a learning rate of $0.1$, a momentum of $0.9$, a weight decay of $0.0005$, and adjusting the learning rate using a cosine annealing learning rate schedule.
The batch size is fixed at $128$ for all experiments.
We freeze the feature extractor, and fine-tune the extra classifier for $10$ epochs.
For our proposed BER baseline, following [] , we set $\tau=1$, $\alpha=0.1$, $p_{in}=-5$ and $p_{out}=-27$ by default. $\lambda=0.002$ is used throughout the experiments.
All results are averaged over three independent runs using different random seeds.
All experiments are performed using $8$ NVIDIA RTX $3090$. More implementation details can be found in Appendix B.

### 4.2 Main Results

**Table 1: Main results on OpenCIL benchmark. We report the average performance over six OOD datasets and all incremental steps. The best, second-best, and third-best performance per dataset are highlighted. Either the post hoc-based or fine-tuning-based OOD methods do not affect the original CIL performance, so their average CIL accuracy on the ID data remains the same.**
|  | ID Dataset: CIFAR100 | ID Dataset: ImageNet1K |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
|  | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 60.08 | 61.68 | 65.88 | 66.01 | 44.44 | 49.63 | 52.22 | 52.29 |  |  |  |  |  |  |  |  |
| Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 67.77 | 88.33 | 66.67 | 86.98 | 70.49 | 86.04 | 71.69 | 84.94 | 63.84 | 89.81 | 67.45 | 91.80 | 65.97 | 89.17 | 67.64 | 89.32 |
| ODIN [] | 70.05 | 81.93 | 69.76 | 80.52 | 71.86 | 80.18 | 74.70 | 75.68 | 68.41 | 87.91 | 71.09 | 87.45 | 67.86 | 89.08 | 70.24 | 88.15 |
| Energy [] | 70.35 | 83.74 | 69.76 | 82.01 | 73.03 | 81.22 | 75.76 | 77.24 | 65.64 | 91.88 | 70.16 | 89.57 | 66.57 | 92.22 | 69.39 | 90.49 |
| MaxLogit [] | 70.34 | 84.45 | 69.54 | 82.78 | 72.98 | 82.33 | 75.71 | 78.11 | 65.83 | 91.38 | 69.93 | 89.42 | 66.75 | 92.03 | 69.56 | 90.02 |
| GEN [] | 70.86 | 84.31 | 69.87 | 82.51 | 73.30 | 81.42 | 76.11 | 77.64 | 66.89 | 90.78 | 69.51 | 89.77 | 68.55 | 88.54 | 69.74 | 89.91 |
| ReAct [] | 68.79 | 84.43 | 70.62 | 82.60 | 74.87 | 81.82 | 76.15 | 78.03 | 55.66 | 94.47 | 66.57 | 89.86 | 63.85 | 90.94 | 65.15 | 91.16 |
| KLM [] | 67.51 | 88.00 | 66.27 | 87.66 | 69.89 | 87.65 | 71.34 | 85.90 | 66.48 | 88.28 | 67.32 | 87.17 | 65.24 | 89.59 | 70.23 | 88.45 |
| Relation [] | 64.40 | 89.57 | 65.48 | 80.85 | 76.12 | 88.37 | 72.17 | 75.29 | 66.38 | 89.92 | 69.10 | 89.63 | 67.51 | 94.59 | 67.93 | 88.99 |
| NNGuide [] | 72.14 | 79.65 | 68.74 | 78.79 | 75.60 | 76.00 | 75.90 | 74.70 | 66.53 | 89.88 | 70.64 | 87.33 | 68.69 | 88.82 | 67.63 | 88.32 |
| Average | 69.13 | 84.93 | 68.52 | 82.74 | 73.13 | 82.78 | 74.39 | 78.61 | 65.07 | 90.48 | 69.09 | 89.11 | 66.78 | 90.55 | 68.61 | 89.42 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 70.44 | 83.78 | 70.64 | 82.46 | 72.04 | 81.33 | 74.74 | 78.28 | 65.15 | 92.51 | 69.54 | 87.26 | 66.30 | 93.16 | 67.84 | 92.39 |
| T2FNorm [] | 70.65 | 84.14 | 71.75 | 82.72 | 71.98 | 82.33 | 74.65 | 78.83 | 66.09 | 91.58 | 69.53 | 89.09 | 67.39 | 91.83 | 68.19 | 90.95 |
| AUGMIX [] | 70.72 | 83.91 | 70.78 | 82.94 | 72.11 | 81.53 | 74.75 | 78.62 | 68.60 | 87.98 | 70.83 | 87.85 | 68.10 | 89.37 | 69.62 | 88.53 |
| REGMIX [] | 71.47 | 84.53 | 71.43 | 82.23 | 72.01 | 82.95 | 75.26 | 79.35 | 66.41 | 90.69 | 69.83 | 89.68 | 67.45 | 90.69 | 69.43 | 90.78 |
| VOS [] | 72.31 | 82.53 | 66.66 | 79.03 | 68.49 | 78.15 | 74.58 | 77.71 | 66.28 | 91.67 | 66.67 | 89.47 | 65.41 | 89.92 | 68.98 | 88.74 |
| NPOS [] | 73.23 | 79.50 | 67.03 | 78.83 | 74.25 | 76.78 | 75.91 | 76.00 | 68.12 | 88.74 | 70.42 | 88.36 | 68.00 | 88.60 | 70.32 | 88.40 |
| BER (Ours) | 74.17 | 78.87 | 71.83 | 77.76 | 76.77 | 75.58 | 76.40 | 74.58 | 68.80 | 87.51 | 71.94 | 86.80 | 69.06 | 87.59 | 70.54 | 87.44 |
| Average | 71.86 | 82.47 | 70.02 | 80.85 | 72.52 | 79.81 | 75.18 | 77.62 | 67.06 | 90.10 | 69.82 | 88.36 | 67.39 | 90.17 | 69.27 | 89.60 |
| Average (All) | 70.33 | 83.85 | 69.18 | 81.92 | 72.86 | 81.48 | 74.74 | 78.18 | 65.94 | 90.31 | 69.41 | 88.78 | 67.04 | 90.38 | 68.90 | 89.50 |

#### CIL Models with Higher CIL accuracy Not Necessarily Have Better OOD Detection Performance.

As shown in Table 1, averaged over all 15 OOD methods, the CIL models with higher CIL accuracy, e.g., WA and FOSTER, often achieve better AUC and FPR performance than the other two CIL models on CIFAR100 (this is generally consistent with the performance at each incremental step, as shown in Fig. 3(a)), but this does not hold on the large ID dataset ImageNet1K.
This may be because better CIL algorithms can keep more essential information about ID data to improve ID classification, which can also prevent OOD data from being misclassified into ID classes; the latter case is mainly due to the poor accuracy for all CIL models.

Figure: (a)
Refer to caption: x5.png

#### CIL Models with Fine-tuning-based OOD Methods often Outperform than that with Post Hoc-based Methods.

This can be observed by looking at the average AUC and FPR results of each CIL model for the post hoc-based and the fine-tuning methods in Table 1. The observation holds for both ID datasets. This demonstrates the advantage of tuning an additional classifier for OOD detection, but it is at the expense of some computational overhead.

#### CIL Models Exhibit Stronger Prediction Confidence on OOD Samples than Old Class Samples.

As shown in Figs. 1(b) and 1(c), the ID samples from old classes/tasks often have lower prediction confidence than different OOD samples, thereby being misclassified as the OOD samples. This phenomenon become more severe with increasing number of incremental steps.
This is because the CIL models tend to be less confident when predicting the samples of old classes due to the catastrophic forgetting problem, making their OOD scores lower than samples from the OOD data.

#### CIL Models often Misclassify OOD Samples into New Classes.

As also shown in Fig. 1(c), the CIL models often yield significantly higher prediction confidence on misclassifying the OOD samples into the new classes than the old classes, due to the presence of large samples from these new classes at each incremental step.
The reason is similar to the case that the CIL models tend to predict the ID samples of the new classes with higher confidence than the old classes.

#### Energy Regularization on old and new classes helps mitigate the biases towards OOD samples and new classes.

This observation is justified by the superiority OOD detection performance of our proposed baseline BER in Table 1, where BER consistently achieves the best performance in the averaged results in all cases on both CIFAR100 and ImageNet1K. More specifically, as shown in
Table 2, either New Task Energy Regularization (NTER) or Old Task Energy Regularization (OTER) helps boost the AUC and reduce the FPR on both near and far OOD data datasets. Since NTER is designed to alleviate the misclassificaiton of OOD samples into new classes while OTER is designed to reduce the misclassification of old class samples as OOD samples, their combination results in a detection model that largely reduces the overall detection errors.

**Table 2: Ablation study of our BER baseline on CIFAR100. Energy [] is used as the baseline that does not use both NTER and OTER.**
| Loss Component | Near OOD Datasets | Far OOD Datasets |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| CIFAR10 [] | TIN [] | MNIST [] | SVHN [] | Texture [] | Places365 [] |  |  |  |  |  |  |  |  |
| NTER | OTER | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ |
| ✗ | ✗ | 70.70 | 84.86 | 72.74 | 83.12 | 86.63 | 62.37 | 62.56 | 96.81 | 56.18 | 92.92 | 73.31 | 82.33 |
| ✓ | ✗ | 71.31 | 84.16 | 73.64 | 82.37 | 89.93 | 52.73 | 65.63 | 95.93 | 59.66 | 91.15 | 75.33 | 79.72 |
| ✗ | ✓ | 70.77 | 83.84 | 73.80 | 82.10 | 88.65 | 55.36 | 64.67 | 95.80 | 59.26 | 91.94 | 74.35 | 81.34 |
| ✓ | ✓ | 71.60 | 83.17 | 75.84 | 81.06 | 91.56 | 46.02 | 67.52 | 93.08 | 62.59 | 90.52 | 75.90 | 79.35 |

#### The Catastrophic Forgetting Problem Applies to OOD Detection Methods too.

This is illustrated in Fig. 1(a), where the performance of different OOD detection methods decreases significantly with increasing incremental steps. This is because, with more incremental steps, more old class samples are wrongly detected as OOD samples, while at the same time, more OOD samples are misclassified into new classes, leading to fast downgraded OOD detection performance.

#### Catastrophic Forgetting is More Persistent in CIL models than OOD Detection Methods.

As shown in Fig. 3(b), the AUC performance for OOD detection drops fast in early incremental steps, and then it slows down and levers off, but the ACC performance for CIL has a continuous, fast decrease throughout all incremental steps.
This is because remembering the forgotten ID samples is more difficult than distinguishing them from OOD samples.
Also, after the CIL models reach a certain level of forgetting for some ID classes, the performance of distinguishing them from OOD data tends to be stable, resulting in relatively stable OOD detection performance at the end of incremental learning.

## 5 Conclusion and Discussion

In this paper, we introduce OpenCIL, the first benchmark for evaluating the capability of OOD detectors in different types of CIL models.
In OpenCIL, we establish a new evaluation protocol to fairly compare diverse OOD detection methods among different incremental steps, and comprehensively evaluate 60 baselines that are composed by 15 OOD detectors and four CIL models. We further propose a new baseline called BER that can effectively mitigate the common issues in the 60 baselines.
Several insights from our large-scale experiments are summarized as follows:
1) OOD detection performance does not necessarily correlate with the incremental CIL accuracy,
2) fine-tuning-based OOD methods often achieve better performance than post hoc-based methods,
3) catastrophic forgetting applies to both CIL classification and OOD detection, but its effect on OOD detection tends to less persistent than on CIL classification,
4) CIL Models are biased towards OOD samples and newly added classes, in which old class samples exhibit high OOD scores and OOD samples exhibit high confidence to be classified into new classes,
and 5) the energy regularization in our proposed BER baseline helps effectively mitigate these biases.

Limitation and Future Work. OpenCIL focuses on assessing the capability of advanced CIL models in detecting OOD samples in the first place.
As a result, the number of the OOD detectors are much more than that of the CIL models.
Therefore, we plan to extend the benchmark with more CIL methods.
We hope this benchmark will push the development of reliable and safe OOD detection methods tailored for CIL.
In addition, it is also important to study the performance of the CIL models in more open environments, where distribution shifts may exist in the ID datasets.

## Appendix A More Evaluation Setting Details

### A.1 Datasets

The in-distribution (ID) datasets for class incremental learning (CIL) include two datasets. 1) CIFAR100 [] contains $50,000$ training images and $10,000$ test images of size $32\times 32$ with $100$ classes, and 2) ImageNet [] is a large-scale classification dataset, which contains $1.28$M training images and $50$K testing images with $1,000$ classes sampled from nature images.

We select six commonly used out-of-distribution (OOD) datasets for OOD detection on CIFRA100. Following the recent large-scale OOD detection benchmark OpenOOD [], our OOD datasets include two near OOD datasets: CIFAR10 [] and Tiny-ImageNet (TIN) [], and four far OOD datasets: MNIST [], Texture [], Places365 [], and SVHN [].
CIFAR10 [] contains $10,000$ images with $10$ classes.
TIN [] contains $7,498$ images, which removes the $2502$ images that have overlapping semantics [] with CIFAR100 classes.
MNIST [] contains $10,000$ images with $10$ classes.
Texture [] contains $5,640$ images with $47$ classes.
Places365 [] contains $33,773$ images, which removes the $2727$ images that have overlapping semantics [] with CIFAR100 classes.
SVHN [] contains $26,032$ images with $10$ class.

We also select six commonly used OOD datasets for OOD detection on ImageNet1k. Following OpenOOD [], these includes four near OOD datasets: ImageNet_O [], iNaturalist [], OpenImage_O [] and Species [], and two far OOD datasets: MNIST [] and Texture [].
ImageNet_O [] contains $2,000$ images from categories not found in the ImageNet1k dataset.
We use a $10,000$ image subset of iNaturalist [], which is based on $110$ mannually selected plant classes not present in ImageNet1k. The OOD samples are randomly sampled images from these $110$ classes []. All images are resized to have a max dimension of $800$ pixels.
OpenImage_O [] contains $15,869$ images with the support of a manual filter.
We use a $10,000$ subset of $713$K images Species [] with $10$ classes.
Two far OOD datasets are the same as CIFAR100.
A summary of the CIL ID and OOD datasets is presented in Table 3.

**Table 3: Key statistics of used CIL ID datasets and OOD datasets.**
| Benchmark | CIFAR100 | ImageNet |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Dataset | Images | Class | Dataset | Images | Class |  |
| ID data (Training) | CIFAR100 | 50,000 | 100 | ImageNet | 1.28M | 1000 |
| ID data (Testing) | CIFAR100 | 10,000 | 100 | ImageNet | 50,000 | 1000 |
| OOD data | CIFAR10 | 10,000 | 10 | ImageNet_O | 2,000 | / |
| TinyImageNet | 7,498 | / | iNaturalist | 10,000 | 110 |  |
| MNIST | 10,000 | 10 | OpenImage_O | 15,869 | / |  |
| Texture | 5,640 | 47 | Species | 10,000 | 10 |  |
| Places365 | 33,773 | / | MNIST | 10,000 | 10 |  |
| SVHN | 26,032 | 10 | Texture | 5,640 | 47 |  |

### A.2 Performance Metrics

This section provides more introduction to the performance measures used in our experiments.
(1) FPR is the false positive rate of OOD examples when the true positive rate of ID examples is at 95%. It measures the portion of falsely recognized OOD when the most of ID samples are recalled.
(2) AUC computes the area under the receiver operating characteristic curve of detecting OOD samples, evaluating the OOD detection performance.
(3) AP measures the area under the precision-recall curve, in which the OOD samples are treated as positive samples.
(4) ACC calculates the classification accuracy of the ID data for the CIL models.
Among all these metrics, only FPR95 is expected to have a lower value for a better model. Higher values indicate better performance for the other three metrics.

### A.3 Codabase

Existing benchmarks are often incorporated into their proposed methods as a data processing procedure, which does not have a module-based framework that can easily alter CIL or OOD detection methods.
Therefore, we have built an extensible modular-based codebase as the basis of OpenCIL, which consists of four core modules in the framework, including Data Processing Module, CIL Pre-train Module, OOD Fine-tune Module (optional), and Evaluation Module.
Users can execute a rearranged script to create a unified YML file for each dataset, CIL algorithm, OOD fine-tuning method, and OOD detectors.
These can provide unified data/model management to guarantee consistency across all incremental inputs, such as data preprocessing, incremental step division, memory data collection, and OOD data growth, facilitating direct comparisons among different combinations of CIL models and OOD detection methods, enhancing the transparency and reproducibility of our performance evaluation.

## Appendix B Baseline Library

We include four different popular CIL models. iCaRL [] and WA [] are the regularization-based algorithms, BiC[] is a replay-based algorithm, and FOSTER[] is a parameter-isolation-based method but it also uses data replay.

For OOD methods, there are two main categories, post hoc-based and fine-tuning-based methods.
For post hoc-based methods, we include $9$ OOD detection methods: MSP [], ODIN [], Energy [], MaxLogit [], and GEN [] use the statistic based on prediction output of each test samples without using any training ID information, while the other four methods:
ReAct [], KLM [], Relation [], and NNGuide [] need the posterior information of training ID samples to obtain the OOD score.
Therefore, if they need a whole label space of training ID data, we feed $T^{train+}_{t}$ to them, which is the combination of memory data and current task training data.
For fine-tuning-based methods, we include $6$ methods: LogitNorm [] and T2FNorm [] are the regularization-based methods that apply the normalization to calibrate the loss function,
AUGMIX[] and REGMIX[] are the augmentation-based methods that apply the data augmentation to ID samples for enhancing the ID data training, while
VOS[] and NPOS[] are synthesis-based methods that generate pseudo-OOD samples to assist the detector training.
Notably, VOS and NPOS focus on outlier synthesis, but applying their original training method to fine-tune the extra classifier is difficult since they are originally design without the final linear classifier. Thus, to have fair comparison, we apply the same energy regularization as our BER for them, which replaces our synthesized pseudo-OOD samples $\bar{x}_{t}$ with their synthesized outlier samples.

## Appendix C Experimental Results for Other Step Sizes and Near/Far-OOD Detection

In this section, we provide more detailed averaged results of the OOD detection performance across all incremental steps in our OpenCIL benchmark.

### C.1 Results for Two Other Incremental Step Sizes

Here we report the averaged results across all incremental steps at the step size of $k=5$ for CIFAR 100 and at the step size of $k=50$ for ImangeNet1K in Table 4, and at the step size of $k=20$ for CIFAR 100 and at the step size of $k=200$ for ImangeNet1K in Table 5. The results for the step size of $k=10$ for CIFAR 100 and the step size of $k=100$ for ImangeNet1K are presented in the main text.

**Table 4: Main results on OpenCIL benchmark. We report the average performance over six OOD datasets and all incremental steps at the step size of $k=5$ for CIFAR 100 and at the step size of $k=50$ for ImangeNet1K. The best, second-best, and third-best performance per dataset are highlighted. Either the post hoc-based or fine-tuning-based OOD methods do not affect the original CIL performance, so their average CIL accuracy on the ID data remains the same.**
|  | ID Dataset: CIFAR100 | ID Dataset: ImageNet1K |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
|  | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 58.20 | 55.87 | 61.44 | 63.51 | 40.86 | 42.26 | 45.99 | 45.96 |  |  |  |  |  |  |  |  |
| Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 66.89 | 87.86 | 68.66 | 87.22 | 69.19 | 86.11 | 70.01 | 85.68 | 61.87 | 91.03 | 63.06 | 92.46 | 61.69 | 92.85 | 65.45 | 90.89 |
| ODIN [] | 70.26 | 79.60 | 70.78 | 78.54 | 71.70 | 80.09 | 72.89 | 75.48 | 66.00 | 88.90 | 67.62 | 90.54 | 63.47 | 91.07 | 69.16 | 87.58 |
| Energy [] | 70.23 | 81.17 | 69.55 | 82.78 | 71.99 | 82.53 | 73.89 | 76.34 | 62.97 | 91.95 | 65.61 | 92.43 | 62.43 | 92.94 | 67.46 | 90.94 |
| MaxLogit [] | 70.16 | 81.87 | 69.87 | 83.31 | 71.89 | 82.96 | 73.82 | 77.34 | 63.94 | 91.58 | 64.67 | 93.32 | 63.51 | 91.44 | 67.31 | 91.30 |
| GEN [] | 70.39 | 82.07 | 70.89 | 79.54 | 72.22 | 82.14 | 74.21 | 76.95 | 55.67 | 93.51 | 60.65 | 92.82 | 61.27 | 93.38 | 59.94 | 94.26 |
| ReAct [] | 70.21 | 81.26 | 69.86 | 84.89 | 73.54 | 82.15 | 74.29 | 76.97 | 63.34 | 89.99 | 62.93 | 89.79 | 63.18 | 89.52 | 66.33 | 87.95 |
| KLM [] | 66.21 | 88.89 | 67.53 | 86.71 | 68.19 | 87.70 | 69.38 | 86.36 | 63.34 | 89.99 | 62.93 | 89.79 | 63.18 | 89.52 | 66.33 | 87.95 |
| Relation [] | 66.33 | 78.06 | 70.64 | 81.42 | 71.89 | 77.19 | 72.49 | 75.74 | 63.13 | 89.86 | 66.39 | 93.63 | 63.11 | 95.41 | 63.48 | 90.83 |
| NNGuide [] | 70.27 | 78.83 | 70.70 | 79.64 | 71.60 | 79.00 | 73.68 | 75.96 | 63.00 | 89.47 | 69.87 | 85.81 | 62.61 | 90.30 | 68.63 | 88.54 |
| Average | 68.99 | 82.18 | 69.83 | 82.67 | 71.36 | 82.21 | 72.74 | 78.54 | 62.58 | 90.70 | 64.86 | 91.18 | 62.72 | 91.83 | 66.01 | 90.03 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 70.21 | 81.18 | 69.22 | 83.39 | 71.13 | 82.65 | 73.31 | 76.78 | 62.02 | 92.73 | 66.30 | 91.20 | 61.58 | 93.94 | 65.87 | 93.26 |
| T2FNorm [] | 70.45 | 81.50 | 69.59 | 83.29 | 70.90 | 83.26 | 73.26 | 77.35 | 62.82 | 92.12 | 65.80 | 92.30 | 62.40 | 92.84 | 66.55 | 92.01 |
| AUGMIX [] | 70.27 | 81.22 | 68.65 | 83.16 | 71.11 | 82.93 | 73.12 | 76.98 | 62.19 | 90.86 | 65.97 | 89.82 | 62.88 | 90.12 | 67.53 | 89.45 |
| REGMIX [] | 70.92 | 81.33 | 68.99 | 83.65 | 71.44 | 84.30 | 73.78 | 77.56 | 63.93 | 91.20 | 66.24 | 90.29 | 62.96 | 91.64 | 67.84 | 91.56 |
| VOS [] | 71.54 | 79.83 | 67.63 | 77.80 | 66.73 | 81.28 | 72.70 | 77.08 | 61.33 | 89.74 | 65.74 | 90.22 | 62.08 | 89.98 | 66.73 | 90.14 |
| NPOS [] | 71.82 | 78.03 | 69.47 | 78.68 | 72.46 | 79.31 | 74.17 | 75.55 | 63.06 | 89.86 | 67.22 | 89.83 | 63.21 | 89.72 | 68.93 | 89.63 |
| BER (Ours) | 72.75 | 77.59 | 71.47 | 77.82 | 72.47 | 78.69 | 74.20 | 74.93 | 63.45 | 89.53 | 67.72 | 88.61 | 64.09 | 89.15 | 69.34 | 88.39 |
| Average | 71.14 | 80.10 | 69.29 | 81.11 | 70.89 | 81.77 | 73.51 | 76.60 | 62.69 | 90.86 | 66.43 | 90.32 | 62.74 | 91.06 | 67.54 | 90.63 |
| Average (All) | 69.93 | 81.27 | 69.59 | 81.99 | 71.15 | 82.02 | 73.08 | 77.69 | 62.63 | 90.77 | 65.55 | 90.80 | 62.73 | 91.49 | 66.68 | 90.29 |

**Table 5: Main results on OpenCIL benchmark. We report the average performance over six OOD datasets and all incremental steps at the step size of $k=20$ for CIFAR 100 and at the step size of $k=200$ for ImangeNet1K. The best, second-best, and third-best performance per dataset are highlighted. Either the post hoc-based or fine-tuning-based OOD methods do not affect the original CIL performance, so their average CIL accuracy on the ID data remains the same.**
|  | ID Dataset: CIFAR100 | ID Dataset: ImageNet1K |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
|  | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ | AUC$\uparrow$ | FPR$\downarrow$ |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 62.65 | 64.14 | 68.05 | 68.75 | 48.85 | 53.30 | 58.42 | 57.84 |  |  |  |  |  |  |  |  |
| Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 68.38 | 87.19 | 69.72 | 87.75 | 72.46 | 84.74 | 72.10 | 85.02 | 66.83 | 87.47 | 71.39 | 86.68 | 70.73 | 87.12 | 71.57 | 84.44 |
| ODIN [] | 71.34 | 81.20 | 71.46 | 80.62 | 74.27 | 78.08 | 74.45 | 76.73 | 71.36 | 83.61 | 75.27 | 82.55 | 71.69 | 84.60 | 73.56 | 82.93 |
| Energy [] | 72.38 | 81.99 | 71.48 | 82.45 | 75.22 | 79.00 | 76.14 | 78.54 | 69.52 | 88.30 | 74.36 | 84.92 | 70.56 | 87.73 | 72.67 | 87.09 |
| MaxLogit [] | 72.32 | 82.20 | 71.60 | 83.73 | 75.22 | 80.02 | 76.08 | 78.84 | 69.62 | 87.90 | 74.17 | 85.99 | 71.08 | 87.26 | 73.15 | 85.57 |
| GEN [] | 72.73 | 82.44 | 71.92 | 83.00 | 75.36 | 79.38 | 76.55 | 78.39 | 70.16 | 87.60 | 74.19 | 85.84 | 71.68 | 85.67 | 73.84 | 84.30 |
| ReAct [] | 71.59 | 82.99 | 73.69 | 82.87 | 76.82 | 78.49 | 77.33 | 78.85 | 59.48 | 92.57 | 69.67 | 85.93 | 63.12 | 88.28 | 64.82 | 89.13 |
| KLM [] | 69.10 | 85.53 | 70.19 | 86.76 | 72.47 | 86.06 | 72.47 | 84.39 | 69.51 | 84.99 | 70.55 | 85.72 | 69.61 | 87.49 | 73.55 | 82.10 |
| Relation [] | 65.37 | 80.08 | 69.10 | 79.46 | 76.23 | 77.46 | 71.27 | 76.53 | 71.24 | 87.09 | 75.12 | 83.34 | 71.45 | 92.75 | 73.27 | 83.31 |
| NNGuide [] | 74.14 | 77.05 | 71.24 | 78.89 | 76.28 | 73.78 | 76.20 | 76.13 | 71.41 | 88.03 | 77.95 | 86.53 | 71.90 | 86.90 | 73.27 | 82.74 |
| Average | 70.82 | 82.30 | 71.16 | 82.84 | 74.93 | 79.67 | 74.73 | 79.27 | 68.79 | 87.51 | 73.63 | 85.28 | 70.20 | 87.53 | 72.19 | 84.62 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 72.41 | 82.32 | 71.27 | 83.80 | 74.29 | 79.49 | 75.38 | 79.28 | 69.40 | 89.17 | 73.18 | 84.05 | 69.14 | 89.57 | 71.62 | 89.30 |
| T2FNorm [] | 72.67 | 82.39 | 71.51 | 83.79 | 74.38 | 80.66 | 75.70 | 79.60 | 70.45 | 87.47 | 74.21 | 82.17 | 70.95 | 87.36 | 71.64 | 87.72 |
| AUGMIX [] | 72.50 | 82.62 | 70.70 | 84.33 | 74.35 | 79.97 | 75.54 | 79.61 | 71.32 | 87.91 | 76.72 | 84.63 | 70.84 | 87.20 | 72.81 | 83.66 |
| REGMIX [] | 72.74 | 82.81 | 71.62 | 84.65 | 74.78 | 80.45 | 75.69 | 80.09 | 69.56 | 89.61 | 72.78 | 87.57 | 70.68 | 88.13 | 71.84 | 88.85 |
| VOS [] | 72.93 | 81.39 | 68.77 | 78.28 | 72.21 | 76.57 | 75.07 | 78.52 | 62.94 | 89.63 | 64.19 | 88.20 | 62.86 | 89.02 | 65.61 | 88.09 |
| NPOS [] | 74.22 | 78.89 | 70.86 | 78.43 | 76.61 | 71.86 | 76.44 | 76.92 | 71.00 | 87.21 | 75.59 | 84.11 | 71.80 | 87.86 | 73.57 | 83.13 |
| BER (Ours) | 74.55 | 76.75 | 71.90 | 77.97 | 76.64 | 71.72 | 76.52 | 75.04 | 71.88 | 87.00 | 76.46 | 83.01 | 71.98 | 86.24 | 73.95 | 82.66 |
| Average | 73.15 | 81.02 | 70.95 | 81.61 | 74.75 | 77.25 | 75.76 | 78.44 | 69.51 | 88.29 | 73.30 | 84.82 | 69.75 | 87.91 | 71.58 | 86.20 |
| Average (All) | 71.84 | 81.74 | 71.06 | 82.30 | 74.85 | 78.61 | 75.18 | 78.91 | 69.10 | 87.85 | 73.49 | 85.08 | 70.00 | 87.70 | 71.92 | 85.31 |

### C.2 Fine-grained Results on Near- and Far-OOD Detection Datasets

Following the experiment results in our paper, we provide more fine-grained results (on near-OOD datasets and far-OOD datasets, respectively) at different step sizes in Tables
6, 7, and 8, in which we also add the AP results of the OOD detection performance.

**Table 6: Detailed results for those in Table 4 on near- and far-OOD detection datasets at the step size of $k=5$ for CIFAR 100 and at the step size of $k=50$ for ImangeNet1K. The best, second-best, and third-best performance per dataset are highlighted. The upper, middle, lower parts of the table are for AUC, AP, FPR performance, respectively.**
| AUC$\uparrow$ | ID Dataset: CIFAR100 | ID Dataset: ImageNet1K |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 58.20 | 55.87 | 61.44 | 63.51 | 40.86 | 42.26 | 45.99 | 45.96 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 67.21 | 66.73 | 67.69 | 69.15 | 70.22 | 68.68 | 70.14 | 69.94 | 61.82 | 61.98 | 65.12 | 58.92 | 62.50 | 60.07 | 64.12 | 68.12 |
| ODIN [] | 70.97 | 69.91 | 69.22 | 71.56 | 72.98 | 71.06 | 72.25 | 73.21 | 64.80 | 68.40 | 68.11 | 66.65 | 63.29 | 63.83 | 66.19 | 75.10 |
| Energy [] | 71.23 | 69.73 | 68.56 | 70.05 | 73.19 | 71.38 | 72.98 | 74.34 | 62.62 | 63.07 | 67.58 | 63.37 | 62.00 | 63.50 | 64.59 | 73.29 |
| MaxLogit [] | 71.16 | 69.66 | 68.85 | 70.38 | 73.16 | 71.26 | 72.98 | 74.25 | 62.87 | 63.17 | 67.11 | 62.62 | 62.65 | 62.01 | 65.02 | 72.34 |
| GEN [] | 71.18 | 69.99 | 69.44 | 71.61 | 73.30 | 71.68 | 73.34 | 74.65 | 63.55 | 64.72 | 65.69 | 62.61 | 64.69 | 61.15 | 65.23 | 71.47 |
| ReAct [] | 68.67 | 70.98 | 66.29 | 71.65 | 72.30 | 74.16 | 70.81 | 76.03 | 57.92 | 51.17 | 63.42 | 55.09 | 62.11 | 59.59 | 59.26 | 61.31 |
| KLM [] | 66.78 | 65.92 | 66.44 | 68.09 | 68.55 | 68.01 | 69.47 | 69.35 | 63.29 | 63.46 | 63.62 | 61.55 | 63.40 | 62.74 | 65.93 | 67.15 |
| Relation [] | 61.30 | 68.85 | 65.38 | 73.27 | 67.53 | 74.07 | 68.42 | 74.52 | 58.73 | 69.95 | 66.19 | 66.80 | 63.22 | 62.87 | 61.05 | 68.37 |
| NNGuide [] | 67.33 | 71.73 | 66.81 | 72.65 | 68.53 | 73.13 | 68.97 | 76.03 | 61.20 | 66.61 | 70.63 | 68.36 | 58.74 | 70.33 | 63.94 | 78.02 |
| Average | 68.43 | 69.28 | 67.63 | 70.93 | 71.08 | 71.49 | 71.04 | 73.59 | 61.87 | 63.61 | 66.39 | 62.89 | 62.51 | 62.90 | 63.93 | 70.57 |
| -Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 71.22 | 69.70 | 68.28 | 69.70 | 72.11 | 70.64 | 72.59 | 73.66 | 61.49 | 63.09 | 66.52 | 65.86 | 61.25 | 62.25 | 63.53 | 70.53 |
| T2FNorm [] | 71.37 | 69.99 | 68.03 | 70.37 | 71.45 | 70.62 | 72.50 | 73.64 | 62.65 | 63.16 | 66.47 | 64.45 | 62.00 | 63.21 | 64.16 | 71.34 |
| AUGMIX [] | 71.47 | 69.67 | 68.12 | 68.92 | 72.15 | 70.58 | 72.74 | 73.32 | 62.79 | 60.99 | 67.43 | 63.05 | 61.96 | 64.72 | 65.97 | 71.09 |
| REGMIX [] | 71.99 | 70.38 | 68.08 | 69.44 | 72.94 | 70.69 | 73.56 | 73.90 | 66.05 | 59.70 | 68.13 | 62.45 | 64.26 | 60.36 | 67.13 | 69.27 |
| VOS [] | 71.96 | 71.33 | 61.91 | 70.48 | 62.23 | 68.98 | 72.18 | 72.96 | 62.17 | 59.65 | 67.03 | 63.16 | 61.48 | 63.28 | 65.24 | 69.71 |
| NPOS [] | 71.54 | 71.96 | 62.03 | 73.20 | 70.37 | 73.50 | 73.40 | 74.56 | 62.22 | 64.74 | 68.45 | 64.76 | 63.56 | 62.51 | 66.97 | 72.85 |
| BER (Ours) | 72.54 | 72.85 | 69.87 | 72.28 | 68.21 | 74.60 | 73.60 | 74.49 | 64.04 | 62.29 | 68.96 | 65.25 | 64.00 | 64.30 | 67.16 | 73.70 |
| Average | 71.73 | 70.84 | 66.62 | 70.63 | 69.92 | 71.37 | 72.94 | 73.79 | 63.06 | 61.95 | 67.57 | 64.14 | 62.64 | 62.95 | 65.74 | 71.21 |
| Average (All) | 69.87 | 69.96 | 67.19 | 70.80 | 70.58 | 71.44 | 71.87 | 73.68 | 62.39 | 62.88 | 66.90 | 63.43 | 62.57 | 62.92 | 64.72 | 70.85 |
| AP$\uparrow$ | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 58.20 | 55.87 | 61.44 | 63.51 | 40.86 | 42.26 | 45.99 | 45.96 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 62.73 | 77.05 | 62.57 | 76.91 | 65.46 | 77.77 | 65.08 | 78.70 | 22.90 | 20.01 | 24.27 | 16.79 | 22.30 | 18.64 | 23.44 | 23.64 |
| ODIN [] | 66.72 | 76.69 | 64.51 | 77.01 | 68.34 | 77.22 | 67.73 | 78.80 | 24.11 | 26.47 | 26.44 | 20.77 | 22.84 | 21.73 | 24.24 | 31.43 |
| Energy [] | 66.91 | 76.88 | 63.27 | 76.01 | 68.42 | 77.48 | 68.44 | 79.37 | 22.40 | 21.21 | 25.73 | 17.93 | 21.63 | 19.61 | 22.91 | 27.15 |
| MaxLogit [] | 66.79 | 76.94 | 63.73 | 76.48 | 68.34 | 77.69 | 68.38 | 79.46 | 22.70 | 21.16 | 25.32 | 17.72 | 22.10 | 19.25 | 23.25 | 26.18 |
| GEN [] | 66.82 | 77.17 | 64.69 | 77.14 | 68.47 | 77.76 | 68.63 | 79.70 | 23.10 | 21.71 | 23.77 | 18.57 | 24.54 | 18.68 | 23.54 | 24.96 |
| ReAct [] | 65.35 | 78.84 | 61.33 | 77.20 | 67.66 | 79.19 | 66.75 | 80.90 | 21.14 | 15.51 | 23.59 | 15.14 | 23.24 | 16.84 | 21.55 | 17.84 |
| KLM [] | 62.17 | 76.80 | 62.02 | 78.65 | 62.92 | 78.44 | 64.09 | 79.07 | 24.13 | 20.25 | 23.96 | 19.58 | 23.91 | 21.54 | 26.15 | 23.51 |
| Relation [] | 61.13 | 78.48 | 61.63 | 80.21 | 64.64 | 82.66 | 65.50 | 84.27 | 25.75 | 20.03 | 26.93 | 21.43 | 22.16 | 19.27 | 22.46 | 25.88 |
| NNGuide [] | 66.45 | 80.63 | 62.47 | 78.56 | 67.86 | 81.45 | 66.92 | 78.62 | 24.47 | 18.74 | 21.35 | 23.34 | 23.98 | 25.34 | 25.62 | 30.94 |
| Average | 65.01 | 77.72 | 62.91 | 77.57 | 66.90 | 78.85 | 66.84 | 79.88 | 23.41 | 20.57 | 24.60 | 19.03 | 22.97 | 20.10 | 23.68 | 25.73 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 66.92 | 76.86 | 63.39 | 76.09 | 67.54 | 77.16 | 68.07 | 79.06 | 21.68 | 20.97 | 25.67 | 19.59 | 21.05 | 19.16 | 22.21 | 23.25 |
| T2FNorm [] | 67.03 | 77.10 | 63.24 | 76.85 | 66.88 | 77.44 | 67.93 | 79.19 | 22.44 | 21.16 | 24.99 | 18.61 | 21.71 | 19.77 | 22.82 | 24.41 |
| AUGMIX [] | 67.12 | 76.84 | 63.17 | 75.63 | 67.52 | 77.13 | 68.22 | 78.88 | 24.63 | 19.31 | 23.96 | 20.14 | 23.79 | 18.54 | 24.83 | 22.17 |
| REGMIX [] | 67.65 | 77.26 | 63.17 | 75.92 | 68.31 | 77.23 | 69.22 | 79.32 | 25.98 | 17.61 | 27.59 | 17.62 | 24.36 | 17.61 | 25.92 | 20.54 |
| VOS [] | 67.50 | 77.56 | 58.66 | 77.45 | 59.23 | 77.19 | 67.59 | 78.69 | 23.73 | 19.60 | 26.89 | 20.23 | 23.48 | 19.63 | 24.86 | 24.53 |
| NPOS [] | 67.78 | 78.56 | 59.06 | 79.51 | 66.09 | 79.16 | 68.62 | 79.60 | 24.96 | 22.63 | 26.89 | 20.78 | 24.64 | 21.03 | 24.02 | 25.70 |
| BER (Ours) | 67.81 | 78.63 | 64.43 | 79.57 | 69.56 | 81.00 | 68.89 | 79.69 | 26.86 | 22.65 | 27.04 | 22.05 | 26.54 | 22.23 | 27.46 | 24.02 |
| Average | 67.40 | 77.54 | 62.16 | 77.29 | 66.45 | 78.04 | 68.36 | 79.20 | 24.33 | 20.56 | 26.15 | 19.86 | 23.65 | 19.71 | 24.59 | 23.52 |
| Average (All) | 66.05 | 77.64 | 62.58 | 77.45 | 66.70 | 78.50 | 67.50 | 79.58 | 23.81 | 20.56 | 25.27 | 19.39 | 23.27 | 19.93 | 24.08 | 24.76 |
| FPR$\downarrow$ | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 58.20 | 55.87 | 61.44 | 63.51 | 40.86 | 42.26 | 45.99 | 45.96 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 87.34 | 88.12 | 88.23 | 86.72 | 85.16 | 86.59 | 86.10 | 85.47 | 91.12 | 90.83 | 91.25 | 94.89 | 92.70 | 93.14 | 91.83 | 89.02 |
| ODIN [] | 83.67 | 77.57 | 85.70 | 74.96 | 82.91 | 78.69 | 83.14 | 71.65 | 90.89 | 84.91 | 89.34 | 92.93 | 91.66 | 89.91 | 90.92 | 80.90 |
| Energy [] | 83.78 | 79.86 | 87.19 | 80.57 | 83.40 | 82.10 | 82.72 | 73.14 | 92.65 | 90.98 | 90.53 | 95.36 | 92.95 | 93.25 | 92.54 | 87.43 |
| MaxLogit [] | 83.91 | 80.85 | 86.80 | 81.57 | 83.07 | 82.90 | 82.94 | 74.54 | 92.46 | 90.94 | 91.00 | 95.30 | 92.92 | 92.97 | 92.41 | 88.00 |
| GEN [] | 83.84 | 81.19 | 85.88 | 76.37 | 82.77 | 81.82 | 82.56 | 74.14 | 92.12 | 90.50 | 92.30 | 95.37 | 90.40 | 93.52 | 92.00 | 89.90 |
| ReAct [] | 83.83 | 79.97 | 88.95 | 82.85 | 83.64 | 81.41 | 83.63 | 73.64 | 92.98 | 94.57 | 91.42 | 95.62 | 92.18 | 95.77 | 93.90 | 94.98 |
| KLM [] | 87.94 | 89.37 | 88.13 | 86.00 | 87.78 | 87.66 | 87.06 | 86.00 | 89.88 | 90.20 | 89.77 | 89.84 | 90.59 | 87.38 | 88.30 | 87.25 |
| Relation [] | 85.39 | 74.39 | 88.48 | 77.88 | 84.44 | 73.56 | 84.10 | 71.57 | 91.78 | 86.00 | 92.85 | 95.19 | 95.52 | 95.19 | 92.56 | 87.38 |
| NNGuide [] | 85.66 | 75.41 | 87.97 | 75.47 | 86.10 | 75.44 | 87.47 | 70.20 | 90.00 | 88.40 | 84.14 | 89.16 | 90.30 | 90.30 | 89.94 | 85.73 |
| Average | 85.04 | 80.75 | 87.48 | 80.27 | 84.36 | 81.13 | 84.41 | 75.59 | 91.54 | 89.70 | 90.29 | 93.74 | 92.14 | 92.38 | 91.60 | 87.84 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 83.77 | 79.88 | 86.75 | 81.70 | 83.71 | 82.11 | 83.01 | 73.66 | 93.23 | 91.73 | 89.61 | 94.38 | 93.92 | 93.98 | 93.39 | 93.03 |
| T2FNorm [] | 83.69 | 80.39 | 86.75 | 81.55 | 83.97 | 82.91 | 82.95 | 74.56 | 92.62 | 91.12 | 90.77 | 95.36 | 92.97 | 92.58 | 92.67 | 90.70 |
| AUGMIX [] | 83.69 | 79.98 | 86.92 | 81.28 | 83.71 | 82.55 | 82.53 | 74.20 | 90.16 | 92.26 | 89.24 | 90.98 | 90.73 | 88.90 | 90.02 | 88.31 |
| REGMIX [] | 83.47 | 80.26 | 87.06 | 81.94 | 83.31 | 84.80 | 81.93 | 75.38 | 89.56 | 94.48 | 88.16 | 94.54 | 90.30 | 94.34 | 89.97 | 94.73 |
| VOS [] | 83.67 | 77.91 | 89.30 | 72.05 | 88.44 | 77.70 | 83.55 | 73.84 | 89.43 | 90.36 | 89.65 | 91.36 | 90.74 | 88.46 | 89.66 | 91.10 |
| NPOS [] | 84.72 | 74.68 | 88.84 | 73.61 | 83.47 | 77.22 | 84.62 | 71.00 | 88.68 | 92.22 | 89.04 | 91.41 | 89.86 | 89.44 | 88.92 | 91.05 |
| BER (Ours) | 82.78 | 75.00 | 86.44 | 73.51 | 83.02 | 76.52 | 82.81 | 70.98 | 88.42 | 91.78 | 87.48 | 90.87 | 88.28 | 90.88 | 87.29 | 90.58 |
| Average | 83.68 | 78.30 | 87.44 | 77.95 | 84.23 | 80.54 | 83.06 | 73.37 | 90.30 | 91.99 | 89.14 | 92.70 | 90.97 | 91.23 | 90.27 | 91.36 |
| Average (All) | 84.45 | 79.68 | 87.46 | 79.25 | 84.31 | 80.87 | 83.82 | 74.62 | 91.00 | 90.70 | 89.78 | 93.28 | 91.63 | 91.88 | 91.02 | 89.38 |

**Table 7: Fine-grained results on near- and far-OOD detection datasets at the step size of $k=10$ for CIFAR 100 and at the step size of $k=100$ for ImangeNet1K. The best, second-best, and third-best performance per dataset are highlighted. The upper, middle, lower parts of the table are for AUC, AP, FPR performance, respectively.**
| AUC$\uparrow$ | ID Dataset: CIFAR100 | ID Dataset: ImageNet1K |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| -Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 60.08 | 61.68 | 65.88 | 66.01 | 44.44 | 49.63 | 52.22 | 52.29 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 67.99 | 67.66 | 71.12 | 64.45 | 71.82 | 69.83 | 71.24 | 71.92 | 63.57 | 64.36 | 67.76 | 66.81 | 66.64 | 64.66 | 66.59 | 69.75 |
| ODIN [] | 71.10 | 69.53 | 73.95 | 67.67 | 74.18 | 70.69 | 73.37 | 75.37 | 67.06 | 71.13 | 70.54 | 72.19 | 67.62 | 68.34 | 67.61 | 75.49 |
| Energy [] | 71.72 | 69.67 | 74.01 | 67.63 | 74.62 | 72.23 | 74.12 | 76.54 | 64.87 | 67.17 | 69.35 | 71.79 | 65.96 | 67.78 | 66.80 | 74.59 |
| MaxLogit [] | 71.65 | 69.69 | 74.04 | 67.29 | 74.62 | 72.16 | 74.23 | 76.45 | 65.13 | 67.22 | 69.50 | 70.81 | 66.80 | 66.64 | 67.36 | 73.94 |
| GEN [] | 71.82 | 70.39 | 74.12 | 67.75 | 74.73 | 72.58 | 74.51 | 76.92 | 65.95 | 68.78 | 69.43 | 69.66 | 69.07 | 67.52 | 67.92 | 73.40 |
| ReAct [] | 67.63 | 69.38 | 73.24 | 69.31 | 73.54 | 75.54 | 75.16 | 76.64 | 58.88 | 49.21 | 66.53 | 66.63 | 65.11 | 61.31 | 65.10 | 65.25 |
| KLM [] | 67.82 | 67.36 | 69.91 | 64.45 | 70.46 | 69.61 | 70.98 | 71.52 | 65.77 | 67.91 | 68.62 | 64.72 | 65.99 | 63.72 | 69.18 | 72.31 |
| Relation [] | 58.11 | 67.55 | 67.34 | 64.55 | 72.13 | 78.12 | 67.40 | 74.55 | 63.33 | 72.47 | 68.51 | 70.30 | 67.61 | 67.31 | 65.67 | 72.42 |
| NNGuide [] | 68.34 | 74.04 | 67.37 | 69.43 | 72.53 | 77.13 | 74.45 | 76.63 | 64.88 | 69.85 | 69.43 | 73.05 | 66.06 | 73.97 | 63.58 | 75.72 |
| Average | 68.46 | 69.47 | 71.68 | 66.95 | 73.18 | 73.10 | 72.83 | 75.17 | 64.38 | 66.46 | 68.85 | 69.55 | 66.76 | 66.81 | 66.65 | 72.54 |
| -Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 71.88 | 69.72 | 73.52 | 69.20 | 73.56 | 71.28 | 73.86 | 75.18 | 64.30 | 66.85 | 68.25 | 72.13 | 65.31 | 68.28 | 66.36 | 70.78 |
| T2FNorm [] | 71.90 | 70.03 | 72.63 | 71.30 | 72.93 | 71.50 | 73.75 | 75.09 | 65.19 | 67.91 | 68.18 | 72.23 | 66.08 | 70.00 | 66.10 | 72.36 |
| AUGMIX [] | 72.50 | 69.82 | 73.19 | 69.58 | 73.63 | 71.35 | 74.33 | 74.96 | 64.94 | 75.91 | 67.75 | 76.97 | 65.71 | 72.87 | 67.23 | 69.06 |
| REGMIX [] | 73.12 | 70.64 | 72.97 | 69.91 | 73.81 | 71.11 | 74.74 | 75.51 | 67.86 | 63.51 | 69.41 | 70.68 | 67.70 | 66.97 | 68.96 | 70.36 |
| VOS [] | 73.41 | 71.76 | 67.55 | 66.23 | 65.31 | 70.08 | 73.80 | 74.97 | 63.69 | 71.47 | 64.94 | 70.13 | 64.14 | 67.94 | 67.19 | 72.57 |
| NPOS [] | 72.74 | 73.48 | 66.17 | 67.46 | 74.00 | 74.37 | 74.28 | 76.73 | 66.79 | 70.78 | 69.12 | 73.02 | 67.19 | 69.62 | 68.07 | 74.81 |
| BER (Ours) | 73.72 | 74.39 | 75.08 | 70.20 | 75.43 | 77.44 | 75.20 | 77.00 | 67.40 | 71.60 | 70.94 | 73.94 | 68.51 | 70.16 | 68.10 | 75.41 |
| Average | 72.75 | 71.41 | 71.59 | 69.13 | 72.67 | 72.45 | 74.28 | 75.63 | 65.74 | 69.72 | 68.37 | 72.73 | 66.38 | 69.41 | 67.43 | 72.19 |
| Average (All) | 70.34 | 70.32 | 71.64 | 67.90 | 72.96 | 72.81 | 73.46 | 75.37 | 64.98 | 67.88 | 68.64 | 70.94 | 66.59 | 67.94 | 66.99 | 72.39 |
| AP$\uparrow$ | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| -Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 60.08 | 61.68 | 65.88 | 66.01 | 44.44 | 49.63 | 52.22 | 52.29 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 62.92 | 77.47 | 66.42 | 75.73 | 66.93 | 78.47 | 66.22 | 79.97 | 24.27 | 21.29 | 26.87 | 22.73 | 25.14 | 19.96 | 25.38 | 25.00 |
| ODIN [] | 66.69 | 76.57 | 69.50 | 75.62 | 69.80 | 77.20 | 68.69 | 80.05 | 26.30 | 27.44 | 28.84 | 26.58 | 26.27 | 23.63 | 26.08 | 30.71 |
| Energy [] | 67.16 | 76.79 | 69.45 | 75.66 | 70.12 | 78.12 | 69.40 | 80.78 | 23.79 | 22.11 | 26.69 | 25.28 | 24.12 | 21.18 | 24.37 | 27.24 |
| MaxLogit [] | 67.02 | 76.91 | 69.47 | 75.73 | 70.03 | 78.37 | 69.41 | 80.90 | 24.18 | 22.41 | 27.17 | 24.60 | 24.90 | 20.69 | 25.00 | 26.82 |
| GEN [] | 67.18 | 77.32 | 69.52 | 75.81 | 70.16 | 78.40 | 69.61 | 81.13 | 24.77 | 23.06 | 27.41 | 23.32 | 28.65 | 21.59 | 25.54 | 26.09 |
| ReAct [] | 64.32 | 78.22 | 68.84 | 77.00 | 69.24 | 80.52 | 67.72 | 82.49 | 21.25 | 13.04 | 25.82 | 20.62 | 25.94 | 18.00 | 25.85 | 20.11 |
| KLM [] | 62.71 | 78.06 | 64.73 | 77.12 | 64.53 | 79.28 | 65.79 | 80.35 | 26.60 | 24.14 | 28.25 | 21.19 | 25.73 | 20.66 | 30.06 | 27.00 |
| Relation [] | 58.70 | 79.28 | 65.41 | 78.51 | 68.32 | 80.80 | 65.20 | 80.88 | 27.17 | 30.24 | 28.16 | 25.52 | 28.12 | 24.41 | 24.89 | 27.99 |
| NNGuide [] | 66.33 | 78.44 | 69.36 | 78.26 | 70.19 | 80.44 | 68.32 | 80.14 | 27.25 | 28.60 | 28.61 | 28.19 | 28.16 | 24.66 | 27.96 | 30.08 |
| Average | 64.78 | 77.67 | 68.08 | 76.60 | 68.81 | 79.07 | 67.82 | 80.74 | 25.06 | 23.59 | 27.54 | 24.23 | 26.34 | 21.64 | 26.13 | 26.78 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 67.34 | 76.81 | 69.06 | 76.32 | 69.11 | 77.66 | 69.05 | 80.11 | 23.42 | 21.69 | 26.54 | 29.55 | 23.43 | 21.51 | 24.01 | 23.52 |
| T2FNorm [] | 67.28 | 77.07 | 68.19 | 77.89 | 68.33 | 78.02 | 68.92 | 80.24 | 24.12 | 22.60 | 25.87 | 27.36 | 24.15 | 22.77 | 24.25 | 24.90 |
| AUGMIX [] | 67.89 | 76.84 | 68.75 | 76.51 | 69.13 | 77.69 | 69.52 | 79.90 | 26.98 | 33.75 | 26.75 | 34.07 | 25.41 | 26.99 | 25.46 | 27.22 |
| REGMIX [] | 68.56 | 77.26 | 68.57 | 76.82 | 69.36 | 77.65 | 69.12 | 80.13 | 27.42 | 18.63 | 27.65 | 22.81 | 26.48 | 19.82 | 27.03 | 21.45 |
| VOS [] | 67.64 | 77.73 | 67.63 | 75.52 | 61.74 | 77.84 | 69.02 | 80.09 | 21.51 | 22.55 | 25.32 | 27.85 | 24.27 | 23.24 | 24.12 | 26.19 |
| NPOS [] | 67.88 | 79.63 | 66.17 | 76.09 | 69.46 | 79.98 | 69.49 | 81.11 | 27.55 | 25.66 | 27.12 | 26.85 | 27.95 | 23.07 | 30.18 | 26.05 |
| BER (Ours) | 68.85 | 79.73 | 70.81 | 78.81 | 69.84 | 80.53 | 69.64 | 81.39 | 28.78 | 30.85 | 28.62 | 27.81 | 28.38 | 25.27 | 28.45 | 30.63 |
| Average | 67.92 | 77.87 | 68.45 | 76.85 | 68.00 | 78.48 | 69.25 | 80.42 | 25.68 | 25.10 | 26.84 | 28.04 | 25.72 | 23.24 | 26.21 | 25.71 |
| Average (All) | 66.15 | 77.76 | 68.24 | 76.71 | 68.46 | 78.81 | 68.45 | 80.60 | 25.34 | 24.25 | 27.23 | 25.90 | 26.07 | 22.34 | 26.16 | 26.31 |
| FPR$\downarrow$ | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| -Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 60.08 | 61.68 | 65.88 | 66.01 | 44.44 | 49.63 | 52.22 | 52.29 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 88.31 | 88.34 | 85.34 | 87.80 | 85.38 | 86.37 | 85.97 | 84.43 | 90.01 | 89.42 | 89.26 | 88.99 | 91.20 | 93.00 | 90.37 | 87.20 |
| ODIN [] | 84.48 | 80.66 | 82.06 | 79.75 | 81.78 | 79.37 | 82.86 | 72.09 | 88.87 | 84.20 | 87.74 | 86.88 | 89.16 | 88.91 | 89.57 | 82.31 |
| Energy [] | 83.99 | 83.61 | 82.41 | 81.82 | 81.75 | 80.95 | 82.58 | 74.57 | 92.13 | 91.38 | 90.36 | 87.98 | 91.81 | 93.03 | 91.78 | 87.91 |
| MaxLogit [] | 84.57 | 84.39 | 82.26 | 83.04 | 82.12 | 82.44 | 82.72 | 75.80 | 91.72 | 90.72 | 89.80 | 88.68 | 91.53 | 93.03 | 91.26 | 87.53 |
| GEN [] | 84.26 | 84.33 | 82.57 | 82.48 | 81.81 | 81.22 | 82.52 | 75.20 | 91.18 | 89.97 | 89.21 | 90.88 | 87.00 | 91.62 | 90.64 | 88.47 |
| ReAct [] | 85.04 | 84.12 | 82.62 | 82.59 | 82.22 | 81.62 | 83.73 | 75.17 | 93.09 | 97.23 | 89.67 | 90.24 | 90.02 | 92.80 | 90.45 | 92.61 |
| KLM [] | 88.24 | 87.89 | 86.78 | 88.09 | 88.03 | 87.46 | 86.18 | 85.76 | 88.79 | 87.27 | 86.30 | 88.93 | 89.25 | 90.26 | 88.33 | 88.69 |
| Relation [] | 90.91 | 88.90 | 84.06 | 79.24 | 90.86 | 87.12 | 84.22 | 70.82 | 91.68 | 86.41 | 89.85 | 89.19 | 94.14 | 95.47 | 89.79 | 87.38 |
| NNGuide [] | 84.23 | 77.32 | 82.26 | 77.06 | 82.06 | 72.97 | 83.57 | 70.27 | 91.40 | 86.84 | 87.44 | 87.11 | 89.02 | 88.42 | 89.17 | 86.63 |
| Average | 86.00 | 84.40 | 83.37 | 82.43 | 84.00 | 82.17 | 83.82 | 76.01 | 90.99 | 89.27 | 88.85 | 88.76 | 90.35 | 91.84 | 90.15 | 87.64 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 83.94 | 83.70 | 82.26 | 82.56 | 82.37 | 80.81 | 82.88 | 75.98 | 92.56 | 92.39 | 89.75 | 82.28 | 92.84 | 93.78 | 92.58 | 92.02 |
| T2FNorm [] | 84.16 | 84.13 | 83.15 | 82.50 | 83.16 | 81.91 | 82.75 | 76.87 | 91.88 | 91.00 | 90.87 | 85.53 | 91.79 | 91.92 | 91.57 | 89.71 |
| AUGMIX [] | 83.61 | 84.07 | 82.48 | 83.17 | 82.33 | 81.14 | 82.47 | 76.70 | 89.26 | 85.42 | 89.94 | 83.66 | 89.29 | 89.55 | 89.42 | 86.75 |
| REGMIX [] | 83.62 | 84.98 | 82.70 | 83.49 | 82.20 | 83.32 | 82.25 | 77.89 | 88.88 | 94.31 | 88.86 | 91.31 | 89.25 | 93.57 | 89.35 | 93.62 |
| VOS [] | 83.34 | 82.12 | 85.81 | 75.64 | 84.71 | 74.86 | 82.75 | 75.18 | 92.36 | 90.29 | 91.35 | 85.71 | 90.14 | 89.48 | 90.24 | 85.72 |
| NPOS [] | 85.45 | 76.52 | 86.09 | 75.20 | 82.76 | 73.78 | 85.58 | 71.21 | 88.92 | 88.38 | 87.60 | 89.88 | 87.36 | 91.10 | 89.22 | 86.75 |
| BER (Ours) | 83.61 | 76.49 | 85.30 | 73.99 | 80.94 | 72.89 | 82.17 | 70.78 | 88.31 | 85.91 | 85.32 | 89.75 | 87.18 | 88.40 | 88.45 | 85.39 |
| Average | 83.96 | 81.72 | 83.97 | 79.51 | 82.64 | 78.39 | 82.98 | 74.94 | 90.31 | 89.67 | 89.10 | 86.87 | 89.69 | 91.11 | 90.12 | 88.57 |
| Average (All) | 85.11 | 83.22 | 83.63 | 81.15 | 83.41 | 80.51 | 83.45 | 75.55 | 90.69 | 89.45 | 88.96 | 87.94 | 90.06 | 91.52 | 90.14 | 88.04 |

**Table 8: Detailed results for those in Table 5 on near- and far-OOD detection datasets at the step size of $k=20$ for CIFAR 100 and at the step size of $k=200$ for ImangeNet1K. The best, second-best, and third-best performance per dataset are highlighted. The upper, middle, lower parts of the table are for AUC, AP, FPR performance, respectively.**
| AUC$\uparrow$ | ID Dataset: CIFAR100 | ID Dataset: ImageNet1K |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 62.65 | 64.14 | 68.05 | 68.75 | 48.85 | 53.30 | 58.42 | 57.84 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 69.42 | 67.86 | 71.44 | 68.87 | 73.14 | 72.11 | 72.33 | 71.99 | 66.14 | 68.23 | 70.66 | 72.84 | 69.22 | 73.77 | 69.35 | 76.01 |
| ODIN [] | 72.67 | 70.68 | 74.14 | 70.11 | 75.06 | 73.87 | 74.02 | 74.67 | 69.56 | 74.95 | 72.70 | 80.40 | 69.88 | 75.29 | 70.83 | 79.00 |
| Energy [] | 73.62 | 71.75 | 74.19 | 70.13 | 75.32 | 75.17 | 75.64 | 76.39 | 67.58 | 73.41 | 71.44 | 80.20 | 68.36 | 74.95 | 69.92 | 78.17 |
| MaxLogit [] | 73.58 | 71.69 | 74.31 | 70.25 | 75.45 | 75.10 | 75.65 | 76.30 | 67.88 | 73.09 | 71.87 | 78.76 | 69.29 | 74.66 | 70.50 | 78.46 |
| GEN [] | 73.74 | 72.22 | 74.56 | 70.59 | 75.58 | 75.25 | 75.88 | 76.88 | 68.77 | 72.94 | 72.27 | 78.04 | 70.97 | 73.09 | 71.27 | 79.00 |
| ReAct [] | 70.33 | 72.22 | 73.64 | 73.72 | 74.16 | 78.15 | 74.81 | 78.60 | 60.87 | 56.69 | 68.14 | 72.73 | 63.74 | 61.87 | 66.82 | 60.83 |
| KLM [] | 69.38 | 68.96 | 71.08 | 69.75 | 72.76 | 72.33 | 72.62 | 72.40 | 68.67 | 71.19 | 71.37 | 68.92 | 69.94 | 68.94 | 72.37 | 75.89 |
| Relation [] | 60.26 | 67.92 | 67.36 | 69.97 | 72.90 | 77.89 | 66.61 | 73.61 | 69.03 | 75.66 | 72.58 | 80.21 | 69.52 | 75.30 | 69.76 | 80.29 |
| NNGuide [] | 71.30 | 75.56 | 71.33 | 71.20 | 73.40 | 77.72 | 72.02 | 78.30 | 68.64 | 76.97 | 75.00 | 83.84 | 68.53 | 78.64 | 70.03 | 79.75 |
| Average | 70.48 | 70.98 | 72.45 | 70.51 | 74.20 | 75.29 | 73.29 | 75.46 | 67.46 | 71.46 | 71.78 | 77.33 | 68.83 | 72.95 | 70.09 | 76.38 |
| -Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 74.05 | 71.59 | 73.34 | 70.23 | 74.81 | 74.03 | 75.63 | 75.25 | 68.55 | 71.11 | 70.12 | 79.30 | 67.75 | 71.92 | 70.00 | 74.85 |
| T2FNorm [] | 74.00 | 72.01 | 72.31 | 71.12 | 74.69 | 74.23 | 75.80 | 75.65 | 68.68 | 73.98 | 70.42 | 81.78 | 69.08 | 74.69 | 69.34 | 76.24 |
| AUGMIX [] | 74.65 | 71.43 | 73.06 | 69.52 | 75.30 | 73.87 | 76.22 | 75.20 | 67.42 | 79.12 | 71.80 | 86.56 | 67.22 | 78.09 | 68.86 | 80.72 |
| REGMIX [] | 74.54 | 71.84 | 72.97 | 70.94 | 75.19 | 74.57 | 76.00 | 75.53 | 69.64 | 69.41 | 70.05 | 78.26 | 69.44 | 73.16 | 70.85 | 73.83 |
| VOS [] | 74.16 | 72.31 | 70.53 | 67.89 | 69.55 | 73.55 | 75.17 | 75.01 | 60.12 | 68.59 | 59.60 | 73.36 | 60.57 | 67.45 | 61.50 | 73.81 |
| NPOS [] | 73.51 | 74.58 | 69.51 | 71.53 | 74.85 | 77.48 | 75.12 | 77.10 | 68.86 | 75.30 | 71.50 | 83.78 | 70.21 | 74.98 | 71.84 | 77.03 |
| BER (Ours) | 74.03 | 74.81 | 70.73 | 72.49 | 74.36 | 77.78 | 75.07 | 77.24 | 69.39 | 76.87 | 73.15 | 83.08 | 70.61 | 74.72 | 70.89 | 80.07 |
| Average | 74.13 | 72.65 | 71.78 | 70.53 | 74.11 | 75.07 | 75.57 | 75.85 | 67.52 | 73.48 | 69.52 | 80.87 | 67.84 | 73.57 | 69.04 | 76.65 |
| Average (All) | 72.08 | 71.71 | 72.16 | 70.52 | 74.16 | 75.19 | 74.29 | 75.63 | 67.49 | 72.34 | 70.79 | 78.88 | 68.40 | 73.22 | 69.63 | 76.50 |
| AP$\uparrow$ | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 62.65 | 64.14 | 68.05 | 68.75 | 48.85 | 53.30 | 58.42 | 57.84 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 64.69 | 77.88 | 66.09 | 77.36 | 68.06 | 79.42 | 67.19 | 79.81 | 26.39 | 25.10 | 29.95 | 27.19 | 27.46 | 28.99 | 28.24 | 33.60 |
| ODIN [] | 68.22 | 76.99 | 68.99 | 75.95 | 70.69 | 78.59 | 69.52 | 79.47 | 29.08 | 32.24 | 31.33 | 37.17 | 29.02 | 31.30 | 29.01 | 36.89 |
| Energy [] | 69.22 | 77.93 | 68.94 | 76.22 | 70.91 | 79.51 | 71.12 | 80.44 | 25.81 | 28.64 | 28.82 | 35.39 | 26.01 | 29.60 | 27.01 | 32.20 |
| MaxLogit [] | 69.11 | 78.06 | 68.98 | 76.48 | 70.94 | 79.75 | 71.03 | 80.58 | 26.38 | 28.39 | 29.68 | 32.27 | 27.10 | 29.57 | 27.87 | 33.98 |
| GEN [] | 69.27 | 78.40 | 69.25 | 76.53 | 71.05 | 79.60 | 71.25 | 80.91 | 27.02 | 27.81 | 30.20 | 31.68 | 30.52 | 27.10 | 28.67 | 36.42 |
| ReAct [] | 66.69 | 79.64 | 68.56 | 78.62 | 70.19 | 81.96 | 70.37 | 82.46 | 22.29 | 17.14 | 28.76 | 26.53 | 25.82 | 20.59 | 28.13 | 18.63 |
| KLM [] | 65.01 | 79.43 | 65.97 | 79.45 | 67.14 | 80.87 | 67.58 | 81.20 | 29.76 | 26.16 | 30.73 | 24.36 | 29.31 | 23.31 | 34.15 | 28.59 |
| Relation [] | 58.89 | 81.39 | 65.14 | 80.71 | 69.00 | 85.23 | 65.80 | 82.40 | 28.20 | 32.80 | 31.11 | 30.76 | 33.80 | 27.37 | 28.26 | 28.12 |
| NNGuide [] | 69.13 | 82.22 | 69.16 | 79.13 | 71.16 | 83.33 | 71.80 | 82.63 | 31.55 | 28.89 | 25.52 | 29.69 | 32.09 | 29.09 | 33.44 | 29.75 |
| Average | 66.69 | 79.10 | 67.90 | 77.83 | 69.90 | 80.92 | 69.52 | 81.10 | 27.39 | 27.46 | 29.57 | 30.56 | 29.01 | 27.44 | 29.42 | 30.91 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 69.65 | 77.80 | 68.22 | 76.29 | 70.29 | 78.78 | 71.03 | 79.71 | 26.76 | 26.25 | 27.62 | 37.77 | 25.39 | 27.70 | 27.39 | 27.84 |
| T2FNorm [] | 69.53 | 78.22 | 67.29 | 77.07 | 70.04 | 79.23 | 71.12 | 80.11 | 26.88 | 29.40 | 27.96 | 41.73 | 26.59 | 30.40 | 27.04 | 29.86 |
| AUGMIX [] | 70.16 | 77.73 | 67.84 | 76.12 | 70.68 | 78.69 | 71.57 | 79.62 | 29.41 | 31.62 | 28.77 | 28.31 | 27.78 | 27.66 | 29.20 | 29.28 |
| REGMIX [] | 70.27 | 78.14 | 68.12 | 76.98 | 71.06 | 79.33 | 71.58 | 79.97 | 28.46 | 22.69 | 28.06 | 29.67 | 27.67 | 25.48 | 29.05 | 24.29 |
| VOS [] | 69.66 | 78.12 | 66.03 | 75.72 | 65.00 | 79.31 | 70.44 | 79.58 | 22.86 | 26.04 | 21.60 | 31.07 | 22.16 | 26.43 | 23.24 | 29.48 |
| NPOS [] | 70.01 | 80.50 | 64.89 | 78.62 | 70.61 | 82.14 | 70.67 | 82.01 | 27.82 | 30.66 | 28.15 | 30.45 | 27.37 | 31.04 | 30.18 | 27.65 |
| BER (Ours) | 70.63 | 81.41 | 68.86 | 79.98 | 71.41 | 83.73 | 72.86 | 82.60 | 30.26 | 31.87 | 30.71 | 36.50 | 29.82 | 33.39 | 30.66 | 32.12 |
| Average | 69.99 | 78.85 | 67.32 | 77.25 | 69.87 | 80.17 | 71.32 | 80.51 | 27.49 | 28.36 | 27.55 | 33.64 | 26.68 | 28.87 | 28.11 | 28.65 |
| Average (All) | 68.13 | 78.99 | 67.65 | 77.58 | 69.89 | 80.59 | 70.31 | 80.84 | 27.43 | 27.86 | 28.69 | 31.91 | 27.99 | 28.06 | 28.85 | 29.92 |
| FPR$\downarrow$ | iCaRL [] | BiC[] | WA [] | FOSTER[] | iCaRL [] | BiC[] | WA [] | FOSTER[] |  |  |  |  |  |  |  |  |
| Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far | Near | Far |  |
| Average CIL accuracy |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
|  | 62.65 | 64.14 | 68.05 | 68.75 | 48.85 | 53.30 | 58.42 | 57.84 |  |  |  |  |  |  |  |  |
| -Post hoc-based OOD methods (w/o Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MSP [] | 86.59 | 87.49 | 86.66 | 88.30 | 84.62 | 84.80 | 85.18 | 84.94 | 88.25 | 85.89 | 86.92 | 86.19 | 89.12 | 83.14 | 87.95 | 77.41 |
| ODIN [] | 83.94 | 79.84 | 84.45 | 78.70 | 82.02 | 76.10 | 82.65 | 73.77 | 85.99 | 78.86 | 85.81 | 76.03 | 86.75 | 80.31 | 87.37 | 74.06 |
| Energy [] | 82.74 | 81.62 | 84.56 | 81.39 | 82.06 | 77.47 | 81.72 | 76.95 | 90.62 | 83.67 | 88.72 | 77.34 | 90.36 | 82.46 | 90.03 | 81.20 |
| MaxLogit [] | 82.62 | 81.99 | 84.72 | 83.23 | 81.85 | 79.10 | 81.44 | 77.54 | 89.86 | 84.00 | 87.78 | 82.40 | 89.54 | 82.69 | 89.14 | 78.44 |
| GEN [] | 82.86 | 82.23 | 84.43 | 82.28 | 81.67 | 78.23 | 81.24 | 76.96 | 89.24 | 84.32 | 87.31 | 82.91 | 85.46 | 86.07 | 88.44 | 76.02 |
| ReAct [] | 84.18 | 82.40 | 84.02 | 82.30 | 81.51 | 76.98 | 81.86 | 77.35 | 92.37 | 92.98 | 86.31 | 85.15 | 88.68 | 87.47 | 87.89 | 91.61 |
| KLM [] | 86.34 | 85.12 | 86.48 | 86.89 | 86.24 | 85.97 | 84.75 | 84.21 | 84.68 | 85.60 | 85.24 | 86.67 | 86.42 | 89.64 | 80.57 | 85.16 |
| Relation [] | 87.14 | 76.55 | 84.52 | 76.94 | 83.35 | 74.51 | 83.91 | 72.84 | 89.50 | 82.30 | 86.36 | 77.31 | 93.46 | 91.33 | 88.11 | 73.72 |
| NNGuide [] | 82.70 | 74.23 | 84.31 | 76.17 | 81.69 | 69.83 | 85.19 | 71.60 | 91.91 | 80.27 | 91.24 | 77.11 | 91.99 | 76.72 | 87.52 | 73.19 |
| Average | 84.35 | 81.27 | 84.91 | 81.80 | 82.78 | 78.11 | 83.10 | 77.35 | 89.16 | 84.21 | 87.30 | 81.23 | 89.09 | 84.43 | 87.45 | 78.98 |
| Fine-tuning-based OOD methods (w/ Fine-tuning) |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| LogitNorm [] | 82.61 | 82.17 | 84.77 | 83.31 | 82.23 | 78.12 | 81.87 | 77.98 | 90.18 | 87.16 | 89.59 | 72.99 | 91.49 | 85.73 | 90.31 | 87.28 |
| T2FNorm [] | 82.63 | 82.28 | 85.28 | 83.05 | 82.63 | 79.67 | 81.59 | 78.60 | 89.78 | 82.85 | 89.25 | 68.00 | 90.11 | 81.85 | 89.61 | 83.92 |
| AUGMIX [] | 82.34 | 82.77 | 84.96 | 84.02 | 81.91 | 78.99 | 81.34 | 78.74 | 89.08 | 85.59 | 88.27 | 77.35 | 92.17 | 77.26 | 88.27 | 74.44 |
| REGMIX [] | 81.88 | 83.27 | 85.08 | 84.43 | 81.84 | 79.75 | 81.07 | 79.61 | 88.40 | 92.04 | 88.75 | 85.22 | 88.44 | 87.50 | 87.88 | 90.78 |
| VOS [] | 82.83 | 80.67 | 85.24 | 74.80 | 86.31 | 71.70 | 82.50 | 76.54 | 91.28 | 86.32 | 88.06 | 88.46 | 91.31 | 84.44 | 90.04 | 84.19 |
| NPOS [] | 85.94 | 75.37 | 88.36 | 73.47 | 81.89 | 66.84 | 84.47 | 73.14 | 87.81 | 86.00 | 86.91 | 78.49 | 88.48 | 86.62 | 87.92 | 73.53 |
| BER (Ours) | 82.70 | 73.77 | 85.96 | 73.97 | 81.59 | 66.79 | 81.14 | 71.99 | 87.71 | 85.57 | 86.06 | 76.91 | 85.40 | 87.91 | 87.63 | 72.72 |
| Average | 82.99 | 80.04 | 85.66 | 79.58 | 82.63 | 74.55 | 82.00 | 76.66 | 89.18 | 86.50 | 88.13 | 78.20 | 89.63 | 84.47 | 88.81 | 80.98 |
| Average (All) | 83.75 | 80.74 | 85.24 | 80.83 | 82.71 | 76.55 | 82.62 | 77.05 | 89.17 | 85.21 | 87.66 | 79.91 | 89.32 | 84.45 | 88.04 | 79.85 |

## Appendix D More Results w.r.t. Increasing Incremental Steps

In this section, we provide more results for the different baselines w.r.t. increasing incremental steps in our paper.
Particularly, following the results of iCaRL [] with CIFAR100 [] in our paper, we provide the results based on Bic [] in Fig. 4, WA [] in Fig. 5, FOSTER[] in Fig. 6.
Furthermore, we show relationship between ACC and OOD detection performance on these three CIL models in Fig. 7.
We also show the average performance of all four CIL models with more OOD detectors (GEN [], KLM [], and NPOS []) in Fig. 8.

Figure: (a)
Refer to caption: x7.png

Figure: (a)
Refer to caption: x10.png

Figure: (a)
Refer to caption: x13.png

Figure: (a)
Refer to caption: x16.png

Figure: (a)
Refer to caption: x19.png

## Appendix E The OpenCIL Benchmark framework and BER Algorithm

The full framework of the OpenCIL Benchmark for post-hoc-based OOD methods are given in Algorithm 1, and for fine-tuning-based OOD methods are given in Algorithm 2 below.
Furthermore, the algorithm of our proposed baseline BER are given in Algorithm 3.

Figure: Algorithm 1 : Post-hoc-based OOD Methods Framework in OpenCIL Benchmark

Figure: Algorithm 2 : Fine-tuning-based OOD Methods Framework in OpenCIL Benchmark

Figure: Algorithm 3 : Bi-directional Energy Regularization (BER)

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: See the weaknesses above.
soundness: 2
strengths: 1. The mixup-based pseudo OOD and old task data generation is interesting.
2. The experiment is extensive and comprehensive including most state-of-the-art OOD and CIL methods.
summary: This paper introduces OpenCIL, a benchmark for evaluating out-of-distribution (OOD) detection in class incremental learning (CIL), which is vital for practical deployment in open-world scenarios. The study identifies biases in CIL models towards OOD samples and new classes and proposes Bi-directional Energy Regularization (BER) to address these issues. BER applies energy regularization to both old and new classes, improving OOD detection. Experiments show that BER achieves effective performance improvements on the OpenCIL benchmark, enhancing CIL models' robustness.
weaknesses: 1. The motivation for benchmarking OOD in CIL is not strong, as OOD detection becomes more and more ineffective during CIL is intuitively predictable. There is no need to spend a lot of resources to illustrate a predictable finding. 
2. Although the mixup-based pseudo-OOD samples and synthetic old task data are interesting, the OOD detection of the proposed BER still relies on existing energy-based methods. In terms of technical novelty, it is more appealing to propose a new OOD scoring mechanism dedicated to CIL.
3. For the OOD classifier, how to initialize it is unclear. Is it initialized from the ID classification classifier or inherited from the latest old OOD classifier?
4. I would disagree with the finding of lines 337-339. In most cases, the performance of fine-tuning-based OOD and post-hoc-based OOD methods are comparable. Besides, I am not sure why BER is only compared with fine-tuning-based OOD approaches. There are some cases in which BER cannot beat all post-hoc-based OOD methods. If fine-tuning-based BER cannot far exceed post-hoc-based OOD methods, I cannot figure out a reason to adopt BER, which even consumes much higher computation costs.
5. The paper presentation and organization have a lot to improve. For example, in Figure 3, there is no legend for dashed curves in subfigure (a); the legend in subfigure (b) denotes ACC performance and different OOD methods, which is confusing. Besides, the curves are the same type; The same curve with the same color denotes different methods among the three subfigures, which is hard to distinguish and read. In addition, the last finding (starts at line 337 on page 7) refers to Tables 1,2,3 that are on pages 9 and 10, which largely hurts the readability.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: + Minor Comments
  - **Page 4, Figure 2 caption**: “principled frameworks for incorporating OOD detection methods” could be clearer if it stated explicitly that these frameworks apply to both post-hoc and fine-tuning-based methods.
  - **Equation (3)**: Consider defining variables like $p_{in}$ and $p_{out}$ more explicitly right after the equation to improve readability.
+ It would be valuable to see a breakdown of the computational trade-offs involved in implementing both NTER and OTER. Are there scenarios where only one of the two might be sufficient or more efficient?
+ Could the OOD detection capability of BER be further enhanced by augmenting the memory with synthesized samples from OOD data?
soundness: 3
strengths: - The paper addresses the gap in **OOD detection** for **class incremental learning**, an area that has seen limited exploration. This contribution is novel, particularly in how it systematically enables a unified protocol for combining CIL models with diverse OOD detectors, which is essential for real-world applications.
- The explanations of the proposed BER method are clear, especially the roles of **New Task Energy Regularization (NTER)** and **Old Task Energy Regularization (OTER)**. These components are effectively tied back to the issues of bias and catastrophic forgetting, making the contributions and their impact easy to grasp.
- The paper is generally well-organized, with a clear structure that systematically introduces the problem, the proposed OpenCIL benchmark, and BER. The visual aids (e.g., Figs 1 and 2) are helpful in understanding the mechanisms and biases associated with CIL models in OOD contexts.
summary: This paper addresses a critical challenge in CIL: training models to learn new classes while retaining knowledge of previous ones and effectively detecting OOD samples, essential for open-world applications like autonomous systems. To evaluate OOD detection methods in CIL, the authors introduce OpenCIL, a benchmark featuring both post-hoc and fine-tuning-based methods across various CIL models and datasets. They also propose BE to mitigate biases toward new classes and OOD samples, using NTER and OTER to enhance OOD detection by synthesizing pseudo-OOD and augmented samples.
weaknesses: + The paper briefly mentions hyperparameters like $\tau$ and $\alpha$ for the BER approach, but there’s no thorough exploration of how sensitive BER is to these parameters. A parameter sensitivity analysis could offer insights into how to optimize BER for different datasets and models, making it more user-friendly.
+ the complexity of the approach might pose challenges for real-world implementation, especially regarding **computational overhead** due to the additional synthesis and regularization components in NTER and OTER. 
+ There are missing citations in the manuscript. For example, the paper introduces generative models, but generative-based methods [1, 2, 3] are missed without corresponding details in the bibliography.   

[1]: Kong, Shu, and Deva Ramanan. "Opengan: Open-set recognition via open data generation." In ICCV. 2021.

[2]: Wang, Qizhou, et al. "Out-of-distribution detection with implicit outlier transformation." In ICLR, 2023.

[3]: Zheng, Haotian, et al. "Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sources." In NeurIPS, 2023.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Could you supplement the references with more recent literature, especially CIL methods from 2023 onward, to better demonstrate the relative advantages of the proposed method?

2. In Sections 4.1 and 4.2, what are the specific meanings of pin and pout? What role do these parameters play in energy regularization? A detailed explanation of these parameters would help readers understand the equations.

3. There is an error in the equation labeling in Algorithm 3 in the Appendix. Could you review and correct the labels to ensure the document's professionalism?

4. Considering the extensibility of the proposed framework and experimental setup, have you considered including more advanced CIL methods as benchmarks to enhance the representativeness and rigor of the experiments?
soundness: 3
strengths: Originality: This paper designs a new benchmark named OpenCIL to establish the performance of the combination between Out-of-Distribution and Class Incremental Learning with two frameworks. Further more this paper proposes Bi-directional Energy Regularization to mitigate the imbalance between old classes and new classes as well as the Out-of-Distribution data.Quality: The experimental is comprehensive, and the results show a slight improvement over other methods. Clarity: The paper clearly defines the problem it aims to address. Significance: The significance of this paper lies in its contribution to provide unified structure for future researchers exploring the integration of OOD detection and CIL.
summary: 1.The variables p_in and p_out in the formulas in Sections 4.1 and 4.2 are not explained clearly. 2.There is an issue with the formula numbering in Algorithm 3 in the Appendix, where Eq.2 is incorrectly labeled as Eq.1. 3.The paper lacks sufficient novelty, as the CIL methods used are outdated, with the most recent work being from 2022. More recent methods should be considered to better demonstrate the novelty of the paper.
weaknesses: 1.Lack of Comprehensive References: The paper does not thoroughly cite recent studies in the field, particularly the latest methods in Class Incremental Learning (CIL). The comparison experiments use CIL methods with the latest from 2022, which may not reflect the most current advancements. More comprehensive references and comparisons could enhance the credibility and persuasiveness of the approach.
2.Unclear Equations: In Sections 4.1 and 4.2, the key parameters pin and pout are not explained, which may hinder readers’ understanding of the equations. Additionally, there is a labeling error in Algorithm 3 in the Appendix, where Eq. 2 is mistakenly labeled as Eq. 1. Correcting these will improve the paper's professionalism and readability.
3.Lack of Advanced Comparison Methods: The comparison experiments do not include the most recent CIL methods, which may affect the reliability of the results. It is recommended to incorporate more recent CIL methods in the comparisons to provide a more thorough evaluation of the proposed framework.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. How would the method perform when the out-of-distribution dataset is dynamic? In other words, would the model be updated so that newly seen classes are no longer considered as out-of-distribution once they have been learned?
2. Further clarifications could be provided in Figure 1. How can we compare the prediction confidence between in-distribution and out-of-distribution samples on old classes and new classes to justify that old class samples are being misclassified as out-of-distribution samples? 
3. For fine-tuning based methods, the prediction of in-distribution classification and out-of-distribution detection is separated. What is the purpose of comparing the 'prediction confidence' between in-distribution and out-of-distribution samples in Figure 1 given that the out-of-distribution branch of the model determines whether a sample is out-of-distribution? 
4. Could the authors provide further clarification on Eq (3) and Eq (5)? It seems to also differ from the loss proposed in the original work (Liu et al., 2020).
soundness: 2
strengths: 1. The paper address an important and challenging problem, which is to perform out-of-distribution detection in the open-world setting and thus, improves the applicability of incremental learning models in real world applications. 
2. The proposed regularization term can be easily integrated into existing class incremental learning models to improve their capability in out-of-distribution detection.
3. Quantitative results demonstrates significant and consistent performance improvement.
summary: This paper addresses the need of detecting out-of-distribution instances by class incremental learners in the open world on top of learning new classes without forgetting old knowledge. The authors propose a OpenCIL benchmark, which utilizes two commonly used class-incremental learning datasets and six out-of-distribution datasets. As baseline of the benchmark, the authors combine existing class-incremental learning and out-of-distribution detection methods and evaluate them on the datasets. To address the issue of increasing bias towards new class and out-of-distribution samples observed in the baseline methods, the authors proposed a Bi-directional Energy Regularization loss to improve the out-of-distribution capability of the class-incremental learning model. Specifically, the new task and old task energy regularization terms are used to modulate the energy prediction of out-of-distribution samples such that they can be distinguished from new class and old class samples respectively. Experimental results on the OpenCIL benchmark demonstrate the effectiveness of the proposed approach.
weaknesses: 1. The out-of-distribution dataset in OpenCIL benchmark is static, that is, these samples or classes remain as out-of-distribution throughout the whole incremental learning process. It does not address the scenario in the open-world where the incremental learner learns the out-of-distribution samples detected. In this case, these samples should no longer be detected as out-of-distribution by the model in the next incremental phase. 
2.  The main findings stated in Section 3.2 needs further clarification. There is no strong justification of why 'catastrophic forgetting is more persistent in CIL models than OOD detection methods' given that the authors are comparing the decreasing rate of two different metrics. It is also unclear how the authors justify that old class samples are detected as out-of-distribution samples. From the prediction confidence in Figure 1, it is difficult to understand how the authors claim that old class samples have lower prediction confidence than out-of-distribution samples. 
3. There seems to be some conflict proposed regularization terms. In Eq (3), the model penalizes in-distribution samples that produce energy lower than $p_{in}$ while penalizing pseudo-OOD samples that produce energy higher than $p_{out}$. On the other hand, in Eq (5), the model is penalizing augmented in-distribution samples that produce energy higher than $p_{in}$.  
4. Limited class-incremental learning methods are used as baselines, where the most recent method is from year 2022. For a more comprehensive evaluation, it would be beneficial to include some of the latest released methods to better demonstrate the robustness and generalizability of proposed method.
5. Novelty is limited as the proposed method is a simple combination of mixup augmentation with the energy-based out-of-distribution detection from previous work (Liu et al., 2020).