SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jinpeng Chen $^{1}$ , Runmin Cong $^{2*†}$ , Yuxuan Luo $^{1}$ , Horace Ho Shing Ip $^{1\dagger}$ , and Sam Kwong $^{3‡}$
$^{1}$ Department of Computer Science, City University of Hong Kong, Hong Kong
$^{2}$ School of Control Science and Engineering, Shandong University, Jinan, Shandong, China
$^{3}$ Lingnan University, Hong Kong
jinpechen2-c@my.cityu.edu.hk, rmcong@sdu.edu.cn, yuxuanluo4-c@my.cityu.edu.hk,
horace.ip@cityu.edu.hk, samkwong@ln.edu.hk
Existing class-incremental semantic segmentation (CISS) methods mainly tackle catastrophic forgetting and background shift, but often overlook another crucial issue. In CISS, each step focuses on different foreground classes, and the training set for a single step only includes images containing pixels of the current foreground classes, excluding images without them. This leads to an overrepresentation of these foreground classes in the single-step training set, causing the classification biased towards these classes. To address this issue, we present STAR, which preserves the main characteristics of each past class by storing a compact prototype and necessary statistical data, and aligns the class distribution of single-step training samples with the complete dataset by replaying these prototypes and repeating background pixels with appropriate frequency. Compared to the previous works that replay raw images, our method saves over 100 times the storage while achieving better performance. Moreover, STAR incorporates an old-class features maintaining (OCFM) loss, keeping old-class features unchanged while preserving sufficient plasticity for learning new classes. Furthermore, a similarity-aware discriminative (SAD) loss is employed to specifically enhance the feature diversity between similar old-new class pairs. Experiments on two public datasets, Pascal VOC 2012 and ADE20K, reveal that our model surpasses all previous state-of-the-art methods. The official code is available at https://github.com/jinpeng0528/STAR.
Typically, neural networks [6, 15, 16] are trained in a one-time fashion. When data updates occur, retraining on the entire dataset is often necessary, as finetuning solely on new data leads to catastrophic forgetting [21] of previously learned knowledge. However, frequent retraining is time-consuming and requires the storage of all past data. Consequently, there is a growing interest in enabling networks to achieve continual learning, which aims to attain performance comparable to one-time training by progressively learning from new data.
*Runmin Cong is also with the Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Jinan, Shandong, China.
†Horace Ho Shing Ip is also with the Centre for Innovative Applications of Internet and Multimedia Technologies, City University of Hong Kong, Hong Kong.
While most continual learning research has focused on image classification, the growing expectation for computers to perform various tasks has driven the expansion of research into other domains. One such domain that has gained considerable attention is class-incremental semantic segmentation (CISS). In CISS, the training process involves multiple steps, with each step focusing on different classes. During each step, only a part of the classes is considered foreground and assigned ground-truth labels, while foreground classes from past or future steps are labeled as background. Compared to other continual learning tasks, CISS must address not only catastrophic forgetting but also background shift [2]. The latter refers to the fact that the background in the current step may include pixels from past or future foreground classes, causing its semantics to change across steps. Without specific countermeasures, this can greatly confuse the network and result in poor performance.
Previous CISS methods [1, 2, 3, 9, 22, 23] effectively addressed catastrophic forgetting and background shift, but they often neglect another critical issue: the discrepancies in class distributions between single-step training sets and the complete dataset. Single-step training sets include only images with pixels from corresponding foreground classes, resulting in an increased proportion of these classes compared to the complete dataset. For instance, under a CISS protocol on the Pascal VOC 2012 dataset [11], overlapped 19-1, 19 foreground classes are trained in the first step, and only tv/monitor is in the second step. In this protocol, tv/monitor constitutes  $16.5\%$  of pixels in the second step's training set but only  $0.8\%$  in the complete dataset. Hence, the network may become biased towards tv/monitor, causing numerous false positives, where pixels not belonging to tv/monitor are misclassified as such. Though adjusting class weights in the loss function can mitigate this issue, the limited quantity and richness of negative samples (pixels not tv/monitor) impede a full grasp of their potential characteristics.
To tackle this issue, we propose a method called STAR (STorage sAving Replay for CISS). After each training step, STAR stores key information, including a compact prototype for each current foreground class, which is derived from the features of this class, and statistical data related to the prototypes or class proportions. Using this information, STAR replays past foreground classes and repeats background pixels at appropriate frequencies during subsequent steps, enabling a thorough reconstruction of single-step training sample distributions with minimal storage cost. Compared to methods that save and replay raw images [1, 3], STAR significantly reduces storage requirements (using only  $\sim 1\%$  of space) while achieving better performance. It also avoids ethical concerns, as the stored data does not contain personal information. Note that although previous replay-based CISS works [1, 3, 20] consider the main function of replay to be combating forgetting, we argue that its more crucial role is in learning new classes, as confirmed by [1, 3], where the versions with replay improve the performance on new classes more than old classes. Since new classes are typically trained individually or with few other new classes, there is a larger overrepresentation in the single-step training set, leading to a noticeable bias in the new-class classifiers that cannot access enough negative samples. The replayed samples, however, can alleviate this problem by serving as negative samples. Extending this idea, we propose to thoroughly align the class distributions of each single-step training set with the complete dataset up to that step by controlling the replay frequency.
In addition, to coordinate with our replay strategy as well as reduce forgetting, it is important to maintain old-class features during subsequent steps. Otherwise, old-class features will deviate from the saved prototypes, rendering the replay ineffective. To address this, we propose an old-class features maintaining (OCFM) loss, which is a spatially-targeted knowledge distillation that constrains the change of old-class features while allowing sufficient flexibility to learn new classes. Moreover, we introduce a similarity-aware discriminative (SAD) loss to selectively enhance discriminability between similar new and old class features, making it easier for the classifiers to distinguish them.
In summary, this paper presents the following contributions: (1) We propose STAR, a method that stores necessary statistical data and compact prototypes for learned foreground classes, enabling a comprehensive reconstruction of single-step training sample distributions to align with the complete dataset, thus eliminating the bias towards specific classes. (2) We introduce an old-class features maintaining (OCFM) loss that retains learned knowledge in a spatially targeted manner, preserving old-class features while ensuring sufficient flexibility for learning new classes. We also design a similarity-aware discriminative (SAD) loss to specifically enhance the discriminability of features between similar old-new class pairs, facilitating the classification. (3) On two public datasets, Pascal VOC 2012 [11] and ADE20K [34], our STAR achieves state-of-the-art performance in CISS.
2.1 Continual learning
Continual learning aims to enable models to incrementally learn new knowledge while avoiding catastrophic forget old knowledge. In computer vision, most continual learning research focuses on image classification and falls into three categories: regularization-based, rehearsal-based, and structure-based methods. Regularization-based methods [17, 8, 10, 4] apply constraints to parameter updates to preserve crucial learned knowledge. Rehearsal-based methods [26, 27, 24] store a small amount of past data or generate additional old-class data for replay during subsequent steps to minimize forgetting. Structure-based methods [19, 28, 18] allocate extra parameters for new classes and keep parameters related to old classes unchanged to preserve knowledge for both.
Similar to our approach, several previous continual learning studies have also aimed at enhancing memory efficiency [14, 30, 32]. For example, [14, 30] utilize compact prototypes to retain old class information and have delved into addressing prototype ineffectiveness caused by shifts in the feature space during incremental training. [32] leverages low-fidelity exemplars rather than prototypes, with a focus on narrowing the domain gap between these low-fidelity exemplars and raw images. Our method distinguishes itself in four aspects: (1) We save a single compact prototype per class for better memory efficiency, unlike some prior works [14, 32] which store multiple prototypes or exemplars. (2) We incorporates class distribution during replay, effectively rectifying the classification bias. (3) We store feature statistics beyond prototypes, offering more comprehensive cues of past classes. (4) All these studies concentrate on continual image classification, but our focus is CISS.
2.2 Class-incremental semantic segmentation
In recent years, a growing number of methods have focused on the CISS task. ILT [22] employs knowledge distillation on both output and intermediate features to reduce catastrophic forgetting. MiB [2] addresses background shift for the first time, using operations on predicted probabilities to eliminate semantic contradictions between steps. SDR [23] leverages prototype matching and contrastive learning to enhance discrimination between features of different classes while applying feature sparsity techniques to reserve space for future knowledge. PLOP [9] preserves old-class knowledge using pseudo labels generated by the previous model for supervision and a spatially-aware variant of POD [10]. SSUL [3] introduces an extra unknown class aided by salient object detection models to tackle background shift. DKD [1] adopts decomposed knowledge distillation to maintain not only the output logit but also its positive and negative components.
RBC [33] is the only prior method that has examined the differences in class distributions between single-step training sets and the complete dataset. During incremental training, it employs pairs of input images, one normal and the other with new-class pixels erased. Additionally, it enhances the weight of old-class pixels in the loss function. These strategies do help mitigate the bias of classifiers towards new classes. However, they do not take into account the actual class proportions within the complete dataset when rectifying the bias, offering no assurance that the classifier's bias can be adjusted to an appropriate level. In contrast, our approach, which replays old-class prototypes based on the recorded class proportion in all past training samples, holds an advantage.
3.1 Problem definition and basic setup
In CISS, a model incrementally learns among  $T$  steps. For each step  $t \in \{1, \dots, T\}$ , the training set  $D^t$  is consisted of  $N^t$  images  $\{x_i^t\}_{i=1}^{N^t} \subseteq \mathbb{R}^{H \times W \times 3}$  and corresponding labels  $\{y_i^t\}_{i=1}^{N^t} \subseteq \mathbb{R}^{H \times W}$ , where  $H \times W$  indicates the image size. During step  $t$ , only a subset of classes,  $L^t$ , is regarded as foreground classes with ground-truth labels in  $y_i^t$ . All other pixels, including the actual background, past classes  $L^{1:t-1}$ , and future classes  $L^{t+1:T}$ , are labeled as background. Note that the sets of foreground classes for different steps are disjoint, i.e.,  $L^t \cap L^{1:t-1} = \emptyset$  and  $L^t \cap L^{t+1:T} = \emptyset$ .
The model at step  $t$  is denoted as  $M^t$ , consisting of a feature extractor  $\Psi_{\theta^t}$  with parameters  $\theta^t$  and classifiers  $\Phi_{\omega^t}$  with parameters  $\omega^t$ .  $\Phi_{\omega^t}$  includes the classifier for each past or current foreground class  $l \in L^{1:t}$ , represented as  $\Phi_{\omega_l^t}$ . Before training at step  $t$ ,  $\theta^t$  and  $\{\omega_l^t\}_{l \in L^{1:t-1}}$  are initialized by
Figure 1: Overview of the proposed STAR. In earlier steps, prototypes for past foreground classes have been saved. At the current step, prototypes are replayed, and background pixels are repeated for classifiers. The OCFM loss maintains similarity between old-class region features from current and previous feature extractors, while the SAD loss differentiates features for similar old-new class pairs.
$\theta^{t-1}$  and  $\{\omega_l^{t-1}\}_{l \in L^{1:t-1}}$ , and  $\{\omega_l^t\}_{l \in L^t}$  is randomly initialized. For an input image  $x_i^t$ , the output of  $M^t$  is denoted as  $p_i^t \in \mathbb{R}^{H \times W}$ , which can be computed by  $p_i^t = M^t(x_i^t) = \Phi_{\omega^t} \circ \Psi_{\theta^t}(x_i^t)$ . The extracted features are denoted as  $f_i^t \in \mathbb{R}^{h \times w \times c}$ , which can be computed by  $f_i^t = \Psi_{\theta^t}(x_i^t)$ . Here,  $h \times w$  and  $c$  represent the spatial size and the number of channels, respectively. When feeding a current image  $x_i^t$  to a previous model  $M^{t-\tau}$  or feature extractor  $\Psi_{\theta^{t-\tau}}$ , the output is given by  $p_i^{t,t-\tau} = M^{t-\tau}(x_i^t)$ , and the features are given by  $f_i^{t,t-\tau} = \Psi_{\theta^{t-\tau}}(x_i^t)$ .
3.2 Overview of the method
The overview of the proposed STAR is shown in Fig. 1. After training at each step  $t - \tau$ , the prototype of each foreground class  $l \in L^{t - \tau}$  and essential statistics are computed and saved. At a subsequent step  $t$ , the input to the classifiers includes not only the features extracted from the input image but also proportionally replayed prototypes of all past foreground classes as well as repetitions of the features in the background region, aligning the sample distribution with the training set accumulated up to this step,  $D^{1:t}$ . Moreover, we employ the OCFM loss,  $\mathcal{L}_{ocfm}$ , between the features  $f_{i}^{t}$  and  $f_{i}^{t,t-1}$  to preserve old-class features and apply the SAD loss,  $\mathcal{L}_{sad}$ , to differentiate the features between similar old-new class pairs. Consequently, our total loss function is:
$$
\mathcal {L} = \mathcal {L} _ {m b c e} + \alpha \mathcal {L} _ {o c f m} + \beta \mathcal {L} _ {s a d}. \tag {1}
$$
Here,  $\mathcal{L}_{mbce}$  denotes a multiple binary cross-entropy (mBCE) loss with a positive weight of  $\gamma$  as in [1, 3], which is proved to be more suitable for CISS than a multi-class cross-entropy loss.  $\mathcal{L}_{ocfm}$  and  $\mathcal{L}_{sad}$  denote the OCFM and SAD losses, respectively, balanced by hyperparameters  $\alpha$  and  $\beta$ .
3.3 Prototype preservation and replay
As previously stated, in a single-step training set, the proportion of pixels from current foreground classes is excessively high, resulting in biased predictions. To address this issue, we aim to comprehensively reconstruct the class distribution of training samples at each step  $t$ , aligning it with the distribution in the union of all training sets up to this step  $D^{1:t}$ . This distribution represents the achievable limit, as it is not possible to know the data from future steps in practice, and it should also be close to the distribution in the complete dataset accumulated up to this step. The most straightforward solution to this problem is to save and replay all past images, but this does not meet the requirements of continual learning, necessitating an alternative solution.
We argue that the feature extractor is relatively generalized in extracting features and does not contain information strongly correlated with the classes. The ability of SSUL [3] to learn new classes even
if the feature extractor parameters are frozen supports this claim. As a result, the network's bias primarily stems from the classifiers. Therefore, we propose creating a prototype for each learned foreground class and replaying these prototypes in subsequent steps, allowing them to go through the classifiers together with the features extracted from input images. This process can correct the foreground class distribution accessed by the classifiers, eliminating its bias. Meanwhile, we have to ensure that each prototype contains the most representative characteristics of its corresponding foreground class with minimal storage cost, thereby efficiently maximizing the correction for the classifiers.
Specifically, after training at each step  $t - \tau$ , we calculate the number of pixels from each current foreground class  $l \in L^{t - \tau}$  in the single-step training set  $D^{t - \tau}$ :
$$
\eta_ {l} = \sum_ {i = 1} ^ {N ^ {t - \tau}} \sum_ {j = 1} ^ {h \times w} \mathbb {1} \left\{\tilde {y} _ {i, j} ^ {t - \tau} = l \right\}. \tag {2}
$$
Here,  $j$  indicates the spatial location,  $\mathbb{1}$  denotes the indicator function.  $\tilde{y}_i^{t - \tau} \in \mathbb{R}^{h \times w}$  represents the downsampled label aligned with the spatial size of the features  $f_i^{t - \tau}$  that fed into the classifiers. It is given by  $\tilde{y}_i^{t - \tau} = DS(y_i^{t - \tau})$ , where  $DS$  denotes a downsampling operation. Since each class is treated as a foreground class in only one step,  $\eta_l$  is the count of pixels from class  $l$  appearing as foreground throughout the entire dataset, which determines the frequency of replaying the prototype of class  $l$  in subsequent steps. Still, due to the possible presence of a small number of  $l$  pixels appearing in other steps' training sets as background, we utilize the previous model to estimate the number of  $l$  pixels in the current training set to fine-tune  $\eta_l$ . Yet, as this aspect has minimal influence on the final outcomes, we deem it optional and provide clarification in the supplementary material.
Next, in order to build replay samples of class  $l$  in future steps, we calculate the feature center of  $l$  to capture its primary characteristics by:
$$
\mu_ {l} = \frac {\sum_ {i = 1} ^ {N ^ {t - \tau}} \sum_ {j = 1} ^ {h \times w} \left(\hat {f} _ {i , j} ^ {t - \tau} \times \mathbb {1} \{\tilde {y} _ {i , j} ^ {t - \tau} = l \}\right)}{\left\| \sum_ {i = 1} ^ {N ^ {t - \tau}} \sum_ {j = 1} ^ {h \times w} \left(\hat {f} _ {i , j} ^ {t - \tau} \times \mathbb {1} \{\tilde {y} _ {i , j} ^ {t - \tau} = l \}\right) \right\| _ {2}} \in \mathbb {R} ^ {c}, \tag {3}
$$
where  $\otimes$  denotes the element-wise multiplication,  $\| \cdot \| _2$  computes the L2-norm, and  $\hat{f}_{i,j}^{t - \tau}$  refers to the L2-normalized feature vector given by  $\hat{f}_{i,j}^{t - \tau} = \frac{f_{i,j}^{t - \tau}}{\left\|f_{i,j}^{t - \tau}\right\|_2}$ .  $\mu_l$  indicates the mean direction of all feature vectors for  $l$ , serving as a prototype of it that embodies the typical characteristic. However, relying solely on this prototype during the replay in subsequent steps may result in inadequately robust training outcomes, as a prototype only provides a single representation of class features without encompassing any variability. Therefore, we also compute the standard deviation of the feature vectors to introduce noise during the replay process. The calculation is given by:
$$
\sigma_ {l} = \sqrt {\frac {1}{\eta_ {l}} \sum_ {i = 1} ^ {N ^ {t - \tau}} \sum_ {j = 1} ^ {h \times w} \left[ \left(\hat {f} _ {i , j} ^ {t - \tau} - \mu_ {l}\right) ^ {2} \times \mathbb {1} \left\{\tilde {y} _ {i , j} ^ {t - \tau} = l \right\} \right]} \in \mathbb {R} ^ {c}. \tag {4}
$$
In addition, since we remove the length information of feature vectors through L2-normalization, we need to restore it during replay. Hence, we also store the mean and standard deviation of the L2-norms of all features for class  $l$ , denoted as  $\mu_{l,norm}$  and  $\sigma_{l,norm}$ , which are formulated by:
$$
\mu_ {l, n o r m} = \frac {1}{\eta_ {l}} \sum_ {i = 1} ^ {N ^ {t - \tau}} \sum_ {j = 1} ^ {h \times w} \left(\left\| \hat {f} _ {i, j} ^ {t - \tau} \right\| _ {2} \times \mathbb {1} \{\tilde {y} _ {i, j} ^ {t - \tau} = l \}\right) \in \mathbb {R} ^ {1}, \tag {5}
$$
$$
\sigma_ {l, n o r m} = \sqrt {\frac {1}{\eta_ {l}} \sum_ {i = 1} ^ {N ^ {t - \tau}} \sum_ {j = 1} ^ {h \times w} \left[ \left(\left\| \hat {f} _ {i , j} ^ {t - \tau} \right\| _ {2} - \mu_ {l , n o r m}\right) ^ {2} \times \mathbb {1} \left\{\tilde {y} _ {i , j} ^ {t - \tau} = l \right\} \right]} \in \mathbb {R} ^ {1}. \tag {6}
$$
With  $\eta_l, \mu_l, \sigma_l, \mu_{l,norm}$  and  $\sigma_{l,norm}$  saved, robust replay in future steps will be available.
At a subsequent step  $t$ , the classifier would originally be fed only the features extracted from the input image,  $f_{i}^{t}$ . To correct the ratio between old and new foreground classes, we also input replayed old
class samples into the classifier. For an old class  $l \in L^{1:t-1}$ , we use a random variable  $r_l$  to represent the replayed sample, which is the product of two Gaussian-distributed random variables, the feature vector direction  $\kappa \sim \mathcal{G}(\mu_l, \sigma_l^2)$  and the L2-norm  $\lambda \sim \mathcal{G}(\mu_{l,\text{norm}}, \sigma_{l,\text{norm}}^2)$ :
$$
r _ {l} = \kappa \times \lambda . \tag {7}
$$
For each old class  $l$ , we replay  $r_l$  for  $\eta_l$  times per training epoch, which represents the number of pixels from  $l$  that appear outside  $D^t$ . In each epoch, all foreground class pixels in  $D^t$  also appear once, so replaying  $\eta_l$  times can align the ratio of old and new foreground class samples with the ratio in  $D^{1:t}$ . Assuming each epoch contains  $e$  iterations, we distribute the  $\eta_l$  times evenly among these iterations. This implies sampling  $r_l$  for  $\frac{\eta_l}{e}$  times in each iteration and concatenating these random samples with  $f_i^t$  to feed the classifiers for calculating the total gradient to update parameters. In this way, the bias towards any foreground classes is eliminated. Simultaneously, as each sampling of  $r_l$  yields different features, and the distribution is controlled by stored standard deviations, the classifiers can capture sufficient potential characteristics of old classes, achieving robust discriminability.
3.4 Background repetition
Apart from the ratio between samples of old and new foreground classes, we have to also restore the proportion of background samples to fully rectify the distribution. However, due to the richer semantics contained in the background and the background shift across different steps, condensing its meaning through a single prototype is impossible. Therefore, we choose to repeat background pixels in the current training set  $D^{t}$  with a proper frequency to restore their proportion.
Similar to the strategy in Sec. 3.3, to control the number of background repetitions in future steps, we calculate the number of appeared background pixels at each step  $t - \tau$ . As numerous background pixels exist in all steps, we not only count the number of background pixels in the single-step training set but also add the background pixel counts saved in previous steps, given by: