Original: Decomposed Knowledge Distillation for Class-incremental Semantic Segmentation
Revised: Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation

Original: Abstract
Revised: Abstract

Original: Class-incremental semantic segmentation (CISS) labels each pixel of an imagewith a corresponding object/stuff class continually.
Revised: Class-incremental semantic segmentation (CISS) labels each pixel of an image with a corresponding object/stuff class continually.

Original: To this end, it is crucial tolearn novel classes incrementally without forgetting previously learned knowledge.
Revised: To this end, it is crucial to learn novel classes incrementally without forgetting previously learned knowledge.

Original: Current CISS methods typically use a knowledge distillation (KD) technique forpreserving classiﬁer logits, or freeze a feature extractor, to avoid the forgettingproblem.
Revised: Current CISS methods typically use a knowledge distillation (KD) technique for preserving classiﬁer logits, or freeze a feature extractor, to avoid the forgetting problem.

Original: The strong constraints, however, prevent learning discriminative featuresfor novel classes.
Revised: The strong constraints, however, prevent learning discriminative features for novel classes.

Original: We introduce a CISS framework that alleviates the forgettingproblem and facilitates learning novel classes effectively.
Revised: We introduce a CISS framework that alleviates the forgetting problem and facilitates learning novel classes effectively.

Original: We have found thata logit can be decomposed into two terms.
Revised: We have found that a logit can be decomposed into two terms.

Original: They quantify how likely an inputbelongs to a particular class or not, providing a clue for a reasoning process of amodel.
Revised: They quantify how likely an input belongs to a particular class or not, providing a clue for a reasoning process of a model.

Original: The KD technique, in this context, preserves the sum of two terms ( i .
Revised: The KD technique, in this context, preserves the sum of two terms ( i .

Original: e .,a class logit), suggesting that each could be changed and thus the KD does notimitate the reasoning process.
Revised: e ., a class logit), suggesting that each could be changed and thus the KD does not imitate the reasoning process.

Original: To impose constraints on each term explicitly, wepropose a new decomposed knowledge distillation (DKD) technique, improvingthe rigidity of a model and addressing the forgetting problem more effectively.
Revised: To impose constraints on each term explicitly, we propose a new decomposed knowledge distillation (DKD) technique, improving the rigidity of a model and addressing the forgetting problem more effectively.

Original: Wealso introduce a novel initialization method to train new classiﬁers for novel classes.
Revised: We also introduce a novel initialization method to train new classiﬁers for novel classes.

Original: In CISS, the number of negative training samples for novel classes is not sufﬁcientto discriminate old classes.
Revised: In CISS, the number of negative training samples for novel classes is not sufﬁcient to discriminate old classes.

Original: To mitigate this, we propose to transfer knowledgeof negatives to the classiﬁers successively using an auxiliary classiﬁer, boostingthe performance signiﬁcantly.
Revised: To mitigate this, we propose to transfer knowledge of negatives to the classiﬁers successively using an auxiliary classiﬁer, boosting the performance signiﬁcantly.

Original: Experimental results on standard CISS benchmarksdemonstrate the effectiveness of our framework.
Revised: Experimental results on standard CISS benchmarks demonstrate the effectiveness of our framework.

Original: Our code will be available online.

Original:  Introduction
Revised:  Introduction

Original: A general way of learning knowledge for neural networks is to tune network weights with examplesfor all object/scene classes at hand.
Revised: A general way of learning knowledge for neural networks is to tune network weights with examples for all object/scene classes at hand.

Original: After ﬁnishing the learning process, the weights are normally ﬁxedfor inference, suggesting that the current learning paradigm is not ﬂexible enough to handle novelclasses unseen at training time.
Revised: After ﬁnishing the learning process, the weights are normally ﬁxed for inference, suggesting that the current learning paradigm is not ﬂexible enough to handle novel classes unseen at training time.

Original: Fine-tuning the weights with additional examples for novel classes addresses the problem in part, but this causes catastrophic forgetting [18].
Revised: Fine-tuning the weights with additional examples for novel classes addresses the problem in part, but this causes catastrophic forgetting [19].

Original: Namely, neural networks rather forget the previously learned knowledge in order to learn new information.
Revised: Namely, neural networks rather forget the previously learned knowledge in order to learn new information.

Original: Class-incrementallearning (CIL) targets to learn novel object/scene classes continually using training samples for thoseclasses only, while minimizingcatastrophically forgetting the knowledge.
Revised: Class-incremental learning (CIL) targets to learn novel object/scene classes continually using training samples for those classes only, while minimizing the forgetting problem.

Original: A key to CIL is to designa learning method that balances between rigidity and plasticity of a model [19].
Revised: A key to CIL is to design a learning method that balances between rigidity and plasticity of a model [20].

Original: On the one hand,network weights should not be altered abruptly in learning new information from novel classes, inorder to preserve the discriminative ability for old ones ( i .
Revised: On the one hand, network weights should not be altered abruptly in learning new information from novel classes, in order to preserve the discriminative ability for old ones ( i .

Original: e ., rigidity), avoiding catastrophic forgetting.
Revised: e ., rigidity), avoiding catastrophic forgetting.

Original: On the other hand, a strong rigidity rather distracts learning knowledge from novel classes.
Revised: On the other hand, a strong rigidity rather distracts learning knowledge from novel classes.

Original: Thenetwork weights should thus be tuned accordingly ( i .
Revised: The network weights should thus be tuned accordingly ( i .

Original: e ., plasticity)
Revised: e ., plasticity).

Original: Class-incremental semantic segmentation (CISS) adopts a CIL paradigm for the task of semantic segmentation.
Revised: Class-incremental semantic segmentation (CISS) adopts a CIL paradigm for the task of semantic segmentation.

Original: CISS methods [5, 9, 20] typically exploit a softmax cross-entropy (CE) term along with knowledge distillation (KD)
Revised: CISS methods [5, 9, 21, 29] typically exploit a softmax cross-entropy (CE) term along with knowledge distillation (KD)

Original: Although the CE term helps to learn novel classes, applyingthe softmax function to all classes, including both old and novel ones, lowers class probabilities of oldones.
Revised: Although the CE term helps to learn novel classes, applying the softmax function to all classes, including both old and novel ones, lowers class probabilities of old ones.

Original: This in turn prevents the model from preserving knowledge learned from old classes, resultingin catastrophic forgetting [1, 22].
Revised: This in turn prevents the model from preserving knowledge learned from old classes, resulting in catastrophic forgetting [1, 23].

Original: The KD technique prevents changing network weights drastically,alleviating the forgetting problem.
Revised: The KD technique prevents changing network weights drastically, alleviating the forgetting problem.

Original: Recently, SSUL [6] proposes to use multiple binary crossentropy (BCE) terms for individual novel classes separately.
Revised: Recently, SSUL [6] proposes to use multiple binary crossentropy (BCE) terms for individual novel classes separately.

Original: This approach handles the forgettingproblem caused by the softmax function, but it is limited in the following: (1) A feature extractor isfrozen in order to enforce the rigidity of the model.
Revised: This approach handles the forgetting problem caused by the softmax function, but it is limited in the following: (1) A feature extractor is frozen in order to enforce the rigidity of the model.

Original: This strong constraint for preserving knowledgefor old classes makes it hard to learn discriminative features for novel classes.
Revised: This strong constraint for preserving knowledge for old classes makes it hard to learn discriminative features for novel classes.

Original: (2) An initializationtechnique for classiﬁers requires an off-the-shelf saliency detector [15], which is computationallydemanding.
Revised: (2) An initialization technique for classiﬁers requires an off-the-shelf saliency detector [15], which is computationally demanding.

Original: In this paper, we present a simple yet effective CISS framework that overcomes the aforementionedproblems.
Revised: In this paper, we present a simple yet effective CISS framework that overcomes the aforementioned problems.

Original: To achieve better plasticity and rigidity for a CISS model, we propose to train a featureextractor, and introduce a decomposed knowledge distillation (DKD) technique.
Revised: To achieve better plasticity and rigidity for a CISS model, we propose to train a feature extractor, and introduce a decomposed knowledge distillation (DKD) technique.

Original: KD encourages amodel to predict logits similar to the ones obtained from an old model.
Revised: KD encourages a model to predict logits similar to the ones obtained from an old model.

Original: We have found that logitscan be represented as the sum of positive and negative reasoning scores that quantify how likely andunlikely an input belongs to a particular class, respectively.
Revised: We have found that logits can be represented as the sum of positive and negative reasoning scores that quantify how likely and unlikely an input belongs to a particular class, respectively.

Original: In this context, KD focuses on preservingthe relative difference between positive and negative reasoning scores only, without considering thechange of each score.
Revised: In this context, KD focuses on preserving the relative difference between positive and negative reasoning scores only, without considering the change of each score.

Original: The DKD technique imposes explicit constraints on each reasoning score,instead of class logits themselves, which is beneﬁcial to improving the rigidity of a CISS modeltogether with KD effectively.
Revised: The DKD technique imposes explicit constraints on each reasoning score, instead of class logits themselves, which is beneﬁcial to improving the rigidity of a CISS model together with KD effectively.

Original: We also propose an initialization technique to train classiﬁers fornovel classes effectively.
Revised: We also propose an initialization technique to train classiﬁers for novel classes effectively.

Original: Note that training samples for novel classes are available only for eachincremental step, suggesting that classiﬁers for the novel classes are trained with a small numberof negative samples.
Revised: Note that training samples for novel classes are available only for each incremental step, suggesting that classiﬁers for the novel classes are trained with a small number of negative samples.

Original: To address this, we propose to train an auxiliary classiﬁer in a current step,and use it to initialize classiﬁers for novel classes in a next step.
Revised: To address this, we propose to train an auxiliary classiﬁer in a current step, and use it to initialize classiﬁers for novel classes in a next step.

Original: To this end, we consider trainingsamples of a current step as potential negatives for novel classes in a next step.
Revised: To this end, we consider training samples of a current step as potential negatives for novel classes in a next step.

Original: We then train theauxiliary classiﬁer, such that all pixels in current training samples as negative ones, transferring priorknowledge of negative samples to the next step for the classiﬁers of novel classes.
Revised: We then train the auxiliary classiﬁer, such that all pixels in current training samples as negative ones, transferring prior knowledge of negative samples to the next step for the classiﬁers of novel classes.

Original: Our initializationtechnique also does not require any pre-trained models, e.g. , for saliency detection [6], in order todifferentiate possibly negative samples.
Revised: Our initialization technique also does not require any pre-trained models, e.g. , for saliency detection [6], in order to differentiate possibly negative samples.

Original: We demonstrate the effectiveness of our framework withextensive ablation studies on standard CISS benchmarks [11, 28].
Revised: We demonstrate the effectiveness of our framework with extensive ablation studies on standard CISS benchmarks [11, 30].

Original: We summarize main contributionsof our work as follows:
Revised: We summarize main contributions of our work as follows:

Original: • We introduce a simple yet effective CISS framework that exploits a novel DKD and multiple BCEterms, achieving a good trade-off between rigidity and plasticity.• We present a novel initialization technique that encodes prior knowledge for negatives to trainclassiﬁers for novel classes effectively.•
Revised: • We introduce a simple yet effective CISS framework that exploits a novel DKD and multiple BCE terms, achieving a good trade-off between rigidity and plasticity. • We present a novel initialization technique that encodes prior knowledge for negatives to train classiﬁers for novel classes effectively.

Original:  We achieve a new state of the art on standard benchmarks for CISS [11, 28], and demonstrate theeffectiveness of our approach through extensive experiments and ablation studies.
Revised: • We achieve a new state of the art on standard benchmarks for CISS [11, 30], and demonstrate the effectiveness of our approach through extensive experiments and ablation studies.

Original:  Related work 
Revised:  Related work 2.1 Class incremental image classiﬁcation

Original: 2.1 Class incremental image classiﬁcation

Original: Many CIL methods [2, 4, 8, 10, 16, 26] have been proposed for image classiﬁcation, attempting topreserve the discriminative ability for old classes.
Revised: Many CIL methods [2, 4, 8, 10, 16, 27] have been proposed for image classiﬁcation, attempting to preserve the discriminative ability for old classes.

Original: Since training samples of novel classes are availableonly in incremental steps, they typically adopt a KD [14] technique to retain classiﬁer logits for oldclasses.
Revised: Since training samples of novel classes are available only in incremental steps, they typically adopt a KD [14] technique to retain classiﬁer logits for old classes.

Original: For example, the works of [8, 10] additionally apply the technique to intermediate featuremaps from a feature extractor.
Revised: For example, the works of [8, 10] additionally apply the technique to intermediate feature maps from a feature extractor.

Original: LwM [8] proposes an attention-based distillation loss to preservevisual information of old classes.
Revised: LwM [8] proposes an attention-based distillation loss to preserve visual information of old classes.

Original: PODNet [10] introduces a spatial-based distillation technique thatencourages pooled feature maps between current and previous steps to be similar to each other.
Revised: PODNet [10] introduces a spatial-based distillation technique that encourages pooled feature maps between current and previous steps to be similar to each other.

Original: CILmethods [2, 4, 16, 26] exploiting an external memory have recently been introduced.
Revised: CIL methods [2, 4, 16, 27] exploiting an external memory have recently been introduced.

Original: They store asubset of training samples for old classes for re-training, which is effective to alleviate catastrophicforgetting.
Revised: They store a subset of training samples for old classes for re-training, which is effective to alleviate catastrophic forgetting.

Original: Due to the data imbalance between old and novel classes, they generally use trainingtricks, such as re-training with a balanced subset of training samples [4, 26] or re-balancing classiﬁerweights [2, 16], which is however computationally expensive and requires additional memory.

Original: Notethat all the aforementioned methods [2, 4, 8, 10, 16, 26] exploit a softmax CE term to learn novelclasses.

Original: Similar to ours, the seminal work of [24] uses BCE losses along with KD for CIL.

Original: Differently,we use a novel DKD loss together with a novel initialization technique specialized for CISS. person car bicycle car bicycle
Revised: Due to the data imbalance between old and novel classes, they generally use training person car bicycle car bicycle

Original: Step
Revised: Step

Original: Step
Revised: Step

Original: Initialize person bicycle sheep person bicycle Initialize
Revised: Initialize person bicycle sheep person bicycle Initialize

Original: Initialize ignoring gradient (a) Training.
Revised: Initialize ignoring gradient (a) Training.

Original: person car bicycle car bicycle
Revised: person car bicycle car bicycle

Original: Step
Revised: Step

Original: Step
Revised: Step

Original: Initialize person bicycle sheep person bicycle Initialize
Revised: Initialize person bicycle sheep person bicycle Initialize

Original: Initialize
Revised: Initialize

Original: Step (b) Initialization.
Revised: Step (b) Initialization.

Revised: tricks, such as re-training with a balanced subset of training samples [4, 27] or re-balancing classiﬁer weights [2, 16], which is however computationally expensive and requires additional memory.

Revised: Note that all the aforementioned methods [2, 4, 8, 10, 16, 27] exploit a softmax CE term to learn novel classes.

Revised: Similar to ours, the seminal work of [25] uses BCE losses along with KD for CIL.

Revised: Differently, we use a novel DKD loss together with a novel initialization technique specialized for CISS.

Original: 2.2 Class incremental semantic segmentation
Revised: 2.2 Class incremental semantic segmentation

Original: Similar to class-incremental classiﬁcation, CISS methods [5, 9] generally adopt the KD techniqueto alleviate catastrophic forgetting.
Revised: Similar to class-incremental classiﬁcation, CISS methods [5, 9] generally adopt the KD technique to alleviate catastrophic forgetting.

Original: For example, MiB [5] applies the technique to pixel-wiseclassiﬁcation scores.
Revised: For example, MiB [5] applies the technique to pixel-wise classiﬁcation scores.

Original: PLOP [9] employs KD to intermediate feature maps, and introduces a local PODloss, specially designed for CISS, by extending the vanilla version in [10].
Revised: PLOP [9] employs KD to intermediate feature maps, and introduces a local POD loss, specially designed for CISS, by extending the vanilla version in [10].

Original: In addition, SDR [21]recently proposes to leverage contrastive learning, complementary to existing KD techniques, toseparate feature clusters, making it easier to learn novel classes.
Revised: In addition, SDR [22] recently proposes to leverage contrastive learning, complementary to existing KD techniques, to separate feature clusters, making it easier to learn novel classes.

Original: These methods also exploit a softmax
Revised: These methods also exploit a softmax

Original: CE term to learn novel classes.
Revised: CE term to learn novel classes.

Original: Different from image classiﬁcation, supervisory signals for CISSare given in pixel-level labels only for regions corresponding to novel classes.
Revised: Different from image classiﬁcation, supervisory signals for CISS are given in pixel-level labels only for regions corresponding to novel classes.

Original: The regions forbackground and old classes are thus unlabeled, suggesting that we have limited information to trainclassiﬁers for CISS using a softmax CE loss.
Revised: The regions for background and old classes are thus unlabeled, suggesting that we have limited information to train classiﬁers for CISS using a softmax CE loss.

Original: To tackle this issue, several works [9, 21] propose togenerate pseudo labels from an old model to provide auxiliary supervisory signals for the unlabeledregions.
Revised: To tackle this issue, several works [9, 22] propose to generate pseudo labels from an old model to provide auxiliary supervisory signals for the unlabeled regions.

Original: Following the work of [24], exploiting multiple BCE losses for CISS is recently introducedto train classiﬁers for novel classes individually [6].
Revised: Following the work of [25], exploiting multiple BCE losses for CISS is recently introduced to train classiﬁers for novel classes individually [6].

Original: This approach enables training a model withoutauxiliary supervisory signals for unlabeled regions.
Revised: This approach enables training a model without auxiliary supervisory signals for unlabeled regions.

Original: It however freezes a feature extractor to alleviatecatastrophic forgetting, constraining the plasticity of classiﬁers for novel classes excessively.
Revised: It however freezes a feature extractor to alleviate catastrophic forgetting, constraining the plasticity of classiﬁers for novel classes excessively.

Original: On thecontrary, our method trains both a feature extractor and classiﬁers, together with novel DKD andinitialization techniques, achieving a better trade-off in terms of plasticity and rigidity.
Revised: On the contrary, our method trains both a feature extractor and classiﬁers, together with novel DKD and initialization techniques, achieving a better trade-off in terms of plasticity and rigidity.

Original:  Approach
Revised:  Approach

Original: We train a CISS model continually with training samples D t at each training step t , where t ∈{ 1 , ..., T } and T is a total number of steps.
Revised: We train a CISS model continually with training samples D t at each training step t , where t ∈ { 1 , ..., T } and T is a total number of steps.

Original: The training dataset D t contains pairs of an imageand a corresponding ground-truth mask y .
Revised: The training dataset D t contains pairs of an image and a corresponding ground-truth mask y .

Original: We denote by C t and C 1: t − 1 sets of novel and oldobject/stuff classes at the step t , respectively.
Revised: We denote by C t and C 1: t − 1 sets of novel and old object/stuff classes at the step t , respectively.

Original: Note that the sets are disjoint, i .
Revised: Note that the sets are disjoint, i .

Original: e ., C 1:
Revised: e ., C 1:

Original: t − 1 ∩ C t = ∅ .
Revised: t − 1 ∩ C t = ∅ .

Original: Note also that regions for background and old classes are labeled as unknown in a current step t .
Revised: Note also that regions for background and old classes are labeled as unknown in a current step t .

Original: Namely, a ground-truth label at position i , y ( i ) , is either one of the classes in C t or the unknownclass c u , i .
Revised: Namely, a ground-truth label at position i , y ( i ) , is either one of the classes in C t or the unknown class c u , i .

Original: e .,
Revised: e .,

Original: y ( i )
Revised: y ( i )

Original: ∈
Revised: ∈

Original: C t ∪ { c u } , at the step t .
Revised: C t ∪ { c u } , at the step t .

Original: 3.1 Overview
Revised: 3.1 Overview

Original: We show in Fig.
Revised: We show in Fig.

Original: 1 an overview of our framework.
Revised: 1 an overview of our framework.

Original: For each step t , we train a CISS model togetherwith an auxiliary classiﬁer H t .
Revised: For each step t , we train a CISS model together with an auxiliary classiﬁer H t .

Original: Our framework mainly consists of a feature extractor F t and a set of classiﬁers G t predicting pixel-level semantic labels for previous and novel classes at thestep t (Fig. 1(a)).
Revised: Our framework mainly consists of a feature extractor F t and a set of classiﬁers G t predicting pixel-level semantic labels for previous and novel classes at the step t (Fig. 1(a)).

Original: In a next step t + 1 , we initialize a feature extractor F t +1 and classiﬁers for oldclasses C 1: t with the previous ones from the step t to preserve knowledge for old classes.
Revised: In a next step t + 1 , we initialize a feature extractor F t +1 and classiﬁers for old classes C 1: t with the previous ones from the step t to preserve knowledge for old classes.

Original: Classiﬁersfor novel classes at the step t + 1 and a new auxiliary classiﬁer H t +1 are initialized with the previousone H t (Fig. 1(b)).
Revised: Classiﬁers for novel classes at the step t + 1 and a new auxiliary classiﬁer H t +1 are initialized with the previous one H t (Fig. 1(b)).

Original: We exploit four loss terms for training (Fig. 1(a)): Multiple binary cross-entropy (mBCE), knowledgedistillation (KD), decomposed knowledge distillation (DKD), and auxiliary classiﬁer (AC) losses.
Revised: We exploit four loss terms for training (Fig. 1(a)): Multiple binary cross-entropy (mBCE), knowledge distillation (KD), decomposed knowledge distillation (DKD), and auxiliary classiﬁer (AC) losses.

Original: The ﬁrst three terms are used to train a CISS model at every step, while the last one is for the auxiliaryclassiﬁer H t .
Revised: The ﬁrst three terms are used to train a CISS model at every step, while the last one is for the auxiliary classiﬁer H t .

Original: Speciﬁcally, the mBCE term encourages the model to learn knowledge from novelclasses.
Revised: Speciﬁcally, the mBCE term encourages the model to learn knowledge from novel classes.

Original: The KD and DKD terms, on the other hand, help to preserve the discriminative ability forold classes.
Revised: The KD and DKD terms, on the other hand, help to preserve the discriminative ability for old classes.

Original: The AC term enables transferring knowledge of negatives to the next step for classiﬁersof novel classes.
Revised: The AC term enables transferring knowledge of negatives to the next step for classiﬁers of novel classes.

Original: 3.2 Training
Revised: 3.2 Training

Original: We train our framework using an objective as follows:
Revised: We train our framework using an objective as follows:

Original: [Equation]
Revised: [Equation]

Original: where L mbce , L kd , L dkd , and L ac are mBCE, KD, DKD, and AC terms, respectively, balanced by thehyperparameters of α and β .
Revised: where L mbce , L kd , L dkd , and L ac are mBCE, KD, DKD, and AC terms, respectively, balanced by the hyperparameters of α and β .

Original: In the following, we describe each term in detail.mBCE loss.
Revised: In the following, we describe each term in detail. mBCE loss.

Original: Given an input image, we ﬁrst obtain a feature map f t at a step t .
Revised: Given an input image, we ﬁrst obtain a feature map f t at a step t .

Original: We then computeclass logits, z t ∈ R HW ×|C 1: t | , with classiﬁers G t , where H and W are the height and width of theinput image, respectively, and | · | is the cardinality of a given set.
Revised: We then compute class logits, z t ∈ R HW ×|C 1: t | , with classiﬁers G t , where H and W are the height and width of the input image, respectively, and | · | is the cardinality of a given set.

Original: Concretely, the class logit isobtained by computing the dot product between features and weights for corresponding classiﬁers,followed by adding a bias:
Revised: Concretely, the class logit is obtained by computing the dot product between features and weights for corresponding classiﬁers, followed by adding a bias:

Original: [Equation]
Revised: [Equation]

Original: where we denote by w t ( c ) and b t ( c ) weights and a bias of a classiﬁer for a class c , respectively, andf t ( i ) is a feature at position i .
Revised: where we denote by w t ( c ) and b t ( c ) weights and a bias of a classiﬁer for a class c , respectively, and f t ( i ) is a feature at position i .

Original: To learn novel classes of C t , current CISS methods typically exploita CE loss computing softmax probabilities w.r.t all classes, C t and C 1: t − 1 , at a step t .
Revised: To learn novel classes of C t , current CISS methods typically exploit a CE loss computing softmax probabilities w.r.t all classes, C t and C 1: t − 1 , at a step t .

Original: This couldlower the softmax probabilities for the old classes C 1: t − 1 , causing catastrophic forgetting [1, 22].
Revised: This could lower the softmax probabilities for the old classes C 1: t − 1 , causing catastrophic forgetting [1, 23].

Original: Weinstead exploit a mBCE loss, and apply it to train classiﬁers for the novel classes C t only as follows:
Revised: We instead exploit a mBCE loss, and apply it to train classiﬁers for the novel classes C t only as follows:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: and 1 [ · ] is an indicator function that outputs 1 if the argument is true, and
Revised: and 1 [ · ] is an indicator function that outputs 1 if the argument is true, and

Original: 0 otherwise.
Revised: 0 otherwise.

Original: Following [3,27 ], we use a weighting strategy with a balance parameter of γ to handle the imbalance between twoterms in (3).
Revised: Following [3, 28], we use a weighting strategy with a balance parameter of γ to handle the imbalance between two terms in (3).

Original: KD loss.
Revised: KD loss.

Original: We adopt a KD term to prevent our model from changing abruptly, mitigating thecatastrophic forgetting problem, deﬁned as follows:
Revised: We adopt a KD term to prevent our model from changing abruptly, mitigating the catastrophic forgetting problem, deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: where p t − 1 is similarly computed as in (4) with z t − 1 , i .
Revised: where p t − 1 is similarly computed as in (4) with z t − 1 , i .

Original: e ., class logits predicted by a previous modelat the step t − 1 .
Revised: e ., class logits predicted by a previous model at the step t − 1 .

Original: This term encourages our model to provide class logits similar to the ones obtainedfrom the previous model, namely, z t ( i, c )
Revised: This term encourages our model to provide class logits similar to the ones obtained from the previous model, namely, z t ( i, c )

Original: ≈
Revised: ≈

Original: z t − 1 ( i, c ) , for old classes, c ∈ C 1: t − 1 , at the step t . 
Revised: z t − 1 ( i, c ) , for old classes, c ∈ C 1: t − 1 , at the step t . [Figure]

Original: DKD loss.
Revised: DKD loss.

Original: Based on that the dot product is the sum of element-wise multiplication between vectors,we decompose the class logit in (2) as follows:
Revised: Based on that the dot product is the sum of element-wise multiplication between vectors, we decompose the class logit in (2) as follows:

Original: [Equation]
Revised: [Equation]

Original: [Figure] where z + t ( i, c ) is the sum of positive elements chosen from the result of element-wise multiplicationbetween f t ( i ) and w t ( c ) , and z − t ( i, c ) is similarly deﬁned using negative elements (Fig.
Revised: where z + t ( i, c ) is the sum of positive elements chosen from the result of element-wise multiplication between f t ( i ) and w t ( c ) , and z − t ( i, c ) is similarly deﬁned using negative elements (Fig.

Original: 2 right).
Revised: 2 right).

Original: We omit the bias of the classiﬁer for ease of notation.
Revised: We omit the bias of the classiﬁer for ease of notation.

Original: We call z + t ( i, c ) and z − t ( i, c ) as positiveand negative reasoning scores , respectively, which quantify how likely and unlikely an input belongsto the class c .
Revised: We call z + t ( i, c ) and z − t ( i, c ) as positive and negative reasoning scores , respectively, which quantify how likely and unlikely an input belongs to the class c .

Original: In this context, the KD technique retains the relative difference between positive andnegative reasoning scores only, suggesting that each reasoning score itself is not preserved (Fig. 2left).
Revised: In this context, the KD technique retains the relative difference between positive and negative reasoning scores only, suggesting that each reasoning score itself is not preserved (Fig.left).

Original: For example, if one of the reasoning scores increases, the other one would decrease accordinglyin order to maintain the sum of reasoning scores, i .
Revised: For example, if one of the reasoning scores increases, the other one would decrease accordingly in order to maintain the sum of reasoning scores, i .

Original: e ., the class logit.
Revised: e ., the class logit.

Original: To address this problem, wepropose a decomposed knowledge distillation (DKD) loss as follows:
Revised: To address this problem, we propose a decomposed knowledge distillation (DKD) loss as follows:

Original: [Equation]
Revised: [Equation]

Original: (7) where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: (8) and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: L − dkd is similarly deﬁned using the negative reasoning score z − t .
Revised: L − dkd is similarly deﬁned using the negative reasoning score z − t .

Original: Note that L + dkd and L − dkd are analogous to the KD term in (5), but they compute losses with the reasoning scores, z + t and z − t , separately, instead of exploiting a class logit z
Revised: Note that L + dkd and L − dkd are analogous to the KD term in (5), but they compute losses with the reasoning scores, z + t and z − t , separately, instead of exploiting a class logit z

Original: t directly.
Revised: t directly.

Original: That is, the DKD term encourages our modelto provide positive and negative reasoning scores similar to the ones obtained from a previousmodel, i .
Revised: That is, the DKD term encourages our model to provide positive and negative reasoning scores similar to the ones obtained from a previous model, i .

Original: e .,
Revised: e .,

Original: z
Revised: z

Original: + t ( i, c ) ≈
Revised: + t ( i, c ) ≈

Original: z
Revised: z

Original: + t − 1 ( i, c ) and z − t ( i, c )
Revised: + t − 1 ( i, c ) and z − t ( i, c )

Original: ≈
Revised: ≈

Original: z − t − 1 ( i, c ) , for the old classes c ∈ C 1: t − 1 .
Revised: z − t − 1 ( i, c ) , for the old classes c ∈ C 1: t − 1 .

Original: Thisexplicit constraint on each reasoning score improves the rigidity of our model, enabling it to preservethe discriminative ability for old classes effectively.
Revised: This explicit constraint on each reasoning score improves the rigidity of our model, enabling it to preserve the discriminative ability for old classes effectively.

Original: AC loss.
Revised: AC loss.

Original: The dataset D t at a step t provides training images mainly depicting one of novel classes C t .
Revised: The dataset D t at a step t provides training images mainly depicting one of novel classes C t .

Original: This suggests that the number of negative samples for the novel classes C t , e .
Revised: This suggests that the number of negative samples for the novel classes C t , e .

Original: g ., images containingobjects for old classes C 1: t − 1 as well, would not be sufﬁcient in the dataset D t .
Revised: g ., images containing objects for old classes C 1: t − 1 as well, would not be sufﬁcient in the dataset D t .

Original: We conjecture thattraining samples in the step t can serve as good negative examples for novel classes C t +1 in the nextstep t + 1 , as C 1: t ∩ C t +1 = ∅ .
Revised: We conjecture that training samples in the step t can serve as good negative examples for novel classes C t +1 in the next step t + 1 , as C 1: t ∩ C t +1 = ∅ .

Original: Based on this, we propose to exploit an auxiliary classiﬁer H t toencode knowledge of the negatives for the novel classes C t +1 in advance of the step t + 1 .
Revised: Based on this, we propose to exploit an auxiliary classiﬁer H t to encode knowledge of the negatives for the novel classes C t +1 in advance of the step t + 1 .

Original: To thisend, we train the classiﬁer H t with the AC term as follows:
Revised: To this end, we train the classiﬁer H t with the AC term as follows:

Original: [Equation]
Revised: [Equation]

Original: (10) where p (cid:48) t ( i ) =
Revised: [Equation] where p (cid:48) t ( i ) =

Original: 1 / (1 + e − z (cid:48)
Revised: 1 / (1 + e − z (cid:48)

Original: t
Revised: t

Original: ( i ) ) , and z (cid:48) t ∈ R
Revised: ( i ) ) , and z (cid:48) t ∈ R

Original: HW × 1 is a logit from the auxiliary classiﬁer H t .
Revised: HW × 1 is a logit from the auxiliary classiﬁer H t .

Original: That is,the classiﬁer H t is trained to classify all pixels in the images of D t as negatives for the next step t + 1 .
Revised: That is, the classiﬁer H t is trained to classify all pixels in the images of D t as negatives for the next step t + 1 .

Original: Note that training images at a step t might contain objects or stuffs for novel classes in the future, butthe number of samples is negligibly small.
Revised: Note that training images at a step t might contain objects/stuff for novel classes in the future. However, we assume that noisy training signals in the step t can be compensated in the future by using sufﬁcient training images for those objects/stuff classes.

Original: Note also that the AC term is used to train the auxiliaryclassiﬁer H t only, and thus a feature extractor does not receive any gradients from this term.
Revised: Note also that the AC term is used to train the auxiliary classiﬁer H t only, and thus a feature extractor does not receive any gradients from this term.

Original: 3.3 Initialization
Revised: 3.3 Initialization

Original: At the beginning of a step t + 1 , we initialize our model, including a feature extractor and classiﬁers,using the model at the step t to transfer previously learned knowledge (Fig. 1(b)).
Revised: At the beginning of a step t + 1 , we initialize our model, including a feature extractor and classiﬁers, using the model at the step t to transfer previously learned knowledge (Fig. 1(b)).

Original: Note that classiﬁers for novel classes are newly added to predict logits for corresponding classes at the step t + 1 .
Revised: Note that classiﬁers for novel classes are newly added to predict logits for corresponding classes at the step t + 1 .

Original: Notealso that the number of negative samples for novel classes might not be sufﬁcient in a dataset D t +1 .
Revised: Note also that the number of negative samples for novel classes might not be sufﬁcient in a dataset D t +1 .

Original: In this context, exploiting a random initialization technique for the new classiﬁers is not effective,degrading the discriminative ability of classiﬁers for the negatives.
Revised: In this context, exploiting a random initialization technique for the new classiﬁers is not effective, degrading the discriminative ability of classiﬁers for the negatives.

Original: To address this, we propose toinitialize new classiﬁers using an auxiliary classiﬁer in a previous step, H t .
Revised: To address this, we propose to initialize new classiﬁers using an auxiliary classiﬁer in a previous step, H t .

Original: The auxiliary classiﬁer
Revised: The auxiliary classiﬁer

Original: H t contains prior knowledge of negatives for novel classes.
Revised: H t contains prior knowledge of negatives for novel classes.

Original: Concretely, we initialize classiﬁers G t +1in the step t + 1 , as follows:
Revised: Concretely, we initialize classiﬁers G t + in the step t + 1 , as follows:

Original: [Equation]
Revised: [Equation]

Original: where we denote by w (cid:48) t and b (cid:48) t weights and a bias of the auxiliary classiﬁer, respectively.
Revised: where we denote by w (cid:48) t and b (cid:48) t weights and a bias of the auxiliary classiﬁer, respectively.

Original: That is, theparameters of new classiﬁers for novel classes
Revised: That is, the parameters of new classiﬁers for novel classes

Original: C t +1 are initialized with the ones from the previousauxiliary classiﬁer H t , and those for other classiﬁers are initialized with the counterparts from theprevious step.
Revised: C t +1 are initialized with the ones from the previous auxiliary classiﬁer H t , and those for other classiﬁers are initialized with the counterparts from the previous step.

Original: 3.4 Inference
Revised: 3.4 Inference

Original: We predict pixel-level semantic labels at a step t , using class probabilities p t , as follows:
Revised: We predict pixel-level semantic labels at a step t , using class probabilities p t , as follows:

Original: [Equation]
Revised: [Equation]

Original: where τ is a threshold, empirically set to 0.5, and we denote by c bg a background class.
Revised: where τ is a threshold, empirically set to 0.5, and we denote by c bg a background class.

Original: We assign aclass label to each pixel, only when one of class probabilities for the pixel is at least larger than thethreshold τ .
Revised: We assign a class label to each pixel, only when one of class probabilities for the pixel is at least larger than the threshold τ .

Original: Otherwise, the pixel is assigned as a background class.
Revised: Otherwise, the pixel is assigned as a background class.

Original:  Experiments 
Revised:  Experiments 4.1 Implementation details

Original: 4.1 Implementation details

Original: Datasets.
Revised: Datasets.

Original: We use PASCAL VOC [11] and ADE20K [28] datasets for evaluation.
Revised: We use PASCAL VOC [11] and ADE20K [30] datasets for evaluation.

Original: PASCAL
Revised: PASCAL

Original: VOC [11] consists of 10 , 582 training and 1 , 449 validation images for 20 object and backgroundclasses.
Revised: VOC [11] consists of 10 , 582 training and 1 , 449 validation images for 20 object and background classes.

Original: ADE20K [28] provides 20 , 210 and 2 , 000 images for training and validation, respectively,with 150 object and stuff classes.
Revised: ADE20K [30] provides 20 , 210 and 2 , 000 images for training and validation, respectively, with 150 object and stuff classes.

Original: Following the protocol in [5], we use ofﬁcial validation splits forevaluation.
Revised: Following the protocol in [5], we use ofﬁcial validation splits for evaluation.

Original: We also exclude 20% of training sets, and use them to tune hyper-parameters.
Revised: We also exclude 20% of training sets, and use them to tune hyper-parameters.

Original: Experimental protocols.
Revised: Experimental protocols.

Original: We follow the experimental protocols in [5].
Revised: We follow the experimental protocols in [5].

Original: First, we evaluate our modelfor various incremental scenarios.
Revised: First, we evaluate our model for various incremental scenarios.

Original: Speciﬁcally, we split all object/stuff classes into base and novelones.
Revised: Speciﬁcally, we split all object/stuff classes into base and novel ones.

Original: We train the model for base classes in an initial step, and update it sequentially for novelclasses in each of the following training steps.
Revised: We train the model for base classes in an initial step, and update it sequentially for novel classes in each of the following training steps.

Original: We denote by ( N b - N n ) the incremental scenario,where N b and N n are the numbers of base and novel classes, respectively.
Revised: We denote by ( N b - N n ) the incremental scenario, where N b and N n are the numbers of base and novel classes, respectively.

Original: For example, given20 object classes in PASCAL VOC [11], for an incremental scenario of (15-1), we learn 15 baseclasses initially, and add a single novel class sequentially, which requires 6 training steps in total.
Revised: For example, given 20 object classes in PASCAL VOC [11], for an incremental scenario of (15-1), we learn 15 base classes initially, and add a single novel class sequentially, which requires 6 training steps in total.

Original: Second, we train our model under two conﬁgurations: Disjoint and overlapped settings.
Revised: Second, we train our model under two conﬁgurations: Disjoint and overlapped settings.

Original: The disjointsetting uses a unique set of training samples for each training step.
Revised: The disjoint setting uses a unique set of training samples for each training step.

Original: Training images in the set depictobject/stuff classes belonging to one of categories to learn in a current step.
Revised: Training images in the set depict object/stuff classes belonging to one of categories to learn in a current step.

Original: The disjoint settinghowever excludes the images if they have any pixels regarding novel classes to be presented in thefuture.
Revised: The disjoint setting however excludes the images if they have any pixels regarding novel classes to be presented in the future.

Original: The overlapped setting leverages all training images that contain at least a single instanceof classes to learn in a current step.
Revised: The overlapped setting leverages all training images that contain at least a single instance of classes to learn in a current step.

Original: Note that the overlapped setting is more realistic in practice,since the disjoint setting assumes that novel classes to learn in the future are known in advance ateach training step.
Revised: Note that the overlapped setting is more realistic in practice, since the disjoint setting assumes that novel classes to learn in the future are known in advance at each training step.

Original: We perform experiments on PASCAL VOC on both disjoint and overlappedsettings with incremental scenarios of (19-1), (15-5), and (15-1).
Revised: We perform experiments on PASCAL VOC on both disjoint and overlapped settings with incremental scenarios of (19-1), (15-5), and (15-1).

Original: For ADE20K [28], we evaluateour model under the overlapped setting only, with the scenarios of (100-50), (100-10), and (50-50),following [6, 9]. 
Revised: For ADE20K [30], we evaluate our model under the overlapped setting only, with the scenarios of (100-50), (100-10), and (50-50), following [6, 9]. [Table] [Table]

Original: Training.
Revised: Training.

Original: We use the DeepLabV3 [7] architecture using ResNet-101 [13] pretrained on ImageNet [25] as a backbone network.
Revised: We use the DeepLabV3 [7] architecture using ResNet-101 [13] pretrained on ImageNet [26] as a backbone network.

Original: Following [5, 6, 9, 21], we adopt different training strategiesfor each dataset.
Revised: Following [5, 6, 9, 22], we adopt different training strategies for each dataset.

Original: For PASCAL VOC [11], we train our model with 60 epochs for both initial andincremental steps, with a batch size of 32 .
Revised: For PASCAL VOC [11], we train our model with 60 epochs for both initial and incremental steps, with a batch size of 32 .

Original: We adopt a polynomial learning rate scheduler, wherelearning rates are set to 0 .
Revised: We adopt a polynomial learning rate scheduler, where learning rates are set to 0 .

Original: 001 and 0 .
Revised: 001 and 0 .

Original: 0001 for initial and incremental steps, respectively.
Revised: 0001 for initial and incremental steps, respectively.

Original: We empirically set γ to 2 during an initial step and 1 for others.
Revised: We empirically set γ to 2 during an initial step and 1 for others.

Original: For ADE20K [28], we train our model[Table] [Table] for 100 epochs with a batch size of 24 .
Revised: For ADE20K [30], we train our model for 100 epochs with a batch size of 24 .

Original: Following [6], we adopt the poly learning rate schedulerwith a linear warm-up [12], where learning rates are set to 0 .
Revised: Following [6], we adopt the poly learning rate scheduler with a linear warm-up [12], where learning rates are set to 0 .

Original: 0025 for an initial step and 0 . 00025for incremental ones, respectively.
Revised: 0025 for an initial step and 0 .for incremental ones, respectively.

Original: We set γ to 35 for all training steps.
Revised: We set γ to 35 for all training steps.

Original: For both datasets, we adoptthe SGD optimizer with momentum of 0 . 9 , and set α and β to 5 .
Revised: For both datasets, we adopt the SGD optimizer with momentum of 0 . 9 , and set α and β to 5 .

Original: We implement our model using
Revised: We implement our model using

Original: PyTorch [23] and train it with four NVIDIA RTX A5000 GPUs.
Revised: PyTorch [24] and train it with four NVIDIA RTX A5000 GPUs.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: Following [5, 6, 9, 21], we report mIoU b , mIoU n , and mIoU all scores, that is,intersection-over-union (IoU) scores averaged over base, novel and all classes, respectively.
Revised: Following [5, 6, 9, 22], we report mIoU b , mIoU n , and mIoU all scores, that is, intersection-over-union (IoU) scores averaged over base, novel and all classes, respectively.

Original: Simplyaveraging the IoU score over all classes ( i .
Revised: Simply averaging the IoU score over all classes ( i .

Original: e ., mIoU all ) is not appropriate to evaluate the performanceof CISS models, especially for the case that the number of novel classes is relatively small comparedto that of base ones.
Revised: e ., mIoU all ) is not appropriate to evaluate the performance of CISS models, especially for the case that the number of novel classes is relatively small compared to that of base ones.

Original: Accordingly, we also report a harmonic mean (hIoU) of mIoU b and mIoU nscores, which is less susceptible to the imbalance between base and novel classes. 
Revised: Accordingly, we also report a harmonic mean (hIoU) of mIoU b and mIoU n scores, which is less susceptible to the imbalance between base and novel classes. [Figure] [Figure]

Original: 4.2 Results
Revised: 4.2 Results

Original: We show in Tables 1 and 2 quantitative comparisons between ours and state-of-the-art CISS methods [5, 6, 9, 21] on PASCAL VOC [11] and ADE20K [28], respectively.
Revised: We show in Tables 1 and 2 quantitative comparisons between ours and state-of-the-art CISS methods [5, 6, 9, 22] on PASCAL VOC [11] and ADE20K [30], respectively.

Original: To better demonstrate theeffectiveness of our approach, we also report results for joint training that serve as upper bounds.
Revised: To better demonstrate the effectiveness of our approach, we also report results for joint training that serve as upper bounds.

Original: Forfair comparison with SSUL-M [6] exploiting an external memory, we also report the results (Ours-M)obtained using previous training samples, following the ofﬁcial implementation provided by theauthors.
Revised: For fair comparison with SSUL-M [6] exploiting an external memory, we also report the results (Ours-M) obtained using previous training samples, following the ofﬁcial implementation provided by the authors.

Original: Qualitative results can be found in the supplement.

Original: We can observe from Tables 1 and 2 three things: (1) CISS methods using a BCE term [6] to learnnovel classes, including ours, perform better than other approaches [5, 9, 21] employing a softmax
Revised: We can observe from Tables 1 and 2 three things: (1) CISS methods using a BCE term [6] to learn novel classes, including ours, perform better than other approaches [5, 9, 22] employing a softmax

Original: CE term, demonstrating the drawback of the softmax function for CISS.
Revised: CE term, demonstrating the drawback of the softmax function for CISS.

Original: (2) Among competitive[Figure] [Table] methods without using an external memory, our method achieves a new state of the art for all casesin terms of mIoU all and hIoU scores.
Revised: (2) Among competitive methods without using an external memory, our method achieves a new state of the art for all cases in terms of mIoU all and hIoU scores.

Original: This suggests that ours preserves knowledge learned frombase classes, while learning novel ones effectively, compared to other methods, achieving a bettercompromise between rigidity and plasticity for CISS.
Revised: This suggests that ours preserves knowledge learned from base classes, while learning novel ones effectively, compared to other methods, achieving a better compromise between rigidity and plasticity for CISS.

Original: We can also see that our method outperformsothers in terms of a hIoU score for all cases by a signiﬁcant margin.
Revised: We can also see that our method outperforms others in terms of a hIoU score for all cases by a signiﬁcant margin.

Original: The external memory providescomplementary information, and this brings additional performance gains.
Revised: The external memory provides complementary information, and this brings additional performance gains.

Original: Our method (Ours-M)clearly outperforms SSUL-M in terms of mIoU all and hIoU scores.
Revised: Our method (Ours-M) clearly outperforms SSUL-M in terms of mIoU all and hIoU scores.

Original: Note that SSUL and SSUL-Malso exploit an off-the-shelf saliency detector [15], pretrained on additional training samples [17],which requires more computational power and memory for training. 
Revised: Note that SSUL and SSUL-M also exploit an off-the-shelf saliency detector [15], pretrained on additional training samples [17], which requires more computational power and memory for training. [Table] [Figure]

Original: We also compare in Fig.
Revised: We also compare in Fig.

Original: 3 our method with the state of the art, including MiB [5], PLOP [9] and
Revised: 3 our method with the state of the art, including MiB [5], PLOP [9] and

Original: SSUL [6], over training steps in terms of mIoU. We can see that our method avoids catastrophicforgetting effectively, maintaining mIoU scores for both base and novel classes over a number ofsteps.
Revised: SSUL [6], over training steps in terms of mIoU. We can see that our method avoids catastrophic forgetting effectively, maintaining mIoU scores for both base and novel classes over a number of steps.

Original: In contrast, other methods often fail to preserve the mIoU scores of old classes in later steps.
Revised: In contrast, other methods often fail to preserve the mIoU scores of old classes in later steps.

Original: Except for the novel class at the fourth incremental step in Fig. 3(d), where SSUL [6] is slightly betterthan ours, our approach outperforms the state of the art for all training steps by a signiﬁcant margin. 
Revised: Except for the novel class at the fourth incremental step in Fig. 3(d), where SSUL [6] is slightly better than ours, our approach outperforms the state of the art for all training steps by a signiﬁcant margin. We show in Fig.

Revised: 4 qualitative results on the PASCAL VOC [11].

Revised: We can see that our method successfully learns novel classes incrementally without forgetting previously learned classes.

Revised: More qualitative results can be found in the supplement.

Original: 4.3 Discussion
Revised: 4.3 Discussion

Original: We show in Table 3 an ablation analysis of our approach.
Revised: We show in Table 3 an ablation analysis of our approach.

Original: The baseline model in the ﬁrst row usesmBCE and KD terms only.
Revised: The baseline model in the ﬁrst row uses mBCE and KD terms only.

Original: Note that the baseline already performs comparable to state-of-the-artmethods [5, 6, 9, 21], conﬁrming once more the signiﬁcance of employing the BCE loss for CISS.
Revised: Note that the baseline already performs comparable to state-of-the-art methods [5, 6, 9, 22], conﬁrming once more the signiﬁcance of employing the BCE loss for CISS.

Original: The last row shows that exploiting the DKD loss along with our initialization technique achieves thebest performance for all metrics.
Revised: The last row shows that exploiting the DKD loss along with our initialization technique achieves the best performance for all metrics.

Original: We further provide a detailed analysis for the two components inthe following.
Revised: We further provide a detailed analysis for the two components in the following.

Original: DKD.
Revised: DKD.

Original: From the ﬁrst and the second rows in Table 3, we can see that the DKD term boosts theperformance for both base and novel classes, as it helps to retain knowledge even after incrementalsteps.
Revised: From the ﬁrst and the second rows in Table 3, we can see that the DKD term boosts the performance for both base and novel classes, as it helps to retain knowledge even after incremental steps.

Original: Moreover, the second row shows that adopting the DKD term performs signiﬁcantly better,compared to freezing a feature extractor as in SSUL [6], in terms of mIoU b score (See the result in
Revised: Moreover, the second row shows that adopting the DKD term performs signiﬁcantly better, compared to freezing a feature extractor as in SSUL [6], in terms of mIoU b score (See the result in

Original: Table 1).
Revised: Table 1).

Original: This suggests that our DKD technique is more effective for maintaining the rigidity of a
Revised: This suggests that our DKD technique is more effective for maintaining the rigidity of a

Original: CISS model.
Revised: CISS model.

Original: We plot in Fig.
Revised: We plot in Fig.

Original: 4 numerical values of (cid:107) z
Revised: 5 numerical values of (cid:107) z

Original: + t − z + t − 1 (cid:107) , (cid:107) z − t − z − t − 1 (cid:107) , and (cid:107) z t − z t − 1 (cid:107)over iterations, where (cid:107) · (cid:107) is the L2 norm.
Revised: + t − z + t − 1 (cid:107) , (cid:107) z − t − z − t − 1 (cid:107) , and (cid:107) z t − z t − 1 (cid:107) over iterations, where (cid:107) · (cid:107) is the L2 norm.

Original: As positive and negative reasoning scores, z + t and z − t ,are similar to previous ones, z + t − 1 and z − t − 1 , respectively, (cid:107) z
Revised: As positive and negative reasoning scores, z + t and z − t , are similar to previous ones, z + t − 1 and z − t − 1 , respectively, (cid:107) z

Original: + t − z + t − 1 (cid:107) and (cid:107) z − t − z − t − 1 (cid:107) becomessmaller.
Revised: + t − z + t − 1 (cid:107) and (cid:107) z − t − z − t − 1 (cid:107) becomes smaller.

Original: We can see from Figs. 4(a) and (b) that the model trained without the DKD term does not[Figure] [Figure] preserve the reasoning scores effectively.
Revised: We can see from Figs. 5(a) and (b) that the model trained without the DKD term does not preserve the reasoning scores effectively.

Original: On the contrary, the DKD term prevents abrupt changesof the scores.
Revised: On the contrary, the DKD term prevents abrupt changes of the scores.

Original: Note that (cid:107) z t − z t − 1 (cid:107) quantiﬁes the consistency of predictions before and after anincremental step, suggesting that minimizing (cid:107) z t − z t − 1 (cid:107) is also crucial to alleviate catastrophicforgetting.
Revised: Note that (cid:107) z t − z t − 1 (cid:107) quantiﬁes the consistency of predictions before and after an incremental step, suggesting that minimizing (cid:107) z t − z t − 1 (cid:107) is also crucial to alleviate catastrophic forgetting.

Original: We can observe in Fig. 4(c) that the DKD term helps to further minimize (cid:107) z t − z t − 1 (cid:107) .
Revised: We can observe in Fig. 5(c) that the DKD term helps to further minimize (cid:107) z t − z t − 1 (cid:107) .

Original: These results verify that the DKD term improves the rigidity of a CISS model together with KD. 
Revised: These results verify that the DKD term improves the rigidity of a CISS model together with KD. [Figure]

Original: Initialization.
Revised: Initialization.

Original: The ﬁrst andthe third rows in Table 3 demonstrate that our initialization techniqueitself provides a considerable performance gain for novel classes in terms of the mIoU n score,verifying that properly initializing classiﬁer weights for novel classes is crucial for CISS.
Revised: The ﬁrst and third rows in Table 3 demonstrate that our initialization technique itself provides a considerable performance gain for novel classes in terms of the mIoU n score, verifying that properly initializing classiﬁer weights for novel classes is crucial for CISS.

Original: Theinitialization technique alleviates the problem, caused by a lack of negatives at each incrementalstep, and provides strong prior knowledge to learn novel classes.
Revised: The initialization technique alleviates the problem, caused by a lack of negatives at each incremental step, and provides strong prior knowledge to learn novel classes.

Original: Note that our model in the thirdrow already gives competitive results compared to SSUL [6] in terms of the mIoU n score, evenwithout exploiting an off-the-shelf saliency detector [15] for initialization.
Revised: Note that our model in the third row already gives competitive results compared to SSUL [6] in terms of the mIoU n score, even without exploiting an off-the-shelf saliency detector [15].

Original: We provide in Fig.
Revised: We provide in Fig.

Original: 5 visualcomparisons of activation maps for a novel class ( i .
Revised: 6 visual comparisons of activation maps for a novel class ( i .

Original: e ., train ), and segmentation labels for all targetclasses ( i .
Revised: e ., train ), and segmentation labels for all target classes ( i .

Original: e ., 15 base classes and the incremental ones of potted plant , sheep , sofa , train ).
Revised: e ., 15 base classes and the incremental ones of potted plant , sheep , sofa , train ).

Original: We obtainthe results using classiﬁers trained with and without our initialization technique.
Revised: We obtain the results using classiﬁers trained with and without our initialization technique.

Original: We can see thatthe classiﬁer without our initialization often activates incorrectly on background regions (Fig. 5(b)bottom) or previous classes ( i .
Revised: We can see that the classiﬁers without our initialization often activates incorrectly on background regions (Fig. 6(b) bottom) or previous classes ( i .

Original: e ., bus in Fig.
Revised: e ., bus in Fig.

Original: 5(b) top).
Revised: 6(b) top).

Original: This distracts classiﬁers for previous classes,resulting in incorrect semantic labels in the regions.
Revised: This distracts classiﬁers for previous classes, resulting in incorrect semantic labels in the regions.

Original: On the contrary, the classiﬁer initialized withour technique successfully suppresses false class probabilities for those regions, providing bettersegmentation results.
Revised: On the contrary, the classiﬁers initialized with our technique successfully suppresses false class probabilities for those regions, providing better segmentation results.

Original: Conclusion
Revised: Note that we train an auxiliary classiﬁer in advance of learning novel classes for incremental steps.

Original: We have presented a novel framework thatshows a good trade-off between rigidity and plasticity for CISS. In particular, we have introduced a new learning paradigm, decompose to distill knowledge,to improve therigidity, and have proposed a novel initialization technique to learn novel classesbetter.
Revised: This suggests that the initialization technique can be adopted only when segmentation models are trained from scratch, and thus pre-trained off-the-shelf segmentation models could not be exploited directly. Other techniques including mBCE and DKD in Sec.

Original: Finally, we have shown that our framework achieves a new state of the art on standard CISSbenchmarks.
Revised: 3.2 can be applied to the off-the-shelf models, which brings signiﬁcant performance gains, compared to current CISS methods [5, 9, 22, 29] (See Tables 1 and 3).

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: Note also that the auxiliary classiﬁer makes segmentation models aware of the fact that novel classes will be added in the future.

Original: [Yes] See Section 1. (b) Did you describe the limitations ofyour work? [Yes] See the supplemental material. (c) Did you discuss any potential negative societal impacts of your work?
Revised: This increment-aware continual learning achieves a new state of the art on standard CISS benchmarks with negligible computational overhead for training, and it will provide a novel paradigm for CIL.

Original: [N/A]

Original:  (d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] If you are including theoretical results...
Revised:  Conclusion

Original: (a) Did you state the full set of assumptions of all theoretical results? [N/A] (b) Did you include complete proofs of all theoretical results? [N/A]
Revised: We have presented a novel framework that shows a good trade-off between rigidity and plasticity for class-incremental semantic segmentation (CISS).

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [No] But our code
Revised: In particular, we have introduced a new learning paradigm, decompose to distill knowledge, to improve the rigidity, and have proposed a novel initialization technique to learn novel classes better.

Original: and models will be made publicly available online.

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,

Original: how they were chosen)? [Yes] See Section 4.1. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: Finally, we have shown that our framework achieves a new state of the art on standard CISS benchmarks.

Original: [Yes] See Section 4.2. (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4.1.
Revised: Acknowledgments and Disclosure of Funding

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4.
Revised: This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT)

Original: (b) Did you mention the license of the assets? [N/A] (c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
Revised: (No.RS-2022-00143524,

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?

Original: [N/A](e)

Original: Did you discuss whether the data you are using/curating contains personally identiﬁableinformation or offensive content?

Original: [N/A]5. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? [N/A]
Revised: Development of Fundamental Technology and Integrated Solution for Next-Generation Automatic

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation? [N/A]
Revised: Artiﬁcial Intelligence System) and the KIST Institutional Program (Project No.2E31051-21-203).