Title: SELECT: SELEctive Context Transfer for Class Incremental Semantic Segmentation
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Knowledge Transfer, Class Incremental Semantic Segmentation

Abstract: Class-Incremental Semantic Segmentation (CISS) is fundamentally challenged by catastrophic forgetting and background shift, where learning new concepts degrades performance on previously seen classes. While existing methods attempt to balance stability (retaining old knowledge) and plasticity (learning new knowledge), they often fail to leverage prior knowledge effectively. These approaches typically rely on indiscriminate knowledge transfer or ambiguous initializations, which can dilute crucial semantic information. To overcome this limitation, we propose SELECT, a novel approach for Selective Context Transfer. SELECT intelligently transfers knowledge by focusing on inter-class semantic similarity. Its core is a Context Transfer Attention mechanism that identifies semantically related past classes and selectively transfers their contextual information to guide the learning of new ones. To ensure this transfer does not corrupt the original representations, we introduce a controlled perturbation and novel discriminative loss that explicitly enforces separation between the latent spaces of the new class and its influential old-class counterparts. Extensive experiments on PASCAL VOC 2012 and ADE20K demonstrate that SELECT significantly mitigates catastrophic forgetting and background shift, providing a robust and effective solution to the stability-plasticity dilemma.