# Feature Transformation-Based Few-Shot Class-Incremental Learning
## 1 Introduction  
Incremental learning refers to methods capable of continuously processing streams of information from the real world, while not only acquiring new knowledge but also retaining, integrating, and potentially optimizing previously learned knowledge [1]. Class-incremental learning is a form of incremental learning that aims to train a unified classifier capable of recognizing newly encountered classes over time [2]. To mitigate the problem of forgetting previously learned knowledge, class-incremental learning commonly employs knowledge distillation techniques, often supported by a small memory buffer that stores exemplars from past classes to guide the loss function during training [3]. The general form of the loss function is as follows [3]:  
\[l(D^{t},P;\theta)=l_{CE}(D^{t},P;\theta)+vl_{DL}\left(\sum_{i=1}^{t-1}D^{i},P; \theta\right), \tag{1}\]  
where \(l_{DL}\) denotes the loss term for knowledge distillation, and \(l_{CE}\) represents the cross-entropy loss for the classifier. The variable \(t\) indicates the current training iteration orgeneration, \(P\) refers to the set of exemplars sampled from the known class set \(D_{i}\), and \(\theta\) denotes the parameters of the neural network. The implementation of the distillation term \(l_{DL}\) varies across different studies. [PERSON] et al. [4] proposed iCaRL, a pioneering method for incremental learning that addresses the problem of catastrophic forgetting in neural networks. iCaRL uses a nearest-mean-of-exemplars classifier, a K-Nearest Neighbors (KNN)-inspired approach that assigns test samples to the closest class mean in feature space, enabling efficient classification. Combined with a strategy for selecting and storing representative exemplars, iCaRL allows models to learn new classes over time while retaining performance on previously learned ones, without requiring access to all past data. [PERSON] et al. [3] introduced an end-to-end incremental learning approach that jointly optimizes classification and distillation losses to mitigate catastrophic forgetting. Unlike methods that decouple feature learning from classification, their framework updates the entire model in an end-to-end manner, allowing the representation and classifier to evolve together as new classes are introduced. This design enables more flexible and adaptive class-incremental learning over time. [PERSON] et al. [5] and [PERSON] et al. [6] both addressed the prediction bias issue that arises in class-incremental learning, where the classifier tends to favor newly introduced classes due to the class imbalance between current task data and limited exemplars from past tasks. [PERSON] et al. [5] proposed a rebalancing strategy that modifies the output layer by replacing the standard dot product with cosine similarity, which reduces the bias towards new classes by normalizing feature magnitudes. In contrast, [PERSON] et al. [6] introduced a bias correction model that post-processes the output logits, learning a set of bias parameters from a held-out validation set to calibrate predictions. These approaches reflect two complementary perspectives: architectural adjustment at the classifier level and output-level calibration via post hoc correction, both aiming to achieve a more balanced and unified classifier in incremental settings. Traditional class-incremental learning methods are not well-suited for few-shot scenarios. There are two main challenges [7; 8]: first is the imbalance between new and old classes, where conventional strategies typically require a large amount of data to mitigate this issue--an assumption that does not hold in few-shot settings where data is scarce; second, balancing the distillation loss and the classification loss becomes more difficult, which often leads to suboptimal performance. To address incremental learning under few-shot conditions, researchers have proposed new methods tailored to these constraints. [PERSON] et al. [9] abandoned the use of distillation loss and introduced a topology-preserving network based on the Neural Gas algorithm.