# Feature Transformation-Based Few-Shot Class-Incremental Learning
## 3 Feature Transformation
### Feature Transformation  
In the scenario considered in this work, the target domain data which is defined as an ideal distribution consistent with the neural collapse phenomenon is not available. In our implementation, the feature transformation is designed to produce class prototypes that approximate the ETF structure implied by neural collapse. By encouraging equiangularity and reduced intra-class variance in the transformed space, we aim to better align the learned features with the theoretically optimal configurations. Therefore, we propose a trainable linear transformation method by introducing an additional feature transformation layer, which is trained after the feature extractor has been learned. Both the feature extractor and the transformation layer are then fixed for incremental learning and evaluation. Specifically, the feature transformation layer is implemented as a fully connected layer with its bias term set to zero, followed by an \(\ell_{2}\) normalization applied to the output. The input and output dimensions are both set to 512, consistent with the dimensionality of the feature extractor output. No Dropout or Batch Normalization is applied within the transformation layer. The weight matrix is initialized using a Gaussian distribution with zero mean and variance \(1/d\), where \(d=512\). The training process for the feature transformation layer is illustrated in Figure 2. The layer is initialized using a Gaussian distribution with zero mean and a variance equal to the reciprocal of the feature dimension. A series of few-shot tasks, each consisting of a support set and a query set, are sampled from the base classes. All samples are first passed through the feature extractor and then through the transformation layer. The transformed query samples are classified using cosine similarity with the transformed support samples. Since the support set is labeled, predictions for the query set can be made based on the similarity scores. Cross-entropy loss is used to supervise the predictions, and the parameters of the transformation layer are updated via gradient descent. Each sampled few-shot task follows a 15-way 1-shot setting, with 15 query samples per class used to construct the query set. The training process of the proposed feature transformation method is shown in Algorithm 1. ```
Data: Base dataset \(D^{(0)}\) Result: Feature transformation module \(g(\cdot)\)
1 Train a standard classification network using the empirical risk minimization in Equation (2);
2 Remove the fully connected layer to obtain the feature extractor;
3 whilethemaximum number of iterations is not reached and the loss is above a threshold do
4 Sample a few-shot task from \(D^{(0)}\), consisting of a support set \(S\) and a query set \(Q\);
5 Use the feature transformation module \(g(\cdot)\) to map data in \(S\) and \(Q\) to the new feature space;
6 In the transformed feature space, classify the query samples in \(Q\) based on cosine similarity to the support samples in \(S\);
7 Compute the classification loss and perform backpropagation to update the parameters of \(g(\cdot)\);
```  
**Algorithm 1**Training procedure of the feature transformation module using the base dataset  
The computational cost of Algorithm 1 is dominated by two parts: (1) feature extraction using the fixed backbone network (e.g., ResNet), and (2) the linear transformation module. In each iteration, the complexity of extracting features for the support and query samples is \(O((N_{S}+N_{Q})\cdot C)\), where \(C\) denotes the computational cost per sample of the convolutional backbone, and \(N_{S}\) and \(N_{Q}\) are the numbers of support and query samples, respectively. The linear transformation, implemented as a \(512\times 512\) fully connected layer, adds \(O((N_{S}+N_{Q})\cdot 512^{2})\) operations per iteration. Computing cosine similarities between every query-support pair requires \(O(N_{Q}\cdot N_{S}\cdot d)\) operations. Overall, for \(T\) iterations, the total training complexity is \(O(T\cdot((N_{S}+N_{Q})\cdot(C+512^{2})+N_{Q}N_{S}d))\).