# SAR Target Incremental Recognition Based on Hybrid Loss Function and Class-Bias Correction
## 3 Methodology
### Bias Correction Layer for Class Imbalance  
Many recent studies [12; 21; 29; 31; 32; 36] have found that the imbalance between old and new classes will lead to bias in the weights of the fully connected layer. The bias refers to the incremental model being more inclined to the new classes. This is because the model is trained on unbalanced dataset, where the model has accessed to many samples from the new task. However, there are few samples from the older classes from earlier times. Although the forgetting prevention mechanisms, such as old sample preservation and knowledge distillation, have been employed, the model prefers newer classes. A direct result found by [PERSON] et al. [28] is that the classifier norms for new classes are larger than old classes. The classifiers are biased towards new classes.  
This paper adopts a simple and practical method to solve the problem of class imbalance, namely a simple bias correction layer is added after the last fully connected layer of the model. It applies a linear function model with two parameters to achieve this function. The linear function is chosen because it is a simple function that does not introduce more hyperparameters to the model and is fast to train. Keep the logits of the output of the old classes \((1,2,\ldots,n)\) and use this linear function model to correct the bias of the output logits of the new classes \((n+1,\ldots,n+m)\):  
\[q_{k} =as(x)+b \tag{9}\] \[s(x) =logit(x)=W^{T}f(x) \tag{10}\]  
where \(a\) and \(b\) are the parameters of the complementary task deviations and \(s(x)\) is the output logits, represented by (10). The bias parameters \(a\) and \(b\) are shared by all new classes, with \(a=1\), \(b=0\) under the initial non-incremental phase.  
For the parameter \(a\), inspired by [37], in the incremental phase, it is computed by normalizing the weight vector in the fully connected layer. Firstly, the weights \(W\) are modified in the fully connected layer in the following form:  
\[W =(W_{0},\ W_{n}) \tag{11}\] \[W_{0} =(w_{1},\ w_{2},\ \ldots,\ w_{n})\in\mathbb{R}^{d\times n}\] (12) \[W_{n} =(w_{n+1},\ w_{n+2},\ \ldots,\ w_{n+m})\in\mathbb{R}^{d\times n} \tag{13}\]  
Then the weight vector parametrization for the old classes and the new classes are calculated separately:  
\[Norm_{0} =(||w_{1}||,\ldots||w_{n}||) \tag{14}\] \[Norm_{n} =(||w_{n+1}||,\ldots,||w_{n+m}||) \tag{15}\]  
Based on the above representation, the parameter \(a\) is calculated by:  
\[a=\frac{Mean(Norm_{0})}{Mean(Norm_{n})} \tag{16}\]  
where \(Mean(\cdot)\) denotes the mean value of the elements in the vector. This will make the average norm of new and old classes weight vectors equal. It is worth noting that the relative magnitudes of the weight vector vanes do not change in the new or old classes. Because we only change the average parametrization and make it equal. The critical point of this design is to enable better separation of data within the old and new classes.  
For the parameter \(b\), it is optimized on the transformed logits \(q_{k}\) by the representative samples preserved in the old sample preservation step. The representative samples set is a balanced dataset with an equal number of samples from the old and new classes. Because of its small size, it is more appropriate to choose a linear function model with fewer and simpler parameters to correct for bias. More importantly, a similar approach to ours, BIC [29], introduces a separate validation set for this linear function model, while we only use the representative samples set for the knowledge retrospection process. Furthermore, the BIC approach is highly dependent on the validation set, which is only effective for the performance of large-scale dataset. When the validation set is small, it leads to a significant performance degradation. In this paper, the parameter \(b\) is optimized by cross-entropy loss, as follows:  
\[L_{b}=-\sum_{k}d_{y=k}log[softmax(q_{k})] \tag{17}\]  
where \(y\) is the label of the ground-truth class, and \(d_{y=k}\) is the indicator function of \(y=k\).