## 3 Materials and Methods
### A Multi-Stage Regularization Method to Realize Fair Classification  
In our CIL process, when training the current classes, our network does not have access to the former class instances, for which the network tends to classify former class instances as the current classes. In other words, this extreme sample imbalance leads to a significant decline in the classification performance of the CIL process. For the last fully connected layer operation, which is also the classifier, its operation is as follows:  
\[\overset{\rightarrow}{\mathbf{y}}=\mathbf{W}\overset{\rightarrow}{\mathbf{ f}}+\left(\overset{\rightarrow}{\mathbf{b}}\right) \tag{6}\]  
where \(y\) is the classification result; \(W\) and \(b\) are the weight and the bias of the fully connected layer. In this paper, the bias of the classifier is ignored. (6) can also be expressed in a vector form, which makes it easier to understand the operation process of the classifier. \[y_{k}=\overset{\rightarrow}{\mathbf{w}_{k}}\bullet\overset{\rightarrow}{ \mathbf{f}} \tag{7}\]  
where \(\overset{\rightarrow}{\mathbf{w}_{k}}\) is the weight vector at \(k\)-th row of \(\mathbf{W}\). \(1\leq k\leq H\) represents different class within \(H\) classes. For convenience, \(\overset{\rightarrow}{\mathbf{w}_{k}}\) is called the weight of classifier \(k\), and \(y_{k}\) is the output of the k-th classifier. (7) can also be written as  
\[y_{k}=\left|\overset{\rightarrow}{\mathbf{w}_{k}}\right|\left|\overset{ \rightarrow}{\mathbf{f}}\right|\cos\theta \tag{8}\]  
where \(\theta\) is the angle between \(\overset{\rightarrow}{\mathbf{w}_{k}}\) and \(\overset{\rightarrow}{\mathbf{f}}\). Now we can analyze these three elements in CIL process, separately. In the incremental learning process, as demonstrated by previous studies on CIL, the impact of sample imbalance is almost unavoidable when there is no access to older instances. Neural network performs classification based on the learnt abstract features, with parameters optimized autonomously. In this case, in the last fully connected layer,  
Figure 4: Schematic diagram of feature space distribution in CIL process: (**a**,**b**) demonstrate the changes in feature clustering observed during the network fitting process, while (**c**,**d**) are the schematic diagrams of the changes in feature clustering during CIL process obtained by our derivation. the network will ultimately achieve preference for the current classes by simultaneously adjusting the three elements on the right side of (8). As for the feature extractor, the scattering center feature which is a manually designed feature plays an important role. It makes the extracted feature less affected by the preference of deep networks for new samples. In this case, rather than saying the old knowledge is forgotten, it is better to say that the old knowledge is \"buried\" by the new knowledge. Therefore, the focus of this part is how to constrain the optimization process of the classifier during the training process and how to exploit the former knowledge as much as possible after the network training. This section adopts a strategy of constraining two of the three elements of (8) so that the impact of sample imbalance is exerted on the other one as much as possible. Then, through subsequent adjustments to this element, we can correct the bias of this element to mitigate the impact of sample imbalance after the network training. In a large number of CIL experiments, there is a common phenomenon [51] about the distributions of \(\left|\overset{\rightarrow}{\mathbf{w}}_{k}\right|\). We make a schematic diagram to illustrate this phenomenon as shown in Figure 5.  
As shown in Figure 5, when training 2 new classes, due to the sample imbalance, the \(l2-norm\) of the current class weights are obviously higher than the former classes, for which the outputs of the current classes (8-th and 9-th) classifiers are much higher than others even when the input instance belongs to the old classes.