## 5 Discussion
### Experiments on Overfitting Mechanism
For class c, the cluster center of positive (\(mean_{c}\)) is calculated as follows:  
\[mean_{c}=\frac{1}{N_{c}}\sum\limits_{i=1}^{N_{c}}\overrightarrow{w_{c}} \bullet\overrightarrow{f_{i}} \tag{19}\]  
where \(N_{c}\) represents the number of instances that belongs to class c. Then, the mean standard deviation of the features belonging to all different classes is calculated as follows:  
\[\overline{std}=\frac{1}{N_{CLASS}}\sum\limits_{c\in Classes}\sqrt{\frac{\sum \limits_{i=1}^{N_{c}}\left(\overrightarrow{w_{c}}\bullet\overrightarrow{f_ {i}}-mean_{c}\right)}{N_{c}}} \tag{20}\]  
Since the samples are projected on the same coordinate axis, the standard deviation can represent not only the stability of feature clustering but also the mean distance between the projected features and the clustering center, which can be used to measure the density of features clustering. The training results with and without dropout are shown in Figure 13.  
The characteristics of Figure 13 resemble those of Figure 10. However, compared with Figure 10, the impact of removing dropout and \"late-stop\" in Figure 13 is even more evident, as the STD curves drop more significantly after around 10 epochs, and reach lower values than the STD with Dropout in the later training stages. This experiment result validates the derivation of our two methods from the perspectives of clustering stability and density. Figure 12: Schematic diagram of the projection of features onto the classifier weight vector. (**a**) Projection diagram of positive and negative features. (**b**) Projection diagram of multiple classes features on one certain classifier weight vector. In our specific experimental setting, the removal of dropout and the use of the \"late-stop\" strategy appear to improve feature clustering. Next, we conduct a CIL experiment to evaluate the effect of the overfitting strategy on CIL performance. Four methods are employed, as illustrated in Table 3, where \"\(\blackcheck\blackcheck\)\" represents the corresponding method is applied and \"-\" means that it is not. As shown in Figure 14, it can be seen that both \"Removing Dropout\" and \"Late-Stop\" lead to improvements in the performance of the CIL stage to varying degrees, among which only applying \"Late-Stop\" has the smallest improvement. It is worth noting that the highest classification accuracy at the basic stage is achieved at 99.147% when using the initial network model. The blue line with the best performance in subsequent CIL, which applies our two methods, is 98.540% at the basic stage. In other words, our overfitting method sacrifices recognition performance to some extent at the basic stage in exchange for improvement in CIL. Among the four models, the \"Dropout and Late Stop\" brings the smallest improvement. Its accuracy in the first and second stages is 88.955% and 84.920%, respectively, while the accuracy in the first and second stages of the initial method is 88.578% and 84.662%, respectively. However, the improvement brought by only removing the dropout is very obvious. In the first and second stages, the accuracy values are 93.099% and 87.827%, respectively. This phenomenon occurs because dropout slows down the network fitting process by probabilistically deactivating neurons, which is more effective than the impact of late-stop. However, comparing the gap between the blue and yellow lines with the gap  
Figure 13: Mean variance of feature clustering distance in feature projection space. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Line** & **Blue** & **Orange** & **Green** & **Red** \\ \hline Removing dropout & \(\blackcheck\blackcheck\) & \(\blackcheck\blackcheck\) & - & - \\ \hline “Late-stop” & \(\blackcheck\check\) & - & \(\blackcheck\check\) & - \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Four methods in the ablation experiment. between the green and red lines, it can be seen that the performance improvement brought by late stopping is different under different conditions.