# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 4 Evaluations
### Evaluation Results  
In order to evaluate the effectiveness of the proposed model, measures such as _precision_, _recall_, _accuracy_, and _F1 score_ are used. The definitions of _precision_, _recall_, _accuracy_, and _F1 score_ are as follows:  
\[Precision=\frac{TP}{TP+FP} \tag{12}\]  
\[Recall=\frac{TP}{TP+FN} \tag{13}\]  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{14}\]  
\[F1 score=\frac{2\times recall\times precision}{precision+recall} \tag{15}\]  
where \(TP\) is true positive, \(TN\) is true negative, \(FP\) is false positive, and \(FN\) is false negative. Precision refers to the proportion of samples that the model predicts as positive examples and that are actually positive examples. Recall refers to the proportion of samples that are actually positive examples and are correctly predicted as positive examples by the model. The F1 score is the harmonic mean of precision and recall, which is used to comprehensively evaluate the performance of the model on imbalanced datasets. The range of the F1 score is between 0 and 1, the closer to 1, the better the model performance. For a detailed description of these evaluation indicators, please refer to [39]. Table 3 shows the precision, recall, and F1 score of the proposed method, as well as the comparison methods, on the SCAN and BreakHis datasets. The **Model** column represents the base model used to generate soup ingredients and the **Strategy** column represents the strategies used for model training. LR\(=0.0001\) represents the results obtained by fine-tuning the model using a constant learning rate of 0.0001. LR-ensemble represents the majority voting ensemble of models trained with different LRs, i.e., 0.0001, 0.0015, and 0.0002. LR-Aver-Soup represents the average soup model, where its ingredients are models trained using the aforementioned different learning rates. CLR-Aver-Soup (SWA) represents the model obtained using the random weight-averaging method [16]. CLR-DDWA-Ensemble represents the ensemble model, which is obtained by selecting the checkpoint models using CLR through the DDWA strategy. CLR-DDWA-Soup represents the average soup model, which is obtained by selecting the checkpoint models using CLR through the DDWA strategy. It can be seen from Table 3 that the model obtained by training with different but fixed LR values demonstrates a relatively similar performance across each evaluation index. Ensemble models trained by different LRs can further improve the performance, showing that the models have diversity. LR-Aver-Soup outperforms LR-Ensemble with a precision improvement ratio of 3.9%. When comparing CLR-Aver-Soup(SWA) and CLR-DDWA-Soup, it can be seen that the latter has better performance, showing the effectiveness of the proposed DDWA ingredient model selection strategy. Figure 5 shows the confusion matrices of the models based on ResNet. The values on the diagonal (green) of the confusion matrix indicate the number of correctly classified samples in each class and their proportions relative to the entire dataset. The values in red cells represent the number of misclassified samples and their proportions of the entire dataset. In the last row, the upper value represents the recall of each class. The last column represents the precision corresponding to each class. The most bottom right values represent the accuracy and error rates of the model. It can be seen that the model does not discriminate well between class 1 and class 4 since, for each model, there are samples from class 4 that are misclassified as class 1. Compared with the other models, the CLR-DDWA-Soup model performs better mainly because it has better classification performance in classes 2, 3, and 5, and fewer misclassified samples in class 1 when compared to the other models. Compared with the constant learning rate models, the ensemble-based model (LR-Ensemble) demonstrates a large improvement in the classification accuracy of class 1. We can see that the ensemble model can make good use of the advantages of the three base models.