# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 4 Evaluations
### Evaluation Results
For CLR-DDWA-Soup, the soup model based on VGG achieves the best performance. The reason may be that the features of the training images in the SCAN dataset are relatively salient and can be effectively extracted via a relatively standard convolutional neural network. In addition, the soup model based on ResNet and DenseNet demonstrates relatively close classification results, which are greatly improved compared to the standard SWA model and the three models with constant learning rates. The highest increase rate reached 8.78%. Figure 10 shows the receiver operating characteristic (ROC) curves of the models on the BreakHis dataset. It is a curve drawn according to a series of different two-classification methods (cut-off value or determination threshold), with the true positive rate (sensitivity) representing the ordinate and the false positive rate representing the abscissa. The ROC curve can easily determine the ability of a model to recognize samples at a certain threshold. We can see that CLR-DDWA-Soup achieves the largest area under the curve (AUC) among all methods. The larger the AUC, the higher the sensitivity; the lower the misclassification rate, the better the performance of the model. As indicated by Figure 1, averaging the weights of the ingredient models that are located on the edge of the error basin and that have large diversity can result in a model whose loss is closer to the expected value of the error basin. We interpolate the weights of the two ingredient models to obtain a path through the error basin according to Equation (8). The interpolation parameter \(\gamma\) is set to \(\{0,\frac{1}{4},\frac{1}{3},\frac{1}{2},\frac{2}{3},\frac{2}{1}\}\). The accuracy of the model with interpolated weights at each point is recorded, as shown in Figure 11. The x-axis coordinates represent various values of \(\gamma\) and the y-axis coordinates represent the model accuracies. We can see from Figure 11 that for each specific model category (ResNet, VGG, DenseNet), the error variation of the weight interpolation model on the two datasets is always kept within a small fluctuation range, i.e., \(\tau\leq 0.017\). This result confirms the conclusion that the model selected by the DDWA strategy is located at the edge of the error basin, which shows its effectiveness. Finally, we present the results of the Friedman test conducted to evaluate the performance of the methods. The Friedman test [40] is a non-parametric test method that uses rank to realize whether there are significant differences in multiple population distributions. The principle hypothesis is that there is no significant difference in multiple population distributions from multiple paired samples. In the field of machine learning, the Friedman test is often used to compare the performances of multiple algorithms, which can verify whether there are significant differences among multiple learning algorithms as a whole. Figure 10: ROC curves of the models on the BreakHis dataset. Let \(I\) and \(J\) denote the numbers of methods and datasets. In the Friedman test, we first calculate the average accuracy of each algorithm running on different datasets, denoted as \(V=(v_{ij})\), where \(v_{ij}\) represents the accuracy of the \(i\)th model on the \(j\)th dataset. We then sort \(v_{ij}\) in descending order for each dataset and determine the rank values of each algorithm. So far, the Friedman test involves converting the table representing the original experimental results, denoted as \(V=(v_{ij})\), into the rank table \(R=(r_{ij})\). In this study, there are 3 base models, 9 methods, and 2 datasets used for the evaluation. We performed the Friedman test on each dataset with different base models. Hence, the size of the ranking matrix \(R\) is \((9,3)\), i.e., \(I=3\times 9\) and \(J=2\). The original hypothesis is that the performances of \(I\) different algorithms are similar.