# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 4 Evaluations
### Evaluation Results
Finally, the CLR-based soup models outperform the models with constant learning rates since their classification performances for class 1 are much better than the other models. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Strategy**} & \multicolumn{3}{c}{**SCAN**} & \multicolumn{3}{c}{**BreakHis**} \\ \cline{3-8}  & & **Precision** & **Recall** & **F1 Score** & **Precision** & **Recall** & **F1 Score** \\ \hline \multirow{8}{*}{ResNet} & LR = 0.0001 & 0.888 & 0.871 & 0.880 & 0.952 & 0.950 & 0.950 \\ \cline{2-8}  & LR = 0.00015 & 0.888 & 0.888 & 0.888 & 0.958 & 0.957 & 0.957 \\ \cline{2-8}  & LR = 0.0002 & 0.871 & 0.854 & 0.862 & 0.924 & 0.923 & 0.923 \\ \cline{2-8}  & LR-Ensemble & 0.877 & 0.871 & 0.874 & 0.958 & 0.957 & 0.957 \\ \cline{2-8}  & LR-Aver-Soup & 0.911 & 0.900 & 0.906 & 0.950 & 0.949 & 0.949 \\ \cline{2-8}  & CLR-Aver-Soup(SWA) & 0.901 & 0.896 & 0.898 & 0.957 & 0.956 & 0.956 \\ \cline{2-8}  & **CLR-DDWA-Ensemble** & **0.907** & **0.902** & **0.904** & **0.965** & **0.963** & **0.962** \\ \cline{2-8}  & **CLR-DDWA-Soup** & **0.929** & **0.926** & **0.931** & **0.979** & **0.978** & **0.978** \\ \hline \multirow{8}{*}{VGG19-BN} & LR = 0.0001 & 0.918 & 0.917 & 0.915 & 0.962 & 0.961 & 0.961 \\ \cline{2-8}  & LR = 0.00015 & 0.925 & 0.925 & 0.924 & 0.972 & 0.971 & 0.971 \\ \cline{2-8}  & LR = 0.0002 & 0.892 & 0.889 & 0.887 & 0.963 & 0.961 & 0.961 \\ \cline{2-8}  & LR-Ensemble & 0.923 & 0.921 & 0.921 & 0.979 & 0.978 & 0.978 \\ \cline{2-8}  & LR-Aver-Soup & 0.924 & 0.960 & 0.940 & 0.978 & 0.978 & 0.978 \\ \cline{2-8}  & CLR-Aver-Soup(SWA) & 0.914 & 0.948 & 0.929 & 0.970 & 0.969 & 0.969 \\ \cline{2-8}  & **CLR-DDWA-Ensemble** & **0.915** & **0.913** & **0.912** & **0.979** & **0.978** & **0.978** \\ \cline{2-8}  & **CLR-DDWA-Soup** & **0.947** & **0.969** & **0.958** & **0.985** & **0.984** & **0.984** \\ \hline \multirow{8}{*}{DenseNet} & LR = 0.0001 & 0.905 & 0.906 & 0.904 & 0.925 & 0.921 & 0.920 \\ \cline{2-8}  & LR = 0.00015 & 0.916 & 0.917 & 0.915 & 0.918 & 0.917 & 0.917 \\ \cline{1-1} \cline{2-8}  & LR = 0.0002 & 0.893 & 0.894 & 0.892 & 0.926 & 0.922 & 0.921 \\ \cline{1-1} \cline{2-8}  & LR-Ensemble & 0.919 & 0.918 & 0.916 & 0.929 & 0.924 & 0.924 \\ \cline{1-1} \cline{2-8}  & LR-Aver-Soup & 0.928 & 0.925 & 0.925 & 0.937 & 0.934 & 0.934 \\ \cline{1-1} \cline{2-8}  & CLR-Aver-Soup(SWA) & 0.905 & 0.904 & 0.903 & 0.956 & 0.956 & 0.956 \\ \cline{1-1} \cline{2-8}  & **CLR-DDWA-Ensemble** & **0.916** & **0.915** & **0.913** & **0.960** & **0.959** & **0.959** \\ \cline{1-1} \cline{2-8}  & **CLR-DDWA-Soup** & **0.931** & **0.931** & **0.931** & **0.971** & **0.971** & **0.971** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Precision, recall, and F1 score on the SCAN and BreakHis datasets. Figure 5: Confusion matrix of the models based on ResNet. (**a**) CLR-Aver-Soup (SWA); (**b**) CLR-DDWA-Ensemble; (**c**) CLR-DDWA-Soup; (**d**) LR = 0.0001; (**e**) LR = 0.00015; (**f**) LR = 0.0002; (**g**) LR-Aver-Soup; (**h**) LR-Ensemble. We also provide the top 8 ranking ingredient models as well as their accuracy and diversity values, as shown in Table 4.  
In Tables 4 and 5, the first column represents the names of ingredient models. The second and third columns represent the accuracy and diversity values of each ingredient model, respectively. Moreover, the last three columns represent the ranks of accuracy, diversity, and the final ranking. The final ranking is obtained by simply adding \(\mathbf{Rank_{Acc}}\) and \(\mathbf{Rank_{Div}}\). We can see from Table 4 that the proposed DDWA method selects ingredient models based on the combination of accuracy and diversity. From the last line of Table 4, it can be seen that even the \(11\  
Figures 8 and 9 show the accuracies of the SCAN and BreakHis datasets for ensemble and soup models based on different ingredient models, i.e., ResNet, VGG, and DenseNet. It can be seen that no single model can achieve the best classification performance in all ingredient models.