# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 5 Discussion  
Our rationale for proposing that the model performs well can be attributed to the following factors. On the one hand, during the training of ingredient models, different hyperparameters are fully utilized. Averaging the model weights in different training stages can result in a model that is closer to the optimal solution. On the other hand, using the DDWA model selection strategy to rank and select the ingredient models, we can further improve the performance of the weight fusion model.  
Compared to ensemble learning and other model fusion strategies, the proposed model soup has the following advantages. Firstly, in terms of training and inference costs, model soup has certain advantages. In terms of training costs, the training strategy used by model soup is to fine-tune models with different training parameters. Each ingredient model is further trained on a pre-trained model, so the overall training cost is much smaller than starting from scratch. In terms of inference costs, the model soup approach involves averaging, selectively averaging, or weighted averaging the weights of all ingredient models, ultimately resulting in a single model. On the other hand, ensemble learning performs voting or selectively weighting the inference results of \(n\) different models, so the inference cost is \(n\) times that of model soup. Secondly, in terms of model interpretability, model soup requires the weight structure of the model to be consistent, and its optimal solution estimate is supported by the error basin theory of the loss functions, which has good interpretability. However, for ensemble learning strategies, the component models have different structures and mainly estimate the error bound of the ensemble model through the diversity and accuracy of the component models, so the interpretability is not as good as model soup.  
The proposed model has the following limitations. First, the premise of merging ingredient models in the weight space is that the weight structures of each model must be consistent, which limits the diversity of ingredient models, including structural diversity  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **ResNet** & **VGG** & **DenseNet** & **Average** \\ \hline LR = 0.0001 & 23.0/17.0/20.0 & 9.5/10.5/10.0 & 13.0/23.0/18.0 & 16.0 \\ LR = 0.00015 & 24.0/16.0/20.0 & 3.5/7.0/5.25 & 9.5/24.0/16.75 & 14.0 \\ LR = 0.0002 & 21.5/21.5/21.5 & 16.0/10.5/13.25 & 15.0/20.0/17.5 & 17.4 \\ LR-Ensemble & 21.5/13.0/17.25 & 5.0/3.5/4.25 & 7.0/21.5/14.25 & 11.9 \\ LR-Aver-Soup & 19.0/18.0/18.5 & 6.0/3.5/4.75 & 3.5/19.0/11.25 & 11.5 \\ CLR-Aver-Soup(SWA) & 18.0/14.5/16.25 & 8.0/8.0/8.0 & 14.0/14.5/14.25 & 12.8 \\ CLR-DDWA-Ensemble & 20.0/9.0/14.5 & 12.0/3.5/7.75 & 11.0/12.0/11.5 & 11.25 \\ CLR-DDWA-Soup & **17.0**/**3.5**/**10.25** & **1.0**/**1.0**/**1.0** & **2.0**/**6.0**/**4.0** & **5.08** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Rankings of the models on SCAN and BreakHis in a Friedman test (SCAN/BreakHis/dataset average).  
and model type diversity. This kind of model diversity is good at preventing over-fitting during training. Secondly, limited by the error basin minimization theory of the loss function, the model soup proposed in this paper seems to perform better when fine-tuning the pre-training model, and model fine-tuning is easier to satisfy the condition that each component model is located at the edge of the error basin; moreover, the essence of the model soup involves the estimation of the optimal point in the error basin. For ingredient models trained from scratch, the model soup strategy does not lead to significant performance gains. Thirdly, in order to improve the diversity of component models, the model fine-tuning process relies heavily on the adjustment of training parameters, such as CLR, the type and strength of data enhancement, and the optimizer choice, which will increase the training workload and make it challenging to achieve the best training plan.