# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 1 Introduction  
Histopathological images are digital images of tissue sections observed under a microscope, and a large number of studies have shown and verified their important value in biomedicine [1; 2]. In particular, histopathological images have been broadly used in the screening of many cancers and are, in fact, considered the gold standard for diagnosing a significant number of cancers. One notable feature of histopathological images is their high information density, which may include microstructures, such as cells, nuclei, stroma, tumors, blood vessels, glands, etc., and histopathological images can be observed at various magnifications, allowing for a multi-scale analysis [3]. The automatic classification of histopathological images plays an important role in computer-aided diagnosis (CAD) systems, which can reduce the burden of pathologists, decrease the influence of human factors in diagnosis, and improve the stability of the results. With the rapid development of deep learning theory and technology [4; 5; 6; 7], histopathological image classification models based on convolutional neural networks have exhibited excellent performance in many applications. In order to improve the inference performance of convolutional neural networks, models with a larger number of layers and channels have been designed. For example, in the famous ImageNet image classification challenge [8],the number of parameters of CoAtNet-4+PT-RA-E150 [9], one of the state-of-the-art models at present, reaches 275M. Large-scale models have high requirements when it comes to the size of the training dataset and the capabilities of computer hardware. However, obtaining a large number of histopathological images is not an easy task for certain medical departments. It is also difficult to fully satisfy the computing resources required to train the model. Different strategies have been proposed to solve the contradiction between the lack of training data and computing resources and the pursuit of model performance. One widely used strategy is transfer learning. It takes the approach of further training (fine-tuning) on top of a well-trained model, e.g., models trained on ImageNet. A common practice is to train only the weights of the last few fully connected layers while fixing the weights of the remaining layers. By using a pre-trained model, one can make full use of its architecture and weights, resulting in significant reductions in the design and training costs of the model. This strategy has been widely used in work on histopathological image analysis [10; 11; 12; 13; 14; 15]. However, setting the model hyperparameters becomes a challenge when using this strategy to train or fine-tune the pre-trained model. Common hyperparameters include the learning rate, weight decay rate, batch size, data augmentation strength, etc. Hyperparameters have a direct impact on the training effect of the model. The optimal hyperparameter setting is usually difficult to obtain. Instead, people determine hyperparameter settings based on experience or by using a grid search on a small verification set. However, the grid search requires a lot of computational resources and may not reach global optima. Another strategy is model ensemble. The main idea of model ensemble is to train several base models with diversity, and then perform majority voting or weighted voting on the output of these models to obtain the final results. By using different training sets, changing the training hyperparameters, using different types of optimizers, etc., the diversity between base models can be improved. Model ensemble can combine multiple base models with suboptimal performance into a model with strong overall performance, which can lead to better results than a single model, provided that the diversity among the base models is ensured. Although the model ensemble strategy avoids the computationally expensive hyperparameter search, it greatly increases the inference cost of the model.