# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 1 Introduction
This is because each test sample needs to be fed to each base model for inference and, hence, the running time is proportional to the number of base models. This limits the application scope of the model ensemble methods, especially in low-specification terminals or mobile devices. Recently, some studies on model fusion in the weight space have been proposed [16; 17; 18; 19]. Researchers proposed different strategies to fuse weights of some trained/pre-trained models to form a single model. Equation (1) shows the weight average strategy for model fusion:  
\[f_{fusion}(x,\theta)=f(x,\frac{1}{M}\sum_{k=1}^{M}\theta_{k}) \tag{1}\]  
where \(F=\{f(x,\theta_{1}),f(x,\theta_{2}),\cdots,f(x,\theta_{M})\}\) represents a set of trained models with the same architecture but different weights. In contrast, model ensemble works in the model space, meaning that it fuses the output of the models to obtain a final result. Equation (2) shows the majority voting strategy on the model set \(F\):  
\[f_{ens}(x)=\frac{1}{M}\sum_{k=1}^{M}f(x,\theta_{k}) \tag{2}\]  
It can be seen that the inference cost of \(f_{fusion}\) is only \(\frac{1}{M}\) of that of \(f_{ens}\) while the training costs of both are almost equivalent. The rationale for model fusion in the weight space lies in the relationship between local minima and the global solution. [PERSON] et al. [20] claimed that the loss minima of a deep neural network are not isolated in a parameter space but rather form a connected manifold. There can be a path between two local minima where the loss does not significantly fluctuate. [PERSON] et al. [16] observed that when training a deep neural network using a high-frequency cyclical learning rate (CLR) [21],the model weights corresponding to the periodic minimum learning rate are located on the periphery of the most desirable solutions (global loss minima). Averaging these weights may lead to a solution closer to the optimal solution than the ones located on the periphery. [PERSON] et al. [17] extended the weight-averaging method from a single training trajectory to the context of fine-tuning models with different training hyperparameters. Their method can also work in the transfer learning mode. Inspired by these studies, we propose a novel model-based fusion method for histopathological image classification tasks. Following the work of [PERSON] et al. [17], we use the term soup in this paper to refer to the models fused in the weight space, and each model's weights are referred to as the ingredients of the soup. We propose two strategies for constructing soup models: the average strategy and the dominant difference weight-averaging (DDWA) strategy. For DDWA, by drawing on the idea of the selective ensemble [22], we select ingredient models that perform satisfactorily enough on the validation set and have large differences from other models for fusion in the weight space. Compared to the strategy of averaging the weights of all ingredient models, DDWA can produce a fusion model that achieves a loss closer to the optimal expected loss. The proposed method can work either in the fine-tuned model mode or in the full model mode. The contributions of this paper are two-fold:  
* We introduce a weight-based model fusion strategy to solve the problem of histopathological image classification. The strategy significantly reduces the computational complexity of model training and inference. * We propose the DDWA method to screen the ingredient models, enabling the weight fusion model to achieve a better approximation to the optimal value of the loss function's error basin, thereby improving the generalization ability of the model. The remainder of this paper is organized as follows. We review some related and important works in Section 2. In Section 3, we present the proposed model and its strategies. The evaluation results are reported and discussed in Section 4. Section 5 discusses the advantages and limitations of the proposed model. The paper is concluded in Section 6.