# A Histopathological Image Classification Method Based on Model Fusion in the Weight Space
## 3 Model Fusion in the Weight Space for Histopathological Image Classification
### Soup Strategies  
Recent studies have shown that the way in which the model weights are generated and the method used for weight fusion have significant impacts on the performance of the soup model. We collectively refer to the generating and fusing of soup ingredients as soup strategies. We implemented two strategies for building the soup model. The first strategy was averaging the soup, which simply involves averaging the weight of each gradient in the soup. The weight-averaging strategy was applied and proved effective in previous studies. Since models trained using methods such as gradient descent often converge near the edge of the error basin, the use of the averaging strategy holds promise in moving a model closer to the center (expected value) of the error basin, which may have better generalization ability. Fine-tuning a pre-trained model with different training hyperparameters and recording the weights in each case would result in a set of models located in the same low-loss basin. The second strategy involves the use of dominant difference weight averaging (DDWA), which is motivated by the classical ensemble pruning method [38]. According to the classification accuracy \(acc\) and diversity \(div\), DDWA selects the ingredients with better performance to construct the soup model. Let \(H=\{h_{w_{1}},h_{w_{2}},\cdots,h_{w_{M}}\}\) represent the set of candidate ingredients and \(D=\{(x_{1},y_{1}),(x_{2},y_{2}),\cdots,(x_{n},y_{N})\}\) represent the evaluation dataset, where \(x_{i}\) represents the \(i\)th instance and \(y_{i}\) represents the corresponding class label. The \(i\)th ingredient's \(acc\) and \(div\) are defined as Equations (9) and (10):  
\[acc_{i}=\frac{1}{N}\sum_{j=1}^{N}L(h_{w_{i}}(x_{j}),y_{j}) \tag{9}\]\[div_{i}=\frac{\sum\limits_{j=1}^{N}(h_{w_{i}}(x_{j})-avg_{j})^{2}\cdot\alpha}{N} \tag{10}\]  
\[avg_{j}=\frac{1}{M}\sum\limits_{k=1}^{M}h_{w_{k}}(x_{j}) \tag{11}\]  
where \(L(\cdot,\cdot)\) is a zero-one loss function, \(avg_{j}\) is the averaging output of all candidate ingredients on \(x_{j}\), and \(\alpha=1\) if \(h_{w_{i}}(x_{j})\) equals the ground truth label \(y_{j}\); otherwise, \(\alpha=0\). \(div_{i}\) evaluates the difference between the \(i\)th component and the entire set under the condition of correct classification. We sort all of the ingredients using non-dominated sort (NDS) according to these two indices, and select the top q% of ingredients to build the soup model, as shown in Algorithm 1. ```
1:\(rank_{A}\):\(acc\) ranking of \(M\) candidate ingredients
2:\(rank_{D}\):\(div\) ranking of \(M\) candidate ingredients
3:\(H\): candidate ingredients
4:\(rank_{NDS}\):\(NDS\) ranking of \(M\) candidate ingredients
5:\(F=\emptyset,rank_{NDS}=\emptyset\)
6:for\(i=1,2,\cdots,M\)do
7:\(S_{i}=\emptyset\), \(b_{i}=0\)
8:for\(j=i,2,\cdots,M\)do
9:if\(h_{i}\) dominates \(h_{j}\)then
10: add \(h_{i}\) to \(S_{i}\)
11:else if\(h_{j}\) dominates \(h_{i}\)then
12:\(b_{i}=b_{i}+1\)
13:endif
14:endfor
15:if\(b_{i}=0\)then
16: Add \(h_{i}\) to \(F\)
17:endif
18:endfor
19: Ascending sort \(F\) according to the sum of \(rank_{A}\) and \(rank_{D}\)
20:while\(F\
eq\emptyset\)do
21:\(Q=\emptyset\)
22:for\(i=1,2,\cdots,M\)do
23:for all \(g\) in \(S_{i}\)do
24:if\(n_{g}\
eq 0\)then
25:\(n_{g}=n_{g}-1\)
26:Addg to \(Q\) if \(n_{g}=0\)
27:endif
28:endfor
29:endfor
30: Ascending sort \(Q\) according to the sum of \(rank_{A}\) and \(rank_{D}\)
31: Add \(Q\) to \(rank_{NDS}\)
32:endwhile
33:return\(rank_{NDS}\)
```  
**Algorithm 1** Non-dominated sort. NDS ensures that the selected soup ingredient models have good accuracy and are different from each other, which is in line with the main idea of model soup. Specifically, a higher accuracy means that the selected ingredient model is close to the error basin, while a larger difference ensures that the spatial distribution of the component model maintains a certain distance, both of which make the soup model more effective. Figure 1 shows the resulting models obtained through weight averaging in two cases.