# Shapley Values as a Strategy for Ensemble Weights Estimation
## 4 Results and Analysis
### Heterogenous Ensembles  
The homogeneous experiment demonstrated weighting capabilities applied to using identical types of machine learning models. In real-world applications, researchers may prefer using different model types to increase the ensemble's diversity and improve its generalization, but also due to varying capabilities [61] of forming internal representations from features available. In the following experiment, weighting strategies were applied to heterogeneous ensembles. The logistic regression monolith model was chosen here as a baseline due to the best performance in the homogeneous experiment. The results for heterogeneous ensembles trained on the Bank Marketing dataset using the Python implementation (Figure 13) were similar to the results presented in the R implementation. The equal ensemble weighting strategy demonstrated the lowest _BCE_ overall with the best performance in an ensemble with 16 models that exhibited 0.233 _BCE_. The maxShap weighting strategy displayed reduced performance over ensembles with a higher number of models and exhibited the worst performance overall. The opposite could be observed for the results of the posShap weighting strategy, although it performed similarly to the Mono approach in ensembles with 4 and 6 members. Ensembles with 10 and 16 members outperformed the Mono approach. The posShap strategy also outperformed Perf when the number of models in the ensemble was 16. The posShap advantage over the Perf strategy demonstrates that Shapley values are able to measure model contribution in greater detail. This advantage stems from the evaluation of all possible permutations revealing combinations that cannot be considered by the simple performance metric of a single model. The ranking of weighting strategies (Figure 14) also reveals that in this scenario, the Equal strategy outperformed other tested approaches. The Rand approach was ranked as second best, and Perf and posShap were ranked third and fourth, respectively. The Mono and maxShap ensemble weighting strategies were the least performant. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Model Type** & **LR 1** & \multicolumn{2}{c}{**DT 2**} & \multicolumn{2}{c}{**LR 1**} & \multicolumn{2}{c}{**DT 2**} \\ \hline
**Dataset** & \multicolumn{2}{c}{**BNG 3**} & \multicolumn{2}{c}{**BNG 3**} & \multicolumn{2}{c}{**Bank 4**} & \multicolumn{2}{c}{**Bank 4**} \\ \hline
**Weighting Strategy** & **Python** & **R** & **Python** & **R** & **Python** & **R** & **Python** & **R** \\ \hline Mono & **2.13** & **2.45** & 5.52 & 4.86 & 3.67 & 3.61 & 5.33 & 6.40 \\ Rand & 4.40 & 4.38 & 3.80 & 3.50 & 4.97 & 5.04 & 4.11 & 4.22 \\ Equal & 4.58 & 4.43 & 3.86 & 3.94 & 4.88 & 4.86 & 3.21 & 3.52 \\ Perf & 3.63 & 3.52 & 2.86 Using the same Python implementation and training the heterogeneous weighted ensembles on the BNG-credit_a dataset, the results (Figure 15) were similar to homogeneous ensemble implementation (Figure 7) for the same dataset. The introduction of logistic regression models further reduced the _BCE_ for ensembles with applied weighting strategies and improved the performance of ensembles. The results of the posShape weighting strategy were the best out of all tested strategies, resulting in a median _BCE_ of 0.312. The Perf and Equal strategies resulted in a somewhat similar performance. The critical difference diagram (Figure 16) did not show a statistically significant difference between these two approaches. The model ranking also revealed that the monolith strategy was ranked the lowest, meaning that all the ensembling strategies increased their performance over their homogeneous counterparts. For ensembles of 16 models, the performance gain over Mono ranged from 1.3% for Equal, Perf and Rand strategies to 1.4 % for posShape strategy.