# Shapley Values as a Strategy for Ensemble Weights Estimation
## 4 Results and Analysis
### Heterogenous Ensembles
The heterogeneous ensemble results (Figure 17) for the Bank Marketing dataset using R language implementation demonstrated that for ensembles with 10 and 16 members, the posShape weighting strategy surpassed the performance of the monolith approach, but the best-performing weighting strategy was Equal with 0.233 _BCE_. With the ensemble consisting of 10 or more members, posShape outperformed the Perf strategy and produced the same median as the Rand strategy of 0.235 _BCE_. Although the weighting performance  
Figure 14: Ranking of heterogenous ensembles: using Bank Marketing dataset and Python implementation. Figure 13: Detection error by _BCE_ for various heterogenous ensemble sizes using Python models and Bank Marketing dataset. gains were only 0.6%, all weighting strategies, except maxShap, outperformed the Mono approach. The ranking of ensembles (Figure 18) revealed that the Mono strategy was ranked as the best-performing one, with posShap and maxShap methods ranked as the least-performing strategies. The results of the heterogeneous ensembles for the BNG-creadit_a dataset implemented in the R language (Figure 19) reveal that all ensemble weighting strategies were outperformed by the Mono approach. Weighting strategies performed similarly with no statistically significant differences. Surprisingly, the best result of 0.339 \(BCE\) was achieved by Rand weighting. As for posShap, its performance dropped when the number of models in an ensemble was 16, but when applied to ensembles with 6 and 10 members, it exhibited the best performance. The ensemble ranking (Figure 20) also reveals that the results for all approaches have no statistically significant difference between them, the Mono approach ranked as the best, and other approaches performed without statistically significant difference between their results. The results of the heterogeneous ensemble experiment are presented in Table 5, which demonstrated that in some cases choosing the right model type is more important than choosing the right weighting strategy. This is especially evident from Figure 19 where the monolithic logistic regression model outperformed the ensembles with performance differences ranging from 1.3% to 2.2%. Figure 16: Ranking of heterogenous ensembles: using BNG-credit_a dataset and Python implementation. Figure 15: Detection error by \(BCE\) for various heterogenous ensemble sizes using Python models and BNG-credit_a dataset. Figure 19: Detection error by \(BCE\) for various heterogenous ensemble sizes using R models and BNG-creadit_a dataset. Figure 17: Detection error by \(BCE\) for various heterogenous ensemble sizes using R models and Bank Marketing dataset. Figure 18: Ranking of heterogenous ensembles: using Bank Marketing dataset and R implementation. One could argue that the difference between the monolith model and the weighting strategies was caused by implementation-specific hyperparameter settings since no such advantage of the monolith can be noticed with the Python implementation (Figure 15) when using the BNG-Credit-A dataset. The differences between implementations were also minimal when comparing Figures 13 and 17. The performances of the two implementations ranged from 0.233 to 0.269 \(BCE\) and exhibited the same best-performing ensembles achieved with an equal weighting strategy. The introduction of additional model types improved ensemble performance in three out of four test cases. Compared to the best result produced in homogeneous experiments, 0.236 \(BCE\), the heterogeneous ensembles increased the performance to 0.233 \(BCE\). Similarly, the experiment results reveal that in three out of four cases, any weighting strategy can outperform a monolithic model, signifying the value of ensembling. PosShap strategy outperformed the Roz strategy in all 3 tested heterogeneous experiment settings except for Python implementation using the Bank Marketing dataset, where results were not statistically different between the two strategies.