## 4 Results and Discussion  
### Combined Pre-Game/In-Game Model Performance  
The confusion matrices for each model are shown in Figure 5 below. The label is whether T1 won or not. Logistic regression had the highest overall accuracy and the highest number of true positive predictions. Gradient Boost had the lowest accuracy; however, it achieved the highest number of true negative predictions, giving it the highest specificity of all the models, but it also had the lowest number of true positives. Random Forest and XGBoost are very similar, with Random Forest slightly outperforming overall, as it correctly classified four more wins than XGBoost, whereas XGBoost correctly classified three more losses.  
Figure 6 below shows the ROC curve for each model, and Figure 7 shows the accuracy and AUC for each model. As seen from both, logistic regression is the most accurate model and has the highest AUC and thus the ability to best differentiate wins and losses.  
Compared with the (LR) model, the RF model demonstrated marginally lower accuracy and AUC. The test results suggest good generalizability, with little evidence of overfitting. Notably, LR attained the highest AUC value at 0.851, and according to Figure 6, it exhibited superior ability in distinguishing between true positives and true negatives. Conversely, Figure 7 indicates that the disparities between the models are negligible. Both GB and XGB lagged behind LR and RF in terms of accuracy, although GB recorded a slightly higher AUC than RF did. The LR combined model is thus the preferred model (given accuracy specifically is our metric of interest) and will act as the base model for comparison to others going forward.  
Figure 5: The confusion matrices show the performance of all the models in this study.  
Figure 6: ROC curves for each model.  
This performance difference could be attributed to multiple factors. First, the use of tenfold cross-validation in conjunction with the internal bootstrapping mechanism inherent in tree-based models might have limited the pool of available information during the hyperparameter tuning phase, possibly leading to the selection of suboptimal hyperparameters and a consequent reduction in average accuracy [72]. Specifically, this could be due to the relatively modest sample size of 3054 combined with the following multiple layers of validation:  
* The external train/test split mechanism that was used (75%/25%) meant only 75% of data were available for training at all (\(n\) = 2290).
* Cross-validation during the training phase meant only 90% of data were available during any given iteration (2290 \(\times\) 0.9 = 2061).
* The internal bootstrapping mechanisms of tree models further reduce the information space. The ranger engine, for instance, internally reserves 36.8% of samples for out-of-bag error estimation during its training phase, meaning each tree would be trained on 2061 \(\times\) 0.632 = 1302 samples. C5.0 and XGBoost also use the sample size hyperparameter to train on a subset of data only.
* This could have meant increased variance in performance across both folds and trees within the models; however, it does mean comprehensive validation of models against unseen data at multiple layers.  
Secondly, the relatively constrained feature space could increase the correlation between trees, thereby increasing the generalization error [73]. Practically, however, the differences are nearly negligible.