Additional file 1 of Don’t dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning
Creators: name: Levy, Joshua J.
name: A. James O’Malley
Publisher: name: figshare
Year: 2020
Subjects: subject: Statistics
subject: FOS: Mathematics
subject: FOS: Mathematics
subject: Artificial Intelligence and Image Processing
subject: FOS: Computer and information sciences
subject: FOS: Computer and information sciences

description: Additional file 1 Appendix Table 1. Glossary of Terms. Appendix Table 2. Pros and Cons of Logistic Regression, Decision Tree, and Random Forest Approaches. Appendix Figure 2. Characteristics of linear modeling approach (blue), decision trees (orange), random forest (green), original generated points (black); (a-b) Predictor X versus response Y for the a) linear continuous predictor, b) non-linear transformed predictor. Appendix Figure 2. Characteristics of generalized linear modeling approach (green), decision trees (purple), random forest (red); (a-b) Predictors X1 and X2 versus binary response Y for the a) linear continuous predictors, b) non-linear transformed predictors. Appendix Figure 3. Decision Diagram for Application of Hybrid Approach. Appendix Figure 4. Performance of Hybrid Approach for select datasets is on par with Random Forest for domain where Random Forest outperforms Logistic Regression by at least 2%; (a) 37 datasets selected with highest positive residual from model regressed on performance gains for Hybrid and Random Forest approaches in the aforementioned domain; (b) Boxenplots of predictive performance of the Logistic Regression, Random Forest and Hybrid methods for these 37 datasets; other comparison methods enforce stricter thresholds/cutoffs for performance gain. Appendix Figure 5. Comparison of Additional Machine Learning Modeling Approaches Across: (a) 277 benchmark datasets featured in the main text; (b) the 77 datasets that remain after selection of datasets where Random Forest significantly outperforms the traditional modeling approaches; the threshold for outperformance for Random Forest was greater than that utilized in previous comparison approaches for effect. Appendix Table 3. Comparison of median AUROC for each machine learning and statistical modeling approach across 277 benchmark datasets and a subset of these datasets where Random Forest significantly outperforms the traditional modeling approaches (n = 77); 95% confidence intervals were derived using a 1000-sample non-parametric bootstrap; the threshold for outperformance for Random Forest was greater than that utilized in previous comparison approaches for effect. Appendix Table 4. Pairwise comparison of median AUROC for each of the modeling approaches for the 277 benchmark datasets; 2-tailed non-parametric Mann-Whitney U tests were corrected using family-wise error rate Bonferroni correction to find median AUROCs that were different between the modeling approaches on the 0.05 alpha significance level; we note here that there still exists 61 datasets from which the Hybrid approach is still able to outperform the Random Forest Approach. Appendix Table 5. Pairwise comparison of median AUROC for each of the modeling approaches for a subset 277 benchmark datasets from which the Random Forest outperforms the traditional modeling approaches (n = 77); 2-tailed non-parametric Mann-Whitney U tests were corrected using family-wise error rate Bonferroni correction to find median AUROCs that were different between the modeling approaches on the 0.05 alpha significance level. Appendix Table 6. Top 10 largest odds-ratios of the Epistasis Logistic Regression Model coefficients after using LASSO to select predictors from a logistic regression model which included all pairwise predictors; note how all main effects have been excluded from this list. Appendix Figure 6. Cluster Heatmap of dataset characteristics of the 277 datasets; rows of matrix indicate datasets; columns of matrix indicate different properties of the dataset; columns were standardized to illustrate trends; left-hand yellow and blue labels were applied to indicate whether Random Forrest model outperformed Logistic Regression model (yellow for gain in performance; blue for no gain in performance, prediction to left of true difference in performance) for a given dataset; column colors indicate relative importance of dataset characteristics as identified by the Gini index.