## 4 Discussion  
### Hypothesis  
The evaluation of seven machine learning models was undertaken to investigate whether their algorithmic distinctions would yield significant differences in performance when predicting innovation outcomes based on company innovation activities. Consistent with hypothesis H1 (\"Machine learning models applied to predict innovation outcomes based on company innovation activities will exhibit differential performance across key predictive metrics, largely driven by each model's inherent algorithmic properties and alignment with the dataset.\"), the results of all statistical tests provide solid support. Significant performance disparities were consistently observed, and no single test failed to reject the null hypothesis for at least one pair of models, indicating that the choice of algorithm is indeed consequential in this predictive context. In contrast, hypothesis H2 (\"Ensemble learning methods (Random Forest, XGBoost, CatBoost, and LightGBM) will yield superior predictive performance compared to individual models (Linear Regression, SVM, and ANN) in forecasting innovation outcomes from firm-level innovation activities.\") received only partial empirical support. Although CatBoost and XGBoost emerged as top performers across most metrics, SVM demonstrated superior recall, underscoring that certain individual models can excel in specific performance dimensions. Random Forest and LightGBM, while generally slightly less effective than CatBoost and XGBoost, also achieved comparatively strong recall values (as evidenced by the Friedman test in Figure 5). Consequently, these findings do not unequivocally affirm hypothesis H2, since the ensemble methods did not uniformly surpass the non-ensemble models across all predictive metrics. With respect to hypothesis H3 (\"CatBoost will outperform other gradient boosting algorithms (XGBoost and LightGBM) owing to its efficient handling of categorical variables, resulting in improved innovation outcome predictions.\"), the comparative evaluation of CatBoost against XGBoost and LightGBM yielded findings that favor CatBoost over LightGBM with statistical significance. Multiple tests, including the Friedman test (for nearly all metrics except recall, where no significant difference was identified, see Figure 5), [PERSON]'s test (Table 6), the 10-fold CV paired \(t\)-test (Table 8), and the corrected 10-fold CV paired \(t\)-test (Table 9), consistently demonstrated that CatBoost performed significantly better than LightGBM. In contrast, the difference between CatBoost and XGBoost was statistically insignificant in most tests, although CatBoost showed a small but not statistically significant promotional advantage. Furthermore, CatBoost displayed better average performance metrics and average ranks (Table 15, Figure 5) when compared directly to XGBoost. Consequently, these results indicate that CatBoost outperformed the other gradient boosting models, thereby confirming Hypothesis H3. Regarding Hypothesis H4 (\"Artificial Neural Networks (ANNs) will not significantly surpass simpler models (e.g., Linear Regression and SVM) in predicting innovation outcomes, primarily due to the limited size and binary nature of the dataset.\"), partial support was observed. Although ANN consistently surpassed Linear Regression in almost all metrics across every statistical test (Tables 6-14), the comparison between ANN and SVM yielded more nuanced outcomes. Most tests indicated that ANN excelled over SVM in terms of precision and ROC-AUC (Tables 7-14). However, ANN underperformed relative to SVM in the recall metric in almost all cases and also underachieved on the F1 metric (as shown by the Wilcoxon signed-rank test in Table 12 and the Friedman test in Figure 5). These mixed findings suggest that neither ANN nor SVM unequivocally dominated across all evaluation criteria. In contrast, ANN demonstrated a clear advantage over Linear Regression, thus providing evidence that ANN can outperform at least one simpler model in a statistically significant manner.