## 4 Discussion
### Limitations  
It must be acknowledged that the chosen approach is primarily based on supervised learning algorithms (ANN, SVM, LR, RF, LightGMB, XGBoost, and CatBoost) and thus excludes potential insights from unsupervised or semi-supervised methods. Although principal component analysis (PCA) was used during data preprocessing to explore the -dimensional structure of the input space, this method was not applied as a stand-alone approach to uncover latent clusters or reveal novel groupings among the firms. The focus on these seven supervised models was based on their proven interpretability and their ability to achieve the classification objective of distinguishing between high- and low-innovative firms. Nevertheless, the exclusive focus on supervised classification is a clear limitation, as further integration of unsupervised methods (e.g., clustering or advanced dimensionality reduction) could potentially reveal deeper structural insights and improve the overall understanding of innovation-related outcomes. An important limitation--which has been pointed out multiple times--is the dependency between training and test sets when performing statistical tests using k-fold cross-validation, repeated cross-validation, and resampled cross-validation to obtain the required number of estimates for statistical evaluation. Although this approach is widely used, it is important to recognize this problem, and that is why new methods have been proposed to mitigate this effect, such as corrected k-fold, corrected repeated k-fold, and corrected resampled k-fold cross-validation. Another limitation of this study results from the variability of the required sample size determined by the power analysis, which varied considerably depending on the model pair and the evaluation metric used. The required sample size ranged from less than five for LR model pairs (for most metrics, with the exception of the precision metric) to over ten thousand for certain model comparisons. For example, in the case of XGB vs. CB on ROC-AUC--where performance differences were minimal--the power analysis yielded a required sample size of 29,442. For all metrics (accuracy, precision, recall, F1, and ROC-AUC), the median required sample size was 192, while the average was 1446. An optimal approach would be to generate a required number of estimates for each pair of models on each metric, based on the results of the power analysis. However, this would significantly increase computational complexity and resource requirements. Therefore, a uniform number of 100 estimates was chosen for this study, which strikes a practical balance between feasibility and statistical rigor, although this remains a limitation. In addition, the [PERSON]'s statistical test uses multiple iterations instead of a single run. Traditionally, the [PERSON]'s test is applied to a benchmark test set (e.g., 10% of the sample in a single split). However, in this study, predictions were made across the entire dataset using a k-fold approach (_'cross_val_predict'_), violating the assumption of independence between training and test sets. This change was made to increase the number of data points and mitigate the effects of fluctuations in small subsamples, albeit at the cost of some bias. Nevertheless, the results of the [PERSON]'s test (Table 6) show that it remains very conservative and primarily identifies statistically significant differences in comparisons with LR models while revealing far fewer differences than other statistical tests. This suggests that the approach did not substantially inflate the Type I error rate, which emphasizes the robustness of the results. Another limitation of this study is that it relies on a two-tailed approach for all statistical tests, which detects significant differences in both directions. As the hypotheses were specifically designed to test whether certain models were superior to others, a one-sided approach would have been more appropriate.