# Startup Sustainability Forecasting with Artificial Intelligence
## 3 Results
### Addressing Key Challenges in Startup Sustainability Forecasting  
In the application of AI to forecasting startup sustainability, three critical challenges must be considered: handling correlated features, preventing overfitting, and accounting for the impact of human factors. Correlated features can lead to the misrepresentation of feature importance and reduce the effectiveness of learning. There are generally two ways to address these issues: feature transformation and feature selection. In feature transformation, the current features are transformed into a new representation where they are not correlated with each other. The primary technique for achieving this is Principal Component Analysis (PCA). This method redefines the feature space into principal components that are orthogonal to each other and capture the largest portions of variance in the data. While PCA effectively eliminates feature correlations, the principal components often lack a natural interpretation--a significant drawback for model explainability. Given that we are considering models that must support critical, high-level decision-making in investment contexts, we instead opt for feature selection methods. Feature selection techniques choose a subset of the original feature space, preserving the interpretability of the original features. For this problem, we propose a two-step feature selection process. In the first step, the process ranks the available features according to their predictive power. This can be achieved using the Mutual Information Criterion (MIC) [21], which reveals the degree of statistical dependence between the feature and the target, i.e., the reduction in target uncertainty from knowing the feature. Tree-based feature importance and Boruta are also well-regarded methods for feature selection, but they do not perform well when there are many correlated features. In the second step, the ranked feature list is scanned sequentially, including features in the subset as long as (b) they are not statistically significantly correlated with the previously selected features and (b) improve the validation performance of the model (step-forward feature selection). Figure 3: A decision tree visualization that reveals the discovered rules for sustainability forecasting. While effective feature selection significantly reduces overfitting, additional methods should be employed to ensure robust model performance. Leave-one-out cross-validation (LOOCV) provides a reliable estimate of model performance; however, the final generalization error should be evaluated using an independent test set. In applications such as startup sustainability forecasting, where data are relatively scarce, fully grown decision trees--being high-variance models--are especially prone to overfitting. Unless the trees are pruned or their depth is restricted, they risk capturing noisy patterns because of their tendency to perfectly fit the training data. A more effective approach is to use ensemble methods such as Random Forest (RF), which mitigate overfitting by combining fully grown trees without the need for pruning or depth constraints. RF reduces variance through averaging tree predictions and decorrelating individual trees by randomly selecting subsets of features for each split. This method has demonstrated strong performance across various problems. Additionally, hybrid ensembles that combine models with different inductive biases can be trained independently, further reducing overfitting by leveraging their complementary strengths. Nevertheless, the most effective direct approach to combat overfitting is to collect more data. Increasing the dataset size also enables the use of deep learning approaches, which generally require more examples than tree-based models. For models based on artificial neural networks (ANNs) and gradient descent, techniques such as early stopping, L1/L2 regularization, and dropout should be employed to prevent the network weights from growing excessively.