# Analyzing Key Features of Open Source Software Survivability with Random Forest
## 4 Results and Analysis
### Model Performance Evaluation and Comparison  
This section evaluates whether the random forest model was appropriately constructed for predicting OSS project survivability and examines its suitability through a performance comparison with the polynomial regression model. The main performance metrics of both models are summarized in Table 4.  
The polynomial regression model was constructed using Lasso regression, which automated both feature selection and the degree setting for each feature. The model's  
\begin{table}
\begin{tabular}{c c} \hline
**Hyperparameter** & **Optimal Value** \\ \hline n\_estimators & 400 \\ max\_depth & 10 \\ min\_samples\_split & 5 \\ min\_samples\_leaf & 1 \\ max\_features & log2 \\ bootstrap & True \\ \hline \end{tabular}
\end{table}
Table 3: Optimal hyperparameter settings.  
performance was evaluated through 5-fold cross-validation, ensuring the selection of the most optimal model for the given dataset.  
The random forest model demonstrated superior performance, as indicated by the various performance metrics and residual distribution. The \(R^{2}\) value for the random forest model was 0.716, meaning that the model explained approximately 71.6% of the data variance. The adjusted \(R^{2}\) value of 0.704 also indicated stability, suggesting that the model effectively captured the complexity and nonlinearity of the data without overfitting. The MSE value was 0.043, signifying a small error between the predicted and actual values and demonstrating high prediction accuracy.  
Additionally, as shown in Figure 4, the residual distribution of the random forest model was generally uniform, indicating that the model effectively learned various data patterns. However, as the target value approached zero, the data became sparse, resulting in increased errors, which represents a limitation of the random forest model. Despite this, the uniform residual distribution across most ranges suggests that the model provides consistent predictive performance.  
The performance of the random forest model was compared with that of the polynomial regression model to identify differences in predictive accuracy. Despite optimization using the Lasso technique, the polynomial regression model exhibited limitations, with an \(R^{2}\) value of 0.19, indicating its inability to adequately capture the nonlinearity of the data. The adjusted \(R^{2}\) value was \(-1.38\), an invalid result, reflecting the model's poor predictive  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Metric** & **Random Forest** & **Polynomial Regression** \\ \hline \(R^{2}\) & 0.716 & 0.19 \\ Adjusted \(R^{2}\) & 0.704 & 1.38 \\ MSE & 0.043 & 0.07 \\ Key Characteristics & Effectively captures nonlinear interactions and complex patterns in the data & Limited by the inability to fully capture nonlinear characteristics due to the constraints of polynomial degrees \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance evaluation results of random forest and polynomial regression models.  
Figure 4: Residual graph of random forest model (Red Line: Smoothed Trend, Blue Dots: Residuals).  
capability. Additionally, the MSE value of the polynomial regression model was 0.07, higher than that of the random forest model, indicating lower predictive accuracy.  
As illustrated in Figure 5, the residual distribution of the polynomial regression model showed a tendency for residuals to concentrate randomly in specific regions, reflecting the model's struggle to capture the nonlinearity of the data. In contrast, the random forest model exhibited uniformly distributed residuals, demonstrating its ability to learn a diverse range of data patterns effectively. These performance metrics and residual distribution comparisons confirmed that the random forest model was more suitable than the polynomial regression model for predicting OSS survivability.