## 4 Analysis and Results
Group 1 includes the features that were further examined against all other features in the dataset to assess their impact on increasing the value of each feature in the group. Mean decrease Gini was a metric used to evaluate the importance of the features in a random forest model. It calculated the overall decrease in Gini impurity across all decision trees in the forest when a specific feature was used to split a node. A higher mean reduction in the Gini value generally indicates a more important feature. Therefore, the two features with a mean decrease Gini value above 1 in this study were further analyzed using multiple linear regression. Only X2 and X27 were used in this analysis as response variables compared to other variables individually in the multiple linear regression model. Figure 5 presents a histogram for each feature in Group 1. Table 4 displays the output for the X2 model. The model employed a stepwise selection procedure based on the Akaike information criterion to enhance accuracy and reduce the number of less significant variables. AIC is a valuable method for stepwise selection as it balances two critical factors in model building. AIC rewards models that can explain much of the data's variation and penalizes those with many parameters. This helps to prevent overfitting, where a model fits the training data too closely but performs poorly on new data. Combining these two elements, AIC provides an overall measure of a model's quality. A lower AIC value indicates a better-fitting model with fewer unnecessary parameters. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Actual Value Is A** & **Actual Value Is B** \\ \hline Predicted A & 4 & 0 \\ Predicted B & 5 & 33 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Confusion matrix. Figure 3: Random forest accuracy vs. number of trees. Figure 4: (**a**) Mean decrease accuracy; (**b**) mean decrease Gini. Figure 5: Histogram for each predicted variable. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Coefficients** & **Estimate** & **Std. Error** & **T-Value** & \(p\)**-Value** & \\ \hline Constant & 0.2329 & 0.0909 & 2.5630 & 0.0120 & * \\ X1 & 0.0000 & 0.0000 & \(-\)2.8970 & 0.0047 & ** \\ X3 & 0.0040 & 0.0012 & 3.3680 & 0.0011 & ** \\ X4 & 0.0121 & 0.0030 & 3.9930 & 0.0001 & ** \\ X5 & 0.0072 & 0.0025 & 2.9540 & 0.0040 & ** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Summary of the regression model for X2. The model was modified by stepwise selection, where variables were added or removed one at a time. The results for the model can be found in Table 5. The residual plots for the regression model were observed in Figure 6 after implementing the stepwise model in both directions. Normality was tested using the Shapiro-Wilk test, while heteroscedasticity was tested using the Breusch-Pagan test. Both tests yielded \(p\)-values greater than 0.05, leading to the rejection of the null hypothesis. This suggests that the residual data followed a normal distribution and is homoscedastic. Furthermore, the Durbin-Watson test showed that the residuals are not correlated. Table 6 displays the VAF values, which were all greater than 1, indicating the possibility of multicollinearity among the predictors. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Coefficients** & **Estimate** & **Std. Error** & **T-Value** & \(p\)**-Value** \\ \hline X6 & 0.0000 & 0.0000 & 1.3180 & 0.1906 \\ X7 & 0.0006 & 0.0007 & 0.8550 & 0.3949 \\ X8 & \(-\)0.0003 &  
The residual plots for X2 and X27 can be seen in Figures 6 and 7. Normality was tested using a Shapiro-Wilk test, while heteroscedasticity was tested using a Breusch-Pagan test. Both tests resulted in \(p\)-values above 0.05, leading to the rejection of the null hypothesis. This suggests that the residual data followed a normal distribution and is homoscedastic. A Durbin-Watson test also showed that the residuals were not correlated. All VIF values were above 1, indicating the potential presence of multicollinearity among the predictors, as seen in Table 6.