# Cyber-Secure IoT and Machine Learning Framework for Optimal Emergency Ambulance Allocation
## 4 Results  
### Model Performance Evaluation and Feature Importance Analysis  
This study evaluates the predictive performance of three ensemble-based regression models for forecasting ambulance demand and dispatch time. The target variables are (1) the estimated number of ambulances required per emergency medical center and (2) the average dispatch duration from departure to hospital arrival. The dataset comprises 2061 monthly observations across 229 EMS centers, integrating operational metrics, population statistics, transportation indicators, and emergency response records. Three state-of-the-art machine learning models were tested: Random Forest (RF), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM). These tree-based algorithms were selected for their proven ability to capture complex nonlinear relationships and interactions among variables. As shown in Table 2, model training and validation were conducted using five-fold cross-validation, and performance was assessed using root mean square error (RMSE), mean absolute error (MAE), and the coefficient of determination (R\({}^{2}\)). Among the models, LightGBM achieved the lowest MSE (0.048) and RMSE (0.218), demonstrating superior overall prediction accuracy. This can be attributed to its use of leaf-wise tree growth, which enhances local precision in high-dimensional spaces [33]. However,XGBoost showed the lowest MAE (0.084) and MAPE (0.040), indicating robustness in minimizing absolute and relative prediction errors. The model's advantage is likely due to its L1/L2 regularization, which effectively prevents overfitting and improves convergence stability through learning rate adjustment. These results suggest that while LightGBM excels in minimizing the total prediction error, XGBoost offers advantages in controlling individual-level deviations. Each model's strengths provide valuable flexibility depending on whether the goal is precise aggregate forecasting or stable individual-level predictions. Meanwhile, the Random Forest model exhibited consistently stable performance across all evaluation metrics, with particularly robust results for RMSLE (0.070), indicating reliable generalization even under log-transformed distributions. This robustness stems from the model's ensemble structure based on bootstrap aggregation (bagging), which effectively reduces variance by averaging multiple decision trees, thereby enhancing prediction stability [34]. Moreover, the model's relative ease of interpretation, particularly in feature importance estimation, aligns well with this study's emphasis on policy applicability. In contrast, LightGBM exhibited a tendency to overemphasize the importance of variables that appear repeatedly across trees. This bias is especially pronounced for high-cardinality or categorical variables, potentially leading to distorted feature importance rankings [35]. Despite this, LightGBM demonstrated the highest overall predictive accuracy and computational efficiency among the tested models. Taken together, all three models demonstrated predictive performance suitable for practical deployment. However, each showed distinct strengths: LightGBM offered high precision and computational efficiency, XGBoost excelled in convergence speed and low mean error, and Random Forest provided strong interpretability and consistent generalization. Given that the objective of this study includes both real-time simulation and policy decision support, factors beyond raw accuracy--such as interpretability, resistance to overfitting, and reliability of feature importance output--were prioritized in model selection. Based on this comprehensive evaluation, Random Forest was adopted as the primary predictive engine for downstream simulation and decision-support tasks.