# Predictive Modeling of Flight Delays at an Airport Using Machine Learning Methods
## 6 Results
### Performance Evaluation
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Phase**} & \multirow{2}{*}{**Accuracy**} & \multirow{2}{*}{**Recall**} & \multirow{2}{*}{**Precision**} & \multirow{2}{*}{**F-1 Score**} & \multicolumn{2}{c}{**Balanced Error Rate**} \\ \hline Logistic & Train & 0.64 & 0.63 & 0.64 & 0.64 & 0.36 \\ \cline{2-6} Regression & Test & 0.64 & **0.61** & 0.33 & **0.43** & **0.37** \\ \hline \multirow{2}{*}{Naïve Bayes} & Train & 0.54 & 0.13 & 0.76 & 0.22 & 0.46 \\ \cline{2-6}  & Test & 0.77 & 0.11 & 0.42 & 0.18 & 0.47 \\ \hline \multirow{2}{*}{ANN} & Train & 0.86 & 0.82 & 0.90 & 0.86 & 0.14 \\ \cline{2-6}  & Test & 0.74 & 0.28 & 0.38 & 0.33 & 0.42 \\ \hline \multirow{2}{*}{XGBoost} & Train & **0.96** & **0.92** & **0.99** & 0.95 & 0.04 \\ \cline{2-6}  & Test & **0.80** & 0.26 & 0.58 & 0.36 & 0.39 \\ \hline \multirow{2}{*}{Random Forest} & Train & 0.90 & 0.80 & **0.99** & **0.89** & **0.10** \\ \cline{2-6}  & Test & 0.79 & 0.12 & 0.62 & 0.20 & 0.45 \\ \hline \multirow{2}{*}{CatBoost} & Train & 0.90 & 0.81 & **0.99** & **0.89** & **0.10** \\ \cline{2-6}  & Test & **0.80** & 0.15 & 0.66 & 0.25 & 0.43 \\ \hline \multirow{2}{*}{LightGBM} & Train & 0.88 & 0.77 & 0.98 & 0.86 & 0.12 \\ \cline{2-6}  & Test & **0.80** & 0.12 & **0.69** & 0.22 & 0.44 \\ \hline \hline \end{tabular} Best values are shown in bold.  
\end{table} 
Table 6: Second trial, airport data with SMOTE. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Phase**} & \multirow{2}{*}{**Accuracy**} & \multirow{2}{*}{**Recall**} & \multirow{2}{*}{**Precision**} & \multirow{2}{*}{**F-1 Score**} & \multicolumn{2}{c}{**Balanced Error Rate**} \\ \cline{2-6}  & Train & 0.79 & 0.07 & 0.58 & 0.13 & 0.47 \\ \cline{2-6} Regression & Test & 0.79 & 0.08 & 0.62 & 0.14 & 0.47 \\ \hline \multirow{2}{*}{Naïve Bayes} & Train & 0.78 & 0.06 & 0.42 & 0.10 & 0.48 \\ \cline{2-6}  & Test & 0.77 & 0.05 & 0.37 & 0.08 & 0.49 \\ \hline \multirow{2}{*}{ANN} & Train & 0.80 & 0.19 & 0.63 & 0.29 & 0.42 \\ \cline{2-6}  & Test & 0.78 & 0.15 & 0.48 & 0.23 & 0.45 \\ \hline \multirow{2}{*}{XGBoost} & Train & 0.83 & **0.26** & 0.78 & **0.39** & 0.38 \\ \cline{2-6}  & Test & **0.80** & **0.20** & 0.65 & **0.31** & **0.41** \\ \hline \multirow{2}{*}{Random Forest} & Train & 0.82 & 0.16 & **0.91** & 0.28 & 0.42 \\ \cline{2-6}  & Test & 0.79 & 0.09 & 0.70 & 0.16 & 0.46 \\ \hline \multirow{2}{*}{CatBoost} & Train & **0.84** & 0.25 & **0.91** & **0.39** & **0.37** \\ \cline{2-6}  & Test & **0.80** & 0.15 & 0.71 & 0.25 & 0.43 \\ \hline \multirow{2}{*}{LightGBM} & Train & 0.82 & 0.17 & 0.83 & 0.28 & 0.42 \\ \cline{2-6}  & Test & **0.80** & 0.13 & **0.72** & 0.22 & 0.44 \\ \hline \hline \end{tabular} Best values are shown in bold.  
\end{table} 
Table 5: Airport dataset results. Table 7 reflects the varying performance of different machine learning models when combined with airport and weather data. XGBoost, CatBoost, and LightGBM continued to emerge as strong candidates, demonstrating excellent accuracy, recall, precision, and balanced error rates. On the other hand, Naive Bayes excelled in recall but at the expense of precision and accuracy. Logistic Regression and ANN showed reasonable performance, while Random Forest exhibited suboptimal results with zero recall, which made it less suitable for this dataset and task. Table 8 reveals the performance of the machine learning models when using a combination of weather and airport data with the application of the SMOTE technique. XGBoost, CatBoost, and LightGBM continued to demonstrate strong performance, showcasing high accuracy, recall, precision, and balanced error rates. Naive Bayes exhibited exceptionally high recall but at the cost of precision, which makes it a suitable choice if recall is the top priority. Logistic Regression and ANN showed reasonable and balanced performance, while Random Forest demonstrated suboptimal results, with low recall and precision.