# A Deep Learning Approach to Analyze Airline Customer Propensities: The Case of South Korea
## 4 Results
### Experimental Results  
Figure 6a,b show the accuracy of different models for predicting customer churn risk and customer satisfaction, respectively. In both graphs, the x-axis represents models, and the y-axis represents the accuracy (in percentage) calculated by Equation (2). The results of precision, recall, F1 scores and accuracy for each model are given in Table 4. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Models** & **Parameters** & **Description** & **Options** & **Selected** \\ \hline kNN & n\_neighbors & Number of neighbors & 3, 5, 7 & 3 \\ \hline Decision Tree & max\_depth & Maximum depth in tree & 4, 8, 16 & 4 \\ \hline RF & max\_depth & Maximum depth in each tree & 4, 8, 16 & 4 \\  & n\_estimators & Number of trees in the forest & 100, 200, 300, 500, 100 \\ \hline XGBost & max\_depth & Maximum depth in each tree & 4, 8, 16 & 4 \\  & n\_estimators & Number of trees in the forest & 100, 200, 300, 500, 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameters of machine learning models. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Models** & **Optimizer** & **Loss** & **Epochs** & **Batch Size** \\ \hline CNN & ADAM & Categorical Cross Entry & 150 & 5 \\ \hline CNN-LSTM & ADAM & Categorical Cross Entry & 150 & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameters of deep learning models. There are two goals of this experiment. The first goal is to determine the most accurate machine learning and deep learning models for the prediction of airline customer propensities. From Figure 6, we can observe that among machine learning models, the RF model achieves the highest accuracy of 84% and 86% in predicting customer churn risk \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multirow{2}{*}{**Prediction Goal**} & \multicolumn{2}{c}{**Precision**} & \multirow{2}{*}{**Recall (\%)**} & \multirow{2}{*}{**F1 Score (\%)**} & \multirow{2}{*}{
\begin{tabular}{c} **Accuracy** \\ **(\%)** \\ \end{tabular} } \\ \hline \multirow{2}{*}{kNN} & Customer Churn Risk & 85 & 84 & 84 & 84 \\  & Customer Satisfaction & 84 & 84 & 84 & 84 \\ \hline \multirow{2}{*}{Decision Tree} & Customer Churn Risk & 79 & 79 & 79 & 79 \\  & Customer Satisfaction & 80 & 79 & 79 & 79 \\ \hline \multirow{2}{*}{XGBoost} & Customer Churn Risk & 82 & 83 & 82 & 82 \\  & Customer Satisfaction & 84 & 84 & 84 & 84 \\ \hline \multirow{2}{*}{RF} & Customer Churn Risk & 85 & 84 & 84 & 84 \\  & Customer Satisfaction & 86 & 86 & 85 & 86 \\ \hline \multirow{2}{*}{CNN} & Customer Churn Risk & 88 & 87 & 87 & 87 \\  & Customer Satisfaction & 88 & 86 & 85 & 86 \\ \hline \multirow{2}{*}{CNN-LSTM} & Customer Churn Risk & 94 & 94 & 93 & 94 \\  & Customer Satisfaction & 92 & 90 & 90 & 90 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Precision, recall, F1 score and accuracy of different models. Figure 6: Accuracy (%) of different models in predicting (a) customer churn risk and (b) customer satisfaction. and satisfaction, respectively. Considering that the RF model is constructed in an ensemble manner, it can overcome the overfitting and underfitting issues of other machine learning models studied in this paper. From Figure 6, we can also observe that deep learning models outperform machine learning models in most cases. Specifically, the CNN-LSTM model achieves the highest accuracy of 94% and 90% in predicting customer churn risk and customer satisfaction, respectively. This occurs because the deep learning models generally learn high-level features from the data incrementally, enabling us to automatically discover essential features for classification. Among deep learning models, CNN-LSTM outperforms the conventional CNN model by 7% and 4% in terms of accuracy in predicting customer churn risk and satisfaction, respectively. Recall from Section 3.4 that the main feature of the CNN-LSTM model is that it modifies the underlying structure of the conventional CNN model by preserving its features.