## 3 Results
### Model Performance  
The results of applying the six ML models using eight variables are presented in Table 2. To compare the performance of the six models, the accuracy, specificity, sensitivity, and AUROC values were obtained and compared [28]. In this study, the final model was selected based on the AUROC value, considering both sensitivity and specificity owing to the high-class imbalance in the data. A grid search was performed to determine the optimal hyperparameter values of each ML model [29]. For each model, Table 2 shows the values of the optimized hyperparameters and the model's performance. Figure 2: Machine learning (ML) modeling process. Figure 3 shows the receiver operating characteristic (ROC) curves of the six ML models. Larger areas indicate larger AUROC values, which in turn denote higher performance. Comparison of ROC curves showed that AdaBoost had the highest AUROC of 0.716. Sensitivity and specificity were 0.741 and 0.691, respectively, with an accuracy of 0.696. The number of trees in AdaBoost was tuned using the grid search algorithm, and it was found that the best AUROC, specificity, sensitivity, and accuracy performance was obtained when the number of trees was 50. The second best-performing model is logistic regression, and it achieved an AUROC of 0.698 with an L1 penalty and C = 0.1 (determined using grid search). We developed the final model using the AdaBoost model, because it achieved the highest AUROC of the six models. In addition, we conducted cross-validation for model generalization verification. Both three-fold and five-fold cross validations showed slightly lower performance than AdaBoost. However, in five-fold cross-validation, the AUROC mean was 0.696, which is close to that of our final model (0.716), confirming that our model was not overfitted to the test dataset. Table 3 shows the mean and standard deviation of AUROC values of ML algorithms. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Variable** & \
\begin{tabular}{c} **Without BM Group** \\ **(_n = 1145_)** \\ \end{tabular} &
\begin{tabular}{c} **With BM Group** \\ **(_n = 137_)** \\ \end{tabular} & \(p\)**-Value** \\ \hline Age & 56.8 \(\pm\) 11.2 & 53.0 \(\pm\) 10.6 & \textless{}0.001 \\ \hline Sex & & 0.005 \\ \hline Male & 902 (78.8\%) & 93 (67.9\%) & \\ Female & 243 (21.2\%) & 44 (32.1\%) & \\ \hline Smoking & & 0.392 \\ \hline Non-smoker & 785 (68.6\%) & 97 (70.8\%) & \\ Ex-smoker & 252 (22.0\%) & 24 (17.5\%) & \\ Current-smoker & 108 (9.4\%) & 16 (11.7\%) & \\ \hline ECOG PS & & \textless{}0.001 \\ \hline
0 & 910 (79.5\%) & 84 (61.3\%) & \\
1 & 198 (17.3\%) & 44 (32.1\%) & \\ \textgreater{}1 & 37 (3.2\%) & 9 (6.6\%) & \\ \hline Pathological tumor stage & & 0.063 \\ \hline
1–2 & 594 (51.9\%) & 59 (43.1\%) & \\
3–4 & 551 (48.1\%) & 78 (56.9\%) & \\ \hline Fuhrmann nuclear grade & & 0.156 \\ \hline
1 & 21 (1.8\%) & 4 (2.9\%) & \\
2 & 277 (24.2\%) & 30 (21.9\%) & \\
3 & 492 (43.0\%) & 49 (35.8\%) & \\
4 & 355 (31.0\%) & 54 (39.4\%) & \\ \hline Heng risk group & & 0.042 \\ \hline Favorable & 526 (46.2\%) & 50 (36.5\%) & \\ Intermediate & 557 (48.6\%) & 75 (54.7\%) & \\ Poor & 59 (5.2\%) & 12 (8.8\%) & \\ \hline Lung metastasis & & 0.001 \\ \hline No & 370 (32.3\%) & 24 (17.5\%) & \\ Yes & 775 (67.7\%) & 113 (82.5\%) & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of characteristics between groups with and without brain metastasis (BM).