## 3 Materials and Methods
### Experimental Setup  
For evaluating the prediction models, we split the data into 80% for the training set and 20% for the evaluation set. The 5-fold stratified cross-validation procedure [43] is applied to the training set. In the procedure of 5-fold stratified cross-validation, the dataset is randomly partitioned into five parts, 4 folds of the training set are used to train classification models, and the remaining 1 fold is used to validate the model. In the presence of class imbalance, stratified cross-validation is a commonly used method because folds are made by an equal number of samples for each class to preserve optimally balanced distributions. This procedure can end up providing enough representatives of minority and majority classes in each fold. Principally, several machine learning classifiers need to set up a few parameters is shown in Table 1.  
Grid search has been applied to hyper-parameter tuning to find the optimal ones. For the MLP models, we use the one and three hidden layers with 5 nodes. These models are optimized by [PERSON], and we set the learning rate is 0.001 with \"sigmoid\" activation function. In the RF classifier, the number of estimators has selected 250, 500, 750, 1000, 1250, 1500 trees, and the quality of split-measured criteria is employed \"gini\" for the Gini impurity and \"entropy\" for the information gain, respectively. The GBT classifier is to use learning rates and the number of estimators, which are calculated as the optimal combinations to materialize better performance. The hyper-parameters of the KNN classifier are \begin{table}
\begin{tabular}{c c c} \hline \hline
**Classifier** & **Parameter** & **Range** \\ \hline \multirow{2}{*}{RF} & Number of estimators & 250, 500, 750, 1000, 1250, 1500 \\  & Criterion & gini; entropy \\ \hline \multirow{2}{*}{GBT} & Learning rate & 0.15, 0.1, 0.05, 0.01, 0.005, 0.001 \\  & Number of estimators & 250, 500, 750, 1000, 1250, 1500 \\ \hline \multirow{2}{*}{KNN} & Number of k range & 3, 6, 9 \\  & Weight options & uniform, distance \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Range of parameters for classification models. weights and number of neighbors. The weights are either set up to \"uniform\", where all points in each neighborhood are weighted the same, or \"distance\", where closer points are more heavily weighted toward the decision. The setting of the neighbor numbers refers to how many neighboring points are to fall inside of one group. In our study, a rank-based non-parametric Friedman test [44] is used to compare the various machine learning classifiers. The Friedman test is computed as follows:  
\[\chi_{F}^{2}=\frac{12D}{K(K+1)}\left[\sum_{k=1}^{K}AvR_{j}^{2}-\frac{K(K+1)^{ 2}}{4}\right]\]  
where \(D\) is the number of datasets and \(K\) is the number of classification algorithms. \(R_{j}\) is the rank of the \(j\)th classification algorithm on a dataset. If chi-square distribution \(\chi_{F}^{2}\) with \((d-1)\) degrees of freedom is larger than a critical value, the null hypothesis is rejected. The f-score, type I error, type II error, balanced accuracy and geometric mean evaluation metrics [45; 46] are utilized to compare the classifier's performance and select the best model. F-score is a metric that considers both precision and sensitivity. Precision is calculated as the number of correct positive results divided by the total number of results that the model identified as positive. For smoking cessation intervention, type I error appears when successful smoking quitters are misclassified as failed, whereas type II error occurs when smokers who failed in during smoking cessation intervention are misclassified as a successful quitter. In more detail, type I error and type II error are defined as the type I error = \(1-\) specificity and type II error = \(1-\) sensitivity, respectively. Most algorithms are designed to maximize the standard accuracy due to class imbalance. Instead of this, balanced accuracy is suggested to evaluate the prediction models.