## Materials and Methods
### Data Analysis
#### 2.4.2 Classification of the Drowsy State Using Machine Learning Algorithms  
To distinguish the drowsy state with high accuracy, two ensemble machine learning algorithms were adopted in this study. The first algorithm is a majority voting classifier using LR, SVM, and kNN. The second algorithm is an RF algorithm. A majority voting classifier (MVC) is an example of a general ensemble algorithm. It is an algorithm that reflects the result of a majority vote based on the results of three or more classifications. RF is an ensemble of decision tree models. An RF algorithm can calculate the importance of features and run efficiently on large databases. In addition, the classification using a decision tree algorithm was performed to compare with these two ensemble machine learning algorithms. The datasets that were used for the above algorithms consisted of target data and predictor data. The target data consisted of the drowsiness levels categorized as a whole number from 0 (alert state) to 4 (extremely drowsy state) based on facial expression, as listed in Table 1. The predictor data consisted of all the extracted features identified by hybrid measures and listed in Table 2. To select the proper features that facilitate high-performance classification, sequential backward selection was performed for the MVC (LR, SVM, and kNN) algorithms. Lasso method [31] was used for performing regularization. For the RF algorithm, the numbers of features and estimators that were used for classification were optimized to improve the classification performance. To validate the classification performance indices, \(k\)-fold cross-validation was performed with \(k\) set to 5. As shown in Figure 4, the dataset was randomly partitioned into five equally sized subsets. Among the subsets, a single subset was used as the validation data for testing the model, and the remaining four subsets were used as training data. After that, the cross-validation process was repeated five times, with each \begin{table}
\begin{tabular}{c c c} \hline \hline
**Measurement** & **Details** & **Extracted Feature** \\ \hline \multirow{4}{*}{Physiological Measures} & \multirow{2}{*}{Electroencephalogram (EEG)} & Delta, theta, alpha, and beta wave content of 16 channels \\  & & (Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T3, \\  & & T4, T5, and T6) \\  & & Mean of R-R interval (RRI) \\  & Electrocardiogram (ECG) & Coefficient of variation R-R interval (CVRR) \\  & & Low frequency per high frequency (LF/HF) \\  & & High frequency (HF) content \\  & & The number of eye blinks \\ Behavioral Measures & Visual Behavior & Percentage of eye closure (PERCLOS) \\  & & Distance of centroid movement \\  & Seat Pressure & X coordinate of the centroid \\  & & Y coordinate of the centroid \\  & & Vehicle velocity \\  & & Longitudinal acceleration \\  & & Offset from lane center (Lateral position) \\ Vehicle based Measures & DS Parameter & Steering wheel acceleration (SWA) \\  & & Standard deviation of lateral position (SDLP) \\  & & Time headway (THW) \\  & & Time to lane crossing (TLC) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Details of extracted features. Figure 3: Procedure of data processing [20]. of the five subsets used once as the validation data. The average of the five results were calculated as an evaluation index (\(e\)). Finally, the performance of the algorithms was evaluated. In detail, the performance values of the two classifiers were calculated using detection accuracy (_Acc._), precision (_Pre._), recall (_Rec._) and _F1_, which were defined in the following formulas (1)-(4). \[Acc.=\frac{TP+TN}{FP+FN+TP+TN} \tag{1}\]  
\[Pre.=\frac{TP}{FP+TP} \tag{2}\]  
\[Rec.=\frac{TP}{FN+TP} \tag{3}\]  
\[F1=2\times\frac{Pre.\times Rec.}{Pre.+Rec.} \tag{4}\]  
(TP, TN, FP, and FN in the above formulas (1-4) indicate true positives, true negatives, false positives, and false negatives, respectively.)