## 4 Experimental Results  
### Experiment Results Based on Feature 1 (Index and Scancode)  
We classified into any number of the training set, validation set, and test set to avoid overfitting and underfitting for the experiment, and Table 4 shows experiment results of the training set, validation set, test set, and the cross-validation using Dataset 1.  
Specifically, the training set had the best score for random forest at 0.93, while the rest of the models had a score at 0.90. The validation set had the worst score of random forest at 0.86, while the rest of the models had a score at 0.88. The test set had the worst score for linear SVC at 0.21, while the rest of the models had a score at 0.88. Cross-validation had the worst score for MLP at 0.746, while the rest of the models had a score at 0.86, except for linear SVC.  
Figure 4 shows performance evaluation results of the real keyboard data input from the keyboard device according to Datasets 1 to 3. Cross-validation splits the data repeatedly, and trains multiple models. Accuracy denotes the number of correctly predicted numbers (true positive and true negative), and precision denotes the number of true positive among the number of true positive and false positive. Recall means the number of true positive among the number of true positive and false negative, and F1-score means the harmonic average of precision and recall. Area Under the Curve (AUC) is a summary of the Receiver Operating Characteristics (ROC) curve, and the AUC score falls between the worst 0 value and the best 1 value.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Parameters**} & **Training Set** & **Validation Set** & **Test Set** & **Cross-Validation** \\  & & **Score** & **Score** & **Score** & **Score** \\ \hline KNN & n\_neighbors = 4 & 0.90 & 0.88 & 0.88 & 0.894 \\ \hline Logistic Regression & C = 0.00001, & 0.90 & 0.88 & 0.88 & 0.895 \\  & L1 regularization & 0.90 & 0.88 & 0.21 & 0.773 \\ \hline Linear SVC & - & 0.90 & 0.88 & 0.88 & 0.895 \\ \hline Decision tree & max\_depth = 1 & 0.90 & 0.88 & 0.88 & 0.895 \\ \hline Random Forest & n\_estimators = 2 & 0.93 & 0.86 & 0.85 & 0.866 \\ \hline Gradient Boosting & learning\_rate = 0.1, & 0.90 & 0.88 & 0.88 & 0.886 \\ \hline SVM (Support Vector Machine) & C = 1 & 0.92 & 0.88 & 0.88 & 0.894 \\ \hline MLP (multi-layer perceptron) & max\_iter = 0.00001, & 0.90 & 0.88 & 0.88 & 0.746 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Experiment results of training set, validation set, test set, and cross-validation using Dataset 1.  
To be more specific about the results for each dataset, gradient boosting had high performance in Dataset 1, while random forest had high performance in Datasets 2 and 3. Nevertheless, most of the models could not measure precision, recall, and F1-score. KNN, logistic regression, decision tree, SVM, and MLP models could not measure in Dataset 1. In Dataset 2, KNN, logistic regression, linear SVC, decision tree, gradient boosting, SVM, and MLP models could not measure, and KNN, logistic regression, linear SVC, decision tree, gradient boosting, and SVM models cannot measure in Dataset 3. In other words, a dataset consisting of index and scancode as a feature cannot classify scancode input from the keyboard and generated random scancode, which means that an attacker is very unlikely to succeed in a password stealing attack.