# IoTDS: A One-Class Classification Approach to Detect Botnets in Internet of Things Devices
## 5 Results and Discussion
### Predictive Performance
#### 5.3.2 Classifiers' Performance
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Classifier** & **Sample Size** & **Precision** & **Recall** & **Specificity** & **Accuracy** & **AUC** & **F1-Score** \\ \hline \multirow{8}{*}{EE} & 0.1 & 0.6617 & 0.8245 & 0.8998 & 0.8846 & 0.8621 & 0.7303 \\  & 0.2 & 0.6605 & 0.8206 & 0.9000 & 0.8840 & 0.8603 & 0.7279 \\  & 0.3 & 0.6598 & 0.8193 & 0.9001 & 0.8838 & 0.8597 & 0.7268 \\  & 0.4 & 0.6599 & 0.8195 & 0.8999 & 0.8837 & 0.8597 & 0.7269 \\  & 0.5 & 0.6601 & 0.8191 & 0.9001 & 0.8838 & 0.8596 & 0.7268 \\  & 0.6 & 0.6592 & 0.8175 & 0.9002 & 0.8835 & 0.8588 & 0.7256 \\  & 0.7 & 0.6593 & 0.8176 & 0.9002 & 0.8835 & 0.8589 & 0.7257 \\  & 0.8 & 0.6592 & 0.8182 & 0.9001 & 0.8836 & 0.8591 & 0.7259 \\  & 0.9 & 0.6591 & 0.8175 & 0.9001 & 0.8834 & 0.8588 & 0.7256 \\ \hline \multirow{8}{*}{IF} & 0.1 & 0.5572 & 0.9856 & 0.7999 & 0.8372 & 0.8928 & 0.7109 \\  & 0.2 & 0.5583 & 0.9855 & 0.8009 & 0.8380 & 0.8932 & 0.7118 \\  & 0.3 & 0.5569 & 0.9856 & 0.7997 & 0.8370 & 0.8927 & 0.7107 \\  & 0.4 & 0.5594 & 0.9855 & 0.8014 & 0.8384 & 0.8934 & 0.7126 \\  & 0.5 & 0.5590 & 0.9855 & 0.8012 & 0.8382 & 0.8933 & 0.7123 \\  & 0.6 & 0.5582 & 0.9856 & 0.8007 & 0.8378 & 0.8931 & 0.7117 \\  & 0.7 & 0.5584 & 0.9853 & 0.8008 & 0.8378 & 0.8931 & 0.7118 \\  & 0.8 & 0.5581 & 0.9851 & 0.8007 & 0.8377 & 0.8929 & 0.7115 \\  & 0.9 & 0.5586 & 0.9855 & 0.8011 & 0.8382 & 0.8933 & 0.7121 \\ \hline \multirow{8}{*}{LOF} & 0.1 & 0.7119 & 0.9835 & 0.8998 & 0.9166 & 0.9416 & 0.8257 \\  & 0.2 & 0.7169 & 0.9915 & 0.9015 & 0.9196 & 0.9465 & 0.8320 \\  & 0.3 & 0.7203 & 0.9919 & 0.9031 & 0.9210 & 0.9475 & 0.8345 \\  & 0.4 & 0.7227 & 0.9918 & 0.9043 & 0.9219 & 0.9481 & 0.8361 \\  & 0.5 & 0.7249 & 0.9915 & 0.9054 & 0.9227 & 0.9484 & 0.8375 \\  & 0.6 & 0.7271 & 0.9911 & 0.9065 & 0.9235 & 0.9488 & 0.8388 \\  & 0.7 & 0.7289 & 0.9904 & 0.9074 & 0.9241 & 0.9489 & 0.8397 \\  & 0.8 & 0.7313 & 0.9889 & 0.9086 & 0.9248 & 0.9488 & 0.8407 \\  & 0.9 & 0.7337 & 0.9873 & 0.9099 & 0.9255 & 0.9486 & 0.8418 \\ \hline \multirow{8}{*}{OSVM} & 0.1 & 0.6882 & 0.8946 & 0.8986 & 0.8978 & 0.8966 & 0.7770 \\  & 0.2 & 0.6890 & 0.8927 & 0.8992 & 0.8979 & 0.8960 & 0.7769 \\ \cline{1-1}  & 0.3 & 0.6896 & 0.8928 & 0.8995 & 0.8982 & 0.8962 & 0.7773 \\ \cline{1-1}  & 0.4 & 0.6901 & 0.8932 & 0.8997 & 0.8984 & 0.8964 & 0.7778 \\ \cline{1-1}  & 0.5 & 0.6903 & 0.8927 & 0.8999 & 0.8985 & 0.8963 & 0.7782 \\ \cline{1-1}  & 0.6 & 0.6904 & 0.8926 & 0.9000 & 0.8985 & 0.8963 & 0.7782 \\ \cline{1-1}  & 0.8 & 0.6904 & 0.8929 & 0.9000 & 0.8985 & 0.8964 & 0.779 \\ \cline{1-1}  & 0.9 & 0.6904 & 0.8928 & 0.8999 & 0.8985 & 0.8964 & 0.7779 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Evaluation of sample size for EE, IF, LOF and OSVM in all datasets. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Classifier** & **Time-Window Size (s)** & **Precision** & **Recall** & **Specificity** & **Accuracy** & **AUC** & **F1-Score** \\ \hline \multirow{4}{*}{EE} & 1 & 0.6599 & 0.8187 & 0.9001 & 0.8837 & 0.8594 & 0.7267 \\  & 5 & 0.6879 & 0.8931 & 0.8992 & 0.8979 & 0.8961 & 0.7759 \\  & 10 & 0.6101 & 0.6883 & 0.8990 & 0.8566 & 0.7937 & 0.6394 \\  & 30 & 0.5542 & 0.6790 & 0. After selecting the time-window and sample size, an optimisation of the classifiers' hyperparameters was performed. We used a random search, a lightweight and effective method [38], to find the best combination of hyperparameters, as presented in Table 5. All the algorithms were submitted to tests to evaluate their best hyperparameters in the same normalised dataset, except for IF, which was not submitted to normalised data because it is a tree ensemble method that can deal with not normalised data. In our evaluation, we used the F1-score to find the best hyperparameters in each algorithm. The optimised hyperparameters for each algorithm are presented in Table 6. Figure 5 presents a boxplot of the F1-scores when using a 1-s time-window, a training sample size corresponding to 10% of legitimate data, and the optmised hyperparameters.