# Intrusion Detection and Prevention in CoAP Wireless Sensor Networks Using Anomaly Detection
## 4 Experimental Evaluation
### A Binary Class Problem Approach  
We proceed by discussing the results obtained by following the binary class problem approach. For this approach, we considered the following two labels in our evaluation: NORMAL with Label \"0\" and ERRONEOUS with Label \"1\". For this approach, we only employ the most accurate feature extraction method, based on the previous obtained multi-class results, and that is LDA. The method of singular value decomposition and two components is applied. Once more, data is pre-processed with a Standard Scaler, with mean 0 and standard deviation 1. The features of the data are reduced by LDA to 2 components, and the method is set to singular value decomposition (SVD). Next, a grid search for  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Scoring Metric Accuracy** & & & & Degree (Valid on & & & \\
**C** & **Kernel** & **Iterations** & **Polynomial Only**) & **Gamma** & **Coef** & **Decision Function Shape** & **Accuracy** \\ \hline
0.452632 & “linear” & 30 & 1 & 0.10000 & 0.000000 & ‘over’ & 0.932782 \\
0.578947 & ‘rb” & 30 & 1 & 0.810526 & 0.000000 & ‘over’ & 0.632368 \\
0.200000 & ‘poly’ & 30 & 1 & 0.147368 & 0.000000 & ‘over’ & 0.933816 \\
0.284211 & ‘sigmoid’ & 30 & 1 & 0.100000 & 0.000000 & ‘over’ & 0.753361 \\ \hline \hline \end{tabular}
\end{table}
Table 2: SVM Parameters obtained using accuracy as the scoring metric (previously processed with multi-class LDA).  
Figure 8: Multi-class problem—Grid Search SVM with sigmoidal Kernel, One vs. Rest decision function shape, 30 iterations, and a scoring criterion of accuracy, confusion Matrix (**a**) and ROC Curves (**b**).  
the best parameters of the SVM classier is performed, with a K-fold of 3 and setting the scoring metric to _Accuracy_. We employ the following SVM Kernels: Linear, RBF, polynomial and sigmoidal, and the obtained results are illustrated in Figures 9 and 10.  
For each Kernel, the best parameters and accuracy results are found by a grid search under the previously described conditions. The corresponding results are presented in Table 3. As can be observed, the Sigmoidal Kernel is able to accurately classify NORMAL observations with a ratio of false positives of 1%, as can be seen in Figure 10a, though with an amount of false negatives of 20%. For the polynomial Kernel, there are 0% false negatives and 28% of false positives, as can be observed in Figure 9a. In terms of recall, both polynomial and sigmoidal kernels present similar results, polynomial being 2% ahead of sigmoidal, as can be seen in Figures 9b and 10b.  
Figure 10: Binary class problem—Grid Search SVM with sigmoidal Kernel, One vs. Rest decision function shape, 30 iterations, and a scoring criterion of accuracy confusion matrix (**a**) and ROC Curves (**b**).  
Figure 9: Binary class problem—Grid Search SVM with polynomial Kernel, One vs. Rest decision function shape, 30 iterations, with scoring criterion of accuracy, confusion Matrix (**a**) and ROC Curves (**b**).  
In conclusion, from our previous discussion we may note that, in our work, we employ pre-processing algorithms and train classifiers for building the intrusion model, according to the multi and binary class approaches. As we have observed from the results of our experimental evaluation, for the multi-class problem approach the proposed system is capable of distinguishing between the four types of misbehaving scenarios implemented, and to reach an accuracy of 93% for the best SVM classier, with accuracy as the scoring metric and when employing LDA to perform feature extraction. As for the binary class problem approach, considering the best SVM parameters, a classification of NORMAL and ERRONEOUS behaviors can reach an accuracy of 92% and an _F_Measure_ of 98%.