# IoT Intrusion Detection Taxonomy, Reference Architecture, and Analyses
## 6 Experimental Results and Discussion
### Evaluation Metrics  
We used the following well-established metrics to measure model performance. Classification can be further categorized into binary classification and multi-classification. In binary classes, the class labels are either \"normal\" or \"attack.\" The outcomes for binary classification must be categorized as follows: (i) true positive (_TP_): when an attack is correctly identified as an attack by the model; (ii) false positive (_FP_): when a benign node is defined as an attack; (iii) true negative (_TN_): when a benign node is correctly identified as non-attack; and (iv) false negative (_FN_): when the model identifies an actual attack as a non-attack (i.e., attackers win and defenders lose). These four categories shape the so-called confusion matrix [120]. To evaluate ML models, the following equations derived from the confusion matrix are (universally) used. These matrices are characterized by the following equations:  
\[ACC=\frac{TP+TN}{TP+TN+FP+FN} \tag{1}\]  
\[\text{Sensitivity}=\frac{TP}{TP+FN} \tag{2}\]  
\[\text{PR}=\frac{FP}{FP+TN} \tag{3}\]  
The accuracy (_ACC_) is the percentage of correctly identified cases divided by total cases that were considered (whole populations). Sensitivity is also known as \"recall,\" or true-positive rate (TPR) or simply the \"detection rate.\" Sensitivity intuitively indicates the model's power to correctly identify attacks. Moreover, it provides the ability of the model classifier to find all the positive samples in the training model, while in the receiver operating characteristic curve, or ROC curve plots, FPR is represented on the \(X\)-axis, and TPR is represented on the \(Y\)-axis. In other words, the upper-left hand corner of the plot (image) is the \"ideal\" point, where TPR = 1 and the FPR = 0. Thus, using the ROC curve, efficiency of the model is measured as best when the Area Under the Curve (AUC) is maximized. Accordingly, _ACC_, ROC curve, and AUC (area under the ROC curve) are used to evaluate the model, whereas the ROC curve is a graph used to visualize the performance of classifiers at various threshold settings. Therefore, traditional protection systems, such as Firewalls and Snort, fall short when challenged by such attacks. Table 5 shows a detailed representation of the distribution of various attacks and their anomalies (in just 5% selected data from the whole dataset) and indicates the number of attacks in each category. Table 5 clearly shows that UDP and TCP are the most important services (protocols) affected by malicious attacks. DoS occurs in some unwanted traffic (source and receiver). An attacker sends many vague packets and repeated packets to the victim (target), and as a result, the victim's services become unavailable to other clients and services. The attacker takes advantage of a weak node to launch an attack through the whole network. Through the vulnerable nodes, the attacker starts sending a large amount of fake data through the network to other nodes. Figure 8 shows targeted attack intensity for each of the seven services (from left to right). The service distribution gives a first impression of the scale of challenges confounding the IoT ecosystem. This is a difficult problem that must be addressed and mitigated. IoITD20 dataset: The IoITD20 dataset was developed for anomalous activity detection for the IoT ecosystem. The data consists of various types of IoT attacks as well as normal traffic (i.e., many features that are present are of a general nature). These attacks include Mirai, DoS, Scan, MITM ARP Spoofing, Scan Host port, and Mirai-UPD flooding. The dataset was generated by including laptops, smartphones, Wi-Fi cameras, and other IoT devices [78]. Effectively, the IoTID20 dataset represents a smart-home environment. The authors derived OpTID20 by splitting the devices within the simulation testbed into two groups. The first group represents the attacking devices, while the second group (e.g., Wi-Fi camera) is the IoT victim devices.