# A Specification-Based IDS for Detecting Attacks on RPL-Based Network Topology
## 4 Evaluation Results and Discussion
### Simulation Results and Discussion  
#### 4.3.1 Detection Efficiency  
We divided the attacks into two groups which have similar results when detected by our IDS. The first group contains the Rank attack, Sinkhole attack, and Neighbor attack--the threats which are detected only by the specified states. The second group includes the local repair and the DIS attacks, which involved both the specification states and statistic collection to reveal. Table 4 below shows the TPR and FPR of the Rank attack, Sinkhole attack and the Neighbor attack after 4 min, when the Rank attack already initiated (at the third minute) and the IDS has just collected the first two IDS data packets from its neighbor. As can be seen from the table, we obtained ideal IDS results, where the TPR is 100% and the FPR is 0%. These results can be explained as follows. The DIO sequence checking in Modules 2 and 3 ensure that there is no inconsistency in DIO and Rank information between the IDS data reported by the neighbours. As a result, the cluster head will know most of the parent--child relationships and their accurate corresponding ranks. Regarding the Rank attack, given its nature which is choosing the worst parent as the preferred parent and changing it frequently, the cluster head will detect the behaviour of breaking the rank rule and raise alarm about the child node for choosing inappropriate parent. Therefore, the Rank attack is detected quickly with high accuracy. Regarding the Sinkhole attack, our implementation lets the attacker keeps informing that it has the rank of the Sink to attract its neighbours. As the attacker is not the actual Sink, before initiating the attacks, it would have a preferred parent. Moreover, this parent-child relationship would be recorded by one of the cluster heads. When the attacker manipulates its new rank to the Sink's rank, such relationship will become illegal, because the child now has a better rank than the parent. This illegal relation will be detected by Module 4 of our IDS. Regarding the neighbor attack, the neighbor attack involves some DIO replications, which will be detected by Modules 2 and 3 through the DIO duplication of the sequence number received. This detection is straightforward, fast and accurate because it does not need to collect much IDS information to decide. Our IDS shows high accuracy results not long after the attack initiating. However, when letting the IDS works for a long time, when the TPR is still ideal, the FPR increases significantly and makes the IDS become less accurate. For example, the TPR and FPR after 10 min detecting RA, SA and \begin{table}
\begin{tabular}{c c c c c c c} \hline  & **FP** & **TP** & **FN** & **TN** & **TPR (\%)** & **FPR (\%)** \\ \hline RA & 0 & 88 & 0 & 7656 & 100 & 0 \\ SA & 0 & 88 & 0 & 7656 & 100 & 0 \\ NA & 0 & 88 & 0 & 7656 & 100 & 0 \\ \hline \end{tabular}
\end{table} 
Table 4: TPR and FPR of Rank attack, sinkhole attack and Neighbor attack after 4 min. NA scenarios are shown in Table 5 below. The figure shows that the FPR increase to about 2%-5%. The reason is that the initiated attacks in the tampered nodes have affected its neighbors around, make those nodes work the same way as the attackers, and therefore become difficult to separate. In order to minimize the FPR, the nodes which are detected as the malicious source should be removed from the network, for example, by adding to a blacklist and asking all other relevant nodes to skip nodes in that list. After removing the nodes, the IDS will stop judging for a certain time to help to stabilize the network before restarting in a new detection cycle. The second group of attacks includes the Local repair and the DIS attack. The difference to compare with the first group is that in this group, observing that a node visit a state is not enough to conclude this node to be malicious. This observation is only considered as part of the statistic evidences. Only when a node visits a state more than a threshold of times during a period, the IDS has the right to raise alarm about the threat.