# Attack Classification Schema for Smart City WSNs
## 5 Proof of Concept
### Analysis
#### 5.2.1 Basic Detection Analysis
If an attacker performing a selective forwarding in the gateway drops a large amount of packets, then the AT1 alarm implemented with a threshold for the number of lost application packetsfrom the sound service would be enough in order to discover the attack. However, the most challenging situations arise when the attacks affect few sound packets and many parking packets, which is the scenario that is implemented in this proof of concept. Thus, we demonstrate the advantages of using correlation rules and the classification schema proposed in this paper. Figure 8: Detection rate comparison between the three techniques at a 5% false positive rate. Figure 7: Evolution of the detection rate and the false positive rate of the three techniques. In the rule-based detection, increasing the packet loss threshold decreases the detection rate and the false positive rate (e.g. defining \(threshold=9\) results in a 10.20% detection rate and a 15.38% false positive rate; defining \(threshold=10\) results in a 8.16% detection rate and a 3.85% false positive rate). In the one-class (OC)-SVM detection, a higher value of the hyperparameter \(\
u\) increases the detection rate and the false positive rate (e.g., defining \(\
u=0.01\) results in a 14.12% detection rate and a 4.41% false positive rate; defining \(\
u=0.02\) results in a 17.75% detection rate and a 9.31% false positive rate). In the detection by correlation, including more alarms in the correlation rule decreases the detection rate and the false positive rate (e.g., aggregating two alarms from any of the two systems results in a 33.47% detection rate and a 5.45% false positive rate; aggregating three alarms results in a 15.14% detection rate and a 1.36% false positive rate). Figures 7 and 8 show the performance of the detection techniques in this scenario. Figure 7 shows that, in general, for a false positive rate lower than 25%, the combination of alarms from the first two techniques in a correlation rule outperforms the other techniques operating separately. In the smart city context, the false positive rate must be kept low to avoid overwhelming system administrators. As seen in Figure 8, setting a maximum 5% false positive rate implies that the detection rate using correlation detection is more than 2.2-times higher than using only OC-SVM and more than 3.7-times higher than using only rule-based detection. Therefore, this outcome shows that administrators can significantly improve detection results by implementing simple correlation rules as shown in this example. These results are particularly interesting when taking into account that the detection scenario is highly complex due to a 20% selective forwarding dropping rate (only 10% more than the normal loss rate for the sound network) and the difficulty of detecting anomalies in nodes spread in a wide area. There is also a need to detect the attacks within a one-hour window, which allows time to apply short-term contingency actions. Therefore, the performance of the proposed detection process would increase in scenarios with a higher dropping rate, where the attack is focused on a narrow area or without short-term contingency requirements. This simulated detection process also shows that alarms triggered by the correlation rule must be considered as highly severe. The other alarms are also important to trace incidents in the smart city, but they are less reliable and provide less information about the incidents than correlation rules. In the next section, starting from the reception of a highly severe alarm, we used the proposed schema to classify the incident into an attack model.