# Attack Classification Schema for Smart City WSNs
## 5 Proof of Concept
### Analysis
#### 5.2.1 Basic Detection Analysis  
In this section, a basic analysis using three different intrusion detection techniques is performed. Firstly, rule-based detection, which generally triggers a low rate of false positives, is used defining a threshold (the AT1 type in Section 3.2.1) to unveil attacks affecting the sound service. Secondly, anomaly-based detection (the AT2 type in Section 3.2.1) is used to complement the previous rule-based detection, thus adding the possibility of detecting unknown attacks. Due to the excellent performance of OC-SVM in the smart city context shown in the comparative study in [28], in this proof of concept, OC-SVM is used to detect attacks on aggregated parking data. Finally, the third technique correlates the alarms triggered by the previous two detection techniques (the AT4 type in Section 3.2.1). In this way, this demonstration shows that different techniques are required to detect attacks in this type of scenario. Additionally, it shows how smart city administrators (who receive data from all of the different subsystems) are capable of improving the attack detection results in the WSNs of an area in the city, especially when they take into account that a single attack can affect at the same time different networks, communication protocols, providers, etc. In order to perform the detection analysis, we first divide the data from this simulation into three sets. Fifty percent of the data are used for training purposes; 25% is used as a validation data to tune the parameters of the algorithms; and the last 25% is used as a test dataset. The selective forwarding attack is only applied to the validation and the test data. The three types of alarms are implemented as follows:  
* We use rule-based detection on the sound data. From this service, we aggregate the number of lost application packets in one-hour windows, and we define a threshold to determine the maximum number of lost application packets that is considered normal (i.e., not due to an attack). Setting a high threshold implies decreasing the detection rate (i.e., the number of detected attacks divided by the number of total attacks) and the false positive rate (i.e., instances that are incorrectly classified as attacks divided by the number of total instances that are not attacks). * We use OC-SVM on the parking data. In this case, the procedure to detect attacks has the following steps: 1. We group the sensor nodes into clusters by location using DBSCAN [35]. Figure 6 shows the division by clusters. 2. We aggregate the number of changes in the parking spots per cluster into one-hour windows. 3. We train an OC-SVM model with the aggregated data. This model is used later to test whether new aggregated parking data have been gathered in a period when the network was working properly or when the network was under attack. In order to evaluate the detection rate considering various false positive rates, we trained various OC-SVM models selecting different values for their hyperparameters. For example, increasing the value of the hyperparameter \(\
u\), which defines the maximum fraction of outliers present in the training data, also increases the detection rate and the false positive rate. * We create a simple correlation rule (the AT4 type in Section 3.2.1) that is triggered when various alarms from any of the previous two detection techniques have been triggered in one hour. In order to build this correlation rule, we compare the detection rate and the false positive rate of varying amounts of alarms triggered by any of the two detection engines. As shown below, by decreasing the number of required alarms in the correlation rule, we can gradually increase both the detection rate and the false positive rate. The results are shown in Figure 7.  
Generally, alarms based on thresholds are more reliable than alarms triggered by machine learning techniques.