# Difficulties and Challenges of Anomaly Detection in Smart Cities: A Laboratory Analysis
## 6 Discussion  
In this section, we discuss several issues related to the simulation and results presented above, presenting the difficulties encountered in the application of two different machine learning techniques in our simulated smart city environment. **Using a single model versus several ones. ** On the one hand, it can be seen that using a single model including variables from all the networks (i.e., models trained with dataset _tr_all_) usually results in good detection rates (Figure 9a,c). Moreover, dealing with a single model instead of dealing with multiple ones from different networks simplifies administrators' task. In this way, administrators only have to review the prediction capabilities of a single model, retrain it if its performance decreases, and monitor the attack predictions of only one model that can be viewed as a summary of the anomalies occurring on a group of networks. On the other hand, generating several models including different variables selected according to predefined meaningful reasons can help administrators to identify the causes of the anomalies or the affected equipment. For example, in the case of our jamming attack, conclusions derived from using all variables from all networks (Figure 9a) are limited to state that there is some malfunction affecting some element in the whole simulation environment. Nevertheless, studying the jamming attack predictions using a different model for each network (Figures 10a, 11a, 12a and 13a) shows that conclusions drawn can be wider. For instance, the data analyst could doubtlessly see that there is an anomaly affecting the RPI and the SCK networks (Figures 11a and 12a). Thus, administrators could easily identify the affected equipment. Therefore, the difficulty here lies in deciding which approach is more appropriate for each smart city context, since there is a trade-off between simplifying the anomaly detection tasks and being able to derive more information from the results. Smart city administrators must be aware that there is no black box technique that can detect any kind of attack and the affected equipment in a scenario such as the smart city. In fact, a complex procedure and a framework are proposed in [25] in order to assist smart city administrators to select the most probable attack, and delimit corrupted data and affected devices in case of incident. **Choosing the right machine learning technique. ** If we performed a literature review on machine learning techniques for anomaly detection, we would probably come to the conclusion that SVM-based techniques are the ones that perform best. In our work, we have seen that both OC-SVM and isolation forest are suitable to detect the random jamming attack (Figure 9a) and OC-SVM models perform slightly better at detecting attacks in most of the situations (higher TPR). Moreover, in general, OC-SVM performs better at not misclassifying attack-free samples (lower FPR). This fact is important in smart cities so as to prevent overwhelming data analysts with too many false positives. However, in certain  
Figure 13: (**a**) prediction results for the jamming attack period using the variables from the WP network (training with dataset _tr_up_ and test with dataset _te_wp_jam_); (**b**) prediction results for the 50% selective forwarding attack period using the variables from the WP network (training with dataset _tr_up_ and test with dataset _te_wp_sf50_); (**c**) prediction results for the 75% selective forwarding attack period using the variables from the WP network (training with dataset _tr_wp_ and test with dataset _te_wp_sf75_). cases (Figures 9(a) and 9(a),b), the isolation forest models are capable of detecting more anomalies without producing more false positives. Moreover, the initialization of the isolation forest was simple and, unlike OC-SVM models, it did not require a grid search to find the best parameters. Therefore, choosing one machine learning technique or another depends on the specific goals and characteristics of the environment.