# Difficulties and Challenges of Anomaly Detection in Smart Cities: A Laboratory Analysis
## 6 Discussion
Data analysts should have tools to allow them to deploy different machine learning techniques in a simplified manner, so that different results can be compared and they can take advantage of several techniques simultaneously, each with its particular strengths. **Attacks or malfunctions that sometimes go unnoticed. ** None of the anomaly detection techniques is able to properly detect the selective forwarding attacks in the Z1 interface at the gateway. The study of the selective forwarding periods with the models including only variables from the Z1 network (Figure 9(b),c) show poor detection results (i.e., low TPR or high FPR). It is worth noting that, although the model trained with dataset _tr_all_ shows good detection results during the 75% selective forwarding attack period (Figure 8(c)), the malfunction raising signals of anomaly is the collapse of the SCK network and not the selective forwarding. This can also be seen examining the SCK network alone (Figure 9(c)). This enhances our previous conclusion that it is necessary not only to consider large models including all the possible variables, but also smaller ones including only meaningful variables. Furthermore, after analyzing in depth the results and reviewing the normal operation of the Z1 network, we can conclude that the lack of good results detecting the selective forwarding attacks is due to the unreliable communication between the Z1 nodes. Figure 9(a) shows that even the jamming attack, which has been clearly detected in other networks, cannot be detected in the Z1 network. The most plausible cause for this is a large packet loss during the attack-free period. Additionally, it should be noted that, in this scenario, smart city administrators cannot notice the original packet loss, since the Z1 nodes have mechanisms to resend lost packets and, eventually, most of the information arrives at the gateway. From this fact, we can draw some conclusions. Firstly, city administrators should not trust that WSN providers properly configure their networks, even though sensor readings are apparently being received correctly at the city datacenters. Malfunctions, such as the one in the Z1 network, are detrimental for the nodes' battery life and also for their already restrained processing power. In addition, these malfunctions fill the electromagnetic spectrum with useless packets, causing interference to other networks. Secondly, analysts should take into account that the response to the same attack can be different depending on the affected network. For instance, in these experiments, all the models should have detected the jamming attacks in all the networks, since the attacks aim at the 2.4 GHz band. However, the anomaly was clearly detected in just two of the networks. **Availability of anomaly-free data. ** Machine learning techniques use training processes to learn the mathematical models that are later used to classify data (normal samples or anomalies in our case). This training process, as previously shown, is based on the availability of a sufficiently large set of anomaly-free data. However, many anomaly detection strategies are designed and implemented once the networks have already been deployed. At this point, logically, it is difficult to guarantee that any set of real data is free of anomalies. Therefore, IT administrators and WSN providers should establish initial periods where networks are monitored carefully until they work as expected and networks generate anomaly-free data. Afterwards, given the fact that the behaviour of these networks evolves, models must be re-trained. At this moment, data analysts should bear in mind that networks can go through long periods of malfunctioning, so this also must be taken into account when re-training the models.