# Difficulties and Challenges of Anomaly Detection in Smart Cities: A Laboratory Analysis
## 2 Background
### Smart City Testbeds  
The growing awareness that data reliability and security are of utmost importance has motivated the development of several initiatives focussed on testing new protocols, devices, security measures, etc. in smart city contexts. Basically, these initiatives can be divided into three types: simulations (1), urban testbeds (2) and laboratory testbeds (3). Each environment has some advantages if used at the right time of project maturity [35; 36]. Initiatives based on simulations (1) usually have the main advantage of flexibility. Teams in charge of simulations can easily change test settings and also change the configuration of the protocols or the devices included in the test scenario, add or remove nodes, change their geographic location, etc. There are powerful simulators for the IoT, such as NS-2 [37], Cooja [38], OMNET++ [39] or Castalia [40]. However, the lack of realism is the main drawback of this type of initiatives. This type of simulation is appropriate to test algorithm and protocols; nonetheless, they are not enough to fully validate proposed solutions in complex environments. On the other hand, urban testbeds (2) aim at deploying a large amount of IoT devices in one or several cities. For instance, City of Things [41] in Antwerpen, Belgium, and SmartSantander [20] in Santander, Spain, are two projects where a broad range of technologies have been deployed to create large smart cities for research purposes. Contrary to what happened with (1), this type of testbed has the advantage of realism. Nonetheless, these initiatives generally lack flexibility to adapt the tests to other configurations or equipment beyond those deployed since the beginning. Moreover, setting up and running the experiments tend to be more expensive and requires more time in real environments than in simulators. Halfway between (1) and (2) lay the initiatives deploying testbeds in laboratories (3). Although this type of testbed is limited in terms of scalability and interaction with the citizens, it has the advantage of using real hardware in a controlled environment. In addition, laboratory testbedsare more flexible than urban testbed, since it is much easier to add/remove devices, deploy new protocols, etc. In addition, testing in a laboratory allows experimenting with extreme conditions that may affect other services beyond those directly involved in the testbed. For example, jamming attacks not only interfere with the communication of testbed nodes, but also with other smart city devices communicating in the same frequency band, and it could even disrupt the neighbors' Wi-Fi connections. Regarding laboratory testbeds, it can be seen that in many cases, the test scenarios are designed to test a single smart city use case, such as the experiment in power systems in the SmartGridLab [42] project. Testbeds of this type provide valuable data for improving a particular service, device or protocol, but they take a silo approach. On the other hand, other laboratory testbeds take a more holistic approach. These initiatives consider the heterogeneity of the smart city and seek to test technological elements taking into account their interaction and integration with other devices. An outstanding initiative of this type is I3 ASensorBed [43]. This is a testbed focused on testing body area networks (BANs) and WSN communications. Another initiative with a stronger focus in cybersecurity is the Smart City Testbed Laboratory [44] deployed by the Center for Cyber Security at New York University Abu Dhabi (CCS-AD). With the aim of improving the safety of cyber-physical systems from a multidisciplinary approach, this laboratory incorporates typical hardware from different domains of a smart city, interconnecting the devices and exchanging data. In order to gather data and to monitor the devices, the laboratory includes an IoT element control platform.