# Difficulties and Challenges of Anomaly Detection in Smart Cities: A Laboratory Analysis
## 1 Introduction  
In the last decade, major cities around the world have been installing technological elements in the streets in order to capture a plethora of urban information to populate smart city information systems that allow authorities to offer new and better services to citizens and to take a data driven city management approach. The installed elements belong to what is called the Internet of Things (IoT), wireless sensor networks (WSNs) being one of the most extended technologies in this area. In smart cities, popular proposals use WSNs to improve parking systems [1] or public lighting [2]. In order to install these networks in the streets, many public administrations have outsourced the installation and operation of these services to external providers. This has raised, however, concerns related to the quality of data [3]. Outsourcing WSN operation usually results in the loss of control over network devices and this, in turn, causes a lack of visibility over the potential data-quality problems affecting these devices. Even when external providers implement proper security mechanisms to ensure data integrity, in practice, smart city administrators cannot determine the extent to which received data are precise and accurate. In fact, the Royal Academy of Engineering has identified data quality as one of the six major barriers to effectively optimize smart infrastructures [4]. To counter this problem, smart city IT administrators need to apply anomaly detection measures, so that they can detect when data stops following the expected patterns. The role of the data analyst, or data scientist, is here of utmost importance. The huge amount of data generated by the smart city WSNs have to be examined to determine the extent to which they are reliable and can be trusted. An overly simplistic approach to this problem could lead to the definition of thresholds on key system variables to detect abnormal behaviours. However, Ref. [5] has shown that complex environments with large volumes of heterogeneous data require more powerful algorithms. The anomaly detection mechanisms must support a multivariate analysis where data patterns are dynamic and where no simple rules can be used to determine whether the behaviour is normal or when it is deviating from the standard. By using these algorithms, the normal state of the system can be learned periodically. This is performed through training mechanisms, which are used to create mathematical models that represent the normal state. Then, these models can be used to determine when a new value must be considered an outlier. In these scenarios, machine learning-based techniques can undoubtedly give very good results, considering the success that machine learning has seen in many other areas of computer science, such as product recommendations systems [6; 7], optical character recognition systems (e.g., [8; 9]), natural language translation [10], and also spam detection [11]. Certainly, a smart city is a complex environment where we can find a huge volume of heterogeneous data. Additionally, most equipment belonging to the smart city's WSNs is especially vulnerable to attacks due to its easy access in public spaces. Thus, the use of advanced anomaly detection techniques in smart cities is critical to contributing to their data reliability and trustworthiness. The main purpose of this article is to evaluate the use of machine learning techniques for anomaly detection in this context. In the WSN research field, machine learning has already been used successfully in the past to detect attacks against the networks and it is very often used to implement intrusion detection systems (IDS) [12; 13; 14]. However, these IDS are designed for controlled environments, where WSN administrators have full access to their nodes. In this article, we want to analyze the use of machine learning in smart city environments, which are much more complex in terms of heterogeneity, number of nodes, accessibility to providers' infrastructure, etc.