# Anomaly Detection for IOT Systems Using Active Learning
## 3 Data Collection
### Exploratory Data Analysis (EDA)  
Before undertaking any data science research, it is imperative to conduct an Exploratory Data Analysis (EDA) [37]. It involves understanding the data and discerning potential patterns, trends, or anomalies within it. The fundamental purpose of an EDA is to ascertain if the data may be effectively employed to inform and guide further modelling and data analysis methodologies. The EDA for this work encompasses the following stages:  
* Data Pre-processing: data preparation encompasses several tasks, such as addressing outliers, removing missing values, and transforming variables [38]. * Descriptive Statistics: measures of central tendency, such as the mean, median, and mode, as well as measures of variability, such as the standard deviation, and the examination of the relationship between variables, such as correlation, are illustrative instances of descriptive statistics that can provide significant insights into the characteristics of the data. * Data Visualization: The utilization of various data visualization techniques, including histograms, scatter plots, box plots, and heat maps, can facilitate the identification of patterns, trends, and outliers within the data. Furthermore, these representations can unveil the associations among the variables. * Dimensionality Reduction: Visualizing and evaluating data with a high number of dimensions poses significant challenges. Principal component analysis (PCA) is one technique that works well for reducing the dimensionality of data while keeping key features. * Feature Selection: Feature selection involves identifying and selecting the most relevant features within a dataset that are essential for anomaly detection. Feature selection might enhance models' accuracy and reduce the data's complexity. Since EDA is a cyclical process, its insights can inform subsequent phases of data analysis and modeling. * Attack Distribution by Category: The analysis of attack distribution by category involves utilizing a bar plot to visually represent the prevalence of different attack categories within the dataset, as shown in Figure 2. This approach allows for a better understanding of the relative frequency of attacks across various categories. It may provide valuable understanding regarding the specific attacks that are more prone to targeting IoT equipment. In the above figure, the distribution of each attack type within the entire dataset is presented. The distribution of attack types in IOT systems follows a normal distribution, with most traffic falling within the normal class. Conversely, the combined occurrence of other attack types is lower than that of the normal class. This observation suggests that the predominant traffic in IOT systems is of the normal type, and the highest level of assault encountered by IOT devices is typically of a generic nature. * Protocol Distribution: Determining the most often utilized protocols in IoT systems can be achieved by employing a visual representation such as a pie chart or bar plot, as shown in Figure 3 to showcase the distribution of these protocols throughout the dataset. It can facilitate the identification of protocols that are more susceptible to attacks and that require implementing supplementary security measures. The term \"protocol\" pertains to a collection of regulations or criteria that dictate how data is conveyed and received across a network. In datasets related to network security, the column labeled \"proto\" commonly denotes the protocol employed for each network connection or packet, encompassing protocols like TCP, UDP, ICMP, and others. A comprehensive knowledge of the protocol used in network communication holds significant importance in network security applications, as distinct protocols may be susceptible to certain forms of attacks. There were many protocol types involved in this dataset, around 131, but we only visualized those with the maximum distribution in the dataset. TCP and UDP are the two most prevalent protocol types.