# Combining Unsupervised Approaches for Near Real-Time Network Traffic Anomaly Detection
## 1 Introduction  
Cyber attacks can impact the performance of networks (corporate or otherwise), allow access to and the modification of confidential data, and compromise the security of virtually any infrastructure belonging to the network itself. This issue can have a significant economic impact on a smart enterprise. Zero-day attacks are unknown, never-before-seen attacks based on undisclosed and unpublished flaws and the widespread knowledge of \"you cannot protect against what is unknown\" [1]. Therefore, it is of fundamental importance to create algorithms that are able to predict 0-day attacks in an automatic and fast way (near real-time). Time is a crucial parameter concerning such attacks: in a system with a large volume of traffic, if it cannot be analyzed in a reasonable time to avoid traffic accumulation, and thus a bottleneck is generated. In a near real-time anomaly detection system, it is necessary to consider the processing speed of the algorithm and the pattern variations typical of attacks caused by the different network traffic that can be monitored in a real context. In particular, network traffic is characterized by behaviors that vary based on holidays, business hours, or times of the year characterized by higher or lower network activity [2]. Therefore, the system must be able to respond to changes in the monitored network traffic dynamically. This work aims to study and design an anomaly detection system through the combination of machine learning algorithms capable of analyzing near real-time network traffic. The research focuses on classical Machine Learning algorithms and an innovative combination of them and how they can be used in a near real-time context, where it is necessary to identify anomalous network traffic promptly. The construction of a complete network traffic dataset is challenging as it is difficult to enumerate all possible cases of \"normality\", and the boundary between normality and anomaly is often not well defined. This happens because creating a labeled dataset that best reflects a real application context is very expensive. Moreover, in particular situations, due to the frequent change of the characteristics of a process or the continuous emergence of different behaviors, it is impossible to determine if the information is still valid. Focusing also on unknown attacks, unsupervised approaches must be considered--anomaly detection represents the primary strategy available. Applying a supervised algorithm can limit the system's effectiveness as it must consider the variability of the networks, their different types, and the number of hosts that assume unknown behavior. For this reason, in this work, it was chosen to use an unsupervised algorithm, since this allows us to isolate anomalous behavior and also identify unknown attacks [3]. These models implicitly assume that normal data are present in a greater quantity than abnormal samples, so the algorithm can classify normal behavior by clustering similar instances together and highlighting as abnormal a sample that deviates from the cluster representing normality. The state of the art of unsupervised Anomaly Detection algorithms shows that they achieve lower performance than algorithms that use a supervised approach. Some algorithms, such as the Extended Isolation Forest, reach performances comparable with supervised algorithms, but the time spent in the prediction phase is not suitable for near real-time applications. In order to improve the performance of unsupervised Anomaly Detection algorithms, an approach combining Deep Learning and Shallow Learning algorithms is proposed in this paper. The idea behind the proposed work is to maximize the prediction speed using Deep Learning techniques and at the same time maximize the accuracy index using robust Shallow Learning algorithms. In a near real-time Anomaly Detection system, it is necessary to minimize the prediction time of the anomaly score in order to allow the experts in the field to detect and respond to cyberattacks promptly.