# Combining Unsupervised Approaches for Near Real-Time Network Traffic Anomaly Detection
## 4 Experiment
### Experimental Setup  
The unsupervised algorithms examined in this work are as follows:  
* Extended Isolation Forest (EIF)
* Ensemble Consensus (EC)
* Deep Autoencoding Gaussian Mixture Model (DAGMM)
* Extended Isolation Forest (DAGMM-EIF)
* Deep Autoencoder (DA)
* Extended Isolation Forest (DA-EIF)
* Deep Structured Energy Based Models (DSEBM)
* Memory-Augmented Deep Autoencoder (MemAE)* Memory-Augmented Deep Autoencoder--Extended Isolation Forest (MemAE-EIF) The metrics examined to assess them are as follows:
* Precision
* Recall
* F1-Score
* Area Under the Receiver Operator Characteristic Curve (AUROC)
* Accuracy  
The Recall, Precision, F1-Score, and Accuracy indices reported in the following tables refer to macro measures; i.e., the weighted average over the number of the instances for each class. In the present experiment, the test dataset samples can be labeled with anomalous and normal classes. In unsupervised algorithm testing, the Anomaly Detection system can distinguish the analyzed traffic into two classes, namely the class related to normal traffic, represented by the label \"0\", and the class indicating any attack, represented by the label \"1\". The reported results were computed through the K-Fold cross-validation [32] technique, in which the value of K was set equal to 10. For the algorithms EIF, DAGMM, DA, DSEBM, MemAE, DAGMM-EIF, MemAE-EIF, and DA-EIF, since they return an anomaly score for each sample analyzed, it is necessary to define a threshold that allows the discrimination of samples considered normal from samples considered anomalous. For the experiments reported in this section, since the percentage of anomalies in the test dataset is known, the anomaly threshold is configured with the value that allows isolating the percentage of higher scores. Only the numerical features were used to analyze the datasets. The following are the configurations used to use each algorithm as an anomaly detector:  
* **Ensemble Consensus**: The configuration reported in [7] is used, with the _\"soft consensus\"_ mode and a threshold of \(-0.2\). * **Deep Autoencoding Gaussian Mixture Model**: The neural network of the DAGMM algorithm used the hyperbolic tangent activation function and was trained through the Adam optimization algorithm [33] with a learning rate of 0.0001. The number of epochs used for the training process is 180. The neural network used by the DAGMM algorithm is described below:  
* Compression network: The compression network has an input layer that contains a dataset record's features. The input layer thus consists of 38 nodes for the KDDCUP99 and NSL-KDD datasets and 78 nodes for the CIC-IDS2017 dataset. There are then 7 hidden layers, consisting of 60, 30, 10, 1 (which represents the compression carried out on the record given as input), 10, 30, and 60 nodes. The output layer contains the reconstructed features of the input record, so it is composed of 38 nodes for the KDDCUP99 and NSL-KDD datasets; it is instead composed of 78 nodes for the CIC-IDS2017 dataset. * Estimation network: The output layer consists of a number of nodes equal to the number of Gaussians used by the GMM algorithm. For the NSL-KDD dataset, the output layer consists of five nodes; instead, for the KDDCUP99 and CIC-IDS2017 datasets, the number of nodes is four. * **Extended Isolation Forest**: For the KDDCUP99 and NSL-KDD datasets, a forest of 200 trees with parameter ndim equal to 38 is used. Parameter ndim represents the number of columns to be combined to produce a subdivision within the trees. In the case of the CIC-IDS2017 dataset, a forest of 200 trees with parameter ndim equal to 10 is used. In addition, for the CIC-IDS2017 dataset, parameters prob_pick_pooled_gain and ntry were configured, respectively, equal to 1 and 10. Parameter prob_pick_pooled_gain denotes the probability of choosing the threshold on which to split a linear combination of variables as the threshold that maximizes an aggregate standard deviation gain criterion on the linear combination [34].