# Deep Learning for Intrusion Detection Systems (IDSs) in Time Series Data +
## 4 Methodology
### Preprocessing  
Before feeding the dataset to the model, we first convert it into the right format. Thus, we initially **clean the data,** discarding \(NaN\) and corrupted values. The latter could be unrealistic attribute values, such as the negative packet length or unrealistic values that are too far from the feature mean (outliers). Such errors could be due to feature extraction tool errors or transportation losses.  
The second pre-processing step is **data scaling**, using the min-max scaler. Scaling is the process of transforming the values of numeric variables in a dataset to a common scale. Since neural networks tend to calculate the distance between data points, we do not want features with larger scales to dominate the ones with lower scales.  
The third pre-processing step is **feature selection**, which is performed using the ANOVA (analysis of variance) \(F\)-value. This is a statistical measure that assesses whether the means of two or more groups are significantly different from each other. The mathematical formula for this is as follows:  
\[F=\frac{MS_{between}}{MS_{within}} \tag{1}\]  
where for \(MS_{between}\) (mean squares between):  
\[MS_{between}=\frac{\sum_{i=1}^{k}n_{i}(\bar{X}_{i}-X)^{2}}{k-1} \tag{2}\]  
and for \(MS_{within}\) (mean squares within):  
\[MS_{within}=\frac{\sum_{j=1}^{n}(X_{ij}-\bar{X}_{i})^{2}}{k_{n}-k} \tag{3}\]  
For these equations, \(n_{i}\) is the number of observations in the \(i\)-th group, \(\bar{X}_{i}\) is the mean of the \(i\)-th group, \(\bar{X}\) is the overall mean, \(k\) is the number of groups, and \(n\) is the number of observations within each group. This statistic is computed between all features and the label.  
The fourth pre-processing step is **one-hot encoding** since we have many categorical features that can take multiple values. This technique gives a column with \(N\) different values and creates \(N\) different binary columns, where 1 indicates the presence of this value in the current instance and 0 indicates the absence. An example is the categorical variable'service', which indicates the type of packet and can be 'dns', 'http','smtp', etc.  
The final stage of pre-processing involves **formatting** the data into a **time series** format. To accomplish this, we begin by sorting the dataset based on the time (starting time) feature. During this pre-processing phase, there is an option to either generate time series objects for the entire sorted dataset or initially group the dataset by (sender IP and receiver IP) pairs, subsequently creating time series objects using a specific pair of IPs. The former analyzes the overall network traffic, while the latter specifically monitors the communication flow between two designated entities. Next, using the sorted dataset, we generate windows  
Figure 1: Methodology for the proactive IDS.  
comprising \(W\) time points and a label, where \(W\) is the size of the input window. Here, the initial \(W\) points serve as the input to the model, and the label is the target that must be predicted. For each of the \(W\) input points, we keep all the features (not only the labels) to formulate the problem of multivariate time series forecasting. The label is calculated based on a value \(T\) and indicates the existence of an attack in the forthcoming \(T\) packets. For example, if \(W\) = 10 and \(T\) = 5, and the labels of the \(T\) packets are \([01001]\), the label of the time series instance would be 1 because there is a 1 in the \(T\) vector. Otherwise, it would be 0. Also, in such a case, we would use ten consecutive instances of the dataset as input for the model.  
We also incorporate overlapping between the different windows to utilize the data to their fullest potential.