## 3 Datasets and Methods
### ML Models and Configuration Parameters  
In total, seven ML models were used in the intrusion detection experiments. State-of-the-art dual-flow LSTM-FCN (Figure 1) and GRU-FCN (Figure 2) models were selected since typically more complex DL configurations tend to generate better results [81]. Those models consist of two flows that are connected to the last decision-making layer: the CNN and RNN flow (LSTM or GRU, respectively). They have been designed for use in time-series datasets, which often are univariate (contains only one observed parameter). System calls from AWSCTD fit perfectly for such methods. Apart from that, several additional experimental models, i.e., not described elsewhere previously, were tested. One of the main criteria influencing the model configuration selection was the least possible number of internal nodes in order to ensure reasonable training/testing times while preserving high accuracy and other metrics values. Selection of models was also based on results achieved by Reference [82; 83; 84], in which DL models (CNN and RNN) were combined in a linear ensemble manner. Combined models returned better accuracy results than separate usage of CNN and RNN models on speech, image, and infrared spectroscopy analysis. The aforementioned research encouraged us to use the linear ensemble with our previously tested [14] models and already recognized as state-of-the-art [85] models:  
* Single-flow model (Figure 3), which contains only CNN flow extracted from LSTM-FCN and was originally named FCN in Reference [85]. * Single-flow CNN model (Figure 4), which demonstrated the best results in our previous experiments [14] and is named AWSCTD-CNN- dynamic (D) (dynamic value of kernels parameter). * New single-flow CNN model (Figure 4), named AWSCTD-CNN-S (static value of kernels parameter). * New dual-flow model (Figure 5), named AWSCTD-CNN-LSTM, combined of AWSCTD-CNN-D and LSTM. * New dual-flow model (Figure 6), named AWSCTD-CNN-GRU, combined of AWSCTD-CNN-D and GRU. Figure 1: Diagram of long short-term memory fully convolutional network (LSTM-FCN) model. CuDNN = CUDA® (Compute Unified Device Architecture) Deep Neural Network library (cuDNN); ReLU = rectified linear unit. Figure 2: Diagram of gated recurrent unit (GRU)-FCN model. Figure 4: Diagram of AWSCTD-CNN-dynamic (D) and AWSCTD-CNN-static (S) models. Figure 3: Diagram of FCN model. It can be seen that the LSTM-FCN, GRU-FCN, and FCN consist of a more complex structure in comparison with the AWSTD-CNN, AWSTCD-CNN-LSTM, and AWSTD-CNN-GRU models. This feature implies that those models can provide better results in intrusion detection, as suggested in Reference [16; 17]. The models were grouped into two families by origin:  
1. FCN family: * LSTM-FCN * GRU-FCN * FCN
2. AWSTD family: * AWSTCD-CNN-D * AWSTCD-CNN-LSTM * AWSTCD-CNN-GRU * AWSTCD-CNN-S  
The following specific parameters were used for FCN family models:  
3. The models from FCN family contain original configuration as described in original papers [16; 17]. * The FCN-GRU part has 8 unfolds and uses aggressive dropout coefficient of 0.8. Figure 5: Diagram of attack-caused Windows OS system calls traces (AWSTD)- convolutional neural network (CNN)-LSTM model. Figure 6: Diagram of AWSTD-CNN-GRU model. The precision (also called detection rate) is the fraction of relevant instances among the retrieved instances [90]. It means that, when the model predicts a positive value, what are the odds that the model has made a correct prediction? The precision is calculated as follows:  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{1}\]  
The recall, also known as sensitivity, is used to measure the fraction of positive patterns that are correctly classified [90]. In other words, recall is a measure that tells us how great our model is when all the actual values are positive. The recall is calculated as follows:  
\[\text{Recall}=\frac{\text{TP}}{\text{FN}+\text{TP}} \tag{2}\]  
The F-score is a metric that combines recall and precision by taking their harmonic mean [90].