# Integration of Machine Learning-Based Attack Detectors into Defensive Exercises of a 5G Cyber Range
## 9 Exercises
### DoH Flooding Attack Detection Scenario
#### 9.2.3 Application and Integration of Machine Learning for DoH Flooding Attack Detection
In that way, the users will be provided with a model that is useless for identifying DoH traffic and which they can use in one of the sections to poison a correct model. To all the models, a random selection of values for the aforementioned hyperparameters was performed to obtain a variety of models with different performance levels. We list these values in Table 5.  
An additional challenge for the blue team in this exercise is to test models created with a different ML algorithm. For this purpose, we created two models using Fully Connected Neural Networks (FCNN). Similarly to the other models, we trained them with a different number of layers to yield varying degrees of performance. The structure of each of the FCNN models is specified in Table 6. The model structures are described using a compact notation that is common in the ML literature [48]. This notation is described below. In the first line, a list of values describes the FCNN architecture: \([X_{0},X_{1},\dots,X_{i},\dots,X_{n}]\) where \(n\) is the number of fully connected layers and \(X_{i}\) is the number of units in the fully connected layer \(i\). In addition. The activation function of each fully connected layer is listed below. Finally, the final activation function that serves as the output of the model is listed at the bottom. The training hyperparameters for the two models used in the DoH exercise were manually tuned to achieve the intended performance. The training hyperparameters of both models are the same, except for the maximum number of epochs, the value of which is given in Table 6 for each model. In particular, we use a batch size of 2048. We also use the Adam optimizer with a learning rate of 0.001. In addition, we use the early stopping technique to automatically terminate the training process if the validation loss does not improve for two epochs. For the validation procedure, we reserve 10% of the training data for the validation split. Finally, as a loss function, we use the categorical cross-entropy function. The quality measures of Table 7 display how the variation in the hyperparameters and structure affects the model's precision and the generation of false positives and false negatives. The quality measures that we have taken into account are the same as the ones we used to evaluate the cryptomining attack detection models: F1-Score, balanced accuracy, and confusion matrix. We also used balanced accuracy in this case, to evaluate the models, as the DoH dataset is not perfectly balanced. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model Name** & **Max. Depth** & **Num. Estimators** \\ \hline \hline \begin{tabular}{c} doh\_rf\_31\_a.pkl \\ (Good performance) \\ \end{tabular} & Unlimited & 100 \\ \hline \hline \
\begin{tabular}{c} doh\_rf\_31\_b.pkl \\ (Medium performance) \\ \end{tabular} & 8 & 3 \\ \hline
\begin{tabular}{c} doh\_rf\_31\_c.pkl \\ (Bad performance) \\ \end{tabular} & 2 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Hyperparameters for each DoH Random Forest Classifier model generated. In the same way as in the cryptomining exercise, by observing Table 7, we can clearly appreciate the didactic value of this exercise. By applying ML models to detect DoH traffic, the blue team can test real traffic against different models observing the contrasting results, varying in confidence and consistency of detecting attacks. This inconsistency in detecting attacks will be clearly observed in the third model, 'doh_rf_31_c.pkl', which will show the blue team that a model can also be modified to always predict the same result, something that they will be able to apply when taking the role of the attacker to poison the model. In the case of our exercise, the model poisoning method consists of obtaining root access to the machine and overwriting the best model with the bad model mentioned above. Although there are other ways of poisoning an ML model, we decided to use this method because it is considered one of the most straightforward methods to accomplish this task, while also being highly effective.