## 3 ENN
### Architectures of Models
#### 3.3.3 Deep Neural Network (DNN)
To combat overfitting, the first hidden block includes a dropout layer with a minimal dropout rate of 0.0000001, followed by another dense layer with 768 neurons, activated by ReLU. This layer is enhanced through batch normalization to stabilize training via normalizing activations. The second hidden block mirrors the first, featuring an additional dropout layer with the same minimal rate and another batch normalization layer. The architecture concludes with a single-neuron output layer activated by the sigmoid function, producing a probability score representing the likelihood of the input belonging to the positive class. This design was optimized for the binary classification tasks, with separate input layers tailored to the CICIDS2017 and NF-BoT-IoT-v2 datasets. (ii) Multi-Class Classification  
The architecture outlined in Table 19 describes the structure of a DNN model tailored for multi-class classification on both the CICIDS2017 and NF-BoT-IoT-v2 datasets. The input layer processes the features of the datasets, with the CICIDS2017 dataset having 69 features and the NF-BoT-IoT-v2 dataset having 28 features. The input layer includes a dense layer with 1024 neurons, activated by ReLU, introducing non-linearity and enabling the model to capture more complex patterns. To combat overfitting, the first hidden block includes a dropout layer with a minimal dropout rate of 0.0000001, followed by another dense layer with 768 neurons, activated by ReLU. This layer is enhanced with batch normalization to stabilize training by normalizing activations. The second hidden block mirrors the first, featuring an additional dropout layer with the same minimal rate and another batch normalization layer. The architecture concludes with an output layer consisting of 14 neurons for the CICIDS2017 dataset and 4 neurons for the NF-BoT-IoT-v2 dataset, each activated by the softmax function. This configuration produces class probabilities, empowering the model to accurately handle multi-class classification tasks with high precision and reliability. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Block** & **Layers** & **Layer Size** & **Activation** \\ \hline CICIDS2017 & Input block & Input layer & 69 & ReLU \\ \hline NF-BoT-IoT-v2 & Input block & Input layer & 18 & ReLU \\ \hline \multirow{6}{*}{Shared structure} & & Dense layer & 1024 & - \\ \cline{2-5}  & & Dropout layer & 0.000001 & - \\ \cline{2-5}  & & Dense layer & 768 & ReLU \\ \cline{2-5}  & & Batch normalization & - & - \\ \cline{2-5}  & & Dropout layer & 0.0000001 & - \\ \cline{2-5}  & & Batch normalization & - & - \\ \cline{2-5}  & & Output block & Output layer & 1 & Sigmoid \\ \hline \hline \end{tabular}
\end{table} 
Table 18: DNN model layers for binary classification. 3. Hyperparameter Configuration for the DNN Model  
The hyperparameters for the DNN models, as shown in Table 20, are configured for both binary and multi-class classification tasks. Both classifiers use a batch size of 128 and the Adam optimizer. The learning rate for both the binary and multi-class classifiers is scheduled using Exponential Decay. The initial learning rate is set to 0.0003, with a decay factor of 0.9 and decay steps of 10,000. This exponential decay schedule reduces the learning rate progressively during training, helping to stabilize the optimization process and ensure more effective convergence over time. The loss function applied is binary cross-entropy for the binary classifier and categorical cross-entropy for the multi-class classifier. Accuracy is used as the evaluation metric for both classifiers.