## 3 ENN
### Architectures of Models
#### 3.3.1 Convolutional Neural Networks (CNNs)
\[ReLU(x)=max(0,x) \tag{11}\]  
The max pooling operation simplifies the input feature map by reducing its spatial dimensions while preserving the most relevant features. This is achieved by selecting the maximum value within a defined pooling window, thus downsampling the data. The process is mathematically represented in Equation (12) [50]. \[P_{i,j}=max\big{(}X_{i;i+p,\ j;j+q}\big{)} \tag{12}\]  
where \(P\) represents the output resulting from the pooling process, while \(p\) and \(q\) denote the dimensions of the pooling window applied to aggregate the input features into the pooled output. The dropout layer introduces regularization by randomly disabling a proportion \(p\) of input units during training. This process reduces overfitting by preventing the model from becoming overly dependent on specific input features, thereby promoting better generalization. The effect of this regularization technique is mathematically represented in Equation (13) [52]. \[Dropout(x)=\left\{ \begin{array}{l}x\ with\ probability\ 1-p\\ 0\ with\ probability\ p\end{array} \right. \tag{13}\]  
For binary classification tasks, the output layer applies the sigmoid function to generate a probability score reflecting the likelihood that a given instance belongs to the positive class. This function outputs a value between 0 and 1, with values closer to 1 suggesting a higher probability of the instance being classified as positive. The mathematical expression for this operation is provided in Equation (14) [53]. \[\sigma(Z)=\frac{1}{1+e^{-z}} \tag{14}\]  
where \(Z\) represents the output generated by the final dense layer in the network. The output layer utilizes the softmax function for multi-class classification, transforming the raw model outputs into a probability distribution over all possible classes. This function ensures that the sum of the probabilities across all classes equals one, providing a clear indication of the model's confidence in each class. The mathematical formulation of this process is presented in Equation (15) [53]. \[Softmax(Z_{i})=\frac{e^{z_{i}}}{\sum\limits_{j}e^{z_{i}}} \tag{15}\]  
where \(Z_{i}\) represents the output generated by the final dense layer in the network for class \(i\). 1. Binary Classification  
The CNN model architecture for binary classification applied to the CICIDS2017 and NF-BoT-IoT-v2 datasets, as outlined in Table 12, began with an input layer tailored to each dataset. For the CICIDS2017 dataset, the input layer consisted of 69 neurons, while the NF-BoT-IoT-v2 dataset used an input layer with 18 neurons. The model then progressed through the three hidden blocks. The first hidden block featured a 1D CNN layer with 256 filters and a ReLU activation function, followed by a 1D max pooling layer with a pool size of 2 and a dropout layer with a minimal rate of 0.00000001 to mitigate overfitting. The second hidden block mirrored the first, incorporating a 1D CNN layer with 256 filters and ReLU activation, followed by a 1D max pooling layer with a pool size of 4, and another dropout layer with the same rate. The third hidden block included a dense layer with 256 neurons and ReLU activation, followed by a dropout layer with a rate of 0.00000001. Finally, the output block consisted of a single neuron in the output layer, activated by the sigmoid function, to generate the binary classification result. This architecture was optimized to process the distinct feature sets of both datasets effectively. (ii) Multi-Class Classification  
The CNN model architecture for multi-class classification applied to the CICIDS2017 and NF-BoT-IoT-v2 datasets, as detailed in Table 13, began with dataset-specific input layers, using 69 neurons for CICIDS2017 and 28 neurons for NF-BoT-IoT-v2. The model progressed through five shared hidden blocks. The first hidden block included a 1D CNN layer with 256 filters, followed by a ReLU activation, a 1D max pooling layer with a pool size of 2 and a dropout layer with a minimal rate of 0.0000001.