## 3 ENN
### Architectures of Models
#### 3.3.2 Autoencoder (AE)
The activation function \(g\), generally selected to be linear, is applied to facilitate reconstruction. The classification layer applies a designated activation function to produce the binary classification output, as outlined in Equation (18) [50]. \[\overline{y}=\sigma\Big{(}W^{(out)}\,h^{(l)}+b^{(out)}\Big{)} \tag{18}\]  
where \(\overline{y}\) represents the predicted probability of the positive class and the output layer's weight matrix is indicated by \(W^{(out)}\), with \(b^{(out)}\) representing the bias term for this layer. The sigmoid function, \(\sigma\), is used to map the output to a probability score ranging from 0 to 1. For multi-class classification, the classification layer uses the softmax activation function, allowing the model to produce a probability distribution over multiple classes. This process is mathematically represented in Equation (19) [53]. \[\overline{y}=\textit{softmax}\Big{(}W^{(out)}\,h^{(l)}+b^{(out)}\Big{)} \tag{19}\]  
where \(\overline{y}\) denotes the vector of predicted probabilities for each class and the output layer is defined by the weight matrix \(W^{(out)}\) and bias \(b^{(out)}\). The softmax function is applied to transform the logits into probabilities, ensuring that the predicted values sum to one across all classes. (i) Binary Classification  
The design of the Autoencoder model for binary classification, as detailed in Table 15, begins with an input layer of 69 neurons for CICIDS2017 and 18 neurons for NF-BoT-IoT-v2. The encoder block consists of three dense layers, the first with 128 neurons, the second with 64 neurons, and the third with 32 neurons, all using the ReLU activation function. The model concludes with an output block containing a single neuron activated by the sigmoid function, providing the binary classification output. (ii) Multi-Class Classification  
The architecture of the Autoencoder model for multi-class classification is in Table 16, starting with an input layer consisting of 69 neurons for the CICIDS2017 dataset and 28 neurons for the NF-BoT-IoT-v2 dataset. In the encoder block, the first dense layer has 128 neurons, followed by two additional dense layers with 64 and 32 neurons, all activatedby the ReLU function, to progressively compress the input data. The model concludes with an output layer consisting of 14 neurons for CICIDS2017 and 4 neurons for NF-BoT-IoT-v2, both activated by the softmax function, to handle the multi-class classification task. (iii) Hyperparameter Configuration for the Autoencoder Model  
The hyperparameters for the Autoencoder model detailed in Table 17 are specified for both the binary and multi-class classification tasks. For both types of classifiers, a batch size of 128 was used, and the Adam optimizer was employed. The learning rate for both the binary and multi-class classification tasks was controlled using the ReduceLROnPlateau scheduling technique. The learning rate started with an initial value of 0.001, reduced by a factor of 0.5 whenever the validation loss did not improve for two consecutive epochs, as specified by the patience parameter. The learning rate does not decrease below a minimum value of \(1\times 10^{-5}\), ensuring that it remains within a reasonable range for effective training. This dynamic adjustment helps optimize model training by preventing the learning rate from becoming too small too quickly, which could hinder convergence. The loss function differed between the two tasks, with binary cross-entropy used for binary classification and categorical cross-entropy applied for multi-class classification. Accuracy was the metric utilized to assess model performance in both scenarios.