## 3 ENN
### Architectures of Models
#### 3.3.2 Autoencoder (AE)  
The Autoencoder model is designed to handle both binary and multi-class classification tasks, starting with an input layer that accepts feature vectors. The architecture includes an encoder composed of multiple dense layers that progressively reduce the dimensionality of the input while utilizing the ReLU activation function to extract significant features. Following the encoder, a classification layer is employed to predict class probabilities. For binary classification, the sigmoid activation function is applied, whereas for multi-class classification, the softmax activation function is used. This allows the model to generate the appropriate output probabilities for each classification type. The model is optimized using the Adam optimizer and applies binary cross-entropy loss for binary classification and categorical cross-entropy loss for multi-class classification, ensuring accurate loss computation for each classification task. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Block** & **Layers** & **Layer Size** & **Activation** \\ \hline \hline \multirow{2}{*}{CICIDS2017} & \multirow{2}{*}{Input block} & \multirow{2}{*}{Input layer} & 69 & - \\ \cline{1-1} \cline{5-5}  & & & Input layer & 28 & - \\ \hline \multirow{5}{*}{Shared structure} & \multirow{2}{*}{Hidden block 1} & \multicolumn{2}{c}{1D CNN layer} & 256 & ReLU \\ \cline{3-5}  & & 1D max pooling layer & 2 & - \\ \cline{3-5}  & & Dropout layer & 0.0000001 & - \\ \cline{3-5}  & & 1D CNN layer & 256 & ReLU \\ \cline{3-5}  & & 1D max pooling layer & 4 & - \\ \cline{3-5}  & & Dropout layer & 0.0000001 & - \\ \cline{3-5}  & & Dense layer & 1024 & ReLU \\ \cline{3-5}  & & Dropout layer & 0.0000001 & - \\ \cline{3-5}  & & Dropout layer & 768 & Softmax \\ \cline{3-5}  & & Dropout layer & 0.0000001 & - \\ \hline \multirow{2}{*}{CICIDS2017} & \multirow{2}{*}{Output block} & \multirow{2}{*}{Output layer} & 14 & Softmax \\ \cline{3-5}  & & & Output layer & 4 & Softmax \\ \hline \hline \end{tabular}
\end{table} 
Table 13: CNN model layers for multi-class classification. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Parameter** & **Binary Classifier** & **Multi-Class Classifier** \\ \hline \hline \multirow{2}{*}{Learning rate} & 128 & 128 \\ \cline{2-3}  & Scheduled: Initial = 0.001, & Scheduled: Initial = 0.001, \\ \cline{2-3}  & Factor = 0.5, Min = \(1\times 10^{-5}\) & Factor = 0.5, Min = \(1\times 10^{-5}\) \\ \cline{2-3}  & (ReduceLROnPlateau) & (ReduceLROnPlateau) \\ \hline \hline Optimizer & [PERSON] & Adam \\ \hline Loss function & Binary cross-entropy & Categorical cross-entropy \\ \hline Metric & Accuracy & Accuracy \\ \hline \hline \end{tabular}
\end{table} 
Table 14: CNN model hyperparameters. In the encoder layers, the dimensionality of the input data is progressively reduced, allowing the model to extract essential features through dense layers. This process of reducing dimensions and extracting features can be mathematically represented as shown in Equation (16) [50]. \[h^{(l)}=f\Big{(}W^{(l)}\,a^{(l-1)}+b^{(l)}\Big{)} \tag{16}\]  
where \(h^{(l)}\) represents the output of encoder layer 1, with \(a^{(l-1)}\) denoting the output of the preceding layer, which acts as the input to the first layer. The weight matrix for layer 1 is denoted by \(W^{(l)}\) and the bias vector for that layer is given by \(b^{(l)}\). In this context, the activation function represented by \(f\) is specifically the ReLU function. In a typical Autoencoder, the decoder layer reconstructs the original input based on the compressed representation produced by the encoder. This reconstruction process is mathematically expressed in Equation (17) [50]. \[\overset{\prime}{\alpha}=g\Big{(}W^{(d)}\,h^{(l)}+b^{(d)}\Big{)} \tag{17}\]  
where \(\hat{\alpha}\) represents the reconstructed output, with \(h^{(l)}\) being the output from the final encoder layer. The weight matrix for the decoder layer is represented by \(W^{(d)}\), and \(b^{(d)}\) denotes the bias vector for the decoder layer.