## 3 ENN
### Architectures of Models
#### 3.3.4 Autoencoder-Convolutional Neural Network (Autoencoder-CNN)
This block comprises a dense layer with an activation function, followed by batch normalization and dropout, to enhance the model's capability to learn intricate patterns for binary classification while minimizing overfitting. Similarly, for multi-classification tasks, the output is flattened and passed through an MLP block that includes a dense layer, batch normalization, activation function, and dropout, effectively strengthening the model's ability to represent complex features and reducing the risk of overfitting. Features from both the CNN and MLP components are then combined into a comprehensive unified feature set. Finally, an output layer employs a sigmoid activation function for binary classification or a softmax activation function for multi-class classification, with binary cross-entropy or categorical cross-entropy as the respective loss functions. The model is optimized with the Adam optimizer to ensure efficient learning and robust convergence across both classification objectives. The encoder layers progressively reduce the dimensionality of the input data, enabling the model to extract key features through dense layers. This process of dimensionality reduction and feature extraction can be mathematically expressed as shown in Equation (24) [50]. \[h^{(l)}=f\left(W^{(l)}\,a^{(l-1)}+b^{(l)}\right) \tag{24}\]  
where \(h^{(l)}\) represents the output of encoder layer l and \(a^{(l-1)}\) denotes the output of the preceding layer, which acts as the input to the first layer. The weight matrix for layer l is denoted by \(W^{(l)}\), and the bias vector for that layer is given by \(b^{(l)}\). In this context, the activation function represented by f is specifically the ReLU function. In a standard Autoencoder, the decoder layer is responsible for reconstructing the original input from the compressed representation generated by the encoder. This process of reconstructing the input from the encoded features can be mathematically described by Equation (25) [50]. \[\overset{{}^{\prime}}{\alpha}=g\left(W^{(d)}\,h^{(l)}+b^{(d)}\right) \tag{25}\]  
where \(\hat{\alpha}\) represents the reconstructed output, \(h^{(l)}\) is the output from the final encoder layer, the weight matrix for the decoder layer is represented by \(W^{(d)}\), and \(b^{(d)}\) denotes the bias vector for the decoder layer. The activation function \(g\), generally selected to be linear, is applied to facilitate reconstruction. The convolution operation in the CNN layers plays a critical role in extracting features by applying filters to the input data. It works by sliding the convolution kernel across the input feature map, computing the dot product at each position and producing a transformed output known as the feature map. The mathematical representation of this convolution process is given in Equation (26) [50]. \[Z_{i,j}=\left(X*K\right)_{i,j}=\sum_{m}\sum_{n}Z_{i+m,j+n}k_{m,n} \tag{26}\]  
where \(Z\) denotes the output feature map and \(X\) represents the input feature map. The convolution kernel denoted by \(K\) is applied during the convolution operation to convert the input features into the output features. The ReLU activation function illustrated in Equation (27) [51] is a simple but powerful non-linear transformation. It sets all negative inputs to zero while maintaining positive values. This mechanism not only mitigates the problem of vanishing gradient but also promotes sparse activations, both of which contribute to faster training and enhanced model performance. \[ReLU(x)=max(0,x) \tag{27}\]  
The max pooling operation reduces the spatial dimensions of the input feature map, effectively downsampling the data while retaining the most important features. This is accomplished by selecting the maximum value from within a specified pooling window, ensuring that the most significant information is preserved while less relevant details are discarded. The mathematical representation of this operation is provided in Equation (28) [50].