## 3 Proposed Approach
### Class Weights
#### 3.3.3 Deep Neural Network (DNN)  
The DNN model for binary and multi-class classification consists of several blocks, including the input block, two hidden blocks, and the output block. The model begins with an input layer, where the number of features varies depending on the dataset, and a dense layer utilizing ReLU activation to learn complex patterns. This is followed by the first hidden block, which includes a dropout layer with a very small rate to prevent overfitting, followed by a dense layer with ReLU activation, then batch normalization. The second hidden block includes another dropout layer with the same small rate and batch normalization to improve training stability. The output layer differs based on the classification type. For binary classification, it features a single neuron with a sigmoid activation function to produce a probability score, while for multi-class classification, it contains multiple neurons with a softmax activation function to provide class probabilities. The model is compiled using the Adam optimizer with a learning rate defined by an exponential decay schedule. It uses binary cross-entropy for binary tasks or categorical cross-entropy for multi-class tasks, enabling effective training. Additionally, a custom callback is implemented to visualize the confusion matrix at the end of each epoch, offering valuable insights into the model's classification performance by comparing predicted and actual labels. The feed-forward operation in a DNN involves passing the input through multiple layers to produce the output. This can be expressed mathematically for a layer l as presented in Equation (18) [93]. \[a^{(l)}=f\left(W^{(l)}\,a^{(l-1)}{+}b^{(l)}\right) \tag{18}\]  
In this context, \(a^{(l)}\) denotes the activation of the current layer l. The weight matrix for this layer is represented by \(W^{(l)}\), while \(b^{(l)}\) signifies the bias vector for layer l. The activation function f is applied element-wise, which may include functions such as ReLU or sigmoid, to introduce non-linearity into the model. The ReLU activation function, shown in Equation (19) [94], is a simple and efficient non-linear function that outputs zero for negative values, promotes sparse activations, and supports effective gradient flow, making it ideal for deep learning. \[ReLU(x)=max(0,x) \tag{19}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Binary Classifier** & **Multi-Class Classifier** \\ \hline Batch size & 128 & 128 \\ \hline \multirow{4}{*}{Learning rate} & Scheduled: Initial = 0.001, & Scheduled: Initial = 0.001, \\  & Factor = 0.5, Min = \(1\times 10^{-5}\) & Factor = 0.5, Min = \(1\times 10^{-5}\) \\  & (ReduceLROnPlateau) & (ReduceLROnPlateau) \\ \hline Optimizer & Adam & Adam \\ \hline Loss function & Binary cross-entropy & Categorical\_crossentropy \\ \hline Metric & Accuracy & Accuracy \\ \hline \hline \end{tabular}
\end{table} 
Table 23: Auto encoder model hyperparameters. For binary classification tasks, the sigmoid function is utilized, which outputs a probability score indicating the likelihood of an instance belonging to the positive class. The sigmoid function transforms the raw score into a value between 0 and 1, effectively serving as a threshold for classification. This can be represented as presented in Equation (20) [92]. \[\sigma\left(Z\right)=\frac{1}{1+e^{-z}} \tag{20}\]  
where \(Z\) is the output from the last dense layer. The output layer employs the softmax function for multi-class classification tasks, enabling the model to produce probability distributions across multiple classes. This function takes a vector of raw scores (logits) and normalizes them into a range between 0 and 1, where the sum of the probabilities equals 1. The softmax function can be mathematically expressed as shown in Equation (21) [92]. \[\mathit{Softmax}\left(Z_{i}\right)=\frac{e^{z_{i}}}{\sum_{j}e^{z_{j}}} \tag{21}\]  
where \(Z_{i}\) is the output from the last dense layer for class \(i\), and \(Z_{j}\) represents the raw score for class \(j\).