## 3 Materials and Methods
### Dimensionality Reduction Using Autoencoder  
In this research, dimensionality reduction is performed using Autoencoders (AE), an unsupervised neural network model that effectively reduces high-dimensional data into a more compact, lower-dimensional form. The AE model consists of three key elements: the encoder, the latent space, and the decoder [30]. Figure 4 illustrates the process flow of preprocessing and dimensionality reduction using the Autoencoder. The encoder compresses the input data, transforming the original 80 features into a more compact, lower-dimensional representation. The optimized 18 features in this reduced representation are selected to maintain essential information while ensuring computational efficiency. The latent space acts as a condensed and informative representation of the original data,  
Figure 3: Operational architecture of proposed scalable hybrid AE-ELM IDS framework. capturing key patterns and potential anomalies. The decoder reconstructs the data from the latent space, with the reconstruction error being a crucial measure for detecting anomalies. A large reconstruction error indicates that there may be anomalies in the network traffic. The AE model is trained with the Adam optimizer, which speeds up convergence, using a learning rate of 0.001 over 37 epochs. The model uses ReLU activation functions in the hidden layers to apply non-linear transformations, while a sigmoid activation function is used in the output layer to maintain accuracy in the reconstruction. Dynamic Thresholding for Convergence  
To improve training efficiency and ensure reliable convergence, dynamic thresholds are applied based on the reconstruction error (Mean Squared Error-MSE) of the Autoencoder. The threshold value \(T\) is computed dynamically using the following statistical Formula (1):  
\[T=\mu\text{MSE}+\text{k}\ ^{*}\sigma\text{MSE} \tag{1}\]  
Figure 4: Process flow of preprocessing and dimensionality reduction using autoencoder. where, uMSE and \(\sigma\)MSE denote the running mean and standard deviation of the reconstruction error over training iterations, and k is a sensitivity constant, which adjusts the threshold's responsiveness. The training is considered to have converged when the change in reconstruction loss between two successive epochs satisfies the following Condition (2):  
\[|\text{MSE}_{\text{t}}-\text{MSE}_{\text{(t-1)}}|<\text{T} \tag{2}\]  
This dynamic thresholding mechanism helps stabilize the training process by ensuring that further adjustments to the Autoencoder's parameters are only made when significant changes in reconstruction error are detected, thereby avoiding unnecessary updates when the model is already converging. The overall process of dynamic thresholding and training is summarized in Algorithm 1 below. ```
Begin
1. Load the dataset (rawD): rawD \(\leftarrow\) load_data('CSE-CIC-IDS2018')
2. Apply preprocessing (ppD) on the data: ppD \(\leftarrow\) min_max_scale(rawD)
3. Initialize Autoencoder parameters: weights, biases \(\leftarrow\) initialize_parameters()
4. Set initial threshold (T): T \(\leftarrow\) u_MSE + k \(\cdot\)\(\sigma\)_MSE (where u_MSE is the mean and \(\sigma\)_MSE is the standard deviation of MSE over initial epochs, k is a sensitivity constant)
5.