# Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
## 3 Width-Adaptive Autoencoder
### Adding and Pruning Nodes
The autoencoder (AE) hyperparameters, including learning rate, batch size, optimizer, and loss function, were configured as detailed in Table 2. To evaluate the performance of the AE, features were extracted from the encoder and subsequently classified using two classifiers: a Support Vector Machine (SVM) and a SoftMax layer. While the training of the WAAE is entirely unsupervised, we use a SoftMax classifier only for post-hoc evaluation purposes. Specifically, once the autoencoder has learned the latent representations, a separate SoftMax layer is trained using the encoded features to assess their discriminative power. This approach does not influence the learning of the autoencoder itself. Furthermore, the structure of the latent space can also be evaluated without labels using metrics such as clustering quality, but labeled data is used here solely to quantify classification accuracy as a means of comparison with other models. #### 4.2.1 Performance Evaluation of the Proposed Approach on Benchmark Datasets

The proposed WAAE is trained using two benchmark datasets, MNIST [36] and CIFAR-10 [37], to assess its ability to dynamically learn the width of an AE in an unsupervised manner. * Learned width dynamics

Figure 6 illustrates the evolution of the number of active nodes (i.e., the learned width) across training iterations for both the MNIST [36] and CIFAR-10 [37] datasets. In the early iterations, the number of nodes increases rapidly, reflecting the model's adaptive expansion to capture complex patterns in the input data. For MNIST, the node count peaks at around iteration 250, whereas for CIFAR-10, the maximum is reached near iteration 300. Beyond these peaks, the model begins pruning less informative or redundant nodes, \begin{table}
\begin{tabular}{c c c c} \hline
**Dataset** & **No. of Instances** & **No. of Attributes** & **No. of Categories** \\ \hline MNIST & 70 k & 784 & 10 \\ CIFAR-10 & 60 k & 3072 & 10 \\ Parkinson & 756 & 754 & 2 \\ Epilepsy & 11,500 & 178 & 5 \\ \hline \end{tabular}
\end{table} 
Table 1: Overview of the considered datasets. \begin{table}
\begin{tabular}{c c c c c} \hline
**Hyperparameter** & **MNIST** & **CIFAR-10** & **Parkinson** & **Epilepsy** \\ \hline Optimizer & [PERSON] & Adam & Adam \\ Loss function & Binary & Binary & Mean Squared Error & Mean Squared Error \\ Learning rate & 0.001 & 0.0001 & 0.0001 & 0.001 \\ Batch size & 256 & 64 & 64 & 128 \\ \hline \end{tabular}
\end{table} 
Table 2: Configuration of hyperparameters. thereby reducing the architecture's size. This pruning behavior helps prevent saturation and encourages model efficiency. Ultimately, the number of nodes stabilizes, converging to 1593 for MNIST and 1896 for CIFAR-10. These final widths reflect the respective complexities of the datasets and confirm the model's ability to dynamically adapt its architecture. * Convergence analysis Figure 7 shows the training and validation loss curves over epochs. For MNIST, both losses decrease rapidly and converge closely within the first few epochs, with minimal discrepancy between the curves, indicating efficient learning and low overfitting. In contrast, CIFAR-10 shows a slower, steady decline in both losses, consistent with the dataset's greater complexity. Notably, the validation loss remains slightly below the training loss throughout training on CIFAR-10, suggesting that the model generalizes well. Overall, both datasets demonstrate smooth and stable convergence, with MNIST exhibiting faster learning dynamics and CIFAR-10 requiring more training epochs to achieve similar performance stability. Figure 6: The number of nodes (the learned width) with respect to each iteration on (**a**) MNIST dataset; (**b**) CIFAR-10 dataset. Figure 7: The training and validation loss with respect to each epoch on (**a**) MNIST dataset; (**b**) CIFAR-10 dataset. Performance metrics

Table 3 presents the final learned widths and classification accuracies for both the MNIST and CIFAR-10 datasets using two classifiers: SoftMax and SVM.