# Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
## 3 Width-Adaptive Autoencoder
### Adding and Pruning Nodes
#### 4.2.2 Performance Evaluation of the Proposed Approach on Real Datasets

The performance of the proposed approach in dynamically learning the width of AE is assessed using two real datasets: Parkinson's Parkinson (1958) and Epilepsy (1960). Since the two datasets are unbalanced, the F1 measure is used to assess the model's performance. * Learned width dynamics

Figure 8 shows the learned width for the two real datasets. As can be seen, the number of nodes for Parkinson's reaches its maximum at iteration 250, when the model converges to 2454 nodes. On the other hand, the Epilepsy dataset reaches its maximum at iteration 200 and converges to 2032 nodes. * Comparative evaluation with manual tuning The results in Table 6 show the comparison between manual tuning and dynamic learning approaches, particularly highlighting the impact of different model widths. In the manual tuning approach, the model's width is varied, and experiments are repeated with different widths to identify the optimal configuration for the dataset. For the Parkinson dataset, the manual approach leads to a substantial number of learned parameters (457,948,938), as the width is likely set to large values in an attempt to capture more complex patterns. Similarly, for the Epilepsy dataset, manual tuning results in 74,522,760 learned parameters. This approach requires significant experimentation with different widths, resulting in longer training times, 10,866.75 s for Parkinson and 6192.44 s for Epilepsy. These extended times reflect the repeated trials to find the best width configuration. On the other hand, the dynamic learning approach, which doesn't involve manual width adjustments but adapts automatically to the data, uses far fewer parameters (3,703,840 for Parkinson and 725,602 for Epilepsy), leading to a much more efficient model. As a result, dynamic learning drastically reduces both training time, 175.01 s for Parkinson and 176.62 s for Epilepsy, and testing time (0.11 s for Parkinson and 0.38 s for Epilepsy). Therefore, while manual tuning requires experimentation with different widths to find the optimal configuration, dynamic learning provides a more efficient alternative by automatically determining the best parameters, resulting in faster training and testing times while reducing model complexity. \begin{table}
\begin{tabular}{c c c c c} \hline
**Dataset** & \multicolumn{2}{c}{**Parkinson**} & \multicolumn{2}{c}{**Epilepsy**} \\ \hline Learned Width & \multicolumn{2}{c}{2454} & \multicolumn{2}{c}{2032} \\ \hline Classifier & SoftMax & SVM & SoftMax & SVM \\ Accuracy & 87.04\% & 90.13\% & 65.32\% & 68.65\% \\ F1\_Score & 86.45\% & 89.89\% & 65.21\% & 68.69\% \\ \hline \end{tabular}
\end{table} 
Table 5: Final learned widths, classification accuracies, and F1-score for both the Parkinson and Epilepsy datasets. Figure 9: The training and validation loss with respect to each epoch on (**a**) Parkinson’s dataset; (**b**) Epilepsy dataset. 2.3 Performance Comparison of the Proposed Approach with State-of-the-Art Models on Benchmark Datasets

In this experiment, the performance of the proposed method is evaluated against several state-of-the-art approaches specifically designed to adaptively determine the optimal width of autoencoders (AEs). The three most relevant and advanced methods considered for comparison are DEVDAN [18], SAQN [21], and NDL [17]. Moreover, two configurations for the SAQN model are considered with and without epochs since the original version considers only one epoch. As previously discussed, DEVDAN (Deep Evolving Denoising Autoencoder Network) is a self-adaptive model that dynamically adjusts the width of its AE by adding or pruning neurons based on the Network Significance (NS) metric. The NS value is computed using the bias and variance of the encoder-decoder model. Before updating network parameters through a combination of generative and discriminative training, a SoftMax layer is appended to facilitate discriminative testing. This process follows a prequential \"test-then-train\" approach.