# Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
## 3 Width-Adaptive Autoencoder
### Adding and Pruning Nodes
DEVDAN's use of only 14 neurons leads to high reconstruction errors and poor generalization due to its constrained representational capacity. This is confirmed by the DEVDAN model's training loss displayed in Figure 10. It is nearly constant, with minimal changes observed across batches. This implies that the model may not be learning effectively. Although SAQN utilizes more neurons, its static allocation (258 nodes) and lack of dynamic width adaptation result in inefficiencies, as reflected by its relatively high test loss. Moreover, as shown in Table 7, by increasing the number of epochs, the SAQN model exhibits minimal improvements in both train and test losses with a little change in accuracy. There is no improvement to the model's width, which implies that it has reached its learning capacity. This may be the result of over-regularization or restricted node adaption. The WAAE, by contrast, adaptively scales its architecture to 1593 neurons, enabling it to learn complex feature representations more effectively. This dynamic capacity, free from manual hyperparameter constraints and structural rigidity, allows it to consistently outperform the baseline models in both accuracy and reconstruction loss. ing that its dynamic architecture and resource allocation lead to improved learning and generalization, justifying the increased training cost. The results in Table 8 further reinforce the effectiveness of the WAAE by contextualizing its computational cost in light of architectural and operational efficiencies. Unlike NDL, which adopts a class-by-class training strategy requiring repeated replay and adjustment after each new class, the WAAE is trained on all classes simultaneously. This enables it to generalize more effectively without excessive reliance on replay mechanisms that inflate FLOPs. In fact, NDL reaches over 34,000 training GFLOPs due to continual reprocessing of prior data. Furthermore, NDL's sequential learning approach risks overfitting to early classes, undermining its generalization. Similarly, while DevDAN is lightweight with only 14 nodes and incurs just 21 GFLOPs during training, its three-step batch processing framework adds overhead and limits convergence. DevDAN struggles to stabilize, continuously pruning and adding nodes even in later training batches, which hinders consistent learning. In contrast, the WAAE achieves stable convergence within a few epochs by dynamically adjusting its width in response to learned patterns, resulting in superior reconstruction performance and the highest accuracy (95.97%) among all models. SAQN, despite requiring up to 467 GFLOPs during training, fails to achieve similar performance due to inadequate iterative learning and a static architecture that underutilizes its computational potential. The model's inability to dynamically expand to meet data complexity results in poor fea

Figure 10: DevDAN model’s training and testing losses with respect to each iteration on the MNIST dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Train Time** & **Test Time** & **Training** & **Testing** & **Model’s** \\  & **(s)** & **(s)** & **FLOPs (G)** & **FLOPs (G)** & **FLOPs (G)** \\ \hline DevDAN & 628.4579 & 0.1964 & 20.5383 & 0.48424 & 0.00007 \\ SAQN & 132.4657 & 1.8377 & 116.6587 & 8 & 0.0008 \\ SAQN with epochs & 417.1866 & 0.5107 & 466.32379 & 8 & 0.0008 \\ NDL & 497.8599 & 0.8997 & 34162.173 & 17.562 & 0.00198 \\ WAAE & 701.0619 & 0.5542 & 2789.455 & 49.9 & 0.00499 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Computational complexity and time comparison of Width-Adaptive AE and state-of-the-art approaches on the MNIST dataset. ture extraction and elevated reconstruction loss. This results in inferior reconstructions, as demonstrated by the sample instances from the MNIST dataset in Figure 11. Notably, the WAAE also excels in inference efficiency. It delivers a faster test time (0.5542 s) than both SAQN and NDL, despite utilizing significantly more nodes (1593).