# Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
## 3 Width-Adaptive Autoencoder
### Adding and Pruning Nodes
In CIFAR-10, accuracies drop overall, but WAAE achieves the highest performance again. On the Parkinson dataset, WAAE shows a notable lead over others, achieving the highest F1-score among all models. Notably, on the challenging Epilepsy dataset, where other models hover around 20-30% F1-score, WAAE significantly outperforms them, achieving over 60%. This highlights its robustness and adaptability across varying data complexities and domains. The experimental results across both benchmark (MNIST, CIFAR-10) and real-world (Parkinson, Epilepsy) datasets demonstrate that the proposed WAAE consistently achieves high classification accuracy while maintaining computational efficiency. The model dynamically adjusted its width by adding and pruning nodes, resulting in architectures that are both compact and effective. Compared to manually tuned and state-of-the-art models, the proposed method showed competitive or superior performance in terms of accuracy, training time, and resource utilization. These findings confirm the model's ability to adapt to varying dataset complexities and support its effectiveness in both simple and challenging learning scenarios. ### Discussion on Parameter Sensitivity

The exponent parameter \(q\) in the cost function \(J_{FC}(l)\) plays a critical role in shaping the distribution of relevance weights. Lower values of \(q\) (closer to 1) result in more uniform relevance distributions, while higher values increase contrast, emphasizing nodes with better inner product preservation. This behavior aligns with principles from fuzzy entropy and attention mechanisms, where higher contrast sharpens selection but risks overfitting. Similarly, the pruning threshold (used for removing low-relevance nodes) controls the trade-off between model compactness and expressiveness. A low threshold may retain redundant nodes, slowing convergence and increasing computational cost, while a high threshold could risk prematurely eliminating useful capacity. In practice, setting the threshold to a fraction of 70% of the maximum relevance weight ensures conservative yet effective pruning. Future work will include an empirical investigation of these parameter sensitivities to validate these theoretical insights and refine the default choices. ## 5 Conclusions

This paper introduced a novel Width-Adaptive Autoencoder (AE) that autonomously determines the optimal width of the network by learning node-level relevance weights. The proposed approach expands the network when necessary and prunes irrelevant nodes based on a dynamic optimization strategy. By integrating a new cost function, it enables simultaneous learning of architecture and parameters, leading to an efficient and adaptive model. Experimental evaluations on both benchmark datasets (MNIST, CIFAR-10) and real-world datasets (Parkinson, Epilepsy) demonstrated the model's ability to achieve competitive or superior accuracy with lower computational overhead compared to manually tuned or state-of-the-art approaches. These results highlight the method's robustness, scalability, and practical applicability in diverse learning scenarios. Despite its advantages, the current framework is limited to a single hidden layer. Future work will focus on extending the approach to multi-layer or hierarchical architectures, allowing for deeper and more expressive representations. Additionally, we aim to adapt the model for real-time and streaming data environments, where on-the-fly width adaptation can further enhance model efficiency. Exploring the joint optimization of both depth and width is also a promising direction that could lead to fully self-configuring deep learning architectures. Moreover, while the proposed WAAE demonstrates promising results in terms of compression flexibility and reconstruction performance, several limitations should be acknowledged. First, although WAAE reduces model size during inference, its training process involves evaluating and optimizing multiple width configurations, which increases computational costs.