# Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
## 2 Literature Review
### Autoencoder Compression and Architectural Adaptability  
Autoencoder compression and architectural adaptability have become prominent areas of research in light of the increasing computational demands and deployment challenges of deep neural networks. Our work builds upon and complements several major lines of research in model pruning, architectural adaptation, and width-scalable representations. Recent advances in model compression and pruning have emphasized the importance of non-uniform and adaptive strategies. For instance, Shapley Value-based pruning [24] and dynamic structure pruning [25] allocate resources based on component significance rather than uniform heuristics. Methods such as AST [26] and LAASP [27] further integrate adaptive training or loss-aware criteria to balance efficiency and accuracy. These approaches are mainly designed for large-scale language models or CNNs. In contrast, our approach introduces a width-adaptive autoencoder architecture that dynamically scales its latent dimension based on node relevance during training, a principle not directly explored in the aforementioned studies. Pruning is a widely used model compression technique that reduces parameter counts by eliminating unimportant weights or structures. Traditional pruning approaches often apply uniform sparsity across layers, which can lead to suboptimal performance. Recent methods have moved towards non-uniform and adaptive pruning strategies to better preserve task-critical parameters. For instance, [PERSON] et al. [24] introduced a Shapley value-based non-uniform pruning approach for large language models (LLMs), assigning different sparsity levels to layers based on their importance. Their sliding-window-based approximation significantly improves computational efficiency while enhancing perplexity and accuracy compared to uniform pruning baselines like SparseGPT. Similarly, [PERSON] et al. [26] proposed the Adaptive Sparse Trainer (AST), a semi-structured sparse training framework for LLMs that avoids the performance degradation common in one-shot pruning by integrating distillation and weight mask learning during training. Other notable approaches, such as [PERSON] and [PERSON] [28], combine knowledge distillation with pruning to achieve compact models without loss in accuracy. This synergy enhances the control over compression levels and yields strong generalization, demonstrated on ResNet and DenseNet architectures. The Reweighted ADMM method by [PERSON] et al. [29] presents another advancement, introducing a reweighting mechanism in the pruning process that outperforms traditional thresholding by dynamically adjusting weight importance. Additionally, [PERSON] et al. [30] proposed IESSP, a sparse stripe pruning strategy that improves feature retention via a novel loss function and information extraction module, achieving substantial FLOP reductions without compromising accuracy. From a structured pruning perspective, [PERSON] et al. [31] developed a loss-aware automatic criterion selection technique that integrates pruning into training. By automating pruning decisions based on loss metrics, their method eliminates manual tuning and improves both accuracy and compression. Furthermore, [PERSON] et al. [25] proposed Dynamic Structure Pruning, which optimizes intra-channel pruning granularities during training using a differentiable group learning strategy, outperforming standard channel pruning in FLOPs and runtime acceleration. Finally, [PERSON] et al. [32] explored post-training pruning for foundation models, targeting both unstructured and semi-structured sparsity without retraining. Their algorithm achieves substantial compression while minimizing performance degradation, making it practical for deployment scenarios with limited computational resources. Alongside pruning, researchers have explored architectural adaptation as a means to optimize network capacity dynamically. Elastic and Slimmable networks [33] enable runtime width adjustment, allowing a single network to operate at multiple capacity levels.