# Dynamic Relevance-Weighting-Based Width-Adaptive Auto-Encoder
## 1 Introduction  
Machine Learning (ML) is a field of Artificial Intelligence (AI) which involves exploiting available data and devising algorithms to emulate human learning. Recently, the Deep Learning (DL) paradigm has added a conceptual and an algorithmic cutting edge to the machine learning field [1, 2]. As such, it has accomplished great success in various fields such as image classification [3], anomaly detection [4], object detection [5], pattern recognition [6], and natural language processing [7], etc. The successful performance of these models is mainly due to the automatic learning of the features from raw data. For instance, Convolutional Neural Networks (CNN) [1] have been exploited broadly for image classification problems [3], and Recurrent Neural networks (RNN) [1] have been employed for sequential data classification such as videos or speech. Alternatively, Autoencoders (AEs) [1] are unsupervised deep learning paradigms, and have been proven to be effective for dimensionality reduction, feature mapping, anomaly detection, image denoising, hyperspectral unmixing, and image generation [1]. Consequently, they have been adopted in many fields such as bioinformatics [8], recommender systems [9], remote sensing [10], and cybersecurity [11]. Moreover, they constitute a main component in successful NLP transformers' architecture, such as Google's BERT [12] and OpenAI's GPT [13]. The greatest challenge when dealing with AEs is choosing the best topology that produces a meaningful and generalizable latent space representation. Typically, the depth andthe width hyperparameters control the topology of a stacked autoencoder and thus affect its performance [14]. Specifically, the depth specifies the number of stacked autoencoders, i.e., the depth of the network, whereas the width specifies the number of hidden units with respect to each layer. The latter corresponds to the number of nodes in the case of a fully connected AE. Determining the most effective topology of an autoencoder (AE) by tuning its depth and width is a challenging task, particularly in unsupervised settings where labeled data are often unavailable for performance evaluation. However, studies such as [14; 15] have shown that even simple architectures with a single hidden layer can achieve state-of-the-art performance when the width is appropriately configured. These findings highlight that the width of an AE can be more influential than its depth in attaining high performance. Consequently, rather than relying on stacked AEs with multiple layers, it becomes essential to identify the optimal number of neurons in the hidden layer of a fully connected AE. Moreover, optimizing width over depth enhances computational efficiency, as wider architectures can leverage parallel matrix operations, whereas deeper networks involve sequential computations. Deep architectures are also more susceptible to challenges such as overfitting [1] and the vanishing gradient problem [2], further supporting the emphasis on width in AE design. Nevertheless, selecting the ideal width remains non-trivial. This is because the width must be defined per dataset and is highly sensitive to data complexity, structure, and noise levels--creating a need for a principled and data-driven approach to determining the latent width rather than relying on arbitrary or fixed configurations. One way of tackling the problem of setting the width of an AE is by empirical experimentation. Nonetheless, manual tuning of the width in such a large search space is computationally expensive and impractical. Moreover, determining the topology of an AE has attracted little attention compared to Supervised Artificial Neural Networks (ANN). In particular, previous attempts were mainly based on genetic and evolutionary optimization approaches, exhibiting expensive computational complexity [16]. This compels the creation of constrained topologies in which the number of nodes is limited [17]. Other alternatives append the AE with a classifier to benefit from the labeled data.