# Variational Autoencoders for Network Lifetime Enhancement in Wireless Sensors
## 3 Methodology
### Architecture of Variational Autoencoders  
VAEs are deep latent space generative models that exhibit excellent results on various applications like language models, protein design, and image generation. The VAEs are found to be one of the successful approaches among the other unsupervised learning techniques. The basic idea of VAE is to create a meaningful data distribution from the encoded information [20; 21]. In simple way, it can be described as: VAEs are utilized to generate a new data sampling from the given input information. For this, the VAEs are equipped with three major blocks called encoder, latent space and decoder. Encoder is the primary block that is used to compress the input information for making it suitable for the latent space. The latent space receives the compressed information from the encoder through the normal distribution with mean (\(\upmu\)) and variance (\(\sigma\)), and reduces the dimensionality of the received data by preserving the major and novel information available in the input. Decoder is the final block that reconstructs the information available in the latent space to its original form, same as of the input. The Autoencoders (AEs) learn the features extracted from the compressed data structures of the input, and decompress the data structures again, to provide a valid output. In contrast, VAEs utilize the Bayesian model for extracting the features from the compressed data structures of the input to generate feature parameters through probability distribution of the given input. From this, the VAEs can create a new form of input data. Hence, the VAEs are represented as a generative architecture and AEs are represented as a reconstruction architecture. In VAEs, the actual input \(x=x_{i=1}^{N}\) generates a random sample \(y=y_{i=1}^{N}\) through discrete or continuous distribution variable. In high dimensional  
Figure 3: Architecture of variable autoencoders. Figure 2: Architecture of autoencoders. space, the output (\(y\)) provides a random vector from the latent space (\(z\)) generated from the encoder output \(e(x)\). VAEs are included with a datapoint (\(x\)) as their input to their neural network block called the encoder, where its output is presented as a latent space (\(z\)). The weights and biases of the latent space (\(z\)) are adjusted with normal distribution parameters of mean (\(\upmu\)) and variance (\(\sigma\)). The latent space (\(z\)) is further connected as an input to the decoder block, where the neural network included in the decoder block provides an outcome as probability distribution of the given input. The Kullback-Leibler (\(KL\)) divergence is utilized to give an approximate distribution for the decoder \(d(z)\) with some \(e(x)\) as shown in Equation (1). [PERSON]'s rule is applied to Equation (1) for reducing the \(KL\) divergence as shown in Equation (2). \[D_{KL}\left[\,e(x)\;||\;d(z)\;\right]=\sum_{z}e(x)\text{log}\bigg{[}\frac{e(x )}{d(z)}\bigg{]}=E\left[\text{log}[\;e(x)-d(z)]\right] \tag{1}\]  
\[E\left[\text{log}[\,e(x)-d(z)]\right]=E\left[\text{log}e(x)-\text{log}d(z)- \text{log}(e)-\text{log}(d)\right] \tag{2}\]  
Since the \(KL\) divergence is expected to be shown with respect to z, the Equation (2) can be further presented as Equation (3) and the \(D_{KL}\) is concluded as Equation (4) as it is always a positive one. \[E\left[\text{log}e(x)-\text{log}d(z)-\text{log}(e)-\text{log}(d)\right]=log \;(d)-[E[\text{log}d(z)]-D_{KL}[e(x)||\;d] \tag{3}\]  
\[\text{log}d\;\geq E[\text{log}d(z)]-D_{KL}[e(x)||\;d] \tag{4}\]  
Evidence Lower Bound (ELBO) of VAEs is shown in Equation (4), and it is represented as the loss function of the neural network on its training process. The \(E[\text{log}d(z)]\) is presented as a reconstruction function, that is, the resultant of latent space (\(z\)). \(D_{KL}[e(x)||d(z)]\) is used to find the similarity between the latent space (\(z\)) and its target distribution \(d(z)\).