# Artificial Intelligence to Reshape the Healthcare Ecosystem
## 2 AI Techniques for Healthcare
### Generative AI
#### 2.4.2 Autoencoders and Variational Autoencoders  
Autoencoders  
Let \(\mathbf{x}\in\mathbb{R}^{d}\) be the input data. They are processed by the encoder function \(\mathbf{z}=f_{\theta}(\mathbf{x})\), where \(f_{\theta}\) is typically a neural network parameterized by \(\theta\). \(\mathbf{z}\in\mathbb{R}^{m}\) is therefore the input data representation in the latent space. The decoder takes \(\mathbf{z}\) as input and converts the data by the function \(\mathbf{\hat{x}}=g_{\phi}(\mathbf{z})\), where \(g_{\phi}\) is typically again a neural network parameterized by \(\phi\). In this way, the input data are reported back into \(\mathbb{R}^{d}\). The autoencoder is trained to minimize the reconstruction error between \(\mathbf{x}\) and \(\mathbf{\hat{x}}\). A commonly used loss function is the root mean squared error (MSE):  
\[\mathcal{L}(\mathbf{x},\mathbf{\hat{x}})=\|\mathbf{x}-\mathbf{\hat{x}}\|^{2} =\|\mathbf{x}-g_{\phi}(f_{\theta}(\mathbf{x}))\|^{2} \tag{37}\]  
The parameters \(\theta\) and \(\phi\) are optimized during the training phase in order to minimize the loss:  
\[\theta^{*},\phi^{*}=\arg\min_{\theta,\phi}\mathbb{E}_{\mathbf{x}\sim p_{ \text{data}}(\mathbf{x})}\Big{[}\|\mathbf{x}-g_{\phi}(f_{\theta}(\mathbf{x})) \|^{2}\Big{]} \tag{38}\]  
Variational Autoencoders  
VAEs include a probabilistic component in their operation. Essentially, the encoder outputs parameters of a probability distribution rather than a simple 'estimate, which is the basis for generating new content. This distribution is referred to as \(q_{\phi}(\mathbf{z}|\mathbf{x})\), and approximates the posterior distribution \(p_{\theta}(\mathbf{z}|\mathbf{x})\). The decoder, in turn, evaluates the likelihood of the data given the latent variables. The VAE's goal is to maximize the so-called evidence lower bound (ELBO):  
\[\log p_{\theta}(\mathbf{x})\geq\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}[ \log p_{\theta}(\mathbf{x}|\mathbf{z})]-\text{KL}(q_{\phi}(\mathbf{z}| \mathbf{x})\|p_{\theta}(\mathbf{z})) \tag{39}\]  
where \(\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}[\log p_{\theta}(\mathbf{x}| \mathbf{z})]\) is the expected log-likelihood of the data, and \(\text{KL}(q_{\phi}(\mathbf{z}|\mathbf{x})\|p_{\theta}(\mathbf{z}))\) is the Kullback-Leibler divergence between the approximate posterior and the prior. In more detail, assuming a Gaussian input data distribution, the encoder outputs the mean \(\mu_{\phi}(\mathbf{x})\) and the standard deviation \(\sigma_{\phi}(\mathbf{x})\) of the latent variable distribution:  
\[q_{\phi}(\mathbf{z}|\mathbf{x})=\mathcal{N}(\mathbf{z};\mu_{\phi}(\mathbf{x}), \sigma_{\phi}^{2}(\mathbf{x})) \tag{40}\]  
In this way, it is not possible to compare two samples: one input and one output. Therefore, the distance must consider the parameters being estimated to allow back-propagation during training. For this purpose, the latent variable \(\mathbf{z}\) is sampled as  
\[\mathbf{z}=\mu_{\phi}(\mathbf{x})+\sigma_{\phi}(\mathbf{x})\odot\epsilon \tag{41}\]  
where \(\epsilon\sim\mathcal{N}(0,\mathbf{I})\). The decoder can sample the reconstructed data \(\mathbf{\hat{x}}\) from the likelihood:  
\[p_{\theta}(\mathbf{x}|\mathbf{z})=\mathcal{N}(\mathbf{\hat{x}};g_{\theta}( \mathbf{z}),\sigma^{2}\mathbf{I}) \tag{42}\]Finally, the VAE loss function combines the reconstruction loss and the Kullback-Leibler (KL) divergence [85]:  
\[\mathcal{L}(\theta,\phi;\mathbf{x})=-\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x}) }[\log p_{\theta}(\mathbf{x}|\mathbf{z})]+\text{KL}(q_{\phi}(\mathbf{z}| \mathbf{x})\|p_{\theta}(\mathbf{z})) \tag{43}\]  
The parameters \(\theta\) and \(\phi\) are optimized by minimizing this loss:  
\[\theta^{*},\phi^{*}=\arg\min_{\theta,\phi}\mathbb{E}_{\mathbf{x}\sim p_{\text {data}}(\mathbf{x})}[\mathcal{L}(\theta,\phi;\mathbf{x})] \tag{44}\]  
The schematic architectures of autoencoders and variational autoencoders are shown in Figure 7.  
Autoencoders and VAEs can be used in different healthcare contexts [82; 84].