## 5 EVAE Parameter Estimation  
To learn our proposed EVAE, we resorted to the reparameterisation trick in [4] to develop the optimisation algorithm. This approach directly employs Monte Carlo sampling to attain samples of latent variables.  
As two sets of latent variables in EVAE are assumed to be continuous and differentiable in the real coordinates, two sets of variational parameters are approximated in mean-field and full-rank Gaussians, i.e., \(z_{b}\sim z_{b}|x\) and \(z_{s}\sim z_{s}|x\). Two valid reparameterisations that apply to \(z_{b}\) and \(z_{s}\) can be specified as \(z_{b}=\mu_{b}+\sigma\cdot\epsilon_{b}\) and \(z_{s}=\mu_{s}+L\cdot\epsilon_{s}\), where \(\epsilon_{b}\) and \(\epsilon_{s}\) are two independent noise variables that both follow normal distribution. Reconstruction errors in \(\mathcal{L}_{VAE}(\theta_{b},\phi_{b},x)\) and \(\mathcal{L}_{VAE}(\theta_{s},\phi_{s},x^{(i)})\) can be further expressed as \(\frac{1}{L}\sum_{l=1}^{L}p_{\theta}(\hat{x}|\mu_{b}+\sigma\cdot\epsilon_{b})\) and \(\frac{1}{L}\sum_{l=1}^{L}p_{\theta}(\hat{x}|\mu_{s}+L\cdot\epsilon_{s})\).  
In terms of KL divergence in \(\mathcal{L}_{VAE}(\theta_{b},\phi_{b},x^{(i)})\) and \(\mathcal{L}_{VAE}(\theta_{s},\phi_{s},x^{(i)})\) under Gaussian approximation, their KL terms can be integrated analytically as \(\frac{1}{2}\sum_{k=1}^{K}(1+\log(\sigma_{k}^{(i)})^{2}-(\mu_{b}^{(i)})^{2}-( \sigma_{k}^{(i)})^{2})\) and \(\frac{1}{2}\sum_{k=1}^{K}(1+\log(L_{k}^{(i)})^{2}-(\mu_{s}^{(i)})^{2}-(L_{k}^{ (i)})^{2})\)[4]. Overall maximisation objective \(\mathcal{J}\) is therefore derived from Equation (9)):  
\[\begin{split}&\mathcal{J}_{(\theta_{b},\theta_{b},\phi_{b})}=\\ &\frac{1}{2}\sum_{k=1}^{K}(1+\log(\sigma_{k}^{(i)})^{2}-(\mu_{b} ^{(i)})^{2}-(\sigma_{k}^{(i)})^{2})+\frac{1}{2}\sum_{k=1}^{K}(1+\log(L_{k}^{(i )})^{2}-(\mu_{s}^{(i)})^{2}-(L_{k}^{(i)})^{2})\\ &+\frac{1}{L}\sum_{l=1}^{L}p_{\theta}(\hat{x}|\mu_{b}+\sigma \cdot\epsilon_{b})+\frac{1}{L}\sum_{l=1}^{L}p_{\theta}(\hat{x}|\mu_{s}+L\cdot \epsilon_{s})+\alpha\big{\{}\mathcal{R}_{\epsilon}(z_{s}|x,z_{b}|x)-\mathcal{ R}_{d}(\hat{x}|z_{b},\hat{x}|z_{s})\big{\}},\end{split} \tag{9}\]  
where \(\epsilon_{b}\sim\mathbb{N}(0,1)\) and \(\epsilon_{s}\sim\mathbb{N}(0,1)\). As two sets of variational parameters stay in one learning objective, we relied on alternating optimisation to iteratively derive the set of \((\theta_{b},\phi_{b})\) and \((\theta_{b},\phi_{b})\). The overall learning algorithm is summarised below as Algorithm 1.  
```
Dataset \(X\); encoder: \(q(z_{b},z_{s}|x;\phi_{s},\phi_{b})\); decoder: \(p(\hat{x}|z_{b},z_{s};\theta_{b},\theta_{b})\)  Initialise the parameters of \(\theta_{b}^{*}\theta_{s};\phi_{b};\phi_{s}\)  Set value of hyperparameter \([\alpha]\) repeat while fix \(\theta_{s},\theta_{s}\)do  random sampling from \(\epsilon_{b}\), where \(\epsilon_{b}\sim\mathbb{N}(0,1)\)  Approximate \(\
abla_{\theta_{b},\phi_{b}}(\mathcal{J})\) via the differentiating the learning objective in Equation (9) w.r.t \(\theta_{b},\phi_{b}\)  Update \(\theta_{b},\phi_{b}\) via an off-shelf gradient-ascent algorithm, e.g., Adam, using \(\
abla_{\theta_{b},\phi_{b}}(\mathcal{J})\) endwhile while fix \(\theta_{b},\phi_{b}\)do  random sampling from \(\epsilon_{s}\), where \(\epsilon_{s}\sim\mathbb{N}(0,1)\)  Approximate \(\
abla_{\theta_{s},\phi_{b}}(\mathcal{J})\) by differentiating learning objective in Equation (9) w.r.t \(\theta_{s},\phi_{s}\)  Update \(\theta_{s},\phi_{s}\) via an off-shelf gradient-ascent algorithm, e.g., Adam, using \(\
abla_{\theta_{s},\phi_{s}}(\mathcal{J})\) endwhile  until elbo (objective) is converged to a certain level Return optimal \(\theta_{b};\theta_{s};\phi_{b};\phi_{s}\)
```  
**Algorithm 1** EVAE learning algorithm.