## 4 Learning Constraints in EVAE
### Learning Objective  
The model is complete by defining a simple factorised joint prior on two latent variables in Equation (5):  
\[p_{\theta_{b},\theta_{s}}(z_{b},z_{s})=p_{\theta_{b}}(z_{b})\cdot p_{\theta_{ s}}(z_{s}). \tag{5}\]  
Armed with the defined joint encoder (cf. Equation (1)), joint decoder (cf. Equation (3)), and the joint prior (cf. Equation (5)), we could finally derive the objective function of our proposed EVAE.  
Recall the learning objective of the original VAE in Equation (6) [4],  
\[\mathcal{L}_{VAE}(\theta,\phi,x)=-\mathbb{D}_{KL}\{q_{\phi}(z|x)||p_{\theta}( z)\}+\frac{1}{L}\sum_{l=1}^{L}p_{\theta}(x|z), \tag{6}\]  
where the first term denotes the regularisation penalty from the variational encoder, and the second term expresses decoder-induced reconstruction loss between generation and original input. Following a similar derivation, the learning objective for the EVAE can then be derived from Equation (7)),  
\[\mathcal{L}_{EVAE}=\mathbb{E}_{q((z_{b},z_{s})|x)}\big{[}-\mathbb{D}_{KL}\big{ }\{q((z_{b},z_{s})|x)||p(z_{b},z_{s})\}+\log p_{\theta_{b},\theta_{s}}(\vec{ x}|(z_{b},z_{s}))\big{]}. \tag{7}\]  
From the previous analytical derivation, the learning objective of EVAE can be seen as comprised of two components: the first corresponds to the regularisation penalty of the derived joint encoder of our EVAE, whereas the residual term relates to the overall reconstruction errors from our derived joint decoder.  
Substituting the defined joint encoder, joint decoder, and the joint prior in Equations (1), (3) and (5) into Equation (7), we can further derive the learning objective into the following form:\[\begin{split}\mathcal{L}_{\text{\emph{EVAE}}}(\theta_{b},\theta_{s}, \phi_{b},\phi_{s},x)&=\mathbb{E}_{q((z_{b},z_{b})|x)}\Big{[}-\{ \log(q_{\phi_{b}}(z_{b}|x))+\log(q_{\phi_{b}}(z_{s}|x))+\mathcal{R}_{e}(z_{b},z _{s})\\ &-\log p_{\theta_{b}}(z_{b})-\log p_{\theta_{s}}(z_{s})\}+\log p _{\theta_{b}}(\vec{x}|z_{b})+\log p_{\theta_{s}}(\vec{x}|z_{s})+\mathcal{R}_{ d}(\vec{x}_{b},\vec{x}_{s})\Big{]}\\ &=\mathbb{E}_{q((z_{b},z_{b})|x)}\Big{[}-\{\log(q_{\phi_{b}}(z_{b }|x))-\log(p_{\theta_{b}}(z_{b}))\}+\log p_{\theta_{b}}(\vec{x}|z_{b})\\ &-\{\log(q_{\phi_{b}}(z_{b}|x))-\log(p_{\theta_{s}}(z_{s}))\}+ \log p_{\theta_{s}}(\vec{x}|z_{s})+\big{[}\alpha\Big{]}\Big{\{}\mathcal{R}_{e }(z_{b},z_{s})-\mathcal{R}_{d}(\vec{x}_{b},\vec{x}_{s})\Big{\}}\Big{]}\\ &=\mathbb{E}_{q((z_{b},z_{b})|x)}\Big{[}-\mathbb{D}_{KL}\{q_{\phi _{b}}(z_{b}|x)||p(z_{b})\}+\log p_{\theta_{b}}(\vec{x}|z_{b})-\mathbb{D}_{KL}\{ q_{\phi_{s}}(z_{s}|x||p(z_{s}))\}\\ &+\log p_{\theta_{s}}(\vec{x}|z_{s})+\big{[}\alpha\Big{]}\Big{\{} \mathcal{R}_{e}(z_{b},z_{s})-\mathcal{R}_{d}(\vec{x}_{b},\vec{x}_{s})\Big{\}} \Big{]}\\ &=\mathcal{L}_{\text{\emph{VAE}}_{b}}(\theta_{b},\phi_{b},x)+ \mathcal{L}_{\text{\emph{VAE}}_{s}}(\theta_{s},\phi_{s},x)+\big{[}\alpha \big{]}\Big{\{}\mathcal{R}_{e}(z_{b},z_{s})-\mathcal{R}_{d}(\vec{x}_{b},\vec{ x}_{s})\Big{\}}.\end{split} \tag{8}\]  
To rewrite the learning objective this way is to group two discrepancy functions together. Remaining terms \(\mathcal{L}_{\text{\emph{vae}}_{b}}(\theta_{b},\phi_{b},x)\) and \(\mathcal{L}_{\text{\emph{vae}}_{s}}(\theta_{s},\phi_{s},x)\) match with the learning objectives of two conventional VAEs.