## 3 Our Proposal
### Mapping Latent Spaces with Regression Flows  
In order to map the latent \(\mathbf{z}\) and parameter \(\mathbf{v}\) spaces, we can first consider that the latent \(\mathbf{z}\) and semantic \(\mathbf{t}\) variables are both unknown latent factors where \(p(\mathbf{z}^{\prime})=p(\mathbf{z},\mathbf{t})\). Hence, we can first address the following reduced formulation  
\[\log p_{\theta}(\mathbf{x},\mathbf{v},\mathbf{z}^{\prime})=\log(p_{\theta}( \mathbf{x}|\mathbf{v},\mathbf{z}^{\prime})p_{\theta}(\mathbf{z}^{\prime}))+ \log p_{\theta}(\mathbf{v}|\mathbf{z}^{\prime}). \tag{9}\]This allows to separately model the variational approximation (detailed in Section 2.1), while solving the inference problem \(p_{\theta}(\mathbf{v}|\mathbf{z})\). To address this inference, we propose to optimize the parameters \(\mathbf{\psi}\) of a transform \(f_{\mathbf{\psi}}\) so that \(\mathbf{v}=f_{\mathbf{\psi}}(\mathbf{z})+\mathbf{e}\), where \(\mathbf{e}\sim\mathcal{N}(\mathbf{0},\mathbf{C}_{v})\) models the inference error as a zero-mean additive Gaussian noise with covariance \(\mathbf{C}_{v}\). Here, we assume that the covariance decomposes into \(\mathbf{C}_{v}^{-1}=\sum_{i}\exp(\lambda_{i})\mathbf{Q}_{i}\), where \(\mathbf{Q}_{i}\) are fixed basis functions and the \(\lambda\) are hyperparameters. Therefore, the full joint likelihood that we will optimize is given by  
\[\mathcal{L}_{f_{\mathbf{\psi}},\lambda}=\log\left[p_{\theta}(\mathbf{v}|f_{\mathbf{ \psi}},\lambda,\mathbf{z})p_{\theta}(f_{\mathbf{\psi}}|\mathbf{z})p_{\theta}( \lambda|\mathbf{z})\right]. \tag{10}\]  
If we know the optimal transform \(f_{\mathbf{\psi}}\) and parameters \(\lambda\), the likelihood of the data is  
\[p_{\theta}(\mathbf{v}\mid f_{\mathbf{\psi}},\lambda,\mathbf{z})=\mathcal{N}( \mathbf{v};f_{\mathbf{\psi}}(\mathbf{z}),\mathbf{C}_{v}) \tag{11}\]  
However, the two posteriors \(p_{\theta}(f_{\mathbf{\psi}}|\mathbf{z})\) and \(p_{\theta}(\lambda|\mathbf{z})\) remain intractable in the general case. In order to solve this issue, we rely again on variational inference by defining an approximation \(q_{\mathbf{\phi}}(f_{\mathbf{\psi}},\lambda|\mathbf{v},\mathbf{z})\) and assume that it factorizes as \(q(f_{\mathbf{\psi}},\lambda|\mathbf{v},\mathbf{z})=q(f_{\mathbf{\psi}}|\mathbf{v}, \mathbf{z})q(\lambda|\mathbf{v},\mathbf{z})\). Therefore, our complete inference is  
\[\mathcal{L}_{f_{\mathbf{\psi}},\lambda}=\log\left[p_{\theta}(\mathbf{v}|f_{\mathbf{ \psi}},\lambda,\mathbf{z})\right]+\mathcal{D}_{KL}\left[q_{\mathbf{\phi}}(f_{\mathbf{ \psi}}|\mathbf{z},\mathbf{v})\|p_{\theta}(f_{\mathbf{\psi}}|\mathbf{z})\right]+ \mathcal{D}_{KL}\left[q_{\mathbf{\phi}}(\lambda|\mathbf{z},\mathbf{v})\|p_{\theta }(\lambda|\mathbf{z})\right] \tag{12}\]  
Hence, we can optimize our approximations through the KL divergence if we find a closed form. To solve for \(\lambda\), we use a Gaussian distribution for both the prior \(p_{\theta}(\lambda|\mathbf{z})=\mathcal{N}(\lambda,\mu_{\lambda},C_{\lambda})\) and posterior \(q_{\mathbf{\phi}}(\lambda|\mathbf{z},\mathbf{v})=\mathcal{N}(\lambda,\mu_{q},C_{q})\). Hence, we obtain a simple analytical solution for \(\lambda\). However, the second part of the objective might be more tedious. Indeed, to perform an accurate inference, we need to rely on a complicated non-linear function, which cannot be assumed to be Gaussian. To address this issue, we introduce the idea of _regression flows_. We consider that the transform \(f_{\mathbf{\psi}(\mathbf{z})}\) is a normalizing flow (see Section 2.1) and provides two different ways of optimizing this approximation.