# Appendix C. Details of VBEM for BNP-GLLIM model

## C.1. VB-E-τ step from Section 3.1

To achieve results from Section 3.1, we make use of (5), (6), (F1), and are only interested in the functional dependence of the right-hand side of (B3) on the variable $τ_k$. Thus, any terms that do not depend on $τ_k$ can be absorbed into the additive normalization constant, giving

$$ 
\begin{aligned}
q_{\tau_k}(\tau_k) &= \exp \left\{ \mathbb{E}_{q_{\alpha,\sigma}} [\log p(\tau_k | \alpha, \sigma)] + \sum_{n=1}^{N} \mathbb{E}_{q_{z_n}} q_{\tau_{\setminus\{k\}}} [\log \pi_{z_n}(\tau)] \right\} + \text{constant} \\
&\propto \exp \left\{ -\mathbb{E}_{q_{\alpha,\sigma}} [\sigma] \log \tau_k + [\mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] - 1] \log (1 - \tau_k) \right. \\
&\qquad \left. + \sum_{n=1}^{N} q_{z_n}(k) \log \tau_k + \sum_{n=1}^{N} \sum_{j=k+1}^{K} q_{z_n}(j) \log (1 - \tau_k) \right\} \\
&= \text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2}).
\end{aligned}
$$

Here,

$$ \hat{\gamma}_{k,1} = 1 - \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{n=1}^{N} q_{z_n}(k) = 1 - \mathbb{E}_{q_{\sigma}} [\sigma] + \sum_{n=1}^{N} q_{z_n}(k), $$

$$ \hat{\gamma}_{k,2} = \mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{n=1}^{N} \sum_{j=k+1}^{K} q_{z_n}(j) = \mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{j=k+1}^{K} \sum_{n=1}^{N} q_{z_n}(j). $$

Furthermore, we used the fact that

$$ \log p(\tau_k | \alpha, \sigma) = \log [\text{Beta}(\tau_k | 1 - \sigma, \alpha + k\sigma)] = \log \left[ \frac{\Gamma(1 - \sigma + k\sigma)}{\Gamma(1 - \sigma)\Gamma(\alpha + k\sigma)} \tau_k^{1-\sigma-1} (1 - \tau_k)^{\alpha+k\sigma-1} \right], $$

$$ \log \pi_{z_n}(\tau) = \log \left[ \tau_{z_n} \prod_{l=1}^{z_n-1} (1 - \tau_l) \right] = \log \tau_{z_n} + \sum_{l=1}^{z_n-1} \log (1 - \tau_l), $$

$$ q_{\tau_{\setminus\{k\}}}(\tau_l) = \prod_{i=1, i \neq k}^{K-1} q_{\tau_i}(\tau_l), \quad d_{\tau_{\setminus\{k\}}} = \prod_{i=1, i \neq k}^{K-1} d_{\tau_i}. $$

Finally, we have for $k \in [K]$, let $N_k = \sum_{n=1}^{N} q_{z_n}(k)$ correspond to the weight of cluster $k$, then

$$ q_{\tau_k}(\tau_k) = \text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2}), $$

$$ \hat{\gamma}_{k,1} = 1 - \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + N_k, \quad \hat{\gamma}_{k,2} = \mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{l=k+1}^{K} N_l. $$
# Appendix C. Details of VBEM for BNP-GLLIM model

## C.1. VB-E-τ step from Section 3.1

To achieve results from Section 3.1, we make use of (5), (6), (F1), and are only interested in the functional dependence of the right-hand side of (B3) on the variable $τ_k$. Thus, any terms that do not depend on $τ_k$ can be absorbed into the additive normalization constant, giving

$$ 
\begin{aligned}
q_{\tau_k}(\tau_k) &= \exp \left\{ \mathbb{E}_{q_{\alpha,\sigma}} [\log p(\tau_k | \alpha, \sigma)] + \sum_{n=1}^{N} \mathbb{E}_{q_{z_n}} q_{\tau_{\setminus\{k\}}} [\log \pi_{z_n}(\tau)] \right\} + \text{constant} \\
&\propto \exp \left\{ -\mathbb{E}_{q_{\alpha,\sigma}} [\sigma] \log \tau_k + [\mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] - 1] \log (1 - \tau_k) \right. \\
&\qquad \left. + \sum_{n=1}^{N} q_{z_n}(k) \log \tau_k + \sum_{n=1}^{N} \sum_{j=k+1}^{K} q_{z_n}(j) \log (1 - \tau_k) \right\} \\
&= \text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2}).
\end{aligned}
$$

Here,

$$ 
\begin{aligned}
\hat{\gamma}_{k,1} &= 1 - \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{n=1}^{N} q_{z_n}(k) = 1 - \mathbb{E}_{q_{\sigma}} [\sigma] + \sum_{n=1}^{N} q_{z_n}(k), \\
\hat{\gamma}_{k,2} &= \mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{n=1}^{N} \sum_{j=k+1}^{K} q_{z_n}(j) = \mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{j=k+1}^{K} \sum_{n=1}^{N} q_{z_n}(j).
\end{aligned}
$$

Furthermore, we used the fact that

$$ 
\begin{aligned}
\log p(\tau_k | \alpha, \sigma) &= \log [\text{Beta}(\tau_k | 1 - \sigma, \alpha + k\sigma)] = \log \left[ \frac{\Gamma(1 - \sigma + k\sigma)}{\Gamma(1 - \sigma)\Gamma(\alpha + k\sigma)} \tau_k^{1-\sigma-1} (1 - \tau_k)^{\alpha+k\sigma-1} \right], \\
\log \pi_{z_n}(\tau) &= \log \left[ \tau_{z_n} \prod_{l=1}^{z_n-1} (1 - \tau_l) \right] = \log \tau_{z_n} + \sum_{l=1}^{z_n-1} \log (1 - \tau_l), \\
q_{\tau_{\setminus\{k\}}}(\tau_l) &= \prod_{i=1, i \neq k}^{K-1} q_{\tau_i}(\tau_l), \quad d_{\tau_{\setminus\{k\}}} = \prod_{i=1, i \neq k}^{K-1} d_{\tau_i}.
\end{aligned}
$$

Finally, we have for $k \in [K]$, let $N_k = \sum_{n=1}^{N} q_{z_n}(k)$ correspond to the weight of cluster $k$, then

$$ 
\begin{aligned}
q_{\tau_k}(\tau_k) &= \text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2}), \\
\hat{\gamma}_{k,1} &= 1 - \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + N_k, \quad \hat{\gamma}_{k,2} = \mathbb{E}_{q_{\alpha,\sigma}} [\alpha] + k \mathbb{E}_{q_{\alpha,\sigma}} [\sigma] + \sum_{l=k+1}^{K} N_l.
\end{aligned}
$$
### C.2. VB-E-(α, σ) step from Section 3.1

In the PY case, to achieve results from Section 3.1, we make use of (5), (4), (F1), (F2), and are only interested in the functional dependence of the right-hand side of (B3) to the variables (α, σ). Thus, any terms that do not depend on (α, σ) can be included in the additive normalization constant. This results in $q_{\alpha,\sigma}(\alpha, \sigma)$ being proportional to

$$ 
\begin{aligned}
\tilde{q}_{\alpha,\sigma}(\alpha, \sigma) &= p(\alpha, \sigma | s_1, s_2, a) \exp \left\{ \mathbb{E} \left[ \prod_{k=1}^{K-1} q_{\tau_k} \left[ \log \prod_{k=1}^{K-1} p(\tau_k | \alpha, \sigma) \right] \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \prod_{k=1}^{K-1} \frac{\Gamma(1 - \sigma + \alpha + k\sigma)}{\Gamma(1 - \sigma)\Gamma(\alpha + k\sigma)} \\
&\quad \times \exp \left\{ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} \left[ -\sigma \log \tau_k + (\alpha - 1 + k\sigma) \log (1 - \tau_k) \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \frac{1}{\Gamma(1 - \sigma)^{K-1}} \prod_{k=1}^{K-1} \frac{[\alpha + (k-1)\sigma]\Gamma(\alpha + (k-1)\sigma)}{\Gamma(\alpha + k\sigma)} \\
&\quad \times \exp \left\{ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} \left[ -\sigma (\log \tau_k - k \log (1 - \tau_k)) + (\alpha - 1) \log (1 - \tau_k) \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \frac{1}{\Gamma(1 - \sigma)^{K-1}} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \frac{\prod_{l=0}^{K-2} \Gamma(\alpha + l\sigma)}{\prod_{k=1}^{K-1} \Gamma(\alpha + k\sigma)} \\
&\quad \times \exp \left\{ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} \left[ -\sigma (\log \tau_k - k \log (1 - \tau_k)) + (\alpha - 1) \log (1 - \tau_k) \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \frac{\Gamma(\alpha)}{\Gamma(1 - \sigma)^{K-1}\Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \\
&\quad \times \exp \left\{ -\sigma \left[ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} [\log \tau_k] - \sum_{k=1}^{K-1} k \mathbb{E}_{q_{\tau_k}} [\log (1 - \tau_k)] \right] + (\alpha - 1) \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} [\log (1 - \tau_k)] \right\},
\end{aligned}
$$

where we used the fact that $\Gamma(x+1) = x\Gamma(x)$. Except in the DP-GLLiM case, i.e., $\sigma = 0$, the normalizing constant, $(\int \tilde{q}_{\alpha,\sigma}(\alpha, \sigma)d(\alpha, \sigma))^{-1}$, for $\tilde{q}_{\alpha,\sigma}$ is not tractable. However, to perform VBEM in Appendix F.1, we do not need the full $q_{\alpha,\sigma}$ distribution, but only the means $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha]$ and $\mathbb{E}_{q_{\alpha,\sigma}}[\sigma]$. One solution, therefore, is to use importance sampling or MCMC to compute these expectations by means of Monte Carlo sums. Via the prior
### C.2. VB-E-(α, σ) step from Section 3.1

In the PY case, to achieve results from Section 3.1, we make use of (5), (4), (F1), (F2), and are only interested in the functional dependence of the right-hand side of (B3) to the variables (α, σ). Thus, any terms that do not depend on (α, σ) can be included in the additive normalization constant. This results in $q_{\alpha,\sigma}(\alpha, \sigma)$ being proportional to

$$ 
\begin{aligned}
\tilde{q}_{\alpha,\sigma}(\alpha, \sigma) &= p(\alpha, \sigma | s_1, s_2, a) \exp \left\{ \mathbb{E} \left[ \prod_{k=1}^{K-1} q_{\tau_k} \left[ \log \prod_{k=1}^{K-1} p(\tau_k | \alpha, \sigma) \right] \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \prod_{k=1}^{K-1} \frac{\Gamma(1 - \sigma + \alpha + k\sigma)}{\Gamma(1 - \sigma)\Gamma(\alpha + k\sigma)} \\
&\quad \times \exp \left\{ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} \left[ -\sigma \log \tau_k + (\alpha - 1 + k\sigma) \log (1 - \tau_k) \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \frac{1}{\Gamma(1 - \sigma)^{K-1}} \prod_{k=1}^{K-1} \frac{[\alpha + (k-1)\sigma]\Gamma(\alpha + (k-1)\sigma)}{\Gamma(\alpha + k\sigma)} \\
&\quad \times \exp \left\{ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} \left[ -\sigma (\log \tau_k - k \log (1 - \tau_k)) + (\alpha - 1) \log (1 - \tau_k) \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \frac{1}{\Gamma(1 - \sigma)^{K-1}} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \frac{\prod_{l=0}^{K-2} \Gamma(\alpha + l\sigma)}{\prod_{k=1}^{K-1} \Gamma(\alpha + k\sigma)} \\
&\quad \times \exp \left\{ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} \left[ -\sigma (\log \tau_k - k \log (1 - \tau_k)) + (\alpha - 1) \log (1 - \tau_k) \right] \right\} \\
&= p(\alpha, \sigma | s_1, s_2, a) \frac{\Gamma(\alpha)}{\Gamma(1 - \sigma)^{K-1}\Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \\
&\quad \times \exp \left\{ -\sigma \left[ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} [\log \tau_k] - \sum_{k=1}^{K-1} k \mathbb{E}_{q_{\tau_k}} [\log (1 - \tau_k)] \right] + (\alpha - 1) \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}} [\log (1 - \tau_k)] \right\},
\end{aligned}
$$

where we used the fact that $\Gamma(x+1) = x\Gamma(x)$. Except in the DP-GLLiM case, i.e., $\sigma = 0$, the normalizing constant, $(\int \tilde{q}_{\alpha,\sigma}(\alpha, \sigma)d(\alpha, \sigma))^{-1}$, for $\tilde{q}_{\alpha,\sigma}$ is not tractable. However, to perform VBEM in Appendix F.1, we do not need the full $q_{\alpha,\sigma}$ distribution, but only the means $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha]$ and $\mathbb{E}_{q_{\alpha,\sigma}}[\sigma]$. One solution, therefore, is to use importance sampling or MCMC to compute these expectations by means of Monte Carlo sums. Via the prior
on $(\alpha, \sigma)$ given in (4), it holds that

$$
\begin{align*}
\tilde{q}_{\alpha, \sigma}(\alpha, \sigma) \\
&= \frac{1}{\Gamma(s_1)} s_2^{s_1} (\alpha + \sigma)^{s_1-1} \exp\{-s_2(\alpha + \sigma)\} \frac{\Gamma(\alpha)p(\sigma | a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \\
&\quad \times \exp\left\{-\sigma \left[\sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log \tau_k] - \sum_{k=1}^{K-1} k \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right] + (\alpha-1) \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right\} \\
&= \frac{1}{\Gamma(s_1)} s_2^{s_1} (\alpha + \sigma)^{s_1-1} \exp\left\{-\left[s_2 - \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right](\alpha + \sigma)\right\} \\
&\quad \times e^{-\sigma\xi} \exp\left\{-\sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right\} \frac{\Gamma(\alpha)p(\sigma | a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \\
&= \frac{1}{\Gamma(s_1)} (\alpha + \sigma)^{s_1-1} \exp\left\{-\left[s_2 - \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right](\alpha + \sigma)\right\} \left[s_2 - \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right]^{s_1-s_2} \\
&\quad \times e^{-\sigma\xi} \exp\left\{-\sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right\} \frac{\Gamma(\alpha)p(\sigma | a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] s_2^{s_1} \\
&\propto \text{Gam}(\alpha + \sigma | \hat{s}_1, \hat{s}_2) e^{-\sigma\xi} \frac{\Gamma(\alpha)p(\sigma | a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} \frac{\alpha + (k-1)\sigma}{\alpha + \sigma}. \tag{C1}
\end{align*}
$$

Here, given that $\psi(\cdot)$ is the digamma function defined by $\psi(z) = \frac{d}{dz} \log \Gamma(z) = \frac{\Gamma'(z)}{\Gamma(z)}$, we have

$$
\begin{align*}
\xi &= \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log \tau_k] - \sum_{k=1}^{K-1} (k-1) \mathbb{E}_{q_{\tau_k}}[\log (1 - \tau_k)], \\
\mathbb{E}_{q_{\tau_k}}[\log \tau_k] &= \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}), \\
\mathbb{E}_{q_{\tau_k}}[\log (1 - \tau_k)] &= \psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}), \\
\hat{s}_1 &= s_1 + K - 1, \quad \hat{s}_2 = s_2 - \sum_{k=1}^{K-1} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})].
\end{align*}
$$

We propose to use the following important distribution $\nu(\alpha, \sigma) =$
Gam $(\alpha + \sigma | \hat{s}_1, \hat{s}_2) p(\sigma | a)$ where $p(\sigma | a)$ is the uniform distribution on $[0, 1]$,
denoted as $\mathcal{U}_{[0,1]}(\sigma)$. Then we obtain an expression for the importance weights,

$$
W(\alpha, \sigma) = \frac{\tilde{q}_{\alpha, \sigma}(\alpha, \sigma)}{\nu(\alpha, \sigma)} = e^{-\sigma \xi} \frac{\Gamma(\alpha)}{\Gamma(1-\sigma)^{K-1} \Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} \frac{\alpha + (k-1)\sigma}{\alpha + \sigma}.
$$

The importance sampling scheme then consists of the following steps

* For $m \in [M]$, first simulate independently $\sigma_m$ from $\mathcal{U}_{[0,1]}(\sigma)$ and then simulate conditionally $\alpha_m$ with the $\sigma_m$-shifted gamma $G(\sigma_m, \hat{s}_1, \hat{s}_2)$. This later simulation is easily obtained by simulating a standard $\gamma(\alpha'_m | \hat{s}_1, \hat{s}_2)$ and then subtracting $\sigma_m$ from the result.

* Compute the importance weights $w_m = W(\alpha_m, \alpha_m)$.
on $(\alpha, \sigma)$ given in (4), it holds that

$$
\begin{align*}
\tilde{q}_{\alpha, \sigma}(\alpha, \sigma) &= \frac{1}{\Gamma(s_1)} s_2^{s_1} (\alpha + \sigma)^{s_1-1} \exp\{-s_2(\alpha + \sigma)\} \frac{\Gamma(\alpha)p(\sigma|a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha+(K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \\
&\quad \times \exp\left\{-\sigma \left[ \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log \tau_k] - \sum_{k=1}^{K-1} k \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)] \right] + (\alpha-1) \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)] \right\} \\
&= \frac{1}{\Gamma(s_1)} s_2^{s_1} (\alpha + \sigma)^{s_1-1} \exp\left\{-\left[s_2 - \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right](\alpha + \sigma)\right\} \\
&\quad \times e^{-\sigma\xi} \exp\left\{-\sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right\} \frac{\Gamma(\alpha)p(\sigma|a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha+(K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] \\
&= \frac{1}{\Gamma(s_1)} (\alpha + \sigma)^{s_1-1} \exp\left\{-\left[s_2 - \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right](\alpha + \sigma)\right\} \left[s_2 - \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right]^{s_1-s_2} \\
&\quad \times e^{-\sigma\xi} \exp\left\{-\sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log(1-\tau_k)]\right\} \frac{\Gamma(\alpha)p(\sigma|a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha+(K-1)\sigma)} \prod_{k=1}^{K-1} [\alpha + (k-1)\sigma] s_2^{s_1} \\
&\propto \text{Gam}(\alpha + \sigma | \hat{s}_1, \hat{s}_2) e^{-\sigma\xi} \frac{\Gamma(\alpha)p(\sigma|a)}{\Gamma(1-\sigma)^{K-1}\Gamma(\alpha+(K-1)\sigma)} \prod_{k=1}^{K-1} \frac{\alpha + (k-1)\sigma}{\alpha + \sigma}. \tag{C1}
\end{align*}
$$

Here, given that $\psi(\cdot)$ is the digamma function defined by $\psi(z) = \frac{d}{dz} \log \Gamma(z) = \frac{\Gamma'(z)}{\Gamma(z)}$, we have

$$
\begin{align*}
\xi &= \sum_{k=1}^{K-1} \mathbb{E}_{q_{\tau_k}}[\log \tau_k] - \sum_{k=1}^{K-1} (k-1) \mathbb{E}_{q_{\tau_k}}[\log (1 - \tau_k)], \\
\mathbb{E}_{q_{\tau_k}}[\log \tau_k] &= \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}), \\
\mathbb{E}_{q_{\tau_k}}[\log (1 - \tau_k)] &= \psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}), \\
\hat{s}_1 &= s_1 + K - 1, \quad \hat{s}_2 = s_2 - \sum_{k=1}^{K-1} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})].
\end{align*}
$$

We propose to use the following important distribution $\nu(\alpha, \sigma) =$
Gam $(\alpha + \sigma | \hat{s}_1, \hat{s}_2) p(\sigma | a)$ where $p(\sigma | a)$ is the uniform distribution on $[0, 1]$,
denoted as $\mathcal{U}_{[0,1]}(\sigma)$. Then we obtain an expression for the importance weights,

$$
W(\alpha, \sigma) = \frac{\tilde{q}_{\alpha, \sigma}(\alpha, \sigma)}{\nu(\alpha, \sigma)} = e^{-\sigma \xi} \frac{\Gamma(\alpha)}{\Gamma(1-\sigma)^{K-1} \Gamma(\alpha + (K-1)\sigma)} \prod_{k=1}^{K-1} \frac{\alpha + (k-1)\sigma}{\alpha + \sigma}.
$$

The importance sampling scheme then consists of the following steps

* For $m \in [M]$, first simulate independently $\sigma_m$ from $\mathcal{U}_{[0,1]}(\sigma)$ and then simulate conditionally $\alpha_m$ with the $\sigma_m$-shifted gamma $G(\sigma_m, \hat{s}_1, \hat{s}_2)$. This later simulation is easily obtained by simulating a standard $\gamma(\alpha'_m | \hat{s}_1, \hat{s}_2)$ and then subtracting $\sigma_m$ from the result.

* Compute the importance weights $w_m = W(\alpha_m, \alpha_m)$.
• Approximate the means

$$
\mathbb{E}_{q_{\alpha, \sigma}}[\alpha] \approx \sum_{m=1}^{M} \frac{w_m}{\sum_{i=1}^{M} w_i} \alpha_m, \quad \mathbb{E}_{q_{\alpha, \sigma}}[\sigma] \approx \sum_{m=1}^{M} \frac{w_m}{\sum_{i=1}^{M} w_i} \sigma_m.
$$

Note that this complication is due to the PY. In the DP-GLLiM case, by substituting
$\sigma = 0$ in (C1), the *E* step is much simpler, as it reduces to computing the approximate
posterior expectation of $\alpha$, namely,

$$
\mathbb{E}_{q_{\alpha,0}}[\alpha] = \frac{\hat{s}_1}{\hat{s}_2}, \quad q_{\alpha,0} = \text{Gam}(\alpha | \hat{s}_1, \hat{s}_2).
$$

### C.3. VB-E-Z step from Section 3.1

In some situations, it is useful to use a 1-of-K binary vector $\mathbf{z}_n$ to represent the latent variable $\mathbf{z}_n$ for each observation ($\mathbf{y}_n, \mathbf{x}_n$). To be more precise, we introduce a K-dimensional binary random variable $\mathbf{z}_n = (z_{nk})_{k \in [K]}$, $K \le \infty$, with a 1-of-K representation in which a particular element $z_{nk}$ is equal to 1, i.e., $z_n = k$, and all other elements are equal to 0. The values of $z_{nk}$ thus satisfy $z_{nk} \in \{0, 1\}$ and $\sum_{k \in [K]} z_{nk} = 1, \forall n \in \mathbb{N}^*$. If there is no confusion, we also denote $\mathcal{Z}$ as the latent matrix $\mathcal{Z} = (z_{nk})_{n \in [\mathbb{N}], k \in [K]}$. It is worth to mentioning that when using a 1-of-K representation of $\mathbf{z}_n$, we can also write down marginal the conditional distributions of $\mathcal{X}$ and $\mathcal{Y} | \mathcal{X}$, corresponding to (7), in the form

$$
p(\mathcal{X} \mid \mathcal{Z}, \mathbf{c}, \mathbf{\Gamma}) = \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_L(\mathbf{x}_n \mid \mathbf{c}_k, \mathbf{\Gamma}_k)^{z_{nk}}, \quad (C2)
$$

$$
p(\mathbf{x}_n | \mathbf{c}, \mathbf{\Gamma}) = \sum_{k=1}^{K} p_{z_n}(k) \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k), \quad p_{z_n}(k) = p(z_n = k) = \pi_k(\boldsymbol{\tau}), \tag{C3}
$$

$$
p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) = \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_D(y_n | A_k x_n + b_k, \Sigma_k)^{z_{nk}}, \quad (C4)
$$

$$
p(\mathbf{y}_n | \mathbf{x}_n; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) = \sum_{k=1}^{K} \frac{p_{z_n}(k) \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)}{\sum_{l=1}^{K} p_{z_n}(l) \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_l, \mathbf{\Gamma}_l)} \mathcal{N}_D(\mathbf{y}_n | \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k).
$$

The observations $\mathcal{X}$ and $\mathcal{Y}$ are therefore i.i.d. and generated from the same GMM (C3)
and infinite GLLiM (C4), respectively. Similarly, (6) can be written down in the form

$$
p(\mathcal{Z} | \tau) = \prod_{n=1}^{N} \prod_{k=1}^{K} \pi_k(\tau)^{z_{nk}}.
$$

By using the decomposition (8), the representation (C2), (C4) and absorbing any
• Approximate the means

$$
\mathbb{E}_{q_{\alpha, \sigma}}[\alpha] \approx \sum_{m=1}^{M} \frac{w_m}{\sum_{i=1}^{M} w_i} \alpha_m, \quad \mathbb{E}_{q_{\alpha, \sigma}}[\sigma] \approx \sum_{m=1}^{M} \frac{w_m}{\sum_{i=1}^{M} w_i} \sigma_m.
$$

Note that this complication is due to the PY. In the DP-GLLiM case, by substituting
$\sigma = 0$ in (C1), the *E* step is much simpler, as it reduces to computing the approximate
posterior expectation of $\alpha$, namely,

$$
\mathbb{E}_{q_{\alpha,0}}[\alpha] = \frac{\hat{s}_1}{\hat{s}_2}, \quad q_{\alpha,0} = \text{Gam}(\alpha | \hat{s}_1, \hat{s}_2).
$$

### C.3. VB-E-Z step from Section 3.1

In some situations, it is useful to use a 1-of-K binary vector $\mathbf{z}_n$ to represent the latent variable $\mathbf{z}_n$ for each observation ($\mathbf{y}_n, \mathbf{x}_n$). To be more precise, we introduce a K-dimensional binary random variable $\mathbf{z}_n = (z_{nk})_{k \in [K]}$, $K \le \infty$, with a 1-of-K representation in which a particular element $z_{nk}$ is equal to 1, i.e., $z_n = k$, and all other elements are equal to 0. The values of $z_{nk}$ thus satisfy $z_{nk} \in \{0, 1\}$ and $\sum_{k \in [K]} z_{nk} = 1, \forall n \in \mathbb{N}^*$. If there is no confusion, we also denote $\mathcal{Z}$ as the latent matrix $\mathcal{Z} = (z_{nk})_{n \in [\mathbb{N}], k \in [K]}$. It is worth to mentioning that when using a 1-of-K representation of $\mathbf{z}_n$, we can also write down marginal the conditional distributions of $\mathcal{X}$ and $\mathcal{Y} | \mathcal{X}$, corresponding to (7), in the form

$$
p(\mathcal{X} \mid \mathcal{Z}, \mathbf{c}, \mathbf{\Gamma}) = \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_L(\mathbf{x}_n \mid \mathbf{c}_k, \mathbf{\Gamma}_k)^{z_{nk}}, \quad (C2)
$$

$$
p(\mathbf{x}_n | \mathbf{c}, \mathbf{\Gamma}) = \sum_{k=1}^{K} p_{z_n}(k) \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k), \quad p_{z_n}(k) = p(z_n = k) = \pi_k(\boldsymbol{\tau}), \tag{C3}
$$

$$
p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) = \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_D(y_n | A_k x_n + b_k, \Sigma_k)^{z_{nk}}, \quad (C4)
$$

$$
p(\mathbf{y}_n | \mathbf{x}_n; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) = \sum_{k=1}^{K} \frac{p_{z_n}(k) \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)}{\sum_{l=1}^{K} p_{z_n}(l) \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_l, \mathbf{\Gamma}_l)} \mathcal{N}_D(\mathbf{y}_n | \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k).
$$

The observations $\mathcal{X}$ and $\mathcal{Y}$ are therefore i.i.d. and generated from the same GMM (C3)
and infinite GLLiM (C4), respectively. Similarly, (6) can be written down in the form

$$
p(\mathcal{Z} | \tau) = \prod_{n=1}^{N} \prod_{k=1}^{K} \pi_k(\tau)^{z_{nk}}.
$$

By using the decomposition (8), the representation (C2), (C4) and absorbing any
terms that are independent on $\mathcal{Z}$ into the additive normalization constant, we obtain

$$
\begin{align*}
\log q_{\mathcal{Z}}(\mathcal{Z}) &\equiv \log q_{\mathcal{Z}}^{(r)}(\mathcal{Z}) \\
&= \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{Y} \mid \mathcal{X}, \mathcal{Z}; \hat{\mathbf{A}}, \hat{\mathbf{b}}, \hat{\mathbf{\Sigma}}\right) p\left(\mathcal{X} \mid \mathcal{Z}; \hat{\mathbf{c}}, \hat{\mathbf{\Gamma}}\right) p\left(\mathcal{Z} \mid \mathbf{\tau}\right) \right] + \text{constant} \\
&\propto \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{Y} \mid \mathcal{X}, \mathcal{Z}; \hat{\mathbf{A}}, \hat{\mathbf{b}}, \hat{\mathbf{\Sigma}}\right) \right] + \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{X} \mid \mathcal{Z}; \hat{\mathbf{c}}, \hat{\mathbf{\Gamma}}\right) \right] + \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{Z} \mid \mathbf{\tau}\right) \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log \mathcal{N}_{D}\left(\mathbf{y}_{n} \mid \hat{\mathbf{A}}_{k} \mathbf{x}_{n}+\hat{\mathbf{b}}_{k}, \hat{\mathbf{\Sigma}}_{k}\right) \right] + \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log \mathcal{N}_{L}\left(\mathbf{x}_{n} \mid \hat{\mathbf{c}}_{k}, \hat{\mathbf{\Gamma}}_{k}\right) \right] \\
&\quad + \mathbb{E}_{q_{\mathcal{Z}}} \left[ \sum_{n=1}^{N} \log \left(\pi_{z_{n}}\left(\mathbf{\tau}\right)\right) \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \log \mathcal{N}_{D}\left(\mathbf{y}_{n} \mid \hat{\mathbf{A}}_{k} \mathbf{x}_{n}+\hat{\mathbf{b}}_{k}, \hat{\mathbf{\Sigma}}_{k}\right) + \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \log \mathcal{N}_{L}\left(\mathbf{x}_{n} \mid \hat{\mathbf{c}}_{k}, \hat{\mathbf{\Gamma}}_{k}\right) \\
&\quad + \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \mathbb{E}_{q_{\mathbf{\tau}}} \left[ \log \left(\pi_{k}\left(\mathbf{\tau}\right)\right) \right] = \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \log \rho_{nk}.
\end{align*}
$$

Here, we used the fact that

$$
\begin{align*}
\log \rho_{nk} &= \log \mathcal{N}_D (\textcolor{green}{y_n} | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) + \log \mathcal{N}_L (\textcolor{red}{x_n} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) + \mathbb{E}_{q_{\tau}} [\log (\pi_k(\tau))] \\
&= -\frac{D}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Sigma}}_k| - \frac{1}{2} (\textcolor{green}{y_n} - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k)^{\top} \hat{\mathbf{\Sigma}}_k^{-1} (\textcolor{red}{y_n} - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k) \\
&\quad - \frac{L}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Gamma}}_k| - \frac{1}{2} (\textcolor{red}{x_n} - \hat{\mathbf{c}}_k)^{\top} \hat{\mathbf{\Gamma}}_k^{-1} (\textcolor{green}{x_n} - \hat{\mathbf{c}}_k) \\
&\quad + \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} \psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2}).
\end{align*}
$$

By taking exponential of both sides and taking into account the normalized constant, it holds that

$$
q_{\mathcal{Z}}(\mathcal{Z}) = \frac{1}{\sum_{l=1}^{K} \rho_{nl}} \prod_{n=1}^{N} \prod_{k=1}^{K} \rho_{nk}^{z_{nk}} = \prod_{n=1}^{N} \prod_{k=1}^{K} r_{nk}^{z_{nk}}, \quad r_{nk} = \frac{\rho_{nk}}{\sum_{l=1}^{K} \rho_{nl}}.
$$

Note also that $z_n = k$ if and only if the latent matrix $\mathcal{Z}$ reduces to a sparse matrix $\mathcal{Z}_{nk}$ which has only one position different from 0, namely $z_{nk} = 1$. This leads to the following simplified notation:

$$
\log q_{z_n}(k) \equiv \log q_{z_n}(z_n = k) \equiv \log q_{\mathcal{Z}}(\mathcal{Z}_{nk}) = r_{nk}.
$$
terms that are independent on $\mathcal{Z}$ into the additive normalization constant, we obtain

$$
\begin{align*}
\log q_{\mathcal{Z}}(\mathcal{Z}) &\equiv \log q_{\mathcal{Z}}^{(r)}(\mathcal{Z}) \\
&= \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{Y} \mid \mathcal{X}, \mathcal{Z}; \hat{\mathbf{A}}, \hat{\mathbf{b}}, \hat{\mathbf{\Sigma}}\right) p\left(\mathcal{X} \mid \mathcal{Z}; \hat{\mathbf{c}}, \hat{\mathbf{\Gamma}}\right) p\left(\mathcal{Z} \mid \mathbf{\tau}\right) \right] + \text{constant} \\
&\propto \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{Y} \mid \mathcal{X}, \mathcal{Z}; \hat{\mathbf{A}}, \hat{\mathbf{b}}, \hat{\mathbf{\Sigma}}\right) \right] + \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{X} \mid \mathcal{Z}; \hat{\mathbf{c}}, \hat{\mathbf{\Gamma}}\right) \right] + \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log p\left(\mathcal{Z} \mid \mathbf{\tau}\right) \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log \mathcal{N}_{D}\left(\mathbf{y}_{n} \mid \hat{\mathbf{A}}_{k} \mathbf{x}_{n}+\hat{\mathbf{b}}_{k}, \hat{\mathbf{\Sigma}}_{k}\right) \right] + \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \mathbb{E}_{q_{\mathcal{Z}}} \left[ \log \mathcal{N}_{L}\left(\mathbf{x}_{n} \mid \hat{\mathbf{c}}_{k}, \hat{\mathbf{\Gamma}}_{k}\right) \right] \\
&\qquad + \mathbb{E}_{q_{\mathcal{Z}}} \left[ \sum_{n=1}^{N} \log \left(\pi_{z_{n}}\left(\mathbf{\tau}\right)\right) \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \log \mathcal{N}_{D}\left(\mathbf{y}_{n} \mid \hat{\mathbf{A}}_{k} \mathbf{x}_{n}+\hat{\mathbf{b}}_{k}, \hat{\mathbf{\Sigma}}_{k}\right) + \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \log \mathcal{N}_{L}\left(\mathbf{x}_{n} \mid \hat{\mathbf{c}}_{k}, \hat{\mathbf{\Gamma}}_{k}\right) \\
&\qquad + \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \mathbb{E}_{q_{\mathbf{\tau}}} \left[ \log \left(\pi_{k}\left(\mathbf{\tau}\right)\right) \right] = \sum_{n=1}^{N} \sum_{k=1}^{K} z_{nk} \log \rho_{nk}.
\end{align*}
$$

Here, we used the fact that

$$
\begin{align*}
\log \rho_{nk} &= \log \mathcal{N}_D (\textcolor{green}{y_n} | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) + \log \mathcal{N}_L (\textcolor{red}{x_n} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) + \mathbb{E}_{q_{\tau}} [\log (\pi_k(\tau))] \\
&= -\frac{D}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Sigma}}_k| - \frac{1}{2} (\textcolor{green}{y_n} - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k)^{\top} \hat{\mathbf{\Sigma}}_k^{-1} (\textcolor{red}{y_n} - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k) \\
&\quad - \frac{L}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Gamma}}_k| - \frac{1}{2} (\textcolor{red}{x_n} - \hat{\mathbf{c}}_k)^{\top} \hat{\mathbf{\Gamma}}_k^{-1} (\textcolor{green}{x_n} - \hat{\mathbf{c}}_k) \\
&\quad + \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} \psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2}).
\end{align*}
$$

By taking exponential of both sides and taking into account the normalized constant, it holds that

$$
q_{\mathcal{Z}}(\mathcal{Z}) = \frac{1}{\sum_{l=1}^{K} \rho_{nl}} \prod_{n=1}^{N} \prod_{k=1}^{K} \rho_{nk}^{z_{nk}} = \prod_{n=1}^{N} \prod_{k=1}^{K} r_{nk}^{z_{nk}}, \quad r_{nk} = \frac{\rho_{nk}}{\sum_{l=1}^{K} \rho_{nl}}.
$$

Note also that $z_n = k$ if and only if the latent matrix $\mathcal{Z}$ reduces to a sparse matrix $\mathcal{Z}_{nk}$ which has only one position different from 0, namely $z_{nk} = 1$. This leads to the following simplified notation:

$$
\log q_{z_n}(k) \equiv \log q_{z_n}(z_n = k) \equiv \log q_{\mathcal{Z}}(\mathcal{Z}_{nk}) = r_{nk}.
$$
### C.3.1. Updating $\Sigma_k$

By using matrix derivatives, the derivative of the log likelihood with respect to $\Sigma_k^{-1}$ is given by

$$ 
\begin{aligned}
& \frac{\partial}{\partial \Sigma_k^{-1}} f_1(\hat{\mathbf{A}}_k, \mathbf{b}_k, \Sigma_k^{-1}) \\
&= -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \frac{\partial}{\partial \Sigma_k^{-1}} \left[ -\log |\Sigma_k^{-1}| + \operatorname{Tr} \left[ \Sigma_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \right] \right] \\
&= -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \left[ -\Sigma_k + (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \right] \\
&= \frac{N_k}{2} \Sigma_k - \frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top}.
\end{aligned}
$$

Finally, setting to zero yields

$$ \hat{\Sigma}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k) (\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k)^{\top}. $$

### C.4. VB-M-(c, $\Gamma$) step from Section 3.2

This step divides into K sub-steps that involve the following optimisations

$$ (\hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \equiv (\hat{\mathbf{c}}_k^{(r)}, \hat{\mathbf{\Gamma}}_k^{(r)}) = \underset{(\mathbf{c}_k, \mathbf{\Gamma}_k)}{\operatorname{argmax}} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [\log p(\mathcal{X} | \mathcal{Z}; \mathbf{c}_k, \mathbf{\Gamma}_k)]. $$

By definition, we have

$$ 
\begin{aligned}
& \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [\log (p(\mathcal{X} | \mathcal{Z}; \mathbf{c}_k, \mathbf{\Gamma}_k))] \\
&= \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} \left[ \log \prod_{n=1}^{N} \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)^{z_{nk}} \right] \\
&= \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [z_{nk} \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)] \\
&= \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [z_{nk}] \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \log |\mathbf{\Gamma}_k| - \frac{1}{2} (\mathbf{x}_n - \mathbf{c}_k)^{\top} \mathbf{\Gamma}_k^{-1} (\mathbf{x}_n - \mathbf{c}_k) \right] \\
&\equiv f_2(\mathbf{c}_k, \mathbf{\Gamma}_k).
\end{aligned}
$$
### C.3.1. Updating $\Sigma_k$

By using matrix derivatives, the derivative of the log likelihood with respect to $\Sigma_k^{-1}$ is given by

$$ 
\begin{aligned}
& \frac{\partial}{\partial \Sigma_k^{-1}} f_1(\hat{\mathbf{A}}_k, \mathbf{b}_k, \Sigma_k^{-1}) \\
& = -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \frac{\partial}{\partial \Sigma_k^{-1}} \left[ -\log |\Sigma_k^{-1}| + \operatorname{Tr} \left[ \Sigma_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \right] \right] \\
& = -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \left[ -\Sigma_k + (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \right] \\
& = \frac{N_k}{2} \Sigma_k - \frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top}.
\end{aligned}
$$

Finally, setting to zero yields

$$ \hat{\Sigma}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k) (\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k)^{\top}. $$

### C.4. VB-M-(c, $\Gamma$) step from Section 3.2

This step divides into K sub-steps that involve the following optimisations

$$ (\hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \equiv (\hat{\mathbf{c}}_k^{(r)}, \hat{\mathbf{\Gamma}}_k^{(r)}) = \underset{(\mathbf{c}_k, \mathbf{\Gamma}_k)}{\operatorname{argmax}} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [\log p(\mathcal{X} | \mathcal{Z}; \mathbf{c}_k, \mathbf{\Gamma}_k)]. $$

By definition, we have

$$ 
\begin{aligned}
& \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [\log (p(\mathcal{X} | \mathcal{Z}; \mathbf{c}_k, \mathbf{\Gamma}_k))] \\
& = \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} \left[ \log \prod_{n=1}^{N} \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)^{z_{nk}} \right] \\
& = \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [z_{nk} \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)] \\
& = \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [z_{nk}] \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k) \\
& = \sum_{n=1}^{N} q_{z_n}(k) \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k) \\
& = \sum_{n=1}^{N} q_{z_n}(k) \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \log |\mathbf{\Gamma}_k| - \frac{1}{2} (\mathbf{x}_n - \mathbf{c}_k)^{\top} \mathbf{\Gamma}_k^{-1} (\mathbf{x}_n - \mathbf{c}_k) \right] \\
& \equiv f_2(\mathbf{c}_k, \mathbf{\Gamma}_k).
\end{aligned}
$$
We aim to solve the following optimisation

$$ (\hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) = \underset{(\mathbf{c}_k, \mathbf{\Gamma}_k)}{\operatorname{argmax}} f_2(\mathbf{c}_k, \mathbf{\Gamma}_k). $$

Similarly with Appendices C.3.1 and C.5.1, we obtain the following update:

$$ \begin{aligned} \hat{\mathbf{c}}_k &= \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k)\mathbf{x}_n, \\ \hat{\mathbf{\Gamma}}_k &= \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{x}_n - \hat{\mathbf{c}}_k)(\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top}. \end{aligned} $$

### C.5. VB-M-(A, b, Σ) step from Section 3.2

By definition, we have

$$ \begin{aligned} & \mathbb{E}_{q_{\mathcal{Z}}^{(r)}}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}; \mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)] \\ &= \mathbb{E}_{q_{\mathcal{Z}}^{(r)}}\left[\log \prod_{n=1}^{N} \mathcal{N}_D\left(y_n \mid \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k\right)^{z_{nk}}\right] \\ &= \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}}[z_{nk} \log \mathcal{N}_D\left(y_n \mid \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k\right)] \\ &= \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}}[z_{nk}] \log \mathcal{N}_D\left(y_n \mid \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k\right) \\ &= \sum_{n=1}^{N} q_{z_n}(k) \log \mathcal{N}_D\left(y_n \mid \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k\right) \\ &= \sum_{n=1}^{N} q_{z_n}(k) \left[-\frac{D}{2} \log(2\pi) - \frac{1}{2} \log |\mathbf{\Sigma}_k| - \frac{1}{2} (y_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \mathbf{\Sigma}_k^{-1} (y_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)\right] \\ &\equiv f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k). \end{aligned} $$

We aim to solve the following optimisation

$$ (\hat{\mathbf{A}}_k, \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) = \underset{(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)}{\operatorname{argmax}} f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k). $$
We aim to solve the following optimisation

$$ (\hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) = \underset{(\mathbf{c}_k, \mathbf{\Gamma}_k)}{\operatorname{argmax}} f_2(\mathbf{c}_k, \mathbf{\Gamma}_k). $$

Similarly with Appendices C.3.1 and C.5.1, we obtain the following update:

$$ \hat{\mathbf{c}}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n, $$

$$ \hat{\mathbf{\Gamma}}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{x}_n - \hat{\mathbf{c}}_k) (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top}. $$

### C.5. VB-M-(A, b, Σ) step from Section 3.2

By definition, we have

$$
\begin{align*}
& \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}; \mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)] \\
&= \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} \left[ \log \prod_{n=1}^{N} \mathcal{N}_D (\mathbf{y}_n | \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k)^{z_{nk}} \right] \\
&= \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [z_{nk} \log \mathcal{N}_D (\mathbf{y}_n | \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k)] \\
&= \sum_{n=1}^{N} \mathbb{E}_{q_{\mathcal{Z}}^{(r)}} [z_{nk}] \log \mathcal{N}_D (\mathbf{y}_n | \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \log \mathcal{N}_D (\mathbf{y}_n | \mathbf{A}_k \mathbf{x}_n + \mathbf{b}_k, \mathbf{\Sigma}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left[ -\frac{D}{2} \log(2\pi) - \frac{1}{2} \log |\mathbf{\Sigma}_k| - \frac{1}{2} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \right] \\
&\equiv f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k).
\end{align*}
$$

We aim to solve the following optimisation

$$ (\hat{\mathbf{A}}_k, \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) = \underset{(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)}{\operatorname{argmax}} f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k). $$
### C.5.1. Updating $\mathbf{b}_k$

The derivative of the log likelihood with respect to $\mathbf{b}_k$ is given by

$$ 
\begin{aligned}
\frac{\partial}{\partial \mathbf{b}_k} f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k) &= -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \frac{\partial}{\partial \mathbf{b}_k} \left[ (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \right] \\
&= -\sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \frac{\partial}{\partial \mathbf{b}_k} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k).
\end{aligned}
$$

Setting this derivative to zero, we obtain the solution for VB-M-b step given by

$$ 
\begin{aligned}
\sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) &= 0 \\
\Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} \mathbf{b}_k &= 0 \\
\Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{b}_k &= 0 \quad (\text{left multiplying by } \mathbf{\Sigma}_k) \\
\Leftrightarrow \mathbf{b}_k &= \frac{1}{\sum_{n=1}^{N} q_{z_n}(k)} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) \equiv \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n). \quad (C5)
\end{aligned}
$$

### C.5.2. Updating $\mathbf{A}_k$

The derivative of the log likelihood with respect to $\mathbf{A}_k$ is given by

$$ 
\begin{aligned}
\frac{\partial}{\partial \mathbf{A}_k} f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k) &= -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \frac{\partial}{\partial \mathbf{A}_k} \left[ (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \right] \\
&= -\sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \frac{\partial}{\partial \mathbf{A}_k} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \mathbf{x}_n^{\top}.
\end{aligned}
$$
### C.5.1. Updating $\mathbf{b}_k$

The derivative of the log likelihood with respect to $\mathbf{b}_k$ is given by

$$ 
\begin{aligned}
\frac{\partial}{\partial \mathbf{b}_k} f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k) &= -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \frac{\partial}{\partial \mathbf{b}_k} \left[ (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \right] \\
&= -\sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \frac{\partial}{\partial \mathbf{b}_k} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k).
\end{aligned}
$$

Setting this derivative to zero, we obtain the solution for VB-M-b step given by

$$ 
\begin{aligned}
\sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) &= 0 \\
\Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} \mathbf{b}_k &= 0 \\
\Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{b}_k &= 0 \quad (\text{left multiplying by } \mathbf{\Sigma}_k) \\
\Leftrightarrow \mathbf{b}_k &= \frac{1}{\sum_{n=1}^{N} q_{z_n}(k)} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) \equiv \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n). \quad (C5)
\end{aligned}
$$

### C.5.2. Updating $\mathbf{A}_k$

The derivative of the log likelihood with respect to $\mathbf{A}_k$ is given by

$$ 
\begin{aligned}
\frac{\partial}{\partial \mathbf{A}_k} f_1(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k) &= -\frac{1}{2} \sum_{n=1}^{N} q_{z_n}(k) \frac{\partial}{\partial \mathbf{A}_k} \left[ (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k)^{\top} \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \right] \\
&= -\sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \frac{\partial}{\partial \mathbf{A}_k} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \mathbf{x}_n^{\top}.
\end{aligned}
$$
Then, we set this derivative w.r.t. $\mathbf{A}_k$ equal to zero, giving

$$ \sum_{n=1}^{N} q_{z_n}(k) \Sigma_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \mathbf{x}_n^{\top} = 0 $$

$$ \Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \mathbf{x}_n^{\top} = 0 \quad (\text{left multiplying by } \mathbf{\Sigma}_k) $$

$$ \Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \mathbf{x}_n^{\top} - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{A}_k \mathbf{x}_n \mathbf{x}_n^{\top} - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{b}_k \mathbf{x}_n^{\top} = 0 $$

$$ \Leftrightarrow \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \mathbf{x}_n^{\top} - \mathbf{A}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \mathbf{x}_n^{\top} - \mathbf{b}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^{\top} = 0 $$

$$ \Leftrightarrow \mathbf{A}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \mathbf{x}_n^{\top} $$

$$ = \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \mathbf{x}_n^{\top} - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^{\top} \quad (\text{using (C5) for } \mathbf{b}_k) $$

$$ \Leftrightarrow \mathbf{A}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \left( \mathbf{x}_n^{\top} - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^{\top} \right) = \sum_{n=1}^{N} q_{z_n}(k) \left( \mathbf{y}_n - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \right) \mathbf{x}_n^{\top} $$

$$ \Leftrightarrow N_k \mathbf{A}_k \mathbf{X}_k \mathbf{X}_k^{\top} = N_k \mathbf{Y}_k \mathbf{X}_k^{\top} \Leftrightarrow \mathbf{A}_k = \mathbf{Y}_k \mathbf{X}_k^{\top} \left( \mathbf{X}_k \mathbf{X}_k^{\top} \right)^{-1} $$

Here, the last equality is obtained by firstly define the following quantities,

$$ \bar{\mathbf{x}}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n, $$

$$ \bar{\mathbf{y}}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n, $$

$$ \mathbf{X}_k = \frac{1}{\sqrt{N_k}} \left( \sqrt{q_{z_1}(k)}(\mathbf{x}_1 - \bar{\mathbf{x}}_k), \dots, \sqrt{q_{z_N}(k)}(\mathbf{x}_N - \bar{\mathbf{x}}_k) \right)_{L \times N}, $$

$$ \mathbf{Y}_k = \frac{1}{\sqrt{N_k}} \left( \sqrt{q_{z_1}(k)}(\mathbf{y}_1 - \bar{\mathbf{y}}_k), \dots, \sqrt{q_{z_N}(k)}(\mathbf{y}_N - \bar{\mathbf{y}}_k) \right)_{D \times N}. $$
Then, we set this derivative w.r.t. $\mathbf{A}_k$ equal to zero, giving

$$
\begin{align*}
& \sum_{n=1}^{N} q_{z_n}(k) \mathbf{\Sigma}_k^{-1} (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \mathbf{x}_n^\top = 0 \\
\Leftrightarrow & \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n - \mathbf{b}_k) \mathbf{x}_n^\top = 0 \quad \text{(left multiplying by } \mathbf{\Sigma}_k \text{)} \\
\Leftrightarrow & \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \mathbf{x}_n^\top - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{A}_k \mathbf{x}_n \mathbf{x}_n^\top - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{b}_k \mathbf{x}_n^\top = 0 \\
\Leftrightarrow & \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \mathbf{x}_n^\top - \mathbf{A}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \mathbf{x}_n^\top - \mathbf{b}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^\top = 0 \\
\Leftrightarrow & \mathbf{A}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \mathbf{x}_n^\top \\
& = \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \mathbf{x}_n^\top - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \mathbf{A}_k \mathbf{x}_n) \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^\top \quad \text{(using (C5) for } \mathbf{b}_k \text{)} \\
\Leftrightarrow & \mathbf{A}_k \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \left( \mathbf{x}_n^\top - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^\top \right) = \sum_{n=1}^{N} q_{z_n}(k) \left( \mathbf{y}_n - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \right) \mathbf{x}_n^\top \\
\Leftrightarrow & N_k \mathbf{A}_k \mathbf{X}_k \mathbf{X}_k^\top = N_k \mathbf{Y}_k \mathbf{X}_k^\top \Leftrightarrow \mathbf{A}_k = \mathbf{Y}_k \mathbf{X}_k^\top (\mathbf{X}_k \mathbf{X}_k^\top)^{-1}.
\end{align*}
$$

Here, the last equality is obtained by firstly define the following quantities,

$$
\bar{\mathbf{x}}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n,
$$

$$
\bar{\mathbf{y}}_k = \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n,
$$

$$
\mathbf{X}_k = \frac{1}{\sqrt{N_k}} \left( \sqrt{q_{z_1}(k)}(\mathbf{x}_1 - \bar{\mathbf{x}}_k), \dots, \sqrt{q_{z_n}(k)}(\mathbf{x}_N - \bar{\mathbf{x}}_k) \right)_{L \times N},
$$

$$
\mathbf{Y}_k = \frac{1}{\sqrt{N_k}} \left( \sqrt{q_{z_1}(k)}(\mathbf{y}_1 - \bar{\mathbf{y}}_k), \dots, \sqrt{q_{z_n}(k)}(\mathbf{y}_N - \bar{\mathbf{y}}_k) \right)_{D \times N}.
$$
Then, we used the fact that

$$
\begin{align*}
& \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \left( \mathbf{x}_n^{\top} - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^{\top} \right) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \left( \mathbf{x}_n^{\top} - \bar{\mathbf{x}}_k^{\top} \right) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right)^{\top} + \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right)^{\top} \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right)^{\top} \\
&= N_k \mathbf{X}_k \mathbf{X}_k^{\top},
\end{align*}
$$

and

$$
\begin{align*}
& \sum_{n=1}^{N} q_{z_n}(k) \left( \textcolor{green}{y_n} - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \textcolor{green}{y_n} \right) \textcolor{red}{x_n^\top} \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \textcolor{green}{y_n} - \bar{\textcolor{green}{y}}_k \right) \textcolor{red}{x_n^\top} \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \textcolor{green}{y_n} - \bar{\textcolor{green}{y}}_k \right) \left( \textcolor{green}{x_n} - \bar{\textcolor{green}{x}}_k \right)^{\top} + \sum_{n=1}^{N} q_{z_n}(k) \left( \textcolor{green}{y_n} - \bar{\textcolor{green}{y}}_k \right) \bar{\textcolor{red}{x}}_k^{\top} \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \textcolor{green}{y_n} - \bar{\textcolor{green}{y}}_k \right) \left( \textcolor{green}{x_n} - \bar{\textcolor{green}{x}}_k \right)^{\top} \\
&= N_k \textcolor{red}{Y}_k \textcolor{red}{X}_k^{\top}.
\end{align*}
$$

Here, we also use the equalities

$$
\begin{align*}
\sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k (\mathbf{x}_n - \bar{\mathbf{x}}_k)^{\top} &= \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \mathbf{x}_n^{\top} - \bar{\mathbf{x}}_k N_k \bar{\mathbf{x}}_k^{\top} = \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \mathbf{x}_n^{\top} - \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \mathbf{x}_n^{\top} = 0, \\
\sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \bar{\mathbf{y}}_k) \bar{\mathbf{x}}_k^{\top} &= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \bar{\mathbf{x}}_k^{\top} - N_k \bar{\mathbf{y}}_k \bar{\mathbf{x}}_k^{\top} = \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \bar{\mathbf{x}}_k^{\top} - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \bar{\mathbf{x}}_k^{\top} = 0,
\end{align*}
$$
Then, we used the fact that

$$
\begin{align*}
& \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \left( \mathbf{x}_n^\top - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n^\top \right) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n \left( \mathbf{x}_n^\top - \bar{\mathbf{x}}_k^\top \right) \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right)^\top + \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right)^\top \\
&= \sum_{n=1}^{N} q_{z_n}(k) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right) \left( \mathbf{x}_n - \bar{\mathbf{x}}_k \right)^\top \\
&= N_k \mathbf{X}_k \mathbf{X}_k^\top,
\end{align*}
$$

and

$$
\begin{align*}
& \sum_{n=1}^{N} q_{z_n}(k) \left( \textcolor{green}{y}_n - \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \textcolor{green}{y}_n \right) \textcolor{red}{x}_n^\top \\
&= \sum_{n=1}^{N} q_{z_n}(k) (\textcolor{green}{y}_n - \bar{\textcolor{green}{y}}_k) \textcolor{red}{x}_n^\top \\
&= \sum_{n=1}^{N} q_{z_n}(k) (\textcolor{green}{y}_n - \bar{\textcolor{green}{y}}_k) (\textcolor{green}{x}_n - \bar{\textcolor{green}{x}}_k)^\top + \sum_{n=1}^{N} q_{z_n}(k) (\textcolor{green}{y}_n - \bar{\textcolor{green}{y}}_k) \bar{\textcolor{red}{x}}_k^\top \\
&= \sum_{n=1}^{N} q_{z_n}(k) (\textcolor{green}{y}_n - \bar{\textcolor{green}{y}}_k) (\textcolor{green}{x}_n - \bar{\textcolor{red}{x}}_k)^\top \\
&= N_k \textcolor{red}{Y}_k \textcolor{red}{X}_k^\top.
\end{align*}
$$

Here, we also use the equalities

$$
\begin{align*}
\sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k (\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top &= \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \mathbf{x}_n^\top - \bar{\mathbf{x}}_k N_k \bar{\mathbf{x}}_k^\top = \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \mathbf{x}_n^\top - \sum_{n=1}^{N} q_{z_n}(k) \bar{\mathbf{x}}_k \mathbf{x}_n^\top = 0, \\
\sum_{n=1}^{N} q_{z_n}(k) (\mathbf{y}_n - \bar{\mathbf{y}}_k) \bar{\mathbf{x}}_k^\top &= \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \bar{\mathbf{x}}_k^\top - N_k \bar{\mathbf{y}}_k \bar{\mathbf{x}}_k^\top = \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \bar{\mathbf{x}}_k^\top - \sum_{n=1}^{N} q_{z_n}(k) \mathbf{y}_n \bar{\mathbf{x}}_k^\top = 0,
\end{align*}
$$
and for each $i, j \in [L]$, it holds that

$$ 
\begin{aligned}
\left[ \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{x}_n - \bar{\mathbf{x}}_k) (\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top \right]_{ij} &= \sum_{n=1}^{N} q_{z_n}(k) \left[ (\mathbf{x}_n - \bar{\mathbf{x}}_k) (\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top \right]_{ij} \\
&= \sum_{n=1}^{N} q_{z_n}(k) [(\mathbf{x}_n - \bar{\mathbf{x}}_k)]_{i1} [(\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top]_{1j} \\
&\equiv \sum_{n=1}^{N} q_{z_n}(k) [(\mathbf{x}_n - \bar{\mathbf{x}}_k)]_i [(\mathbf{x}_n - \bar{\mathbf{x}}_k)]_j \\
&= \left[ \left( \sqrt{q_{z_1}(k)}(\mathbf{x}_1 - \bar{\mathbf{x}}_k), \dots, \sqrt{q_{z_n}(k)}(\mathbf{x}_N - \bar{\mathbf{x}}_k) \right) \right]_i \left[ \left( \sqrt{q_{z_1}(k)}(\mathbf{x}_1 - \bar{\mathbf{x}}_k), \dots, \sqrt{q_{z_n}(k)}(\mathbf{x}_N - \bar{\mathbf{x}}_k) \right) \right]_j^\top \\
&= N_k [\mathbf{X}_k \mathbf{X}_k^\top]_{ij}.
\end{aligned}
$$

## Appendix D. Details on the ELBO

In this section, we provide the closed-form expressions for the ELBO stated in Proposition 3.1. Let us recall that when $\sigma = 0$, the ELBO in the BNP-GLLIM is derived as follows:

$$ \mathcal{F}(q_{\mathcal{Z}}, q_{\Theta}, \hat{\phi}) = \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] \\ + \mathbb{E}[\log p(\Theta; \hat{\phi})] - \mathbb{E}[\log q(\mathcal{Z})] - \mathbb{E}[\log q(\Theta)], $$

The terms of the right-hand side of the above equation have the following closed-form expressions:

$$ \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_D (\mathbf{y}_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \quad (D1) $$

$$ \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k), \quad (D2) $$

$$ \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] = \sum_{k=1}^{K} N_k \left[ \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})] \right], \quad (D3) $$

$$ \mathbb{E}[\log p(\Theta; \hat{\phi})] = \sum_{k=1}^{K-1} \mathbb{E}[\log p(\tau_k | \alpha)] + \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)], \quad (D4) $$

$$ \mathbb{E}[\log p(\tau_k | \alpha)] = \frac{\hat{s}_1 - \hat{s}_2}{\hat{s}_2} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})] + \psi(\hat{s}_1) - \log(\hat{s}_2), $$

$$ \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$
and for each $i, j \in [L]$, it holds that

$$
\begin{aligned}
& \left[ \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{x}_n - \bar{\mathbf{x}}_k) (\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top \right]_{ij} = \sum_{n=1}^{N} q_{z_n}(k) \left[ (\mathbf{x}_n - \bar{\mathbf{x}}_k) (\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top \right]_{ij} \\
& = \sum_{n=1}^{N} q_{z_n}(k) [(\mathbf{x}_n - \bar{\mathbf{x}}_k)]_{i1} [(\mathbf{x}_n - \bar{\mathbf{x}}_k)^\top]_{1j} \\
& \equiv \sum_{n=1}^{N} q_{z_n}(k) [(\mathbf{x}_n - \bar{\mathbf{x}}_k)]_i [(\mathbf{x}_n - \bar{\mathbf{x}}_k)]_j \\
& = \left[ \left( \sqrt{q_{z_1}(k)}(\mathbf{x}_1 - \bar{\mathbf{x}}_k), \dots, \sqrt{q_{z_n}(k)}(\mathbf{x}_N - \bar{\mathbf{x}}_k) \right) \right]_i \left[ \left( \sqrt{q_{z_1}(k)}(\mathbf{x}_1 - \bar{\mathbf{x}}_k), \dots, \sqrt{q_{z_n}(k)}(\mathbf{x}_N - \bar{\mathbf{x}}_k) \right) \right]_j^\top \\
& = N_k [\mathbf{X}_k \mathbf{X}_k^\top]_{ij}.
\end{aligned}
$$

## Appendix D. Details on the ELBO

In this section, we provide the closed-form expressions for the ELBO stated in Proposition 3.1. Let us recall that when $\sigma = 0$, the ELBO in the BNP-GLLIM is derived as follows:

$$ \mathcal{F}(q_{\mathcal{Z}}, q_{\Theta}, \hat{\phi}) = \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] \\ + \mathbb{E}[\log p(\Theta; \hat{\phi})] - \mathbb{E}[\log q(\mathcal{Z})] - \mathbb{E}[\log q(\Theta)], $$

The terms of the right-hand side of the above equation have the following closed-form expressions:

$$ \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_D (\mathbf{y}_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \quad (\text{D1}) $$

$$ \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k), \quad (\text{D2}) $$

$$ \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] = \sum_{k=1}^{K} N_k \left[ \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})] \right], \quad (\text{D3}) $$

$$ \mathbb{E}[\log p(\Theta; \hat{\phi})] = \sum_{k=1}^{K-1} \mathbb{E}[\log p(\tau_k | \alpha)] + \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)], \quad (\text{D4}) $$

$$ \mathbb{E}[\log p(\tau_k | \alpha)] = \frac{\hat{s}_1 - \hat{s}_2}{\hat{s}_2} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})] + \psi(\hat{s}_1) - \log(\hat{s}_2), $$

$$ \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$
$$ \mathbb{E}[\log q(\mathcal{Z})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log q_{z_n}(k), \qquad (D5) $$

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha,0}(\alpha)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)], \qquad (D6) $$

$$ \mathbb{E}[\log q_{\alpha,0}(\alpha)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$

$$ \mathbb{E}[\log q_{\tau_k}(\tau_k)] = \sum_{l=1}^{2} (\hat{\gamma}_{k,l} - 1) \{\psi(\hat{\gamma}_{k,l}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})\} + \log \frac{\Gamma(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})}{\Gamma(\hat{\gamma}_{k,1}) \Gamma(\hat{\gamma}_{k,2})}. $$

# Appendix E. Technical proofs

## E.1. Proof of Lemma 5.2

We first want to prove (18). Using the partition of a joint Gaussian with $x_b \equiv x$, $\mu_b = \mu_k^x, \Sigma_{bb} \equiv V_k^{xx}, x_a \equiv y, \mu_a \equiv \mu_k^y, \Sigma_{aa} \equiv V_k^{yy}$, we obtain

$$ \begin{aligned} p(x_a | x_b) &= \mathcal{N}(x_a | \mu_{a|b}, \Gamma_{aa}^{-1}), & \mu_{a|b} &= \mu_a - \Gamma_{aa}^{-1} \Gamma_{ab} (x_b - \mu_b) = \mu_a - \Sigma_{ab} \Sigma_{bb}^{-1} (x_b - \mu_b), \\ p(x_b) &= \mathcal{N}(x_b | \mu_b, \Sigma_{bb}). \end{aligned} \qquad (E1) $$

Recall that

$$ \begin{aligned} p(y | x, Z = k; \psi) &= \mathcal{N}_D(y | A_k x + b_k, \Sigma_k), \\ p(x | Z = k; \psi) &= \mathcal{N}_L(x | c_k, \Gamma_k), p(Z = k; \psi) = \pi_k. \end{aligned} \qquad (E2) $$

By identifying the parameters of (E1) and (E2), it holds that

$$ \begin{aligned} \pi_k &= \rho_k \\ c_k &= \mu_k^x, \\ \Gamma_k &= V_k^{xx}, \\ A_k &= -\Gamma_{aa}^{-1}\Gamma_{ab} = V_k^{xy\top} (V_k^{xx})^{-1}, \\ b_k &= \mu_a + \Gamma_{aa}^{-1}\Gamma_{ab}\mu_b = \mu_k^y - V_k^{xy\top} (V_k^{xx})^{-1} \mu_k^x, \\ \Sigma_k &= \Gamma_{aa}^{-1} = \Sigma_{aa} - \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba} = V_k^{yy} - V_k^{xy\top} (V_k^{xx})^{-1} V_k^{xy}. \end{aligned} $$

The following decomposition of the joint probability distribution will be used:

$$ \begin{aligned} p(\mathbf{w} | \psi) &= \sum_{k=1}^{K} p(y | \mathbf{x}, Z = k; \psi) p(\mathbf{X} = \mathbf{x} | Z = k; \psi) p(Z = k; \psi) \\ &= \sum_{k=1}^{K} \pi_k \mathcal{N}_D(y | A_k \mathbf{x} + b_k, \Sigma_k) \mathcal{N}_L(x | c_k, \Gamma_k) \\ &\equiv \sum_{k=1}^{K} \rho_k \mathcal{N}_{L+D}(\mathbf{w} | \mu_k, \mathbf{V}_k). \end{aligned} $$
$$ \mathbb{E}[\log q(\mathcal{Z})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log q_{z_n}(k), \qquad (D5) $$

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha,0}(\alpha)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)], \qquad (D6) $$

$$ \mathbb{E}[\log q_{\alpha,0}(\alpha)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$

$$ \mathbb{E}[\log q_{\tau_k}(\tau_k)] = \sum_{l=1}^{2} (\hat{\gamma}_{k,l} - 1) \{\psi(\hat{\gamma}_{k,l}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})\} + \log \frac{\Gamma(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})}{\Gamma(\hat{\gamma}_{k,1}) \Gamma(\hat{\gamma}_{k,2})}. $$

# Appendix E. Technical proofs

## E.1. Proof of Lemma 5.2

We first want to prove (18). Using the partition of a joint Gaussian with $x_b \equiv x$, $\mu_b = \mu_k^x, \Sigma_{bb} \equiv V_k^{xx}, x_a \equiv y, \mu_a \equiv \mu_k^y, \Sigma_{aa} \equiv V_k^{yy}$, we obtain

$$ \begin{aligned} p(x_a | x_b) &= \mathcal{N} \left( x_a \mid \mu_{a|b}, \Gamma_{aa}^{-1} \right), & \mu_{a|b} &= \mu_a - \Gamma_{aa}^{-1} \Gamma_{ab} (x_b - \mu_b) = \mu_a - \Sigma_{ab} \Sigma_{bb}^{-1} (x_b - \mu_b), \\ p(x_b) &= \mathcal{N} \left( x_b \mid \mu_b, \Sigma_{bb} \right). & & (E1) \end{aligned} $$

Recall that

$$ \begin{aligned} p(y | x, Z = k; \psi) &= \mathcal{N}_D(y | A_k x + b_k, \Sigma_k), \\ p(x | Z = k; \psi) &= \mathcal{N}_L(x | c_k, \Gamma_k), p(Z = k; \psi) = \pi_k. & & (E2) \end{aligned} $$

By identifying the parameters of (E1) and (E2), it holds that

$$ \begin{aligned} \pi_k &= \rho_k \\ c_k &= \mu_k^x, \\ \Gamma_k &= V_k^{xx}, \\ A_k &= -\Gamma_{aa}^{-1}\Gamma_{ab} = V_k^{xy\top} (V_k^{xx})^{-1}, \\ b_k &= \mu_a + \Gamma_{aa}^{-1}\Gamma_{ab}\mu_b = \mu_k^y - V_k^{xy\top} (V_k^{xx})^{-1} \mu_k^x, \\ \Sigma_k &= \Gamma_{aa}^{-1} = \Sigma_{aa} - \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba} = V_k^{yy} - V_k^{xy\top} (V_k^{xx})^{-1} V_k^{xy}. \end{aligned} $$

The following decomposition of the joint probability distribution will be used:

$$ \begin{aligned} p(\mathbf{w} | \psi) &= \sum_{k=1}^{K} p(y | \mathbf{x}, Z = k; \psi) p(\mathbf{X} = \mathbf{x} | Z = k; \psi) p(Z = k; \psi) \\ &= \sum_{k=1}^{K} \pi_k \mathcal{N}_D(y | A_k \mathbf{x} + b_k, \Sigma_k) \mathcal{N}_L(x | c_k, \Gamma_k) \\ &\equiv \sum_{k=1}^{K} \rho_k \mathcal{N}_{L+D}(\mathbf{w} | \mu_k, \mathbf{V}_k). \end{aligned} $$
By using result for the joint Gaussian, see e.g., (E6), we obtain the desired result (19).

Finally, Lemma 5.2 is proved via using the following two statements (Deleforge et al., 2015, Lemmas 1 and 2):

(i) For any $\rho_k \in \mathbb{R}$, $\mu_k \in \mathbb{R}^{L+D}$, and $V_k \in S_+^{L+D}$, there is a set of parameters $c_k \in \mathbb{R}^L$, $\lambda_k \in S_+^L$, $\pi_k \in \mathbb{R}$, $A_k \in \mathbb{R}^{D \times L}$, $b_k \in \mathbb{R}^D$, $\Sigma_k \in S_+^D$ such that (18) holds.

(ii) Reciprocally, for any $c_k \in \mathbb{R}^L$, $\Lambda_k \in S_+^L$, $\pi_k \in \mathbb{R}$, $A_k \in \mathbb{R}^{D \times L}$, $b_k \in \mathbb{R}^D$, $\Sigma_k \in S_+^D$, there is a set of parameters $\rho_k \in \mathbb{R}$, $\mu_k \in \mathbb{R}^{L+D}$ and $V_k \in S_+^{L+D}$ such that (19) holds.

## E.2. Proof of Proposition 3.1

Using the sum and product rules for both discrete and continuous variables, the ELBO in BNP-GLLIM (B1) is given by

$$ 
\begin{aligned}
\mathcal{F}(q_Z, q_\Theta, \hat{\phi}) &= \mathbb{E}_{q_Z q_\Theta} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] \\
&\equiv \mathbb{E} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] \\
&= \sum_{\mathcal{Z}} \iint \int q(\mathcal{Z}) q(\Theta) \log \left[ \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] d\mathcal{Z} d\Theta \\
&= \mathbb{E} \left[ \log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\mathcal{Z} | \Theta; \hat{\phi}) \right] \\
&\quad + \mathbb{E} \left[ \log p(\Theta; \hat{\phi}) \right] - \mathbb{E} \left[ \log q(\mathcal{Z}) \right] - \mathbb{E} \left[ \log q(\Theta) \right].
\end{aligned}
\quad (E3) $$

Next, we evaluate the various terms in the ELBO (E3).

### Proof of (D1)

Via the mean field approximation and the truncation, we have the following computations:

$$ 
\begin{aligned}
\mathbb{E} \left[ \log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi}) \right] &= \mathbb{E} \left[ \log \prod_{n=1}^{N} p \left( y_n | \mathbf{x}_n, z_n, \Theta; \hat{\phi} \right) \right] \\
&= \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_D \left( y_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right)^{z_{nk}} \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E} \left[ z_{nk} \log \mathcal{N}_D \left( y_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right) \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_Z} [z_{nk}] \log \mathcal{N}_D \left( y_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right) \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_D \left( y_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right),
\end{aligned}
$$
By using result for the joint Gaussian, see e.g., (E6), we obtain the desired result (19).

Finally, Lemma 5.2 is proved via using the following two statements (Deleforge et al., 2015, Lemmas 1 and 2):

(i) For any $\rho_k \in \mathbb{R}$, $\mu_k \in \mathbb{R}^{L+D}$, and $V_k \in S_+^{L+D}$, there is a set of parameters $c_k \in \mathbb{R}^L$, $\lambda_k \in S_+^L$, $\pi_k \in \mathbb{R}$, $A_k \in \mathbb{R}^{D \times L}$, $b_k \in \mathbb{R}^D$, $\Sigma_k \in S_+^D$ such that (18) holds.

(ii) Reciprocally, for any $c_k \in \mathbb{R}^L$, $\Lambda_k \in S_+^L$, $\pi_k \in \mathbb{R}$, $A_k \in \mathbb{R}^{D \times L}$, $b_k \in \mathbb{R}^D$, $\Sigma_k \in S_+^D$, there is a set of parameters $\rho_k \in \mathbb{R}$, $\mu_k \in \mathbb{R}^{L+D}$ and $V_k \in S_+^{L+D}$ such that (19) holds.

## E.2. Proof of Proposition 3.1

Using the sum and product rules for both discrete and continuous variables, the ELBO in BNP-GLLIM (B1) is given by

$$ 
\begin{aligned}
\mathcal{F}(q_Z, q_\Theta, \hat{\phi}) &= \mathbb{E}_{q_Z q_\Theta} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] \\
&\equiv \mathbb{E} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] \\
&= \sum_{\mathcal{Z}} \iint q(\mathcal{Z}) q(\Theta) \log \left[ \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] d\mathcal{Z} d\Theta \\
&= \mathbb{E} \left[ \log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\mathcal{Z} | \Theta; \hat{\phi}) \right] \\
&\quad + \mathbb{E} \left[ \log p(\Theta; \hat{\phi}) \right] - \mathbb{E} \left[ \log q(\mathcal{Z}) \right] - \mathbb{E} \left[ \log q(\Theta) \right].
\end{aligned}
\quad (E3) $$

Next, we evaluate the various terms in the ELBO (E3).

### Proof of (D1)

Via the mean field approximation and the truncation, we have the following computations:

$$ 
\begin{aligned}
\mathbb{E} \left[ \log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi}) \right] &= \mathbb{E} \left[ \log \prod_{n=1}^{N} p \left( y_n \mid \mathbf{x}_n, z_n, \Theta; \hat{\phi} \right) \right] \\
&= \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_D \left( y_n \mid \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right)^{z_{nk}} \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E} \left[ z_{nk} \log \mathcal{N}_D \left( y_n \mid \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right) \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_Z} [z_{nk}] \log \mathcal{N}_D \left( y_n \mid \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right) \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_D \left( y_n \mid \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right),
\end{aligned}
$$
where

$$ \log \mathcal{N}_D (\mathbf{y}_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) = -\frac{D}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Sigma}}_k| \\ -(\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k)^{\top} \hat{\mathbf{\Sigma}}_k^{-1} (\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k). $$

Proof of (D2)

Similarly to the previous proof, we obtain

$$ \begin{aligned} \mathbb{E} [\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] &= \mathbb{E} \left[ \log \prod_{n=1}^{N} p(\mathbf{x}_n | z_n, \Theta; \hat{\phi}) \right] \\ &= \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k)^{z_{nk}} \right] \\ &= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E} [z_{nk} \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k)] \\ &= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_{\mathcal{Z}}} [z_{nk}] \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \\ &= \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k), \end{aligned} $$

where

$$ \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) = -\frac{L}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Gamma}}_k| - \frac{1}{2} (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top} \hat{\mathbf{\Gamma}}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{c}}_k). $$

Proof of (D3)

Via calculation, it follows the expressions of the following quantities,

$$ \begin{aligned} \mathbb{E}_{q_{\tau_k}} [\log (\tau_k)] &= \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}), \\ \mathbb{E}_{q_{\tau_k}} [\log (1 - \tau_k)] &= \psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}). \end{aligned} \quad (E4) $$
where

$$ \log \mathcal{N}_D (\mathbf{y}_n | \hat{\mathbf{A}}_k \mathbf{x}_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) = -\frac{D}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Sigma}}_k| \\ -(\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k)^{\top} \hat{\mathbf{\Sigma}}_k^{-1} (\mathbf{y}_n - \hat{\mathbf{A}}_k \mathbf{x}_n - \hat{\mathbf{b}}_k). $$

Proof of (D2)

Similarly to the previous proof, we obtain

$$ \begin{aligned} \mathbb{E} \left[ \log p(\mathcal{X} \mid \mathcal{Z}, \Theta; \hat{\phi}) \right] &= \mathbb{E} \left[ \log \prod_{n=1}^{N} p(\mathbf{x}_n \mid z_n, \Theta; \hat{\phi}) \right] \\ &= \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k)^{z_{nk}} \right] \\ &= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E} \left[ z_{nk} \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \right] \\ &= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_{\mathcal{Z}}} [z_{nk}] \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \\ &= \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k), \end{aligned} $$

where

$$ \log \mathcal{N}_L (\mathbf{x}_n | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) = -\frac{L}{2} \log(2\pi) - \frac{1}{2} \log |\hat{\mathbf{\Gamma}}_k| - \frac{1}{2} (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top} \hat{\mathbf{\Gamma}}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{c}}_k). $$

Proof of (D3)

Via calculation, it follows the expressions of the following quantities,

$$ \begin{aligned} \mathbb{E}_{q_{\tau_k}} [\log (\tau_k)] &= \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}), \\ \mathbb{E}_{q_{\tau_k}} [\log (1 - \tau_k)] &= \psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}). \end{aligned} \quad (E4) $$
Via (E4), it holds that

$$
\begin{align*}
\mathbb{E} [\log p(\mathcal{Z} | \Theta; \hat{\phi})] &= \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} [\pi_k(\tau)]^{z_{nk}} \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_{\mathcal{Z}}} [z_{nk}] \mathbb{E}_{q_{\Theta}} \left[ \log \left[ \tau_k \prod_{l=1}^{k-1} (1 - \tau_l) \right] \right] \\
&= \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_{nk}} \left[ \mathbb{E}_{q_{\tau_k}} [\log \tau_k] + \sum_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}} [\log (1 - \tau_l)] \right] \\
&= \sum_{k=1}^{K} N_k \left[ \mathbb{E}_{q_{\tau_k}} [\log \tau_k] + \sum_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}} [\log (1 - \tau_l)] \right] \\
&= \sum_{k=1}^{K} N_k \left[ \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})] \right].
\end{align*}
$$

Proof of (D4)

Given a chosen truncated value $K \in \mathbb{N}^*$, it holds that

$$
\mathbb{E}_{q_{\Theta}} \left[ \log p(\Theta; \hat{\phi}) \right] = \sum_{k=1}^{K-1} \mathbb{E}_{q_{\Theta}} \left[ \log p(\tau_k | \alpha, \sigma) \right] + \mathbb{E}_{q_{\Theta}} \left[ \log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2, \hat{a}) \right].
$$

Here, we have

$$
\begin{align*}
\mathbb{E}_{q_{\Theta}} [\log p(\tau_k | \alpha, \sigma)] &= \mathbb{E}_{q_{\Theta}} [\log \text{Beta}(\tau_k | 1 - \sigma, \alpha + k\sigma)] \\
&= \mathbb{E}_{q_{\Theta}} \left[ \log \tau_k^{-\sigma} (1 - \tau_k)^{\alpha+k\sigma-1} + \log C(\alpha, \sigma) \right], \\
&= -\mathbb{E}_{q_{\alpha,\sigma}} [\sigma] \mathbb{E}_{q_{\tau_k}} [\log \tau_k] + \mathbb{E}_{q_{\alpha,\sigma}} [\alpha + k\sigma - 1] \mathbb{E}_{q_{\tau_k}} [\log(1 - \tau_k)] \\
&\quad + \mathbb{E}_{q_{\alpha,\sigma}} [\log C(\alpha, \sigma)],
\end{align*}
$$

where we have defined

$$
C(\alpha, \sigma) = \frac{\Gamma(1 - \sigma + \alpha + k\sigma)}{\Gamma(1 - \sigma)\Gamma(\alpha + k\sigma)}.
$$

Next, for the sake of simplicity, for $\sigma$, we use a uniform prior $\mathcal{U}_{[0,1]}(\sigma)$ so that parameter $a$ does not have to be taken into account. Then it holds that

$$
\begin{align*}
\mathbb{E}_{q_{\Theta}} [\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2)] &= \mathbb{E}_{q_{\alpha, \sigma}} [\log \text{Gam}(\alpha + \sigma | \hat{s}_1, \hat{s}_2)] + \mathbb{E}_{q_{\alpha, \sigma}} [\log \mathcal{U}_{[0,1]}(\sigma)] \\
&= \log \left[ \frac{1}{\Gamma(\hat{s}_1)} \hat{s}_2^{\hat{s}_1} \right] + (\hat{s}_1 - 1) \mathbb{E}_{q_{\alpha, \sigma}} [\log(\alpha + \sigma)] - \hat{s}_2 \mathbb{E}_{q_{\alpha, \sigma}} [\alpha + \sigma] \\
&= \log \left[ \frac{1}{\Gamma(\hat{s}_1)} \hat{s}_2^{\hat{s}_1} \right] + (\hat{s}_1 - 1) \mathbb{E}_{q_{\alpha, \sigma}} [\log(\alpha + \sigma)] - \hat{s}_2 \mathbb{E}_{q_{\alpha, \sigma}} [\alpha + \sigma].
\end{align*}
$$

When $\sigma \neq 0$, the normalizing constant for $q_{\alpha,\sigma}(\alpha, \sigma)$ is not tractable. Nevertheless, to compute the ELBO, we do not need the full $q_{\alpha,\sigma}$ distribution but only the means $\mathbb{E}_{q_{\alpha,\sigma}}[\sigma]$, $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha + k\sigma - 1]$, $\mathbb{E}_{q_{\alpha,\sigma}}[\log C(\alpha, \sigma)]$, $\mathbb{E}_{q_{\alpha,\sigma}}[\log(\alpha + \sigma)]$ and $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha + \sigma]$. One
Via (E4), it holds that

$$
\begin{align*}
\mathbb{E} [\log p(\mathcal{Z} | \Theta; \hat{\phi})] &= \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} [\pi_k(\tau)]^{z_{nk}} \right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_{\mathcal{Z}}} [z_{nk}] \mathbb{E}_{q_{\Theta}} \left[ \log \left[ \tau_k \prod_{l=1}^{k-1} (1 - \tau_l) \right] \right] \\
&= \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_{nk}} \left[ \mathbb{E}_{q_{\tau_k}} [\log \tau_k] + \sum_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}} [\log (1 - \tau_l)] \right] \\
&= \sum_{k=1}^{K} N_k \left[ \mathbb{E}_{q_{\tau_k}} [\log \tau_k] + \sum_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}} [\log (1 - \tau_l)] \right] \\
&= \sum_{k=1}^{K} N_k \left[ \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})] \right].
\end{align*}
$$

Proof of (D4)

Given a chosen truncated value $K \in \mathbb{N}^*$, it holds that

$$
\mathbb{E}_{q_{\Theta}} \left[ \log p(\Theta; \hat{\phi}) \right] = \sum_{k=1}^{K-1} \mathbb{E}_{q_{\Theta}} \left[ \log p(\tau_k | \alpha, \sigma) \right] + \mathbb{E}_{q_{\Theta}} \left[ \log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2, \hat{a}) \right].
$$

Here, we have

$$
\begin{align*}
\mathbb{E}_{q_{\Theta}} [\log p(\tau_k | \alpha, \sigma)] &= \mathbb{E}_{q_{\Theta}} [\log \text{Beta}(\tau_k | 1 - \sigma, \alpha + k\sigma)] \\
&= \mathbb{E}_{q_{\Theta}} \left[ \log \tau_k^{-\sigma} (1 - \tau_k)^{\alpha+k\sigma-1} + \log C(\alpha, \sigma) \right], \\
&= -\mathbb{E}_{q_{\alpha,\sigma}} [\sigma] \mathbb{E}_{q_{\tau_k}} [\log \tau_k] + \mathbb{E}_{q_{\alpha,\sigma}} [\alpha + k\sigma - 1] \mathbb{E}_{q_{\tau_k}} [\log(1 - \tau_k)] \\
&\quad + \mathbb{E}_{q_{\alpha,\sigma}} [\log C(\alpha, \sigma)],
\end{align*}
$$

where we have defined

$$
C(\alpha, \sigma) = \frac{\Gamma(1 - \sigma + \alpha + k\sigma)}{\Gamma(1 - \sigma)\Gamma(\alpha + k\sigma)}.
$$

Next, for the sake of simplicity, for $\sigma$, we use a uniform prior $\mathcal{U}_{[0,1]}(\sigma)$ so that parameter $a$ does not have to be taken into account. Then it holds that

$$
\begin{align*}
\mathbb{E}_{q_{\Theta}} [\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2)] &= \mathbb{E}_{q_{\alpha, \sigma}} [\log \text{Gam}(\alpha + \sigma | \hat{s}_1, \hat{s}_2)] + \mathbb{E}_{q_{\alpha, \sigma}} [\log \mathcal{U}_{[0,1]}(\sigma)] \\
&= \log \left[ \frac{1}{\Gamma(\hat{s}_1)} \hat{s}_2^{\hat{s}_1} \right] + (\hat{s}_1 - 1) \mathbb{E}_{q_{\alpha, \sigma}} [\log(\alpha + \sigma)] - \hat{s}_2 \mathbb{E}_{q_{\alpha, \sigma}} [\alpha + \sigma] \\
&= \log \left[ \frac{1}{\Gamma(\hat{s}_1)} \hat{s}_2^{\hat{s}_1} \right] + (\hat{s}_1 - 1) \mathbb{E}_{q_{\alpha, \sigma}} [\log(\alpha + \sigma)] - \hat{s}_2 \mathbb{E}_{q_{\alpha, \sigma}} [\alpha + \sigma].
\end{align*}
$$

When $\sigma \neq 0$, the normalizing constant for $q_{\alpha,\sigma}(\alpha, \sigma)$ is not tractable. Nevertheless, to compute the ELBO, we do not need the full $q_{\alpha,\sigma}$ distribution but only the means $\mathbb{E}_{q_{\alpha,\sigma}}[\sigma]$, $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha + k\sigma - 1]$, $\mathbb{E}_{q_{\alpha,\sigma}}[\log C(\alpha, \sigma)]$, $\mathbb{E}_{q_{\alpha,\sigma}}[\log(\alpha + \sigma)]$ and $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha + \sigma]$. One
solution is therefore to use importance sampling or MCMC to compute these expectations via Monte Carlo sums.

When $\sigma = 0$, using integration by parts, it holds that $\Gamma(\alpha + 1) = \alpha\Gamma(\alpha)$ and hence $C(\alpha, \sigma) \equiv C(\alpha) = \alpha$. Furthermore, the posterior $q_{\alpha,\sigma} \equiv q_{\alpha}$ is again a gamma distribution Gam $(\alpha | \hat{s}_1, \hat{s}_2)$ with $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha] \equiv \mathbb{E}_{q_{\alpha}}[\alpha] = \frac{\hat{s}_1}{\hat{s}_2}$ and $\mathbb{E}_{q_{\alpha,\sigma}}[\log \alpha] \equiv \mathbb{E}_{q_{\alpha}}[\log \alpha] = \psi(\hat{s}_1) - \log(\hat{s}_2)$. Therefore, we have the following tractable formulas:

$$ 
\begin{aligned}
\mathbb{E}_{q_{\oplus}}[\log p(\tau_k | \alpha, \sigma)] &\equiv \mathbb{E}_{q_{\oplus}}[\log p(\tau_k | \alpha)] \\
&= [\mathbb{E}_{q_{\alpha,0}}[\alpha] - 1] \mathbb{E}_{q_{\tau_k}}[\log(1 - \tau_k)] + \mathbb{E}_{q_{\alpha,0}}[\log \alpha], \\
&= \frac{\hat{s}_1 - \hat{s}_2}{\hat{s}_2} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})] + \psi(\hat{s}_1) - \log(\hat{s}_2),
\end{aligned}
$$

$$ 
\begin{aligned}
\mathbb{E}_{q_{\ominus}}[\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2)] &\equiv \mathbb{E}_{q_{\ominus}}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] \\
&= \log \left[ \frac{1}{\Gamma(\hat{s}_1)} \hat{s}_2^{\hat{s}_1} \right] + (\hat{s}_1 - 1) [\psi(\hat{s}_1) - \log(\hat{s}_2)] - \hat{s}_1.
\end{aligned}
$$

### Proof of (D5)

Due to the mean-field approximation (9) and truncation, this step is analytically computed as follows:

$$ 
\begin{aligned}
\mathbb{E}_{q_Z}[\log q(\mathcal{Z})] &= \mathbb{E}_{q_Z}\left[\log \prod_{n=1}^{N} q_{Z_n}(Z_n)\right] = \mathbb{E}_{q_Z}\left[\log \prod_{n=1}^{N} \prod_{k=1}^{K} q_{Z_n}(k)^{Z_{nk}}\right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \log q_{Z_n}(k) \mathbb{E}_{q_Z}[Z_{nk}] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{Z_n}(k) \log q_{Z_n}(k).
\end{aligned}
$$

### Proof of (D6)

We have

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha,\sigma}(\alpha, \sigma)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)]. $$

Note that these terms involving expectations of the logs of the $q$ distributions simply represent the negative entropies of those distributions.

Since $q_{\alpha,\sigma}(\alpha, \sigma)$ is not tractable, when $\sigma \neq 0$, we cannot calculate analytically $\mathbb{E}[\log q_{\alpha,\sigma}(\alpha, \sigma)]$. Furthermore, it is also difficult to approximate it using MCMC or importance sampling.

When $\sigma = 0$, the posterior $q_{\alpha,\sigma} \equiv q_{\alpha}$ is again a gamma distribution Gam $(\alpha | \hat{s}_1, \hat{s}_2)$ with

$$ 
\begin{aligned}
\mathbb{E}[\log q_{\alpha,0}(\alpha)] &\equiv \mathbb{E}[\log \text{Gam}(\alpha | \hat{s}_1, \hat{s}_2)] \\
&= -\text{H}[\text{Gam}(\alpha | \hat{s}_1, \hat{s}_2)] \\
&= -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1.
\end{aligned}
$$
solution is therefore to use importance sampling or MCMC to compute these expectations via Monte Carlo sums.

When $\sigma = 0$, using integration by parts, it holds that $\Gamma(\alpha + 1) = \alpha\Gamma(\alpha)$ and hence $C(\alpha, \sigma) \equiv C(\alpha) = \alpha$. Furthermore, the posterior $q_{\alpha,\sigma} \equiv q_{\alpha}$ is again a gamma distribution Gam $(\alpha | \hat{s}_1, \hat{s}_2)$ with $\mathbb{E}_{q_{\alpha,\sigma}}[\alpha] \equiv \mathbb{E}_{q_{\alpha}}[\alpha] = \frac{\hat{s}_1}{\hat{s}_2}$ and $\mathbb{E}_{q_{\alpha,\sigma}}[\log \alpha] \equiv \mathbb{E}_{q_{\alpha}}[\log \alpha] = \psi(\hat{s}_1) - \log(\hat{s}_2)$. Therefore, we have the following tractable formulas:

$$ 
\begin{aligned}
\mathbb{E}_{q_{\oplus}}[\log p(\tau_k | \alpha, \sigma)] &\equiv \mathbb{E}_{q_{\oplus}}[\log p(\tau_k | \alpha)] \\
&= [\mathbb{E}_{q_{\alpha,0}}[\alpha] - 1] \mathbb{E}_{q_{\tau_k}}[\log(1 - \tau_k)] + \mathbb{E}_{q_{\alpha,0}}[\log \alpha], \\
&= \frac{\hat{s}_1 - \hat{s}_2}{\hat{s}_2} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})] + \psi(\hat{s}_1) - \log(\hat{s}_2), \\
\mathbb{E}_{q_{\ominus}}[\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2)] &\equiv \mathbb{E}_{q_{\ominus}}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] \\
&= \log \left[ \frac{1}{\Gamma(\hat{s}_1)} \hat{s}_2^{\hat{s}_1} \right] + (\hat{s}_1 - 1) [\psi(\hat{s}_1) - \log(\hat{s}_2)] - \hat{s}_1.
\end{aligned}
$$

### Proof of (D5)

Due to the mean-field approximation (9) and truncation, this step is analytically computed as follows:

$$ 
\begin{aligned}
\mathbb{E}_{q_Z}[\log q(\mathcal{Z})] &= \mathbb{E}_{q_Z}\left[\log \prod_{n=1}^{N} q_{Z_n}(Z_n)\right] = \mathbb{E}_{q_Z}\left[\log \prod_{n=1}^{N} \prod_{k=1}^{K} q_{Z_n}(k)^{Z_{nk}}\right] \\
&= \sum_{n=1}^{N} \sum_{k=1}^{K} \log q_{Z_n}(k) \mathbb{E}_{q_Z}[Z_{nk}] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{Z_n}(k) \log q_{Z_n}(k).
\end{aligned}
$$

### Proof of (D6)

We have

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha,\sigma}(\alpha, \sigma)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)]. $$

Note that these terms involving expectations of the logs of the $q$ distributions simply represent the negative entropies of those distributions.

Since $q_{\alpha,\sigma}(\alpha, \sigma)$ is not tractable, when $\sigma \neq 0$, we cannot calculate analytically $\mathbb{E}[\log q_{\alpha,\sigma}(\alpha, \sigma)]$. Furthermore, it is also difficult to approximate it using MCMC or importance sampling.

When $\sigma = 0$, the posterior $q_{\alpha,\sigma} \equiv q_{\alpha}$ is again a gamma distribution Gam $(\alpha | \hat{s}_1, \hat{s}_2)$ with

$$ 
\begin{aligned}
\mathbb{E}[\log q_{\alpha,0}(\alpha)] &\equiv \mathbb{E}[\log \text{Gam}(\alpha | \hat{s}_1, \hat{s}_2)] \\
&= -\text{H}[\text{Gam}(\alpha | \hat{s}_1, \hat{s}_2)] \\
&= -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1.
\end{aligned}
$$
Since we had $q_{\tau_k}(\tau_k) = \text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2})$, its differential entropy is given by

$$ \begin{aligned} \mathbb{E}[\log q_{\tau_k}(\tau_k)] &= -H[\text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2})] \\ &= \sum_{l=1}^{2} (\hat{\gamma}_{k,l} - 1) \{\psi(\hat{\gamma}_{k,l}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})\} + \log \frac{\Gamma(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})}{\Gamma(\hat{\gamma}_{k,1}) \Gamma(\hat{\gamma}_{k,2})}. \end{aligned} $$

### E.3. Proof of Theorem 4.1

Recall that $\Theta = (\tau, \alpha, \sigma)$. Then,

$$ \begin{aligned} p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y}) &= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{x} | \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{z} | \Theta, \mathcal{X}, \mathcal{Y}) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\ &= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\hat{x} | \hat{z}, \mathbf{c}, \mathbf{\Gamma}) p(\hat{z} | \tau; \beta) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \equiv D_1. \end{aligned} \tag{E5} $$

Note that in (E5), $p(\Theta | \mathcal{X}, \mathcal{Y})$ is in fact the (unknown) true posterior distribution of the parameters given a sample $(\mathcal{X}, \mathcal{Y})$. Because the integrations w.r.t. true posterior distribution are intractable, we approximate the predictive conditional density by replacing the true posterior distribution $p(\Theta | \mathcal{X}, \mathcal{Y})$ with its truncated variational posterior of parameters $\Theta$ given by

$$ q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) = q_{\alpha, \sigma}(\alpha, \sigma | \mathcal{X}, \mathcal{Y}) \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k | \mathcal{X}, \mathcal{Y}). $$

Recall that the infinite state space for each $z_j$ is dealt with by choosing a truncation of the state space to a maximum label $K$ (Blei and Jordan, 2006). In practice, this consists of assuming that the variational distributions $q_{z_n}$ for $n \in [N]$, satisfy $q_{z_n}(k) = 0$ for $k > K$ and that the variational distribution on $\tau$ also factorizes as $q_{\tau}(\tau) = \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k)$ with the additional condition that $\tau_K = 1$. In particular, here we choose $K = \hat{K}$, where $\hat{K}$ is estimated from some suitable procedures.
Since we had $q_{\tau_k}(\tau_k) = \text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2})$, its differential entropy is given by

$$ 
\begin{aligned}
\mathbb{E}[\log q_{\tau_k}(\tau_k)] &= -H[\text{Beta}(\tau_k | \hat{\gamma}_{k,1}, \hat{\gamma}_{k,2})] \\
&= \sum_{l=1}^{2} (\hat{\gamma}_{k,l} - 1) \{\psi(\hat{\gamma}_{k,l}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})\} + \log \frac{\Gamma(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})}{\Gamma(\hat{\gamma}_{k,1}) \Gamma(\hat{\gamma}_{k,2})}.
\end{aligned}
$$

### E.3. Proof of Theorem 4.1

Recall that $\Theta = (\tau, \alpha, \sigma)$. Then,

$$ 
\begin{aligned}
p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y}) &= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{x} | \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{z} | \Theta, \mathcal{X}, \mathcal{Y}) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\hat{x} | \hat{z}, \mathbf{c}, \mathbf{\Gamma}) p(\hat{z} | \tau; \beta) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \equiv D_1.
\end{aligned}
\tag{E5} $$

Note that in (E5), $p(\Theta | \mathcal{X}, \mathcal{Y})$ is in fact the (unknown) true posterior distribution of the parameters given a sample $(\mathcal{X}, \mathcal{Y})$. Because the integrations w.r.t. true posterior distribution are intractable, we approximate the predictive conditional density by replacing the true posterior distribution $p(\Theta | \mathcal{X}, \mathcal{Y})$ with its truncated variational posterior of parameters $\Theta$ given by

$$ q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) = q_{\alpha, \sigma}(\alpha, \sigma | \mathcal{X}, \mathcal{Y}) \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k | \mathcal{X}, \mathcal{Y}). $$

Recall that the infinite state space for each $z_j$ is dealt with by choosing a truncation of the state space to a maximum label $K$ (Blei and Jordan, 2006). In practice, this consists of assuming that the variational distributions $q_{z_n}$ for $n \in [N]$, satisfy $q_{z_n}(k) = 0$ for $k > K$ and that the variational distribution on $\tau$ also factorizes as $q_{\tau}(\tau) = \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k)$ with the additional condition that $\tau_K = 1$. In particular, here we choose $K = \hat{K}$, where $\hat{K}$ is estimated from some suitable procedures.
For simplicity, we consider the case when $\beta = 0, \sigma = 0$. Then we have

$$
\begin{align*}
D_1 &\approx \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \hat{A}, \hat{b}, \hat{\Sigma}) p(\hat{x} | \hat{z}, \hat{c}, \hat{\Gamma}) p(\hat{z} | \tau) q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&= \sum_{k=1}^{\infty} \int \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \pi_k(\tau) q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&\approx \sum_{k=1}^{K} \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \int \pi_k(\tau) q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&= \sum_{k=1}^{K} \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \int \pi_k(\tau) q_{\tau}(\tau | \mathcal{X}, \mathcal{Y}) d\tau \underbrace{\int q_{\alpha,0}(\alpha | \mathcal{X}, \mathcal{Y}) d\alpha}_{=1} \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \\
&\equiv \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_{L+D}(\hat{w} | \mathbb{E}[w], \operatorname{cov}[w]) \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}}[1 - \tau_l] \mathcal{N}_{L+D}(\hat{w} | \mathbb{E}[w], \operatorname{cov}[w]).
\end{align*}
$$

Here, by defining $\hat{w} \equiv [\hat{x}; \hat{y}]$, we used the fact that

$$
\mathbb{E}[\mathbf{w}] = \begin{pmatrix} \hat{c}_k \\ \hat{A}_k \hat{c}_k + \hat{b}_k \end{pmatrix}, \quad \operatorname{cov}[\mathbf{w}] = \begin{pmatrix} \hat{\Gamma}_k & \hat{\Gamma}_k \hat{A}_k^\top \\ \hat{A}_k \hat{\Gamma}_k & \hat{\Sigma}_k + \hat{A}_k \hat{\Gamma}_k \hat{A}_k^\top \end{pmatrix}.
$$

Indeed, we made use of the following result for the joint Gaussian, see, e.g., Bishop (2006, Eq. (2.115), page 93). Given a marginal Gaussian distribution for x and a conditional Gaussian distribution for y given x in the form

$$
\begin{align*}
p(\mathbf{x}) &= \mathcal{N}(\mathbf{x} | \boldsymbol{\mu}, \mathbf{\Gamma}^{-1}), \\
p(\mathbf{y} | \mathbf{x}) &= \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{x} + \mathbf{b}, \mathbf{L}^{-1}),
\end{align*}
$$

then the joint distribution of $\mathbf{w} \equiv [\mathbf{x}; \mathbf{y}]$ is given by

$$
p(\mathbf{w}) = \mathcal{N}(\mathbf{w} | \mathbb{E}[\mathbf{w}], \operatorname{cov}[\mathbf{w}]), \quad \text{where}
\operatorname{cov}[\mathbf{w}] = \begin{pmatrix} \mathbf{\Gamma}^{-1} & \mathbf{\Gamma}^{-1}\mathbf{A}^\top \\ \mathbf{A}\mathbf{\Gamma}^{-1} & \mathbf{L}^{-1} + \mathbf{A}\mathbf{\Gamma}^{-1}\mathbf{A}^\top \end{pmatrix}, \quad \mathbb{E}[\mathbf{w}] = \begin{pmatrix} \boldsymbol{\mu} \\ \mathbf{A}\boldsymbol{\mu} + \mathbf{b} \end{pmatrix}. \tag{E6}
$$

In our situation, the desired result is obtained via using $\mathbf{y} \equiv \hat{\mathbf{y}}$, $\mathbf{A} \equiv \hat{\mathbf{A}}_k$, $\mathbf{b} \equiv \hat{\mathbf{b}}_k$, $\mathbf{L}^{-1} = \hat{\mathbf{\Sigma}}_k$, $\mathbf{x} \equiv \hat{\mathbf{x}}$, $\boldsymbol{\mu} \equiv \hat{\mathbf{c}}_k$, $\mathbf{\Gamma}^{-1} \equiv \hat{\mathbf{\Gamma}}_k$.
For simplicity, we consider the case when $\beta = 0, \sigma = 0$. Then we have

$$
\begin{align*}
D_1 &\approx \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \hat{A}, \hat{b}, \hat{\Sigma}) p(\hat{x} | \hat{z}, \hat{c}, \hat{\Gamma}) p(\hat{z} | \tau) q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&= \sum_{k=1}^{\infty} \int \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \pi_k(\tau) q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&\approx \sum_{k=1}^{K} \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \int \pi_k(\tau) q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \\
&= \sum_{k=1}^{K} \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \int \pi_k(\tau) q_{\tau}(\tau | \mathcal{X}, \mathcal{Y}) d\tau \underbrace{\int q_{\alpha,0}(\alpha | \mathcal{X}, \mathcal{Y}) d\alpha}_{=1} \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \\
&\equiv \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_{L+D}(\hat{w} | \mathbb{E}[w], \operatorname{cov}[w]) \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}}[1 - \tau_l] \mathcal{N}_{L+D}(\hat{w} | \mathbb{E}[w], \operatorname{cov}[w]).
\end{align*}
$$

Here, by defining $\hat{w} \equiv [\hat{x}; \hat{y}]$, we used the fact that

$$
\mathbb{E}[\mathbf{w}] = \begin{pmatrix} \hat{c}_k \\ \hat{A}_k \hat{c}_k + \hat{b}_k \end{pmatrix}, \quad \operatorname{cov}[\mathbf{w}] = \begin{pmatrix} \hat{\Gamma}_k & \hat{\Gamma}_k \hat{A}_k^\top \\ \hat{A}_k \hat{\Gamma}_k & \hat{\Sigma}_k + \hat{A}_k \hat{\Gamma}_k \hat{A}_k^\top \end{pmatrix}.
$$

Indeed, we made use of the following result for the joint Gaussian, see, e.g., Bishop (2006, Eq. (2.115), page 93). Given a marginal Gaussian distribution for x and a conditional Gaussian distribution for y given x in the form

$$
\begin{align*}
p(\mathbf{x}) &= \mathcal{N}(\mathbf{x} | \boldsymbol{\mu}, \mathbf{\Gamma}^{-1}), \\
p(\mathbf{y} | \mathbf{x}) &= \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{x} + \mathbf{b}, \mathbf{L}^{-1}),
\end{align*}
$$

then the joint distribution of $\mathbf{w} \equiv [\mathbf{x}; \mathbf{y}]$ is given by

$$
p(\mathbf{w}) = \mathcal{N}(\mathbf{w} | \mathbb{E}[\mathbf{w}], \operatorname{cov}[\mathbf{w}]), \quad \text{where} \\
\operatorname{cov}[\mathbf{w}] = \begin{pmatrix} \mathbf{\Gamma}^{-1} & \mathbf{\Gamma}^{-1}\mathbf{A}^\top \\ \mathbf{A}\mathbf{\Gamma}^{-1} & \mathbf{L}^{-1} + \mathbf{A}\mathbf{\Gamma}^{-1}\mathbf{A}^\top \end{pmatrix}, \quad \mathbb{E}[\mathbf{w}] = \begin{pmatrix} \boldsymbol{\mu} \\ \mathbf{A}\boldsymbol{\mu} + \mathbf{b} \end{pmatrix}. \tag{E6}
$$

In our situation, the desired result is obtained via using $\mathbf{y} \equiv \hat{\mathbf{y}}$, $\mathbf{A} \equiv \hat{\mathbf{A}}_k$, $\mathbf{b} \equiv \hat{\mathbf{b}}_k$, $\mathbf{L}^{-1} = \hat{\mathbf{\Sigma}}_k$, $\mathbf{x} \equiv \hat{\mathbf{x}}$, $\boldsymbol{\mu} \equiv \hat{\mathbf{c}}_k$, $\mathbf{\Gamma}^{-1} \equiv \hat{\mathbf{\Gamma}}_k$.
Furthermore, we also used the fact that

$$
\begin{align*}
\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] &= \int \tau_k q_{\tau_k}(\tau_k | \mathcal{X}, \mathcal{Y}) d\tau_k \int \prod_{l=1}^{k-1} (1-\tau_l) \prod_{j=1, j \neq k}^{K-1} q_{\tau_j}(\tau_j | \mathcal{X}, \mathcal{Y}) \prod_{j=1, j \neq k}^{K} d\tau_j \\
&= \mathbb{E}_{q_{\tau_k}}[\tau_k] \int \prod_{l=1}^{k-1} (1-\tau_l) \prod_{j=1}^{k-1} q_{\tau_j}(\tau_j | \mathcal{X}, \mathcal{Y}) \underbrace{\int \prod_{j=k+1}^{K-1} q_{\tau_j}(\tau_j | \mathcal{X}, \mathcal{Y}) \prod_{j=k+1}^{K-1} d\tau_j}_{=1} \prod_{j=1}^{k-1} d\tau_j \\
&= \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \int (1-\tau_l) q_{\tau_l}(\tau_l | \mathcal{X}, \mathcal{Y}) d\tau_l \\
&= \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}}[1-\tau_l].
\end{align*}
$$

Next, we aim to prove that

$$
\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] = 1.
$$

Indeed, recall that we have defined

$$
\begin{align*}
& \tau_k | \alpha, \sigma \stackrel{\text{ind}}{\sim} \text{Beta}(\tau_k | 1 - \sigma, \alpha + k\sigma), \quad k \in \mathbb{N}^*, \\
& \pi_k(\tau) = \tau_k \prod_{l=1}^{k-1} (1 - \tau_l), \quad k \in \mathbb{N}^*, \\
& p(\mathcal{Z} | \tau) \propto \prod_{n=1}^{N} \pi_{z_n}(\tau),
\end{align*}
$$

and to deal with the infinite state space for each $z_j$, we considered a truncation of the
state space to a maximum label $K \equiv K_{\max}$, $K_{\max} \in \mathbb{N}^*$ (Blei and Jordan, 2006). In
practice, this consists of assuming that the variational distributions $q_{z_n}$ for $n \in [N]$,
satisfy $q_{z_n}(k) = 0$ for $k > K$ and that the variational distribution on $\tau$ also factorizes
as $q_{\tau}(\tau) = \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k)$ with the additional condition that $\tau_K = 1$. Based on the
proof from Ghosal and Van der Vaart (2017, Lemma 3.4), it holds that a necessary and
sufficient condition to guarantee that these $\pi_k$'s sum to 1 almost surely, i.e.,

$$
\sum_{k=1}^{\infty} \pi_k(\tau) = \sum_{k=1}^{\infty} \tau_k \prod_{l=1}^{k-1} (1 - \tau_l) = 1,
$$

is that the expectation $\mathbb{E}[\prod_{l=1}^{k-1} (1 - \tau_l)]$ tends to 0 as $k$ tends to $\infty$. In particular, if $\tau_1, \tau_2, \dots$ are i.i.d., e.g., when $\sigma = 0$, it suffices that $p(\tau_1 > 0) > 0$. Then

$$
1 = \mathbb{E}_{q_{\tau}} \left[ \sum_{k=1}^{\infty} \pi_k(\tau) \right] = \sum_{k=1}^{\infty} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)] = \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)].
$$
Furthermore, we also used the fact that

$$
\begin{align*}
\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] &= \int \tau_k q_{\tau_k}(\tau_k | \mathcal{X}, \mathcal{Y}) d\tau_k \int \prod_{l=1}^{k-1} (1-\tau_l) \prod_{j=1, j \neq k}^{K-1} q_{\tau_j}(\tau_j | \mathcal{X}, \mathcal{Y}) \prod_{j=1, j \neq k}^{K} d\tau_j \\
&= \mathbb{E}_{q_{\tau_k}}[\tau_k] \int \prod_{l=1}^{k-1} (1-\tau_l) \prod_{j=1}^{k-1} q_{\tau_j}(\tau_j | \mathcal{X}, \mathcal{Y}) \underbrace{\int \prod_{j=k+1}^{K-1} q_{\tau_j}(\tau_j | \mathcal{X}, \mathcal{Y}) \prod_{j=k+1}^{K-1} d\tau_j}_{=1} \prod_{j=1}^{k-1} d\tau_j \\
&= \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \int (1-\tau_l) q_{\tau_l}(\tau_l | \mathcal{X}, \mathcal{Y}) d\tau_l \\
&= \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}}[1-\tau_l].
\end{align*}
$$

Next, we aim to prove that

$$
\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] = 1.
$$

Indeed, recall that we have defined

$$
\begin{align*}
\tau_k | \alpha, \sigma &\stackrel{\text{ind}}{\sim} \text{Beta}(\tau_k | 1 - \sigma, \alpha + k\sigma), \quad k \in \mathbb{N}^*, \\
\pi_k(\tau) &= \tau_k \prod_{l=1}^{k-1} (1 - \tau_l), \quad k \in \mathbb{N}^*, \\
p(\mathcal{Z} | \tau) &\propto \prod_{n=1}^{N} \pi_{z_n}(\tau),
\end{align*}
$$

and to deal with the infinite state space for each $z_j$, we considered a truncation of the
state space to a maximum label $K \equiv K_{\max}$, $K_{\max} \in \mathbb{N}^*$ (Blei and Jordan, 2006). In
practice, this consists of assuming that the variational distributions $q_{z_n}$ for $n \in [N]$,
satisfy $q_{z_n}(k) = 0$ for $k > K$ and that the variational distribution on $\tau$ also factorizes
as $q_{\tau}(\tau) = \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k)$ with the additional condition that $\tau_K = 1$. Based on the
proof from Ghosal and Van der Vaart (2017, Lemma 3.4), it holds that a necessary and
sufficient condition to guarantee that these $\pi_k$'s sum to 1 almost surely, i.e.,

$$
\sum_{k=1}^{\infty} \pi_k(\tau) = \sum_{k=1}^{\infty} \tau_k \prod_{l=1}^{k-1} (1 - \tau_l) = 1,
$$

is that the expectation $\mathbb{E}[\prod_{l=1}^{k-1} (1 - \tau_l)]$ tends to 0 as $k$ tends to $\infty$. In particular, if $\tau_1, \tau_2, \dots$ are i.i.d., e.g., when $\sigma = 0$, it suffices that $p(\tau_1 > 0) > 0$. Then

$$
1 = \mathbb{E}_{q_{\tau}} \left[ \sum_{k=1}^{\infty} \pi_k(\tau) \right] = \sum_{k=1}^{\infty} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)] = \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)].
$$
### E.4. Proof of Theorem 4.2

From the product rule of probability, we see that this conditional distribution can be evaluated from the joint and marginal distributions. Furthermore, by integrating out $\hat{z}$ and $\Theta$, the predictive conditional density is then given by

$$p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}) = \frac{p(\hat{y}, \hat{x} | \mathcal{X}, \mathcal{Y})}{p(\hat{x} | \mathcal{X}, \mathcal{Y})} = \frac{\sum_{\hat{z}} \int p(\hat{y}, \hat{x}, \hat{z}, \Theta | \mathcal{X}, \mathcal{Y}) d\hat{z} d\Theta}{\sum_{\hat{z}} \int p(\hat{x}, \hat{z}, \Theta | \mathcal{X}, \mathcal{Y}) d\hat{z} d\Theta} \equiv \frac{D_1}{D_2}$$

Next, with a similar step as in the proof of Theorem 4.1, we also obtain

$$D_2 \approx \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k).$$

Therefore, we obtain

$$\begin{align*}
p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}) &\approx \frac{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)} \\
&= \sum_{k=1}^{K} \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)} \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \\
&\equiv \sum_{k=1}^{K} g_k(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_D(\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k),
\end{align*}$$

which is a mixture of Gaussian experts since we have

$$g_{Lk}(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)}, \quad k \in [K],$$

belongs to a $K-1$ dimensional probability simplex.

### E.5. Proof of Theorem 4.3

To deal with high-dimensional regression data, namely high-to-low regression, given the inverse conditional density $p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y})$, we want to compute the following forward conditional density

$$p(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) = \frac{p(\hat{x}, \hat{y} | \mathcal{X}, \mathcal{Y})}{p(\hat{y} | \mathcal{X}, \mathcal{Y})} = \frac{p(\hat{x}, \hat{y} | \mathcal{X}, \mathcal{Y})}{\int_{\hat{x}} p(\hat{x}, \hat{y} | \mathcal{X}, \mathcal{Y}) d\hat{x}} = \frac{D_1}{\int_{\hat{x}} D_1(\hat{x}) d\hat{x}} \equiv \frac{D_1}{D_3}.$$
### E.4. Proof of Theorem 4.2

From the product rule of probability, we see that this conditional distribution can be evaluated from the joint and marginal distributions. Furthermore, by integrating out $\hat{z}$ and $\Theta$, the predictive conditional density is then given by

$$p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}) = \frac{p(\hat{y}, \hat{x} | \mathcal{X}, \mathcal{Y})}{p(\hat{x} | \mathcal{X}, \mathcal{Y})} = \frac{\sum_{\hat{z}} \int p(\hat{y}, \hat{x}, \hat{z}, \Theta | \mathcal{X}, \mathcal{Y}) d\hat{z} d\Theta}{\sum_{\hat{z}} \int p(\hat{x}, \hat{z}, \Theta | \mathcal{X}, \mathcal{Y}) d\hat{z} d\Theta} \equiv \frac{D_1}{D_2}$$

Next, with a similar step as in the proof of Theorem 4.1, we also obtain

$$D_2 \approx \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_L (\hat{x} | \hat{c}_k, \hat{\Gamma}_k).$$

Therefore, we obtain

$$\begin{align*}
& p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}) \\
& \approx \frac{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_L (\hat{x} | \hat{c}_k, \hat{\Gamma}_k) \mathcal{N}_D (\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_L (\hat{x} | \hat{c}_k, \hat{\Gamma}_k)} \\
& = \sum_{k=1}^{K} \frac{\mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_L (\hat{x} | \hat{c}_k, \hat{\Gamma}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_L (\hat{x} | \hat{c}_k, \hat{\Gamma}_k)} \mathcal{N}_D (\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k) \\
& \equiv \sum_{k=1}^{K} g_k (\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_D (\hat{y} | \hat{A}_k \hat{x} + \hat{b}_k, \hat{\Sigma}_k),
\end{align*}$$

which is a mixture of Gaussian experts since we have

$$g_{Lk}(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_L(\hat{x} | \hat{c}_k, \hat{\Gamma}_k)}, \quad k \in [K],$$

belongs to a $K-1$ dimensional probability simplex.

### E.5. Proof of Theorem 4.3

To deal with high-dimensional regression data, namely high-to-low regression, given the inverse conditional density $p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y})$, we want to compute the following forward conditional density

$$p(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) = \frac{p(\hat{x}, \hat{y} | \mathcal{X}, \mathcal{Y})}{p(\hat{y} | \mathcal{X}, \mathcal{Y})} = \frac{p(\hat{x}, \hat{y} | \mathcal{X}, \mathcal{Y})}{\int_{\hat{x}} p(\hat{x}, \hat{y} | \mathcal{X}, \mathcal{Y}) d\hat{x}} = \frac{D_1}{\int_{\hat{x}} D_1(\hat{x}) d\hat{x}} \equiv \frac{D_1}{D_3}.$$
Then, we have to compute or approximate $D_3$. Using Theorem 4.1, we obtain

$$D_3 \approx \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \int \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{x}} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) d\hat{\mathbf{x}} \\ = \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{c}}_k + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k \hat{\mathbf{\Gamma}}_k \hat{\mathbf{A}}_k^{\top}).$$

Indeed, we made use of the following results for marginal and conditional Gaussians, see, e.g., Bishop (2006, Eq. (2.115), page 93). Given a marginal Gaussian distribution for x and a conditional Gaussian distribution for y given x in the form

$$p(\mathbf{x}) = \mathcal{N}(\mathbf{x} | \mathbf{\mu}, \mathbf{\Gamma}^{-1}), \\ p(\mathbf{y} | \mathbf{x}) = \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{x} + \mathbf{b}, \mathbf{L}^{-1}),$$

then the marginal distribution of y and the conditional distribution of x given y are
given by

$$p(\mathbf{y}) = \int p(\mathbf{y} | \mathbf{x})p(\mathbf{x})d\mathbf{x} = \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{\mu} + \mathbf{b}, \mathbf{L}^{-1} + \mathbf{A}\mathbf{\Gamma}^{-1}\mathbf{A}^{\top}), \\ p(\mathbf{x} | \mathbf{y}) = \mathcal{N}(\mathbf{x} | \mathbf{\Sigma}[\mathbf{A}^{\top}\mathbf{L}(\mathbf{y} - \mathbf{b}) + \mathbf{\Gamma}\mathbf{\mu}], \mathbf{\Sigma}), \mathbf{\Sigma} = (\mathbf{\Gamma} + \mathbf{A}^{\top}\mathbf{L}\mathbf{A})^{-1}.$$

In our situation, the desired result is obtained via using $\mathbf{y} \equiv \hat{\mathbf{y}}, \mathbf{A} \equiv \hat{\mathbf{A}}_k, \mathbf{b} \equiv \hat{\mathbf{b}}_k, \mathbf{L}^{-1} = \hat{\mathbf{\Sigma}}_k, \mathbf{x} \equiv \hat{\mathbf{x}}, \mathbf{\mu} \equiv \hat{\mathbf{c}}_k, \mathbf{\Gamma}^{-1} \equiv \hat{\mathbf{\Gamma}}_k$.

Finally, we obtain

$$p(\hat{\mathbf{x}} | \hat{\mathbf{y}}, \mathcal{X}, \mathcal{Y}) \\ \approx \sum_{k=1}^{K} \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{x}} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{c}}_k + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k \hat{\mathbf{\Gamma}}_k \hat{\mathbf{A}}_k^{\top})} \\ = \sum_{k=1}^{K} g_{Dk}(\hat{\mathbf{y}} | \hat{\mathbf{\Theta}}^{*}, \hat{\mathbf{\phi}}^{*}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{A}}_k^{*} \hat{\mathbf{y}} + \hat{\mathbf{b}}_k^{*}, \hat{\mathbf{\Sigma}}_k^{*}),$$

where

$$g_{Dk}(\hat{\mathbf{y}} | \hat{\mathbf{\Theta}}^{*}, \hat{\mathbf{\phi}}^{*}, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{c}}_k^{*}, \hat{\mathbf{\Gamma}}_k^{*})}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{c}}_k^{*}, \hat{\mathbf{\Gamma}}_k^{*})}.$$

Here, we used the fact that $p(\hat{\mathbf{y}}, \hat{\mathbf{x}} | \hat{z} = k) = p(\hat{\mathbf{x}} | \hat{\mathbf{y}}, \hat{z} = k)p(\hat{\mathbf{y}} | \hat{z} = k)$, namely,

$$\mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{x}} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{\Sigma}}_k^* [\hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1} (\hat{\mathbf{y}} - \hat{\mathbf{b}}_k) + \hat{\mathbf{\Gamma}}_k^{-1} \hat{\mathbf{c}}_k], \hat{\mathbf{\Sigma}}_k^*) \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{c}}_k + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k \hat{\mathbf{\Gamma}}_k \hat{\mathbf{A}}_k^\top) \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{A}}_k^* \hat{\mathbf{y}} + \hat{\mathbf{b}}_k^*, \hat{\mathbf{\Sigma}}_k^*) \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{c}}_k^*, \hat{\mathbf{\Gamma}}_k^*),$$
Then, we have to compute or approximate $D_3$. Using Theorem 4.1, we obtain

$$D_3 \approx \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \int \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{x}} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) d\hat{\mathbf{x}} \\ = \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{c}}_k + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k \hat{\mathbf{\Gamma}}_k \hat{\mathbf{A}}_k^{\top}).$$

Indeed, we made use of the following results for marginal and conditional Gaussians, see, e.g., Bishop (2006, Eq. (2.115), page 93). Given a marginal Gaussian distribution for x and a conditional Gaussian distribution for y given x in the form

$$p(\mathbf{x}) = \mathcal{N}(\mathbf{x} | \mathbf{\mu}, \mathbf{\Gamma}^{-1}), \\ p(\mathbf{y} | \mathbf{x}) = \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{x} + \mathbf{b}, \mathbf{L}^{-1}),$$

then the marginal distribution of y and the conditional distribution of x given y are
given by

$$p(\mathbf{y}) = \int p(\mathbf{y} | \mathbf{x})p(\mathbf{x})d\mathbf{x} = \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{\mu} + \mathbf{b}, \mathbf{L}^{-1} + \mathbf{A}\mathbf{\Gamma}^{-1}\mathbf{A}^{\top}), \\ p(\mathbf{x} | \mathbf{y}) = \mathcal{N}(\mathbf{x} | \mathbf{\Sigma}[\mathbf{A}^{\top}\mathbf{L}(\mathbf{y} - \mathbf{b}) + \mathbf{\Gamma}\mathbf{\mu}], \mathbf{\Sigma}), \mathbf{\Sigma} = (\mathbf{\Gamma} + \mathbf{A}^{\top}\mathbf{L}\mathbf{A})^{-1}.$$

In our situation, the desired result is obtained via using $\mathbf{y} \equiv \hat{\mathbf{y}}, \mathbf{A} \equiv \hat{\mathbf{A}}_k, \mathbf{b} \equiv \hat{\mathbf{b}}_k, \mathbf{L}^{-1} = \hat{\mathbf{\Sigma}}_k, \mathbf{x} \equiv \hat{\mathbf{x}}, \mathbf{\mu} \equiv \hat{\mathbf{c}}_k, \mathbf{\Gamma}^{-1} \equiv \hat{\mathbf{\Gamma}}_k$.

Finally, we obtain

$$p(\hat{\mathbf{x}} | \hat{\mathbf{y}}, \mathcal{X}, \mathcal{Y}) \\ \approx \sum_{k=1}^{K} \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{x}} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k)}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{c}}_k + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k \hat{\mathbf{\Gamma}}_k \hat{\mathbf{A}}_k^{\top})} \\ = \sum_{k=1}^{K} g_{Dk}(\hat{\mathbf{y}} | \hat{\mathbf{\Theta}}^{*}, \hat{\mathbf{\phi}}^{*}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{A}}_k^{*} \hat{\mathbf{y}} + \hat{\mathbf{b}}_k^{*}, \hat{\mathbf{\Sigma}}_k^{*}),$$

where

$$g_{Dk}(\hat{\mathbf{y}} | \hat{\mathbf{\Theta}}^{*}, \hat{\mathbf{\phi}}^{*}, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{c}}_k^{*}, \hat{\mathbf{\Gamma}}_k^{*})}{\sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{c}}_k^{*}, \hat{\mathbf{\Gamma}}_k^{*})}.$$

Here, we used the fact that $p(\hat{\mathbf{y}}, \hat{\mathbf{x}} | \hat{\mathbf{z}} = k) = p(\hat{\mathbf{x}} | \hat{\mathbf{y}}, \hat{\mathbf{z}} = k)p(\hat{\mathbf{y}} | \hat{\mathbf{z}} = k)$, namely,

$$\mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{x}} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{c}}_k, \hat{\mathbf{\Gamma}}_k) \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{\Sigma}}_k^* [\hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1} (\hat{\mathbf{y}} - \hat{\mathbf{b}}_k) + \hat{\mathbf{\Gamma}}_k^{-1} \hat{\mathbf{c}}_k], \hat{\mathbf{\Sigma}}_k^*) \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{A}}_k \hat{\mathbf{c}}_k + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k \hat{\mathbf{\Gamma}}_k \hat{\mathbf{A}}_k^\top) \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{A}}_k^* \hat{\mathbf{y}} + \hat{\mathbf{b}}_k^*, \hat{\mathbf{\Sigma}}_k^*) \mathcal{N}_D(\hat{\mathbf{y}} | \hat{\mathbf{c}}_k^*, \hat{\mathbf{\Gamma}}_k^*),$$
with

$$ \hat{\Sigma}_k^* = (\hat{\Gamma}_k^{-1} + \hat{A}_k^\top \hat{\Sigma}_k^{-1} \hat{A}_k)^{-1}, $$

$$ \hat{A}_k^* = \hat{\Sigma}_k^* \hat{A}_k^\top \hat{\Sigma}_k^{-1}, $$

$$ \hat{b}_k^* = \hat{\Sigma}_k^* [\hat{\Gamma}_k^{-1} \hat{c}_k - \hat{A}_k^\top \hat{\Sigma}_k^{-1} \hat{b}_k], $$

$$ \hat{c}_k^* = \hat{A}_k \hat{c}_k + \hat{b}_k, $$

$$ \hat{\Gamma}_k^* = \hat{\Sigma}_k + \hat{A}_k \hat{\Gamma}_k \hat{A}_k^\top. $$

When required, it is straightforward to approximate the expectation and covariance matrix of $\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}$ as follows:

$$ \mathbb{E}[\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}] \approx \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} $$

$$ = \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} $$

$$ = \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) (\hat{A}_k^* \hat{y} + \hat{b}_k^*), $$

$$ \begin{aligned} \text{var}[\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}] &= \mathbb{E}[(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top] - \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top \\ &\approx \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} \\ &\quad - \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top \\ &\approx \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} \\ &\quad - \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top \\ &\approx \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) [\hat{\Sigma}_k^* + (\hat{A}_k^* \hat{y} + \hat{b}_k^*) (\hat{A}_k^* \hat{y} + \hat{b}_k^*)^\top] \\ &\quad - \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E}(\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^\top, \end{aligned} $$

where we used the following definitions

$$ \text{cov}(X, Y) = \mathbb{E}(XY^\top) - \mathbb{E}(X)\mathbb{E}(Y)^\top, \text{var}(X) = \text{cov}(X, X). $$

## Appendix F. BNP-GLLIM2: a model with an hyperprior on the gating parameters

### F.1. VBEM for BNP-GLLIM2

A more general BNP-GLLIM model, referred to as BNP-GLLIM2 can be considered by specifying a prior on the gating parameters ($c_k, \Gamma_k$) as a normal-inverse-Wishart (NIW)
with

$$ \hat{\Sigma}_k^* = (\hat{\Gamma}_k^{-1} + \hat{A}_k^\top \hat{\Sigma}_k^{-1} \hat{A}_k)^{-1}, $$

$$ \hat{A}_k^* = \hat{\Sigma}_k^* \hat{A}_k^\top \hat{\Sigma}_k^{-1}, $$

$$ \hat{b}_k^* = \hat{\Sigma}_k^* [\hat{\Gamma}_k^{-1} \hat{c}_k - \hat{A}_k^\top \hat{\Sigma}_k^{-1} \hat{b}_k], $$

$$ \hat{c}_k^* = \hat{A}_k \hat{c}_k + \hat{b}_k, $$

$$ \hat{\Gamma}_k^* = \hat{\Sigma}_k + \hat{A}_k \hat{\Gamma}_k \hat{A}_k^\top. $$

When required, it is straightforward to approximate the expectation and covariance matrix of $\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}$ as follows:

$$ \mathbb{E}[\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}] \approx \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} $$

$$ = \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} $$

$$ = \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) (\hat{A}_k^* \hat{y} + \hat{b}_k^*), $$

$$ \mathrm{var}[\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}] = \mathbb{E} \left[ (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top} \right] - \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top} $$

$$ \approx \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top} \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} $$

$$ - \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top} $$

$$ \approx \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \int (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top} \mathcal{N}_L (\hat{x} | \hat{A}_k^* \hat{y} + \hat{b}_k^*, \hat{\Sigma}_k^*) d\hat{x} $$

$$ - \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top} $$

$$ \approx \sum_{k=1}^{K} g_{Dk} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) [\hat{\Sigma}_k^* + (\hat{A}_k^* \hat{y} + \hat{b}_k^*) (\hat{A}_k^* \hat{y} + \hat{b}_k^*)^{\top}] $$

$$ - \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y}) \mathbb{E} (\hat{x} | \hat{y}, \mathcal{X}, \mathcal{Y})^{\top}, $$

where we used the following definitions

$$ \mathrm{cov}(\mathbf{X}, \mathbf{Y}) = \mathbb{E}(\mathbf{X}\mathbf{Y}^{\top}) - \mathbb{E}(\mathbf{X})\mathbb{E}(\mathbf{Y}^{\top}), \mathrm{var}(\mathbf{X}) = \mathrm{cov}(\mathbf{X}, \mathbf{X}). $$

# Appendix F. BNP-GLLIM2: a model with an hyperprior on the gating parameters

## F.1. VBEM for BNP-GLLIM2

A more general BNP-GLLIM model, referred to as BNP-GLLIM2 can be considered by specifying a prior on the gating parameters ($c_k, \Gamma_k$) as a normal-inverse-Wishart (NIW)
<figure><img src="image_14.png" /><figcaption>**Figure F1.** Graphical representation of BNP-GLLIM2: the plate denotes N i.i.d. observations, white-filled circles correspond to unobserved (latent) variables and random or unknown parameters represented in red, while grey-filled circles correspond to observed variables represented in green. Hyperparameters are represented in blue.</figcaption></figure>

distribution parameterized by $\rho_k = (\mathbf{m}_k, \lambda_k, \mathbf{\Psi}_k, \nu_k)$ with a PDF

$$p(\mathbf{c}_k, \Gamma_k | \rho_k) \equiv \mathcal{N} \mathcal{I} \mathcal{W}(\mathbf{c}_k, \Gamma_k | \rho_k) = \mathcal{N}(\mathbf{c}_k | \mathbf{m}_k, \lambda_k^{-1} \Gamma_k) \mathcal{I} \mathcal{W}(\Gamma_k | \mathbf{\Psi}_k, \nu_k).$$

The assumptions on the other parameters are not changed, so that hyperparameters and parameters are now as follows:

$$\phi = (s_1, s_2, a, (\rho_k, \mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)_{k \in \mathbb{N}^*}), \text{ while } \Theta = (\tau, \alpha, \sigma, \theta^*), \theta^* = (\theta_k^*)_{k \in \mathbb{N}^*} \equiv (\mathbf{c}_k, \Gamma_k)_{k \in \mathbb{N}^*}.$$

BNP-GLLIM2 can be represented graphically as in Figure F1. The joint distribution of the observed data $\mathcal{X}$, $\mathcal{Y}$ and all latent variables can be expressed hierarchically as

$$\begin{aligned}
p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \phi) &= \prod_{n=1}^{N} p(y_n | \mathbf{x}_n, z_n, \Theta; \phi) p(\mathbf{x}_n | z_n, \Theta; \phi) p(\mathcal{Z} | \Theta; \phi) p(\Theta; \phi) \\
&= \prod_{n=1}^{N} p(y_n | \mathbf{x}_n, z_n; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\mathbf{x}_n | z_n, \mathbf{c}, \mathbf{\Gamma}) p(\mathcal{Z} | \tau) \\
&\qquad \prod_{k \in \mathbb{N}^*} p(\tau_k | \alpha, \sigma) p(\alpha, \sigma | s_1, s_2, a) \prod_{k \in \mathbb{N}^*} p(\mathbf{c}_k, \Gamma_k; \rho_k).
\end{aligned}
\quad (\text{F1})$$

Following the same idea as in Section 3, we only consider the truncated variational posterior of parameters $\Theta$ as follows

$$q_{\Theta}(\Theta) = q_{\alpha, \sigma}(\alpha, \sigma) \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k) \prod_{k=1}^{K} q_{\theta_k^*}(\theta_k^*). \quad (\text{F2})$$

These forms of $q_{\mathcal{Z}}$ and $q_{\Theta}$ lead to our four VB-E steps and three VB-M steps, summarized below with details in Appendix C. Set the initial value of $\phi$ to $\phi^{(0)}$. Then, repeat iteratively the following steps. The iteration index is omitted in the update formulas for simplicity.

### VB-E steps

Note that the VB-E-$\tau$, VB-E-$(\alpha, \sigma)$ steps are the same as in Section 3. We only highlight the modified steps as follows.

We first consider the derivation of the update equation for the factor $q_{\mathcal{Z}}(\mathcal{Z})$.
<figure><img src="image_13.png" /><figcaption>**Figure F1.** Graphical representation of BNP-GLLIM2: the plate denotes *N* i.i.d. observations, white-filled circles correspond to unobserved (latent) variables and random or unknown parameters represented in red, while grey-filled circles correspond to observed variables represented in green. Hyperparameters are represented in blue.</figcaption></figure>

distribution parameterized by $\rho_k = (\mathbf{m}_k, \lambda_k, \mathbf{\Psi}_k, \nu_k)$ with a PDF

$$p(\mathbf{c}_k, \Gamma_k | \rho_k) \equiv \mathcal{N} \mathcal{I} \mathcal{W} (\mathbf{c}_k, \Gamma_k | \rho_k) = \mathcal{N} (\mathbf{c}_k | \mathbf{m}_k, \lambda_k^{-1} \Gamma_k) \mathcal{I} \mathcal{W} (\Gamma_k | \mathbf{\Psi}_k, \nu_k).$$

The assumptions on the other parameters are not changed, so that hyperparameters and parameters are now as follows:

$$\phi = (s_1, s_2, a, (\rho_k, \mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)_{k \in \mathbb{N}^*}), \text{ while } \Theta = (\tau, \alpha, \sigma, \theta^*), \theta^* = (\theta_k^*)_{k \in \mathbb{N}^*} \equiv (\mathbf{c}_k, \Gamma_k)_{k \in \mathbb{N}^*}.$$

BNP-GLLIM2 can be represented graphically as in Figure F1. The joint distribution of the observed data $\mathcal{X}$, $\mathcal{Y}$ and all latent variables can be expressed hierarchically as

$$\begin{aligned}
p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \phi) &= \prod_{n=1}^{N} p(y_n | \mathbf{x}_n, z_n, \Theta; \phi) p(\mathbf{x}_n | z_n, \Theta; \phi) p(\mathcal{Z} | \Theta; \phi) p(\Theta; \phi) \\
&= \prod_{n=1}^{N} p(y_n | \mathbf{x}_n, z_n; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\mathbf{x}_n | z_n, \mathbf{c}, \mathbf{\Gamma}) p(\mathcal{Z} | \tau) \\
&\qquad \prod_{k \in \mathbb{N}^*} p(\tau_k | \alpha, \sigma) p(\alpha, \sigma | s_1, s_2, a) \prod_{k \in \mathbb{N}^*} p(\mathbf{c}_k, \Gamma_k; \rho_k).
\end{aligned} \quad (\text{F1})$$

Following the same idea as in Section 3, we only consider the truncated variational posterior of parameters $\Theta$ as follows

$$q_{\Theta}(\Theta) = q_{\alpha, \sigma}(\alpha, \sigma) \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k) \prod_{k=1}^{K} q_{\theta_k^*}(\theta_k^*). \quad (\text{F2})$$

These forms of $q_{\mathcal{Z}}$ and $q_{\Theta}$ lead to our four VB-E steps and three VB-M steps, summarized below with details in Appendix C. Set the initial value of $\phi$ to $\phi^{(0)}$. Then, repeat iteratively the following steps. The iteration index is omitted in the update formulas for simplicity.

### VB-E steps

Note that the VB-E-$\tau$, VB-E-$(\alpha, \sigma)$ steps are the same as in Section 3. We only highlight the modified steps as follows.

We first consider the derivation of the update equation for the factor $q_{\mathcal{Z}}(\mathcal{Z})$.
### F.1.1. VB-E-Z step

By using the mean-field approximation (9) and the truncation, see Appendix C.3 for more details, for all $n \in [N]$ and for $k \in [K]$, this step consists of computing

$$ q_{z_n}(k) = \frac{\rho_{nk}}{\sum_{l=1}^{K} \rho_{nl}}. \qquad (F3) $$

Here, given $\mathcal{N}_n$ represents the neighbors of $n$, we define $\log \rho_{nk}$ by

$$ \begin{aligned} & -\frac{1}{2} \left\{ \log |\hat{\Sigma}_k| + (y_n - \hat{\mathbf{A}}_k x_n - \hat{\mathbf{b}}_k)^{\top} \hat{\Sigma}_k^{-1} (y_n - \hat{\mathbf{A}}_k x_n - \hat{\mathbf{b}}_k) \right. \\ & \qquad \left. + \log \left| \frac{\hat{\Psi}_k}{2} \right| - \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + (1-l)}{2} \right) + \hat{\nu}_k (x_n - \hat{\mathbf{m}}_k)^{\top} \hat{\Psi}_k^{-1} (x_n - \hat{\mathbf{m}}_k) + \frac{L}{\hat{\lambda}_k} \right\} \\ & \qquad + \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})]. \end{aligned} \qquad (F4) $$

Note that in the above formula, symbols $(\hat{\mathbf{m}}_k, \hat{\lambda}_k, \hat{\Psi}_k, \hat{\nu}_k)$ and $(\hat{\mathbf{A}}_k, \hat{\mathbf{b}}_k, \hat{\Sigma}_k)$ are the hyperparameters. Specifically defined in the following Appendix F.1.2 and Section 3.2.

**Proof of (F4).** With respect to the VBEM for BNP-GLLIM2 model from Appendix F.1.1, it is almost similar to the previous step in Appendix C.3, except that we have to take into account the randomness of c and $\Gamma$. Namely, we have

$$ \begin{aligned} q_{z_n}(z_n) & \propto \exp \mathbb{E}_{q_{\Theta}} \left[ \log \left( p \left( y_n \mid \mathbf{x}_n, z_n; \hat{\mathbf{A}}_{z_n}, \hat{\mathbf{b}}_{z_n}, \hat{\mathbf{\Sigma}}_{z_n} \right) p \left( \mathbf{x}_n \mid z_n, \mathbf{c}_{z_n}, \Gamma_{z_n} \right) p \left( \mathbf{z} \mid \boldsymbol{\tau} \right) \right) \right] \\ &= \exp \left\{ \log p \left( y_n \mid \mathbf{x}_n, z_n; \hat{\mathbf{A}}_{z_n}, \hat{\mathbf{b}}_{z_n}, \hat{\mathbf{\Sigma}}_{z_n} \right) + \mathbb{E}_{q_{\theta^*}} \left[ \log p \left( \mathbf{x}_n \mid z_n, \mathbf{c}_{z_n}, \Gamma_{z_n} \right) \right] + \mathbb{E}_{q_{\tau}} \left[ \log \pi_{z_n}(\boldsymbol{\tau}) \right] \right\}. \end{aligned} \qquad (F5) $$

Here, for $z_n = k$, it holds that

$$ \begin{aligned} \mathbb{E}_{q_{\theta^*}} \left[ \log p \left( \mathbf{x}_n \mid z_n, \hat{\mathbf{c}}_{z_n}, \hat{\Gamma}_{z_n} \right) \right] &= \mathbb{E}_{q_{\theta^*}} \left[ \log \mathcal{N}_L \left( \mathbf{x}_n \mid \hat{\mathbf{c}}_k, \hat{\Gamma}_k \right) \right] \\ &= -\frac{L}{2} \log(2\pi) - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}} \left[ \log |\hat{\Gamma}_k| \right] - \frac{1}{2} \mathbb{E}_{q_{\theta^*}} \left[ (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top} \hat{\Gamma}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{c}}_k) \right], \end{aligned} $$

where we used the fact that

$$ \begin{aligned} \mathbb{E}_{q_{\Gamma_k}} \left[ \log |\hat{\Gamma}_k| \right] &= \log \left| \frac{\hat{\Psi}_k}{2} \right| - \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + (1-l)}{2} \right), \\ \mathbb{E}_{q_{\theta^*}} \left[ (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top} \hat{\Gamma}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{c}}_k) \right] &= \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k)^{\top} \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) + \frac{L}{\hat{\lambda}_k}. \end{aligned} $$

Plugging in all of the above expression back into (F5) yields the desired results in (F4). □
### F.1.1. VB-E-Z step

By using the mean-field approximation (9) and the truncation, see Appendix C.3 for more details, for all $n \in [N]$ and for $k \in [K]$, this step consists of computing

$$q_{z_n}(k) = \frac{\rho_{nk}}{\sum_{l=1}^{K} \rho_{nl}}. \qquad (F3)$$

Here, given $\mathcal{N}_n$ represents the neighbors of $n$, we define $\log \rho_{nk}$ by

$$-\frac{1}{2}\left\{\log|\hat{\Sigma}_k| + (y_n - \hat{\mathbf{A}}_k x_n - \hat{\mathbf{b}}_k)^{\top}\hat{\Sigma}_k^{-1}(y_n - \hat{\mathbf{A}}_k x_n - \hat{\mathbf{b}}_k)\right. \\ + \log\left|\frac{\hat{\mathbf{\Psi}}_k}{2}\right| - \sum_{l=1}^{L} \psi\left(\frac{\hat{\nu}_k + (1-l)}{2}\right) + \hat{\nu}_k(x_n - \hat{\mathbf{m}}_k)^{\top}\hat{\mathbf{\Psi}}_k^{-1}(x_n - \hat{\mathbf{m}}_k) + \frac{L}{\hat{\lambda}_k}\right\} \quad (F4) \\ + \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1}[\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})].$$

Note that in the above formula, symbols $(\hat{\mathbf{m}}_k, \hat{\lambda}_k, \hat{\mathbf{\Psi}}_k, \hat{\nu}_k)$ and $(\hat{\mathbf{A}}_k, \hat{\mathbf{b}}_k, \hat{\Sigma}_k)$ are the hyperparameters. Specifically defined in the following Appendix F.1.2 and Section 3.2.

**Proof of (F4).** With respect to the VBEM for BNP-GLLiM2 model from Appendix F.1.1, it is almost similar to the previous step in Appendix C.3, except that we have to take into account the randomness of c and $\Gamma$. Namely, we have

$$\begin{aligned} q_{z_n}(z_n) & \propto \exp \mathbb{E}_{q_{\Theta}} \left[ \log \left( p \left( y_n \mid \mathbf{x}_n, z_n; \hat{\mathbf{A}}_{z_n}, \hat{\mathbf{b}}_{z_n}, \hat{\mathbf{\Sigma}}_{z_n} \right) p \left( \mathbf{x}_n \mid z_n, \mathbf{c}_{z_n}, \Gamma_{z_n} \right) p \left( \mathbf{z} \mid \boldsymbol{\tau} \right) \right) \right] \\ &= \exp \left\{ \log p \left( y_n \mid \mathbf{x}_n, z_n; \hat{\mathbf{A}}_{z_n}, \hat{\mathbf{b}}_{z_n}, \hat{\mathbf{\Sigma}}_{z_n} \right) + \mathbb{E}_{q_{\theta_{z_n}^*}} \left[ \log p \left( \mathbf{x}_n \mid z_n, \mathbf{c}_{z_n}, \Gamma_{z_n} \right) \right] + \mathbb{E}_{q_{\tau}} \left[ \log \pi_{z_n}(\boldsymbol{\tau}) \right] \right\}. \end{aligned} \qquad (F5)$$

Here, for $z_n = k$, it holds that

$$\begin{aligned} \mathbb{E}_{q_{\theta_{z_n}^*}} \left[ \log p \left( \mathbf{x}_n \mid z_n, \hat{\mathbf{c}}_{z_n}, \hat{\Gamma}_{z_n} \right) \right] &= \mathbb{E}_{q_{\theta_{z_n}^*}} \left[ \log \mathcal{N}_L \left( \mathbf{x}_n \mid \hat{\mathbf{c}}_k, \hat{\Gamma}_k \right) \right] \\ &= -\frac{L}{2} \log(2\pi) - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}} \left[ \log |\hat{\Gamma}_k| \right] - \frac{1}{2} \mathbb{E}_{q_{\theta_{z_n}^*}} \left[ (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top} \hat{\Gamma}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{c}}_k) \right], \end{aligned}$$

where we used the fact that

$$\mathbb{E}_{q_{\Gamma_k}} \left[ \log |\hat{\Gamma}_k| \right] = \log \left| \frac{\hat{\mathbf{\Psi}}_k}{2} \right| - \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + (1-l)}{2} \right),$$

$$\mathbb{E}_{q_{\theta_{z_n}^*}} \left[ (\mathbf{x}_n - \hat{\mathbf{c}}_k)^{\top} \hat{\Gamma}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{c}}_k) \right] = \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k)^{\top} \hat{\mathbf{\Psi}}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) + \frac{L}{\hat{\lambda}_k}.$$

Plugging in all of the above expression back into (F5) yields the desired results in (F4). □
### F.1.2. VB-E-$\theta^*$ step

This step is divided into K parts where the computation is similar to that in standard Bayesian GMM with a choice of conjugate prior. Hence, for each $k \le K$, the variational posterior is a Normal-inverse-Wishart density defined as

$$ q_{\theta_k^*}(c_k, \Gamma_k) = \mathcal{NIW}(c_k, \Gamma_k | \hat{\mathbf{m}}_k, \hat{\lambda}_k, \hat{\Psi}_k, \hat{\nu}_k). \quad (F6) $$

Here, the hyperparameters are updated as follows (see, e.g., Bishop (2006, Section 10.2.1)):

$$ \begin{align*} \hat{\lambda}_k &= \lambda_k + N_k, \quad \hat{\nu}_k = \nu_k + N_k, \quad N_k = \sum_{n=1}^{N} q_{z_n}(k) \\ \hat{\Psi}_k &= \Psi_k + N_k \mathbf{S}_k + \frac{\lambda_k N_k}{\lambda_k + N_k} (\mathbf{m}_k - \bar{\mathbf{c}}_k)(\mathbf{m}_k - \bar{\mathbf{c}}_k)^{\top}, \\ \hat{\mathbf{m}}_k &= \frac{\lambda_k \mathbf{m}_k + N_k \bar{\mathbf{c}}_k}{\lambda_k + N_k} = \frac{\lambda_k \mathbf{m}_k + N_k \bar{\mathbf{c}}_k}{\hat{\lambda}_k}, \quad (F7) \\ \bar{\mathbf{c}}_k &= \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n, \\ \mathbf{S}_k &= \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{x}_n - \bar{\mathbf{c}}_k)(\mathbf{x}_n - \bar{\mathbf{c}}_k)^{\top}. \end{align*} $$

### VB-M steps

The maximisation step consists of updating the hyperparameters $\phi = (s_1, s_2, a, (\rho_k, \mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)_{k \in [K]}$), where $\rho_k = (\mathbf{m}_k, \lambda_k, \Psi_k, \nu_k), k \in [K]$, by maximizing the free energy, if they are not set heuristically:

$$ \phi^{(r)} = \underset{\phi}{\operatorname{argmax}} \mathbb{E}_{q_{\theta}^{(r)} q_{\tau}^{(r)} q_{\alpha, \sigma}^{(r)} q_{\theta^*}^{(r)}} [\log p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \tau, \alpha, \sigma, \theta^*; \phi)]. \quad (F8) $$

The VB-M-step can therefore be divided into four independent sub-steps as listed below. From the conditional independence of $(s_1, s_2, a, \rho)$ and $(\mathcal{Y}, \mathcal{X}, \mathcal{Z})$ given $(\tau, \alpha, \sigma, \theta^*)$, the solutions for the VB-M-$(s_1, s_2)$ (in the DP case) and VB-M-$\rho$ steps are straightforward. Only the M-$(s_1, s_2, a)$ step (in the PYP case) and $(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)_{k \in [K]}$ are more involved. Note that the VB-M-$(s_1, s_2, a)$, VB-M-$(\mathbf{A}, \mathbf{b}, \mathbf{\Sigma})$ steps are the same as in Section 3. We only highlight the modified step below.

### F.1.3. VB-M-$\rho$ step

This step divides into K sub-steps that involve again cross-entropies,

$$ \rho_k^{(r)} = \underset{\rho}{\operatorname{argmax}} \mathbb{E}_{q_{\theta_k^*}^{(r)}} [\log p(\mathbf{c}_k, \Gamma_k; \rho_k)] = \hat{\rho}_k^{(r)}, $$

where $\hat{\rho}_k^{(r)} = (\hat{\lambda}_k^{(r)}, \hat{\nu}_k^{(r)}, \hat{\Psi}_k^{(r)}, \hat{\mathbf{m}}_k^{(r)})$ is given in (F7).
### F.1.2. VB-E-$\theta^*$ step

This step is divided into K parts where the computation is similar to that in standard Bayesian GMM with a choice of conjugate prior. Hence, for each $k \le K$, the variational posterior is a Normal-inverse-Wishart density defined as

$$ q_{\theta_k^*}(c_k, \Gamma_k) = \mathcal{NIW}(c_k, \Gamma_k | \hat{\mathbf{m}}_k, \hat{\lambda}_k, \hat{\Psi}_k, \hat{\nu}_k). \quad (F6) $$

Here, the hyperparameters are updated as follows (see, e.g., Bishop (2006, Section 10.2.1)):

$$
\begin{align*}
\hat{\lambda}_k &= \lambda_k + N_k, \quad \hat{\nu}_k = \nu_k + N_k, \quad N_k = \sum_{n=1}^{N} q_{z_n}(k) \\
\hat{\Psi}_k &= \Psi_k + N_k \mathbf{S}_k + \frac{\lambda_k N_k}{\lambda_k + N_k} (\mathbf{m}_k - \bar{\mathbf{c}}_k)(\mathbf{m}_k - \bar{\mathbf{c}}_k)^{\top}, \\
\hat{\mathbf{m}}_k &= \frac{\lambda_k \mathbf{m}_k + N_k \bar{\mathbf{c}}_k}{\lambda_k + N_k} = \frac{\lambda_k \mathbf{m}_k + N_k \bar{\mathbf{c}}_k}{\hat{\lambda}_k}, \tag{F7} \\
\bar{\mathbf{c}}_k &= \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) \mathbf{x}_n, \\
\mathbf{S}_k &= \frac{1}{N_k} \sum_{n=1}^{N} q_{z_n}(k) (\mathbf{x}_n - \bar{\mathbf{c}}_k)(\mathbf{x}_n - \bar{\mathbf{c}}_k)^{\top}.
\end{align*}
$$

### VB-M steps

The maximisation step consists of updating the hyperparameters $\phi = (s_1, s_2, a, (\rho_k, \mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)_{k \in [K]}$), where $\rho_k = (\mathbf{m}_k, \lambda_k, \Psi_k, \nu_k), k \in [K]$, by maximizing the free energy, if they are not set heuristically:

$$ \phi^{(r)} = \underset{\phi}{\operatorname{argmax}} \mathbb{E}_{q_{\theta}^{(r)} q_{\tau}^{(r)} q_{\alpha, \sigma}^{(r)} q_{\theta^*}^{(r)}} [\log p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \tau, \alpha, \sigma, \theta^*; \phi)]. \quad (F8) $$

The VB-M-step can therefore be divided into four independent sub-steps as listed below. From the conditional independence of $(s_1, s_2, a, \rho)$ and $(\mathcal{Y}, \mathcal{X}, \mathcal{Z})$ given $(\tau, \alpha, \sigma, \theta^*)$, the solutions for the VB-M-$(s_1, s_2)$ (in the DP case) and VB-M-$\rho$ steps are straightforward. Only the M-$(s_1, s_2, a)$ step (in the PYP case) and $(\mathbf{A}_k, \mathbf{b}_k, \mathbf{\Sigma}_k)_{k \in [K]}$ are more involved.

Note that the VB-M-$(s_1, s_2, a)$, VB-M-$(\mathbf{A}, \mathbf{b}, \mathbf{\Sigma})$ steps are the same as in Section 3. We only highlight the modified step below.

### F.1.3. VB-M-$\rho$ step

This step divides into K sub-steps that involve again cross-entropies,

$$ \rho_k^{(r)} = \underset{\rho}{\operatorname{argmax}} \mathbb{E}_{q_{\theta_k^*}^{(r)}} [\log p(c_k, \Gamma_k; \rho_k)] = \hat{\rho}_k^{(r)}, $$

where $\hat{\rho}_k^{(r)} = (\hat{\lambda}_k^{(r)}, \hat{\nu}_k^{(r)}, \hat{\Psi}_k^{(r)}, \hat{\mathbf{m}}_k^{(r)})$ is given in (F7).
### F.1.4. ELBO for BNP-GLLIM2

**Proposition F.1.** When $\sigma = 0$, the ELBO in BNP-GLLIM2 is determined analytically as follows:

$$ \mathcal{F}(q_Z, q_\Theta, \hat{\phi}) = \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] \\ + \mathbb{E}[\log p(\Theta; \hat{\phi})] - \mathbb{E}[\log q(\mathcal{Z})] - \mathbb{E}[\log q(\Theta)]. \quad (F9) $$

Here, we have the following update formulas:

$$ \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_D \left( y_n \mid \hat{\mathbf{A}}_k x_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k \right), \quad (F10) $$

$$ \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] = \frac{1}{2} \sum_{k=1}^{K} N_k \left[ \log \tilde{\Gamma}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} - \hat{\nu}_k \operatorname{Tr}(\mathbf{S}_k \hat{\Psi}_k^{-1}) - \hat{\nu}_k (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \right], \quad (F11) $$

$$ \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] = \sum_{k=1}^{K} N_k \left[ \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})] \right], \quad (F12) $$

$$ \mathbb{E}[\log p(\Theta; \hat{\phi})] = \sum_{k=1}^{K-1} \mathbb{E}[\log p(\tau_k | \alpha)] + \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] + \sum_{k=1}^{K} \mathbb{E}[\log p(c_k, \Gamma_k; \hat{\rho}_k)], \quad (F13) $$

$$ \mathbb{E}[\log p(\tau_k | \alpha)] = \frac{\hat{s}_1 - \hat{s}_2}{\hat{s}_2} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})] + \psi(\hat{s}_1) - \log(\hat{s}_2), $$

$$ \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$

$$ \mathbb{E}[\log p(c_k, \Gamma_k; \hat{\rho}_k)] = \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{L}{2} - \frac{L}{2} \hat{\nu}_k + \log B \left( \hat{\Psi}_k^{-1}, \hat{\nu}_k \right) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k, $$

$$ \log \tilde{\Gamma}_k = \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + 1 - l}{2} \right) + L \log 2 + \log |\hat{\Psi}_k|, $$

$$ \mathbb{E}[\log q(\mathcal{Z})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log q_{z_n}(k), \quad (F14) $$

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha,0}(\alpha)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)] + \sum_{k=1}^{K} \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)], \quad (F15) $$

$$ \mathbb{E}[\log q_{\alpha,0}(\alpha)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$

$$ \mathbb{E}[\log q_{\tau_k}(\tau_k)] = \sum_{l=1}^{2} (\hat{\gamma}_{k,l} - 1) \{\psi(\hat{\gamma}_{k,l}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})\} + \log \frac{\Gamma(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})}{\Gamma(\hat{\gamma}_{k,1}) \Gamma(\hat{\gamma}_{k,2})}, $$

$$ \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)] = \frac{L}{2} \log \frac{\hat{\lambda}_k}{2\pi} - \frac{L}{2} + \log B \left( \hat{\Psi}_k^{-1}, \hat{\nu}_k \right) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k - \frac{\hat{\nu}_k L}{2}. $$
### F.1.4. ELBO for BNP-GLLIM2

**Proposition F.1.** When $\sigma = 0$, the ELBO in BNP-GLLIM2 is determined analytically as follows:

$$ \mathcal{F}(q_Z, q_\Theta, \hat{\phi}) = \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] + \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] \\ + \mathbb{E}[\log p(\Theta; \hat{\phi})] - \mathbb{E}[\log q(\mathcal{Z})] - \mathbb{E}[\log q(\Theta)]. \qquad (\text{F9}) $$

Here, we have the following update formulas:

$$ \mathbb{E}[\log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log \mathcal{N}_D (y_n | \hat{\mathbf{A}}_k x_n + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k), \qquad (\text{F10}) $$

$$ \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] = \frac{1}{2} \sum_{k=1}^{K} N_k \left[ \log \tilde{\Gamma}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} - \hat{\nu}_k \operatorname{Tr}(\mathbf{S}_k \hat{\Psi}_k^{-1}) - \hat{\nu}_k (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \right], \qquad (\text{F11}) $$

$$ \mathbb{E}[\log p(\mathcal{Z} | \Theta; \hat{\phi})] = \sum_{k=1}^{K} N_k \left[ \psi(\hat{\gamma}_{k,1}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}) + \sum_{l=1}^{k-1} [\psi(\hat{\gamma}_{l,2}) - \psi(\hat{\gamma}_{l,1} + \hat{\gamma}_{l,2})] \right], \qquad (\text{F12}) $$

$$ \mathbb{E}[\log p(\Theta; \hat{\phi})] = \sum_{k=1}^{K-1} \mathbb{E}[\log p(\tau_k | \alpha)] + \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] + \sum_{k=1}^{K} \mathbb{E}[\log p(c_k, \Gamma_k; \hat{\rho}_k)], \qquad (\text{F13}) $$

$$ \mathbb{E}[\log p(\tau_k | \alpha)] = \frac{\hat{s}_1 - \hat{s}_2}{\hat{s}_2} [\psi(\hat{\gamma}_{k,2}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})] + \psi(\hat{s}_1) - \log(\hat{s}_2), $$

$$ \mathbb{E}[\log p(\alpha | \hat{s}_1, \hat{s}_2)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$

$$ \mathbb{E}[\log p(c_k, \Gamma_k; \hat{\rho}_k)] = \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{L}{2} - \frac{L}{2} \hat{\nu}_k + \log B \left( \hat{\Psi}_k^{-1}, \hat{\nu}_k \right) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k, $$

$$ \log \tilde{\Gamma}_k = \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + 1 - l}{2} \right) + L \log 2 + \log |\hat{\Psi}_k|, $$

$$ \mathbb{E}[\log q(\mathcal{Z})] = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \log q_{z_n}(k), \qquad (\text{F14}) $$

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha,0}(\alpha)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)] + \sum_{k=1}^{K} \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)], \qquad (\text{F15}) $$

$$ \mathbb{E}[\log q_{\alpha,0}(\alpha)] = -\log \Gamma(\hat{s}_1) + (\hat{s}_1 - 1) \psi(\hat{s}_1) + \log(\hat{s}_2) - \hat{s}_1, $$

$$ \mathbb{E}[\log q_{\tau_k}(\tau_k)] = \sum_{l=1}^{2} (\hat{\gamma}_{k,l} - 1) \{\psi(\hat{\gamma}_{k,l}) - \psi(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})\} + \log \frac{\Gamma(\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2})}{\Gamma(\hat{\gamma}_{k,1}) \Gamma(\hat{\gamma}_{k,2})}, $$

$$ \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)] = \frac{L}{2} \log \frac{\hat{\lambda}_k}{2\pi} - \frac{L}{2} + \log B \left( \hat{\Psi}_k^{-1}, \hat{\nu}_k \right) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k - \frac{\hat{\nu}_k L}{2}. $$
## F.2. Predictive conditional density for BNP-GLLIM2

### F.2.1. Joint density

We first show how to compute the joint density $p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y})$ via Theorem F.2, which is proved in Appendix F.4.

**Theorem F.2.** We approximate the joint density of BNP-GLLIM2 by a mixture of product between Gaussian and Student's t-distributions as follows:

$$p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y}) \approx \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k + 1 - L) \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k). \qquad (F16)$$

Here, the positive semidefinite shape matrices of Student's t-distributions are given by

$$\mathbf{L}_k = \frac{(\hat{\nu}_k + 1 - L) \hat{\lambda}_k}{1 + \hat{\lambda}_k} \hat{\Psi}_k. \qquad (F17)$$

### F.2.2. Inverse conditional density

We then show how to approximate the inverse conditional density $p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y})$. This predictive density in BNP-GLLIM2 is approximated by a MoE via Theorem F.3 with the proof in Appendix F.5.

**Theorem F.3.** We approximate the inverse conditional density of BNP-GLLIM2 by a MoE as follows:

$$p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}) \approx \sum_{k=1}^{K} g_k(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k).$$

Here, the gating posteriors are defined as

$$g_k(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k + 1 - L)}{\sum_{l=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_l(\tau)] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_l, \mathbf{L}_l, \hat{\nu}_l + 1 - L)}, \quad k \in [K].$$

Furthermore, for any $k \in [K]$, it holds that

$$\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] = \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}}[1 - \tau_l],$$

$$\mathbb{E}_{q_{\tau_k}}[\tau_k] = \frac{\hat{\gamma}_{k,1}}{\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}}, \quad \mathbb{E}_{q_{\tau_k}}[1 - \tau_k] = 1 - \mathbb{E}_{q_{\tau_k}}[\tau_k] = \frac{\hat{\gamma}_{k,2}}{\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}},$$

$$\hat{\gamma}_{k,1} = 1 - \mathbb{E}_{q_{\alpha, \sigma}}[\sigma] + N_k, \quad \hat{\gamma}_{k,2} = \mathbb{E}_{q_{\alpha, \sigma}}[\alpha] + k \mathbb{E}_{q_{\alpha, \sigma}}[\sigma] + \sum_{l=k+1}^{K} N_l, \quad N_k = \sum_{n=1}^{n} q_{z_n}(k).$$
## F.2. Predictive conditional density for BNP-GLLIM2

### F.2.1. Joint density

We first show how to compute the joint density $p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y})$ via Theorem F.2, which is proved in Appendix F.4.

**Theorem F.2.** We approximate the joint density of BNP-GLLIM2 by a mixture of product between Gaussian and Student's t-distributions as follows:

$$p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y}) \approx \sum_{k=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k + 1 - L) \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k). \qquad (F16)$$

Here, the positive semidefinite shape matrices of Student's t-distributions are given by

$$\mathbf{L}_k = \frac{(\hat{\nu}_k + 1 - L) \hat{\lambda}_k}{1 + \hat{\lambda}_k} \hat{\Psi}_k. \qquad (F17)$$

### F.2.2. Inverse conditional density

We then show how to approximate the inverse conditional density $p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y})$. This predictive density in BNP-GLLIM2 is approximated by a MoE via Theorem F.3 with the proof in Appendix F.5.

**Theorem F.3.** We approximate the inverse conditional density of BNP-GLLIM2 by a MoE as follows:

$$p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}) \approx \sum_{k=1}^{K} g_k(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k).$$

Here, the gating posteriors are defined as

$$g_k(\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k + 1 - L)}{\sum_{l=1}^{K} \mathbb{E}_{q_{\tau}}[\pi_l(\tau)] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_l, \mathbf{L}_l, \hat{\nu}_l + 1 - L)}, \quad k \in [K].$$

Furthermore, for any $k \in [K]$, it holds that

$$\mathbb{E}_{q_{\tau}}[\pi_k(\tau)] = \mathbb{E}_{q_{\tau_k}}[\tau_k] \prod_{l=1}^{k-1} \mathbb{E}_{q_{\tau_l}}[1 - \tau_l],$$

$$\mathbb{E}_{q_{\tau_k}}[\tau_k] = \frac{\hat{\gamma}_{k,1}}{\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}}, \quad \mathbb{E}_{q_{\tau_k}}[1 - \tau_k] = 1 - \mathbb{E}_{q_{\tau_k}}[\tau_k] = \frac{\hat{\gamma}_{k,2}}{\hat{\gamma}_{k,1} + \hat{\gamma}_{k,2}},$$

$$\hat{\gamma}_{k,1} = 1 - \mathbb{E}_{q_{\alpha, \sigma}}[\sigma] + N_k, \quad \hat{\gamma}_{k,2} = \mathbb{E}_{q_{\alpha, \sigma}}[\alpha] + k \mathbb{E}_{q_{\alpha, \sigma}}[\sigma] + \sum_{l=k+1}^{K} N_l, \quad N_k = \sum_{n=1}^{n} q_{z_n}(k).$$
The prediction task is carried out via the following approximation

$$ \mathbb{E}[\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}] \approx \sum_{k=1}^{K} g_k (\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) [\hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k]. $$

### F.2.3. Forward conditional density

To deal with high-dimensional regression data, namely high-to-low regression, given the inverse conditional density $p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y})$, we want to approximate the following forward conditional density via Theorem F.4, whose proof is provided in Appendix F.6.

**Theorem F.4.** It holds that

$$ p(\hat{\mathbf{x}} | \hat{y}, \mathcal{X}, \mathcal{Y}) \approx \sum_{i=1}^{I} \sum_{k=1}^{K} g_{ki} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{\mathbf{x}} | \hat{\mathbf{A}}_k^*(\eta_i) \hat{y} + \hat{\mathbf{b}}_k^*(\eta_i), \hat{\mathbf{\Sigma}}_k^*(\eta_i)), $$

which is a mixture of Gaussian experts, where, for all $k \in [K]$, $i \in [I]$,

$$ g_{ki} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_D (\hat{y} | \hat{\mathbf{c}}_k^*, \hat{\Gamma}_k^*(\eta_i)) \mathrm{Gam} (\eta_i | \frac{\hat{\nu}_k+1-L}{2}, \frac{\hat{\nu}_k+1-L}{2})}{\sum_{i=1}^{I} \sum_{l=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_l(\tau)] \mathcal{N}_D (\hat{y} | \hat{\mathbf{c}}_l^*, \hat{\Gamma}_l^*(\eta_i)) \mathrm{Gam} (\eta_i | \frac{\hat{\nu}_l+1-L}{2}, \frac{\hat{\nu}_l+1-L}{2})}, $$
$$ \hat{\mathbf{\Sigma}}_k^*(\eta_i) = (\eta_i \mathbf{L}_k + \hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1} \hat{\mathbf{A}}_k)^{-1}, $$
$$ \hat{\mathbf{A}}_k^*(\eta_i) = \hat{\mathbf{\Sigma}}_k^*(\eta_i) \hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1}, $$
$$ \hat{\mathbf{b}}_k^*(\eta_i) = \hat{\mathbf{\Sigma}}_k^*(\eta_i) [\eta \mathbf{L}_k \hat{\mathbf{m}}_k - \hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1} \hat{\mathbf{b}}_k], $$
$$ \hat{\mathbf{c}}_k^* = \hat{\mathbf{A}}_k \hat{\mathbf{m}}_k + \hat{\mathbf{b}}_k, $$
$$ \hat{\Gamma}_k^*(\eta_i) = \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k (\eta_i \mathbf{L}_k)^{-1} \hat{\mathbf{A}}_k^\top. $$

Here, $\eta_i, i \in [I]$, are chosen via discretizing $\eta$-space, $[0, U_\eta]$, into a grid, e.g., uniform. Note that for simplicity, we evaluate the integrand as a Riemann integral with a truncated value $0 < U_\eta < \infty$ and a number of point $I \in \mathbb{N}^*$ for approximating the integration but we can use any scheme to approximate such 1-dimensional integration.

### F.3. Proof of Proposition F.1

Using the sum and product rules for both discrete and continuous variables, the ELBO in BNP-GLLIM (B1) is given by

$$ \begin{aligned} \mathcal{F}(q_Z, q_\Theta, \hat{\phi}) &= \mathbb{E}_{q_Z q_\Theta} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q_\Theta(\Theta)} \right] \equiv \mathbb{E} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] \\ &= \sum_Z \iint q(\mathcal{Z}) q(\Theta) \log \left[ \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] d\mathcal{Z} d\Theta \\ &= \mathbb{E} \left[ \log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi}) \right] \quad (\text{F18}) \\ &\quad + \mathbb{E} \left[ \log p(\mathcal{Z} | \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\Theta; \hat{\phi}) \right] - \mathbb{E} \left[ \log q(\mathcal{Z}) \right] - \mathbb{E} \left[ \log q(\Theta) \right]. \end{aligned} $$
The prediction task is carried out via the following approximation

$$ \mathbb{E}[\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y}] \approx \sum_{k=1}^{K} g_k (\hat{x} | \hat{\Theta}, \hat{\phi}, \mathcal{X}, \mathcal{Y}) [\hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k]. $$

### F.2.3. Forward conditional density

To deal with high-dimensional regression data, namely high-to-low regression, given the inverse conditional density $p(\hat{y} | \hat{x}, \mathcal{X}, \mathcal{Y})$, we want to approximate the following forward conditional density via Theorem F.4, whose proof is provided in Appendix F.6.

**Theorem F.4.** It holds that

$$ p(\hat{\mathbf{x}} | \hat{y}, \mathcal{X}, \mathcal{Y}) \approx \sum_{i=1}^{I} \sum_{k=1}^{K} g_{ki} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) \mathcal{N}_L (\hat{\mathbf{x}} | \hat{\mathbf{A}}_k^*(\eta_i) \hat{y} + \hat{\mathbf{b}}_k^*(\eta_i), \hat{\mathbf{\Sigma}}_k^*(\eta_i)), $$

which is a mixture of Gaussian experts, where, for all $k \in [K]$, $i \in [I]$,

$$ g_{ki} (\hat{y} | \hat{\Theta}^*, \hat{\phi}^*, \mathcal{X}, \mathcal{Y}) = \frac{\mathbb{E}_{q_{\tau}} [\pi_k(\tau)] \mathcal{N}_D (\hat{y} | \hat{\mathbf{c}}_k^*, \hat{\Gamma}_k^*(\eta_i)) \mathrm{Gam} (\eta_i | \frac{\hat{\nu}_k+1-L}{2}, \frac{\hat{\nu}_k+1-L}{2})}{\sum_{i=1}^{I} \sum_{l=1}^{K} \mathbb{E}_{q_{\tau}} [\pi_l(\tau)] \mathcal{N}_D (\hat{y} | \hat{\mathbf{c}}_l^*, \hat{\Gamma}_l^*(\eta_i)) \mathrm{Gam} (\eta_i | \frac{\hat{\nu}_l+1-L}{2}, \frac{\hat{\nu}_l+1-L}{2})}, $$
$$ \hat{\mathbf{\Sigma}}_k^*(\eta_i) = (\eta_i \mathbf{L}_k + \hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1} \hat{\mathbf{A}}_k)^{-1}, $$
$$ \hat{\mathbf{A}}_k^*(\eta_i) = \hat{\mathbf{\Sigma}}_k^*(\eta_i) \hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1}, $$
$$ \hat{\mathbf{b}}_k^*(\eta_i) = \hat{\mathbf{\Sigma}}_k^*(\eta_i) [\eta \mathbf{L}_k \hat{\mathbf{m}}_k - \hat{\mathbf{A}}_k^\top \hat{\mathbf{\Sigma}}_k^{-1} \hat{\mathbf{b}}_k], $$
$$ \hat{\mathbf{c}}_k^* = \hat{\mathbf{A}}_k \hat{\mathbf{m}}_k + \hat{\mathbf{b}}_k, $$
$$ \hat{\Gamma}_k^*(\eta_i) = \hat{\mathbf{\Sigma}}_k + \hat{\mathbf{A}}_k (\eta_i \mathbf{L}_k)^{-1} \hat{\mathbf{A}}_k^\top. $$

Here, $\eta_i, i \in [I]$, are chosen via discretizing $\eta$-space, $[0, U_\eta]$, into a grid, e.g., uniform. Note that for simplicity, we evaluate the integrand as a Riemann integral with a truncated value $0 < U_\eta < \infty$ and a number of point $I \in \mathbb{N}^*$ for approximating the integration but we can use any scheme to approximate such 1-dimensional integration.

### F.3. Proof of Proposition F.1

Using the sum and product rules for both discrete and continuous variables, the ELBO in BNP-GLLIM (B1) is given by

$$ \begin{aligned} \mathcal{F}(q_Z, q_\Theta, \hat{\phi}) &= \mathbb{E}_{q_Z q_\Theta} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q_\Theta(\Theta)} \right] \equiv \mathbb{E} \left[ \log \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] \\ &= \sum_Z \iint q(\mathcal{Z}) q(\Theta) \log \left[ \frac{p(\mathcal{Y}, \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi})}{q(\mathcal{Z}) q(\Theta)} \right] d\mathcal{Z} d\Theta \\ &= \mathbb{E} \left[ \log p(\mathcal{Y} | \mathcal{X}, \mathcal{Z}, \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi}) \right] \quad (\text{F18}) \\ &\quad + \mathbb{E} \left[ \log p(\mathcal{Z} | \Theta; \hat{\phi}) \right] + \mathbb{E} \left[ \log p(\Theta; \hat{\phi}) \right] - \mathbb{E} \left[ \log q(\mathcal{Z}) \right] - \mathbb{E} \left[ \log q(\Theta) \right]. \end{aligned} $$
Note that the proofs of (F10), (F12), (F14) are the same as in the proof of Proposition 3.1.

Proof of (F11)

$$ \mathbb{E} \left[ \log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi}) \right] = \mathbb{E} \left[ \log \prod_{n=1}^{N} p(\mathbf{x}_n | z_n, \Theta; \hat{\phi}) \right] = \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)^{z_{nk}} \right] \\ = \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E} [z_{nk} \log \mathcal{N}_L (\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)] $$

$$ = \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_{\mathcal{Z}}}[z_{nk}] \mathbb{E}_{q_{c_k, \Gamma_k}}[\log \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \mathbf{\Gamma}_k)] $$

$$ = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \mathbb{E}[\log |\mathbf{\Gamma}_k|] - \frac{1}{2} \mathbb{E}[(\mathbf{x}_n - \mathbf{c}_k)^{\top} \mathbf{\Gamma}_k^{-1} (\mathbf{x}_n - \mathbf{c}_k)] \right] $$

$$ = \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_n}(k) \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\mathbf{\Gamma}_k|] - \frac{1}{2} \mathbb{E}_{q_{c_k, \Gamma_k}}[(\mathbf{x}_n - \mathbf{c}_k)^{\top} \mathbf{\Gamma}_k^{-1} (\mathbf{x}_n - \mathbf{c}_k)] \right] \quad (\text{Lemma F.5}) $$

$$ = \frac{1}{2} \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_n}(k) \left[ -\log \tilde{\mathbf{\Gamma}}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} - \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) \right] $$

$$ = \frac{1}{2} \sum_{k=1}^{K} N_k \left[ \log \tilde{\mathbf{\Gamma}}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} \right] - \frac{1}{2} \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_n}(k) \left[ \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) \right] $$

$$ = \frac{1}{2} \sum_{k=1}^{K} N_k \left[ \log \tilde{\mathbf{\Gamma}}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} - \hat{\nu}_k \mathrm{Tr} \left( \mathbf{S}_k \hat{\Psi}_k^{-1} \right) - \hat{\nu}_k (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \right] $$

(using (F21) from Lemma F.5). \qquad (F19)

To obtain (F19), we have to use the following Lemma F.5.

**Lemma F.5.** We can compute the expectations w.r.t. the variational distributions of the parameters as follows:

$$ \log \tilde{\mathbf{\Gamma}}_k \equiv \mathbb{E}_{q_{\Gamma_k}}[\log |\mathbf{\Gamma}_k|] = \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + 1 - l}{2} \right) + L \log 2 + \log |\hat{\Psi}_k|, $$

$$ \mathbb{E}_{q_{c_k, \Gamma_k}}[(\mathbf{x}_n - \mathbf{c}_k)^{\top} \mathbf{\Gamma}_k^{-1} (\mathbf{x}_n - \mathbf{c}_k)] = L \hat{\lambda}_k^{-1} + \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k). \quad (\text{F20}) $$

Furthermore, for each $k \in [K]$, it holds that

$$ \sum_{n=1}^{N} q_{z_n}(k) \left[ \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) \right] = N_k \left[ \hat{\nu}_k \mathrm{Tr} \left( \mathbf{S}_k \hat{\Psi}_k^{-1} \right) + \hat{\nu}_k (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \right]. \quad (\text{F21}) $$
Note that the proofs of (F10), (F12), (F14) are the same as in the proof of Proposition 3.1.

### Proof of (F11)

$$ \mathbb{E}[\log p(\mathcal{X} | \mathcal{Z}, \Theta; \hat{\phi})] = \mathbb{E} \left[ \log \prod_{n=1}^{N} p(\mathbf{x}_n | z_n, \Theta; \hat{\phi}) \right] = \mathbb{E} \left[ \log \prod_{n=1}^{N} \prod_{k=1}^{K} \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \Gamma_k)^{z_{nk}} \right] \\ = \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}[z_{nk} \log \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \Gamma_k)] $$

$$ = \sum_{n=1}^{N} \sum_{k=1}^{K} \mathbb{E}_{q_{\mathcal{Z}}}[z_{nk}] \mathbb{E}_{q_{c_k, \Gamma_k}}[\log \mathcal{N}_L(\mathbf{x}_n | \mathbf{c}_k, \Gamma_k)] $$

$$ = \sum_{n=1}^{N} \sum_{k=1}^{K} q_{z_n}(k) \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \mathbb{E}[\log |\Gamma_k|] - \frac{1}{2} \mathbb{E}[(\mathbf{x}_n - \mathbf{c}_k)^{\top} \Gamma_k^{-1} (\mathbf{x}_n - \mathbf{c}_k)] \right] $$

$$ = \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_n}(k) \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \mathbb{E}_{q_{c_k, \Gamma_k}}[(\mathbf{x}_n - \mathbf{c}_k)^{\top} \Gamma_k^{-1} (\mathbf{x}_n - \mathbf{c}_k)] \right] \quad (\text{Lemma F.5}) $$

$$ = \frac{1}{2} \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_n}(k) \left[ -\log \tilde{\Gamma}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} - \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) \right] $$

$$ = \frac{1}{2} \sum_{k=1}^{K} N_k \left[ \log \tilde{\Gamma}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} \right] - \frac{1}{2} \sum_{k=1}^{K} \sum_{n=1}^{N} q_{z_n}(k) \left[ \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) \right] $$

$$ = \frac{1}{2} \sum_{k=1}^{K} N_k \left[ \log \tilde{\Gamma}_k - L \log(2\pi) - L \hat{\lambda}_k^{-1} - \hat{\nu}_k \mathrm{Tr}(\mathbf{S}_k \hat{\Psi}_k^{-1}) - \hat{\nu}_k (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \right] $$

(using (F21) from Lemma F.5). \qquad (F19)

To obtain (F19), we have to use the following Lemma F.5.

**Lemma F.5.** We can compute the expectations w.r.t. the variational distributions of the parameters as follows:

$$ \log \tilde{\Gamma}_k \equiv \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] = \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + 1 - l}{2} \right) + L \log 2 + \log |\hat{\Psi}_k|, $$

$$ \mathbb{E}_{q_{c_k, \Gamma_k}}[(\mathbf{x}_n - \mathbf{c}_k)^{\top} \Gamma_k^{-1} (\mathbf{x}_n - \mathbf{c}_k)] = L \hat{\lambda}_k^{-1} + \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k). \quad (\text{F20}) $$

Furthermore, for each $k \in [K]$, it holds that

$$ \sum_{n=1}^{N} q_{z_n}(k) \left[ \hat{\nu}_k (\mathbf{x}_n - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\mathbf{x}_n - \hat{\mathbf{m}}_k) \right] = N_k \left[ \hat{\nu}_k \mathrm{Tr}(\mathbf{S}_k \hat{\Psi}_k^{-1}) + \hat{\nu}_k (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \hat{\Psi}_k^{-1} (\bar{\mathbf{x}}_k - \hat{\mathbf{m}}_k) \right]. \quad (\text{F21}) $$
### Proof of (F13)

Given a chosen truncated value $K \in \mathbb{N}^*$, it holds that

$$ \mathbb{E}_{q_{\Theta}}[\log p(\Theta; \hat{\phi})] = \sum_{k=1}^{K-1} \mathbb{E}_{q_{\Theta}}[\log p(\tau_k | \alpha, \sigma)] + \mathbb{E}_{q_{\Theta}}[\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2, \hat{a})] \\ + \sum_{k=1}^{K} \mathbb{E}_{q_{\Theta}}[\log p(c_k, \Gamma_k; \hat{\rho}_k)]. $$

Note that $\mathbb{E}_{q_{\Theta}}[\log p(\tau_k | \alpha, \sigma)]$ and $\mathbb{E}_{q_{\Theta}}[\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2, \hat{a})]$ are calculated in the same way as in Proposition 3.1.

Finally, we have to compute the remaining term

$$ \begin{aligned} & \mathbb{E}_{q_{\Theta}}[\log p(c_k, \Gamma_k; \hat{\rho}_k)] \\ &= \mathbb{E}_{q_{c_k, \Gamma_k}} \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \log \hat{\lambda}_k^{-L} |\Gamma_k| - \frac{1}{2} (c_k - \hat{m}_k)^{\top} (\hat{\lambda}_k^{-1} \Gamma_k)^{-1} (c_k - \hat{m}_k) \right] \\ & \quad + \mathbb{E}_{q_{\Gamma_k}} \left[ \log \mathcal{W} (\Gamma_k^{-1} | \hat{\Psi}_k^{-1}, \hat{\nu}_k) \right] \\ &= -\frac{1}{2} L \log(2\pi) + \frac{1}{2} L \log \hat{\lambda}_k - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \hat{\lambda}_k \mathbb{E}_{q_{c_k, \Gamma_k}} \left[ (c_k - \hat{m}_k)^{\top} \Gamma_k^{-1} (c_k - \hat{m}_k) \right] \\ & \quad + \mathbb{E}_{q_{\Gamma_k}} \left[ \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) + \frac{\hat{\nu}_k - L - 1}{2} \log |\Gamma_k^{-1}| - \frac{1}{2} \mathrm{Tr}(\hat{\Psi}_k \Gamma_k^{-1}) \right] \\ &= \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \hat{\lambda}_k \mathbb{E}_{q_{c_k, \Gamma_k}} \left[ (c_k - \hat{m}_k)^{\top} \Gamma_k^{-1} (c_k - \hat{m}_k) \right] \\ & \quad + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L - 1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \mathrm{Tr}(\hat{\Psi}_k \mathbb{E}_{q_{\Gamma_k}}[\Gamma_k^{-1}]) \\ &= \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{1}{2} \hat{\lambda}_k \left[ L \hat{\lambda}_k^{-1} + \hat{\nu}_k (\hat{m}_k - \hat{m}_k)^{\top} \hat{\Psi}_k^{-1} (\hat{m}_k - \hat{m}_k) \right] \\ & \quad + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k - \frac{1}{2} \hat{\nu}_k \mathrm{Tr}(\hat{\Psi}_k \hat{\Psi}_k^{-1}) \quad (\text{using Lemma F.5}) \\ &= \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{L}{2} - \frac{L}{2} \hat{\nu}_k + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k, \end{aligned} $$

where

$$ \log \tilde{\Gamma}_k \equiv \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] = \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + 1 - l}{2} \right) + L \log 2 + \log |\hat{\Psi}_k|. $$

### Proof of (F15)

We have

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha, \sigma}(\alpha, \sigma)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)] + \sum_{k=1}^{K} \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)]. $$

Note that these terms involving expectations of the logs of the $q$ distributions simply represent the negative entropies of those distributions. In particular, the first two terms
### Proof of (F13)

Given a chosen truncated value $K \in \mathbb{N}^*$, it holds that

$$ \mathbb{E}_{q_{\Theta}}[\log p(\Theta; \hat{\phi})] = \sum_{k=1}^{K-1} \mathbb{E}_{q_{\Theta}}[\log p(\tau_k | \alpha, \sigma)] + \mathbb{E}_{q_{\Theta}}[\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2, \hat{a})] \\ + \sum_{k=1}^{K} \mathbb{E}_{q_{\Theta}}[\log p(c_k, \Gamma_k; \hat{\rho}_k)]. $$

Note that $\mathbb{E}_{q_{\Theta}}[\log p(\tau_k | \alpha, \sigma)]$ and $\mathbb{E}_{q_{\Theta}}[\log p(\alpha, \sigma | \hat{s}_1, \hat{s}_2, \hat{a})]$ are calculated in the same way as in Proposition 3.1.

Finally, we have to compute the remaining term

$$ \begin{aligned} & \mathbb{E}_{q_{\Theta}}[\log p(c_k, \Gamma_k; \hat{\rho}_k)] \\ &= \mathbb{E}_{q_{c_k, \Gamma_k}} \left[ -\frac{L}{2} \log(2\pi) - \frac{1}{2} \log \hat{\lambda}_k^{-L} |\Gamma_k| - \frac{1}{2} (c_k - \hat{m}_k)^{\top} (\hat{\lambda}_k^{-1} \Gamma_k)^{-1} (c_k - \hat{m}_k) \right] \\ & \quad + \mathbb{E}_{q_{\Gamma_k}} \left[ \log \mathcal{W} (\Gamma_k^{-1} | \hat{\Psi}_k^{-1}, \hat{\nu}_k) \right] \\ &= -\frac{1}{2} L \log(2\pi) + \frac{1}{2} L \log \hat{\lambda}_k - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \hat{\lambda}_k \mathbb{E}_{q_{c_k, \Gamma_k}} \left[ (c_k - \hat{m}_k)^{\top} \Gamma_k^{-1} (c_k - \hat{m}_k) \right] \\ & \quad + \mathbb{E}_{q_{\Gamma_k}} \left[ \log B (\hat{\Psi}_k^{-1}, \hat{\nu}_k) + \frac{\hat{\nu}_k - L - 1}{2} \log |\Gamma_k^{-1}| - \frac{1}{2} \mathrm{Tr}(\hat{\Psi}_k \Gamma_k^{-1}) \right] \\ &= \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \hat{\lambda}_k \mathbb{E}_{q_{c_k, \Gamma_k}} \left[ (c_k - \hat{m}_k)^{\top} \Gamma_k^{-1} (c_k - \hat{m}_k) \right] \\ & \quad + \log B (\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L - 1}{2} \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] - \frac{1}{2} \mathrm{Tr}(\hat{\Psi}_k \mathbb{E}_{q_{\Gamma_k}}[\Gamma_k^{-1}]) \\ &= \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{1}{2} \hat{\lambda}_k \left[ L \hat{\lambda}_k^{-1} + \hat{\nu}_k (\hat{m}_k - \hat{m}_k)^{\top} \hat{\Psi}_k^{-1} (\hat{m}_k - \hat{m}_k) \right] \\ & \quad + \log B (\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k - \frac{1}{2} \hat{\nu}_k \mathrm{Tr}(\hat{\Psi}_k \hat{\Psi}_k^{-1}) \quad (\text{using Lemma F.5}) \\ &= \frac{1}{2} L \log \left( \frac{\hat{\lambda}_k}{2\pi} \right) - \frac{L}{2} - \frac{L}{2} \hat{\nu}_k + \log B (\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k, \end{aligned} $$

where

$$ \log \tilde{\Gamma}_k \equiv \mathbb{E}_{q_{\Gamma_k}}[\log |\Gamma_k|] = \sum_{l=1}^{L} \psi \left( \frac{\hat{\nu}_k + 1 - l}{2} \right) + L \log 2 + \log |\hat{\Psi}_k|. $$

### Proof of (F15)

We have

$$ \mathbb{E}[\log q(\Theta)] = \mathbb{E}[\log q_{\alpha, \sigma}(\alpha, \sigma)] + \sum_{k=1}^{K-1} \mathbb{E}[\log q_{\tau_k}(\tau_k)] + \sum_{k=1}^{K} \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)]. $$

Note that these terms involving expectations of the logs of the $q$ distributions simply represent the negative entropies of those distributions. In particular, the first two terms
are calculated in the same way as in Proposition 3.1.

Similarly, we obtain

$$
\begin{align*}
& \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)] \\
&= \mathbb{E}[\log \mathcal{N}_L(c_k | \hat{\mathbf{m}}_k, \hat{\lambda}_k^{-1} \Gamma_k)] + \mathbb{E}[\log \mathcal{W}(\Gamma_k^{-1} | \hat{\Psi}_k^{-1}, \hat{\nu}_k)] \\
&= -H[\mathcal{N}_L(c_k | \hat{\mathbf{m}}_k, \hat{\lambda}_k^{-1} \Gamma_k)] - H[\mathcal{W}(\Gamma_k^{-1} | \hat{\Psi}_k^{-1}, \hat{\nu}_k)] \\
&= \frac{L}{2} \log \frac{\hat{\lambda}_k}{2\pi} + \frac{1}{2} \mathbb{E}[\log |\Gamma_k|] - \frac{L}{2} + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L - 1}{2} \mathbb{E}[\log |\Gamma_k|] - \frac{\hat{\nu}_k L}{2} \\
&= \frac{L}{2} \log \frac{\hat{\lambda}_k}{2\pi} - \frac{L}{2} + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k - \frac{\hat{\nu}_k L}{2}.
\end{align*}
$$

F.4. *Proof of Theorem F.2*

Recall that we defined $\Theta = (\tau, \alpha, \sigma, \theta^*)$, $\theta^* = (\theta_k^*)_{k \in \mathbb{N}^*} \equiv (c_k, \Gamma_k)_{k \in \mathbb{N}^*}$. Then,

$$
\begin{align}
p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y}) &= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{x} | \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{z} | \Theta, \mathcal{X}, \mathcal{Y}) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \nonumber \\
&= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\hat{x} | \hat{z}, \mathbf{c}, \mathbf{\Gamma}) p(\hat{z} | \tau; \beta) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \equiv T_1. \tag{F22}
\end{align}
$$

Note that in (F22), $p(\Theta | \mathcal{X}, \mathcal{Y})$ is in fact the (unknown) true posterior distribution of the parameters given a sample $(\mathcal{X}, \mathcal{Y})$. Because the integrations w.r.t. true posterior distribution are intractable, we approximate the predictive conditional density by replacing the true posterior distribution $p(\Theta | \mathcal{X}, \mathcal{Y})$ with its truncated variational posterior of parameters $\Theta$ given by

$$
q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) = q_{\alpha, \sigma}(\alpha, \sigma | \mathcal{X}, \mathcal{Y}) \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k | \mathcal{X}, \mathcal{Y}) \prod_{k=1}^{K} q_{\theta_k^*}(\theta_k^* | \mathcal{X}, \mathcal{Y}).
$$

Recall that the infinite state space for each $z_j$ is dealt with by choosing a truncation of
the state space to a maximum label $K$ (Blei and Jordan, 2006). In practice, this consists
of assuming that the variational distributions $q_{z_n}$ for $n \in [N]$, satisfy $q_{z_n}(k) = 0$ for $k >$
$K$ and that the variational distribution on $\tau$ also factorizes as $q_{\tau}(\tau) = \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k)$
with the additional condition that $\tau_K = 1$. In particular, here we choose $K = \tilde{K}$, where
$\tilde{K}$ is estimated from some suitable procedures.
are calculated in the same way as in Proposition 3.1.

Similarly, we obtain

$$
\begin{align*}
& \mathbb{E}[\log q_{c_k, \Gamma_k}(c_k, \Gamma_k)] \\
&= \mathbb{E}[\log \mathcal{N}_L(c_k | \hat{\mathbf{m}}_k, \hat{\lambda}_k^{-1} \Gamma_k)] + \mathbb{E}[\log \mathcal{W}(\Gamma_k^{-1} | \hat{\Psi}_k^{-1}, \hat{\nu}_k)] \\
&= -H[\mathcal{N}_L(c_k | \hat{\mathbf{m}}_k, \hat{\lambda}_k^{-1} \Gamma_k)] - H[\mathcal{W}(\Gamma_k^{-1} | \hat{\Psi}_k^{-1}, \hat{\nu}_k)] \\
&= \frac{L}{2} \log \frac{\hat{\lambda}_k}{2\pi} + \frac{1}{2} \mathbb{E}[\log |\Gamma_k|] - \frac{L}{2} + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L - 1}{2} \mathbb{E}[\log |\Gamma_k|] - \frac{\hat{\nu}_k L}{2} \\
&= \frac{L}{2} \log \frac{\hat{\lambda}_k}{2\pi} - \frac{L}{2} + \log B(\hat{\Psi}_k^{-1}, \hat{\nu}_k) - \frac{\hat{\nu}_k - L}{2} \log \tilde{\Gamma}_k - \frac{\hat{\nu}_k L}{2}.
\end{align*}
$$

F.4. *Proof of Theorem F.2*

Recall that we defined $\Theta = (\tau, \alpha, \sigma, \theta^*)$, $\theta^* = (\theta_k^*)_{k \in \mathbb{N}^*} \equiv (c_k, \Gamma_k)_{k \in \mathbb{N}^*}$. Then,

$$
\begin{align}
p(\hat{y}, \hat{x}, \mathcal{X}, \mathcal{Y}) &= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{x} | \hat{z}, \Theta, \mathcal{X}, \mathcal{Y}) p(\hat{z} | \Theta, \mathcal{X}, \mathcal{Y}) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \nonumber \\
&= \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\hat{x} | \hat{z}, \mathbf{c}, \mathbf{\Gamma}) p(\hat{z} | \tau; \beta) p(\Theta | \mathcal{X}, \mathcal{Y}) d\Theta \equiv T_1. \tag{F22}
\end{align}
$$

Note that in (F22), $p(\Theta | \mathcal{X}, \mathcal{Y})$ is in fact the (unknown) true posterior distribution of the parameters given a sample $(\mathcal{X}, \mathcal{Y})$. Because the integrations w.r.t. true posterior distribution are intractable, we approximate the predictive conditional density by replacing the true posterior distribution $p(\Theta | \mathcal{X}, \mathcal{Y})$ with its truncated variational posterior of parameters $\Theta$ given by

$$
q_{\Theta}(\Theta | \mathcal{X}, \mathcal{Y}) = q_{\alpha, \sigma}(\alpha, \sigma | \mathcal{X}, \mathcal{Y}) \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k | \mathcal{X}, \mathcal{Y}) \prod_{k=1}^{K} q_{\theta_k^*}(\theta_k^* | \mathcal{X}, \mathcal{Y}).
$$

Recall that the infinite state space for each $z_j$ is dealt with by choosing a truncation of
the state space to a maximum label $K$ (Blei and Jordan, 2006). In practice, this consists
of assuming that the variational distributions $q_{z_n}$ for $n \in [N]$, satisfy $q_{z_n}(k) = 0$ for $k >$
$K$ and that the variational distribution on $\tau$ also factorizes as $q_{\tau}(\tau) = \prod_{k=1}^{K-1} q_{\tau_k}(\tau_k)$
with the additional condition that $\tau_K = 1$. In particular, here we choose $K = \tilde{K}$, where
$\tilde{K}$ is estimated from some suitable procedures.
For simplicity, we consider the case when $\beta = 0, \sigma = 0$. Then, we obtain

$$
\begin{align*}
T_1 &\approx \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\hat{x} | \hat{z}, \mathbf{c}, \mathbf{\Gamma}) p(\hat{z} | \mathbf{\tau}) q_{\Theta}(\mathbf{\Theta} | \mathcal{X}, \mathcal{Y}) d\mathbf{\Theta} \\
&= \sum_{k=1}^{\infty} \int \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) \pi_k(\mathbf{\tau}) q_{\Theta}(\mathbf{\Theta} | \mathcal{X}, \mathcal{Y}) d\mathbf{\Theta} \\
&\approx \sum_{k=1}^{K} \int \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) \int \pi_k(\mathbf{\tau}) q_{\mathbf{\tau}}(\mathbf{\tau} | \mathcal{X}, \mathcal{Y}) d\mathbf{\tau} \\
&\qquad \times \underbrace{\int q_{\alpha,0}(\alpha | \mathcal{X}, \mathcal{Y}) d\alpha}_{=1} \prod_{k=1}^{K} q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\mathbf{\tau}}}[\pi_k(\mathbf{\tau})] \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \int \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k \\
&\qquad \times \underbrace{\int \prod_{j=1, j \neq k}^{K} q_{\theta_j^*}(\mathbf{c}_j, \mathbf{\Gamma}_j | \mathcal{X}, \mathcal{Y}) \prod_{j=1, j \neq k}^{K} d\mathbf{c}_j d\mathbf{\Gamma}_j}_{=1} \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\mathbf{\tau}}}[\pi_k(\mathbf{\tau})] \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \underbrace{\int \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k}_{=\mathrm{St}(\hat{x}|\hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\mathbf{\nu}}_k+1-L) \text{(Lemma F.6)}} \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\mathbf{\tau}}}[\pi_k(\mathbf{\tau})] \mathrm{St}(\hat{x} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\mathbf{\nu}}_k+1-L) \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k).
\end{align*}
$$

Here we used the following Lemma F.6

**Lemma F.6.** For each $k \in [K]$, it holds that

$$
\int \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \mathbf{\Gamma}_k) q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k = \mathrm{St}(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\mathbf{\nu}}_k + 1 - L).
$$

Proof of Lemma F.6

By definition, we obtain

$$
\begin{align*}
& \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) q(\boldsymbol{\pi} | \mathcal{X}) q(\mathbf{c}_k, \Gamma_k | \mathcal{X}) d\mathbf{c}_k d\Gamma_k \\
&= \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\mathbf{c}_k d\Gamma_k \\
&= \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\mathbf{c}_k d\Gamma_k \\
&= \iint \left[ \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) d\mathbf{c}_k \right] W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\Gamma_k \\
&= \iint \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\Gamma_k. \tag{F23}
\end{align*}
$$
For simplicity, we consider the case when $\beta = 0, \sigma = 0$. Then, we obtain

$$
\begin{align*}
T_1 &\approx \sum_{\hat{z}} \int p(\hat{y} | \hat{x}, \hat{z}; \mathbf{A}, \mathbf{b}, \mathbf{\Sigma}) p(\hat{x} | \hat{z}, \mathbf{c}, \mathbf{\Gamma}) p(\hat{z} | \mathbf{\tau}) q_{\Theta}(\mathbf{\Theta} | \mathcal{X}, \mathcal{Y}) d\mathbf{\Theta} \\
&= \sum_{k=1}^{\infty} \int \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) \pi_k(\mathbf{\tau}) q_{\Theta}(\mathbf{\Theta} | \mathcal{X}, \mathcal{Y}) d\mathbf{\Theta} \\
&\approx \sum_{k=1}^{K} \int \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) \int \pi_k(\mathbf{\tau}) q_{\mathbf{\tau}}(\mathbf{\tau} | \mathcal{X}, \mathcal{Y}) d\mathbf{\tau} \\
&\qquad \times \underbrace{\int q_{\alpha,0}(\alpha | \mathcal{X}, \mathcal{Y}) d\alpha}_{=1} \prod_{k=1}^{K} q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\mathbf{\tau}}}[\pi_k(\mathbf{\tau})] \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \int \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k \\
&\qquad \times \underbrace{\int \prod_{j=1, j \neq k}^{K} q_{\theta_j^*}(\mathbf{c}_j, \mathbf{\Gamma}_j | \mathcal{X}, \mathcal{Y}) \prod_{j=1, j \neq k}^{K} d\mathbf{c}_j d\mathbf{\Gamma}_j}_{=1} \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\mathbf{\tau}}}[\pi_k(\mathbf{\tau})] \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k) \underbrace{\int \mathcal{N}_L(\hat{x} | \mathbf{c}_k, \mathbf{\Gamma}_k) q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k}_{=\text{St}(\hat{x}|\hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k+1-L) \text{(Lemma F.6)}} \\
&= \sum_{k=1}^{K} \mathbb{E}_{q_{\mathbf{\tau}}}[\pi_k(\mathbf{\tau})] \text{St}(\hat{x} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k+1-L) \mathcal{N}_D(\hat{y} | \hat{\mathbf{A}}_k \hat{x} + \hat{\mathbf{b}}_k, \hat{\mathbf{\Sigma}}_k).
\end{align*}
$$

Here we used the following Lemma F.6

**Lemma F.6.** For each $k \in [K]$, it holds that

$$
\int \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \mathbf{\Gamma}_k) q_{\theta_k^*}(\mathbf{c}_k, \mathbf{\Gamma}_k | \mathcal{X}, \mathcal{Y}) d\mathbf{c}_k d\mathbf{\Gamma}_k = \text{St}(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k + 1 - L).
$$

Proof of Lemma F.6

By definition, we obtain

$$
\begin{align*}
& \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) q(\boldsymbol{\pi} | \mathcal{X}) q(\mathbf{c}_k, \Gamma_k | \mathcal{X}) d\mathbf{c}_k d\Gamma_k \\
&= \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\mathbf{c}_k d\Gamma_k \\
&= \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\mathbf{c}_k d\Gamma_k \\
&= \iint \left[ \iint \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) d\mathbf{c}_k \right] W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\Gamma_k \\
&= \iint \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\Gamma_k. \tag{F23}
\end{align*}
$$
When the size of the data set is large, i.e., $N \to \infty$, this predictive distribution (F23) becomes a mixture of Gaussians with component means $\hat{\mathbf{m}}_k$ and precisions $\mathbf{L}_k$. In particular, we made use of the following results for marginal and conditional Gaussians, see, e.g., Bishop (2006, Eq. (2.115), page 93). Given a marginal Gaussian distribution for $x$ and a conditional Gaussian distribution for $y$ given $x$ in the form

$$p(\mathbf{x}) = \mathcal{N}(\mathbf{x} | \mu, \Gamma^{-1}), \\ p(\mathbf{y} | \mathbf{x}) = \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{x} + \mathbf{b}, \mathbf{L}^{-1}),$$

then the marginal distribution of $y$ and the conditional distribution of $x$ given $y$ are given by

$$p(\mathbf{y}) = \mathcal{N}(\mathbf{y} | \mathbf{A}\mu + \mathbf{b}, \mathbf{L}^{-1} + \mathbf{A}\Gamma^{-1}\mathbf{A}^{\top}), \\ p(\mathbf{x} | \mathbf{y}) = \mathcal{N}(\mathbf{x} | \mathbf{\Sigma}\{\mathbf{A}^{\top}\mathbf{L}(\mathbf{y} - \mathbf{b}) + \Gamma\mu\}, \mathbf{\Sigma}),$$

where

$$\mathbf{\Sigma} = (\mathbf{\Gamma} + \mathbf{A}^{\top}\mathbf{L}\mathbf{A})^{-1}.$$

In our situation, via using $\mathbf{y} \equiv \hat{\mathbf{x}}, \mathbf{x} \equiv \mathbf{c}_k, \mathbf{A} \equiv \mathbf{I}, \mathbf{b} \equiv \mathbf{0}, \mathbf{L}^{-1} = \Gamma_k^{-1}, \mu \equiv \hat{\mathbf{m}}_k, \Gamma^{-1} \equiv (\hat{\lambda}_k \Gamma_k)^{-1}$, we obtain

$$p(\hat{\mathbf{x}} | \Gamma_k^{-1}, \mathcal{X}) = \int \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) d\mathbf{c}_k \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \Gamma_k^{-1} + (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \left(\frac{1 + \hat{\lambda}_k}{\hat{\lambda}_k}\right) \Gamma_k^{-1}, \mathcal{X}).$$

Notice that the Wishart distribution is a conjugate prior for the Gaussian distribution with known mean and unknown precision. Therefore, it holds that the product of

$$\mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X})$$

is again a Wishart distribution without normalized. This can be verified by focusing on the dependency on $\Gamma_k$. More precisely, by using the trace trick of quadratic form,
When the size of the data set is large, i.e., $N \to \infty$, this predictive distribution (F23) becomes a mixture of Gaussians with component means $\hat{\mathbf{m}}_k$ and precisions $\mathbf{L}_k$. In particular, we made use of the following results for marginal and conditional Gaussians, see, e.g., Bishop (2006, Eq. (2.115), page 93). Given a marginal Gaussian distribution for $x$ and a conditional Gaussian distribution for $y$ given $x$ in the form

$$p(\mathbf{x}) = \mathcal{N}(\mathbf{x} | \mu, \Gamma^{-1}), \\ p(\mathbf{y} | \mathbf{x}) = \mathcal{N}(\mathbf{y} | \mathbf{A}\mathbf{x} + \mathbf{b}, \mathbf{L}^{-1}),$$

then the marginal distribution of $y$ and the conditional distribution of $x$ given $y$ are given by

$$p(\mathbf{y}) = \mathcal{N}(\mathbf{y} | \mathbf{A}\mu + \mathbf{b}, \mathbf{L}^{-1} + \mathbf{A}\Gamma^{-1}\mathbf{A}^{\top}), \\ p(\mathbf{x} | \mathbf{y}) = \mathcal{N}(\mathbf{x} | \mathbf{\Sigma}\{\mathbf{A}^{\top}\mathbf{L}(\mathbf{y} - \mathbf{b}) + \mathbf{\Gamma}\mu\}, \mathbf{\Sigma}),$$

where

$$\mathbf{\Sigma} = (\mathbf{\Gamma} + \mathbf{A}^{\top}\mathbf{L}\mathbf{A})^{-1}.$$

In our situation, via using $\mathbf{y} \equiv \hat{\mathbf{x}}, \mathbf{x} \equiv \mathbf{c}_k, \mathbf{A} \equiv \mathbf{I}, \mathbf{b} \equiv \mathbf{0}, \mathbf{L}^{-1} = \Gamma_k^{-1}, \mu \equiv \hat{\mathbf{m}}_k, \Gamma_k^{-1} \equiv (\hat{\lambda}_k \Gamma_k)^{-1}$, we obtain

$$p(\hat{\mathbf{x}} | \Gamma_k^{-1}, \mathcal{X}) = \int \mathcal{N}_L(\hat{\mathbf{x}} | \mathbf{c}_k, \Gamma_k^{-1}) \mathcal{N}_L(\mathbf{c}_k | \hat{\mathbf{m}}_k, (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) d\mathbf{c}_k \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \Gamma_k^{-1} + (\hat{\lambda}_k \Gamma_k)^{-1}, \mathcal{X}) \\ = \mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \left(\frac{1 + \hat{\lambda}_k}{\hat{\lambda}_k}\right) \Gamma_k^{-1}, \mathcal{X}).$$

Notice that the Wishart distribution is a conjugate prior for the Gaussian distribution with known mean and unknown precision. Therefore, it holds that the product of

$$\mathcal{N}_L(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W(\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X})$$

is again a Wishart distribution without normalized. This can be verified by focusing on the dependency on $\Gamma_k$. More precisely, by using the trace trick of quadratic form,
$(\hat{x} - \hat{m}_k)^{\top} \Gamma_k (\hat{x} - \hat{m}_k) = \text{Tr}((\hat{x} - \hat{m}_k)(\hat{x} - \hat{m}_k)^{\top} \Gamma_k)$, we obtain

$$
\begin{align*}
& N_L (\hat{x} | \hat{m}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W (\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) \\
&= \underbrace{\frac{B(\hat{\Psi}_k, \hat{\nu}_k)}{(2\pi (1 + \hat{\lambda}_k^{-1}))^{L/2} |\Gamma_k|^{1/2 + (\hat{\nu}_k - L - 1)/2}}}_{\equiv C(\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k)} \\
&\quad \times \exp \left\{ -\frac{1}{2(1 + \hat{\lambda}_k^{-1})} (\hat{x} - \hat{m}_k)^{\top} \Gamma_k (\hat{x} - \hat{m}_k) - \frac{1}{2} \text{Tr} (\hat{\Psi}_k^{-1} \Gamma_k) \right\} \\
&= C (\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k) |\Gamma_k|^{(\hat{\nu}_k + 1 - L - 1)/2} \exp \left\{ -\frac{1}{2} \text{Tr} \left( (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^{\top} \Gamma_k + \hat{\Psi}_k^{-1} \Gamma_k \right) \right\} \\
&= C (\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k) |\Gamma_k|^{(\hat{\nu}_k + 1 - L - 1)/2} \exp \left\{ -\frac{1}{2} \text{Tr} \left\{ \left[ (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^{\top} + \hat{\Psi}_k^{-1} \right] \Gamma_k \right\} \right\} \\
&= \frac{C (\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k)}{B (\hat{\Psi}_k^*, \hat{\nu}_k^*)} W (\Gamma_k | \Psi_k^*, \hat{\nu}_k^*).
\end{align*}
$$

Here, $\hat{\nu}_k^* = \hat{\nu}_k + 1$, and

$$
\begin{align*}
\Psi_k^* &= \left[ (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^{\top} + \hat{\Psi}_k^{-1} \right]^{-1}, \\
|\hat{\Psi}_k^*|^{(\hat{\nu}_k+1)/2} &= \left| (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^{\top} + \hat{\Psi}_k^{-1} \right|^{-(\hat{\nu}_k+1)/2} \\
&= |\hat{\Psi}_k^{-1}| \left[ (1 + \hat{\lambda}_k^{-1})^{-1} \hat{\Psi}_k (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^{\top} + \mathbf{I} \right]^{-(\hat{\nu}_k+1)/2} \\
&= |\hat{\Psi}_k|^{(\hat{\nu}_k+1)/2} \left| (1 + \hat{\lambda}_k^{-1})^{-1} \hat{\Psi}_k (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^{\top} + \mathbf{I} \right|^{-(\hat{\nu}_k+1)/2} \\
&= |\hat{\Psi}_k|^{(\hat{\nu}_k+1)/2} \left[ 1 + (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k)^{\top} \hat{\Psi}_k (\hat{x} - \hat{m}_k) \right]^{-(\hat{\nu}_k+1)/2}.
\end{align*}
$$
$(\hat{x} - \hat{m}_k)^\top \Gamma_k (\hat{x} - \hat{m}_k) = \text{Tr}((\hat{x} - \hat{m}_k)(\hat{x} - \hat{m}_k)^\top \Gamma_k)$, we obtain

$$
\begin{align*}
& N_L (\hat{x} | \hat{m}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W (\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) \\
&= \underbrace{\frac{B(\hat{\Psi}_k, \hat{\nu}_k)}{(2\pi (1 + \hat{\lambda}_k^{-1}))^{L/2} |\Gamma_k|^{1/2 + (\hat{\nu}_k - L - 1)/2}}}_{\equiv C(\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k)} \\
&\quad \times \exp \left\{ -\frac{1}{2(1 + \hat{\lambda}_k^{-1})} (\hat{x} - \hat{m}_k)^\top \Gamma_k (\hat{x} - \hat{m}_k) - \frac{1}{2} \text{Tr} (\hat{\Psi}_k^{-1} \Gamma_k) \right\} \\
&= C (\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k) |\Gamma_k|^{(\hat{\nu}_k + 1 - L - 1)/2} \exp \left\{ -\frac{1}{2} \text{Tr} \left( (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^\top \Gamma_k + \hat{\Psi}_k^{-1} \Gamma_k \right) \right\} \\
&= C (\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k) |\Gamma_k|^{(\hat{\nu}_k + 1 - L - 1)/2} \exp \left\{ -\frac{1}{2} \text{Tr} \left\{ \left[ (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^\top + \hat{\Psi}_k^{-1} \right] \Gamma_k \right\} \right\} \\
&= \frac{C (\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k)}{B (\hat{\Psi}_k^*, \hat{\nu}_k^*)} W (\Gamma_k | \Psi_k^*, \hat{\nu}_k^*).
\end{align*}
$$

Here, $\hat{\nu}_k^* = \hat{\nu}_k + 1$, and

$$
\begin{align*}
\Psi_k^* &= \left[ (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^\top + \hat{\Psi}_k^{-1} \right]^{-1}, \\
|\hat{\Psi}_k^*|^{(\hat{\nu}_k+1)/2} &= \left| (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^\top + \hat{\Psi}_k^{-1} \right|^{-(\hat{\nu}_k+1)/2} \\
&= |\hat{\Psi}_k^{-1}| \left[ (1 + \hat{\lambda}_k^{-1})^{-1} \hat{\Psi}_k (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^\top + \mathbf{I} \right]^{-(\hat{\nu}_k+1)/2} \\
&= |\hat{\Psi}_k|^{(\hat{\nu}_k+1)/2} \left| (1 + \hat{\lambda}_k^{-1})^{-1} \hat{\Psi}_k (\hat{x} - \hat{m}_k) (\hat{x} - \hat{m}_k)^\top + \mathbf{I} \right|^{-(\hat{\nu}_k+1)/2} \\
&= |\hat{\Psi}_k|^{(\hat{\nu}_k+1)/2} \left[ 1 + (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{x} - \hat{m}_k)^\top \hat{\Psi}_k (\hat{x} - \hat{m}_k) \right]^{-(\hat{\nu}_k+1)/2}.
\end{align*}
$$
Via the normalization constant we have

$$
\begin{align*}
& \int \mathcal{N}_L (\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, (1 + \hat{\lambda}_k^{-1}) \Gamma_k^{-1}, \mathcal{X}) W (\Gamma_k | \hat{\Psi}_k, \hat{\nu}_k, \mathcal{X}) d\Gamma_k \\
&= \frac{C(\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k)}{B(\hat{\Psi}_k^*, \hat{\nu}_k^*)} \underbrace{\int W (\Gamma_k | \Psi_k^*, \hat{\nu}_k^*) d\Gamma_k}_{=1} = \frac{C(\hat{\Psi}_k, \hat{\nu}_k, \hat{\lambda}_k)}{B(\hat{\Psi}_k^*, \hat{\nu}_k^*)} = \frac{B(\hat{\Psi}_k, \hat{\nu}_k)}{(2\pi (1 + \hat{\lambda}_k^{-1}))^{L/2}} \frac{1}{B(\hat{\Psi}_k^*, \hat{\nu}_k^*)} \\
&= \frac{1}{(2\pi (1 + \hat{\lambda}_k^{-1}))^{L/2}} \frac{|\hat{\Psi}_k|^{-\hat{\nu}_k/2} (2^{\hat{\nu}_k L/2} \pi^{L(L-1)/4} \prod_{l=1}^L \Gamma(\frac{\hat{\nu}_k+1-l}{2}))^{-1}}{|\hat{\Psi}_k^*|^{-(\hat{\nu}_k+1)/2} (2^{(\hat{\nu}_k+1)L/2} \pi^{L(L-1)/4} \prod_{l=1}^L \Gamma(\frac{(\hat{\nu}_k+1)+1-l}{2}))^{-1}} \\
&= \frac{1}{(\pi (1 + \hat{\lambda}_k^{-1}))^{L/2}} \frac{|\hat{\Psi}_k|^{-\hat{\nu}_k/2} \Gamma(\frac{\hat{\nu}_k+1}{2}) \Gamma(\frac{\hat{\nu}_k}{2}) \dots \Gamma(\frac{\hat{\nu}_k+2-L}{2})}{|\hat{\Psi}_k^*|^{-(\hat{\nu}_k+1)/2} \Gamma(\frac{\hat{\nu}_k}{2}) \Gamma(\frac{\hat{\nu}_k-1}{2}) \dots \Gamma(\frac{\hat{\nu}_k+2-L}{2}) \Gamma(\frac{\hat{\nu}_k+1-L}{2})} \\
&= \frac{\Gamma(\frac{\hat{\nu}_k+1}{2})}{\Gamma(\frac{\hat{\nu}_k+1-L}{2}) \pi^{L/2}} \frac{|\hat{\Psi}_k|^{-\hat{\nu}_k/2}}{(1 + \hat{\lambda}_k^{-1})^{L/2}} |\hat{\Psi}_k|^{(\hat{\nu}_k+1)/2} [1 + (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{\mathbf{x}} - \hat{\mathbf{m}}_k)^{\top} \hat{\Psi}_k (\hat{\mathbf{x}} - \hat{\mathbf{m}}_k)]^{-(\hat{\nu}_k+1)/2} \\
&= \frac{\Gamma(\frac{\hat{\nu}_k+1}{2})}{\Gamma(\frac{\hat{\nu}_k+1-L}{2}) \pi^{L/2}} \frac{|\hat{\Psi}_k|^{1/2}}{(1 + \hat{\lambda}_k^{-1})^{L/2}} [1 + (1 + \hat{\lambda}_k^{-1})^{-1} (\hat{\mathbf{x}} - \hat{\mathbf{m}}_k)^{\top} \hat{\Psi}_k (\hat{\mathbf{x}} - \hat{\mathbf{m}}_k)]^{-(\hat{\nu}_k+1)/2} \\
&= \text{St}(\hat{\mathbf{x}} | \hat{\mathbf{m}}_k, \mathbf{L}_k, \hat{\nu}_k + 1 - L).
\end{align*}
$$

Here,

$$
\mathbf{L}_k = \frac{(\hat{\nu}_k + 1 - L) \hat{\lambda}_k}{1 + \hat{\lambda}_k} \hat{\Psi}_k,
$$

and $\Delta^2$ is the squared Mahalanobis distance defined by

$$
\Delta^2 = (\hat{\mathbf{x}} - \hat{\mathbf{m}}_k)^{\top} \mathbf{L}_k (\hat{\mathbf{x}} - \hat{\mathbf{m}}_k).
$$