## 3 Solution Framework
### Mean-Field CEVTI  
This subsection focuses on the hierarchical Bayesian model. In this model, all users share a global variable that determines the distribution of channel coefficients, while each local hidden variable belongs to each user. We denote \(\mathbf{r}=r_{1:N}\) as observed variables, \(\mathbf{d}=d_{1:N}\) as local hidden values, and \(\alpha\) as global hidden values. Using a variational distribution \(q(\mathbf{a}|\alpha)\) to approximate \(p(\mathbf{a}|\alpha)\), and a variational distribution \(q(\mathbf{d}|\theta)\) to approximate \(p(\mathbf{d}|\theta)\), the joint density of the model, according to Equation (2), then becomes:  
\[p(\mathbf{a},\mathbf{r}\mathbf{d})=q(\mathbf{a}|\alpha)q(\mathbf{d}|\theta) \prod_{i=1}^{N}p(r_{i},d_{i}|\theta), \tag{4}\]  
of which \(\alpha\) is the global variable, i.e., the hyper-parameter of this model, and \(\theta\) is the hidden local variable. The main computing problem of the Bayesian model is the posterior inference, i.e., the output signal (or received signal). Our objective is to calculate \(p(\textbf{a,d}\,\textbf{l}\,\textbf{r})\), the conditional probability of the hidden variable (the transmitted signal) and the global hidden variables (the hyper-parameter of channel coefficients) given the observations (received signal). In many cases, this computation is intractable, and approximation is required. CEVTI proposes a parameterized family of hidden variables and tries to determine the family of members that are closest to the posterior distribution by KL divergence. That is to say, CEVTI tries to use the variational distribution to replace the target distribution. In VI, this is equal to computing the ELBO in terms of a variational parameter, since maximizing ELBO and minimizing KL divergence between the variational distribution and target distribution, are equivalent to maximizing the posterior probability:  
\[ \begin{split}\text{log}p(\textbf{r})&=\mathbb{E}_ {q}[\text{log}(\frac{p(\textbf{a,d}\,\textbf{r})}{q(\textbf{a}|\alpha)q( \textbf{d}|\theta)})]-\mathbb{E}_{q}[\text{log}(\frac{p(\textbf{a,d}\, \textbf{l}\,\textbf{r})}{q(\textbf{a}|\alpha)q(\textbf{d}|\theta)})]\\ &=\mathbb{E}_{q}[\text{log}p(\textbf{a,d}\,\textbf{r})]-\mathbb{E }_{q}[\text{log}q(\textbf{a,d}|v)]-\mathbb{E}_{q}[\text{log}(\frac{p(\textbf{ a,d}\,\textbf{l}\,\textbf{r})}{q(\textbf{a}|\alpha)q(\textbf{d}|\theta)})]\\ &=\mathcal{L}(v)+\text{KL}(q(\textbf{a,d})||p(\textbf{a,d}\, \textbf{l}\,\textbf{r}))\end{split}  \tag{5}\]  
of which  
\[\mathcal{L}(v)=\mathbb{E}_{q}[\text{log}p(\textbf{a,d}\,\textbf{r})]-\mathbb{ E}_{q}[\text{log}q(\textbf{a,d}|v)] \tag{6}\]  
Using the fully factored family, we assume that each user and its transmitted signals are independent  
\[q(\textbf{d,}\textbf{a}|v)=q(\textbf{d}|\lambda)\prod q(a_{i}|\phi_{i}) \tag{7}\]  
where \(v\) is the variational parameter, \(\lambda\) is the global variational variable, \(\phi_{i}\) is the local variational variable of user \(i\), and therefore \(v=\{\lambda,\phi\}\). This kind of simplification method is also called the mean-field method, and offers several computational advantages, especially when deriving the gradients of the objective function. VI optimizes Equation (6) using a gradient or coordinate ascent. To best determine the local optimal, we use tempered variational inference. Tempered variational inference applies tempering into mean-field variational inference. We first use an additional temperature parameter \(T\geq 1\). Given \(T\), the joint probability becomes:  
\[p(\textbf{d,a,r}|T)=\frac{p(\textbf{d,r}\,\textbf{l}\,\textbf{a})^{1/T}p( \textbf{a}|\alpha)}{c(T)}. \tag{8}\]  
where \(c(T)\) is the normalizing constant, or the tempered partition function:  
\[c(T)=\int p(\textbf{d,r}\,\textbf{l}\,\textbf{a})^{1/T}p(\textbf{a}|\alpha) \text{drd}\textbf{add}. \tag{9}\]  
The tempered joint means the tempered posterior. Tempered variational inference optimizes the variational distribution \(q(\dot{)}\) against a tempered posterior. We start from a higher temperature and end when \(T=1\).