# A Bayesian Modeling Approach to Situated Design of Personalized Soundscaping Algorithms
## 2 Methods
### Factor Graphs and Message Passing-Based Inference
#### 2.3.3 Variational Message Passing  
In some instances, the integrals in the sum-product update rule (20) can become intractable. Linear Gaussian models are an example of a class of models in which sum-product messages can be calculated in closed-form expressions. However, in models such as the Algonquin model of (8) the integrals become intractable. In these cases, we can resort to an approximate message passing algorithm, called variational message passing (VMP) [40,41], which gives closed-form expressions for conjugate pairs of distributions from the exponential family. If closed-form expressions with VMP are still not available, we might resort to approximation methods, such as importance sampling or [PERSON]'s method [42].  
Suppose that we are dealing with a generative model \(p(y,z)\) with an intractable posterior distribution \(p(z\ |\ y)\), where \(y\) and \(z\) denote the observed and latent variables, respectively. The goal of variational inference is to approximate the intractable true posterior with a tractable variational distribution \(q(z)\) through minimization of a variational free energy functional  
\[F[q]=\mathrm{D}_{\mathrm{KL}}[q(z)\ ||\ p(z\ |\ y)]-\ln p(y)\,. \tag{21}\]  
where \(\mathrm{D}_{\mathrm{KL}}\) is the Kullback-Leibler divergence and which is, in the machine learning literature, also known as the negative Evidence Lower BOund (ELBO) [43] as it bounds the negative log-evidence \(-\ln p(y)\), because \(\mathrm{D}_{\mathrm{KL}}\geq 0\) for any choice of \(q\). Since the second term of (21) is independent of \(q(z)\), free energy minimization is equivalent to the minimization of the Kullback-Leibler divergence. Furthermore, the variational free energy can be used as an approximation to the negative log-evidence for techniques such as Bayesian model selection, Bayesian model averaging [44] and Bayesian model combination [45].  
In practice, the optimization of (21) is performed by imposing additional constraints on \(q(z)\), e.g., by limiting \(q(z)\) to a family of distributions, or by additional factorization assumptions (e.g., \(q(z)=\prod_{i}q_{i}(z_{i})\) which is known as the mean-field assumption). Depending on the constraints on \(q(z)\), the minimization of (21) can be achieved through  
Figure 7: Forney-style factor graph of (17) with sum-product messages as indicated in (19) for the calculation of the marginal distribution of \(x_{4}\).  
sum-product message passing or variants of VMP. In the latter case, the goal is to iteratively update the variational distributions through coordinate descent on (21). In general, the variational message \(v(x_{j})\) from a generic node \(f(x_{1},\ x_{2},\ldots,x_{n})\) with incoming marginals \(q(x_{\setminus j})\) (see Figure 8) can be written as [41]  
\[\vec{v}(x_{j})\propto\exp\int\prod_{i\in\{1,\ldots,n\}\setminus j}q(x_{i})\ln f (x_{1},\ x_{2},\ldots,x_{n})\ \mathrm{d}x_{\setminus j}. \tag{22}\]  
Given these messages, the variational distributions can be updated through the multiplication of the forward and backward message on that respective edge as  
\[q(x_{j})\propto\vec{v}(x_{j})\overleftarrow{v}(x_{j}). \tag{23}\]