# Variational Inference for a Recommendation System in IoT Networks Based on [PERSON]'s Identity
## 3 Svfs  
The standard procedure of the recommendation system can be seen in Figure 1. To begin with, we pre-train a set of user-item scores. Define \(u_{i}\) to be the feature vector of user \(i\), the value of which is in \(\mathcal{U}\subset\mathcal{R}^{d_{u}}\); then, \(v_{j}\) the feature vector of item \(j\), the value of which is in \(\mathcal{V}\subset\mathcal{R}^{d_{u}}\). The number of users is \(|U|=N\), and the number of items is \(|V|=M\). Denote the rating matrix as \(L\). Therefore, we reach the fundamental objective, i.e., to calculate \(p(U,V|L)\). Note that the matrix factorization way decomposes \(L\) to obtain \(U,V\). After this, we use \(U\) and \(I\), the ground truth of user and item feature, to further predict ratings that users have not given. Specifically, the objective of SVRS is to obtain the \(U\) and \(V\) that can maximize the likelihood, or minimize the Stein discrepancy. The user feature density is  
\[p(u):=\bar{p}(u)/Z, \tag{7}\]  
of which \(\bar{p}(u):=p_{0}(u)\prod_{i=1}^{N}p(L_{i}|u)\). \(Z=\int\bar{p}(u)\mathrm{d}u\) is the normalization. Notice that \(U\) and \(V\) are calculated alternatively, and that, in order to save space, we may skip the derivation of \(V\) (see Figure 2 for the procedure of SVRS). Figure 1: Procedure of the recommendation system. Traditionally, \(p(u)\) is maximized by maximizing the function that has the same monotonicity with joint probability, a highly-demanding process to apply directly in VI. To calculate the log joint density, we can instead compute the evidence lower bound (ELBO), or the metrics that measure the difference (or distance) of \(p(L|u)\) with \(q(u)\)--of which \(q(u)\) is the approximate variational distribution [28]:  
\[ \begin{split}\log(p(L))&=\int q(u)\text{log}(p(L,u ))\text{d}L-\int q(u)\text{log}(q(u))\text{d}L+\\ &(-\int q(u)\text{log}(\frac{p(L|u)}{q(u)})\text{d}L)\\ &=\mathcal{L}(u)+\text{KL}(q||p),\end{split}  \tag{8}\]  
where \(\text{KL}(q||p)=-\int q(u)\text{log}(\frac{p(L|u)}{q(u)})\text{d}L\), \(q(u)=\prod_{i=q}^{N}q_{i}(u_{i})\). Thus, two methods can be used to obtain the target distribution that can maximize the log likelihood of \(L\), maximizing ELBO or minimizing \(\text{KL}(q||p)\). We choose the second one. In order to minimize the KL divergence, we need to differentiate it with respect to \(q(u)\). This is not easy. Thus, we have to set \(q(u)\) to simple forms, or use MCMC sampling instead. In this work, we choose the first popular way. SVRS requires some assumptions before we can proceed. To begin with, we assume the rating noise is Gaussian,  
\[ \begin{split} p(L_{ij}|U_{i},V_{j})&=\mathcal{N}(U_ {i}V_{j},\delta^{2})\\ &=\log\frac{1}{\sqrt{2\pi}\delta}\text{exp}(-\frac{L_{ij}-U_{i}V_ {j}}{2\delta^{2}}).\end{split}  \tag{9}\]  
Since we use \(q(u)\) and \(q(v)\) to replace \(p(L|u,v)\), we need to first define which form of \(q(u)\) / \(q(v)\) is best-suited. Clearly, the optimal \(q(u)\) and \(q(v)\) have the smallest divergence with \(p(L|u,v)\). Thus, the best \(q(u)\) and \(q(v)\) have the following form:  
\[ \begin{split} q^{*}(u)&=\text{arg min}_{q\in \mathcal{Q}_{u}}\text{KL}(q||p)\\ &=\mathbb{E}_{q}[\text{log}q(u)]-\mathbb{E}_{q}[\text{log}\beta( u)]+const.,\\ q^{*}(v)&=\text{arg min}_{q\in\mathcal{Q}_{v}}\text {KL}(q||p)\\ &=\mathbb{E}_{q}[\text{log}q(v)]-\mathbb{E}_{q}[\text{log}\beta( v)]+const.,\end{split}  \tag{10}\]  
of which \(\mathcal{Q}_{u}\) is \(q(u)\)'s space, and \(\mathcal{Q}_{v}\) is \(q(v)\)'s space. Intuitively, the best \(\mathcal{Q}\) possesses the feature of accuracy and tractability. Therefore, it is rational to choose linear \(\mathcal{Q}\) that is  
Figure 2: Schematic of SVRS. (Training data is Douban Movie, left picture of Item Feature is block-buster movie in China). continuous and differentiable (in another perspective described in other papers, this is the trick of reparameterization) [29]. In this way, using  
\[T(u)=u+\rho\phi(u), \tag{12}\]  
we can simplify the computing and differentiating process.