# Event-Based Probabilistic Embedding for POI Recommendation
## 3 Model Inference and Learning  
Our goal with model inference is to learn the parameters that maximize the likelihood of observed random variables and minimize the loss function of neural network. TARE's optimization goal can be expressed as follows. \[\mathcal{J}=\mathcal{J}_{PGM}+\mathcal{J}_{neural} \tag{10}\]  
where \(\mathcal{J}_{PGM}\) is the optimization target of event embedding and \(\mathcal{J}_{neural}\) is the loss function of the neural network. The \(\mathcal{J}_{PGM}\) is defined as follows. \[\mathcal{J}_{PGM} \tag{11}\] \[=-P(\vec{z},\vec{r},\vec{w},\vec{l},\vec{v}|\vec{u},\theta_{embedding})\] \[=-P(\vec{z}|\vec{u},\theta_{z})P(\vec{r}|\vec{u},\theta_{r})P( \vec{l}|\vec{r},\vec{z},\theta_{t})P(\vec{v}|\vec{r},\vec{z},\theta_{v})P( \vec{w}|\vec{z},\theta_{w})P(\vec{l}|\vec{u},\vec{z})\] \[=-\prod_{u=1}^{N}\prod_{i=1}^{|\mathcal{D}_{u}|}\prod_{w\in W_{w,i}}\theta_{w|z_{u}|}^{(w)}\theta_{z|u}^{(z_{u},i)}\theta_{r|u}^{(t_{u,i})} \theta_{|t|r_{u},z_{u},t_{q}}^{(t_{u,i})}\theta_{|r|u_{u},z_{u},t_{q}}^{(t_{u, i})}\theta_{|r|u_{u},z_{u},t_{q}}^{(t_{u,i})}p(l_{v_{u},t_{q}}|\mu_{r_{u,i}} \Sigma_{r_{u,i}})\]  
The loss function \(\mathcal{J}_{neural}\) of the neural network is defined as follows. \[\mathcal{J}_{neural}=\sum_{u=1}^{N}\sum_{i=1}^{|\mathcal{D}_{u}|}\left\|y_{u,v_{u},t_{u},t_{u},t_{q}}-\mathcal{H}(E_{u,v_{u},t_{u},t_{u},t_{q}})\right\| +\lambda\left\|W\right\|_{F}^{2} \tag{12}\]  
where \(D_{u}\) represents the set of \(u\)'s check-in activities, and \(|D_{u}|\) represents the number of \(u\)'s check-in activities. We use L2 regularization to prevent overfitting. However, it is difficult to minimize \(\mathcal{J}\). Therefore, we apply the Gibbs EM algorithm [21], which is a mixture between EM and a Monte Carlo sampler, to minimize \(\mathcal{J}\). In the E-step, we sample regions and topics by using Gibbs sampling. In the M-step, we optimize the model parameters \(\theta_{neural}\) and \(\theta_{PGM}\) by fixing all topic and region assignments. The two steps are conducted repeatly until convergence. More specifically, we draw region \(r\) and topic \(z\) for each check-in activity in the E-step. We use Gibbs sampling to sample region \(r\) and topic \(z\) according to Equations (13) and (14). When we sample region \(r\), we assume all other variables are fixed. \(r_{u,i}\) represents the region assignments for all user activities except the \(i\)th activity of user \(u\), and the sampling topic \(z\) is similar to the sampling region \(r\). \[P(z_{u,i}|z_{-u,i},\vec{u},\vec{v},\vec{w},\vec{l},\vec{l}) \tag{13}\] \[\propto\sum_{r=1}^{R}\theta_{z|u}^{(z_{u,i})}*\theta_{r|u}^{(r)}* \theta_{|t|r,z_{u}|}^{(t_{u,i})}*\theta_{v|r,z_{u}|}^{(\theta_{v,i})}*p(l_{v_{ u,i}}|\mu_{r},\Sigma_{r})*\prod_{w\in W_{z}}\theta_{w|z_{u,i}}^{w|}\]\[ \begin{split}& P(r_{u,i}|r_{-u,i},\vec{u},\vec{w},\vec{l},\vec{t})\\ &\propto\sum_{z=1}^{Z}\theta_{z|u}^{(z)}*\theta_{r|u}^{(r_{u,i})}* \theta_{t|r_{u,j},z}^{(u_{u,i})}*\theta_{v|r_{u,j},z}^{(u_{u,i})}*p(l_{v_{u,i} }|u_{r_{u,j}},\Sigma_{r_{u,j}})*\prod_{w_{i}\in W_{v}}\theta_{w|z}^{w_{i}}\end{split}  \tag{14}\]  
In the M-step, we optimize the parameters \(\theta_{neural}\) and \(\theta_{PGM}\) to minimize \(\mathcal{J}\) with all topic and region assignments fixed. We use Stochastic Gradient Descent (SGD) to update the parameters \(\theta_{neural}\) and \(\theta_{PGM}\) for minimizing \(\mathcal{J}_{neural}\). There are no negative samples in the training data because this is one-class problem. So, we generate negative samples by negative sampling. For each positive sample, we generate five negative samples by randomly changing one of the elements (\(u\), \(v\), \(t_{q}\) or \(l_{q}\)) in positive samples. Due to space limitations, the gradient of the weight of the neural network is not listed, and the gradients of model parameters \(\theta_{t|r,z}^{t}\), \(\theta_{v|r,z}^{v}\), \(\theta_{r|u}^{r}\), \(\theta_{z|u}^{z}\) and \(\theta_{w|z}^{w}\) are provided as follows. \[