# BiTTM: A Core Biterms-Based Topic Model for Targeted Analysis
## 3 BiTTM
### Inference  
Following BTM [2] and TTM [1], we choose Gibbs Sampling [34] to infer the model parameters. All notations used in this section are shown in Table 2.  
We first sample the status of every biterm. Intuitively, if a biterm contains a query keyword, then it is relevant with the target aspect. Let \(d\) be a binary variable and \(d=1\) indicates a biterm contains the keyword provided by users. Then, we define the probability that a biterm is relevant as \(P(r|x=d,\beta^{r},\beta^{ir},\gamma)=1\) if \(d=1\). Otherwise, we define the probability as shown below:  
\[P(r|x=d,\beta^{r},\beta^{ir},\gamma)= \tag{1}\]  
\[\left\{ \begin{array}{ll}1&d=1\\ \frac{n^{r}_{-i}+\gamma}{|B|+2-1}*\\ \frac{\prod_{\text{l=1,r}}\Gamma(\beta^{r}_{-i}n^{r}_{-i,m}+\beta^{r}_{+m} \delta+\epsilon+1)}{\Gamma(\sum_{\text{l=1,r}}^{m}(\beta^{r}_{+m},\omega+1)+| \beta^{r}_{+}|+|W|\epsilon)}&d=0,r=relevant\\ \frac{n^{r}_{-i}+\gamma}{|B|+2\gamma-1}*\\ \frac{\prod_{\text{l=1,r}}\Gamma(n^{r}_{-i,m}+\beta^{ir}_{+m})}{\Gamma(\sum_{ \text{l=1,r}}^{m}n^{r}_{-i,m}+1|\beta^{r}_{+}|)}&d=0,r= irrelevant\\ \end{array} \right.\]  
Next, we sample word selector \(\beta^{r}_{w}\) for all words \(w\in W\). Applying Gibbs Sampling similar to TTM [1], we can obtain the equation \(P(\beta^{r}_{w}|\beta^{r}_{-w},w,\delta,\epsilon,p,q|)\propto P(\beta^{r},w| \delta,\epsilon,p,q)\). Then,  
\[P(\beta^{r}_{w}|\beta^{r}_{-w},\delta,\epsilon,p,q|)\propto\iint P(\beta^{r}, w,\omega,\phi|\delta,\epsilon,p,q)dwd\phi \tag{2}\]  
\[\propto\left\{ \begin{array}{ll}\Gamma(n^{r}_{+}+\delta+\epsilon)*\Gamma(| \beta^{r}_{-w},|\delta+|W|\epsilon+n^{r}_{-w,\alpha})*\\ \Gamma(|\beta^{r}_{-w,\alpha}|\delta+\delta+|W|\epsilon)*(p+|\beta^{r}_{-w, \alpha}|)&\beta^{r}_{w}=1\\ \Gamma(\delta+\epsilon)*\Gamma(|\beta^{r}_{-w,\alpha}|\delta+\delta+|W| \epsilon+n^{r}_{-w,\alpha})*\\ \Gamma(|\beta^{r}_{-w,\alpha}|\delta+|W|\epsilon)*(q+|W|-|\beta^{r}_{-w, \alpha}|-1)&\beta^{r}_{w}=0\\ \end{array} \right.\]  
For a biterm \(b_{i}(w_{i,1},w_{i,2})\), the probability of sampling \(k\) as the topic for \(w_{i,m}\) can be computed as Equation (3). \[P(z_{i,m}=k|\alpha,\beta^{r},\delta^{ir},\delta,\epsilon)\propto \tag{3}\]  
\[\left\{ \begin{array}{ll}P(z_{i,m}=k|\alpha,\beta^{r},\delta,\epsilon)&r_{b _{i}}=\text{relevant}\\ P(z_{i,m}=k|\alpha,\beta^{ir})&r_{b_{i}}=\text{irrelevant}\\ \end{array} \right.\]As mentioned before, two words in a bitterm may be assigned different topics if the bitterm is relevant. Therefore, we can have  
\[P(z_{i,m}=k|\alpha,\beta^{r},\delta,\epsilon)\varpropto \tag{4}\]  
\[\int P(z_{i,m}=k|\theta)P(\theta|\alpha)d\theta\int P(w_{i,m}|\phi_{k})P(\phi_{ k}|\beta^{r}_{k}\delta+\epsilon)d\phi\]  
\[\varpropto\frac{n^{r,-im}_{s|k}+\alpha}{\sum_{k}(n^{r,-im}_{s|k}+\alpha)} \frac{\beta^{r}_{w_{i,m}|k}n^{r,-im}_{w_{i,m}|k}+\beta^{r}_{w_{i,m}|k}\delta+\epsilon} {\sum_{k}(n^{r,-im}_{s|k}+\alpha)\sum_{k}(\beta^{r}_{w_{i,m}|k}n^{r,-im}_{w_{i,m }|k}+\beta^{r}_{w_{i,m}|k}\delta+\epsilon)}\]  
\[\varpropto\frac{n^{r,-im}_{s|k}+\alpha}{\sum_{k}n^{r,-im}_{s|k}+\alpha}\frac{ \beta^{r}_{w_{i,m}|k}n^{r,-im}_{w_{i,m}|k}+\beta^{r}_{w_{i,m}|k}\delta+\epsilon} {\sum_{k}n^{r,-im}_{s|k}+\alpha}\frac{n^{r,-im}_{s|k}+\beta^{r}_{w_{i,m}|k} \delta+|W|\epsilon}\]  
If a bitterm \(b_{i}\) is irrelevant (i.e., \(r_{b_{i}}=0\)), then we directly sample a topic from \(\phi^{ir}\) for two words \(w_{j,m}\). Therefore, we can obtain the conditional probability:  
\[P(z_{i,m}=k|\alpha,\beta^{r},\beta^{ir},\delta,\epsilon)\varpropto \tag{5}\]  
\[\left\{ \begin{array}{ll}\frac{n^{r,-im}_{s|k}+\alpha}{\sum_{k}n^{r,-im}_{s|k} +Ka}&\frac{\beta^{r}_{w_{i,m}|k}n^{r,-im}_{w_{i,m}|k}+\beta^{r}_{w_{i,m}|k} \delta+\epsilon}{\sum_{k}n^{r,-im}_{s|k}+Ka}&r_{b_{i}}=1\\ \frac{n^{r,-im}_{s|k}+\alpha}{\sum_{k}n^{r,-im}_{s|k}+Ka}&\frac{(n^{r,-im}_{s|k }+\beta^{r})(n^{r,-im}_{w_{i,m}|k}+\beta^{r})}{(n^{r,-im}_{s|k}+|W|\beta^{r} )(n^{r,-im}_{s|k}+|W|\beta^{ir}+1)}&r_{b_{i}}=0\end{array} \right.\]  
At last, we sample a word selector \(\beta^{r}_{w|k}\) for a topic \(k\), where \(w\in W\) and \(k\in K\).