# BiTTM: A Core Biterms-Based Topic Model for Targeted Analysis
## 3 BiTTM
### Model Description & Generative Process  
In this subsection, we describe the model and the generative process of BiTTM. Table 2 lists the notations used in this paper. The generative process is as follows:  
1. Draw \(\theta\sim Dirichlet(\alpha)\). 2. Draw \(\phi^{ir}\sim Dirichlet(\beta^{ir})\). 3. For each target-relevant topic \(k\in\{1,2,\cdots,K\}\) 1. Draw \({}_{k}\sim Beta(p,q)\). 2. For each word \(w\in\{1,2,\cdots,W\}\) 2. Draw \(\beta^{r}_{k,w}\sim Bernoulli(\omega_{k})\). 3. Draw \(\phi^{r}_{k}\sim Dirichlet(\beta^{r}_{k}\delta\ +\epsilon)\). 4. For each biterm \(b_{i}(w_{i,1},w_{i2})\in B\) 1. Draw \(\pi_{b}\sim Beta(\gamma)\). 2. Compute \(r\) based on \(x\) and \(Bernoulli(\pi_{b})\). 3. If \(b_{i}\) is relevant to the target 1. Draw \(z_{1},z_{2}\sim Multinomial(\theta)\). 2. Draw \(w_{i,1}\sim Multinomial(\phi^{r}_{z_{1}})\), and 3. Draw \(w_{i,2}\sim Multinomial(\phi^{r}_{z_{2}})\). 4. If \(b_{i}\) is irrelevant 1. Draw \(w_{i,1},w_{i,2}\sim Multinomial(\phi^{ir})\). Graphical representation of BiTTM is shown in Figure 3. Following the above procedure, the generative process can be summarised into three parts. Firstly, we draw two global parameters \(\theta\) and \(\phi^{ir}\). The former is a topic distribution which models on the entire corpus instead of one document, and the latter is a topic-word distribution of irrelevant topic. In other words, two words in an irrelevant biterms are drawn from only one irrelevant vant topic. Secondly, \(\phi^{r}_{k}\) is drawn for each target-relevant topic \(k\in\{1,2,\cdots,K\}\). Please note that two smoothing parameters, smoothing prior \(\delta\) and weak smoothing prior \(\epsilon\), are used for dual-sparsity [25]. Thirdly, status \(r\) of \(b_{i}\) is determined by both target indicator \(x\) and \(Bernoulli(\tau_{b})\). According to two different types of status, relevant or irrelevant, we draw a word from \(\phi^{r}\) or \(\phi^{ir}\). \begin{table}
\begin{tabular}{l l} \hline \hline
**Notation** & **Meaning** \\ \hline \(B\) & the set of (core) biterms \\ \(W\) & the set of words \\ \(D\) & the set of documents \\ \(\pi_{b}\) & the bernoulli distribution over biterm \(b\) \\ \(\gamma_{r},\beta^{ir},\alpha\) & beta prior of \(\pi_{b}\), Dirichlet prior of \(\phi^{ir},\theta\) \\ \(\phi^{ir}\) & topic-word distribution over the irrelevant topic \\ \(\phi^{r}_{k}\) & topic-word distribution over the \(k\)th relevant topic \\ \(p,q\) & beta prior of \(\omega\) \\ \(\delta,\epsilon\) & word smoothing prior, weak word smoothing prior \\ \(\omega_{k}\) & bernoulli distribution of word selector \(\beta^{r}_{k}\) \\ \(x,w,z,r\) & target indicator, word, topic, status \\ \(\beta^{r}_{w|k},\beta^{r}_{*|k}\) & word selector of word \(w\) under topic \(k\), the sum of word selector \(\beta^{r}_{w|k}\). \\ \(\beta^{r}_{w},\beta^{r}_{*}\) & word selector of word \(w\), the sum of word selector \(\beta^{r}_{w}\) \\ \(\beta^{r}_{-w,*}\) & the sum of word selector except the word \(w\) \\ \(n^{r}_{-i,w},n^{i}_{-i,w}\) & the number of times word \(w\) is relevant (irrelevant) excluding biterm \(b_{i}\) \\ \(n^{r}_{-i},n^{r}_{-i}\) & the number of relevant (irrelevant) biterms excluding biterm \(b_{i}\). \\ \(w_{i,m}\) & the \(m\)th word in biterm \(b_{i}\), where \(m=1,2\) \\ \(n^{r}_{w}\) & the number of times that word \(w_{i,m}\) assigned to topic \(k\) under relevant status excluding \(w_{i,m}\) \\ \(n^{r}_{*|k}\) & the total number of words assigned to topic \(k\) under relevant status excluding \(w_{i,m}\) \\ \(n^{i}_{w_{i,m}|k}\) & the number of times that word \(w_{i,m}\) assigned to topic \(k\) under irrelevant status excluding biterm \(b_{i}\) \\ \(n^{i}_{*|k}\) & the total number of words assigned to topic \(k\) under irrelevant status excluding biterm \(b_{i}\) \\ \(n^{r}_{w}\) & the number of times that word \(w\) is relevant \\ \(n^{r}_{-w,*}\) & the total number of words that are relevant excluding word \(w\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Symbol description. Figure 3: The graphical model of BiTTM.