# LDA-Based Topic Modeling Sentiment Analysis Using Topic/Document/Sentence (TDS) Model
## 3 Proposed Method
### Topic Document Sentence (TDS) Model  
The existing LDA framework represents three hierarchical layers, where topics are associated with documents and words are associated with topics. However, many researchers [21] have implemented a joint sentiment topic (JST) unsupervised ML model by adding an additional layer to the LDA model. More specifically, sentiment labels of JST are associated with documents, topics are associated with sentiment labels, and words are associated with both sentiment labels and topics of the entire corpus. The difference between TDS and LDA is that LDA supports three layers for processing, while TDS has one additional sentiment label layer for giving higher classified sentiment. It is important  
Figure 2: Latent Dirichlet Allocation model. to note that, other than the one additional layer, the TDS model is similar to the JST model. However, what distinguishes the TDS model from the JST model is the implementation of sentiment label analysis in three parts of sentiment analysis sections, such as topic sentiment analysis, document sentiment analysis, and word sentiment analysis. Let us assume that we have a corpus with a collection of \(\mathbf{D}\) documents denoted by \(\mathbf{D}=\{d_{1}d_{2}d_{3},\ldots d_{M}\}\), and the corresponding vocabulary of the document collection is denoted by \(\mathbf{V}=\{w_{1}w_{2}w_{3},\ldots w_{V}\}\). Each document in the corpus is a sequence with \(N_{d}\) words denoted by \(\mathbf{d}=\{w_{d1}w_{d2}w_{d3},\ldots w_{N_{d}}\}\), where \(d\) is the document number of the collection \(d_{D}\), \(V\) is the number of words, and \(w_{N_{d}}\) is the nth word in document \(\mathbf{d}\), \(w_{N_{d}}\in V\). Assume that \(\mathbf{S}\) is the number of distinct sentiment labels, and \(\mathbf{T}\) is the total number of topics. The generative process shows in Figure 2(a) representing of the original JST model [35]. Like the JST model, the proposed TDS model development parameters are as follows:  
For every document \(\mathbf{d}\), choose a distribution \(\pi_{d}\sim Dir(\gamma)\). For each sentiment label \(\mathbf{l}\) under document \(\mathbf{d}\), choose a distribution \(\theta_{d,l}\sim Dir(\alpha)\). For each word \(\omega_{i}\) in document \(\mathbf{d}\)  
* choose a sentiment label \(l_{i}\sim\pi_{d}\),
* choose a topic \(z_{i}\sim\theta_{d,l_{i}}\),
* choose a word \(\omega_{i}\) from the distribution over words defined by the topic \(z_{i}\) and sentiment label \(l_{i},\ \varphi_{i}^{l_{i}}\). The JST model has two determinant parameters, \(\alpha\) and \(\beta\), for the document collection layer. In this example, \(\alpha\) is a prior parameter of topic distribution in the document collection, and \(\beta\) is the word distribution of every latent topic. Sentiment label \(\mathbf{l}\) is associated with the number of topics, and parameter \(\gamma\) can be integrated into \(\mathbf{l}\) in corpus observations. The JST consists of three sets of latent variables:  
* JST with document distribution--\(\theta\),
* JST with word distribution--\(\varphi\),
* Sentiment document distribution--\(\pi\). As mentioned in [18], \(\pi\) plays an important role in identifying the document polarity. In our model implementation, \(\pi\) is the main change. It is worth noting that sentiment document distribution is applied to the total number of topics, \(\mathbf{T}\) stage. Formula implementation of TDS model with joint sentiment. \[P(w,z,l)=P(w|z,l)P(z,l)=P(w|z,l)*P(z|l,d)P(l|d) \tag{2}\]  
Figure 3: Graphical models of JST and TDS.