# Topic Word Embedding-Based Methods for Automatically Extracting Main Aspects from Product Reviews
## 3 The Proposed Approaches
### Automatic Generation of Main Aspects from Clean Topics
Given \(\theta_{i}\) and a sentence \(s_{i}\), the probability value that \(s_{i}\) is relevant with \(\theta_{i}\) is first computed by Equation (7) and then only top-k sentences with the highest \(s^{*}\) value are selected from a total of sentences in the collection. \[s^{*}=argmax_{s_{i}}P(s_{i}|\theta_{i}) \tag{1}\] \[=argmax_{s_{i}}\frac{P(\theta_{i}|s_{i})P(s_{i})}{P(\theta_{i})}\] (2) \[=argmax_{s_{i}}\frac{P(s_{i})}{P(\theta_{i})}P(\theta_{i}|s_{i})\] (3) \[=argmax_{s_{i}}Z(\theta_{i})\prod_{w_{i}\in\theta_{i}}P(w_{i}|s_{i})\] (4) \[=argmax_{s_{i}}Z(\theta_{i})\prod_{w_{i}\in\theta_{i}}P_{s_{i}}(w _{i}|\theta_{i})\] (5) \[=argmax_{s_{1}}\prod_{w_{i}\in\theta_{i}}logP_{s_{i}}(w_{i}|\theta _{i})\] (6) \[=argmax_{s_{i}}\sum_{w_{i}\in\theta_{i}}logP_{s_{i}}(w_{i}|\theta _{i}), \tag{7}\]  
where \(P_{s_{i}}(w_{i}|\theta_{i})=\frac{P(w_{i})}{\sum_{w\in\theta_{i}\cap k\text{ on}(s_{i})P(w_{i})}}\), where \(P(w_{i})\) is a probability of the \(i\)-th topic word computed by a topic model and \(P(w)\) is one probability of topic words appearing in both the topic \(\theta_{i}\) and the \(i\)-th sentence \(s_{i}\). By [PERSON] theorem, Equation (1) is converted to Equation (2). The \(\frac{P(s_{i})}{P(\theta_{i})}\) in Equation (3) is the normalized term so it can replaced by \(Z(\theta_{i})\). Since \(Z(\theta_{i})\) is treated as a constant, it can be omitted because it does not significantly affect the results. If we replace \(P(\theta_{i}|s_{i})\) in Equation (3) with the term for each word, it is represented as \(\prod_{w_{i}\in\theta_{i}}P_{s_{i}}(w_{i})\) in Equation (5). Finally, we obtain Equation (7) by taking a log on Equation (5). Then, for \(s_{i}\in\) top-\(k\) sentences \(\{s_{1},\,\dots,\,s_{k}\}\), the dependency parsing is carried out to find head words from \(s_{i}\). In the dependency parsing, there exist governor and dependent words in a given sentence. Let us assume one sentence like \"I saw a beautiful flower on the road\". In this case, the word 'beautiful' is the dependent and the word 'flower' is the governor. Throughout this article, we call governors head words for convenience. If some head words appear frequently cross top-\(k\) sentences, such words can be considered to be key words representing the topic \(\theta_{i}\). Then, such head words \(w_{1}^{H}\), \(w_{2}^{H}\),...are converted to the vectors \(v_{w_{1}^{H}}\), \(v_{w_{2}^{H}}\),...through the domain-customized word embedding model. The next step is similar to the first method. \(v_{*}\) is computed by averaging element-wise sum of the vectors. Finally, a vocabulary \(w_{*}\) close to \(v_{*}\) is chosen in the domain-customized word embeddingmodel, where \(w_{*}\) is considered as an aspect describing \(\theta_{i}\) to the best. Algorithm 4 is the pseudo code for the second aspect auto-generation method. ```
Input: A topic \(\theta_{i}=\{w_{1}:P(w_{1}),\ldots,w_{l}:P(w_{l})\}\) and a set of sentences in a collection; Output: A topic label \(w_{*}\); for(\(s_{i}\in\text{ A set of sentences in the collection}\)) do \(s_{i}^{*}=\sum_{w_{i}\in\theta_{i}}log(\frac{P(w_{i})}{\sum_{w\in\theta_{i} \cap w_{\text{dom}}(s_{i})P(w)}})\);  end for HeadWords = \(\phi\); for(\(s_{i}\in\text{ Top-}k\) sentences with the highest \(s_{i}^{*}\)) do  HeadWords = HeadWords \(\cup\) DependencyParser(\(s_{i}\));  end for(\(w_{i}\in\text{ Frequently-used head words}\)) do \(f_{w_{i}}(w_{i})=v_{w_{i}}\);  end for\(v_{*}=\frac{v_{w_{1}}+\ldots+v_{w_{l}}}{|\theta_{i}|}\); \(w_{*}=f_{w_{0}}^{-1}(v_{*})\);
```  
**Algorithm 4**The second aspect auto-generation method  
The third method is a hybrid of the first and second methods. In other words, the third method is working based on the consideration of both word-level approach (from the first method) and sentence-level approach (from the second method). Technically, it selects both top-\(k\) topic words from the first method and top-\(k\) head words from the second method. Then, the words are converted to the vectors and \(v_{*}\) is computed by averaging element-wise sum of the vectors.