# Topic Word Embedding-Based Methods for Automatically Extracting Main Aspects from Product Reviews
## 2 Background
### Related Work
used the dependency parsing to find all possible aspect candidates from review documents [23]. In recent years, learning-based approaches have been widely used. [PERSON] et al. proposed a semi-supervised learning method based on Expectation-Maximization (EM) with sharing words (e.g., battery, battery life, and battery power) and lexical similarity based on WordNet to label the aspect of the extracted word [24]. In addition, [PERSON] et al. proposed a multi-aspect sentence labelling method in which each sentence in a review document is labelled to an aspect. The authors modified the existing topic model with minimal seed words to encourage a correspondence between topics and aspects, and then predict overall multi-aspect rating scores [25]. To analyze people's opinions and sentiments toward certain aspects, References [26; 27; 28] attempted to learn emotion and knowledge information based on a senti Long Short-Term Memory (LSTM)-based model that extends LSTM with a hierarchical attention mechanism. Unlike their approaches, the proposed method focuses on finding main aspects automatically using clean topics. To improve the performance of text classification, Reference [29] proposed a capsule network using dynamic routing that can learn hierarchically. Because the capsule network integrates multiple sub layers layer, text classification is improved from single label text classification to multi-label one. By and large, most recent aspect extraction methods are grouped to three categories: (1) Unsupervised methods, (2) Supervised methods, and (3) Semi-supervised methods. The unsupervised methods are based on Dependency Parsing [30], Association Rule Mining [31], Mutual/ Association [32], Hierarchy [33], Ontology [34], Generative Model [35; 36], Co-occurrence [37], Rule-based [38], and Clustering [39]. The supervised methods are based on Association Rule Mining [40], Hierarchy with Co-occurrence [41], Co-occurrence [42], Classification [43], Fuzzy Rule [44], Rule-based [45], and Conditional Random Fields [46]. The semi-supervised methods are based on Association Rule Mining [47], Clustering [48], and Topic Model [49]. Rather than traditional topic models, [PERSON] [50] used only word2 vec to find topics. All vocabularies are mapped to word embedding spaces and clustered to topics, each of which is a group of relevant words in the corpus. In contrast, the proposed method basically uses some traditional topic model because the traditional topic models are likely to work well to obtain brief summary from large documents. In the approach, a topic model is first applied to find topics and then main aspects are exploited from the clean topics. To clean the topics, various word embedding methods such as word2 vec, fastText, GloVe, ELMo, and BERT are used. Reference [51] extracted some topics from the embedding space using only word-level embedding, while the proposed method takes advantage of both word-level and sentence-level embedding to detect main topics from a given large corpus. All existing approaches based on any topic model [25; 35; 36; 49] are quite different from ours in that the Bayesian network of the original topic model is modified and prior knowledge such as pre-defined aspects [25], explicit aspects [35], opinion bearing words [36], aspect dependent sentiment lexicon [52], and cannot/must-links [49] are used. Alternatively, we do not change any existing topic model directly and do not use any prior knowledge like opinion words. In addition, to enforce a direct correspondence between aspects and topics, we propose optimal starting points for learning, topic cleaning, and aspect labelling based on topic structures.