Venue: NAACL 2016

# Clustering for Simultaneous Extraction of

Aspects and Features from Reviews

 Lu Chen1, Justin Martineau2, Doreen Cheng2, Amit Sheth1

1 Kno.e.sis Center, Wright State University

Fairborn, OH 45435 USA

2 Samsung Research America, Silicon Valley

San Jose, CA 95134 USA

{chen, amit}@knoesis.org

justin.m@samsung.com

doreenycheng@yahoo.com

###### Abstract

This paper presents a clustering approach that simultaneously identifies product features and groups them into aspect categories from online reviews. Unlike prior approaches that first extract features and then group them into categories, the proposed approach combines feature and aspect discovery instead of chaining them. In addition, prior work on feature extraction tends to require seed terms and focus on identifying explicit features, while the proposed approach extracts both _explicit_ and _implicit_ features, and does not require seed terms. We evaluate this approach on reviews from three domains. The results show that it outperforms several state-of-the-art methods on both tasks across all three domains.

+
Footnote †: This author’s research was done during an internship with Samsung Research America.

## 1 Introduction

If you are thinking of buying a TV for watching football, you might go to websites such as Amazon to read customer reviews on TV products. However, there are many products and each of them may have hundreds of reviews. It would be helpful to have an aspect-based sentiment summarization for each product. Based on other customers' opinions on multiple aspects such as size, picture quality, motion-smoothing, and sound quality, you might be able to make the decision without going through all the reviews. To support such summarization, it is essential to have an algorithm that extracts product features and aspects from reviews.

_Features_ are components and attributes of a product. A feature can be directly mentioned as an opinion target (i.e., explicit feature) or implied by opinion words (i.e., implicit feature). Different feature expressions may be used to describe the same _aspect_ of a product. Aspect can be represented as a group of features. Consider the following review sentences, in which we denote **explicit** and _implicit_ features in boldface and italics, respectively.

1. This phone has great **display** and perfect **size**. It's very _fast_ with all great **features**.
2. Good **features** for an _inexpensive_ android. The **screen** is _big_ and _vibrant_. Great **speed** makes _smooth_ viewing of tv programs or sports.
3. The phone runs _fast_ and _smooth_, and has great **price**.

In review 1, _display_ is an explicit feature, and opinion word "fast" implies implicit feature _speed_. The task is to identify both explicit and implicit features, and group them into aspects, e.g., {speed, fast, smooth}, {size, big}, {price, inexpensive}.

Many existing studies Hu and Liu (2004); Su et al. (2006); Qiu et al. (2009); Hai et al. (2012); Xu et al. (2013) have focused on extracting features without grouping them into aspects. Some methods have been proposed to group features given that feature expressions have been identified beforehand Zhai et al. (2010); Moghaddam and Ester (2011); Zhao et al. (2014), or can be learned from semi-structured Pros and Cons reviews Guo et al. (2009); Yu et al. (2011). In recent years, topic models have been widely studied for their use in aspect discovery with the advantage that they extract features and group them simultaneously. However, researchers have found some

limitations of such methods, e.g., the produced topics may not be coherent or directly interpretable as aspects [31; 2], the extracted aspects are not fine-grained [40], and it is ineffective when dealing with aspect sparsity [56].

In this paper, we present a clustering algorithm that extracts features and groups them into aspects from product reviews. Our work differs from prior studies in three ways. First, it identifies both features and aspects simultaneously. Existing clustering-based solutions [78; 9, 10] take a two-step approach that first identifies features and then employs standard clustering algorithms (e.g., k-means) to group features into aspects. We propose that these two steps can be combined into a single clustering process, in which different words describing the same aspect can be automatically grouped into one cluster, and features and aspects can be identified at the same time. Second, both explicit and implicit features are extracted and grouped into aspects. While most existing methods deal with explicit features (e.g., "speed", "size"), much less effort has been made to identify implicit features implied by opinion words (e.g., "fast", "big"), which is challenging because many general opinion words such as "good" or "great" do not indicate product features, therefore they should not be identified as features or grouped into aspects. Third, it is unsupervised and does not require seed terms, hand-crafted patterns, or any other labeling efforts.

Specifically, the algorithm takes a set of reviews on a product (e.g., TV, cell phone) as input and produces aspect clusters as output. It first uses a part-of-speech tagger to identify nouns/noun phrases, verbs and adjectives as candidates. Instead of applying the clustering algorithm to all the candidates, only the frequent ones are clustered to generate _seed clusters_, and then the remaining candidates are placed into the closest seed clusters. This does not only speed up the algorithm, but it also reduces the noise that might be introduced by infrequent terms in the clustering process. We propose a novel _domain-specific similarity measure_ incorporating both statistical association and semantic similarity between a pair of candidates, which recognizes features referring to the same aspects in a particular domain. To further improve the quality of clusters, several problem-specific _merging constraints_ are used to prevent the clusters referring to different aspects from being merged during the clustering process. The algorithm stops when it cannot find another pair of clusters satisfying these constraints.

This algorithm is evaluated on reviews from three domains: cell phone, TV and GPS. Its effectiveness is demonstrated through comparison with several state-of-the-art methods on both tasks of feature extraction and aspect discovery. Experimental results show that our method consistently yields better results than these existing methods on both tasks across all the domains.

## 2 Related Work

Feature and aspect extraction is a core component of aspect-based opinion mining systems. Zhang and Liu (2014) provide a broad overview of the tasks and the current state-of-the-art techniques.

Feature identification has been explored in many studies. Most methods focus on explicit features, including unsupervised methods that utilize association rules [12, 13], dependency relations [15, 56], or statistical associations [14] between features and opinion words, and supervised ones that treat it as a sequence labeling problem and apply Hidden Markov Model (HMM) or Conditional Random Fields (CRF) [13, 12] to it. A few methods have been proposed to identify implicit features, e.g., using co-occurrence associations between implicit and explicit features [78; 14, 40], or leveraging lexical relations of words in dictionaries [11]. Many of these techniques require seed terms, hand-crafted rules/patterns, or other annotation efforts.

Some studies have focused on grouping features and assumed that features have been extracted beforehand or can be extracted from semi-structured Pros and Cons reviews. Methods including similarity matching [10], topic modeling [15, 16], Expectation-Maximization (EM) based semi-supervised learning [41, 42], and synonym clustering [54] have 

been explored in this context.

To extract features and aspects at the same time, topic model-based approaches have been explored by a large number of studies in recent years. Standard topic modeling methods such as pLSA [12] and LDA [30] are extended to suit the peculiarities of the problem, e.g., capturing local topics corresponding to ratable aspects [94, 57, 68], jointly extracting both topic/aspect and sentiment [10, 11, 12, 13], incorporating prior knowledge to generate coherent aspects [95, 11, 12], etc.

Very limited research has focused on exploring clustering-based solutions. Su et al. su:2008 presented a clustering method that utilizes the mutual reinforcement associations between features and opinion words. It employs standard clustering algorithms such as k-means to iteratively group feature words and opinion words separately. The similarity between two feature words (or two opinion words) is determined by a linear combination of their intra-similarity and inter-similarity. Intra-similarity is the traditional similarity, and inter-similarity is calculated based on the degree of association between feature words and opinion words. To calculate the inter-similarity, a feature word (or an opinion word) is represented as a vector where each element is the co-occurrence frequency between that word and opinion words (or feature words) in sentences. Then the similarity between two items is calculated by cosine similarity of two vectors. In each iteration, the clustering results of one type of data items (feature words or opinion words) are used to update the pairwise similarity of the other type of items. After clustering, the strongest links between features and opinion words form the aspect groups. Mauge et al. mauge:2012 first trained a maximum-entropy classifier to predict the probability that two feature expressions are synonyms, then construct a graph based on the prediction results and employ greedy agglomerative clustering to partition the graph to clusters. Bancken et al. bancken:2014 used k-medoids clustering algorithm with a WordNet-based similarity metric to cluster semantically similar aspect mentions.

These existing clustering methods take two steps. In the first step, features are extracted based on association rules or dependency patterns, and in the second step features are grouped into aspects using clustering algorithms. In contrast, our method extracts features and groups them at the same time. Moreover, most of these methods extract and group only explicit features, while our method deals with both explicit and implicit features. The method proposed in [12] also handles implicit features (opinion words), but their similarity measure largely depends on co-occurrence between features and opinion words, which may not be efficient in identifying features that are semantically similar but rarely co-occur in reviews.

## 3 The Proposed Approach

Let \(X=\{x_{1},x_{2},...,x_{n}\}\) be a set of candidate features extracted from reviews of a given product (e.g., TV, cell phone). Specifically, by using a part-of-speech tagger1, nouns (e.g., "battery") and two consecutive nouns (e.g., "battery life") are identified as candidates of _explicit_ features, and adjectives and verbs are identified as candidates of _implicit_ features. Stop words are removed from \(X\). The algorithm aims to group similar candidate terms so that the terms referring to the same aspect are put into one cluster. At last, the important aspects are selected from the resulting clusters, and the candidates contained in these aspects are identified as features.

Footnote 1: [http://nlp.stanford.edu/software/tagger.shtml](http://nlp.stanford.edu/software/tagger.shtml)

### A Clustering Framework

Algorithm 1 illustrates the clustering process. The algorithm takes as input a set \(X\) that contains \(n\) candidate terms, a natural number \(k\) indicating the number of aspects, a natural number \(s\) (\(0<s\leq n\)) indicating the number of candidates that will be grouped first to generate the _seed clusters_, and a real number \(\delta\) indicating the upper bound of the distance between two mergeable clusters. Instead of applying the _agglomerative clustering_ to all the candidates, it first selects a set \(X^{\prime}\subseteq X\) of \(s\) candidates that appear most frequently in the corpus for clustering. The reasons for this are two-fold. First, the frequently mentioned terms are more likely the actual features of customers' interests. By clustering these terms first, we can generate high quality seed clusters.

 

Second, as the clustering algorithm requires pair-wise distances between candidates/clusters, it could be very time-consuming if there are a large number of candidates. We can speed up the process by clustering only the most frequent ones.

```
Input: \(X=\{x_{1},...,x_{n}\}\), \(k\), \(s\), \(\delta\) Output: \(\{A_{j}\}^{k}_{j=1}\)
1 Select the top \(s\) most frequent candidates from \(X\): \(X^{\prime}=\{x^{\prime}_{1},...,x^{\prime}_{s}\}\); Set \(C_{1}\leftarrow\{x^{\prime}_{1}\},...,C_{s}\leftarrow\{x^{\prime}_{s}\}\); Set \(\Theta\leftarrow\{C_{1},...,C_{s}\}\); whilethere exist a pair of mergeable clusters from \(\Theta\)do
2 Select a pair of closest clusters \(C_{l}\) and \(C_{m}\) such that violate-constraints(\(C_{l}\), \(C_{m}\), \(\delta\)) is false; \(C_{m}\gets C_{l}\cup C_{m}\); \(\Theta\leftarrow\Theta-\{C_{l}\}\); for\(x_{i}\in X-X^{\prime}\)do
3 Select the closest clusters \(C_{d}\) from \(\Theta\) such that violating-constraints(\(\{x_{i}\}\), \(C_{d}\), \(\delta\)) is false; ifthere exist such cluster \(C_{d}\)then
4\(C_{d}\gets C_{d}\cup\{x_{i}\}\); \(\{A_{j}\}^{k}_{j=1}\leftarrow\textsc{select($\Theta$,$k$)}\);
```

**Algorithm 1**Clustering for Aspect Discovery

The clustering starts with every frequent term \(x^{\prime}_{i}\) in its own cluster \(C_{i}\), and \(\Theta\) is the set of all clusters. In each iteration, a pair of clusters \(C_{l}\) and \(C_{m}\) that are most likely composed of features referring to the same aspect are merged into one. Both a similarity measure and a set of constraints are used to select such pair of clusters. We propose a _domain-specific similarity measure_ that determines how similar the members in two clusters are regarding the particular domain/product. Moreover, we add a set of _merging constraints_ to further ensure that the terms from different aspects would not be merged. The clustering process stops when it cannot find another pair of clusters that satisfy the constraints. We call the obtained clusters in \(\Theta\) the _seed clusters_. Next, the algorithm assigns each of the remaining candidate \(x_{i}\in X-X^{\prime}\) to its closest seed cluster that satisfies the merging constraints. At last, \(k\) clusters are selected from \(\Theta\) as aspects2. Based on the idea that the frequent clusters are usually the important aspects of customers' interests, we select the top \(k\) clusters having the highest sum of members' frequencies of occurrence. From the \(k\) aspects, the nouns and noun phrases (e.g., "speed", "size") are recognized as explicit features, and the adjectives and verbs (e.g., "fast", "big"), are recognized as implicit features.

Footnote 2: If \(k\) is larger than the number of clusters obtained, all the clusters are selected as aspects.

### Domain-specific Similarity

The similarity measure aims to identify terms referring to the same aspect of a product. Prior studies [68, 7] have shown that general semantic similarities learned from thesaurus dictionaries (e.g., WordNet) do not perform well in grouping features, mainly because the similarities between words/phrases are domain dependent. For example, "ice cream sandwich" and "operating system" are not relevant in general, but they refer to the same aspect in _cell phone_ reviews3; "smooth" and "speed" are more similar in _cell phone_ domain than they are in _hair dryer_ domain. Methods based on distributional information in a domain-specific corpus are usually used to determine the domain-dependent similarities between words/phrases. However, relying completely on the corpus may not be sufficient either. For example, people usually use "inexpensive" or "great price" instead of "inexpensive price"; similarly, they use "running fast" or "great speed" instead of "fast speed". Though "inexpensive" and "price" or "fast" and "speed" refer to the same aspect, we may not find they are similar based on their context or co-occurrences in the corpus.

Footnote 3: _Ice cream Sandwich_ is a version of the Android mobile operating system.

We propose to estimate the domain-specific similarities between candidates by incorporating both general semantic similarity and corpus-based statistical association. Formally, let \(G\) be a \(n\times n\) similarity matrix, where \(G_{ij}\) is the _general semantic similarity_ between candidates \(x_{i}\) and \(x_{j}\), \(G_{ij}\in[0,1]\), \(G_{ij}=1\) when \(i=j\), and \(G_{ij}=G_{ji}\). We use UMBC Semantic Similarity Service4 to get \(G\). It combines both WordNet knowledge and statistics 

from a large web corpus to compute the semantic similarity between words/phrases Han et al. (2013).

Let \(T\) be a \(n\times n\) association matrix, where \(T_{ij}\) is the pairwise _statistical association_ between \(x_{i}\) and \(x_{j}\) in the domain-specific corpus, \(T_{ij}\in[0,1]\), \(T_{ij}=1\) when \(i=j\), and \(T_{ij}=T_{ji}\). We use normalized pointwise mutual information (NPMI) Bouma (2009) as the measure of association to get \(T\), that is,

\[NPMI(x_{i},x_{j})=\frac{\log\frac{Nf(x_{i},x_{j})}{f(x_{i})f(x_{ j})}}{-\log\frac{f(x_{i},x_{j})}{N}},\]

where \(f(x_{i})\) (or \(f(x_{j})\)) is the number of documents where \(x_{i}\) (or \(x_{j}\)) appears, \(f(x_{i},x_{j})\) is the number of documents where \(x_{i}\) and \(x_{j}\) co-occur in a sentence, and \(N\) is the total number of documents in the domain-specific corpus. NPMI is the normalization of pointwise mutual information (PMI), which has the pleasant property \(NPMI(x_{i},x_{j})\in[-1,1]\)Bouma (2009). The values of NPMI are rescaled to the range of \([0,1]\), because we want \(T_{ij}\in[0,1]\).

A candidate \(x_{i}\) can be represented by the \(i\)-th row in \(G\) or \(T\), i.e., the row vector \(g_{i:}\) or \(t_{i:}\), which tells us what \(x_{i}\) is about in terms of its general semantic similarities or statistical associations to other terms. The cosine similarity between two vectors \(\vec{u}\) and \(\vec{v}\) can be calculated as:

\[cosine(\vec{u},\vec{v})=\frac{\vec{u}\cdot\vec{v}}{\parallel\vec{ u}\parallel\parallel\vec{v}\parallel}=\frac{\sum_{i=1}^{n}u_{i}v_{i}}{ \sqrt{\sum_{i=1}^{n}u_{i}^{2}}\sqrt{\sum_{i=1}^{m}v_{i}^{2}}}.\]

By calculating the cosine similarity between two vectors of \(x_{i}\) and \(x_{j}\) (\(i\neq j\)), we get the following similarity metrics:

\[sim_{g}(x_{i},x_{j})=cosine(g_{i:},g_{j:}),\\ sim_{t}(x_{i},x_{j})=cosine(t_{i:},t_{j:}),\\ sim_{gt}(x_{i},x_{j})=max(cosine(g_{i:},t_{j:}),cosine(t_{i:},g_{j:})).\]

\(sim_{g}(x_{i},x_{j})\) provides the comparison between \(g_{i:}\) and \(g_{j:}\). Similar row vectors in \(G\) indicate similar semantic meanings of two terms (e.g., "price" and "inexpensive"). \(sim_{t}(x_{i},x_{j})\) provides the comparison between \(t_{i:}\) and \(t_{j:}\). Similar row vectors in \(T\) indicate similar context of two terms in the domain, and terms that occur in the same contexts tend to have similar meanings Harris (1954) (e.g., "ice cream sandwich" and "operating system"). \(sim_{gt}(x_{i},x_{j})\) provides the comparison between the row vector in \(G\) and the row vector in \(T\) of two terms. \(sim_{gt}(x_{i},x_{j})\) is designed to get high value when the terms strongly associated with \(x_{i}\) (or \(x_{j}\)) are semantically similar to \(x_{j}\) (or \(x_{i}\)). By this measure, the domain-dependent synonyms such as "smooth" and "speed" (in _cell phone_ domain) can be identified because the word "smooth" frequently co-occurs with some other words (e.g., "fast", "run") that are synonymous with the word "speed".

Because \(G_{ij}\in[0,1]\) and \(T_{ij}\in[0,1]\), the values of \(sim_{g}(x_{i},x_{j})\), \(sim_{t}(x_{i},x_{j})\), and \(sim_{gt}(x_{i},x_{j})\) range from \(0\) to \(1\). In addition, \(sim_{g}(x_{i},x_{j})=sim_{g}(x_{j},x_{i})\), \(sim_{t}(x_{i},x_{j})=sim_{t}(x_{j},x_{i})\) and \(sim_{gt}(x_{i},x_{j})=sim_{gt}(x_{j},x_{i})\). When \(i=j\), we set all the similarity metrics between \(x_{i}\) and \(x_{j}\) to \(1\). Finally, the domain-specific similarity between \(x_{i}\) and \(x_{j}\) (\(i\neq j\)) is defined as the weighted sum of the above three similarity metrics: \(sim(x_{i},x_{j})=w_{g}sim_{g}(x_{i},x_{j})+w_{t}sim_{t}(x_{i},x_{j})+w_{gt}sim_ {gt}(x_{i},x_{j})\), where \(w_{g}\), \(w_{t}\) and \(w_{gt}\) denote the relative weight of importance of the three similarity metrics, respectively. The values of the weight ranges from \(0\) to \(1\), and \(w_{g}+w_{t}+w_{gt}=1\).

Based on the domain-specific similarities between candidates, we now define the distance measures of clustering as:

\[dist_{avg}(C_{l},C_{m})=\frac{\sum_{x_{i^{\prime}}\in C_{l}}\sum_{ x_{j^{\prime}}\in C_{m}}(1-sim(x_{i^{\prime}},x_{j^{\prime}}))}{|C_{l}|\times|C_{ m}|},\] \[r(C_{l})=argmax_{x_{i^{\prime}}\in C_{l}}f(x_{i^{\prime}}),\\ dist_{rep}(C_{l},C_{m})=1-sim(r(C_{l}),r(C_{m})),\]

where \(dist_{avg}(C_{l},C_{m})\) is the average of candidate distances between clusters \(C_{l}\) and \(C_{m}\), \(r(C_{l})\) is the most frequent member (i.e., _representative term_) in cluster \(C_{l}\), and \(dist_{rep}(C_{l},C_{m})\) is the distance between the representative terms of two clusters. The two clusters describing the same aspect should be close to each other in terms of both average distance and representative distance, thus the final distance is defined as the maximum of these two:

\[dist(C_{l},C_{m})=max(dist_{avg}(C_{l},C_{m}),dist_{rep}(C_{l},C_{m})).\]

### Merging Constraints

Prior studies Wagstaff et al. (2001) have explored the idea of incorporating background knowledge as constraints on the clustering process to further improve the performance. Two types of constraints are usually considered: must-link constraints specifying 

that two objects (e.g., words) must be placed in the same cluster, and cannot-link constraints specifying that two objects cannot be placed in the same cluster. We also add problem-specific constraints that specify which clusters cannot be merged together, but instead of manually creating the cannot-links between specific words, our cannot-link constraints are automatically calculated during the clustering process.

Specifically, two clusters cannot be merged if they violate any of the three merging constraints: (1) The distance between two clusters must be less than a given value \(\delta\) (see Algorithm 1). (2) There must be at least one noun or noun phrase (candidate of explicit feature) existing in one of the two clusters. Because we assume an aspect should contain at least one explicit feature, and we would not get an aspect by merging two non-aspect clusters. (3) The sum of frequencies of the candidates from two clusters co-occurring in the same sentences must be higher than the sum of frequencies of them co-occurring in the same documents but different sentences. The idea is that people tend to talk about different aspects of a product in different sentences in a review, and talk about the same aspect in a small window (e.g., the same sentence).

## 4 Experiments

In this section, we evaluate the effectiveness of the proposed approach on feature extraction and aspect discovery. Table 1 describes the datasets from three different domains that were used in the experiments. The cell phone reviews were collected from the online shop of a cell phone company, and the GPS and TV reviews were collected from Amazon.

Three human annotators manually annotate the datasets to create gold standards of features and aspects. These annotators first identify feature expressions from reviews independently. The expressions that were agreed by at least two annotators were selected as features. Then the authors manually specified a set of aspects based on these features, and asked three annotators to label each feature with an aspect category. The average inter-annotator agreement on aspect annotation was \(\kappa=0.687\) (\(stddev=0.154\)) according to Cohen's Kappa statistic. To obtain the gold standard annotation of aspects, the annotators discussed to reach an agreement when there was a disagreement on the aspect category of a feature. _We are making the datasets and annotations publicly available5_.

Footnote 5: [http://knoesis.wright.edu/researchers/luchen/download/naacl16_aspects.zip](http://knoesis.wright.edu/researchers/luchen/download/naacl16_aspects.zip)

Table 1 shows the number of reviews, aspects, unique explicit/implicit features manually identified by annotators, and candidates of explicit (i.e., noun and noun phrase) and implicit (i.e., adjective and verb) features extracted from the datasets in three domains.

We use "CAFE" (Clustering for Aspect and **F**eature **E**xtraction) to denote the proposed method. We assume the number of aspects \(k\) is specified by the users, and set \(k=50\) throughout all the experiments. We use \(s=500\), \(\delta=0.8\), \(w_{g}=w_{t}=0.2,w_{gt}=0.6\) as the default setting of CAFE, and study the effect of parameters in Section "Influence of Parameters". In addition, we evaluate each individual similarity metric - "CAFE-g", "CAFE-t" and "CAFE-gt" denote the variations of "CAFE" that use \(sim_{g}(x_{i},x_{j})\), \(sim_{t}(x_{i},x_{j})\), and \(sim_{gt}(x_{i},x_{j})\) as the similarity measure, respectively. We empirically set \(\delta=0.4\) for "CAFE-g", \(\delta=0.84\) for "CAFE-t" and \(\delta=0.88\) for "CAFE-gt".

### Evaluations on Feature Extraction

We compared CAFE against the following two state-of-the-art methods on feature extraction:

* **PROP**: A double propagation approach Qiu et al. (2009) that extracts features using handcrafted rules based on dependency relations between features and opinion words.
* **LRTBOOT**: A bootstrapping approach Hai et al. (2012) that extracts features by mining pairwise feature-feature, feature-opinion, opinion-opinion associations between terms in the corpus, where the association is measured by the likelihood ratio tests (LRT). Both methods require seeds terms. We ranked the feature candidates by descending document fre

\begin{table}
\begin{tabular}{|l|c|c|c|} \hline  & Cell phone & GPS & TV \\ \hline \# Reviews & 500 & 500 & 500 \\ \hline \# Aspects & 46 & 37 & 34 \\ \hline \# Features (expl.) & 419 & 637 & 485 \\ \hline \# Features (impl.) & 339 & 492 & 277 \\ \hline \# Candidates (expl.) & 1,248 & 2,078 & 2,333 \\ \hline \# Candidates (impl.) & 1,115 & 1,779 & 1,690 \\ \hline \end{tabular}
\end{table}
Table 1: Data sets and gold standards.

 

quency and manually selected the top 10 genuine features as seeds for them. According to the study [2], the performance for LRTBOOT remained almost constant when increasing the seeds from 1 to 50. Three association thresholds need to be specified for LRTBOOT. Following the original study in which the experiments were conducted on cell-phone reviews, we set \(ffth=21.0,~{}ooth=12.0\), and performed grid search for the value of \(foth\). The best results were achieved at \(foth=9.0\) for cell-phone reviews, and at \(foth=12.0\) for GPS and TV reviews.

The results were evaluated by precision \(=\frac{N_{agree}}{N_{result}}\), recall \(=\frac{N_{agree}}{N_{gold}}\), and F-score \(=\frac{2\times\text{precision/recall}}{\text{precision+recall}}\), where \(N_{result}\) and \(N_{gold}\) are the number of features in the result and the gold standard, respectively, and \(N_{agree}\) is the number of features that are agreed by both sides. Because PROP and LRTBOOT extract only explicit features, the evaluation was conducted on the quality of explicit features. The performance of identifying implicit features will be examined by evaluation on aspect discovery, because implicit features have to be merged into aspects to be detected.

Table 2 shows the best results (in terms of F-score) of feature extraction by different methods. Both PROP and LRTBOOT obtain high recall and relatively low precision. CAFE greatly improves precision, with a relatively small loss of recall, resulting in 21.68% and 9.36% improvement in macro-averaged F-score over PROP and LRTBOOT, respectively. We also plot precision-recall curves at various parameter settings for CAFE and LRTBOOT in Figure 1. For CAFE, we kept \(s=500\), \(w_{g}=w_{t}=0.2,w_{gt}=0.6\), and increased \(\delta\) from 0.64 to 0.96. For LRTBOOT, we kept \(ffth=21.0,~{}ooth=12.0\), and increased \(foth\) from 6.0 to 30.0. For PROP, only one precision-recall point was obtained. From Figure 1, we see that the curve of CAFE lies well above those of LRTBOOT and PROP across three datasets. Though LRTBOOT achieved similar precision as CAFE did at the recall rate of approximately 0.37 for GPS reviews and at the recall rate of approximately 0.49 for TV reviews, it performed worse than CAFE at increasing recall levels for both datasets.

The key difference between CAFE and the baselines is that CAFE groups terms into clusters and identifies the terms in the selected aspect clusters as features, while both baselines enlarge a feature seed set by mining syntactical or statistical associations between features and opinion words. The results suggest that features can be more precisely identified via aspect clustering. Generally, CAFE is superior to its variations, and CAFE-g outperforms CAFE-gt and CAFE-t.

### Evaluations on Aspect Discovery

For comparison with CAFE on aspect discovery, we implemented the following three methods:

* **MuReinf**: A clustering method [12] that utilizes the mutual reinforcement ass

Figure 1: Precision-recall curves at various parameter settings for three feature extraction methods.

 

sociation between features and opinion words to iteratively group them into clusters. Similar to the proposed method, it is unsupervised, clustering-based, and handling implicit features.
* **L-EM**: A semi-supervised learning method [21] that adapts the Naive Bayesian-based EM algorithm to group synonym features into categories. Because semi-supervised learning needs some labeled examples, the proposed method first automatically generates some labeled examples (i.e., the groups of synonym feature expressions) based on features sharing common words and lexical similarity.
* **L-LDA**: A baseline method [21] that is based on LDA. The same labeled examples generated by L-EM are used as seeds for each topic in topic modeling.

These three methods require features to be extracted beforehand, and focus on grouping features into aspects. Both LRTBOOT and CAFE are used to provide the input features to them. We set \(\alpha=0.6\) for MuReinf, because their study [34] showed that the method achieved best results at \(\alpha>0.5\). All three methods utilize dictionary-based semantic similarity to some extent. Since CAFE uses the UMBC Semantic Similarity Service, we use the same service to provide the semantic similarity for all the methods.

The results were evaluated using Rand Index [7], a standard measure of the similarity between the clustering results and a gold standard. Given a set of \(n\) objects and two partitions of them, the Rand Index is defined as \(\frac{2(a+b)}{n\times(n-1)}\). The idea is that the agreements/disagreements between two partitions are checked on \(n\times(n-1)\) pairs of objects. Among all the pairs, there are \(a\) pairs belonging to the same cluster in both partitions, and \(b\) pairs belonging to different clusters in both partitions. In this study, the gold standard and the aspect clusters may not share the exact same set of features due to the noise in feature extraction, therefore we consider \(n\) the number of expressions in the union of two sets.

Table 3 shows the Rand Index achieved by different methods. Among the methods that generate partitions of the same features provided by CAFE, CAFE achieves the best macro-averaged Rand Index, followed by CAFE + MuReinf, CAFE + L-LDA, and CAFE + L-EM. CAFE outperforms the variations using the single similarity metric, i.e., CAFE-g, CAFE-t and CAFE-gt. The results imply the effectiveness of our domain-specific similarity measure in identifying synonym features in a particular domain. Using the input features from LRTBOOT, the performance of MuReinf, L-EM and L-LDA decrease on all three domains, compared with using the input features from CAFE. The decrease is more significant for L-EM and L-LDA than for MuReinf, which suggest that the semi-supervised methods L-EM and L-LDA are more dependent on the quality of input features.

Table 4 illustrates a sample of the discovered aspects and features by CAFE. The algorithm identifies the important _aspects_ in general sense as well as the important aspects that are not so obvious thus could be easily missed by human judges, e.g., _suction cup_ for GPS and _glare_ for TV. In addition, both explicit and implicit features are identified and grouped into the aspects, e.g., _expensive_ and _price_, _big_ and _size_, _sensitive_ and _signal_, etc.

### Influence of Parameters

We varied the value of \(\delta\) (distance _upper bound_), \(s\) (the number of frequent candidates selected to generate seed clusters) and \(w_{gt}\) (the weight of \(sim_{gt}\)) to see how they impact the results of CAFE, for both feature extraction (in terms of F-Score) and aspect discovery (in terms of Rand Index). Both F-score and Rand Index increases rapidly at first and then slowly decreases as we increase \(\delta\) from 0.64 to 0.96 (see the left subplot in Figure 2). Because more clusters are allowed to be merged as we increase \(\delta\), which is good at first but then it introduces more noise than benefit. Based on the experiments on

\begin{table}
\begin{tabular}{|c|r|r|r|r|} \hline  & Cell-phone & GPS & TV & macro-average \\ \hline LRTBOOT + MuReinf & 0.7182 & 0.8031 & 0.7747 & 0.7653 \\ \hline LRTBOOT + L-EM & 0.6633 & 0.6893 & 0.7138 & 0.6888 \\ \hline LRTBOOT + L-LDA & 0.7653 & 0.7198 & 0.7664 & 0.7505 \\ \hline CAFE + MuReinf & 0.7973 & 0.8212 & **0.8334** & 0.8173 \\ \hline CAFE + L-EM & 0.7581 & 0.7772 & 0.7879 & 0.7744 \\ \hline CAFE + L-LDA & 0.7904 & 0.8144 & 0.8247 & 0.8098 \\ \hline CAFE & 0.8041 & **0.8238** & 0.8326 & **0.8202** \\ \hline CAFE-g & 0.7382 & 0.7534 & 0.8205 & 0.7707 \\ \hline CAFE-t & 0.7868 & 0.8050 & 0.7965 & 0.7961 \\ \hline CAFE-gt & **0.8073** & 0.7716 & 0.7906 & 0.7898 \\ \hline \end{tabular}
\end{table}
Table 3: Rand Index of aspect identification.

 

three domains, the best results can be achieved when \(\delta\) is set to a value between 0.76 and 0.84. The middle subplot illustrates the impact of \(s\), which shows that CAFE generates better results by first clustering the top 10%-30% most frequent candidates. Infrequent words/phrases are usually more noisy, and the results could be affected more seriously if the noises are included in the clusters in the early stage of clustering. Experiments were also conducted to study the impact of the three similarity metrics. Due to the space limit, we only display the impact of \(w_{gt}\) and \(w_{g}\) given \(w_{t}=0.2\). As we can see from the right subplot in Figure 2, setting \(w_{gt}\) or \(w_{g}\) to zero evidently decreases the performance, indicating both similarity metrics are useful. The best F-score and Rand Index can be achieved when we set \(w_{gt}\) to 0.5 or 0.6 across all three domains.

## 5 Conclusion

In this paper, we proposed a clustering approach that simultaneously extracts features and aspects of a given product from reviews. Our approach groups the feature candidates into clusters based on their domain-specific similarities and merging constraints, then selects the important aspects and identifies features from these aspects. This approach has the following advantages: (1) It identifies both aspects and features simultaneously. The evaluation shows its accuracy on both tasks outperforms the competitors. (2) Both explicit and implicit features can be identified and grouped into aspects. The mappings of implicit features into explicit features are accomplished naturally during the clustering process. (3) It does not require labeled data or seed words, which makes it easier to apply and broader in application. In our future work, instead of selecting aspects based on frequency, we will leverage domain knowledge to improve the selection.

We would like to thank Lushan Han and UMBC Ebiquiry Lab for kindly allowing us to access their Semantic Similarity Service. This research was partially supported by NSF awards CNS-1513721 "Context-Aware Harassment Detection on Social Media" and EAR-1520870 "Hazards SEES: Social and Physical Sensing Enabled Decision Support for Disaster Management and Response".

Figure 2: CAFE parameter tuning: feature quality in terms of F-score (F1) and aspect quality in terms of Rand Index (RI). The default setting is \(s=500\), \(\delta=0.8\), \(w_{g}=w_{t}=0.2,w_{gt}=0.6\). We keep other parameters as the default setting when we tune an individual parameter.

 



## References

* [Bancken et al.2014] Wouter Bancken, Daniele Alfarone, and Jesse Davis. 2014. Automatically detecting and rating product aspects from textual customer reviews. In _DMNLP Workshop at ECML/PKDD_.
* Blei et al. [2003] David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. _the Journal of machine Learning research_, 3:993-1022.
* Bouma [2009] Gerlof Bouma. 2009. Normalized (pointwise) mutual information in collocation extraction. _GSCL_, pages 31-40.
* Brody and Elhadad [2010] Samuel Brody and Noemie Elhadad. 2010. An unsupervised aspect-sentiment model for online reviews. In _NAACL_, pages 804-812.
* Carenini et al. [2005] Giuseppe Carenini, Raymond T Ng, and Ed Zwart. 2005. Extracting knowledge from evaluative text. In _K-CAP_, pages 11-18.
* Chen et al. [2013] Zhiyuan Chen, Arjun Mukherjee, Bing Liu, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. 2013. Exploiting domain knowledge in aspect extraction. In _EMNLP_, pages 1655-1667.
* Chen et al. [2014] Zhiyuan Chen, Arjun Mukherjee, and Bing Liu. 2014. Aspect extraction with automated prior knowledge learning. In _ACL_, pages 347-358.
* Fei et al. [2012] Geli Fei, Bing Liu, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. 2012. A dictionary-based approach to identifying aspects im-plied by adjectives for opinion mining. In _COLING_, page 309.
* Guo et al. [2009] Honglei Guo, Huijia Zhu, Zhili Guo, XiaoXun Zhang, and Zhong Su. 2009. Product feature categorization with multilevel latent semantic association. In _CIKM_, pages 1087-1096.
* Hai et al. [2011] Zhen Hai, Kuiyu Chang, and Jung-jae Kim. 2011. Implicit feature identification via co-occurrence association rule mining. In _Computational Linguistics and Intelligent Text Processing_, pages 393-404.
* Hai et al. [2012] Zhen Hai, Kuiyu Chang, and Gao Cong. 2012. One seed to find them all: mining opinion features via association. In _CIKM_, pages 255-264.
* Han et al. [2013] Lushan Han, Abhay Kashyap, Tim Finin, James Mayfield, and Jonathan Weese. 2013. Umbc ebiquitycore: Semantic textual similarity systems. In _the Second Joint Conference on Lexical and Computational Semantics_, pages 44-52.
* Harris [1954] Zellig S Harris. 1954. Distributional structure. _Word_.
* Hofmann [2001] Thomas Hofmann. 2001. Unsupervised learning by probabilistic latent semantic analysis. _Machine learning_, 42(1-2):177-196.
* Hu and Liu [2004] Minqing Hu and Bing Liu. 2004. Mining opinion features in customer reviews. In _AAAI_, pages 755-760.
* Jin et al. [2009] Wei Jin, Hung Hay Ho, and Rohini K Srihari. 2009. Opinionminer: a novel machine learning system for web opinion mining and extraction. In _SIGKDD_, pages 1195-1204.
* Jo and Oh [2011] Yohan Jo and Alice H Oh. 2011. Aspect and sentiment unification model for online review analysis. In _WSDM_, pages 815-824.
* Kim et al. [2013] Suin Kim, Jianwen Zhang, Zheng Chen, Alice H Oh, and Shixia Liu. 2013. A hierarchical aspect-sentiment model for online reviews. In _AAAI_.
* Lin and He [2009] Chenghua Lin and Yuan He. 2009. Joint sentiment/topic model for sentiment analysis. In _CIKM_, pages 375-384.
* Liu et al. [2005] Bing Liu, Minqing Hu, and Junsheng Cheng. 2005. Opinion observer: analyzing and comparing opinions on the web. In _WWW_, pages 342-351.
* Lu et al. [2009] Yue Lu, ChengXiang Zhai, and Neel Sundaresan. 2009. Rated aspect summarization of short comments. In _WWW_, pages 131-140.
* Moghaddam and Ester [2011] Samaneh Moghaddam and Martin Ester. 2011. Ilda: interdependent lda model for learning latent aspects and their ratings from online product reviews. In _SIGIR_, pages 665-674.
* Mukherjee and Liu [2012] Arjun Mukherjee and Bing Liu. 2012. Aspect extraction through semi-supervised modeling. In _ACL_, pages 339-348.
* Qiu et al. [2009] Guang Qiu, Bing Liu, Jiajun Bu, and Chun Chen. 2009. Expanding domain sentiment lexicon through double propagation. In _IJCAI_, pages 1199-1204.
* Rand [1971] William M Rand. 1971. Objective criteria for the evaluation of clustering methods. _Journal of the American Statistical association_, 66(336):846-850.
* Su et al. [2006] Qi Su, Kun Xiang, Houfeng Wang, Bin Sun, and Shiwen Yu. 2006. Using pointwise mutual information to identify implicit features in customer reviews. In _ICCPOL_, pages 22-30.
* Su et al. [2008] Qi Su, Xinying Xu, Honglei Guo, Zhili Guo, Xian Wu, Xiaoxun Zhang, Bin Swen, and Zhong Su. 2008. Hidden sentiment association in chinese web opinion mining. In _WWW_, pages 959-968.
* Titov and McDonald [2008a] Ivan Titov and Ryan McDonald. 2008a. Modeling online reviews with multi-grain topic models. In _WWW_, pages 111-120.
* Titov and McDonald [2008b] Ivan Titov and Ryan T McDonald. 2008b. A joint model of text and aspect ratings for sentiment summarization. In _ACL_, pages 308-316.
* Wagstaff et al. [2001] Kiri Wagstaff, Claire Cardie, Seth Rogers, Stefan Schrodl, et al. 2001. Constrained k-means clustering with background knowledge. In _ICML_, volume 1, pages 577-584.
* Wang and Ester [2014] Hao Wang and Martin Ester. 2014. A sentiment-aligned topic model for product aspect rating prediction. In _EMNLP_, pages 1192-1202

* [Xu et al.2013] Liheng Xu, Kang Liu, Siwei Lai, Yubo Chen, and Jun Zhao. 2013. Mining opinion words and opinion targets in a two-stage framework. In _ACL_, pages 1764-1773.
* [Xu et al.2014] Yinqing Xu, Tianyi Lin, Wai Lam, Zirui Zhou, Hong Cheng, and Anthony Man-Cho So. 2014. Latent aspect mining via exploring sparsity and intrinsic information. In _CIKM_, pages 879-888.
* [Yang and Cardie2013] Bishan Yang and Claire Cardie. 2013. Joint inference for fine-grained opinion extraction. In _ACL_, pages 1640-1649.
* [Yu et al.2011] Jianxing Yu, Zheng-Jun Zha, Meng Wang, and Tat-Seng Chua. 2011. Aspect ranking: identifying important product aspects from online consumer reviews. In _ACL_, pages 1496-1505.
* [Zhai et al.2010] Zhongwu Zhai, Bing Liu, Hua Xu, and Peifa Jia. 2010. Grouping product features using semi-supervised learning with soft-constraints. In _COLING_, pages 1272-1280.
* [Zhai et al.2011] Zhongwu Zhai, Bing Liu, Hua Xu, and Peifa Jia. 2011. Clustering product features for opinion mining. In _WSDM_, pages 347-354.
* [Zhang and Liu2014] Lei Zhang and Bing Liu. 2014. Aspect and entity extraction for opinion mining. In _Data Mining and Knowledge Discovery for Big Data_, pages 1-40.
* [Zhang and Zhu2013] Yu Zhang and Weixiang Zhu. 2013. Extracting implicit features in online customer reviews for opinion mining. In _WWW companion_, pages 103-104.
* [Zhao et al.2014] Li Zhao, Minlie Huang, Haiqiang Chen, Junjun Cheng, and Xiaoyan Zhu. 2014. Clustering aspect-related phrases by leveraging sentiment distribution consistency. In _EMNLP_, pages 1614-1623.