Venue: EMNLP 2017

# Document-Level Multi-Aspect Sentiment Classification as

Machine Comprehension

Yichun Yin\({}^{1}\), Yangqiu Song\({}^{2}\), Ming Zhang\({}^{1}\)

\({}^{1}\)School of Electronics Engineering and Computer Science, Peking University, Beijing, China

\({}^{2}\)Department of Computer Science and Engineering, HKUST, Hong Kong {yichunyin, mzhang_cs}@pku.edu.cn, yqsong@cse.ust.hk

###### Abstract

Document-level multi-aspect sentiment classification is an important task for customer relation management. In this paper, we model the task as a machine comprehension problem where pseudo question-answer pairs are constructed by a small number of aspect-related keywords and aspect ratings. A hierarchical iterative attention model is introduced to build aspect-specific representations by frequent and repeated interactions between documents and aspect questions. We adopt a hierarchical architecture to represent both word level and sentence level information, and use the attention operations for aspect questions and documents alternatively with the multiple hop mechanism. Experimental results on the TripAdvisor and BeerAdvocate datasets show that our model outperforms classical baselines.

## 1 Introduction

Document-level sentiment classification is one of the pragmatical sentiment analysis tasks Pang and Lee (2007); Liu (2010). There are many Web sites having platforms for users to input reviews over products or services, such as TripAdvisor, Yelp, Amazon, etc. Most of reviews are very comprehensive and thus long documents. Analyzing these documents to predict ratings of products or services is an important complementary way for better customer relationship management. Recently, neural network based approaches have been developed and become state-of-the-arts for long-document sentiment classification Tang et al. (2015a, b); Yang et al. (2016). However, predicting an overall score for each long document is not enough, because the document can mention different aspects of the corresponding product or service. For example, in Figure 1, there could be different aspects for a review of hotel. These aspects help customer service better understand what are the major pros and cons of the product or service. Compared to the overall rating, users are less motivated to give aspect ratings. Therefore, it is more practically useful to perform document-level multi-aspect sentiment classification task, predicting different ratings for each aspect rather than an overall rating.

One straightforward approach for document-level multi-aspect sentiment classification is multi-task learning Caruana (1997). For neural networks, we can simply treat each aspect (e.g., rating from one to five) as a classification task, and let different tasks use softmax classifier to extract task-specific representations at the top layer while share the input and hidden layers to mutually enhance the prediction results Collobert et al. (2011); Luong et al. (2016). However, such approach ignores the fact that the aspects themselves have semantic meanings. For example, as human beings, if we were asked to evaluate the aspect rating of a document, we simply read the review, and find aspect-related keywords, and see around comments. Then, we aggregate all the related snippets to make a decision.

In this paper, we propose a novel approach to treat document-level multi-aspect sentiment clas

Figure 1: Example: hotel review with aspects.



sification as a machine comprehension Kumar et al. (2016); Sordoni et al. (2016) problem. To mimic human's evaluation of aspect classification, we create a list of keywords for each aspect. For example, when we work on the _Room_ aspect, we generate some keywords such as "room," "bed," "view," etc. Then we can ask pseudo questions: "How is the room?" "How is the bed?" "How is the view?" and provide an answer "Rating 5." In this case, we can train a machine comprehension model to automatically attend corresponding text snippets in the review document to predict the aspect rating. Specifically, we introduce a hierarchical and iterative attention model to construct aspect-specific representations. We use a hierarchical architecture to build up different representations at both word and sentence levels interacting with aspect questions. At each level, the model consists of input encoders and iterative attention modules. The input encoder learns memories1 of documents and questions with Bi-directional LSTM (Bi-LSTM) model and non-linear mapping respectively. The iterative attention module takes into memories as input and attends them sequentially with a multiple hop mechanism, performing effective interactions between documents and aspect questions.

Footnote 1: Following the work Weston et al. (2015); Sukhbaatar et al. (2015), we refer the memory to a set vectors which are stacked together and could be attended.

To evaluate the effectiveness of the proposed model, we conduct extensive experiments on the TripAdvisor and BeerAdvocate datasets and the results show that our model outperforms typical baselines. We also analyze the effects of numbers of the hop and aspect words on performances. Moreover, a case study for attention results is performed at both word and sentence levels.

The contributions of this paper are two-fold. First, we study the document-level multi-aspect sentiment classification as a machine comprehension problem and introduce a hierarchical iterative attention model for it. Second, we demonstrate the effectiveness of proposed model on two datasets, showing that our model outperforms classical baselines. The code and data for this paper are available at [https://github.com/HKUST-KnowComp/DMSCMC](https://github.com/HKUST-KnowComp/DMSCMC).

## 2 Method

In this section, we introduce our proposed method.

### Problem Definition and Hierarchical Framework

We first briefly introduce the problem we work on. Given a piece of review, our task is to predict the ratings of different aspects. For example, in Figure 1, we predict the ratings of _Cleanliness_, _Room_, and _Value_. To achieve this, we assume that there are existing reviews with aspect ratings for machines to learn. Formally, we denote the review document as \(d\) containing a set of \(T_{d}\) sentences \(\{s_{1},s_{2},\ldots s_{T_{d}}\}\). For the \(t\)-th sentence \(s_{t}\), we use a set of words \(\{w_{1},w_{2},\ldots w_{|s_{t}|}\}\) to represent it, and use **wi**, \(\textbf{w}^{w}_{i}\) and \(\textbf{w}^{p}_{i}\) as the one-hot encoding, word embedding, and phrase embedding for \(w_{i}\) respectively. The phrase embedding encodes the semantics of phrases where the current word \(w_{i}\) is the center (e.g., hidden vectors learned by Bi-LSTM shown in Section 2.2). For each \(q_{k}\) of \(K\) aspects

Figure 2: The architecture of our model. Left: multi-task learning. Right: hierarchical attention module which includes input encoders and iterative attention modules.

 

\(\left\{q_{1},q_{2},\ldots,q_{K}\right\}\), we use \(N_{k}\) aspect-related keywords, \(\left\{q_{k_{1}},q_{k_{2}}\ldots q_{k_{N_{k}}}\right\}\), to represent it. Similarly, we use \(\mathbf{q}_{k_{i}}\), \(\mathbf{q}^{w}_{k_{i}}\) as the one-hot encoding and word embedding for \(q_{k_{i}}\) respectively.

There are several sophisticated methods for choosing aspect keywords (e.g., topic model). Here, we consider a simple way where five seeds were first manually selected for each aspect and then more words were obtained based on their cosine similarities with seeds2

Footnote 2: For example, the words “value,”’ price,” worth, ” cost,” and “S” are selected as seeds for aspect _Price_. The information for seeds can be found in our released resource.

As shown in Figure 2 (left), our framework follows the idea of multi-task learning, which learns different aspects simultaneously. In this case, all these tasks share the representations of words and architecture of semantic model for the final classifiers. Different from straightforward neural network based multi-task learning Collobert et al. (2011), for each document \(d\) and an aspect \(q_{k}\), our model uses both the content of \(d\) and all the related keywords \(\left\{q_{k_{1}},q_{k_{2}}\ldots q_{k_{N_{k}}}\right\}\) as input. Since the keywords can cover most of the semantic meanings of the aspect, and we do not know which document mentions which semantic meaning, we build an attention model to automatically decide it (introduced in Section 2.3). Assuming that the keywords have been decided, we use a hierarchical attention model to select useful information from the review documents. As shown in Figure 2 (right), the hierarchical attention of keywords is applied to both sentence level (to select meaningful words) and document level (to select meaningful sentence). Thus, our model builds aspect-specific representations in a bottom-up manner.

Specifically, we obtain sentence representations \(\left\{\mathbf{s}_{1}^{k},\mathbf{s}_{2}^{k},\ldots\mathbf{s}_{T}^{k}\right\}\) using the input encoder (Section 2.2) and iterative attention module (Section 2.3) at the word level. Then we take sentence representations and \(k\)-th aspect as input and apply the sentence-level input encoder and attention model to generate the document representation \(\mathbf{d}_{k}\) for final classification. As shown in Figure 2 (right), the attention model is applied twice at different levels of the representation.

### Input Encoder

The input module builds memory vectors for the iterative attention module and is performed both at word and sentence levels. For a document, it converts word sequence into word level memory \(\mathbf{M}_{w}^{d}\) and sentence sequence into sentence level memory \(\mathbf{M}_{s}^{d}\) respectively. For an aspect question \(q_{k}\), it takes a set of aspect-specific words \(\left\{q_{k_{i}}\right\}_{1\leq i\leq N_{k}}\) as input and derives word level memory \(\mathbf{M}_{w}^{q}\) and sentence level memory \(\mathbf{M}_{s}^{q}\).

To construct \(\mathbf{M}_{v}^{d}\), we obtain word embeddings \(\left\{\mathbf{w}_{1}^{w},\mathbf{w}_{2}^{w},\ldots\mathbf{w}_{|s_{t}|}^{w}\right\}\) from an embedding matrix \(\textbf{E}^{A}\) applied to all words shown in the corpus. Then, LSTM Hochreiter and Schmidhuber (1997) model is used as the encoder to produce hidden vectors of words based on the word embeddings. At each step, LSTM takes input \(\mathbf{w}_{t}^{w}\) and derives a new hidden vector by \(\mathbf{h}_{t}=\text{LSTM}(\mathbf{w}_{t}^{w},\mathbf{h}_{t-1})\). To preserve the subsequent context information for words, another LSTM is ran over word sequence in a reverse order simultaneously. Then the forward hidden vector \(\overrightarrow{\mathbf{h}}_{t}\) and backward hidden vector \(\overleftarrow{\mathbf{h}}_{t}\) are concatenated as phrase embedding \(\mathbf{w}_{t}^{p}\). We stack these phrase embeddings together as word level memory \(\mathbf{M}_{w}^{d}\). Similarly, we feed sentence representations into another Bi-LSTM to derive the sentence level memory \(\mathbf{M}_{s}^{d}\). Note that, the sentence representations are obtained using the iterative attention module which is described as Eq. (5) in Section 2.3.

Since we have question keywords as input, to allow the interactions between questions and documents, we also build question memory in following way. We obtain \(\textbf{Q}_{k}=\left\{\mathbf{q}_{k_{i}}^{w}\right\}_{1\leq i\leq N_{k}}\) by looking up an embedding matrix 3\(\mathbf{E}^{B}\) applied to all question keywords. Then a non-linear mapping is applied to obtain the question memory at word level:

Footnote 3: \(\mathbf{E}^{A}\) and \(\textbf{E}^{B}\) are initialized by the same pre-trained embeddings but are different embedding matrices with different updates.

\[\mathbf{M}_{w}^{q_{k}}=\tanh(\textbf{Q}_{k}\mathbf{W}_{w}^{q}), \tag{1}\]

where \(\mathbf{W}_{w}^{q}\) is the parameter matrix to adapt \(q_{k}\) at word level. Similarly, we use another mapping to obtain the sentence level memory:

\[\mathbf{M}_{s}^{q_{k}}=\tanh(\textbf{Q}_{k}\mathbf{W}_{s}^{q}), \tag{2}\]

where \(\mathbf{W}_{s}^{q}\) is the parameter matrix to adapt \(q_{k}\) at sentence level.

### Iterative Attention Module

The iterative attention module (IAM) attends and reads memories of questions and documents alternatively with a multi-hop mechanism, deriving 

aspect-specific sentence and document representations. As we discussed in the introduction, the set of selected question keywords may not best characterize the aspect for different documents. Thus, the IAM module introduces a backward attention to use document information (word or sentence) to select useful keywords of each aspect as the document-specific question to build attention model.

The illustration of IAM is shown in Figure 3. To obtain sentence representations, it takes \({\bf M}_{w}^{d}\) and \({\bf M}_{w}^{q}\) as the input and performs \(m\) iterations (hops). For each iteration, IAM conducts four operations: (1) attends the question memory by the selective vector **p** and summarizes question memory vectors into a single vector \(\hat{\textbf{q}}\); (2) updates the selective vector by the previous one and \(\hat{\textbf{q}}\); (3) attends document (content) memory based on the updated selective vector and summarizes memory vectors in to a single vector \(\hat{\textbf{c}}\); (4) updates the selective vector by the previous one and \(\hat{\textbf{c}}\).

We unify operations (1) and (3) by an attention function \(\hat{\textbf{x}}=\mathcal{A}(\textbf{p},\mathbf{M})\), where \(\mathbf{M}\) could be \({\bf M}_{w}^{d}\) or \({\bf M}_{w}^{q}\) which corresponds \(\tilde{\textbf{x}}=\hat{\textbf{c}}\) or \(\tilde{\textbf{x}}=\hat{\textbf{q}}\). The attention function \(\mathcal{A}\) is decomposed as:

\[\begin{split}&\textbf{H}=\tanh(\mathbf{MW}_{a}\odot(\mathds{1} \textbf{p}))\\ &\textbf{a}=\text{softmax}(\textbf{Hy}_{a}^{T})\\ &\dot{\textbf{x}}=\sum\textbf{a}_{i}\textbf{M}_{i},\end{split} \tag{3}\]

where \(\mathds{1}\) is a vector with all elements are 1, which copies the selective vector to meet the dimension requirement. The \(\textbf{W}_{a}\) and \(\textbf{v}_{a}\) are parameters, **a** is attention weights for memory vectors, and \(\textbf{M}_{i}\) means \(i\)-th row in \(\mathbf{M}\).

Operations (2) and (4) are formulated as an update function \(\textbf{p}_{2i-l\left\{l\right\}}=\mathcal{U}(\hat{\textbf{x}},\textbf{p}_{2 i-l\left\{l\right\}-1})\), where \(i\) is the hop index, \(l\) can be 0 or 1 which corresponds to \(\hat{\textbf{x}}=\hat{\textbf{c}}\) or \(\hat{\textbf{x}}=\hat{\textbf{q}}\) respectively. We initialize \(\textbf{p}_{0}\) by a zero vector. The update function \(\mathcal{U}\) can be a recurrent neural network Xiong et al. (2017) or other heuristic weighting functions. In this paper, we introduce a simple strategy:

\[\textbf{p}_{2i-\left\{l\right\}}=\hat{\textbf{x}}, \tag{4}\]

which ignores the previous selective vector but succeeds to obtain comparable results with other more complicated function in the initial experiments.

Multi-hop mechanism attends different memory locations in different hops Sukhbaatar et al. (2015), capturing different interactions between documents and questions. In order to preserve the information of various kinds of interactions, we concatenate all \(\hat{\textbf{c}}\)'s in each hop as the final representations of sentences:

\[\textbf{s}=[\hat{\textbf{c}}_{1};\hat{\textbf{c}}_{2};\cdots\hat{\textbf{c}}_ {m}]. \tag{5}\]

After obtaining sentence representations, we feed them into the sentence-level input encoder, deriving the memories \({\bf M}_{s}^{d}\) and \({\bf M}_{s}^{q}\). Then, the aspect-specific document representation \({\bf d}_{k}\) is obtained by the sentence-level IAM in a similar way.

### Objective Function

For each aspect, we obtain aspect-specific document representations \(\{{\bf d}_{k}\}_{1\leq k\leq K}\). All these representations are fed into classifiers, each of which includes a softmax layer. The softmax layer outputs the probability distribution over \(|\mathcal{Y}|\) categories for the distributed representation, which is defined as:

\[\textbf{p}^{\prime}(d,k)=\text{softmax}(\textbf{W}_{k}^{class}\textbf{d}_{k}), \tag{6}\]

where \(\textbf{W}_{k}^{class}\) is the parameter matrix.

We define the cross-entropy objective function between gold sentiment distribution \(\textbf{p}(d,k)\) and predicted sentiment distribution \(\textbf{p}^{\prime}(d,k)\) as the classification loss function:

\[-\sum_{d\in\mathcal{D}}\sum_{k=1}^{K}\sum_{i=1}^{|\mathcal{Y}|}\textbf{p}(d,k) \mathrm{log}(\textbf{p}^{\prime}(d,k)), \tag{7}\]

where \(\textbf{p}(d,k)\) is a one-hot vector, which has the same dimension as the number of classes, and only the dimension associated with the ground truth label is one, with others being zeros.

Figure 3: The iterative attention module.

 



## 3 Experiment

In this section, we show experimental results to demonstrate our proposed algorithm.

### Datasets

We conduct our experiments on TripAdvisor Wang et al. (2010) and BeerAdvocate McAuley et al. (2012); Lei et al. (2016) datasets, which contain seven aspects (_value_, _room_, _location_, _cleanliness_, _check infront desk_, _service_, and _business service_) and four aspects (_feel_, _look_, _smell_, and _taste_) respectively. We follow the processing step Lei et al. (2016) by choosing the reviews with different aspect ratings and the new datasets are described in Table 1. We tokenize the datasets by Stanford corenlp4 and randomly split them into training, development, and testing sets with 80/10/10%.

Footnote 4: [http://nlp.stanford.edu/software/corenlp.shtml](http://nlp.stanford.edu/software/corenlp.shtml)

### Baseline Methods

To demonstrate the effectiveness of the proposed method, we compare our model with following baselines:

_Majority_ uses the majority sentiment label in development sets as the predicted label.

_SVM_ uses unigram and bigram as text features and uses Liblinear Fan et al. (2008) for learning.

_SLDA_ refers to supervised latent Dirichlet allocation Blei and Mcauliffe (2010) which is a statistical model of labeled documents.

_NBoW_ is a neural bag-of-words model averaging embeddings of all words in a document and feeds the resulted embeddings into SVM classifier.

_DAN_ is a deep averaging network model which consists of several fully connected layers with averaged word embeddings as input. One novel word dropout strategy is employed to boost model performances Iyyer et al. (2015).

_CNN_ continuously performs a convolution operation over a sentence to extract words neighboring features, then gets a fixed-sized representation by a pooling layer Kim (2014).

_LSTM_ is one variant of recurrent neural network and has been proved to be one of state-of-the-art models for document-level sentiment classification Tang et al. (2015). We use LSTM to refer Bi-LSTM which captures both forward and backward semantic information.

_HAN_ means the hierarchical attention network which is proposed in Yang et al. (2016) for document classification. Note that, the original _HAN_ depends GRU as the encoder. In our experiments, LSTM-based HAN obtains slightly better results. Thus, we report the results of HAN with LSTM as the encoder.

We extend _DAN_, _CNN_, _LSTM_ with the hierarchical architecture and multi-task framework, the corresponding models are _MHDAN_, _MHCNN_ and _MHLSTM_ respectively. Besides, _MHAN_ is also evaluated as one baseline, which is _HAN_ with the multi-task learning.

### Implementation Details

We implement all neural models using Theano Theano Development Team (2016). The model parameters are tuned based on the development sets. We learn 200-dimensional word embeddings with Skip-gram model Mikolov et al. (2013) on in-domain corpus, which follows Tang et al. (2015). The pre-trained word embeddings are used to initialize the embedding matrices \({\bf E}^{A}\) and \({\bf E}^{B}\). The dimensions of all hidden vectors are set to 200. For TripAdvisor dataset, the hop numbers of word-level and sentence-level iterative attention modules are set to 4 and 2 respectively. For BeerAdvocate dataset, the hop numbers are set to 6 and 2. The number of selected keywords \(N_{k}=N\) is set to 20. To avoid model over-fitting, we use dropout and regularization as follows: (1) the regularization parameter is set to _1e-5_; (2) the dropout rate is set to 0.3, which is applied to both sentence and document vectors. All parameters are trained by ADADELTA Zeiler (2012) without needing to set the initial learning rate. To ensure fair comparisons, we make baselines have same settings as the proposed model, such as word embeddings, dimensions of hidden vectors and optimization details and so on.

### Results and Analyses

We use accuracy and mean squared error (MSE) as the evaluation metrics and the results are shown in Table 2.

\begin{table}
\begin{tabular}{l|c|c|c} \hline Dataset & \#docs & \#words/doc & \#words/sent \\ \hline TripAdvisor & 29,391 & 251.7 & 18.0 \\ BeerAdvocate & 51,020 & 144.5 & 12.1 \\ \hline \end{tabular}
\end{table}
Table 1: Statistics of the datasets. The rating scale of TripAdvisor dataset is 1-5. The rating scale of BeerAdvocate dataset is 1-10.



Compared to SVM and SLDA, NBoW achieves higher accuracy by 3% in both datasets, which shows that embedding features are more effective than traditional ngram features on these two datasets. All neural network models outperform NBoW. It shows the advantages of neural networks in the document sentiment classification.

From the results of neural networks, we can observe that DAN performs worse than LSTM and CNN, and LSTM achieves slightly higher results than CNN. It can be explained that the simple composition method averaging embeddings of words in a document but ignoring word order, may not be as effective as other flexible composition models, such as LSTM and CNN, on aspect classification. Additionally, we observe that the multi-task learning and hierarchical architecture are beneficial for neural networks. Among all baselines, MHAN and MHLSTM achieve comparable results and outperform others.

Compared with MHAN and MHLSTM, our method achieves improvements of 1.5% (3% relative improvement) and 1.0% (2.5% relative improvement) on TripAdvisor and BeerAdvocate respectively, which shows that the incorporation of iterative attention mechanism helps the deep neural network based model build up more discriminative aspect-aware representation. Note that BeerAdvocate is relatively more difficult since the predicted ratings are from 1 to 10 while TripAdvisor is 1 to 5. Moreover, t-test is conducted by randomly splitting datasets into train/dev/test sets and random initialization. The results on test sets are described in Table 3 which show performance of our model is stable.

### Case Study for Attention Results

In this section, we sample two sentences from TripAdvisor to show the visualization of attention results for case study. Both word-level and sentence-level attention visualizations are shown in Figure 4. We normalize the word weight by the sentence weight to make sure that only important words in a document are highlighted.

From the top figures in (a) and (b), we observe that our model assigns different attention weights for each aspect. For example, in the first sentence, the words _confortable_ and _bed_ are assigned higher

\begin{table}
\begin{tabular}{l|c c|c c|c c|c c} \hline  & \multicolumn{6}{c|}{**TripAdvisor**} & \multicolumn{6}{c}{**BeerAdvocate**} \\ \cline{2-9} Model & Dev & Test & \multicolumn{2}{c|}{Dev} & \multicolumn{2}{c}{Test} \\ \cline{2-9}  & Accuracy & MSE & Accuracy & MSE & Accuracy & MSE & Accuracy & MSE \\ \hline Majority & 24.47 & 2.533 & 23.89 & 2.549 & 24.48 & 4.706 & 24.41 & 4.545 \\ SVM & 34.30 & 1.982 & 35.26 & 1.963 & 25.70 & 3.286 & 25.79 & 3.270 \\ SLDA & 31.58 & 2.131 & 32.81 & 2.110 & 25.39 & 3.372 & 25.73 & 3.391 \\ NBoW & 38.43 & 1.866 & 39.09 & 1.808 & 28.99 & 2.883 & 28.85 & 2.919 \\ \hline DAN & 40.30 & 1.569 & 40.93 & 1.531 & 31.25 & 2.569 & 32.44 & 2.279 \\ CNN & 43.25 & 1.474 & 43.35 & 1.456 & 34.17 & 2.173 & 33.37 & 2.217 \\ LSTM & 43.85 & 1.525 & 44.02 & 1.470 & 35.23 & 2.112 & 34.78 & 2.097 \\ HAN & 44.47 & 1.312 & 44.68 & 1.301 & 36.57 & 1.903 & 36.03 & 1.920 \\ \hline MHDAN & 42.22 & 1.554 & 42.47 & 1.549 & 32.76 & 2.358 & 32.54 & 2.376 \\ MHCNN & 44.19 & 1.329 & 43.79 & 1.398 & 36.10 & 1.966 & 35.33 & 1.976 \\ MHLSTM & 44.53 & 1.308 & 44.72 & 1.272 & 38.14 & 1.785 & 37.04 & 1.809 \\ MHAN & 44.72 & 1.294 & 44.94 & 1.210 & 37.98 & 1.783 & 36.82 & 1.813 \\ \hline Our & 46.21 & 1.091 & **46.56** & **1.083** & 39.43 & 1.696 & **38.06** & **1.755** \\ \hline \end{tabular}
\end{table}
Table 2: Comparison of our model and other baseline methods.

\begin{table}
\begin{tabular}{l|c c|c c} \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c|}{**TripAdvisor**} & \multicolumn{2}{c}{**BeerAdvocate**} \\ \cline{2-5}  & Accuracy & MSE & Accuracy & MSE \\ \hline MHLSTM & 44.75 (0.24) & 1.256 (0.05) & 37.28 (0.43) & 1.802 (0.17) \\ MHAN & 45.02 (0.33) & 1.221 (0.12) & 37.02 (0.22) & 1.810 (0.15) \\ \hline Our & **46.65’**(0.29) & **1.084°**(0.06) & **38.25′**(0.35) & **1.749°**(0.18) \\ \hline \end{tabular}
\end{table}
Table 3: The results of average accuracy/MSE and standard deviation of models on test sets. We choose _MHAN_ and _MHLSTM_ as comparison baselines for TripAdvisor and BeerAdvocate respectively. In t-tests, the marker \({}^{*}\) refers to p-value \(<\) 0.05 and the marker † refers to p-value \(<\) 0.01.

 

weights in the aspect _Room_, and the word _clean_ are highlighted by the aspect _Cleaniness_. In the second sentence, the word _internet_ is assigned a high attention value for _Business_. Moreover, the bottom figures in (a) and (b) show that (1) word weights of different hops are various; (2) attention values in higher hop are more reasonable. Specifically, in the first sentence, the weight of word _clean_ is higher than the word _comfortable_ in first hop, while _comfortable_ surpasses _clean_ in higher hops. In the second sentence, we observe that the value of word _internet_ increases with the number of hop. Thus, we can see that the more sensible weights are obtained for words through the proposed iterative attention mechanism. Similarly, the figures (c) and (d) show that the conclusion from words is also suitable for sentences. For the first sentence, the sentence weight regarding the aspect _Room_ is lower than _Cleanliness_ in the first hop, but surpasses _Cleanliness_ in the second hop. For the second sentence, the weight for _Business_ becomes higher in the second hop.

### Effects of Hop and Aspect Keywords

In this experiment, we investigate the effects of hop number \(m\) and size of aspect keywords \(N\) on performances. All the experiments are conducted on the development set. Due to lack of space, we only present the results of TripAdvisor and the results of BeerAdvocate have a similar behavior as TripAdvisor.

For the hop number, we vary \(m\) from 1 to 7 and the results are shown in Figure 5 (left). We can see that: (1) at the word level, the performance increases when \(m\leq 4\), but shows no improvement after \(m>4\); (2) at the sentence level, model performs best when \(m=2\). Moreover, we can see that the hop number of word level leads to larger variation than the hop number of sentence level.

For the size of aspect keywords, we vary \(N\) from 0 to 35, incremented by 5. Note that, we set a learnable vector to represent question memory when \(N=0\). The results are shown in Figure 5 (right). We observe that the performance increases when \(N\leq 20\), and has no improvement after \(N>20\). This indicates that a small number of keywords can help the proposed model achieve competitive results.

## 4 Related Work

**Multi-Aspect Sentiment Classification.** Multi-aspect sentiment classification has been studied extensively in literature. Lu et al. (2011) used support vector regression model based on hand

Figure 4: The attention visualization of words and sentences. Darker color means higher weight. (a) and (b) are the visualization of word weights; (c) and (d) are the visualization of sentence weights. The top figures in (a) and (b) show the word weights of fourth hop for each aspect. The bottom figures in (a) and (b) visualize the word weights of different hops for the aspects _Room_ and _Business_ respectively.

 

crafted features to predict aspect ratings. To handle the correlation between aspects, McAuley et al. (2012) added a dependency term in final multi-class SVM objective. There were also some heuristic based methods and sophisticated topic models where multi-aspect sentiment classification is solved as a subproblem Titov and McDonald (2008); Wang et al. (2010); Diao et al. (2014); Pappas and Popescu-Belis (2014). However, these approaches often rely on strict assumptions about words and sentences, for example, using the word syntax to determine if a word is an aspect or a sentiment word, or relating a sentence with an specific aspect. Another related problem is called aspect-based sentiment classification Pontiki et al. (2014, 2016); Poria et al. (2016), which first extracts aspect expressions from sentences Poria et al. (2014); Balahur and Montoyo (2008); Chen et al. (2014, 2013), and then determines their sentiments. With the developments of neural networks and word embeddings in NLP, neural network based models have shown the state-of-the-art results with less feature engineering work. Tang et al. (2016) employed a deep memory network for aspect-based sentiment classification given the aspect location and Lakkaraju et al. (2014) employed recurrent neural networks and its variants for the task of extraction of aspect-sentiment pair. However, these tasks are sentence-level. Another related research field is document-level sentiment classification because we can treat single aspect sentiment classification as an individual document classification task. This line of research includes Tang et al. (2015); Chen et al. (2016); Tang et al. (2016); Yang et al. (2016) which are based on neural networks in a hierarchical structure. However, they did not work on multiple aspects.

**Machine Comprehension.** Recently, neural network based machine comprehension (or reading) has been studied extensively in NLP, with the releases of large-scale evaluation datasets Hermann et al. (2015); Hill et al. (2016); Rajpurkar et al. (2016). Most of the related studies focus on attention mechanism Bahdanau et al. (2014) which is firstly proposed in machine translating and aims to solve the long-distance dependency between words. Hermann et al. (2015) used BiLSTM to encode document and query, and proposed Attentive Reader and Impatient Reader. The first one attends document based on the query representation, and the second one attends document by the representation of each token in query with an incremental manner. Memory Networks Weston et al. (2015); Sukhbaatar et al. (2015) attend and reason document representation in a multihop fashion, enriching interactions between documents and questions. Dynamic Memory Network Kumar et al. (2016) updates memories of documents by re-running GRU models based on derived attention weights. Meanwhile, the query representation is refined by another GRU model. Gated-Attention Reader Dhingra et al. (2016) proposes a novel attention mechanism, which is based on multiplicative interactions between the query embeddings and the intermediate states of a recurrent neural network document reader. Directional Attention Model Xiong et al. (2017); Seo et al. (2017) fuses co-dependent representations of queries and documents in order to focus on relevant parts of both. Iterative Attention model Sordoni et al. (2016) attends question and document sequentially, which is related to our model. Different from Iterative Attention model, our model focuses on the document-level multi-aspect sentiment classification, which is proposed

Figure 5: Results of different hops and different sizes of question keywords. Left: different the hop numbers; Right: different sizes of keywords.

 

in a hierarchical architecture and has different procedures in the iterative attention module. Another related research problem is visual question answering which uses an image as question context rather than a set of keywords as question. Neural network based visual question answering [16, 20] is similar as the proposed models in text comprehension.

## 5 Conclusion

In this paper, we model the document-level multi-aspect sentiment classification as a text comprehension problem and propose a novel hierarchical iterative attention model in which documents and pseudo aspect-questions are interleaved at both word and sentence-level to learn aspect-aware document representation in a unified model. Extensive experiments show that our model outperforms the other neural models with multi-task framework and hierarchical architecture.

## 6 Acknowledgments

This paper is partially supported by the National Natural Science Foundation of China (NSFC Grant Nos. 61472006 and 91646202) as well as the National Basic Research Program (973 Program No. 2014CB340405). This work was also supported by NVIDIA Corporation with the donation of the Titan X GPU, Hong Kong CERG Project 26206717, China 973 Fundamental R&D Program (No.2014CB340304), and the LORELEI Contract HR0011-15-2-0025 with DARPA. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. We also thank the anonymous reviewers for their valuable comments and suggestions that help improve the quality of this manuscript.

## References

* Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. In _Proceedings of ICLR_.
* Balahur and Montoyo (2008) Alexandra Balahur and Andres Montoyo. 2008. A feature dependent method for opinion mining and classification. In _Natural Language Processing and Knowledge Engineering_. pages 1-7.
* Blei and Mcauliffe (2010) David M. Blei and Jon D. Mcauliffe. 2010. Supervised topic models. _Advances in Neural Information Processing Systems_ 3:327-332.
* Caruana (1997) Rich Caruana. 1997. Multitask learning. _Machine Learning_ 28(1):41-75.
* Chen et al. (2016) Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. 2016. Neural sentiment classification with user and product attention. In _Proceedings of EMNLP_. pages 1650-1659.
* Chen et al. (2014) Zhiyuan Chen, Arjun Mukherjee, and Bing Liu. 2014. Aspect extraction with automated prior knowledge learning. In _ACL_. pages 347-358.
* Chen et al. (2013) Zhiyuan Chen, Arjun Mukherjee, Bing Liu, Meichun Hsu, Malu Castellanos, and Riddhiman Ghosh. 2013. Exploiting domain knowledge in aspect extraction. In _EMNLP_. pages 1655-1667.
* Collobert et al. (2011) Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. _Journal of Machine Learning Research_ 12(Aug):2493-2537.
* Dhingra et al. (2016) Bhuwan Dingra, Hanxiao Liu, William W Cohen, and Ruslan Salakhutdinov. 2016. Gated-attention readers for text comprehension. _arXiv preprint arXiv:1606.01549_.
* Diao et al. (2014) Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J Smola, Jing Jiang, and Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In _Proceedings of KDD_. ACM, pages 193-202.
* Fan et al. (2008) Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. Liblinear: A library for large linear classification. _Journal of machine learning research_ 9(Aug):1871-1874.
* Hermann et al. (2015) Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In _Proceedings of NIPS_. pages 1693-1701.
* Hill et al. (2016) Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2016. The goldilocks principle: Reading children's books with explicit memory representations. In _Proceedings of ICLR_.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural computation_ 9(8):1735-1780.
* Iyyer et al. (2015) Mohit Iyyer, Varun Manjunatha, Jordan L BoydGraber, and Hal Daume III. 2015. Deep unordered composition rivals syntactic methods for text classification. In _Proceedings of ACL_. pages 1681-1691.
* Kim (2014) Yoon Kim. 2014. Convolutional neural networks for sentence classification. In _Proceedings of EMNLP_. pages 1746-1751.
* Kumar et al. (2016) Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, and Richard Socher. 2016. Ask me anything: Dynamic memory networks for natural language processing. In _Proceedings of ICML_ 

Himabindu Lakkaraju, Richard Socher, and Chris Manning. 2014. Aspect specific sentiment analysis using hierarchical deep learning. In _NIPS Workshop on Deep Learning and Representation Learning_.
* Lei et al. (2016) Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In _Proceedings of EMNLP_. Austin, Texas, pages 107-117.
* Liu (2010) Bing Liu. 2010. Sentiment analysis and subjectivity. In _Handbook of Natural Language Processing, Second Edition._, pages 627-666.
* Lu et al. (2011) Bin Lu, Myle Ott, Claire Cardie, and Benjamin K Tsou. 2011. Multi-aspect sentiment analysis with topic models. In _ICDM Workshops_. IEEE, pages 81-88.
* Lu et al. (2016) Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016. Hierarchical question-image co-attention for visual question answering. In _Proceedings of NIPS_. pages 289-297.
* Luong et al. (2016) Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task sequence to sequence learning. In _Proceedings of ICLR_.
* McAuley et al. (2012) Julian McAuley, Jure Leskovec, and Dan Jurafsky. 2012. Learning attitudes and attributes from multi-aspect reviews. In _Proceedings of ICDM_. IEEE, pages 1020-1025.
* Mikolov et al. (2013) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In _Proceedings of NIPS_. pages 3111-3119.
* Pang and Lee (2007) Bo Pang and Lillian Lee. 2007. Opinion mining and sentiment analysis. _Foundations and Trends in Information Retrieval_ 2(1-2):1-135.
* Pappas and Popescu-Belis (2014) Nikolaos Pappas and Andrei Popescu-Belis. 2014. Explaining the stars: Weighted multiple-instance learning for aspect-based sentiment analysis. In _Proceedings of EMNLP_. pages 455-466.
* Pontiki et al. (2016) Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphee De Clercq, Veronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia Loukachevitch, Evgeniy Kotelnikov, Noria Bel, Salud Maria Jimenez-Zafra, and Gulsen Eryigit. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In _Proceedings of SemEval_. pages 19-30.
* Pontiki et al. (2014) Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In _Proceedings of SemEval_. pages 27-35.
* Poria et al. (2014) Soujanya Poria, Erik Cambria, Lun-Wei Ku, Chen Gui, and Alexander Gelbukh. 2014. A rule-based approach to aspect extraction from product reviews. In _Proceedings of the second workshop on natural language processing for social media (SocialNLP)_. pages 28-37.
* Poria et al. (2016) Soujanya Poria, Iti Chaturvedi, Erik Cambria, and Federica Bisio. 2016. Sentic lda: Improving on Ida with semantic similarity for aspect-based sentiment analysis. In _International Joint Conference on Neural Networks_. pages 4465-4473.
* Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In _Proceedings of EMNLP_. pages 2383-2392.
* Seo et al. (2017) Minjoom Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. _Proceedings of ICLR_ .
* Sordoni et al. (2016) Alessandro Sordoni, Philip Bachman, Adam Trischler, and Yoshua Bengio. 2016. Iterative alternating neural attention for machine reading. _arXiv preprint arXiv:1606.02245_ .
* Sakhbaatar et al. (2015) Sainbayar Sakhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In _Proceedings of NIPS_. pages 2440-2448.
* Tang et al. (2015a) Duyu Tang, Bing Qin, and Ting Liu. 2015a. Document modeling with gated recurrent neural network for sentiment classification. In _EMNLP_. pages 1422-1432.
* Tang et al. (2015b) Duyu Tang, Bing Qin, and Ting Liu. 2015b. Learning semantic representations of users and products for document level sentiment classification. In _ACL_. pages 1014-1023.
* Tang et al. (2016) Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. In _Proceedings of EMNLP_. pages 214-224.
* Team (2016) Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. _arXiv e-prints_ abs/1605.02688. [http://arxiv.org/abs/1605.02688](http://arxiv.org/abs/1605.02688).
* Titov and McDonald (2008) Ivan Titov and Ryan T McDonald. 2008. A joint model of text and aspect ratings for sentiment summarization. In _Proceedings of ACL_. Citeseer, volume 8, pages 308-316.
* Wang et al. (2010) Hongning Wang, Yue Lu, and Chengxiang Zhai. 2010. Latent aspect rating analysis on review text data: a rating regression approach. In _Proceedings of KDD_. ACM, pages 783-792.
* Weston et al. (2015) Jason Weston, Sumit Chopra, and Antoine Bordes. 2015. Memory networks. In _Proceedings of ICLR_.
* Xiong et al. (2016) Caiming Xiong, Stephen Merity, and Richard Socher. 2016. Dynamic memory networks for visual and textual question answering. In _Proceedings of ICML_. pages 1378-1387 

Caiming Xiong, Victor Zhong, and Richard Socher. 2017 Dynamic coattention networks for question answering. _Proceedings of ICLR_ .
* Yang et al. (2016) Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016 Hierarchical attention networks for document classification. In _Proceedings of NAACL-HLT_. pages 1480-1489.
* Zeiler (2012) Matthew D Zeiler. 2012. Adadelta: an adaptive learning rate method. _arXiv preprint arXiv:1212.5701_ .