Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding (2020)

Abstract: Aspect-based sentiment analysis of review texts is of great value for understanding user feedback in a fine-grained manner. It has in general two sub-tasks: (i) extracting aspects from each review, and (ii) classifying aspect-based reviews by sentiment polarity. In this paper, we propose a weakly-supervised approach for aspect-based sentiment analysis, which uses only a few keywords describing each aspect/sentiment without using any labeled examples. Existing methods are either designed only for one of the sub-tasks, or are based on topic models that may contain overlapping concepts. We propose to first learn <sentiment, aspect> joint topic embeddings in the word embedding space by imposing regularizations to encourage topic distinctiveness, and then use neural models to generalize the word-level discriminative information by pre-training the classifiers with embedding-based predictions and self-training them on unlabeled data. Our comprehensive performance analysis shows that our method generates quality joint topics and outperforms the baselines significantly (7.4% and 5.1% F1-score gain on average for aspect and sentiment classification respectively) on benchmark datasets.

## introduction with the vast amount of reviews emerging on platforms like amazon and yelp, aspect-based sentiment analysis, which extracts opinions about certain facets of entities from text, becomes increasingly essential and benefits a wide range of downstream applications @xcite . aspect-based sentiment analysis contains two sub-tasks: aspect extraction and sentiment polarity classification. the former identifies the aspect s1: eye-pleasing with semiprivate booths, place for a date. s2: mermaid inn is an overall good restaurant with really good seafood. figure 1 : two sample restaurant reviews. pure aspect words are in red and wavy-underlined, and general opinion words are in blue and framed in boxes. words implying both aspects and opinions (which we define as joint topics) are underlined and in purple. covered in the review, whereas the latter decides its sentiment polarity. various methods have been proposed for the task. neural network models @xcite have outperformed rule-based models @xcite ), but they require large-scale fine-grained labeled data to train, which can be difficult to obtain. some other studies leverage word embeddings to solve the aspect extraction problem in an unsupervised @xcite or weaklysupervised setting @xcite , without using any annotated documents. in this work, we study the weakly-supervised setting, where only a few keywords are provided for each aspect and sentiment. we show two sample restaurant reviews in fig. 1 together with their expected output-aspect and sentiment labels. with a closer look at these two example reviews, we observe that s2 includes a general opinion word "good" and a pure aspect word "seafood", which are separate hints for sentiment and aspect classification respectively. s1, on the other hand, does not address the target with plain and general words, but instead use more specific words like "semi-private" and "date" which are uniquely used when people feel good about the ambience instead of other aspects. humans can interpret these unique and fine-grained terms as hints for a joint topic of good, ambience , but this is hard for models that are solely trained for one sub-task. if a model can automatically learn the semantics of each joint topic of sentiment, aspect , it will be able to identify representative terms of the joint topics such as "semi-private" which provide information for aspect and sentiment simultaneously, and will consequently benefit both aspect extraction and sentiment classification. therefore, leveraging more fine-grained information by coupling the two subtasks will enhance both. several lda-based methods consider learning joint topics @xcite @xcite , but they rely on external resources such as part-of-speech (pos) tagging or opinion word lexicons. a recent lda-based model @xcite uses pre-trained word embedding to bias the prior in topic models to jointly model aspect words and opinion words. though working fairly well, topic models are generative models and do not enforce topic distinctivenesstopic-word distribution can largely overlap among different topics, allowing topics to resemble each other. besides, topic models yield unstable results, causing large variance in classification results. we propose the jasen model for joint aspect-sentiment topic embedding. our general idea is to learn a joint topic representation for each sentiment, aspect pair in the shared embedding space with words so that the surrounding words of topic embeddings nicely describe the semantics of a joint topic. this is accomplished by training topic embeddings and word embeddings on in-domain corpora and modeling the joint distribution of usergiven keywords on all the joint topics. after learning the joint topic vectors, embedding-based predictions can be derived for any unlabeled review. however, these predictions are sub-optimal for sentiment analysis where word order plays an important role. to leverage the expressive power of neural models, we distill the knowledge from embeddingbased predictions to convolutional neural networks (cnns) @xcite which perform compositional operations upon local sequences. a self-training process is then conducted to refine cnns by using their high-confident predictions on unlabeled data. we demonstrate the effectiveness of jasen by conducting experiments on two benchmark datasets and show that our model outperforms all the baseline methods by a large margin. we also show that our model is able to describe joint topics with coherent term clusters. our contributions can be summarized as follows: (1) we propose a weakly-supervised method jasen to enhance two sub-tasks of aspect-based sentiment analysis. our method does not need any annotated data but only a few keywords for each aspect/sentiment. (2) we introduce an embedding learning objective that is able to capture the semantics of fine-grained joint topics of sentiment, aspect in the word embedding space. the embedding-based prediction is effectively leveraged by neural models to generalize on unlabeled data via self-training. (3) we demonstrate that jasen generates high-quality joint topics and outperforms baselines significantly on two benchmark datasets. ## related work the problem of aspect-based sentiment analysis can be decomposed into two sub-tasks: aspect extraction and sentiment polarity classification. most previous studies deal with them individually. there are various related efforts on aspect extraction @xcite , which can be followed by sentiment classification models @xcite . other methods @xcite jointly solve these two sub-tasks by first separating target words from opinion words and then learning joint topic distributions over words. below we first review relevant work on aspect extraction (sec 2.1) and then turn to studies that jointly extract aspects and sentiment polarity (sec 2.2). ## problem definition our weakly-supervised aspect-based sentiment analysis task is defined as follows. the input is a training corpus @xmath0, d 2 , . . . , d |d| } of text reviews from a certain domain (e.g., restaurant or laptop) without any label for aspects or sentiment. a list of keywords l a for each aspect topic (denoted as @xmath1. for each unseen review in the same domain, our model outputs a set of s, a labels. ## model figure 2 shows the workflow of jasen. our goal is to generate a set of sentiment, aspect predictions for each review. we first learn an embedding space to explicitly represent the semantics of the topics (including both pure aspect/sentiment and joint sentiment, aspect ones) as embedding vectors, which are surrounded by the embeddings of the representative words of the topics. we also impose discriminative regularization on the embedding space to push different topics apart. to model the local sequential information which is crucial for sentiment analysis, we use cnn as the classifier by pre-training it on pseudo labels given by the cosine similarity between document embeddings and topic embeddings, and self-training it on unlabeled data to iteratively refine its parameters. below we introduce the details of jasen. ## evaluation we conduct a series of quantitative and qualitative evaluation on benchmark datasets to demonstrate the effectiveness of our model. ## conclusion in this paper we propose to enhance weaklysupervised aspect-based sentiment analysis by learning the representation of sentiment, aspect joint topic in the embedding space to capture more fine-grained information. we introduce an