Adversarial Learning of P oisson Factorisation Model for Gauging Brand Sentiment in User Reviews (2021)

Abstract: In this paper, we propose the Brand-Topic Model (BTM) which aims to detect brand-associated polarity-bearing topics from product reviews. Different from existing models for sentiment-topic extraction which assume topics are grouped under discrete sentiment categories such as ‘positive’, ‘negative’ and ‘neural’, BTM is able to automatically infer real-valued brand-associated sentiment scores and generate fine-grained sentiment-topics in which we can observe continuous changes of words under a certain topic (e.g., ‘shaver’ or ‘cream’) while its associated sentiment gradually varies from negative to positive. BTM is built on the Poisson factorisation model with the incorporation of adversarial learning. It has been evaluated on a dataset constructed from Amazon reviews. Experimental results show that BTM outperforms a number of competitive baselines in brand ranking, achieving a better balance of topic coherence and unique-ness, and extracting better-separated polarity-bearing topics.

## introduction market intelligence aims to gather data from a company's external environment, such as customer surveys, news outlets and social media sites, in order to understand customer feedback to their products and services and to their competitors, for a better decision making of their marketing strategies. since consumer purchase decisions are heavily influenced by online reviews, it is important to automatically analyse customer reviews for online brand monitoring. existing sentiment analysis models either classify reviews into discrete polarity categories such as 'positive', 'negative' or 'neural', or perform more fine-grained sentiment analysis, in which aspect-level sentiment label is predicted, though still in the discrete polarity category space. we argue that it is desirable to be able to detect subtle topic changes under continuous sentiment scores. this allows us to identify, for example, whether customers with slightly negative views share similar concerns with those holding strong negative opinions; and what positive aspects are praised by customers the most. in addition, deriving brand-associated sentiment scores in a continuous space makes it easier to generate a ranked list of brands, allowing for easy comparison. existing studies on brand topic detection were largely built on the latent dirichlet allocation (lda) model @xcite which assumes that latent topics are shared among competing brands for a certain market. they however are not able to separate positive topics from negative ones. approaches to polarity-bearing topic detection can only identify topics under discrete polarity categories such as 'positive' and 'negative'. we instead assume that each brand is associated with a latent real-valued sentiment score falling into the range of @xcite in which -1 denotes negative, 0 being neutral and 1 positive, and propose a brand-topic model built on the poisson factorisation model with adversarial learning. example outputs generated from btm are shown in figure 1 in which we can observe a transition of topics with varying topic polarity scores together with their associated brands. more concretely, in btm, a document-word count matrix is factorised into a product of two positive matrices, a document-topic matrix and a topic-word matrix. a word count in a document is assumed drawn from a poisson distribution with its rate parameter defined as a product of a documentspecific topic intensity and its word probability under the corresponding topic, summing over all topics. we further assume that each document is associated with a brand-associated sentiment score and a latent topic-word offset value. the occurrence count of a word is then jointly determined by both the brand-associated sentiment score and the topic-word offset value. the intuition behind is that if a word tends to occur in documents with positive polarities, but the brand-associated sentiment score is negative, then the topic-word offset value will have an opposite sign, forcing the occurrence count of such a word to be reduced. furthermore, for each document, we can sample its word counts from their corresponding poisson distributions and form a document representation which is subsequently fed into a sentiment classifier to predict its sentiment label. if we reverse the sign of the latent brand-associated sentiment score and sample the word counts again, then the sentiment classifier fed with the resulting document representation should generate an opposite sentiment label. our proposed btm is partly inspired by the recently developed text-based ideal point (tbip) model @xcite in which the topic-specific word choices are influenced by the ideal points of authors in political debates. however, tbip is fully unsupervised and when used in customer reviews, it generates topics with mixed polarities. on the contrary, btm makes use of the document-level sentiment labels and is able to produce better separated polarity-bearing topics. as will be shown in the experiments section, btm outperforms tbip on brand ranking, achieving a better balance of topic coherence and topic uniqueness measures. the contributions of the model are three-fold: the rest of the paper is organised as follows. section 2 presents the related work. section 3 describes our proposed brand-topic model. section 4 and 5 discusses the experimental setup and evaluation results, respectively. finally, section 5 concludes the paper and outlines the future research directions. our work is related to the following research: poisson factorisation models poisson factorisation is a class of non-negative matrix factorisation in which a matrix is decomposed into a product of matrices. it has been used in many personalise application such as personalised budgets recommendation @xcite , ranking @xcite , or content-based social recommendation @xcite . poisson factorisation can also be used for topic modelling where a document-word count matrix is factorised into a product of two positive matrices, a document-topic matrix and a topic-word matrix @xcite . in such a setup, a word count in a document is assumed drawn from a poisson distribution with its rate parameter defined as a product of a document-specific topic intensity and its word probability under the corresponding topic, summing over all topics. polarity-bearing topics models early approaches to polarity-bearing topics extraction were built on lda in which a word is assumed to be generated from a corpus-wide sentiment-topicword distributions @xcite . in order to be able to separate topics bearing different polarities, word prior polarity knowledge needs to be incorporated into model learning. in recent years, the neural network based topic models have been proposed for many nlp tasks, such as information retrieval @xcite , aspect extraction @xcite and sentiment classification @xcite . most of them are built upon variational autoencode (vae) (kingma and welling, 2014) which constructs a neural network to approximate the topic-word distribution in probabilistic topic models @xcite @xcite . intuitively, training the vae-based supervised neural topic models with class labels @xcite @xcite can introduce sentiment information into topic modelling, which may generate better features for sentiment classification. ## brand-topic model (btm) we propose a probabilistic model for monitoring the assessment of various brands in the beauty market from amazon reviews. we extend the text-based ideal point (tbip) model with adversarial learning and gumbel-softmax to construct document features for sentiment classification. the overall architecture of our proposed btm is shown in figure 2 . in what follows, we will first give a brief introduction of tbip, followed by the presentation of our proposed btm. ## experimental setup datasets we construct our dataset by retrieving reviews in the beauty category from the amazon review corpus foot_0 @xcite . each review is accompanied with the rating score (between 1 and 5), reviewer name and the product meta-data such as product id, description, brand and image. we use the product meta-data to relate a product with its associated brand. by only selecting brands with relatively more and balanced reviews, our final dataset contains a total of 78,322 reviews from 45 brands. reviews with the rating score of 1 and 2 are grouped as negative reviews; those with the score of 3 are neutral reviews; and the remaining are positive reviews. the statistics of our dataset is shown in baselines we compare the performance of our model with the following baselines: • joint sentiment-topic (jst) model @xcite , built on lda, can extract polaritybearing topics from text provided that it is supplied with the word prior sentiment knowledge. in our experiments, the mpqa subjectivity lexicon 3 is used to derive the word prior sentiment information. parameter setting since documents are represented as the bag-of-words which result in the loss of word ordering or structural linguistics information, frequent bigrams and trigrams such as 'without doubt', 'stopped working', are also used as features for document representation construction. tokens, i.e., n-grams (@xmath0, 2, 3}), occurred less than twice are filtered. in our experiments, we set aside 10% reviews (7,826 reviews) as the test set and the remaining (70,436 reviews) as the training set. for hyperparameters, we set the batch size to 1,024, the maximum training steps to 50,000, the topic number to 30, the temperature in the gumbel-softmax equation in section 3.2 to 1. since our dataset is highly imbalanced, we balance data in each mini-batch by oversampling. for a fair comparison, we report two sets of results from the baseline models, one trained from the original data, the other trained from the balanced training data by oversampling negative reviews. the latter results in an increased training set consisting of 113,730 reviews. ## experimental results in this section, we will present the experimental results in comparison with the baseline models in brand ranking, topic coherence and uniqueness measures, and also present the qualitative evaluation of the topic extraction results. we will further discuss the limitations of our model and outline future directions. ## conclusion in this paper, we presented the brand-topic model, a probabilistic model which is able to generate polarity-bearing topics of commercial brands. compared to other topic models, bmt infers realvalued brand-associated sentiment scores and extracts fine-grained sentiment-topics which vary smoothly in a continuous range of polarity scores. it builds on the poisson factorisation model, combining it with an adversarial learning mechanism to induce better-separated polarity-bearing topics. experimental evaluation on amazon reviews against several baselines shows an overall improvement of topic quality in terms of coherence, uniqueness and separation of polarised topics.