Sentiment Analysis using the Relationship between Users and Products (2023)

Abstract: In product reviews, user and product aspects are useful in sentiment analysis. Nevertheless, previous studies mainly focus on modeling user and product aspects without considering the relationship between users and products. The relationship between users and products is typically helpful in estimating the bias of a user toward a product. In this paper, we, therefore, introduce the Graph Neural Network-based model with the pre-trained Language Model (GNNLM), where the relationship between users and products is incorporated. We conducted experiments on three well-known benchmarks for sentiment classification with the user and product information. The experimental results show that the relationship between users and products improves the performance of sentiment analysis. Furthermore, GNNLM achieves state-of-the-art results on yelp-2013 and yelp-2014 datasets.

## introduction sentiment analysis aims to understand a user's opinion toward a product. it is to infer the sentiment polarity or intensity on a review of a document @xcite . recently, user and product information in a review has been proven to be helpful for sentiment analysis models @xcite . consequently, many studies investigate how to model user and product aspects and incorporate them into deep neural network models. nevertheless, none of them focuses on the relationship between users and products. this relationship between users and products typically provides the bias of a user's sentiment toward a product. for example, users a and b share similar sentiments on many products. if there is a product for which we do not know user a's sentiment, but we know user b's sentiment, we might be able to infer user a's sentiment from user b's sentiment. in addition, if a user has a high expectation toward the product, but the product does not meet the expectation, it would greatly impact the user's sentiment. meanwhile, the interaction between users and products has proven to be useful in other tasks, such as spam detection @xcite and citation recommendation @xcite . based on these observations, we assume that the relationship between users and products could provide a clue to help sentiment analysis. in this paper, we, therefore, propose a new approach using graph neural networks with the pre-trained language model, namely gnnlm. in gnnlm, the relationship between the user and the product is captured by the graph neural network model as distributed representations and then combined with a distributed representation of reviews obtained from a pre-trained language model to predict the sentiment label. we conduct experiments on three benchmarks @xcite for sentiment classification with the user and product information. the results show that combining the relationship between the user and the product could help improve the performance of the sentiment analysis model. ## related work recent studies have shown that user and product information is useful for sentiment analysis. the first study @xcite argues that user and product information are consistent with a sentiment from a review. they propose upnn that incorporates the user and product preference matrix into a cnn-based model to modify the meaning of word representation. updmn @xcite uses a deep memory network to capture the user and product preferences with the lstm-based model. nsc @xcite is the model using a hierarchical neural network with the attention mechanism to capture global user and product information. hcsc @xcite investigates the cold start problem for sentiment analysis with the user and product information by introducing shared user and product representations. dupmn @xcite uses a hierarchical lstm-based model to encode the document with dual memory networks, one for user information and the other for production information. cma @xcite encodes the document using a hierarchical lstm-based model, in which user and product information are injected hierarchically. bilstm + basis-cust @xcite is a model that combines categorical metadata of users and products into the neural network model. chim (amplayo, 2019) utilizes chunk-wise matrices to represent the user and product aspects and injects them into different locations of the model. iupc @xcite ) is a model built on stacked attention with bert to memorize historical reviews of a user and all reviews of a product. ma-bert @xcite ) is a multiattribute bert, where user and product aspects are incorporated into the bert model. based on our survey, none of them investigates the relationship between users and products for sentiment analysis. ## our approach as shown in fig. 1 , our approach, gnnlm, consists of three components: 1) graph neural networks, 2) pre-trained language model, and 3) classification layer. the task definition and the details of each component are described as follows. ## conclusion this paper introduces gnnlm, gnns with the pre-trained language model for sentiment analysis with user and product information. unlike previous studies, we incorporate the relationship between users and products into the model using gnns. experimental results show that the representations learned from the relationship between users and products contribute to sentiment analysis models. in the future, we will attempt to model user and product aspects from reviews into the graph.