Sentiment Analysis on Streaming User Reviews via Dual-Channel Dynamic Graph Neural Network (2023)

Abstract: Sentiment analysis on user reviews has achieved great success thanks to the rapid growth of deep learning techniques. The large number of online streaming reviews also provides the opportunity to model temporal dynamics for users and products on the timeline. However, existing methods model users and products in the real world based on a static assumption and neglect their time-varying characteristics. In this paper, we present DC-DGNN, a dual-channel framework based on a dynamic graph neural network (DGNN) that models temporal user and product dynamics for sentiment analysis. Specifically, a dual-channel text encoder is employed to extract current local and global contexts from review documents for users and products. Moreover, user review streams are integrated into the dynamic graph neural network by treating users and products as nodes and reviews as new edges. Node representations are dynamically updated along with the evolution of the dynamic graph and used for the final score prediction. Experimental results on five real-world datasets demonstrate the superiority of the proposed method.

## introduction sentiment analysis on user reviews, inferring the overall sentiment polarity (e.g. 1-5 stars on the review site amazon) of a user-written review document for a product, has gained popularity with the rapid growth of online review sites such as amazon, yelp, and imdb. compared to other sentiment analysis tasks @xcite @xcite , not only the text itself but also the user and product information are crucial for the final rating score prediction. for sentiment analysis on user reviews, early methods incorporate user and product embeddings by training randomly initialized embedding with the classifier @xcite @xcite negative words with gradually positive ratings p slow, steady boring. very much unlike a coben book. i was very disappointed, but needed to see why others think this is a wonderful novel. reiew #2 09 03, 2016 i read his latest book, fool me once, which i not only disliked, i thought it was terrible. it is written in a fastpaced style that makes it a very quick read. unfortunately, that's the one and only good thing i can say about it. reiew #1 04 @xcite this wasn't one of harlan coben's great stories. the main character was unlikable. there was no one in the entire story i could associate with. reiew #3 10 03, 2016 5 months 1 month positive words with gradually negative ratings u i was surprised that i liked this book as much as i did. i listened to the audio version of frances fuller, her reading speed was really good; i like a fast reader. reiew #1 04 20, 2015 i liked the nostalgic tone of the book and the style of writing. the characters felt very much a part of the world they were set in. 08 26, 2015 reiew #2 09 21, 2015 reiew #3 4 months 1 month i really enjoyed this book. the flow was very good and the writing was well done. i'm also glad that the majority of the book was set years after the "dark times", in which crime was nearly all-consuming. figure 1 : an illustration of streaming reviews from a user or to a product. the left part indicates an example of a user who drafts the reviews with the positive words but scores lower with time. the right part indicates an example for a product that receives reviews with negative words but is scored higher with time. @xcite . later, the framework of dual model with text was applied to learn separated representations for user and product @xcite . studies have shown that users and products often do not appear as independent individuals and exhibit shared preferences and social interactions @xcite . therefore, recent attempts improve the embedding quality by implicitly incorporating user-user or user-product relationships @xcite @xcite . however, all these aforementioned approaches assume that the user and product characteristics are static, which is unreasonable in the ever-evolving real world. in fact, the users' preferences and the popularity of products are continuously changing over time. figure 1 illustrates a real case of shifts in user rating preference (user u ) and product rating preference (product p ) in amazon book review 1 . user u represents an example whose words remain positive while scoring gradually negative. specifically, in the review on april 20, 2015, some positive words such as enjoyed, well done, good were used with a score of 5 given to the book. in the subsequent two reviews, user u still used positive words, but the ratings were changed to 4 stars and 3 stars. it might indicate that user u 's word habits remain consistent, while the scoring preference has shifted, suggesting a more cautious behavior in rating. for product p , negative words such as disliked, boring, and unlikable are more likely to appear in the reviews during the display period. however, when analyzing rating changes, product p receives more positive ratings over time, indicating that it has grown in popularity even with negative review documents. based on the above observations, we assume that dynamic modeling with the temporal user and product dynamics is essential for sentiment analysis on user reviews in real-world changing contexts. therefore, in this paper, we propose a new study on sentiment analysis that focuses on streaming user reviews, which takes chronological reviews as a stream and aims to predict the rating score with the dynamic user and product representations. since modeling pairwise relations between users and products in a graph has been proven useful in sentiment analysis on user reviews @xcite , we provide an intuitive way to model streaming user reviews in a dynamic graph and propose a dual-channel dynamic graph neural network (dc-dgnn) to learn the temporal dynamics of users and products. to interconnect users and products while still preserving their specific properties, dc-dgnn is designed with two dual-channel components based on a first-separatethen-gather strategy. first, a dual-channel text encoder is proposed to encode review documents into the local and global channels to learn separate userrelated and product-related contexts respectively. then, inspired by jodie @xcite , a dual-channel dynamic graph updating module is designed to aggregate information of the implicit user-review-product bipartite graph. in this way, dynamic information flows smoothly from one moment to the next with the evolution of the dynamic graph and is integrated into current user and prod-1 https://nijianmo.github.io/amazon/index.html uct representations. finally, the updated user and product representations along with the text representation are used for the current time score prediction. we evaluate the proposed dc-dgnn on five self-built datasets. the experimental results show that dc-dgnn outperforms all existing state-ofthe-art methods and the motivation for modeling the temporal dynamics of users and products is verified. ## methodology in sentiment analysis on streaming user reviews, reviews are sorted in chronological order in the form we propose a dual-channel dynamic graph neural network (dc-dgnn) to model the dynamics of the streaming review data, as shown in figure 2 , which takes a sequence of reviews as input and integrates the review information into users and products through two dual-channel components. more concretely, for a document written by a user, a dual-channel text encoding component is first applied to encode text into user-related and product-related channels. in addition, a dual channel dynamic graph updating component is proposed to integrate new review information into users and products. finally, by concatenating the original text representation, the updated user representation as well as the updated product representation and feeding them into a prediction layer, we can obtain the dynamic scoring result for the current moment. the details of each component are discussed as follows. ## experiments we construct 5 datasets for sentiment analysis on streaming user reviews to evaluate the performance of our model and promote the development of this research. we first collect 5-core data from the amazon reviews, which are built by @xcite and @xcite in a variety of categories to ensure the diversity of our datasets. the former dataset we refer covers the period from may 1996 to july 2014, and the latter ranges from may 1996 to oct 2018. the statistics of the 5 datasets are shown in for more on our modifications to the jodie structure, see the appendix. we formalize the score prediction task as a classification problem, and the following seven evaluation indicators are used to evaluate our model: accuracy, precision, recall, f1, mse, rmse, and mae. for the split of the training set and test set, we adopt a ratio of 9:1. ## conclusion in this paper, we present novel research on sentiment analysis on streaming user reviews and propose a dual-channel dynamic graph neural network, namely dc-dgnn, to model the temporal dynamics of users and products. dc-dgnn dynamically updates user and product representations through the dual-channel evolution of the graph. on our 5 self-constructed datasets, by comprehensive evaluations and ablation study, we confirm the superiority of our dc-dgnn and the impact of its modules. through additional analytical experiments, we further demonstrate the importance of modeling user and product dynamics, hence verifying the conjecture in this paper.