Venue: EMNLP 2023

# Sentiment Analysis on Streaming User Reviews via Dual-Channel Dynamic Graph Neural Network

Xin Zhang Linhai Zhang  Deyu Zhou1

School of Computer Science and Engineering, Key Laboratory of Computer Network

and Information Integration, Ministry of Education, Southeast University, China

{zhangxin, lzhang472, d.zhou}@seu.edu.cn

Footnote 1: Corresponding author.

###### Abstract

Sentiment analysis on user reviews has achieved great success thanks to the rapid growth of deep learning techniques. The large number of online streaming reviews also provides the opportunity to model temporal dynamics for users and products on the timeline. However, existing methods model users and products in the real world based on a static assumption and neglect their time-varying characteristics. In this paper, we present DC-DGNN, a dual-channel framework based on a dynamic graph neural network that models temporal user and product dynamics for sentiment analysis. Specifically, a dual-channel text encoder is employed to extract current local and global contexts from review documents for users and products. Moreover, user review streams are integrated into the dynamic graph neural network by treating users and products as nodes and reviews as new edges. Node representations are dynamically updated along with the evolution of the dynamic graph and used for the final prediction. Experimental results on five real-world datasets demonstrate the superiority of the proposed method.

+
Footnote †: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7208–7220

December 6-10, 2023 @2023 Association for Computational Linguistics

## 1 Introduction

Sentiment analysis on user reviews, inferring the overall sentiment polarity (e.g. 1-5 stars on the review site Amazon) of a user-written review document for a product, has gained popularity with the rapid growth of online review sites such as Amazon, Yelp, and IMDB. Compared to other sentiment analysis tasks Yang et al. (2019); Zhou et al. (2020, 2021a), not only the text itself but also the user and product information are crucial for the final rating score prediction.

For sentiment analysis on user reviews, early methods incorporate user and product embeddings by training randomly initialized embedding with the classifier Tang et al. (2015); Chen et al. (2016); Ma et al. (2017); Dou (2017). Later, the framework of dual model with text was applied to learn separated representations for user and product Long et al. (2018); Lyu et al. (2020). Studies have shown that users and products often do not appear as independent individuals and exhibit shared preferences and social interactions Kim and Srivastava (2007); Mittal et al. (2022). Therefore, recent attempts improve the embedding quality by implicitly incorporating user-user or user-product relationships Amplayo et al. (2018); Wen et al. (2020); Zhou et al. (2021c, b).

However, all these aforementioned approaches assume that the user and product characteristics are static, which is unreasonable in the ever-evolving real world. In fact, the users' preferences and the popularity of products are continuously changing over time. Figure 1 illustrates a real case of shifts

Figure 1: An illustration of Streaming Reviews from a user or to a product. The left part indicates an example of a user whorafts the reviews with the positive words but scores lower with time. The right part indicates an example for a product that receives reviews with negative words but is scored higher with time.



in user rating preference (user \(U\)) and product rating preference (product \(P\)) in Amazon Book Review1. User \(U\) represents an example whose words remain positive while scoring gradually negative. Specifically, in the review on April 20, 2015, some positive words such as _enjoyed_, _well done_, _good_ were used with a score of 5 given to the book. In the subsequent two reviews, user \(U\) still used positive words, but the ratings were changed to 4 stars and 3 stars. It might indicate that user \(U\)'s word habits remain consistent, while the scoring preference has shifted, suggesting a more cautious behavior in rating. For product \(P\), negative words such as _disliked_, _boring_, and _unlikeable_ are more likely to appear in the reviews during the display period. However, when analyzing rating changes, product \(P\) receives more positive ratings over time, indicating that it has grown in popularity even with negative review documents. Based on the above observations, we assume that dynamic modeling with the temporal user and product dynamics is essential for sentiment analysis on user reviews in real-world changing contexts.

Footnote 1: [https://nijianmo.github.io/amazon/index.html](https://nijianmo.github.io/amazon/index.html)

Therefore, in this paper, we propose a new study on sentiment analysis that focuses on streaming user reviews, which takes chronological reviews as a stream and aims to predict the rating score with the dynamic user and product representations. Since modeling pairwise relations between users and products in a graph has been proven useful in sentiment analysis on user reviews Tan et al. (2011); Zhou et al. (2021c), we provide an intuitive way to model streaming user reviews in a dynamic graph and propose a dual-channel dynamic graph neural network (DC-DGNN) to learn the temporal dynamics of users and products. To interconnect users and products while still preserving their specific properties, DC-DGNN is designed with two dual-channel components based on a first-separate-then-gather strategy. First, a dual-channel text encoder is proposed to encode review documents into the local and global channels to learn separate user-related and product-related contexts respectively. Then, inspired by JODIE Kumar et al. (2019), a dual-channel dynamic graph updating module is designed to aggregate information of the implicit user-review-product bipartite graph. In this way, dynamic information flows smoothly from one moment to the next with the evolution of the dynamic graph and is integrated into current user and product representations. Finally, the updated user and product representations along with the text representation are used for the current time score prediction.

We evaluate the proposed DC-DGNN on five self-built datasets. The experimental results show that DC-DGNN outperforms all existing state-of-the-art methods and the motivation for modeling the temporal dynamics of users and products is verified.

## 2 Related Work

### Sentiment Analysis on User Reviews

Incorporating user and product information into models through reviews is the main idea of user review sentiment analysis methods. Tang et al. (2015) shows that modeling the reviewer as well as the product being reviewed is valuable for polarity prediction. Several works Chen et al. (2016); Ma et al. (2017); Dou (2017) utilize different semantic-level attentions to encode user and product information. Amplayo (2019) has also looked into the effects of various locations and methods for incorporating attributes into the model. Long et al. (2018) first proposed a dual user and product memory network model to learn their representation separately. Similarly, Lyu et al. (2020) stacks all available historical reviews separately for users and products to enrich their representations.

Recently, to exploit more knowledge from user-user or user-product relations, Amplayo et al. (2018) introduces shared vectors that are constructed from similar users/products to address the cold start problem. Yuan et al. (2019) creates inferred representations from representative users or products to use the inherent correlation between users or products. Wen et al. (2020) aggregates documents written by similar users toward the same product to improve classification accuracy. Zhou et al. (2021b) applies a group-wise model to use group information to supplement user information. However, none of the above works considered the critical factor of temporal information in the real world. In fact, the characteristics of users and products are changing over time. Hence, relying solely on static modeling is not enough to capture their dynamic attributes.

### Dynamic Graph Neural Network

Previous graph representation learning mainly focused on static graphs with a predefined number of 

nodes and edges. Real-world graphs, on the other hand, typically change over time as their graph structures change with the nodes and edges coming in or disappearing. In order to tackle real-world situations, Continuous Time Dynamic Graph Neural Network (Nguyen et al., 2018; Kumar et al., 2019) and Discrete Time Dynamic Graph Neural Network (Sankar et al., 2020; Yang et al., 2021) for time dynamic graph representation learning have been proposed. However, despite its potential in modeling real-world graph integration and dynamics, previous studies only focus on its application in the fields of social networks, citation graphs, traffic networks, etc., while research on sentiment analysis still remains blank.

## 3 Methodology

In sentiment analysis on streaming user reviews, reviews are sorted in chronological order in the form of \(E=\left\{\mathcal{E}_{1},\ldots,\mathcal{E}_{T}\right\}\). Each of them is denoted as \(\mathcal{E}_{i}=(u_{i},p_{i},t_{i},d_{i})\), where \(t_{i}\) is timestamp for the review \(d_{i}\), \(u_{i}\) is the user who wrote the review \(d_{i}\) and \(p_{i}\) is the product being reviewed. The task aims to predict the user's rating \(y\) towards the product under current condition \(\mathcal{E}_{t}\) based on the historical information \(\left\{\mathcal{E}_{1},\ldots,\mathcal{E}_{t-1}\right\}\), and learns a mapping function between user's rating \(y\) and condition \(\mathcal{E}_{t}\), denoted as \(y=\text{f}(\mathcal{E}_{t}|\big{\{}\mathcal{E}_{1},\ldots,\mathcal{E}_{t-1} \big{\}})\). Table 1 lists the important mathematical notations used throughout the paper.

We propose a Dual-Channel Dynamic Graph Neural Network (DC-DGNN) to model the dynamics of the streaming review data, as shown in Figure 2, which takes a sequence of reviews as input and integrates the review information into users and products through two dual-channel components. More concretely, for a document written by a user, a dual-channel text encoding component is first applied to encode text into user-related and product-related channels. In addition, a dual channel dynamic graph updating component is proposed to integrate new review information into users and products. Finally, by concatenating the original text representation, the updated user representation as well as the updated product representation and feeding them into a prediction layer, we can obtain the dynamic scoring result for the current moment. The details of each component are discussed as follows.

### Dual Channel Text Encoding

Earlier research has highlighted the necessity to differentiate between users and products to create distinct representations (Ji et al., 2020). We consider that the global information of the text corresponds to the user, while the local information corresponds to the product. Hence, we adopt a

Figure 2: The architecture of Dual-Channel Dynamic Graph Neural Network (DC-DGNN). The dynamic graph evolves through two steps: (1) the review document is sent to _Dual Channel Tax Encoding_ for Text Encoding, where text representation is encoded into user-related and product-related contexts; (2) the separated contexts are fed into User Part and Product Part in _Dual-Channel Dynamic Graph Updating_ for User Encoding and Product Encoding, respectively.

 

parallel multi-scale representation learning method called MUSE (Zhao et al., 2019) to encode text into global and local channels for users and products respectively. We first embed the document \(d=\big{[}x_{1},x_{2},\cdots,x_{L}\big{]}\) through BERT (Devlin et al., 2019) to get its pretrained word embeddings:

\[\big{[}\mathbf{h}_{1},\mathbf{h}_{2},\cdots,\mathbf{h}_{L}\big{]}=\mathrm{BERT }\big{(}x_{1},x_{2},\cdots,x_{L}\big{)} \tag{1}\]

where \(h_{i}\) is a d-dimensional feature vector for representing corresponding tokens \(x_{i}\). We denote text representation as \(H\). Then we follow the subsequent steps to obtain global and local context.

_Channel 1: Self-Attention Mechanism for User's Global Context._ We first project the document representation \(H\) into into three parts, key \(K\), query \(Q\), and value \(V\). The global context for users is then obtained by means of a self-attention mechanism:

\[H_{u}=\operatorname*{softmax}\bigl{(}\frac{Q_{u}K^{T}_{u}}{\sqrt{d_{k}}} \bigr{)}V_{u} \tag{2}\]

where \(Q_{u},K_{u},V_{u}\) are transformed from \(H\) through the linear layer.

_Channel 2: Convolution for Product's Local Context._ DynamicConv (Wu et al., 2019) is applied here, whose each convolution sub-module contains multiple cells with different kernel sizes to capture different-range features. The output of the convolution cell with kernel size k is:

\[\mathrm{Conv}_{k}(H)=\mathrm{Depth.conv}_{\text{k}}\big{(}V_{p}\big{)}W^{ \text{out}} \tag{3}\]

where \(V_{p}\) is transformed from \(H\) by a shared projection with _Channel 1_, and \(\mathrm{Depth.conv}_{\text{k}}\big{(}V_{p}\big{)}\) is a traditional depthwise convolutional function, which formulas as follows:

\[\begin{split}\mathrm{Depth.conv}_{\text{k}}\big{(}X\big{)}& {=\sum_{j=1}^{k}\bigl{(}\operatorname*{softmax}(\sum_{c=1}^{d} W^{Q}_{j,c}\\ X_{i,c})\cdot X_{i+j-\left\lceil\frac{k+1}{2}\right\rceil} \hskip-4.5pt\bigr{]}\hskip-4.5pt,c}\\ \end{split} \tag{4}\]

where the output corresponds to the calculation results of the document's \(i\)-th element of output channel \(c\). In the end, the weight of different convolution cells is automatically selected through a gating mechanism for the product's local context:

\[H_{p}=\sum_{i=1}^{n}\frac{\exp\bigl{(}\alpha_{i}\bigr{)}}{\sum_{j=1}^{n}\exp \bigl{(}\alpha_{j}\bigr{)}}\mathop{Conv}_{k_{i}}(H) \tag{5}\]

### Dual Channel Dynamic Graph Updating

Our proposed dual-channel dynamic graph updating component is primarily inspired by JODIE (Kumar et al., 2019), with adaptive adjustments and designs specially made for sentiment analysis on streaming user reviews.

Given a set of reviews with N users and M products, we first create two embedding look-up tables for both users and products as \(\mathbf{E}_{u}=\big{[}\mathbf{u}_{1},\ldots,\mathbf{u}_{N}\big{]}\) and \(\mathbf{E}_{p}=\big{[}\mathbf{p}_{1},\ldots,\mathbf{p}_{M}\big{]}\), which also act as a current representation storage database. At time 0, each user \(u\) and product \(p\) in the database is initialized randomly from a uniform distribution into an r-dimensional vector \(\mathbf{u}(0)\in\mathbb{R}^{r}\) and \(\mathbf{p}(0)\in\mathbb{R}^{r}\). When new input arrives, the representation stored in the database is taken out as \(\mathbf{u}\big{(}t^{-}\big{)}\) and \(\mathbf{p}\big{(}t^{-}\big{)}\) and added to the updating process. After this, the updated information will be renewed to the database as \(\mathbf{u}(t)\) and \(\mathbf{p}(t)\). Note that we only maintain the most recent representation of the users and products in these two databases.

The entire updating process is designed as a dual-channel structure of user-part and product-part, where the user part contains a projection module and an updating module, and the product part only has an updating module.

**User Part.** The first part for the user is a Projection Module to process temporal projections. We assume that user preference is continuously shifting even when there are no review actions. Therefore, for user embeddings after the elapsed time \(\Delta_{u}\), we perform the following projection function:

\[\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}{=(1+\mathbf{w})*\mathbf{u}\big{(}t^ {-}\big{)}} \tag{6}\]

where \(\mathbf{w}\in\mathbb{R}^{r}\) is a time-context vector converted from \(\Delta_{u}\), and the larger \(\Delta_{u}\) is, the more the projected embedding vector differs from the input embedding vector.

\begin{table}
\begin{tabular}{c c} \hline
**Notations** & **Descriptions** \\ \hline \(\mathcal{E}_{i}\) & The i-th review \\ \(\mathbf{E}_{u}\) & Database for users \\ \(\mathbf{E}_{p}\) & Database for products \\ \(H\) & Text embedding of the review document \\ \(H_{u}\) & User-related context \\ \(H_{p}\) & Product-related context \\ \(\mathbf{u}(t)\) & User’s embedding at time t \\ \(\mathbf{p}(t)\) & Product’s embedding at time t \\ \(\mathbf{u}\big{(}t^{-}\big{)}\) & User’s embedding before time t \\ \(\mathbf{p}(t^{-}\big{)}\) & Product’s embedding before time t \\ \(\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}\) & Projected user embedding \\ \(\mathbf{p}(t^{-}+\Delta_{p})\) & Projected product embedding \\ \(\tilde{\mathbf{p}}(t^{-}+\Delta_{u})\) & Predicted user’s next product embedding \\ \hline \end{tabular}
\end{table}
Table 1: Important notations and descriptions 

Later, a Next Product Prediction (NIP) cell, which predicts the product that the user is likely to review, is proposed to enhance the representation of the user:

\[\begin{split}\tilde{\mathbf{p}}(t^{-}+\Delta_{u})&=W_{1 }\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}\\ &+W_{2}\mathbf{p}^{-}\big{(}t^{-}\big{)}{+B}\end{split} \tag{7}\]

where \(\mathbf{p}^{-}\big{(}t^{-}\big{)}\) is product embedding before time \(t\) corresponding to the product from \(u\)'s previous review, and \(W_{1}\in\mathbb{R}^{r\times r}\), \(W_{2}\in\mathbb{R}^{r\times r}\) and \(B\in\mathbb{R}^{r}\) are trainable parameters in linear layer.

For the NIP unit, our aim is to minimize the difference between the predicted product embedding \(\tilde{\mathbf{p}}(t^{-}+\Delta_{u})\) and the real product embedding \(\mathbf{p}(t^{-}+\Delta_{p})\), where \(\Delta_{p}\) is the time difference between the current review and the last review for product \(p\), which is different from \(\Delta_{u}\). Since we assume that the products do not change during the time interval, here \(\mathbf{p}(t^{-}+\Delta_{p})\) equals \(\mathbf{p}(t^{-})\). Thus, the loss function can be represented as:

\[L_{nip}=\sum_{\big{(}u,p,t,d\big{)}\in\mathcal{E}}\left\|\hat{\mathbf{p}}(t^{ -}+\Delta_{u})-\mathbf{p}(t^{-})\right\|_{2} \tag{8}\]

where we use the L2 loss function to push the predicted product representation closer to the true product representation.

The second part for the user is an Update Module, implemented based on an RNN cell, which generates the updated representation of the user after this review. It takes the projected user embedding \(\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}\), the previous product embedding \(\mathbf{p}\big{(}t^{-}\big{)}\), and the user's time interval \(\Delta_{u}\) as input, then integrates them into the current user representation:

\[\begin{split}\mathbf{u}\big{(}t\big{)}&=\tanh \bigl{(}W_{1}^{u}\big{(}\mathbf{p}\big{(}t^{-}\big{)}{\oplus}\Delta_{u}\oplus H _{u}\big{)}\\ &+W_{2}^{u}\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}{+b_{u}}\bigr{)} \end{split} \tag{9}\]

where \(W_{1}^{u}\in\mathbb{R}^{\binom{r+1+d}{\times r}}\), \(W_{2}^{u}\in\mathbb{R}^{r\times r}\) and \(b_{u}\in\mathbb{R}^{r}\) are parameters of user's update RNN cell. Through these two modules, the user's representation flows smoothly from \(t^{-}\) to \(t\).

**Product Part.** The product part is similar to the user side, except that there is no projection module, as we consider the product to be static for the duration. In contrast to the user part, the positions of \(\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}\) and \(\mathbf{p}\big{(}t^{-}\big{)}\) are switched, and the context used for product updating is \(H_{p}\):

\[\begin{split}\mathbf{p}\big{(}t\big{)}&=\tanh \bigl{(}W_{1}^{p}\big{(}\mathbf{u}\big{(}t^{-}+\Delta_{u}\big{)}{\oplus}\Delta _{p}\\ &{\oplus}H_{p}\big{)}{+}W_{2}^{p}\mathbf{p}\big{(}t^{-}\big{)}{+ b_{p}}\big{)}\end{split} \tag{10}\]

where \(W_{1}^{p}\in\mathbb{R}^{\binom{r+1+d}{\times r}}\), \(W_{2}^{p}\in\mathbb{R}^{r\times r}\) and \(b_{p}\in\mathbb{R}^{r}\) are parameters of product's update RNN cell.

### Training

In order to ensure the quality of user and product embeddings, we apply the L2 loss between \(t^{-}\) and \(t\) for users and products to prevent sudden changes in continuous time:

\[L_{usmooth} =\big{\lVert}\mathbf{u}\big{(}t\big{)}{-}\mathbf{u}\big{(}t^{-} \big{)}\big{\rVert}_{2} \tag{11}\] \[L_{p_{smooth}} =\big{\lVert}\mathbf{p}\big{(}t\big{)}{-}\mathbf{p}\big{(}t^{-} \big{)}\big{\rVert}_{2} \tag{12}\]

where \(\mathbf{u}\big{(}t\big{)}\) and \(\mathbf{u}\big{(}t^{-}\big{)}\) represent the user's previous and current representation respectively, while \(p\) corresponds to the product.

For score prediction at time \(t\), we calculate cross-entropy loss between the predicted score \(\widehat{y}\) and true score \(y\):

\[\begin{split} L_{score}&=-\frac{1}{N}{\sum_{i=1}^{N }}{\sum_{c=1}^{K}}ylog(\widehat{y})\\ &\text{where $\widehat{y}$}=\operatorname{MLP}\bigl{(}\mathbf{ u}\big{(}t\big{)}{\oplus}\mathbf{p}\big{(}t\big{)}{\oplus}H\big{)}\end{split} \tag{13}\]

Finally, our training objective can be formulated as follows:

\[\begin{split} L&=\lambda_{1}L_{score}+\lambda_{2}L_{ nip}\\ &+\lambda_{3}L_{u_{smooth}}+\lambda_{4}L_{p_{smooth}}\end{split} \tag{14}\]

where \(\lambda_{1},\lambda_{2}\) and \(\lambda_{3}\) as well as \(\lambda_{4}\) are tradeoff parameters for each loss.

## 4 Experiments

We construct 5 datasets for sentiment analysis on streaming user reviews to evaluate the performance of our model and promote the development of this research. We first collect 5-core data from the Amazon Reviews, which are built by McAuley et al. (2015) and Ni et al. (2019) in a variety of categories to ensure the diversity of our datasets. The former dataset we refer covers the period from May 1996 to July 2014, and the latter ranges from May 1996 to Oct 2018. The statistics of the 5 datasets are shown in Table 2. For more construction details, see the Appendix.

We adopt BERT (base-uncased) Devlin et al. (2019) as the pretrained encoder for texts. For user and product embeddings, we explore it in dimensions [8; 16; 32; 64; 28; 256]. The batch size is 8, and the learning rate is 3e-5, the number of epochs is 2. \(\lambda_{1}\), \(\lambda_{2}\), \(\lambda_{3}\) and \(\lambda_{4}\) are all set to 1. The dropout 

of MUSE is set to 0.1, and three different kernel size of [1; 3; 5] is applied for deepwise convolution. For more on our modifications to the JODIE structure, see the Appendix. We formalize the score prediction task as a classification problem, and the following seven evaluation indicators are used to evaluate our model: Accuracy, Precision, Recall, F1, MSE, RMSE, and MAE. For the split of the training set and test set, we adopt a ratio of 9:1.

### Baseline Methods

As the task of sentiment analysis on streaming user reviews has not yet been investigated, we create two different baselines to contrast with our DC-DGNN method:

* _Text-based model_: The **Bert-Sequence** model, a Bert Model transformer Devlin et al. (2019) with a sequence classification head on top, which is a frequent winner in classification tasks, is still applied here as a very competitive baseline. Similarly, the structure of the combination of Bi-directional LSTM with Attention (**BiLSTM+Att**) is also used as another baseline.
* _User and Product-based model_: **JODIE**Kumar et al. (2019) applies a coupled recurrent neural network model to learn embedding trajectories of users and items, which has the ability to predict a state change in users of the recommendation system. Here we use it as a baseline for score prediction. Due to its t-batch strategy, only the regression task can be implemented here. **CHIM**Amplayo (2019) adopts a chunk-wise matrix representation for user/product attributes and injects user/product information in different locations. **IUPC**Lyu et al. (2020) uses a dual channel modeling approach similar to ours by stacking text information into user and product through multi-head attention. **NGSAM**Zhou et al. (2021b) exploits group information to supplement user information by a group-wise model, which is designed as a regression task with bias.

### Main Results

The experimental results on the datasets from May 1996 to October 2018 are shown in the first four rows of Table 3, and the dataset from May 1996 to July 2014 is shown in the last row of Table 3. It can be observed that:

1. On all five datasets, the performance of the Bert-based model (Bert-Sequence, IUPC, and our DC-DGNN) is better than that of the Glove-based model (BiLSTM+Att, NGSAM, and CHIM), which proves that a high-quality feature extractor is still a necessity for sentiment analysis.
2. Our DC-DGNN model outperforms all other baselines on 5 datasets, confirming the superiority of DC-DGNN in modeling user and product temporal dynamics. Meanwhile, compared to JODIE, our DC-DGNN has significant improvements, which shows that we have successfully adapted the dynamic graph structure to our sentiment analysis task.
3. Comparing the results on the first four datasets with the last dataset, we observe that our model shows convincible and considerable performance when datasets have a relatively longer timespan, which indicates that our DC-DGNN has the potential to model on more real-world ever-increasing streaming datasets.

### Ablation Study

In order to verify the impact of the modules proposed in this paper, ablation experiments were designed as follows: 1) w/o DCTE means removing the Dual Channel Text Encoding (DCTE) structure and directly updating the user and product with

\begin{table}
\begin{tabular}{c c c c c c c c} \hline Category & Total Num & User Num & Product Num & Avg r/u & Avg r/p & Test Avg r/u & Test Avg r/p \\ \hline _Books_ & 25164 & 500 & 497 & 50.328 & 50.632 & 52.019 & 56.066 \\ _Grocery\_and\_Gourtnet\_Food_ & 9725 & 500 & 500 & 19.45 & 19.45 & 20.456 & 24.088 \\ _Kindle\_Store_ & 11742 & 500 & 488 & 23.484 & 24.061 & 24.632 & 28.373 \\ _Sports\_and\_Outdoors_ & 6002 & 500 & 493 & 12.004 & 12.174 & 12.278 & 15.537 \\ _reviews\_CDs\_and\_Vinyl_ & 19583 & 500 & 500 & 39.166 & 39.166 & 44.819 & 41.610 \\ \hline \end{tabular}
\end{table}
Table 2: Statistical information of our self-constructed datasets. ’Avg r/u’ means the average associated reviews number for users, and ‘Avg r/p’ means the average associated reviews number for products. Test corresponding to the test set.

 

the BERT cls representation; 2) w/o user&product removes user and product information, and only completes score prediction through dynamically learned text representation. As shown in Table 4, w/o DCTE and w/o user&product all cause a performance drop on each dataset except for _Grocery__and__Gournet__Food_, where its text content is relatively short, and MUSE in DCTE generally has advantages in encoding long texts rather than short texts. We also find that both w/o DCTE and user&product result in significant drops on _Sports__and__Outdoors_, indicating that even with relatively sparse user/product information but a long timespan, DC-DGNN's dual modeling of user and product dynamics still plays a crucial role in score prediction.

### Variants of DC-DGNN Structure

**Discussion of User and Product Embedding Dimensions.** Different datasets have their own char

\begin{table}
\begin{tabular}{c|l l c c c c c c c} \hline \hline  & \multicolumn{1}{c}{_Bools_} & \multicolumn{3}{c}{_Grocery__and__} & \multicolumn{3}{c}{_Kindle__Store_} & \multicolumn{3}{c}{_Sports__and_} & \multicolumn{3}{c}{_reviewes__CDs__} \\  & & \multicolumn{3}{c}{_Gourmet__Good_} & \multicolumn{3}{c}{_Ouddoors_} & \multicolumn{3}{c}{_and__Vinyl_} \\ \hline  & RMSE(↓) & MAE(↓) & RMSE(↓) & MAE(↓) & RMSE(↓) & MAE(↓) & RMSE(↓) & MAE(↓) & RMSE(↓) \\ \hline w/o DCTE & 0.6553 & 0.3643 & **0.6648** & **0.2980** & 0.6451 & 0.3566 & 0.7664 & 0.3977 & 0.6406 & 0.3175 \\ w/o user@product & 0.6553 & 0.3651 & 0.7096 & 0.3124 & 0.6588 & 0.3745 & 0.7477 & 0.3993 & 0.6446 & 0.3267 \\ DC-DGNN & **0.6360** & **0.3449** & 0.6994 & 0.3227 & **0.6318** & **0.3532** & **0.7089** & **0.3561** & **0.6390** & **0.3134** \\ \hline \end{tabular}
\end{table}
Table 4: Compare the results under the original structure with the results of w/o DCTE and w/o user&product.

\begin{table}
\begin{tabular}{|r|l|c c c c c c c|} \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Method**} & \multicolumn{6}{c}{**Criteria**} & \multicolumn{3}{c}{_Greurica_} \\ \cline{3-10}  & & Accuracy(↑) & Precision(↑) & Recall(↑) & F1(↑) & MSE(↓) & RMSE(↓) & MAE(↓) \\ \hline  & BiLSTM+Att & 0.5793 & 0.4219 & 0.3528 & 0.3458 & 0.6726 & 0.8201 & 0.4970 \\  & Bert-Sequence & 0.6687 & 0.6032 & 0.4941 & 0.5269 & 0.4203 & 0.6483 & 0.3584 \\ \multirow{-2}{*}{_Books_} & JODIE & - & - & - & - & 17.6734 & 4.2040 & 4.2035 \\  & NGSAM & - & - & - & - & 0.7027 & 0.8383 & 0.6401 \\  & CHIM & 0.6293 & 0.5338 & 0.4310 & 0.4557 & 0.6293 & 0.7933 & 0.4466 \\  & IUPC & 0.6722 & **0.6237** & **0.5130** & 0.5473 & 0.4168 & 0.6456 & 0.3548 \\  & **DC-DGNN** & **0.6814** & 0.6086 & **0.5363** & **0.6527** & **0.4044** & **0.6360** & **0.3449** \\ \hline  & BiLSTM+Att & 0.6814 & 0.3925 & 0.3755 & 0.3816 & 0.8243 & 0.9079 & 0.4543 \\  & Bert-Sequence & 0.7235 & 0.5672 & 0.4032 & 0.4202 & 0.5036 & 0.7096 & 0.3453 \\ \multirow{-2}{*}{_Grocery__and_} & JODIE & - & - & - & - & 19.5945 & 4.4266 & 4.4246 \\  & NGSAM & - & - & - & - & 0.6628 & 0.8141 & 0.5462 \\  & CHIM & 0.7050 & 0.5425 & 0.4514 & 0.4660 & 0.7328 & 0.8560 & 0.4080 \\  & IUPC & 0.7235 & 0.3879 & 0.3666 & 0.3563 & 0.5971 & 0.7727 & 0.3669 \\  & **DC-DGNN** & **0.7451** & **0.6207** & **0.5191** & **0.5021** & **0.4892** & **0.6994** & **0.3227** \\ \hline  & BiLSTM+Att & 0.5804 & 0.3325 & 0.3561 & 0.3337 & 0.7864 & 0.8868 & 0.5226 \\  & Bert-Sequence & 0.6511 & 0.5540 & 0.4897 & 0.5086 & 0.4230 & 0.6504 & 0.3719 \\ \multirow{-2}{*}{_Kindle_Store_} & JODIE & - & - & - & - & 17.4456 & 4.1768 & 4.1761 \\  & NGSAM & - & - & - & - & 0.5454 & 0.7385 & 0.5515 \\  & CHIM & 0.6485 & 0.4535 & 0.3986 & 0.4035 & 0.6026 & 0.7762 & 0.4238 \\  & IUPC & 0.6545 & 0.5672 & 0.4994 & 0.5198 & 0.4383 & 0.6620 & 0.3719 \\  & **DC-DGNN** & **0.6664** & **0.5912** & **0.5157** & **0.5370** & **0.3991** & **0.6318** & **0.3532** \\ \hline  & BiLSTM+Att & 0.6689 & 0.1843 & 0.2016 & 0.1637 & 1.2396 & 1.1134 & 0.5441 \\  & Bert-sequence & 0.6872 & 0.3879 & 0.3447 & 0.3545 & 0.5391 & 0.7342 & 0.3794 \\ \multirow{-2}{*}{_Sports__and_} & JODIE & - & - & - & - & 19.8610 & 4.4566 & 4.4558 \\  & NGSAM & - & - & - & - & 0.7356 & 0.8577 & 0.5840 \\  & CHIM & 0.6905 & 0.4392 & 0.3109 & 0.3341 & 0.9967 & 0.9983 & 0.4742 \\  & IUPC & 0.6872 & 0.4034 & 0.3713 & 0.3791 & 0.5541 & 0.7444 & 0.3844 \\  & **DC-DGNN** & **0.7038** & **0.4227** & **0.3950** & **0.4063** & **0.5025** & **0.7089** & **0.3561** \\ \hline  & BiLSTM+Att & 0.6605 & 0.4227 & 0.3988 & 0.3939 & 0.8474 & 0.9205 & 0.4655 \\  & Bert-Sequence & 0.7182 & 0.5635 & 0.5218 & 0.5396 & 0.4436 & 0.6660 & 0.3252 \\ \multirow{-2}{*}{_reives__CDs_} & JODIE & - & - & - & - & 18.5957 & 4.3123 & 4.3091 \\  & NGSAM & - & - & - & - & 0.7320 & 0.8556 & 0.6885 \\ \multirow{-2}{*}{_and__Vinyl_} & CHIM & 0.7009 & 0.5340 & 0.4803 & 0.4912 & 0.5365 & 0.7325 & 0.3640 \\ \multirow{-2}{*}{_CIFM_} & IUPC & 0.7070 & 0.5363 & 0.5056 & 0.5177 & 0.4273 & 0.6537 & 0.3313 \\  & **DC-DGNN** & **0.7243** & **0.5691** & **0.5435** & **0.5542** & **0.4084** & **0.6390** & **0.3134** \\ \hline \end{tabular} \end{ 

acteristics in distribution and domain, so it is unreasonable to set the same embedding dimension for modeling on all datasets. To this end, we conduct a dimension exploration experiment to compare the impact of different dimension sizes on 5 datasets. Overall in Figure 4, we can see that the better results (i.e., the darker the blue color) are basically on the right side of the graph, indicating that in most cases, it's appropriate to set a relatively large dimension. This is because our dataset is basically at the level of ten thousand, with relatively abundant data, and modeling with large dimensions is beneficial. However, the largest is not always the best when some datasets are relatively small. For example, _Sports_and_Outdoors_ achieves the best results when the dimension is 64.

**Variants of Processing Text Information for DCTE.** After obtaining global and local contexts through DCTE, it has an additional dimension related to sequence length compared to user and product representation. To align them, we have multiple processing methods. Other than the _cls_ result reported in the main experiment, we also tried two different methods: _mean_ and _max_. _cls_ takes the encoding at position 0 of the global \(H_{u}\) and local representations \(H_{p}\) respectively for updating users and products; _mean_ averages the representation of the entire dimension 1 of \(H_{u}\) and \(H_{p}\); _max_ conducts max_pool1d on the entire dimension 1. For the sake of fairness, we compare their performance with both the user and product dimensions set to 128. Figure 5 shows that, in general, _cls_ can achieve relatively good results. However, in specific, the method corresponding to the best results still depends on the characteristics of different datasets.

### Case Study

Figure 3 shows a user's reviews sampled from the _Books_ dataset. The goal is to predict the rating score from the same user of the review on February 10, 2017. CHIM makes the worst prediction that differs the most from the actual 1-star. This is because CHIM focuses on attribute incorporation and cannot obtain user- or product-biased sentiment information. IUPC and BERT get the same results, indicating that IUPC cannot satisfy the need for user dynamics modeling when the user characteristic is complex. JODIE's result also indicates that its original structure is not an appropriate setting.

Figure 4: The visualize RMSE scores by setting the user and product dimensions to [8, 16, 32, 64, 128, 256] on different datasets

Figure 5: RMSE comparison of different text processing methods in DCTE

Figure 3: The case study (grey indicates neutral expression, red indicates positive expression, and green indicates negative expression).

 

as explained in detail in the Appendix. Our model, on the contrary, can accurately capture the user's constantly negative scoring trend when the three expressions are very close and predict a score of 1, which shows the importance of dynamic modeling.

### Computational Efficiency

In this section, we will discuss the time efficiency comparison of our proposed DC-DGNN and other models. We conduct a time test to compare our model's performance with that of IUPC and CHIM, which also learn user and product embeddings simultaneously. As shown in Table 5, DC-DGNN demonstrates better computational efficiency on most datasets. This can be attributed to the continuous storage of updated embeddings in a database, in contrast to previous methods that train two separate embedding instances. As a result, our method's time complexity is \(O(L\cdot e)+O(L\cdot H\cdot d)\), which is a reduction compared to the \(O(D\cdot L\cdot H\cdot d)\) complexity of other methods. Furthermore, the time advantage becomes even more obvious as any of these dimensions increases. While CHIM, which relies on GloVe Pennington et al. (2014) encoding, may show faster processing on specific datasets, it requires a larger number of training epochs.

## 5 Conclusion

In this paper, we present novel research on sentiment analysis on streaming user reviews and propose a dual-channel dynamic graph neural network, namely DC-DGNN, to model the temporal dynamics of users and products. DC-DGNN dynamically updates user and product representations through the dual-channel evolution of the graph. On our 5 self-constructed datasets, by comprehensive evaluations and ablation study, we confirm the superiority of our DC-DGNN and the impact of its modules. Through additional analytical experiments, we further demonstrate the importance of modeling user and product dynamics, hence verifying the conjecture in this paper.

## Limitations

Although our model has shown excellent performance in sentiment analysis on streaming user reviews, we still believe it has some limitations:

* Comparing datasets with longer timespan and shorter timespan, we find that improvement is not noticeable for the datasets with shorter timespan, which is a limitation for analysis only with short-term data. At the same time, our DC-DGNN model is also not friendly for datasets with sparse user/product information.
* In early experimental attempts, for the structure built directly from JODIE, we found significant differences in prediction performance at different times. When making predictions on data with a relatively recent time, the performance is great, and when the time is farther away, the performance is sharply decreased.
* The current structure we proposed considers forward integration and ignores backpropagation. In fact, the addition of subsequent reviews will also have an impact on past node representation. At the same time, we also ignore the high-order correlations between user-user and product-product. For example, user and user can be connected through a second-order homogeneous graph. The above-mentioned more refined design in graph updating may be our future improvement direction.

## Acknowledgement

The authors would like to thank the anonymous reviewers for their insightful comments. This work is funded by the National Natural Science Foundation of China (62176053). This work is supported by the Big Data Computing Center of Southeast University.

## References

* Amplayo (2019) Reinald Kim Amplayo. 2019. Rethinking attribute representation and injection for sentiment classification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 5601-5612. Association for Computational Linguistics.
* Amplayo et al. (2018) Reinald Kim Amplayo, Jihyeok Kim, Sua Sung, and Seung-won Hwang. 2018. Cold-start aware user and

\begin{table}
\begin{tabular}{l c c c} \hline Category & DC-DGNN & IUPC & CHIM \\ \hline _Books_ & **9.3172** & 12.1095 & 21.5937 \\ _Grocev\_and\_Gourent\_Food_ & **3.3225** & 3.7920 & 2.2807 \\ _Kindle\_Store_ & **4.1851** & 4.2269 & 4.7659 \\ _Sports\_and\_Outdoors_ & 2.0984 & 2.1240 & **1.2126** \\ _reviews\_CDs\_and\_Vinyl_ & **7.1875** & 7.7507 & 14.4805 \\ \hline \end{tabular}
\end{table}
Table 5: Comparison of Training Time (minutes/epoch) 

product attention for sentiment classification. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_, pages 2535-2544. Association for Computational Linguistics.
* Chen et al. (2016) Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. 2016. Neural sentiment classification with user and product attention. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_, pages 1650-1659. The Association for Computational Linguistics.
* Cong et al. (2023) Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. 2023. Do we really need complicated model architectures for temporal networks? In _The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023_. OpenReview.net.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_, pages 4171-4186. Association for Computational Linguistics.
* Dou (2017) Zi-Yi Dou. 2017. Capturing user and product information for document level sentiment analysis with deep memory network. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_, pages 521-526. Association for Computational Linguistics.
* Ji et al. (2020) Shuyi Ji, Yifan Feng, Rongrong Ji, Xibin Zhao, Wanwan Tang, and Yue Gao. 2020. Dual channel hypergraph collaborative filtering. In _Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 2020-2029.
* Kim and Srivastava (2007) Young Ae Kim and Jaideep Srivastava. 2007. Impact of social influence in e-commerce decision making. In _Proceedings of the 9th International Conference on Electronic Commerce: The Wireless World of Electronic Commerce, 2007, University of Minnesota, Minneapolis, MN, USA, August 19-22, 2007_, volume 258 of _ACM International Conference Proceeding Series_, pages 293-302. ACM.
* Kumar et al. (2019) Srijan Kumar, Kikun Zhang, and Jure Leskovec. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In _Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining_, pages 1269-1278.
* Long et al. (2018) Yunfei Long, Mingyu Ma, Qin Lu, Rong Xiang, and Chu-Ren Huang. 2018. Dual memory network model for biased product review classification. In _Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, WASSA@EMNLP 2018, Brussels, Belgium, October 31, 2018_, pages 140-148. Association for Computational Linguistics.
* Lyu et al. (2020) Chenyang Lyu, Jennifer Foster, and Yvette Graham. 2020. Improving document-level sentiment analysis with user and product context. In _Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020_, pages 6724-6729. International Committee on Computational Linguistics.
* December 1, 2017
- Volume 1: Long Papers_, pages 634-643. Asian Federation of Natural Language Processing.
* McAuley et al. (2015) Julian J. McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based recommendations on styles and substitutes. In _Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9-13, 2015_, pages 43-52. ACM.
* Mittal et al. (2022) Amit Mittal, Meenal Arora, and Ajay Ranga. 2022. Impe: Influence maximization on social media with the impact of e-commerce products. In _5th International Conference on Contemporary Computing and Informatics, IC3I 2022, Uttar Pradesh, India, December 14-16, 2022_, pages 1789-1793. IEEE.
* Nguyen et al. (2018) Giang Hoang Nguyen, John Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Bunyee Koh, and Sungchul Kim. 2018. Continuous-time dynamic network embeddings. In _Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23-27, 2018_, pages 969-976. ACM.
* Ni et al. (2019) Jianmo Ni, Jiacheng Li, and Julian J. McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 188-197. Association for Computational Linguistics.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL_, pages 1532-1543. ACL 

Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. 2020. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In _WSDM '20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020_, pages 519-527. ACM.
* Tan et al. (2011) Chenhao Tan, Lillian Lee, Jie Tang, Long Jiang, Ming Zhou, and Ping Li. 2011. User-level sentiment analysis incorporating social networks. In _Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, August 21-24, 2011_, pages 1397-1405. ACM.
* Tang et al. (2015) Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning semantic representations of users and products for document level sentiment classification. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers_, pages 1014-1023. The Association for Computer Linguistics.
* Wang et al. (2021) Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. 2021. Inductive representation learning in temporal networks via causal anonymous walks. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
* Wen et al. (2020) Jiahui Wen, Guangda Zhang, Hongyun Zhang, Wei Yin, and Jingwei Ma. 2020. Speculative text mining for document-level sentiment classification. _Neurocomputing_, 412:52-62.
* Wu et al. (2019) Felix Wu, Angela Fan, Alexei Baevski, Yann N. Dauphin, and Michael Auli. 2019. Pay less attention with lightweight and dynamic convolutions. In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
* Yang et al. (2021) Menglin Yang, Min Zhou, Marcus Kalander, Zengfeng Huang, and Irwin King. 2021. Discrete-time temporal network embedding via implicit hierarchical learning in hyperbolic space. In _KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021_, pages 1975-1985. ACM.
* Yang et al. (2019) Yang Yang, Deyu Zhou, Yulan He, and Meng Zhang. 2019. Interpretable relevant emotion ranking with event-driven attention. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 177-187. Association for Computational Linguistics.
* Yu et al. (2023) Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. 2023. Towards better dynamic graph learning: New architecture and unified library. _CoRR_, abs/2303.13047.
* Yuan et al. (2019) Zhigang Yuan, Fangzhao Wu, Junxin Liu, Chuhan Wu, Yongfeng Huang, and Xing Xie. 2019. Neural review rating prediction with user and product memory. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019_, pages 2341-2344. ACM.
* Zhao et al. (2019) Guangxiang Zhao, Xu Sun, Jingjing Xu, Zhiyuan Zhang, and Liangchen Luo. 2019. Muse: Parallel multi-scale attention for sequence to sequence learning. _arXiv preprint arXiv:1911.09483_.
* Zhou et al. (2021a) Deyu Zhou, Jianan Wang, Linhai Zhang, and Yulan He. 2021a. Implicit sentiment analysis with event-centered text representation. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021_, pages 6884-6893. Association for Computational Linguistics.
* Zhou et al. (2020) Deyu Zhou, Shuangzhi Wu, Qing Wang, Jun Xie, Zhaopeng Tu, and Mu Li. 2020. Emotion classification by jointly learning to lexiconize and classify. In _Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020_, pages 3235-3245. International Committee on Computational Linguistics.
* Zhou et al. (2021b) Deyu Zhou, Meng Zhang, Linhai Zhang, and Yulan He. 2021b. A neural group-wise sentiment analysis model with data sparsity awareness. In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_, pages 14594-14601. AAAI Press.
* Zhou et al. (2021c) Xiabing Zhou, Zhongqing Wang, Min Zhou, Qifa Wang, Shoushan Li, Min Zhang, and Guodong Zhou. 2021c. Sentiment classification via user and product interactive modeling. _Sci. China Inf. Sci._, 64(12).

 

Experiment Details

**Dataset construct details.** Firstly, the data was subjected to a cleaning operation where we removed duplicates, as well as the data where their texts are empty, and then removed special characters from the texts. Later, the processing steps to build the datasets for sentiment analysis on streaming user reviews are as follows:

* Step1: Sort users by the number of reviews they have made, then filter the data containing the top 500 users from the original data according to their review counts.
* Step2: For the data obtained in the first step, sort the products according to the number of reviews they received, and filter the data containing the top 500 products.
* Step3: Normalise the data. For time processing, convert it to timestamp format and subtract the smallest timestamp in the dataset. For users and products, map the unique identifier to a numeric number.
* Step4: Finally, the data are sorted by timestamp to fit in our chronological setting.

**The reason why _Text-based model_ is necessary here.** Although many previous methods that incorporate user and product information have shown strong potential, however, in some cases, they not only fail to learn high-quality representations but have counterproductive effects on score prediction. For example, in some datasets, only a few products are widely reviewed, and the number of reviews received by different products shows an imbalance. In fact, this data imbalance is very common in the real world, and even if we try to avoid it when constructing datasets, it still exists to some extent. Therefore, in this paper, we still take _text-based model_ as a strong baseline.

**Experimental adjustments for adapting JODIE to sentiment analysis tasks.** The original JODIE structure first divides the data by timespan and then further divides the data by t-batch. T-batch is a strategy to prevent the overlapping of users and items in each batch. However, we believe this strategy may be inappropriate as it disrupts the original time order, which contradicts the chronological setup. To address this, we only divide the data into multiple time periods according to timespan and abandon the t-batch strategy to ensure the original order of the data.

Additionally, JODIE has a problem with gradients vanishing during testing. As JODIE was designed for recommendation systems, it keeps gradients propagation during testing, which is not ideal for our sentiment analysis task. To solve this, we propose a strategy that saves and updates the user and product representations in the databases during training. And during the testing phase, we only take the representations out of the database without updating them.

In JODIE, there is also a small experimental problem. We find that its static embedding setting conflicts with successful training. JODIE sets one-hot vectors as static embedding for each user and item in a rude way, which is not friendly to the situation where there are a large number of users and items, and will directly cause calculation overload and make it impossible to train. Therefore, in this paper, we deal with this problem by directly abandoning static embedding.

## Appendix B Supplement Experimental Results

Based on the unified dynamic graph library implemented by Yu et al. (2023), we present in Figure 6 a comparative analysis of our DC-DGNN alongside several other dynamic graph models. **CAWN**Wang et al. (2021) leverages causal anonymous walks and recurrent neural networks to extract node representations by exploring network dynamics and causality. **DyGFormer**Yu et al. (2023) utilizes a neighbor co-occurrence encoding scheme and a patching technique to capture long-term temporal dependencies. **GraphMixer**Cong et al. (2023) incorporates a fixed-time encoding function into a link encoder, and employs a node encoder with neighbor mean-pooling for feature summarization. From the results, it is obvious that their performance is actually on par with JODIE's, which is unsatisfactory. All of these models also rely on text information for training, similar to our model. However, it is clear that they do not fully exploit the potential of review texts, resulting in subpar task performance, which further reflects the significance of our work 



\begin{table}
\begin{tabular}{|c|l|c c c c c c c|} \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Method**} & \multicolumn{8}{c|}{**Criteria**} \\ \cline{3-10}  & & Accuracy(\(\uparrow\)) & Precision(\(\uparrow\)) & Recall(\(\uparrow\)) & F1(\(\uparrow\)) & MSE(\(\downarrow\)) & RMSE(\(\downarrow\)) & MAE(\(\downarrow\)) \\ \hline \multirow{6}{*}{_Books_} & JODIE & 0.4849 & 0.1927 & 0.2046 & 0.1409 & 1.5655 & 1.2512 & 0.7977 \\  & CAWN & 0.3144 & 0.0629 & 0.2000 & 0.0957 & 0.9756 & 0.9877 & 0.7688 \\  & DyGFormer & 0.4785 & 0.0957 & 0.2000 & 0.1295 & 1.5965 & 1.2635 & 0.8112 \\  & GraphMixer & 0.0201 & 0.0040 & 0.2000 & 0.0079 & 11.1066 & 3.3327 & 3.1888 \\  & **DC-DGNN** & **0.6814** & **0.6086** & **0.5363** & **0.5627** & **0.4044** & **0.6360** & **0.3449** \\ \hline \multirow{6}{*}{_Grocery__and_ \text{...}Gourmet\_Food_} & jodie & 0.1942 & 0.0634 & 0.2116 & 0.0871 & 1.6637 & 1.2898 & 1.0323 \\  & CAWN & 0.2080 & 0.0416 & 0.2000 & 0.0689 & 1.0748 & 1.0367 & 0.8703 \\  & DyGFormer & 0.0302 & 0.0060 & 0.2000 & 0.0117 & 6.7165 & 2.5916 & 2.4585 \\  & GraphMixer & 0.0240 & 0.0048 & 0.2000 & 0.0094 & 12.5374 & 3.5408 & 3.4104 \\  & **DC-DGNN** & **0.7451** & **0.6207** & **0.5191** & **0.5021** & **0.4892** & **0.6994** & **0.3227** \\ \hline \multirow{6}{*}{_Kindle\_Store_} & JODIE & 0.3589 & 0.1394 & 0.2005 & 0.1111 & 1.0006 & 1.0003 & 0.7416 \\  & CAWN & 0.3606 & 0.0721 & 0.2000 & 0.1060 & 0.9216 & 0.9600 & 0.7195 \\  & DyGFormer & 0.0380 & 0.0076 & 0.2000 & 0.0147 & 5.5645 & 2.3589 & 2.2027 \\  & GraphMixer & 0.0210 & 0.0042 & 0.2000 & 0.0082 & 10.8859 & 3.2994 & 3.1607 \\  & **DC-DGNN** & **0.6664** & **0.5912** & **0.5157** & **0.5370** & **0.3991** & **0.6318** & **0.3532** \\ \hline \multirow{6}{*}{_Sports\_and_ \text{..}Outdoors_} & JODIE & 0.2427 & 0.1988 & 0.2065 & 0.0971 & 1.3662 & 1.1689 & 0.9224 \\  & CAWN & 0.2056 & 0.0411 & 0.2000 & 0.0682 & 1.0911 & 1.0446 & 0.8733 \\  & DyGFormer & 0.0189 & 0.0038 & 0.2000 & 0.0074 & 7.1000 & 2.6646 & 2.5622 \\  & GraphMixer & 0.0300 & 0.0060 & 0.2000 & 0.0117 & 13.1044 & 3.6200 & 3.5022 \\  & **DC-DGNN** & **0.7038** & **0.4227** & **0.3950** & **0.4063** & **0.5025** & **0.7089** & **0.3561** \\ \hline \multirow{6}{*}{_reviews\_CDs_} & JODIE & 0.1600 & 0.0402 & 0.1557 & 0.0639 & 2.2033 & 1.4844 & 1.2033 \\  & CAWN & 0.2491 & 0.0498 & 0.2000 & 0.0798 & 1.1668 & 1.0802 & 0.8652 \\ \cline{1-1}  & DyGFormer & 0.5830 & 0.1166 & 0.2000 & 0.1473 & 1.5650 & 1.2510 & 0.6991 \\ \cline{1-1}  & GraphMixer & 0.0364 & 0.0073 & 0.2000 & 0.0141 & 11.9721 & 3.4601 & 3.3009 \\ \cline{1-1}  & **DC-DGNN** & **0.7243** & **0.5691** & **0.5435** & **0.5542** & **0.4084** & **0.6390** & **0.3134** \\ \hline \end{tabular}
\end{table}
Table 6: Supplement experimental results of our DC-DGNN model and comparison with previous dynamic graph works on datasets in the May 1996 - Oct 2018 time period (first 4 datasets) and May 1996 - July 2014 time period (last dataset). \(\downarrow\) indicates the smaller the metrics, the better the method, while \(\uparrow\) indicates the larger the metrics, the better the method. The score marked as bold means the best performance among all the methods.