Parameter-free Dynamic Graph Embedding for Link Prediction
Authors: Jiahao Liu
Dongsheng Li
Hansu Gu
Tun Lu
Peng Zhang
Ning Gu
Venue: NeurIPS 2022 Accept (2022)

Abstract: Dynamic interaction graphs have been widely adopted to model the evolution of user-item interactions over time. There are two crucial factors when modelling user preferences for link prediction in dynamic interaction graphs: 1) collaborative relationship among users and 2) user personalized interaction patterns. Existing methods often implicitly consider these two factors together, which may lead to noisy user modelling when the two factors diverge. In addition, they usually require time-consuming parameter learning with back-propagation, which is prohibitive for real-time user preference modelling. To this end, this paper proposes FreeGEM, a parameter-free dynamic graph embedding method for link prediction. Firstly, to take advantage of the collaborative relationships, we propose an incremental graph embedding engine to obtain user/item embeddings, which is an Online-Monitor-Offline architecture consisting of an Online module to approximately embed users/items over time, a Monitor module to estimate the approximation error in real time and an Offline module to calibrate the user/item embeddings when the online approximation errors exceed a threshold. Meanwhile, we integrate attribute information into the model, which enables FreeGEM to better model users belonging to some under represented groups. Secondly, we design a personalized dynamic interaction pattern modeller, which combines dynamic time decay with attention mechanism to model user short-term interests. Experimental results on two link prediction tasks show that FreeGEM can outperform the state-of-the-art methods in accuracy while achieving over 36X improvement in efficiency. All code and datasets can be found in https://github.com/FudanCISL/FreeGEM.

Parameter-free Dynamic Graph Embedding for Link Prediction
Jiahao Liu Dongsheng Li³, Hansu Gu Tun Lu Peng Zhang Ning Gu
'School of Computer Science, Fudan University, Shanghai, China 2 Shanghai Key Laboratory of Data Science, Fudan University, Shanghai, China
Miirosooft Research Asia, Shanghai, China *Seattle, United States
jiahaoliu2em fudan. edu. en, dongsli@mi crosoft. com, hansug@acm, org (lutun, zhangpeng_, ninggu)ofuuan uuda edu. cn
Abstract
Dynamic interaction graphs have been widely adopted to model the evolution of user-item interactions over time. There are two crucial factors when modelling user preferences for link prediction in dynamic interaction graphs: 1) collaborative relationship among users and 2) user personalized interaction patterns. Existing methods often implicitly consider these two factors together, which may lead to noisy user modelling when the two factors diverge. In addition, they usually require time-consuming parameter learning with back- puekppoopagttion, which is prohibitive for real-time user preference modelling. To this end, this paper proposes FreeGEMM parameter-free dynamic graph embedding method for link prediction. Firstly, to take advantage the collaborative relationships, we propose an incremental graph embedding engine obtain user/item embeddings, which is Online- Conitor-Offline architecture consisting of an Online module to approximately embed users/items over time, Monitor module to estimate the approximation error in real time and an Offline module to calibrate the user/item embeddings integrate when the online approximation errors exceed threshold. Meanwhile, attribute information into the model, which enables FreeGEM to better model users belonging to under represented groups. Secondly, design a personalized dynamic interaction pattern modeller, which combines dynamic time decay with attention mechanism model user short-term interests. Experimental results on two link prediction tasks show that FreeGEM can outperform the state-of-the-art methods in accuracy while achieving over 36X improvement in efficiency. All code and datasets can be found in wttphttt://// thub. com/ FudanCISL/FreeCE)
1 Introduction
Dynamic interaction graphs have been adopted in wide range of link prediction tasks to model the evolution of user-item interactions over time I22 14 37]. The evolution dynamic interaction graph can be reflected by its historical interaction (edge) sequence, in which two crucial factors should explicitly considered for link prediction tasks: 1) collaborative relationship among users, i.e. users with similar historical interactions will interact with similar items in the future, which is also the basic assumption of collaborative filtering 7 [3 20 and 2) user personalized dynamic interaction patterns, i.c., users could have unique short- term interaction patterss, which not shared among like-mine minded users but can only be reflected by their own interaction sequences.
Many existing methods learn user preferences from dynamic interaction graphs using temporal point process (TPP) 28
41, recurrent neural network (RNN) [ 14 32 and graph neural network (GNN) 4 23 37 etc., in which the following key challenges arise. Firstly, these methods do not
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Historical Data <>
Incremental Graph Embedding Engine
User E mbedding
Personalized Dynamic Interaction
Prediction
Stream Data
Item Embedding
Pattern Modeller
Figure 1: The high-level design of the proposed FreeGEM method.
consider the above two key factors separately, so that collaborative relationship may bring noises to personalized patterns when they diverge, and vice versa. Secondly, the above methods often require time-consuming parameter learning with pack-propagation. In dynamic interaction graphs, the model training should follow chronological order of the interactions to capture the temporal dynamics, which raises efficiency issue even for applications with moderate number of interactions.
In this paper, we propose Parameter- Dynamic Graph EMbedding (FreeGEM) method for link prediction. Here, parameter-free means that we do not incorporate any parameters which need to be learned via back-propagation. As shown in Figure [ components: incremental graph embedding engine and personalized dynamic interaction pattern modeller. The incremental graph embedding engine takes both historical data and online stream data as input and generates user/item embeddings as the output, which exploits the collaborative relationship among users. Its core innovation is the proposed which can achieve online embedding updates by solving closed-form solutions in real time and keep the approximation errors caused by online singular value decomposition (SVD) 2 within any predefined threshold. In the Offline step, we propose a frequency-avee aware preference matrix reconstruction method to alleviate the oversmoothing problem and an attribute-intes integrratee SVD to alleviate the cold-start issue. Surprisingly, the integration of attribute information enables FreeGEM better model users belonging to under represented groups. In the Online step, we convert the offline truncated SVD into an online SVD to generate embeddings estimate the online approximation error in real time by analyzing the relationship between approximation error and the update of online algorithm. The personalized dynamic interaction pattern modeller also a parameter-free component, which combines the dynamic time decay with attention mechanism to model user short-term interests. It takes the user and item embeddings as input and outputs the prediction results. Specifically, selectively forgets the early interactions through dynamic time decay and hence focuses on more recent interactions for prediction. Then, it leverages the attention mechanism to capture user personalized dynamic interaction patterns over the "decayed" item embedding sequences. Experimental results on two link prediction tasks (future item recommendation and next interaction prediction) show that FreeGEMM can substantially outperform the state-of-the-- link prediction methods in accuracy while achieving over 36X improvement in computational efficiency. Besides, our empirical studies also confirm that FreeGEM can alleviate the cold-start issue and achieve high robustness on very sparse data.
FreeGEM consists of two key
architecture,
real time. In the Monitor module,
2 Related Work
Link prediction methods on dynamic interaction graphs are mostly based on TPP, RNN and GNN, etc., which need time-consuming training process. To the best of our knowledge, the proposed FreeGEM is the only parameter-free method in this task, with high accuracy and efficiency.
Tap-based methods Know-Evolve (28 and HTNE 41] model interactions as multivariate point processes and Hawkes processes, respectively. Wang et al. [30] propose a co-evolutionary process model the co-evolving nature of users and items. Shchur al. [2 propose to directly model the conditional distribution of inter-event times. DSPP 3 incorporates topology and long-term dependencies into the intensity function.
RNN-based methods RRN 32 models user and item interaction sequences with separate RNNs.
Time-LSTM [40] proposes time gates to represent the time intervals. LatentCross incorporates contextual data into embeddings. DeepCoevolve [6] and JODIE [14] generate node embeddings using two intertwined RNNs. JODIE [14] can also estimate user embeddings trajectories. DeePRed [3] employs non-recursive mutual RNNs to model interactions.
GNN-based methods TDIG-MPNN 4 captures the global and local information on the graph. DGCF [6 uses three update mechanisms update users and items. SDGNN (27) takes the

---

R
H
Online
User Embedding
R
G¹
RH
RH
Monitor
tr]
U.g.th,t (ug.ith.))
Offline
Item Embedding
Figure 2: Illustration of the proposed incremental graph embedding engine.
state changes of neighbor nodes into account. MRATE [5] combines different relations to realize multi-relation awareness. OGN [ updates nodes in online fashion with a constant memory
cost. TCL [29] proposes a graph-topology-awar transformer to learn the representations of nodes.
CoPE 37 uses an ordinary differential equation-based GNN to model the evolution of network. TGL 39 proposes a unified framework for large-scale temporal GNN training. MetaDyGNN [34] proposes a model based on a meta-learning framework for few-shot link prediction in dynamic networks. TREND 31 uses Hawkes process- based GNN for temporal graph representation learning.
3 Incremental Graph Embedding Engine
As shown in Figure the incremental graph embedding engine 1) takes the historical -eem interaction, user/item attribute and online stream data as the input, 2) embeds users and items via the architecture, and 3) outputs the user and item embeddings in real time.
novel
Maline-Monitor-OOfling Architecture The Offline module first decomposes the historical data matrices by offline SVD to obtain user and item embeddings. capture real-time information from the interaction stream, the model needs to efficiently update the corresponding embeddings when new interaction occurs. Thus, we propose the Online module, which uses online SVD to approximately update user and item embeddings, to meet the real-time requirements. However, since the approximation error of online SVD accumulates over time, we propose a Monitor module to estimate the accumulated approximation error of the Online module in real time. When the accumulated error exceeds a threshold, we restart the Offline module to calibrate the user and item embeddings. Otherwise, continue to execute the Online module for real- real-time user/item embedding.
3.1 Offline Module
The input data of FreeGEM includes six matrices as defined in Table in which R E R is user-item interaction matrix, E Rmxp is user attribute matrix and E TR=x00 is item attribute matrix, where m/n is the number of users/items, g/p is the dimension of user/item attribute vector and (R); indicates the number interactions between user i and item j. Each stream data sample can be represented as 5-tuple (u, g, i, h, l). where i, l are user id, item id and timestamp respectively, and g E R2 and h E I2 are user and item attribute vectors, respectively.
3.1.1 Frequency-aware Preference Matrix Reconstruction
There are three steps to realize collaborative filtering 19]: 1) decompose the interaction matrix R using truncated SVD to obtain U (u,,... ..; Ug) E g=* diag(si,.... 5k) E k 81 > 8k > and V = (V1,..., Vv) E R 2) the embeddings of users and items are Ey = US1 and F, //2 respectively; and use R = Fg F.J the low rank approximation of R. Our method makes non-trivial improvements based on this, including frequency- preference matrix reconstruction in this section and attribute-nttegrated SVD in the Section 3.1.2
The raw interaction matrix R cannot perfectly reflect user preferences due to potential biases 25
we propose to normalize R as follows: R' = Di RD, where 0 is hyperparameter, Dv and Dj are diagonal matrices, and (Du \ss = nn 1 R.y and (D1);, = mm Ray- Compared with R, R' can more accurately represent user preferences. Intuitively, the more users an item interacts with, the less it can reflect each user's preference, and vice versa. The normalization can alleviate the popular bias and help to achieve more accurate user representation.
By applying offline truncated SVD on R', we have the approximation of R' as: 1== = which only retains the low-frequency signals of R', so it can be regarded as R' passing through an
3

---

Table 1: Correspondence between the decomposed matrix and the constructed embedded matrix.
Description of the decomposed matrix
Decomposed mainx
Constructed embedding (lef)
Constructed embbediig (right)
user-item
R R
E R R
E R
altribute
G R=x
Eg x
Eg R
item attribute
H Ruxq
E7 e Ruxig
Ej E Rexkg
user attribute item
G¹ R R=**
F2 E RPXX4
EJ E R=XXq
user-item_ attribute
RH E R
F E Ruckk
F E Byx
user, berr_atlribute altribbue
G¹ RH E R2x
Eo E RR
Eir E R9x/g
Table 2: Correspondence between path and embedding matrix.
No.
Path
User embedking
Item embedding
1
user-ilem
EU, Ed E [
Er, E; E &R
2
uscr-user_ ussr-user__atriibutt-iie otttribbuto-iiem
Bu² = (g T RRax
Et, (E2) RUXP
3
user-item attribute-item
Rdg Fig T E mm=x
Etg E I ER
4
user-user_ sser--ser__ttributte-ii httpeeetteemtttttteeeeeeeeee-
Eug Eyj (EG) E Rm
Er, E) Eg BRaXNG
5
aver-ilem aitribbuee wat.attiiuueeeee item
EU; (E((E)) E xm
Ei, (EE) E Ru
ideal low-pass graph filter. Thus, it will suffer from the oversmoothing problem when k is small. To alleviate this problem, we introduce a hyperparameter y to control the ratio of high-frequency signals to low-frequency signals as follows: Eg = US? E, VS (7 < 0.5). The original intensity of each frequency is 8t, which becomes "/" after applying the frequency control. The attenuation ratio /s., which hyperbola about s;. The low-frequency signal corresponds to larger s;. which means that the attenuation of the low-frequency signal is stronger than that of the high-frequency signal, improving the proportion of the high-frequency signal in the reconstructed matrix N²' In summary, this is equivalent to reducing the proportion of low-frequency signals after ideal low-pass filtering through truncated SVD (more discussion can be found in the Appendix (A.)) Finally, the predicted interaction matrix is obtained by inverse normalization as follows: R D9.
3.1.2 Attribute-integrated SVD
In addition to frequency-aware preference matrix reconstruction, compared with the plain SVD, we also integrate attribute information to better model users and items. Integrating user and item attributes can help to improve the prediction accuracy and alleviate the cold-start issue in the R. G and H are three basic matrices, which respectively describe the co-occurrence relationship of user-item, user-user attribute and ttem-item attribute. To connect them, further obtain three derived matrices G' RH RPxq, G R R and RH E R= which describe the co-occurrence relationship of user attribute-item attribute, user attribute-item and user-item eser-item_attribute respectively. After decomposing and reconstructing the six matrices using the method described in Section 1.1.I we can get 6 x 2 = 12 embedding matrices, as shown in different dimensions. As shown in Figure 3 we have four objects: user, user attribute, item and item attribute. There are C7 = 6 relationships between them, which are represented by the six matrices respectively. The six edges represent six embedding spaces, and each object has a representation in three of them. There are five paths between user and item without revisiting, each of which corresponds embedding matrix and an item embedding matrix, as shown in Table Except for the user-item path where user and item are directly connected, all other paths go through user/item attribute and thus integrate the user/item attributes in the embeddings. Finally, we only concatenate the embeddings from the first three paths as the output of the Offline module, for the last two paths consider user/item attributes repeatedly, and obtain user embedding and item embedding as follows:
recommender system 33 36
Table [ whose superscript indicate embedding space, corresponding
user
Eu = (a. F, (on Fr,, Fin Fr,) (a +p+4)
(=)(()+++))
(1)
(2)
og is the hyperparameter that controls the weight of the i-th path, and
represents concatenation.
3.2 Online Module
The only difference between Online module and Offline module is the way to obtain truncated SVD, where the Offline module uses ordinary truncated SVD but the Online module uses online SVD 22 Online SVD
22 provides approximated method to calculate the truncated SVD of updated

---

E1
R
E7
R
online
user
item
Rg
SVD
Ed
RH
G
R,
EA
E
offline
item
Rg
R1
R2 Rs
SVD
attribute
FG
RH
Eg
attribute
Figure 3: Attribute-integrated SVD.
Figure 4: Motivating example of the Monitor.
matrix in linear time. Let Rg _R be the interaction matrix at time Assuming that have finished the truncated SVD of Rg and then at time t + 1, user u interacts with item i. Thus, we have R++1 R2 ui where R and E R" are one-hot vectors indicating user id and item id, respectively. Therefore, we realize the incremental calculation of truncated SVD of R&41 given the truncated SVD of R, USVT, where E Rm k S E R V B and h1 + p + q as shown in Equation and Equation [
The details are described as follows.
Firstly, we calculate:
m U n n i, p
Um, q= i
Vn, P
|p "ppl''pp Q 0=11911-2a
(3)
Secondly, we calculate:
K
s
0 0
m
n
(4)
Thirdly, we calculate the full SVD of K and get SKSK with UK E R(+))(++)) Vx E 1(++1)x(+++) and SK E (+++)) Then, we have the following result:
R4+1= = R2 + T x (UU PIUK)Sx([V Q[Vx))
(5)
Finally, we can use the first is columns of IU PJUK. Sk and (V Q] as the estimate of the truncated SVD of Re+1-
3.3 Monitor Module
In the Online module, the approximation will accumulate time. To analyze the accumulated error, TIMERS (38] was proposed to calculate the lower bound the approximation error of online SVD through matrix perturbation (26). However, TIMERS depends on a time-consuming eigen- decomposition, so it can not monitor the етгоr in real time but based on the granularity of time slice for digen-dccomposition. Nevertheless, TIMERS provides
insights for the design of the Monitor: 1) the accumulated approximation error does not change uniformly with time and 2) there are no shared correlation patterns between the running time of online SVD, the number new observations in the interaction matrices and the accumulated eтror among different datasets.
The timing of restart Offline module is very important because untimely restart will lead excessive error and reduce the embedding quality and too frequent restart will lead to low efficiency. There could be two restart heuristics: 1) restart after certain time and 2) restart after certain number of new interactions. However, both heuristics are not optimal due to unawareness of the online approximation error, leading potentially unnecessary or inaccurate updates.
The motivation of the Monitor is illustrated Figure 4 At time step O, we initialize the offline truncated SVD of the user-item interaction matrix Rg. At time step i, we can have matrices: I) R, if we use offline SVD and 2) R, if we use online SVD to approximate the matrix when new interaction occurs. For ease illustration, of the reconstructed matrices, and arrows to represent the evolution directions of the results. The angle between the evolution directions of offline SVD and online SVD should be less than n/3 (more discussion can be found in the Appendix A.2), otherwise the results of online SVD is even worse than not updating at all. Since there is no online approximation error in offline SVD, we take Å, where 1, 2,3, as the ground truth and fty, where i = 1, correspond to the results obtained by online SVD after each update. The lengths of Rg Ri (Ro RG Rg (R₂ Rx) and RgRs (Roy Ray)
straight lines represent the F-norm distance
5

---

also pwy finil vector
user embedding
long-term
user id
embedding
weighted sum
engine with time decay
long
prefecmnnee
Attention
item embedding
Time Decay
Attention
Self
K, V
Item sequence
Figure 5: Personalized dynamic interaction pattern modeling module.
are not equal, for different interactions have different impacts on the evolution of SVD [38 The dotted line represents the F-norm distance between the matrices reconstructed by offline SVD and online SVD, = || R [r. i..., the online approximation error. The dotted line become longer over time, indicating that the error will accumulate with the occurrence of new interactions.
We use distance to specifically refer to the F-norm distance between the reconstructed matrix by online SVD at the current time step and the initial reconstructed matrix: d = || RollF. Calculating R1. hope to monitor the length of the dotted line without calculating it. While calculating Rr. Rz Ry is efficient due to the linearity of online SVD and it can be from the Figure 4 that there is a positive correlation between distance and approximation error (empirically verified in the Appendix (A.3), we estimate the online approximation error though the distance. Every time the Online module is executed, the Monitor will calculate the distance in real time. When the distance exceeds the predefined threshold, we believe the corresponding approximation reaches another threshold for restarting the Offline module.
R2, R3 is time come-consuming.
Compared with the two simple heuristics, Monitor estimates errors in data-driven way with the following benefits: 1) the Monitor estimates the error according to the interaction events, avoiding the negative impacts caused by uneven interaction distributions in different time intervals; and 2) the Monitor estimates the error according to the positive correlation between the distance and the error, avoiding the varying impacts of different interactions the error estimation. It should be noted that, compared with the two heuristics, Monitor only adds the process of calculating the distance, which is very efficient and thus can achieve real-time monitoring.
4 Personalized Dynamic Interaction Pattern Modeller
Both Offline module and Online module cannot model the evolution trends of the dynamic interaction graph due to lacking the ability of memorization. The incremental graph embedding engine captures collaborative relationships shared among similar users, but the interaction pattern of each user in his/her interaction sequence is irrelevant his/her preference, i.e., interaction patterns are unique instead of collaborative with like-minded users. Therefore, we propose an additional downstream module to capture user personalized dynamic interaction patterns illustrated Figure 5
4.1 Dynamic Time Decay
We define the process from the execution an Offline module to the execution of the next Offline module stage. At the beginning of the i-th stage, when constructing the historical interaction matrix, we decay the historical interaction score as exp(,6(t/T, - 1)]. where T; is the beginning time the i-th stage and 3 >> 0 is the decay coefficient. This implies that, for each stage, the score of historical interaction (t < T:) is less than 1, while the score interactions current stage (t > T,) is greater than equal to 1. The decay should be memoryless, i.c., the decay factor is unchanged for the same time difference (more discussion can found the Appendix A.4). This requires that 3/T is constant. Thus, for stage and stage their decay coefficients have the following relationship: 8./3, = T./T,
Time decay makes the model more relevant to recent interactions through selective forgetting. It can help the model capture the interaction pattern that users often interact with the recently interacted items in many link prediction tasks. However, it cannot automatically capture the personalized

---

dynamic interaction pattern of each user. Thus, we propose to combine attention mechanism with dynamic time decay achieve personalized interaction pattern modelling.
4.2
Attention Module
For user, we use the weighted average of recent user embeddings as his/her long-term embedding:
elong
(6)
where e! E R is the corresponding user embedding, that is, row of the Ev, at the r-th time step. Then, we concatenate the decayed embeddings of the user's recent items as S. E k and apply self-attention and attention to obtain the short term preference embedding Cshrrt follows:
S e
-1, (b)
Sy
Su Su
ST @short = (Sy) S Clong
Vk
Vk
Finally, we obtain the final user embedding by weighted average:
Xeshart + (1 X) etmng.
(8)
We use the dot product between current item embeddings and user final embedding: E, e as the scores for predicting the link between the user and all items. It should be noted that both self-attention and attention operations in our design require no weight matrices, i... they are parameter- free. The whole process is shown in the Figure
Discussion user has his/her own historical interactions. corresponding to personalization, and the recent historical interactions change over time, corresponding to temporal dynamic. A user's s long-term preference vector should changing smoothly, which is in line with the characteristies of long-term interests. A user's short-term preference vector is linear combination of the embeddings of recently interacted items, which changes rapidly over time and is in line with the characteristies of short-term interests. Time decay of item sequences plays key role in position embedding, which indicates that we will pay more attention to the recently interacting items. Self-attention can integrate the information of the whole sequence, and attention makes items with high similarity to users more likely to be interacted again. Compared with time decay, the attention module is data-driven and automatically adapt the personalized and dynamic interaction patterns by making full use of the recent interactions. However, compared with time decay, attention may be disturbed by many unnecessary patterns. Our empirical studies confirm that time decay and attention complementary to each other and should be combined in the model.
Experiments
Datasets For the future item recommendation task, we use the Amazon Video, Amazon Game 9 MovieL.ens-1M (ML-IM) and KovieLLens-100K (MI-100K) [8). For the next interaction prediction task, we use Wikipedia and LastEM [14). For all datasets, we use the first 80% interactions as training set, the following 10% interactions validation set, and the last 10% interactions test set.
Metrics For the future item recommendation task, we use Recall@ 10 to evaluate models. For the next interaction prediction task, we use MRR and Hir@ 10 to evaluate models. For all experiments, we report the results on the test set when the models achieve the optimal results on verification sets.
Baselines For the future item recommendation task, we compare with LightGCN [10] Time- LSTM [40), RRN (32, DeepCoevolve (6) JODIE [14] and CoPE (37] For the next interaction
prediction task, we compare with Time-LSTM 40) RRN (2. LatentCross M. CTDNE [17). DeepCoevolve [6) JODIE [14] and CoPE 37 Among all baselines, LightGCN (10 is the only static graph representation learning method for recommendation tasks. These baselines include TPp-based, RNN-based and GNN-based methods. For clarity, we discuss and compare these methods with FreeGEM in the Appendix A.5
7

---

Table 3: Accuracy comparison with state-of- aart--rrthe-rtt methods on two link prediction tasks.
(a) Future item recommendation
(b) Next interaction prediction
Video
Game
ML-100K
ML-IM
Wikipedia
LastFM
Rocall
Rocall
LighTGCN
0.056
0.026
0.025
0.029
Time-LSTM
0.247
0.342
0.068
0.137
Time I.STM
0.044
0.020
0.058
0.033
RRN
0.522
0.617
0.089
0.182
RRN
0.068
0.029
0.065
0.043
LatenCross
0.424
0.481
0.148
0.227
DespCocvolve
0,030
0.027
0.069
0.030
CTDNE
0.035
0.056
0.010
0.010
JODIE*
0.035
DoepCoevaive
CoPE:
0
0.047
0.081
0.049
JODIE
0,746
0.822
0.195
0.307
FreeGEM *(no attr)
0.446
FreeGEM "(with aftr)
0.149
0.065
FreeGEM
0.786
0.852
0.195
0.453
Table 4: The ablation studies on the future item recommendation task.
Pattern Modeller
Matrix Recoustruction
Video
Game
ML-100K
ML-IM
0.073
0.023
0.051
0.045
0.107
0.041
0.074
0.051
0.080
0.025
0.052
FreeGEM *(mo attr)
V
0.113
0.050
0.114
0.053
5.1 Future item recommendation
We use this task to verify whether the model can accurately predict user future interactions according to their historical interactions, which is a typical application of dynamic interaction graphs in recommender system. In this task, a user interacts with an item only once at most.
The results are shown Table 3(a) in which the results of baselines reported by CoPE (37. To be fair, we do not allow JODIE, CoPE and FreeGEM to update models during test phase (marked with *). In addition, since all baselines do not integrate attribute, we provide both the results FreeGEM with (with--ttr) and without (no-attr) attributes. It can be observed that FreeGEM *(no-attr) achieves better accuracy than all baselines on all datasets. Compared with truncated SVD, FreeGEM *(no-attr) only adds the proposed trequency-award reconstruction module and the personalized interaction pattern modeller. Further, FreeGEM "(with-tttr outperforms FreeGEM *(no-attr) due to the proposed attribute-integrated SVD, which verifies the effectiveness of this module. Among all baselines. LightGCN performs the worst, because it is the only static GNN model which cannot capture the dynamic characteristics of the interaction graph.
5.2 Next interaction prediction
We use this task to verify whether the model can accurately predict a user's next interaction according to the user's historical interactions up the current timestamp, which is kind of user behavior prediction problem. In this task, a user may interact with an item for many times.
The results are shown in Table 3(b) (b), in which the results of baselines are also reported by CoPE [37 There are no user/item attributes Wikipedia and LastFm, so we use all proposed modules except the attribute-integrated SVD in FreeGEM. Since JODIE, CoPE and FreeGEM can update models in test time, they significantly outperform the other methods without test time training. FreeGEM can outperform JODIE and achieve comparable accuracy with CoPE. Although the accuracy of FreeGEM has no obvious advantage over CoPE in this task, we will show later that FreeGEM can significantly outperform CoPE in computation efficiency due to no learnable parameters.
5.3 Ablation Studies
Future item recommendation We use A, B, C to refer to ablative variants FreeGEM, and check cross marks indicate whether the corresponding module exists. As shown in Table 4 when the personalized interaction pattern modeller or frequency-awart reconstruction module is not adopted, the results are suboptimal, which confirms the effectiveness of these two modules.
Next interaction prediction We use D-I to refer to ablative variants of FreeGEM, and check or cross marks indicate whether the corresponding module exists. In addition, we use an intuitively effective baseline, called Last-k, which takes the recent & items that users interacted with as predictions. The

---

Table 5: The ablation studies on the next interaction prediction task.
Wikipedia
LasiFM
Olliine
Online
Decay
Atlention
MRR
Hiter
MRR
Hit0 10
Last-]
0.775
0.775
0,058
0.098'
Last-10
0.792
0,842
0.139
0.263
D
x
x
K
X
0.441
0.620
0.074
0.186
X
X
0.222
x
/
X
X
0.497
0,715
0.104
0.251
//
X
X
0.530
0.739
0.11
0.256
/
X
0.779
0.851
0.163
0.348
/
/
X
0.541
0.747
0.190
0.446
FreeGEM
0.786
0.453
wikipadia
lastim
1.90
380
0.8
Wildocdo Hit
360
0.7
Lastin Hit
Neen
140
0.6
Lead MIR
Willocdia HRR
0.4
120
20%
60% Porcontage oftinnning sata
40%
80%
676 75
89
99
1.0%
6%
7%
9%
10%
Figure 6: Performance comparison of three restart methods.
Figure 7: Robustness.
results are shown in Table Note that the result of Last-/ is Hit@ so we marked its result with D dose not update in test phase; E only uses offinn SVD to update guided by Monitor in test phase; F updates in real time, but without offline restart. G better than D, and F, which confirms the effectiveness of the Sediie-Monito--Offliii the effectiveness of both dynamic time decay module and attention module. Dynamic time decay is very effective on Wikipedia, while attention has stronger effects on LastFm. This is due to the phenomenon that many users continuously interact with the same item in Wikipedia, which can be proved by the excellent results of Last-k on Wikipedia. It should be emphasized that, we think the upstream incremental graph embedding engine and the downstream personalized dynamic interaction pattern modeller are independent, and different implementations of the downstream modules can make our model deal with different downstream tasks. Therefore, during the ablation study, we do not perform cross-component ablation studies between the upstream engine and the downstream modeller, but only perform ablation studies within each component (i.c., within the upstream engine and within the downstream modeller).
architecture. FreeGEM is better than H and I, indicating
Monitor We compare the average approximation errors of Monitor, Interaction (restart with fixed number of interactions) and Time (restart with fixed time). when the number of restarts are the same. First, we execute online SVD to obtain the total approximation error, take 6% 10% of the total error as the threshold of Monitor, and finally run Time and Interaction according to the number of restarts of Monitor. The results are shown in Figure Compared with the average approximation errors of Monitor, the errors of Interaction and Time are 10.93% and 5.28% higher on Wikipedia and 16. 16.80% and 9.67% higher on LastFm, respectively, indicating that Monitor is more effective.
5.4 Other Studies
Table 6: Total running time comparison. The times of speedup is shown in the parentheses.
Wikipedia
Lastim
FreGEM
JODIE
CoPE
FreeCEM
JODIE
CoPE
9.7min
350.0min (3).X) 3,389.Imii (370.0X)
34,8min
15.790.0mi (288.1X) 31.212.5mia (934.3X)
Running time We use the next interaction prediction task to study the efficiency of JODIE, CoPE and FreeGEM. Other baselines, such as Time-LSTM [40]. RRN 32), LatentCross [I] and CTDNE [17] show comparable efficiency with JODIE [14 and thus are omitted. JODIE and CoPE both run epochs and choose the best performing model on validation set as the optimal model, but FreeGEM only requires multiple runs of offline/online SVD. As shown in Table 6 FreeGEM is at least 36X faster than JODIE and at least 370X faster than COPE, demonstrating
high efficiency. Besides, the running time of JODIE and CoPE increase significantly when the number of interactions

---

increases, but the running time of FreeGEM only increases moderately which is more desirable for real-world applications.
Cold-start We use the future item recommendation task to study the effects of atrribute-integrated SVD in cold-start setting. We define the users who have not appeared in training set and verification set as cold-start users. For ML-IM, there are 29 cold-start users. When attributes are not integrated, FreeGEM randomly recommends items to these users. For 17 of the cold-start users whose Recall @ 10 is not O, FreeGEM increases their average Recall @ 10 from 0.011 to 0.029, achieving a relative increase of 166 %. For 7 cold-start users whose Recall 10 is O, FreeGEM increases their average Recall@ 10 to 0.100, which is very high compared with the average Recall @ 10 of all users in ML-IM.
Robustness For the next interaction prediction task, we change the proportion of the training set to verify the robustness of FreeGEM in different levels of data sparsity. We change the percentage of the training set from 10% to 80%, next 10% interactions after the training set the validation set, and next 10% interactions after the validation set as the test set. The results are shown in Figure 7 in which we can observe that the accuracy of FreeGEM is almost unaffected. This experiment demonstrate that that FreeGEM has strong robustness to the scale of training data. The results of many baselines can also be found in JODIE [4]
Under represented groups There could be recommendation bias concerns on under represented groups in specific applications and the existence of under represented groups usually comes from the biases during the data collection process. Some bias mitigation techniques are specially designed to solve this problem (2I). We carry out further experiments on ML-100K. First, we group the users by gender and find that there are 670 male users (71.0% and 273 female users (29.0%) in this dataset. Male users had 753,313 interaction records (74.4%), and female users had 246,298 interaction records (25.6%). This shows that there is bias of gender distribution in this dataset. Then, we remove the attribute-integrated SVD module from our method and calculate the performance of our model on male and female users. We find that without using attributes, the Recall of male users is 0.124 and Recall of female users is 0.085. Male users are with 45.9% higher Recall than female users. Finally, we take the attribute-intege integratee SVD module back to see how the performance differs. We find that after using attributes, the Recall of male users increases to 0. 156, Recall of female users increases to 0. 0.1 Male users are with 22.8% higher Recall than female users. To sum up, due to the bias of gender distribution in the ML-1006 dataset, there a clear bias in recommendation accuracy, i.e., male users are with 45. 59. higher Recall than female users even without using user attributes in the model. However, to our surprise, the bias in the recommendation results becomes less significant after using user attributes in model, i.e., male users are with only 22.8% higher Recall than female users when user attributes are incorporated. We think the reason is that user attributes can help these under represented groups (female users) to improve their embedding quality.
6 Conclusion
We propose FreeGEM, a parameter-free dynamic graph embedding method for link prediction. By modelling collaborative relationships and personalized dynamic interaction patterns separately, FreeGEM can alleviate the noises when the two key factors diverge, leading to higher accuracy. Specifically, we propose an incremental graph embedding engine for real-time dynamic graph embedding via a novel Online-Monitor-Offinn pattern modeller with dynamic time decay and attention. Since there are no learnable parameters,
architecture and a personalized dynamic interaction
FreeGEM can avoid the time-consuning back-propagation, leading to significantly higher efficiency. One limitation of this work is that we only explore the link prediction task, leaving other downstream machine learning tasks, such as node classification, as future work. Another line of future work is to extend the high level design of our method to other kinds of parameterized models. Although we empirically find that our method can mitigate rating bias by utilizing user/item attributes, practitioners should pay more attention to the biases during data collection before using our method.
Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (NSFC) under Grants 61932007 and 62172106.
10

---

References
[1] A. Beutel, P. Covington, S. Jain, C. Xu, J. Li, V. Gatto, and E. H. Chi. Latent cross: Making use of context in recurrent recommender systems. In Proceedings the Eleventh ACM International Conference on Web Search and Data Mining, pages 46-54, 2018.
[2] M. Brand. Fast low-rank modifications of the thin singular value decomposition. Linear algebra and its applications, 415(1):20-34) 2006.
[3] J. Cao, X. Lin, X. Cong, S. Guo, H. Tang, T. Liu, and B. Wang. Deep structural point process for learning temporal interaction networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 305-320. Springer, 2021.
[4] X. Chang, X. Liu, J. Wen, Li, Y. Fang. L. Song, and Y. Qi. Continuous-time dynamic graph learning via neural interaction processes. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 145-154, 2020.
[5] L. Chen, S. Yu, D. Lyu, and D. Wang. Multi-relation aware temporal interaction network embedding. arXiv preprint vrXiv:2110.045000 2021.
[6] H. Dai, Y. Wang. R. Trivedi, and L. Song. Deep coevolutionary network: Embedding user and item features for recommendation. arXiv preprint arXiv: 1609.03675, 2016.
[7] D. Goldberg, D. Nichols, B. M. Oki, and D. Terry. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12):61-70, 1992.
[8] M. Harper and J. A. Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1-19, 2015.
[9] R. He and J. McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference world wide web, pages 507-517, 2016.
[10] X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference research and development in Information Retrieval, pages 639-649 -48, 2020.
[11] H. Kang, J.-H. Ho, D. Mesquita, J. Pérez, and Souza. Online graph nets. 2021.
[12] S. M. Kazemi, R. Goel, K. Jain, I. Kobyzev, A. Sethi, P. Forsyth, and P. Poupart. Representation learning for dynamic graphs: A survey. J. Mach. Learn. Res., 21(70):1-73, 2020.
[13] Z. Kefato, S. Girdzijauskas, N. Sheikh, and A. Montresor. Dynamic embeddings for interaction prediction. In Proceedings of the Web Conference 2021, pages 1609-1618, 2021.
[14] S. Kumar, X. Zhang, and J. Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1269-1278, 2019.
[15] D. Li, C. Chen, W. Liu, T. Lu, N. Gu, and S. Chu. Mixture-ram matrix approximation for collaborative filtering. In Advances in Neural Information Processing Systems, pages 477-485, 2017.
[16] X. Li, M. Zhang, S. Wu, Z. Liu, L. Wang, and S. Y. Philip. Dynamic graph collaborative filtering. In 2020 IEEE International Conference Data Mining (ICDM), pages 322-331 IEFF, 2020.
[17] G. H. Nguyen, J. B. Lee, R. A. Rossi, N. K. Ahmed, E. Koh, and S. Kim. Continuous -time dynamic network embeddings. In Companion Proceedings of the The Web Conference 2018, pages 969-976. 2018.
[18] H. Nt and T. Maehara. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint 190500500.005550 2019.

---

[19] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Application of dimensionality reduction in recommender system-a case study. Technical report, Minnesota Univ Minneapolis Dept of Computer Science, 2000.
[20] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Item base- collaborative filtering recommenda- tion algorithms. In Proceedings of the 10th international conference on World Wide Web, pages 285-295, 2001.
[21] S. Saxena and S. Jain. Exploring and mitigating gender bias in recommender systems with explicit feedback. arXiv preprint arXiv:21 2021.
[22] O. Shchur, M. Biloš, and S. Günnemann. Intensity-free learning of temporal point processes. arXiv preprint arXiv: 201,
[23] Y. Shen, Y. Wu, Y. Zhang. C. Shan, J. Zhang, B. K. Letaief, and D. Li. How powerful is graph convolution for recommendation? In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 1619-1629, 2021.
[24] H. Steck. Item popularity and recommendation accuracy. In Proceedings of the fifth ACM conference on Recommender systems, pages 125-132, 2011.
[25] H. Steck. Markov random fields for collaborative filtering. Advances in Neural Information Processing Systems, 32, 2019.
[26] G. W. Stewart. Matrix perturbation theory. 1990.
[27] S. Tian, T. Xiong, and L. Shi. Streaming dynamic graph neural networks for continuous -time temporal graph modeling. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1361-1366. IEEE, 2021.
(28) R. Trivedi, H. Dai, Y. Wang, and L. Song. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In international conference on machine learning. pages 3462-3471. PMLR, 2017.
[29] I. Wang, X. Chang, S. Li, Y. Chu, H. Li, W. Zhang, X. He, I. Song. J. Zhou, and H. Yang. Tel: Transformer-bass based dynamic graph modelling via contrastive learning. arXiv preprint arXiv:2/05.07944, 2021.
[30] Y. Wang. N. Du, R. Trivedi, and L. Song. Coevolutionary latent feature processes for continuous- time user-item interactions. Advances in neural information processing systems, 29, 2016.
[31] Z. Wen and Y. Fang. Trend: Temporal event and node dynamics for graph representation learning, In Proceedings of the ACM Web Conference 2022, pages 1159-1169, 2022..
[32] C.-Y. Wu, A. Ahmed, A. Beutel, A. Smola, and H. Jing. Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining, pages 495-503, 2017.
[33] J. Xia, D. Li, H. Gu, J. Liu, T. Lu, and N. Gu. Fire: Fast incremental recommendation with graph signal processing. Proceedings of the ACM Web Conference 2022, WWW '22, page 1360-2369. ACM, 2022.
[34] C. Yang, C. Wang, Y. Lu, X. Gong, C. Shi, W. Wang, and X. Zhang. Few-shot link prediction in dynamic networks. In Proceedings of the Fifleenth ACM International Conference on Web Search and Data Mining, pages 1255-1255, 2022.
[35] W. Yu and Z. Qin. Graph convolutional network for recommendation with low-pass collaborative filters. In International Conference on Machine Learning, pages 10936-10945. PMLR, 2020.
[36] Z. Zeng, C. Xiao, Y. Yao, R. Xie, Z. Liu, Lin, L. Lin, and M. Sun. Knowledge transfer via pre-training for recommendation: A review and prospect. Frontiers in big Data, page 4, 2021.
[37] Y. Zhang, Y. Xiong, D. Li, C. Shan, K. Ren, and Y. Zhu. Cope: Modeling continuous propagation and evolution on interaction graph. In Proceedings of the 30th ACM International Conference on Information& & Knowledge Management, pages 2627-2636, 2021.
12

---

[38] Z. Zhang, P. Cui. J. Pei, X. Wang, and W. Zhu. Timers: Error-bounded svd restart on dynamic networks, Thirty-second AAAI conference on artificial intelligence, 2018.
[39] H. Zhou, D. Zheng, Nisa, V. loannidis, Song, and G. Karypis. Tgl: A general framework for temporal gnn training on billion-scale graphs. arXiv preprint arXiv:2203. 14883, 2022.
[40] Y. Zhu, H. Li, Y. Liao, B. Wang, Z. Guan, H. Liu, and D. Cai. What do next: Modeling user behaviors by time-Istm. In IJCAI, volume 17, pages 3602-3608, 2017.
[41] Y. Zuo, G. Liu, H. Li Guo, X. Hu, and Wu. Embedding temporal network via neighborhood formation. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2857-2866, 2018.
13


Reviews:
[{"review_id": "RQcSU2ZWcJ7", "reviewer": "Reviewer_coS6", "summary": "This paper introduces a parameter-free dynamic graph embedding (FreeGEM) method for link prediction tasks. For modeling user preference in dynamic integration graphs, there are two crucial factors to consider: 1) collaborative relationships among users and 2) user personalized interaction patterns. Existing methods usually implicitly consider these two factors together and suffer from noisy distributions. The proposed method considers the two factors separately instead: with an incremental graph embedding engine to model the former and a personalized dynamic interaction pattern modeler to model the latter. For the incremental graph embedding engine, an Online-Monitor-Offline architecture is proposed, with an online module to approximate the user/item embeddings, a monitor module to estimate the errors of the online module, and an offline module to calibrate the embeddings, such that the module can learn the embeddings efficiently without exceeding the error threshold. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of the proposed method. ", "questions": "1) In  section 4.2, it says u^(r) is the user embedding as r-th time step. Since it never appears in the paper before, is u^(r) a learned embedding from the model? In section 3.2, it also uses the notation u as a one-hot vector indicating the user id. Are they the same thing?\n2) In section 3.1.1, it is not clear for me how E_I and E_U are involved in obtaining the interaction matrix(R-hat) with the low-frequency approximation. Can you explain more about the logic flow here?\n3) Through the description of section 3,  it seems that Online-Monitor-Offline architecture eventually estimates the interaction matrix. How do you transform the interaction matrix to user/item embedding? Some clarifications are recommended here. \n4) It would be helpful to label all the equations in this paper. \n5) Line 155, a missing space between the sentences.\n", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- Originality: The proposed method with the two modules to model the two factors and the Online-Monitor-Offline architecture is novel. \n\n- Quality: The content of the paper is well organized. The motivation is clear. Sufficient experiments and ablation studies are conducted. \n\n- Significance: This paper presents a new method that can estimate the user/item embedding for link prediction tasks efficiently. \n\nWeaknesses:\n\nThe methodology part needs more clarification. Some equations or concepts are not well-explained. I point out these issues in Questions.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GR5c25b9xBy", "reviewer": "Reviewer_vE5r", "summary": "The authors present a parameter-free graph embedding model to handle the missing link prediction tasks, including the future-item recommendation and the next-item recommendation. In short, the framework consists of 1) a graph embedding module and 2) a personalization module, where the former is built with an online-monitor-offline structure, and the latter introduces the concept of time decay and attention.", "questions": "Is it possible to apply such an architecture to parameterized models?\n", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n- The idea is novel and interesting.\n- The proposed model achieves a good improvement in efficiency.\n- The experiments are comprehensive, including the evaluation of two link prediction tasks, the ablation studies, and results regarding the efficiency. \n\nWeakness\n- Some parts of the paper are unclear. \n1. In line 181, why π/3 is the boundary?\n2. In line 183, what does 1-3 or 1’-3’ mean?\n3. In line 215, there are variant ways to do time decay. Why this one? \n4. In table 5, is it possible to apply H and I without D/E/F/G?\n- It is a bit hard to totally understand the key to the solution as the whole model contains many submodules, which are not comparable to other existing models. For example, the decay mechanism is essential, but do some other models not consider it? I suggest the authors compile a table explaining the differences among the compared models.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "CRNH67PqTnA", "reviewer": "Reviewer_uRAL", "summary": "This work proposed an approach to model the dynamic user-item interaction graphs. Ths proposed approach is a framework comprising an online-offine architechture with majorly matrix-factorization techniques.", "questions": "I understand it is tedious to prepare the code and data sets. But I still strongly suggest the authors provide the code and data sets to make the process more transparent. Maybe there are some concerns (e.g. ethical, anonymity) the author(s) can elaborate more in the author rebuttal phase?", "limitations": "I have my reservation related to the content of checklist where the author(s) wrote \"To the best of our knowledge, this work does not have potential negative social impacts.\" I like the confidence signaled. However, when it comes to user-item data analysis, a common ethical issue is that some users and items may suffer because they belong to some under represented groups. Especially when the training data is splited into fragments. I think the paper can be better if the author(s) elaborate more on this direction and futhermore the potentials of this work or continuations of this work can help mediate the mentioned issue.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I like the combination of matrix factorization and dynamic graph-structured data. The experiment results demonstrate that this combination does have a positive impact.\n\nThe idea of parameter-free is always exciting in data analysis. However, I cannot fully agree with the definition proposed by the author(s). I expect to see a model without or with less complicated hyper-parameter or training parameter set-ups. However, according to the definition:\n\n\"...we do not incorporate any parameters which need to be learned via back-propagation...\" the reality is far from the expectation.\n\nIn the end, this work still requrieds a search of (a wide range of) hyper-parameters. Given the required computation for model training, it can be better if the author(s) can provide more insight on how to choose the hyper-parameters (e.g. rule-of-thumb) instead of through experiments.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept