TUT4CRS: Time-aware User-preference Tracking for Conversational Recommendation System

Authors: dongxiao  he
jinghan  zhang
xiaobao  wang
meng  ge
zhiyong  feng
longbiao  wang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 10:15:34

Abstract: The Conversational Recommendation System (CRS) aims to capture user dynamic preferences and provide item recommendations based on multi-turn conversations. However, effectively modeling these dynamic preferences faces challenges due to conversational limitations, which mainly manifests as limited turns in a conversation (quantity aspect) and low compliance with queries (quality aspect). Previous studies often address these challenges in isolation, overlooking their interconnected nature. The fundamental issue underlying both problems lies in the potential abrupt changes in user preferences, to which CRS may not respond promptly. We acknowledge that user preferences are influenced by temporal factors, serving as a bridge between conversation quantity and quality. Therefore, we propose a more comprehensive CRS framework called Time-aware User-preference Tracking for Conversational Recommendation System (TUT4CRS), leveraging time dynamics to tackle both issues simultaneously. Specifically, we construct a global time interaction graph to incorporate rich external information and establish a local time-aware weight graph based on this information to adeptly select queries and effectively model user dynamic preferences. Extensive experiments on two real-world datasets validate that TUT4CRS can significantly improve recommendation performance while reducing the number of conversation turns.

Main paper: 
 introduction: 
 The recommendation system plays a crucial role in efficiently delivering personalized recommendations to users based on their interests [31] . By modeling user preferences, these systems effectively address the problem of information overload [24] . However, traditional recommendation systems rely on the user's historical interactions to model their interests, having the assumption that these interests remain static over time [4] . Consequently, it is challenging to capture fine-grained and dynamic user preferences accurately.Conversational recommendation systems (CRS) [17] are designed to capture user dynamic preferences in a more fine-grained manner by allowing the system to directly ask users for their preferences and gather feedback. CRS can be divided into two scenarios: single-turn and multi-turn conversational recommendation. In the single-turn conversational recommendation (SCR), the system stops after recommending items to the user, regardless of whether the user is satisfied or not. On the other hand, the multi-turn conversational recommendation (MCR) continues to interact with the user by asking questions or making recommendations until it successfully meets the user's needs or reaches a maximum number of turns [5, 18] . For a more interactive and personalized recommendation experience, MCR aligns better with real-world requirements, allowing for ongoing interaction with users and ensuring that their needs are fully addressed.Existing methods in multi-turn conversational recommendation (MCR) typically rely on direct user input to model their preferences, known as user dynamic preferences. These preferences are then used to provide personalized recommendations. However, the effectiveness of these methods in achieving their intended outcomes is limited due to the issue of conversation limitation. This limitation manifests in two distinct ways: limited turns in a conversation (quantity aspect) and low compliance of queries (quality aspect). The first aspect, limited turns in a conversation, refers to the constraint on the number of turns allowed in a conversation. This limitation poses a challenge as it hinders the system's ability to gather sufficient information and accurately model user dynamic preferences within the given timeframe. The second aspect, low compliance of queries, pertains to the difficulty in selecting the most appropriate query to ask the user in each turn. With a large set of query candidates to choose from, it becomes increasingly challenging for the system to accurately identify the correct query.To address the issue of quantity, some works [9, 33] have attempted to obtain additional information beyond conversational information and mine the user's cyclical behavior through the user's historical interactions to model user static preferences. When there is no clear dynamic preference in the user's conversation, recommendations can be made using the user static preference. Regarding the issue of quality, some works [18, 34] model CRS as interactive path reasoning on a graph, constraining the candidate query set to only include the neighbor nodes of the current node, thereby greatly reducing the candidate space and selecting the correct query.Previous research has often addressed these two issues in isolation, neglecting their intrinsic interconnections, which can lead to various challenges. As depicted in Figure 1 , the left side illustrates that most existing methods typically tackle the quantity aspect by modeling user static preferences (i.e., represented as a "hamburger"), achieved by filtering out occasional behaviors from the user order history. Consequently, the agent's query selection tends to prioritize static preferences. However, inaccurate additional information may result in inappropriate queries, rendering the candidate set unfilterable, exacerbating the quality aspect problem. Thus, these two issues are interrelated and mutually influence each other. It is imperative to address them jointly to devise a comprehensive solution. Fortunately, user behavior often demonstrates temporal influences, reflecting differences in users' consumption habits at different times. This is exemplified on the right side of Figure 1 , where the occasional behavior of ordering roast duck in the user's historical orders is attributed to Valentine's Day, reflecting users' tendency to select more formal and expensive meals on such occasions. Considering time enables us to acquire accurate external information to tackle the quantity aspect problem while aiding in candidate set filtration. Additionally, the attributes within the candidate set inherently possess temporal properties, allowing us to further refine the candidate set using time information to address the quality aspect problem. Motivated by this observation, we propose utilizing the time factor as a means to address both problems simultaneously, serving as a bridge between them.However, incorporating time factors to tackle the aforementioned challenges presents significant difficulties. When integrating time factors into the fine-grained modeling of user static preferences, occasional yet significant behaviors may inadvertently be filtered out as noise. Moreover, a vast number of candidate items and attributes can diminish the efficiency and accuracy of recommendations and queries. Addressing how to leverage time factors for effective filtering remains a formidable challenge-specifically, determining how to efficiently discern the relationships among time, items, and attributes, and subsequently filter candidate sets based on these relationships. Most importantly, achieving integration between these two modules in a mutually reinforcing manner poses a critical challenge.To solve these problems, we propose a novel framework, i.e., Time-aware User-preference Tracking for Conversational Recommendation System (TUT4CRS), which consists of two main components. Firstly, we utilize user historical interactions to build a Global Time Interaction Graph. This graph serves as a bridge between the user and the item, allowing us to model the user static preferences. We can learn about the user's consumption habits from their occasional behaviors. The item can also gather information about its implied temporal features. This approach enables us to capture both the user long-term preferences and short-term preferences. Secondly, we construct a Local Time Aware Weight Graph to leverage the previously learned information. This graph dynamically learns the user's perception of time and helps us make query selections based on this understanding. By incorporating this component, we can efficiently model the user's finer-grained dynamic preferences.To summarize, the contributions of this paper include:• We acknowledge the temporal impact on user behavior and propose linking the quality and quantity aspects of conversational recommendation through the time factor, opening a new direction for future research. • We propose a novel framework that utilizes time factor to capture incidental yet significant behaviors beyond users' dynamic preferences. This framework integrates this information to aid query selection during conversations, leading to a more precise depiction of user dynamic preferences in a shorter number of turns. • We conduct experiments on two real-world datasets and demonstrate that TUT4CRS can effectively improve the performance of conversational recommendations. 
related works: 
 Unlike traditional recommendation systems [2, 7, 8] , Conversational Recommendation Systems (CRS) aim to interact with users and capture their feedback to infer their dynamic preferences, then use the dynamic preferences to make recommendations. Due to its ability to dynamically get the user's feedback, CRS has become an effective solution for capturing dynamic user preferences and solving the explainability problem. CRS can be divided into two main parts of the task: the conversational module for language understanding and generation [14, 19] , and the recommendation module for learning strategies for querying and recommendation [16, 18, 20] . This work focuses on the recommendation module. The Multi-Round Conversational Recommendation (MCR) [5, 16, 18, 32] task is the most realistic assumption in the currently proposed problem scenarios for CRS. In MCR task, the agent continues to interact with the user by asking questions or making recommendations until it successfully meets the user's needs or reaches a maximum number of turns. The main challenge for MCR is how to dynamically learn user preferences and select the right action accordingly, i.e., what attributes to ask for (query action) or what items to recommend (recommend action). CRM [25] and EAR [16] learn user preferences with a factorization-based method under the pairwise Bayesian Personalized Ranking (BPR) framework [23] . SCPR [18] models conversational recommendation as an interactive path reasoning problem on a graph, which is able to prune off many irrelevant candidate attributes. Unicorn [5] builds a weighted graph to model the dynamic preference of users and choose actions from the candidate action space. Despite effectiveness, previous works typically addressed the query action and the recommend action in isolation, without considering their inherent connection. It is crucial to address them jointly for a comprehensive solution. 
problem definition: 
 In this section, we formulate the problem of multi-turn conversational recommendation (MCR), which aims to recommend the target item to users by asking for attributes and recommending items in the limited turns of the conversation.Specifically, we define the user set as U, the item set as V, the attribute set as P, and the set of timestamps as T = {T 𝑦𝑒𝑎𝑟 , T 𝑚𝑜𝑛𝑡ℎ , T 𝑑𝑎𝑦 , T ℎ𝑜𝑢𝑟 , T 𝑤𝑒𝑒𝑘 }. Besides, each item 𝑣 ∈ V is associated with a set of attributes P 𝑣 ⊆ P. And each user 𝑢 ∈ U has a list of items that interacted at different timestamps, denoted by 𝐼 𝑢 = {(𝑣 ]. At the beginning of each conversation, the user is initialized with a target item 𝑣 𝑡𝑎𝑟𝑔𝑒𝑡 , an attribute belonging to the target item 𝑝 0 ∈ P 𝑣 𝑡𝑎𝑟𝑔𝑒𝑡 and the current timestamp 𝑡 𝑐𝑢𝑟 . Then, the CRS can ask the user preference on an attribute selected from the candidate attribute set P 𝑐𝑎𝑛𝑑 or recommend a certain number of items from the candidate item set V 𝑐𝑎𝑛𝑑 . Based on the user's feedback, MCR will update the candidate attribute set P 𝑐𝑎𝑛𝑑 and the candidate item set V 𝑐𝑎𝑛𝑑 . The conversation will continue until the CRS hits the target item 𝑣 𝑡𝑎𝑟𝑔𝑒𝑡 or reaches the maximum number of turns Q. 
framework: 
 Each conversational turn of TUT4CRS can be divided into four steps: state modeling, action, transition, and reward.At first, we define the state 𝑠 𝑞 , which contains all the conversational information of the prior 𝑞-1 turns:𝑠 𝑞 = [𝐼 𝑞 𝑢 ,𝑇 𝑐𝑢𝑟 ],where 𝐼 𝑞 𝑢 = [𝑢, P 𝑞 𝑎𝑐𝑐 , P 𝑞 𝑟𝑒 𝑗 , V 𝑞 𝑟𝑒 𝑗 , P 𝑞 𝑐𝑎𝑛𝑑 , V 𝑞 𝑐𝑎𝑛𝑑 ] records the user u's di- alogue,According to the current state 𝑠 𝑞 , the agent takes action 𝑎 𝑞 ∈ A 𝑞 , where 𝑎 𝑞 can be selected to ask an attribute 𝑝 𝑞 𝑎𝑠𝑘 ∈ P 𝑞 𝑐𝑎𝑛𝑑 or take a recommendation with the items set V 𝑞 𝑟𝑒𝑐 ⊆ V 𝑞 𝑐𝑎𝑛𝑑 . Inspired by [5] ,EQUATIONwhere V P 𝑞 𝑎𝑐𝑐 denotes the items that satisfy all the accepted at-tributes P 𝑞 𝑎𝑐𝑐 , V P 𝑞 𝑟𝑒 𝑗denotes the items that satisfy one of the rejected attributes P 𝑞 𝑟𝑒 𝑗 , P V 𝑞 𝑐𝑎𝑛𝑑 denotes the attributes which belong to the candidate items. The details of how the scores are calculated for sorting will be described in section 5.3.After the agent has selected an action, the state is updated to the next state 𝑠 𝑞+1 based on the user's feedback. Specifically, when the agent asks for attribute 𝑝 𝑞 𝑎𝑠𝑘 and the user accepts it, the accepted attribute set is updated asP 𝑞+1 𝑎𝑐𝑐 = P 𝑞 𝑎𝑐𝑐 ∪ 𝑝 𝑞 𝑎𝑠𝑘 .Conversely, if the user rejects the attribute, the rejected attribute set is updated asP 𝑞+1 𝑟𝑒 𝑗 = P 𝑞 𝑟𝑒 𝑗 ∪ 𝑝 𝑞 𝑎𝑠𝑘 .When the agent decides to recommend the items V 𝑞 𝑟𝑒𝑐 , if the user rejects all the items, the state is updated asV 𝑞+1 𝑟𝑒 𝑗 = V 𝑞 𝑟𝑒 𝑗 ∪ V 𝑞 𝑟𝑒𝑐 .If the user accepts any of the recommended items, the conversation is successful.Following previous CRS study [5, 18] , have devised five types of rewards: (1) 𝑟 𝑟𝑒𝑐_𝑠𝑢𝑐 : Give strongly positive rewards when the user accept recommended items; (2) 𝑟 𝑟𝑒𝑐_𝑓 𝑎𝑖𝑙 : Give negative rewards when the user rejects the recommended items; (3) 𝑟 𝑎𝑠𝑘_𝑠𝑢𝑐 : Give slightly positive rewards when the user accepts the asked attribute; (4) 𝑟 𝑎𝑠𝑘_𝑓 𝑎𝑖𝑙 : Give negative rewards when the user rejects the asked attribute; (5) 𝑟 𝑞𝑢𝑖𝑡 : Give strongly negative rewards when the maximum number of turns is reached.  
tut4crs policy learning: 
 The overview of our framework is shown in Figure 2 , and we describe this framework in detail in this section.Existing methods [9, 33] model user static preferences from user order history and use this as external information, with unintentional filtering of occasional behavior. However, there is also information implicit in these occasional behaviors, i.e., users' consumption habits at different times, which plays a great role in our subsequent query selection and modeling of user dynamic preference. However, a small amount of occasional behaviors would be filtered out as noises. To solve this problem and obtain information from occasional behaviors, we use the timestamp as a bridge to construct a triplet relationship (𝑢, 𝑡, 𝑣). Unlike the binary relationship (𝑢, 𝑣), the more complex triple relationship can improve robustness and make each interaction unique, allowing for equal learning of information from them. Based on the triple relationship, we construct a global time interaction graph. With the use of time information as a bridge between the user and the item, we can not only model the user static preferences, but also learn the user's consumption habits at different times, and the item can also gather information about its implied temporal features.We define the global graph structure as 𝐺 𝑔𝑙𝑜𝑏𝑎𝑙 = {𝑁 𝑔𝑙𝑜𝑏𝑎𝑙 , E 𝑔𝑙𝑜𝑏𝑎𝑙 }, where 𝑁 𝑔𝑙𝑜𝑏𝑎𝑙 = U ∪ V ∪ P ∪ T and E = E 𝑢𝑡 ∪ E 𝑡 𝑣 ∪ E 𝑣𝑝 , i.e., there are edges between user nodes and time nodes, time nodes and item nodes, item nodes and attribute nodes. A time node 𝑡 𝑖 ∈ T represents a specific timestamp, and its embedding is obtained by fusing the embeddings of its different timestamp dimensions as:EQUATIONwhere each𝑡 𝑑𝑖𝑚 𝑖 ∈ 𝑡 𝑖 is the one-hot encode embedding, 𝑑𝑖𝑚 ∈ {𝑦, 𝑚, 𝑑, ℎ, 𝑤𝑒𝑒𝑘 },𝑊 0 ∈ R (5 * 𝑑 𝑡𝑖𝑚𝑒 ) ×𝑑 ℎ𝑖𝑑 ,𝑊 𝑦 ∈ R | T 𝑦𝑒𝑎𝑟 | ×𝑑 𝑡𝑖𝑚𝑒 ,𝑊 𝑚 ∈ R | T 𝑚𝑜𝑛𝑡ℎ | ×𝑑 𝑡𝑖𝑚𝑒 , 𝑊 𝑑 ∈ R | T 𝑑𝑎𝑦 | ×𝑑 𝑡𝑖𝑚𝑒 , 𝑊 ℎ ∈ R | T ℎ𝑜𝑢𝑟 | ×𝑑 𝑡𝑖𝑚𝑒 and 𝑊 𝑤𝑒𝑒𝑘 ∈ R | T 𝑤𝑒𝑒𝑘 | ×𝑑 𝑡𝑖𝑚𝑒are trainable parameters. Thus, different time nodes 𝑡 𝑖 can have different representations.Training. We employ a 𝐿 𝑔𝑙𝑜𝑏𝑎𝑙 -layer GCN [10] [11] [12] 15] to extract the external preferences of users. The initial input embedding of the first layer are 𝑒 𝑡 denote the output embedding of nodes after the propagation of 𝑙-th layer. For each layer, GCN aggregates [13, 28] neighborhood embedding for each node 𝑛 𝑖 and combines it with the last layer embedding of the node 𝑛 𝑖 :EQUATIONThe final embedding of node 𝑛 𝑖 is as follows:EQUATION𝑖 , ..., 𝑒EQUATIONThe similarity between the two representations can be determined through the vector dot product, denoted as 𝑓 (𝑒 𝑎 , 𝑒 𝑏 ) = 𝑒 𝑇 𝑎 𝑒 𝑏 . Additionally, the similarity score between an instance 𝑖 and a set of instances 𝑆 can be calculated as 𝑔(𝑖, 𝑆) = tanh( 𝑗 ∈𝑆 𝑓 (𝑒 𝑖 , 𝑒 𝑗 )). To get more external information from the global graph 𝐺 𝑔𝑙𝑜𝑏𝑎𝑙 , inspired by [9] , we use multi-tasks to train the representation offline and obtain external information from different perspectives.Attribute-Item Classification Task. Through this task, we can strengthen the correlation between items and their attributes. For each attribute 𝑝 𝑖 , we classify the items that contain the attribute 𝑝 𝑖 belonging to the positive class 𝑣 𝑖 and the items that do not belong to the negative class 𝑣 ′ 𝑖 . Then we strengthen the relation between by using the cross-entropy loss function:EQUATIONwhere the training dataD 1 = {(𝑝 𝑖 , 𝑣 𝑖 , 𝑣 ′ 𝑖 )|𝑣 𝑖 ∈ V 𝑝 𝑖 , 𝑣 ′ 𝑖 ∈ V/V 𝑝 𝑖 }.The attribute 𝑝 𝑖 is a sample from P, 𝑉 𝑝 𝑖 denotes the positive item set contains the attribute 𝑝 𝑖 and V/𝑉 𝑝 𝑖 denotes the negative item set doesn't contains the attribute.User-Time-Item Prediction Task. For users' consumption habits at different times, we obtain coarse-grained preference information by predicting users' target items under different times, while the items learn their hidden temporal features. Specifically, for each user, his interaction occurs at time 𝑡 = {𝑡 𝑦 , 𝑡 𝑚 , 𝑡 𝑑 , 𝑡 ℎ , 𝑡 𝑤𝑒𝑒𝑘 }, and we incorporate the time information of the user:EQUATIONwhere 𝑊 𝑓 𝑢𝑠𝑖𝑜𝑛 ∈ R 2𝑑 ℎ𝑖𝑑 ×𝑑Inspired by [9, 16] , we adopt two types of BPR [23] loss with two types of negative examples. The loss function is defined as:EQUATIONwhere 𝐷 2 = {(𝑢, 𝑡, 𝑣, 𝑣 -)|𝑣 ′ ∈ V/V 𝑢 }, the item v is the target item and the 𝑣 -is sampled from the set of non-interacted items of user u, and V 𝑢 is the set of items historically interacted by user. Also 𝐷 3 = {(𝑢, 𝑡, 𝑣, 𝑣 -)|𝑣 ′ ∈ V 𝑞 𝑐𝑎𝑛𝑑 /V 𝑢 },we also employ BPR loss as:EQUATIONwhere D 4 = {(𝑢, 𝑡, 𝑝, 𝑝 -)|𝑝 ∈ P 𝑣 , 𝑝 -∈ P/P 𝑣 } and P 𝑣 denotes the attribute set belong to the target item v.Finally, the multi-task training objective is:EQUATION)where ||Θ|| 2 is the regularizer term to avoid overfitting, and 𝜆 is the regularization parameter. Through the offline training, we can obtain rich external preferences that include user static preferences, user consumption habits, and hidden temporal features of items.With the approach in the previous section, we captured the rich external preference of the user, which can be of great help to us in the selection of queries and the modeling the user dynamic preferences. To select useful information for the current state from the user external preferences to help us with query selection and user dynamic preference modeling, we construct EQUATIONThe edge (𝑢, 𝑡) ∈ E 𝑢𝑡 denotes the conversation happening in 𝑡 ∈ 𝑡 𝑐𝑢𝑟 ; (𝑢, 𝑝) ∈ E 𝑢𝑝 𝑎𝑐𝑐 denotes the user likes attribute p; (𝑢, 𝑣) ∈ E 𝑢𝑣 𝑐𝑎𝑛𝑑 denotes the user may like the item, and (𝑣, 𝑝) ∈ E 𝑣 𝑐𝑎𝑛𝑑 𝑝 𝑐𝑎𝑛𝑑 denotes the item contains attribute p.To dynamically sense the user's preference, i.e., what exactly the user is interested in, we use GAT [27] to dynamically compute the user's attention during message propagation by:EQUATION)The input representations of user nodes, item nodes, and attribute nodes are obtained through Equation ( 4), and the representations of time nodes are calculated by the following:EQUATIONEQUATIONEQUATIONEQUATION𝑤𝑒𝑒𝑘 = 𝑊 𝑇 𝑙𝑡 (𝑡 𝑤𝑒𝑒𝑘 𝑐𝑢𝑟 𝑊 𝑤𝑒𝑒𝑘 ), where 𝑊 𝑦 ,𝑊 𝑚 ,𝑊 𝑑 ,𝑊 ℎ , and 𝑊 𝑤𝑒𝑒𝑘 are pretrained by Equation ( 2) and 𝑊 𝑙𝑡 ∈ R 𝑑 𝑡𝑖𝑚𝑒 ×𝑑 ℎ𝑖𝑑 are trainable parameters.After applying the 𝐿 𝑙𝑜𝑐𝑎𝑙 -layer GAT, we obtain the final representation 𝑧 𝑖 of a node 𝑛 𝑖 ∈ N 𝑙𝑜𝑐𝑎𝑙 by using the output 𝑧 (𝐿 𝑙𝑜𝑐𝑎𝑙 ) 𝑖 of the last layer. The user u utilizes personalized learning to incorporate the current conversation and time information. This enables the user to gather the most relevant and useful information for their current status from external sources. Following [5] , we employ a Transformer encoder [26] to capture the sequential information of the conversation history: X * = TransformerEncoder(X), where X ∈ R 𝑙 ×𝑑 ℎ𝑖𝑑 denotes the embedding of input, and 𝑙 the sequence length. The input sequence X includes the user u and the accepted attributes 𝑃 𝑞 𝑎𝑐𝑐 at turn 𝑞 with the embedding 𝑧 𝑖 learned by GAT layers. Finally, we get the state representation 𝑠 𝑞 by a mean pooling layer: 𝑠 𝑞 = MeanPool(X * ).In order to develop a strategy for decision-making, we employ reinforcement learning [29] , which allows us to learn optimal actions. Following [5] , we define our action space A 𝑞 as the top-𝐾 𝑣 items and top-𝐾 𝑝 attributes. To determine the order of the action space at turn 𝑞, we calculate the 𝑠𝑐 score by:EQUATIONwhere we use user-rejected items as the punishing factor to keep the action in the action space as far away as possible from those items with negative user feedback. After obtaining the state representation 𝑠 𝑞 and action space A 𝑞 , we use Dueling Q-Network [30] to decide the next action, which contains two deep neural networks learning the value function 𝑓 𝑉 (𝑠) and the advantage function 𝑓 𝐴 (𝑠, 𝑎) respectively. The Q-function can be calculated by:EQUATION)Under the standard assumptions of Bellman's equation, the discount factor for delayed rewards is taken to be 𝛾, and the optimisation objective is to find out the action that achieves the maximum reward by learning the strategy 𝜋 * :EQUATION)We sample mini-batch experiences from the replay buffer D and define the loss function as follows:EQUATION)6 Experiments 
datasets: 
 We utilize two widely used recommendation datasets for the experimental design. The statistics of them are presented in Table 1 . The Yelp dataset is sourced from the 2018 iteration of the Yelp Business Recommendation Challenge. [16] established a hierarchical catalog for Yelp attributes, comprising 29 coarse-grained attributes at the first level and 590 attributes at the second level. We specifically utilize the second level of attributes for items in our study. The MovieLens-20M dataset serves as a prevalent benchmark dataset for recommendation systems. We filter the user-item interactions to include only ratings greater than 3 and subsequently employ K-means clustering to categorize attribute types. To achieve this, we have developed a user simulator that is based on [5, 18] . To simulate a conversation, we consider each observed user-times-item interaction triplet (𝑢, 𝑡, 𝑣). In this context, the item v serves as the ground-truth, and we define its attribute set 𝑃 𝑣 as the accepted attribute by the user when asked. The simulated conversation begins with the simulated user specifying a certain attribute 𝑝 0 randomly selected from 𝑃 𝑣 . During the MCR process, when it comes to selecting the recommendation action, we examine whether the top-𝑘 𝑣 items include the target item v. If this is the case, we consider the recommendation accepted by the user and deem the conversation successful. However, if the target item is not included, we continue the conversation until either success is achieved or the maximum number of turns is reached. By utilizing this user simulator, we can enhance the MCR's ability to learn and improve its performance.6.2.2 Parameter Settings. We sort the dataset by time and then split the dataset by 7:1.5:1.5 for training, validation, and testing. And we set the size 𝐾 𝑝 of the attribute list and the size 𝐾 𝑣 of the recommendation list as 10, the maximum turn Q as 15. We train the global graph described in Section 5 with the training set for 2500 epochs, the learning rate is 1e-4, the embedding size of attributes and items is set to 64, the embedding size for different timestamp dimensions are set to 16 respectively. And the number of GCN layers 𝐿 𝑔𝑙𝑜𝑏𝑎𝑙 is set to 2.We utilize the user simulator to interact with the CRS. This allows us to train our framework effectively using data from the validation set. The settings of the rewards are the same as previous works [5, 33] : 𝑟 𝑟𝑒𝑐_𝑠𝑢𝑐 = 1, 𝑟 𝑟𝑒𝑐_𝑓 𝑎𝑖𝑙 = -0.1, 𝑟 𝑎𝑠𝑘_𝑠𝑢𝑐 = 0.01, 𝑟 𝑎𝑠𝑘_𝑓 𝑎𝑖𝑙 = -0.1, 𝑟 𝑞𝑢𝑖𝑡 = -0.3. The numbers of GAT layers 𝐿 𝑙𝑜𝑐𝑎𝑙 for YELP and MovieLens dataset are set to be 4 and 1, respectively. During the training procedure of Dueling Q-Network, the size of the experience replay buffer is 50,000, and the size of the mini-batch is 128. The learning rate and the discount factor 𝛾 are set to be 1e-4 and 0.999.To evaluate the performance of the framework TUT4CRS, we choose four categories of compared models, namely: (1) Rule-based methods, i.e., Abs Greedy [3] and Max Entropy [6] , which only recommends items to user and selects query based on the maximum entropy; (2) RL-based SCR (Single-turn Conversational Recommendation) method, i.e., CRM [25] . A reinforcement learning-based method in SCR scenario, recording the user preferences into a belief tracker; (3) RL-based MCR method, i.e., EAR [16] , SCPR [18] , UNICORN [5] . These methods use reinforcement learning to learn the policy in the MCR setting. In addition, EAR proposes a three-stage solution to enhance the interaction between the conversational component and the recommendation component, SCPR models MCR as an interactive path reasoning problem and UNICORN unifies the two decision strategies via graph neural networks. For a fair comparison, we use FM [22] and TransE [1] to generate initial representations, which we name UNICORN-FM and UNICORN-TransE respectively. Besides, we adjust SCPR (referred to as T*-SCPR) by replacing the FM with the representation learned by Global Graph and concatenate the time representation with the user representation; (4) RL-based MIMCR (Multi-Interest Multi-round Conversational Recommendation) method, i.e., MCMIPL [33] and HutCRS [21] . MCMIPL proposes the new conversational recommendation scenario MIMCR, where users may have multiple interests. HutCRS portrays the conversation as a hierarchical interest tree consisting of two stages based on MIMCR.Following [5] , we utilize success rate (SR@Q) to measure the cumulative ratio of successful recommendation with the maximum turn Q, and average turn (AT) to evaluate the average number of turns. SR@15 can measure the overall performance of our framework, while SR@10 can represent the successful performance of our framework in fewer turns. Besides, hDCG@(Q, K) is used to additionally evaluate the ranking performance of recommendations. For SR@Q and hDCG@(Q, K), higher values represent better performance, while the opposite is true for AT, where lower values represent better.Based on the comprehensive performance analysis presented in Table 2 , our TUT4CRS framework demonstrates remarkable effectiveness across both datasets, showcasing superior results in terms of four metrics compared to baselines. Particularly noteworthy are the significant enhancements observed in SR@10 and AT metrics. These improvements can be attributed to several factors: (1) The global graph construction method incorporates a diverse array of external information, encompassing user static preferences, temporal consumption patterns, and item-specific temporal features. These factors collectively contribute to informed query selection and precise modeling of user dynamic preferences. (2) Integration of time information into the modeling of user dynamic preferences, coupled with dynamic learning through attention mechanisms, enables the selection of pertinent information based on current context from previously acquired knowledge. This enables efficient query selection and accurate modeling of user preferences. Moreover, the utilization of external information proves advantageous in modeling user dynamic preferences. T*-SCPR exhibits notable performance improvements over the original method (i.e., SCPR), particularly evident in SR@10 and AT metrics on the Movie-Lens dataset. Besides, the choice of initial embedding significantly influences user preference modeling. In the case of UNICORN, we explore two distinct embedding initialization approaches: FM and TransE. Our results reveal that UNICORN-TransE surpasses UNICORN-FM across multiple performance metrics. This underscores the pivotal role of initial embedding quality in harnessing external information effectively, suggesting that incorporating a broader array of diverse external information can yield further enhancements in results. Furthermore, while MCMIPL and HutCRS demonstrate strong performance on the YELP dataset, their performance on the Movie-Lens dataset is notably poorer. This variance can be ascribed to the MIMCR scenario inherent in these models, where tasks involve classifying attributes into two distinct levels. However, it's worth noting that while YELP dataset benefits from expert categorization, the MovieLens dataset relies on a clustering algorithm, leading to inferior results. This underscores the significance of expert knowledge in attaining superior performance in the MIMCR scenario, which may not always be readily available in practical applications. In order to verify the effectiveness of some key designs, we conduct a series of ablation experiments on the YELP and MovieLens datasets. The results are shown in Table 3 .6.4.1 External Information. In the section 5, we describe the process of constructing a global graph to extract external information from user order history. This information includes user static preference, user consumption habits, and item temporal features. To evaluate the effectiveness of our global time interaction graph mining information, we replace it with other methods, i.e., Random and FM [22] . Interestingly, we observe a significant drop in the performance of the model when these alternative methods are used instead. This finding highlights the importance and effectiveness of our approach in constructing a global graph.Components. Here, we remove GAT, transformer encoder, and timestamp from the local graph respectively. We observe that: (1) When replacing the GAT, we notice a decline in the performance of TUT4CRS. We believe that this is due to the significance of higher-order relationships in effectively modeling user preferences for CRS. (2) Similarly, when the transformer encoder is replaced, TUT4CRS experiences a degradation in performance. This can be attributed to the transformer's strong ability to summarize implicit information within a sequence. (3) The impact of removing timestamps is more pronounced in the YELP dataset compared to the MovieLens dataset. This suggests that the external information we have learned is particularly beneficial for smaller datasets. However, for larger datasets, it is crucial to carefully select the external information that is most relevant to our current state.The Role of Timestamp. To verify the validity of time information, we remove the timestamp entirely from our framework. We observe a decrease in performance on both datasets, underscoring the crucial role of integrating time information into our framework. By stacking more layers, collaborative information from multihop neighbors would be distilled. We investigate how the number of local time weight graph layer 𝐿 𝑙𝑜𝑐𝑎𝑙 influences the performance of TUT4CRS. The range of 𝐿 𝑙𝑜𝑐𝑎𝑙 tested here is {1, 2, 3, 4}, and the results are shown in Figure 3 .For YELP dataset, we observe that the performance of the model improves further as the number of layers increases. This would be attributed to the larger size of the local graph, which allows for more effective information to be obtained with deeper layers. For MovieLens dataset, a single layer achieves the best performance, and deeper layers lead to a decrease in performance. It may be because the size of the local graph is small, and deep layers may introduce over-smoothing problems.We conduct a case study on the YELP dataset, focusing on the UNICORN and TUT4CRS models to analyze their performance. Figure 4a shows the detailed MCR process for both models, while Figure 4b displays the attention scores of the user during the first three turns of the TUT4CRS model across different time dimensions.In this case study, the user initiates a conversation on February 14, 2018, seeking dining options, with the target item being "Choolaah BBQ". In conversation (a), the user's ordering history indicates (a) Detailed MCR process for TUT4CRS and UNICORN. The "price level" refers to the price range of the item, which will be different according to different datasets; "star level" is similar, which refers to the overall rating of the item. a preference for pizza places or coffee shops. Consequently, the agent initially queries attributes such as "price level 2", "Nightlife", and "Coffee & Tea". However, these queries are rejected as they are unrelated to the target item. In conversation (b), the user's attention scores for the year, month, and day are higher than the week during turns 1 and 2. Given that the user consumed steak on February 14th of the previous year, the agent infers the user's consumption habits at specific times. Consequently, the agent queries attributes such as "star 4" and "price level 4". In turn 3, the focus shifts to the "day", and the attribute "restaurants" is selected for querying. Ultimately, the recommendation provided by the TUT4CRS model proves successful within ten turns. 
conclusion: 
 This work presents a novel framework called Time-aware Userpreference Tracking for Conversational Recommendation Systems (TUT4CRS). The framework addresses both quantitative and qualitative challenges in CRS by incorporating the time factor as a crucial element. TUT4CRS utilizes a global time interaction graph to capture valuable external information from the user's historical interactions. This information is then used to construct a local time-aware weight graph and personalize the use of external information based on the user current state, which helps the agent to select relevant queries. In this way, the user's dynamic preferences can be learned efficiently and accurately. The effectiveness of TUT4CRS is validated through extensive experiments conducted on two datasets. 
 
 Appendix:  


Decision: This paper introduces the Time-aware User-preference Tracking for Conversational Recommendation Systems (TUT4CRS) framework, which incorporates time dynamics into CRS to enhance user interaction models. The use of a global time interaction graph and a local time-aware weight graph is commendable and validated through extensive experiments on two datasets. Despite criticisms regarding the limited novelty beyond time embeddings and the complexity of its implementation, the approach marks a step forward in the field. Concerns about consistency in model naming and notation complexity should be addressed to improve clarity. 
Anyway, the strengths and potential impact of TUT4CRS justify its acceptance.

Reviews: review: paper_summary: paper_summary: The article introduces TUT4CRS (Time-aware User-preference Tracking for Conversational Recommendation System), a framework aimed at enhancing Conversational Recommendation Systems (CRS) by leveraging temporal dynamics to model user preferences. The authors propose a method that integrates a global time interaction graph and a local time-aware weight graph to address the limitations of conversational recommendations, specifically the issues of limited conversation turns and low query compliance. Through extensive experiments on two real-world datasets, the framework demonstrates significant improvements in recommendation performance and efficiency.
strength_weakness: strength_weakness: 1. Innovative Use of Temporal Dynamics: The incorporation of time as a critical factor in modeling user preferences is a novel approach that bridges the gap between conversation quantity and quality. This adds a valuable dimension to understanding and predicting user behavior.

2. Comprehensive Framework: The dual-layer approach of using both global and local graphs to capture user preferences provides a robust mechanism for handling both long-term and short-term user interests, which enhances the adaptability and accuracy of the recommendations.

3. Detailed Problem Analysis: The paper offers a thorough analysis of the challenges in CRS, particularly the issues arising from limited conversation turns and query compliance. This depth of analysis provides a strong foundation for the proposed solution.
limitations: limitations: 1. Figure 3 will appear blurry when enlarged.
2. Are you sure that the two commonly used datasets for recommendation tasks can adequately support conversational recommendation tasks?
paper_summary: paper_summary: The authors focus on the challenges in Conversational Recommendation Systems. To better leverage the temporal information, they propose a Time-aware User-preference Tracking method to model the dynamic preference. The framework constructs a global time interaction graph to incorporate rich external information for offline training. During conversational recommendation, 
a local time-aware weight graph is constructed to model user dynamic preferences. They conduct experiments on two real-world datasets with addtional analysis.
strength_weakness: strength_weakness: 1.The introduction of timestamp information into CRS is a reasonable idea.
2.The paper is well-organized and easy to follow.
limitations: limitations: 1.The novelty of the proposed method appears limited. The primary difference from existing methods, such as UNICON, is the incorporation of time embeddings. The other parts such as the graph model in offline training, graph with transformer in CRS and RL are the same. Other components, such as the graph model in offline training, the graph with transformer in CRS, and reinforcement learning , are similar.While this is a useful enhancement, it may not represent a significant departure from existing approaches.
2.It would be beneficial to see how TUT4CRS performs against other baselines that also utilize offline training with timestamp information.
3.There is inconsistency in the model names used throughout the paper, with different names such as TUT4CRS, HUT4CRS, and TAT4CRS
paper_summary: paper_summary: The paper presents a novel framework for Conversational Recommendation Systems (CRS) called Time-aware User-preference Tracking for Conversational Recommendation System (TUT4CRS). Traditional CRS face challenges in capturing dynamic user preferences due to limitations in conversation length (quantity) and relevance of queries (quality). TUT4CRS addresses these interconnected issues by leveraging temporal dynamics. It constructs a global time interaction graph to incorporate rich external information and a local time-aware weight graph to dynamically select queries and model user preferences effectively.
strength_weakness: strength_weakness: The paper tackles a compelling and complex problem of accurately modeling dynamic user preferences in conversational recommendation systems.
limitations: limitations: - In problem definition, a set of timestamps is defined as $\mathcal{T} = \mathcal{T}_{𝑦𝑒𝑎𝑟}, \mathcal{T}_{𝑚𝑜𝑛𝑡ℎ},\mathcal{T}_{𝑑𝑎𝑦},\mathcal{T}_{ℎ𝑜𝑢𝑟}, \mathcal{T}_{𝑤𝑒𝑒𝑘}$. Why do we need $\mathcal{T}_{𝑤𝑒𝑒𝑘}}$? Isn’t sufficient to just use  $\mathcal{T} = {\mathcal{T}_{𝑦𝑒𝑎𝑟}, \mathcal{A}_{𝑚𝑜𝑛𝑡ℎ}, \mathcal{A}_{𝑑𝑎𝑦}, \mathcal{T}_{ℎ𝑜𝑢𝑟}$? Also, authors stated that at the beginning of each conversation, the user is initialized with a target item $𝑣_{𝑡𝑎𝑟𝑔𝑒𝑡}$. How is it defined? It’s unclear at this stage.
- The proposed TUT4CRS framework is quite complex, involving multiple components like the global time interaction graph and local time-aware weight graph. This complexity could pose challenges in implementation and require significant computational resources.
- The notation used in the paper is overly complex and extensive, which significantly impacts its readability. The excessive use of symbols, multiple layers of subscripted and superscripted variables, and intricate mathematical expressions make it challenging for readers to follow the core ideas and methodologies. This level of detail, while aiming for precision, can obscure the main contributions and makes comprehension difficult. Simplifying the notation and providing summary tables or diagrams could greatly enhance clarity, making the paper more accessible to a broader audience, including those less familiar with the specific technical nuances.
- The experiments are conducted on only two datasets (YELP and MovieLens). Additional datasets from different domains would help generalize the findings and validate the robustness of the approach.
paper_summary: paper_summary: This paper presents a novel framework Time-aware User-preference Tracking for Conversational Recommendation Systems (TUT4CRS). The framework addresses both quantitative and qualitative challenges in CRS by incorporating the time factor. The authors utilize a global time interaction graph to capture external information from the user’s historical interactions, which is used to construct the local time-aware weight graph. This graph then helps to select relevant queries based on the users’ current state. Finally, the effectiveness of TUT4CRS is validated via extensive experiments.
strength_weakness: strength_weakness: This paper makes a good contribution to the CRS which uses the simple yet effective temporal factor to handle the quantity and quality problems in CRS. The proposed framework shows great potential in improving a more precise depiction of user dynamic preferences in a shorter amount of time, and therefore improving the performance of CRS.
limitations: limitations: I do not see the obvious weakness of the paper. I am just curious what is the computation complexity of the current framework (i.e., the training and testing time) and how the proposed method scales with an increasing number of users and interactions.