HutCRS: Hierarchical User-Interest Tracking for Conversational Recommender System
Authors: Mingjie Qian
Yongsen Zheng
Jinghui Qin
Liang Lin
Venue: EMNLP 2023 Main (2023)
Keywords: Conversational Recommender Rystem
Multi-round Conversations
Hierarchical Interest Tree
Graph Neural Network
TL;DR: We propose a more realistic, user-friendly, and explainable framework HutCRS, which portrays the conversation as a hierarchical interest tree consisting of two stages.

Abstract: Conversational Recommender System (CRS) aims to explicitly acquire user preferences towards items and attributes through natural language conversations. However, existing CRS methods ask users to provide explicit answers (yes/no) for each attribute they require, regardless of users' knowledge or interest, which may significantly reduce the user experience and semantic consistency. Furthermore, these methods assume that users like all attributes of the target item and dislike those unrelated to it, which can introduce bias in attribute-level feedback and impede the system's ability to accurately identify the target item. To address these issues, we propose a more realistic, user-friendly, and explainable CRS framework called Hierarchical User-Interest Tracking for Conversational Recommender System (HutCRS). HutCRS portrays the conversation as a hierarchical interest tree that consists of two stages. In stage I, the system identifies the aspects that the user prefers while the system asks about attributes related to these positive aspects or recommends items in stage II. In addition, we develop a Hierarchical-Interest Policy Learning (HIPL) module to integrate the decision-making process of which aspects to ask and when to ask about attributes or recommend items. Moreover, we classify the attribute-level feedback results to further enhance the system’s ability to capture special information, such as attribute instances that are accepted by users but not presented in their historical interactive data. Extensive experiments on four benchmark datasets demonstrate the superiority of our method. The implementation of HutCRS is publicly available at https://github.com/xinle1129/HutCRS.

HutCRS:
Hierarchical User-Interest Tracking for Conversational Recommender System
Mingjie Qian', Yongsen Zheng", Jiinhhui Qin² and Liang Lin't
'Sun Yat-sen University, *Guangdong University of Technology (z-yongsensmile, scape 1989) ) gmail.com
qianmj7 @ mail2.sysuu.ddu.cm linliang @ ieee.org
Abstract
cent years(Lei al.. 2020a), utilizing the user's online feedback conduct dynamic and explain- able recommendations through interactive conver- sations with the user. One of the challenges for CRS is how to efficiently acquire user preferences and quickly narrow down the recommendation can- didates. In this regard, attribute-based CRS has been extensively studied, whether attribute can significantly reduce the recommenda- tion candidates(Gao et al., 2021).
Conversational Recommender System (CRS) aims to explicitly acquire user preferences to- wards items and attributes through natural lan- guage conversations. However, existing CRS methods ask users provide explicit answers (yes/no) for each attribute they require, regard- less of users' knowledge interest, which may significantly reduce the user experience and semantic consistency. Furthermore, these methods assume that users like all attributes of the target item and dislike those unrelated it, which can introduce bias in attribute- level feedback and impede the system's is ability to accurately identify the target item. To ad- dress these issues, we propose a more realis- tic, user-friennte user-friendly, and explainable CRS frame- work called Hicrarchical User-Interes: Track- ing for Conversational Recommender System (HutCRS). HutCRS portrays the conversation hierarchical interest tree that consists of two stages. In stage the system identifies the aspects that the user prefers while the system asks about attributes related these positive aspects recommends items in stage II. In addition, we develop a Hierarchical- Interest Policy Learning (HIPI.) module integrate the decision- backsign-making process of which aspects to ask and when to about attributes or rec- ommend items. Moreover, attribute-level feedback results to further en- hance the system's ability to capture special information, such as attribute instances that are accepted by users but not presented in their historical interactive data. Extensive experi- ments on four benchmark datasets demonstrate the superiority our method. The imple- mentation of HutCRS is publicly available https///gihub.com//inl/ 129/HuiCRS.
likes
Among the different proposed problem settings for attribute-based CRS, the single- round Chunissstttkk et al., 2018; Sun and Zhang, 2018) setting is the earliest to be intro- duced. However, this setting is impractical in real-world deployments because the CRS only makes recommendations once, ending the session regardless of the results. To address this issue, the multi-round(Lei al., 2020a,b; Xu et al., 2021; Ren et al., 2021; Deng et al., 2021; Tu et al., 2022; Hu et al., 2022: Zhang et al., 2022) setting has been proposed, wherein the CRS interacts with the user by asking attributes and recommending items multiple times until the task succeeds or the user terminates the session. Multi-round Conversational Recommendation(MCCR aims to make successful recommendations with fewer conversation turns.
classify the
Although MCR is widely acknowledged the most realistic setting for CRS, the existing MCR assumptions regarding user interest deviate from real-world scenarios. First, MCR presupposes that users have definitive answers(yes or no) for all queried attributes. However, users might be un- aware of or indifferent to certain attributes, and compelling them to respond could reduce the user experience and generate inconsistent conversation utterances. Additionally, MCR assumes that users will accept all attributes that belong to the target item and reject those unrelated to it. In practice, users might not necessarily like attributes the target item, nor dislike attributes that are not
1 Introduction
Conversational recommender systems (CRSs) have become one of the trending research topics in re-
"Both authors contributed equally this research. *Corresponding author.

---

Sausuung
Tor I'm looking for Samsing nocile phona.
Whish sepeet do you cae alout? (A) Butty (By Color (C (DISCC (Exclers
B. and
Which oftte fillowing
Bllack Srepdingon fier jerhers
Srupdiigm
Wakd the follving like? White (B) (Cjothers
Galley
and
: Lanneittad airinaz: spcca stribac
(Negrtie aepect
mauraay Bise this Samoung fixioy 525 Ulra
Porliie Negative hee slked sorixue
Yest Thonk you!
Eppada postie raguhs atriAa
Targe aoe
Figure 1: An illustration of conversation in HutCSS. The hierarchical interest tree (right side) based on the user's feedback enables the system to better understand the needs and preferences of the user.
included in the target item. This discrepancy can result in invalid attribute-level feedback and pre- vent the system from identifying the target item accurately. For instance, considering target item "an esports phone with a large battery capacity but wwwpperformance camera", the user doesn't 't priori- tize the performance of the camera, but it doesn't 't imply that the user prefers a low-performance cam- era. If the system asks about the flo-peefformaee camera", it may receive negative feedback which could cause it miss
of which aspects to ask and when to ask attributes or when to recommend items. In addition, we
classify the attribute-level feedback results further enhance the system's ability to capture special information. Positive attributes that are not present in the historical data and negative attributes that are present in the historical data are marked as "special". Extensive experiments on Yelp, LastFM, Amazon-Boon Book, and MovieLen demonstrate the superiority of HutCRS compared with state-of-the-art baselines.
the target item.
In summary, our contributions can be concluded as follows:
To address the aforementioned issues, we pro- pose novel framework Hierarchical User-Interest Tracking for Conversational Recommender System (HurCRS). As shown in Figure represent the user's preferences in the current session as a hierarchical tree that depicts the user's interests and priorities, facilitating a better understanding of their needs and preferences. Specifically, the system first prompts the user to specify their aspects of interest based on user-initialized attributes. Once this information is obtained by the system, it will either ask about attributes associated with the positive aspects or recommend items. The interaction based on the hierarchical interest tree enables the system to ask about attributes that the user is indeed interested in, thereby improving user experience and consistency in the conversation. Besides, when attribute instances belong to the target item and align with the aspects of interest to the user, they more likely to be accepted. This clear preference will lead to the generation of more authentic feedback.
We extend existing CRS to a more realistic, user- friendly, and explainable setting.
We propose the hierarchical interest tree and uti- lize the classification results of attribute-level feedback to further enhance the system's 's abil- ity to comprehend the needs and preferences users.
Experiments on four benchmarks show the supe- riority of HutCRS compared with state-of-the baselines.
2 Related Works
CRS is a recommendation system that can elicit the dynamic preferences users and take actions based on their current needs through real-time mul- titurn interactions(Gas et al., 2021). In this work, we focus on attribute- bassdd CRS(Sun and Zhang, 2018; 8; Lei et al., 2020a,b; Deng al., 2021; Zhang al., 2022) which ask users attributes to efficiently acquire user preferences and quickly narrow down the recommendation candidates. The problem set- tings of attribute-baser CRS can be mainly divided into two categories: single-round and multi-round.
Inspired by Zhang et al. (2022), we develop a Hierarchical Interest Policy Learning (HIPL)
module to integrate the decision-making process

---

In single-round sennario et al.. 2018; Sun and Zhang, 2018), the CRS only makes recommendations once, ending the session regard- less of the results. In multi-round scenario, the CRS can interacts with the user by asking attributes and recommending items multiple times until the task succeeds or the user terminates the session.
ask about attributes associated with the positive aspects or recommend items until least one ac- ceptable item successfully recommended to the user or the system reaches the maximum number of turn T.
HutCRS
Due to the wide acknowledgement of Multi- round Conversational Recommendation (MCR) as the most realistic setting(Lei et al., 2020a) for CRS, this work is conducted based on the multi- round scenario. EAR(Lei et al., 2020a) extends the single-round setting into multi-round set- ting where the system recommend multiple times. SCPR(Lei et al., 2020b) models the MCR as an interactive path reasoning problem on graph. which able to prune off many irrelevant candi- date attributes. FPAN(Xu et al., 2021) utilizes a gating mechanism to aggregate online feedback from users. UNIcORN(Deng et al., 2021) pro- poses to formulate three decision-making problems in CRS as unified policy learning task based on dynamic weighted graph. MIMCPL.(Zhang al., 2022) extends MCR more realistic scenario setting named MIMCR where the user may ac- cept multiple attribute instances with the same type and multiple items with partially overlapped at- tributes. However, the existing MCR assumptions about user interest still deviate from real-world scenarios. Hence, we propose a novel framework called HutCRS to tackle these issues.
To improve user experience and semantic consis- tency, we propose novel framework Hierarchical User-Interest Tracking for Conversation Recom- mender System (HutCRS) for MCR. Similarly to Zhang et al. (2022); Lei et al. (2020b); Deng et al. (2021), our framework also aims to learn the policy network x (0)]))) to maximize the expected cumu- lative rewards as: π arg max BI ri], where st denotes the current state, de denotes the tion taken by the agent and the ry is the intermediate reward. As shown in Figure 2, our framework por- trays the conversation as a hierarchical interest tree that consists of two stages where each stage com- prises two components: Hierarchical User Interest Tracking and wwwtrchhical-nntteee. Policy Learning, In stage I, the system aims to identify the aspects that the user prefers, while in stage II, the system will either ask about attributes related to these posi- tive aspects or recommend items.
4.1 Hierarchical User Interest Tracking
In order learn the policy network (aalss)) it is necessary to characterize states and actions. To this end, we use the Hierarchical User State Tracking module to track the current state and the Interest- based Candidate Selection module to rank the can- didates.
3 Definition and Preliminary
To better describe and understand the HutCRS, we set up some basic notions in this section. We define the sets of users and items as l/ and V, respectively. Additionally, we separately define the sets of as- pect (i.e. type) and attribute instances as C and P. Each item instance of attribute instances P,,. Each attribute instance p P has its corresponding aspect Cp E C. In each episode, there is a set Vu, of items that are accept- able to the user u E U. The set is represented as Vu = {u1, U2 uN. where N₂ is the number of acceptable items, Po, 0 Pry 0 ... n PuN. P Psomm: # and P. * Py. As shown in Figure 1, session starts with preferred attribute instance Po Psame specified by user u. Then, the agent asks about which aspects the user prefers from the candidate aspect set Cound until obtaining positive feedback. In the remaining turns, will repeatedly
4.1.1 Hierarchical User State Tracking
We utilize hierarchical interest tree to represent the user's preferences in the current session. Un- like the approach presented in Zhu et al. (2018), our hicrarchical interest tree does not necessitate extra time and resources for its construction. As shown in Figure 2, the full tree consists of the user- initialized attribute po as the root node, all aspect instances as the first layer nodes, all attribute in- stances as the second layer nodes, and all items the last layer nodes. As the interaction proceeds, prune the full tree to obtain dynamic trec based on user feedback. This dynamic tree only retains nodes that have been interacted with by the user in the current session, as well as those nodes that belong to the candidates. In addition, to en- hance the system's 's ability to capture special infor-
is associated with a set

---

Current Graph
Current GNN
Action Space
Dueling Q-network
Advantugee
Interest-based
MI
Candidale Selection
Global Graph
Global GNN
multi-
attention
Value
Full Tree
User State
Dynamic Tree
Hierarchical Tracking
User Current user
Aspect / Attribute Item
Positive Negntive
Special positive Special negalive
User initialized attribute
Figure 2: Overview of the proposed framework, HutCRS. HutCRS consists of two stages, each comprising two components: Hierarchical User Interest Tracking and Hferarchical-Intere Policy Learning.
p(t)
our ruu_rey" Polt! = pale U P!!l p((+1)
mation, we also classify the attribute-level feedback results, where positive attributes Pace not present in the historical data and negative attributes Prej present in the historical data will be marked as "spe- cial". Hence, the current state includes four types components: St = =u, p!.), v! where c(t)
p(t) U p(t)
p(t-1)
p(t)
cur_rej' sepeial acc sepeial
U
rej
p (t)
p((t1) P sepoial_rej
p(()
our cur_scpoial_art p(t)
sepoial_rej
cur_ por_scpoiol_rey In this stage, we employ the at-
tribute instance-basee union set strategy to update v(t++)) cand Unlike Zhang et al. (2022), we don't 't re- move all items that contain the negative attributes, as in Lei et al. (2020b); Deng et al. (2021); Lei et al. (2020a): v((+!) {ulv E Vpo v and Pa N P(u) + 0).
c. (Clle, Cejz clt) denotes the state of as-
pects, and p!! Pale, p(,) p(t) p(()
special occ- spenial rejs ((t) caad] f denotes the state of attributes,
rej
D! (D(e) v&t canal denotes the state of items.
In stage I, aspects that jects in turn I can be defined as Ccur_ace and Cour_ned jrj_rej respectively. Before the system gets the accepted aspects Cuers some components are up- clt)
accepts re-
4.1.2 Interest-based Candidate Selection
To reduce the action space in subsequent steps, we need sort the candidate nodes the dynamic tree. Following Deng et (2021), we construct a dynamic weighted graph to rank candidate items. Given user u, we denote the dynamic graph at t-th turn as g( (100000))
dated by c(tll) c(t) -c(t) p( p(t)
caand" Cour_rej' repesia_rej = segrzial_re rej (P(!)
car cor_seeppal__e) It's worth noting that
((t)
we don't add P (ptc!t) to cur_rej directly, but
curures
we get ((() ser_sepcial_re) from epecty our mij
In this stage, we employ the aspect instance-based union set strategy update v((+1))y( cand {ulp e pp and Cy n c(t) * 0}. where Vpo is the item set in which all items are associated to po. and C. is the aspect
Ar0 = {u} UP! U UP!! U Uv!!)
(1)
are
cand
w!!
if n4 = u, CV
1,
if n4 E V,n PP
(2)
0.
otherwise
which all aspects are associated to v. Once the system gets the accepted aspects Cuocs the system will transition to stage II.
where E denotes the weighted edges between modes n, and ny. wi is scalar indicating the recommendation score of the item in the cur-
In stage II. the candidate attribute set Peand is initialized as PegECaec- Some components are updated by p(l+1) = p(t)
rent state. To incorporate the special negative information, such ((()) is calculated as: m!!")
p(!)
cand - Puuraa

---

( Co cp o Ce Epcrie co Excrle mp!!) (p!)
$(42)) (n) similarly, where the Tpoor denotes that the item v is associated with the attribute instance
E Epeplt)
special apecial meg snr 9 epe)). where a()) the sig- moid function, Cu. By and ep are the embeddings of the user, item, and attribute, respectively.
p. Then we can get node embeddings Sy ((++1) of
users, attributes and items respectively; Su
((+1)
ReLU(s (1=1 (n)), Sp ((+1)
ReLU (1+1 (m)),
Different from item ranking, those attributes with high scores should be able to not only cap- ture the user's preferences but also better reduce the uncertainty of candidate items. Since HutCRS only asks about attributes related to the positive aspects which have already captured user's pref- erences, we utilize the entropy method( Wu et al.. 2015) to rank the candidates.
(11) Se ((+)) = ReLU (Sowe (n). spon
(tD (n))). We
(1a)
denote the output the last layer Sn
as the final
embedding sy of the node.
HiierachicallLnteees Extractor Following Zhang et al. (2022), we use multi-attention mech- anism capture user's diverse interests. First, we fuse the embeddings of the rejected items and tribute instances to represent the negative interest of the user:
4.2 Hierarchical-Intere Policy Learning
The purpose of this module is to decide which as- pects to ask and when to ask attributes when recommend items achieve successful recom- mendations in the fewest turns. We construct current graph and global graph balance the user's short-term and long-term preferences. More- Extractor
VN+1 = rej
[Nres]
s
(4)
nENres
where Nroj Vrej U Proj. and Wrej E Rdxd trainable parameters. Then based on the dy- namic tree, we get the special labels for accepted attribute instance embeddings [01, U2, uN] and UN+1:
over, we utilize the Hierarchical- and Dueling Q-network to better understand users' hierarchical interests and decide the next action, Graph Representation. For the current graph g! which is introduced in 4.1.2, we employ Le-layer GCN(Kipf and Welling, 2016) to refine the node embeddings as in Zhang al. (2022):
[1,0]. if p: P Pspecial_ace ace
1.
(5)
[0, 1]. otherwise
where 1, N, + It is worth noting that we use the embedding e,, of the current graph ett) for d1 to UN balance the user's short-term and long-term preferences.
(1+1) !
((+1) ReLU
+ !!!! s(t) E,
E
(t) n.
JENZO
Finally, we utilize K, attention networks with M iterations to adjust the weights between [U1, v2, UN UN+1]. The method for calculating the initial iteration of each attention network obtain the interest embedding qi
(1)
where en denotes the output node embedding of I-th layer, (Le) denotes the final embedding e,, of the node, denotes the node n's neighbor nodes
set turn and (I+1) E Rdxd are trainable parameters.
(1) is as follows:
For the global graph Gg = EN,)), where N = -uu MUVUP and È = Eu, U &p,,. we employ Laylr Global Graph Neural Network (SGNN((Schlicnkrrul et al., 2018; Chen et al., 2020, 2019) extract long-term preferences of users, and global correlations of items and attribute instances as in Zhang et al. (2022): s (n)
N+1 kn Vn k E (e,....KK (6)
exp ( σ (Wx (v))))
(1)
(7)
N+1 Ln exp (h, (W& (v. (ww))))))
where h& and We are trainable parameters. Based on the m- 1-th iteration results, we can get the (m)
bg (1-1) + EiENr.... (n) /W (a) Ar (n)
where Nr..o (n) denotes the neighbor nodes of node n with the edge type Pa-es which denotes
q as follows:
the user u has interacted the item (L+1) and
N
(m)
(m)
by (1+1) are both trainable parameters. We can get
q
V
(8)

---

stage II, we adopt the top-based strategy. The system first selects the action with the max Q- value. If the selected action points to an item, the system will recommend top- items. Otherwise, the system will ask top-Katt attribute instances. Moreover, we define a replay buffer D to store the experience (st, at, Tt, S1+1,,4+1 and deñine loss function:
exp(h o (Wa(Va ))
(m)
nl= exp(h o (WWilw
(9)
takm (m-1))
(m 11)) + akn ()) ) 1/ 9
We define the output of the M-th iteration as the final embeddings. These embeddings are represented by . .
Q St, oe; Qq: 0M)) where Oar is the set of pa- rameters to capture hierarchical-intrre embed- dings, fg = { 8A), and y/. is the target value, which is based on the optimal Q-function: Ye
"K.....,),, M ..... Ix where -M q .... M are the embedded vectors that capture the hierarchical interests.
Ti + y max Q ($1-11 a++1; fgg fas).
Action Decision Policy Learning. A good strategy not only makes the recommendation at the appro- priate time but also exhibits flexible adaptation to users' feedback. Additionally, it maintains conver- sational topies and adapts to various scenarios to ensure user comfort throughout the interaction et al., 2021). Previous works focus on the strategy for determining when to ask attributes or when to recommend items, while our strategy also requires determining which aspects to ask. Therefore, we divide the mabisiionmmkinng process into two stages. In stage the system aims to identify the aspects that the user prefers, while in stage II, the system will ask about attributes related these positive aspects or recommend items.
We utilize the double DQN (Van Hasselt et al., 2016) method to combat the overestimation bias present in the original DQN. Specifically, ploy a periodic copy of a target network Q' from the online network train the model, in accor- dance with Deng et al. (2021); Zhou et al. (2020); Zhang et al. (2022).
em-
5 Experiment
5.1 Datasets
Following Zhang al. (2022), we conduct experi- ments on four benchmark datasets: Yclp, LastFM, Amazon-Boon Book,, and MovieLens. For each conver- sation episode, we sample N2 items with partially overlapped attribute instances as acceptable items for the user.
Following Deng et al. (2021); Zhang et al. (2022), we select top-K, items based on the rec- (t) and top-K, attribute in- stances based on the entropy score as the action space Ap- Inspired by Zhang et al. (2022), we adopt a ducling Q-network to determine the next action. The Q-value Q(s:, at) of the state 8t and the action a₂ is calculated by: Q (st, at) = max; (Jey (qs) + Sea (4k, a4)), k E (1....K) K7). where Jey and Sex (-) are two separate multi- layer perceptions (MLP). Following the Bellman equation (Bellman and Kalaba, 1957), the optimal Q-value (s,,,, a.) is calculated by: Q" (st, a:) Ex [r: + y maXa EA441 Q" ( ax41 1 at)). where r is the reward based on the user's feedback and y is the discounted factor.
ommendation score wn!
5.2 Experiments Setup
User Simulator. We conduct a simulation of a conversation session for each observed user-item interaction pair (u, Vu). In this simulation, we con- sider each item 02g E Vu as the target item that serves as the ground truth. To initiate the session, the simulated user specifies an attribute instance Po E P Psaae.. Given conversation, the simulated user's feedback of each turn follows the rules: (1) when the system asks a question. they will accept the aspects instances or attribute instances which are associated with any item in Vu and reject others; (2) when the system recommends a list of items, it will be accepted by the user if at least one item in Vu is included; (3) the user's 's patience will expire when the maximum number of turn T is reached. Baselines. To evaluate model performance, we compare our model with the following baselines: (1) Max Entropy(Lei et al., 2020a), which asks the user for an attribute based on the maximum en- tropy or recommends items with a certain probabil-
In stage I, we adopt the sum- based strategy. For each candidate aspect node of the dynamic tree, we sum the Q-values of its child nodes in A, obtain the aspect-level score by which we se-
lect top-Kasp aspect instances to ask. Once the user accepts some aspects, the dynamic tree no longer updates the first layer nodes and the sys- tem will transition from stage to stage II. In

---

Yelp
LastFM
Amazon Book
MovieLens
Models
$R@15
AT
hDCG
15@15
AT
hDCG SR 15 AT hDCG
SR@ @ 15
AT hDCG
Abs Greedy
0.222
13.48
0.073
0.635
8.66 0.267
0.189
13.43 0.089
0.273
12.19 0.138
Max Entropy
0.375
12.57
0.139
0.640
9.62 0.288
0.343
12.21 0.125
0.704
6.93 0.448
CRM
0.223
13.83
0.073
0.597
10.60 0.269
0.309
12.47 0.117
0.654
7.86 0.413
BAR 0.263
13.79
0.098
0.612
9.66 0.276
0.354
12.07 0.132
0.714
6.53 0.457
SCPR
0.413
12.45
0.149
0.751
8.52
0.339
0.428
11.50 0.159
0.812
4.03 0.547
UNICORN
0.438
12.28
0.151
0.843
7.25
0.363
0.466
11.24 0.170
0.836
3.82 0.576
MCMIPL
0.482
11.87
0.160
0.874
6.35
0.396
0.545
10.83 0.223
0.882
3.61 0.599
HutCRS
0.528
11.33'
0.175'
0.900
6.52
0.348
0.638
9.84 0.227 0.902
4.16 0.475
Table 1: Performance comparison of different models the four datasets. improvement (p < 0.01) over all baselines. hDCG stands for hDCG@(15, 10).
indicates statistically significant
ity. (2) Abs Greedy(Crrisakoppalla al., 2016), which only recommends items and updates itself. (3) CRM(Sun and Zhang, 2018), which is single- round CRS that learns the policy deciding the next action. Following Lei et al. (2020a), we adapt CRM to MCR scenario. (4) EAR(Lei et al., 2020a),
0.3
0.2
Relative Success Rate
0.0
-0.1
which proposes three-stage model to better con- verse and recommend item to users. (5) SCPR(Lei et al., 2020b), which conducts interactive path rea- soning on the graph to prune candidate attribute nodes in the graph, and employs the DQN to select an action. (6) UNICORN(Deng et al., 2021), which proposes a unified policy learning framework and adopts a dynamic weighted graph-based RL. to select action. (7) MCMIPL.ZZhng et al., 2022), which generates multiple choice questions, utilizes a union set strategy to select candidate items and the exact multi-interes: of the user to select the next action. It is the
-0.2
0.3
MCMIPLK
HutCRS Yelp
-0.4
HutCRR-LastFM
-0.5
HutCRS-Amazon Book
HutCRS-MovieLens
-0.6
10
12
14
Conversation Turns
Figure 3: Comparisons at Different Conversation Turns.
tract the user's hierarchical interests with K₁ 4 attention networks, M iterations and the ac- tion space consists of Kp = 10 attribute instances and Kg = 10 items. The reward settings are as follows: Tree_sue = 1, Free fail = -0.1, Task_ate suc 0.01, Task_ all fail = -0.1, Task Task_asp_nn asp_sue = 0.02, -0.2, Tquit = -0.3. Inspired by Zhang et al. (2022), ing question follows: In stage I, Pt Ege ace while in stage II. I' = Ep(t) Task atl suc +
method.
Parameters Setting We randomly split each dataset into training, validation, and test sets in a 7:1.5:1.5 ratio. The embedding dimension is set 64, and the batch size to 128. In stage I, We ask most Kesp 4 aspect instances in each turn. stage II, we recommend top K 10 items or ask Katt 2 attribute instances each turn. The maximum number of turn T for a conversation is set to 15. We utilize the Adam optimizer with a learning rate of le-4. Discount factor is set to be 0.999. Following Deng et al. (2021), we pretrain the node embeddings in the constructed KG with the training set using TransE (Bordes et al., 2013) via OpenKE (Han al., 2018). We construct the global graph based on the training set. The num- bers of current GNN layers Le and global GNN layers Ly are set to 2 and 1, respectively. We ex-
"'ask_aep_fe"
set the reward of ask-
Task Task_asp_smn aup + Ege0
Conerree Task Pask_asp__ae fanls
Eper Per Task_att_fail We set the maximum num- curures ber N,, of acceptable items as 2 and explore other settings in the hyper-parameter analysis.
Evaluation Metries. Following Lei et al. (2020a);
Deng et al. (2021): Zhang et al. (2022), we utilize the success rate turn (SR @t) (Sun and Zhang, 2018) 8) to measure the cumulative ratio of successful recommendation with the maximum turn T, and average turn (AT) to evaluate the average number

---

I'm looking for a rhythmic noise music artist.
Hit I'm looking rhythmic noise music artist.
Which aspect do you care about?
Which rock music do you like? (A) German (B) Others
(A) Electronic (I3 Rock (C) Industrial (D) Others
A and C.
B.
Which of the following do you like?
Which rock music do like? (A) Gothic (B) Industrial rock (C) Others
(A) Dark electro (B) Electronica (C) Others
Which electronic music do you like? (A) Techno (B) Electronica (C) Others
You may like music artist S.L.NA!
Thank you!
Success! (a) HutCRS
Contiauc...
MCMIPL
Figure 4: Sample conversations generated by HutCRS and MCMIPL.
of turns. Besides, we adopt hDCG@(T.K) (Deng al., 2021) to additionally evaluate the ranking performance recommendations. Higher values SR @t and hDCG@ (T,K) indicate better perfor- mance, while a lower value of AT indicates overall higher efficiency.
Amazon Book
MovieLens
Models
SR @ 15 AT hDCG SR@15 AT hDCG
HutCRS
0.638
9.84 0.227
0.902 4.16 0.475
(a) 0.630
9.85 0.225
0.890
4.25 0.467
(b) 0.619
10.00 0.221
0.898
4.26 0.468
(c) 0.615
10.05 0.218
0.871
4.77 0.432
5.3 Performance Comparison
(d) 0.541
10.79 0.189
0.882
4.69 0.438
The comparison experimental results the base-
(e) 0.629 9.86 0.227
0.875
4.42 0.444
line models and our models are shown in Table 1. In terms of SR @ 15, our model outperforms all the comparison methods with improvements of 10%, 3%, 17%, and 2% four datasets. But in terms of AT and hDCG, model only shows improve- ment on the Yelp and Amazon Book datasets. How- ever, this result does not imply that our model inefficient. We intuitively present the performance comparison of success rate at each turn in Figure 3. Relative success rate denotes the difference be- tween HutCRS and the most competitive baseline MCMIPL, where the blue line of MCMIPL is set to = 0 in the figure. We observe that the absence of recommendations in stage for HuICRS leads a lower success rate compared to MCMIPL in the initial turns. As result, this negatively affects the model's performance on the AT and hDCG metrics, especially for relatively straightforward datasets such as lastfm and movielens. But for the more challenging Yelp and Amazon Book datasets, our model achieves improvement across all metrics. This shows that method attains better perfor- mance than previous baselines.
0.592
10.06 0.217
0.878 4.34 0.468
Table 2: Results of the Ablation Study.
LastFM
Movielens
1.0
0.0
1.0
5.0
15
5RB13
AT
500991
6.5
4.02
5.5
0.7
5.0
0.7
3.0
No. of asked aspect instances
No. asked aspect instances
Figure 5: Performance comparisons w.r.t. Kaap
lation models: (a) removing the classification; (b) removing the global graph; re)roing the cur- rent graph; (d) adopting binary questions stage I; (c) replacing the sum-based strategy for selecting aspect instances with a random strategy; (f) replac- ing the top-based strategy for selecting attribute instances with random strategy. All ablation mod- cls perform worse than HutCRS in terms of all metries, which demonstrates the preeminence of HuICRS.
5.4 Ablation Studies
5.5 Hyper-parameter Analysis
order verify the effectiveness of some key de- signs, we conduct series of ablation experiments on the Amazon- Book and MovieLens datasets. The results are shown in Table 2. We design four ab-
Inquiring users with questions that incorporate a varying number of aspect instances (Kasp) influ- ences the performance of the model. As illustrated

---

in Figure 5, the AT metric exhibits improvement when the value of Kasp rises, whereas the SR @ 15 metric does not consistently display enhancement. This observation suggests that increasing Kasp be- yond a certain threshold may yield limited perfor- mance gains. The ongoing decline of the AT metric can be attributed to reduction in the number of turns taken during stage as Kaep escalates.
to the stage I after entering the stage II, as this may enhance performance. Lastly, our method solely classifies feedback at the attribute level, lacking item-level classification, which also presents an opportunity for improvement.
Ethics Statement
The data utilized in our study are sourced from open-access repositories, and do not pose any privacy concerns. We are confident that our re- search adheres to the ethical standards set forth by EMNLP.
5.6 Case Study
We further conduct qualitative analysis to demon- strate the role played by the hierarchical interest structure in successful recommendations. We ran- domly sample real-world interaction from our model and the state-of-the-art method on LastFM based on the same test instance. The generated con- versations by HutRRS and MCMIPL with the user simulator are presented in Figure 4. It can be ob- served that HuCRS first identifies the aspects that the user prefers, then asks about attributes related to the accepted aspects, and finally recommends the item. Compare to HutCRS, MCMIPL repeat- edly asks about attributes of the aspect that the user is not interested in, resulting in lower efficiency. Therefore, the hierarchical interest structure makes conversations more user-friendly and explainable.
Acknowledgements
This work is supported in part by the Na- tional Key R&D Program of China under Grant 10.2021200111601 in part by the National Nat- ural Science Foundation of China under Grant No.U21 Grant No.61836011 and Grant No. 62206314; in part by the GuangDong Ba- sic and Applied Basic Research Foundation un- der Grant No. 2023A1515011374 and Grant No. 2022A151501 1835; in part by the Guangdong Province Key Laboratory of Information Security Technology; and in part by the China Postdoctoral Science Foundation Funded Project under Grant No. 2023M77.
Conclusion
In this paper, we propose more realistic, user- friendly, and explainable framework HutCRS, which portrays the conversation as a hierarchical interest tree consisting of two stages. In addition, we design HIPL module to integrate the decision- making process and classify attribute-level feed- back capture special information. Extensive ex- periments four benchmark datasets demonstrate the superiority of our method.
References
Richard Bellman and Robert Kalaba. 1957. On the role of dynamic programming in statistical communica- tion theory, IRE Transactions on Information Theory, 3(3):197-203.
Antoine Bordes, Nicolas Usunier, Alberto Garcia- Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. Advances in neural information pro- cessing systems, 26.
Limitations
While our model achieves a new state-of-the-ar art performance, it still has several limitations. Firstly, our framework employs a hierarchical interest tree structure, necessitating the inclusion of both at- tribute and aspect data in the dataset. Secondly, the conversation is divided into two stages. During the stage the system asks about aspects until the user accepts specific aspects and then proceeds to the stage II. Once in the stage 1I, the system unable revert to the stage Ideally, the agent should be able to recommend directly based on negative feedback regarding aspects in the stage I and return
Yu Chen, Lingfei Wu, and Mohammed Zaki. 2020. Iter- ative deep graph learning for graph neural networks: Better and robust node embeddings. Advances neural information processing systems, 33:19314 14- 19326.
Yu Chen, Lingfei Wu, and Mohammed Zaki. 2019, Re- inforcement learning based graph-to-sequence model for natural question generation. arXiv preprint 19X.v:1908.04942.
Konstantina Christakopoulou, Alex Beutel, Rui Li, Sagar Jain, and Ed H Chi. 2018. Q&R: A two-stage approach toward interactive recommendation. In Pro- ceedings of the 24th ACM SIGKDD International

---

Conference on Knowledge Discovery & Data Mining, pages 139-148.
2018. Modeling relational data with graph convolu- tional networks. In The Semantic Web: 15th Inter- national Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings 15, pages 593- 607. Springer.
Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards conversational recom- mender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge dis- covery and data mining, pages $15-824.
Yueming Sun and Yi Zhang. 2018. Conversational rec- ommender system. In The 41xt international acm sigir conference on research & development in infor- mation retrieval, pages 235-244.
Yang Deng. Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. 2021. Unified conversational recommendation graph-based policy learning via graph- based reinforeement learn- ing. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1431-1441.
Quan Tu, Shen Gao, Yanran Li, Jianwei Cui, Bin Wang, and Rui Yan. 2022. Conversational recommendation via hierarchical information modeling. In Proceed- ings of the 45th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, pages 2201-2205.
Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: survey, Open, 2:100-126.
Hado Van Hasselt, Arthur Guez, and David Silver. 2016.
Deep reinforcement learning with double q- q-learming, In Proceedings of the AAAI conference on artificial intelligence, volume 30.
Xu Han, Shulin Cao, Xin Lv, Yankai Lin, Zhiyuan Liu, Maosong Sun, and Juanzi Li. 2018. Openke: An open toolkit for knowledge embedding. In Proceed- ings of the 2018 conference empirical methods in natural language processing: system demonstrations. pages 139-144.
Wu, Miao Li, and Chin-Hui Lee. 2015. A proba- bilistic framework for representing dialog systems and cntropy-based dialog management through dy- namic stochastic state evolution. IEEF/ACM Truns- actions on Audio, Speech, and Language Processing, 2035..
Chenhao Hu, Shuhua Huang, Yansen Zhang, and Yubao Liu. 2022. Learning to infer user implicit preference in conversational recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 256-266.
Kerui Xu, Jingxuan Yang, Jun Xu, Sheng Gao, Jun Guo, and Ji- Ji-Rong Wen. 2021. Adapting user preference to online feedback in multi-round conversational rec- ommendation. In Proceedings of the 14th ACM inter- national conference on web search and data mining, pages 364-372.
Thomas N Kipf and Max Welling. 2016. Semi- supervised classification with graph convolutional networks. arXiv preprint arXiv: (iKi::6600029000
Yiming Zhang. Lingfei Wu, Qi Shen, Yitong Pang, Zhi- hua Wei, Fangli Xu, Bo Long, and Jian Pei. 2022. Multiple choice questions based multi-interest policy learning for conversational recommendation. In Pro- ceedings of the ACM Web Conference 2022, pages 2153-2162.
Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020a. Estimation Estimation-action-oe action-reflection. Towards deep interaction between conversational and recom- mender systems. In Proceedings of the 13th Interna- tional Conference on Web Search and Data Mining, pages 304-312.
Sijin Zhou, Xinyi Dai, Haokum Chen, Weinan Zhang, Kan Ren, Ruiming Tang, Xiuqiang He, and Yong Yu. 2020. Interactive recommender system via knowl- edge graph graph-enhanced reinforcement learning. In Pro- ceedings of the 43rd international ACM SIGIR con- ference on research and development in information retrieval, pages 179-188.
Wenqiang Lei, Gangyi Zhang, Xiangnan Hc, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020b. Interactive path reasoning on graph for con- versational recommendation. 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2073- 2083.
Proceedings of the
Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie
He, Han Li, and Kun Gai. 2018 Learning tree-based deep model for recommender systems. In Proceed- ings of the 24th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining, pages 1079-1088.
Xuhui Ren, Hongzhi Yin. Tong Chen, Hao Wang, Zi Huang, and Kai Zheng. 2021. Learning to ask appropriate questions in conversational recommenda- tion. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 808-817.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titoy, and Max Welling.


Reviews:
[{"review_id": "jKo9foHJoM", "reviewer": "Reviewer_2e3F", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\n​    ​    ​    ​    ​    ​    Thank you for reviewing our paper and providing valuable comments and suggestions. We sincerely appreciate your attention to our research and the time you have dedicated to it. Based on your feedback, we have engaged in thorough discussions and analysis, and we will make the necessary revisions to enhance the quality and accuracy of the paper. In the following response, we provide detailed explanations of our responses to each question and the specific measures taken for the revisions.\n\n**Comment1** The authors should conduct a t-test to verify the proposed model's effectiveness, as the reinforcement learning technique is hard to implement. \n\n**Response:** Thanks for your suggestion. We have conducted one-sample t-tests on all four datasets with a sample size of 40. The results indicate a significant difference between our model's performance and the strongest baseline MCMIPL (p < 0.01). Besides, we will provide the t-test on all datasets in the final paper.\n\n**Comment2** The task of conversational recommendation is still interesting, but I feel that this task is still difficult to evaluate because of the lack of real datasets. It would be nice if the authors could try to make an effort to build real datasets.\n\n**Response:** Thanks for your suggestion. We truly appreciate the value of using real datasets for evaluating this task, as the existing works mostly rely on the user simulator to simulate real users, which introduces biases. Therefore, in this study, we aim to extend the existing CRS to a more realistic, user-friendly, and explainable setting to ensure the effectiveness of the evaluation as much as possible. In the future, we will make an effort to construct real datasets.\n\n**Comment3** The method proposed in the article is still complex and open-source code is recommended.\n\n**Response:**  Thanks for your suggestion. We will release our code and model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.  HutCRS introduces interest-based policy learning and a hierarchical interest tree approach to dialog modeling, allowing the system to understand users' interests and preferences better and dynamically adjust based on user feedback.\n2.  The experimental results show that HutCRS has higher recommendation accuracy as well as shorter conversation lengths compared to the traditional conversational recommender system approach.\n3.  HutCRS introduces a deep reinforcement learning-based conversation management approach that allows the system to make more intelligent conversation management and recommendation decisions.", "reasons_to_reject": "1. The authors should conduct a t-test to verify the proposed model's effectiveness, as the reinforcement learning technique is hard to implement. \n2.  The task of conversational recommendation is still interesting, but I feel that this task is still difficult to evaluate because of the lack of real datasets. It would be nice if the authors could try to make an effort to build real datasets.\n3.  The method proposed in the article is still complex and open source code is recommended.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper's main topic is a Conversational Recommender System (CRS) that utilizes a hierarchical interest tree and attribute-level feedback to enhance the system's ability to comprehend the needs and preferences of users. The system is designed to generate a personalized recommendation for each user through a multi-turn dialogue process.\n\nThe main contributions of the paper are:\n1. The proposal of a hierarchical interest tree and the utilization of the classification results of attribute-level feedback to further enhance the system's ability to comprehend the needs and preferences of users.\n2. Demonstrating the superiority of HutCRS compared with state-of-the-art baselines through extensive experiments on four benchmarks, including Yelp, LastFM, Amazon-Book, and MovieLen datasets.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ahTE4Cev5f", "reviewer": "Reviewer_DXwT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\n​    ​    ​    ​    ​    ​    Thank you for reviewing our paper and providing valuable comments and suggestions. We sincerely appreciate your attention to our research and the time you have dedicated to it. Based on your feedback, we have engaged in thorough discussions and analysis, and we will make the necessary revisions to enhance the quality and accuracy of the paper. In the following response, we provide detailed explanations of our responses to each question and the specific measures taken for the revisions.\n\n**Comment1** The paper does not perform convincingly in empirical evaluations. Notably, it does not outperform the baseline MCMIPL on the LastFM and MovieLens datasets. This raises questions about the practicality and efficacy of the proposed method.\n\n**Response:** It is indeed true that our model does not outperform the baseline MCMIPL in terms of AT and hDCG on the LastFM and MovieLens datasets. However, as mentioned in lines 541-547 of the paper, this does not imply that our method is inefficient. Firstly, the primary objective of our work is to extend the existing CRS to a more realistic, user-friendly, and explainable setting. In this new and more challenging setting, our model still achieves better overall performance. Secondly, unlike MCMIPL, our model does not perform recommendations in stage I; instead, it invests additional turns to gather aspects of user interest. Intuitively, this necessitates more turns for our model to complete the recommendations, particularly for simpler datasets like LastFM and MovieLens, resulting in lower AT and hDCG metrics. Lastly, regarding SR@15, our model outperforms MCMIPL on four datasets, indicating that our model has better potential.\n\n**Comment2** It appears that the improvements seem to be more from allowing multiple choice feedback than the proposed hierarchical tree and policy learning contributions. More evidence is needed to demonstrate their effectiveness.\n\n**Response:** Thanks for your comment. We believe it is highly reasonable that adopting binary choice questions (i.e., yes/no) in stage I leads to the most significant decline in the model's performance. The purpose of stage I is to gather the aspects they are interested in, and only those positive users' feedback can be transformed into stage II for recommendations. Therefore, if the system can only inquire about one aspect instance each turn, it proves not only less user-friendly but also results in the system spending more turns in stage I, consequently leading to a noticeable decrease in the model's performance. We conduct statistical analysis on the Amazon Book dataset. If the system only asks about one aspect in each turn, the average number of turns required in stage I would increase by approximately 37\\%.\n\n**Comment3** The necessity of the current and global graphs in the ablation study is not well justified. There is a lack of evidence that these components significantly contribute to the system's performance.\n\n**Response:** Thank you for your suggestions. We conduct ablation studies on the Amazon Book and MovieLens datasets to investigate the effects of the global graph and the current graph. The experimental results are presented in the table below:\n| model         | datasets      | SR@15 | AT   | hDCG  |\n|---------------|---------------|-------|------|-------|\n| HutCRS        | Amazon Book   | 0.638 | 9.84 | 0.227 |\n| -global graph | Amazon Book   | 0.619 | 10.00| 0.221 |\n| -current graph| Amazon Book   | 0.615 | 10.05| 0.218 |\n| HutCRS        | MovieLens     | 0.902 | 4.16 | 0.475 |\n| -global graph | MovieLens     | 0.898 | 4.26 | 0.468 |\n| -current graph| MovieLens     | 0.871 | 4.77 | 0.432 |\n\nAs can be seen from the table, both the global graph and the current graph have a positive impact on the model's performance, with the current graph having a relatively larger influence.\n\n**Comment4** The connection between the proposed method and the challenges mentioned in the introduction is unclear. For example, the paper does not explain how the hierarchical tree framework handles situations where \"users might not necessarily like all attributes of the target item, nor dislike attributes that are not included in the target item.\"\n\n**Response:** Thanks for your suggestion. As mentioned in lines 105-111 of the paper, we first allow users to select the aspects they are interested in, and then ask about relevant attribute instances. We believe that users have clear preferences for these attribute instances: if an attribute instance belongs to the target item and falls within the aspects the user is interested in, the user will accept it. We will emphasize it in our paper later.\n\n**Comment5** The paper's clarity could be improved. For instance, Figure 2 lacks clarity in portraying the two-stage process, and Figure 3 is placed too far from the corresponding text. \n\n**Response:** Thanks for your suggestion. We will polish our paper and the layout of the figures accordingly.\n\n**Comment6** \"Hrarchical\" in line 093 should be \"Hierarchical\".\n\n**Response:** Thanks for pointing out the errors. We will correct these errors in the final paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Classifying attribute feedback as special positive/negative is an interesting idea to capture non-obvious preferences that improve recommendation accuracy.\n* The proposed method is more user-friendly, as it allows users to provide feedback on multiple aspects, unlike traditional systems that only allow single-item feedback.\n* Experiments on four benchmarks show the effectiveness of the proposed method.", "reasons_to_reject": "* The paper does not perform convincingly in empirical evaluations. Notably, it does not outperform the baseline MCMIPL on the LastFM and MovieLens datasets. This raises questions about the practicality and efficacy of the proposed method.\n* It appears that the improvements seem to be more from allowing multiple choice feedback than the proposed hierarchical tree and policy learning contributions. More evidence is needed to demonstrate their effectiveness.\n* The necessity of the current and global graphs in the ablation study is not well justified. There is a lack of evidence that these components significantly contribute to the system's performance.\n* The connection between the proposed method and the challenges mentioned in the introduction is unclear. For example, the paper does not explain how the hierarchical tree framework handles situations where \"users might not necessarily like all attributes of the target item, nor dislike attributes that are not included in the target item.\"\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes HutCRS, a conversational recommender system using a hierarchical interest tree to model user preferences. The model consists of two stages: hierarchical user state tracking and hierarchical-interest policy learning. \nBesides, the paper classifies the attribute-level feedback results to further enhance the system’s ability to comprehend the needs and preferences of users.\nExperiments on four datasets show improvements over baselines.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "* The paper's clarity could be improved. For instance, Figure 2 lacks clarity in portraying the two-stage process, and Figure 3 is placed too far from the corresponding text.\n* \"Hrarchical\" in line 093 should be \"Hierarchical\""}}, {"review_id": "BMvOcAVZct", "reviewer": "Reviewer_9hoF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\n​    ​    ​    ​    ​    ​    Thank you for reviewing our paper and providing valuable comments and suggestions. We sincerely appreciate your attention to our research and the time you have dedicated to it. Based on your feedback, we have engaged in thorough discussions and analysis, and we will make the necessary revisions to enhance the quality and accuracy of the paper. In the following response, we provide detailed explanations of our responses to each question and the specific measures taken for the revisions.\n\n**Comment1** The authors' construction of a hierarchical interest tree to represent user preferences in each session raises concerns about the time complexity of the overall process. To demonstrate the efficiency of the proposed framework, it would be beneficial if the paper includes a detailed analysis of the time complexity and compares it with other methods.\n\n**Response:** It is worth noting that our hierarchical interest tree hardly incurs additional time and memory overhead. Although we portray the conversation as a hierarchical interest tree, it is not necessary to store the full tree. It is enough to promote dialogue by saving the nodes (such as candidate attribute sets) and the relationships between nodes which can be obtained from the knowledge graph. In this regard, existing methods also store nodes (except for aspect nodes and special information) and make use of a knowledge graph, so our hierarchical interest tree is low-cost. Furthermore, compared to [1], our model only asks about attributes related to aspects that the user is interested in. This can reduce the size of the candidate attribute set and consequently decrease the number of scoring computations performed on attribute instances, making the conversation more efficient.\n\nSpecifically, in a turn of scoring attribute instances, [1] has a time complexity of $O(n_1 \\cdot t_1)$, while our model has a time complexity of $O(n_2 \\cdot t_2)$, where $n$ represents the size of the candidate attribute set, and $t$ represents the time complexity of scoring a single attribute instance. It is worth noting that $t_1$ is approximately equal to $t_2$, and $n_1$ is approximately k times larger than $n_2$, where k represents the average factor by which $n_1$ is larger than $n_2$. Since the four datasets on average contain around 30 aspect instances, k is expected to be greater than 10. Therefore, our model is more efficient in comparison.\n\n**Comment2** Although the framework bears similarities to [1], the paper should emphasize and better showcase the improvements and advancements achieved by the proposed approach over [1]. This clarification would add more clarity and value to the research, enabling readers to understand the novel contributions of the proposed framework.\n\n**Response:** Thanks for your suggestion. We will emphasize and better showcase the improvements and advancements achieved by the proposed approach over [1]. Compared to [1], we have made several improvements in our dialogue process. Firstly, our system now starts by capturing the aspects that interest the user before asking for attributes or recommending items. This not only enhances user-friendliness but also ensures more authentic feedback at the attribute-level. Secondly, we have utilized a hierarchical interest tree to reduce the size of the candidate attribute set, resulting in a more efficient computation process. Additionally, we have classified the feedback at the attribute-level and incorporated special information into the embeddings, enabling the model to better understand user’s feedback with greater precision. Lastly, our model's overall performance outperforms [1], especially on datasets that are inherently more challenging for successful recommendations. This demonstrates the superior potential of our model.\n\n**Comment3** For a comprehensive understanding of the proposed approach's performance, it would be valuable to include information on the success rate at each conversation turn. This data can offer insights into the framework's comparative performance throughout the recommendation process, helping readers to assess its effectiveness in a more nuanced manner.\n\n**Response:** Thanks for your suggestion. In fact, our paper has already included Success Rate at turn $t$ (SR@15) metric where turn $t$ ranges from 1 to 15, which is a most popular metric to evaluate the models' ability to minimize number of conversation turns in conversational recommendation. Most researches [1] [2] [3] [4] [5] consider that SR@15 (i.e., $t \\in [0, 15]$) can be effectively to evaluate the conversation task since a robust CRS could successfully recommend items in the shortest possible number of rounds, and thus $t \\in [0, 15]$ contains enough turns. Besides, we will draw a figure to show the success rate at each conversation turn on four datasets to make it be more intuitively understood in the final paper.\n\n**Comment4** Is the process of constructing the full hierarchical interest tree and then pruning it to obtain a dynamic tree based on user feedback computationally expensive in terms of memory and time?\n\n**Response:** Our hierarchical interest tree is somewhat similar to the one mentioned in Paper [6]. We will include [6] in the list of references to help users better understand the hierarchical interest tree. However, as mentioned earlier, since we can directly retrieve the relationships between nodes from the knowledge graph, and both the knowledge graph and the information of the nodes themselves are also utilized in other methods, it hardly incurs any additional time and space overhead.\n\n[1] Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhi-hua Wei, Fangli Xu, Bo Long, and Jian Pei. 2022. Multiple choice questions based multi-interest policy learning for conversational recommendation. In Proceedings of the ACM Web Conference 2022, pages 2153–2162.\n\n[2] Lei W, He X, Miao Y, et al. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems[C]//Proceedings of the 13th International Conference on Web Search and Data Mining. 2020: 304-312.\n\n[3] Deng Y, Li Y, Sun F, et al. Unified conversational recommendation policy learning via graph-based reinforcement learning[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021: 1431-1441.\n\n[4] Lei W, Zhang G, He X, et al. Interactive path reasoning on graph for conversational recommendation[C]//Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020: 2073-2083.\n\n[5] Xu K, Yang J, Xu J, et al. Adapting user preference to online feedback in multi-round conversational recommendation[C]//Proceedings of the 14th ACM international conference on web search and data mining. 2021: 364-372.\n\n[6] Zhu H, Li X, Zhang P, et al. Learning tree-based deep model for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 1079-1088.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe authors recognize the limitations of current Multi-Choice Conversational Recommender (MCR) methods and enhance them to address a more realistic scenario, considering the user's authentic interest characteristics.\n2.\tIn order to enhance the system's comprehension of users' preferences and requirements, the authors introduce the concept of a hierarchical interest tree, which accurately depicts users' interests and priorities.\n3.\tThe authors extensively evaluate the proposed framework through rigorous experiments conducted on four benchmark datasets, providing compelling evidence of its effectiveness.\n", "reasons_to_reject": "1.\tThe authors' construction of a hierarchical interest tree to represent user preferences in each session raises concerns about the time complexity of the overall process. To demonstrate the efficiency of the proposed framework, it would be beneficial if the paper includes a detailed analysis of the time complexity and compares it with other methods.\n2.\tAlthough the framework bears similarities to [1], the paper should emphasize and better showcase the improvements and advancements achieved by the proposed approach over [1]. This clarification would add more clarity and value to the research, enabling readers to understand the novel contributions of the proposed framework.\n3.\tFor a comprehensive understanding of the proposed approach's performance, it would be valuable to include information on the success rate at each conversation turn. This data can offer insights into the framework's comparative performance throughout the recommendation process, helping readers to assess its effectiveness in a more nuanced manner.\n\n[1] Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhi-hua Wei, Fangli Xu, Bo Long, and Jian Pei. 2022. Multiple choice questions based multi-interest policy learning for conversational recommendation. In Proceedings of the ACM Web Conference 2022, pages 2153–2162.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "Addressing the limitations of current attribute-based Conversational Recommender Systems (CRS) in accurately capturing user interest and reflecting real-world scenarios, the authors present a new framework named Hierarchical User-Interest Tracking for Conversational Recommender System (HutCRS) to enhance the user experience.  In HutCRS, the ongoing conversation is represented as a hierarchical interest tree, with the user initialized attribute, aspect instances, attribute instances, and items forming the root, first, second, and last layers respectively. The tree is dynamically pruned based on user feedback. Additionally, attribute-level feedback results are classified to capture special information effectively. To optimize the recommendation process, the authors employ a Hierarchical-Interest Policy Learning module to make decisions that lead to successful recommendations in fewer interactions. The proposed system's effectiveness is evaluated through extensive experiments conducted on four benchmark datasets.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "1.\tIs the process of constructing the full hierarchical interest tree and then pruning it to obtain a dynamic tree based on user feedback computationally expensive in terms of memory and time?"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an approach to produce personalized recommendation through multi-turn dialogue. User interests are represented hierarchically through a tree during dialogue, which is pruned and used to make recommendations (in a policy learning phase).

**Pros**: Reviewers agree the paper introduces new concepts (hierarchical interest tree and associated policy learning) that provide practical benefit to the to the conversational recommendation problem. Some reviewers note the solution has "interesting" new ideas, is more "user friendly", and has extensive evaluation (4 benchmarks).

**Cons**: Most reviewers raised technical concerns including: missing ablations, the use of simulated data, statistical significance of results, consistency of improvements compared to baselines as well as some concerns about motivation/hypotheses. Still, during the rebuttal authors appear to address many of these concerns. All reviewers acknowledge these responses post-rebuttal, and no review indicates serious technical concerns in the scoring. At the same time, most reviewers are ambivalent - there is no champion among the reviewers.

Overall a fairly middle ground paper: reviewers technical concerns are not serious enough to warrant low scores, and excitement among reviewers is not high enough to elicit a champion.
Decision: Accept-Main