Lending Interaction Wings to Recommender Systems with Conversational Agents
Authors: Jiarui Jin
Xianyu Chen
Fanghua Ye
Mengyue Yang
Yue Feng
Weinan Zhang
Yong Yu
Jun Wang
Venue: NeurIPS 2023 poster (2023)
Keywords: Conversational Agent
Recommender System
Conversational Recommendation
TL;DR: We propose CORE to incorporate a conversational agent into any recommendation platform in a plug-and-play style.

Abstract: An intelligent conversational agent (a.k.a., chat-bot) could embrace conversational technologies to obtain user preferences online, to overcome inherent limitations of recommender systems trained over the offline historical user behaviors. In this paper, we propose CORE, a new offline-training and online-checking framework to plug a COnversational agent into REcommender systems. Unlike most prior conversational recommendation approaches that systemically combine conversational and recommender parts through a reinforcement learning framework, CORE bridges the conversational agent and recommender system through a unified uncertainty minimization framework, which can be easily applied to any existing recommendation approach. Concretely, CORE treats a recommender system as an offline estimator to produce an estimated relevance score for each item, while CORE regards a conversational agent as an online checker that checks these estimated scores in each online session. We define uncertainty as the sum of unchecked relevance scores. In this regard, the conversational agent acts to minimize uncertainty via querying either attributes or items. Towards uncertainty minimization, we derive the certainty gain of querying each attribute and item, and develop a novel online decision tree algorithm to decide what to query at each turn. Our theoretical analysis reveals the bound of the expected number of turns of CORE in a cold-start setting. Experimental results demonstrate that CORE can be seamlessly employed on a variety of recommendation approaches, and can consistently bring significant improvements in both hot-start and cold-start settings.

Lending Interaction Wings to Recommender Systems with Conversational Agents
Jiarui Jin l,, Xianyu Chen', Fanghua Ye?, Mengyue Yang", Yue Feng",
Weinan Zhang -1, Yong Jun Wang2
Shanghai Jiao Tong University, University College London {jinjiarui97, wnzhang, xi anyujun yyu) edu, on (fanghua. ye. 19, mengyue. yang. 20, yue. feng. 20, jun. wang ac .u
Abstract
Recommender systems trained offline historical user behaviors are embracing conversational techniques to online query user preference. Unlike prior conversa- tional recommendation approaches that systemically combine conversational and recommender parts through a reinforcement learning framework, we propose CORE, new tffliin-training and online-checking paradigm that bridges COnversational agent and REcommender systems via a unified uncertainty minimization framework. It can benefit any recommendation platform in alug-and-play style. Here, CORE treats recommender system as an offline relevance score estimator to produce an estimated relevance score for each item; while conversational agent is regarded an online relevance score checker to check these estimated scores in each session. We define uncertainty the summation of unchecked relevance scores. In this regard, the conversational agent acts to minimize uncertainty via querying either attributes items. Based on the uncertainty minimization framework, we derive the expected certainty gain of querying each attribute and item, and develop a novel online decision tree algorithm decide what to query each turn. We reveal that CORE can be extended to query attribute values, and we establish a new Human-Al recommendation simulator supporting both open questions querying attributes and closed questions of querying attribute values. Experimental results on industrial datasets show that CORE could be seamlessly employed on popular recommendation approaches, and can consistently bring significant improvements, compared against either recently proposed reinforcement learning-basee or classi- cal statistical methods, in both hot-start and cold-start recommendation settings. We further demonstrate that our conversational agent could communicate as human if empowered by pre-trained large language model, c.g., gpt-3. 3.t.3..-turbb
Introduction
Recommender systems are powerful tools to facilitate users' information seeking [28, 38, 4, 20, 22, 21, 52, 51]; however, most prior works solely leverage offline historical data to build recommender system. The inherent limitation of these recommendation approaches lies in their offline focus on users' historical interests, which would not always align with users' present needs. As intelligent conversational assistants (a.k.a,, chat-bots such as ChatGPT and Amazon Alexa, have entered the daily life of users, these conversational techniques bring unprecedented opportunity to online obtain users' current preferences via conversations. This possibility has been envisioned as conversational
Work done during Jiarui Jin's visit University College London. Coorreppnndenc:: Weinan Zhang, Jun Wang.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

An Example of CORE: an Offline-training and Online-checking Framework
(a) Matrix Candidate Items with Offline
(b) Online Decision Tree Decide What to Query [Either an Attribute an Item)
Estimated Relevance Scores
My offline estimated relevanee scores
0.
is
0..
O
Query Breakfast Service
P
A
build an online decision tree according to your estimation.
Yes
Conversatonal Agant
99
3
N
0.1
uoppls appasnsiinn
Query Hotel
Query
P
5
Y
0.4
Do you raquire Breakfast Service?
Yes
Ves
Yes.
Cheers!
No
No
5
Y
N
0.4
recommmed Hotel B for you
Query Hotel
A
O
Query Hotel D
Yes
No
Yes
No
3
N
0.1
Great, Thank you
Cheers!
Cheers!
No Item Found
No Found
I will offline update estimations for Hotel A, Hotel B, Hotel C, Hotel
M
An illustrated example of CORE, effline-training and online-checking framework, where recom- mender system operates as an offline relevance score estimator (colored in green), while conversational agent acts as an online relevance score checker (colored in blue). Concretely, given a matrix of candidate items, as shown in (a). the recommender system could offline assign an estimated relevance score to each item, and then the conversational agent would online check these scores by querying either items or attributes, depicted in (b).
Figure I:
recommender systems and has inspired a series of conversational recommendation methods [26, 30, 44]. Unfortunately, all of these approaches try to model the interactions between users and systems using a reinforcement learning-basse based framework, which inevitably suffers from data insufficiency and deployment difficulty, because most recommendation platforms are based on supervised learning.
In this paper, we propose CORE that can bridge a COnversational agent and REcommender systems in plug-and-play style. In our setting, conversational agent can choose either to query (a.k.a., to recommend) an item (e.g., Hotel A) or to query an attribute (e.g., Hotel Level), and the user should provide their corresponding preference. Here, the goal of the conversational agent is to find (a.k.a., to query) an item that satisfies the user, with a minimal number of interactions.
We formulate the cooperation between conversational agent and recommender system into a novel and online-checking framework. Specifically, CORE treats a recommender system as an offline relevance score estimator that offline assigns a relevance score to each item, while a conversational agent regarded as an online relevance score checker that online checks whether these estimated relevance scores could reflect the relevance between items and the user's 's current needs. Here, "checked items" means those items, we can certainly say that they can not satisfy the user according to already queried items and attributes. We introduce a new uncertainty metric defined as the summation of estimated relevance scores of those unchecked items. Then, the goal of our conversational agent can be formulated as minimizing the uncertainty via querying items attributes during the interactions. To this end, we derive expected certainty gain to measure the expectation of uncertainty reduction by querying each item and attribute. Then, during each interaction, our conversational agent selects an item or an attribute with the maximum certainty gain, resulting in an online decision tree algorithm. We exemplify the above process in Figure 1.
Notice that users usually do not hold clear picture of their preferences on some attributes (i.e.. attribute IDs), e.g., what Hotel Level they need, instead, they could have clear preference on a specific value of an attribute (i.e., attribute value), e.g., Hotel Level=5 is too expensive for a student user. Also, asking an open question of querying attributes could result in an unexpected answer, c.g., a user answers 3. 5.5 to Hotel Level. In this regard, querying attribute values leading to closed questions (i.c., Yes or No questions) could be a better choice. We reveal that CORE could be directly applied to the above querying strategies. We also develop a new Human-A Al recommendation simulator that supports both querying attributes and attribute values.
In practice, we extend CORE to handle continuous attributes and to consider the dependence among attributes. Moreover, we demonstrate that our conversational agent could straightforwardly be em- powered by a pre-trained language model, e.g., 3.t-3.5-turbe. to communicate as a human. Note that CORE poses no constraint on recommender systems, only requiring the estimated relevance scores. Therefore, CORE can be seamlessly applied to any recommendation platform. We conduct experiments on 8 industrial datasets (including both tabular data, sequential behavioral data and graph-

---

data) with 9 popular recommendation approaches (e.g., DeepFM [20], DIN [52]). Experimental results show that CORE can bring significant improvements in both hot-start recommendation (i.e., the recommender system is offline trained) and cold-start recommendation (i.e., the recommender system not trained) settings. We compare CORE against recently proposed reinforcement learning based methods and classical statistical methods, and CORE could consistently show better performance.
2 Bridging Conversational Agents and Recommender Systems
2.1 Problem Formulation
Let denote a set of users, V = {ui,..., far} be a set of M items, X' {21, ... NN be a set of N attributes (a.k.a., features) of items. We consider a recommender system as mapping function, denoted as VR : U x V- + R that assigns an estimated relevance score to each item regarding a user. Then, during each online session, conversational agent also can be formulated as mapping function, denoted as co : l x - R that chooses either an item or an attribute to query user, and A = V/u U X denotes the action space of the conversational agent. For convenience, we focus on the most favorite item of the given user each session. In other words, our goal is to identify the item that will receive the first click when all items are equally presented. We call this item as larget item.
For this purpose, (-) acts as an offline estimator to produce an estimated relevance distribution of items regarding each user through offline training on previous behavioral data; while y (-) operates as an online checker to check whether these estimated scores fit the user's current needs through online interactions. Here, "checked items" denote those items that can not be the target item according to queried items and attributes. For example, as Figure I illustrates, after querying Breakfast Service, we have items Hotel C and Hotel D checked. We introduce uncertainty as the summation of estimated relevance scores of unchecked items. Formally, we have:
Definition (Uncertainty and Certainty Gain). For the k-th turn, we define uncertainty, denoted as Up, to measure how many estimated relevance scores are still unchecked, i.e.,
Ug+ == SUM( ( "aaE(Um))unn E Vv+),,
(1)
where VnE(Um) outputs the estimated relevance score for item Ums Vi is the set of all the unchecked items after k interactions, and V4 is initialized as Vo = V. Then, the certainty gain of k-th interaction is defined as AU, == U Un i.e., how many relevance scores are checked at the k-th turn. Since our goal is find the target item, if Vco(-) successfully finds one the k-th turn, then set all the items checked, namely Ug= = O.
In this regard, our conversational agent is to minimize Up at each k-th turn via removing those checked items from V/g. Considering that online updating (nz(-) is infeasible in practice due to the high latency and computation costs, the objective of Co can be expressed as:
min K, s.L., Ux O, Wi
(2)
where K is the number of turns, and V means that the recommender system is frozen. To this end, the design of our conversational agent could be organized as an uncertainty minimization problem.
2.2 Comparisons to Previous Work
Bridging conversational techniques and recommender systems has become an appealing solution to model the dynamic preference and weak explainability problems in recommendation task [15, 26]. where the core sub-task is to dynamically select attributes to query and make recommendations upon the corresponding answers. Along this line, the main branch of previous studies is to combine the conversational models and the recommendation models from a systematic perspective. Namely conversational and recommender models are treated and learned as two individual modules [30, 2, 44, 48] in the system. The system is developed from a single-turn conversational recommender system [5, 6, 47] to a multiple-turn one [49]. To decide when to query attributes and when to make recommendations (i.e., query items), recent papers [30, 31, 44, 30] develop reinforcement
In this paper, our conversational agent only faces one user u E 24 in each session, we omit the input in mapping functions V. (-)s for simplicity.
3

---

learning-basee -aaind solutions, which are innately suffering from insufficient usage of labeled data and high complexity costs of deployment. However, reinforcement learning approaches often demand substantial number of training samples, a condition referred to as data insufficiency, and require operating within a relatively compact action space. Instead, our conversational agent can be regarded as generalist agent that can query either items or attributes. In addition, our querying strategy is derived based on the uncertainty minimization framework, which only requires estimated relevance scores from the recommender system. Hence, CORE can be straightforwardly applied to any supervised learning-basee recommendation platform, in a plug-and-play way, free of meticulous reward function design in reinforcement learning methods. CORE is more friendly open- open-world world scenarios especially when training data is limited.
We present the connections to other previous work (e.g., decision tree algorithms) in Appendix A5.
3 Making the Conversational Agent a Good Uncertainty Optimizer
3.1 Building an Online Decision Tree
As described in Section 2.1, the aim of our conversational agent is to effectively reduce uncertainty via querying either items or attributes. The core challenge is how to decide which item or attribute to query. To this end, we begin by introducing expected certainty gain to measure the expectation of how much uncertainty could be eliminated by querying each item and each attribute. Then, we can choose an item or an attribute with the maximum expected certainty gain to query.
Formally, let Xg denote the set of unchecked attributes after k interactions. Then, for each k-th turn, we define Aquery as an item or an attribute to query, which is computed following:
arg max Woa(query(a)),
Qquery
where (-a(-) denotes the expected certainty gain of querying a, and a can be either an unchecked item (from V4-1) or an unchecked attribute (from xx1)).
I (-) for Querying an Item. We first consider the case where E VA-1- Let e* denote the target
item in the session. Since we only need to find one target item, therefore, if a E VR 1, we can derive:
You Vos(query(a) =²) Pr((a =t)) + Voo(a + v²) Pr * v*)
Σ Vne(um)) pr(a= w*) + (a)(a) Pr(a + v*).
(4)
On VV
where a = 0² and * v" denote that queried is the target item and not. If a v², the session is done, and therefore, the certainty gain (i.e., Mo o²)) is the summation of all the relevance scores in VA-1- Otherwise, only is checked, and the certainty gain (i.e., la o vz) is the relevance score of a, and we have V = Ma) and X = X
Considering that being the target item means being the most relevant item, we leverage the user's previous behaviors to estimate the user's current preference. With relevance scores estimated by (RE(.), we estimate Pr(a- = 0*) as:
RR (a) Pr(a U*) SUM( V (mm) Um E Vk-
(5)
and Pr(a v*) 1- 1 o*).
(.) for Querying an Attribute. We then consider the case where a E 24-1. For each queried attribute a, let Wi denote the set of all the candidate attribute values, and let w W. denote the user preference on a, e.g., a is Hotel Level, W, is 3. Then, if a E 2 we have:
(ca(query(a))- Σ We EW.
(Voaww. = Pr(wa = =w²)
(6)
where w. = " means that when querying a, the user's answer (represented by wx)) wa. Voa (wa w:)) is the certainty gain when w. = " happens, and Pr(w.- = w")) is the probability of wa w occurring. w. = w.. holds, then all the unchecked items whose value of is not equal to wa should be removed from Vk-1. 1. as they certainly not satisfying the user's 's needs.

---

Formally, let Vanim= denote the set of all the items whose value of is equal to wa and let V Vonten of denote the set of rest items. Then, (en(wa wa) can be computed as:
(waa = wx) SUM( (eee(u (Mm)) E Vk-1 Va }),
(7)
which indicates that the certainty gain, when wa is the user's answer, is the summation of relevance scores of those items not matching the user preference.
To finish ca (cuequery())) we also need to estimate Pr(w, wy). To estimate the user preference on attribute a, we leverage the estimated relevance scores given by WRE( (.) as:
Pr(wa- = = = SUM( ( (tm)))))))))) Hum E Vx-1 n Va,am==..)
(8)
SUM( ( (V))(Um) emm Vk-1))
In this case, we remove VR-1 n V Fer from Vk-1, namely we have Vy- = VA-1 Vensattt As attribute a is checked, we have Ai-- = 1 Here, Wy is provided by the user after querying a.
By combining Eqs. (4), (6), and (7), we can derive a completed form of Va Van (qu(ee)y((a) for a Vk-1 U - (See Appendix A1.1 for details). Then, at each k-th turn, we can always follow Eq. (3) to obtain the next query 'query- As depicted in Figure (bb), the above process results in online decision tree, where the nodes in each layer are items and attributes to query, and the depth of the tree the number of turns (see Appendix A4.3 for visualization of real-world case).
3.2 From Querying Attributes to Querying Attribute Values
We note that the online decision tree introduced above is general framework; while applying it to real-world scenarios, there should be some specific designs.
L (-) for Querying an Attribute Value. One implicit assumption in the above online decision tree that the user's preference on queried attribute always falls into the set of attribute values, namely w E Wa holds. However, it can not always hold, due to (i) a user would not have clear picture of an attribute, (ii) user's answer would be different from all the candidate attribute values, e-g.. a is Hotel Level, " 3.5, and W. f3. 5), as shown in Figure ((a). In these cases, querying attributes would not be good choice. Hence, we propose to query attribute values instead of attribute IDs, because (i) a user is likely to hold clear preference for specific value of an attribute, e.g.. a user would not know an actual Hotel Level of her favoring hotels, but she clearly knows she can not afford a hotel with Hotel Level=5, and (ii) since querying attribute values leads to closed questions instead of open questions, a user only needs to answer Yes or No, therefore, avoiding the user's answer to be out of the scope of all the candidate attribute values.
Formally, in this case, x E Xx In light of this, we compute the expected certainty gain of querying attribute value as:
Wa x X=11 which indicates we need to choose a value wi E W, where
oo (x) = w.) Mes( ww= = w Pr(w. = w + Mea(w, # w Pr(w. + wx). (9)
where = E W. denotes the user preference on attribute x. Here, different from querying attributes, a user would only respond with Yes (i.e., Wa = wx) or No (i.e., wx # w Therefore, we only need to estimate the certainty gain for the above two cases. Voa(w, w can be computed following Eq. (7) and cca * w²) can be calculated by replacing V Vinsutt with V V =w. Pr(w, wx) is estimated in Eq. (8) and Pr(w # wx) = Pr(w, w.). In this case, if all the values of x have been checked, we have Xx = (xr) otherwise, X'- = X and V2 = Va-1 Vznawtw, if receiving Yes from the user, V/- = VR-1 =ai.= otherwise.
We reveal the connection between querying attributes (i.e., querying attribute IDs) and querying attribute values in the following proposition.
Proposition 1. For any altribute x E Xa-1, Voo(query(a)) 2 Vic Von(query(z) -w)) holds for all the possible w, W..
This proposition shows that if users could give clear preference for the queried attribute and their preferred attribute value is one of the candidate attribute values, then querying attributes would be an equivalent or a better choice than querying attribute values. In other words, querying attributes and querying attribute values can not operate on the same attributes (otherwise, Ven(-) would always choose to query attributes). Therefore, we can combine querying items and querying attribute values
5

---

Algorithm 1 CORE for Querying Items and Attributes
Input: A recommender system VnE(+), an item set V. an attribute set X, an offline dataset D. Output: Updated recommender system VRE(.). up-to-date dataset D. D. Offline-Traininn
1: Train (-E(-) on D.
2: for each session (i.e., the given user) do
3: Initialize k = 1 and Vo- V, Xo X.
4: repeat
5:
Compute aquery following Eq. (3) for querying items and attributes or following Eq. (10) for querying items and attribute values.
Moline-Checking
6:
Query Aquery to the user and receive the answer.
Online ChalineCheeckigg Checcing
7:
Generate V2 and X₁. from V and Xz-1- Go next turn: k 4- k 1.
8:
9:
until Querying the target item or k > Knax where Keax is the maximal number of turns. Collect session data and add to D.
10:
11:
end for
Update VEE(-) using data in D.
Offline- Training
by setting the action space to A = Wz x Xx-1 U Va-1. Then, we can re-formulate Eq. (3) as:
To (onquery(()) = wz), max Voa(query(v))) (10)
Qquery = arg max af [ W. where -E
max
wEVe-1
In the context of querying attribute values, we further reveal what kind of attribute value is an ideal one in the following theorem.
Proposition 2. In the context of querying attribute values, an ideal choice is always the one that can partition all the unchecked relevance scores into two equal parts (i.e., the ideal W. E Wa, x E Xx is the one that makes V (w.- w") SUM( (V (fm) I8m Vx-13)/2 hold), if it is achievable. And the certainty gain in this case is cc (query(I) = wa) = SUM( WEEE (1m) (mm E Vx- 1//2.
Then, consider the bound of the expected number of turns. To get rid of the impact of (-E(.), we introduce a cold-start setting [43]. where VRE(.) knows nothing about the user, and equally assigns relevance scores all M items, resulting in VRE(Um) 1/M holds for any Un V.
Lemma 1. In the context of querying attribute values, suppose that aa (mm)) ) = 1 1/M holds for any Um V. then the expected number of turns (denoted as R) is bounded by log < R (M+1)/2.
Here, the good case lies in that our conversational agent is capable of finding an attribute value to form an ideal partition at each turn, while the bad case appears when can only check item at each turn. We provide detailed proofs of Propositions and 2, and Lemma in Appendix A1.2.
(o for Querying Attributes in Large Discrete Continuous Space. All the above querying strategies are designed in the context that for each attribute, the range of its candidate values is a "small" discrete space, namely [Wa|= << |V-il where X When it comes cases where W. a large discrete space or a continuous space, then either querying attribute x or any attribute value w, W. would not be good choice. For example, let be Hotel Price, then when querying r, the user would not respond with an accurate value, and querying r=one possible value could be ineffective. To address this issue, we propose generate new attribute value w, and query whether the user's preference is not smaller than it or not. Formally, we have:
Veo (quuee)))()) 2 wx) oo (w= 2 w: Pr(w wG)= + To Pwolw. < wx) .P),,, < <w), (11)
where x E X and W. can be either in or out of W,.. Compared to querying attribute values (i.e., Eq. (9)), the new action space is A = IR Xt-1. Notice that Proposition 2 is also suitable for this case (see detailed description in Appendix 12. where the best partition is to divide the estimated relevance scores into two equal parts. Therefore, we produce w., by averaging all the candidate attribute values weighted by the corresponding relevance scores. Formally, for each x E Xx-1, we compute We as:
Ue AVERAGE({ V (Um) Wem I' E Vx-13),
(12)
where Wo.,, is the value of attribute x in item Um. e.g., in Figure 1(a), let a be Hotel Level, and Fm be Hotel A, then Wo... = 3.

---

Query Breakfast
(a]
SF
Question Generator
Do you require Breakfast Service
(b)
Conversational Agent
Breakfast Service: Not
Care
Answer Extractor
do not care about Breakdast Service, and rreally want hotel with Shower.
Empowered
Hotel Level: Not Know
Shower: Yes
(c)
User
Conversational Agent
Chat-bot Empowered Conversational Agent
We illustrate the empowerment of our conversational agent through the utilization of a pre-trained chat-bot. In this context, the red box signifies the chal-bol-empowered conversational agent. To accomplish this, we input the output queries produced by the original conversational agent, such as Breakfast Service ce into the question generator, as depicted in (a). In (b), a user is expected to input the generated question in a free-text format and provide the corresponding answer in a free-text format. Subsequently, in (c), the answer extractor extracts key information from the user's response and provides it to the original conversational agent.
Figure 2:
In this case, X2 -1,,, and V Vk-1 \ Vonai <on if receiving Yes from the user when querying
whether user preference is not smaller than Wg. V/= = Vx-1 1 V Vaniie 2w, otherwise. Vanca w. is the set of all the items whose value of x is smaller than we and V 2w is the set of the rest items.
3.3 Plugging the Conversational Agent into Recommender Systems
Overall Algorithm. We begin by summarizing CORE for querying items and attributes or querying items and attribute values in Algorithm 1. From the algorithm, we can clearly see that our Van(.) puts no constraints on WRE(.) and only requires the estimated relevance scores from V.E(+), therefore, CORE can be seamlessly integrated into any recommendation platform. We note that in conversational agent, querying attributes and querying attribute values can be compatible, but can not simultaneously operate the same attribute, due to Proposition 1. See Appendix A2.3 for detailed discussion.
Making c.a(-) Consider Dependence among Attributes. We notice that the above formulations of either querying attributes or querying attribute values, does not consider the dependence among attributes (e.g., as Figure ((a) shows, attribute Hotel Level can largely determine attribute Shower Service). To address this issue, we take Vaa(.) in Eq. (6) as an example (see detailed descriptions of the other Voa(-) issinn in Appendix A2.2), and re- re-formulate it as:
Vor (quuery()))
Σ (Vaa(queryld') Pr(query(d))querr(d)) a'EX2-1
(13)
where a 1 and Pr(query(a") query(a)) measures the probability of the user preference on a determining the user preference on a'. Compared to Vo Waa(ques (aa(quer(aa), V/O (Qquer((a)) further considers the impact of querying attribute a on other attributes. To estimate Pr Pr(query(a (a') (quer((()) we develop two solutions. We notice that many widely adopted recommendation approaches are developed on factorization machine (FM) [38], e.g., DeepFM [20]. Therefore, when applying these FM-based recommendation approaches, one approach is to directly adopt their learned weight for each pair of attributes (a, a') as the estimation of Pr(query (a")) (a'))query(aa). When applying CORE to any other recommendation method (e.g., DIN [52]). we develop a basedd approach that does estimations by computing this conditional probability &r (a((query(a)) based on the given candidate items. We leave the detailed computations of VD (query(u)) in both ways in Appendix A2.2.
Empowering Veo(.) to Communicate with Humans. When applying CORE into real-world scenar- ios, users may provide a Not Care attitue regarding the queried attributes or queried attribute values. In these cases, we generate Vμ and X₂ by V4- = V4-1 and X' Xx-1 Ma). because querying is non-informative. To capture the user's different attitudes on queried items and attributes or attribute values, we can incorporate pre-trained large language model (LLM) (c.g., (.). As our cause the -idiifferrentable issue. In light of this, we exemplify some task-specific prompts to enable the conversational agent to (i) produce questions by prompting queried items and attributes, and (ii) extract the key messages from the user's answers. As shown in Figure 2, a conversational agent consisting of an L--M-chtt-bot and our online decision tree algorithm would communicate like humans. We also provide some case studies of the conversational agent the context of a question generator and answer extractor. See Appendix A4.2 for a detailed description.
5..--turbo) in part does not require training, simply plugging an LLM would not
7

---

4 Experiments
4.1 Experimental Configurations
We summarize different experimental settings as follows. (i) We design two different quering strategies regarding attributes (shown line 5 in Algorithm 1). One is querying attributes (i.e.. attribute IDs); and the other is querying attribute values. (ii) We introduce two different recommender system settings. One is the hot- start setting (shown in line in Algorithm 1) that initializes the estimated relevance scores of items by a given pre-trained recommender sys- tem; and the other is the cold-start setting where those estimated relevance scores uniformly generated (corresponding to the case where the recommender system knows nothing about the given user). Because the conversational agent (+) operates in dynamic process, we de-
Table I: Results comparison in the context of querying attributes. See Table Al for the full version.
(.n(()
Ven(-)
Amazon
T@3
S@3
T@5
S@5
ME
3.04
0.98
5.00
1.00
COLD START
CORE
2.88
1.00
2.87
1.00
CORE,
2.84
1.00
2.86
1.00
AG
2.76
0.74
2.97
0.83
CRM
3.07
0.98
3.37
1.00
EAR
2.98
0.99
3.13
1.00
FM
CRIF
2.84
1.00
2.64
1.00
UNICORN
2.65
1.00
2.45
1.00
CORE
2.17
1.00
2.16
1.00
CORE
2.14
1.00
2.14
1.00
velop a new simulator to simulate the Human-AI recommendation interactions, which consists of a conversational agent and user agent. Specifically, for each user, we use her browsing log as session data, and treat all the items receiving positive feedback (e.g., chick) as target items. Then, for each k-th turn, when the conversational agent queries an attribute E - the user agent returns a specific attribute value if all the target items hold the same value for æ;; otherwise, the user agent returns Not Care. When the conversational agent queries an attribute value w,, E Wa, the user agent returns Yes if at least one target item holds wa the value of attribute otherwise, returns No.
For each experimental setting, we first set Knxx and then evaluate the performance in terms of the average turns needed to end the sessions, denoted as T@ KuAx (where for each session, if (-) successfully queries the target item within KNXX turns, then return the success turn; otherwise, we enforce ce (-) to query an item at (Kax + 1) turn, if succeeds, return Keax + 1, otherwise return KNax + 3); and the average success rate, denoted as S@ KHAX (where for each session, if co does not successfully query the target item within Knax turns, then we enforce c-)(-) to query an item after Knax turns, succeeds, return 1, otherwise return 0).
To verify CORE can be applied to a variety of recommendation platforms, we conduct evaluations on three tubular datasets: Amazon [8, 331, LastFM [9] and Yelp [12], three sequential datasets: Taobao [10]. Tmall [11] and Alipay [7], two graph-structured datasets: Douban Movie [35, 53] and Douban
Book [35, 53]. The recommendation approaches used in this paper, i.c., R-) include FM [38],
DEEP FM [20]. PNN [37], DIN [52], GRU [23]. LSTM [18], MMOE [32], GCN [27] and GAT [46]. We also use COLD START to denote the cold-start recommendation setting. The conversational methods used in this paper, i.e., (.a(,)s, include (i) Abs Greedy (AG) always queries an item with the highest relevance score at each turn: (ii) Max Entropy (ME) always queries the attribute with the maximum entropy in the context of querying attributes,
queries the attribute value of the chosen attribute, with the highest frequency in the context of querying attribute values; (iii) CRM [44], (iv) EAR [30], (v) CRIF [25], (vi) UNICORN [13]. Here, AG can be regarded as a strategy of solely applying WRE(.). Both CRM and EAR are reinforcement learning based approaches, originally proposed on the basis of FM recommender system. Thus, we also evaluate their performance with hot-start FM-based recommendation methods, because when applying them to a cold-start recommendation platform, their strategies would reduce to a random strategy. Consider that ME is (+), independent of WEE(.) (namely, the performance of hot-start and cold-start recommendation settings are the same); and therefore, we only report their results in the cold-start recommendation setting. We further introduce a variant of CORE, denoted as CORE where we compute and use & (-)s instead of Ven(-) in line 5 in Algorithm 1.
We provide detailed descriptions of datasets and data pre-processing, simulation design, baselines, and implementations in Appendix A3.1, A3.2, A3.3, and A3,4.
8

---

Table 2: Results comparison of querying attribute values on tabular datasets. See Table A2 for the full version.
[ml()
Pen(-)
Amazon
LastFM
Yelp
T@3
S@3
T@5
S@5
T@3
S
T@S
S@5
T@3
S@3
T@5
S@5
AG
6.47
0.12
7.83
0.23
6.77
0.05
8.32
0.14
6.65
0.08
8.29
0.13
COLD START
ME
6.50
0.12
8.34
0.16
6.84
0.04
8.56
0.11
6.40
0.15
8.18
0.20
CORE
6.02
0.25
6.18
0.65
5,84
0.29
5.72
0.74
5,25
0.19
6.23
0.65
CORE,
6,00
0.26
6.01
0.67
5.79
0.30
5,70
0.75
5.02
0.21
6.12
0.68
AG
2.76
0.74
2.97
0.83
4.14
0.52
4.67
0.64
3.29
0.70
3.39
0.81
4.58
0.28
0.38
4.23
0.34
5.87
4.12
0.25
0.69
EAR
4.13
0.32
6.32
0.42
4.02
0.38
5.45
0.67
4.10
0.28
5.95
0.72
CRIF
4.34
0.28
6,24
0.45
3.98
0.58
4.11
0.76
4.07
0.31
6.02
0.70
FM
UNICORN
4.43
0.30
0.52
4.00
0.54
7.56
4.40
0.25
5.38
0.77
CORE
3.26
0.83
3.19
0.99
3.79
0.72
3.50
0.99
3.14
0.84
3.20
0.99
CORE
3.16
0.85
3,22
1.00
3.75
0.774
3.53
1,00
3.10
0.85
3.23
1.00
AG
3.07
0.71
3,27
0.82
3.50
0.68
3.84
0.79
3.09
0.74
3.11
0.88
DEEP
4.51
0.29
0.40
4.18
5.88
4.11
0.23
0.71
EAR
4.47
0.30
6.35
0.43
4.01
0.37
5,43
0.69
4.01
0.32
5.74
0.75
CORE
3.23
0.85
3,22
0.99
3.47
0.81
3.34
1.00
2.98
0.93
3.11
1.00
PNN
AG
3.02
0.74
3,10
0.87
3.44
0.67
3.53
0.84
2.83
0.77
2.82
0.91
CORE
3.01
0.88
3.04
0.99
3.10
0.87
3.20
0.99
2.75
0.88
2.76
1.00
Table Results comparison querying attribute values on sequential datasets. See Table A3 for the full version.
Vse
Voo(-)
Taobao
Tmall
Alipay
T@3
S@3
T@S
S@5
T@3
S@3
T@5
S@5
T@3
S@3
T@5
S@S
AG
6.30
0.15
7.55
0.27
6.80
0.04
8.54
0.09
6.47
0.11
7.95
0.19
COLD START
ME
6.43
0.14
7.82
0.29
6.76
0.05
8,50
0.12
6.71
0.07
8.46
0.11
CORE
5.42
0.39
5.04
0.89
6.45
0.13
7.38
0.37
5.98
0.25
6.17
0.65
AG
2.71
0.85
2.83
0.95
4.14
0.51
4.81
0.59
3.10
0.82
3.35
0.85
DIN
CORE
2.45
0.97
2.54
1,00
4.12
0.64
4.16
0.89
3.25
0.83
3.32
0.96
AG
2.80
0.80
2.64
0.97
3.82
0.56
4.40
0.64
3.17
0.83
3.29
0.87
GRU
CORE
2.31
0.98
2.44
1.00
3.81
0.72
3,91
0.92
3.10
0.84
3.11
0.96
AG
2.60
0.85
2.52
0.97
4.73
0.41
5.63
0.49
3.43
0.78
3.27
0.89
LSTM
CORE
2.37
0.97
2.49
1.00
4.58
0.55
4.36
0.90
3.03
0.84
3.16
0.97
AG
3.04
0.75
2.98
0.92
4.10
0.54
4.56
0.62
3.58
0.83
3.90
0.92
MMOE
CORE
2.48
0.96
2.60
1.00
3.92
0.65
4.19
0.85
3.21
0.91
3.17
0.98
4.2
Experimental Results
We report our results of querying attributes and items in Table 1, and the results of querying attribute values and items in Tables 2, and 3, 4. and summarize our findings as follows.
Reinforcement learning based methods work well in querying items and attributes but perform poorly in querying items and attribute values. By comparing Table I to Table 2, performance reduction of CRM and EAR. One possible explanation is that compared to attribute IDs, the action space of querying attribute values is much larger. Thus, it usually requires much more collected data to train a well-performed policy.
huge
T@ Krax could not align with S@ KrAx- A higher success rate might not lead to fewer turns, and ME gains a worse performance than AG in some cases in the cold-start recommendation setting. The main reason is that although querying an attribute value can obtain equivalent more certainty gain than querying an item at most times, however, only querying (a.k.a., recommending) an item could end session. Therefore, sometimes, querying an attribute value is too conservative. It explains why CORE outperforms AG in terms of $@3 but gets a lower score of T@3 in the Amazon dataset and FM recommendation base.
Our conversational agent can consistently improve the recommendation performance terms of suecess rate. CORE can consistently outperform AG, in terms of success rate, especially for the coad-start recommendation setting. As AG means solely using recommender systems, it indicates that c- can consistently help V/ RE One possible reason is that our uncertainty minimization

---

Table 4: Results comparison of querying attribute values graph datasets. See Table A4 for the full version.
Douban Movie
Douban Book
Vaz(-)
co
T@3
S@3
T@5
S@5
T@3
S
T@5
S@5
AG
6.52
0.11
7.94
0.21
6.36
7.68
0.26
COLD START
ME
6.60
0.10
8.16
0.21
6.40
0.15
8.04
0.24
CORE
5.48
0.38
4.84
0.94
5.96
0.26
5.08
0.92
AG
3.75
0.63
3.65
0.87
3.56
0.64
3.41
0.87
GAT
CORE
2.89
0.91
2.97
1.00
2.80
0.92
2.91
1.00
AG
3.21
0.69
3.33
0.83
3.20
0.71
3.18
0.89
GCN
CORE
2.76
0.92
2.81
1.00
2.85
0.91
2.85
1.00
Table Result comparisons in the context of querying attribute values and items tabular datasets, where reduce the 50% training data of FM and DEEP FM.
Man(
Pcn(-)
Amazon
LastFM
T633
S
T@5
S
T683
S@3
T@5
S
CRIF
5.14
0.18
6.78
0.41
4.98
0.33
6.34
0.56
UNICORN
5.13
0.23
6.89
0.42
5.01
0.34
5.94
0.45
FM
CORE
3,86
0.73
3,98
0.80
3,99
0.65
4.04
0.86
CRIF
5.42
0.39
5,98
0.57
4.61
0.53
4.46
0.67
DEEP FM
UNICORN
4.34
0.45
5.12
0.62
4.35
0.45
4.67
0.65
CORE
3.96
0.78
4.23
0.85
3.75
0.72
3.83
0.86
framework unifies querying attribute values and items. In other words, AG is special case of CORE, where only querying items is allowed.
Considering Dependence among attributes is helpful. Comparisons between CORE and CORE, reveal that considering the dependence among attributes could improve the performance of CORE in most cases.
We further investigate the impact Kyax by assign- ing Kидх 1, 5, 7.9 and reporting the results of CORE and AG on Amazon dataset in the context of the cold-start and hot-start recommendation settings in Figure 3. The results further verify the superiority of CORE, especially with cold-start (-)
Success nate
0.6 04
Our conversational agent is more stable when the training data is limited. For further assessment of the baseline meth-
CORE's stability in comparison ods, we conducted an evaluation by randomly select- ing a subset of the training set comprising only 50% of the samples. The results of this evaluation are sented in Table 5. The table clearly demonstrates that CORE exhibits a higher level of stability in its per- formance when compared to existing reinforcement beasing-bbseed frameworks.
DOME COLD
AGLCOLD
Figure 3: Comparisons of CORE and AG with dif-
ferent Kux in both cold-start and hot-start settings.
We also provide a case study of incorporating an LLM into CORE to handle free-text inputs and output human language Appendix A4.2, where we provide detailed prompts. We further offer a visualization of online decision tree in A4.3.
5 Conclusions and Future Work
In this paper, we propose CORE that can incorporate conversational agent into any recommendation platform in a plug-and-play fashion. Empirical results verify that CORE outperforms existing rein- forcement baasdd and statistics-based approaches in both the setting of querying items and attributes, and the setting querying items and attribute values. In the future, it would be interesting to evaluate CORE in some online real-world recommendation platforms.
10

---

Acknowledgement. The Shanghai Jiao Tong University team is supported by National Key R&D Program of China (2022ZD01 14804), Shanghai Municipal Science and Technology Major Project (2021SHZDZZ0112) and National Natural Science Foundation of China (62076161, 62177033). Jiarui Jin would like to thank Wu Wen Jun Honorary Doctoral Scholarship from Al Institute, Shanghai Jiao Tong University.
References
[1] Deepak Agarwal and Bee- Che-Chung Chen. Regression-base based latent factor models. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. pages 19-28, 2009.
[2] Keping Bi, Qingyao Ai, Yongfeng Zhang, and W Bruce Croft. Conversational product search based on negative feedback. In Proceedings of the 28th aem international conference on information and knowledge management, pages 359-368, 2019.
[3] Haibin Chen, Qianli Ma, Zhenxi Lin, and Jiangyue Yan. Hierarchy- aware label semantics matching network for hierarchical text classification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 1ihh International Joint Conference on Natural Language Processing (Volume Long Papers), pages 4370-4379, 2021.
[4] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, pages 7-10, 2016.
[5] Konstantina Christakopoulou, Alex Beutel, Rui Li, Sagar Jain, and Ed H Chi. Q&r: A two- stage approach toward interactive recommendation. In Proceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pages 139-148, 2018.
[6] Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. Towards conversational recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 815-8244 2016.
[7] Alipay Dataset. http::/waacaaaaiii.cccmmmm//u/at
[8] Amazaon Dataset. http:://nnaaullyyuussddddddduaan
[9] LastFM Dataset. http:///goooooassogggddaassscccc
[10] Taobao Dataset. http://aanaaaaaggnooooona/aaaaaa
[11] Tmall Dataset. http://iaaaaaaaauucccoc/a//a//et
[12] Yelp Dataset. https//wwww.yepp.oo/aaaass
[13] Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. Unified conversational recom- mendation policy learning via graph-based reinforcement learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1431-1441, 2021.
[14] Mehdi Elahi, Francesco Ricci, and Neil Rubens. A survey of active learning in collaborative filtering recommender systems. Computer Science Review, 20:29- 50, 2016.
[15] Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. Advances and challenges in conversational recommender systems: A survey. Al Open, 2:100-126 126, 2021.
[16) Nadav Golbandi, Yehuda Koren, and Ronny Lempel. On bootstrapping recommender systems. In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 1805-1808, 2010.
[17] Nadav Golbandi, Yehuda Koren, and Ronny Lempel. Adaptive bootstrapping of recommender systems using decision trees. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 595-604, 2011.

---

[18] Alex Graves and Alex Graves. Long short-term memory. Supervised sequence labelling with recurrent neural networks, pages 37-45, 2012.
[19] Asela Gunawardana and Christopher Meek. Tied boltzmann machines for cold start recommen- dations. In Proceedings of the 2008 ACM conference on Recommender systems, pages 19-26, 2008.
[20] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: ciariiaatio-macchin based neural network for ctr prediction. arXiv preprint 173.:21703.04277. 2017.
[21] Xiangnan He, Kuan Deng. Xiang Wang. Yan Li, Yongdong Zhang, and Meng Wang. Lightenn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 339-648, 2020.
[22] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173-182, 2017.
[23] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv: (tGiv:///100993399 2015.
[24] Badr Hssina, Abdelkarim Merbouha, Hanane Ezzikouri, and Mohammed Erritali. A comparative study of decision tree id3 and c4. 5. International Journal of Advanced Computer Science and Applications, 4(2):13-19, 2014.
[25] Chenhao Hu, Shuhua Huang, Yansen Zhang, and Yubao Liu. Learning to infer user implicit preference in conversational recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 256-266, 2022.
[26] Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. A survey on conversational recommender systems. ACM Computing Surveys (CSUR), 14(5))1-36, 2021.
[27] Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. arXiv preprint /009:0::600000900000 2016.
[28] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, 42(8):30-33 2009.
[29] Xuan Nhat Lam, Thuc Vu, Trong Due Le, and Anh Due Duong. Addressing cold-start problem in recommendation systems. In Proceedings of the 2nd international conference on Ubiquitous information management and communication, pages 208 211, 2008.
[30] Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong. Min-Yen Yen Kan, and Tat-Seng Chua. Estimation-acto cetiiiioonaaiion-oooo reflectionn Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 304-312, 2020.
[31] Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. Towards deep conversational recommendations. Advances in neural information processing systems, 31, 2018
[32] Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi-gate mixture-of-expers. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1930-1939, 2018
[33] Julian McAuley and Alex Yang. Addressing complex and subjective product-related queries with customer reviews. Proceedings of the 25th International Conference on World Wide Web, pages 625-635, 2016.
12

---

[34] Nima Mirbakhsh and Charles X Ling. Improving top-n recommendation for cold-start users via cross-domain information. ACM Transactions on Knowledge Discovery from Data (TKDD), 14):1-19, 2015.
[35] Douban Movie and Douban Book Datasets, http://///wwwoaaegeeeo/oaaoaaaaa wtwpaatnnnneieeiiidddiinaaaaaaae
[36] Seung-Taek Park and Wei Chu. Pairwise preference regression for cold-start recommendation. In Proceedings of the third ACM conference Recommender systems, pages 21 21-28, 2009.
[37] Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu. Ying Wen, and Jun Wang. Product- based neural networks for user response prediction. In 2016 IEEE 16th international conference on data mining (ICDM), pages 149-11554 IEEE, 2016.
[38] Steffen Rendle. Factorization machines. In 2010 IEEE International conference data mining, pages 995-1000. IEEE, 2010.
[39] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint 12XXiv:2005.66: 2012.
[40] Neil Rubens, Mehdi Elahi, Masashi Sugiyama, and Dain Kaplan. Active learning in recom- mender systems. Recommender systems handbook, pages 809-846, 2015.
[41] Neil Rubens and Masashi Sugiyama. Influence-based collaborative active learning. In Proceed- ings of the 2007 ACM conference on Recommender systems, pages 145-148, 2007.
[42] Martin Saveski and Amin Mantrach. Item cold-start recommendations: learning local collective embeddings. In Proceedings of the 8th ACM Conference on Recommender systems, pages 89-96, 2014.
[43] Andrew Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. Methods and metries for cold-start recommendations. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, pages 253-260, 2002.
[44] Yueming Sun and Yi Zhang. Conversational recommender system. In The 41st international acm sigir conference on research & development in information retrieval, pages 235-244, 2018.
[45] Alex Tamkin, Kunal Handa, Avash Shrestha, and Noah Goodman. Task ambiguity in humans and language models. arXiv preprint arXiv:2212.10711 2022.
[46] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 2017.
[47] Tong Yu, Yilin Shen, and Hongxia Jin, A visual dialog augmented interactive recommender system. In Proceedings of the 25th ACM SIGKDD international conference knowledge discovery & data mining, pages 157-165, 2019.
[48] Xiaoying Zhang, Hong Xie, Hang Li, and John CS Lui. Conversational contextual bandit: Algorithm and application. In Proceedings of the web conference 2020, pages 662-672, 2020.
[49] Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th acm international conference on information and knowledge management, pages 177-186. 2018.
[50] Cheng Zhao, Chenliang Li, Rong Xiao, Hongbo Deng, and Aixin Sun. Catn: Cross- domain recommendation for cold-start users via aspect transfer network. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 239-238, 2020.
[51] Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for thick-through rate prediction. In Proceedings of the AAAI conference artificial intelligence, volume 33, pages 5941-5948. 2019.
13

---

[52] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan. Junqi Jin, Han Li, and Kun Gai. Deep interest network for dlck-throught rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1059-1068, 2018.
[53] Feng Zhu, Chaochao Chen, Yan Wang, Guanfeng Liu, and Xiaolin Zheng. Dtedr; A framework for dual-target cross-domain recommendation. In Proceedings the 28th ACM International Conference on Information and Knowledge Management, pages 1533 1542, 2019.
14

---

A1
Conversational Agent Design on Uncertainty Minimization Framework
A1.1 Detailed Deviations
This paper introduces conversational agent built upon the recommender system to interact with human user. We begin by summarizing the interaction principles, taking Figure as an example.
Definition A1 (Conversational Agent and Human User Interactions). Our conversational agent designed to act in the following four ways.
(i) Query an item u E Vk-1, where Vk-1 the set of unchecked items after 1 interactions (e.g.. recommend Hotel A to the user).
(i) Query an attribute (e.g., query what Hotel Level does the user want). (ii) Query whether the user's preference value W2 E W2 where Wa is the set of values of attribute x (e.g.. query whether the user likes a hotel with Hotel Level=5).
2x-1, where Xx-1 is the set of unchecked attributes after k- interactions
an attribute Rg-1 is equal specific attribute
(iv) Query whether the user's preference on an attribute is not smaller than a specific value Ws E IR (e-g., query whether the user likes a hotel with Hotel Level23.5).
The human user is supposed to respond in the following ways.
(i) For queried item v, the user should answer Yes (if satisfies the user) or No (otherwise) (e.g.. answer Yes, if the user likes Hotel A).
(i) For queried attribute x, the user should answer her preferred attribute value, denoted as w E Wa (e.g,, answer 3 for queried attribute Hotel Level), or answer Not Care to represent that any attribute value works.
(iii For queried attribute value Wa. the user should answer Yes (if wa matches the user preference) or No (otherwise) (e.g., answer Yes, if the user wants a hotel with Hotel Level=5), or answer Not Care to represent that any attribute value works.
(iv) For queried attribute value Wg. the user should answer Yes (if the user wants an item whose value of altribute x is not smaller than wa) or No (otherwise) (e.g., answer Yes, if the user wants Not Care represent that any attribute value works.
a hotel with Hotel Level=5),
Then, we separately describe the key concepts, including uncertainty, certainty gain, and expected certainty gain, introduced this paper.
Definition A2 (Uncertainty). For the k-th turn, we define uncertainty, denoted as Up to measure how many estimated relevance scores are still unchecked, which can be formulated as:
Ux- == SUM (Wr(Um))Em E Vva)).
(14)
where V (V)( outputs the estimated relevance score for item Vm- The above equation tells us that the uncertainty of each turn is decided by the unchecked items.
It is straightforward to derive the certainty gain, as the uncertainty reduction each turn.
Definition A3 (Certainty Gain). For the k-th turn, we define certainty gain of k-th interaction as:
AU := Uk-1 U SUM( Ven (vm) Um E AVv))):
(15)
where AV, V4-1 VR. For simplicity, we use a to denote the k-tth action of the conversational agent. According to the Human-Al interactions introduced in Definition Al, we can derive:
VE: {a},
a E Vk-1 and the answer to querying (i) is Yes,
E Vx R and the answer querying (i) is No,
V
Vaniia #0ng
V4-1:
a E X and the answer to querying (ii) is w
Vk-1.
a E We where x E Xx-1 and the answer to querying (iii) Yes,
AV,
V Vomm== w.
V4-1,
a E W. where I E Xx-1 and the answer to querying (iii) is No.
Vinin
Vk-1,
a GER,IE X and the answer querying (iv) is Yes,
Sermm ms
1V-11
a E X and the answer to querying (iv) is No. the answer to querying either (ïi, (ii) or (iv) is Not Care,
(16)
15

---

where of is the set of unchecked items whose value of attribute a is not equal to the user answer wa. V to is the set of unchecked items whose value of attribute x not equal to the queried attribute value Was V2 =w,, subset of VR-1, is the set of unchecked items whose value of attribute x is equal to the queried attribute value Wz. Va-am <w, the set of unchecked items whose value of attribute x is smaller than the queried attribute value Wx Vasiim 2 2 is the set of unchecked items whose value of attribute x not smaller than the queried attribute value wz-
To estimate the certainty gain from taking each possible action, we introduce the expected certainty gain as follows.
Definition A4 (Expected Certainty Gain). For the k-th turn, we define expected certainty gain to estimate AU; on (-) taking a different action.
Eq. (4),
a E VR-1: i... querying (i),
Eq. (6),
E Rg i.e. querying (ii),
Vca(.)
(17)
Eq. (9),
a W.. i.e., querying (iti),
Eq. (11), a R, x Xk-1, i.e. querying (iv).
Then, at each turn, we can compute the candidate action, getting the maximum expected certainty gain, as the action to take, denoted as dquery- In practice, as shown in Proposition 1, for each attribute, querying attribute IDs, i.e., (ii), and querying attribute values, i.e., (iii), is not compatible. And, (iv) is particularly designed for large discrete or continuous value space, which can be regarded as specific attribute value generation engineering for (iii) (i.e., using Eq. (12) to directly compute the queried value for each attribute), and thus, we treat (iv) as part of (iii). Therefore, we organize two querying strategies. One is querying (i) and (ii), whose objective can be formulated as Eq. (3). The other one is querying (i) and (iii), and the objective can be written as Eq. (10).
Besides Vk. we further summarize the update of X' as follows. Similarly, we define A& 1x-1 \. Ar. then Ad's can be written as:
{a} )
querying (ii), { querying either (iii) or (iv), and there is no unchecked attribute value in æ, querying either (i) (iv).
AX
0,
(18)
Based on the above, CORE runs Algorithm shows.
Remark. One of the advantages of querying attribute values, compared to querying attributes, is that the user's s answer to queried attribute would be out of the candidate attribute values (i.e., W. for queried attribute x... We are also aware that one possible solution is that the conversational agent would list all the candidate attribute values in the query. However, we argue that this approach would work only when the number of candidate values is small (namely, [Wa] is small) such as attributes Color and Hotel Level, but not work when there many candidate values, c.g., attribute Brand, since listing all of them would significantly reduce the user satisfaction.
A1.2 Proofs
Proposition A1. For any attribute x E Xx-1, v Vou(query(z)) 2 Woa(query(z) = wx) holds for all the possible value w, W..
Proof. For consistency, we re-formulate Eq. (6) as:
Lea (cu(query(z))
(Vos(ug =:)) Pr(w.. wx)) Σ
(19)
Wi E.
where x is the queried attribute, and = represents the user preference on (corresponding to the notations a and w respectively). We can also re-write Vox (w,, = w()) as:
T (ww, = w.)) =SUM( (RR (em))m)) E Vk-1 n Vuamm }) Σ
(SUM( SUR((VRE(VM))ee Va-1 O n Varaia =w
wHEW )
(20)
Σ
Veo(w" + wx) 2 Voo # wx),
weeW. ((w))
16

---

where wa is an arbitrary attribute value in Wa W. The above equation is built upon the simple fact that after an attribute z, the answer of the user preferring w' equivalent to the answer of the user not preferring all the other w/s. which can remove all the unchecked items whose value is equal to any wn. Thus, the expected certainty gain of knowing the user preferring wy not smaller than knowing the user not preferring any one Wa We V "MA("' ). and the equality holds only in the case where W2 = [we, w' J. namely there are only two candidate attribute values.
Based on the above equations, we can derive:
Vx Von(queryxx) - Σ
(Voa(W> = =w²)= Pr = w²))
WnEWa
[w, = Pr(w2= = ($)))
E
(Voo(w's =w)) Pr(w', w.))
" (ww)
2oo ( w Pr(w2z w$))
Σ
Voo(wx F wg) Pr(m1 wx))
(11
w WLEWWMMu M
2o00 = Pr(wy wx)) + oo (wx, F w.)
Σ
(Pr(w's wx))
my E.W. Maat 2 Vw, = = Pr(wz= = w")) + (o + wx) Pr(wz= # wx))
2 oo (query(x) w.).
Since we put constraint on Xx-1, thus it proves the proposition.
Proposition A2. In the context of querying attribute values, an ideal choice is always the one that can partition all the unchecked relevance scores into two equal parts (i.e., the ideal w, Wa, x Xx-1 is the one that makes & (w, = wx SUM( { And the certainty gain in this case is c (query(r) = wg) = SUM({ (RRE (Um) Umm E Vx-1))/2.
RR (vm) Ium E ///-1)//2 hold), if it is achievable.
Proof. Without loss of generalizability, in the context of querying attribute values, we recap the formulation of (ca(query(x) = wx). shown in Eq. (9) as:
Tca (a(querr(()) = wx) oo (w. = w Pr(w= w") + 1 oo (w, F wg) Pr(w> + wn)
SUM( We Uo Vk-1 0 Vamin==aa)) }) SUM( ( (1m) I'm E Vk-13)
-SUM((VRR( ( ) E Vk-1 V Van })
SUM( SUM({RRE (eee(o (um Hum E Va-1 O tha }) (22)
+UUM (SDM((Vert (VRE(UU))) E VK-1 Vw,)
SUM( Uaz (1m) Um E Vk-1))
R ROES R
RYES
=Rygs-
+ (R RYeS)
R
where we use RYES to denote SUM( ( VR (vm Ium E V--1 n Vanana Aw. }). the expected certainty gain of the event w/ = " happening (i.e., the user answers Yes to querying w²), ), and use Rt to denote the summation of relevance scores of all the unchecked items, i.e., SUM( { Ra (Mn(()) Dmm E Vg-1)). For convenience, we use to denote Cca (cu(que)yxz) = wz). Then, can be regarded as a function of RyES. Namely, Ryes is the independent variable and v is the dependent variable.
To maximize V, we have:
2 (R-2 2. RYES) = 0. R
NV
(23)
Therefore, we have Ryee= R/2, and in this case, y R/2. Then, we can reach the conclusion that the idcal partition is the one dividing all the unchecked relevance scores, i.c., R. into two equal parts; and in this case, (ca(query(z) that querying Wa can check half of the relevance scores in expectation.
w = R/2 = SUM( T Ree( (mm Vm Va-1))/2, which indicates
O
Lemma A1 In the context of querying attribute values, suppose that VRE(Um) = 1 1/M holds for any M+1 < R < (M+1)/2.
2m E V, then the expected number of turns (denoted as K) is bounded by log
Proof. We begin by considering the best case. According to Proposition 2, if we can find an attribute value Wx, where querying w. can partition the unchecked relevance scores into two equal parts, then
17

---

we can build binary tree, where we can check M/2² at the k-th turn. Therefore, we have:
1 1+2+ + ... +2 R1 M, +
(24)
which derives K = logg2- M-1 In the worst case, we only can query one item during one query. Then, the expected number turns is:
1 M
M-1
1 -
M
=
+2 +2
+ M
II ==
(1
.1=
(25)
M
M -11
M -
2
Combining Eqs. (24) and (25) together, we can draw log2
M+1
K < (M 1)/2.
A2 Plugging the Conversational Agent into Recommender Systems
A2.1
V V-o(-) for Querying Attributes in Large Discrete or Continuous Space
The main idea of our conversational agent is to recursively query the user to reduce uncertainty. The core challenge is that there exist some cases where querying any attribute values items can not effectively reduce uncertainty. Most of these cases occur when some key attributes have a large discrete space continuous space, leading to broad decision tree. Formally, for key attribute x E -x, a "small" discrete space usually means [Wal < [Vx-1l. For example, for attribute Hotel Price. then querying z, the user would not respond with accurate value, and querying z=one possible value could be ineffective.
To address this issue, propose to find a E R instead of wa W,, and then we can query whether the user's preference is not smaller than or not, i.c. query(æ) 2 wy instead of whether the user's preference is equal to wa not, i.e. query(æ) = Wz. Then, the expected certainty gain, this case, can be written as:
ca qqerr((x) 2 wx) Woo(we, wx) Pr(ws, w²) + Voo(wo w(t) Pr(w, <w),, (26)
where
Voo 2 w/) SUM( (RR((Mm))ug E Vaew, n Vk-1)), oo W22 < w;) = SUM( (mm))))u E Vezwz n 1x-1)),
(27)
where V=zw, is the set of items whose value of attribute is not smaller than Wa, and Veew, is the of the rest items, namely V Vezu. UVz4wa Vk-1i and
Pr(w, 2 w SUM( ( VRE(UU)) Ium Vazw, n VK-13)
SUM( { (RE(Um)) Iumm E Vk-1)) Pr(w, < w SUM( ( RE (mm (mm))m Iu E Vaews O Vk-1))
(28)
SUM( ( WR (Wm)) |Um Vk-13)
Therefore, the same as query(x) War query(æ) > wy also divides the unchecked items into two parts, and the user is supposed to answer Yes or No, corresponding to either one of the two parts. Then, Proposition 2 also works here. Namely, for each attribute X the oracle was denoted as wo is the one that can partition the relevance scores into two equal parts. Formally, we have:
(Vaz (ae))))) E Vazw. Vk-1)) SUM( (V (um Hom VV-13)] (29)
- = arg min SUM( wy, ER
2
Since it is infeasible find an exact oracle one, approximate w.
AVERAGE({ ( W (Um) We.. I'm E Vx-13),
(30)
where Wr,,. the value of attribute in item Un- indicates that our estimation is the average the attribute values for the items in V weighted by their relevance scores.
18

---

A2.2 Making (-)(-) Consider Dependence among Attributes
The following techniques allow CORE take the dependence among attributes into account. We provide two ways, where one requires an FM-based recommender system, while the other one poses no constraint.
Taking (-) in Eq. (6) as an example, we re-formulate Eq. (6) as, when a E X4 and then we can compute V (query(a)) as:
Te (query(a)))
Σ
ooo (oooquury(a')) Pr(query(a')querrynn)
(31)
a'EA
where we use (-) to denote this variant of Vca(-).
Estimation from a Pre-trained FM--ased Recommender System. If our recommender system applies factorization machine (FM) based recommendation approach, then we can directly adopt the learned weights as the estimation of Pr Pr(query(d') query((a) in Eq. (31). Taking DeepFM [20] as an example, we begin by recapping its FM component:
N
Wi +
Σ In +
Σ
Σ
(V Vneenn
(32)
m
where the model parameters should be estimated in the recommender system (in line in Algorithm 1), including wo E IR, w E RRV. V E RND and D is the dimension of embedding. And, (s.) :) is the dot product of two vectors of size D, defined as (Vi, Vj) Ede Vid Ujd- In this regard, for each pair of attributes (e.g.. (a, a') in Eq. (31)), we can find the corresponding normalized (Vn, /nr)//val [V,/ as the estimation of Pr(query(c') (((uury(a))quuer((()) qquer((a))
Estimation in a Statistical Way. If appplynng any other recommendation approach to the recommender system, we design statistical way. We first decompose 4Pa (query((a) according to Eq. (6):
De (queryy()))))
Σ E
(Woo w/) Pr(w. =wa)
(33)
where we define Co (w, = wx) as:
T (wa w/) = Σ Σ
(Voa(wa = w.)) Pr(we' = w2 = =wa)
(34)
a' d'EA w.l EWW,
where Pr(w.- w We w measures the probability of how likely getting the user's preference on attribute a (i.c., w. = w()) determinates the user's preference on other attributes (i.c., Wa = w For example, in Figure 1, if the user's preference
attribute Hotel Level is 5 (i.e., a is Hotel Level, Wa is 5 and the user's answer is Yes), then we could be confident to say that the user preference attribute Shower Service is Yes (i.e., a' is Shower Service, wa is Yes, and the user's ' answer is Yes), i.e., Pr(w,' = w W,, = wx)) is close to 1.
We estimate Pr(w w (wa = wx) by using the definition of the conditional probability:
Maantt W.r) Va-al
Pr(w.: war/10. = w:)
(35)
Vaoii Vk-11
where Veram=w, is the set of items whose value of a equals Was and V (a (a - = is the set of items whose value of a equals wa and value of equals wa By incorporating Eqs. (34) and (35) into Eq. (33), we can compute tro ({u(que))(a)) for any a E Xx-1-
Extensions to Other Cases. Besides querying attributes, we also introduce another querying strategy query attribute values. Formally, can have:
Vealquery(a (x) = w.) = vo (w. wx) Pr(w,, = = + DD (w. # w:)) Pr(w # w"), (36)
where (eo(w, = wx) can be computed by Eq. (34), and the formulation of Va (w,, + w could
be directly extended from Eq. (34) by replacing Von (w., = w:) with Voa (wa # wx), and replacing Pr(war = w., w. = wi)) with Pr(waa * w (w, * w.). Pr(wa) * w # wx) could be computed by replacing V Vaminww, with V tua. and replacing wo. with
19

---

Vanaaa Am Mararm fwa)" Vaninn is the set of items whose value of does not equal wa and V (amato fw, (Mn.)(ad ]A(a fw,) is the set of items whose value of does not equal Wa and value of a' does not equal Wa!
Then, we have made our conversational agent consider the dependence among attributes for cases (ii) and (iii), summarized in Definition A1. There is need to consider the dependence in case (i), and, as concluded in Appendix AI.1, (iv) can be regarded as special engineering technique in (iii), and thus, one just needs to follow the same way handle case (iv).
A2.3 Overall Algorithm
We summarize the overall algorithm in Algorithm 1. CORE follows traiiing-tainng--... -and-onlin- checking paradigm, where offline- trrining is represented in lines and 12, and ouline-checking is represented in lines 5, and 7.
As shown in line 5, there are two querying settings, i.e., querying items and attributes, and querying items and attribute values. We note that querying attributes and querying attribute values can be compatible, but can not simultaneously operate on the same attribute. We recap that Proposition I says that for each attribute, assuming users could give clear answer showing their preference, querying an attribute can always obtain certainty gain not smaller than querying any attribute value of the attribute.
Therefore, in practice, we would select those attributes that are likely to receive a clear preference from users (e.g., attributes Category, Brand) in the setting of querying items and attributes, and use the rest of attributes (e.g., attribute Price) in the setting of querying items and attribute values. Also, as stated at the end of Appendix A1.1, we can further select several attributes with small space of attribute values, use them in the setting of querying items and attributes, and list all the candidate attribute values in the queries. In this regard, for any attribute, since the space of attribute values is changing in the context of querying attribute values, then we may transfer from the setting of querying attribute values to querying attributes, when there are few unchecked candidate attribute values.
All the above operations need careful feature engineering, which should be task-specific and dataset- specific. We argue that this is out of the scope of this paper, and we leave for future work.
A3
Experimental Configuration
A3.1 Dataset Descriptions and Data Pre-processing
We summarize the datasets used in this paper as follows.
Amazon dataset [8, 33] is a dataset collected by Amazon from May 1996 to July 2014. There are 1,114,563 reviews of 133,960 users and 431,827 items and 6 attributes.
LastFM dataset [9] is dataset collected from Lastfm, music artist recommendation platform. There are 76,693 interactions 1,801 users and 7,432 items and 33 attributes.
Yelp dataset [12] is a dataset collected from Yelp, a business recommendation platform. There are 1,368,606 6066 interactions of 27,675 users and 70,311 items and 590 attributes. We follow [30] to create 29 (parents) attributes upon 590 original attributes, and we use the newly created ones in our experiments.
Taobao dataset [10] is a dataset collected by Taobao from November 2007 to December 2007. It consists of 100, 150,807 interactions of 987,994 users and 4, 162,024 items with an average sequence length of 101 and attributes.
Tmall dataset [1 of 54.925,331 interactions of 424,170 users and 1,090,390 items with an average length of 129 and attributes.
dataset collected by Tmall from May 2015 to November 2015. consists
Alipay dataset [7] is a dataset collected by Alipay, from July 2015 to November 2015. There are 35,179.371 interactions of 498,308 users and 2,200,191 items with an average sequence length of 70 and 6 attributes.
Douban Movie dataset [35, 53] a dataset collected from Douban Movie, a movie recommenda- tion platform. There are 1,278,401 interactions of 2,712 users and 34,893 items with 4 attributes.
20

---

Table AI: Results comparison in the context of querying attributes and items tabular datasets.
Pea(-)
Amazon
LastFM
Yelp
T@3
S@3
T@5
S@5
T@3
S
T@S
S@5
T@3
S@3
T@5
S@5
AG
6.47
0.12
7.83
0.23
6.77
0.05
8.32
0.14
6.65
0.08
8.29
0.13
ME
3.04
0.98
5.00
1.00
3.00
1.00
5.00
1.00
3.00
1.00
5.00
1.00
COLD START
CORE
2.88
1.00
2.87
1.00
2.73
1.00
2.75
1.00
2.92
1.00
2.94
1.00
CORE
2,84
1.00
2,86
1.00
2.74
1,00
2.73
1.00
2.90
1.00
2.91
1.00
AG
2.76
0.74
2.97
0.83
4.14
0.52
4.67
0.64
3.29
0.70
3.39
0.81
CRM
3.07
0.98
0.83
2.98
0.99
3.43
3.08
0.98
0,96
FM
EAR
2.98
0.99
3.13
1.00
3.02
1.00
3.51
1.00
2.94
1.00
3.02
0.99
CORE
2.17
1.00
2.16
1.00
2.06
1.00
2.07
1,00
2.09
1.00
2.10
1.00
CORE
2.14
1.00
2.14
1.00
2.05
1.00
2.05
1.00
2.10
1.00
2.08
1.00
AG
3.07
0.71
3.27
0.82
3.50
0.68
3.84
0.79
3.09
0.74
3.11
0.88
CRM
2.68
0.99
2.99
0.99
2.94
0.99
3.05
0.99
2.92
1.00
2.99
1.00
DEEP
EAR
2.70
1.00
2.88
1.00
2.95
1.00
3.21
0.98
2.87
1.00
2.97
1.00
CRIF
2.84
1.00
2.64
1.00)
2.78
0.999
2.81
2.93
0.99
2.01
0.99
UNICORN
2.65
1.00
2.45
1.00
2.87
1,00
2.90
1.00
2.88
1.00
2.92
1.00
CORE
2.07
1.00
2.06
1.00
2.07
1.00
2.08
1.00
2.06
1.00
2.07
1.00
CORE
2.08
1.00
2.02
1.00
2.05
1.00
2.03
1.00
2.03
1.00
2.06
1.00
AG
3.02
0.74
3.10
0.87
3.44
0.67
3.53
0.84
2.83
0.77
2.82
0.91
PNN
CORE CORE
2.71
1.00
1.000
2.05
2.06
2.15
1.00
1.00
2,68
1.00
2.98
1.00
2.07
1.00
2.02
1.00
2.08
1.00
2.11
1.00
Douban Book dataset [35, 53] is dataset collected from Douban Book, a book recommendation platform. There are 96,041 interactions of 2,110 users and 6,777 items with 5 attributes.
In summary, our paper includes three tubular datasets (i.e., Amazon, LastFM, Yelp), three sequential datasets (i.e., Taobao, Tmall, Alipay), and two graph-structured datasets (i.e.. Douban Book, Douban Movie). First, we follow the common setting of recommendation evaluation [22, 39] that reduces the data sparsity by pruning the users that have less than 10 historical interactions and the users that have at least 1 positive feedback (e.g., clicks in Taobao). We construct each session by sampling one user and 30 items from her browsing log (if less than 30 items, we randomly sample some items that are not browsed, as the items receive negative feedback, into the session). During sampling, we manage the ratio of the number of items receiving positive feedback and the number of negative feedback falls into the range from 1:10 to I:30. We use a one-to-one mapping function to map all the attribute values into discrete space to operate. From those attributes with continuous spaces, we directly apply our proposed method introduced in Section 3.2.
A3.2 Simulator Design
As summarized in Definition A1, there are two main agents in our simulator, namely conversational agent and a user agent. The conversational agent is given the set of candidate items (i.e.. V), and the set of candidate attributes (i.e., 2)) (together with their candidate values, i.e., W2 for every E X). Then, at k-th turn, the conversational agent is supposed to provide an action of querying. either one from (i), (ii), (iii) and (iv) shown in Definition A1, and the user agent is supposed to generate the correspouding answer and derive the set of unchecked items (i.c., V&), and the set of unchecked attributes (i.e., Xx)) (together with the unchecked values of each attribute æ)) Let W be the set of the unchecked values of x, then its update function is simple. Firstly, we assign W Wa, and we can further define AWk- = W #
W then AW # can be written as:
{
{ querying (iii), and selecting an attribute value in otherwise.
AWAh x
(37)
0,
For simplicity, we omit the above update in the main text.
From the above description, we know that the conversational agent and the user agent are communicat- ing through exchanging the set of unchecked items and unchecked attributes (and unchecked attribute values). We also develop a port function in the conversational agent that leverages pre-trained large language model to generate the human text for each action. See Appendix A4.2 for detailed descriptions and examples.
21

---

Table A2: Results comparison of querying attribute values and items tabular datasets.
Pcn(-)
Amazon
LastFM
Yelp
T@3
S
T@5
S@5
T@3
S
T@S
S@5
T@3
S@3
T@5
S@5
AG
6.47
0.12
7.83
0.23
6.77
0.05
8.32
0.14
6.65
0.08
8.29
0.13
ME
6.50
0.12
8.34
0.16
6.84
0.04
8.56
0.11
6.40
0.15
8.18
0.20
COLD START
CORE
6,02
0.25
6.18
0.65
5.84
0.29
5.72
0.74
5.25
0.19
6.23
0.65
CORE,
6,00
0.26
6.01
0.67
5.79
0.30
5,70
0.75
5.02
0.21
6.12
0.68
AG
2.76
0.74
2.97
0.83
4.14
0.52
4.67
0.64
3.29
0.70
3.39
0.81
4.58
5.87
4.12
0.25
0.69
EAR
4.13
0.32
6.32
0.42
4.02
0.38
5.45
0.67
4.10
0.28
5.95
0.72
CRIF
4.34
0.28
6,24
0.45
3.98
0.58
4.11
0.76
4.07
0.31
6.02
0.70
FM
UNICORN
4.43
0.30
0.52
0.54
7.56
4.40
0.25
5.38
0.77
CORE
3.26
0.83
3.19
0.99
3,79
0.72
3.50
0.99
3.14
0.84
3.20
0.99
CORE
3.16
0.85
3.22
1.00)
3.75
0.74
3.53
1.00
3.10
0.85
3.23
1.00
AG
3.07
0.71
3,27
0.82
3,50
0.68
3.84
0.79
3.09
0.74
3.11
0.88
4.51
0.40
5.88
4.11
DEEP
EAR
4.47
0.30
6.35
0.43
4.01
0.37
5.43
0.69
4.01
0.32
5.74
0.75
FM
CORE
3.23
0.85
3,22
0.99
3.47
0.81
3,34
1.00
2.98
0.93
3.11
1.00
CORE
3.16
0.87
3,21
1.00
3.45
0.83
3.30
1.00
2.97
0.94
3.10
1.00
3.02
0.74
3.10
0.87
3.44
0.67
3.53
2.83
0.77
2.82
0.91
PNN
0.99
3.20
2.75
1.00
CORE
3.00
0.92
1.00
3.05
0.88
3.12
1.00
2.74
0.88
2.76
1.00
A3.3
Baseline Descriptions
We first summarize the recommendation approaches, denoted as R-E(-), used in this paper as follows.
of items uniformly generated. In other words, for the item set = {vm ] m==1 we set the relevance score for each item Um V by (RE(Um) 1/M.
COLD START denotes the cold-start setting, where all the relevance
FM [38] is a factorization machine- based recommendation method working on tabular data, which considers the second-order interactions among attributes (i.e., feature fields).
DEEP FM [20] combines an FM component and a neural network component together to produce the final prediction.
PNN [37] includes an embedding layer to Iearn a representation of the categorical data and a product layer to capture interactive patterns among categories.
DIN [52] designs a deep interest network that uses a local activation unit to adaptively learn representation of user interests from historical behaviors.
GRU [23] applies a gated recurrent unit (GRU) to encode the long browsing histories of users.
LSTM [18] applies a long short term memory unit (LSTM) to encode the historical browsing logs of users.
MMOE [32] develops multi-gate mixture-of-expert that can model the user's multiple behaviors by sharing the expert sub-models across all the behaviors.
GCN [27] designs graph convolutional network that learns representations of nodes (either users or items) by passing and aggregating their neighborhood information.
GAT [46] designs graph attention network that adopts an attention mechanism to consider the different contributions from the neighbor nodes in representing the central nodes (either users or items).
We then summarize the conversational techniques, denoted Voo(.), used in this paper follows.
AG (Abs Greedy) always queries an item with the highest relevance score at each turn, which is equivalent to solely using the recommender system conversational agent.
ME (Max Entropy) always generates a query in the attribute level. In the setting querying items and attributes, it queries the attribute with the maximum entropy, which can be formulated as:
Denttaww, 0 D [Vr -raal=w, n MV--1l).
Qquary arg
Σ DEALt- EW2
D4-1l
log
M4-1al
(38)
22

---

In the setting of querying items and attribute values, we first apply Eq. (38) to obtain the chosen attribute and then we select the attribute value with the highest frequency of the chose attribute as:
Aquary
arg max V v. O V/-1l: Mr EWW
(39)
where xx is computed following Eq. (38). To evaluate the success rate, during the evaluation turn, we apply AG after employing ME.
CRM [44] integrates the conversational component and the recommender component by feeding the belief tracker results to an FM-based recommendation method. It is originally designed for the single-round setting, and we follow [30] to extend it to the multiple-round setting.
EAR [30] consists of three stages, i.e., the estimation stage to build predictive models to estimate user preference on both items and attributes based on an FM-based recommendation approach, the action stage to determine whether to query attributes or recommend items, the reflection stage to update the recommendation method.
CRIF [25] formulates the conversational recommendation scheme as a four-phase process consist- ing of offline representation learning, tracking, decision, and inference. In the inference module, by fully utilizing the relation between users' attribute-level and item-level feedback, CRIF can explicitly deduce users' implicit preferences.
UNICORN [13] formulate the conversational recommendation problem as a unified policy learning task. UNICORN uses dynamic weighted graph-based enforcement learning method to learn policy to select the action at each conversation turn, either asking an attribute or recommending items.
The proposed methods are listed as follows.
CORE is our proposed method calculating Vea(-)s in line in Algorithm 1.
CORE a variant of CORE that computes 4/ (o))) instead of Voa(-)s, making 4 (-)s consider the dependence among attributes.
A3.4 Implementation Details
For each recommendation approach, we directly follow their official implementations with the following hyper-parameter settings. The learning rate is decreased from the initial value 1 10-2 to 1 10-6 during the training process. The batch size is set as 100. The weight for the L2 regularization term is 4 10-5, The dropout rate is set as 0.5. The dimension of embedding vectors is set as 64. For those FM-based methods (i.e., FM, DEEP FM), we build a representation vector for each attribute. We treat it as the static part of each attribute embedding, while the dynamic part is the representation of attribute values stored in the recommendation parameters. In practice, we feed the static and dynamic parts together as a whole into the model. After the training process, store the static part and use to estimate the dependence among attributes, as introduced in Appendix A2.2. All the models are trained under the same hardware settings with 16-Core AMD Ryzen 9 5950X (2. 194GHZ), 62.78GB RAM, NVIDIA GeForce RTX 3080 cards.
A4 A Additional Experimental Results
A4.1
Performance Comparisons
We conduct the experiment in two different experimental settings. One is the setting of querying items and attributes, and the other is the setting of querying items and attribute values. We report the results of the former setting on tabular datasets (i.c., Amazon, LastFM, Yelp) in Table AI, and also report the results of the latter setting on these tabular datasets in Table A2. We also evaluate the performance of CORE in sequential datasets and graph--tructured datasets, and report their results in Table A3 and Table A4 respectively.
By combining these tables, our major findings are consistent with those shown Section 4.2. Moreover, we also note that the performance of CORE in querying items and attributes is close to the oracle, and thus considering the dependence among attributes in CORE does not bring much improvement.
23

---

Table A3: Results comparison of querying attribute values and items on sequential datasets.
Taohao
Tmall
Alipay
(-a(+)
co(
Tw3
S@3
TØ5
S@5
T@3
S@3
T@5
S@5
T@3
S
T@5
S@5
AG
6.30
0.15
7.55
0.27
6.80
0.04
8.54
0.09
6.47
0.11
7.95
0.19
MB
6.43
0.14
7.82
0.29
6.76
0.05
8.50
0.12
6.71
0.07
8,46
0.11
COLD START
CORE
5.42
0.39
5.04
0.89
6.45
0.13
7.38
0.37
5.98
0.25
6.17
0.65
C
5.41
0.40
5.05
0.90
6.34
0.17
7.14
0.40
5.91
0.28
6.12
0.68
AG
3.03
0.70
3.17
0.81
3.57
0.58
4.32
0.61
2.99
0.84
3.20
0.87
FM
CORE
3.01
0.87
2.95
1.00
3,53
4.14
0.86
3.37
0.90
3.29
0.97
CORE
3.02
0.88
2.91
1.00
3.50
0.71
4.11
0.87
3.32
0.91
3.14
0.97
AG
2.99
0.72
2.93
0.52
3.03
0.83
3.22
0.87
DEEP
CORE
2.73
0.92
2.78
0.99
4.43
0.84
3.17
0.87
3.18
0.97
FM
CORE,
2.68
0.94
2.80
1.00
4.13
0.65
4.42
0.85
3.12
0.87
3.17
0.97
AG
2,93
0.76
2.87
0.92
3.98
0.52
4.60
0.61
3.18
0.88
2.94
0.91
PNN
CORE
2.51
0.98
2.64
1.00
3.20
0.64
4.11
0.90
3.19
0.88
3.15
0.98
CORE
2.48
0.98
2.61
1.00
3.20
0.65
4.02
0.94
3.18
0.88
3.11
0.98
AG
2.71
0.85
2.83
0.95
4.14
0.51
4.81
0.59
3.10
0.82
3.35
0.85
DIN
CORE
2.45
0.97
2.54
1.00
4.12
0.64
4.16
0.89
3.25
0.83
3.32
0.96
CORE;
2.44
0.97
2.50
1.00
4.10
0.66
4.12
0.91
3.22
0.85
3.30
0.97
AG
2.80
0.80
2.64
0.97
3.82
0.56
4.40
0.64
3.17
0.83
3.29
0.87
GRU
CORE
2.31
0.98
2.44
1.00
3.81
0.72
3.91
0.92
3,10
0.84
3.11
0.96
CORE,
2.96
0.99
2.40
1.00
3.78
0.74
3.90
0.93
3.10
0.84
3.12
0.95
AG
2.60
0.85
2.52
0.97
4.73
0.41
5.63
0.49
3.43
0.78
3.27
0.89
LSTM
CORE
2.37
0.97
2.49
1.00
4.58
0.55
4.36
0.90
3.03
0.84
3.16
0.97
CORE,)
2.30
0.98
2.49
1.00
4.56
0.57
4.34
0.91
3.05
0.85
3.18
0.97
AG
3.04
0.75
2.98
0.92
4.10
0.54
4.56
0.62
3.58
0.83
3.90
0.92
MMOE
CORE
2.48
0.96
2.60
1.00
3.92
0.65
4.19
0.85
3.21
0.91
3.17
0.98
CORE
2.46
0.97
2.61
1.00
3.90
0.66
4.20
0.84
3.19
0.89
3.12
0.99
Table A4: Results comparison of querying attribute values and items on graph datasets.
Douban Movie
Douban Book
V REE (+)
Vec (+)
T@3
S@3
T@5
s@5
T@3
S@3
T@5
S@5
AG
6.52
0.11
7.94
0.21
6.36
0.15
7.68
0.26
ME
6.60
0.10
8.16
0.21
6.40
0.15
8.04
0.24
COLD START
CORE
5.48
0.38
4.84
0.94
5.96
0.26
5,08
0.92
CORE,"
5.45
0.40
4.81
0.94
5.91
0.28
4.98
0.94
AG
3.75
0.63
3.65
0.87
3.56
0.64
3.41
0.87
GAT
CORE
2.89
0.91
2.97
1.00
2.80
0,92
2,91
1.00
CORE
2.87
0.92
2.96
1.00
2.81
0.93
2.90
1.00
AG
3.21
0.69
3.33
0.83
3.20
0.71
3.18
0.89
GCN
CORE
2.76
0.92
2.81
1.00
2.85
0.91
2.85
1.00
CORE
2.74
0.93
2.80
1.00
2.83
0.93
2.78
1.00
A4.2 Incorporating Our Conversational Agent with a Frozen Chat-bot
With the development of pre-trained large language models (LLMs), chat-bots built based on these LLMs are capable of communicating like humans, which is a powerful tool to allow our conversational agent to extract the key information from the user's free text feedback and generate free text for querying attributes and items. Concretely, chat-bot can act as either a question generator or a answer extractor. As shown in Figure 2, if our conversational agent decides to query attribute breakfast service, then the command passes to the question generator to generate a free text question "Do you require breakfast service?" The user answers the question by free text "I do not care about breakfast service, and I really want a hotel with shower", and then the answer extractor extracts the user preference on the given answer, namely the user does not care about attribute breakfast service and gives positive feedback on attribute shower.
24

---

For this purpose, we follow a short OpenAl tutorial for prompt engineering to design the following prompts based on gpt-3. turbo model.
# Load the API key and relevant Python libaries. import openai import o8
def get_ commpletion(prropppp
messages [["role": "user", "content": prompt)] response openai ChatCompletion.creen create( model=model, messages-nessages,
temperature=0, # this is the degree of randomness of the model's 's output
return response... choices [0] message content""
We first evaluate using LLM as question generator by an example of generating question to query an attribute, e.g., breakfast service.
# Large language model as question generator. text f"n"
Attribute, Breakfast Service, Hotel .n"
prompt = f"""
You will be provided with text delimited by triple quotes. If it starts with the word "item", it denotes an item, then you should generate a text to recommend the item to the user.
Otherwise, it denotes an attribute,
then you should generate a text to query the user's preference on the attribute. You should be gentle. (' .*"
response get_completion (prompt) print (response)
The following is the corresponding output provided by the LLM.
Good day! May I ask for your preference regarding breakfast service in hotel? Would you like to have a complimentary breakfast or do you prefer to have the option to purchase breakfast at the hotel restaurant?
We then evaluate using an LLM as a question generator by an example of generating question to query (i.e., recommend) item, e.g., hotel A.
# Large language model as question generator. text f"nn
Item, Hotel u*"
prompt = f"""
You will be provided with text delimited by triple quotes. If it starts with the word "item", it denotes an item, then you should generate text to recommend the item to the user.
Otherwise, it denotes an attribute,
then you should generate text to query the user's preference on attribute. You should be gentle.
'' ('
response dete.competiionpprompt print (responese)
hnttps://ww.deppph http://awwaaaaaa/ataeeetoorereaa cenitt-proop-tett wwweeeeennnnoonooooooeeeeeee
25

---

The following is the corresponding output provided by the LLM.
Great choice! Hotel A wonderful option. Would you like me provide more information about the hotel or help you book room?
Also, we evaluate using an LLM as an answer extractor by an example of extracting the user preference attributes, e.g., breakfast service, hotel level, and shower.
text f"."
I do not care about breakfast service, and I really want hotel with shower. unn
prompt Euun
You will be provided with text delimited by triple quotes.
If you can infer the user preference attributes, then re-write the text in the following format:
[attribute namel [user perference]
Attribute names include Breakfast Service, Hotel Level, and Shower. User preference includes Yes to denote the positive preference, No denote the negative preference, and Not Care to denote the user does not care. If you can not infer the user preference attributes, then re-write the text in the following format: lattribute namel: Not Know "<((txx))<c u*"
response get_.comlletion(proom (prompt) print (response)
The following is the corresponding output by the LLM.
Breakfast Service: Not Care
Hotel Level: Not Know
Shower: Yes
Similarly, also can evaluate using an LLM as question generator by example of generating question to query an attribute value, e.g., Hotel Level=5.
# Large language model as question generator.
text f"."
Attribute, Hotel Level is 5, Hotel unn
prompt f"n
You will be provided with text delimited by triple quotes. If it starts with the word "item", it denotes item then you should generate a text to recommend the item to the user.
Otherwise, it denotes attribute
then you should generate text to query the user's preference on the attribute. You should be gentle. ' (text). "*"
response get_completion (prooptt (prompt) print (response)
The following is the corresponding output by the LLM.
Excuse me, may ask for your preference on hotel level? Would you prefer 5-star hotel or are you open to other options?
According to Definition A1, we have exemplified querying (i), (ii), and (iii) the above examples. We further evaluate querying (iv). Namely, we evaluate using an LLM question generator by an example of generating a question to query whether the user preference is not smaller than an attribute value, e.g., Hotel Level not smaller than 3.
26

---

Query Attribute Value:
whether Children's Books or not
No
Query Item: whether 0001061100 or not
No
Yes
Query Attribute Value: whether Price is not smaller than $5.5 or not
Yes
No
Query Item: whether 0001062395
Yes
No
An illustrated example of online decision tree in the setting of querying items and attribute values, where the target item 0001062395.
Figure A1:
# Large language model as question generator. text f"n"
Attribute, Hotel Level is not smaller than 3, Hotel unn
prompt = f"nn
You will be provided with text delimited by triple quotes.
If it starts with the word "item", it denotes an item then you should generate text to recommend the item to the user.
Otherwise, it denotes attribute
then you should generate a text to query the user's preference on the attribute. You should be gentle. c ('(
response get_ cet.:coomleeioorrorroe (prompt) print (response)
The following the corresponding output by the LLM.
Excuse me, may ask for your preference on hotel level? Would you prefer hotel with a level of or higher?
We note that if there are too many attribute IDs (or too many attribute values) in the use case, then might need to further incorporate some hierarchical designs [3] and ambiguous matching [45] into the above system, which is out of the scope of this paper, and we leave it for future work.
A4.3 Visualization and Case Study
We investigate case in Amazon dataset in the setting of querying items and attribute values, where the target item is 0001062395. We depict the online decision tree in Figure Al. From the figure, can see that the conversational agent first queries a value of attribute Category, then queries (i.e., recommends) an item 000161100; and after that, it queries another attribute, i.e., Price, and finally queries an item 0001062395.
Compared to Figure (bb), this example seems like chain. The main reason is that in practice, the user would give the corresponding answer to the query each turn. Therefore, the binary tree (in the setting of querying items and attribute values) would reduce to chain.
27

---

From this case, we also can observe that our conversational agent is capable of jointly considering the items and attributes search for the target items in the session.
A5
Connections to Existing Approaches
A5.1
Connections to Conversational Recommender Systems
Bridging recently emerged conversational techniques and recommender systems becomes an appeal- ing solution to model the dynamic preference and weak explainability problems in recommendation task [15, 26], where the core sub-task is to dynamically select attributes to query and make recom- mendations upon the corresponding answers. Along this line, one popular direction is to build conversational recommender system, which combines the conversational models and the recommen- dation models from systematic perspective. In other words, these models are treated and learned as two individual modules [30, 2, 44, 48]. For example, compared to previous literature [5, 6, 47J, recent work [49] builds the systems upon the multiple turn scenarios; unfortunately, it does not investigate when to query attributes and when to make recommendations (i.e., query items). To solve this issue, prior works [30, 31] develop reinforcement learning- -aaned solutions. However, all these previous methods based on a reinforcement learning framework are innately suffering from insufficient usage of labeled data and high complexity costs of deployment.
Instead, CORE can be seamlessly adopted to any recommendation method (especially those widely adopted supervised learning-bass based recommendation methods), and is easy implement due to conversational strategy based on the uncertainty minimization theory.
A5.2 Connections to (Offline) Decision Tree Algorithms
Decision tree algorithms [24] such as ID3 and C4.5 proposed based information theory, which measures the uncertainty in each status by calculating its entropy. If we want to directly adopt the entropy measurement for the conversational agent, then one possible definition of entropy is
[V Braten = n&4
I Woatun n (VA)).
H₂ Σ
log
(40)
[&
[V<
where Hą is the empirical entropy for k-th turn, y the set of all the labels, and V Vy.a is the set of items whose label is y. For convenience, we call this traditional decision tree as offline decision tree.
The main difference between the previous offline decision tree and our online decision tree lies in that our online decision tree algorithm does not have labels to measure the "uncertainty", instead, we have access to the estimated relevance scores given by recommender systems. We also note that directly using the user's previous behaviors as the labels would lead to sub-optimal solution, because (i) offline labels in collected data are often biased and can only cover a small number of candidate items, and (ii) offline labels only can reflect the user's previous interests, but the user's preferences are always shifting.
To this end, we measure the uncertainty in terms of the summation of the estimated relevance scores of all the unchecked items after previous (k - interactions. Formally, we define our uncertainty as:
U SUM( V (RE(U E VsJ),
(41)
where (-) denotes the recommender system. Similar to the information gain in the offline decision tree, we then derive the definition of certainty gain (as described Definition 1), and formulate the conversational agent into an uncertainty minimization framework.
A5.3 Connections to Recommendation Approaches to Addressing Cold-start Issue
Cold-start issues are situations where no previous events, e.g., ratings, are known for certain users or items [29, 43]. Commonly, previous investigations have revealed that the more (side) information, the better the recommendation results. In light of this, roughly speaking, there are two main branches to address the cold-start problem. One direction is to combine the content information into collaborative filtering to perform hybrid recommendation [1, 19, 36, 42), and a recent advance [34, 50] proposes to further combine the cross-domain information to the recommender system. The other direction
28

---

is to incorporate an active learning strategy into the recommender system [14, 40], whose target is select items for the newly-signed users rate. For this purpose, representative methods include the popularity strategy [17]. the coverage strategy [16], and the uncertainty reduction strategy [41 ], where the first one selects items that have been frequently rated by users, the second one selects items that have been highly co-rated with other items, and the third one selects items that can help the recommender system to better learn the user preference.
Integrating the conversational agent with the recommender system offers promising solution to address the cold start issue by effectively leveraging user interactions. This approach allows for querying additional information from users, thereby enhancing the system's 's understanding of their preferences and requirements.
distinctive aspect of our conversational agent's strategy compared to existing active learning strate- gies, lies in its primary objective. While active learning strategies aim improve the recommender system's performance, our conversational agent prioritizes meeting the user's immediate needs during the ongoing session. Consequently, the offine-traininn and online-checking paradigm employed in our approach assigns higher relevance scores to items that are considered "uncertain. This diverges from conventional settings where uncertainty estimation is typically conducted independently of relevance estimation.
By incorporating the conversational agent into recommender system, leverage interactions to gather more comprehensive information, which can effectively address the challenges associated with star-start scenarios. Our primary focus is to provide optimal user experience in real-time rather than solely optimizing the recommender system's 's accuracy. This novel paradigm emphasizes the importance of uncertainty estimation in relevance estimation, setting approach apart from traditional methods.
29


Reviews:
[{"review_id": "MBBtLqjjd2", "reviewer": "Reviewer_URPy", "summary": "The authors assert that there is a gap between traditional recommender systems trained on offline (historical) preference data and conversational assistants that aim to elicit user feedback about their *current* preferences. The authors assert that conversational assistant training relies on reinforcement learning-based frameworks and suffer from data insufficiency. They propose the CORE method for learning a conversational recommender system based on the idea of maximizing \"certainty gain\" by using a pre-trained recommender system as a scoring model for an online decision tree process. Experiments across a wide range of datasets demonstrates consistent improvements over reinforcement learning methods for learning conversational recommenders.", "questions": "N/A", "limitations": "The authors do not really discuss limitations of the work in a substantive way. I would like to see a greater discussion of challenges and limitations.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The problem formulation makes sense and the authors organize the methodology in 3.1 and 3.2 in an intuitive manner. However, as mentioned in the weaknesses section there could be improvements in clarity.\n- It is important that the authors extend attribute certainty gain computation to continuous attributes (Lemma 1) as this is a realistic setting for many attributes of interest in real-world situations (see prices, interest rates, or other continuous values).\n- As a general extension of the above, the authors carefully consider multiple cases that bring the problem setting closer to real-world user interactions and preference considerations (neutral responses, attribute dependence). They extend the derivation of certainty gain across these situations.\n- Experiments across a variety of datasets and base recommender systems demonstrate that CORE performs better in most cases compared to CRM and EAR baselines.", "weaknesses": "- The authors should clarify the reasoning for the statement on lines (135-136) that \"if [the queried item is in the target set], the sesion is done and therefore the certainty gain is the summation of all relevance scores in V_{k-1}\". Why is this the certainty gain? It is rooted in the definition in (1) that the certainty gain of a target item sets all items to \"checked for certainty\" but it is unclear what the basis is for this or whether it's a choice made to simplify the modeling.\n- The authors omit discussion of a large area of conversational recommendation focusing on eliciting user feedback via critiques of surfaced attributes [including 1-4]. This area is directly related to the \"attribute query\" in the online-checking framework of CORE, and merits discussion in the literature review.\n- The reference to ChatGPT-3.5-turbo seems gratuitous, as there is little mention of it in the main paper other than a statement on (254) that a pre-trained language model can be used as the Psi_{CO} component.\n- The experimental analysis seems relatively sparse. It would be good to see a deeper analysis (case studies, for example, or human evaluations) in the main paper body. It would also help to see a discussion of the limitations, edge cases, or remaining challenges in the space and what other challenging experimental settings were considered.\n\n\nReferences:\n[1] Antognini, D. \"Interacting with Explanations through Critiquing\" (2021)\n[2] Antognini, D. \"Positive and Negative Critiquing for VAE-based Recommenders\" (2022)\n[3] Li, S. \"Self-Supervised Bot Play for Conversational Recommendation with Rationales\" (2022)\n[4] Wu, G. \"Deep language-based critiquing for recommender system\" (2019)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions. Please also see the main responses above.\n\n> Why is this certainty gain from finding an item satisfying user needs in lines 135-136?\n\nAs defined in Eq. (2), our objective is to find an item satisfying user needs, and then we assume that the session is finished because the user would click the item to jump to the item page. Therefore, as shown in Definition 1 (lines 95-96), the certainty gain from finding an item satisfying user needs should be the maximum value. Therefore, we set it as the summation of all the unchecked relevance scores. We will further clarify this in our revision.\n\n> More literature on focusing on eliciting user feedback via critiques of surfaced attributes.\n\nThanks for your suggestion. We plan to add these papers to our revision and discuss the connections between them and our method.\n\n> Little mention ChatGPT-3.5.\n\nThanks for pointing it out. Our conversational component has two parts: one is how to decide what items or attribute values to query (which corresponds to our online decision tree algorithm), and the other one is how to generate querying texts and analyze user responses. As stated in Section 3.3 and Appendix 4.2, ChatGPT-3.5 is used to scale CORE up to free-text real-world use cases, which indeed is an alternative to other Chatbots such as Llama 2. We will re-organize the main text and the appendix to better explain the utility of ChatGPT here.\n\n> Deeper analysis (e.g., case studies, for example, or human evaluations) is needed. \n\nThanks for your suggestions. We provide a case study of CORE in Figure A2, and some examples of generated texts can be found in Appendix 4.2. We plan to include more examples and human evaluations of the generated conversations in the revision.\n\n> Further discussion of the limitations, edge cases, or remaining challenges.\n\nThanks for your suggestion. Tables 1 to 4 (A1 to A4) show that CORE can well address cold-start problem on user side (where RS knows nothing about the user, and therefore offer equivalent treatment on all the items). But, when it comes to cold-start problems on item side (where RS knows nothing about the item), it is still an opening problem of how to use CORE to help RS to address this issue. We will include this edge case in our revision. \n"}, {"review_id": "QwEcewWcTs", "reviewer": "Reviewer_48VR", "summary": "In this paper,  the authors propose a learning framework called CORE that can incorporate a conversational agent into any recommendation platform, to complementarily check estimated offline relevance scores in each online user session. Experiments results on comprehensive benchmark datasets show that CORE outperforms existing reinforcement learning-based and statistics-based approaches in querying items and attributes/attribute values. \n\n", "questions": "1. What kind of data/features in each dataset are used to estimate the offline relevance scores in the first place?  Are the accuracy of the offline relevance scores critical to the performance of recommendations after online-checking framework via conversational agent? ", "limitations": "NA", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The proposed offline-training and online-checking paradigm bridges a conversational agent and recommender systems via a unified uncertainty minimization framework. It's effective and efficient, and flexible to handle recommendations in both cold-start and hot-start settings. \n\n2. Various large-scale benchmark datasets are used in experiments, and the results are convincing. \n\n\n", "weaknesses": "\n1. Are the comparison results in Tables 1, 2, 3 and 4 statistically significant? It'd be great if t-test results could be provided. \n\n2. The central idea in this paper is to bring a conversational agent for recommendation systems. If possible, it'd be great to conduct an experiment on a real online platform to have user sessions to show the superiority of the proposed algorithm.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions. Please also see the main response above.\n\n1. It is better to provide t-test results for tables 1, 2, 3, and 4.\n\nThanks for your suggestion. We will provide t-test results in our revision.\n\n2. It is great to conduct CORE on online platforms.\n\nThanks for your advice. We are thrilled about bringing CORE online to further verify its performance and use cases. However, as it requires deep incorporation with the industry, therefore, we are still looking for potential incorporation. \n\n\n3. What kind of feature/data is used?\n\nWe use all the features provided in the public datasets including both discrete and continuous features, e.g., seller ID, item ID, category ID, action ID (showing the type of user behaviors), and date (showing the time of user behaviors) in Taobao for training RS offline and generating estimated scores online. We will add these statistics in our revision.  \n\n4. Are the accuracy of offline recommendations critical to CORE?\n\nWe evaluate the effect of RS performance on CORE from the following three perspectives: \n1. Tables 1 to 4 (A1 to A4) provide the results of cold-start setting (on user side). In these cases, RS knows nothing about the user, and our results show that CORE also could outperform other baselines. Besides, Lemma 1 also provides a bound for the expected number of turns in the context of cold-start setting. These empirical results and theoretical analysis guarantee our performance when RS performs poorly (i.e., assigning the same scores to all the items).\n2. Tables 1 to 4 (A1 to A4) also include different RS methods (with different RS performance). Results show that CORE can consistently benefit their RS compared to other baselines. \n3. Table R4 shows the result of ablation studies. Different amounts of training data often lead to different RS performances, and CORE can stably outperform other baselines. \n\nWe will make it clear in our revision.\n"}, {"review_id": "TJJKVkcee8", "reviewer": "Reviewer_7XeM", "summary": "In this paper, a conversational part of a recommender system is proposed. It is assumed that a recommendation model is available that assigns scores to user-item pairs (which estimate the probabilities of acceptance of the corresponding recommendations), items have a number of important attributes (numerical and categorical), and each user has preferences over the values of these attributes. A conversational model should sequentially decide (based on the recommendation scores) whether to ask the current user about the preferenced values of one attribute, or to try recommending an item, which can be accepted or rejected by the user. The goals are to maximize the rate of successful dialogs which accepted recommendations and to minimize the average number of rounds until the user accepts a recommendation.\n\nIn the current paper, authors propose a simple greedy algorithm that chooses an action that minimizes the expected sum of estimated recommendation scores of the unchecked items, that is, items that are still can be chosen by the user in the light of the obtained information so far.\n\nAuthors compare their algorithm with two previous baselines based on RL (CRM and EAR) and conclude that the proposed method outperforms RL-based approaches.", "questions": "-\tCould you summaries the proposed learning method?\n-\tWhat is the theoretical foundation behind the proposed greedy algorithm of expected gain maximization? Is it suboptimal in some assumptions?", "limitations": "I do not see any limitations of the proposed work", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "-\tPaper is mostly well-written, the algorithmic ideas and propositions are clear.\n-\tThe idea of expected uncertainty minimization is reasonable", "weaknesses": "1.\tThe main questionable point is the contribution of the paper. I have the following doubts:\n\n-\tPapers on conversational RS referenced in related work are very old, I do not see any works dated by 21-23 years. Expected gain maximization resembles expected improvement criterion, one of the most widely-used Bayesian optimization algorithms (see \"Efficient global optimization of expensive black-box functions.\" by Jones et al.) Both algorithms are based on the estimation of the expected gain using the posterior distribution. I think, corresponding references are needed here.\n\n-\tMotivation behind the proposed method in comparison with RL-based approaches is not convincing for me. Problem setting does inherently lead to RL approaches: it includes actions, states, rewards, and requires a policy for an optimal trajectory. Why a simple definite empirical greedy algorithm should outperform RL-based models?\n\n2.\tTheoretical part is not perfect. \n\nIn problem setting, it would be useful to state explicitly what assumptions underly the proposed method. For example, it in implicitly assumed that there is a unique item the user needs (therefore the sum of probabilities over items equals 1, see Eq. 5). Another example is that information on user preferences on an attribute carries binary information on user preferences on items, what underlies their division on “cheched” and “unchecked”.  The motivation behind the approach is based on these assumptions, which are rather controversial in practical cases considered in experiments.\n\nTheoretical results are very simple. Claimed propositions are self-evident, but do not carry much understanding on the specific properties, novelty, and contribution of the proposed EG maximization technique.\n\nThere are some inaccuracies in equations. For example:\n\n-\tEquation for the action space at line 175 looks formally incorrect. As $W_x$ is different for different $x$, we cannot obtain $A$ as a set product.\n-\tEq. 10 Is not formally accurate: the argmax operator should be applied to a function.\n\n\n3.\tExperiments and practice.\n\nThe baselines are rather old (CRM and EAR). What about CRIF from “Learning to Infer User Implicit Preference in Conversational Recommendation”?\n\nThere is no discussion on the possibility of efficient implementation of the proposed algorithm. How the calculation of uncertainty gains can be implemented in practice? Naïve summation over all items with the given attribute (see, e.g., eqs. 7-8) for each possible attribute values (see eq. 7) looks impractical in real-time conversational recommender systems.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions. Please also see the main response above.\n\n> The main questionable point is the contribution of the paper.\n\nPlease see the summary of our contributions in the main response above.  \n\n> Reference is old.\n\nThanks for your suggestion. We will add more recent literature in our revision.  \n\n> Why the proposed method outperforms RL based methods?\n\nIndeed, RL can consider complicated states and encode more information. However, RL often requires large numbers of training samples (known as data insufficiency) and relatively small action space (corresponding to querying attributes in our paper). Also, without sufficient training data, RL could not well generalize to open-world cases. Moreover, RL’s performance largely relies on careful reward function design, while CORE could be easily deployed in various use cases. \n\nTherefore, in many real-world RS cases, the above requirements of RL could not be fully reached. As a result, RL might not get its best performance. In contrast, CORE, a learning-free method (our online decision tree does not have any parameters to tune online), could achieve stable performance and could achieve better performance as few training samples and a huge action space (i.e., querying attribute values). \n\n> Implicit assumption on a unique item of a user and binary information on user preferences.\n\nWe apologize that we might not fully understand the review’s meaning here (if the reviewer could elaborate further, we would be happy to respond more precisely). But as a brief background context, we do believe CORE does not need these assumptions: (i) Target items do not need to be unique, since we can support a set of target items, as stated as a \\in \\mathcal{V}^* in lines 131-132, and Eq. 5 is a normalization trick to normalizes the score of each item over all the unchecked items. (ii) We consider the case where users may not care about specific attributes (or attribute values), as stated in line 251. Our experiments are also conducted in this setting, as stated in lines 278. \n\nWe will further clarify this in our reversion. \n\n> Theoretical part is not perfect. \n\nThanks for your suggestion. We will correct them and carefully check our analysis in our revision.\n\n> Baseline is old.\n\nComparison with [1] (CRIF) and [2] (UNICORN). Results (Tables R1 and R2) verify that CORE can perform well for querying attribute values, while RL methods excel at querying attributes because querying attribute values holds a huge action space which is not friendly for RL.\n\nWe will the completed version of the results in our revision.\n\n[1] Learning to Infer User Implicit Preference in Conversational Recommendation. 2022.\n\n[2] Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning. 2021.\n\n> How to implement it online?\n\nWe believe CORE can be easily implemented online, due to the following three reasons: (i) Our RS is offline-trained, therefore there are no training costs online. (ii) In practice, there are multiple stages in recommendation platform including matching, pre-ranking, ranking, re-ranking. CORE only needs to perform on re-ranking stage where there are only tens of items needed to be computed. (iii) In real-world cases, all the RS need to assign scores to each re-ranked item, CORE only brings extra computations on the expectation of certainty gain for each recommendation. We will clarify this in our revision.    \n\n> Could you summarize the proposed learning method?\n\nWe have summarized the main contributions of CORE in the main response above. We are very glad to further discuss if you have any confusion.  \n\n> What are the theoretical foundations behind it?\n\nWe apologize that we might not fully understand the review’s meaning in terms of “theoretical foundations” (if the reviewer could elaborate further, we would be happy to respond more precisely). An intuitive theoretical explanation of CORE is that RS function can be regarded as a prior from offline training, which is unlikely to perform well in the online setting, so we propose CORE to refine the prior online to better capture user needs. We propose Proposition 2 and Lemma 1 to analyze CORE’s performance on an ideal RS case and a bad RS case respectively.\n"}, {"review_id": "TQhe10g2aL", "reviewer": "Reviewer_8sN6", "summary": "The paper is about conversational recommender systems (CRS), which are systems that can interact with users through natural language and provide personalized recommendations. The paper addresses the challenge of incorporating a conversational agent into any existing recommender system in a plug-and-play fashion, without requiring reinforcement learning or data collection. The paper proposes CORE, a novel offline-training and online-checking framework that bridges a conversational agent and a recommender system via a unified uncertainty minimization objective. The paper claims that CORE can benefit any recommendation platform and can handle different types of data, attributes, and queries. The paper reports that CORE outperforms existing methods in both hot-start and cold-start recommendation settings, and can communicate as a human if empowered by a pre-trained language model.", "questions": "1) How does CORE handle the situation when the user’s answer is not clear or consistent with their previous answers?\n2) How does CORE compare with other conversational recommender systems that use natural language generation or understanding techniques?\n3) How does CORE deal with the trade-off between exploration and exploitation in querying items or attributes?\n4) How does CORE adapt to different domains or scenarios of recommendation, such as books, movies, etc.?\n5) How does CORE cope with the noise or bias in the offline estimated relevance scores or the online user responses?\n6) How does CORE handle the scalability and efficiency issues when dealing with large-scale datasets or action spaces?\n7) How does CORE incorporate user preferences on multiple attributes or items simultaneously?", "limitations": "NA", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) Originality: The paper proposes a novel offline-training and online-checking framework, CORE, that bridges a conversational agent and a recommender system via a unified uncertainty minimization objective. This approach is original in its ability to incorporate a conversational agent into any existing recommender system in a plug-and-play fashion, without requiring reinforcement learning or data collection.\n2) Quality: The paper develops a new human-AI recommendation simulator and conducts extensive experiments on eight industrial datasets with nine popular recommendation approaches. The results show that CORE outperforms existing methods in both hot-start and cold-start recommendation settings.\n3) Significance: The proposed CORE framework has the potential to benefit any recommendation platform and can handle different types of data, attributes, and queries. This makes it a significant contribution to the field of conversational recommender systems.", "weaknesses": "1) The introduction does not provide enough background and motivation for the problem of conversational recommender systems. It should explain why this problem is important and challenging, and what are the existing gaps and limitations in the literature. A possible way to improve it is to cite more relevant works and compare them with the proposed approach.\n2) The proposed approach in Section 3 is not clearly explained and justified. It does not provide enough details and intuition for how the uncertainty minimization framework works, how the expected certainty gain is derived, how the online decision tree algorithm is implemented, and how the dependence among attributes is considered. It also does not discuss the advantages and disadvantages of the proposed approach compared to other methods. A possible way to improve it is to provide more examples, figures, pseudocode, and analysis to illustrate the proposed approach.\n3) The experimental setup in Section 4 is not comprehensive and fair. It does not describe how the datasets are preprocessed, how the hyperparameters are tuned, how the baselines are implemented, and how the evaluation metrics are calculated. \n4) The experimental results in Section 4 are not convincing and insightful. They do not show the statistical significance of the performance differences, the impact of different factors/hyperparameters, or the qualitative analysis of the generated conversations. They also do not discuss the limitations and challenges of the proposed approach, such as scalability, robustness, diversity, etc. \n5) The references in Section 6 are incomplete and inconsistent. Some references are missing important information such as authors, titles, venues, pages, etc., or have different formats or styles.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions. Please also see the main responses above.\n\n> Cite more relevant work.\n\nWe will include more relevant literature in our revision.\n\n> More examples, figures, pseudocode, and analysis.\n\nWe provide detailed derivations of certainty gain and expected certainty gain in Appendix 1.1. We provide a figure to illustrate an example of the online decision tree in Figure A2. The pseudocode of CORE is available in Algorithm 1. The dependence among attributes is encoded by introducing the dependence into the formulation of certainty gain as shown in Eq. (13). Please check the main response above to see the detailed advantages and disadvantages between CORE and other RL methods.  \n\n> The experimental setup in Section 4 is not comprehensive and fair.\n\nAll the raw data (a.k.a., all the features) in these datasets are used to train our RS, and serve as the candidate attributes (and attribute values). CORE has two stages: (i) One is the offline stage where RS is tuned following the classical supervised learning paradigm and hyper-parameters are listed in Appendix A3.4. (ii) The other one is the online stage where there is no tuned needed and there is no hyper-parameter, as summarized in Algorithm 1. All the baselines are described in Appendix 3.3 and we directly follow their official implementations. The computations of evaluation metrics are described in lines 281 to 287.\n\n> The experimental results in Section 4 are not convincing and insightful.\n\nWe will add the t-test evaluation metrics of each table in our revision. As mentioned above, there are no hyperparameters in our online component, we do not discuss the impact of hyperparameters. We have discussed the effect of the offline component: RS from the following three perspectives as shown in the main response above.\n\n> They also do not discuss the limitations and challenges.\n\nWe summarize the comparisons between CORE and RL methods in the main response.\n\n> The references in Section 6 are incomplete and inconsistent\n\nWe will fix it in our revision.\n\n> How does CORE handle the situation when the user’s answer is not clear or consistent with their previous answers?\n\nFirstly, we allow the user to answer Not Care when the user is not clear about her preference in conversations, as introduced in lines 250-258. Secondly, the inconsistency with their previous answers could be handled by the LLMs (i.e., Chatbots). As stated in Appendix A 4.2, we can use some prompts to ask LLMs about the user preference for specific attributes, and in this case, LLMs are expected to encode the contextual conversations to provide the answers.\n\n> How does CORE compare with other conversational RS that use natural language generation or understanding techniques?\n\nThere are two aspects of evaluating the conversational RS. One is to evaluate whether the system can find an item satisfying the user within the minimum number of turns, which corresponds to the evaluation metrics such as average turns and success rate in the paper. The other one is to what extent the generated texts are friendly to users. From this perspective, we plan to do some case studies on the generated texts by CORE. We will list some examples and compare them with the texts generated by other conversational RS algorithms in terms of human evaluations. Due to time limitation of rebuttal period, we plan to add them to our revision. Also, we want to emphasize that CORE does not focus on the quality of generated language, since we can borrow power from pre-trained Chatbots, as described in Section 3.3 and Appendix A4.2. \n\n> How does CORE deal with the trade-off between exploration and exploitation?\n\nWe introduce Eq. (10) to tradeoff the exploration by querying attributes and exploitation by recommending items. We also note that Eq. (10) is extendable because one can add some weights or regularization terms to fit some particular use cases (i.e., if the use cases prefer exploration, then one can add a penalty on querying item v; otherwise, one can add a penalty on querying attribute w_x). We will discuss this further in our revision. \n\n> How does CORE adapt to different domains?\n\nFrom Algorithm 1, one can see CORE only requires the estimated scores of candidate items from RS. Therefore, CORE can be directly adopted in different domains of recommendations once an RS is available. As for the conversational agent, our conversational agent using pre-trained Chatbots could easily adapt to different domains since these Chatbots such as ChatGPT-3.5 carry rich domain knowledge.   \n\n>  How does CORE cope with the noise or bias?\n\nUnbiased RS is actually another big topic in the recommendation field, which discussed how to debias from offline data; also addressing ambiguous texts is another big topic in natural language models. Therefore, we argue that these problems are out of the scope of this paper. Moreover, CORE acts as a bridge connecting offline RS and online Chatbots, and therefore, CORE is unbiased and denoised if offline RS and online Chatbots are unbiased and denoised.\n\n> How does CORE handle the scalability and efficiency issues?\n\nWe believe CORE is efficient when scaling up to large-scale datasets, due to the following three reasons: (i) Our RS is offline-trained, therefore there are no training costs online. (ii) In practice, there are multiple stages in recommendation platform including matching, pre-ranking, ranking, re-ranking. CORE only needs to perform on re-ranking stage where there are only tens of items needed to be computed. (iii) In real-world cases, all the RS need to assign scores to each re-ranked item, CORE only brings extra computations on the expectation of certainty gain for each recommendation.\n\n> How does CORE incorporate user preferences?\n\nAs introduced in Algorithm 1, CORE holds sets of unchecked items and unchecked attributes. Then, once the user shows her preferences, CORE can update the sets correspondingly. We summarize how CORE updates in Appendix A 1.1.  "}, {"review_id": "qhQ7PqhgqY", "reviewer": "Reviewer_HpBp", "summary": "The paper proposes a novel framework called CORE that bridges conversational agents and recommender systems via an uncertainty minimization principle. The framework treats a recommender system as an offline relevance score estimator and a conversational agent as an online relevance score checker. The conversational agent can query either items or attributes (or attribute values) to reduce the uncertainty of the user’s preference and find a target item. The paper shows that CORE can be applied to various recommendation platforms and datasets, and can outperform existing reinforcement learning-based or statistical methods in both hot-start and cold-start settings. The paper also demonstrates how to empower the conversational agent with a pre-trained language model to communicate more naturally with the user. ", "questions": "1.\tThe modeling approach preferred by CORE users appears to be more of a heuristic estimate compared to some dynamic decision-making methods in reinforcement learning. How does your method ensure that decision-making performance is better than RL-based methods, given that your method seems more like an improved version of Max-entropy?\n2.\tAs mentioned in the Weaknesses Section, all of your decisions rely on the estimated scores from the offline recommendation model. Using this score as a benchmark for uncertainty measurement is dubious. How can you ensure that the recommendation model's estimates are accurate and reliable?\n", "limitations": "1.\tPlease refer to the Weaknesses Section.\n2.\tThe lack of comparison with the latest CRS methods, such as CPR[1] and UNICORN[2]. These new methods share some similarities with the core settings of the paper, such as UNICORN's decision space, which also includes item space and attribute space.\n\n[1] Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020. Interactive Path Reasoning on Graph for Conversational Recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’20). 2073–2083.\n[2] Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. 2021. Unified conversational recommendation policy learning via graph-based reinforcement learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1431–1441.\n", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1.\tThe paper presents a comprehensive study of the core task of conversational recommendation by considering the complex situations that arise in conversational contexts.\n2.\tThe authors conduct an extensive experimental analysis and exploration.\n3.\tThe proposed method achieves lower complexity and latency compared to existing state-of-the-art approaches.\n", "weaknesses": "1.\tThe innovations and core contributions of the paper are questionable. Most conversational recommendation papers consist of offline and online components, but CORE does not appear to have any notable novel features despite considering more complex interaction settings. The improvements in these scenarios do not seem to be sufficient to warrant acceptance of this paper.\n2.\tThe formatting and organization of the paper could be improved to facilitate readability, with inconsistencies in notation and an excessive number of annotations. The description in Section 3 is too verbose, with the core method not being highlighted adequately. A concise explanation of Algorithm 1 would suffice to clarify the scenario.\n3.\tThe core of CORE still relies on an offline recommendation system, and even the online decision making depends on scores from the offline recommendation system. The key challenge of conversational recommendation is determining how to make dynamic decisions based on user interests and conversation context. Existing reinforcement learning-based methods evidently consider more factors, whereas the proposed heuristic method in this paper depends on recommendation scores that themselves have uncertainty in dynamic interactions. The methodological foundations of CORE appear weak and unconvincing.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions. Please also see the main response above.\n\n> The innovation and motivation of the paper.\n\nWe emphasize that the core idea of CORE is not to introduce an online component but to propose a plug-and-play framework to enable any offline RS to online query user preferences without the need for a heavy change on industrial supervised learning based recommendation platform. Moreover, CORE can use powerful Chatbot APIs while RL methods with joint optimizations over RS components and conversational components can not, since there is no gradient through APIs. Please refer to the main response above for detailed comparison between CORE and RL methods. We are very happy to further discuss if you have any further concern. \n\n> Heavy notations.\n\nThanks for your suggestion. We will clarify them in the revision.\n\n\n> How do you ensure the CORE outperforms RL?\n\nIndeed, RL can consider complicated states and encode more factors. However, RL often requires large numbers of training samples (known as data insufficiency) and relatively small action space (corresponding to querying attributes in our paper). Also, without sufficient training data, RL could not well generalize to open-world cases. Moreover, RL’s performance largely relies on careful reward function design, while CORE could be easily deployed in various use cases. \n\nTherefore, in many real-world RS cases, the above requirements of RL could not be fully reached. As a result, RL might not get its best performance. In contrast, CORE, a learning-free method (our online decision tree does not have any parameters to tune online), could achieve stable performance and could achieve better performance as few training samples and a huge action space (i.e., querying attribute values). \n\nPlease refer to the main response for detailed comparison and a list of experiments for comparison. \n\n> The methodological foundations of CORE are weak and unconvincing.\n\nAn intuitive theoretical explanation of CORE is that RS function can be regarded as a prior from offline training, which is unlikely to perform well in the online setting, so we propose CORE to refine the prior online to better capture user needs. We propose Proposition 2 and Lemma 1 to analyze CORE’s performance on an ideal RS case and a bad RS case respectively.\n\nWe are very glad to further discuss if you could provide some specific examples of unconvincing foundations.\n\n> Lack of comparison between recent RL methods.\n\nThanks for your suggestions. We have included CRIF [1] and UNICORN [2] as two new baselines. Results (Tables R1 and R2) verify that CORE can perform well for querying attribute values, while RL methods excel at querying attributes because querying attribute values holds a huge action space which is not friendly for RL.\n\nWe will the completed version of the results in our revision.\n\n[1] Learning to Infer User Implicit Preference in Conversational Recommendation. 2022.\n\n[2] Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning. 2021."}, {"review_id": "i0l7wlI7F7", "reviewer": "Reviewer_SPx5", "summary": "This paper proposes a method that combines offline recommendation learning with online decision tree learning to enable recommendation in a conversational format with a few rounds of interaction. The proposed approach is evaluated on multiple datasets and in various validation settings to assess its effectiveness.", "questions": "It seems that the paper does not address the importance of considering finer item characteristics derived from diverse evaluations in user reviews during conversational interactions with AI. Could you please provide some insights into this aspect and its potential impact on the proposed approach?\n\nAre there any existing conversational recommender systems that could serve as comparative methods to the proposed approach in this paper?", "limitations": "The paper lacks consideration for finer item characteristics obtained from user reviews with language models (LM) to capture linguistic variations and users' intuitive expressions. This limitation restricts the ability to provide more nuanced and personalized recommendations in conversational recommender systems.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This paper proposes a novel framework called CORE for uncertainty reduction. CORE improves the performance of recommender systems by revealing user preferences through querying attributes and attribute values.\n\nThe paper evaluates the performance of CORE using multiple datasets and different recommendation methods. The experimental results demonstrate that CORE improves the success rate of recommendations.", "weaknesses": "(1) This paper heavily relies on decision trees, and there is a significant design cost involved in creating those decision trees. Decision trees are domain-specific and lack clear design guidelines, and the paper does not discuss this aspect or provide any discussion on it in relation to each experimental dataset. Therefore, the weakness lies in the inability to evaluate this aspect.\n\n\n(2) The reliance on metadata such as price, location, and hotel rank in decision trees makes the approach simplistic. In conversational recommender systems, it is important to consider finer item characteristics that can be obtained from diverse evaluations in user reviews while engaging in conversational interactions with AI. Furthermore, in conversational recommender systems, it is crucial to incorporate language models (LM) to capture linguistic variations and users' intuitive expressions in order to reflect them in the recommendations. However, this aspect is not well addressed in the paper (though it states a little in Section 4).\n\n\n(3) The evaluation setup of this paper raises some concerns. While the main evaluation metric is the number of turns, the criteria used to determine correctness seem to be based solely on whether an item was checked or not. In the case of hotels, for example, it is important to consider whether a reservation was actually made, and relying solely on item checking may lead to an inflated number of correct predictions.\nAnd, while evaluations are performed on different datasets, there is a lack of detailed explanations about the specifics of the conversations and the actual recommendations made in each dataset. Without this information, it becomes difficult to fully understand the effectiveness and characteristics of the proposed method.\nFurthermore, the experimental results of this paper do not include comparisons with other methods or existing approaches. It is important to compare the proposed method with other recommendation techniques in order to clearly demonstrate its advantages and improvements.\nMoreover, it appears that there is a lack of qualitative evaluation. While the paper utilizes multiple experimental datasets for validation, it is difficult to grasp the nature of the interactions and how satisfactory item recommendations are achieved in each dataset. Given that the conversation is a crucial aspect, the absence of information on the nature of the conversations makes it challenging to form a clear picture of the system's performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your suggestions. Please also see the main response above.\n\n> CORE heavily relies on decision trees.\n\nWe want to emphasize that the core idea of our online decision tree is how to compute the certainty gain, as shown in Section 3.1. As the estimated score for each item is calculated offline, there are no extra computation costs to get these scores online. Therefore, CORE also requires the computation to form a chain (as exemplified in Figure A2) instead of an entire tree. \n\nFurthermore, since the computation of certainty gain is an non-parametric equation, therefore, the computationally costly part would lie in addressing continuous features. But in the context of RS, most of features (e.g., 88% features in Tmall) are discrete features that is efficient to compute.       \n\n> It is needed to consider finer characters that can be obtained from diverse evaluations in user reviews.\n\nWe apologize that we might not fully understand the review’s meaning in terms of “finer characters”. But as a brief background context, CORE could get these “finer characters” (some complicated user perference hidden in textual responses) by designing specific prompts. For example, a teenager would say that her budget is limited as she is a student in conversations. Our Chatbot APIs would encode these information, and if we ask Chatbot APIs by what her preference on attribute “Price”. Chatbot APIs would return “Price: $1-100”. In this case, we can remove attribute Price from our unchecked attributes. We will show more case studies in our revisions to show that CORE would encode these characters by prompt designs. \n\n> The evaluation setup of this paper raises some concerns.\n\nFirstly, these evaluation metrics are widely used in previous conversational RS papers. Secondly, we agree that these metrics lack of qualitative evaluation. Therefore, we will list some examples of generated converations by CORE and compare them with other conversational RS algorithms by human evaluations. In this regards, each conversation would be evaluated both from how likely the queried items reach user needs and to what extent the texts are friendly to user. Due to time limitation of rebuttal period, we plan to add them in our revision.\n\n\n> Are there any existing conversational recommender systems that could serve as comparative methods to the proposed approach in this paper?\n\nOur main topic focus on how to bridge RS to conversational agent, and therefore, our baselines lie in existing conversational RS methods including CRIF [1], UNICORN [2], EAR [3], and CRM [4]. And, the ability of analyzing user textual response largely depends on what kind of LMs (or conversational components) are used in the method. As CORE can utilize Chatbot APIs, therefore, our ability of analyzing textual responses totally depend on what LLMs are used.\n\n[1] Learning to Infer User Implicit Preference in Conversational Recommendation. 2022.\n\n[2] Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning. 2021.\n\n[3] Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. 2020. \n\n[4] Conversational recommender system. 2018. \n"}]

Author rebuttal:
We summarize our responses and the results of the suggested experiments here. We also respond to every specific concern of each reviewer as individual comments below.

To summarize our contributions: (i) CORE is a plug-and-play method that can enable any (offline) RS to recommend (i.e., query) items and attribute values online. For this purpose, we develop an offline-training and online-checking paradigm, where we regard RS as an offline estimator and an online conversational agent as an online checker. Our proposed online decision tree algorithm can leverage offline estimations from RS to decide what to query online. (ii) CORE can utilize pre-trained Chatbot APIs, since many powerful Chatbots are too heavy to be jointly optimized with RS, even some of them can be only accessed through APIs such as ChatGPT-3.5. (iii) RL often requires large numbers of training samples (known as data insufficiency) and relatively small action space (corresponding to querying attributes in our paper). Also, without sufficient training data, RL could not well generalize to open-world cases. Therefore, in many real-world RS cases, RL might not get its best performance; while CORE, a learning-free method (our online decision tree does not have any parameters to tune online), could achieve stable performance.   

As one of the main concerns lies in the detailed comparisons between CORE and RL, below is a list of new experiments.
1. Comparison with [1] (CRIF) and [2] (UNICORN). Results (Tables R1 and R2) verify that CORE can perform well for querying attribute values, while RL methods excel at querying attributes because querying attribute values holds a huge action space which is not friendly for RL.
2. Comparisons with a new user simulation with a diversity metric. Results (Table R3) show that RL excels at giving diverse recommendations and therefore can well address the Matthew effect when there are multiple rounds in real-world cases, while CORE is good at giving specific recommendations.
3. Ablation studies with different amounts of training data. Results (Table R4) show that CORE is relatively stable with few training samples since RL often requires a huge number of training data.

Tables R1 to R4 can be found in uploaded PDF. Due to time limitations, we only conduct the experiments on LastFM and Amazon datasets, we plan to provide a completed form in our final version. 

Another concern lies in whether CORE would be significantly affected by RS. Below is a list of old and new experiments answering this:
1. Tables 1 to 4 (A1 to A4) provide the results of cold-start setting (on user side). In these cases, RS knows nothing about the user, and our results show that CORE also could outperform other baselines. Besides, Lemma 1 also provides a bound for the expected number of turns in the context of cold-start setting. These empirical results and theoretical analysis guarantee our performance when RS performs poorly (i.e., assigning the same scores to all the items).
2. Tables 1 to 4 (A1 to A4) also include different RS methods (with different RS performance). Results show that CORE can consistently benefit their RS compared to other baselines. 
3. Table R4 shows the result of ablation studies. Different amounts of training data often lead to different RS performances, and CORE can stably outperform other baselines.

[1] Learning to Infer User Implicit Preference in Conversational Recommendation. 2022.

[2] Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning. 2021.


Meta-review: Although scores are ever-so-slightly borderline, they tend toward accepting the paper. The discussion was fairly active and rebuttal had a positive impact on scores.
Decision: Accept (poster)