SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1 CORE for Querying Items and Attributes
Input: A recommender system  $\Psi_{\mathrm{RE}}(\cdot)$ , an item set  $\nu$ , an attribute set  $\mathcal{X}$ , an offline dataset  $\mathcal{D}$ .  
Output: Updated recommender system  $\Psi_{\mathrm{RE}}(\cdot)$ , up-to-date dataset  $\mathcal{D}$ .  
1: Train  $\Psi_{\mathrm{RE}}(\cdot)$  on  $\mathcal{D}$ .  
2: for each session (i.e., the given user) do  
3: Initialize  $k = 1$  and  $\mathcal{V}_0 = \mathcal{V}$ ,  $\mathcal{X}_0 = \mathcal{X}$ .  
4: repeat  
5: Compute  $a_{\mathrm{query}}$  following Eq. (3) for querying items and attributes or following Eq. (10) for querying items and attribute values.  
6: Query  $a_{\mathrm{query}}$  to the user and receive the answer.  
7: Generate  $\mathcal{V}_k$  and  $\mathcal{X}_k$  from  $\mathcal{V}_{k-1}$  and  $\mathcal{X}_{k-1}$ .  
8: Go to next turn:  $k \gets k + 1$ .  
9: until Querying the target item or  $k > K_{\mathrm{MAX}}$  where  $K_{\mathrm{MAX}}$  is the maximal number of turns.  
10: Collect session data and add to  $\mathcal{D}$ .  
11: end for  
12: Update  $\Psi_{\mathrm{RE}}(\cdot)$  using data in  $\mathcal{D}$ .  
> Offline-Training
by setting the action space to  $\mathcal{A} = \mathcal{W}_x\times \mathcal{X}_{k - 1}\cup \mathcal{V}_{k - 1}$ . Then, we can re-formulate Eq. (3) as:
$$
a _ {\text {q u e r y}} = \underset {a \in \{w _ {x}, v \}} {\arg \max } \left(\underset {w _ {x} \in \mathcal {W} _ {x} \text {w h e r e} x \in \mathcal {X} _ {k - 1}} {\max } \Psi_ {\mathrm {C G}} (\text {q u e r y} (x) = w _ {x}), \underset {v \in \mathcal {V} _ {k - 1}} {\max } \Psi_ {\mathrm {C G}} (\text {q u e r y} (v))\right). \tag {10}
$$
In the context of querying attribute values, we further reveal what kind of attribute value is an ideal one in the following theorem.
Proposition 2. In the context of querying attribute values, an ideal choice is always the one that can partition all the unchecked relevance scores into two equal parts (i.e., the ideal  $w_{x} \in \mathcal{W}_{x}, x \in \mathcal{X}_{k-1}$  is the one that makes  $\Psi_{\mathrm{CG}}(w_{x} = w_{x}^{*}) = \mathrm{SUM}(\{\Psi_{\mathrm{RE}}(v_{m}) | v_{m} \in \mathcal{V}_{k-1}\}) / 2$  hold), if it is achievable. And the certainty gain in this case is  $\Psi_{\mathrm{CG}}(\text{query}(x) = w_{x}) = \mathrm{SUM}(\{\Psi_{\mathrm{RE}}(v_{m}) | v_{m} \in \mathcal{V}_{k-1}\}) / 2$ .
Then, we consider the bound of the expected number of turns. To get rid of the impact of  $\Psi_{\mathrm{RE}}(\cdot)$ , we introduce a cold-start setting [43], where  $\Psi_{\mathrm{RE}}(\cdot)$  knows nothing about the user, and equally assigns relevance scores to all  $M$  items, resulting in  $\Psi_{\mathrm{RE}}(v_m) = 1 / M$  holds for any  $v_m \in \mathcal{V}$ .
Lemma 1. In the context of querying attribute values, suppose that  $\Psi_{\mathrm{RE}}(v_m) = 1 / M$  holds for any  $v_m \in \mathcal{V}$ , then the expected number of turns (denoted as  $\widehat{K}$ ) is bounded by  $\log_2^{M + 1} \leq \widehat{K} \leq (M + 1) / 2$ .
Here, the good case lies in that our conversational agent is capable of finding an attribute value to form an ideal partition at each turn, while the bad case appears when we can only check one item at each turn. We provide detailed proofs of Propositions 1 and 2, and Lemma 1 in Appendix A1.2.
$\Psi_{\mathbb{CG}}(\cdot)$  for Querying Attributes in Large Discrete or Continuous Space. All the above querying strategies are designed in the context that for each attribute, the range of its candidate values is a "small" discrete space, namely  $|\mathcal{W}_x| \ll |\mathcal{V}_{k-1}|$  where  $x \in \mathcal{X}_{k-1}$ . When it comes to cases where  $\mathcal{W}_x$  is a large discrete space or a continuous space, then either querying attribute  $x$  or any attribute value  $w_x \in \mathcal{W}_x$  would not be a good choice. For example, let  $x$  be Hotel Price, then when querying  $x$ , the user would not respond with an accurate value, and querying  $x=$  one possible value could be ineffective. To address this issue, we propose to generate a new attribute value  $w_x$  and query whether the user's preference is not smaller than it or not. Formally, we have:
$$
\Psi_ {\mathrm {C G}} (\text {q u e r y} (x) \geq w _ {x}) = \Psi_ {\mathrm {C G}} \left(w _ {x} \geq w _ {x} ^ {*}\right) \cdot \Pr \left(w _ {x} \geq w _ {x} ^ {*}\right) + \Psi_ {\mathrm {C G}} \left(w _ {x} <   w _ {x} ^ {*}\right) \cdot \Pr \left(w _ {x} <   w _ {x} ^ {*}\right), \tag {11}
$$
where  $x \in \mathcal{X}_{k-1}$  and  $w_x$  can be either in or out of  $\mathcal{W}_x$ . Compared to querying attribute values (i.e., Eq. (9)), the new action space is  $\mathcal{A} = \mathbb{R} \times \mathcal{X}_{k-1}$ . Notice that Proposition 2 is also suitable for this case (see detailed description in Appendix A2.1), where the best partition is to divide the estimated relevance scores into two equal parts. Therefore, we produce  $w_x$  by averaging all the candidate attribute values weighted by the corresponding relevance scores. Formally, for each  $x \in \mathcal{X}_{k-1}$ , we compute  $w_x$  as:
$$
w _ {x} = \text {A V E R A G E} \left(\left\{\Psi_ {\mathrm {R E}} \left(v _ {m}\right) \cdot w _ {v _ {m}} \mid v _ {m} \in \mathcal {V} _ {k - 1} \right\}\right), \tag {12}
$$
where  $w_{v_m}$  is the value of attribute  $x$  in item  $v_m$ , e.g., in Figure 1(a), let  $a$  be Hotel Level, and  $v_m$  be Hotel A, then  $w_{v_m} = 3$ .
Figure 2: We illustrate the empowerment of our conversational agent through the utilization of a pre-trained chat bot. In this context, the red box signifies the chat bot-empowered conversational agent. To accomplish this, we input the output queries produced by the original conversational agent, such as Breakfast Service into the question generator, as depicted in (a). In (b), a user is expected to input the generated question in a free-text format and provide the corresponding answer in a free-text format. Subsequently, in (c), the answer extractor extracts key information from the user's response and provides it to the original conversational agent.
In this case,  $\mathcal{X}_k = \mathcal{X}_{k-1}$ , and  $\mathcal{V}_k = \mathcal{V}_{k-1} \setminus \mathcal{V}_{x_{\mathrm{value}} < w_x}$  if receiving Yes from the user when querying whether user preference is not smaller than  $w_x$ ,  $\mathcal{V}_k = \mathcal{V}_{k-1} \setminus \mathcal{V}_{x_{\mathrm{value}} \geq w_x}$  otherwise.  $\mathcal{V}_{x_{\mathrm{value}} < w_x}$  is the set of all the items whose value of  $x$  is smaller than  $w_x$  and  $\mathcal{V}_{x_{\mathrm{value}} \geq w_x}$  is the set of the rest items.
3.3 Plugging the Conversational Agent into Recommender Systems
Overall Algorithm. We begin by summarizing CORE for querying items and attributes or querying items and attribute values in Algorithm 1. From the algorithm, we can clearly see that our  $\Psi_{\mathsf{C0}}(\cdot)$  puts no constraints on  $\Psi_{\mathrm{RE}}(\cdot)$  and only requires the estimated relevance scores from  $\Psi_{\mathrm{RE}}(\cdot)$ , therefore, CORE can be seamlessly integrated into any recommendation platform. We note that in a conversational agent, querying attributes and querying attribute values can be compatible, but can not simultaneously operate on the same attribute, due to Proposition 1. See Appendix A2.3 for a detailed discussion.
Making  $\Psi_{\mathrm{CG}}(\cdot)$  Consider Dependence among Attributes. We notice that the above formulations of either querying attributes or querying attribute values, does not consider the dependence among attributes (e.g., as Figure 1(a) shows, attribute Hotel Level can largely determine attribute Shower Service). To address this issue, we take  $\Psi_{\mathrm{CG}}(\cdot)$  in Eq. (6) as an example (see detailed descriptions of the other  $\Psi_{\mathrm{CG}}(\cdot)$ s in Appendix A2.2), and re-formulate it as:
$$
\Psi_ {\mathrm {C G}} ^ {\mathrm {D}} (\operatorname {q u e r y} (a)) = \sum_ {a ^ {\prime} \in \mathcal {X} _ {k - 1}} \left(\Psi_ {\mathrm {C G}} (\operatorname {q u e r y} \left(a ^ {\prime}\right)) \cdot \Pr (\operatorname {q u e r y} \left(a ^ {\prime}\right) | \operatorname {q u e r y} (a))\right), \tag {13}
$$
where  $a \in \mathcal{X}_{k-1}$ , and  $\operatorname*{Pr}(\text{query}(a')|\text{query}(a))$  measures the probability of the user preference on  $a$  determining the user preference on  $a'$ . Compared to  $\Psi_{\mathrm{CG}}(\text{query}(a))$ ,  $\Psi_{\mathrm{CG}}^{\mathrm{D}}(\text{query}(a))$  further considers the impact of querying attribute  $a$  on other attributes. To estimate  $\operatorname*{Pr}(\text{query}(a')|\text{query}(a))$ , we develop two solutions. We notice that many widely adopted recommendation approaches are developed on factorization machine (FM) [38], e.g., DeepFM [20]. Therefore, when applying these FM-based recommendation approaches, one approach is to directly adopt their learned weight for each pair of attributes  $(a,a')$  as the estimation of  $\operatorname*{Pr}(\text{query}(a')|\text{query}(a))$ . When applying CORE to any other recommendation method (e.g., DIN [52]), we develop a statistics-based approach that does estimations by computing this conditional probability  $\Psi_{\mathrm{CG}}^{\mathrm{D}}(\text{query}(a))$  based on the given candidate items. We leave the detailed computations of  $\Psi_{\mathrm{CG}}^{\mathrm{D}}(\text{query}(a))$  in both ways in Appendix A2.2.
Empowering  $\Psi_{\mathsf{C0}}(\cdot)$  to Communicate with Humans. When applying CORE into real-world scenarios, users may provide a Not Care attitude regarding the queried attributes or queried attribute values. In these cases, we generate  $\mathcal{V}_k$  and  $\mathcal{X}_k$  by  $\mathcal{V}_k = \mathcal{V}_{k-1}$  and  $\mathcal{X}_k = \mathcal{X}_{k-1} \setminus \{a\}$ , because querying  $a$  is non-informative. To capture the user's different attitudes on queried items and attributes or attribute values, we can incorporate a pre-trained large language model (LLM) (e.g., gpt-3.5-turbo) in  $\Psi_{\mathsf{C0}}(\cdot)$ . As our online-checking part does not require training, simply plugging an LLM would not cause the non-differentiable issue. In light of this, we exemplify some task-specific prompts to enable the conversational agent to (i) produce questions by prompting queried items and attributes, and (ii) extract the key messages from the user's answers. As shown in Figure 2, a conversational agent consisting of an LLM-chat bot and our online decision tree algorithm would communicate like humans. We also provide some case studies of the conversational agent in the context of a question generator and an answer extractor. See Appendix A4.2 for a detailed description.
4.1 Experimental Configurations
We summarize different experimental settings as follows. (i) We design two different quering strategies regarding attributes (shown in line 5 in Algorithm 1). One is querying attributes (i.e., attribute IDs); and the other is querying attribute values. (ii) We introduce two different recommender system settings. One is the hot-start setting (shown in line 1 in Algorithm 1) that initializes the estimated relevance scores of items by a given pre-trained recommender system; and the other is the cold-start setting where those estimated relevance scores are uniformly generated (corresponding to the case where the recommender system knows nothing about the given user). Because the conversational agent  $\Psi_{\mathbb{C0}}(\cdot)$  operates in a dynamic process, we de
Table 1: Results comparison in the context of querying attributes. See Table A1 for the full version.
ΨRE(·)	ΨCO(·)	Amazon
T@3	S@3	T@5	S@5
COLD START	ME	3.04	0.98	5.00	1.00
CORE	2.88	1.00	2.87	1.00
CORED+	2.84	1.00	2.86	1.00
FM	AG	2.76	0.74	2.97	0.83
CRM	3.07	0.98	3.37	1.00
EAR	2.98	0.99	3.13	1.00
CRIF	2.84	1.00	2.64	1.00
UNICORN	2.65	1.00	2.45	1.00
CORE	2.17	1.00	2.16	1.00
CORED+	2.14	1.00	2.14	1.00
velop a new simulator to simulate the Human-AI recommendation interactions, which consists of a conversational agent and a user agent. Specifically, for each user, we use her browsing log as session data, and treat all the items receiving positive feedback (e.g., chick) as target items. Then, for each  $k$ -th turn, when the conversational agent queries an attribute  $x \in \mathcal{X}_{k-1}$ , the user agent returns a specific attribute value if all the target items hold the same value for  $x$ ; otherwise, the user agent returns Not Care. When the conversational agent queries an attribute value  $w_x \in \mathcal{W}_x$ , the user agent returns Yes if at least one target item holds  $w_x$  as the value of attribute  $x$ ; otherwise, returns No.
For each experimental setting, we first set  $K_{\mathrm{MAX}}$ , and then evaluate the performance in terms of the average turns needed to end the sessions, denoted as  $\mathrm{T} @ K_{\mathrm{MAX}}$  (where for each session, if  $\Psi_{\mathrm{C0}}(\cdot)$  successfully queries the target item within  $K_{\mathrm{MAX}}$  turns, then return the success turn; otherwise, we enforce  $\Psi_{\mathrm{C0}}(\cdot)$  to query an item at  $(K_{\mathrm{MAX}} + 1)$ -th turn, if succeeds, return  $K_{\mathrm{MAX}} + 1$ , otherwise return  $K_{\mathrm{MAX}} + 3$ ); and the average success rate, denoted as  $S @ K_{\mathrm{MAX}}$  (where for each session, if  $\Psi_{\mathrm{C0}}(\cdot)$  does not successfully query the target item within  $K_{\mathrm{MAX}}$  turns, then we enforce  $\Psi_{\mathrm{C0}}(\cdot)$  to query an item after  $K_{\mathrm{MAX}}$  turns, if succeeds, return 1, otherwise return 0).
To verify CORE can be applied to a variety of recommendation platforms, we conduct evaluations on three tubular datasets: Amazon [8, 33], LastFM [9] and Yelp [12], three sequential datasets: Taobao [10], Tmall [11] and Alipay [7], two graph-structured datasets: Douban Movie [35, 53] and Douban Book [35, 53]. The recommendation approaches used in this paper, i.e.,  $\Psi_{\mathrm{RE}}(\cdot)\mathrm{s}$ , include FM [38], DEEP FM [20], PNN [37], DIN [52], GRU [23], LSTM [18], MMOE [32], GCN [27] and GAT [46]. We also use COLD START to denote the cold-start recommendation setting. The conversational methods used in this paper, i.e.,  $\Psi_{\mathrm{CO}}(\cdot)\mathrm{s}$ , include (i) Abs Greedy (AG) always queries an item with the highest relevance score at each turn; (ii) Max Entropy (ME) always queries the attribute with the maximum entropy in the context of querying attributes, or queries the attribute value of the chosen attribute, with the highest frequency in the context of querying attribute values; (iii) CRM [44], (iv) EAR [30], (v) CRIF [25], (vi) UNICORN [13]. Here, AG can be regarded as a strategy of solely applying  $\Psi_{\mathrm{RE}}(\cdot)$ . Both CRM and EAR are reinforcement learning based approaches, originally proposed on the basis of FM recommender system. Thus, we also evaluate their performance with hot-start FM-based recommendation methods, because when applying them to a cold-start recommendation platform, their strategies would reduce to a random strategy. Consider that ME is a  $\Psi_{\mathrm{CO}}(\cdot)$ , independent of  $\Psi_{\mathrm{RE}}(\cdot)$  (namely, the performance of hot-start and cold-start recommendation settings are the same); and therefore, we only report their results in the cold-start recommendation setting. We further introduce a variant of CORE, denoted as  $\mathrm{CORE}_{\mathrm{D}}^{+}$  where we compute and use  $\Psi_{\mathrm{CG}}^{\mathrm{D}}(\cdot)\mathrm{s}$  instead of  $\Psi_{\mathrm{CG}}(\cdot)\mathrm{s}$  in line 5 in Algorithm 1.
We provide detailed descriptions of datasets and data pre-processing, simulation design, baselines, and implementations in Appendix A3.1, A3.2, A3.3, and A3.4.
Table 2: Results comparison of querying attribute values on tabular datasets. See Table A2 for the full version.
ΨRE(·)	ΨCO(·)	Amazon	LastFM	Yelp
T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5
COLD START	AG	6.47	0.12	7.83	0.23	6.77	0.05	8.32	0.14	6.65	0.08	8.29	0.13
ME	6.50	0.12	8.34	0.16	6.84	0.04	8.56	0.11	6.40	0.15	8.18	0.20
CORE	6.02	0.25	6.18	0.65	5.84	0.29	5.72	0.74	5.25	0.19	6.23	0.65
CORED+	6.00	0.26	6.01	0.67	5.79	0.30	5.70	0.75	5.02	0.21	6.12	0.68
FM	AG	2.76	0.74	2.97	0.83	4.14	0.52	4.67	0.64	3.29	0.70	3.39	0.81
CRM	4.58	0.28	6.42	0.38	4.23	0.34	5.87	0.63	4.12	0.25	6.01	0.69
EAR	4.13	0.32	6.32	0.42	4.02	0.38	5.45	0.67	4.10	0.28	5.95	0.72
CRIF	4.34	0.28	6.24	0.45	3.98	0.58	4.11	0.76	4.07	0.31	6.02	0.70
UNICORN	4.43	0.30	6.15	0.52	4.00	0.54	7.56	0.11	4.40	0.25	5.38	0.77
CORE	3.26	0.83	3.19	0.99	3.79	0.72	3.50	0.99	3.14	0.84	3.20	0.99
CORED+	3.16	0.85	3.22	1.00	3.75	0.74	3.53	1.00	3.10	0.85	3.23	1.00
DEEP FM	AG	3.07	0.71	3.27	0.82	3.50	0.68	3.84	0.79	3.09	0.74	3.11	0.88
CRM	4.51	0.29	6.32	0.40	4.18	0.38	5.88	0.63	4.11	0.23	6.02	0.71
EAR	4.47	0.30	6.35	0.43	4.01	0.37	5.43	0.69	4.01	0.32	5.74	0.75
CORE	3.23	0.85	3.22	0.99	3.47	0.81	3.34	1.00	2.98	0.93	3.11	1.00
PNN	AG	3.02	0.74	3.10	0.87	3.44	0.67	3.53	0.84	2.83	0.77	2.82	0.91
CORE	3.01	0.88	3.04	0.99	3.10	0.87	3.20	0.99	2.75	0.88	2.76	1.00
Table 3: Results comparison of querying attribute values on sequential datasets. See Table A3 for the full version.
ΨRE(·)	ΨCO(·)	Taobao	Tmall	Alipay
T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5
COLD START	AG	6.30	0.15	7.55	0.27	6.80	0.04	8.54	0.09	6.47	0.11	7.95	0.19
ME	6.43	0.14	7.82	0.29	6.76	0.05	8.50	0.12	6.71	0.07	8.46	0.11
CORE	5.42	0.39	5.04	0.89	6.45	0.13	7.38	0.37	5.98	0.25	6.17	0.65
DIN	AG	2.71	0.85	2.83	0.95	4.14	0.51	4.81	0.59	3.10	0.82	3.35	0.85
CORE	2.45	0.97	2.54	1.00	4.12	0.64	4.16	0.89	3.25	0.83	3.32	0.96
GRU	AG	2.80	0.80	2.64	0.97	3.82	0.56	4.40	0.64	3.17	0.83	3.29	0.87
CORE	2.31	0.98	2.44	1.00	3.81	0.72	3.91	0.92	3.10	0.84	3.11	0.96
LSTM	AG	2.60	0.85	2.52	0.97	4.73	0.41	5.63	0.49	3.43	0.78	3.27	0.89
CORE	2.37	0.97	2.49	1.00	4.58	0.55	4.36	0.90	3.03	0.84	3.16	0.97
MMOE	AG	3.04	0.75	2.98	0.92	4.10	0.54	4.56	0.62	3.58	0.83	3.90	0.92
CORE	2.48	0.96	2.60	1.00	3.92	0.65	4.19	0.85	3.21	0.91	3.17	0.98
We report our results of querying attributes and items in Table 1, and the results of querying attribute values and items in Tables 2, and 3, 4, and summarize our findings as follows.
Reinforcement learning based methods work well in querying items and attributes but perform poorly in querying items and attribute values. By comparing Table 1 to Table 2, we can see a huge performance reduction of CRM and EAR. One possible explanation is that compared to attribute IDs, the action space of querying attribute values is much larger. Thus, it usually requires much more collected data to train a well-performed policy.
$\mathbf{T}@\mathbf{K}_{\mathrm{MAX}}$  could not align with  $\mathbf{S}@\mathbf{K}_{\mathrm{MAX}}$ . A higher success rate might not lead to fewer turns, and ME gains a worse performance than AG in some cases in the cold-start recommendation setting. The main reason is that although querying an attribute value can obtain an equivalent or more certainty gain than querying an item at most times, however, only querying (a.k.a., recommending) an item could end a session. Therefore, sometimes, querying an attribute value is too conservative. It explains why CORE outperforms AG in terms of S@3 but gets a lower score of T@3 in the Amazon dataset and FM recommendation base.
Our conversational agent can consistently improve the recommendation performance in terms of success rate. CORE can consistently outperform AG, in terms of success rate, especially for the cold-start recommendation setting. As AG means solely using recommender systems, it indicates that  $\Psi_{\mathsf{CO}}(\cdot)$  can consistently help  $\Psi_{\mathrm{RE}}(\cdot)$ . One possible reason is that our uncertainty minimization
Table 4: Results comparison of querying attribute values on graph datasets. See Table A4 for the full version.
ΨRE(·)	ΨC0(·)	Douban Movie	Douban Book
T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5
COLD START	AG	6.52	0.11	7.94	0.21	6.36	0.15	7.68	0.26
ME	6.60	0.10	8.16	0.21	6.40	0.15	8.04	0.24
CORE	5.48	0.38	4.84	0.94	5.96	0.26	5.08	0.92
GAT	AG	3.75	0.63	3.65	0.87	3.56	0.64	3.41	0.87
CORE	2.89	0.91	2.97	1.00	2.80	0.92	2.91	1.00
GCN	AG	3.21	0.69	3.33	0.83	3.20	0.71	3.18	0.89
CORE	2.76	0.92	2.81	1.00	2.85	0.91	2.85	1.00
Table 5: Result comparisons in the context of querying attribute values and items on tabular datasets, where we reduce the  ${50}\%$  training data of FM and DEEP FM.
ΨRE(·)	ΨCO(·)	Amazon	LastFM
T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5
FM	CRIF	5.14	0.18	6.78	0.41	4.98	0.33	6.34	0.56
UNICORN	5.13	0.23	6.89	0.42	5.01	0.34	5.94	0.45
CORE	3.86	0.73	3.98	0.80	3.99	0.65	4.04	0.86
DEEP FM	CRIF	5.42	0.39	5.98	0.57	4.61	0.53	4.46	0.67
UNICORN	4.34	0.45	5.12	0.62	4.35	0.45	4.67	0.65
CORE	3.96	0.78	4.23	0.85	3.75	0.72	3.83	0.86
framework unifies querying attribute values and items. In other words, AG is a special case of CORE, where only querying items is allowed.
Considering Dependence among attributes is helpful. Comparisons between  $\mathrm{CORE}$  and  $\mathrm{CORE}_{\mathrm{D}}^{+}$  reveal that considering the dependence among attributes could improve the performance of CORE in most cases.
We further investigate the impact of  $K_{\mathrm{MAX}}$  by assigning  $K_{\mathrm{MAX}} = 1,3,5,7,9$  and reporting the results of CORE and AG on Amazon dataset in the context of the cold-start and hot-start recommendation settings in Figure 3. The results further verify the superiority of CORE, especially with a cold-start  $\Psi_{\mathrm{RE}}(\cdot)$ .
Our conversational agent is more stable when the training data is limited. For further assessment of CORE's stability in comparison to the baseline methods, we conducted an evaluation by randomly selecting a subset of the training set comprising only  $50\%$  of the samples. The results of this evaluation are presented in Table 5. The table clearly demonstrates that CORE exhibits a higher level of stability in its performance when compared to existing reinforcement learning-based frameworks.
Figure 3: Comparisons of CORE and AG with different  $K_{\mathrm{MAX}}$  in both cold-start and hot-start settings.
We also provide a case study of incorporating an LLM into CORE to handle free-text inputs and output human language Appendix A4.2, where we provide detailed prompts. We further offer a visualization of an online decision tree in A4.3.
In this paper, we propose CORE that can incorporate a conversational agent into any recommendation platform in a plug-and-play fashion. Empirical results verify that CORE outperforms existing reinforcement learning-based and statistics-based approaches in both the setting of querying items and attributes, and the setting of querying items and attribute values. In the future, it would be interesting to evaluate CORE in some online real-world recommendation platforms.
Acknowledgement. The Shanghai Jiao Tong University team is supported by National Key R&D Program of China (2022ZD0114804), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102) and National Natural Science Foundation of China (62076161, 62177033). Jiarui Jin would like to thank Wu Wen Jun Honorary Doctoral Scholarship from AI Institute, Shanghai Jiao Tong University.