SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table A3: Results comparison of querying attribute values and items on sequential datasets.
ΨRE(·)	ΨCO(·)	Taobao	Tmall	Alipay
T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5
COLD START	AG	6.30	0.15	7.55	0.27	6.80	0.04	8.54	0.09	6.47	0.11	7.95	0.19
ME	6.43	0.14	7.82	0.29	6.76	0.05	8.50	0.12	6.71	0.07	8.46	0.11
CORE	5.42	0.39	5.04	0.89	6.45	0.13	7.38	0.37	5.98	0.25	6.17	0.65
CORED+	5.41	0.40	5.05	0.90	6.34	0.17	7.14	0.40	5.91	0.28	6.12	0.68
FM	AG	3.03	0.70	3.17	0.81	3.57	0.58	4.32	0.61	2.99	0.84	3.20	0.87
CORE	3.01	0.87	2.95	1.00	3.53	0.69	4.14	0.86	3.37	0.90	3.29	0.97
CORED+	3.02	0.88	2.91	1.00	3.50	0.71	4.11	0.87	3.32	0.91	3.14	0.97
DEEP FM	AG	2.99	0.72	2.93	0.89	4.38	0.46	5.23	0.52	3.03	0.83	3.22	0.87
CORE	2.73	0.92	2.78	0.99	4.31	0.62	4.43	0.84	3.17	0.87	3.18	0.97
CORED+	2.68	0.94	2.80	1.00	4.13	0.65	4.42	0.85	3.12	0.87	3.17	0.97
PNN	AG	2.93	0.76	2.87	0.92	3.98	0.52	4.60	0.61	3.18	0.88	2.94	0.91
CORE	2.51	0.98	2.64	1.00	3.20	0.64	4.11	0.90	3.19	0.88	3.15	0.98
CORED+	2.48	0.98	2.61	1.00	3.20	0.65	4.02	0.94	3.18	0.88	3.11	0.98
DIN	AG	2.71	0.85	2.83	0.95	4.14	0.51	4.81	0.59	3.10	0.82	3.35	0.85
CORE	2.45	0.97	2.54	1.00	4.12	0.64	4.16	0.89	3.25	0.83	3.32	0.96
CORED+	2.44	0.97	2.50	1.00	4.10	0.66	4.12	0.91	3.22	0.85	3.30	0.97
GRU	AG	2.80	0.80	2.64	0.97	3.82	0.56	4.40	0.64	3.17	0.83	3.29	0.87
CORE	2.31	0.98	2.44	1.00	3.81	0.72	3.91	0.92	3.10	0.84	3.11	0.96
CORED+	2.96	0.99	2.40	1.00	3.78	0.74	3.90	0.93	3.10	0.84	3.12	0.95
LSTM	AG	2.60	0.85	2.52	0.97	4.73	0.41	5.63	0.49	3.43	0.78	3.27	0.89
CORE	2.37	0.97	2.49	1.00	4.58	0.55	4.36	0.90	3.03	0.84	3.16	0.97
CORED+	2.30	0.98	2.49	1.00	4.56	0.57	4.34	0.91	3.05	0.85	3.18	0.97
MMOE	AG	3.04	0.75	2.98	0.92	4.10	0.54	4.56	0.62	3.58	0.83	3.90	0.92
CORE	2.48	0.96	2.60	1.00	3.92	0.65	4.19	0.85	3.21	0.91	3.17	0.98
CORED+	2.46	0.97	2.61	1.00	3.90	0.66	4.20	0.84	3.19	0.89	3.12	0.99
Table A4: Results comparison of querying attribute values and items on graph datasets.
ΨRE(·)	ΨCO(·)	Douban Movie	Douban Book
T@3	S@3	T@5	S@5	T@3	S@3	T@5	S@5
COLD START	AG	6.52	0.11	7.94	0.21	6.36	0.15	7.68	0.26
ME	6.60	0.10	8.16	0.21	6.40	0.15	8.04	0.24
CORE	5.48	0.38	4.84	0.94	5.96	0.26	5.08	0.92
CORED+	5.45	0.40	4.81	0.94	5.91	0.28	4.98	0.94
GAT	AG	3.75	0.63	3.65	0.87	3.56	0.64	3.41	0.87
CORE	2.89	0.91	2.97	1.00	2.80	0.92	2.91	1.00
CORED+	2.87	0.92	2.96	1.00	2.81	0.93	2.90	1.00
GCN	AG	3.21	0.69	3.33	0.83	3.20	0.71	3.18	0.89
CORE	2.76	0.92	2.81	1.00	2.85	0.91	2.85	1.00
CORED+	2.74	0.93	2.80	1.00	2.83	0.93	2.78	1.00
A4.2 Incorporating Our Conversational Agent with a Frozen Chat bot
With the development of pre-trained large language models (LLMs), chat-bots built based on these LLMs are capable of communicating like humans, which is a powerful tool to allow our conversational agent to extract the key information from the user's free text feedback and generate free text for querying attributes and items. Concretely, chat bot can act as either a question generator or a answer extractor. As shown in Figure 2, if our conversational agent decides to query attribute breakfast service, then the command passes to the question generator to generate a free text question "Do you require breakfast service?" The user answers the question by free text "I do not care about breakfast service, and I really want a hotel with shower", and then the answer extractor extracts the user preference on the given answer, namely the user does not care about attribute breakfast service and gives positive feedback on attribute shower.
For this purpose, we follow a short OpenAI tutorial for prompt engineering to design the following prompts based on gpt-3.5-turbo model.
# Load the API key and relevant Python libraries.  
import openai  
import os  
def get Completion(prompt, model="gpt-3.5-turbo"):  
    messages = ["role": "user", "content": prompt]  
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=0, # this is the degree of randomness of the model's output)  
    return responsechoices[0].message["content"]
We first evaluate using an LLM as a question generator by an example of generating a question to query an attribute, e.g., breakfast service.
Large language model as question generator.   
text  $=$  f""   
Attribute, Breakfast Service, Hotel   
""   
prompt  $=$  f""   
You will be provided with text delimited by triple quotes.   
If it starts with the word "item", it denotes an item,   
then you should generate a text to recommend the item to the user.   
Otherwise, it denotes an attribute,   
then you should generate a text to query the user's preference on the attribute.   
You should be gentle.   
{text}""   
""   
response  $\equiv$  get Completion(prompt)   
print(response)
The following is the corresponding output provided by the LLM.
Good day! May I ask for your preference regarding breakfast service in a hotel? Would you like to have a complimentary breakfast or do you prefer to have the option to purchase breakfast at the hotel restaurant?
We then evaluate using an LLM as a question generator by an example of generating a question to query (i.e., recommend) an item, e.g., hotel A.
Large language model as question generator.   
text  $=$  f""   
Item, Hotel A   
""   
prompt  $=$  f""   
You will be provided with text delimited by triple quotes.   
If it starts with the word "item", it denotes an item,   
then you should generate a text to recommend the item to the user.   
Otherwise, it denotes an attribute,   
then you should generate a text to query the user's preference on the attribute.   
You should be gentle.   
{text}""   
""   
response  $\equiv$  get Completion(prompt)   
print(response)
$^{2}$ https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
Great choice! Hotel A is a wonderful option. Would you like me to provide more information about the hotel or help you book a room?
Also, we evaluate using an LLM as an answer extractor by an example of extracting the user preference on attributes, e.g., breakfast service, hotel level, and shower.
text  $=$  f""   
I do not care about breakfast service, and I really want a hotel with a shower.   
""   
prompt  $=$  f""   
You will be provided with text delimited by triple quotes.   
If you can infer the user preference on attributes,   
then re-write the text in the following format: [attribute name]: [user perference]   
Attribute names include Breakfast Service, Hotel Level, and Shower.   
User preference includes Yes to denote the positive preference, No to denote the negative preference, and Not Care to denote the user does not care.   
If you can not infer the user preference on attributes,   
then re-write the text in the following format: [attribute name]: Not Know   
"{'text}'""   
""   
response  $\equiv$  get Completion(prompt)   
print(response)
The following is the corresponding output by the LLM.
Breakfast Service: Not Care Hotel Level: Not Know Shower: Yes
Similarly, we also can evaluate using an LLM as a question generator by an example of generating a question to query an attribute value, e.g., Hotel Level=5.
Large language model as question generator.   
text  $=$  f""   
Attribute, Hotel Level is 5, Hotel   
""   
prompt  $=$  f""   
You will be provided with text delimited by triple quotes.   
If it starts with the word "item", it denotes an item \   
then you should generate a text to recommend the item to the user.   
Otherwise, it denotes an attribute \   
then you should generate a text to query the user's preference on the attribute.   
You should be gentle.   
{text}""   
""   
response  $=$  get Completion(prompt)   
print(response)
Excuse me, may I ask for your preference on hotel level? Would you prefer a 5-star hotel or are you open to other options?
According to Definition A1, we have exemplified querying (i), (ii), and (iii) in the above examples. We further evaluate querying (iv). Namely, we evaluate using an LLM as a question generator by an example of generating a question to query whether the user preference is not smaller than an attribute value, e.g., Hotel Level not smaller than 3.
Figure A1: An illustrated example of an online decision tree in the setting of querying items and attribute values, where the target item is 0001062395.
Large language model as question generator.   
text  $=$  f""   
Attribute, Hotel Level is not smaller than 3, Hotel   
""   
prompt  $=$  f""   
You will be provided with text delimited by triple quotes.   
If it starts with the word "item", it denotes an item \   
then you should generate a text to recommend the item to the user.   
Otherwise, it denotes an attribute \   
then you should generate a text to query the user's preference on the attribute.   
You should be gentle.   
{"{text}""   
""   
response  $=$  get Completion(prompt)   
print(response)
Excuse me, may I ask for your preference on hotel level? Would you prefer a hotel with a level of 3 or higher?
We note that if there are too many attribute IDs (or too many attribute values) in the use case, then it might need to further incorporate some hierarchical designs [3] and ambiguous matching [45] into the above system, which is out of the scope of this paper, and we leave it for future work.
A4.3 Visualization and Case Study
We investigate a case in Amazon dataset in the setting of querying items and attribute values, where the target item is 0001062395. We depict the online decision tree in Figure A1. From the figure, we can see that the conversational agent first queries a value of attribute Category, then queries (i.e., recommends) an item 000161100; and after that, it queries another attribute, i.e., Price, and finally queries an item 0001062395.
Compared to Figure 1(b), this example seems like a chain. The main reason is that in practice, the user would give the corresponding answer to the query at each turn. Therefore, the binary tree (in the setting of querying items and attribute values) would reduce to a chain.
From this case, we also can observe that our conversational agent is capable of jointly considering the items and attributes to search for the target items in the session.
A5 Connections to Existing Approaches
A5.1 Connections to Conversational Recommender Systems
Bridging recently emerged conversational techniques and recommender systems becomes an appealing solution to model the dynamic preference and weak explainability problems in recommendation task [15, 26], where the core sub-task is to dynamically select attributes to query and make recommendations upon the corresponding answers. Along this line, one popular direction is to build a conversational recommender system, which combines the conversational models and the recommendation models from a systematic perspective. In other words, these models are treated and learned as two individual modules [30, 2, 44, 48]. For example, compared to previous literature [5, 6, 47], recent work [49] builds the systems upon the multiple turn scenarios; unfortunately, it does not investigate when to query attributes and when to make recommendations (i.e., query items). To solve this issue, prior works [30, 31] develop reinforcement learning-based solutions. However, all these previous methods based on a reinforcement learning framework are innately suffering from insufficient usage of labeled data and high complexity costs of deployment.
Instead, CORE can be seamlessly adopted to any recommendation method (especially those widely adopted supervised learning-based recommendation methods), and is easy to implement due to our conversational strategy based on the uncertainty minimization theory.
A5.2 Connections to (Offline) Decision Tree Algorithms
Decision tree algorithms [24] such as ID3 and C4.5 were proposed based on information theory, which measures the uncertainty in each status by calculating its entropy. If we want to directly adopt the entropy measurement for the conversational agent, then one possible definition of entropy is
$$
\mathrm {H} _ {k} = - \sum_ {y \in \mathcal {Y}} \left(\frac {\left| \mathcal {V} _ {\text {y} \text {v a l u e}} = y \cap \mathcal {V} _ {k} \right|}{\left| \mathcal {V} _ {k} \right|} \log \frac {\left| \mathcal {V} _ {\text {y} \text {v a l u e}} = y \cap \mathcal {V} _ {k} \right|}{\left| \mathcal {V} _ {k} \right|}\right), \tag {40}
$$
where  $\mathbb{H}_k$  is the empirical entropy for  $k$ -th turn,  $\mathcal{V}$  is the set of all the labels, and  $\mathcal{V}_{y\mathrm{value}} = y$  is the set of items whose label is  $y$ . For convenience, we call this traditional decision tree as offline decision tree.
The main difference between the previous offline decision tree and our online decision tree lies in that our online decision tree algorithm does not have labels to measure the "uncertainty", instead, we have access to the estimated relevance scores given by recommender systems. We also note that directly using the user's previous behaviors as the labels would lead to a sub-optimal solution, because (i) offline labels in collected data are often biased and can only cover a small number of candidate items, and (ii) offline labels only can reflect the user's previous interests, but the user's preferences are always shifting.
To this end, we measure the uncertainty in terms of the summation of the estimated relevance scores of all the unchecked items after previous  $(k - 1)$  interactions. Formally, we define our uncertainty as:
$$
\mathrm {U} _ {k} = \operatorname {S U M} \left(\left\{\Psi_ {\mathrm {R E}} \left(v _ {m}\right) \mid v _ {m} \in \mathcal {V} _ {k} \right\}\right), \tag {41}
$$
where  $\Psi_{\mathrm{RE}}(\cdot)$  denotes the recommender system. Similar to the information gain in the offline decision tree, we then derive the definition of certainty gain (as described in Definition 1), and formulate the conversational agent into an uncertainty minimization framework.
A5.3 Connections to Recommendation Approaches to Addressing Cold-start Issue
Cold-start issues are situations where no previous events, e.g., ratings, are known for certain users or items [29, 43]. Commonly, previous investigations have revealed that the more (side) information, the better the recommendation results. In light of this, roughly speaking, there are two main branches to address the cold-start problem. One direction is to combine the content information into collaborative filtering to perform a hybrid recommendation [1, 19, 36, 42], and a recent advance [34, 50] proposes to further combine the cross-domain information to the recommender system. The other direction
is to incorporate an active learning strategy into the recommender system [14, 40], whose target is to select items for the newly-signed users to rate. For this purpose, representative methods include the popularity strategy [17], the coverage strategy [16], and the uncertainty reduction strategy [41], where the first one selects items that have been frequently rated by users, the second one selects items that have been highly co-rated with other items, and the third one selects items that can help the recommender system to better learn the user preference.
Integrating the conversational agent with the recommender system offers a promising solution to address the cold start issue by effectively leveraging user interactions. This approach allows for querying additional information from users, thereby enhancing the system's understanding of their preferences and requirements.
A distinctive aspect of our conversational agent's strategy compared to existing active learning strategies, lies in its primary objective. While active learning strategies aim to improve the recommender system's performance, our conversational agent prioritizes meeting the user's immediate needs during the ongoing session. Consequently, the offline-training and online-checking paradigm employed in our approach assigns higher relevance scores to items that are considered "uncertain." This diverges from conventional settings where uncertainty estimation is typically conducted independently of relevance estimation.
By incorporating the conversational agent into recommender system, we leverage user interactions to gather more comprehensive information, which can effectively address the challenges associated with cold-start scenarios. Our primary focus is to provide an optimal user experience in real-time rather than solely optimizing the recommender system's accuracy. This novel paradigm emphasizes the importance of uncertainty estimation in relevance estimation, setting our approach apart from traditional methods.