LLM - REDIAL : A Large-Scale Dataset for Conversational Recommender Systems Created from User Behaviors with LLM s (2024)

Abstract: The large-scale conversational recommendation dataset is pivotal for the development of conversational recommender systems (CRS). Most existing CRS datasets suffers from the problems of data inextensibility and semantic inconsistency. To tackle these limitations and establish a benchmark in the conversational recommendation scenario, in this paper, we introduce the LLM-REDIAL dataset to facilitate the research in CRS. LLM-REDIAL is constructed by leveraging large language models (LLMs) to generate the high-quality dialogues. To provide the LLMs with detailed guidance, we integrate historical user behavior data with dialogue templates that are carefully designed through the combination of multiple pre-defined goals. LLM-REDIAL has two main advantages. First, it is the largest multi-domain CRS dataset which consists of 47.6k multi-turn dialogues with 482.6k utterances across 4 domains. Second, dialogue semantics and the users’ historical interaction information is highly consistent. Human evaluation are conducted to verify the quality of LLM-REDIAL. In addition, we evaluate the usability of advanced LLM-based models on LLM-REDIAL.

## introduction in recent years, conversational recommender systems (crs) have been widely explored in both academia and industry @xcite @xcite , which leverage natural language conversations to provide users with personalized and context-aware recommendations. unlike the conventional recommender systems that rely solely on user-item interactions, crs incorporates the conversational aspect, allowing users to interact with the system through natural language. the existing crs methods are primarily data-driven, requiring large-scale conversational datasets for model training. in this connection, an increasing emphasis has been placed on dataset construction in the field of crs. there are a few efforts to build datasets for conversational recommendation @xcite @xcite . while these existing datasets have propelled the development of conversational recommendation to some extent, there are still the following limitations of two aspects: (1) data inextensibility. most of previous dataset construction require a lot of human annotations significantly limiting the dataset scalability. additionally, the quality of dialogue texts obtained through sentence retrieval or crowd-workers can not be guaranteed. insufficient quantity and quality of dialogues would hinder the training of crs models. even with the emergence of large language models (llms), this situation persists. while current llms demonstrate superior capabilities in text generation and data annotation @xcite , they exhibit less promising performance in conversational recommendation. consequently, large-scale conversational recommendation datasets remain a bottleneck in the development of crs. (2) semantic inconsistency. the surge of llms making the response generation in crs less challenging, and the [agent] that's great to hear! i have a movie recommendation for you based on your likes. how about checking out world war z? ## dataset statistics our llm-redial is constructed based on the amazon review dataset. there are 24 different domains and this work selects 4 of them to be the data sources. more domains will be used to generate more conversations in our future work. the llm-redial consists of 47,651 dialogues with 482,684 utterances across 4 domains. the statistics of our llm-redial are shown in ## conclusion this paper presents a large-scale multi-turn dialogue dataset for conversational recommendation which is constructed with llms based on the users' historical behaviours. we fill the user behaviour data into the well-designed dialogue template to guide the llms to generate high-quality dialogues. benefitting from the powerful generation capability of llms, llm-redial is the largest multidomain crs dataset with 47.6k dialogues covering recommendation process. comprehensive experiments are conducted to verify the quality and usability of our llm-redial. we believe that llm-redial can serve as a rich resource for advancing research in crs, assisting the community in proposing better methods for conversational recommendation within the context of llms.