R ec I n D ial: A Unified Framework for Conversational Recommendation with Pretrained Language Models (2022)

Abstract: Conversational Recommender System (CRS), which aims to recommend high-quality items to users through interactive conversations, has gained great research interest recently. A CRS is usually composed of a recommendation module and a generation module. In the previous work, these two modules are loosely connected in the model training and are shallowly integrated during inference, where a simple switching or copy mechanism is adopted to incorporate recommended items into generated responses. Moreover, the current end-to-end neural models trained on small crowd-sourcing datasets (e.g., 10K dialogs in the ReDial dataset) tend to overfit and have poor chit-chat ability. In this work, we propose a novel unified framework that integrates recommendation into the dialog (RecInDial) generation by introducing a vocabulary pointer. To tackle the low-resource issue in CRS, we finetune the large-scale pretrained language models to generate fluent and diverse responses, and introduce a knowledge-aware bias learned from an entity-oriented knowledge graph to enhance the recommendation performance. Furthermore, we propose to evaluate the CRS models in an end-to-end manner, which can reflect the overall performance of the entire system rather than the performance of individual modules, compared to the separate evaluations of the two modules used in previous work. Experiments on the benchmark dataset ReDial show our RecInDial model significantly surpasses the state-of-the-art methods. More extensive analyses show the effectiveness of our model.

## introduction in recent years, there have been fast-growing research interests to address conversational recommender system (crs) @xcite @xcite , due to the booming of intelligent agents in e-commerce platforms. it aims to recommend target items to users through interactive conversations. traditional recommender systems perform personalized recommendations based on user's previous implicit feedback like clicking or purchasing histories, while crs can proactively ask clarification questions and extract user preferences from conversation history to conduct precise recommendations. existing generative methods @xcite @xcite are generally composed of two modules, i.e., a recommender module to predict precise items and a dialogue module to generate free-form natural responses containing the recommended items. such methods usually utilize copy mechanism @xcite or pointer network @xcite to inject the recommended items into the generated replies. however, these strategies cannot always incorporate the recommended items into the generated responses precisely and appropriately. on the other hand, most of the existing crs datasets @xcite @xcite are relatively small (∼10k dialogues) due to the expensive crowd-sourcing labor. the end-toend neural models trained on these datasets from scratch are prone to be overfitting and have undesirable quality on the generated replies in practice. encouraged by the compelling performance of pre-training techniques, we present a pre-trained language models (plms) based framework called recindial to address these challenges. recin-dial integrates the item recommendation into the dialogue generation under the pretrain-finetune schema. specifically, recindial finetunes the powerful plms like dialogpt @xcite together with a relational graph convolutional network (rgcn) to encode the node representation of an item-oriented knowledge graph. the former aims to generate fluent and diverse dialogue ... user: that sounds good. i could go with a classic. have you seen troll 2 (1990)? i'm looking for a horrible movie. cheesy horror human: tuesday 13, you like? redial: black panther (2018) is a good one too. kbrd: or it (2017) kgsf: i would recommend watching it. our: yes i have seen that one. it was good. i also liked the movie it (2017). ... to better illustrate the motivation of our work, furthermore, to better investigate the end-to-end crs system, we argue to evaluate the performance of recommendation by checking whether the final responses contain the target items. existing works separately evaluate the performance of the two modules, i.e., dialogue generation and item recommendation. however, a copy mechanism or pointer network cannot always inject the recommended items into generated replies precisely and appropriately as we mentioned before. the performance of the final recommendations is actually lower than that of the recommender module. for instance, the recall@1 of the recommender module in kgsf @xcite is 3.9% while the actual performance is only 0.9% when evaluating the final integrated responses (see we conduct extensive experiments on the popular benchmark redial @xcite . our recindial model achieves a remarkable improvement on the recommendation over the state-of-theart, and the generated responses are also significantly better on automatic metrics as well as human evaluation. further ablation studies and quantitative and qualitative analyses demonstrate the superior performance of our approach. the contributions of this work can be: existing works in crs can be mainly divided into two categories, namely attribute-based crs and open-ended crs. attribute-based crs. the attribute-based crs can be viewed as a question-driven task-oriented dialogue system @xcite . this kind of system proactively asks clarification questions about the item attributes to infer user preferences, and thus search for the optimal candidates to recommend. there are various ask-ing strategies studied by existing works, such as entropy-ranking based approach @xcite , generalized binary search based approaches @xcite , reinforcement learning based approaches @xcite @xcite , adversarial learning based approach @xcite and graph based approaches @xcite @xcite . another line of research on this direction address the tradeoff issue between exploration (i.e., asking questions) and exploitation (i.e., making recommendations) to achieve both the engaging conversations and successful recommendations, especially for the cold-start users. some of them leverage bandit online recommendation methods to address cold-start scenarios @xcite @xcite , while others focus on the asking strategy with fewer turns @xcite @xcite . open-ended crs. existing works @xcite @xcite @xcite @xcite on this direction explore crs through more free-form conversations, including proactively asking clarification questions, chatting with users, providing the recommendation, etc. multiple datasets have been released to help push forward the research in this area, such as redial @xcite , tg-redial (chinese) @xcite , inspired @xcite and durecdial @xcite . @xcite make the first attempt on this direction and contribute the benchmark dataset re-dial by the paired crowd-workers (i.e., seeker and recommender). follow-up studies @xcite leverage the multiple external knowledge to enhance the performance of open-ended crs. cr-walker @xcite is proposed to perform the tree-structured reasoning on the knowledge graph to introduce relevant items, while mgcg @xcite addresses the transition policy from a non-recommendation dialogue to a recommendation-oriented one. besides, @xcite develop an open-source toolkit crslab to further facilitate the research on this direction. most of these works utilize pointer network @xcite or copy mechanism @xcite to inject the recommended items into generated replies. our work lies in the research of open-ended crs. while … @xmath0 another line of related work lies in the end-toend task-oriented dialogs @xcite @xcite , which also require response generation based on a knowledge base but not for recommendations. ## methodology in this section, we present our proposed recindial model. figure 1 shows the model overview. we first formalize the conversational recommendation task and then detail our plm-based response generation module together with the vocabulary pointer. after that, we introduce how to incorporate the knowledge from an item-oriented knowledge graph with an rgcn into the model. finally, we describe the model training objectives. ## experimental setup datasets. we evaluate our model on the benchmark dataset redial @xcite . due to the collection difficulty of the real world data, most the previous work @xcite @xcite only conducts experiments on this single dataset. the statistics of re-dial dataset is shown in parameter setting. we finetune the small size pre-trained dialogpt model foot_0 , which consists of 12 transformer layers. the dimension of embeddings is 768. it is trained on 147m multi-turn dialogues from reddit discussion threads. for the knowledge graph (kg), both the entity embedding size and the hidden representation size are set to 128, and we set the layer number for r-gcn to 1. for bart baseline, we finetune the base model foot_1 with 6 layers in each of the encoder and decoder, and a hidden size of 1024. for gpt-2 baseline, we finetune the small model foot_2 . for all model's training, we adopt adam optimizer and the learning rate is chosen from {1e-5, 1e-4}. the batch size is chosen from {32, 64}, the gradient accumulation step is set to 8, and the warm-up step is chosen from {500, 800, 1000}. all the hyper-parameters are determined by grid-search. baselines and comparisons. we first introduce two baselines for recommender and dialogue modules, respectively. (1) popularity. it ranks the movie items according to their historical frequency in the training set without a dialogue module. (2) transformer @xcite . it utilizes a transformer-based encoder-decoder to generate responses without recommender module. we then compare the following baseline models in the experiment: (3) redial @xcite . it consists of a dialogue generation module based on hred @xcite , a recommender module based on auto-encoder @xcite , and a sentiment analysis module. (4) kbrd @xcite . it utilizes a knowledge graph from dbpedia to model the relational knowledge of contextual items or entities, and the dialogue generation module is based on the transformer architecture. (5) kgsf @xcite . it incorporates and fuses both word-level and entity-level knowledge graphs to learn better semantic representations for user preferences. (6) gpt-2. we directly finetune gpt-2 and expand its vocabulary to include the item vocabulary. (7) bart. we directly finetune bart and expand its vocabulary to include the same item vocabulary. (8) dialogpt. we directly finetune dialogpt and expand its vocabulary to include same item vocabulary. for our recindial, in addition to the full model ( 9 ) recindial, we also evaluate two variants: (10) recindial w/o vp, where we remove the vocabulary pointer; and (11) recindial w/o kg, where the knowledge graph part is removed. evaluation metrics. as we discussed above, the previous works evaluate the recommender and dialogue modules separately. following the previous setting @xcite , we evaluate the recommender module by recall@k (@xmath1, 10, 50). besides, we also evaluate recall@k in an end-to-end manner, i.e., to check whether the final produced response contains the target item. in such a setting, the recall@k score not only depends on whether the ground truth item appears in the top k recommendation list but also reply on if the recommended item is successfully injected into the generated sentences. therefore, the end-to-end evaluation is fair for all models and applicable for @xmath2, 10, 50. for the dialogue module, automatic metrics include: (1) fluency: perplexity (ppl) measures the confidence of the generated responses. ## experimental results in this section, we first report the comparison results on recommendation and response generation. then we discuss the human evaluation results. after that, we show an example to illustrate how our model works, followed by qualitative analysis. ## conclusion this paper presents a novel unified plm-based framework called recindial for crs, which integrates the item recommendation into the generation process. specifically, we finetune the largescale plms together with a relational graph con-volutional network on an item-oriented knowledge graph. besides, we design a vocabulary pointer mechanism to unify the response generation and item recommendation into the existing plms. extensive experiments on the crs benchmark dataset redial show that recindial significantly outperforms the state-of-the-art methods.