Conversational Recommendation as Retrieval: A Simple, Strong Baseline (2023)

Abstract: Conversational recommendation systems (CRS) aim to recommend suitable items to users through natural language conversation. However, most CRS approaches do not effectively utilize the signal provided by these conversations. They rely heavily on explicit external knowledge e.g., knowledge graphs to augment the models’ understanding of the items and attributes, which is quite hard to scale. To alleviate this, we propose an alternative information retrieval (IR)-styled approach to the CRS item recommendation task, where we represent conversations as queries and items as documents to be retrieved. We expand the document representation used for retrieval with conversations from the training set. With a simple BM25-based retriever, we show that our task formulation compares favorably with much more complex baselines using complex external knowledge on a popular CRS benchmark. We demonstrate further improvements using user-centric modeling and data augmentation to counter the cold start problem for CRSs.

## introduction recommendation systems have become ubiquitous in recent years given the explosion in massive item catalogues across applications. in general, a recommendation system learns user preference from historical user-item interactions, and then recommends items of user's preference. in contrast, crss directly extract user preferences from live dialog history to precisely address the users' needs. an example dialogue from the popular redial benchmark @xcite for crss is shown in generally, a crs integrates two modules: a dialogue module which generates natural language responses to interact with users, and a recommendation module which recommends desirable items to users using the dialog context and external knowledge. we focus on the latter module in this work: we posit that once the correct item to recommend is identified, newer pretrained language models (plms) can easily generate fluent agent responses. it is notable that the conversational context provides sufficient signal to make good recommendations @xcite . e.g., in most approaches to crs rely heavily on external knowledge sources, such as knowledge graphs (kgs) and reviews @xcite . such approaches require specific sub-modules to encode information from these sources like graph neural networks (kipf and welling, 2016), which are hard to scale with catalog additions. existing approaches require either re-training the entire system when the kg structure changes @xcite or adding complex architectures on top to adapt @xcite . newer approaches utilize plms @xcite ), but they often encode item information in model parameters, making it hard to scale to new items without retraining. looking for a fast, more scalable approach, we re-formulate the item recommendation task for crss as an information retrieval (ir) task, with recommendation-seeking conversations as queries and items to recommend as documents. the document content for retrieval is constructed using plain text metadata for the item paired with conversations where the said item is recommended, in order to enhance semantic overlap between the queries which are themselves conversations. we apply a standard non-parametric retrieval baseline -bm25 -to this task and show that the resulting model is fast and extensible without requiring complex external knowledge or architectures, while presenting improvements over more complex item recommendation baselines. our contributions are summarized as follows: ## related work conversational recommendation systems constitute an emerging research area, helped by datasets like redial @xcite , tg-redial @xcite , inspired (hayati et al., 2020), durecdial @xcite , and cpcd @xcite . we next describe the recommender module architectures of crs baselines. redial @xcite uses an autoencoder to generate recommendations. crss commonly use knowledge graphs (kgs) for better understanding of the item catalog: dbpedia @xcite is a popular choice of kg. kbrd @xcite uses item-oriented kgs, while kgsf @xcite further incorporates a word-based kg @xcite . cr-walker @xcite performs tree-structured reasoning on the kg, crfr @xcite does reinforcement learning and multi-hop reasoning on the kg. uni-crs @xcite uses knowledge-added prompt tuning with and kg & a fixed plm. some methods also incorporate user information: cola @xcite uses collaborative filtering to build a user-item graph, and @xcite aims to find lookalike users for user-aware predictions. eschewing kgs, mese @xcite trains an item encoder to convert flat item metadata to embeddings then used by a plm, and tscr @xcite trains a transformer with a cloze task modified for recommendations. most above approaches, however, either rely on complex models with kgs and/or need to be retrained for new items, which is very frequent in present-day item catalogs. ## model we formally define the item recommendation task, followed by our retrieval framework, details of the bm25 retrieval model used, and finally our useraware recommendation method on top of bm25. ## cold start and data augmentation conversational recommenders often suffer from the cold start problem: it is difficult for a new item i.e. not seen during training, to be recommended, since not much is known about it beyond metadata. our model is not immune to this problem. the red lines in figure 1 show r@10 values for the bm25 model for different sets of movies in re-dial based on how many times they are seen in the training set: the model never or rarely recommends movies with 10 or fewer occurrences in training. to counteract this, we perform data augmentation using few-shot prompting @xcite . in particular, we randomly select 6 conversations from redial's training set, use them to prompt a palm 2-l model @xcite , and generate up to 20 dialogues per movie. we do this only for movies seen 10 or fewer times during training, since the model does the worst on these. figure 1 's blue curve shows notably improved r@10 for the movies for which data was augmented, without hurting r@10 for more frequent movies. overall r@10 also improves by ~8% using @xmath0. further combining augmentation with user selection lifts r@1 to 5.9, r@10 to 22.3, and r@50 to 40.7. figure 2 plots recall for bm25 model with the number of artificial dialogues added for lowfrequency movies. based on this plot, we opted to generate at most 20 conversations per movie. ## conclusion we present a retrieval-based formulation of the item recommendation task, used to build crss, by modeling conversations as queries and items as documents. we augment the item representation with conversations recommending that item; the retrieval task then reduces to matching conversations to conversations. using bm25-based retrieval with this task results in a model that is very fast and inexpensive to train (~5 min on cpu) while being flexible to add-ons like user selection. we also show that new items can be easily added without retraining the model, and that simple data augmentation with as few as 20 conversations counters the cold start problem for new items: fewer than most neural network finetuning methods would need.