Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation (2023)

Abstract: Conversational Recommendation System (CRS) is a rapidly growing research area that has gained significant attention alongside advancements in language modelling techniques. However, the current state of conversational recommendation faces numerous challenges due to its relative novelty and limited existing contributions. In this study, we delve into benchmark datasets for developing CRS models and address potential biases arising from the feedback loop inherent in multi-turn interactions, including selection bias and multiple popularity bias variants. Drawing inspiration from the success of generative data via using language models and data augmentation techniques, we present two novel strategies, ‘Once-Aug’ and ‘PopNudge’, to enhance model performance while mitigating biases. Through extensive experiments on ReDial and TG-ReDial benchmark datasets, we show a consistent improvement of CRS techniques with our data augmentation approaches and offer additional insights on addressing multiple newly formulated biases.

## introduction conversational recommendation system (crs) is a growing research topic and application area, along with the recent advance in natural language processing (nlp) and conversational techniques. in contrast to traditional recommendation approaches, which provide item suggestions in a non-interactive manner, conversational recommendation involves multi-turn and mix-initiative interactions between users and the system @xcite . hence, a growing body of research studies @xcite @xcite has introduced diverse conversational recommendation models, which address the natural language understanding from user utterances, user intent estimation, user preference estimation and generation of appropriate responses that encapsulate the recommended items. ## related work this section offers an overview of the biases from data collection and the development of recommendation systems to the conversational recommendation system. in addition, we further discuss the existing bias mitigation strategies that leverage generative data or so-called data imputation. ## bias analysis, formulation and mitigation for crs here we formally define the conversational recommendation task and its associated components, susceptible to various biases originating in the feedback loop. we then propose to consider the impact of multiple biases, and account for the unique characteristics of crss. finally, we introduce two novel data augmentation strategies, once-aug and popnudge, to mitigate the impact of biases. ## experimental setup our objective is to improve the evaluation of crss techniques based on various bias effects while validating the performance of our proposed data augmentation approaches. to achieve this, we conduct a series of experiments using recent crs models on two benchmark datasets: redial @xcite and tgredial @xcite . these experiments allow us to access the recommendation performance and bias effect on existing approaches and determine the effectiveness of our proposed approach. in our analysis, we consider several commonly used crs models that serve as baselines in the literature. we summarise the recommendation component of these models: 1) redial @xcite , an autoencoder conversational recommender and uses sentiment-analysed user opinions as input. 2) kbrd @xcite , it links entities in dialogue content to an external knowledge graph (dbpedia) and employs a self-attention mechanism to learn entity representations for recommendations. 3) tg-redial @xcite uses the pretrained bert (kenton and toutanova, 2019) and the sequential recommender sasrec @xcite ) to model the historical utterances and user interactions, respectively. 4) kgsf @xcite leverages both entity-oriented and word-oriented to enrich data representations and enhance item recommendations. the implementation and evaluation of these models are supported by the open-sourced crslab @xcite ) toolkit foot_1 . for recommendation accuracy evaluation, we follow @xcite and apply a set of commonly used evaluation metrics, including hit ratio (hit), ndcg and mrr with cutoffs at 10 and 50. to ensure fair performance comparisons, our data augmentation strategies solely enrich the training data and do not modify the validation and test data. specifically, for the popnudge strategy, we explore augmentation effects with varying numbers of sampled dialogues (k) ranging from {1,5,10,50}. ## result analysis in this section, we present and discuss the experimental results to illustrate the effectiveness of applying once-aug and popnudge to baseline models while examining the recommendation accuracy, the effect on selection and popularity biases. ## conclusions our study investigates several biases within the feedback loop of developing a conversational recommendation system, including selection bias, popularity bias, cross-episode popularity (cep) effect and user intent-oriented popularity (uiop) effect. these proposed biases and effects shed light on crs model performance and provide valuable insights for effective model development. additionally, we leverage the recent advance of large language models to generate effective synthetic data and propose novel and effective data augmentation strategies, once-aug and popnudge, to enhance model performance. our experimental results demonstrate the effectiveness of our data augmentation strategies in improving crs model performance while considering the recommendation accuracy as well as the mitigation across selection bias and popularity biases.