Interactive Content Diversity and User Exploration in Online Movie Recommenders: A Field Experiment

AbstractRecommender systems often struggle to strike a balance between matching users’ tastes and providing unexpected recommendations. When recommendations are too narrow and fail to cover the full range of users’ preferences, the system is perceived as useless. Conversely, when the system suggests too many items that users don’t like, it is considered impersonal or ineffective. To better understand user sentiment about the breadth of recommendations given by a movie recommender, we conducted interviews and surveys and found out that many users considered narrow recommendations to be useful, while a smaller number explicitly wanted greater breadth. Additionally, we designed and ran an online field experiment with a larger user group, evaluating two new interfaces designed to provide users with greater access to broader recommendations. We looked at user preferences and behavior for two groups of users: those with higher initial movie diversity and those with lower diversity. Among our findings, we discovered that different levels of exploration control and users’ subjective preferences on interfaces are more predictive of their satisfaction with the recommender.Keywords: Human-recommender interactionpigeonholinginteractive recommendationuser exploration Disclosure statementNo potential conflict of interest was reported by the author(s).Notes1 Active users were considered as those who have logged in to the MovieLens website over 12 times and rated over 20 movies in the year of 2021.2 MovieLens is a non-commercial, personalized recommender that gathers users’ ratings on movies they have watched and provides predictions of new movies users might like to watch with different recommendation algorithms. The authors would like to thank the MovieLens team for their support and resources.3 MovieLens allows users who rated over 15 movies to unlock the feature of selecting one of four different algorithms for movie recommendation. More details can be found in section 4.2.4 More details of the existing algorithms on MovieLens can be found in section 4.2.5 More details of tag genomes can be found in section 4.2.1.6 We selected 72 to ensure real-time calculation feasibility and avoid affecting user experience with potential delay in recommendation serving time.Additional informationNotes on contributorsRuixuan SunRuixuan Sun is a third-year Computer Science PhD student at the GroupLens Lab of University of Minnesota, Twin Cities. Her research interests lie in the intersection of Human–Computer Interaction and Recommender Systems. She is passionate about explainable AI, social computing, and human-centric recommender system design and evaluations.Avinash AkellaAvinash Akella graduated from the University of Minnesota with a Master’s degree in Data Science in 2023. His research interests lie at the intersection of Human–Computer Interaction, Data Science, Computer Science, and Psychology. He is now a Data Scientist at IBM.Ruoyan KongRuoyan Kong graduated from the University of Minnesota, Twin Cities with a PhD in Computer Science in 2023. Her research interests include recommender systems, email personalization, and social computing. She is now a machine learning engineer at TikTok.Moyan ZhouMoyan Zhou is a third-year PhD student in Computer Science at the University of Minnesota, Twin Cities in GroupLens Lab. Her research is focused on Human–Computer Interaction and Peer Production and Online Collaboration Communities.Joseph A. KonstanJoseph A. Konstan is Distinguished McKnight Professor of Computer Science and Engineering at the University of Minnesota. He has been working in the field of Recommender Systems since 1995, mostly with focus on human-centered design and evaluation of recommender algorithms and interfaces.