Leveraging Unpaired Feedback for Long-Term LLM -based Recommendation Tuning (2025)

Abstract: Most recommender systems focus on short-term objectives such as click-through rate, often at the expense of long-term user satisfaction. This can lead to echo chambers, where users are repeatedly exposed to redundant content. While recent efforts integrate Large Language Models (LLMs) into recommendation, they typically inherit this short-sighted focus. In this work, we highlight unpaired feedback—implicit signals such as continued engagement (positive) or silent disengagement (negative) that lack explicit contrastive labels—as a key challenge for long-term recommendation. Effectively learning from such feedback is crucial for improving LLM-based recommenders in dynamic user environments. To this end, we propose ULRec (Unpaired Feedback for Long-Term LLM-based Recommendation Tuning), a simple framework that fine-tunes LLMs using both positive and negative unpaired feedback. ULRec leverages the KTO algorithm to incorporate these signals without requiring paired supervision. Despite its simplicity, ULRec consistently improves long-term recommendation performance, demonstrating the value of modeling unpaired user feedback.

## introduction recommendation systems play a central role in filtering information and shaping user experiences @xcite @xcite . while most existing work focuses on short-term objectives such as click-through rate (ctr) @xcite @xcite , this often leads to long-term issues like echo chambers, where users are repeatedly exposed to similar content @xcite @xcite . such short-sighted optimization can degrade user satisfaction over time * work is done during internship at meituan. ## preliminary in this section, we first present the problem formulation and briefly describe the interactive environment used for simulation. ## experiment we conducted detailed experiments to address the following three research questions (rqs): ## conclusion in this work, we revisit the long-standing echo chamber problem in recommendation systems from the perspective of llms, emphasizing the importance of fine-tuning by effectively modeling users' time-varying feedback to enhance long-term user satisfaction. we identify an under-explored yet critical challenge: how to learn from unpaired feedback, especially negative signals from users' nonengagement behaviors. to address this, we propose ulrec, a lightweight framework that integrates unpaired feedback into the llm fine-tuning process, which utilizes the kto algorithm to handle implicit negative signals without relying on paired preference data. our extensive experiments demonstrate that ulrec significantly improves the longterm recommendation quality of recommendation systems, proving the value of explicitly modeling unpaired feedback. this work not only deepens our understanding of long-term feedback mechanisms in llm-based recommender systems but also suggests new directions for future research on long-term recommendation strategies. moreover, although we have optimized the simulation environment based on previous work @xcite to better adapt to real-world scenarios, there still inevitably exists a gap between the simulation and the real environment. how to further reduce the discrepancy between the real world and the simulated environment remains a question worthy of exploration.