Typical Recommender systems adopt a static view of the recommendation process and treat it as a prediction problem. We argue that it is more appropriate to view the problem of gen erating recommendations as a sequential deci sion problem and, consequently, that Markov de cision processes (MDP) provide a more appro priate model for Recommender systems. MDPs introduce two benefits: they take into account the long-term effects of each recommendation, and they take into account the expected value of each recommendation. To succeed in practice, an MDP-based Recommender system must em ploy a strong initial model; and the bulk of this paper is concerned with the generation of such a model. In particular, we suggest the use of an n-gram predictive model for generating the ini tial MDP. Our n-gram model induces a Markov chain model of user behavior whose predictive accuracy is greater than that of existing predic tive models. We describe our predictive model in detail and evaluate its performance on real data. In addition, we show how the model can be used in an MDP-based Recommender system.
A Markov Decision Process based recommender system was developed in REF to take into the long-term effects of each recommendation.
An MDP-based Recommender System
arXiv: Learning
Mathematics Computer Science