A Comparison of Algorithms for Collaborative Filtering on RBMs

Almost all web retailers employ some form of recommender system to tailor the products and services o ered to their customers. A common approach to recommendation tasks is collaborative ltering, which uses a database of the preferences of di erent users i.e. collaboration to predict user preferences i.e. ltering [3]. Collaborative ltering embodies a nearest-neighbor philosophy in that predictions are largely based upon the preferences of other 'like-minded' users. For example, if students A and B both positively rated the text 'Principles of Data Mining' and student A also favorably rated the text 'Pattern Recognition and Machine Learning', then it is reasonable to think that student B would also enjoy this text. In principle, collaborative ltering can be used to make recommendations to any user on any item. In practice, though, the database of preferences will be very sparse as only a few (if any) customers may have rated each item. In such situations it is not immediately clear how to identify similar users, much less make predictions that lead to purchases. Early approaches to collaborative ltering relied on simple metrics to nd similar users. GroupLens, for example, was an early Internet news system that used the correlation coe cient between user ratings of articles as a similarity measure [8]. However, memory-based approaches like GroupLens do not scale without further approximation because they require access to the ratings of the entire user community. For this reason, approaches such as singular value decomposition (SVD) that are based on matrix factorization have also been proposed (see e.g. [4, 7]). However, application of matrix factorization methods to sparse ratings matrices can be a non-trivial challenge. As such, Ho man proposed a formal statistical model of user preference using hidden variables over user-item-rating triplets [6]. Unlike traditional clustering models that use latent variables to indicate which cluster (i.e. group of like minded people) a user belongs to, Ho man's probabilistic latent semantic analysis (pLSA) model links users and their ratings to di erent latent causes. More recently, Salakhutdinov et al. proposed an approach to collaborative ltering using Restricted Boltzmann Machines (RBMs)[9]. These models were trained using an e cient learning procedure, known as Contrastive Divergence (CD), that maximizes an approximation to the true likelihood function[5]. Furthermore, the authors report performance on the Net ix data set that is superior to the competitive SVD model posed in [7]. In this report, we investigate the use of an alternative learning procedure known as Persistent CD[10]. In addition, we demonstrate that outperforming SVD models using the RBM models is a non-trivial matter, requiring careful parameter tuning, performance-enhancing tricks and lots of CPU time for training. The remainder of this report is structured as follows. In the next section I present some necessary background on RBMs and also discuss the learning CD and PCD learning procedure. In Section 3, I present Salakhutdinov et al.'s models for collaborative ltering using RBMs. Section 4 contains experimental results and Section 5 contains some concluding remarks.