Paper:
A Latent Source Model for Online Collaborative Filtering
Despite the prevalence of collaborative filtering in recommendation systems, there has been little theoretical development on why and how well it works, especially in the "online" setting, where items are recommended to users over time. We address this theoretical gap by introducing a model for online recommendation systems, cast item recommendation under the model as a learning problem, and analyze the performance of a cosine-similarity collaborative filtering method. In our model, each of n users either likes or dislikes each of m items. We assume there to be k types of users, and all the users of a given type share a common string of probabilities determining the chance of liking each item. At each time step, we recommend an item to each user, where a key distinction from related bandit literature is that once a user consumes an item (e.g., watches a movie), then that item cannot be recommended to the same user again. The goal is to maximize the number of likable items recommended to users over time. Our main result establishes that after nearly log(km) initial learning time steps, a simple collaborative filtering algorithm achieves essentially optimal performance without knowing k. The algorithm has an exploitation step that uses cosine similarity and two types of exploration steps, one to explore the space of items (standard in the literature) and the other to explore similarity between users (novel to this work).

Review:
** COMMENTS AFTER AUTHOR FEEDBACK **This is a high quality paper.Minor comment. Not addressed in the rebuttal is"1. The definition of neighbourhood is very unclear, because of the use of “jointly”. Line 178 ... contradicts Lemma 8’s proof (Line 576)."which might be good to clarify in the paper.--------------------------------------------------Disclaimer: the reviewer is not an expert in Learning Theory, but in Collaborative Filtering. In assessing the paper’s theoretical significance, he/she will submit to the viewpoint of expert Learning Theorists. MAIN IDEAS OF THE PAPER:We know that collaborative filtering works in practice: users who buy A also tend to buy B. The paper takes a bold stab at providing theoretical guarantees on how well it works, albeit in a restricted and slightly hypothetical setting. The hypothetical setting is a simplified sketch of reality, necessary to obtain a theoretically analysable handle on the problem.The paper analyses a very simple collaborative filtering algorithm, where collaborative filtering relies on looking for a set of neighbouring users with the most similar like/dislike patters (within some cosine distance) and using their extra like/dislike votes to score items to provide next item to a user. The goal is to maximize the expected number of liked items presented to each user within a fixed time window of 1 to T recommendations.The paper shows that the users are “learnable” after around log(km) = log(number of user types) + log(number of items) rounds of recommendations (up to log^{7/3}(km), depending on a parameter choice). Essentially in time logarithmic in the number of items, roughly optimal performance can be expected. The whole set-up relies on some conditions, like having enough users.The result relies on two additional steps in the collaborative filtering algorithm: (1) random exploration, and (2) structured exploration. Both are necessary, (1) to interrogate potential future items for recommendation, and (2) to have enough of an overlap in like/dislike items to learn about the similarity between users.The experimental results try to simulate a real scenario by streaming a curated subset of the Netflix prize data to the data set’s users. In the simulation, the average cumulative reward is higher than those of two existing methods (PAF and DM, incidentally neither of which has ever been used in practice, to my knowledge and those of peers).RELATIONSHIP TO PREVIOUS WORK:The paper is set apart by using two exploration steps, one of which learns the similarity between users (setting it apart from standard multi-arm bandits). Furthermore, no arm can be pulled more than once.QUALITY and CLARITY:The paper is very well explained, and beautifully written.I worked through most of the proofs in the supplementary material, and they seem technically correct.I would like to comment on some textual issues for the benefit of the authors:1.  The definition of neighbourhood is very unclear, because of the use of “jointly”. Line 178 suggests that *only* user-item indexes that were recommended in the joint exploration step in the algorithm are eligible for defining neighbours (and hence cosine similarity). Why are other overlapping items not considered in defining the neighbourhood? The confusion arises as Line 178 is intentional, but if we read Lemma 8’s proof (Line 576), it contradicts Line 178, as now jointly means the items that two users rated in common, and *not* the subset of that set, which was tagged with “jointly” in your Algorithm (see Line 157).2. Sec 2, Lines 103 and others. The paper exclaims the point that one should not recommend a consumed item to a user again (especially a disliked one). This is rather obvious in the recommendations community, and I assume you drive home the point to distinguish your work from a standard multi-arm bandit setting?3. Typos.Line 165 should refer to \tilde{p}_{ui} and not index j?Line 524 (Lemma 5’s proof). \bar{Z}_s = 1 / s^\alpha – Z_s would give a zero mean.Line 610 [unnecessary inequality in proof]Line 634 user u has is...ORIGINALITY and SIGNIFICANCE:This paper is the first attempt (to my knowledge) to characterize learning rates in a (pseudo-) real recommendations setting.The (new) practical take-home message is that joint exploration is required for the algorithm (and maybe more generally?) to achieve optimality in time logarithmic in the number of items.PRACTICAL WEAKNESSES TO BE AWARE OF:To achieve the main results (that is, how long and how much should one explore and exploit before attaining optimal performance) the proofs and algorithm rely on a number of assumptions. These assumptions already give useful insights (like how many users one needs relative to an item catalogue size and tolerance), but from a practitioner’s viewpoint, they break down on many levels.1. The assumption that users belong to each type with odds 1/k is invalid. In reality, these are typically according some power law.2. Random (and joint) exploration, whilst theoretically appealing (and necessary) is a dangerous game, as it can hurt user retention in systems like those described in the Introduction. The algorithm doesn’t make provision for a user never returning because of it.3. From calibration plots in real systems, we know that condition A1 does not hold in practice. There are pairs for which the odds are really, well, coinflip!These are not criticisms, but differences to be aware of.