Original: Under review as a conference paper at ICLR

Original: O NLINE L OW R ANK M ATRIX C OMPLETION
Revised: O NLINE L OW R ANK M ATRIX C OMPLETION

Original: Anonymous authors Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We study the problem of online low-rank matrix completion with M users, N items and T rounds.
Revised: We study the problem of online low-rank matrix completion with M users, N items and T rounds.

Original: In each round, the algorithm recommends one item per user, for which it gets a (noisy) reward sampled from a low-rank user-item preference matrix.
Revised: In each round, the algorithm recommends one item per user, for which it gets a (noisy) reward sampled from a low-rank user-item preference matrix.

Original: The goal is to design a method with sub-linear regret (in T ) and nearly optimal dependence on M and N .
Revised: The goal is to design a method with sub-linear regret (in T ) and nearly optimal dependence on M and N .

Original: The problem can be easily mapped to the standard multi-armed bandit problem where each item is an independent arm, but that leads to poor regret as the correlation between arms and users is not exploited.
Revised: The problem can be easily mapped to the standard multi-armed bandit problem where each item is an independent arm, but that leads to poor regret as the correlation between arms and users is not exploited.

Original: On the other hand, exploiting the low-rank structure of reward matrix is challenging due to non-convexity of the low-rank manifold.
Revised: On the other hand, exploiting the low-rank structure of reward matrix is challenging due to non-convexity of the low-rank manifold.

Original: We ﬁrst demonstrate that the low-rank structure can be exploited using a simple explore-then-commit (ETC) approach that ensures a regret of O ( polylog ( M + N ) T 2 / 3 ) .
Revised: We ﬁrst demonstrate that the low-rank structure can be exploited using a simple explore-then-commit (ETC) approach that ensures a regret of O ( polylog ( M + N ) T 2 / 3 ) .

Original: That is, roughly only polylog ( M + N ) item recommendations are required per user to get a non-trivial solution.
Revised: That is, roughly only polylog ( M + N ) item recommendations are required per user to get a non-trivial solution.

Original: We then improve our result for the rank- 1 setting which in itself is quite challenging and encapsulates some of the key issues.
Revised: We then improve our result for the rank- 1 setting which in itself is quite challenging and encapsulates some of the key issues.

Original: Here, we propose OCTAL (Online Collaborative ﬁlTering using iterAtive user cLustering) that guarantees nearly optimal regret of O ( polylog ( M + N ) T 1 / 2 ) .
Revised: Here, we propose OCTAL (Online Collaborative ﬁlTering using iterAtive user cLustering) that guarantees nearly optimal regret of O ( polylog ( M + N ) T 1 / 2 ) .

Original: OCTAL is based on a novel technique of clustering users that allows iterative elimination of items and leads to a nearly optimal minimax rate.
Revised: OCTAL is based on a novel technique of clustering users that allows iterative elimination of items and leads to a nearly optimal minimax rate.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Collaborative ﬁltering based on low-rank matrix completion/factorization techniques are the cornerstone of most modern recommendation systems (Koren, 2008).
Revised: Collaborative ﬁltering based on low-rank matrix completion/factorization techniques are the cornerstone of most modern recommendation systems (Koren, 2008).

Original: Such systems model the underlying user-item afﬁnity matrix as a low-rank matrix, use the acquired user-item recommendation data to estimate the low-rank matrix and subsequently, use the matrix estimate to recommend items for each user.
Revised: Such systems model the underlying user-item afﬁnity matrix as a low-rank matrix, use the acquired user-item recommendation data to estimate the low-rank matrix and subsequently, use the matrix estimate to recommend items for each user.

Original: Several existing works study this ofﬂine setting (Candès & Recht, 2009; Deshpande &
Revised: Several existing works study this ofﬂine setting (Candès & Recht, 2009; Deshpande &

Original: Montanari, 2012; Jain et al., 2013; Chen et al., 2019; Abbe et al., 2020).
Revised: Montanari, 2012; Jain et al., 2013; Chen et al., 2019; Abbe et al., 2020).

Original: However, typical recommendation systems are naturally online and interactive – they recommend items to users and need to adapt quickly based on users’ feedback.
Revised: However, typical recommendation systems are naturally online and interactive – they recommend items to users and need to adapt quickly based on users’ feedback.

Original: The goal of such systems is to quickly identify each user’s preferred set of items, so it is necessary to identify the best items for each user instead of estimating the entire afﬁnity matrix.
Revised: The goal of such systems is to quickly identify each user’s preferred set of items, so it is necessary to identify the best items for each user instead of estimating the entire afﬁnity matrix.

Original: Moreover, items/users are routinely added to the system, so it should be able to quickly adapt to new items/users by using only a small amount of recommendation feedback.
Revised: Moreover, items/users are routinely added to the system, so it should be able to quickly adapt to new items/users by using only a small amount of recommendation feedback.

Original: In this work, we study this problem of the online recommendation system.
Revised: In this work, we study this problem of the online recommendation system.

Original: In particular, we study the online version of low-rank matrix completion with the goal of identifying top few items for each user using say only logarithmic many exploratory recommendation rounds for each user.
Revised: In particular, we study the online version of low-rank matrix completion with the goal of identifying top few items for each user using say only logarithmic many exploratory recommendation rounds for each user.

Original: In each round (out of T rounds) we predict one item (out of N items) for each user (out of M users) and obtain feedback/reward for each of the predictions – e.g. did the users view the recommended movie.
Revised: In each round (out of T rounds) we predict one item (out of N items) for each user (out of M users) and obtain feedback/reward for each of the predictions – e.g. did the users view the recommended movie.

Original: The goal is to design a method that has asymptotically similar reward to a method that can pick best items for each user.
Revised: The goal is to design a method that has asymptotically similar reward to a method that can pick best items for each user.

Original: As mentioned earlier, we are speciﬁcally interested in the setting where T ⌧ N i.e. the number of recommendation feedback rounds is much smaller than the total number of items.
Revised: As mentioned earlier, we are speciﬁcally interested in the setting where T ⌧ N i.e. the number of recommendation feedback rounds is much smaller than the total number of items.

Original: Moreover, we assume that the expected reward matrix is low-rank.
Revised: Moreover, we assume that the expected reward matrix is low-rank.

Original: That is, if R ( t ) ij is the reward obtained in the t th round for predicting item- j for user-
Revised: That is, if R ( t ) ij is the reward obtained in the t th round for predicting item- j for user-

Original: i , then E R ( t ) ij = P ij , where P 2 R M ⇥ N is a low-rank matrix.
Revised: i , then E R ( t ) ij = P ij , where P 2 R M ⇥ N is a low-rank matrix.

Original: A similar low rank reward matrix setting has been studied in online multidimensional learning problems (Katariya et al., 2017b; Kveton et al., 2017; Trinh et al., 2020).
Revised: A similar low rank reward matrix setting has been studied in online multidimensional learning problems (Katariya et al., 2017b; Kveton et al., 2017; Trinh et al., 2020).

Original: But in these problems, the goal is to ﬁnd the matrix entry with the highest reward.
Revised: But in these problems, the goal is to ﬁnd the matrix entry with the highest reward.

Original: Instead, our goal is to recommend good items to all users which is a signiﬁcantly harder challenge.
Revised: Instead, our goal is to recommend good items to all users which is a signiﬁcantly harder challenge.

Original: A trivial approach is Under review as a conference paper at ICLRto ignore the underlying low-rank structure and solve the problem using standard multi-arm bandit methods.
Revised: A trivial approach is to ignore the underlying low-rank structure and solve the problem using standard multi-arm bandit methods.

Original: That is, model each (user, item) pair as an arm .
Revised: That is, model each (user, item) pair as an arm .

Original: Naturally, that would imply exploration of almost all the items for each user, which is also reﬂected in the regret bound (averaged over users) of O ( p NT )
Revised: Naturally, that would imply exploration of almost all the items for each user, which is also reﬂected in the regret bound (averaged over users) of O ( p NT )

Original: (Remark 1).
Revised: (Remark 1).

Original: That is, as expected, the regret bound is vacuous when the number of recommendation rounds T ⌧ N .
Revised: That is, as expected, the regret bound is vacuous when the number of recommendation rounds T ⌧ N .

Original: In contrast, most of the existing online learning techniques that leverage structure amongst arms assume a parametric form for the reward function and require the reward function to be convex in the parameters (Shalev-Shwartz et al., 2011; Bubeck, 2011).
Revised: In contrast, most of the existing online learning techniques that leverage structure amongst arms assume a parametric form for the reward function and require the reward function to be convex in the parameters (Shalev-Shwartz et al., 2011; Bubeck, 2011).

Original: Thus, due to the non-convexity of the manifold of low-rank matrices, such techniques do not apply in our case.
Revised: Thus, due to the non-convexity of the manifold of low-rank matrices, such techniques do not apply in our case.

Original: While there are some exciting recent approaches for non-convex online learning (Agarwal et al., 2019; Suggala &
Revised: While there are some exciting recent approaches for non-convex online learning (Agarwal et al., 2019; Suggala &

Original: Netrapalli, 2020), they do not apply to the above mentioned problem.
Revised: Netrapalli, 2020), they do not apply to the above mentioned problem.

Original: Our Techniques and Contributions: We ﬁrst present a method based on the explore-then-commit (ETC) approach (Algorithm 2).
Revised: Our Techniques and Contributions: We ﬁrst present a method based on the explore-then-commit (ETC) approach (Algorithm 2).

Original: For the ﬁrst few rounds, the algorithm runs a pure exploration strategy by sampling random items for each user.
Revised: For the ﬁrst few rounds, the algorithm runs a pure exploration strategy by sampling random items for each user.

Original: We use the data obtained by pure exploration to learn a good estimate of the underlying reward matrix P ; this result requires a slight modiﬁcation of the standard matrix completion result in Chen et al.
Revised: We use the data obtained by pure exploration to learn a good estimate of the underlying reward matrix P ; this result requires a slight modiﬁcation of the standard matrix completion result in Chen et al.

Original: We then run the exploitation rounds based on the current estimate.
Revised: We then run the exploitation rounds based on the current estimate.

Original: In particular, for the remaining rounds, the algorithm commits to the arm with the highest estimated reward for each user.
Revised: In particular, for the remaining rounds, the algorithm commits to the arm with the highest estimated reward for each user.

Original: With the ETC algorithm, we achieve a regret bound of
Revised: With the ETC algorithm, we achieve a regret bound of

Original: O ( polylog ( M + N ) T 2 / 3 ) (Thm. 1).
Revised: O ( polylog ( M + N ) T 2 / 3 ) (Thm. 1).

Original: This bound is able to get rid of the dependence on N , implying non-trivial guarantees even when T ⌧ N .
Revised: This bound is able to get rid of the dependence on N , implying non-trivial guarantees even when T ⌧ N .

Original: That is, we require only polylog ( M + N ) exploratory recommendations per user.
Revised: That is, we require only polylog ( M + N ) exploratory recommendations per user.

Original: However, the dependence of the algorithm on T is sub-optimal.
Revised: However, the dependence of the algorithm on T is sub-optimal.

Original: To address this, we study the special but critical case of rank-one reward matrix.
Revised: To address this, we study the special but critical case of rank-one reward matrix.

Original: The rank- 1 setting is itself technically challenging (Katariya et al., 2017b) and encapsulates many key issues.
Revised: The rank- 1 setting is itself technically challenging (Katariya et al., 2017b) and encapsulates many key issues.

Original: We provide a novel algorithm OCTAL in Algorithm 3 and a modiﬁed version in Algorithm 8 that achieves nearly optimal regret bound of O ( polylog ( M + N ) T 1 / 2 ) (see Theorems 2 and E).
Revised: We provide a novel algorithm OCTAL in Algorithm 3 and a modiﬁed version in Algorithm 8 that achieves nearly optimal regret bound of O ( polylog ( M + N ) T 1 / 2 ) (see Theorems 2 and E).

Original: The key insight is that in rank-one case, we need to cluster users based on their true latent representation to ensure low regret.
Revised: The key insight is that in rank-one case, we need to cluster users based on their true latent representation to ensure low regret.

Original: Our method OCTAL consists of multiple phases of exponentially increasing number of rounds.
Revised: Our method OCTAL consists of multiple phases of exponentially increasing number of rounds.

Original: Each phase reﬁnes the current estimate of relevant sub-matrices of the reward matrix using standard matrix completion techniques.
Revised: Each phase reﬁnes the current estimate of relevant sub-matrices of the reward matrix using standard matrix completion techniques.

Original: Using the latest estimate, we jointly reﬁne the cluster of users and the estimate of the best items for users in each cluster.
Revised: Using the latest estimate, we jointly reﬁne the cluster of users and the estimate of the best items for users in each cluster.

Original: We can show that the regret in each phase decreases very quickly and since the count of phases required to get the correct clustering is small, we obtain our desired regret bounds.
Revised: We can show that the regret in each phase decreases very quickly and since the count of phases required to get the correct clustering is small, we obtain our desired regret bounds.

Original: Finally, we show that our method achieves a regret guarantee that scales as e
Revised: Finally, we show that our method achieves a regret guarantee that scales as e

Original: O ( T 1 / 2 ) with the number of rounds T .
Revised: O ( T 1 / 2 ) with the number of rounds T .

Original: We also show that the dependence on T is optimal (see
Revised: We also show that the dependence on T is optimal (see

Original: Theorem 3).
Revised: Theorem 3).

Original: Below we summarize our main contributions ( e
Revised: Below we summarize our main contributions ( e

Original: [Equation] hides logarithmic factors):
Revised: O ( · ) hides logarithmic factors):

Original: • We formulate the online low rank matrix completion problem and deﬁne the appropriate notion of regret to study.
Revised: • We formulate the online low rank matrix completion problem and deﬁne the appropriate notion of regret to study.

Original: We propose Algorithm 2 based on the explore-then-commit approach that suffers a regret of e O ( T 2 / 3 ) .
Revised: We propose Algorithm 2 based on the explore-then-commit approach that suffers a regret of e O ( T 2 / 3 ) .

Original: (Thm. 1) •
Revised: (Thm. 1) •

Original: We propose a novel algorithm
Revised: We propose a novel algorithm

Original: OCTAL (Online Collaborative ﬁlTering using iterAtive user cLustering) and a modiﬁed version (Alg. 3 and Alg. 8 respectively) for the special case of rank- 1 reward matrices and guarantee a regret of e O ( T 1 / 2 ) .
Revised: OCTAL (Online Collaborative ﬁlTering using iterAtive user cLustering) and a modiﬁed version (Alg. 3 and Alg. 8 respectively) for the special case of rank- 1 reward matrices and guarantee a regret of e O ( T 1 / 2 ) .

Original: Importantly, our algorithms provide non-trivial regret guarantees in the practical regime of M  T and N  T i.e when the number of users and items is much larger than the number of recommendation rounds.
Revised: Importantly, our algorithms provide non-trivial regret guarantees in the practical regime of M  T and N  T i.e when the number of users and items is much larger than the number of recommendation rounds.

Original: Moreover, OCTAL does not suffer an issue of large cold-start time (possibly large exploration period) as in the ETC algorithm.
Revised: Moreover, OCTAL does not suffer an issue of large cold-start time (possibly large exploration period) as in the ETC algorithm.

Original: • We conducted detailed empirical study of our proposed algorithms (see Appendix A) on synthetic and multiple real datasets, and demonstrate that our algorithms can achieve signiﬁcantly lower regret than methods that do not use collaboration between users.
Revised: • We conducted detailed empirical study of our proposed algorithms (see Appendix A) on synthetic and multiple real datasets, and demonstrate that our algorithms can achieve signiﬁcantly lower regret than methods that do not use collaboration between users.

Original: Furthermore, we show that for rank- 1 case, while it is critical to tune the exploration period in ETC (as a function of rounds and sub-optimality gaps) and is difﬁcult in practice (Lattimore & Szepesvári, 2020)[Ch. 6], OCTAL still suffers lower regret without such side-information (see Figures 1a and 2b).
Revised: Furthermore, we show that for rank- 1 case, while it is critical to tune the exploration period in ETC (as a function of rounds and sub-optimality gaps) and is difﬁcult in practice (Lattimore & Szepesvári, 2020)[Ch. 6], OCTAL still suffers lower regret without such side-information (see Figures 1a and 2b).

Original: Technical Challenges: For rank- 1 case, i.e., P = uv T , one can cluster users in two bins: ones with u i  0 and ones with u i < 0 .
Revised: Technical Challenges: For rank- 1 case, i.e., P = uv T , one can cluster users in two bins: ones with u i  0 and ones with u i < 0 .

Original: Cluster- 2 users dislike the best items for Cluster- 1 users and vice-versa.
Revised: Cluster- 2 users dislike the best items for Cluster- 1 users and vice-versa.

Original: Thus, we require algorithms that can learn this cluster structure and exploit the fact that within the same cluster, relative ranking of all items remain the same.
Revised: Thus, we require algorithms that can learn this cluster structure and exploit the fact that within the same cluster, relative ranking of all items remain the same.

Original: This is a challenging information to exploit, and requires learning the latent structure.
Revised: This is a challenging information to exploit, and requires learning the latent structure.

Original: Note that an algorithm that ﬁrst attempts to cluster all users and then optimize the regret will suffer the same T 2 / 3 dependence in the worst case as in Under review as a conference paper at ICLRthe ETC algorithm; this is because the difﬁculty of clustering the users are widely different.
Revised: Note that an algorithm that ﬁrst attempts to cluster all users and then optimize the regret will suffer the same T 2 / 3 dependence in the worst case as in the ETC algorithm; this is because the difﬁculty of clustering the users are widely different.

Original: Instead, our proposed OCTAL algorithm (Algorithms 3 and 8) in each iteration/phase performs two tasks: i) it tries to eliminate some of the items similar to standard phase elimination method (Lattimore &
Revised: Instead, our proposed OCTAL algorithm (Algorithms 3 and 8) in each iteration/phase performs two tasks: i) it tries to eliminate some of the items similar to standard phase elimination method (Lattimore &

Original: Szepesvári, 2020)[Ch 6, Ex 6.8] for users that are already clustered, ii) it simultaneously tries to grow the set of clustered users.
Revised: Szepesvári, 2020)[Ch 6, Ex 6.8] for users that are already clustered, ii) it simultaneously tries to grow the set of clustered users.

Original: For partial clustering, we ﬁrst apply low rank matrix completion guarantees over carefully constructed reward sub-matrices each of which correspond to a cluster of users and a set of active items that have high rewards for all users in the same cluster, and then use the partial reward matrix for eliminating some of the items in each cluster.
Revised: For partial clustering, we ﬁrst apply low rank matrix completion guarantees over carefully constructed reward sub-matrices each of which correspond to a cluster of users and a set of active items that have high rewards for all users in the same cluster, and then use the partial reward matrix for eliminating some of the items in each cluster.

Original: 1.1 R ELATED W ORKS
Revised: 1.1 R ELATED W ORKS

Original: To the best of our knowledge, we provide ﬁrst rigorous online matrix completion algorithms.
Revised: To the best of our knowledge, we provide ﬁrst rigorous online matrix completion algorithms.

Original: But, there are several closely related results/techniques in the literature which we brieﬂy survey below.
Revised: But, there are several closely related results/techniques in the literature which we brieﬂy survey below.

Original: A very similar setting was considered in Sen et al.
Revised: A very similar setting was considered in Sen et al.

Original: (2017) where the authors considered a multi-armed bandit problem with L contexts and K arms with context dependent reward distributions.
Revised: (2017) where the authors considered a multi-armed bandit problem with L contexts and K arms with context dependent reward distributions.

Original: The authors assumed that the L ⇥ K reward matrix is low rank and can be factorized into non-negative components which allowed them to use recovery guarantees from non-negative matrix factorization.
Revised: The authors assumed that the L ⇥ K reward matrix is low rank and can be factorized into non-negative components which allowed them to use recovery guarantees from non-negative matrix factorization.

Original: Moreover, the authors only showed ETC algorithms that resulted in T 2 / 3 regret guarantees.
Revised: Moreover, the authors only showed ETC algorithms that resulted in T 2 / 3 regret guarantees.

Original: Our techniques can be used to improve upon the existing guarantees in Sen et al.
Revised: Our techniques can be used to improve upon the existing guarantees in Sen et al.

Original: (2017) in two ways 1)
Revised: (2017) in two ways 1)

Original: Removing the assumption of the low rank components being non-negative as we use matrix completion with entry-wise error guarantees.
Revised: Removing the assumption of the low rank components being non-negative as we use matrix completion with entry-wise error guarantees.

Original: The dependence on T can be improved from T 2 / 3 to T 1 / 2 when the reward matrix P is rank- 1 .
Revised: The dependence on T can be improved from T 2 / 3 to T 1 / 2 when the reward matrix P is rank- 1 .

Original: Multi-dimensional online decision making problems namely stochastic rank-1 matrix bandits was introduced in Katariya et al.
Revised: Multi-dimensional online decision making problems namely stochastic rank-1 matrix bandits was introduced in Katariya et al.

Original: (2017b;a); Trinh et al.
Revised: (2017b;a); Trinh et al.

Original: In their settings, at each round t 2 [ T ] , the learning agent can choose one row and one column and observes a reward corresponding to an entry of a rank- 1 matrix.
Revised: In their settings, at each round t 2 [ T ] , the learning agent can choose one row and one column and observes a reward corresponding to an entry of a rank- 1 matrix.

Original: Here, the regret is deﬁned in terms of the best (row ,column) pair which corresponds to the best arm.
Revised: Here, the regret is deﬁned in terms of the best (row ,column) pair which corresponds to the best arm.

Original: This setting was extended to the rank r setting (Kveton et al., 2017), rank 1 multi-dimensional tensors (Hao et al., 2020), bilinear bandits (Jun et al., 2019; Huang et al., 2021) and generalized linear bandits (Lu et al., 2021).
Revised: This setting was extended to the rank r setting (Kveton et al., 2017), rank 1 multi-dimensional tensors (Hao et al., 2020), bilinear bandits (Jun et al., 2019; Huang et al., 2021) and generalized linear bandits (Lu et al., 2021).

Original: Although these papers provide tight regret guarantees, they cannot be translated to our problem.
Revised: Although these papers provide tight regret guarantees, they cannot be translated to our problem.

Original: This is because, we solve a signiﬁcantly different problem with an underlying rank- 1 reward matrix P where we need to minimize the regret for all users (rows of P ) jointly.
Revised: This is because, we solve a signiﬁcantly different problem with an underlying rank- 1 reward matrix P where we need to minimize the regret for all users (rows of P ) jointly.

Original: Hence, it is essential to ﬁnd the entries (columns) of P with large rewards for each user(row) of P ; contrast this with the multi-dimensional online learning problem where it is sufﬁcient to infer only the entry ((row,column) pair) in the matrix/tensor with the highest reward.
Revised: Hence, it is essential to ﬁnd the entries (columns) of P with large rewards for each user(row) of P ; contrast this with the multi-dimensional online learning problem where it is sufﬁcient to infer only the entry ((row,column) pair) in the matrix/tensor with the highest reward.

Original: Since the rewards for each user have different gaps, the analysis becomes involved for our OCTAL algorithm.
Revised: Since the rewards for each user have different gaps, the analysis becomes involved for our OCTAL algorithm.

Revised: Finally, (Dadkhahi & Negahban, 2018; Zhou et al., 2020) also consider our problem setting but they only provide heuristic algorithms without any theoretical guarantees.

Original: Another closely related line of work is the theoretical model for User-based Collaborative Filtering (CF) studied in Bresler et al.
Revised: Another closely related line of work is the theoretical model for User-based Collaborative Filtering (CF) studied in Bresler et al.

Original: (2014; 2016); Heckel & Ramchandran (2017); Bresler & Karzand (2019);
Revised: (2014; 2016); Heckel & Ramchandran (2017); Bresler & Karzand (2019);

Original: Huleihel et al.
Revised: Huleihel et al.

Original: In particular, these papers were the ﬁrst to motivate and theoretically analyze the collaborative framework with the restriction that the same item cannot be recommended more than once to the same user.
Revised: In particular, these papers were the ﬁrst to motivate and theoretically analyze the collaborative framework with the restriction that the same item cannot be recommended more than once to the same user.

Original: Here, it is assumed that the M users can be partitioned into K latent clusters such that the preference vectors of users inthe same clusterare close; the preference vectors of users in different clusters have small inner product.
Revised: Here a signiﬁcantly stricter cluster structure assumption is made over users where users in same cluster have similar preferences.

Original: Such models areknown to be restrictive and typically collaborative ﬁltering via low-rank matrix completion based methods are preferred in real-world recommendation systems.
Revised: Such models are restrictive as they provide theoretical guarantees only on a very relaxed notion of regret (termed pseudo-regret ).

Original: In the past decade, several papers have studied the problem of ofﬂine low rank matrix completion on its own (Mazumder et al., 2010; Negahban & Wainwright, 2012; Chen et al., 2019; Deshpande &
Revised: In the past decade, several papers have studied the problem of ofﬂine low rank matrix completion on its own (Mazumder et al., 2010; Negahban & Wainwright, 2012; Chen et al., 2019; Deshpande &

Original: Montanari, 2012; Abbe et al., 2020; Jain et al., 2013; Jain & Kar, 2017) and also in the presence of side information such as social graphs or similarity graphs (Xu et al., 2013; Ahn et al., 2018; 2021;
Revised: Montanari, 2012; Abbe et al., 2020; Jain et al., 2013; Jain & Kar, 2017) and also in the presence of side information such as social graphs or similarity graphs (Xu et al., 2013; Ahn et al., 2018; 2021;

Original: Elmahdy et al., 2020; Jo & Lee, 2021; Zhang et al., 2022).
Revised: Elmahdy et al., 2020; Jo & Lee, 2021; Zhang et al., 2022).

Original: Some of these results namely the ones that provide k · k 1 norm guarantees on the estimated matrix can be adapted into Explore-Then-Commit (ETC) style algorithms (see Sec. 4).
Revised: Some of these results namely the ones that provide k · k 1 norm guarantees on the estimated matrix can be adapted into Explore-Then-Commit (ETC) style algorithms (see Sec. 4).

Original: Finally, there is signiﬁcant amount of related theoretical work for online non-convex learning (Suggala & Netrapalli, 2020;
Revised: Finally, there is signiﬁcant amount of related theoretical work for online non-convex learning (Suggala & Netrapalli, 2020;

Original: Yang et al., 2018; Huang et al., 2020) and empirical work for online Collaborative Filtering (Huang et al., 2020;
Revised: Yang et al., 2018; Huang et al., 2020) and empirical work for online Collaborative Filtering (Huang et al., 2020;

Original: Lu et al., 2013; Zhang et al., 2015)
Revised: Lu et al., 2013; Zhang et al., 2015)

Original: but they do not study the regret in online matrix completion setting.
Revised: but they do not study the regret in online matrix completion setting.

Original: Organization:

Original: In Section 2, we deﬁne the problem and the notion of regret that we minimize.

Original: In Section 3, we introduce several preliminary results (proofs delegated to Appendix B).

Original: In Section 4,

Original: Under review as a conference paper at ICLRwe discuss the Explore-Then-Commit (ETC) algorithm (proof in Appendix C) and in Section 5, we introduce and give an overview of the OCTAL algorithm (proofs delegated to Appendices D and E).

Original: In Appendix A, we provide detailed experimental results on synthetic and real world datasets.

Original:  P ROBLEM D EFINITION
Revised:  P ROBLEM D EFINITION

Original: Notations: We write [ m ] to denote the set { 1 , 2 , . . .
Revised: Notations: We write [ m ] to denote the set { 1 , 2 , . . .

Original: , m } .
Revised: , m } .

Original: For a vector v 2 R m , v i denotes the i th element; for any set
Revised: For a vector v 2 R m , v i denotes the i th element; for any set

Original: U ✓ [ m ] , let v U denote the vector v restricted to the indices in U .
Revised: U ✓ [ m ] , let v U denote the vector v restricted to the indices in U .

Original: A i denotes the i th row of A and A ij denotes the ( i, j ) -th element of matrix A .
Revised: A i denotes the i th row of A and A ij denotes the ( i, j ) -th element of matrix A .

Original: [ n ] denotes the set { 1 , 2 , . .
Revised: [ n ] denotes the set { 1 , 2 , . .

Original: . , n } .
Revised: . , n } .

Original: For any set
Revised: For any set

Original: U ⇢ [ m ] , V ⇢ [ n ] , A U , V denotes the matrix A restricted to the rows in U and columns in
Revised: U ⇢ [ m ] , V ⇢ [ n ] , A U , V denotes the matrix A restricted to the rows in U and columns in

Original: V .
Revised: V .

Original: Also, let k
Revised: Also, let k

Original: A
Revised: A

Original: k 2 !1 be the maximum ` 2 norm of the rows of A and k
Revised: k 2 !1 be the maximum ` 2 norm of the rows of A and k

Original: A k 1 be the absolute value of the largest entry in A .
Revised: A k 1 be the absolute value of the largest entry in A .

Original: We write E X to denote the expectation of a random variable X .
Revised: We write E X to denote the expectation of a random variable X .

Original: Consider a system with a set of M users and N items.
Revised: Consider a system with a set of M users and N items.

Original: Let P = UV T 2 R M ⇥
Revised: Let P = UV T 2 R M ⇥

Original: N be the unknown reward matrix of rank r < min( M , N ) where U 2 R M ⇥ r and V 2 R N ⇥ r denote the latent embeddings corresponding to users and items respectively.
Revised: N be the unknown reward matrix of rank r < min( M , N ) where U 2 R M ⇥ r and V 2 R N ⇥ r denote the latent embeddings corresponding to users and items respectively.

Original: In other words, we can denote P ij , h u i , v j i where u i , v j 2 R r denotes the r -dimensional embeddings of i -th user and the j -th item, respectively.
Revised: In other words, we can denote P ij , h u i , v j i where u i , v j 2 R r denotes the r -dimensional embeddings of i -th user and the j -th item, respectively.

Original: Often, we will also use the SVD decomposition of P = ¯U ⌃ ¯V where ¯ U 2 R M ⇥ r , ¯ V 2 R N ⇥ r are orthonormal matrices i.e. ¯ U T ¯ U =
Revised: Often, we will also use the SVD decomposition of P = ¯U ⌃ ¯V where ¯ U 2 R M ⇥ r , ¯ V 2 R N ⇥ r are orthonormal matrices i.e. ¯U T ¯U =

Original: I and ¯ V T ¯ V
Revised: I and ¯V T ¯V

Original: = I and ⌃ , diag (  1 ,  2 , . .
Revised: = I and ⌃ , diag (  1 ,  2 , . .

Original: ,  r ) 2 R r ⇥ r is a diagonal matrix.
Revised: ,  r ) 2 R r ⇥ r is a diagonal matrix.

Original: We will denote the condition number of the matrix P by  , (max i  i )
Revised: We will denote the condition number of the matrix P by  , (max i  i )

Original: (min i  i )  1 .
Revised: (min i  i )  1 .

Original: Consider a system that recommends one item to every user, in each round t 2 [ T ] .
Revised: Consider a system that recommends one item to every user, in each round t 2 [ T ] .

Original: Let, R ( t )
Revised: Let, R ( t )

Original: u ⇢ u ( t ) be the reward for recommending item ⇢ u ( t ) 2 [ N ] for user u .
Revised: u ⇢ u ( t ) be the reward for recommending item ⇢ u ( t ) 2 [ N ] for user u .

Original: Also, let:
Revised: Also, let:

Original: [Equation]
Revised: [Equation]

Original: where E ( t ) u ⇢ u ( t ) denotes the unbiased additive noise.
Revised: where E ( t ) u ⇢ u ( t ) denotes the unbiased additive noise.

Original: Each element of { E ( t ) u ⇢ u ( t ) } u 2 [ M ] ,t 2 [ T ] is assumed to be i.i.d.
Revised: Each element of { E ( t ) u ⇢ u ( t ) } u 2 [ M ] ,t 2 [ T ] is assumed to be i.i.d.

Original: zero mean sub-gaussian random variables with variance proxy  2 .
Revised: zero mean sub-gaussian random variables with variance proxy  2 .

Original: That is, E [ E ( t ) u ⇢ u ( t ) ] =and E [exp( s E ( t ) u ⇢ u ( t ) )]
Revised: That is, E [ E ( t ) u ⇢ u ( t ) ] =and E [exp( s E ( t ) u ⇢ u ( t ) )]

Original:  exp(  2 s 2 / 2) for all u 2 [ M ] , t 2 [ T ] .
Revised:  exp(  2 s 2 / 2) for all u 2 [ M ] , t 2 [ T ] .

Original: The goal is to minimize the expected regret where the expectation is over randomness in rewards and the algorithm:
Revised: The goal is to minimize the expected regret where the expectation is over randomness in rewards and the algorithm:

Original: [Equation]
Revised: [Equation]

Original: In this problem, the interesting regime is ( N , M )  T as is often the case for most practical recommendation systems.
Revised: In this problem, the interesting regime is ( N , M )  T as is often the case for most practical recommendation systems.

Original: Here, treating each user separately will lead to vacuous regret bounds as each item needs to be observed at least once by each user to ﬁnd the best item for each user.
Revised: Here, treating each user separately will lead to vacuous regret bounds as each item needs to be observed at least once by each user to ﬁnd the best item for each user.

Original: However, low-rank structure of the rewards can help share information about items across users.
Revised: However, low-rank structure of the rewards can help share information about items across users.

Original: Remark 1.
Revised: Remark 1.

Original: If T  N , then we can treat each user as a separate multi-armed bandit problem.
Revised: If T  N , then we can treat each user as a separate multi-armed bandit problem.

Original: In that case, in our setting, the well-studied Upper Conﬁdence Bound (UCB) algorithm achieves an expected regret of at most O
Revised: In that case, in our setting, the well-studied Upper Conﬁdence Bound (UCB) algorithm achieves an expected regret of at most O

Original: (  p NT log T ) (Theorem 2.1 in Bubeck & Cesa-Bianchi (2012)).
Revised: (  p NT log T ) (Theorem 2.1 in Bubeck & Cesa-Bianchi (2012)).

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Let us introduce a different observation model from (1).
Revised: Let us introduce a different observation model from (1).

Original: Consider an unknown rank r matrix P 2 R M ⇥ N .
Revised: Consider an unknown rank r matrix P 2 R M ⇥ N .

Original: For each entry i 2 [ M ] , j 2 [ N ] , we observe:
Revised: For each entry i 2 [ M ] , j 2 [ N ] , we observe:

Original: [Equation]
Revised: [Equation]

Original: where E ij are independent zero mean sub-gaussian random variables with variance proxy  2 > 0 .
Revised: where E ij are independent zero mean sub-gaussian random variables with variance proxy  2 > 0 .

Original: We now introduce the following result from Chen et al.
Revised: We now introduce the following result from Chen et al.

Original: Lemma 1 (Theorem 1 in Chen et al.
Revised: Lemma 1 (Theorem 1 in Chen et al.

Original: Let rank r
Revised: Let rank r

Original: = O (1) matrix P 2 R d ⇥ d with SVD decomposition P = ¯U ⌃ ¯V T satisfy k ¯U k 2 , 1  p µr/d, k ¯V k 2 , 1  p µr/d and condition number 
Revised: = O (1) matrix P 2 R d ⇥ d with SVD decomposition P = ¯U ⌃ ¯V T satisfy k ¯U k 2 , 1  p µr/d, k ¯V k 2 , 1  p µr/d and condition number 

Original: =
Revised: =

Original: O (1) .
Revised: O (1) .

Original: Let 1  p 
Revised: Let 1  p

Original: Cµ 2 d  1 log 3 d for some sufﬁciently large constant C > 0 ,  =
Revised: Cµ 2 d  1 log 3 d for some sufﬁciently large constant C > 0 ,  =

Original: O ⇣q
Revised: O ⇣q

Original: [Equation]
Revised: [Equation]

Original: ⌘ .
Revised: ⌘ .

Original: Suppose we observe noisy entries of P according to observation model 
Revised: Suppose we observe noisy entries of P according to observation model in (4).

Revised: Then, with probability exceeding 1  O ( d  3 ) , we can compute a matrix b

Revised: P by using Algorithm 4 (Appendix B) with parameters ( U =

Revised: [ M ] , V =

Revised: [ N ] ,  2 , r, p ) s.t.,

Revised: [Equation]

Original: Algorithm
Revised: Algorithm

Original: Require: Set of users U ✓ [ M ] , set of items V ✓ [ N ] , total rounds m , set of indices ⌦ ✓ U ⇥ V , rounds in each iteration b = max u 2 U | v 2 V | ( u, v ) 2 ⌦ | , regularization parameter  .
Revised: Require: Set of users U ✓ [ M ] , set of items V ✓ [ N ] , total rounds m , set of indices ⌦ ✓ U ⇥ V , rounds in each iteration b = max u 2 U | v 2 V | ( u, v ) 2 ⌦ | , regularization parameter  .

Original: Index of round t is relative to the ﬁrst round when the algorithm is invoked; hence t = 1 , 2 , . . .
Revised: Index of round t is relative to the ﬁrst round when the algorithm is invoked; hence t = 1 , 2 , . . .

Original: , m .
Revised: , m .

Original: 1: for ` = 1 , 2 , . . .
Revised: 1: for ` = 1 , 2 , . . .

Original: , m/b do 2:
Revised: , m/b do 2:

Original: For all ( i, j ) 2 ⌦ , set Mask ij = 0 .
Revised: For all ( i, j ) 2 ⌦ , set Mask ij = 0 .

Original: 3: for ` 0 = 1 , 2 , . . .
Revised: 3: for ` 0 = 1 , 2 , . . .

Original: , b do 4: for each user u 2 U in round t =
Revised: , b do 4: for each user u 2 U in round t =

Original: ( `  1) b
Revised: ( `  1) b

Original: + ` 0 do 5:
Revised: + ` 0 do 5:

Original: Recommend an item ⇢ u ( t ) in { j 2 V | ( u, j ) 2 ⌦ , Mask uj = 0 } and set Mask u ⇢ u ( t )
Revised: Recommend an item ⇢ u ( t ) in { j 2 V | ( u, j ) 2 ⌦ , Mask uj = 0 } and set Mask u ⇢ u ( t )

Original: = 1 .
Revised: = 1 .

Original: If not possible then recommend any item ⇢ u ( t ) in V s.t.
Revised: If not possible then recommend any item ⇢ u ( t ) in V s.t.

Original: ( u, ⇢ u ( t )) 62 ⌦ .
Revised: ( u, ⇢ u ( t )) 62 ⌦ .

Original: Observe R ( t ) u ⇢ u ( t ) .
Revised: Observe R ( t ) u ⇢ u ( t ) .

Original: 6: end for 7: end for 8: end for 9: For each ( u, j ) 2 ⌦ , compute Z uj to be average of b m/b c observations corresponding to user u being recommended item j i.e. Z uj =
Revised: 6: end for 7: end for 8: end for 9: For each ( u, j ) 2 ⌦ , compute Z uj to be average of b m/b c observations corresponding to user u being recommended item j i.e. Z uj =

Original: avg { R ( t ) u ⇢ u ( t ) for t 2 [ m ] | ⇢ u ( t ) =
Revised: avg { R ( t ) u ⇢ u ( t ) for t 2 [ m ] | ⇢ u ( t ) =

Original: j } .
Revised: j } .

Original: Discard all other observations corresponding to indices not in ⌦ .
Revised: Discard all other observations corresponding to indices not in ⌦ .

Original: 10: Without loss of generality, assume |U|  |V| .
Revised: 10: Without loss of generality, assume |U|  |V| .

Original: For each i 2 V , independently set  i to be a value in the set [ d |V| / |U| e ] uniformly at random.
Revised: For each i 2 V , independently set  i to be a value in the set [ d |V| / |U| e ] uniformly at random.

Original: Partition indices in V into V (1) , V (2) , . .
Revised: Partition indices in V into V (1) , V (2) , . .

Original: , V ( k ) where k = d |V| / |U| e and V ( q )
Revised: , V ( k ) where k = d |V| / |U| e and V ( q )

Original: = { i 2 V |  i = q } for each q 2 [ k ] .
Revised: = { i 2 V |  i = q } for each q 2 [ k ] .

Original: Set ⌦ ( q ) ⌦ \ ( U ⇥ V ( q ) ) for all q 2 [ k ] .
Revised: Set ⌦ ( q ) ⌦ \ ( U ⇥ V ( q ) ) for all q 2 [ k ] .

Original: # If |U|  |V| , we partition the indices in U .
Revised: # If |U|  |V| , we partition the indices in U .

Original: 11: for q 2 [ k ] do 12:
Revised: 11: for q 2 [ k ] do 12:

Original: Solve convex program min
Revised: Solve convex program min

Original: [Equation]
Revised: [Equation]

Original: where k Q ( q ) k ? denotes nuclear norm of matrix Q ( q ) and ⇡ ( j ) is index of j in set V ( q ) .
Revised: where k Q ( q ) k ? denotes nuclear norm of matrix Q ( q ) and ⇡ ( j ) is index of j in set V ( q ) .

Original: 13: end for 14: Return e
Revised: 13: end for 14: Return e

Original: [Equation]
Revised: [Equation]

Original: ) for all q 2 [ k ] and for every ( i, j )62 U ⇥ V ,
Revised: all q [Equation] [Equation] ] and for every ( i, j )

Original: [Equation]
Revised: [Equation]

Original: in (4). Then, with probability exceeding 1  O ( d  3 ) , we can compute a matrix b P by using Algorithm 4 (Appendix B) with parameters ( U = [ M ] , V =
Revised: U ⇥ V ,

Original: [ N ] ,  2 , r, p ) s.t.,

Original: [Equation]
Revised: [Equation]

Original: Note there are several difﬁculties in using Lemma 1 directly in our setting which are discussed below:
Revised: Note there are several difﬁculties in using Lemma 1 directly in our setting which are discussed below:

Original: Remark 2 (Matrix Completion for Rectangular Matrices) .
Revised: Remark 2 (Matrix Completion for Rectangular Matrices) .

Original: Lemma 1 is described for square matrices and a trivial approach to use Lemma 1 for rectangular matrices with M rows and N columns (say
Revised: Lemma 1 is described for square matrices and a trivial approach to use Lemma 1 for rectangular matrices with M rows and N columns (say

Original: N  M ) by appending N  M zero rows leads to an undesirable ( N / M ) 1 / 2 factor (Lemma 5) in the error bound (the ( N / M ) 1 / 2 factor does not arise if we care about spectral/Frobenius norm instead of
Revised: N  M ) by appending N  M zero rows leads to an undesirable ( N / M ) 1 / 2 factor (Lemma 5) in the error bound (the ( N / M ) 1 / 2 factor does not arise if we care about spectral/Frobenius norm instead of

Original: L 1 norm).
Revised: L 1 norm).

Original: One way to resolve the issue is to partition the columns into N / M groups by assigning each column into one of the groups uniformly at random.
Revised: One way to resolve the issue is to partition the columns into N / M groups by assigning each column into one of the groups uniformly at random.

Original: Thus, we create N / M matrices which are almost square and apply Lemma 1 to recover an estimate that is close in L 1 norm.
Revised: Thus, we create N / M matrices which are almost square and apply Lemma 1 to recover an estimate that is close in L 1 norm.

Original: Thus we can recover an estimate of the entire matrix which is close in L  to the  the undesirable  M )  10-12 in 
Revised: Thus we can recover an estimate of the entire matrix which is close in L 1 norm up to the desired accuracy without suffering the undesirable ( N / M ) 1 / 2 factor (Lemma 6 and Steps 10-12 in Algorithm 1).

Original: Remark 3 (Observation Models) .
Revised: Remark 3 (Observation Models) .

Original: The observation model in equation 4 is signiﬁcantly different from equation 1.
Revised: The observation model in equation 4 is signiﬁcantly different from equation 1.

Original: In the former, a noisy version of each element of P is observed independently with probability p while in the latter, in each round t 2 [ T ] , for each user u 2 [ M ] , we observe noisy version of a chosen element ⇢ u ( t ) .
Revised: In the former, a noisy version of each element of P is observed independently with probability p while in the latter, in each round t 2 [ T ] , for each user u 2 [ M ] , we observe noisy version of a chosen element ⇢ u ( t ) .

Original: Our approach to resolve this discrepancy theoretically is to ﬁrst sample a set ⌦ of indices according to equation 4 and subsequently use equation 1 to observe the indices in ⌦ (see Steps 3-6 in Algorithm 1 and Corollary 1).
Revised: Our approach to resolve this discrepancy theoretically is to ﬁrst sample a set ⌦ of indices according to equation 4 and subsequently use equation 1 to observe the indices in ⌦ (see Steps 3-6 in Algorithm 1 and Corollary 1).

Original: Of course, this implies obtaining observations corresponding to indices in a super-set of ⌦ (see Step 5 in Algorithm 1) and only using the observations in ⌦ for obtaining an estimate of the underlying matrix.
Revised: Of course, this implies obtaining observations corresponding to indices in a super-set of ⌦ (see Step 5 in Algorithm 1) and only using the observations in ⌦ for obtaining an estimate of the underlying matrix.

Original: In practice, this is not necessary and we can use all the observed indices to obtain the estimate in Step 12 of Algorithm 1.
Revised: In practice, this is not necessary and we can use all the observed indices to obtain the estimate in Step 12 of Algorithm 1.

Original: Remark 4 (Repetition and Median Tricks) .
Revised: Remark 4 (Repetition and Median Tricks) .

Original: The smallest error that is possible to achieve by using Lemma 1 is by substituting p = 1 and thereby obtaining k
Revised: The smallest error that is possible to achieve by using Lemma 1 is by substituting p = 1 and thereby obtaining k

Original: [Equation]
Revised: [Equation]

Original: Under review as conference paper

Original: [Equation] ⌘ and moreover, the probability of failure is polynomially small in the dimension d ; however, this is insufﬁcient when d is not large enough.
Revised: ⌘ and moreover, the probability of failure is polynomially small in the dimension d ; however, this is insufﬁcient when d is not large enough.

Original: Two simple tricks allow us to resolve this issue: 1) First we can obtain repeated observations from the same entry of the reward matrix and take its average; s repetitions can bring down the noise variance to  2 /s 2)
Revised: Two simple tricks allow us to resolve this issue: 1) First we can obtain repeated observations from the same entry of the reward matrix and take its average; s repetitions can bring down the noise variance to  2 /s 2)

Original: Second, we can use the median trick where we obtain several independent estimates of the reward matrix and compute the element-wise median to boost the success probability (see proof of Lemma 2).
Revised: Second, we can use the median trick where we obtain several independent estimates of the reward matrix and compute the element-wise median to boost the success probability (see proof of Lemma 2).

Original: We address all these issues (see Appendix B for detailed proofs) and arrive at the following lemma:
Revised: We address all these issues (see Appendix B for detailed proofs) and arrive at the following lemma:

Original: Lemma 2.
Revised: Lemma 2.

Original: Let rank r
Revised: Let rank r

Original: = O (1) reward matrix P 2 R M ⇥ N with SVD decomposition P = ¯U ⌃ ¯V T satisfy k ¯U k 2 , 1  p
Revised: = O (1) reward matrix P 2 R M ⇥ N with SVD decomposition P = ¯U ⌃ ¯V T satisfy k ¯U [Equation]  p

Original: [Equation]
Revised: [Equation]

Original: p µr/ N and condition number  =
Revised: p µr/ N and condition number  =

Original: O (1) .
Revised: O (1) .

Original: Let d 1 = max( M , N ) , d 2 = min( M , N ) such that d 2 = ⌦ ( µr log( rd 2 )) and 1  p  Cµ 2 d  1 2 log 3 d 2 for sufﬁciently large constant C > 0 .
Revised: Let d 1 = max( M , N ) , d 2 = min( M , N ) such that d 2 = ⌦ ( µr log( rd 2 )) and 1  p  Cµ 2 d  1 2 log 3 d 2 for sufﬁciently large constant C > 0 .

Original: Suppose we observe noisy entries of P according to observation model in (1).
Revised: Suppose we observe noisy entries of P according to observation model in (1).

Original: For any positive integer s > 0 satisfying  ps =
Revised: For any positive integer s > 0 satisfying  p

Revised: [Equation]

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original:  , there exists an algorithm A with parameters s, p,  that uses m =
Revised: ⌘ , there exists an algorithm A with parameters s, p,  that uses m =

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original: ⌘ rounds to compute a matrix b
Revised: ⌘ rounds to compute a matrix b

Original: P such that with probability exceeding 1  O (  log( MN   1 ))
Revised: P such that with probability exceeding 1  O (  log( MN   1 ))

Original: [Equation]
Revised: [Equation]

Revised: P

Revised: [Equation]

Original: Remark 5.
Revised: Remark 5.

Original: Alg.
Revised: Alg.

Original: A repeats the following process O
Revised: A repeats the following process O

Original: (log( MN   1 )) times: 1) sample subset of indices ⌦ ✓ [ M ] ⇥ [ N ] such that every ( i, j ) 

Revised: (log( MN   1 )) times: 1) sample subset of indices ⌦ ✓ [ M ] ⇥ [ N ] such that every ( i, j )

Original: [ M ] ⇥ [ N ] is inserted into ⌦ independently with probability p .
Revised: [ M ] ⇥ [ N ] is inserted into ⌦ independently with probability p .

Original: By setting b = max i 2 [ M ] | j 2 [ N ] | ( i, j ) 2 ⌦ | , Algorithm A invokes Alg. 1 with total rounds bs , number of rounds in each iteration b , set ⌦ , set of users [ M ] , items [ N ] and regularization parameter
Revised: By setting b = max i 2 [ M ] | j 2 [ N ] | ( i, j ) 2 ⌦ | , Algorithm A invokes Alg. 1 with total rounds bs , number of rounds in each iteration b , set ⌦ , set of users [ M ] , items [ N ] and regularization parameter

Original:  = C
Revised:  = C

Original:   p min( M , N ) p for a suitable constant C  > 0 in order to compute an estimate of P .
Revised:   p min( M , N ) p for a suitable constant C  > 0 in order to compute an estimate of P .

Original: The ﬁnal estimate b
Revised: The ﬁnal estimate b

Original: P is computed by taking an entry-wise median of each individual estimates obtained as output from each invocation of Alg. 1.
Revised: P is computed by taking an entry-wise median of each individual estimate obtained as output from several invocations of Alg. 1.

Original: Alternatively, Alg.
Revised: Alternatively, Alg.

Original: A is detailed in Alg.
Revised: A is detailed in Alg.

Original: 7 in Appendix B.
Revised: 7 in Appendix B.

Original: Note that the total number of noisy observations made from the matrix P is m · M  MN · p · s .
Revised: Note that the total number of noisy observations made from the matrix P is m · M  MN · p · s .

Original: Therefore, informally speaking, the average number of observations per index is p · s which results in an error of e
Revised: Therefore, informally speaking, the average number of observations per index is p · s which results in an error of e

Original: O (  / p sp ) ignoring other terms (contrast with error e
Revised: O (  / p sp ) ignoring other terms (contrast with error e

Original: O (  / p p ) in equation 5.)
Revised: O (  / p p ) in equation 5.)

Original: Remark 6 (Setting parameters s, p in lemma 2) .
Revised: Remark 6 (Setting parameters s, p in lemma 2) .

Original: Lemma 2 has three input parameters namely s 2 Z , 0  p  1 and 0    1 .
Revised: Lemma 2 has three input parameters namely s 2 Z , 0  p  1 and 0    1 .

Original: For any set of input parameters ( ⌘ , ⌫ ) , our goal is to set s, p,  as functions of known  , r, µ, d 2 such that we can recover k P  b
Revised: For any set of input parameters ( ⌘ , ⌫ ) , our goal is to set s, p,  as functions of known  , r, µ, d 2 such that we can recover k P  b

Original: P k 1  ⌘ with probability 1  ⌫ for which the conditions on  and p are satisﬁed.
Revised: P k 1  ⌘ with probability 1  ⌫ for which the conditions on  and p are satisﬁed.

Original: From (17), we must have p sp =
Revised: From (17), we must have p sp =

Original: c  r p d 2 p µ 3 log d 2 ⌘ for some appropriate constant c > 0 .
Revised: c  r p d 2 p µ 3 log d 2 ⌘ for some appropriate constant c > 0 .

Original: If r = O (1) and ⌘  k P k 1 , then an appropriate choice of c also satisﬁes the condition  p s =
Revised: If r = O (1) and ⌘  k P k 1 , then an appropriate choice of c also satisﬁes the condition  p s =

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original: ⌘ .

Original: More precisely, we are going to set p =
Revised: More precisely, we are going to set p =

Original: Cµ 2 d  1 2 log 3 d 2 and s =
Revised: Cµ 2 d  1 2 log 3 d 2 and s =

Original: l⇣ c  r p µ ⌘ log d⌘ m in order to obtain the desired guarantee.
Revised: l⇣ c  r p µ ⌘ log d⌘ m in order to obtain the desired guarantee.

Original:  E XPLORE -T HEN -C OMMIT (ETC) A LGORITHM
Revised:  E XPLORE -T HEN -C OMMIT (ETC) A LGORITHM

Original: In this section, we present an Explore-Then-Commit (ETC) based algorithm for online low-rank matrix completion.
Revised: In this section, we present an Explore-Then-Commit (ETC) based algorithm for online low-rank matrix completion.

Original: The algorithm has two disjoint phases of exploration and exploitation.
Revised: The algorithm has two disjoint phases of exploration and exploitation.

Original: We will ﬁrst jointly explore the set of items for all users for a certain number of rounds and compute an estimate b
Revised: We will ﬁrst jointly explore the set of items for all users for a certain number of rounds and compute an estimate b

Original: P of the reward matrix P .
Revised: P of the reward matrix P .

Original: Subsequently, we commit to the estimated best item found for each user and sample the reward of the best item for the remaining rounds in the exploitation phase for that user.
Revised: Subsequently, we commit to the estimated best item found for each user and sample the reward of the best item for the remaining rounds in the exploitation phase for that user.

Original: Note that the exploration phase involves using a matrix completion estimator in order to estimate the entire reward matrix P from few observed entries.
Revised: Note that the exploration phase involves using a matrix completion estimator in order to estimate the entire reward matrix P from few observed entries.

Original: Our regret guarantees in this framework is derived by carefully balancing exploration phase length and the matrix estimation error.
Revised: Our regret guarantees in this framework is derived by carefully balancing exploration phase length and the matrix estimation error (detailed proof provided in Appendix C).

Original: Theorem 1.
Revised: Theorem 1.

Original: Consider the rank- r online matrix completion problem with M users, N items, T recommendation rounds.
Revised: Consider the rank- r online matrix completion problem with M users, N items, T recommendation rounds.

Original: Set d 2 = min( M , N ) .
Revised: Set d 2 = min( M , N ) .

Original: Let R ( t ) u ⇢ u ( t ) be the reward in each round, deﬁned as in equation 1.
Revised: Let R ( t ) u ⇢ u ( t ) be the reward in each round, deﬁned as in equation 1.

Original: Suppose d 2 = ⌦ ( µr log( rd 2 )) .
Revised: Suppose d 2 = ⌦ ( µr log( rd 2 )) .

Original: Let P 2 R M ⇥
Revised: Let P 2 R M ⇥

Original: N be the expected reward matrix 
Revised: N be the expected reward matrix that satisﬁes the conditions stated in Lemma 2 , and let  2 be the noise variance in rewards. Then, Algorithm 2, applied to the online rank- r matrix completion problem guarantees the following regret: [Equation]

Original: Require: users M , items N , rounds T , noise  2 , rank r of P , no. of estimates f
Revised: Require: users M , items N , rounds T , noise  2 , rank r of P , upper bound on magnitude of expected rewards

Original:  = O (log( MNT )) .
Revised: || P || 1 , no. of estimates f = O (log( MNT )) .

Original: 1: Set d 2 = min( M , N ) and v = ( N k P k 1 )  2 /⇣
Revised: 1: Set d 2 = min( M , N ) and v = ( N k P k 1 )  2 /⇣

Original: T  r p dp µ 3 log d⌘ 2 / 3 .
Revised: T  r p dp µ 3 log d⌘ 2 / 3 .

Original: Set p
Revised: Set p

Original: =
Revised: =

Original: Cµ 2 d  1 2 log 3 d 2 , s =
Revised: Cµ 2 d  1 2 log 3 d 2 , s =

Original: d vp  1 e and  =
Revised: d vp  1 e and  =

Original: C   p d 2 p for some constants C, C  > 0 .
Revised: C   p d 2 p for some constants C, C  > 0 .

Original: 2: for k = 1 , 2 , . . .
Revised: 2: for k = 1 , 2 , . . .

Original: , f do 3:
Revised: , f do 3:

Original: For each tuple of indices ( i, j ) 2 [ M ] ⇥ [ N ] , independently set  ij = 1 with probability p and  ij = 0 with probability 1  p . 4:
Revised: For each tuple of indices ( i, j ) 2 [ M ] ⇥ [ N ] , independently set  ij = 1 with probability p and  ij = 0 with probability 1  p . 4:

Original: Denote ⌦ =
Revised: Denote ⌦ =

Original: { ( i, j ) 

Revised: { ( i, j )

Original: [ M ] ⇥
Revised: [ M ] ⇥

Original: [ N ] |  ij = 1 } and b = max i 

Revised: [ N ] |  ij = 1 } and b = max i

Original: [ M ] | | j 2 [ N ] | ( i, j ) 2 ⌦
Revised: [ M ] | | j 2 [ N ] | ( i, j ) 2 ⌦

Original: | to be the maximum number of index tuples in a particular row.
Revised: | to be the maximum number of index tuples in a particular row.

Original: Set total number of rounds to be bs .
Revised: Set total number of rounds to be bs .

Original: Compute the k th estimate b
Revised: Compute the k th estimate b

Original: P ( k ) =
Revised: P ( k ) =

Original: E STIMATE ([ M ] , [ N ] , bs, ⌦ , b,  ) .
Revised: E STIMATE ([ M ] , [ N ] , bs, ⌦ , b,  ) .

Original: # (Algorithm 1 is used to recommend items to every user for bs rounds.
Revised: # (Algorithm 1 is used to recommend items to every user for bs rounds.

Original: 6: end for 7: Compute ﬁnal estimate estimate b
Revised: 6: end for 7: Compute ﬁnal estimate estimate b

Original: P by taking the entry-wise median of b
Revised: P by taking the entry-wise median of b

Original: P (1) , b
Revised: P (1) , b

Original: P (2) , . . .
Revised: P (2) , . . .

Original: , b
Revised: , b

Original: P ( f ) .
Revised: P ( f ) .

Original: 8: for each of remaining rounds do 9:
Revised: 8: for each of remaining rounds do 9:

Original: Recommend argmax j 2 [ N ] b
Revised: Recommend argmax j 2 [ N ] b

Original: P ij for each user
Revised: P ij for each user

Original: i 2 [ M ] .
Revised: i 2 [ M ] .

Original: # Number of remaining rounds is T  bsf .
Revised: # Number of remaining rounds is T  bsf .

Original: 10: end forthat satisﬁes the conditions stated in Lemma 2 , and let  2 be the noise variance in rewards. Then, Algorithm 2, applied to the online rank- r matrix completion problem guarantees the following regret: Reg
Revised: 10: end for

Original: [Equation]

Original: Remark 7 (Non-trivial regret bounds) .
Revised: Remark 7 (Non-trivial regret bounds) .

Original: Theorem 1 provides non-trivial regret guarantees in the key regime when N  T and M > N where the regret scales only logarithmically on M , N .
Revised: Theorem 1 provides non-trivial regret guarantees in the key regime when N  T and M > N where the regret scales only logarithmically on M , N .

Original: This is intuitively satisfying since in each round we are obtaining M observations, so more users translate to more information which in-turn allows better understanding of the underlying reward matrix.
Revised: This is intuitively satisfying since in each round we are obtaining M observations, so more users translate to more information which in-turn allows better understanding of the underlying reward matrix.

Original: However, the dependence of regret on T (namely T 2 / 3 ) is sub-optimal.
Revised: However, the dependence of regret on T (namely T 2 / 3 ) is sub-optimal.

Original: In the subsequent section, we provide a novel algorithm to obtain regret guarantees with T 1 / 2 for rank- 1 P .
Revised: In the subsequent section, we provide a novel algorithm to obtain regret guarantees with T 1 / 2 for rank- 1 P .

Original: Remark 8 (Gap dependent bounds) .
Revised: Remark 8 (Gap dependent bounds) .

Original: Deﬁne the minimum gap to be  = min
Revised: Deﬁne the minimum gap to be  = min

Original: u [-
-] [ M ] [Equation]
Revised: u [-
-]2 [ M ]

Original:    where ⇡ u (1) , ⇡ u (2) corresponds to the items with the highest and second highest reward for user u respectively.
Revised: P u ⇡ u (1)  P u ⇡ u (2)  where ⇡ u (1) , ⇡ u (2) corresponds to the items with the highest and second highest reward for user u respectively.

Original: If the quantity  is known then it is possible to design ETC algorithms where  the exploration phase is tuned accordingly in order to obtain regret bounds that scale logarithmically with the number of rounds T .
Revised: If the quantity  is known then it is possible to design ETC algorithms where length of the exploration phase is tuned accordingly in order to obtain regret bounds that scale logarithmically with the number of rounds T .

Original:  OCTAL A LGORITHM
Revised:  OCTAL A LGORITHM

Original: In this section we present our algorithm OCTAL (Algorithm 3) for online matrix completion where the reward matrix P is rank 1 .
Revised: In this section we present our algorithm OCTAL (Algorithm 3) for online matrix completion where the reward matrix P is rank 1 .

Original: The set of users is described by a latent vector u 2 R M and the set of items is described by a latent vector v 2 R N .
Revised: The set of users is described by a latent vector u 2 R M and the set of items is described by a latent vector v 2 R N .

Original: Thus P = uv T with SVD decomposition P = ¯  ¯u¯v T .
Revised: Thus P = uv T with SVD decomposition P = ¯  ¯u¯v T .

Original: Algorithm Overview: Our ﬁrst key observation is that as P is rank-one, we can partition the set of users into two disjoint clusters
Revised: Algorithm Overview: Our ﬁrst key observation is that as P is rank-one, we can partition the set of users into two disjoint clusters

Original: C 1 , C 2 where C 1 ⌘ { i 2 [ M ] | u i  0 } and C 2 ⌘ [ M ] \ C 1 .
Revised: C 1 , C 2 where C 1 ⌘ { i 2 [ M ] | u i  0 } and C 2 ⌘ [ M ] \ C 1 .

Original: Clearly, for all users u 2 C 1 , the item that results in maximum reward is j max = argmax t 2 [ N ] v t .
Revised: Clearly, for all users u 2 C 1 , the item that results in maximum reward is j max = argmax t 2 [ N ] v t .

Original: On the other hand, for all users u 2 C 2 , the item that results in maximum reward is j min = argmin t 2 [ N ] v t .
Revised: On the other hand, for all users u 2 C 2 , the item that results in maximum reward is j min = argmin t 2 [ N ] v t .

Original: Thus, if we can identify
Revised: Thus, if we can identify

Original: C 1 , C 2 and estimate items with high reward (identical for users in the same cluster) using few recommendations per user, we can ensure low regret.
Revised: C 1 , C 2 and estimate items with high reward (identical for users in the same cluster) using few recommendations per user, we can ensure low regret.

Original: But, initially C 1 , C 2 are unknown, so all users are unlabelled i.e., their cluster is unknown.
Revised: But, initially C 1 , C 2 are unknown, so all users are unlabelled i.e., their cluster is unknown.

Original: In each phase (the outer loop indexed by ` ), Algorithm 3 tries to label at least a few unlabelled users correctly.
Revised: In each phase (the outer loop indexed by ` ), Algorithm 3 tries to label at least a few unlabelled users correctly.

Original: This is achieved by progressively reﬁning estimate e
Revised: This is achieved by progressively reﬁning estimate e

Original: Q of the reward matrix P restricted to the unlabelled users and all items (Step 12).
Revised: Q of the reward matrix P restricted to the unlabelled users and all items (Step 12).

Original: Subsequently, unlabelled users for which the difference in maximum and minimum reward (inferred from estimated reward matrix) is large are labelled (Step 19).
Revised: Subsequently, unlabelled users for which the difference in maximum and minimum reward (inferred from estimated reward matrix) is large are labelled (Step 19).

Original: At the same time, in Step 13 users labelled in previous phases are partitioned into two clusters (denoted by M ( ` , 1) and M ( ` , 2) ) and for each of them, the algorithm reﬁnes an estimate of two distinct sub-matrices of the reward matrix P by recommending items only from a reﬁned set ( N ( ` , 1) and N ( ` , 2) respectively) containing the best item ( j max or j min ).
Revised: At the same time, in Step 13 users labelled in previous phases are partitioned into two clusters (denoted by M ( ` , 1) and M ( ` , 2) ) and for each of them, the algorithm reﬁnes an estimate of two distinct sub-matrices of the reward matrix P by recommending items only from a reﬁned set ( N ( ` , 1) and N ( ` , 2) respectively) containing the best item ( j max or j min ).

Original: We also identify a small set 
Revised: We also identify a small set of good items for each labelled user (including users labelled in previous phases), which correspond to large estimated rewards. We partition all these users into two clusters ( M ( ` +1 , 1) and M ( ` +1 , 2) ) such that the set of good items for users in different clusters are disjoint. We can prove that such a partitioning is possible; users in same cluster have same sign of user embedding. Algorithm

Original: Require: Number of users M , items N , rounds T , noise 1: Set M (1 , 1) =
Revised: Require: Number of users M , items N , rounds T , noise  2 , bound on the entry-wise magnitude of expected rewards || P || 1 , incoherence µ . 1: Set

Original: M (1 ,  2) =
Revised: M (1 , 1) = M (1 , 2) =

Original:  and B (1) =
Revised:  and B (1) =

Original: [ M ] .
Revised: [ M ] .

Original: Set N (1 , 1) = N (1 , 2) =
Revised: Set N (1 , 1) = N (1 , 2) =

Original:  .
Revised:  .

Original: Set f
Revised: Set f

Original: = O (log( MNT )) and suitable constants a, c, C, C 0 , C  > 0 .
Revised: = O (log( MNT )) and suitable constants a, c, C, C 0 , C  > 0 .

Original: 2: for ` = 1 , 2 , . .
Revised: 2: for ` = 1 , 2 , . .

Original: . , do 3:
Revised: . , do 3:

Original: Set  ` = C 0 2  ` min ⇣ k
Revised: Set  ` = C 0 2  ` min ⇣ k

Original: P k 1 ,  p µ log N ⌘ .
Revised: P k 1 ,  p µ log N ⌘ .

Original: 4: for k = 1 , 2 , . . .
Revised: 4: for k = 1 , 2 , . . .

Original: , f do 5: for each pair of non-null sets ( B ( ` ) , N ) , ( M ( ` , 1) , N ( ` , 1) ) , ( M ( ` , 2) , N ( ` , 2) )
Revised: , f do 5: for each pair of non-null sets ( B ( ` ) , N ) , ( M ( ` , 1) , N ( ` , 1) ) , ( M ( ` , 2) , N ( ` , 2) )

Original: ✓ [ M ] ⇥
Revised: ✓ [ M ] ⇥

Original: [ N ] do 6:
Revised: [ N ] do 6:

Original: Denote ( T (1) , T (2) ) to be the considered pair of sets and i 2 { 0 , 1 , 2 } to be its index.
Revised: Denote ( T (1) , T (2) ) to be the considered pair of sets and i 2 { 0 , 1 , 2 } to be its index.

Original: Set d 2 ,i = min( |T (1) | , |T (2) | ) .
Revised: Set d 2 ,i = min( |T (1) | , |T (2) | ) .

Original: Set p
Revised: Set p

Original: `
Revised: `

Original: ,i =
Revised: ,i =

Original: Cµ 2 d  1 2 ,i log 3 d 2 ,i and s ` ,i =
Revised: Cµ 2 d  1 2 ,i log 3 d 2 ,i and s ` ,i =

Original: l⇣ c  p µ  ` log d 2 ,i ⌘ m .
Revised: l⇣ c  p µ  ` log d 2 ,i ⌘ m .

Original: For each tuple of indices ( u, v ) 2 T (1) ⇥ T (2) , independently set  uv = 1 with probability p ` ,i and  uv = 0 with probability 1  p ` ,i .
Revised: For each tuple of indices ( u, v ) 2 T (1) ⇥ T (2) , independently set  uv = 1 with probability p ` ,i and  uv = 0 with probability 1  p ` ,i .

Original: Denote ⌦ ( i ) =
Revised: Denote ⌦ ( i ) =

Original: { ( u, v ) 2 T (1) ⇥ T (2) |  uv = 1 } and b ` ,i = max u 2 U | v 2 V | ( u, v ) 2 ⌦ | .
Revised: { ( u, v ) 2 T (1) ⇥ T (2) |  uv = 1 } and b ` ,i = max u 2 U | v 2 V | ( u, v ) 2 ⌦ | .

Original: Set total number of rounds to be m ` ,i =
Revised: Set total number of rounds to be m ` ,i =

Original: b ` ,i s ` ,i .
Revised: b ` ,i s ` ,i .

Original: 10: end for 11:
Revised: 10: end for 11:

Original: Set m ` = max i 2 { 0 , 1 , 2 } m ` ,i .
Revised: Set m ` = max i 2 { 0 , 1 , 2 } m ` ,i .

Original: Compute e
Revised: Compute e

Original: Q ( ` ,k ) =
Revised: [Equation]

Original: E STIMATE ( |B ( ` ) | , [ N ] , m ` , ⌦ (0) , b ` , 0 ,  =
Revised: [Equation] [Equation] m [Equation]

Original: C
Revised: p d [Equation] [Equation]

Original:   p d 2 , 0 p ` ) .
Revised: p [Equation]

Original: # Algorithm 1 is used to recommend items to every user in B ( ` ) for m ` rounds.
Revised: # Algorithm 1 is used to recommend items to every user in B ( ` ) for m ` rounds.

Original: For i 2 { 1 , 2 } , compute e
Revised: For i 2 { 1 , 2 } , compute e

Original: P ( ` ,i,f ) =
Revised: P ( ` ,i,f ) =

Original: E STIMATE ( |M ( ` ,i ) | , |N
Revised: E STIMATE ( |M ( ` ,i ) | , |N

Original: ( ` ,i ) | , m ` , ⌦ ( i ) , b ` ,i ,  =
Revised: ( ` ,i ) | , m ` , ⌦ ( i ) , b ` ,i ,  =

Original: C   p d 2 ,i p ` ) .
Revised: C   p d 2 ,i p ` ) .

Original: # Algorithm 1 recommends items to every user in M ( ` ,i ) for m ` rounds .
Revised: # Algorithm 1 recommends items to every user in M ( ` ,i ) for m ` rounds .

Original: 14: end for 15:
Revised: 14: end for 15:

Original: Compute e
Revised: Compute e

Original: Q ( ` ) =
Revised: Q ( ` ) =

Original: Entrywise Median ( { e
Revised: Entrywise Median ( { e

Original: Q ( ` ,k ) } fk =1 ) , e
Revised: Q ( ` ,k ) } fk =1 ) , e

Original: P ( ` ,i ) =
Revised: P ( ` ,i ) =

Original: Entrywise Median ( { e
Revised: Entrywise Median ( { e

Original: P ( ` ,i,k ) } fk =1 ) for i 2 { 1 , 2 } .
Revised: P ( ` ,i,k ) } fk =1 ) for i 2 { 1 , 2 } .

Original: Set B ( ` +1) ⌘ n u 2 B ( ` ) |  maxt
Revised: Set B ( ` +1) ⌘ n [Equation] [Equation]  max

Original: [Equation]
Revised: [Equation]

Original: [ N ] e
Revised: ] e

Original: Q ( ` ) ut  min t 2 [ N ] e

Original: Q ( ` ) ut   2 a  ` o 17:
Revised: [Equation] e [Equation] o 17:

Original: Compute T ( ` +1) u 
Revised: Compute T ( ` +1) u =

Revised: [Equation]

Original: = { j 2 [ N ] } | e
Revised: [ N ] [Equation] | e

Original: Q ( ` ) uj +  ` > max t 2 [ N ] e
Revised: [Equation] max [Equation] ] e

Original: Q ( ` ) ut } for all u 2 B ( ` ) \ B ( ` +1) .
Revised: Q ( ` ) ut } for all u 2 B ( ` ) \ B ( ` +1) .

Original: For i 2 { 1 , 2 } , for all users u 2 M ( ` ,i ) , compute T ( ` +1) u
Revised: For i 2 { 1 , 2 } , for all users u 2 M ( ` ,i ) , compute T ( ` +1) u

Original: = { j 2 N ( ` ,i ) | e
Revised: = { j [Equation] N ( ` ,i ) | e

Original: P ( ` ,i ) uj +  ` > max t 2 N ( ` ,i ) e
Revised: P ( ` ,i ) [Equation]  ` > max t 2 N ( ` ,i ) e

Original: P ( ` ,i ) ut } .
Revised: P ( ` ,i ) ut } .

Original: Set v to be any user in [ M ] \B ( ` +1) .
Revised: Set v to be any user in [ M ] \B ( ` +1) .

Original: Set M
Revised: Set M

Original: ( ` +1 , 1) = { u 2 [ M ] \B ( ` +1)
Revised: ( ` +1 , 1) = { u 2 [ M ] \B ( ` +1)

Original: | T ( ` +1)
Revised: | T ( ` +1)

Original: u \ T ( ` +1) v 6 =  } .
Revised: u \ T ( ` +1) v 6 =  } .

Original: Set
Revised: Set

Original: M ( ` +1 , 2) =
Revised: M ( ` +1 , 2) =

Original: [ M ] \ ( B ( ` +1)
Revised: [ M ] \ ( B ( ` +1)

Original: [ M ( ` +1 , 1) ) .
Revised: [ M ( ` +1 , 1) ) .

Original: Compute N ( ` +1 , 1) =
Revised: Compute N ( ` +1 , 1) =

Original: T u 2 M ( ` +1 , 1) T ( ` +1) u , N ( ` +1 , 2) =
Revised: T u 2 M ( ` +1 , 1) T ( ` +1) u , N ( ` +1 , 2) =

Original: T u 2 M ( ` +1 , 2) T ( ` +1) u .
Revised: T u 2 M ( ` +1 , 2) T ( ` +1) u .

Original: For i 2 { 1 , 2 } , if |M ( ` +1 ,i ) |  M p T , then set B ( ` +1)
Revised: For i 2 { 1 , 2 } , if |M ( ` +1 ,i ) |  M p T , then set B ( ` +1)

Original: B ( ` +1)
Revised: B ( ` +1)

Original: [ M ( ` +1 ,i ) and M ( ` +1 ,i )  .
Revised: [ M ( ` +1 ,i ) and M ( ` +1 ,i )  .

Original: 22: end forof good items for each labelled user (including users labelled in previous phases), which correspond to large estimated rewards. We partition all these users into two clusters ( M ( ` +1 , 1) and M ( ` +1 , 2) ) such that the set of good items for users in different clusters are disjoint. We can prove that such a partitioning is possible; users in same cluster have same sign of user embedding.
Revised: 22: end for

Original: We also prove that the set of good items contain the best item for each labelled user ( j max or j min ).
Revised: We also prove that the set of good items contain the best item for each labelled user ( j max or j min ).

Original: So, after each phase, for each cluster of users, we compute the intersection of good items over all users in the cluster.
Revised: So, after each phase, for each cluster of users, we compute the intersection of good items over all users in the cluster.

Original: This subset of items ( joint good items) must contain the best item for that cluster and therefore we can discard the other items (Step 20).
Revised: This subset of items ( joint good items) must contain the best item for that cluster and therefore we can discard the other items (Step 20).

Original: We can show that all items in the set of joint good items ( N ( ` +1 , 1) and N ( ` +1 , 2) ) have rewards which is close to the reward of the best item.
Revised: We can show that all items in the set of joint good items ( N ( ` +1 , 1) and N ( ` +1 , 2) ) have rewards which is close to the reward of the best item.

Original: Therefore the algorithm suffers small regret if for each group of labelled users, the algorithm recommends items from the set of joint good items (Step 13) in the next phase.
Revised: Therefore the algorithm suffers small regret if for each group of labelled users, the algorithm recommends items from the set of joint good items (Step 13) in the next phase.

Original: We can further show that for the set of unlabelled users, the difference in rewards between the best item and worst item is small and hence the regret for such users is small, irrespective of the recommended item (Step 12).
Revised: We can further show that for the set of unlabelled users, the difference in rewards between the best item and worst item is small and hence the regret for such users is small, irrespective of the recommended item (Step 12).

Original: Note that until the number of labelled users is sufﬁciently large, we do not consider them separately (Step 21).
Revised: Note that until the number of labelled users is sufﬁciently large, we do not consider them separately (Step 21).

Original: A crucial part of our analysis is to show that for any subset of users and items considered in Step 5, the number of rounds sufﬁcient to recover a good estimate of the expected reward sub-matrix is small irrespective of the number of considered items (if the number of users is sufﬁciently large).
Revised: A crucial part of our analysis is to show that for any subset of users and items considered in Step 5, the number of rounds sufﬁcient to recover a good estimate of the expected reward sub-matrix is small irrespective of the number of considered items (if the number of users is sufﬁciently large).

Original: Remark 9 (Practical considerations) .
Revised: Remark 9 (Practical considerations) .

Original: In general OCTAL (Alg. 3) is computationally faster than the ETC Algorithm (Alg. 2) with a higher exploration length.
Revised: In general OCTAL (Alg. 3) is computationally faster than the ETC Algorithm (Alg. 2) with a higher exploration length.

Original: This is because OCTAL eliminates large Under review as a conference paper at ICLRchunks of items in every phase and therefore has to solve easier optimization problems; on the other hand, ETC has to solve a low rank matrix completion problem in MN variables that becomes slower with the exploration length (datapoints).
Revised: This is because OCTAL eliminates large chunks of items in every phase and therefore has to solve easier optimization problems; on the other hand, ETC has to solve a low rank matrix completion problem in MN variables that becomes slower with the exploration length (datapoints).

Original: Moreover, OCTAL algorithm runs in phases with the initial phases being very small; hence the users do not have to wait for a long time to even get personalized recommendations like in ETC.
Revised: Moreover, OCTAL algorithm runs in phases with the initial phases being very small; hence the users do not have to wait for a long time to even get personalized recommendations like in ETC.

Original: These features make OCTAL much more practical than ETC.
Revised: These features make OCTAL much more practical than ETC.

Original: To summarize, in Algorithm 3, the entire set of rounds [ T ] is partitioned into phases of exponentially increasing length.
Revised: To summarize, in Algorithm 3, the entire set of rounds [ T ] is partitioned into phases of exponentially increasing length.

Original: In each phase, for the set of unlabelled users, we do pure exploration and recommend random items from the set of all possible items (Step 12).
Revised: In each phase, for the set of unlabelled users, we do pure exploration and recommend random items from the set of all possible items (Step 12).

Original: The set of labelled users are partitioned into two clusters; for each, we follow a semi-exploration strategy where we recommend random items from a set of joint good items (Steps 13).
Revised: The set of labelled users are partitioned into two clusters; for each, we follow a semi-exploration strategy where we recommend random items from a set of joint good items (Steps 13).

Original: We now introduce the following deﬁnition:
Revised: We now introduce the following deﬁnition:

Original: Deﬁnition 3 ( ( ↵ , µ ) -Local Incoherence) .
Revised: Deﬁnition 3 ( ( ↵ , µ ) -Local Incoherence) .

Original: For 0  ↵  1 , a vector v 2 R m is ( ↵ , µ ) -local incoherent if for all sets U ✓ [ m ] satisfying |U|  ↵ m , we have k v U k 1 
Revised: For 0  ↵  1 , a vector v 2 R m is ( ↵ , µ ) -local incoherent if for all sets U ✓ [ m ] satisfying |U|  ↵ m , we have k v U k 1 

Original: [Equation]
Revised: [Equation]

Revised: Local incoherence for a vector v implies that any sub-vector of v having a signiﬁcant size must be incoherent as well.

Original: Note that the local incoherence condition is trivially satisﬁed if the magnitude of each vector entry is bounded from below.
Revised: Note that the local incoherence condition is trivially satisﬁed if the magnitude of each vector entry is bounded from below.

Original: We are now ready to state our main result:
Revised: We are now ready to state our main result:

Original: Theorem 2.
Revised: Theorem 2.

Original: Consider the rank- 1 online matrix completion problem with T rounds, M users s.t.
Revised: Consider the rank- 1 online matrix completion problem with T rounds, M users s.t.

Original: M  p T and N items.
Revised: M  p T and N items.

Original: Denote d 2 = min( M , N ) .
Revised: Denote d 2 = min( M , N ) .

Original: Let R ( t ) u ⇢ u ( t ) be the reward in each round, deﬁned as in equation 1.
Revised: Let R ( t ) u ⇢ u ( t ) be the reward in each round, deﬁned as in equation 1.

Original: Let  2 be the noise variance in rewards and let P 2 R M ⇥ N be the expected reward matrix with SVD decomposition P =
Revised: Let  2 be the noise variance in rewards and let P 2 R M ⇥ N be the expected reward matrix with SVD decomposition P =

Original:  ¯u¯v T such that ¯u is ( T  1 / 2 , µ ) -locally incoherent,
Revised:  ¯u¯v T such that ¯u is ( T  1 / 2 , µ ) -locally incoherent,

Original: [Equation]
Revised: [Equation]

Original: ¯v
Revised: ¯v

Original: [Equation]
Revised: [Equation]

Original: 1  p µ/ N , d 2 = ⌦ ( µ log d 2 ) and | ¯v j min | = ⇥ ( | ¯v j max | ) .
Revised: 1  p µ/ N , d 2 = ⌦ ( µ log d 2 ) and | ¯v j min | = ⇥ ( | ¯v j max | ) .

Original: Then, by suitably choosing parameters {  ` } ` , positive integers { s ( ` , 0) , s ( ` , 1) , s ( ` , 2) } ` and 1  { p ( ` , 0) , p ( ` , 1) , p ( ` , 2) } `  0 as described in Algorithm 3, we can ensure a regret guarantee of Reg ( T ) =
Revised: Then, by suitably choosing parameters {  ` } ` , positive integers { s ( ` , 0) , s ( ` , 1) , s ( ` , 2) } ` and 1  { p ( ` , 0) , p ( ` , 1) , p ( ` , 2) } `  0 as described in Algorithm 3, we can ensure a regret guarantee of Reg ( T ) =

Original: O ( p T k
Revised: O ( p T k

Original: P
Revised: P

Original: k
Revised: k

Original: + J p TV )
Revised: + J p TV )

Original: where
Revised: where

Original: J =
Revised: J =

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original: [Equation] ⌘⌘⌘ andV = ⇣
Revised: and

Original: [Equation]
Revised: [Equation]

Original: [Equation] ⌘ . Similar to Algorithm 2, Algorithm 3 allows non-trivial regret guarantees even when N  T provided the number of users is signiﬁcantly large as well i.e. M = e ⌦ ( N p T ) .
Revised: Similar to Algorithm 2, Algorithm 3 allows non-trivial regret guarantees even when N  T provided the number of users is signiﬁcantly large as well i.e. M = e ⌦ ( N p T ) .

Original: Remark 10.
Revised: Remark 10.

Original: Under slightly stronger local incoherence conditions on the vector ¯ u , we can analyze a modiﬁed version of OCTAL (Alg.
Revised: Under slightly stronger local incoherence conditions on the vector ¯ u , we can analyze a modiﬁed version of OCTAL (Alg.

Original: 8) without requiring users M to be large.
Revised: 8) without requiring users M to be large.

Original: When |C 1 | ⇡ |C| 2 , the regret guarantee (Thm. E) scales as e
Revised: When |C 1 | ⇡ |C| 2 , the regret guarantee (Thm. E) scales as e

Original: O ( p
Revised: O ( p

Original: NT / M ) .
Revised: NT / M ) .

Original: Due to space limitations, details of Algorithmand the proof of Theorem E can be found in Appendix E
Revised: Due to space limitations, details of Algorithmand the proof of Theorem E can be found in Appendix E

Original: Finally, we show that the above dependence on N , M , T matches the lower bound that we obtain by reduction to the well-known multi-armed bandit problem.
Revised: Finally, we show that the above dependence on N , M , T matches the lower bound that we obtain by reduction to the well-known multi-armed bandit problem.

Original: Theorem 3.
Revised: Theorem 3.

Original: Let P 2 [0 , 1] M ⇥ N be a rank 1 reward matrix and the noise variance  2 = 1 .
Revised: Let P 2 [0 , 1] M ⇥ N be a rank 1 reward matrix and the noise variance  2 = 1 .

Original: In that case, any algorithm for online matrix completion problem will suffer regret of ⌦ ( p NTM  1 ) .
Revised: In that case, any algorithm for online matrix completion problem will suffer regret of ⌦ ( p NTM  1 ) .

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We studied the problem of online rank-one matrix completion in the setting of repeated item recommendations and blocked item recommendations, which should be applicable for several practical recommendation systems.
Revised: We studied the problem of online rank-one matrix completion in the setting of repeated item recommendations and blocked item recommendations, which should be applicable for several practical recommendation systems.

Original: We analyzed an explore-then-commit (ETC) style method which is able to get the regret averaged over users to be nearly independent of number of items.
Revised: We analyzed an explore-then-commit (ETC) style method which is able to get the regret averaged over users to be nearly independent of number of items.

Original: That is, per user, we require only logarithmic many item recommendations to get non-trivial regret bound.
Revised: That is, per user, we require only logarithmic many item recommendations to get non-trivial regret bound.

Original: But, the dependence on the number of rounds T is sub-optimal.
Revised: But, the dependence on the number of rounds T is sub-optimal.

Original: We further improved this dependence by proposing OCTAL that carefully combines exploration, exploitation and clustering for different users/items.
Revised: We further improved this dependence by proposing OCTAL that carefully combines exploration, exploitation and clustering for different users/items.

Original: Our methods iteratively reﬁnes estimate of the underlying reward matrix, while also identifying users which can be recommended certain items conﬁdently.
Revised: Our methods iteratively reﬁnes estimate of the underlying reward matrix, while also identifying users which can be recommended certain items conﬁdently.

Original: Our algorithms and proof techniques are signiﬁcantly different than existing bandit learning literature.
Revised: Our algorithms and proof techniques are signiﬁcantly different than existing bandit learning literature.

Original: We believe that our work only scratches the surface of an important problem domain with several open problems.
Revised: We believe that our work only scratches the surface of an important problem domain with several open problems.

Original: For example,
Revised: For example,

Original: Algorithm 3 requires rank- 1 reward matrix.
Revised: Algorithm 3 requires rank- 1 reward matrix.

Original: Generalizing the result to rank- r reward matrices would be interesting.
Revised: Generalizing the result to rank- r reward matrices would be interesting.

Original: Furthermore, relaxing assumptions on the reward matrix like stochastic noise or additive noise should be relevant for several important settings.
Revised: Furthermore, relaxing assumptions on the reward matrix like stochastic noise or additive noise should be relevant for several important settings.

Original: Finally, collaborative ﬁltering can feed users related items, hence might exacerbate their biases.
Revised: Finally, collaborative ﬁltering can feed users related items, hence might exacerbate their biases.

Original: Our method might actually help mitigate the bias due to explicit exploration, but further investigation into such challenges is important.
Revised: Our method might actually help mitigate the bias due to explicit exploration, but further investigation into such challenges is important.