Original: PAC-Bayesian Matrix Completion with a Spectral Scaled Student Prior
Revised: PAC-Bayesian Matrix Completion with a Spectral Scaled Student Prior

Original: Abstract
Revised: Abstract

Original: We study the problem of matrix completion in this paper.
Revised: We study the problem of matrix completion in this paper.

Original: A spectral scaled Student prior is exploited to favour the underlying low-rank structure of the data matrix.
Revised: A spectral scaled Student prior is exploited to favour the underlying low-rank structure of the data matrix.

Original: We provide a thorough theoretical investigation for our approach through PAC-Bayesian bounds.
Revised: We provide a thorough theoretical investigation for our approach through PAC-Bayesian bounds.

Original: More precisely, our PAC-Bayesian approach enjoys a minimax-optimal oracle inequality which guarantees that our method works well under model misspeciﬁcation and under general sampling distribution.
Revised: More precisely, our PAC-Bayesian approach enjoys a minimax-optimal oracle inequality which guarantees that our method works well under model misspeciﬁcation and under general sampling distribution.

Original: Interestingly, we also provide eﬃcient gradient-based sampling implementations for our approach by using Langevin Monte Carlo.
Revised: Interestingly, we also provide eﬃcient gradient-based sampling implementations for our approach by using Langevin Monte Carlo.

Original: More speciﬁcally, we show that our algorithms are signiﬁcantly faster than Gibbs sampler in this problem.
Revised: More speciﬁcally, we show that our algorithms are signiﬁcantly faster than Gibbs sampler in this problem.

Original: To illustrate the attractive features of our inference strategy, some numerical simulations are conducted and an application to image inpainting is demonstrated
Revised: To illustrate the attractive features of our inference strategy, some numerical simulations are conducted and an application to image inpainting is demonstrated

Original: 1. Introduction
Revised: 1. Introduction

Original: Arising in various applications such as recommender systems Bennett and Lanning (2007);
Revised: Arising in various applications such as recommender systems Bennett and Lanning (2007);

Original: Xiong et al.
Revised: Xiong et al.

Original: ; Adomavicius and Tuzhilin (2011), genotype imputation Chi et al.
Revised: ; Adomavicius and Tuzhilin (2011), genotype imputation Chi et al.

Original: Jiang et al.
Revised: Jiang et al.

Original: image processing Cabral et al.
Revised: image processing Cabral et al.

Original: ; Luo et al. (2015); He and Sun (2014) and quantum state tomography Gross et al.
Revised: ; Luo et al. (2015); He and Sun (2014) and quantum state tomography Gross et al.

Original: (2010); Mai and Alquier (2017), matrix completion aims at rebuilding a matrix from its partially observed entries.
Revised: (2010); Mai and Alquier (2017), matrix completion aims at rebuilding a matrix from its partially observed entries.

Original: Most of the recent methods for this problem are usually based on penalized optimizations which are considered both from theoretical and computational point of views.
Revised: Most of the recent methods for this problem are usually based on penalized optimizations which are considered both from theoretical and computational point of views.

Original: The seminal results were done in Cand`es and Recht (2009); Cand`es and Tao (2010); Cand`es and Plan (2010);
Revised: The seminal results were done in Cand`es and Recht (2009); Cand`es and Tao (2010); Cand`es and Plan (2010);

Original: Koltchinskii et al.
Revised: Koltchinskii et al.

Original: ; Negahban and Wainwright (2012).
Revised: ; Negahban and Wainwright (2012).

Original: Several eﬃcient algorithms had also been proposed and studied as in Mazumder et al.
Revised: Several eﬃcient algorithms had also been proposed and studied as in Mazumder et al.

Original: (2010); Recht and R´e (2013);
Revised: (2010); Recht and R´e (2013);

Original: Hastie et al.
Revised: Hastie et al.

Original: On the other hand, various Bayesian approaches have also been proposed in the problem of matrix completion largely from a computational direction Lim and Teh (2007);
Revised: On the other hand, various Bayesian approaches have also been proposed in the problem of matrix completion largely from a computational direction Lim and Teh (2007);

Original: Salakhutdinov and Mnih (2008); Zhou et al. (2010); Alquier et al.
Revised: Salakhutdinov and Mnih (2008); Zhou et al. (2010); Alquier et al.

Original: ; Lawrence and Urtasun (2009); Cottet and Alquier (2018); Babacan et al.
Revised: ; Lawrence and Urtasun (2009); Cottet and Alquier (2018); Babacan et al.

Original: ; Yang et al.
Revised: ; Yang et al.

Original: These Bayesian estimators are mostly based on conjugate low-rank factorization priors which allow to use
Revised: These Bayesian estimators are mostly based on conjugate low-rank factorization priors which allow to use

Original: Gibbs sampling Alquier et al.
Revised: Gibbs sampling Alquier et al.

Original: Salakhutdinov and Mnih (2008).
Revised: Salakhutdinov and Mnih (2008).

Original: Nevertheless, these
Revised: Nevertheless, these

Original: Gibbs samplers require to calculate a number of matrix inversions or singular value decompositions at each iteration which is costly and can slow down signiﬁcantly the algorithm for large data.
Revised: Gibbs samplers require to calculate a number of matrix inversions or singular value decompositions at each iteration which is costly and can slow down signiﬁcantly the algorithm for large data.

Original: Variational Bayes methods based on optimization have been also considered in this problem Lim and Teh (2007); Babacan et al.
Revised: Variational Bayes methods based on optimization have been also considered in this problem Lim and Teh (2007); Babacan et al.

Original: ; Yang et al.
Revised: ; Yang et al.

Original: However, the theoretical understanding of Bayesian estimators has received quite limited attention, up to ourknowledge, Mai and Alquier (2015) and Alquier and Ridgway (2020) are the only prominent works.
Revised: However, the theoretical understanding of Bayesian estimators has received quite limited attention, up to ourknowledge, Mai and Alquier (2015) and Alquier and Ridgway (2020) are the only prominent works.

Original: More speciﬁcally, they showed that a Bayesian estimator with a low-rank factorization prior reaches the minimax-optimal rate up to a logarithmic factor.
Revised: More speciﬁcally, they showed that a Bayesian estimator with a low-rank factorization prior reaches the minimax-optimal rate up to a logarithmic factor.

Original: The paper Alquier and Ridgway (2020) further shows that the same rate can be obtained by using a Variational Bayesian estimator and the concentration rate of posterior is also studied in their works.
Revised: The paper Alquier and Ridgway (2020) further shows that the same rate can be obtained by using a Variational Bayesian estimator and the concentration rate of posterior is also studied in their works.

Original: In this paper, we study the problem of Bayesian matrix completion where a spectral scaled Student is exploited to favour the (approximate) low-rank structure of the underlying matrix.
Revised: In this paper, we study the problem of Bayesian matrix completion where a spectral scaled Student is exploited to favour the (approximate) low-rank structure of the underlying matrix.

Original: We prove that our PAC-Bayesian estimator enjoys a general minimax-optimal oracle inequality.
Revised: We prove that our PAC-Bayesian estimator enjoys a general minimax-optimal oracle inequality.

Original: As a result, it shows that our estimator works well in general cases which are under model misspeciﬁcation and under general sampling distributon.
Revised: As a result, it shows that our estimator works well in general cases which are under model misspeciﬁcation and under general sampling distributon.

Original: While this theory is similar to Mai and Alquier (2015), where a diﬀerent prior (a low-rank factorizaton prior) is studied, our result presents an absolute improvement over that paper.
Revised: While this theory is similar to Mai and Alquier (2015), where a diﬀerent prior (a low-rank factorizaton prior) is studied, our result presents an absolute improvement over that paper.

Original: More speciﬁcally, the leading constant in our minimax-optimal oracle inequality is strictly smaller thanwhile the leading constant in Mai and Alquier (2015) is 3.
Revised: More speciﬁcally, the leading constant in our minimax-optimal oracle inequality is strictly smaller thanwhile the leading constant in Mai and Alquier (2015) is 3.

Original: Up to our knowledge, a sharp minimax-optimal oracle inequality with leading constant 1 has not yet been obtained for
Revised: Up to our knowledge, a sharp minimax-optimal oracle inequality with leading constant 1 has not yet been obtained for

Original: Bayesian matrix completion.
Revised: Bayesian matrix completion.

Original: 2. Bayesian matrix completion 2.1. Model
Revised: 2. Bayesian matrix completion 2.1. Model

Original: Let M ∗ ∈ R m × p be an unknown (expected to be low-rank) matrix of interest.
Revised: Let M ∗ ∈ R m × p be an unknown (expected to be low-rank) matrix of interest.

Original: We observe a random subset of noisy entries of M ∗ as
Revised: We observe a random subset of noisy entries of M ∗ as

Original: [Equation]
Revised: [Equation]

Original: where Ω is a subset of indices { 1 , . .
Revised: where Ω is a subset of indices { 1 , . .

Original: . , m } × { 1 , . . .
Revised: . , m } × { 1 , . . .

Original: , p } and E ij are independently generated noise at the location ( i, j ) with E ( E ij ) = 0.
Revised: , p } and E ij are independently generated noise at the location ( i, j ) with E ( E ij ) = 0.

Original: Let Π ij denote the probability to observe the ( i, j )-th entry.
Revised: Let Π ij denote the probability to observe the ( i, j )-th entry.

Original: Then, the problem of recovering M ∗ with n = | Ω | < mp under the assumption that rank( M ∗ ) (cid:28) min( m, p ) is called the noisy matrix completion problem.
Revised: Then, the problem of recovering M ∗ with n = | Ω | < mp under the assumption that rank( M ∗ ) (cid:28) min( m, p ) is called the noisy matrix completion problem.

Original: We consider the following (pseudo-)Bayesian mean estimator with a given prior distribution π ( dM ),
Revised: We consider the following (pseudo-)Bayesian mean estimator with a given prior distribution π ( dM ),

Original: [Equation]
Revised: [Equation]

Original: where ˆ ρ λ ( dM ) ∝ exp( − λr ( M ))
Revised: where ˆ ρ λ ( dM ) ∝ exp( − λr ( M ))

Original: π ( dM ) is the posterior in which r ( M )
Revised: π ( dM ) is the posterior in which r ( M )

Original: =
Revised: =

Original: (cid:80) ( i,j
Revised: (cid:80) ( i,j

Original: ) ∈ Ω ( Y ij − M ij ) 2 /n.
Revised: ) ∈ Ω ( Y ij − M ij ) 2 /n.

Original: The choice λ = n/ (2 σ 2 ) is corresponding exactly to the Bayesian mean estimator that would be obtained for a Gaussian noise.
Revised: The choice λ = n/ (2 σ 2 ) is corresponding exactly to the Bayesian mean estimator that would be obtained for a Gaussian noise.

Original: However, by using λ , this will allow us to obtain the optimality of the estimator under a wider class of noises.
Revised: However, by using λ , this will allow us to obtain the optimality of the estimator under a wider class of noises.

Original: Moreover, this kind of fractional posterior has been shown to work well in misspeciﬁcation model Alquier and Ridgway (2020); Bhattacharya et al.
Revised: Moreover, this kind of fractional posterior has been shown to work well in misspeciﬁcation model Alquier and Ridgway (2020); Bhattacharya et al.

Original: (2019); Gr¨unwald and Van Ommen (2017)
Revised: (2019); Gr¨unwald and Van Ommen (2017)

Original: ; Bissiri et al.
Revised: ; Bissiri et al.

Original: (2016) which in our setup can be used for approximate low-rank model.
Revised: (2016) which in our setup can be used for approximate low-rank model.

Original: 2.2. Low-rank promoting prior: a spectral scaled Student prior
Revised: 2.2. Low-rank promoting prior: a spectral scaled Student prior

Original: We consider the following prior,
Revised: We consider the following prior,

Original: [Equation]
Revised: [Equation]

Original: where τ > 0 is a tuning parameter and I m is the m × m identity matrix .
Revised: where τ > 0 is a tuning parameter and I m is the m × m identity matrix .

Original: To illustrate that this prior has the potential to encourage the low-rankness of M , one can check that
Revised: To illustrate that this prior has the potential to encourage the low-rankness of M , one can check that

Original: [Equation]
Revised: [Equation]

Original: where s j ( M ) denotes the j -th largest singular value of M .
Revised: where s j ( M ) denotes the j -th largest singular value of M .

Original: It is well known that the log-sum function (cid:80) mj
Revised: It is well known that the log-sum function (cid:80) mj

Original: =1 log( τ 2 + s j ( M ) 2 ) encourages a sparsity on { s j ( M ) } , see Candes et al.
Revised: =1 log( τ 2 + s j ( M ) 2 ) encourages a sparsity on { s j ( M ) } , see Candes et al.

Original: Yang et al.
Revised: Yang et al.

Original: Alternatively, one can recognize a scaled Student distribution evaluated at s j ( M ) in the last display above which induces sparsity on { s j ( M ) } , Dalalyan and
Revised: Alternatively, one can recognize a scaled Student distribution evaluated at s j ( M ) in the last display above which induces sparsity on { s j ( M ) } , Dalalyan and

Original: Tsybakov (2012).
Revised: Tsybakov (2012).

Original: Thus the resulting matrix M enjoys a low-rank structrure, approximately.
Revised: Thus the resulting matrix M enjoys a low-rank structrure, approximately.

Original: Although this prior is not conjugate in our problem, it is particularly convenient to implement the Langevin Monte Carlo algorithm, a gradient-based sampling method, see
Revised: Although this prior is not conjugate in our problem, it is particularly convenient to implement the Langevin Monte Carlo algorithm, a gradient-based sampling method, see

Original: Section 3.
Revised: Section 3.

Original: 2.3. Theoretical guarantees
Revised: 2.3. Theoretical guarantees

Original: Before we present the theoretical guarantees for our procedure, let us formulate some assumptions.
Revised: Before we present the theoretical guarantees for our procedure, let us formulate some assumptions.

Original: Assumption 1:
Revised: Assumption 1:

Original: There is a known L > 0 such that (cid:107) M ∗(cid:107)
Revised: There is a known L > 0 such that (cid:107) M ∗

Original: [Equation]
Revised: [Equation]

Original: Assumption 2:
Revised: Assumption 2:

Original: The noise variables E 1 , . . .
Revised: The noise variables E 1 , . . .

Original: , E n are independent and independent of X 1 , . . .
Revised: , E n are independent and independent of X 1 , . . .

Original: , X n .
Revised: , X n .

Original: There exist two known constants σ > 0 and ξ > 0 such that
Revised: There exist two known constants σ > 0 and ξ > 0 such that

Original: [Equation]
Revised: [Equation]

Original: For any matrix A m × p , let (cid:107) A (cid:107) F denote the Frobenius norm, i.e, (cid:107) A (cid:107) 2 F =
Revised: For any matrix A m × p , let (cid:107) A (cid:107) F denote the Frobenius norm, i.e, (cid:107) A (cid:107) 2 F =

Original: Tr( A T A ).
Revised: Tr( A T A ).

Original: We deﬁne a (general-Frobenius) “norm” as follow (cid:107) A (cid:107) 2 F, Π = (cid:80) ij ( A ij ) 2 Π ij .
Revised: We deﬁne a (general-Frobenius) “norm” as follow (cid:107) A (cid:107) 2 F, Π = (cid:80) ij ( A ij ) 2 Π ij .

Original: Note that when the sampling distribution Π is uniform, then (cid:107) A (cid:107) 2 F, Π = (cid:107) A (cid:107) 

Revised: Note that when the sampling distribution Π is uniform, then (cid:107) A (cid:107) 2 F, Π = (cid:107) A (cid:107)

Original: F / ( mp ) .
Revised: F / ( mp ) .

Original: Put C 1 := 2 (cid:2) 4 σ 2 + (3 L ) 2 (cid:3) and C 2 :=
Revised: Put C 1 := 2 (cid:2) 4 σ 2 + (3 L ) 2 (cid:3) and C 2 :=

Original: 12 L (2 ξ +3 L ) and take C such that C > C 2 +3 C 1 / 2.
Revised: 12 L (2 ξ +3 L ) and take C such that C > C 2 +3 C 1 / 2.

Original: We are now ready to state our main theoretical result and the proof of this theorem is postponed to Appendix A.
Revised: We are now ready to state our main theoretical result and the proof of this theorem is postponed to Appendix A.

Original: Theorem
Revised: Theorem

Original: Let Assumption 1 and 2 be satisﬁed and take λ = λ ∗ := n/C .
Revised: Let Assumption 1 and 2 be satisﬁed and take λ = λ ∗ := n/C .

Original: Then, with probability at least 1 − (cid:15), (cid:15) ∈ (0 , 1) , one has for any matrix ¯  M  with its rank at most r that
Revised: Then, with probability at least 1 − (cid:15), (cid:15) ∈ (0 , 1) , one has for any matrix ¯ M, s.t. sup i,j | ¯ M ij | ≤ L with its rank at most r that

Original: [Equation]
Revised: [Equation]

Original: where δ is in (0 , 1) and C > 6 C is a universal constant that depends on σ 2 , L, ξ only.
Revised: where δ is in (0 , 1) and C > 6 C is a universal constant that depends on σ 2 , L, ξ only.

Original: We remark that our oracle inequality (4) comes with a leading constant 1+ δ < 2 (thank to a more careful calculation) with 0 < δ < 1 that is absolutely smaller than the leading constant 3 in Theorem 1 in Mai and Alquier (2015).
Revised: We remark that our oracle inequality (4) comes with a leading constant 1+ δ < 2 (thank to a more careful calculation) with 0 < δ < 1 that is absolutely smaller than the leading constant 3 in Theorem 1 in Mai and Alquier (2015).

Original: Thus our work presents an improved oracle inequality for Bayesian matrix completion.
Revised: Thus our work presents an improved oracle inequality for Bayesian matrix completion.

Original: We would like to note that a sharp oracle inequality with a leading constant 1 is not yet obtained for Bayesian matrix completion and remains as an important open question.
Revised: We would like to note that a sharp oracle inequality with a leading constant 1 is not yet obtained for Bayesian matrix completion and remains as an important open question.

Original: We note that Theorem 1 is stated under a general setting.
Revised: We note that Theorem 1 is stated under a general setting.

Original: More speciﬁcally, it holds without any assumption on the sampling distribution of the observations as done in Mai and
Revised: More speciﬁcally, it holds without any assumption on the sampling distribution of the observations as done in Mai and

Original: Alquier (2015) while other works require some, see e.g. Foygel et al.
Revised: Alquier (2015) while other works require some, see e.g. Foygel et al.

Original: (2011); Klopp (2014);
Revised: (2011); Klopp (2014);

Original: Negahban and Wainwright (2012).
Revised: Negahban and Wainwright (2012).

Original: Moreover, Theorem 1 can also be used in various setup where the underlying matrix M ∗ is for example low-rank, approximate low-rank. . .
Revised: Moreover, Theorem 1 can also be used in various setup where the underlying matrix M ∗ is for example low-rank, approximate low-rank. . .

Original: Several special cases are derived in the Appendix A.
Revised: Several special cases are derived in the Appendix A.

Original: It is remarked that the convergence rate r ( m + p ) log ( mp ) /n is minimax-optimal up to a log-factor.
Revised: It is remarked that the convergence rate r ( m + p ) log ( mp ) /n is minimax-optimal up to a log-factor.

Original: A lower bound for low-rank matrix completion is provided in Koltchinskii et al.
Revised: A lower bound for low-rank matrix completion is provided in Koltchinskii et al.

Original: (2011) that is r ( m + p ) /n , whereas the sharp upper bounds are obtained by penalized minimization methods in Klopp (2015); Chen et al.
Revised: (2011) that is r ( m + p ) /n , whereas the sharp upper bounds are obtained by penalized minimization methods in Klopp (2015); Chen et al.

Original: Up to our knowledge, however, a sharp rate for Bayesian estimators in the problem of matrix completion still remains open.
Revised: Up to our knowledge, however, a sharp rate for Bayesian estimators in the problem of matrix completion still remains open.

Original: The paper Mai (2021b), based on numerical comparisons between Bayesian methods and a de-biased estimator in Chen et al.
Revised: The paper Mai (2021b), based on numerical comparisons between Bayesian methods and a de-biased estimator in Chen et al.

Original: (2019) which sharply reached the minimiax-optimal rate, conjectures that the Bayesian methods could actually reach this rate sharply and the additonal logarithmic factor could be due to the teachnical proofs.
Revised: (2019) which sharply reached the minimiax-optimal rate, conjectures that the Bayesian methods could actually reach this rate sharply and the additonal logarithmic factor could be due to the teachnical proofs.

Original: 3. Numerical Studies 3.1. Unadjusted Langevin Monte Carlo algorithm
Revised: 3. Numerical Studies 3.1. Unadjusted Langevin Monte Carlo algorithm

Original: Let P Ω ( · ) : R m × p (cid:55)→ R m × p be the orthogonal projection onto the observed entries in the index set
Revised: Let P Ω ( · ) : R m × p (cid:55)→ R m × p be the orthogonal projection onto the observed entries in the index set

Original: Ω that P Ω ( Y ) ij =
Revised: Ω that P Ω ( Y ) ij =

Original: Y ij , if ( i, j ) ∈ Ω , and 0 otherwise.
Revised: Y ij , if ( i, j ) ∈ Ω , and 0 otherwise.

Original: We propose to compute an approximation of the posterior in (2) by a suitable version of the Langevin Monte Carlo algorithm, a gradient-based sampling method.
Revised: We propose to compute an approximation of the posterior in (2) by a suitable version of the Langevin Monte Carlo algorithm, a gradient-based sampling method.

Original: Let us remind that we have
Revised: Let us remind that we have

Original: [Equation]
Revised: [Equation]

Original: In this work, we aim at using the constant step-size unadjusted Langevin Monte Carlo algorithm (denoted by LMC), see Durmus and Moulines (2019) for detail.
Revised: In this work, we aim at using the constant step-size unadjusted Langevin Monte Carlo algorithm (denoted by LMC), see Durmus and Moulines (2019) for detail.

Original: This algorithm is deﬁned by selecting an initial matrix M 0 and then by using the recursion
Revised: This algorithm is deﬁned by selecting an initial matrix M 0 and then by using the recursion

Original: [Equation]
Revised: [Equation]

Original: where h > 0 is the step-size and W 0 , W 1 , . .
Revised: where h > 0 is the step-size and W 0 , W 1 , . .

Original: . are independent random matrices with i.i.d.
Revised: . are independent random matrices with i.i.d.

Original: standard Gaussian entries.
Revised: standard Gaussian entries.

Original: For small values of the step-size h , the posterior mean ˆ M = (cid:80) Tk M k /T is very close to the integral (2) of interest.
Revised: For small values of the step-size h , the posterior mean ˆ M = (cid:80) Tk M k /T is very close to the integral (2) of interest.

Original: However, for some h that may not be small enough, the Markov process can be transient and as a consequence the sum explodes Roberts and Stramer (2002).
Revised: However, for some h that may not be small enough, the Markov process can be transient and as a consequence the sum explodes Roberts and Stramer (2002).

Original: Several strategies are available to address this issue: one can take a smaller h and restart the algorithm or a Metropolis–Hastings correction can be included in the algorithm (denoted MALA, see details in the Appendix A).
Revised: Several strategies are available to address this issue: one can take a smaller h and restart the algorithm or a Metropolis–Hastings correction can be included in the algorithm (denoted MALA, see details in the Appendix A).

Original: The Metropolis–Hastings approach ensures the convergence to the desired distribution, however, the algorithm is greatly slowed down because of an additional acception/rejection step at each iteration.
Revised: The Metropolis–Hastings approach ensures the convergence to the desired distribution, however, the algorithm is greatly slowed down because of an additional acception/rejection step at each iteration.

Original: Taking a smaller h also slows down the algorithm but we keep some control on its time of execution.
Revised: Taking a smaller h also slows down the algorithm but we keep some control on its time of execution.

Original: 3.2. Simulation studies
Revised: 3.2. Simulation studies

Original: In order to access the behaviour of our algorithms, we ﬁrst conducted a series of experiments with simulated data:
Revised: In order to access the behaviour of our algorithms, we ﬁrst conducted a series of experiments with simulated data:

Original: • Setting I: In the ﬁrst setting, a rank- r matrix M ∗ is generated as the product of two rank- r matrices,
Revised: • Setting I: In the ﬁrst setting, a rank- r matrix M ∗ is generated as the product of two rank- r matrices,

Original: [Equation]
Revised: [Equation]

Original: where the entries of U ∗ and V ∗ are i.i.d N (0 , 1).
Revised: where the entries of U ∗ and V ∗ are i.i.d N (0 , 1).

Original: With a missing rate υ = 20% , 50% and 80%, we observe the entries of the matrix M ∗ using a uniform sampling.
Revised: With a missing rate υ = 20% , 50% and 80%, we observe the entries of the matrix M ∗ using a uniform sampling.

Original: Then, this sampled set is corrupted by noise as in (1), where E i are i.i.d N (0 , 1).
Revised: Then, this sampled set is corrupted by noise as in (1), where E i are i.i d N (0 , 1).

Original: The dimensions are alternated by ﬁxing m = 100 and varying p = 100 and 500.
Revised: The dimensions are alternated by ﬁxing m = 100 and varying p = 100 and 500.

Original: The rank r is varied between r = 2 and r = 5.
Revised: The rank r is varied between r = 2 and r = 5.

Original: • Setting II: The second series of simulations is similar to the ﬁrst one, except that the matrix M ∗ is no longer rank- r , but it can be well approximated by a rank- r matrix:
Revised: • Setting II: The second series of simulations is similar to the ﬁrst one, except that the matrix M ∗ is no longer rank- r , but it can be well approximated by a rank- r matrix:

Original: [Equation]
Revised: [Equation]

Original: where the entries of A and B are i.i.d N (0 , 1).
Revised: where the entries of A and B are i.i.d N (0 , 1).

Original: We compare our method with the state-of-the-art method in Bayesian matrix completion from Alquier et al.
Revised: We compare our method with the state-of-the-art method in Bayesian matrix completion from Alquier et al.

Original: (2014); Alquier and Ridgway (2020) which employed a Gibbs sampler algorithm.
Revised: (2014); Alquier and Ridgway (2020) which employed a Gibbs sampler algorithm.

Original: This method is denoted by ‘Gibbs’.
Revised: This method is denoted by ‘Gibbs’.

Original: For each setting, we simulate 50 data sets (simulation replicates).
Revised: For each setting, we simulate 50 data sets (simulation replicates).

Original: Then, we report the average and the standard deviation for a measure of error of each method over the replicates.
Revised: Then, we report the average and the standard deviation for a measure of error of each method over the replicates.

Original: The performance of a method (say (cid:99) M ) is measured by the mean squared error (MSE) per entry MSE :
Revised: The performance of a method (say (cid:99) M ) is measured by the mean squared error (MSE) per entry MSE :

Original: = (cid:107) (cid:99) M − M ∗ (cid:107) 2 F / ( mp ) and the normalized mean square error (NMSE)
Revised: = (cid:107) (cid:99) M − M ∗ (cid:107) 2 F / ( mp ) and the normalized mean square error (NMSE)

Original: NMSE := (cid:107) (cid:99) M − M ∗ (cid:107) 2 F / (cid:107) M ∗ (cid:107) 2 F .
Revised: NMSE := (cid:107) (cid:99) M − M ∗ (cid:107) 2 F / (cid:107) M ∗ (cid:107) 2 F .

Original: We also measure the prediction error by using Pred :=
Revised: We also measure the prediction error by using Pred :=

Original: (cid:107)P ¯Ω ( (cid:99) M − M ∗ ) (cid:107) 2 F / ( mp − n ) , where ¯Ω is the set of un-observed entries.
Revised: (cid:107)P ¯Ω ( (cid:99) M − M ∗ ) (cid:107) 2 F / ( mp − n ) , where ¯Ω is the set of un-observed entries.

Original: The results are reported in Table 1 and 2.
Revised: The results are reported in Table 1 and 2.

Original: The choice of the step-size parameters is set as h = 1 / ( pm 400) which is selected such that the acceptance rate of MALA is approximate 0.5.
Revised: The choice of the step-size parameters is set as h = 1 / ( pm 400) which is selected such that the acceptance rate of MALA is approximate 0.5.

Original: We ﬁxed τ = 1 in all settings.
Revised: We ﬁxed τ = 1 in all settings.

Original: The LMC, MALA and Gibbs sampler are used with tuning parameter λ
Revised: The LMC, MALA and Gibbs sampler are used with tuning parameter λ

Original: = 1 / (4 σ 2 ).
Revised: = 1 / (4 σ 2 ).

Original: In this simulation, LMC and MALA are initialized by using the output from the Gibbs sampler (after 50 iterations run).
Revised: In this simulation, LMC and MALA are initialized by using the output from the Gibbs sampler (after 50 iterations run).

Original: The parameters for the prior of the ‘Gibbs’ method are K = 10 , a = 1 , b = 1 / 100.
Revised: The parameters for the prior of the ‘Gibbs’ method are K = 10 , a = 1 , b = 1 / 100.

Original: These algorithms are run with T =iterations
Revised: These algorithms are run with T =iterations

Original: and we take the ﬁrst 100 steps as burn-in.
Revised: and we take the ﬁrst 100 steps as burn-in.

Original: Resuts from our simulations in Table 1 and 2 show that our proposed algortihms (LMC and MALA) perform quite similar to the Gibbs sampler.
Revised: Resuts from our simulations in Table 1 and 2 show that our proposed algortihms (LMC and MALA) perform quite similar to the Gibbs sampler.

Original: However, it is shown that LMC and MALA are much more faster than the Gibbs sampler, see Figure 1.
Revised: However, it is shown that LMC and MALA are much more faster than the Gibbs sampler, see Figure 1.

Original: More speciﬁcally, we compare the running time of the LMC, MALA against the Gibbs sampler with k = min( m, p ) and k = min( m, p ) / 2, in which we ﬁxed m = 100 , r = 2 , υ = 20% and p is varied by 50 , 100 , 200 , 500.
Revised: More speciﬁcally, we compare the running time of the LMC, MALA against the Gibbs sampler with k = min( m, p ) and k = min( m, p ) / 2, in which we ﬁxed m = 100 , r = 2 , υ = 20% and p is varied by 50 , 100 , 200 , 500.

Original: 3.3. Real application to Image inpainting
Revised: 3.3. Real application to Image inpainting

Original: We further applied our proposed algorithms on image inpainting to evaluate their performance on real data.
Revised: We further applied our proposed algorithms on image inpainting to evaluate their performance on real data.

Original: The aim of image inpainting is to complete an image with missing pixels.
Revised: The aim of image inpainting is to complete an image with missing pixels.

Original: Here, we applied our methods on the well-known benchmark image Lena Gonzalez and Woods (2007) which is of size 256 × 256.
Revised: Here, we applied our methods on the well-known benchmark image Lena Gonzalez and Woods (2007) which is of size 256 × 256.

Original: This data has been used before in the context of matrix completion in Yang et al.
Revised: This data has been used before in the context of matrix completion in Yang et al.

Original: The LMC and MALA are initialized from an output from the SoftImpute method in Mazumder et al.
Revised: The LMC and MALA are initialized from an output from the SoftImpute method in Mazumder et al.

Original: (2010) which is available in You (2020); Hastie and Mazumder (2015).
Revised: (2010) which is available in You (2020); Hastie and Mazumder (2015).

Original: We consider the cases where υ = 20% and υ = 50% of pixels are missing uniformly at random.
Revised: We consider the cases where υ = 20% and υ = 50% of pixels are missing uniformly at random.

Original: The experiments are repeated 30 times and the average of the considered errors is reported.
Revised: The experiments are repeated 30 times and the average of the considered errors is reported.

Original: The original Lena 256 × 256 image with missing pixels and these images recovered by respective algorithms and the estimation errors results are given in Table 3.
Revised: The original Lena 256 × 256 image with missing pixels and these images recovered by respective algorithms and the estimation errors results are given in Table 3.

Original: From the results in Table 3 and Figure 2, we see that our proposed methods outperform the method based on Gibbs sampler.
Revised: From the results in Table 3 and Figure 2, we see that our proposed methods outperform the method based on Gibbs sampler.

Original: This can be explained as that the Gibbs sampler algorithm based on Gaussian low-rank factorizaton prior where all the entries of the image’s matrix are positive and thus the Gaussian low-rank factorizaton prior may not be an appropriate prior for image inpainting.
Revised: This can be explained as that the Gibbs sampler algorithm based on Gaussian low-rank factorizaton prior where all the entries of the image’s matrix are positive and thus the Gaussian low-rank factorizaton prior may not be an appropriate prior for image inpainting.

Original: On the other hand, our approach indicates that the spectral scaled student prior can capture the (apprioximate) low-rank structure and the smoothness of the image data matrix and as a consequence it helps obtain a better visual quality image, see Figure 2.
Revised: On the other hand, our approach indicates that the spectral scaled student prior can capture the (apprioximate) low-rank structure and the smoothness of the image data matrix and as a consequence it helps obtain a better visual quality image, see Figure 2.

Original: 4. Conclusion
Revised: 4. Conclusion

Original: In this paper, we have studied the problem of Bayesian matrix completion by using a spectral scaled Student prior to promoting the low-rank structure of the underlying matrix.
Revised: In this paper, we have studied the problem of Bayesian matrix completion by using a spectral scaled Student prior to promoting the low-rank structure of the underlying matrix.

Original: We have provided a thorough theoretical evaluation for our Bayesian estimator under both model misspeciﬁcation and general sampling distribution.
Revised: We have provided a thorough theoretical evaluation for our Bayesian estimator under both model misspeciﬁcation and general sampling distribution.

Original: We have also provided eﬃcient gradientbased sampling algorithms for our estimator by using Langevin Monte Carlo approach.
Revised: We have also provided eﬃcient gradientbased sampling algorithms for our estimator by using Langevin Monte Carlo approach.

Original: These attractive features of our inference strategy are demonstrated through numerical simulations and real application to image inpainting.
Revised: These attractive features of our inference strategy are demonstrated through numerical simulations and real application to image inpainting.

Original: Acknowledgements
Revised: Acknowledgements

Original: TTM is supported by the Norwegian Research Council grant number 309960 through the
Revised: TTM is supported by the Norwegian Research Council grant number 309960 through the

Original: Centre for Geophysical Forecasting at NTNU.
Revised: Centre for Geophysical Forecasting at NTNU.

Original: I would like to warmly thank the anonymous referees for their useful comments on this work.
Revised: I would like to warmly thank the anonymous referees for their useful comments on this work

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Appendix A. Appendix A.1. Brief review on futher related works
Revised: Appendix A. Appendix A.1. Brief review on futher related works

Original: It is worth noting that most studies in matrix completion consider that the sampling distribution is uniform, see Alquier et al.
Revised: It is worth noting that most studies in matrix completion consider that the sampling distribution is uniform, see Alquier et al.

Original: (2014); Cand`es and Plan (2010); Cand`es and Recht (2009); Cand`es and Tao (2010); Koltchinskii et al.
Revised: (2014); Cand`es and Plan (2010); Cand`es and Recht (2009); Cand`es and Tao (2010); Koltchinskii et al.

Original: (2011); Alquier and Ridgway (2020)
Revised: (2011); Alquier and Ridgway (2020)

Original: ; Lim and Teh (2007) among others.
Revised: ; Lim and Teh (2007) among others.

Original: However, in practice the observed entries are not assured to be uniformly distributed: for example, some items are more popular and as a consequence receive much more ratings than others.
Revised: However, in practice the observed entries are not assured to be uniformly distributed: for example, some items are more popular and as a consequence receive much more ratings than others.

Original: More importantly, it is noted that the sampling distribution is usually not known in practice.
Revised: More importantly, it is noted that the sampling distribution is usually not known in practice.

Original: Several studies have been performed for general sampling schemes rather than uniform distribution, see e.g. Foygel et al.
Revised: Several studies have been performed for general sampling schemes rather than uniform distribution, see e.g. Foygel et al.

Original: (2011); Klopp (2014); Negahban and Wainwright (2012).
Revised: (2011); Klopp (2014); Negahban and Wainwright (2012).

Original: Moreover, the paper Mai and Alquier (2015) is the ﬁrst line of works that studied (Bayesian) matrix completion under a general sampling model without any restriction.
Revised: Moreover, the paper Mai and Alquier (2015) is the ﬁrst line of works that studied (Bayesian) matrix completion under a general sampling model without any restriction.

Original: The spectral scaled Student prior has been considered before in the context of image denoising Dalalyan (2020) and in reduced rank regression Mai (2021a).
Revised: The spectral scaled Student prior has been considered before in the context of image denoising Dalalyan (2020) and in reduced rank regression Mai (2021a).

Original: Moreover, it is shown that if a random matrix M has the density distribution as in (3), then the random vectors
Revised: Moreover, it is shown that if a random matrix M has the density distribution as in (3), then the random vectors

Original: M i are all drawn from the p -variate scaled Student distribtuion ( τ/ √ 3) t 3 ,p , (Dalalyan, 2020,
Revised: M i are all drawn from the p -variate scaled Student distribtuion ( τ/ √ 3) t 3 ,p , (Dalalyan, 2020,

Original: Lemma 1).
Revised: Lemma 1).

Original: On the other hand, this prior can be seen as the marginal distribution of the
Revised: On the other hand, this prior can be seen as the marginal distribution of the

Original: Gaussian-inverse Wishart prior that is explored in Yang et al.
Revised: Gaussian-inverse Wishart prior that is explored in Yang et al.

Original: (2018) in the context of matrix completion where the precision matrix is integrated out.
Revised: (2018) in the context of matrix completion where the precision matrix is integrated out.

Original: Therefore, in some ways, our work provide theoretical assessment for that paper.
Revised: Therefore, in some ways, our work provide theoretical assessment for that paper.

Original: However, we note that the paper
Revised: However, we note that the paper

Original: Yang et al.
Revised: Yang et al.

Original: (2018) consider an opimization approach to Bayesian matrix completion using variational inference while our work propose an eﬃcient sampling approach.
Revised: (2018) consider an opimization approach to Bayesian matrix completion using variational inference while our work propose an eﬃcient sampling approach.

Original: A.2. Short review on low-rank factorization priors
Revised: A.2. Short review on low-rank factorization priors

Original: Up to our knowledge, the paper Geweke (1996) was the ﬁrst work that introduced a lowrank factorization prior in a context of low-rank matrix estimation.
Revised: Up to our knowledge, the paper Geweke (1996) was the ﬁrst work that introduced a lowrank factorization prior in a context of low-rank matrix estimation.

Original: The idea is to express the matrix M as M =
Revised: The idea is to express the matrix M as M =

Original: U m × k V (cid:62) p × k with k ≤ min( p, m ) and then the prior is diﬁned on U and V rather than on M as
Revised: U m × k V (cid:62) p × k with k ≤ min( p, m ) and then the prior is diﬁned on U and V rather than on M as

Original: [Equation]
Revised: [Equation]

Original: for some τ > 0.
Revised: for some τ > 0.

Original: This approach has been used in matrix completion for the ﬁrst time in
Revised: This approach has been used in matrix completion for the ﬁrst time in

Original: Lim and Teh (2007).
Revised: Lim and Teh (2007).

Original: However, this approach is faced with the problem of choosing k .
Revised: However, this approach is faced with the problem of choosing k .

Original: One could perform model selection for any possible k as done in Kleibergen and Paap (2002);
Revised: One could perform model selection for any possible k as done in Kleibergen and Paap (2002);

Original: Corander and Villani (2004) but the computation may be expensive for large data set.
Revised: Corander and Villani (2004) but the computation may be expensive for large data set.

Original: Recent approaches as in Babacan et al.
Revised: Recent approaches as in Babacan et al.

Original: (2012) focus on ﬁxing a large k , e.g. k = min( p, m ), then sparsity-promoting priors are placed on the columns of U and V such that most columns are almost null.
Revised: (2012) focus on ﬁxing a large k , e.g. k = min( p, m ), then sparsity-promoting priors are placed on the columns of U and V such that most columns are almost null.

Original: As a consequence, the resulting matrix M = UV (cid:62) is approximately lowrank.
Revised: As a consequence, the resulting matrix M = UV (cid:62) is approximately lowrank.

Original: See Alquier (2013); Alquier et al.
Revised: See Alquier (2013); Alquier et al.

Original: (2014) for the details and dicussions on low-rank factorization priors.
Revised: (2014) for the details and dicussions on low-rank factorization priors.

Original: Using low-rank factorization priors, most authors use the Gibbs sampler to simulate from the posterior as the conditional posterior distributions can be derived explitcitely, e.g. in
Revised: Using low-rank factorization priors, most authors use the Gibbs sampler to simulate from the posterior as the conditional posterior distributions can be derived explitcitely, e.g. in

Original: Salakhutdinov and Mnih (2008).
Revised: Salakhutdinov and Mnih (2008).

Original: However, it is noted that these Gibbs sampling algorithms update the factor matrices in a row-by-row manner and invole a number of matrix inverse or singular value decomposition operations at each iteration.
Revised: However, it is noted that these Gibbs sampling algorithms update the factor matrices in a row-by-row manner and invole a number of matrix inverse or singular value decomposition operations at each iteration.

Original: This is expensive and slow down the algorithm for large data set signiﬁcantly.
Revised: This is expensive and slow down the algorithm for large data set signiﬁcantly.

Original: A.3. Theoretical result in special cases: uniformly sampling and exact low-rank
Revised: A.3. Theoretical result in special cases: uniformly sampling and exact low-rank

Original: Firstly, when the sampling distribution is uniform, we obtain the following result for the
Revised: Firstly, when the sampling distribution is uniform, we obtain the following result for the

Original: Frobenius norm directly from Theorem 1.
Revised: Frobenius norm directly from Theorem 1.

Original: Corollary 2 Let Assumption 1 and 2 be satisﬁed and take λ = λ ∗ := n/C .
Revised: Corollary 2 Let Assumption 1 and 2 be satisﬁed and take λ = λ ∗ := n/C .

Original: Then, with probability at least 1 − (cid:15), (cid:15) ∈ (0 , 1) , one has for any matrix ¯ M with its rank at most r that
Revised: Then, with probability at least 1 − (cid:15), (cid:15) ∈ (0 , 1) , one has for any matrix ¯ M with its rank at most r that

Original: [Equation]
Revised: [Equation]

Original: where δ is in (0 , 1) and C > C is a universal constants that depends on σ 2 , L, ξ only.
Revised: where δ is in (0 , 1) and C > C is a universal constants that depends on σ 2 , L, ξ only.

Original: Secondly, as soon as the true matrix M ∗ is low-rank where its rank is atmost r , we can pick ¯ M = M ∗ and obtain the following results.
Revised: Secondly, as soon as the true matrix M ∗ is low-rank where its rank is atmost r , we can pick ¯ M = M ∗ and obtain the following results.

Original: Corollary 3 Let Assumption 1 and 2 be satisﬁed and take λ = λ ∗ := n/C .
Revised: Corollary 3 Let Assumption 1 and 2 be satisﬁed and take λ = λ ∗ := n/C .

Original: Then, with probability at least 1 − (cid:15), (cid:15) ∈ (0 , 1) , one has
Revised: Then, with probability at least 1 − (cid:15), (cid:15) ∈ (0 , 1) , one has

Original: [Equation]
Revised: [Equation]

Original: where δ is in (0 , 1) and C > C is a universal constants that depends on σ 2 , L, ξ only.
Revised: where δ is in (0 , 1) and C > C is a universal constants that depends on σ 2 , L, ξ only.

Original: A.4. A Metropolis-adjusted Langevin algorithm
Revised: A.4. A Metropolis-adjusted Langevin algorithm

Original: Here, we propose a Metropolis-Hasting correction to the LMC.
Revised: Here, we propose a Metropolis-Hasting correction to the LMC.

Original: This approach guarantees the convergence to the posterior and it also provides a useful way for choosing h .
Revised: This approach guarantees the convergence to the posterior and it also provides a useful way for choosing h .

Original: More precisely, we consider the update rule in (5) as a proposal for a new state,
Revised: More precisely, we consider the update rule in (5) as a proposal for a new state,

Original: [Equation]
Revised: [Equation]

Original: Note that ˜ M k +1 is normally distributed with mean M k − h ∇ log ˆ ρ λ ( M k ) and the covariance matrices equal to 2 h times the identity matrices.
Revised: Note that ˜ M k +1 is normally distributed with mean M k − h ∇ log ˆ ρ λ ( M k ) and the covariance matrices equal to 2 h times the identity matrices.

Original: This proposal is accepted or rejected according to the Metropolis-Hastings algorithm that the proposal is accepted with probabiliy:
Revised: This proposal is accepted or rejected according to the Metropolis-Hastings algorithm that the proposal is accepted with probabiliy:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: is the transition probability density from x to x (cid:48) .
Revised: is the transition probability density from x to x (cid:48) .

Original: Compared to random-walk Metropolis–Hastings, the advantage of MALA is that it usually proposes moves into regions of higher probability, which are then more likely to be accepted.
Revised: Compared to random-walk Metropolis–Hastings, the advantage of MALA is that it usually proposes moves into regions of higher probability, which are then more likely to be accepted.

Original: We note that the step-size h is chosen such that the acceptance rate is approximate
Revised: We note that the step-size h is chosen such that the acceptance rate is approximate

Original: 0 .following Roberts and Rosenthal (1998).
Revised: 0 .following Roberts and Rosenthal (1998).

Original: See Section 3.2 for some choices in special cases in our simulations.
Revised: See Section 3.2 for some choices in special cases in our simulations.

Original: It is also noted that an immediate application of the LMC algorithm in (5) needs to calculate a p × p matrix inversion at each iteration.
Revised: It is also noted that an immediate application of the LMC algorithm in (5) needs to calculate a p × p matrix inversion at each iteration.

Original: This might be expensive and can slow down signiﬁcantly the algorithm.
Revised: This might be expensive and can slow down signiﬁcantly the algorithm.

Original: Therefore, in the case of very big data or huge p , we could replace this matrix inversion by its accurately approximation through a convex optimization.
Revised: Therefore, in the case of very big data or huge p , we could replace this matrix inversion by its accurately approximation through a convex optimization.

Original: More precisely, it can be easily veriﬁed that the matrix B = ( τ 2 I m + MM (cid:124) ) − 1 M is the solution to the following convex optimization problem
Revised: More precisely, it can be easily veriﬁed that the matrix B = ( τ 2 I m + MM (cid:124) ) − 1 M is the solution to the following convex optimization problem

Original: [Equation]
Revised: [Equation]

Original: The solution of this optimization problem can be conveniently obtained by using the package ’glmnet’ Friedman et al.
Revised: The solution of this optimization problem can be conveniently obtained by using the package ’glmnet’ Friedman et al.

Original: (2010) (with family option ’mgaussian’).
Revised: (2010) (with family option ’mgaussian’).

Original: This avoid to perform matrix inversion or other costly calculation.
Revised: This avoid to perform matrix inversion or other costly calculation.

Original: However, we note here that the LMC algorithm is being used with approximate gradient evaluation, theoretical assessment of this approach can be found in Dalalyan and Karagulyan (2019).
Revised: However, we note here that the LMC algorithm is being used with approximate gradient evaluation, theoretical assessment of this approach can be found in Dalalyan and Karagulyan (2019).

Original: A.5. Proofs
Revised: A.5. Proofs

Original: The main argument of the proof is based on the so-called PAC-Bayesian bounds which were introduced in Shawe-Taylor and Williamson (1997); McAllester (1998).
Revised: The main argument of the proof is based on the so-called PAC-Bayesian bounds which were introduced in Shawe-Taylor and Williamson (1997); McAllester (1998).

Original: However, our results are derived by following the Catoni’s works Catoni (2007) where the author shown how to derive powerful oracle inequalities from PAC-Bayesian bounds.
Revised: However, our results are derived by following the Catoni’s works Catoni (2007) where the author shown how to derive powerful oracle inequalities from PAC-Bayesian bounds.

Original: Since then this technique has been use many times to obtain oracle inequalities in many diﬀerent problems
Revised: Since then this technique has been use many times to obtain oracle inequalities in many diﬀerent problems

Original: Dalalyan and Tsybakov (2008); Mai and Alquier (2015); Alquier and Lounici (2011); Guedj and Robbiano (2018); Ridgway et al.
Revised: Dalalyan and Tsybakov (2008); Mai and Alquier (2015); Alquier and Lounici (2011); Guedj and Robbiano (2018); Ridgway et al.

Original: ; Cottet and Alquier (2018); Mai and Alquier (2017); Alquier et al.
Revised: ; Cottet and Alquier (2018); Mai and Alquier (2017); Alquier et al.

Original: For the sake of simplicity, let us deﬁne
Revised: For the sake of simplicity, let us deﬁne

Original: [Equation]
Revised: [Equation]

Original: In order to understand what follows, keep in mind that our optimal estimator comes with λ = λ ∗ = nC , so α and β are of order n .
Revised: In order to understand what follows, keep in mind that our optimal estimator comes with λ = λ ∗ = nC , so α and β are of order n .

Original: First, we state a general PAC-Bayesian inequality for matrix completion from Mai and
Revised: First, we state a general PAC-Bayesian inequality for matrix completion from Mai and

Original: Alquier (2015), in the style of Catoni (2004); Dalalyan and Tsybakov (2008).
Revised: Alquier (2015), in the style of Catoni (2004); Dalalyan and Tsybakov (2008).

Original: Theorem 4 Let Assumption 1 and 2 be satisﬁed.
Revised: Theorem 4 Let Assumption 1 and 2 be satisﬁed.

Original: Then, for any (cid:15) ∈ (0 , 1) , with probability at least 1 − (cid:15) , one has
Revised: Then, for any (cid:15) ∈ (0 , 1) , with probability at least 1 − (cid:15) , one has

Original: [Equation]
Revised: [Equation]

Original: where M 1+ ( M ) is the set of all positive probability measures over M .
Revised: where M 1+ ( M ) is the set of all positive probability measures over M .

Original: This theorem is the result from the Step 1 in the proof of Theorem 1 in Mai and Alquier (2015).
Revised: This theorem is the result from the Step 1 in the proof of Theorem 1 in Mai and Alquier (2015).

Original: Now, we are ready to present the proof of Theorem 1.
Revised: Now, we are ready to present the proof of Theorem 1.

Original: Proof [ of Theorem 1 ] Let’s ﬁx an arbitrary matrix ¯ M .
Revised: Proof [ of Theorem 1 ] Let’s ﬁx an arbitrary matrix ¯ M .

Original: Let ¯ ρ be the distribution obtained from the prior π by a translation
Revised: Let ¯ ρ be the distribution obtained from the prior π by a translation

Original: [Equation]
Revised: [Equation]

Original: Now, we apply Theorem 4 and upper bound the right hand side of (8) evaluated at the distribution ¯ ρ .
Revised: Now, we apply Theorem 4 and upper bound the right hand side of (8) evaluated at the distribution ¯ ρ .

Original: We obtain
Revised: We obtain

Original: [Equation]
Revised: [Equation]

Original: Firstly, we deal with the integral term that
Revised: Firstly, we deal with the integral term that

Original: [Equation]
Revised: [Equation]

Original: By using the translation invariance of the Lebesgue measure and the fact that (cid:82) Mπ ( M ) dM = 0, we obtain
Revised: By using the translation invariance of the Lebesgue measure and the fact that (cid:82) Mπ ( M ) dM = 0, we obtain

Original: [Equation]
Revised: [Equation]

Original: For the last integral, we have that
Revised: For the last integral, we have that

Original: [Equation]
Revised: [Equation]

Original: using the Lemma 1 in Dalalyan (2020).
Revised: using the Lemma 1 in Dalalyan (2020).

Original: Secondly, we deal with the Kullback-Leibler term.
Revised: Secondly, we deal with the Kullback-Leibler term.

Original: We have that
Revised: We have that

Original: KL
Revised: KL

Original: [Equation]
Revised: [Equation]

Original: that is followed by using Lemma 2 in Dalalyan (2020)
Revised: that is followed by using Lemma 2 in Dalalyan (2020)

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: we have for any matrix ¯ M with rank( ¯ M ) =
Revised: we have for any matrix ¯ M with rank( ¯ M ) =

Original: r that
Revised: r that

Original: [Equation]
Revised: [Equation]

Original: Taking now λ
Revised: Taking now λ

Original: = λ ∗ := n/C with C > C 2 +3 C 1 / 2, we get that βα ≤ 1+ δ with 0 < δ < 1, and that 4 α ≤ 6 C/n .
Revised: = λ ∗ := n/C with C > C 2 +3 C 1 / 2, we get that βα ≤ 1+ δ with 0 < δ < 1, and that 4 α ≤ 6 C/n .

Original: Thus, the inequality (9) yeilds
Revised: Thus, the inequality (9) yeilds

Original: [Equation]
Revised: [Equation]

Original: Now, by taking τ 2 = r ( m + p ) / ( mpn ), we obtain that
Revised: Now, by taking τ 2 = r ( m + p ) / ( mpn ), we obtain that

Original: [Equation]
Revised: [Equation]

Original: where C > 6 C is a universal constants that depends on σ 2 , L, ξ only.
Revised: where C > 6 C is a universal constants that depends on σ 2 , L, ξ only.

Original: This completes the proof of the theorem.
Revised: This completes the proof of the theorem.