Title: Fast exact recovery of noisy matrix from few entries: the infinity norm approach
Venue: NeurIPS 2025
Accepted: yes
Keywords: Matrix Completion

Matrix Perturbation

Recommendation System

Random Matrices
Abstract: The matrix recovery (completion) problem, a central problem in data science, involves recovering a matrix $A$ from a relatively small random set of entries. While such a task is generally impossible, it has been shown that one can recover $A$ exactly in polynomial time, with high probability, under three basic and necessary assumptions: (1) the rank of $A$ is very small compared to its dimensions (low rank), (2) $A$ has delocalized singular vectors (incoherence), and (3) the sample size is sufficiently large. Various algorithms address this task, including convex optimization by Candes, Recht, and Tao (2009, 2010), alternating projection by Hardt and Wooters (2014), and low-rank approximation with gradient descent by Keshavan, Montanari, and Oh (2009, 2010). In applications, Candes and Plan (2009) noted that it is more realistic to assume noisy observations. In such cases, the above approaches provide approximate recovery with small root mean square error, which is difficult to convert into exact recovery. Recently, results by Abbe et al. (2017) and Bhardwaj et al. (2023) on approximation in the infinity norm showed that one can recover $A$ even in the noisy case, provided $A$ has bounded precision. However, beyond the three basic assumptions, they either required that the condition number of $A$ be small (2017) or that the gaps between consecutive singular values be large (2023). These additional assumptions conflict, with one requiring singular values to be close together and the other suggesting they should be far apart. It is thus natural to conjecture that neither is necessary. In this paper, we demonstrate that this is indeed the case. We propose a simple algorithm for exact recovery of noisy data, relying solely on the three basic assumptions. The core step of the algorithm is a straightforward truncated singular value decomposition, which is highly efficient. To analyze the algorithm, we prove a new infinity norm version of the classical Davis-Kahan perturbation theorem, improving an earlier result in (2023). Our proof employs a combinatorial contour integration argument and is entirely distinct from all previous approaches.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript considers the problem of exact recovery of a low rank matrix from noisy observations, under an additional assumption that\nthe underlying matrix entries have fixed precision (i.e. they are all integer multiples of some epsilon). \n\nThe authors study a simple SVD-based algorithm followed by rounding. They prove that, under certain conditions, their method can exactly recover the target matrix even in the presence of additive noise. The analysis combines tools from random matrix theory and from matrix perturbation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \nThe use of sophisticated tools from random matrix theory enables the authors to establish recovery guarantees that surpass existing \nresults in similar settings.\nThe paper is relatively clearly written and accessible.\n\nWeaknesses: \nThe problem considered in this manuscript is uncommon and non-standard in the matrix completion literature. \nIn terms of originality, the method considered is (up to the rounding step and how to choose the number of components) standard as an initialization in the matrix completion literature, and so there is limited novelty here. \nThese issues raise major concerns regarding the importance of the problems studied, the significance of their findings for the community and their practical relevance. \n\nWhile the authors motivate their work by practical applications, the manuscript lacks an empirical evaluation to support the theoretical claims or illustrate the method's performance in practice.\n\nFit to NeurIPS: as the authors write, one of their theorems can be seen as an infinity norm variant of the classical Davis-Kahan Theorem. Perhaps a more suitable venue for this work would be a journal focused on matrix analysis. \n\nAdditional comments: \n- Problem formulation: the authors assume the underlying matrix has finite precision while observations contain noise. It would strengthen the manuscript if the authors were to provide concrete examples where this setting holds. To the best of my knowledge, the more common setting is one in which the observations are quantized (e.g, have finite precision) with the underlying matrix assumed low rank but not with bounded precision. Examples include movie ratings (e.g. observed values are 1-5) with various stochastic models relating how observed ratings are related to the underlying matrix entries, and the 1-bit matrix completion problem, where observed entries are +1/-1, again where they underlying matrix is assumed low rank. \n\n- Abstract is very long. In addition, the abstract cites various papers, which is rather uncommon.\n\n- The authors write that the paper by Abbe et al [9] showed that one can recover A even in the noisy case, provided A has bounded precision [p1 L15], and a similar statement on p3 L101. This seems to imply that [9] also considered the precise problem studied in this manuscript of exact recovery in a matrix completion problem. Unless I missed something, I find this rather misleading. Ref. [9] discussed exact recovery for stochastic block models, which is a different problem. To the best of my understanding, Section 3.3 of [9], titled matrix completion with noisy entries does not assume bounded precision. \n\n- p1 L32 to be precise, the task is to recover A given A_\\Omega *as well as* the set Omega itself. \n\n- p1 L66 the authors claim that many papers assume r is bounded while m,n\\to\\infty. I find this inaccurate for the matrix completion literature. It often holds in mathematical statistics / RMT papers. \n\n- p3 L113-end: the authors show the eigenvalue decay for the Yale face dataset. The relation between this example and the manuscript is unclear. At best, this result shows that many practical datasets do not have an exact low rank, but at best are only approximately low rank. \n\n- A small remark is that matrix completion becomes challenging when the number of observed entries is very small and yet above the information limit. In this case, the SVD of the observed entries is not sufficiently accurate, and yet there are various methods that empirically are still able to recover the matrix.\n\nQUESTIONS:\n- P4 L127 the authors write \"this not only provides a mathematically ... .but also significantly expands the range of applications (regarding real datasets). Can the authors provide convincing examples for this claim ?\n\n- Can the authors discuss how to estimate the parameters K_A and K_Z in the proposed algorithm ? \n\n- P5 L160 RMSE recovery with the same error margin. Can you please explain this sentence ?\n\nLIMITATIONS:\nThis work is theoretical and has no societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper analyzes the performance of the proposed approximate-and-round 2 algorithm for matrix completion. By leveraging contour integrals and refined bounds on bilinear forms of matrix powers, it eliminates the dependence on the condition number and singular value gaps in the recovery guarantees and establishes an infinity-norm bound between the estimated matrix and the ground truth.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n\nAs mentioned in the summary, the paper presents a novel result on the performance of the approximate-and-round 2 algorithm for matrix completion, removing the dependence on the condition number and singular value gaps in the recovery guarantees. This is a meaningful contribution that offers new theoretical insights to the community. While I was not able to check all the details of the proofs due to time constraints, the overall approach appears solid and well-motivated.\n\n**Weaknesses:**\n1. I recommend adding numerical experiments to demonstrate that the algorithm's performance does not depend on the condition number $\\kappa$ if time permits.\n2. The assumption that $A$ has a distortion level of $\\varepsilon$ feels somewhat unnatural. It effectively implies that $A=$ $\\varepsilon B$ for some integer matrix $B$, where $B$ is bounded by $K_A / \\varepsilon$ and has rank $r$. Requiring an integer matrix to be exactly low-rank can be restrictive and unnatural in practice. Moreover, exact recovery may not be necessary, as a refined entrywise error bound would be sufficient for many applications, particularly under noisy observation settings.\n\nQUESTIONS:\n1. Lines 261-272 suggest that a fine-grained bound on $\\left|e_j^{\\top} E_{\\text {sym}}^a w_i\\right|$ is needed. Such bounds are common in the literature, for example, in Section C.4 of Estimating Mixed Memberships with Sharp Eigenvector Deviations by Mao et al. (JASA 2021) and Lemmas 4 and 5 of Asymptotic Theory of Eigenvectors for Random Matrices With Diverging Spikes by Fan et al. (JASA 2022). (I am referring to the arXiv versions of these papers.)\n    If I understand correctly, the bounds in Lemmas D.3 and D.4 in the Appendix could be derived using bounds on $\\left|e_j^{\\top} E_{\\text {sym}}^a w_i\\right|$. Could you briefly discuss whether the results in these papers can be applied directly to obtain Lemmas D.3 and D.4, or clarify if additional arguments are needed?\n2. In the proof of Theorem 2.3, the signal-to-noise assumption changes from $\\sigma_1 \\geq 100 r K \\sqrt{r_{\\max } N / p}$ to $\\sigma_1 \\geq 100 r \\kappa \\sqrt{r_{\\max } N}$. It appears that you are implicitly using $\\kappa=K / \\sqrt{p}$, which conflicts with the notation where $\\kappa$ denotes the condition number of the matrix $A$. Perhaps this is a typo, and you intended to use a different symbol, such as $\\varsigma$ as introduced in Theorem B.4?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article looks into the matrix recovery/completion problem, focusing on the assumptions typically utilized to ensure exact recovery is possible. The article identifies three basic assumptions including low rank, incoherence, and sampling density as the necessary assumptions for exact recovery. It then looks into two extra spectral assumptions used in recent works on the small condition number and the large gaps between consecutive singular values, which seem to contradict with each other. It turns out that neither is in fact necessary.\nFrom this the paper establishes the main contribution (Theorem 2.3) that given a sufficiently large spectral norm of the matrix and sufficient sampling rate, the matrix can be exactly recovered with a high probability with a small error. The approximation algorithm is correspondingly based on a truncated SVD. Additionally, in Theorem 3.1, the paper introduces an upper bound on the $l_\\infty$-norm of errors in recovery, with a new approach to proof using contour integrals.\n\nSTRENGTHS AND WEAKNESSES:\nThe target problem of the article has long been of interest, including the algorithm to recovery/complete a noisy and partially observed matrix, and theoretical results on the sampling rate to achieve accurate recovery. The literature review on the seminar works and more recent works is quite extensive, including the discussion on the necessity of the basic assumptions and the extra spectral ones. The key contributions include the low-rank approximation algorithm with a proposed method to find the cutoff, and a lower bound on the sampling rate, which are quite significant.\n\nIn terms of weakness, my opinion is that I am hoping to see more direct comparison of the sampling rate between the theoretical results proposed with that of the earlier works on matrix completion for their similarity and difference. For example, the results in ''Matrix Completion With Noise'' and ''Exact Low-rank Matrix Completion via Convex Optimization''. I believe that will better highlight the significance of the results by this article.\n\nQUESTIONS:\nA couple questions:\n\n(1) The two spectral assumptions addressed by the paper in Section 1.3 are that the condition number is small, and the gaps between consecutive singular values are large. My impression is that these statements are rather hard to quantitatively verify. The example provided in the article claims that $\\kappa=10.45$ is a large condition number, with which I am not sure I can agree since typically we would call it large when it is in the range of $10^3$. Similarly, in Algorithm 2.2, the largest index $s$ is found by looking for the last 'big' drop in singular values, which is in some way similar to the second spectral statement. I wonder whether the authors can clarify on the refutation of these two assumptions.\n\n(2) The recovery algorithm uses a truncated SVD with an additional scheme to ascertain the cutoff $s$. Since truncated SVD for matrix completion has been widely applied, I am curious if the authors can provide more insight on how the thresholding rule is established.\n\nLIMITATIONS:\nNot limitations but a few minor points:\n\n(1) I wonder whether the authors have considered applying the proposed completion algorithm to a real-world application to demonstrate its performance. \n\n(2) The article denotes the fixed precision of the matrix by $\\epsilon$, however, later in the paper it is referred to as the distortion level. Although both are probably supposed to refer to the same entity, I fear this causes some confusion.\n\n(3) In Section 2,1, it is claimed the algorithm is efficient consisting a truncated SVD on a sparse matrix. I am not seeing the sparsity being properly defined so I wonder whether this point can be clarified.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper studies exact matrix completion in a setting in which the underlying matrix has rank r and  entries that are multiple of a known fixed number epsilon. The authors show that exact reconstruction is possible (in the case of fixed rank) as soon as the number of observed entries is larger than $(m+n)*log(m+n)^4$ which is nearly optimal.
The important point is that no assumption on the condition number or separation between singular values is made.

STRENGTHS AND WEAKNESSES:
Strenghts:
1. Removing the dependence on the condition number and separation between singular values is significant.
2. The proof technique is interesting.

Weaknesses:
1. Several key ideas are actually based on Tran Vu (2024)
2. The algorithm is a small variation over method studied before.
3. The assumption of discrete entries together with small rank is unrealistic. For instance the netflix ratings matrix is discrete but does not have small rank.

QUESTIONS:
I do not have questions.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-05