Title: RGNMR: A Gauss-Newton method for robust matrix completion with theoretical guarantees
Venue: NeurIPS 2025
Accepted: yes
Keywords: Robust Matrix Completion

Outlier Detection

Gauss-Newton Method

Matrix Factorization

Robust PCA

Matrix Completion
Abstract: Recovering a low rank matrix from a subset of its entries, 
some of which may be corrupted, is known as the robust matrix completion (RMC) problem.
Existing RMC methods have  several limitations: they require a relatively large number of observed entries; 
they may fail under overparametrization, when their assumed rank is higher than the correct one;
and many of them fail to recover even mildly ill-conditioned matrices.
In this paper we propose a novel RMC method, denoted $\texttt{RGNMR}$, which overcomes these limitations. 
$\texttt{RGNMR}$ is a simple factorization-based iterative algorithm, which combines a Gauss–Newton linearization with removal of entries suspected to be outliers.
On the theoretical front, we prove that under suitable assumptions, 
$\texttt{RGNMR}$ is guaranteed exact recovery of the underlying low rank matrix.
Our theoretical results improve upon the best currently known for factorization-based methods. 
On the empirical front, 
we show via several simulations
the advantages of $\texttt{RGNMR}$ over existing RMC methods, and in particular its ability to handle a small number of observed entries, overparameterization of the rank and ill-conditioned matrices.
In addition, we propose a novel scheme for  estimating the number of corrupted entries.
This scheme may be used by other RMC methods that require as input the number of corrupted entries.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of matrix completion with corrupted (and unobserved) entries. It proposes to use a Guass-Newton linearization step and outlier removal to improve the assumptions needed for accurate reconstruction. They provide theoretical guarantees for their method and show through experiments that it does well even in challenging settings.\n\nSTRENGTHS AND WEAKNESSES:\nThe problem is well-motivated and the various challenging settings identified. The theoretical guarantees are clear, transparent, and well-explained. I found the entire paper to be very clear and forthcoming.\n\nFigure 1 -- in the caption, it would be helpful to describe what is meant by failure probablity (since the paper studies the robust version, what is failure?)\n\nRemark 2.1 -- what is meant here by runtime savings? It seems that doing additional updates would increase runtime -- is it that the small increase in runtime is far outweighed by the improved convergence (and if so, perhaps this could just be rephrased). \n\nPerhaps a remark after (2) could be added to enhance some intuition -- given that the hope is that this approach allows for a lower number of non-uniformly sampled entries yet also removes outliers, it seems on first glance that it is very possible for the samples to \"miss\" some directions, making those appear as outliers. In this case, the method actually does itself more damage, but further trying to ignore those directions. Can some intuition be discussed here about this case (rather than waiting for the theory below)?\n\nQUESTIONS:\n1) Per the discussion above, could a remark after (2) be added to enhance some intuition?\n2) How sensitive is the approach to parameter selection (i.e. number of corruptions, \\mu, \\alpha...). Remark 3.2 only states this is a common assumption, but are these practical issues or not?\n3) Perhaps 2) could be addressed by some sensitivity experiments (dare I say ablation experiments) if they can't be addressed by other works?\n\nLIMITATIONS:\nThey discussed limitations in the problem approaches.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new method for robust matrix completion with strong theoretical guarantees. The algorithm can be described in the following: For robustness, a set of $k$ indices $\\Lambda$ is maintained and used to track the estimated outliers based on the largest residual entries. For matrix completion, a factorization method based on a gauss-newton linearization on the factors is used (GNMR by Zilber and Nadler 2022), where only the set of indices $(i,j) \\not \\in \\Lambda$ (or the estimated \"inliers\") are used to update the factors. \n\nFor the outlier indices, $$\\Lambda_{t+1} = \\arg \\underset{\\Lambda \\subset \\Omega, |\\Lambda| = k}{\\min} \\|\\|U_t V_{t+1}^T + U_{t+1}V_t^T - U_tV_t^T - X\\|\\|_{F(\\Omega \\setminus \\Lambda)}^2$$\n\nFor the update of the factors, \n$$(U_{t+1}, V_{t+1}) = \\arg \\underset{U,V}{\\min} \\|\\|U_t V^T + U V^T - U_tV_t^T - X\\|\\|_{F(\\Omega \\setminus \\Lambda_t)}^2$$\n\nSince the number of outliers $k$ must be estimated, the authors also propose a simple yet interesting binary search scheme with justified intuition. \n\nThe authors provide recovery guarantees for a slightly modified version of the proposed method. The first theorem establishes the fact that given a sufficiently accurate balanced initialization with bounded row norms, which are relatively standard assumptions, the modified algorithm recovers the true matrix with high probability and linear convergence rate. The recovery guarantees has small sample complexity, and holds the weakest conditions for factorization based methods. \n\nThe second theorem establishes the possibility to construct an accurate enough initialization under suitable conditions, such that the recovery guarantee's assumptions can be satisfied. This initialization's success depends on the fraction of outliers to be slightly lower than other algorithms. \n\nThe authors also provide a set of simulations comparing against other robust matrix completion methods and demonstrates the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\nThe paper provides a simple yet effective algorithm for robust matrix completion, mainly making good use of the existing results from GNMR (Zilber and Nadler 2022). Performance is good compared with other RMC methods with strong theoretical guarantees (especially sample complexity and fraction of outliers for the recovery guarantee), and experiments on a real computer vision dataset is included. The paper is presented in a concise and clear manner. \n\nWeaknesses: \n\n1. I don't see significant novelty in the work, as it mostly relies on the existing work GNMR, which was simple and had strong theoretical properties. The paper's main contribution employs a simple robustness mechanism similar to existing methods to an already powerful matrix completion algorithm. \n\n2. Including a set of experiments comparing directly with the original GNMR without the robust mechanism may help me to understand the true effectiveness of RGNMR. It is unclear to me how effective the robustness mechanism actually is. \n \n3. In applying factorization methods in general, the rank used for the factorization is always difficult to choose. Even for applications where the true rank of a certain matrix under noiseless conditions are known, it may still be difficult, as the best rank to use for matrix completion may be higher than the true rank due to noise and corruption. The authors didn't fully discuss the affect of rank in the factorization and \"did not consider the case of overparametrization\", and this may limit the practicality of the proposed method. \n\n4. The initialization procedure in theorem 3.2 improves the sample complexity of existing works, but requires a lower fraction of outliers. It is possible that the initialization procedure doesn't provide a sufficiently accurate initialization when data is highly corrupted. \n\nIn equation (1) between lines 106 and 107, a square is missing for the norm of the residual.\n\nQUESTIONS:\n1. Can the authors provide some insight as to how to choose the rank for the factorization in practice? If possible, can a set of small experiments be added to show the effects of overparametrization? I understand that the time is short, but even a small set of experiments may help. \n\n2. How different are the solutions of RGNMR-BS and the normal variant of RGNMR? Under what conditions may they vary much and how much does it affect the runtime? Are there cases where BS takes a long time to find the correct k and greatly affects the quality of the solution and runtime? \n\n3. Including a set of experiments comparing RGNMR directly with the original GNMR without the robust mechanism may be helpful to understand and validate the effectiveness of the simple robust mechanism. \n\n4. If I recall correctly, GNMR had different variants according to some change of variables. In the experiments for RGNMR, was there a specific variant of GNMR that worked better than the others?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the classic problem of low-rank matrix completion in presence of outliers. It is noted that previous robust matrix completion (RMC) algorithms fail when the number of observed entries in the measurement matrix is relatively small, the low-rank matrices ($U$ and $V$) are overparameterized or the condition number is small. To mitigate these shortcomings, the authors propose an alternation-based algorithm whereby the outlier-corrupted elements are estimated in each step and subsequently excluded from estimating the decomposed matrices.  Since this involves estimating the number of outlier-corrupted entries ($k$) in each iteration, the authors test two versions, i) the oracle case whereby $k$ is known (RGNMR) and ii) the binary search scheme implemented to find an upper bound of $k$.\nThe authors present a theorem (Theorem 3.1) to provide recovery guarantee under certain assumptions such as small enough fraction of outliers and large enough number of observed entries. The theorem is supplemented by numerical simulation results using largely synthetic data (except for the background extraction example) demonstrating improved performance compared to competitors under various scenarios changing oversampling ratio, fraction of outliers, overparameterized rank, etc.\n\nSTRENGTHS AND WEAKNESSES:\n[ Strengths ]\n\n- Clarity: the paper is mostly comprehensible despite use of heavy math notations.\n- Results: experimental results largely support the claims made in this paper and is mostly superior compared to the competing algorithms.\n- Reproducibility: Supplementary code is provided for review.\n\n[ Major weaknesses ]\n\n- Limited motivation: While it is true that low-rank matrix factorization (LRMF) still serves as a baseline model for recommender systems, it is subsequently being more replaced by deep models that are able to capture nonlinear relationships between the user traits and item traits (e.g., Li et al., A Survey on Deep Neural Networks in Collaborative Filtering Recommendation Systems, arxiv). The paper would benefit from a more concrete motivation of why robust low-rank matrix completion is still an important problem (cited works in l.20 are already 4 to 9 years old).\n\n- Novelty of the proposed method: In terms of the algorithmic structure, RGNMR is largely based on the work of GNMR (Zilber and Nadler, 2022) for estimating decomposed matrices. The only major difference is in re-estimating the corrupted entries in each iteration of the algorithm.\n\n- Practicality of the experimental settings: One area where LRMF can still accel compared to deep networks is where the observation matrix ($L$) itself is very large but the number of observed entries is very small (e.g. Bennett et al., The Netflix Prize, Proceedings of the KDD Cup Workshop 2007). Nevertheless, evaluations in this paper is limited to largely synthetic observation matrices of relatively small sizes (3200 x 400). I am not entirely sure if such experimental setting would support the usefulness of this algorithm for real-world applications, e.g. in (large-scale) recommender systems.\nAnother major weakness in my opinion is that, as demonstrated in Fig. 7, the performance of the (non-oracle) RGNMR-BS is not stronger than others even when the standard deviation of the additive noise is sufficiently small (1e-6 or greater). In practice, additive noise and gross outliers usually occur simultaneously and thus should be the default testing ground in my opinion. \n\n- Runtime and/or algorithmic complexity is an essential component for practical consideration but is not compared against other algorithms.\n\n- Missing connections to other literature: The approach of subsequently improving the set of inliers is similar to the concept of  graduated non-convexity (e.g., Sidhartha et al., Adaptive Annealing for Robust Averaging, ECCV 2024) but is not discussed in the related work.\n\n[ Minor weakness ]\n\n- It seems the positions of figures are slightly unsynchronized compared to the location of paper contents. Perhaps Figures 1 to 3 could be moved to the latter half of the paper? Also, the paper in its current form is not self-contained within the 9 pages limit --- many essential experimental results (e.g. Fig. 7) are in the supplementary material.\n\nQUESTIONS:\n1. I wonder if Figures 1 to 3 are drawn assuming some degree of additive noise in each observed entries (like in Fig. 7) or only consider the effect of gross outliers. (In practical viewpoint, assuming no or very little additive noise can be very rare.)\n\n2. I wonder if the authors have tried out various challenging different visibility patterns besides uniform sampling and power-law sampling (Meka et al., 2009) techniques. For instance in structure-from-motion in computer vision (Tomasi and Kanade, IJCV 1992), the visibility pattern of the observation matrix can often be diagonal if the underlying motion of the camera is circular. I wonder if the algorithmic convergence would be severly affected by the ``diagonal-ness'' of the visibility matrix as for other algorithms such as Damped Wiberg/Variable Projection (Okatani et al., Damped Wiberg, ICCV 2011, Hong et al., Revisiting Variable Projection..., CVPR 2017)\n\n3. Since RGMNR is only tested on matrices with comparatively small fraction of outliers (0 to 25%), I wonder if this method would outperform approaches based on non-graduated convexity in which the width of the robust kernel is gradually narrowed in each iteration (e.g., see Le et al., A Graduated Filter Method for Large Scale Robust Estimation, CVPR 2020 --- this is not a paper handling LRMF but a similar technique could be applied).\n\n4. In l. 259, it is reported that RGMNR succeeds even from random initialization. Does this mean all elements of $\\Lambda$, $U$ and $V$ are sampled from a particular probability distribution? What is the assumed underlying distribution? I am asking this because from my experience, I feel the convergence would depend significantly on how $\\Lambda$ (the predicted outliers) is estimated but is not discussed in the paper.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA new method for robust matrix completion, RGNMR, is introduced to overcome the limitations of existing methods. It is empirically shown that RGNMR has significant advantages when only a limited number of entries is observed, and, moreover, that it copes well with both overparameterization of the rank and with ill-conditioned matrices. Besides, it is shown that RGNMR recovers the underlying matrix exactly under suitable assumptions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The paper is well-written; ideas are clearly presented.\n- The numerical results look promising.\n- A theoretical analysis reveals that RGNMR recovers the underlying low-rank matrix exactly under suitable assumptions, and improves upon the best-known results for factorization-based methods.\n\nWeaknesses:\n- There is a discrepancy between the method that is theoretically analyzed, and the one that is used in the experiments. Although not uncommon in this field, I believe it would strengthen the paper if the authors could explain in more detail the connection between Alg. 2 and Alg. 1. \n- The main novelty of RGNMR is the procedure for estimating the number of corrupted entries $k^\\*$, which is then combined with the (existing) GNMR scheme. As mentioned in Remark 2.2, various existing RMC methods require an estimate of $k^*$, and therefore they could be equipped with a similar scheme. This raises the important question whether such an approach may yield a performance similar to that of RGNMR.\n\nTypos:\n1. Line 28: Remove / rephrase `both’, since it is followed by three adjectives\n2. Line 76: RGNMR performs well on real data\n3. Line 166: The vertical pipe for “such that” is doubled\n4. Line 188: Maybe good to explicitly define the support?\n5. Inconsistency: over-removal versus over removal\n6. Line 266: The remain open directions\n7. Line 472: The subscript init should be upright\n8. Line 583: Eq. eq. (14) -> Eq. (14)\n\nQUESTIONS:\n- Is there a connection (formal or intuitive) between the estimation procedure of the corrupted entries in Alg. 1 and Alg. 2? \n- Does this scheme for updating $\\Lambda_t$ in Alg. 2 also work decently in practice? Or is there a (significant) gap between theory and practice?\n- Could you attribute the observed properties of RGNMR to either the estimation procedure of the corrupted entries, or to the Gauss-Newton based method GNMR? What is the contribution of each of these to the overall method?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}