One-Sided Matrix Completion from Ultra-Sparse Samples
Authors: Hongyang R. Zhang
Zhenshuo Zhang
Huy Nguyen
Guanghui Lan
Venue: Accepted by TMLR ()

Abstract: Matrix completion is a classical problem that has received recurring interest from a wide range of fields. In this paper, we revisit this problem in an ultra-sparse sampling regime, where each entry of an unknown, $n\times d$ matrix $M$ (with $n \ge d$) is observed independently with probability $p = C / d$, for a fixed integer $C \ge 2$. This setting is motivated by applications involving large, sparse panel datasets, where the number of rows (users) far exceeds the number of columns (items). When each row contains only $C$---fewer than the rank of $M$---accurate imputation of $M$ is impossible. Instead, we focus on estimating the row span of $M$, or equivalently, the averaged second-moment matrix $T = M^{\top} M / n$.

The empirical second-moment matrix computed from observational data exhibits non-random and sparse missingness. We propose an unbiased estimator that normalizes each nonzero entry of the second moment by its observed frequency, followed by gradient descent to impute the missing entries of $T$. This normalization divides a weighted sum of $n$ binomial random variables by their total number of ones---a nonlinear operation. We show that the estimator is unbiased for any value of $p$ and enjoys low variance. When the row vectors of $M$ are drawn uniformly from a rank-$r$ factor model satisfying an incoherence condition, we prove that if $n \ge O({d r^5 \epsilon^{-2} C^{-2} \log d})$, any local minimum of the gradient-descent objective is approximately global and recovers $T$ with error at most $\epsilon^2$.

Experiments on both synthetic and real-world data validate our approach. On three MovieLens datasets, our algorithm reduces bias by $88\%$ relative to several baseline estimators. We also empirically evaluate the linear sampling complexity of $n$ relative to $d$ using synthetic data. Finally, on the Amazon reviews dataset with sparsity $10^{-7}$, our method reduces the recovery error of $T$ by $59\%$ and $M$ by $38\%$ compared to existing matrix completion methods.

Published in Transactions on Machine Learning Research (01/2026)
One-Sided Matrix Completion from Ultra-Sparse Samples
Hongyang R. Zhang
ho..shangghhhrthhesssee....t
Northeastern University, Boston
Zhenshuo Zhang
thonrggtteeetttttteeeeseee..... edu
Northeastern University, Boston
Huy L. Nguyen Northeastern University, Boston
hw..nggwgenChhorthssssee...
Guanghui Lan
george.lan@iisee.geeeh..
Georgia Institute of Technology, Atlanda
Reviewed on OpenReview: http:///weeermettten//////vvvvve
Abstract
Matrix completion is a classical problem that has received recurring interest from a wide range of fields. In this paper, we revisit this problem in an ultro-sparse sampling regime, d matrix M (with
where each entry of an unknown, with probability p C/d, for fixed integer C C 2. This setting is motivated by applications involving large, sparse panel datasels, where the number of rows (users) far exceeds the number of columns (iterns). When each row contains only ('-fewer than the rank of M-accurate imputation of M is impossible. Instead, we focus on estimating the row span of M, or equivalently, the averaged second-moment malriz T M M/n.
d) is observed independently
The empirical recond-moment matrix computed from observational data exhibits non-random and sparse missingness. We propose an unbiased estimator that normalizes each nonzero entry of the second moment by its observed frequency, followed by gradient descent impute the missing entries of T. This normalization divides weighted sum of binomial random variables by their total number of ones nonlinear operation. We show that the estimator is unbiased for any value of and enjoys low variance. When the row vectors of M are drawn uniformly from a rank-r factor model satisfying an incoherence condition, we prove that if n 2 E 2C log d), any local minimum of the gradient-descent objective is approximately global and recovers T with error most e².
Experiments on both synthetic and world data validate our approach. On three Movie- Lens datasets, our algorithm reduces bias by 88% relative to several bascline estimators. We also empirically evaluate the linear sampling complexity of relative to d using synthetic data. Finally, the Amazon reviews dataset with sparsity 10-7, our method reduces the recovery error of T by 59% and M by 38% compared to existing matrix completion methods.
1 Introduction
Matrix completion is classical problem that has received sustained attention across several disciplines, including compressive sensing (Candes & Plan, 2010), machine learning (Srebro et al., 2004), and signal processing (Chi et al., 2019). A seminal result in the literature states that when the unknown matrix M low rank and satisfies an incoherence assumttion (i.e., the row norms of the low-rank factors are approximately balanced), exact recovery from a uniformly random subset of entries is achievable (Candes Recht, 2012).
In this paper, we study matrix completion an ultra-sparse sampling regime, where the sampling probability p C/d, for some constant C 2, resulting in roughly Cn observed entries in total. This is less than

---

Published in Transactions on Machine Learning Research (01/2026)
nr log the number of bits required to store the rank-r factors of an by d matrix. Although recovering M itself is M specifically, the second-moment matrix T = M T M /n in this regime, In other words, when M is tall matrix, our goal is to recover the subspace spanned by the row vectors of M.
impossible (Chen, 2015), we show that it is possible to recover one side of
There are several motivations for revisiting matrix completion the ultra-sparse sampling regime. First, many large-scale panel datasets are extremely sparse in practice. For example, consider the Amazon reviews dataset (Hou et al., 2024), which contains roughly 54 million users and 48 million items, with about 571 million observed reviews in total. The resulting sparsity level is low as 10-7. Therefore, we need matrix completion methods that can remain effective under this level of ultra-sparsity. Second, many recommendation systems must preserve data privacy. In such settings, each user may contribute only a small portion of data to a central server, often after adding white noise for differential privacy (Liu et al., 2015). The central server then aggregates these contributions by computing summary statistic such as the low-vank subspace spanned by the row vectors. This summary statistic can then be shared publicly, allowing each user to project their data locally onlo the learned subspace (Wang et al., 2023). In these scenarios, the number of users can be orders of magnitude larger than the number of items (see Table 2 for concrete examples), which motivates studying recovery of the row space of M when n > d.
Recent work by Cao et al. (2023) initiates the study of one-sided matrir completion, whose goal is to estimate the averaged momod- matrix of the row vectors of M, denoted by
1 M,M, n
T
under the setting where two entries per row are observed. In other words, the matrix T' captures the row-space structure of M. Their analysis leaves open the general case where more than two entries are observed per row. To illustrate the challenge, let M denote the observed data matrix and I the corresponding binary mask indicating observed entries. Because M is highly sparse, the information it contains is insufficient to recover M directly rendering standard matrix completion guarantees inapplicable to this regime. Let Ω denote the set of nonzero entries of the empirical second-moment matrix M M, which has the same dimension as T. We make two key observations about MT M that together highlight the technical challenges in recovering T under ultra-sparse sampling:
1. The missing patterns in Ω is highly non-random: diagonal entries are almost always observed, while off-diagonal entries are extremely sparse. For a detailed calculation, see Claim 2.1. Moreover, the observation frequencies in the off-diagonal entries of I are sparse and unevenly distributed.
2. Although stochastic gradient algorithms are widely used in practice (Lan, 2020), providing theoretical guarantees for matrix completion is technically challenging (Sun & Luo, 2016) and has not been done under non-random missingness, to the best of our knowledge.
To address the first issue, we analyze a weighted estimator that normalizes each nonzero entry of M/ M by the corresponding entry at ITI This approach corresponds to the Hájek estimator (Hájek, 1971; Särndal et al., 2003), which divides two random quantities making its analysis more subtle due to nonlinearity. Our key observation is that the Hájek eslimator is eraclly unbiased on the observed subset Q, unlike prior results that show only asymptotical unbiasedness as + co. This finite-sample unbiasedness arises from the symmetry of pairwise products in computing the sccond-moment matrix.
Furthermore, a first-order approximation analysis on the variance of the Hájek estimator reveals that this estimator achieves lower variance than the classical Horvitz- Thrmmpso estimator (Horvitz & Thompson, 1952), leading to more stable and less biased empirical estimates, a result corroborated by our experiments.
To tackle the second issue, we establish recovery guarantees for any local minimizer of a loss function defined between the normalized empérical second-moment matria and its low-rank factorization used to impute missing eutries of T that lie outside Q. Under rauk-r factor model for the rows of M, we show that if
n 2 O(dr C log
2

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Comparison between this work and several closely related results on matrix completion under an ultra-sparse sampling regime. Here, M E Rn with n 2 d, and T 1 M denotes the averaged moment matrix of the rows of M. We focus on the one-sided or ultra-sparse settings where exact recovery of M itself is infeasible.
Estimand
Sampling Regime
Main Approach (Reference)
M
p p=O@. Two entries per row
Thresholded Alternating Minimization (Gamarnik et al., 2017)
T
Nuclear-Norn Regularization (Cao et al., 2023) HÁJEK-GD with Incoherence Regularization (This paper)
T
p= C 2C
then any local minimizer of the loss objective is within e² Frobenius norm distance to T. The big-O notation above ignores dependencies of the bound on the condition number of M as well as the incoherence condition number (See Assumption 3.3 for the definition). This bound assumes standard incoherence conditions- impossible (Candes & Recht, 2012) and is optimal
without which recovery is innemaation-thoreeiiii in scaling relative to d. Importantly, the result holds for any constant C, thus generalizing prior results and closing an open question left by Cao et al. (2023). key technical stop is establishing a concentration inequality to lackle the non-random missingness of S2, by leveraging a conditional independence property al each row of M M. This conditional independence property enables us to analyze and bound the bias of the Hájek estimator row by row, ensuring consistency even under heterogeneous observation patterns.
Numerical results. We extensively validate our approach on both synthetic and real-workd datasets. We find that the Hájek estimator is particularly effective under extreme sparsity. For instance, when each row has only two observed entries, our method outperforms a prior nuclear-norm regularization baseline by 70% recovering T. We also explore using one-sided matrix completion as a subroutine for imputing missing entries of M. On the Amazon reviews dataset, we extend our estimator to impute M via least-squares regression after estimating T, achieving 21% and 38% lower root mean squared error (RMSE) than alternating gradient descent and soft-impute with alternating least squares (Hastic et al., 2015; Chi et al., 2019; Li et al., 2020a), respectively. Ablation studies further show that the Hájek estimator reduces bias on Ω by 99% relative to the Horvitz-Thompson Thompson estimator on synthetic data, and by 88% on average across three MovieLens datasets. Incorporating an incoherence regularization term (Ge et al., 2016) into the loss further decreases recovery error of T by 22% on synthetic data and by 52% on the Amazon review dataset.
Summary of contributions. This paper revisits the matrix completion problem under an ultra-sparse sampling regime, where only O'n entries are observed from an unknown n x d matrix for a constant C 2 2. Our contributions are threefold:
1. We design new algorithm for one-sided matrix completion based on the Hájek estimator, accompanied by a detailed theoretical analysis. We prove that the Hájek estimator is an unbiased estimator of the second-moment matrix for any sampling probability p and achieves lower variance than the Horvitz Thompson estimator.
2. We establish near-optimal sample complexity bounds for one-sided matrix completion under a low- rank, incoherent factor model. Our results apply to gradient-based optimization and extend to general sampling patterns beyond those considered in prior work. See Table for a comparison of sample complexity and main approach.
3. We perform extensive experiments on synthetic and real-world datasets, demonstrating the practical effectiveness of our approach and showing that one-sided matrix completion can be used as a robust subroutine for full matrix recovery. The implementation and reproducíble code are available at: http:///ggtttbbcco.oi/ttttteeeee
One limitation of our sample complexity result lics in the common-means assumption the rank-r factor model. We believe this condition can be relaxed if each factor is perturbed by random noise (see Remark 3.5). Another limitation concerns the existence of exact low-rank factor models for recovering T and the need
3

---

Published in Transactions on Machine Learning Research (01/2026)
to specify the true rank in the gradient-descent procedure (see Remark 4.3). We discuss possible extensions to address these issues in Section 7.
Organizations. The remainder of this paper is organized as follows. Section 2 describes the problem set and notations. Section 3 presents the estimator and the main theoretical results. Section 4 outlines the proof sketch and two extensions. Section 5 and provide empirical results and related work, respectively. Section concludes the paper and discusses several open directions. Complete proofs appear in Appendix A-B, and additional experiments are discussed in Appendix C.
2 Preliminaries and Notations
Let M denote an by d, unknown matrix. Assume the rank of M is equal to Without loss of generality, suppose that 2 d. For example, each row of M may include the ratings of user, while each column may correspond to the ratings of item. We observe partial matrix, denoted by M. Let T 10, denote the indicator matrix on the observed entries of M. Our goal is to recover the averaged second-moment matris T M M/n, given M.
We assume that each entry of M is observed independently with probability p E (0, 1), following prior literature (e.g., Candes Recht (2012); Sun & Luo (2016); Ge al. (2016)). We focus on constant sampling regime, where p C/d, for some fixed integer C 2 2. In such regime, is roughly Cn, less than the nunber of bils to represent an by r malrix.
An important quantity for working with one-sided estimation is 1 1,, which is a symmetric matrix in dimension d. This matrix provides the empirical froquency counts for the empirical second moment matrix M M. For example, consider an off-diagonal entry (i,j), where i A j and both i, are in (1. d]. The (i,j)-th entry of T I, denoted as (T T),,,, is equal to the number of overlapping, nonzero entries between the i-th and j-th column of M. That is,
a
(T))
Σ I...R.
(1)
Let S denote the indices of the nonzero entries of IT. We now make the following observation regarding the observed entries within Q. In particular, we find that the sampling patterns of f2 are non-random in the following sense.
Claim 2.1. For every i = 1,2, .2...,,,, the diagonal entries satisfy that
p)" Pr[(i, i) E [0]==11 = 1 p
The off-diogonal entries satisfy that for every i == 1,...,d,, d, and j = -1, 1,....d such that j + i,
Pr((i,)) E ? = 1 (1 pp"
To illustrate, recall that p C/d. Suppose n is at the order of Odd d). The probability that (1, E Ω is roughly 1 - e pm se d Then by a union bound, with probability at least 1 - O(d-1), it must be the case that (i, E Ω, for every i = 1,2.....
For each off-diagonal entry, it is observed in Ω with probability equal to (1 p²)". For simplicity of notation, let us denote this as g in the rest of the paper. Note that q np² is roughly equal to O(C2d- d).
To account for the difference in sampling probabilities between diagonal and off-diagonal entries of Ω, let Po : [ddd + dxd be a weighted projection operator defined as follows:
oZ
if (i, Ω
(Pa(Z)) =
Zij
if (i,j) E Ω and j 4 i,
0,
(i,5) & Ω.
4

---

Published in Transactions on Machine Learning Research (01/2026)
Estimators. Having introduced the co-occurrence matrix I and its nonzero index set 52 next, discuss the normalization of the empirical second moment of the observed matrix, given by M M. One way to normalize M M for one-sided matrix completion is via the Horvitz-Thompson (HT) estimator (Horvitz & Thompson, 1952), which inversely reweights each entry with the true sampling probability (i.c., p on the diagonal entries and p² on the ofl-diagonal entries):
Σ Me
if =- J.
T
up
if if j.
One can verify that [[[7,,]]= = T... Notice that np² is the expectation of (TT The (cf. equation (1)), while up is the expectation when i j. When p is too small, np² becomes small, resulting in high fluctuation in the off-diagonal entries. Another approach, known as the Hájek estimator (Hájek, 1971), is to normalize cach entry in Ω by the estimated sampling probability, which is canceled out since the probability value is the same across different k, leading to the following expression, for any (i,j) E Ω2
if J.
kiz IN.s
Mr. Me, Ei Ja,ekx
if i + j.
The theoretical analysis of the Hájek estimator is challenging due to its nonlinear form, which involves dividing one random variable by another. It known that the Hájek estimator is asymptotically unbiased (Hájek, 1971) in the large limit, incurring an error of (nn-1/2). In addition, the Hájek estimator provides variance reduction effect compared to the Tervitt-Thomsoon estimator in causal inference (Hirano et al., 2003). Little is known in the matrix completion problem, and several natural questions arise. First, how does this estimator work for one-sided matrix completion? What is the bias of T, and how does the Hájek estimator compare to the Thomppon estimator? Second, how can we use T to impute the missing entries outside ?? The rest of this paper is dedicated to answering these questions.
Notations. Before continuing, we provide a list of notations for describing the results. Following the convention of big-O notations, given two functions f(n) and g(n), let ['m) S O(g(n)) indicate that there exists constant e independent of n such that when n 2 ng for some large enough no, then f(n) c- (nng.).. We use the notation f(n) & g(n) as a shorthand for indicating that f(n) = Ogg(n)). Let Ö(g(n)) denote that ((log((n)ggn))) for some constant independent of n. We also use the little-o notation fnn S o(g(n)), which means that f(n)/g(n) goes to zero as goes to infinity.
Let [d] = {1, 2,.,,,..,, d} denote a shorthand notation for the set from up to d. We use W(a, b) to denote a Gaussian distribution with mean set at a and variance equal to b. We use the notation (x)+ = max(r, 0) to denote a truncation operation set above zero.
Let ||-I|, denote the Frobenius norm of an input matrix. Let |1-1|2 denote the spectral norm of the input matrix and let ||-I| denote the Euclidean norm of a vector. Let (X,Y) Tr(X Y] denote the inner product between two matrices that have the same dimensions. Let I1- denote the infinity norm of matrix, which corresponds to its largest entry in absolute value.
3 Estimation and Recovery Guarantees
In this scction, we present a thorough analysis of the Hájek estimator in the onc-sided matrix completion problem. We begin by analyzing the bias of T compared to T the observed entries Ω. A key observation is that I provides unbiased estimate of T, as stated formally below.
Lemma 3.1. Suppose the entries of M are sampled from M independently with a fized probability p E (0, 1). Let Ω denole the inder set correspouding to the non-zero entries of M M. Then, the following must be true:
E (T.s(( s2] -T.s
(2)
5

---

Published in Transactions on Machine Learning Research (01/2026)
for any 1 1 1ii S d. As corollary of equation (2), have that
Tio Pr ((,,) E S
Unlike conventional results showing that the Hájek estimator is asymplotically consistent (Särndal el al., 2003), in the one-sided matrix completion setting, we find that the Hájek estimator unbiased in the presence of finitely many samples. As remark, the same unbiased estimation result can also be stated in the setting where every row has k uniformly random entries from M, for arbitrary integers k 2 2. The proof is similar to that of Lemma 3.1 and is omitted.
We now describe the main idcas for proving Lemma 3.1. First, we note that for any k: > 1, conditioned on Mn.-Mn..
(I Dr, = k, M M must be equal to the sum of pairs chosen from M1,M M2,1 Furthermore, the choice of such pairs out of uniformly at random among all possible (x) combinations, because the choices are symmetrie. Thus, the average of randomly chosen pairs must be equal to the average of all pairs in expectation. This is stated preciscly in the following equation:
E
M i.
( =
Σ
Ma Ma. Ti.j
(3)
k
which leads the conclusion of equation (2). See the full proof in Appendix A.
3.1 Variance Reduction
Having established that T provides an unbiased estimator on the entries of T, we turn to studying the variance of F. We show that the Hájek estimator incurs lower variance relative to the Horvitz- Thromst-Thomsoon estimator. We first derive an approximation
Var(T) to tackle the nonlinearity the Hájek estimator.
Theorem 3.2. Suppose == - for some fized integer C² 2 2. Suppose n 2 O(d d). Then, with probability at least 1 O), (i) all of the diagonal entries of T are in St and (ti) for any diogonal entries of T, the variance of Tr.s, for any i = 1.2.....d. ,d, is approzimated by
Var (T.))
1-p
Mr.
1 1-P
T2
+0
1
(4)
np
np
npp
km
For any nonzero, off-diogonal entries in Ω2 the variance of Pigj for any 1 S 1ii I d, is approrimated by
(logd)2) d)²
(log
Var Tig (t.) (.)
n Σ M Mr
Thy +0
(5)
We now compare the variance of the Hájek estimator, Var(T), with the variance of the Horvitz-Thompaet estimator, VTr(T), since both estimators are unbiased. We first derive the variance of T to make the comparison. For diagonal entries, we have that T., (np) M M) for = I,..., d. For off-diagonal entries, we have Ti, = (np²)-1 (app)--(MM (M Myss, for i + j. Then, we calculate the variance of T as:
Var (Tis) = P
M k.s: Vi= I,...,d,
(6)
n2p
k=
Var Tws)
M k.j' Vj = I....,, and j j #i,
(7)
n²p²
k=1
which can be verified based on the definition of T Section Now we can compare equations (6) and (7) with equations (4) and (5), respectively.

---

Published in Transactions on Machine Learning Research (01/2026)
10
T (Ilájek)
T (IIT)
Variance
10
10
10-12
0.01
0.05
p U.1 C= == C 100
Figure 1: An illustration of the variance reduction using the Hájek estimator vs. the Horvitz-Thompson (HT) estimator, measured on synthetic data with n = 10 and d = 10", by repeating the data sampling procedure 100 times and calculating the variance across either uniform sampling with probability p, or sampling C entries per row. In particular, Var(T) is generally 10-3 lower than Var(T). For details regarding the simulation sctup, see Section 5.
Notice that the diagonal entries are nonzero with high probability over all the diagonal entries. By comparing equation (4) to equation (6), we see a reduction in the second term of equation (4), (11-p)T?./mmp) which is at the same order as the first term.
The off-diagonal entries are nonzero with probability equal to q R np² - o(d By comparing equation (5) to (7), we see that equation (7) is larger than the first term of equation (5) by factor of mp²/ (11 - p²). In addition, the second term of equation (5), which is always negative, further reduces to the variance of Tij compared with Tūj-
Algorithmic implications. One consequence of variance reduction is that the empirical performance of the Hájek estimator tends to be much more stable than the Torvitx-Thomppes setting. This is illustrated in Figure 1, measured on synthetic datasets with n = 10 and d d= 10 for various values of p (uniform sampling) and C (i.c., sampling C entrics per row without repetition).
estimator in the ultra-sparse
3.2 Sample Complexity
Next, we consider the entries outside Ω. We minimize a reconstruction error objective between T and XXT, for some X E Rdxr, plus a regularization penalty of R(X), as follows:
((X) 2 P - [| + +R(X),
where À > O is a regularization parameter. R(X) is defined as EL_ (I)x, - a)4, where X, E R" is the i-th row vector of X, for i i= 1....,d, d, and a is a scalar that roughly corresponds to the row vector norms of X. This penalty regularizes the maximum fo-row-norm of X, and is shown to provide theoretical properties in the optimization landscape of matrix completion (Ge et al., 2016; 2017). We summarize Algorithm 1, which uses gradient descent to minimize E(X). As a remark, in the loss objective f(X), we recover the diagonal and off-diagonal entries of T together via XXT, while reweighting the diagonal entries of XXT, based on the calculation in Lemma. 3.1.1
procedure in
We analyze this algorithm in common mcans model, where each row of M follows mixture of r vectors U1, .2,...,, E Rd. Let M; denote the i-th row vector of M. For every = 1.2..... n. assume that M, is drawn uniformly at random from u1, U2,..., Mp- Let U = [u1, u2, u2,..., / denote a by r matrix corresponding to the combined rank-r factors. Let k = mmax (max(U)) (U)/omin(U)) denote the condition number of U. A critical condition to ensure guaranteed recovery in matrix completion the following assumption regarding the row norms of U.
We remark that there other algorithms besides gradient descent for the imputation step, such subspace recovery the Crassmaun manifold (Boumal & Absil, 2011). Zilber & Nadler (2022) design an itcrative Gaussian- algorithm for matrix recovery based linearization of the recovery objective. A natural extension of HÁJRK-GE GD involves using the Hajek estimator in the first step, and then applying these methods (instead stochastic gradients) in the second step.
7

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Hájek estimation with gradient descent (HAJEK-GD) for one-sided matrix completion
Input: partially-observed data matrix M E Raxd
Require: Rank of the recond-momen matrix 7, number of iterations t, and learning rate Output: A by d matrix
1: Te Raxd f- The 0-1 indicator mask corresponding to the nonzero entries of M
2: CC [d] x [d] + The set of indices corresponding to the nonzero entries of I
Te Rdxd f The element-wise division between M M and I I on ?
4: Xo E Raxr + A random Gaussian matrix whose entries are sampled independently from N00 d-!)
5: for m=i=l l,..., do
6: X X - ()(((_))),, where E(-) is defined in equation (8)
7: end for
return T plus the entries of X,X, outside ?
Assumption 3.3 (See also Definition 1.1 in Recht. (2011) and Assumption 1 in Ge et al. (2016)). Let Ue denote the i-th row vector of a d by matrix U, where E Rd is the i-th basis vector, for any = 1, 1.2...., d. The coherence of U is given by
d
|Uee 1sisd |
Assuming that ((U) is fixed value that does not grow with d; That is, n(U) does not increase with the dimensionality of the problem. We show the following performance guarantee of gradient descent for recovering T, measured in terms of the Frobenius norm distance between local minimizer and T. In particular, recall that À refers to the strength of the regularization penalty, while a refers to the magnitude of the norm of the row vectors. See also the description of the incoherence penalty term below equation (8).
Theorem 3.4. Suppose the rows of M follow a mirture of common factors given by U E Rdxr, Additionally, the coherence of U is at most ps for some fized 221 1 that does not grow with d. Suppose each entry of M is observed with probability p = _ for some fixed integer C 2, and let q for some fixed constant e and e 1), with probability at least - Odd-1) over the randommess of Ω when d is large enough, any local minimizer X of E(-) salisfies
(1 - 12)= Let = 4x2r
and X (r+1)dg
16 When n 2 ndr""n"""
C²e
x -T sc.
(10)
This result implics that any local minimum solution of the loss objective also approximately a global minimum solution. We defer a proof sketch of this result to Section 4. We remark that prior results have established onc-sided matrix completion guarantees when each row has two observed entries (Cao et al., 2023).
By contrast, Theorem 3.4 applies to arbitrary values of C 2 2 in each row.
Notice that Theorem 3.4 crucially relies on the assumption that n is much larger than d. To provide several concrete examples, Table 2 in Section 5 presents the values of n and d for several real-world datasets, including MovieLens and Amazon reviews. assumption that n can be much larger d in practice. Our experiments, presented later in Section 5.2, further validate the sample complexity result using synthetic data.
these examples, the average n/d is roughly 10, which justifies the
Algorithmic implications. A crucial step in the proof is to leverage the incoherence assumption, which reduces local optimality conditions from finite samples to the entire population. Later in Section 5.3, we also empirically demonstrate that the use of the incoherence regularization penalty helps improve performance on real-world datasets.
Remark 3.5. The common means model often used to study learning from heterogeneous datasets such as multitask learning and distributed learning. See, e.g., Kolar et al. (2011); Dobriban Sheng (2020). In particular, Kolar et al. (2011) study union support recovery under the common means model, further assuming
8

---

Published in Transactions on Machine Learning Research (01/2026)
that every mean vector is perturbed by another noise addition of c, (added to M.). Despite the simplicity of this model, several learning problems can be reduced to the Normal means model (Brown & Low, 1996). Dobriban & Sheng (2020) study a distributed ridge regression problem, where each every individual performs a ridge regression on a local machine, and then, the ridge estimators are aggregated together on a central server. In particular, the sumples of each individual are assumed to follow a linear model with the same B3 plus independent random noise with mean zero and voriance o². One way to extend the common means model is by positing a noise addition, such as a noise vector E, added to Mi, whose entries are drawn independently from Gaussian distribution (0, o² /d). Our proof can be naturally applied to this extension. In particulor, there are three places that need to be modified to account for the noise addition, including: 1) The proof of Lemma B.1, in particular equations (27), (28), and (29). 2) The proof of Corollary B.2, in particular, adding the dependence of o² to the numeralor in equalions (31) and (32). 3) The proof of Corollary B.3, which inherits the dependence on o² from Corollary B.2. These extensions will rely on concentration bounds on Gaussian random variables. Finally, we would also need to incorporate the noise variance in the diagonal entries of T. We can de-bios this by subtracting a suilably scaled diagonal malrir in the objective ((X).
Another plausible extension is to instead assume that each M, is linear combination of the low-rank factors. This setting oppears to be sigmificantly more challenging and would likely go beyond the techniques developed in this paper, as one would first need to disentangle the linear dependence and design a new estimator capable of learning the low-rank subspace sponned by the latent factors. One promising direction for tackling this problem is to use the spectral estimator developed in the work of Chen et al. (2021). This is left as an open question for future work.
Discussions. In the proof of Theorem 3.4, we tackle the error terms arising from the diagonal and off-diagonal entries separately. This is because the diagonal entries are fully observed with high probability. By contrast, the off-diagonal entries are only sparsely observed. At a high level, as n increases, the concentration errors in the diagonal entries will decrease (at a rate of n -1/2). Meanwhile, q will also increase, leading to more observed samples in the off-diagonal entries. Thus, we can set n carefully to ensure that the error between T and T, when restricted to Ω2 sufficienlly small. The details can be found in Appendix 1.11..2.
We now discuss work from the perspective of prior work by Fan et al. (2013). Their work examines low-rank plus noise common factor model. Their estimators involve low-rank estimator, plus a diagonal matrix. In particular, this diagonal matrix is designed to offset the bias contributed by the noise term. As an extension, if we incorporate the noise term into the latent factor model of Mi, we would similarly need to de-bias the diagonal entries to account for the norms of the noise. The proof can be extended to handle this setting by estimating the magnitude of the noise variance, and then adding this bias to the objective f(X).
4 Proof Techniques and Extensions
Next, we present a sketch of our proof. There are two major challenges in analyzing the Hájek estimator in the ultra-sparse sampling regime: First, the estimator involves dividing one random variable by another random variable, resulting in a nonlinear estimator. Further, the missing patterns in Z are non-random, depending on the diagonal and off-diagonal entries, respectively. Our approach to tackle this non-linearity is via a first-order approximation technique for analyzing the Hájek estimator on the diagonal entries. Further, we carefully analyze the bias the off-diagonal entrics.
Second, the sampling patterns of Ω are not fully independent. To this end, we establish concentration inequality that handles the concentration error row by row in Ω. This is based on the observation that, conditioned on one row, expect the diagonal entries, the randomness of each entry in that row would be independent across different entries. This row-by-row concentration argument also allows us to analyze the spectral norm of the bias matrix. As a remark, this type of argument has been used in matrix completion with non-random missing data (Athey et al., 2021). However, prior work focuses on analyzing nuclear norm regularization methods, while our result now applies to gradient- based optimization. Next, we outline the main results in addressing the above two challenges.
9

---

Published in Transactions on Machine Learning Research (01/2026)
4.1 Main Proof Ideas
Bias and variance of the Häjek estimator. Recall that T is unbiased by Lemma 3.1. Thus, the variance of T is the same the expected squared error between T and T. A key result for deriving the variance of the Hájek estimator is as follows.
Lemma 4.1. In the setting of Theorem 3.4, suppose p = C/n for some fized integer 2 2 and 2 d(log d)/e². Let S, {j: (i.j) E Q.j + i} denote the set of indices corresponding to the nonzero entries in the i-th row of T. With probability at least O( the following must be true, for every = 1,2 1.2...... d,
1²r dnpp
6C-1
3 3log(d/2)
Var (Tis)
1
1 +
1 1+
(11)
d
2n
3 3log dd/2
Σ JES:
Var Tiis
1 113rV
+ (d
(12)
2d
Proof sketch. The proof regarding the diagonal entries in equation (11) is based on first-order approximation of Var(T.g). For every 1,2...... d, let
A1, := Σ Jk. and B == Σ I..
(13)
We perform Taylor's expansion of Ti around (B(,.)) [B.,i) as follows (See also Chapter 5.5, Särndal et al. (2003)):
E [Ar,]] E [
1
E [Ar.]
Th
(Ar. (E(A,)) E [i,s]
(Br, - D.Bii-[[BL, ((B + EiS--
(14)
(E [2,,))
Above, the first term to the right of equation (14) stems from the zeroth-order expansion. The second and third terms arise from taking the partial derivatives over Ar,,, Bi respectively. fi is an error term that is at the order of the variance of Ar,s Ba,, which can be calculated in close form based on equation (13).
Given that f is unbiased, Vfr(f..)) is equal to the expectation after squaring both sides of equation (14). As result, we derive the variance approximation of Ti, by carefully analyzing each term, which will lead equation (4).
Figure 2 provides illustration of equation (14) (except the e.s error term) and the corresponding variance estimate from equation (4). We find that both approximations hold with negligible errors for various sampling probabilities (uniform sampling with probability p) and C (fixed number of entries per row).
This simulation uses = 10 and d 10 and follows the setup stated in the synthetic data generation process, which can be found in Section 5.
As for the off-diagonal entries of f, note that when (i,j) E Ω, with high probability the (i.j)-th entry of IT must be one. Thus, Var(Ti,) is equal to the variance of values M1, Mn, (plus some small errors). This leads to the variance approximation of equation (5). See Appendix B.1 for the full proof.
A concentration inequality for operator Pq. In order to analyze the optimization landscape of the loss objective E(-), we will first derive the first-order and second-order optimality conditions. However, these are stated on the observed set 2 and we need to turn them into the full set on T. To this end, we develop the following concentration inequality, which helps reduce finite-sample local optimality conditions to the full matrix.
10

---

Published in Transactions on Machine Learning Research (01/2026)
2.0 x 10
x 1005
x x00
1.5 X 10
2.0
1.0
Error
1.5
Error
1.5
Error
Error
1.0
0.5
1.0
0.5
0.5
0.5
0.0
0.0
0.05
0.1 0.15
0.2
0.25
5
10
15
20
25
0.05
0.1 0.15
0.2 0.25
5
10
15
20
25
p
C
C
(a) Taylor's expansion
(b) Taylor's expansion
(c) Variance approximation
(d) Variance approximation
We illustrate the results from applying a first-order approximation to the variance of the diagonal entries of HÁJEK-GD, run on synthetic data with = 10 and = 10 Figures 2a and 2c: Sample cach entry with probability p. Figures 2b and 2d: Sample C entries per row without repetition. In particular, the approximation errors incurred by both first-order Taylor's expansion and the variance approximation are generally less than 10-6.
Figure 2:
Lemma 4.2. Let W E Rdxd be d by d symmetric matriz such that [ 2. for some fized positive value of v. Then, with probability at least 1 - O(t-1), the following slatement must be true:
16log(2d)
[Pa((W) gWll2 S V
(15)
dg
The proof of this concentration result can be found in Appendix B.2.
The last ingredient involves analyzing the population loss of one-sided matrix completion, building the machinery of Ge al. (2016). These are discussed in detail in Appendix B.3 and B.4, which involve analyzing the population landscape and then reducing empirical case to the population case, respectively.
In particular, we extend their analysis to account for the bias of T and the non-random missingness of Ω due to the diagonal and off-diagonal entries.
The treatment of the bias of T relative to T is especially tedious and requires careful calculation. We build on Lemma 4.1 and 4.2 to give the bias of each diagonal entry, and the bias of each row See Corollary B.2.
The final proof of Theorem 3.4 can be found in Appendix B.5. We also discuss our work from the perspective of the work of Ge et al. (2016) in Remark B.12.
The above sample complezity result assumes the existence of a low-rank factor model, and the gradient descent algorillum requires knowing the exacl runk of the faclor model. In practice, one can adjust the rank parameter in the gradient descent algorithm via cross-validation See detailed ablation analysis in Appendir C.3. It is interesting question to eramine adaptive optimization methods that can automatically learn or estimate the true runk without knowing il. One natural exlension of our technique is by runming SVD on T and from the decomposition infer the rank of T. For example, this can be achieved by analyzing the spectrum induced by the noise added to T. See Corollary B.2 in Appendia B.1 for the analysis of the bias (i.e., T) on Q.
Remark 4.3.
4.2 Extensions
We now describe two extensions of our approach. In the first extension, we use HÁJEK-GD to recover the unobserved entries M. To accomplish this, we solve a least-squares regression problem that projects the observed entries of M onto the span of the recovered sccond-moment matrix. An illustration of this overall procodure provided in Figure 3, and the complete algorithm is detailed in Algorithm 2. In particular, when C O(r log d), one show that this least-squares procedure accurately recover every of M.
11

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm 2 Missing data imputation using one-sided matrix completion
Input: A partially observed M Raxd
Require: Rank r; number of iterations i, learning rate y)
Output: An & by d matrix corresponding to the imputed rows of S
< HÁJEK-GD M; r, 1,), U,D,U," rank-r SVD of
// May add Gaussian noise the non-zero entries of M
3: < set of indices corresponding to the nonzero entries of M M
Q 4 arg min 1/2 EiEs,(,)eE (QQU)) Maa)
return QXT
As discussed in the introduction, recovery guarantees are generally impossible when the number of samples is extremely limited. However, one practical way to view this algorithm as a user-level recovery procedure: when a user performs the local least-squares regression, it is likely that they will have access to additional local data beyond the ultra-sparse global observations. In such cases, the least-squares step can accurately roconstruct the entire row based on the recovered row space.
In the second extension, we further generalize our recovery guarantees to account for differential privacy (DP) using the Gaussian mechanism. We begin by recalling the notion of differential privacy in the context of matrix completion. Given an input M, an algorithm A is said to satisfy (c, 6)-joint differential privacy if, for any i E 21,2,.... .,,,,, n), replacing the i-th row of M (and the corresponding row of M) by another vector x E X Rd yields matrices M and M' that differ in only one row. Then, for any set of measurable events S,
Pr A(M) s) S exp(() Pr A(M') E S +
(16)
This definition follows prior work on user-level DP for matrix completion (Liu et al., 2015; Wang et al., 2023).
To satisfy Os, we perturb the nonzero entries of A with Gaussian noise drawn from NO0, o²). To determine the appropriate noise level o, we measure the fz-sensitivity the algorithm A Rexd - Rdxd as:
A2(A) = max ||A(M) A(,')ll, M~M
(17)
where M ~ M denotes that the two matrices differ in at most one row.
By standard results in the DP literature, when
σ 2 2n (1.258 (1.25-11)42(4 (AA)
the algorithm HÁJEK-GD combined with the Gaussian mechanism satisfics the (c, ()joint diffcrential privacy with high probability. See Theorem A.1 in Dwork & Roth (2014) for the full statement. In Appendix C.3, we empirically estimate the sensitivity of HÁJEK-GD and find that A2(A) remains small typically between and 19 across several datasets.
Together, these two extensions show that our approach can (i) leverage the recovered space to im- pute missing entries of M, and (ii) ensure privacy-preserving estimation under standard Gaussian noise perturbation.
5 Experiments
In this section, we evaluate our proposed approach through extensive experiments on both synthetic data and real-world datasets.
Synthetic data generation. We generate synthetic data by sampling each entry of M Ra independently from a Gaussian distribution ( 1/d), where denotes the number of columns. Unless otherwise stated, fix 10 and d 103. To construct. low-rank ground-truth matrix, we perform SVD of M and
12

---

Published in Transactions on Machine Learning Research (01/2026)
2
2
U
X (= U,D,U,)
Q
M Input: partially- observed data matrix
Step 1: Using Häjek estimator
Step 2: Impute missing entries Tf using gradient descent
Extension: Impute missing entries of M using least squares projection
Figure Musstration of the proposed one-sided malria completion pipeline. Input: A partially observed matrix M E 1xxd Each row represents user's data (e.g,, movie ratings), and each column represents item. Thus, M is a tall-and-skinny matrix, with being larger than d. Step 1: Apply the Hájek estimator to correct for non-random missingness in the empirical pecond-moment matrix M M over the observed index set Q. Step 2: Impute the remaining unobserved entries of T by running gradient descent on a low-rank reconstruction loss between the estimated and true second-moment matrices. The final combines the observed entries from T with the missing entries from X,X,", where X, denotes the t-th gradient descent iterate. Extension: Perform full matrix imputation by projecting onto the recovered low-rank subspace U, (obtained via rank-r SVD of the estimated T) and solving least-squares regression problem estimate the missing entries of M.
retain the top-10 singular values and corresponding singular vectors, truncating the lower tail. The observed matrix M is then obtained by sampling each entry independently with probability p = C/d. We also consider an alternative sampling scheme that selects exactly C' entries per row without replacement, and find that the results are qualitatively similar under both schemes.
Real-world datasets. We further evaluate our algorithm on several large, sparse matrix datasets: (i) three MovieLens datasets with 20 million and 32 million nonzero entries; (ii) an Amazon Reviews dataset (Hou et al., 2024); and (iii) a Genomics dataset from the 1, 000 Genomes Project (Cao et al., 2023). These datasets are standard testbeds for sparse matrix complction and cover wide range of sparsity leves and scales representative real-world recommendation and biological data. Their detailed statistics are summarized in Table 2. We formulate the prediction task on each dataset as a regression problem.
Baselines. We compare our method against three strong basclines widely used the matrix-completion literature:
Nuclear-norm regularization (Cao et al., 2023), which coincides with the same weighted estimator as HÁJEK-GD when each row contains exactly two entries. One difference between our implementation and theirs is that we also add the incoherence regularization term to the loss function.
2. Alternating Gradient Descent (alternating- GD).
3. Soft-Impute with Alternating Least Squares (softlmpute-ALS) (Hastie et al., 2015).
To adapt the latter two methods for one-sided matrix completion, we first apply each baseline to recover the full matrix M and then compute the corresponding second-moment matrix T M M/n. Additional dataset descriptions and implementation details are provided in Appendix C.
Evaluation metrics. For one-sided matrix completion, we measure compare the recovery error as the Frobenius norm between the estimated and true T matrices. For full matrix imputation, we compare Algorithm to alternating-GD and softImputo-ALS, reporting the root mean squared error (RMSE) between the imputed and ground-truth values. All experiments are implemented in PyTorch and executed on an Ubuntu server with 16 Tntel Xeon CPUs and one Nvidia Quadro 6000 GPU.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: The detailed statistics of five real-world matrix datasets used in our experiments.
Dataset
Genomes
Amazon Reviews
Mov
Mov
-oV
# Rows
100, 000
100, 000
138, 000
162, 000
200, 948
# Columns
2, 504
50, 000
27, 000
62, 000
87, 585
# Nonzero
2.5 x 10
1.3 x 10
2 x 10"
2.5 107
3.2 107
Summary of numerical results. For one-sided matrix completion in the sparsest regine- where cach row has only two observed entries we find that HÁJEK-GD reduces recovery error by 70% compared to nuclear-norm regularization on synthetic data. Across all three real-world datasets (each with sparsity below 1%), HÁJEK-GD reduces the recovery error of T by at least 42% relative to alternating- GD and by 59% relative to softImpute-ALS,
When using HÁJEK- GD to impute the missing entries of M as part of Algorithm 2, the overall approach achieves an 85% reduction in RMSE compared to alternating-Gi GD and softImpute-ALS on synthetic data where each row has only two entries. On 21% relative to alternating-Gi and by 38% relative to softImpute-AI.S.
datasets, HÁJEK-GI GD further reduces RMSE by at least
Finally, conduct ablation studies to validate our algorithmic design choices. In particular, we highlight the benefits of incorporating the incoherence regularization R(X) in the loss and demonstrate that HÁJEK- GD remains robust under random noise perturbations in the input. Together, these findings indicate that our approach is particularly effective and stable in ultra-sparse matrix regimes.
5.1 Measuring Bias on Observed Entries
We begin by comparing the bias between î and T, measured by the sum of squared errors between f (and T) with T on Ω. Our finding is illustrated in Figure 4. First, find that the bias of f compared to T, is at the order of 10-4 to 10-2 for various p. In Figure 4a, we find that on synthotic datasots, the bias of 7 is 9 10-4 and the bias of T is 0.3 averaged over various sampling probabilities, resulting in a reduction of 99%. In Figure 4b, observe that for three MovieLens datasets, the bias of T is 6 10-4 and the bias of T is 5 x 10-3, resulting in a reduction of 88%.
Additionally, we Lest a biased sampling procedure, where users who watch more movies tend to rale more movies as well (also known as "snowball effects" (Chen et al., 2020)). For each i =,,...,; n, we sample its entries with probability p; equal to C/d times the number of non-zeros in row i. We find that under this biased sampling procedure, the bias of T is 3 x 10-4 and the bias 7 is 5 x 10-3, resulting in a reduction of 93%. As explained carlier, these results stem from the variance reduction effect of T.
5.2 Recovery Results for Synthetic Matrix Data
We now examine one-sided recovery error by keeping the number of samples m fixed while varying p and d separately. Since the sample complexity on n grows linearly with d times log(d), the error should remain roughly constant as we fix m/d while slowly increasing d. We report simulation results under three regimes. For all three setting, we fix the number of rows n 1
First, we set the rank r 10, noise injection variance o² 1, and vary d from 10x to 5 10 and p from 2/d to 10/d. As shown in Figure 5a, the estimation error remains flat for different values of p and m, which suggests that sample complexity does not grow with d, after adjusting for m/d.
Second, we set the sampling probability p- = 2/d, o² = 1, and vary from 10x to 5 x 10x and r from 5 to 25. As shown in Figure 5b, the estimation error remains flat for a given rank r, as d grows.
Third, we set the dimension d 10 p= 2/d, r 10, while varying o² from to 50. We find that as o² increases, the estimation error grows lincarly, as shown in Figure 5c. This is consistent with our bounds in Theorem 3.4, where grows proportionally with o².
14

---

Published in Transactions on Machine Learning Research (01/2026)
10
P: Uniform
Uniform T: Non-unilorm
T (Hájek)
T (ITT)
10
100
T. Non-uniform
Bias
Bios
10-2
10-2
10-3
10
10-1
p p= 0.01
p= 0.05
p= U.i C=10 C C= 50
C 100
20M
25M
32M
-00
2551
7IL 32M
m
mn
11
m..
(a) Comparing the bias of T with the bias of
(b) Comparing the bias of 7 with the bias of
Figure 4: Comparing the bias of the Hájek estimator vs. the Horvitz- Thriitz-Thoopoon estimator, measured as the mcan squared error between f (or T) and T on the index set Q. Figure 4a: Reporting the bias on synthetic matrix data with n = 10 and d = 10 Figure 4b: Reporting the bias on MovieLens datasels. We consider both uniform sampling with probability p as well as sampling C entries from each row. In Figure 4b, we further consider non-uniform sampling sctting, where the sampling probability at cach row is proportional to the nunber of nonzero entries in that row.
0.011
0.011
0.6
0.009
0.010
0.4
Eror
Error
Error
0.007
0.009
0.2
0.005
0.008
0.003 2
0.007 2
0.0
3
10
20
30
40
50
d
x10
d
x10
0
(a) Varying d for different p
(b) Varying d for different
(c) Varying noise variance
Figure 5: Illustration of the recovery error of Algorithm 1. Figure 5a: We vary the sampling probability and find that the error roughly stays constant after we also fix the number of samples. Figure 5b: We similar
results as we fix the number of samples but vary the rank of the underlying malrix. Figure 5c: We gradually increase the noise level and find that the estimation errors also increase. The reported results are aggregated based on five independent runs.
Similar results on synthetic data are further illustrated in Figure 7 (See Appendix C.2). We see consistent trend in which our approach reduces the recovery error across different settings of C (expected number of entries at each row) and n, compared to all three baseline methods, including alternating-Gi and muclear norm regularization. In the setting with the fewest observed entrics (C 2 and 10*), HÁJEK-GD achieves the lowest error of 0.10, representing 70% reduction compared nuclear norm regularization, which yields an error of 0.48. In the setting with C = 10 observed entries per row, HÁJEK-GD achieves an error of 0.06, outperforming alternative-GD and nuclear norm regularization, whose error rates are 0.09 and 0.17, respectively.
lompte---AALS.
the closest baseline,
5.3 Recovery Results for Real-World Matrix Data
Next, report the results three real-world datasets in Table 3. We find that our approach consistently achieves the lowest error for recovering T and M, compared to two bascline methods. For onc-sided matrix completion, HÁJEK-GD improves upon alternating-GN by 42% and softlmute-AAL by 59% on average. For recovering the entire matrix, our approach reduces recovery error by 21% relative to alternating-Gi and by 38% relative mopttetAt,, averaged over the three datascts. In particular, on the Amazon reviews dataset, HÁJEK-GF reduces estimation of T by 59% and recovery of M by 21% % relative the best performing baseline.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: Results from applying our approach on three real-world datasets with up to 32 million entries, as compared with two baseline methods, namely alternating-Co and softImpute-ALS. We report both the recovery errors and the running time (measured in the number of seconds when evaluated on an Ubuntu server). We run cach experiment with five random sceds to calculate the mean and the standard deviation.
Dataset
Moviel.ens-32M Amazon Reviews
Genomes
MovieI.ens-32M
Amazon Reviews
Genomes
T
One-sided recovery error
Running time
AlternatingGD
9.1 x 10-3
3.3 x 10-3
7.0 x 10-5
7.2 x 1002
4.8 x 10a
23.2
Sof
9.1 x 10
3.2.0.3 10-3
1.940.5 x 10-
6.344 x 104
5.1_0.3 x 103
959.8440
Algorithm
4.7 x 10-3
1.3 1.3 10.2 x 10-3
5.8 x 10
1.3 x 102
2.8 L0.2 x 102
1.3
M
Root mean squared recovery error
Running time
AlternatingGD
1.7
2.6
0.2g0.1
5.7 x 10
2.2 x 10a
23.2+5.0
SoftImuutAALS
1.4g0.1
3.3
0.4
6,344 x 10*
4.8_0.3 x 10
959.9
Algorithm 2
1.110.1
1.940.1
0.2
5.610.1 x 102
2.8 X 10
14.7
An important design in HÁJEK- GD the use of an incoherence regularization penalty of R(-) (see Section 3.2). We now describe the results from varying regularization paramcter À and the threshold a (used in the truncation of R(-)), part of R(-), on synthetic dataset with p 10/d and also the Amazon Review dataset. We vary À between 10-1, 10-3, 10-2, and we vary between 10-5, 10-1 10-3 10-2, 10-1 We find that on synthetic datasets, the lowest recovery error is achieved when À = 10 and a = 10-3 leading to an error of 2.8 x 10-3, compared to the error rate of 3.6 x 10-3 without using the regularization. In other words, we can reduce the recovery error by 22%.
On the Amazon Reviews dataset, the lowest recovery error achieved when À 10-2 2 and a = 10-1, at 1.3 10-?, compared to 2.7 10-2 without using the incoherence regularizer. As result, incorporating the incoherence regularization penalty of R(-) reduces the recovery error by 52%.
Runtime scaling as the dimensions grow. Below, report experiment results to illustrate the runtime scaling of HÁJEK-GD as a function of both d (mumber of columns) and |S]] (mumber of nonzero entries in F)) In particular, we consider both synthetic data and real-world data. In Figure 6a, we illustrate the rumning time (measured in seconds) as a function of the dimension d. We also vary C to ensure the robustness of our findings. In Figure 6b, we illustrate the running time from HÁJEK-GD on the MovieLens datasets with various sampling probabilities p. We find that the runtime of HÁJEK-GU ADD scales near linearly with the number of observed entries Ω. It is an interesting question to precisely analyze the runtime scaling of gradient descent in low-rank matrix completion. See, e.g., several recent works for references (Li et al., 2020b; Ju et al., 2022; 2023; Ma & Fattahi, 2024; Zhang et al., 2024).
6 Related Work
The problem of recovering low-rank matrix from a few potentially noisy entries has a rich history of studies in the literature, spanning a wide variety of areas, including collaborative filtering, machine learning, and compressed sensing (Candes & Plan, 2010). One of the earliest results from the matrix completion literature involves a maximum-margin matrix factorization approach (Srebro et al., 2004), which minimizes the reconstruction error plus a trace norm penalty on the low-rank factors. Under an incoherent condition on the low-rank factors plus joint incoherence condition, Candes & Recht (2012) show that exact recovery of the unknown matrix is possible by minimizing the nuclear norm of the reconstructed matrix, subject to equality constraints on the observed entrics. The joint incoherence condition on the factors is further shown to be unnecessary (Chen, 2015). Ding & Chen (2020) sharpen the known bounds through leave-one-out analysis, which leads to a convergence guarantee for projected gradient descent on a constconstr formulation. Complementary to these results, Zhang et al. (2019b) show error bounds on the muclcar norm regularizod objective, relative to the error of the best rank-r approximation of M. Further references about different. algorithmic approaches to matrix completion can be found in the survey by Sun (2015).
16

---

Published in Transactions on Machine Learning Research (01/2026)
1.3
160
p= 0.8
1.0
120
p= 0.5
Seconds
# Seconds
p= 0.2
0.7
80-
#:
0.4
40
0
0.1
0.2
0.4
0.6
0.8
1.0
0.1
0.5
0.9 ||]
1.3
d
x10
x10
(a) Runtime on synthetic data by varying d
(b) Runtime on real-world data by varying [
Figure 6: Illustration of the runtime scaling of HÁJEK-GD d and m increase. Figure Ga: We vary the dimension d and find that the runtime of HÁJEK-GD increases roughly linearly in d as d increases. To ensure the robustness of our findings, we report the results for three values of C' (recall that the sampling probability p p= C/d). Figure 6b; We vary the number of entries m between three MovieLens datasets and find that the runtime of HÁJEK-GI GD increases as m increascs. The reported results are based on five independent runs.
Recent literature has studied the optimization geometry of the loss landscape of matrix completion using rank-constrained first-order optimization methods (Sun & Luo, 2016; Ge et al., 2016). Provided with a regularization on the incoherence (balance between the rows of low-rank factors), it is possible to prove that all local minimum solutions of the matrix completion reconstruction error objective are also global minimum (Ge ct al., 2017). By carcfully analyzing the local goometry of the matrix factorization objective, it is possible to achieve exact recovery under the incoherence condition (Sun & Luo, 2016). Provided with finding local minima in
this characterization, one can obtain convergence rates through the literature nonconvex optimization problems (Nesterov & Polyak, 2006; Wang et al., 2019). In particular, it is known that many optimization algorithms, including cubic regularization, trust-region methods, and stochastic gradient descent, can efficiently find a local minimizer. See a recent textbook on first-order and stochastic optimization methods for further references (Lan, 2020).
There is a closely related problem of low-rank matrix recovery, whose aim to reconstruct low-rank matrix based on linear system of measurement equations of the unknown matrix (Recht et al., 2010). The optimization landscape of low-rank matrix recovery from random linear measurements in the presence of arbitrary outliers is studied by Li et al. (2020a). Besides, there are studies on the dynamics of gradient descent assuming the factorization is over par-aattmteeizee (Li et al., 2018; Ma & Fattahi, 2024). Additionally, matrix factorization has connections to two-layer neural networks and random matrices, which have inspired studies on the dynamics of deep linear networks (Li et al., 2018). Another related problem is nonnegative matrix factorization, which has also been studied for "tall-and-skinny" matrices (Benson et al., 2014). There is also a line of work on the recovery of a low-rank plus sparse matrix (Hsu et al., 2011). Another plausible direction in the ultra-sparse sampling setting is to instead recover a low-rank approximation of the underlying matrix from just (n)) samples. See Gamarnik et al. (2017) for further references of this setting.
The one-sided matrix completion problem has been recently formulated and studied in the special case where each row contains only two randomly observed entries (Cao et al., 2023). This work complements their paper in four aspects. First, their sample complexity bound focuses on the setting where every row has two observed entries, while our result applics to more general settings that allow for any constant C. Sccond, their result builds on proof techniques for matrix completion with nuclear norm regularization penalty. By contrast, our result applies to wradient-based optimization algorithms, which are faster and easier to implement in practice. Third, our result builds on the incoherence assumption from the MC literature. Finally, we explore the use of one-sided matrix completion for full matrix completion and find that this can also lead to improved results for imputing missing entries of the full matrix.
First-order stochastic gradient methods also have applications in largo-scale matrix completion (Mackey et al., 2011). In particular, one can apply stochastic gradient updates in an asynchronous protocol, making suitable for distributed platforms (Recht & Ré, 2013). Besides the nuclear norm mininization approach, alternating
17

---

Published in Transactions on Machine Learning Research (01/2026)
minimization such as alternating least squares and low-rank matrix factorization are also widely used in practice (Hastie et al., 2015). More recently, Wang et al. (2023) consider a low-rank matrix factorization approach to private matrix completion. Assuming the M matrix is indeed of low rank, their work shows a sample complexity bound that scales lincarly in dimension. The difference between our work and this work is that, we focus on an ultra-sparse sampling regime where there are only O(z) randomly sampled entries, rendering the accurate recovery of the entire matrix information theoretically impossible. In light of our new results, it may also be worth resisting the dynamics of zeroth-order and first-order methods for nonconvex optimization (Ghadimi & Lan, 2013; Sun, 2020) when high rates of noise are added during each iteration.
The idea of using the estimated probability to reweight population statistic is known as the Hájek estimator (Hájek, 1971). It has often been used to tackle sparse and non-random sampling data (Särndal et al., 2003). This estimator is particularly effective for tackling sparse observational data such as in experimental design (Xiong et al., 2024). It is known from the causal inference literature (Hirano et al., 2003) that the Hájek estimator incurs lower variance than the Horvitz- Thomiiz-T estimator, which weights each observation inversely with the true probability. To our knowledge, the analysis of this estimator for one-sided matrix completion appears to be new for the matrix completion literature. Unlike conventional results (Hájek, 1971) where the Hájek estimator is shown to be asymptotically unbiased, for onc-sided matrix completion, we find that it is unbiased even in the finite sample regime. Recent work (Bai & Ng, 2021) develops the estimation of counterfactual when potential outcomes follow a factor model of block-missing panel data. Xiong & Pelger (2023) and Duan et al. (2023) develop the inferential theory for latent factor models in large-dimensiona panel data with general non-random missing patterns by proposing a PCA-based estimator on an adjusted covariance matrix.
7 Conclusion
In this paper, we study the problem of matrix completion in an ultra-sparse sampling regime, where each row, only a constant number of entries are observed in each row. While recovering the entire matrix is not possible, we focus on the one-sided matrix completion problem. We apply the Hájek estimator from the econometrics literature to this problem, and present a thorough analysis of this estimator. We demonstrate that this estimator is unbiased and achieves a lower variance compared to the Horvitz-Thompson estimator. Then, we use gradient descent to impule the missing entries of the second-moment matrix and analyze its sample complexity under a low-rank mixture model. The sample complexity bound is optimal in its dependence on the dimension d. Our result applics to the case of observing C entries at cach row, resolving an open question from prior work.
Extensive experiments on both synthetic and real-world datasets demonstrate that our approach is more efficient for sparse matrix datasets compared to several commonly used matrix completion methods. Ablation studies validate the use of a low-rank matrix incoherence regularizer in the algorithm.
Our paper opens up several promising avenues for future work. First, it would be valuable to theoretically analyze the sensitivity of the Hájek estimator to noise injection, which could inform the design of privacy- preserving matrix completion algorithms in the ultra-sparse sampling regime. Second, our current sample- complexity bound exhibits high dependence on the rank of the latent factor model; improving this rank dependence remains another interesting open question. It would also be interesting to understand to what extent linear sample complexity of n relative to d can be achieved under ultra-sparse sampling, by further relaxing the assumptions required in our analysis. Finally, extending our framework to other forms of panel data such as low-rank tensors or low-rank mixture models, is another promising avenue left for future work.
Acknowledgment
Thanks to Steven Cao, Daniel Hsu, Oliver Hinder, and Jason Lee for discussions about this work during various stages. We appreciate the editors and the anonymous referees for their constructive comments and suggestions. The work of H. R. Zhang and Z. Zhang supported in part by NSF award ITS-2412008. The views and opinions expressed in this paper are solely those of the authors and do not necessarily reflect those of the NSF.
18

---

Published in Transactions on Machine Learning Research (01/2026)
References
Susan Athey, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi. Matrix completion methods for causal panel data models. Journal of the American Statistical Association, 116 (536):1716-177))) 2021.
Jushan Bai and Serena Ng. Matrix completion, counterfactuals, and factor analysis of missing data. Journal of the American Stalistical Associalion, 116(636) 1763, 2021.
Austin R Benson, Jason D Lee, Bartek Rajwa, and David Gleich. Scalable methods for nonnegative matrix factorizations of near-separable tall-and-skinny matrices. Advances in Neural Information Processing Systems, 2014.
Nicolas Boumal and Pierre-antoine Absil. Rtrme: riemannian trust-region method for low-rank matrix complction. Advances in neural information processing systems, 24, 2011.
Lawrence D Brown and Mark G Low. Asymptotic equivalence of nonparametric regression and white noise. The Annals of Statistics, (366))2384423888 1996.
Emmanuel Candes and Benjamin Recht. Exact matrix completion via convex optimization. Communications of the ACM, 55(6):111-119, 2012.
Emmanuel Candes and Yaniv Plan. Matrix completion with noise. Proceedings of the IEEE, 98(6):925 936, 2010.
Steven Cao, Percy Liang, and Gregory Valiant. One-sided matrix completion from two observations per row. International Conference on Machine Learning, 2023.
Justin Chen, Gregory Valiant, and Paul Valiant. Worst-case analysis for randomly collected data. Advances in Neural Information Processing Systems, 33:18183-18193, 2020.
Yanxi Chen, Cong Ma, H Vincent Poor, and Yuxin Chen. Learning mixtures of low-rank models. IEEE Transactions on Information Theory, (177:461334636, 2021.
Yudong Chen. Incoherence-optimall matrix completion. IEEE Tronsactions on Information Theory, 61(5): 2909 2923, 2015.
Yuejic Chi, Yue Lu, and Yuxin Chen. Nonconvex optimization meets low-rank matrix factorization: An overview. IEEE Transactions on Signal Processing, 67(20):5239 5269, 2019.
Lijun Ding and Yudong Chen. Leave-onc-out approach for matrix completion: Primal and dual analysis. IEEE Trunsactions on Information Theory, 66(11):7274-75 7301, 2020.
Edgar Dobriban and Yue Sheng. Wonder: Weighted one-shot distributed ridge regression in high dimensions. Journal of Machine Learning Research, 21 (66)):11-52 2020.
Zhaorui Dong, Yushun Zhang, Jianfeng Yao, and Ruoyu Sun. Towards quantifying the hessian structure of neural networks. ar preprint orXiv:2505.00 02809, 2025.
Junting Duan, Markus Pelger, and Ruoxuan Xiong. Target pea: Transfer learning large dimensional panel data. Journal of Ecmometricss pp. 105521, 2023.
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and TrendsD in Theoretical Computer Science, 9(3 4):211 407, 2014.
Jianqing Fan, Yuan Liao, and Martina Mincheva. Large covariance estimation by thresholding principal orthogonal complements. Journal of the Royal Statistical Society Series B: Statistical Methodology, 75(4): 603 680, 2013.
David Gamarnik, Quan Li, and Hongvi Zhang. Matrix completion from O(n) samples in lincar time. In Conference Learning Theory, 2017.
19

---

Published in Transactions on Machine Learning Research (01/2026)
Rong Ge, Jason D Lee, and Tengyu Ma. Matrix completion has spurious local minimum. Advances in
Neural Information Processing Systems, 2016.
Rong Ge, Chi Jin, and Yi Zheng. No spurious local minima in nonconvex low rank problems: A unified geometric analysis. In International Conference on Machine Learning, 2017.
Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM Journal Optimization, 23(4)22341-2366 2013.
Jaroslav Hájek. Comment on "an essay on the logical foundations of survey sampling, part one". The foundations of survey sampling, 236, 1971.
Trevor Hastie, Rahul Mazumder, Jason D) Lee, and Reza Zadeh. Matrix completion and low-rank svd via fast. alternating least squares. The Journal of Machine Learning Research, 16(1):3367 3402, 2015.
Keisuke Hirano, Guido W Imbens, and Geert Ridder. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(44:1161-1189, 2003.
Daniel G Horvitz and Donovan J Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American statistical Association, 47(260):663 685, 1952.
Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation. arXiv preprint (rXi::243300352, 2024.
Daniel Hsu, Sham M Kakade, and Tong Zhang, Robust matrix decomposition with sparse corruptions. IEEE Transactions on Information Theory,
2011.
Haotian Ju, Dongyue Li, and Hongyang R. Zhang. Robust fine-tuning of deep neural networks with hessian- based generalization guarantees. In International conference on machine learning, pp. 10431-10461. PMLR, 2022.
Haotian Ju, Dongyuc Li, Ancesh Sharma, and Hongyang R. Zhang. Generalization in graph neural networks: Improved pac-bayesian bounds on graph diffusion. In International conference on artificial intelligence and statistics, pp. 6314 1841- PMLR, 2023.
Mladen Kolar, John Lafferty, and Larry Wasserman. Union support recovery in multi-task learning. Journal of Machine Learning Research, 12(7), 2011.
Guanghui Lan. First-order and stochastic optimization methods for machine learning, volume 1. Springer, 2020.
Yuanxin Li, Yuejie Chi, Huishuai Zhang, and Yingbin Liang. Non-convex low-rank matrix recovery with gradient descent. Information and Inference: A Journal of the
arbitrary outliers via IMA, ((2):289-33) -25, 2020a.
Yuanzhi Li, Tengyu Ma, and Hongyang R. Zhang. Algorithmic regularization in over-parameterized matrix sensing and neural networks with quadratic activations. In Conference On Learning Theory, 2018.
Yuanzhi Li, Tengyu Ma, and Hongyang R. Zhang. Learning over-parametrized two-layer neural networks beyond ntk. In Conference on learning theory, pp. 2613-2682. PMTRR 2020b.
Ziqi Tiu, Yu-Xiang Wang, and Alexander Smola. Fast differentially private matrix factorization. In Proceedings of the 9th ACM Conference Recommender Systems, pp. 171 178, 2015.
Jianhao Ma and Salar Fattahi. Convergence of gradient descent with small initialization for unregularized matrix completion. In Conference on Learning Theory, 2024.
Lester Mackey, Michael Jordan, and Amect Talwalkar. Divide-and-conquer matrix factorization. Advances in Neural Information Processing Systems, 2011.
20

---

Published in Transactions on Machine Learning Research (01/2026)
Yurii Nesterov and Boris T Polyak. Cubic regularization of newton method and its global performance. Mathemalical Programming, 108(1):177-28) 205, 2006.
Benjamin Recht. A simpler approach to matrix completion. Journal of Machine Learning Research, 12(12), 2011.
Benjamin Recht, and Christopher Ré. Parallel stochastic gradient algorithms for large-scale matrix completion. Mathematical Programming Computation, 5(2):201 226, 2013.
Benjamin Recht, Maryam Fazel, and Pablo A Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM review, 52(3):471 501, 2010.
Carl-Erik Särndal, Bengt Swensson, and Jan Wretman. Model assisted survey sampling. Springer Science & Business Media, 2003.
Nathan Srebro, Jason Rennie, and Tommi Jaakkola. Maximum- margin matrix factorization. Advances in Neural Information Processing Systems, 2004.
Ruo- Yu Sun. Optimization for deep learning: An overview. Journal of the Operations Research Society of China, 8(2):249 294, 2020.
Ruoyu Sun. Matriz completion via nonconver factorization: Algorithms and theory. PhD thesis, University of Minnesota, 2015.
Ruoyu Sun and Zhi-Quan Luo. Guaranteed matrix completion via. non-convex factorization. IEFE Transactions on Information Theory, 62(11):6535 6579, 2016.
Lingxiao Wang, Boxin Zhao, and Mladen Kolar. Differentially private matrix completion through low-rank matrix factorization. In International Conference on Artificial Intelligence and Statistics, 2023.
Zhe Wang, Yi Zhou, Yingbin Liang, and Guanghui Lan. Stochastic variance-reduced cubic regularization for nonconvex optimization. In The 22nd International Conference on Artificial Intelligence and Statistics, 2019.
Ruoxuan Xiong and Markus Pelger. Large dimensional latent factor modeling with missing observations and applications to causal inference. Journal of Econometries, 233(1):271-300 2023.
Ruoxuan Xiong, Alex Chin, and Sean J Taylor. Data-driven switchback experiments: Theoretical tradeoffs and empirical bayes designs. arXiv preprint arXiv:2406 0666666, 2024.
Hongyang R. Zhang, Vatsal Sharan, Moses Charikar, and Yingyu Liang. Recovery guarantees for quadratic tensors with limited obsorvations. In International Conference on Artificial Intelligence and Statistics, 2019a.
Hongyang R. Zhang, Dongyue Li, and Haotian Ju. Noise stability optimization for finding flat minima: A Lessian-based regularization approach. Transactions on Machine Learning Research, 2024.
Lijun Zhang, Tianbao Yang, Rong Jin, and Zhi-Hua. Zhou. Relative error bound analysis for nuclear norm regularized matrix completion. Journal of Machine Learning Research, 20(97):1-22, 2019b.
Zhenshuo Zhang, Minxuan Duan, Youran Ye, and Hongyang R. Zhang. Scalable multi-objective and meta reinforcement learning via gradient estimation. arXiv preprint arXww:5111.1277, 2025.
Pini Zilber and Boaz Nadler. Gumr: A provable one-line algorithm for low rank matrix recovery. SIAM journal on mathematics of data science, 4(2):909 934, 2022.
21

---

Published in Transactions on Machine Learning Research (01/2026)
A Proof of Theorem 3.2
First, we present the proof of the unbiasedness of the Hájek estimator on the observed set Q.
Proof of Lemma 3.1. Suppose that (TT = k, for any 2 1. Then, conditioned on (1 Digs = k. the expectation of Tig is equal to
Tig (T" Dis =k =
M M)
=k
k
Ma. Ma
(TT =k _
k
Σ Pr Laitas 1 = MM, -
Notice that for a fixed a, the event that la, = are pairwise independent for all a = 1.2..... Given that out of the n pairs are chosen, the probability that q is chosen equal to (x=1)/(() = k/n. Thus, the above is equal to
Ma. = Ts
which concludes the proof of equation (3). To finish the proof that f unbiased, notice that
E
Ti
* o0 =
Pr (1T)) = (1 1as + 0 E Teis (1 (IT)) k)
k=1
Σ Pr 1) = - D) z0]
Túg Ti.j
k=1
which concludes the proof of equation 2.
Next, present the proof of Theorem 3.2, which provides first-order approximation to the variance of the Hájek estimator on Ω.
Proof of Theorem 3.2. For equation (4), we use Taylor's expansion on [ar[T For case of presentation, let denote Ar, = (M M),,, and let Bt, ((TT))) 1)is, for any sii d. Thus, conditioned on the event that BA, # 0, the (i, i)thh entry of T, given by Ti., is equal to At,s/ Bi..
By Taylor's expansion, we approximate the ratio statistic using the delta method as:
Ari E [AA, B E B.,]
1
E [A,,
EE [B,]] (Ar, - E[A,)) (E [B.,))
(Br, E [B,,)) 64,5.
(18)
where 6,, is the order of the variance of AG, and B which ((1((m))). Thus, based on the first-order expansion, the variance of Ars is approximated by: Be,
A
Var [Arl
(E (B [Bal
2 [A,d Cou(A Br.)
1
Var
+0
(19)
B
[E ()
(E (B,))
[E
np
where the error term is based on the approximation error that stems from fy in equation (18).
Now we look into simplifying equation (19). Based on the definition of A Brss, we get the following facts, for any = 1,...,d: d:
E [B,s]] = np, Var [Bad] = np(1 -p);
(20)
[A,,,] p(M TMTTM M)i,, CoulAi, Bi.s) (p -D- M)
(21)
22

---

Published in Transactions on Machine Learning Research (01/2026)
To verify the above results, let us introduce Bernoulli random variable X3, which is equal to with probability or 0 with probability 1 p, for k = 1,,,,-. 1,,,...,,. Thus, we can write
A Mt Xas Xs, and B Xx.
k=
As a result,
CoulArs Br,s] = E [A, - E A E [B,d
]
E
Xx
Σ Xx
- 'Mp((M MM)
)
E
X
Σ Mr
nppp(M (MT MMss (note that x2 = Xx))
n
Mh. (p + (n (m-1))²) - 2pp (M* M)
(p (p-p²) (M M).
Therefore, by plugging in the results from equations (20) and (21) back into equation (19), we obtain an approximate variance the ratio statistic as:
Var[Ar, p²(M M) mp(1 p) 2p(M Mys (p (p- (MM M) n²p²
n ntp²
n²pp
Var[Ard n2yp
(M M)
(22)
n2pp
This applies to any diagonal entries of T. In particular, notice that the first term of equation (22) is precisely the variance of T.,, which reweights the observed entries based the true probability. This leads to the following variance estimate for the diagonal entries:
V Var [ar[Ard)
(1 (M M),
(23)
p22p2
n
Finally, we write down the variance of A as follows:
Var Var A = p(1 -
Σ 1m1
M
(24)
Notice that the probability that Br, is equal to zero is at most O(d-9). We can add this error analysis into the above proof, which does not affect the order of the error term. This concludes the proof of equation (4) in Theorem 3.2 regarding the variance approximation of the diagonal entries.
Next, we consider the case of off-diagonal entries equation (5). The probability that an off-diagonal entry of T is nonzero is
11 (1-
which is approximately up² = CO d/2/d. Thus, conditioned on the fact that (1 D).s * 0, the dominating event is when (I" = The error to this, which when there are least two nonzero entries in the sum,
is less than
Pr[([ 1). 2 Pr[(TT T).s #
as np².
When the (i. entry of / [T] / is equal to one, then T. (M M) The variance of Ti,,, conditioned on (T) = 1, thus given by
Σ Me Mr,MMes ))²
2
Mk, -
23

---

Published in Transactions on Machine Learning Research (01/2026)
After simplifying the above formula, we reach the conclusion stated in equation (5). The proof is thus completed.
Remark A.1 (Potential applications of the Hájek estimator). Beyond one-sided matria completion, Häjek estimators may serve as useful tool in other high- digeesidmensi estimation problems given few observational samples. In tensor completion (Zhang et al., 2019a), naive proppes-ropynstty weighting estimators can exhibit large variance due to rare obsercations, whereas self-normalization can stabilize the estimation of low-rank factors given ulbra-su ulbre-pprse samples.
Similarly, in off-policy reinforcement learning (Zhang al., 2025), importonce weighted estimators are widely used to correct for distribution mismatch but are often unstoble in sparse-reward settings. Hájek-style eslimalors provide principled variance-reduction mechanism and may be useful for estimating value-funclos matrices under partial trajectory observations.
B Proof of Theorem 3.4
We analyze the sample complexity of Algorithm 1. In particular, we will analyze the loss function ((X), including the incoherence regularization penalty R(X). See equation (8). Without loss of generality, let us
assume that ||U|| = within this section. This also implics mmax (U) 2 2 a (U).
A key part of the proof is analyzing the bias of T (relative to T). Let N 12d denote the bias of the Hájek estimator, where
Nt. T..j -Taj v(i,)) E Ω Ni. = 0, v(i,j) & Ω.
Here is road map of this section:
First, in Appendix B.1, derive upper bounds the variance of the Hájek estimator, the observed set Ω.
Second, in Appendix B.2, we derive a concentration bound that applies to the weighted projection Po- By similar argument, also derive the spectral norm of the noise matrix N.
Third, Appendix B.3, we derived the local optimality conditions from analyzing the loss function ((-), and then we argue that any local minimum solution of E(-) must satisfy the incoherence assumption with suitable condition number.
Fourth, in Appendix B.4, we use the incoherence assumption to reduce the local optimality conditions on the empirical loss to the population loss.
Finally, in Appendix B.5, we analyze the population loss and based on that, complete the proof of Theorem 3.4. The key is to use the regularization penalty R(X) to control the norm of each factor, leading to concentration estimates between the population loss and the empirical loss. We provide a detailed comparison between this work and the work of Go al. (2016) Remark B.12.
B.1 Proof of Lemma 4.1
In the following proof, we proceed to bound the variance of the diagonal entries î and also its off-diagonal entries grouped by each row.
Proof. Based on Lemma 3.1, we already know that T unbiased on Ω. Next, we derive the variance of the diagonal and off-diagonal entries of T separately, since
E (Tr. To,))
Var[Tagh, for any (i.j) E Ω.
(25)
24

---

Published in Transactions on Machine Learning Research (01/2026)
For the diagonal entries of T, with probability at least 1- Idd="),, all of them observed in Q. In particular, when n 2 d log d/e2, by the Chernoff bound and union bound, with probability at least - Od,-t), for all = ,,,,
60-1
[ Drs E [(TT)
np.
d
Thus, we can get that
60'-1
Aia np
Var [Txp]] 1
(1+3ey ++
V
Next, notice that
A1 Σ
Mr Xa. Xx.
(26)
R=1
Therefore, the variance of
Asi
is equal to
up
A
1
Var
Mk.i
npp
n
k=
max Sast u.o
3 3log(d/2) 2n
-
ui,
1
(by Hooffding's incquality)
np
r-
Σ
2
1
3 llo(dd/2 (d/2)
2
1 1
(27)
np
2n
1==1
1 p
p²
3 log (d/2)
1 1
np
d2
2n
which concludes the proof of equation can also show that
(11). Similar to this calculation, by applying Hoeffding's inequality,
A
1
Blog(d/2)
p
1 1+rl
np
r
11
3 log (d/)
1 1
(28)
d
2n
Next, examine the off-diagonal entries of T. From Theorem 3.2, the variance of the off-diagonal entries is approximated by the variance of single entry. Consider the case where we condition taking two overlapping entries from columns i and j. Denote the two entries as a1,02. We have that
+ 02 2
Var[ai] + Var[az]
V ar
+ Cou(ai,
4
Varlas] + Varlaz]
2
by the Cauchy-Schwarz inequality on the covariance. The probability that we observe three overlapping entries is less than O(d-3), 3). Since the size of Ω is ((d2g) O(d=1). The probability that such events happen is less than O(d-1). In summary, we have shown that for every (i.j) E Ω where i + j,
[ar(
Σ M MR.MM..) -TR.)
) T
Mr, M2, + O(d
25

---

Published in Transactions on Machine Learning Research (01/2026)
We now look at the sum of the variances of all the observed entries in the i-th row. Let the set be denoted by S = (j (7,5) 2 * it, for all i = .,2,..., d. We have
Ma. Mi.
(29)
JES,
| max 1=1
u7 "
log (d/2)
(by Hoeffding's inequality)
a js
Σ
u
u?
2n
JE8
1
max 1 3
3 Blog(d/2)
3 llo((d/2)
L q
al
1t r-1
(by Hoeffding's inequality)
-j=1
2d
2n
max 15
||u.| 1|
Σ un.
1 1
3log(d/2
1+
3 Blog(d/2
(relax ||u,]] by its max over 8)
r
2d
2n
S _4 d
1
3log(d (d/2)
(by the incoherence assumption on U)
2d
The last line uses the premise that the maximum & norm of factor is at most 1, and also 2 d. In particular, the above Hoeffding's inequality applies uniformly to all i with probability at least 1 - 0(d )) In summary, we have shown that
Σ
Var(Tigl
pq 1 3r/
3 log(d/2)
+
nq 1
2d
JES:
where we use the fact that with probability at least 1 - (d-,,150 s 2nq. Therefore, we conclude that equation (12) is true.
As a remark, one corollary we can draw from the above calculation is that
2222
3 log(d/2)
Var Ti,s
1 1+TV
+ ((d
(30)
d²
2n
This can be seen by following the steps from equation (29) for fixed pair of (i,j) E Ω.
Remark B.1. In the case that Mj is a noisy perturbation of one of the factors, we can extend the proof following equation (29). Another natural extension is to assume that every M; is a linear combination of the r factors, pr. This seems to a much more challenging problem, as one would need to design estimators to first disentangle the linear combination patterns. This left for future work.
Next, we build on Lemma 3.1 to examine the bias of T on the observed set Ω. Another corollary from the above variance calculation is the following bound on the bias of T. We consider the diagonal and off-diagonal entries separately.
Corollary B.2. In the setting of Lemma 4.1, suppose n 2 dr. Then, with probability at least O(d for every i 11,,...,d, the following must hold:
-2C- 32C-112log(d) log(d)
N1. S V
(31)
nd?
Natl 5 2p + (d-11/2)
(32)
JES.
26

---

Published in Transactions on Machine Learning Research (01/2026)
Proof. First, have that
Aii
-(1zeg
6C'-1
A
[Na.l
Ti,:
Ti,
Tip
1 ev
Ti,
Bi,i
d
np
A
60'-
Ari
Tai
+
np
d
np
By equation (28), the latter is at most
60
3 (d/()
8C 8C-1 20- log(d)
1
d d
2n
dnn
since S / d/o.///. As for the former, by equation (31), we have
A
Var[Ned c 6
Recall that A, is the sum of n Bernoulli random variables (ef. equation (26)), by Bernstein's inequality, with probability at least O(d (dd-)), 1) for all 1,2.....d, d,
Ari ni
War( No.)) ) log(d/2
3 (max, ug,,) log (d/2)
Túps S
n
3n
r log(1/2
dnrpp + log(d//
nd
for large enough values of d. Specifically, we have used the bound on the variance of Tye in Lemma 4.1, using equation (11). Notice that the second term is lower-order term relative to the first. This shows that equation (31) holds for large enough values of d.
For equation (32), based on the proof of Lemma 4.1,
Σ Nesl S JES,
15:1-
Σ
NZ
(by the Cauchy-Schwarz incquality)
jES;
2dq
No,
(since |S < 2dg)
jES,
puq
3 log blggdd/2) + Odd-))
2p2r2
/3log(d/2)
2dq
3rl
(33)
2d
d²
2d
where the last line is by applying Hoeflding's inequality to the sequence {N?, : E Si} (which are all independent from each other), and holds with probability at least d=1 over all possible i. In particular, the expectation on this sequence based on equation (12) and the maximum is based on equation (30). From the last line above, we conclude that equation (32) is true.
B.2 Proof of Lemma 4.2
Proof. By the weighted operation in Pg and the condition that all the diagonals have been observed, we can cancel out the diagonal entries between Pa) and qW. In the rest of the following, we focus on the off-diagonal entries. Let Rd denote unit vector. We have that
|| [PnnWW) gWIl2 = max T (Pa(W) - -W)))
=: ==|===
27

---

Published in Transactions on Machine Learning Research (01/2026)
Next, expand the right-hand side above
(Pa(W) qW)x
Σ
Σ
Wi -
Σ
Wiyzzy
im1
JES
Isisajji
Σ JES:of
Wiyzia
E
Wijnzy
i=1
IS/Sdjzi
We focus fixed i above. By Bernstein's inoquality, with probability at least 1 - O(d-)), we have
6 - 4g(1 9)
Σ
log(2d)
||
las max |ayl 1sjsd
| |zs| 44 q) (2g( + ||
aaal max IsiEd
Lzyl-
Above, we have plugged in the variance and the maximum values at the i-th row. As result,
Σ c و 10(1 9) log(2d)
Σ Iaa max Jzyl.
(34)
IsiEd
In particular, we have used the bound || S Sv/vd and the fact that Ee |x; S vd via the Cauchy- Schwarz inequality. Next, we notice that
a max Isisd
xzح S 1.
To see this, notice that if we look a particular pair Fi, Ty. Conditioned on fixed a; + +2 and |aya S X, |xyl S X, Jaral + |zyl maximized when x; zy. This implics that when E Jas] is maximized, all the non-zero r,'s must be equal to each other. Suppose there are k of them, and they are all equal to Then, we must have ka² < 1, < X. As result,
max |z kad S 1.
By rearranging equation (34), and using the fact that dg 2 4lo((2d) to relax the second term in equation (34), we have concluded the proof of equation (15).
With similar proof, we can derive bound on the spectral norm of N, stated as follows.
Corollary B.3. In the setting of Lemma 4.1, as long as q 2 the spectral norm of Ps(N) sotisfies:
4log(2d) then with probability least O(d-t),
d
22 log(2d)
log(d)
[Pa(N))22 S SoV
O aV nd?
(35)
dg
Proof. Let Id be any unit vector. Note that N symmetric matrix. Therefore, we expand T Na follows:
x T Nr =
N 2 +
Σ Σ Nijuzy
jES
28

---

2pr Published in Transactions on Machine Learning Research (01/2026)
By equation (31), Corollary B.2,
32C log
d
32C
1 2² log
e1 S eV
2 = VV
ndf
ndf
As for e2, notice that |Nasl (34)), we have that
T -Ti
s 2pr/d By following the proof of Lemma 4.2 above (cf. equation
2ur
62 S Vd
v 4q(1 g) log(2d) + d
Since dg 2 4 log(2d), the second term on the right less than or equal to the first term on the right. Put together, we can see that equation (35) is true.
Another implication is the inner product of two matrices under the projection operator.
Proposition B.4. Suppose || || z Q, for any two matrices X,Y
With probability at least 1 - O(d over the randomness of Y E Rdxd, we have
16 16log(2d)
- (Pa(X), 9Xx, (((x))) V
(36)
dq
Proof. With probability at least. - O(d-), the diagonal entries of (Pa(X), Y) are canceled out with that of (), As for the off-diagonal entries, let
Σ Σ JES,jjE
Y.yY - Σ Σ Y.
e2
Notice that for fixed i, whether E S, another j' S; independent events. Thus, apply Bernstein's 's inequality to each row, similar to the steps following equation (34), which leads to the bound on e2 stated in equation (36).
B.3 Local Optimality and Incoherence
First, we derive the first-order and second-order optimality conditions of ((X), accounting for the weighted projection Ps to offset the imbalance between diagonal and off-diagonal entries in Q.
Proposition B.5 (See also Proposition 5.1, Ge et al. (2016)). Suppose X is local optimum of ((X) (See equation (8)). Then, the first order optimality condition is equivalent to
2P +
(37)
The second-order oplimality condition is equivalent to:
VVERR (Po(VX +XVT), XV +VX NV, V²R(X))
2 2(Paff - -XX))VVVV
(38)
Proof. The gradient formula of equation (37) can be seen by directly taking the gradient on {(X), and separating the gradient from the squared loss vs the regularizer. As for the Hessian formula of equation (38), when V is sufficiently small, one has that Ve(X + V) [V((())V. Thus, we have that
VI(X+ +V)
=2Xx + +VV +VV +xx (X++) 2P) +) +AVR(X +V).
29

---

Published in Transactions on Machine Learning Research (01/2026)
By the second-order optimality condition, one has that V + + 0, when V tends to zero. Plugging this into the above, we get
vVv((++V)
=2(Po(VXT= +XV 2( - -
+ =(Pa(VX' X,V),, + +VX" (Xx +V) XV. te =(Pa(VXT + +XVT), + +XV 2PP(XX" - -T), + A VVV +
where e = O( denotes an error term. When V tends to zero, we have that equation (38) must be true for any V Rd based on the second-order optimality condition.
Next, we show that the regularizer of R(X) leads the row norms of X to be bounded by at most 20, which builds on the first-order optimality conditions above. As result, the first- and sccond-order optimality conditions for their population versions. Based on that, show that X must be bounded away from zero.
concentration tools to reduce
We begin by analyzing the effect of the regularizer. We show the following property, which results from adding the incoherence regularizer and can be derived based on the gradient of the regularizer.
Proposition B.6. The gradient of R(X) satisfies that for any Y RR
(Ix31) - ) 3 (X,, YTeT)
(VR(X).)) 4X
X
where e E Rd is the i-th basis vector. As consequence,
((V((X)); X.) 2 0, for every i = 1.,,......
We first notice that because of the regularizer, any matrix X that satisfies the first-order optimality condition must also satisfy the incoherence condition.
Lemma B.7. Let S; denote the set of observed entries in I at the i-th row, for i 1.2..... ., d. Suppose [S]] < 2dq, for any i 1,,2,..,, In the setting of Theorem 3.4, for any X that satisfies the first-order optimality of equation (37), have
max |X S max (2a, 2/r p(r 1)g/x).
(39)
Proof. Let it = arg max ||X be the index of the row that has the highest norm in X. Recall that X; IR" refers the i-th row vector of X. Suppose the i-th row of Ω consists of entries with index [ S, where S, is the set of indices in the i-th row that are observed in Q. If [Xxx S 2a, then equation (39) is true. Otherwise, let us assume that X, 2 2a.
We will compare the -t row of the left and right-hand sides of equation (37). First, we have the following identity based on the projection matrix Pa:
(Pa(f)x).. = (Pa(UUT + +)))), = (Pa(UU )). X No.,,
where the sub-index on it refers the i--th row of the enclosed matrix. Then, the (--nor of Pa(UUT) is at most
)
Σ U,)
(40)
JES,
Σ
||
|| S Σ pr JES, d
(by the incoherence assumption on U)
jES.
2prdg
2urq.
(by | S 2dg)
30

---

Published in Transactions on Machine Learning Research (01/2026)
Second, we look at the f--morm of the i"-th row of N. By using Corollary B.2, bound the Eg-norm of the left of equation (37) by
(Pa(T) ((((Pa(UU +I
mmxx |||l|
==1
$ (2pr + + + ((n-)) + x
(41)
where we use equations (40), (31), and (32) above, along with the fact that S. Next, we lower bound the norm of the right- hand side equation (37). We have that (Xa, Xx),,.
(aXX))X); - Σ
JES
Thus, by taking an inner product with X we get
(Pa((X))x)) Xx)=
Σ (Xir, X,) 20.
JESμ
Using Proposition B.6, we obtain that
(xx -X -a)² Ejes, Xx) ||x+1||
2 0.
(42)
It follows that
((Po(XXT)X) (AVR(X)); I(AA)(()))=
(by equation (42)
4X 11 a0 ||x
(by Proposition B.6)
IV Lee |
(since ||X,- || 2 2a)
Therefore, plugging in the equation above and the equation (41) into the first-order optimality condition (37). We obtain equation (39). Thus, the proof completed.
The condition that. |S|] S 2dg for all i can be shown via Chernoff bounds (recall that 1 (1 - p2))). To see this, notice that conditioned on row i, the event of whether j St (for any j between and but not equal to i) is independent. from each other. In expectation, the size of [Sa]] should be equal to dg. In the setting of Theorem 3.4, dq is at least log(d). Therefore, with probability at least 1 O(d ?), |S,] 2dg, for all i 1,2.....d. d.
B.4 Reduction of Local Optimality Conditions
One consequence of the above result is the following population version of the first-order optimality condition.
Lemma B.8. Suppose S Vu(r
(r + 1)//A and X 2 dg/8. In the setting of Theorem 3.4, with high probability
over the rundomness of Qt for any E RRdxr that satisfis the firs--order optimality condition (37), we have that
|X|| I (max(()) Vo, where 8 3//33/32222 22 log(2d)/(dg)
(43)
222d log (2d) (2d)
| x x I 420 (U) (
(44)
dq
Proof. By the incoherence assumption on U, we have that
3/2
| 11.0 W d
pr | <
|
31

---

Published in Transactions on Machine Learning Research (01/2026)
by applying the Cauchy- Cauchy-Schwarz inequality. By setting pr | ||r in Lemma 4.2, from equation (15) we get
Pa(UUT) - qUE x 1 PAlUU )x - qUE 11 ||X||
S s ||
(45)
for 51 1og
Next, by the incoherence condition on X in equation (39). we have that
|1 v 4u(r + 1)g
Using equation (15) from Lomma 4.2 with v = 4p(r + 1)dg/A, A, we have
[P(xX - - X11 s || qxxx ||11
- q62 1x
(46)
where
82 4 1)
dogg log((d))> S 32p(r 1) (ol/oog((2)(()))))
for À 2 dg/8.
We also need to add the error induced by N, given by
[ F S [N)]l)) ||X||
S 9ōg |
(47)
where 83 V 22 log (g(d))/(dg) + - and the bound on the spectral norm of Pa(N) is shown in Corollary B.3.
Now, by plugging equations (45), (46), and (47) into the first-order optimality condition (ef. equation (37)), we have shown that
| x xxxx - 1111
(48)
where 8= 8, + 82 83.
When || S max (U) both equations (43) and (44) are proved. Otherwise, by the triangle inequality
a alIUU X F 2 (a(UUT)x(1 qô, ||
Pa(XX )x XR(X) 1- qō1 || (|Pa(N))) |X
V Pa(XX x - 4(0; + +50) ||
2 |||xx xI| (6) + 6g + 63) |X
(49)
where the second step is by equation (42) and the spectral norm bound on N, and the last step is again by the triangle inequality. Then, we have that
LOUT XIp < |OUT 1.. -1X1I < (amax (U)) |
On the other hand, |XxX x 1 Ele og where a; is the i-th singular value of X, for i = 1,2,.... Notice that
Σ
Σ o <
of
+
r
Σ of
1=1
by Hölder's inequality. Thus, plugging in this fact into the above equation (49), we obtain that
| Σ of Sr. Fmax (U))² (51 + 82 + 5z)r Sr. (o max (U))2
which implics that || S 1/2 o max (U) + 812 This concludes the proof of equation (43), which implies equation (44) based equation (48).
32

---

Published in Transactions on Machine Learning Research (01/2026)
Second, we look at the second-order optimality condition and show that this condition implics that the smallest singular value of X is lower bounded from below.
Lemma B.9. In the setting of Theorem 3.4, suppose X satisfies the second-order optimality condition (38) and equation (39). Suppose 4Kr AKT/p/// and X p(r 1)g/a?.
When q 2 ox 1og(e) for fixed constant > 0, and d is large enough, with probability at least - O(d=1) over the randomness of Ω2 we have that
d
(omin (x))² 8
mii (U))
(50)
Proof. Let A = {i: || I at be the index set of row vectors of X whose l2 norm is at most a. Let UA be the matrix that has the same i-th row as the i-th row U for every A, and 0 elsewhere. Let v E R' be a unit vector such that ||Xı|| min (X).
We show that
minn (UA) 2 (1 (32k)) (32k)) Jm (U).
Let B (1,2,.... ..... d) \A. Since for any i E B, |X 2 a, we have that |B|a² < || I (m (U)) + by equation (43). It also follows that B I mmaax r 8)/a².
Next, we have that
m (UA)
O (U - Un)
min (U)
T (UB)
min (U) |UB F
min (U) - V B rrudd
(o (Qmax(U)2r + 8)mF
z Omin (U) mmin(U)
da²
2 (1 1 1)min(())
(51)
where the last line is bocause a 2 4kr 4Kr /u/d. d. It also follows that UA has column rank equal to
Notice that there must exist unit vector un in the column span of UA such that ||X [XTuall S Amin (X). To see this, into the orthogonal subspace. Since U. is unit vector, we can also write u us U for some 3 E R" 1
simply decompose the column span of UA into one in the column span of X and another
where || S Gman (Ua) S 30) Further,
(máx |[aeell) 1=1
4 ur/d 30 (U)
| 5
|||]|| S
Next, we plug in V T in the sccond-order optimality condition of equation (38). Note that since uA is in the column span of UA, it is supported on the subset of columns spanned by A, and therefore (V2(())VV = 0.2 Therefore, the term about the Hessian of the regularization term in equation (38) is equal to zero. Thus, by taking V = unv T in equation (38), we get that
(Pa(ua(Xo)T + XeuJ), (A(o)) + Xvun) 2 2u PaJUU - X NuA
(52)
Note that
[Xell < (máx X+l) |||| p 2a,
For details about the calculation regarding R²)()) see Lemma 18, (2017).
33

---

Published in Transactions on Machine Learning Research (01/2026)
based Lemma B.7 and the fact that is unit vector. As result,
80 / 81 min
where 61 , 30min(U) By applying Proposition B.4 to the left-hand side of equation (52), we have that
16 16lg((2))
(Pa(ua(Xo) + usu)), + X [a(Xo) + Xou s /
dq
Next, by applying Lemma 4.2 to the first term in the right-hand side of equation (52), we get
(6lo((2))
Pa - Xx jua - quAUU - xx Jual S sd2y
dq
where
82 = =2-d Xx 11.0 I pr + 4da² S 65kur.
Finally, apply the spectral norm bound on N from equation (B.3) to Nua. Put together, get
/ Xml + (Xo) 20(UU - XXT, (nun) 54,
where
16 log(
32 l2g( Ö(n 112
8 (81 + +02)
dg
dg
By simplifying the above calculation and using the fact that 4s is a unit vector, we get that
||] + + uA 2 2 2 wAll -6. 6.
(53)
Recall that uA unit vector and us the column span of UA. Therefore,
|IUT un ||TTua|| uAll (amin (Ux))?
and the right hand side of equation (53) is greater than 2( (UA))² 6.
Moreover, recall that ||Xxu|| = miin (X) and ||XTuall uall S Tmin (X). Therefore, the left hand side of equation (53) is at most 6(0min (X))² , in which we apply Cauchy-Schwarz to the cross term. Put together, from equation (53) we conclude that
3 (amin (U)) 6'
(x)) (O (Ux))²
6 16
by equation (51). Thus, when
dq 2 cx 3r (og(d),
for c 16x 652 9, we have that
min (U))2
16
which concludes the proof of equation (50).
34

---

Published in Transactions on Machine Learning Research (01/2026)
B.5 Characterization of The Population Loss
In the next result, we show that could remove the effect of the regularizer term in the residual error. We use the fact that the regularizer follows the same direction as X, as stated in Proposition B.6.
Lemma B.10. In the setting of Theorem 3.4, suppose X E gaxr satisfies that
(mmin (X))² 1 min (U))²
and a 2 4x²r /
| x xx [|p & X
for any Y 2 0.
(54)
Proof. Let S= fie [d] |x, 2 a} be the index set of vectors of X whose & norm is greater than By the definition of the regularizer R(X), for any not in S, we have that (VR(X)); 0. Thus, for any i $ S we have that
|0:0 x - x,X x|| = 10,0
For any that is in S, we have that
| x X,X x = | x- -
where U,, X, are the i-h row vectors of U.X X E Raxe, for every = 1,2, 2....... .d. As for cach i E S, we will
show that
(VR(X)), U.U X) 20.
(55)
By Proposition B.6, we have
(Xx a)2 X;
(VR(X)): 41
X; a:Xi,
where a; 2 0 denotes the multiplier in front of X,. Then, we have that
(VR(()), a. | |
2 a, |x miin (x x)
MV
| (amin (U))
by the condition the smallest singular value of U. On the other hand,
(,R x) =a; (X,, U x)
Saz ||x||l -|UA||- -max (U) xMax(X)
Sa. |xx|| (O (U))2xU
(by equation (43), plus Fmax mmax(X) < ||X||>> p) (since max (U) T (U))
Sa; X&I| (mmin (U) VF
x.1 (amin (U))?
16
where the last step is because
P 4h 2 11
| 2
4 T
since |lA|l < / ur/d by Assumption 3.3. Putting both sides together, we therefore conclude that equation (55) must hold. The proof is completed.
Finally, we show that the above result also implies that U close to XXT
35

---

Published in Transactions on Machine Learning Research (01/2026)
Lemma B.11. In the setting of Theorem 3.4, suppose X Raxr satisfies equations (44) and (54). Then, with probability at least. 1 - )( the following must. be true:
| - -UU op2 log(2d)
(56)
dq
for some fixed constant e > 0.
Proof. We separate U into one part that is the column span of X and another part that is orthogonal
the column span of X. Let U = Z + V where Z is the projection of U to the column span of X, and V is the projection of U to the orthogonal column subspace of X. In particular, we have V X 0. Thus,
UU = (Z + () = ZZ + VZ
= x - x = ZZZ x xx 1| + 1| S0
(57)
for
p2 12741og(2d)
8 max max (U)
dg
by applying equation (44) and also equation (54). Notice that Z has the same column span as X. As result,
Izz x xx || oi (x) 2z
-x |
which can be verified by inserting the SVD of X into the above inequality. Combined with equation (57), we thus have that
6²
882
|zz xX 11 S
(amin (X))2 S
(omia (U))2*
since (arin (X))2 2 (amin /8))
Next, let a E Rd,b e R² be two unit vectors such that
a 77 Xb Omin (77 X).
From equation (57) we can infer that
miin (ZZ X) - Txx Xb 88.
(58)
Now we consider two cases. Suppose a XXTXXXX Xb is positive. Then the left-hand side of equation (58) must. be at least
a xx Xb min (ZZ X) omin (XXT x) omin (Zz X).
The other case is that if a x.x Xb is negative, then the left hand side of equation (58) must be at least
la xx Xt| mmin (ZZ X) 2 mmnn (Xx X) miin (ZZ x)).
We know that
Omin (XXT X) > (amin ( (8)88/88/8)
Thus, if (amin (1)66 then
mmin (ZZ X) > (amin (U)))/(16V8)
On the other hand,
mmin (27 X) S m (7) Omin (7 X)
|1ll| Jmin (Z X),
36

---

Published in Transactions on Machine Learning Research (01/2026)
which implics
niia (U))
min (Z x)
x)>
8v8 |0
Also note that | X s 83, which implies that
82
5126
|
5126
| - moin ( X))2
m (U))
O (1)
Finally;
|zv|| $ || 1 - -
since Z is a projection of U to the column space of X. In summarr, have that
L00 - XXT 2
1| Xx
1| + 2 ZV
F V 1
(omin (U))2 + 102452 xx
1652
51225' nr (amin (U))*
+
(amin (U))
(512x²r 1024x* + 16)5²
(U))²
since d < (0min()))//(888) which concludes the proof of equation (56), for =42 x 512.
Now we are ready complete the proof of Theorem 3.4.
Proof of Theorem 3.4. Suppose X satisfies the first-order and second-order optimality conditions. Then by Lemma B.7 and Lemma B.8, we have that X satisfies the incoherence condition in equation (39). As a result, (amin (X))² 2 (amin (U)) /8 by Lemma B.9. In particular, we require a to be at least 4x2r 10/2r//// //dd.
In addition, based on Lemma B.8, we require a I Vulr + 1)g/A, which requires
A p(r + 1)g/a² 2 dg/8.
Recall that
q 9=1- p3) = - /d)" ~ Cn/dd.
When
s 6 log(2d)
C
the right hand side of equation (56) at most 2 This concludes the proof of equation (10).
Remark B.12. Our proof has been inspired by the important early work of Ge et al. (2016). We build on and expand their work in three aspects.
First, in the one-sided completion setting, the sampling patterns are non-random. In particular, the diagonal entries are fully observed, while the off- off-oiagnnal entries sparsely observed. Thus, there is mix of dense and sparse samplings in Q. To address this problem, we use weighted projection as defined in Section 2.
Second, the bias of the Hájek estimator is compler due to the nonlinear form of the estimator, and requires delicate analysis of the noise matrix N. This is studied in detail in Corollary B.2 and Corollary B.3.
37

---

Published in Transactions on Machine Learning Research (01/2026)
Third, the observation patterns in the off-diagonal entries of Ω not completely independent, requiring new, rou-by-row concentration. inequality that leverages conditional independence property, i.e., Lemma 4.2.
More broudly, non- non-uniform sampling patterns very common in real-world datasets and have received lots of recent interest in causal inference (Athey et al,, 2021; Xiong & Pelger, 2023). We hope this work inspires further studies of modeling non- con-uniform sampling on sparse panel data from an optimization perspective.
Recent work such as Dong et al. (2025) eramine the optimization landscape of neural networks through evaluating the Hessian structure, such as the Hessian spectrum. In light of the proof techniqqess developed in this work, it might also be interesting revisit these insights in the setting of low-rank matria factorization, which can be reloted to two-layer quadratic neural networks (Li et al, 2018).
C Omitted Experiments
In this section, we describe the dataset statistics and pre-processing procedures. The MovieLens-32MM dataset comprises 32 million user ratings approximately 87, 000 movies, collected from the MovieLens recommendation platform, with ratings ranging from 1 to 5. The sparsity ratio is roughly 3 x 10-1 We also consider two related MovicLens datasets with 20 million and 25 million entries to ensure the robustness of our findings. We split 80% into training and hold out the rest for testing.
The Amazon Reviews dataset includes 571 million user ratings 48 million products, collected from Amazon, with ratings ranging from 1 to 5. The sparsity ratio is roughly x 10-7. For the Amazon reviews dataset, we choose the Automotive category with 50, 000 items rated by 100, 000 users as M. We use 80-20 split.
The Genomes dataset contains phased genotype data for 2, 054 individuals gathered from diverse populations. The dataset is represented as dense matrix, with rows representing genomic sites and columns corresponding to individuals. Each entry represents a biallelic genotype, encoded as or 1. We map them to and 2, and use to indicate missing entries. This dataset is fully observed. Thus, we use 1% for training and hold out the rest for lesting.
For MovicLens and Amazon Reviews datasets, we use f on M M to obtain the ground truth second- moment. matrix. Then, the est imation error is evaluated against. this matrix. In our experiments, we set the sampling probability p = 0.8. Given that the original datasets have an average sparsity of approximately 0.3%, even with p = 0.8, the number of observed entries remains very sparse. When is not fully observed, we focus on measuring the mean squared error within the observed entries instead and treat the missing entries zero.
C.1 Implementation
We provide detailed description of all the baseline methods we have implemented in our experiments.
Alternating-GD minimizes the squared reconstruction error on the observed entries of M. We minimize the following objective:
min XER YER
[Pa(XT -M)I
where Pn denotes the projection onto the observed entries. We initialize X and Y from an isotropic Caussian distribution. We perform the optimization using Adam with learning rate of 0.1 over 300 training epochs.
SoftImputo-AIS alternates between matrix completion using singular value decomposition (SVD) and imputing missing values (Hastie al., 2015). At each iteration, missing entries are filled with the current estimates, followed by SVD and soft thresholding of the singular values. The resulting
38

---

Published in Transactions on Machine Learning Research (01/2026)
Alternating GD
Nuclear Norm Reg.
Alternating GD
Nuelear Norm Reg.
SoftImpute ALS
Algorithm
SoftImpute ALS
Algorithm 2
Recovery Error
4.00
Recovery Error
4.00
0.06
0.06
3.00
3.00
RMSE
0.04
RMSE
2.00
2.00
0.03
0.02
1.00
1.00
0.00
0.00
0.00
0.00
2
4
2
3
2
3
7
x10
x10
(a) p
(b) C =
(c) p =
C
(d) C 02
4.00
4.00
0.06
0.06
Recovery Erron
Recovery Error
3.00
3.00
RMSE
RMSE
2.00
2.00
0.03
0.03
1.00
1.00
0.00
0.00
0.00
0.00
2
3
x10
7 (f) C = 10
x10
71 (g) C = 5
x10
11
x10
(e) C
=
(h) C'
10
Figure 7: One-sided recovery error of I (shown in the left two columns) and the root mean squared recovery error of the original matrix M (shown the right two columns) using our algorithms on synthetic matrix data. Figures 7a, 7b: One-sided recovery error under uniform sampling by varying p, and sampling two entries per row (C 2) while varying the number of rows n. Figures Te, 7d: Recovery error on M, also under uniform sampling, and sampling two entries per row. Overall, we find that our approach consistently provides more accurate estimates compared with the three baseline methods, all of which are widely used in the matrix completion literature. Figures 7e, 7f, 7g, 7h: Vary C and repeat the same experiment.
truncated low-rank approximation is used to update the matrix. We repeat this process until convergence, as measured by the Frobenius norm difference between successive reconstructions, or until a maximum of 500 iterations. We set the target rank the same as r and the convergence threshold to 10-5
Nuclear norm regularization is another commonly used approach for matrix completion. In one-sided ostimation, we compute empirical averages based on the observed data (Cao et al., 2023). For ofT-diagonal entries (i,j); we identify all samples which the pair (i,j) appears and average the product of the corresponding entries from the two relevant columns. For diagonal entries (i, we consider all samples where index appears in either position of a pair, square the corresponding data matrix entry for each such sample, and average these squared values. We then estimate the full matrix using symmetric alternating gradient descent by solving the following optimization problem:
min XeRdx
Pa(XXX - +
where ||X|| denotes the nuclear norm of X and as 0.01. We perform the optimization using Adam with a learning rate of 0.1 for 1, 000 steps.
Hyperparameter configurations. We report the hyperparameter configurations used our experiments.
The learning rate 7 is varied from 10 to 105, the regularization cocfficient À from 10-2 10-5, and the regularization parameter from 10-4 to 1. Based on vross-validation, we select 17 = 10 À 10-4, and
10-3 which yield the lowest error. For the Amazon Review dataset, set X = 0.01 and 0.1.
39

---

Published in Transactions on Machine Learning Research (01/2026)
Table 4: We report the recovery error for both one-sided matrix completion (i.e., recovering T), and recovering M, along with the corresponding running time (measured in seconds on an Ubuntu server) on three MovieLens datasets. We run each experiment with five random seeds and report the mean and standard deviation.
Datasel
MovieL
Mov
MovieL.nns-20 20M Mo
Estimating T
One-sided recovery error
Running time (Seconds)
Alternating-Gi
0.015+4.000
0.006+0.002
1.9
10
4.4 x 10a
SoftImpute-ALS
0.008 005
0.006+0.002
3.4
103
9.4
103
Algorithm 1
0.002 0.001
0.0021..10
4.8 10.2
10
1.1
102
Estimating MM
Root mean squared crror
Running time (Seconds)
Alternating- GD
1.11
1.27
1.940.1
102
4.4
102
1.09
1.25
3.440.1
103
9.440.7 103
Algorithm 2
0.99
1.04
1.0 x 10a
2.6 x 102
C.2 Omitted Comparison Results
First, we report the runtime comparison between HÁJEK-GI GD and nuclear norm regularization. We run gradient descent with 1, 000 iterations until it has converged. We also fix the number of observed entries as 200d while varying d from 10 to 10a. We find that our approach requires 0.95 seconds for d 10sh and 1.61 seconds for d = 10 By contrast, solving a convex program with muclear norm penalty takes 5.68 seconds for 10 and 100.34 seconds for d = 10*, measured on the Ubuntu server.
Second, we present. the recovery error for imputing M, illustrated in Figure 7. Our approach consistently reduces the recovery error compared to all three baseline methods. For instance, when C (sampling two entries per row),
approach incurs error of 0.0003, compared to 0.002 for nuclear norm regularization. We also observe similar results on synthetic data when sampling five entries per row (C = 5) or sampling ten entries per row (C = 10) in Figures 7e and 7f.
Similar results are observed when our approach is applied to recover the missing entries M. For recovering M, our approach reduces estimation error by 94% relative to alternating GD and by 98% compared softImpute-ALS.
Finally, we report the results on MovieLens-225 20M and MovieLens-225 25M in Table 4. For one-sided matrix completion, our approach reduces the Frobenius norm error between the estimate and the true T by up to 87% compared to alternating-Gi GD and up to 75% compared to softimpute-Ale ALS, while also achieving lower running times.
For recovering the missing entries of M, our approach reduces the RMSE by up to 10% compared alternating- GD and up to 6% compared to softlmpute-A1s,
C.3 Ablation Analysis
Setting the rank of the target matriz. In Scction 5, we use a target rank of for all the reported experiments. To further assess the robustness of our approach, we conduct additional experiments on three real-world datasets using Algorithm 1, with the target rank in the range of 1 to 30. The results are shown in Table 5. We find that larger ranks may lead to overfitting in sparse settings; for example, on the Amazon dataset, increasing beyond 10 results in worse performance when recovering M from the estimated T.
We also run similar analyses on synthetic data and find that the algorithm is not particularly sensitive to different choices in this context. In particular, we conduct different simulations by varying the rank of M between 1 and 50. We vary the target rank used in Algorithm 1 between and 50. We find that the estimation errors are comparable between different choices of target ranks.
40

---

Published in Transactions on Machine Learning Research (01/2026)
Table 5: We vary the rank of the variable matrix HÁJEK-GU corresponding to the results we reported Table 3. The results are averaged over five independent runs.
Dataset
MovieLens-20M Amazon Reviews
Genomes
Estimating T (Using Algorithm 1)
One-sided recovery error T
r = 1
2.0
10-3
4.5_0.1 x 10 10-3
5.8 x 10-3
r = 10
4.7
10-3
1.3 x 10-3
5.8 X 10-5
r 20
1.9
10-3
1.3_0.1
10-3
5.8 10-5
30
1.8 10.1
10-3
1.3
10-3
5.8
10-3
Estimating (Using Algorithm 2)
Root mean squared recovery error on M
1
3.740.1
4.640.1
0.1
r = 10
1.1
1.940.1
0.2+0.1
r 20
1.0
2.3
0.2
r 30
1.040.1
2.7
0.2
Sensitivity of Algorithm 1. We evaluate the sensitivity of adding noise to M on the final output of Algorithm 1. We use synthetic datasets with varied numbers of rows bet 5d and 20d, and fixed dimension of 1, 000.
We also test different ranks of M, ranging from 5 to 20. We add Gaussian noise with standard deviation that ranges from 0 to 0.01 to the input. Interostingly, we find that cstimation error increases lincarly with o. Moreover, when varying n, the sensitivity level of HÁJEK-GD (recall its definition in equation (17)) is generally low. When n = 5d, the slope of the line is 19. When n = 10d, the slope decreases to 14. When n = 20d, the slope further decreases to 13. Similar results hold after varying the rank between 20, 10, 5, and the sensitivity level ranges between 16, 13, 11. See illustration in Figure
Recovery Ervor
0.2
5d
Remoery Error
0.2
20
10d
10
0.1
20d
0.1
5
0.0
0.0
1.00
0.75
0.50
0.25
0.00
1.00
0.75
0.50
0.25
0.00
0
X x10
0
x x10
(a) Varying noise level for different number of rows
(b) Varying noise level for different rank of M
Figure 8: Illustratin the cstimation error after adding Gaussian noise with mean zero and variance o² to M the non-zero entries. In Figure Sa, vary the number of vary rank (of M) with fixed 20d. Across all six cases, the sensitivity level, measured by the slope of the line, is at most 19 and drops to 11 as the sample size increases or the rank decreases. We report the mean and standard deviation from five independent runs.
with fixed rank 10. In Figure 8b,
41


Reviews:
[{"review_id": "cE3WAxPxsf", "reviewer": "Reviewer_SM4K", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. The paper provides a strong theoretical analysis of matrix row-space completion under an ultra-sparse sampling regime. The analysis includes a variance comparison between the Hájek and Horvitz–Thompson estimators over the observed entries, as well as a study of the sample complexity of the gradient descent algorithm applied to the unobserved entries.\n\n2. The proposed method, grounded in solid theoretical foundations, demonstrates significant performance improvements on both synthetic and real-world datasets.\n\n3. The incorporation of incoherence regularization effectively reduces recovery error, highlighting the practical value of the theoretically motivated penalty term.\n\nWeakness\n\n1. The paper assumes that the rank $r$ is known for both the theoretical analysis and empirical evaluation. However, in real-world applications, the true rank is typically unknown, which may limit the practical applicability of the proposed approach.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the recovery of the row space of a matrix under an ultra-sparse sampling regime. It first analyzes the variance between the Hájek estimator and the Horvitz–Thompson estimator over the observed entries. Then, the author proposes a gradient descent algorithm for the unobserved entries and investigates its sample complexity. Finally, the paper provides empirical validation on both synthetic and real-world datasets to demonstrate the effectiveness of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Typo\n\n1. In the first line of Section 2, \"With loss of generality\" should be corrected to \"Without loss of generality.\"\n\n2. In the first line after Assumption 3.3 \"Assuming that $µ(U)$ is a fixed value that does not row with d\". What does this mean?\n\nQuestion: \n\n1. In Equation (4), the first term is $M^4_{k, i}$ whereas in Equation (6) it is $M^2_{k, i}$. Why the paper states \"By comparing equation (4) to equation (6), we see a reduction in the second term from equation (4), which is at the same order as the leading term\" ?", "broader_impact_concerns": "No concerns towards ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mRAxtIlphc", "reviewer": "Reviewer_8mat", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- The paper tackles a challenging and practically important problem: matrix completion in the ultra-sparse regime, which arises frequently in large-scale recommendation systems where the number of users and items increases rapidly.\n- The proposed Hájek-GD method is novel and well-motivated. Theoretical analysis of Hájek-type estimators in this context is nontrivial, and the paper provides rigorous and insightful results, including variance reduction and recovery guarantees.\n- The empirical evaluation is thorough, covering a number of real-world datasets, and demonstrates substantial improvements over standard matrix completion baselines in terms of bias, variance, and recovery accuracy.\n\nWeaknesses\n- The paper is dense and presents a large number of technical results. The exposition could be improved to make the paper more accessible to a general audience.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of one-sided matrix completion in the ultra-sparse regime, where only a small subset of entries from a tall matrix $M \\in \\mathbb{R}^{n \\times d}$ (with $n \\gg d$) are observed. Specifically, it focuses on recovering the row space of $M$ by estimating its second-moment matrix $T = \\frac{1}{n} M^\\top M$, despite extreme sparsity in the observations.\n\nThe proposed method, namely Hájek-GD, estimates the second-moment matrix $T$ in two steps. In the first step, for each pair of columns in $M$ that are co-observed in at least one row, it computes the average inner product between the corresponding column entries, normalized by the number of such co-observations. This yields a Hájek-type estimator that adjusts for the non-random and sparse observation pattern. However, due to extreme sparsity, many column pairs may never be jointly observed, leaving large portions of the second-moment matrix $T$ missing. To address this, the second step imputes the unobserved entries by fitting a low-rank factorization via gradient descent, with an incoherence regularization term to stabilize optimization and promote identifiability. The completed second-moment matrix, denoted by $\\hat{T}$, is then used to recover the row span of $M$ via gradient descent.\n\nThe paper makes two main technical contributions. First, it demonstrates that the Hájek-type estimator is unbiased in finite samples and achieves substantially lower variance than the Horvitz-Thompson estimator, with greater variance reduction in sparser regimes. Second, it establishes a near-optimal recovery guarantee for the full second-moment matrix under a low-rank factor model with an incoherence condition.\n\nExtensive experiments on real-world datasets, including MovieLens, Amazon Reviews, and 1000 Genomes, demonstrate that the proposed Hájek-GD method significantly outperforms standard matrix completion baselines in recovering both the second-moment matrix and the original matrix.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- In the recovery of row span of $M$ from the completed matrix $\\hat{T}$, the paper uses both diagonal and off-diagonal entries (as in Equation (8)). However, prior work on covariance and second-moment matrix estimation (e.g., Fan, Liao, and Mincheva, 2013) often treats the diagonal separately under a low-rank plus diagonal structure. It would be helpful to clarify under what conditions it is appropriate to include both diagonal and off-diagonal entries when estimating the low-rank component (i.e., the row span of $M$).\n- It would be helpful to provide a precise and explicit definition of the \"one-side matrix completion\" problem in both the abstract and the introduction. In addition, it would be helpful to add some discussion about why this problem is important and challenging.\n- Referring to $M$ as a tall matrix and stating that the goal is to estimate its row span would help concretize the problem setup for the reader.\n- In Theorem 3.4, it would be helpful to discuss the assumption that the number of users $n$ must be much larger than the number of items $d$, and explain why this assumption is reasonable in real-world applications (e.g., in recommendation systems).\n- The Hájek estimator is known to exhibit finite-sample bias in general. Since the paper shows it is unbiased in this setting, it would be helpful to highlight and explain this result more clearly in the introduction.\n- The first extension, recovering individual entries of $M$, is closely aligned with the standard matrix completion problem. Therefore, it would be helpful to expand this extension a bit and discuss the practical scenarios where this extension is feasible (e.g., when the number of observed entries per row exceeds the rank of $M$).\n- The second extension, concerning differential privacy, is currently a bit hard to follow and would benefit from revision for clarity.\n\nMinor Issues and Typos\n- In Figure 3, Step 1, the dimension of $\\hat{T}$ is $d$ by $d$ as opposed to $n$ by $d$.\n- The notation $S_i$ in Lemma 4.1 lacks a definition. \n- “Bias of Hájek’s estimator” in the paragraph header on page 8 should be “Bias and Variance of Hájek’s estimator”.\n- Hajek’s estimator -> Hajek estimator.\n- Horvitz-Thompson’s estimator -> Horvitz-Thompson estimator.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oWFiQe6S6v", "reviewer": "Reviewer_PeU4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nI think this is a strong paper overall. It studies an important problem: matrix completion when one could only observe a constant number of entries per row. To tackle this challenge, authors propose to use Hajek estimator, which is novel in the matrix completion literature. They conduct rigorous theoretical studies on Hajek estimator for matrix completion, prove its conditional unbiasedness and variance reduction from HT estimator. This is not particularly easy as Hajek estimator involves dividing random variables. To do so they analyze the variance via first-order approximation. I think the introduction of Hajek estimator to the matrix completion community is a nice contribution. Experiments are comprehensive, they cover various aspects (variance, relationship between different parameters and $d$, recovery error and runtime).\n\nWeaknesses:\n\nThere are two weaknesses, one major, one minor.\n\n1 (Major). The \"common sense\" assumption when analyzing the Hajek-GD algorithm. This model assumes the rows of $M$ are drawn uniformly from $r$ vectors $u_1,\\ldots,u_r$. This seems to be a very restrictive assumption --- in particular, this means the other rank-$r$ factor is essentially rearranging and duplicating columns of the factor matrix $U$. This deserves many more discussions than a footnote. Further, the references justifying this assumptions are all related to regressions, I assume this means that this assumption directly impacts the proof of the gradient descent step. Authors should thoroughly discuss and justify this setting, how strongly the proofs are \"dependent\" on the assumption, etc.\n\n2 (Minor). One of the major advantages of gradient descent is its efficiency, authors might consider conduct a preliminary asymptotic runtime analysis of Hajek-GD.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the matrix completion problem when the number of samples being taken is extremely small, only $Cn$. In this setting, recover the matrix $M$ is not possible, so authors consider recovering the second moment $T=\\frac{1}{n}M^\\top M$. The major challenge here is that the binary mask matrix $\\Omega$ for $T$ is not uniform, as it has much higher probabilities on the diagonal than the off-diagonals. To address this issue, the authors introduce the Hajek estimator which obtains lower variance than the more commonly-used HT estimator. And then, they show that use a properly regularized objective, one could recover $T$ up to $\\epsilon^2$ error in squared Frobenius norm via first compute the Hayek estimator, then run gradient descent and use the entries outside of $\\Omega$ obtained by gradient descent. Extensive experiments are performed to show that Hayek estimator indeed has lower variance than HT estimator on synthetic dataset. Further, on real-world datasets, the proposed Hajek-GD algorithm outperforms methods such as alternating GD and softImpute ALS in both recovery error and runtime.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Major: see weaknesses, particularly the major weakness.\n\nMinor typos:\n\n* Hja&#769;ke has its punctuation misplaced in multiple places, sometimes it's missed, sometimes it's typed as Hj&#769;ake. I suggest authors to carefully proofread the manuscript and make sure the use of punctuation is consistent.\n\n* In Theorem 3.4, it might be better to first remind readers the definition of $\\alpha$ and $\\lambda$ for regularization before setting their values.", "broader_impact_concerns": "No such concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)