Title: Sample-Efficient Geometry Reconstruction from Euclidean Distances using Non-Convex Optimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: Euclidean distance geometry

non-convex optimization

iteratively reweighted least squares

low-rank

data efficiency

convergence guarnatees

restricted isometry property

dual basis
Abstract: The problem of finding suitable point embedding or geometric configurations given only Euclidean distance information of point pairs arises both as a core task and as a sub-problem in a variety of machine learning applications. In this paper, we aim to solve this problem given a minimal number of distance samples. 
To this end, we leverage continuous and non-convex rank minimization formulations of the problem and establish a local convergence  
guarantee for a variant of iteratively reweighted least squares (IRLS), which applies if a minimal random set of observed distances is provided.
 As a technical tool, we establish a restricted isometry property (RIP) restricted to a tangent space of the manifold of symmetric rank-$r$ matrices given random Euclidean distance  measurements, which might be of independent interest for the analysis of other non-convex approaches.  Furthermore, we assess data efficiency, scalability and generalizability of different reconstruction algorithms through numerical experiments with simulated data as well as real-world data, demonstrating the proposed algorithm's ability to identify the underlying geometry from fewer distance samples compared to the state-of-the-art.
 
 The Matlab code can be found at \href{https://github.com/ipsita-ghosh-1/EDG-IRLS}{github\_SEGRED}

Sample-Efficient Geometry Reconstruction from Euclidean Distances using Non-Convex Optimization

Abstract

The problem of finding suitable point embedding or geometric configurations given only Euclidean distance information of point pairs arises both as a core task and as a sub-problem in a variety of machine learning applications. In this paper, we aim to solve this problem given a minimal number of distance samples. To this end, we leverage continuous and non-convex rank minimization formulations of the problem and establish a local convergence guarantee for a variant of iteratively reweighted least squares (IRLS), which applies if a minimal random set of observed distances is provided. As a technical tool, we establish a restricted isometry property (RIP) restricted to a tangent space of the manifold of symmetric rankr matrices given random Euclidean distance measurements, which might be of independent interest for the analysis of other non-convex approaches. Furthermore, we assess data efficiency, scalability and generalizability of different reconstruction algorithms through numerical experiments with simulated data as well as real-world data, demonstrating the proposed algorithm's ability to identify the underlying geometry from fewer distance samples compared to the state-of-the-art.

The Matlab code can be found at github_EDG-IRLS

1 Introduction

Euclidean Distance Geometry (EDG) problems have applications spanning diverse domains, from comprehending protein structures through molecular conformations [MMLL19, MMWL22], prediction of molecular conformations in computational chemistry [DPRV15, SWBB97] and aiding dimensional reduction in machine learning [TSL00] to facilitating localization in sensor networks [FQX19], coupled with its role in solving partial differential equations on manifolds [LL17]. This emphasizes its broad impact in computational sciences. [LLMM14] proposes embedding entities based on Distance Geometry Problems (DGP), where object positions are determined based on a subset of pairwise distances or inner products which significantly reduces computational complexity compared to traditional word embedding methods.

̸

Problem 1 . Mathematically, consider a collection of n points p i in an r -dimensional Euclidean space with coordinates P = [ p 1 , p 2 , ..., p n ] ∈ R r × n whose pairwise squared Euclidean distances are given by d 2 ij = || p i -p j || 2 for each 1 ≤ i = j ≤ n where || · || denotes the Euclidean norm. Given only partial information of the { d ij } such that only a subset of cardinality m<n ( n -1) / 2 is known, the goal is to reconstruct the geometry of the points, that is to recover the point coordinates P .

If all the distances between the points are provided, the problem is known as multidimensional scaling [LT24, GCKG23], and closed solution formula exists. However, this is not the case for the incomplete setup which is the focus of this work, and in which only partial information of the pairwise distances is available.

For instance, AlphaFold [SEJ + 20, JEP + 21] has shown effectiveness in predicting the threedimensional structure of protein given as input its amino acid sequence. AlphaFold2, which uses an attention mechanism-based transformer architecture [VSP + 17], is trained on known sequences and structures from the Protein Data Bank [HMBFGG + 00] and determines the distances between the C α atoms of all residue pairs in a protein. Subsequently, as a subproblem, this distance information is used to predict the protein's structure by identifying the foldings within the protein molecule. In the context of this subproblem, where the input is the predicted pairwise distances, a linear mapping can be used to predict geometric coordinates. However, many of the predicted distances might not be accurate, which is why low confidence regions of the resulting structures, as indicated by the predicted local-distance difference test (pLDDT) [JEP + 21], could be masked, and a new structure could subsequently be re-computed based on the distance entries of the medium-to-high confidence regions using high-accuracy solution algorithms for Problem 1.

In the context of the Problem 1, we have access to a subset of entries of this D = [ d 2 ij ] matrix, defined by the index set Ω ⊂ { 1 , . . . , n } 2 . Now, we formulate the Gram matrix X 0 = P ⊤ P , residing in the set of real-valued symmetric matrices S n . This matrix has a lower rank r compared to the symmetric distance matrix D ∈ S n = { X ∈ R n × n : X = X ⊤ } , which has a rank of r +2 [DPRV15].

To make the solution translation invariant, the centroid of the points must be the origin, i.e., ∑ n ℓ =1 p ℓ = 0 . To ensure this, the Gram matrix should satisfy also satisfy X 0 · 1 = 0 , where 1 ∈ R n is the vector of all ones. Based on these observations, we can frame Problem 1 as an instance of the computationally challenging NP-hard rank minimization [Rec11] problem defined by

incorporating also the positive semi-definiteness constraint X ⪰ 0 , which comes from the observation that X 0 = P ⊤ P is also PSD.

From an optimization perspective, this rank-minimization problem is highly complex due to its nonconvexity and non-smoothness. Being an NP-hard problem [TL18, Hen95, MW97], it is extremely difficult to solve directly. Consequently, a significant amount of existing research [Rec11, DR16, CR09, CW18, Gro11, TL18] has focused on minimizing the convex envelope of the rank function, known as the nuclear norm, instead. However, it was observed that for related low-rank matrix completion problem, such convex relaxations are not as data-efficient as other formulations [ALMT14, TW13], while posing also computational limitations [CC14].

In this paper, we study the question of, given incomplete distance information Problem 1, how many samples are necessary to ensure accurate recovery? [TL18] established convergence with O ( νrn log 2 n ) uniformly random samples for solving (1) using a nuclear norm surrogate. While there are numerous results for non-convex methods in other low-rank recovery problems [KV21, CLC19, Van13], there are currently only few non-convex algorithm specifically designed for EDG problems available [TL18, SLCT23, ZCZ22, NKKS19, LS24]. To the best of our knowledge, only [NKKS19, LS24] provide convergence guarantees in the non-convex setting. Unlike generic lowrank matrix recovery problems [RFP10], EDG problems share the complexities of low-rank matrix completion problems, such as the absence of a direct uniform null space property [RXH11] or a restricted isometry property [RFP10]. Additionally, the underlying measurement basis in EDG problems is not orthonormal, making it impossible to directly use the analysis of standard matrix completion to provide guarantees for techniques using the Gram matrix-based low-rank modelling (1) of Problem 1.

1.1 Our Contribution

In this paper, we propose and analyze an algorithm for the EDG problem based on the iteratively reweighted least squares framework (MatrixIRLS, see Section 3), for which we show that m = Ω( νrn log( n )) (where ν is the coherence factor) randomly sampled distances are sufficient to guarantee local convergence to the ground truth, X 0 with a quadratic rate as shown in Theorem 4.3, from which the geometry of the points P is trivially recovered. The sample complexity assumption

of Theorem 4.3 matches the lower bound of low-rank matrix completion problems as established in [CT10]. In Section 4, we construct a dual basis (Lemma 4.4) that spans S n which enables us to show the restricted isometry property (RIP) restricted to a tangent space of the manifold of symmetric rankr in Theorem 4.5 for our approach, which can be of independent interest for the analysis of other nonconvex algorithms.

While the convergence statement of Theorem 4.3 only applies in a local neighborhood of a ground truth, the indicated data-efficiency of the proposed method is numerically validated through different experiments in Section 5 on synthetic and real data in comparison to the state-of-the-art methods. Furthermore, we demonstrate that MatrixIRLS method is robust to ill-conditioned data, further highlighting its flexibility. In the Supplementary material, we discuss the limitations in Appendix B. Additionally, we provide proofs related to the the theoretical results of Section 4 in Appendices B and C. We further discuss the numerical considerations of our experiments in Appendix E. We discuss about the computational complexity of the algorithm in detail in Appendix F.

2 Related Work

Early research on EDG problems focused on establishing its mathematical properties like defining the conditions under which a distance matrix can be represented in Euclidean space[Gow85, YH38, YH38, Alf05].A comprehensive overview of EDG applications, including molecular conformations, wireless sensor networks, robotics, and manifold learning, is available in [Lib20]. Motivated by the molecular conformation problem, [BJ95] relates this Euclidean distance matrix completion to a graph realization problem by showing that such matrices can be completed if the graph is chordal. Other than graph theoretic approaches, as a technical tool, various optimization strategies [Hen95, MW97, Tro00] have been deployed to solve EDG problems. [AKW99] proposes a primaldual interior point algorithm that solves an equivalent semi-definite programming problem. However, none these works provide theoretical reconstruction guarantees in the incomplete setup of Problem 1.

While providing an accurate modelling of Problem 1, the rank minimization formulation (1) poses challenges due to its non-convex and non-smooth nature. There is a mature existing literature [DR16, CR09, CW18, Gro11] around replacing the rank function, rank( X ) , with the sums of its singular values σ i ( X ) (also known as the nuclear norm). Building on the rank minimization formulation (1) of the EDG problem, [TL18] minimizes the convex nuclear norm surrogate of the inferred Gram matrix. They propose a dual basis approach that enables a theoretical guarantee for this type of nuclear norm minimization formulation of the EDG problem.

From a practical point of view, it is well-known that the convex approach is computationally intensive, as the arithmetic complexity is cubic in n , the dimension of X 0 [CC14]. It also tends to demand more data samples than non-convex alternatives, making it less efficient in terms of data [ALMT14, TW13].

To mitigate these issues, recent studies have shifted focus to non-convex methods such as matrix factorization [SL16, MWCC20, ZCZ22]. These methods optimize a function involving a data-fit objective regularized by squared Frobenius norms of the factor matrices, which are computationally more feasible and data-efficient. Few 'non-convex' algorithms are based on matrix factorization like the work in [BM03].

Based on a similar formulation, ScaledSGD [ZCZ22] is a preconditioned stochastic gradient descent method aimed at robustness for ill-conditioned problems. Additionally, some of the most effective techniques for low-rank matrix completion involve minimizing smooth objectives on the Riemannian manifold of fixed-rank matrices, providing scalability and the potential to reconstruct the matrix with fewer samples, although they lack strong performance guarantees [Van13, WCCL20, BA15, BNZ21]. ReiEDG [SLCT23] is a Riemannian-based gradient descent strategy utilizing the sampling operator on Ω , which is non-convex approach to solving the EDG problem. However, it does not provide convergence guarantees. To the best of our knowledge, [NKKS19, LS24] are the non-convex approaches for solving the EDG problem in the Gram-matrix-based low-rank modeling (1) of Problem 1. The work in [NKKS19] proposes an algorithm based on Riemannian optimization over a manifold and provides convergence guarantees. These guarantees are derived from extended Wolfe conditions. However, it is not explicitly detailed how these convergence guarantees depend on problem parameters such as the sampling model and the number of samples (see Remark III.8 in [NKKS19]). Similarly, the study in [LS24] employs a Riemannian framework and provides local convergence guarantees under Bernoulli sampling. Nonetheless, [LS24] does not clarify whether

the proposed algorithm achieves local linear convergence. In contrast to these studies, the algorithm proposed in this paper achieves local quadratic convergence under uniform sampling of the distances.

To handle the non-smoothness and non-convexity of rank minimization problems of the type (1), iteratively reweighted least squares (IRLS) algorithms take a different route than methods based on [BM03] or Riemannian methods by minimizing a sequence of quadratic majorizing functions of smoothed rank surrogates [DDFG10, FRW11, MF10, KS18, KV21]. IRLS algorithms have been studied extensively over the years, as indicated by [Law61, GR97, Bur12]. In the context of lowrank matrix completion, IRLS algorithms are known to be among the most data-efficient methods available, while being amenable to a rigorous convergence analysis [FRW11, MF10, KS18, KV21]. Most recently, [KV21] provided an improvement on previous instantiations of the IRLS framework [FRW11, MF10, KS18] for low-rank optimization problems by providing an improved reweighting strategy, for which the authors show a local convergence guarantee that is applicable for low-rank matrix completion, given random entrywise samples of minimal sample complexity. The algorithm we propose in Section 3 is similar to [KV21, Algorithm 1] and Theorem 4.3 follows partially the proof strategy of a related result in [KV21]. However, the setup of Problem 1 does not allow a direct adaptation of both the implementation and analysis of [KV21] due to the non-orthogonality of the measurement basis.

3 MatrixIRLS for Euclidean Distance Geometry

In this section, we provide a detailed outline and description of the iteratively reweighted least squares method in the context of the EDG reconstruction problem. To this end, we define preliminaries for stating the algorithm. MatrixIRLS, defined in Algorithm 1 below, can be interpreted as a hybrid of a smoothing method [Che12] and a Majorization-Minimization algorithm [SBP17]. In particular, the proposed algorithm minimizes smoothed log-det objectives defined as F ϵ ( X ) := ∑ n i =1 f ϵ ( σ i ( X ))

which is a continuously differentiable function with ϵ -2 -Lipschitz gradient [KV21].

We can decompose X by X = U dg( γσ ) U ⊤ , where γ ∈ { +1 , -1 } , It is clear that the optimization landscape of F ϵ crucially depends on the smoothing parameter ϵ . Instead of minimizing F ϵ directly, our method minimizes, for k ∈ N , ϵ k > 0 and X ( k ) a quadratic model that is related to the secondorder Taylor expansion of the function f ϵ at the current iterate and its information is encoded in a weight operator [KV21] defined below in Definition 3.1.

Definition 3.1 ([KV21]) . X ∈ S n be a matrix with singular value decomposition X ( k ) = U dg( γσ ) U ⊤ , where γ ∈ { +1 , -1 } i.e., U ∈ S n are orthonormal matrices. We define the weight operator core matrix H X ,ε ∈ S n of X for smoothing parameter ε > 0 such that

where ◦ denotes the entrywise ore Hadamard product of two matrices.

Our method is designed to provide iterates that satisfy the constraints of the formulation (1) at each iteration. They can be encoded using the following definition.

Definition 3.2. Given n ∈ N , let I = { α = ( i, j ) | 1 ≤ i < j ≤ n } be the index set of upper triangular indices.

We define the operator basis { w α } α ∈ I ∪{ ( i,i ): i ∈{ 1 ,...,n }} where

where e i ∈ R n is the i -th standard basis vector.

Given a set (or multiset) of indices Ω = { α 1 , . . . , α m } ⊂ I of cardinality m = | Ω | , we define the measurement operator A = A Ω : S n → R m + n which maps X ∈ S n to A ( X ) whose ℓ -th coordinate is defined as A ( X ) ℓ = ⟨ w α ℓ , X ⟩ for ℓ ≤ m and as A ( X ) ℓ = ⟨ w ( ℓ -m,ℓ -m ) , X ⟩ for ℓ > m .

The basis { w α } α of Definition 3.2 can be considered as an extended definition of the primal basis used in [TL18, LT24], but additionally is able to encode the constraint X · 1 = 0 which guarantees that the Gram matrix corresponds to points whose centroid is located at the origin. Accordingly, we can define the constraint set corresponding to Gram matrices of points that are centered and simultaneously satisfy the pairwise distance constraints of Problem 1 as { X ∈ S n : A ( X ) = [ D Ω ; 0 ] } . The algorithm below minimizes the quadratic, majorizing model of the objective F ϵ k ( · ) given k , while satisfying the measurement operator based on the sampled distances. Equivalently, we can define the main computational step of the method as

where W k : S n ↦→ S n is defined as W k := W X ( k ) ,ϵ k with the definition of the weight operator Definition 3.1 above. With this preparation, we provide an outline of MatrixIRLS in Algorithm 1 below. Equation (7) provides suitable update rule for the smoothing parameter sequence ( ϵ k ) k ∈ N that enables the computation of only r = O ( ˜ r ) singular triplets of each algorithmic iterate [KV21].

Algorithm 1 MatrixIRLS for Euclidean Distance Geometry Problems

Input: Index pairs Ω ⊂ 1 , distances D Ω = ( d ij ) ( i,j ) ∈ Ω , rank estimate ˜ r . Output: X ( k ) after suitable stopping condition. Initialize k = 0 , ϵ 0 = ∞ and W 0 = Id .

Solve weighted least squares: Solve (5) by

for

k

= 1

,

2

, . . . , do

Update smoothing: Compute ˜ r +1 -th singular value of X ( k ) to update

Update weight operator: For r k := |{ i ∈ [ n ] : σ i ( X ( k ) ) > ϵ k }| , compute the first r k singular values σ ( k ) i := σ i ( X ( k ) ) and matrices U ( k ) ∈ R n × r k with leading r k left singular vectors of X ( k ) to update W k = W X ( k ) ,ϵ k defined in Definition 3.1. end for

3.1 Computational Considerations

The computational complexity of this above algorithm can be computed from the steps (6) and (7). In our numerical implementation, we largely follow the tangent space formulation of the weighted least squares step (6), cf. [KV21, Section 3], which involves the solution of an order O ( nr k ) = O ( nr ) linear system if ˜ r = r is chosen as the ground truth rank. An additional difficulty we overcame in the provided reference implementation arises from the fact that AA ∗ is not the identity. The per-iteration time complexity of our method is dominated by O (( mr + r 2 n ) N 0 inner ) , where N 0 inner is the number of inner iteration bound of the iterative linear system solver. Detailed FLOPs calculation is shown in Appendix F.

4 Theoretical Analysis

In this section, we discuss about the local convergence of MatrixIRLS in Section 4.1 and establish RIP restricted to the tangent space of the manifold of symmetric rankr matrices at the ground truth Gram matrix X 0 in Section 4.2.

4.1 Local Convergence Analysis of Algorithm 1

It is well-known in the literature on low-rank matrix completion that for an entrywise measurement basis, recovery from generic measurements is more difficult if most information of the low-rank

matrix is concentrated in few entries, and this observation is typically captured by the notion of incoherence [CR09, Rec11, Che15]. For the purpose of the EDG problem of interest, we use the following coherence notion, which has appeared in similar form in [TL18].

Definition 4.1 (Coherence for Gram matrices in the EDG problem, [TL18]) . Let X ∈ S n be of rank r . Let T = T X = { ZX + XZ ⊤ : Z ∈ R n × n } be the tangent space onto the rank-r manifold M r = { Z ∈ S n : rank( Z ) = r } at X . We say that X has coherence ν with respect to the basis { w α } α ∈ I of the subspace { X ∈ S n : X1 = 0 } if

where P T : S n → S n denotes the projection operator onto T and { v } α ∈ I is a dual basis of { w α } α ∈ I , which means that ⟨ w α , v β ⟩ = δ α,β for each α, β ∈ I ( δ α,β = 1 for α = β and equal to 0 otherwise).

Remark 4.2 . In [TL18, Definition 1], the coherence constant ν was required to satisfy a third condition (see [TL18, (Ineq. 14)]). However, this condition is not needed for our proofs, which is why we can use the weaker definition of Definition 4.1. Similar improvements for the standard basis incoherence notion were achieved in [Che15]. In [TL18, Lemma 21], it was shown that up constants, the definition above is equivalent to a coherence condition with respect to the standard basis [Rec11, Che15].

Following the conventional sampling approach in the existing literature [Rec11, Che15, CWB08], the index set is Ω = ( i ℓ , j ℓ ) m ℓ =1 ⊂ I contains m samples drawn uniformly at random without replacement.

(The values of the constants C ∗ = 10 5 , ˜ C = 21 √ 5 , C = 4900 are explicitly derived in the Supplementary material.)

Theorem 4.3 ( Local convergence of MatrixIRLS for EDG with Quadratic Rate ) . Let X 0 ∈ S n be a matrix of rank r that is ν -incoherent, and let A : S n → R m + n be the measurement operator corresponding to an index set Ω ⊂ I of size m = | Ω | that is drawn uniformly without replacement. There exist constants C ∗ , ˜ C and C such that the following holds. (a) If the sample complexity fulfills m ≥ Cνrn log n , and if (b) the output matrix X ( k ) ∈ S n of the k -th iteration of MatrixIRLS for EDG with inputs y = A ( X 0 ) and ˜ r = r updates the smoothing parameter in (7) such that ϵ k = σ r +1 ( X ( k ) ) and fulfills ∥ X ( k ) -X 0 ∥ S ∞ ≲ m 3 2 C ∗ κL 2 (log n ) 3 2 √ n -r σ r ( X 0 ) where κ = σ 1 ( X 0 ) /σ r ( X 0 ) is the condition number of X 0 , then the local convergence rate is quadratic in the sense that ∥ X ( k +1) -X 0 ∥ S ∞ ≤ min( µ ∥ X ( k ) -X 0 ∥ 2 S ∞ , ∥ X ( k ) -X 0 ∥ S ∞ ) with µ = ( m ˜ C 2 L log n ) 1 4(1+6 κ ) σ r ( X 0 ) and furthermore X ( k + ℓ ) ℓ →∞ - - - → X 0 with high probability.

In other words, Theorem 4.3 indicates Algorithm 1 converges to the ground truth with high probability with a sample complexity of Ω( νrn log n ) , if initialized close to the ground truth Gram matrix. We refer to Appendix D for its proof.

Our theorem's sample complexity requirement aligns with the lower bound for generic low-rank matrix completion problems, as established in [CT10]. We note that this theorem only provides a local convergence guarantee for Algorithm 1. This is in line with the strongest known results for IRLS algorithms optimizing non-convex objectives [For10, KV21, PKV22]. We provide numerical evidence in Section 5 that the minimal sample complexity assumption (a) of Theorem 4.3 indeed captures the generic reconstruction ability of the method.To the best of our knowledge, Theorem 4.3 represents the first convergence guarantee for any algorithm for Problem 1 that applies at the optimal order Ω( νrn log n ) of provided pairwise distances, and furthermore, the first theoretical guarantee for any non-convex optimization framework for Problem 1.

4.2 Dual Basis Construction and Local Restricted Isometry Property on Tangent Spaces

While a convergence result similar to Theorem 4.3 had been previously obtained for an IRLS algorithm for low-rank matrix completion [KV21], an adaptation of the proof of [KV21] to the EDG setting is not possible due to non-orthogonality of the basis { w α } α of Definition 3.2.

In order to prove Theorem 4.3, we establish a restricted isometry property of a suitably defined sampling operator (see (9)) with respect to the tangent space of the manifold of symmetric rankr matrices at the ground truth Gram matrix X 0 = P ⊤ P . To formulate this sampling operator and the respective RIP condition, we construct a dual basis to the measurement basis { w α } α of Definition 3.2.

Lemma 4.4 (Dual Basis Construction) . Let n ∈ N , I = { ( i, j ) | 1 ≤ i < j ≤ n } be the index set and { w α } α ∈ I ∪ I D be the primal basis of Definition 3.2 with I D = { ( i, i ) : i ∈ { 1 , . . . , n }} . If where a i = e i -1 n 1 for i ∈ { 1 , . . . , n } , then { v α } α ∈ I ∪ I D is a dual basis with respect to { w α } α ∈ I ∪ I D , i.e., { v α } α and { w α } α are bi-orthogonal.

Lemma 4.4 extends the dual basis construction of [TL18, LT24], in which the duality of { v α } α ∈ I with respect to { w α } α ∈ I was shown. The proof of Lemma 4.4 is detailed in Appendix B.1.

Unlike the basis pair of [TL18, LT24], our bases span the entire space of symmetric matrices S n (see Appendix B.2), which enables us to show the following restricted isometry property.

Theorem 4.5 (Restricted Isometry Property for Sampling Operator Q Ω ) . Let L = n ( n -1) / 2 , 0 < ϵ ≤ 1 2 , and Ω ⊂ I be a multiset of size m sampled independently with replacement. Define the sampling operator Q Ω : S n → S n such that

where w α and v α as in (4) and (8) , respectively. Let X 0 ∈ S n be a ν -incoherent matrix whose tangent space onto the manifold M r of symmetric rankr matrices is denoted as T 0 = T X 0 . Let P T 0 : S n → S n be the projection operator associated to T 0 . Then ∥ ∥ P T 0 Q * Ω P T 0 -P T 0 ∥ ∥ S ∞ ≤ ϵ holds with probability at least 1 -2 n provided that

The dual basis as discussed in Section 3 along with the extension for the diagonal entries together spans the space of n × n symmetric matrices. This construction has been crucial in proving the RIP restricted to the tangent space T X of rank constrained smooth manifold M r . This construction could also be valuable for analyzing other non-convex algorithms. A detailed proof of Theorem 4.5 is provided in Appendix C This proof is achieved by using concentration inequalities like the Matrix Bernstein inequality theorem C.1 in multiple lemmas stated and proved in detail in the supplemental material.

Since the existing literature for nonconvex approaches for solving Problem 1 lacks this property, this approach of establishing RIP by restricting it to the tangent space can be useful for the analysis of other nonconvex methods. The RIP condition, originally introduced in the context of compressed sensing [Can08], is a fundamental assumption in the literature on low-rank matrix recovery ([ZL12, Che15, KV21, CZ13]). A tangent-space restricted RIP has been has been useful for analyzing the convergence and performance properties of other non-convex methods [GLM16, LZ23, LHLZ20, ZLTW18] in a non-EDG setting.

5 Numerical Experiments

We evaluate the performance of MatrixIRLS, Algorithm 1, for instances of the EDG reconstruction Problem 1 in terms of data efficiency across multiple datasets in comparison to other state-of-the-art methods in the literature. We compare the performance of MatrixIRLS with three other algorithms: (a) ALM, an augmented Lagrangian method that minimizes the non-convex formulation defined by min P ∈ R r × n Tr( P ⊤ P ) subject to R Ω ( P ⊤ P ) = R Ω ( X 0 ) , with R Ω being equal to the sampling operator Q Ω restricted to Ω , which is based on a Burer-Monteiro factorization of the Gram matrix, and which has been studied in the numerical experiments of [TL18], (b) ScaledSGD [ZCZ22] which is a preconditioned stochastic gradient descent method designed to be robust with respect to ill-conditioned problems, (c) RieEDG [SLCT23] which is a Riemannian-based gradient descent approach based on the sampling operator (9) restricted to Ω . The choice of these algorithms is based on their robustness to noise as claimed in the respective papers.

5.1 Synthetic Data

We first consider a synthetic data, where we select n = 500 points P 0 = [ p 1 , . . . , p n ] ∈ R r × n from a standard Gaussian distribution at random such that ( p i ) j i.i.d. ∼ N (0 , 1) for all i, j , which

defines the ground truth Gram matrix X 0 = P 0 ⊤ P 0 . We are provided with m = | Ω | Euclidean distances, where the point index pair set Ω ⊂ { ( i, j ) ∈ [ n ] × [ n ] , i < j } is sampled uniformly at random. This is parametrized by the oversampling factor ρ = m nr -r ( r -1) / 2 where the denominator is the degrees of freedom (discussed in Appendix E.4). To understand the efficiency of the algorithm over a range of ranks r and across different oversampling factors ρ , we conduct phase transition experiments for all the above mentioned algorithms. We define a successful recovery as the case of the relative Procrustes distance d Procrustes ( P rec , P 0 ) (discussed in Appendix E.3) between the recovered matrix P rec and ground truth coordinate matrix P 0 does not exceed a tolerance threshold tol rec = 10 -3 . We chose the Procrustes distance at it is a shape preserving distance that accounts also for differences in scaling and alignment between the reconstructed geometries. We observe the performance of the algorithms as shown in the Figure 1 for ranks between 2 to 5 and a oversampling factor ranging from 1 to 4 over 24 instances.

In Figure 1, each entry on the figure represents the probability of success of an algorithm for the given rank ground truth rank r and oversampling factor ρ over 24 instances. 1 In terms of the recovery of the ground truth, we notice a comparable performance for the MatrixIRLS and ALM. However, the other two algorithms RieEDG and ScaledSGD, for the given success tolerance, the recovery of a ground truth is only possible if more samplies corresponding to a larger oversampling factor are provided, for any of the ranks r considered. This emphasizes that MatrixIRLS is able to achieve state-of-the-art data efficiency.

1

2

3

4

2

3

4

5

Oversampling factor

ρ

Rank

r

(a) MatrixIRLS

1

2

3

4

2

3

4

5

Oversampling factor

ρ

Rank

r

(b) ALM

1

2

3

4

2

3

4

5

Oversampling factor

ρ

Rank

r

(c) RieEDG

1

2

3

4

2

3

4

5

Oversampling factor

ρ

Rank

r

0

0

.

2

0

.

4

0

.

6

0

.

8

1

(d) ScaledSGD

In order to understand the scalability of the proposed algorithm, we have provided the time of completion of MatrixIRLS in Table 1. The first two rows shows that for fixed oversampling factor ρ = 3 , we are able to recover the points in the magnitude of 10 4 , is just 13 . 7 minutes. So to further test the scalability, we also looked at the time to completion when less number of samples are provided ( ρ = 2 . 5 ). In that setup n = 10000 takes 57 . 5 minutes to recover with high precision.

5.2 Ill-conditioned Data

We now assess the performance of the different EDG methods on ill-conditioned data. Ill-conditioning in the point matrix in particular arises in situations where, for example, r -dimensional points are approximately following an r -1 -dimensional geometry, a situation which often arises when the available distance information if affected by outliers [ZCZ22].

Similar to the setup above, we construct Gram matrices X 0 ∈ S n of rank r = 5 with condition number κ = σ 1 ( X 0 ) /σ r ( X 0 ) = 10 5 corresponding to n = 400 random data points P 0 ∈ R r × n generated with random orthogonal singular vectors and singular values σ i ( X 0 ) interpolated between κ and 1 with decay of order O ( i -2 ) . Figure 2 shows the success probability of different algorithms for ill-conditioned ground truths.

To have a deeper insight into the algorithm's performance, Figure 3 shows the per-iterate relative reconstructed error of the four algorithms on this ill-conditioned data at an oversampling factor of ρ = 2 for a representative problem instance. This reconstructed error refers to the Procrustes distance between the ground truth P 0 and the recovered P k at each iteration k . We observe that for ill-conditioned data, MatrixIRLS is the only method that can recover the ground truth up to a

reasonable precision, achieving a relative error of ≈ 10 -8 after around 35 iterations (top of Figure 3), whereas the other methods do not achieve errors below 10 -3 even after 100000 iterations. While one iteration of MatrixIRLS typically has a longer runtime than an iteration of any of the other algorithms due to its second-order nature, we also provide a visualization of the observed runtimes of the methods in the bottom of Figure 3. It can be seen that MatrixIRLS is able reconstruct the geometry of the challenging problem in around 200 seconds up to a high precision. Additionally, we run a study on the algorithms' behavior across different oversampling factors between ρ = 1 and ρ = 4 for 24 random problem instances, visualized in Figure 4. The box plots indicate visualize median relative Procrustes error together with the relevant 25% and 75% quantiles of the observed error distribution. We observe that MatrixIRLS has consistent convergence for ill-conditioned data even for lower oversampling rates as long as ρ ≥ 1 . 5 .

(a) MatrixIRLS

(b) ALM

(c) ScaledSGD

(d) RieEDG

8'0

Rank

0.6

3

0.4

0.2

2

2

3

Oversampling Rate

0.8

Rank

0.6

3

0.4

0.2

2

2

3

Oversampling Rate

0.8

Rank

0.6

0.4

0.2

2

CN

3

Oversampling Rate

0.25

0.2

Rank

0.15

3

0.1

0.05

2

2

3

Oversampling Rate

5

10

0

10

1

10

2

10

3

10

4

10

10

-

8

10

-

4

10

0

Iterations

Relative Error

ScaledSGD

RieEDG

ALM

MatrixIRLS

10

-

2

10

-

1

10

0

10

1

10

2

10

3

10

-

8

10

-

4

10

0

Runtime in seconds

Relative Error

ScaledSGD

RieEDG

ALM

MatrixIRLS

5.3 Real Data

To assess the performance of Algorithm 1 in realistic setups, we consider the task of molecular conformation, i.e., we aim to reconstruct the 3D structure of a molecule from partial information about the distances of its atoms. For our experiments, we determine the structures of a protein molecule (1BPM) from the Protein Data Bank [HMBFGG + 00], which is collected using X-ray diffraction experiments or nuclear magnetic resonance (NMR) spectroscopy.

The goal of this experiment is to reconstruct the point matrix P 0 from | Ω | distance samples as defined in Problem 1. For the 1BPM protein data the rank of the input matrix is 3 . We conduct the experiments corresponding to oversampling factors ρ between 1 and 4 . Like the previous setup, here successful recovery refers to the case where the relative Procrustes distance, that is the metric d Procrustes ( P rec , P 0 ) between the recovered matrix P rec and ground truth coordinate matrix P 0 does not exceed a tolerance threshold tol rec across 24 independent realizations. The error analysis for this experiment is shown in Figures 7a and 7b in Appendix E.

It is evident that MatrixIRLS and ALM have comparable results in recovering the geometry of the data given lesser samples, where as algorithms like ScaledSGD or RieEDG are unable to reconstruct geometries even when the oversampling rate is as high as ρ = 4 . In the fig. 5, we see a convergence

1

2

3

4

10

-

8

10

-

4

10

0

Oversampling

Relative Procrustes Error

ScaledSGD

RieEDG

ALM

MatrixIRLS

with high precision for MatrixIRLS from ρ ∼ 2 . 5 for Protein which means that it recovered the ground truth with around 0 . 5% samples.

In the Figure 6 the reconstructed protein in blue is aligned with the ground truth structure of the protein in pink. We can see that when the oversampling is 3 . 5 , which is when there is 0 . 6% samples available, the reconstruction exactly matches with the ground truth.

In order to understand the runtime of the different algorithms Table 2, reports the reconstruction times for each of the four algorithms applied to the 1BPM Protein data (with datapoints n = 3672 ) in a low-data regime with oversampling factor of ρ = 3 . . It can be seen that MatrixIRLS is with 7 . 08 minutes around 3 times faster than ALM, which needed 23 . 04 minutes until convergence. While these times certainly depend on the precise choice of stopping criteria for each algorithm (we used the ones indicated in Appendix E ), this shows that the proposed method is competitive in terms of clock time.

×

For the implementation of the other algorithms, we use the authors' code for the respective approaches (discussed in Appendix E). We include another set of experiments on US cities data [UU20], in the Appendix E.

6 Conclusion

In this paper, we address the challenge of reconstructing suitable geometric configurations using minimal Euclidean distance samples. By leveraging continuous and non-convex rank minimization formulations, we develop a variant of the iteratively reweighted least squares (IRLS) algorithm and establish a local convergence guarantee under the condition of a minimal random set of observed distances. Our contribution also includes the proof of a restricted isometry property (RIP) restricted to the tangent space of the manifold of symmetric rankr matrices, a result which might be of independent interest for the analysis of other non-convex methods. As future work further analysis on the global convergence can be established. Through numerical validation we conclude that the algorithm is able to achieve state-of-the-art data efficiency.

References

[ADSVF23] A. Andreella, R. De Santis, A. Vesely, and L. Finos. Procrustes-based distances for exploring between-matrices similarity. Statistical Methods & Applications , 32(3):867-882, 2023.

[AKW99] A. Y. Alfakih, A. K. Khandani, and H. Wolkowicz. Solving Euclidean Distance Matrix Completion Problems Via Semidefinite Programming. Computational Optimization and Applications , 12:13-30, 1999.

[Alf05] A. Y. Alfakih. On the uniqueness of Euclidean distance matrix completions: the case of points in general position. Linear Algebra and its Applications , 397:265277, 2005.

[ALMT14] D. Amelunxen, M. Lotz, M. B. McCoy, and J. A. Tropp. Living on the edge: Phase transitions in convex programs with random data. Information and Inference: A Journal of the IMA , 3(3):224-294, 2014.

[BA15] N. Boumal and P.-A. Absil. Low-rank matrix completion via preconditioned optimization on the Grassmann manifold. Linear Algebra and its Applications , 15(475):200-239, 2015.

[BJ95] M. Bakonyi and C. R. Johnson. The Euclidian Distance Matrix Completion Problem. SIAM J. Matrix Anal. Appl. , 16:646-654, 1995.

[BM03] S. Burer and R. D. Monteiro. A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Mathematical Programming , 95(2):329-357, 2003.

[BNZ21] J. Bauch, B. Nadler, and P. Zilber. Rank 2r iterative least squares: efficient recovery of ill-conditioned low rank matrices from few entries. SIAM J. Math. Data Sci. , 3(1):439-465, 2021.

[Bou20] N. Boumal. An introduction to optimization on smooth manifolds. Available online at http://sma.epfl.ch/~nboumal/book/IntroOptimManifolds_ Boumal_2020.pdf , November , 2020.

[Bur12] C. S. Burrus. Iterative reweighted least squares. OpenStax CNX. Available online: http://cnx. org/contents/92b90377-2b34-49e4-b26f-7fe572db78a1 , 12, 2012.

[Can08] E. J. Candès. The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique , 346(9):589-592, 2008.

[CC14] Y. Chen and Y. Chi. Robust Spectral Compressed Sensing via Structured Matrix Completion. IEEE Trans. Inf. Theory , 60(10):6576-6601, 2014.

[Che12] X. Chen. Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. , 134(1):71-99, 2012.

[Che15] Y. Chen. Incoherence-Optimal Matrix Completion. IEEE Trans. Inf. Theory , 61(5):2909-2923, 2015.

[CLC19] Y. Chi, Y. M. Lu, and Y. Chen. Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview. IEEE Trans. Signal Process. , 67(20):5239-5269, 2019.

[CR09] E. J. Candès and B. Recht. Exact matrix completion via convex optimization. Found. Comput. Math. , 9(6):717-772, 2009.

[CT10] E. J. Candès and T. Tao. The Power of Convex Relaxation: Near-Optimal Matrix Completion. IEEE Trans. Inf. Theory , 56(5):2053-2080, 2010.

[CW18] J.-F. Cai and K. Wei. Exploiting the Structure Effectively and Efficiently in LowRank Matrix Recovery. Processing, Analyzing and Learning of Images, Shapes, and Forms , 19:21 pp., 2018.

[CWB08] E. Candès, M. B. Wakin, and S. Boyd. Enhancing Sparsity by Reweighted ℓ 1 -Minimization. The Journal of Fourier Analysis and Applications , 14:877-905, 2008.

[CZ13] T. T. Cai and A. Zhang. Sharp RIP bound for sparse signal and low-rank matrix recovery. Applied and Computational Harmonic Analysis , 35(1):74-93, 2013.

[DDFG10] I. Daubechies, R. DeVore, M. Fornasier, and C. Güntürk. Iteratively Reweighted Least Squares Minimization for Sparse Recovery. Commun. Pure Appl. Math. , 63:1-38, 2010.

[DPRV15] I. Dokmanic, R. Parhizkar, J. Ranieri, and M. Vetterli. Euclidean Distance Matrices: Essential theory, algorithms, and applications. IEEE Signal Processing Magazine , 32(6):12-30, 2015.

[DR16] M. A. Davenport and J. Romberg. An Overview of Low-Rank Matrix Recovery From Incomplete Observations. IEEE J. Sel. Topics Signal Process. , 10:608-622, 2016.

[For10] M. Fornasier. Numerical Methods for Sparse Recovery. In M. Fornasier, editor, Theoretical Foundations and Numerical Methods for Sparse Recovery , volume 9 of Radon Series on Computational and Applied Mathematics , pages 93-200. De Gruyter, Berlin, 2010.

[FQX19] J. Fliege, H.-D. Qi, and N. Xiu. Euclidean distance matrix optimization for sensor network localization. In Cooperative Localization and Navigation , pages 99-126. CRC Press, 2019.

[FRW11] M. Fornasier, H. Rauhut, and R. Ward. Low-rank Matrix Recovery via Iteratively Reweighted Least Squares Minimization. SIAM J. Optim. , 21(4):1614-1640, 2011.

[GCKG23] B. Ghojogh, M. Crowley, F. Karray, and A. Ghodsi. Multidimensional Scaling, Sammon Mapping, and Isomap , pages 185-205. Springer International Publishing, Cham, 2023.

[GLM16] R. Ge, J. D. Lee, and T. Ma. Matrix Completion has No Spurious Local Minimum. In Advances in Neural Information Processing Systems (NIPS) , pages 2973-2981, 2016.

[Gow85] J. C. Gower. Properties of Euclidean and non-Euclidean distance matrices. Linear Algebra and its Applications , 67:81-97, 1985.

[GR97] I. F. Gorodnitsky and B. D. Rao. Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm. IEEE Trans. Signal Process. , pages 600-616, 1997.

[Gro11] D. Gross. Recovering Low-Rank Matrices From Few Coefficients in Any Basis. IEEE Trans. Inf. Theory , 57(3):1548-1566, 2011.

[Hen95] B. Hendrickson. The Molecule Problem: Exploiting Structure in Global Optimization. SIAM Journal on Optimization , 5(4):835-857, 1995.

[HMBFGG + 00] J. W. Helen M. Berman, Z. Feng, T. N. B. Gary Gilliland, I. N. S. Helge Weissig, and P. E. Bourne. The Protein Data Bank. Nucleic Acids Research, Volume 28, Issue 1, 1 January 2000, Pages 235-242 , 2000.

[HS52] M. R. Hestenes and E. Stiefel. Methods of Conjugate Gradients for Solving Linear Systems. Journal of research of the National Bureau of Standards , 49(1), 1952.

[JEP + 21] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature , 596(7873):583-589, 2021.

[KS18] C. Kümmerle and J. Sigl. Harmonic Mean Iteratively Reweighted Least Squares for Low-Rank Matrix Recovery. J. Mach. Learn. Res. , 19(47):1-49, 2018.

[KV21] C. Kümmerle and C. M. Verdun. A scalable second order method for ill-conditioned matrix completion from few samples. In International Conference on Machine Learning , pages 5872-5883, 2021.

[Law61] C. Lawson. Contributions to the Theory of Linear Least Maximum Approximation . Ph. D. Thesis, Univ. of Calif., Los Angeles, 1961.

[LHLZ20] Y. Luo, W. Huang, X. Li, and A. R. Zhang. Recursive Importance Sketching for Rank Constrained Least Squares: Algorithms and High-order Convergence. arXiv preprint arXiv:2011.08360 , 2020.

[Lib20] L. Liberti. Distance geometry and data science. Top , 28(2):271-339, 2020.

[LL17] R. Lai and J. Li. Solving Partial Differential Equations on Manifolds From Incomplete Inter-Point Distance, 2017.

[LLMM14] L. Liberti, C. Lavor, N. Maculan, and A. Mucherino. Euclidean Distance Geometry and Applications. SIAM Review , 56(1):3-69, 2014.

[LS24] Y. Li and X. Sun. Sensor Network Localization via Riemannian Conjugate Gradient and Rank Reduction. IEEE Transactions on Signal Processing , 2024.

[LT24] S. Lichtenberg and A. Tasissa. A dual basis approach to multidimensional scaling. Linear Algebra and its Applications , 682:86-95, 2024.

[LZ23] Y. Luo and A. R. Zhang. Low-rank Tensor Estimation via Riemannian GaussNewton: Statistical Optimality and Second-Order Convergence. The Journal of Machine Learning Research , 24(1):18274-18321, 2023.

[Meu06] G. Meurant. The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations . Society for Industrial and Applied Mathematics' 2006.

[MF10] K. Mohan and M. Fazel. Reweighted nuclear norm minimization with application to system identification. In Proceedings of the American Control Conference , pages 2953-2959. IEEE, 2010.

[Mir60] L. Mirsky. Symmetric Gauge Functions And Unitarily Invariant Norms. The Quarterly Journal of Mathematics , 11(1):50-59, 1960.

[MM15] C. Musco and C. Musco. Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition. In Advances in Neural Information Processing Systems (NIPS) , pages 1396-1404, 2015.

[MMLL19] T. Malliavin, A. Mucherino, C. Lavor, and L. Liberti. Systematic Exploration of Protein Conformational Space Using a Distance Geometry Approach. Journal of Chemical Information and Modeling , 59, 08 2019.

[MMWL22] M. Masters, A. H. Mahmoud, Y. Wei, and M. A. Lill. Deep Learning Model for Flexible and Efficient Protein-Ligand Docking. In ICLR2022 Machine Learning for Drug Discovery , 2022.

[MW97] J. J. Moré and Z. Wu. Global Continuation for Distance Geometry Problems. SIAM Journal on Optimization , 7(3):814-836, 1997.

[MWCC20] C. Ma, K. Wang, Y. Chi, and Y. Chen. Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution. Foundations of Computational Mathematics , 20:451-632, 2020.

[NKKS19] L. T. Nguyen, J. Kim, S. Kim, and B. Shim. Localization of IoT networks via lowrank matrix completion. IEEE Transactions on Communications , 67(8):5833-5847, 2019.

[PKV22] L. Peng, C. Kümmerle, and R. Vidal. Global Linear and Local Superlinear Convergence of IRLS for Non-Smooth Robust Regression. In Advances in Neural Information Processing Systems (NeurIPS) , volume 35, pages 28972-28987, 2022.

[Rec11] B. Recht. A Simpler Approach to Matrix Completion. J. Mach. Learn. Res. , 12:3413-3430, 2011.

[RFP10] B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization. SIAM Rev. , 52(3):471501, 2010.

[RXH11] B. Recht, W. Xu, and B. Hassibi. Null space conditions and thresholds for rank minimization. Mathematical programming , 127:175-202, 2011.

[SBP17] Y. Sun, P. Babu, and D. P. Palomar. Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning. IEEE Trans. Signal Process. , 65(3):794-816, 2017.

[SEJ + 20] A. W. Senior, R. Evans, J. Jumper, J. Kirkpatrick, L. Sifre, T. Green, C. Qin, A. Žídek, A. W. Nelson, A. Bridgland, et al. Improved protein structure prediction us3ing potentials from deep learning. Nature , 577(7792):706-710, 2020.

[SL16] R. Sun and Z.-Q. Luo. Guaranteed Matrix Completion via Non-Convex Factorization. IEEE Transactions on Information Theory , 62(11):6535-6579, November 2016.

[SLCT23] C. M. Smith, S. P. Lichtenberg, H. Cai, and A. Tasissa. Riemannian Optimization for Euclidean Distance Geometry. In OPT 2023: Optimization for Machine Learning , 2023.

[Ste06] M. Stewart. Perturbation of the SVD in the presence of small singular values. Linear Algebra Appl. , 419(1):53-77, 2006.

[SWBB97] D. C. Spellmeyer, A. K. Wong, M. J. Bower, and J. M. Blaney. Conformational analysis using distance geometry methods. Journal of Molecular Graphics and Modelling , 15(1):18-36, 1997.

[TL18] A. Tasissa and R. Lai. Exact reconstruction of euclidean distance geometry problem using low-rank matrix completion. IEEE Transactions on Information Theory , 65(5):3124-3144, 2018.

[Tro00] M. W. Trosset. Distance matrix completion by numerical optimization. Computational Optimization and Applications , 17:11-22, 2000.

[Tro11] J. A. Tropp. User-Friendly Tail Bounds for Sums of Random Matrices. Foundations of Computational Mathematics , 12(4):389-434, August 2011.

[TSL00] J. B. Tenenbaum, V. d. Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. science , 290(5500):2319-2323, 2000.

[TW13] J. Tanner and K. Wei. Normalized Iterative Hard Thresholding for Matrix Completion. SIAM J. Sci. Comput. , 35(5):S104-S125, 2013.

[UU20] U.S. Geological Survey and U.S. Census Bureau. US Cities Dataset. Available at https://simplemaps.com/data/us-cities , 2020.

[Van13] B. Vandereycken. Low-Rank Matrix Completion by Riemannian Optimization. SIAM J. Optim. , 23(2), 2013.

[VSP + 17] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

[WCCL20] K. Wei, J.-F. Cai, T. F. Chan, and S. Leung. Guarantees of Riemannian optimization for low rank matrix completion. Inverse Probl. Imaging , 14(2):233-265, 2020.

[Woo50] M. A. Woodbury. Inverting modified matrices. Memorandum report , 42(106):336, 1950.

[YH38] G. M. Young and A. S. Householder. Discussion of a set of points in terms of their mutual distances. Psychometrika , 3:19-22, 1938.

[ZCZ22] J. Zhang, H.-M. Chiu, and R. Y. Zhang. Accelerating SGD for Highly IllConditioned Huge-Scale Online Matrix Completion. Advances in Neural Information Processing Systems , 35:37549-37562, 2022.

[ZL12] Y.-B. Zhao and D. Li. Reweighted ℓ 1 -minimization for sparse solutions to underdetermined linear systems. SIAM J. Optim. , 22(3):1065-1088, 2012.

[ZLTW18] Z. Zhu, Q. Li, G. Tang, and M. B. Wakin. Global Optimality in Low-Rank Matrix Optimization. IEEE Trans. Signal Process. , 66(13):3614-3628, 2018.

Supplementary Material

A Limitations

For our method, the limited convergence radius leads to the convergence guarantee locally. Additionally, the runtime per iteration is larger compared to other non-convex methods [ZCZ22, TL18, SLCT23] but our method achieves convergence in less than a tenth of the number of iterations of the other methods. For some datasets like the Protein (1BPM) [HMBFGG + 00], ALM has a superior performance for oversampling factor ( ρ ) between 1 and 1 . 5 .

B Dual Basis Construction

In this section, we provide the proof the bi-orthogonality of the basis defined in Lemma 4.4 with respect to the measurement basis Definition 3.2, and furthermore establish properties of the basis pair in Appendix B.2 that will be useful later on.

B.1 Proof of Lemma 4.4

Proof of Lemma 4.4. For simplicity of the proof, we denote the two cases of Definition 3.2, separately. This helps us to divide the proof into multiple subcases.

With L = n ( n -1) 2 , let V = [ v (1 , 2) , . . . , v ( n -1 ,n ) ] ∈ R n 2 × L be a matrix whose columns enumerate the vectorized dual basis elements v α where α = ( i, j ) ∈ I , and let V E ∈ R n 2 × n consist of vectorized dual basis vectors v ( i,i ) with i ∈ { 1 , . . . , n } . Similarly, W ∈ R n 2 × L and W E ∈ R n 2 × n are defined using vectorized basis elements { w α } α ∈ I ∪ I D . As establishing duality between the two basis sets is equivalent to bi-orthogonality of the basis elements, it remains to show that the extended basis matrices ˜ V = [ V | V E ] ∈ R n 2 × L + n and ˜ W = [ W | W E ] ∈ R n 2 × L + n satisfy the relationship

where Id is the identity matrix.

In coordinate-wise notation, this means that for α = ( i, j ) and β = ( k, l ) , with 1 ≤ i ≤ j ≤ n and 1 ≤ k ≤ l ≤ n , we must show that ⟨ ˜ w α , ˜ v β ⟩ = δ α,β where ˜ w α and ˜ v β are columns of ˜ W and ˜ V respectively.

From Claim 3 . 1 of [LT24], it is clear that V is the dual of W . We require to show the biorthogonality for the extended parts.

First we show that, for α of the form ( i, i ) ,

Let us decompose the summand ⟨ e i · 1 ⊤ , a i a ⊤ i ⟩ in detail, so that we can derive the remaining calculations in a similar manner,

Putting this value back in (10), we can complete the rest of the calculation in a similar manner and finally arrive at the following,

̸

Now, let us look at the case where α = ( i, i ) , and β = ( j, j ) and i = j . Using a similar calculation we can show that,

Also, between the w α and v β where α = ( i, j ) with 1 ≤ i < j ≤ n and β = ( k, k ) with 1 ≤ k ≤ n , we can show a similar relation by

Finally, we need to show for w α with α = ( i, i ) for 1 ≤ i ≤ n and v β where β = ( k, l ) with 1 ≤ k < l ≤ n that the respective dot product is 0 . This is verified by the calculation

Hence combining the above parts we have proved that ˜ V is the dual of ˜ W .

B.2 Properties of Dual Basis

Lemma B.1. ˜ W˜ V ⊤ maps n × n symmetric matrices to itself.

Proof. Let x be a vectorized representation of a symmetric matrix X . Then, using the dual basis expansion, x can be represented as: x = ˜ W˜ V T x . We now apply the the operator ˜ W˜ V T to x .

where the last equality follows from the fact that ˜ V is dual to ˜ W .

Lemma B.2. For orthonormal basis { w α } and { v α } as defined in Definition 3.2 , the spectral norm of ˜ W˜ V ⊤ is 1, that is

Proof. By definition, ∥ ˜ W˜ V T ∥ S ∞ = max ∥ x ∥ 2 =1 ∥ ˜ W˜ V T x ∥ 2 . Note that

By construction of the dual basis, ( ˜ W T ˜ W ) = ( ˜ V T ˜ V ) -1 . Therefore, ˜ V ( ˜ W T ˜ W ) ˜ V T is an orthogonal projection operator onto the column space of ˜ V . If x is a vectorized representation of any symmetric matrix, using Lemma B.1 the projection operator is an identity operator. Hence, its operator norm is 1.

C Sampling Operator Bounds and Proof of Tangent Space Restricted Isometry Properties

In this section, we first state the matrix Bernstein [Tro11] concentration inequality for rectangular matrices, which is repeatedly used in the proofs further in the section. We then provide a proof for the bound of our sampling operator Q Ω and then we conclude this section by proving the RIP restricted to the tangent space of manifold of symmetric rankr matrices at the ground truth X 0 .

Theorem C.1 (Matrix Bernstein for rectangular matrices [Tro11, Theorem 1.6]) . Consider the finite sequence { Z k } of independent, random matrices with dimension d 1 × d 2 . Assume that each random matrix satisfies

Define

Then, for all t ≥ 0 ,

Lemma C.2 (Spectral Norm Bound for Sampling Operator Q Ω ) . Let n be the dimension of the input matrix and let Ω = ( i ℓ , j ℓ ) m ℓ =1 ⊂ I be a multiset of double indices fulfilling m < L = n ( n -1) / 2 that are sampled independently with replacement. Consequently, we have that with probability of at least 1 -2 n 2 , the sampling operator Q Ω : S n → S n of (9)

fulfills

Proof. We define,

So from (13),

So from (16), we get

In order to provide a bound for Q Ω , we first evaluate a bound for R Ω . We would use concentration inequality of Theorem C.1, for computing the bound of Q Ω Let us define

We compute the expectation of z k by,

Now let us compute the bound for z k

We use triangle inequality in the first inequality and since the spectral norm of VW ⊤ is 1, we arrive at the second inequality.

Now, we can further bound the spectral norm of v α k w ⊤ α k by,

We take the product of z k and z ∗ k as,

Now taking expectation on both side of the above relation we get;

Since expectation is a linear operator we can preserve the equality in the first equality, then we use triangle inequality in the first inequality. We further use that the spectral norm of WV ⊤ is 1 and arrive at the second inequality. Further using (17) we bound the expression in the last inequality. As per Theorem C.1 we now compute the product of z k and z ∗ k

Now taking expectation on both side of the above relation we get;

The value of the bound of z k is denoted by R in Theorem C.1. So R in this case is

Further σ is defined by max[ ∑ k E ( z k z ∗ k ) , ∑ k E ( z ∗ k z k )] as per Theorem C.1. We have,

So as per Theorem C.1,

Recalling the definition of z k from (14)

with a probability 1 -2 n exp( -t 2 2 σ 2 + Rt 3 ) . We have arrived at the above inequality from the definition of Q Ω and that the spectral norm of the extended basis of the dual pair, V E W ⊤ E is 1.

Since L = n ( n -1) 2 so, L 2 ≤ n 2 L 2 . We can further simplify the denominator by the following,

Therefore,

We arrive at the first inequality by using 1 m ≤ L m and 4 L +1 3 m ≤ 6 L 3 m

So using this relation, let us assume that t = 20 L √ log n m .

Further let us simply Equation (23), by using this value of t. Since by Theorem 4.3, m ≥ Kn log n , where K = Cνr , so log n m ≤ 1 Kn , hence we can say simplify Equation (23), further by the following,

We arrive at the third inequality bu using the expression for L and using logn m ≤ 1 Kn . Then, we can deduce the following from Equation (24),

So the probability in (21) becomes 2 n · 1 n 3 If we use (22), then we can conclude that that the spectral norm of Q Ω is bounded by ( 20 L √ log n m +1 ) . Hence, we can say that the spectral norm of Q Ω is bounded by ( 20 L √ log n m +1 ) with a probability of 1 -2 n 2 .

Now we prove the restricted isometry property of the sampling operator Q Ω with respect to the tangent space of the rankr matrix manifold at the ground truth, as stated in Theorem 4.5.

Proof of Theorem 4.5. Define z k = L m P T 0 w α k v ⊤ α k P T 0 -1 m P T 0 WV ⊤ P T 0 . Then,

Now we calculate the spectral norm of z k

The third inequality follows from the coherence bounds as in Definition 4.1 .

We use that P 2 T 0 = P T 0 in the second equality to further calculate the expectation of the above equality,

since ˜ W˜ V ⊤ = Id , we can adjust summand at the seocond equality. Defining the random operator ˜ z k := L m P T 0 w α k v ⊤ α k P T 0 , we see that z k from above satisfies z k = ˜ z k -1 m P T 0 WV ⊤ P T 0 and therefore

With this definition, and for the fact that positive semi-definite matrices satisfies

∥ A -B ∥ S ∞ ≤ max( ∥ A ∥ S ∞ , ∥ B ∥ S ∞ ) , we can bound the spectral norm of the expectation in the following way

For any M ∈ R n × n , it follows from the definition of ˜ z k

and thus

Now using the incoherence condition such as Definition 4.1 and [TL18, Lemma 22]. we can argue that

Now we get from (27)

So by Triangle Inequality,

Now we need to compute similar bounds for E [ z k z ∗ k ] .

So similar to (29) we apply Triangle Inequality on (30) and get,

Now, we can approximate this bound in the following way, Since L = n ( n -1) 2 , so L ≤ n 2 2 this bound can be written as

Comparing the equations (31) and (32), we can argue that the bound on (31) is larger than that of (32) as L 2 is larger than νr . So

The value of the bound of z k is denoted by R in Theorem C.1. So R in this case is

Further σ is defined by max E ( z k z ∗ k ) , E ( z ∗ k z k )] as per Theorem C.1. We have,

So as per Theorem C.1,

Now we would calculate the bound for the above probability, In this scenario, since L = n ( n -1) 2 , we can bound ( σ 2 + Rϵ 3 ) by the following inequality in the second,

We use ϵ 3 is less than 1 in the first inequality and the the second summand is less than νrL nm in the second inequality. Here further we used that ϵ ≤ 1 2 and ν ≥ 1 , r ≥ 1 and bound the probability as follows,

So, given that m ≥ 49 νnr ϵ 2 log n holds true, we can arrive at the first inequality above, further, since L = n ( n -1) 2 so we can say, L ≤ n 2 / 2 hence we get the second inequality. We can bound this further by,

Hence we can conclude from here that || ∑ m k =1 z k || S ∞ ≤ ϵ ) holds with a probability of 1 -2 n 3 . Now we can derive the bound ∥ ∥ P T 0 Q * Ω P T 0 -P T 0 ∥ ∥ S ∞ ≤ ϵ in the statement of Theorem 4.5 by the following argument,

We get the get implication by adjusting the equation with addition and subtraction of P T 0 w E v ⊤ E P T 0 because P T 0 WV ⊤ P T 0 + P T 0 w E v ⊤ E P T 0 = P T 0 ˜ W˜ V ⊤ P T 0 and ˜ W˜ V ⊤ P T 0 = P T 0 So, we have proved the assertion of Theorem 4.5 is true with a probability of 1 -2 n 3 .

Following Proposition 3 . 1 in [RXH11], if we change the sampling model to sampling without replacement we should have the same bound with same failure probability.

D Proof of Local Quadratic Convergence

In this section we provide the proof of the local convergence theorem as stated in Theorem 4.3.

Lemma D.1. Let 0 < ϵ ≤ 1 2 , let X 0 ∈ S n be a ν -incoherent matrix. Let P T 0 : S n → S n be the projection operator associated to T 0 . Then assume that the following three conditions hold:

(a) For Q Ω : S n → S n be defined as in (13) from m independent uniformly sampled locations, we have :

(b) The tangent space T 0 = T X 0 onto the rankr manifold T 0 = T X 0 M r fulfills :

(c) The spectral norm distance between X and X 0 fulfills:

Then the tangent space T = T X onto the rank-r manifold at X fulfills:

Lemma D.1 is a literal adaptation of [KV21, Lemma B.3] to the operator Q * Ω , which is why we omit its proof. Similarly, we can use Lemmas B.8 and B.9 of [KV21] in the same way for our proofs.

Lemma D.2. Let X 0 ∈ S n be a matrix of rank r that is ν -incoherent, and let Ω = ( i ℓ , j ℓ ) m ℓ =1 be a random index set of cardinality | Ω | = m that is sampled uniformly without replacement, or, alternatively, sampled independently with replacement. There exists constants C, ˜ C,C 1 such that if

then, with probability at least 1 -2 n 2 , the following holds: For each matrix X ( k ) ∈ S n fulfilling it follows that the projection P T k : S n → S n onto the tangent space T k := T T r ( X ( k ) ) M r satisfies

for each matrix η ∈ ker Q Ω in the null space of the operator Q Ω : S n → S n .

Proof of Lemma D.2. Assume that there are m locations Ω = ( i ℓ , j ℓ ) m ℓ =1 in n × n sampled independently uniformly with replacement , where m fulfills (38) with C := 49 /ε 2 and ε = 0 . 1 . By Lemma C.2, it follows that the corresponding operator Q Ω from Lemma C.2 fulfills

on an event called e Ω , which occurs with a probability of at least 1 -2 n , and by Theorem 4.5, the tangent space T 0 = T X 0 M r corresponding to the µ 0 -incoherent rankr matrix X 0 fulfills

on an event called e Ω ,T 0 , which occurs with a probability of at least 1 -n -2 . Let ˜ ϵ = 1 10 . If X ( k ) ∈ R n × n is such that ∥ X ( k ) -X 0 ∥ S ∞ ≤ ˜ ξσ r ( X 0 ) with

it follows by Lemma D.1 that on the event E Ω ∩ E Ω ,T 0 , the tangent space T k := X ( k ) onto the rankr manifold at X ( k ) fulfills

Next, we claim that on the event E Ω ∩ E Ω ,T 0 ,

for any for each matrix η ∈ ker Q Ω in the null space of the operator Q Ω : S n → S n . Let η ∈ ker Q Ω .

and furthermore,

Then

Using (42) in the last inequality, implies that

Dividing by ∥P T k ( η ) ∥ F on both sides we get,

Since spectral norm of P T k . is bounded by 1 . Furthermore, we used that ϵ ≤ 1 10 in the last inequality. Since η ∈ ker Q Ω , it holds that so that

where we used (40) in the last inequality. Inserting this above, we obtain

Since L m log n > 1 So we get

for the constant ˜ C defined by,

Moreover, we observe that for C 1 := 1 20 ˜ C where C 1 is the constant of (39), it holds that

implying that the two statements of Lemma D.2 are satisfied on the event E Ω ∩ E Ω ,T 0 if (39) holds. By the above mentioned probability bounds and a union bound, E Ω ∩ E Ω ,T 0 occurs with a probability of at least 1 -2 n -2 , finishing the proof for the sampling with replacement model. By the argument of Proposition 3 of [Rec11], the result extends to the model of sampling locations drawn uniformly at random without replacement, with the same probability bound. This concludes the proof of Lemma D.2.

The following lemma will also play a role in the proof of Theorem 4.3.

Lemma D.3. Let C, ˜ C,C 1 be the constants of Lemma D.2 and µ 0 be the incoherence factor of a rankr matrix X 0 . If and if η ( k ) = X ( k ) -X 0 fulfills

with

then, on the event of Lemma D.2, it holds that

Proof. First, we compute that

where 0 < ζ < 1 such that ∥ X ( k ) -X 0 ∥ S ∞ ≤ ζσ r ( X 0 ) , using Lemma D.4 twice in the fourth inequality and ∥ AB ∥ F ≤ ∥ A ∥ S ∞ ∥ B ∥ F all matrices A and B , referring to the notations of lemma B. 8 in [KV21](see below) for U 0 , Σ 0 , V 0 , U ( k ) ⊥ and V ( k ) ⊥ .

Using Lemma D.2 for η ( k ) = X ( k ) -X 0 , we obtain on the event on which the statement of Lemma D.2 holds that

since C 1 = 1 20 ˜ C , after rearranging we get,

which implies the statement of this lemma.

Lemma D.4 (Wedin's bound [Ste06]) . Let X and ̂ X be two matrices of the same size and their singular value decompositions

where the submatrices have the sizes of corresponding dimensions. Suppose that δ, α satisfying 0 < δ ≤ α are such that α ≤ σ min (Σ) and σ max ( ̂ Σ ⊥ ) < α -δ . Then

Now using the above lemma let us conclude the proof for the theorem,

Proof of Theorem 4.3. Let k = k 0 and X ( k ) be the k -th iterate of MatrixIRLS fro EDG with the parameters stated in Theorem 4.3. Under the sampling model of Theorem 4.3, if the number of samples m fulfills m ≥ Cνrn log n , where C is the constant of Lemma D.2, we know from Lemma D.2

if furthermore η ( k ) := X ( k ) -X 0 fulfills with

holds with a probability of at least 1 -2 n -2 . Then, by lemma B. 9 of [KV21],

We denote the event that this is fulfilled by E . Furthermore, on this event, if ξ ≤ 1 / 2 in (48) and denoting the condition number by κ = σ 1 ( X 0 ) /σ r ( X 0 ) , it follows from ,lemma B. 8 of [KV21], that

Furthermore, if X ( k ) r ∈ R n × n denotes the best rankr approximation of X ( k ) in any unitarily invariant norm, we estimate that

Inserting these two bounds into (50), we obtain

Finally, if, additionally, (48) is satisfied for

we conclude that

and also, we observe a quadratic decay in the spectral error such that

with a constant µ = ( m ˜ C 2 L log n ) 1 4(1+6 κ ) σ r ( X 0 ) . This shows condition b of Theorem 4.3.

To show the remaining statement, we can use Lemma D.3 to show that if X ( k ) is close enough to X 0 , we can ensure that the ( r +1) -st singular value σ r +1 ( X ( k ) ) of the current iterate is strictly decreasing. More precisely, assume now the stricter assumption of

In fact, if ξ fulfills (49) and (51), we can conclude that on the event E ,

using Lemma D.3 for one factor ∥ η ( k ) ∥ S ∞ and (52) for the other factor ∥ η ( k ) ∥ S ∞ in the third inequality, and (51) in the last inequality. Taking the update rule (7) for the smoothing parameter into account, this implies that ϵ k +1 = σ r +1 ( X ( k +1) ) , which ensures that the first statement of Theorem 4.3 is fulfilled likewise for iteration k +1 . By induction, this implies that X ( k + ℓ ) ℓ →∞ - - - → X 0 , which finishes the proof of Theorem 4.3.

E Numerical Considerations

E.1 Experiments on more real data

As a continuation of Section 5, we provide the error analysis for the 1BPM protein data whose corresponding recovery visualizations are found in Figures 7a and 7b. The figure Figure 7a shows the Procrustes distance between the recovered matrix from the samples provided and the ground-truth for both datasets. Similar to the phase transition diagram of the Gaussian data, we observe the probability of success observed over these 24 instances in fig. 7b.

1

1.5

2

2.5

3

3.5

4

10

-

8

10

-

4

10

0

Oversampling

Relative Procrustes Error

ScaledSGD

RieEDG

ALM

MatrixIRLS

1

2

3

4

0

0

.

2

0

.

4

0

.

6

0

.

8

1

Oversampling

Success Probability

ScaledSGD

RieEDG

ALM

MatrixIRLS

Next, we evaluate the performance of MatrixIRLS (Algorithm 1), on the US cities dataset [UU20] in comparison to the aforementioned algorithms. In this setup, we are given m = | Ω | Euclidean

distances √ ( λ i -λ j ) 2 +( ϕ i -ϕ j ) 2 between vectors containing longitude and latitude values λ i , λ j and ϕ i , ϕ j of n = 2920 cities in the United States, whose squares can be arranged in an incomplete distance matrix D ∈ S n that serves as an input for the reconstruction algorithms. Like the previous setup of Section 5, the set Ω ⊂ I of point index pairs is sampled uniformly at random, and we consider different choices of m parameterized by the oversampling factor ρ . Here, for the US cities the rank of the input matrix is 2 . In Figure 8a we observe a similar pattern to that of Figure 7a, for the US cities data. The success probability Figure 8b also shows a similar pattern to that Figure 4.

1

1.5

2

2.5

3

3.5

4

10

-

8

10

-

4

10

0

Oversampling

Relative Procrustes Error

Scaled SGD

ALM

Matrix IRLS

1

2

3

4

0

0

.

2

0

.

4

0

.

6

0

.

8

1

Oversampling

Success Probability

Scaled SGD

AlM

Matrix IRLS

(a) The relative Procrustes error for all the algorithms for different oversampling rates for US cities data (b) The probability of success for all the algorithms for different oversampling rates for US cities

We visualize the reconstruction of the US cities data as shown in Figure 9.

-

120

-

100

-

80

-

60

30

40

50

60

(a)

-

120

-

100

-

80

-

60

30

40

50

60

(b)

-

120

-

100

-

80

-

60

30

40

50

60

(c)

In this paper, we have adapted the authors' codes of ScaledSGD [ZCZ22], ALM [TL18] and MatrixIRLS [KV21] from their respective githubs. The code of RieEDG [SLCT23][KV21], has been obtained from the author through personal communication.

E.2 Choice of parameters

MatrixIRLS : The algorithm is configured with the following options:

The input parameter for MatrixIRLS includes the number of outer IRLS iterations N 0 , the number of inner iterations N 0 inner , the tolerance, which is the stopping criteria for the algorithm tol 0 inner . For large datasets like the UScities data and Protein data N = 400 , although the algorithm converges within 120 iterations. We run the experiments with N 0 inner = 2000 and tol inner = 10 -10 . In a smaller setup of the synthetic data we perform the experiments with n = 500 points with same parameters. although convergence of MatrixIRLS is observed much faster. To put more emphasis on the per iteration error, we study the experiment on per-iterate analysis with more iterations.

Augmented Lagrangian Method : For the Augmented Lagrangian method(ALM) we have same number iterations which is parametrized as N 0 firstorder for the Barzilai-Borwein gradient method in the code. There are 3 stopping criteria for the BB gradient method which are parametrised by xtol , ftol and gtol on the iterate, functional value of the iterate and the gradient of the function respectively. As mentioned in [TL18], the relative change in

Energy is the stopping criterion for the algorithm and the tolerance for that measure is set at 10 -10 for our experiments. Similar to the above setup, for larger dataset, more iterations are observed so that we can achieve a fair comparison between all the algorithms.

Scaled SGD : This method uses the learning rate and the number of iterations N 0 firstorder as the parameters. For all the experiments the value of N 0 firstorder are kept constant across the methods. however, based on the dimension of the input which is n , the iterations are modified to achieve a fair comparison. The learning rate is set at 0 . 2 , which is based on the respective paper [ZCZ22].

Riemmanian Gardient : This method has parameters, number of iterations which is same N 0 firstorder , the thresholding tolerance which has same value as the tol inner .

E.3 Distance metric

The success of proabbility in the experiments of Section 5, is with respect to the Procrustes distance. The Procrustes analysis uses similarity transformations like scaling, rotation and maps into a common reference frame [ADSVF23]. If we consider our problem, this distance is considered with the ground truth as the reference frame and is defined as

E.4 Degrees of freedom

̸

We count the degrees of freedom in the spectral decomposition of a rank-r symmetric matrix: A = UDU ⊤ where U is an orthogonal matrix and D is a diagonal matrix consisting of the eigenvalues of A on the diagonal. In U there are total r unit norm constraints and there are total r ( r -1) 2 constraints to the orthogonality of the columns vectors of U which follow from ⟨ u i , u j ⟩ = 0 for i = j and i ∈ { 1 , 2 , ..., r } and j ∈ { 1 , 2 , ..., r } . Therefore, the total number of constraints is r ( r -1) 2 + r = r ( r +1) 2 . The total degrees of freedom in D is r . Hence, the total degrees of freedom of A is:

F Computational Complexity

For a symmetric matrix X ∈ S n , we write its eigendecomposition (with in magnitude decaying eigenvalues) such that

where U ∈ R n × r , U ⊥ ∈ R n × ( n -r ) are matrices with orthonormal columns and Λ = diag( γσ 1 , . . . γσ r ) and Λ ⊥ := dg( γσ r +1 , . . . γσ d ) diagonal matrices with entries λ i = σ i γ i where σ i is the i -th singular value of X and γ i ∈ {± 1 } a sign. Furthermore, we denote by T r ( X ) the best rankr approximation of X , which can be written such that

where ∥ · ∥ can be any unitarily invariant norm, due to the Eckardt-Young-Mirsky theorem [Mir60], using the established notation.

The computational complexity of Algorithm 1 is dominated by the solution of the weighted least squares problem (5) and the computational of spectral information used in the update of the smoothing parameter ϵ k of (7) and the weight operator update (7) (see also Definition 3.1).

For solving (5), we consider detailed in Appendix F.1, based on solving a linear system with the dimensionality of tangent space T T r k ( X ( k ) ) M r k . The tangent space formulation can be derived from (6) via the Sherman-Morrison-Woodbury [Woo50] formula using the weight operator structure.

In order to update the smoothing parameter and the weight operator we compute the SVD of the iterate by approximating top singular vectors and corresponding values following Randomized Block Krylov Methods [MM15]. This computation takes O ((( m + nr ) r log n ) .

F.1 Tangent Space Implementation

Let M r := { X ∈ S n : rank( X ) = r } the manifold of symmetric rankr matrices and let X ∈ S n be as in (53). In this case, given r ∈ N , we can write the tangent space of M r at T r ( X ) as see also [Van13], [Bou20, Chapter 7.5].

If P T : S n → S n is the orthogonal projection operator that projects symmetric matrices onto T as used in Theorem 4.5, we note that

which can be decomposed such that where the action of P T : R r ( n + r ) → S n can be described as

with Γ 1 ∈ S r being the result of an ( r 2 × 1) to ( r × r ) reshaping of the first r 2 coordinates of γ and Γ 2 ∈ R n × r the result of an ( rn × 1) to ( n × r ) reshaping of the remaining coordinates of γ . Further, we used the notation of the adjoint operator P ∗ T : S n → R r ( n + r ) of P T which maps X ∈ S n to the vectorization γ of [Γ 1 , Γ 2 ] := { U ⊤ XU , ( I -UU ⊤ ) XU } .

Due to the fact that the weight operator W k = W X ( k ) ,ϵ k of Definition 3.1 associated to the iterate X ( k ) ∈ S n and the smoothing parameter ϵ k > 0 is self-adjoint and invertible, we can write its inverse as where D -1 k ∈ R r k ( n + r k ) × r k ( n + r k ) is a diagonal matrix with entries max( σ i ( X ( k ) ) , ϵ k ) max( σ j ( X ( k ) ) , ϵ k ) , where either i or j are smaller or equal than r k , see also [KV21, Eq. (12)], and T k = T T r k ( X ( k ) ) M r k is the tangent space at T r k ( X ( k ) ) onto the rankr k manifold.

Recall from (7) that the the defining equation for the next iterate X ( k +1) given W k is

Using (56), we see that

and, using the Sherman-Morrison-Woodbury [Woo50] formula, we have that

with linear system matrix M := ( ϵ 2 k C -1 + P ∗ T k A ∗ ( AA ∗ ) -1 A P T k ) , noting that C := ( D -1 k -ϵ 2 k I ) is invertible since ( D -1 k ) ii > ϵ 2 k for all diagonal indices. The observation of (59) can be turned an efficient implementation of the weighted least squares computing X ( k +1) , which is presented in Algorithm 2. It can be shown that Algorithm 2 indeed computes X ( k +1) implicitly, cf. Lemma F.1. We omit its proof, which follows the proof of [KV21, Lemma A.1].

Algorithm 2 Tangent space implementation of weighted least squares step in Algorithm 1

Input: Set Ω , observation distances D Ω = ( d ij ) ( i,j ) ∈ Ω , singular vectors U ∈ R d 1 × r k , V ( k ) ∈ R n × r k and singular values σ ( k ) 1 , . . . , σ ( k ) r k , smoothing parameter ϵ k , projection γ (0) k = P ∗ T k P T k -1 ( γ k -1 ) ∈ R r k ( n + r k ) of solution γ k -1 ∈ R r k -1 ( n + r k -1 ) of linear system (60) for previous iteration k -1 .

2: Solve

for ∆ γ k ∈ S k by the conjugate gradient method [HS52, Meu06], where

3: Compute γ k = γ (0) k +∆ γ k .

Output: r k +1 ∈ R m and γ k ∈ R r k ( n + r k ) .

4: Compute residual r k +1 := ( AA ∗ ) -1 ( y -A P T k ( γ k )) ∈ R m .

Lemma F.1. If r k +1 ∈ R m and γ k ∈ R r k ( n + r k ) is the output of Algorithm 2, then X ( k +1) as in step (6) of Algorithm 1 fulfills

where Γ 1 ∈ R r k × r k is the matricization of the first r 2 k elements of γ k and Γ 2 ∈ R n × r k the matricization of the reamining elements.

Algorithm 3 Implementation of U ⊤ A ∗ : R m + n → R r × n

S

S

S

Input: Input vector y = [ y 1 , y 2 , ....., y m + n ] , Ω = ( i ℓ , j ℓ ) m ℓ =1 ⊂ I be a multiset of double indices.

1

2

=

=

3

S

=

=

V

V

V

m

ℓ

=1:(

∑

m

ℓ

∑

=1:(

m

ℓ

∑

S

1

=

2

=

3

=

V

=

=1:(

1

+

U

2

S

⊤

1

2

U

1

2

U

1

V

Output:

S

⊤

⊤

+

V

Algorithm 4 Implementation of AA ∗ : R m + n → R m + n

saved as a sparse matrix.

▷

4

rm

▷ rn flops.

▷

2

rn flops

Total of rm +3 rn flops.

Input: Ω = ( i ℓ , j ℓ ) m ℓ =1 ⊂ I be a multiset of double indices fulfilling m < L = n ( n -1) / 2 that are sampled independently with replacement, the vector y = [ y 1 , y 2 , ....., y m + n ] . Load S from Algorithm 3 which computes ∗ y .

A ( )

▷ 4 m +2 n flops ▷ n flops ] ▷ n flops Output: Total of 4 m +4 n flops

ℓ

i

,j i,j

ℓ

i

ℓ

,j

-

·

)

)

∈

∈

ℓ

Ω

Ω

)

∈

S

[

y

for some for some

Ω

y

3

m

1

V

2

ℓ

i

e

ℓ

-

+1

·

[

3

S

T

.

, ..., y

y

m

+

m

+

+1

j

i

y

y

e

T

j

ℓ

n

, ..., y

V

3

]

m

ℓ

i

e

ℓ

ℓ

e

j

⊤

+

ℓ

T

i

ℓ

e

e

·

n

T

j

ℓ

1

]

⊤

·

flops.

Algorithm 5 Implementation of P ∗ T A ∗ : R m + n → R r ( n + r )

A 1 = V ∈ R r × n from Algorithm 3. r

Input: Input vector y = [ y 1 , y 2 , ....., y m + n ] ∈ R m + n , index set Ω , left singular vector matrix U ∈ R n × r .

Γ

1

M

=

A

= Γ

1

1

U

U

R

×

r

.

∈

R

r

×

n

.

Γ

2

= (

A

Output:

⊤

{

1

Γ

1

⊤

∈

-

M

Γ

,

2

⊤

}.

Lemma F.1 shows that an iterate X ( k +1) can be represented via only m + r k ( n + r k ) parameters. In the remainder of this discussion, we will assume that r k = r , which is the case in most cases in practice if the rank estimate ˜ r of Algorithm 1 is chosen as ˜ r = r .

To quantify the computational cost of Algorithm 2, we assume a fixed number N 0 inner of CG iterations solving (60). When applying the system matrix M via matrix-vector multplication, we observe that its first summand ϵ 2 k ( D -1 k -ϵ 2 k I ) -1 is diagonal and thus results in O ( r ( n + r )) flops per CG iteration. To quantify the matrix-vector multiplication cost of its second summand P ∗ T k A ∗ ( AA ∗ ) -1 A P T k , we define below algorithms that efficiently implement the application of the operators P ∗ T A ∗ : R m + n → R r ( n + r ) (Algorithm 5), AA ∗ : R m + n → R m + n (Algorithm 4) and A P T : R r ( n + r ) → R m + n (Algorithm 6). As an auxiliary function, we also provide an implementation of U ⊤ A ∗ : R m + n → R r × n (Algorithm 3) below.

The application of ( AA ∗ ) -1 : R m + n → R m + n can be achieved by an inexact iterative solver that applies Algorithm 4 a fixed number ˜ N of iterations, which costs O (( m + n ) ˜ N ) . Since the time complexity of Algorithm 5 and Algorithm 6 are O ( r 2 n + rm ) , we obtain one matrix vector multiplication with M from Algorithm 2in a time complexity of O ( r 2 n + rm +( m + n ) ˜ N ) .

Since N 0 inner CGiterations are used, we obtain a total time complexity of O ( N 0 inner ( r 2 n + rm +( m + n ) ˜ N )) for Algorithm 2. In our experiments, we observe that a small number ˜ N = 10 of iterations for solving the system associated to ( AA ∗ ) -1 is sufficient to obtain high-accuracy solutions.

This breakdown of the computational costs of each algorithm is shown in Algorithm 3 to Algorithm 6.

)

r

(

m

▷ O

▷ O

+

(

(

r

r

n

2

2

))

n

n

)

)

flops flops

flops

▷ O

(

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Main claims of the paper are weaved in Abstract and Introduction. There is a separate subsection in the Introduction dedicated to discuss the contributions of the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: In the Appendix B of this paper, we mention the limitation of this paper.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer:[Yes]

Justification: Every lemma/ theorems stated in the paper (Section 4 and appendices C and D) include the assumptions in the respective statements.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The related link to the anonymized Github respository is provided in the Appendix E

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All references of code and reference code are provided in the paper. [TODO] Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Experimental details are provided in Section 5 and discussion on the choice of parameters is provided in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer:[Yes]

Justification: In the Section 5, the experiments are performed on multiple independant realizations and respective boxplots of error are shown in Appendix E. All the figures provide statistical significance of the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The experiements were performed on a single node of a compute cluster equipped with dual 24-core Intel Xeon Gold 6248R CPUs, utilizing 32 parallel tasks.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: [TODO]

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: In the introduction and in related work we highlight the impact of the work. These provide foundational algorithmic research to reconstruction algorithms which has applications in molecular conformation, sensor network localization. There is no potential negative docial impact of this work.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper uses data that are publicly available. The data sources are cited properly in Section 5.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer:[Yes]

Justification: Throughout the paper we cite the different information that we used from the existing literature. Additionally, in the Appendix E, we provide links to the Github repositories that we used for reference.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The link to the anonymized version of the repository containing our experiments have been provided in Appendix E.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: NA

Justification: The paper does not involve crowdsourcing nor research with human subjects.r Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

[TODO]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper proposed a method called MatrixIRLS to solve the EDG problem, an important problem in various fields, especially when pairwise distance is partially observed. The proposed method improves the number of random sampled distances to guarantee local convergence to the ground truth.

STRENGTHS:
The related work is discussed comprehensively. The method and theory seem solid. Experiments are convincing.

WEAKNESSES:
1.	There are quite a few typos or undefined notation that make some of the paragraphs hard to read. For example:

  a.	Line 66, $\nu$ is not defined. Although it’s briefly mentioned in line 80, its rigorous definition is on Page 5. I suggest the following change to line 66: $O()$ (where $\nu$ is the coherence factor, see \cref{def:4.1} for details).

  b.	Line 158, $\sigma_i(\cdot)$ is not defined, tho I can guess the meaning.

  c.	Line 165: $X^{(k)}$ should be $X$. 

  d.	Eq (5) gives the update from k to k+1, while Eq (6) gives the rule from k-1 to k.

  e.	Eq(9), at the end, there is a missing comma.

2.	Figure 2 looks great, but it will be even better if we can observe the claimed rate $O(\nu rn\log n)$ empirically.

QUESTIONS:
1.	There are some vertical space issues, like Line 286 for Section 5. 
2.	Can the authors briefly discuss the main challenges for global convergence?

LIMITATIONS:
Discussed in the supplementary material. I suspect the file in the supplementary zip file should be the main submitted file.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the under-constrained problem to reconstruct n ordered points in Euclidean space R^r, which are given by fewer than all n(n-1)/2 pairwise distances.\n\nSTRENGTHS:\nThe authors should be highly praised for clearly stating main Problem 1 on page 1 and for formulating main Theorems 4.2 and 4.5 almost rigorously. \n\nProblem 1 is well-motivated by the AlphaFold approach to protein folding prediction when a protein backbone in 3D is reconstructed form partially known distances between alpha-carbon atoms.\n\nWEAKNESSES:\nDifferent from the initial Problem 1, the actual considered problem is the rank minimization in (1) after line 54. Does the paper explain anywhere why it makes sense to minimize this integer rank of a Gram matrix even if we forget about initial points. Another weakness is the practical impossibility of solving this rank minimization, which is claimed in line 54 to be NP-hard. It would be useful to re-phrase a problem in another way that allows a polynomial solution. \n\nMore importantly, the rank minimization in (1) and algorithm MatrixILRS appeared under the same name but with slightly different notations in the earlier paper [KV21]. So the problem and algorithm in the submitted paper look incremental updates of the past work, which even uses the same names, includes similar theorems, example structures and pseudo-codes.\n\nQUESTIONS:\nFor a simple example, if n=4 points in the plane are given by 4 distances that are edge lengths of a quadrilateral on these 4 points, then what's the meaning of this rank minimization in this case? Can Problem 1 be explicitly solved in some partial cases, e.g. for 5 distances on 4 points in the plane? \n\nIn Def 3.2, is w_alpha a matrix n x n? In lines 177-178, how is measurement operator <w_alpha_l,X> computed? Are angle brackets missed for l>m in line 178?  \n\nThe concept of a nu-incoherent matrix appears in Theorems 4.2, 4.5 without earlier explanations. What are these matrices?\n\nThe words \"coherence factor\" appear for the first time in line 80 without defining this factor nu. Also, nu appears earlier in line 66, still without explanations. Is it the same \"coherence\" appearing much later in Definition 4.1?\n \nThe word \"probability\" appears for the first time in Theorem 4.2, later again in Theorem 4.5. How is the probability space defined to make the convergence with \"high probability\" meaningful?\n\nIn Theorem 4.3, can mu be large enough so that the resulting inequality becomes equality and hence no convergence is guaranteed? What are the values of mu in the experiments form section 5?\n\nLines 251-252 say that \"an adaptation of the proof of [KV21] to the EDG setting is not possible due to non-orthogonality of the basis {w_alpha}_alpha of Definition 3.2\". What specific step in the past proof requires this orthogonality?\n\nLines 857-858 say that \"The Procrustes analysis uses similarity transformations like scaling, rotation and maps into a common reference frame [ADSVF23]\". However the minimization after line 859 includes only translations and orthogonal maps. Was uniform scaling missed?  Why is the minimum achieved over infinitely many possible translations? Does this Procrustes distance satisfy the metric axioms and what is its computational complexity in the number of points?\n\nThe biggest concern is line 82 saying that \"from which the geometry of the points P is trivially recovered\". Main Algorithm 1 (MatrixIRLS) outputs a matrix X^(k) through some iterative optimization. Why is this final n x n matrix realizable by n points in a Euclidean space R^r? For instance, we cannot arbitrarily choose 6 pairwise distances between 4 points in the plane. Even if these distances are positive and satisfy triangle inequalities, they should also satisfy a complicated polynomial equation saying that the tetrahedron on these 4 points has volume 0. For n>4 points, there are many more such equations, so a random symmetric matrix of positive numbers is realized by points with probability 0.\n\nLIMITATIONS:\nThis submission seems an incremental improvement of the past work [KV21] with little comparison between the two papers.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of reconstructing geometric configurations from minimal Euclidean distance samples. The authors propose an innovative algorithm based on iteratively reweighted least squares (IRLS) within a non-convex optimization framework. They provide a local convergence guarantee for the method, ensuring accurate geometry recovery from a minimal number of random distance samples. Additionally, the paper establishes a restricted isometry property (RIP) relevant to other non-convex approaches. Numerical experiments demonstrate the algorithm’s efficiency and robustness compared to state-of-the-art methods.\n\nSTRENGTHS:\n1. The paper introduces a novel application of IRLS for non-convex optimization in Euclidean Distance Geometry (EDG) problems. Establishing the local convergence guarantee and RIP is a significant theoretical contribution.\n\n2. The paper is well-researched, with rigorous theoretical analysis and proofs supporting the proposed method’s convergence and efficiency. The empirical validation through both synthetic and real-world datasets adds robustness to the claims.\n\n3. The paper is well-structured and clearly written, making complex mathematical concepts accessible. The detailed algorithmic steps enhance understanding.\n\n4. The proposed method addresses a fundamental problem in various applications, such as molecular conformation and sensor network localization. The ability to achieve accurate reconstruction with fewer samples has significant implications for data efficiency in these fields.\n\nWEAKNESSES:\n1. The paper compares the proposed method with a few state-of-the-art algorithms. Including more diverse comparisons, particularly with newer methods covered in the existing literature, would provide a more comprehensive evaluation.\n\n2. The experiments could be enhanced by adding more parameter analysis or robustness analysis. For example, testing the performance under different parameters (n, r, k) on the ill-conditioned data will further validate the effectiveness of the proposed method.\n\n3. The paper provides a local convergence guarantee, but a more detailed discussion on the global convergence properties of the algorithm would be beneficial.\n\nQUESTIONS:\n1.How sensitive is the proposed algorithm to the choice of parameters, such as rank estimate in the IRLS framework? Providing guidelines or heuristics for parameter selection would be helpful.\n\n2.In the additional experiments presented in Appendix E.1, why does the MatrixIRLS method exhibit a large standard deviation when rho is in the range of 2.5 to 3.5? This suggests that the method is not very stable or robust across all settings. Considering the success probability, ALM performs better than MatrixIRLS. Could you explain the potential reasons for this discrepancy?\n\n3.The time complexity of the MatrixIRLS algorithm appears to be missing a component in Step 7, which involves singular value decomposition with a complexity of O(n^3). Given this, what is the scalability of the MatrixIRLS method, such as the maximum problem size that can be solved within one hour?\n\n4.Will the results of the MatrixIRLS method converge to the same solution as MDS when there is no incomplete information?\n\n*Small suggestions:*\n\n1.The linewidth and point size is too large in Figs. 2, 6(b), and 7(b).\n\n2.No captions in Figs. 6 and 7.\n\n3.Mixed use of fig. and Figure.\n\n4.Some typos: \n\nLine 52: should satisfy also satsify\n\nLine 813: figure Figure\n\nLIMITATIONS:\nIt would be beneficial to include the scalability limitations, such as the maximum problem size that can be solved.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper examines the problem of recovering point locations based on incomplete pairwise Euclidean distance information. The problem is cast as a low-rank matrix recovery problem, and an algorithm based on iteratively-reweighted least squares is proposed. The authors derive a lower bound for the minimum number of required samples for reconstruction, and show that their algorithm has quadratic convergence. Experimental results show that their method has comparable results to other state-of-the-art methods for most data, but is also uniquely robust to ill-conditioned cases where other methods fail to converge.\n\nSTRENGTHS:\n* Provides the first theoretical convergence guarantee at the optimal amount of samples for the studied pairwise distance problem under non-convex optimization. \n* Performance comparable to state of the art methods are demonstrated on \n* The method converges quickly and effectively even for ill-conditioned matrices, where previous state of the art methods fail. \n* Detailed theoretical derivations and proofs. \n* An complete anonymized code repository is provided for reproducibility.\n\nWEAKNESSES:\n* Example data is somewhat small-scale and synthetic, but appears sufficient for comparison with other recovery methods. \n* While this is not a weakness of the paper, my own theoretical background is insufficient to verify some of the proofs, such as the construction of the dual basis followed by the establishment of the restricted isometry property condition.\n\nQUESTIONS:\n* While number of steps/time to convergence is shown for ill-conditioned matrix experiments, they are not shown for real data experiments. How much faster does the proposed MatrixIRLS derivative converge compared to ALM? Are they comparable?\n* Are there applicable downstream machine learning/deep learning applications that require this kind of point recovery, where improved results could be further demonstrated?\n\nLIMITATIONS:\nThe authors address the limitations of their work in detail in the paper, and there are no societal impacts to be addressed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-13"}