Low-rank Matrix Recovery with Unknown Correspondence

1 INTRODUCTION . In the era of big data , one usually needs to utilize data gathered from multiple disparate platforms when accomplishing a specific task . However , the correspondence among the data samples from these different sources are often unknown due to either missing identity information or privacy reasons ( Unnikrishnan et al. , 2018 ; Gruteser et al. , 2003 ; Das & Lee , 2018 ) . Examples include the multi-image matching problem studied in ( Ji et al. , 2014 ; Zeng et al. , 2012 ; Zhou et al. , 2015 ) , the record linkage problem ( Chan & Loh , 2001 ) and the federated recommender system ( Yang et al. , 2020 ) . In the simplest scenario , we have two data matrices A = [ a1 , ... , an ] ⊤ , B = [ b1 , ... , bn ] ⊤ with ai ∈ RmA and bi ∈ RmB , which are from two different platforms ( data sources ) . As discussed above , the correspondence ( ai , bi ) may not be available , and thereby the goal is to recover the underlying correspondence between a1 , ... , an and bπ̃ ( 1 ) , ... , bπ̃ ( n ) , where π̃ ( · ) denotes an unknown permutation . We can translate such problem described above as a matrix recovery problem , i.e. , to recover the matrix M = [ A , B ] from the permuted observation Mo = [ A , P̃B ] , where P̃ ∈ Pn is an unknown permutation matrix and Pn denotes the set of all n× n permutation matrices . We term this problem as Matrix Recovery with Unknown Correspondence ( MRUC ) . Inspired by the classical low-rank model for matrix recovery ( Wright & Ma , 2021 ; Mazumder et al. , 2010 ; Hastie et al. , 2015 ) , we especially focus on the scenario where the matrix M features a certain low-rank structure . Such low-rank model has achieved great success in many applications like the recommender system ( Schafer et al. , 2007 ; Mazumder et al. , 2010 ) and the image recovery and alignment problem ( Zeng et al. , 2012 ; Zhou et al. , 2015 ) . By denoting Bo = P̃B , we want to solve the following rank minimization problem for MRUC , min P∈Pn rank ( [ A , PBo ] ) . ( 1 ) Practical applications . It is known that the recommender system often suffers from data sparsity ( Zhang et al. , 2012 ) because users typically only provide ratings for very few items . To enlarge the set of observable ratings for each user , we may harness extra data from multiple platforms ( Netflix , Amazon , Youtube , etc. ) . One classical work on this problem is the multi-domain recommender system considered in ( Zhang et al. , 2012 ) . Unfortunately , their work neglects a crucial issue that data from these diverse platforms ( or domains ) are not always well aligned for two primary reasons . The first is that the same user may use different identities , or even leave nothing about their identities , on these platforms . Another reason is that , those platforms are not allowed to share with each other the identity information about their users for preserving privacy . Another application is the visual permutation learning problem ( Santa Cruz et al. , 2017 ) , where one needs to recover the original image from the shuffled pixels . Both of the two applications give rise to a challenging extension of the MRUC problem , where we not only need to recover multiple correspondence across different data sources , but also face the difficulty of dealing with the missing values in data matrix . Relationship to the multivariate unlabeled sensing problem . Problem ( 1 ) is closely related to the Multivariate Unlabeled Sensing ( MUS ) problem , which has been studied in ( Pananjady et al. , 2017 ; Zhang et al. , 2019a ; b ; Zhang & Li , 2020 ; Slawski et al. , 2020b ; a ) . Specifically , the MUS is the multivariate linear regression problem with unknown correspondence , i.e. , it solves min P∈Pn , W∈Rm2×m1 ∥Y − PXW∥2F , ( 2 ) where W ∈ Rm2×m1 is the regression coefficient matrix , Y ∈ Rn×m1 and X ∈ Rn×m2 denotes the output and the permuted input respectively , and ∥·∥F is the matrix Frobenius norm . In fact , a concurrent work ( Yao et al. , 2021 ) studies the same rank minimization problem as ( 1 ) , but their approach is to solve it using the algorithm developed for MUS problem . Despite of the similarity to the MUS problem , we remark that MRUC problem has it own distinct features and , as shown in Section 4 , the algorithm for the MUS algorithm can not be directly and effectively applied , especially when there are multiple unknown correspondence and missing entries to be considered . Related works . To the best of our knowledge , the concurrent and independent ( Yao et al. , 2021 ) is the only work that also considers the MRUC problem . Theoretically , ( Yao et al. , 2021 ) showed that there exists an non-empty open subset U ⊆ Rn× ( m1+m2 ) , such that ∀M ∈ U , solving ( 1 ) is bound to recover the original correspondence . However , such results only prove its existence for the subset U and do not provide a concrete characterization . Regarding the algorithm design , ( Yao et al. , 2021 ) follows the idea of ( Slawski et al. , 2020b ; a ) and treats problem ( 1 ) heuristically as a MUS problem . However , there are two main drawbacks in their algorithm that largely limit its practical value . First , in the dense permutation scenario , it ignores the interaction among the shuffled columns and hence can not utilize the prior knowledge on the unknown permutation to have an improved performance ; Second , their method can not deal with data with missing values . Contributions of this work . Our contributions in this work lie in both theoretical and practical aspects . Theoretically , we are the first to rigorously study how the rank of the data matrix is perturbed by the permutation , and show that problem ( 1 ) can be used to recover a generic low-rank random matrix almost surely . Besides , we also propose a nuclear norm minimization problem as a surrogate for problem ( 1 ) . The most important theoretical result in this work is that we provide a non-asymptotic analysis to bound the error of the nuclear norm minimization problem under a mild assumption . Practically , we propose an efficient algorithm M3O that solves the nuclear norm minimization problem , which overcomes the aforementioned two shortcomings in ( Yao et al. , 2021 ) . Notably , M3O works very well even for an extremely difficult task , where we need to recover multiple unknown correspondence from the data that are densely permuted and contain missing values . We remark that this is so far a challenging problem unexplored in the existing literature . Outline . For conciseness , we will first study the MRUC problem with single unknown correspondence , and then show that the theoretical results and the algorithm can be readily extended to the more complicated scenarios . We start with building the theoretical results for ( 1 ) and its convex relaxation in Section 2 . Then , the algorithm is developed in Section 3 . The simulation results are presented in Section 4 and the conclusions are drawn in Section 5 . Notations . Given two matrices X , Y ∈ Rn×m , we denote ⟨X , Y ⟩ = ∑n i=1 ∑m j=1 XijYij as the matrix inner product . We denote X ( i ) as the ith row of the matrix X and X ( i , j ) as the element at the ith row and the jth column . We denote 1m ∈ Rm and 1n×m ∈ Rn×m as the all-one vector and matrix , respectively , and In be the n × n identity matrix . For α ∈ Rm , β ∈ Rn , we define the operator ⊕ as α⊕ β = α1⊤n + 1mβ⊤ ∈ Rm×n . We denote ∥ · ∥∗ as the nuclear norm for matrices . For vectors , we denote ∥ · ∥0 , ∥ · ∥1 as the zero norm and 1-norm respectively . 2 MATRIX RECOVERY VIA A LOW-RANK MODEL . How is the matrix rank perturbed by the row permutation ? To answer this fundamental question , we first introduce the cycle decomposition of a permutation . Definition 1 ( Cycle decomposition of a permutation ( Dummit & Foote , 1991 ) ) . Let S be a finite set and π ( · ) be a permutation on S. A cycle ( a1 , ... , an ) is a permutation sending aj to aj+1 for 1 ≤ j ≤ n− 1 and an to a1 . Then a cycle decomposition of π ( · ) is an expression of π ( · ) as a union of several disjoint cycles1 . It can be verified that any permutation on a finite set has a unique cycle decomposition ( Dummit & Foote , 1991 ) . Therefore , we can define the cycle number of a permutation π ( · ) as the number of disjoint cycles with length greater than 1 , which is denoted as C ( π ) . We also define the non-sparsity of a permutation as the Hamming distance between it and the original sequence , i.e. , H ( π ) =∑ s∈S I [ π ( s ) ̸= s ] . It is obvious that H ( π ) > C ( π ) if π is not an identity permutation . As a simple example , we consider the permutation π ( · ) that maps the sequence ( 1,2,3,4,5,6 ) to ( 3,1,2,5,4,6 ) . Now the cycle decomposition for it is π ( · ) = ( 132 ) ( 45 ) ( 6 ) , and C ( π ) = 2 , H ( π ) = 5 . In all the following theoretical results , we denote the original matrix as M = [ A , B ] ∈ Rn×m with A ∈ Rn×mA , B ∈ Rn×mB , and rank ( M ) = r , rank ( A ) = rA , rank ( B ) = rB . We denote the corresponding permutation as πP ( · ) for any permutation matrix P ∈ Pn . The following proposition says that the perturbation effect of a permutation π on the rank of M becomes stronger , if π permutes more rows and contains less cycles . Proposition 1 . ∀P ∈ Pn , we have rank ( [ A , PB ] ) ≤ min { n , m , rA + rB , r +H ( πP ) − C ( πP ) } . ( 3 ) We have similar result for the case with multiple permutations , which is summarized in Corollary 1 in Appendix A.1 . It turns out that , without any assumption on M , ( 3 ) is the tightest upper bound for the rank of a perturbed matrix . Notably , the following proposition says that the upper bound in ( 3 ) is attained with probability 1 for a generic low-rank random matrix . Definition 2 . A probability distribution on R is called a proper distribution if its density function p ( · ) is absolutely continuous with respect the Lebesgue measure on R. Proposition 2 . If the original matrix M is a random matrix with M = RE where R ∈ Rn×r and E ∈ Rr×m are two random matrices whose entries are i.i.d and follow a proper distribution on R , and r ≤ min { √ n 2 , mA , mB } , then ∀P ∈ Pn , the equality rank ( [ A , PB ] ) = min { 2r , r +H ( πP ) − C ( πP ) } ( 4 ) holds with probability 1 . Convex relaxation for the rank function . Despite the previous theoretical justification for problem ( 1 ) , it is non-convex and non-smooth . Another crucial issue is that we often have a noisy observation matrix and it is well known that the rank function is extremely sensitive to the additive noise . In this paper , we assume that the observation matrix is corrupted by i.i.d Gaussian additive noise , i.e. , Mo = [ Ao , Bo ] = [ A , P̃B ] +W , where W ( i , j ) ∼ N ( 0 , σ2 ) , where σ2 reflects the strength of the noise . We first denote the singular values of a matrix X ∈ Rn×m as σ1X , ... , σ k X where k = min { n , m } . Since rank ( X ) = ∥ [ σ1X , ... , σkX ] ∥0 , from Proposition 2 we can view the perturbation effect of a permutation to a low-rank matrix as breaking the sparsity of its singular values . This view leads naturally to the well-known 1-norm minimization problem which has been proven robust to additive noise and can yield a sparse solution ( Wright & Ma , 2021 ) , i.e. , min P∈Pn ∥ [ Ao , PBo ] ∥∗ = ∥ [ σ1Mo , ... , σ k Mo ] ∥1 . ( 5 ) Since for an arbitrary matrix , the 1-norm of its singular values is equivalent to its nuclear norm , we refer problem ( 5 ) as the nuclear norm minimization problem . Theoretical justification for the nuclear norm . Nuclear norm has a long history used as a convex surrogate for the rank , and it has been theoretically justified for applications like low-rank matrix completion ( Candès & Tao , 2010 ; Wright & Ma , 2021 ) . It is also important to see whether the nuclear norm is still a good surrogate for the rank minimization problem ( 1 ) . In this work , we establish a sufficient condition on A and B under which problem ( 5 ) is provably justified for correspondence recovery . We denote A = ∑rA i=1 σ i Au i Av i⊤ A , B = ∑rB i=1 σ i Bu i Bv i⊤ B as the singular values decomposition of A and B , where the σiA and σ i B are the non-zero singular values . Firstly , from the definition of nuclear norm , it can be simply verified for any P ∈ Pn that −Z/N ≤ ( ∥ [ A , PB ] ∥∗ − ∥M∥∗ ) /∥M∥∗ ≤ Z/N , ( 6 ) where we denote N = max { ∥A∥∗ , ∥B∥∗ } and Z = min { ∥A∥∗ , ∥B∥∗ } . The inequality ( 6 ) indicates that A and B should have comparable magnitude , i.e. , ∥A∥∗ ≈ ∥B∥∗ , otherwise the influence of the permutation will be less significant . Therefore , we are interested in the scenario where the singular values of A and B are comparable , which is described as the following Assumption 1 . 1Two cycles are disjoint if they do not have common elements Assumption 1 . There exists a constant ϵ1 ≥ 0 such that |σiA − σiB | ≤ ϵ1 , ∀i = 1 , .. , r , ( 7 ) where we denote that σiA = 0 if i > rA , and σ i B = 0 if i > rB . Similar to the matrix rank , we also need a proper low-rank assumption on the matrix M for the nuclear norm . In this work , we particularly study the scenario that the left singular vectors of A and B are similar , which we formally describe as Assumption 2 . We refer Assumption 2 as a proper low-rank assumption , because it indicates that the column space of M can be approximated by the column space of one of its submatrices . Assumption 2 . There exists a constant ϵ2 ≥ 0 such that ∥uiA − uiB∥ ≤ ϵ2 , ∀i = 1 , ... , T , ( 8 ) where we denote T = min { rA , rB } . Furthermore , we also need that all the column singular vectors u1A , ... , u T A , u 1 B , ... , u T B are variant under any P ∈ Pn with P ̸= In : we define a vector u ∈ Rn to be variant under a P ∈ Pn if Pu ̸= u . One simple and weak condition for a vector u to satisfy such property is that u dose not contains duplicated elements , which leads to the following Assumption 3 . Assumption 3 . There exists a constant ϵ3 ≥ 0 such that min u∈U min i ̸=j |u ( i ) − u ( j ) | ≥ ϵ3 > 0 , ( 9 ) where U = { u1A , ... , uTA , u1B , ... , uTB } . In summary , the assumptions mentioned above feature a typical low-rank structure in M , and implies that the nuclear norm of M is sensitive to permutation . With the three assumptions , we have the following important theorem , which provides high probability bound for the approximation error of ( 5 ) . We denote the solution to ( 5 ) as P ∗ , and let π∗ and π̃ be the corresponding permutation to the permutation matrices P ∗⊤ and P̃ , respectively . We define the difference between the two permutations π∗ and π̃ as the Hamming distance dH ( π ∗ , π̃ ) def . = n∑ i=1 I ( π∗ ( i ) ̸= π̃ ( i ) ) . Theorem 1 . Under Assumptions 1 , 2 and 3 , if additionally ϵ1 ≤ M4r , ϵ2 ≤ min { 1 2 √ 2T , √ 2M 2N } , and σ ≤ M16L2 , then the following bound for the Hamming distance dH ( π ∗ , π̃ ) ≤ 2 ϵ23 2− ( √2D D + ( √ 2 + 2 ) ϵ1r + √ 2ϵ2N + 2 √ 2DLσ − √ Tϵ2 ) 2 ( 10 ) holds with probability at least 1− 2 exp { − D8Lσ } , where L = max { n , m } , D = ∥A∥∗ + ∥B∥∗ . The proof to all the aforementioned theoretical results are provided in Appendix A.1 . Remark 1 . From Theorem 1 we can see that when ϵ3 > 0 , and ϵ1 → 0 , ϵ2 → 0 , σ → 0 , the error dH ( π∗ , π̃ ) will converge to zero with probability 1 . Furthermore , we can also discover that the correspondence can be difficult to recover when : • The rank of original matrix M is high , which can be seen from ( 10 ) . • The magnitude of A and B w.r.t rank or nuclear norm are not comparable , which can be seen from ( 6 ) and ( 7 ) . • The strength of noise is high , which can be seen from ( 10 ) and the probability in Theorem 1 . Notably , the numerical experiments in Section 4.1 corroborate our claims as well . Remark 2 . Additionally , from the proof of Theorem 1 we find that the fundamental reason for the success of ( 5 ) is that if M satisfies the previous assumptions , we have ∥ [ A , PB ] ∥∗/∥M∥∗ ≈ O ( ( 1−H ( πP ) /2n ) − 1 2 ) . ( 11 ) In many applications , we can only observe part of the full data . Therefore , it is also worthwhile to investigate whether ( 11 ) still holds when we can only access a small subset of the entries in Mo . Notably , Figure 1 gives the positive answer and shows that the relationship ( 11 ) is gracefully degraded when the percentage of observable entries is decreasing . This phenomenon is remarkable since it indicates the original correspondence can be recovered from only part of the full data . The matrices used to generate Figure 1 are the same as those in Section 4.1 , and the nuclear norm is computed approximately by first filling the missing entries using Soft-Impute algorithm ( Mazumder et al. , 2010 ) .

Summary: This paper studies the following matrix recovery problem, given an observation matrix $M_o = [A, \tilde P B]$, where $\tilde P$ is an unknown permutation matrix, recover the underlying matrix $M = [A, B]$. The underlying assumption is that the original matrix $M$ is low-rank. They study this in the settings where there are missing entries as well as in the presence of bounded additive Gaussian noise.   Prior work has more extensively considered the version of the problem of multivariate linear regression with unknown correspondence, where the goal is to solve $\min_{P, W} \|Y - PXW\|^2_F$ where $W$ are the regression coefficients, $X$ are the permuted input, and $Y$ is the output.   [1] considers the problem that is solved in this paper as well, but use an extension of the algorithm meant for the regression version of the problem which seems to suffer from the inability to extend to more aggressive noise models.   Their proof relies on studying how the permutation $P$ changes the rank of the matrix $M$, the authors demonstrate that properties of the cycle decomposition of the permutation $P$ control the rank, qualitatively, the effect of $P$ on $\text{rank}(M)$ becomes stronger the more rows it permutes and contains less cycles. The program they analyze is the standard nuclear-norm relaxation for the rank function.   They show that if (1) the singular values of $A, B$ are comparable, (2) the column-space of $M$ can be approximated by the column space of one of its submatrices and (3) The columns of $M$ do not contain any duplicated elements; then the difference between the recovered permutation and original permutation is small (as long as the rank of $M$ is small, the additive noise is bounded.   To extend this to the case of missing entries requires some technical work which changes the program to nuclear-norm regularized, frobenius norm regression on the coordinates that are present. They solve the problem via an alternating optimization approach -- the regression problem is convex in the unknown matrix $\widehat M$, and they interpret the problem of recovering the permutation $P$ as an "entropic optimal transport" problem, which is strongly convex.   The authors also run a fairly detailed set of experiments that demonstrate the performance of their algorithm.   [1] Unlabeled principal component analysis. -- Yunzhen Yao, Liangzu Peng, and Manolis C Tsakiris'21