Optimal detection of the feature matching map in presence of noise and outliers

We consider the problem of finding the matching map between two sets of d-2 dimensional vectors from noisy observations , where the second set contains outliers.3 The matching map is then an injection , which can be consistently estimated only4 if the vectors of the second set are well separated . The main result shows that,5 in the high-dimensional setting , a detection region of unknown injection may be6 characterized by the sets of vectors for which the inlier-inlier distance is of order7 at least d1/4 and the inlier-outlier distance is of order at least d1/2 . These rates8 are achieved using the estimated matching minimizing the sum of logarithms of9 distances between matched pairs of points . We also prove lower bounds establishing10 optimality of these rates . Finally , we report the results of numerical experiments on11 both synthetic and real world data that illustrate our theoretical results and provide12 further insight into the properties of the estimators studied in this work.13 1 Introduction14 Finding the best match between two clouds of points is a problem encountered in many real problems.15 In computer vision , one can look for correspondences between two sets of local descriptors extracted16 from two images . In text analysis , one can be interested in matching vector representations of the17 words of two similar texts , potentially in two different languages . The goal of the present work is to18 gain theoretical understanding of the statistical limits of the matching problem.19 In the sequel , we use the notation [ n ] = { 1 , . . . , n } for any integer n , and define ‖ · ‖ as the Euclidean20 norm in Rd . Assume that two independent sequences X = ( Xi ; i ∈ [ n ] ) and Y = ( Yi ; i ∈ [ n ] ) of21 independent vectors are generated such that Xi and Yi are drawn from the same distribution Pi on Rd,22 for every i ∈ [ n ] . The statistician observes the sequenceX and a shuffled versionX # of the sequence23 Y . More precisely , X # is such that X # i = Yπ∗ ( i ) for some unobserved permutation π ∗ . The goal of24 matching is to estimate the permutation π∗ from data ( X , X # ) . In the case of Gaussian distributions25 Pi , this problem has been studied in ( Collier and Dalalyan , 2013 , 2016 ) . Clearly , consistent estimation26 of the matching map π∗ is impossible if there are two data generating distributions Pi and Pj that are27 very close . In ( Collier and Dalalyan , 2013 , 2016 ) , a precise quantification of the separation between28 these distributions is given that enables consistent estimation of π∗ . Furthermore , it is shown that the29 permutation estimator minimizing the sum of logarithms of pairwise distances between the elements30 ofX and the elements of the shuffled versionX # is an optimal estimator of π∗.31 In this paper , we extend the model studied in ( Collier and Dalalyan , 2016 ) to the case when the setX # 32 is contaminated by outliers . The number of outliers is supposed to be known and is equal to m− n,33 where n = |X| and m = |X # | are the sizes of considered two sequences , however the indices of the34 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . outliers are unknown . All the distributions are assumed in this paper to be spherical Gaussian , although35 all the probabilistic tools used in the proofs have their sub-Gaussian counterparts . Thus , we consider36 that two spherical Gaussian distributions 1 P1 = Nd ( µ1 , σ21Id ) and P2 = Nd ( µ2 , σ22Id ) are well37 separated if the “ distance to noise ratio ” κ ( P1 , P2 ) = ‖µ1 − µ2‖/ √ σ21 + σ 2 2 is large . Main findings38 of ( Collier and Dalalyan , 2016 ) , in terms of smallest separation distance κ̄ = mini 6=j κ ( Pi , Pj ) are39 summarized in the second columns of Table 1 . Likewise , the last column of the table provides40 a summary of the contributions of the present paper in terms of κ̄in-in = mini6=j κ ( Pi , Pj ) and41 κ̄in-out = mini , j κ ( Pi , Qj ) , where Q1 , . . . , Qm−n are the distributions of the outliers.42 An unexpected finding of this work is that the “ degree ” of heteroscedasticity of the model has a strong43 impact on the separation distances and the detection regions ( sets of values of ( κ̄in-in , κ̄in-out ) for which44 it is possible to detect the feature map π∗ ) . This is in sharp contrast with the outlier-free case , where45 consistent estimation requires κ̄ to be at least of order ( d log n ) 1/4 irrespective from the behaviour46 of variances of Pi . We prove in this work that in the high dimensional regime d ≥ c log n , which is47 arguably more appealing than the low dimensional regime d ≤ c log n , the following statements are48 true:49 • If there is no heteroscedasticity , i.e. , when all the variances are equal , consistent estimation50 of π∗ is possible if and only if κ̄ = κ̄in-in ∧ κ̄in-out is at least of order ( d log ( nm ) ) 1/4 . This is51 the same rate as in the outlier-free case.52 • If the heteroscedasticity is mild , i.e. , all the variances are of the same order , the condition53 κ̄in-in & ( d log ( nm ) ) 1/4 is the same as in the previous item , but the stronger condition54 κ̄in-out & d1/2 is needed for the inlier-outlier separation distance.55 • Finally , in the general heteroscedastic setting both κ̄in-in and κ̄in-out should be at least of56 order d1/2 . Furthermore , in all these cases consistent estimation is performed by the same57 estimator : the Least Sum of Logarithms ( LSL ) .58 Note also that the empirical evaluation done in this paper shows that LSL is interesting not only from59 the theoretical but also from the practical point of view.60 Agenda Section 2 describes the framework of the vector matching problem and introduces the ter-61 minology used throughout this paper . Precise statements of the main theoretical results are gathered in62 Section 3 . The prior work is briefly discussed in Section 4 . Section 5 contains numerical experiments63 carried out both for synthetic and real data . A brief summary and some concluding remarks are64 presented in Section 6 . Proofs of all theoretical claims are deferred to the supplemental material.65 2 Problem Formulation66 We begin with formalizing the problem of matching two sequences of feature vectors ( X1 , . . . , Xn ) 67 and ( X # 1 , . . . , X # m ) with different sizes n and m such that m ≥ n ≥ 2 . In what follows , we assume68 1We use the notation Id for the d× d identity matrix that the observed feature vectors are randomly generated from the model69 { Xi = θi + σiξi , X # j = θ # j + σ # j ξ # j , i = 1 , . . . , n and j = 1 , . . . , m. ( 1 ) In this model , illustrated in Figure 1 , it is assumed that70 • θ = ( θ1 , . . . , θn ) and θ # = ( θ # 1 , . . . , θ # m ) are two sequences of vectors from Rd , corre-71 sponding to the original features , which are unavailable,72 • σ = ( σ1 , . . . , σn ) > , σ # = ( σ # 1 , . . . , σ # m ) > are positive real numbers corresponding to the73 magnitudes of the noise contaminating each feature,74 • ξ1 , . . . , ξn and ξ # 1 , . . . , ξ # m are two independent sequences of i.i.d . random vectors drawn75 from the Gaussian distribution with zero mean and identity covariance matrix.76 The simplest special case of ( 1 ) , considered in ( Collier and Dalalyan , 2016 ) , corresponds to the77 situation where a perfect matching exists between the two sequences θ and θ # . This means that78 m = n and , for some bijective mapping π∗ : [ n ] → [ n ] , θi = θ # π ( i ) for all i ∈ [ n ] . In the general case,79 both X = ( X1 , . . . , Xn ) and X # = ( X # 1 , . . . , X # m ) may contain outliers , i.e . feature vectors that80 have no corresponding pair . In such a situation , it is merely assumed that there exists a set S ⊂ [ n ] 81 and an injective mapping π∗ : S → [ m ] such that82 θi = θ # π∗ ( i ) and σi = σ # π∗ ( i ) , ∀ i ∈ S. ( 2 ) In this case we say that the vectors { Xi : i ∈ [ n ] \ S } and { X # j : j ∈ [ m ] \ π∗ ( S ) } are outliers . The83 ultimate goal is to estimate the feature matching map π∗.84 In this work we consider the case when S = [ n ] and m > n. This means that only the larger set of85 feature vectors , namelyX # , contains outliers . Let us also define the set Oπ∗ , [ m ] \ Im ( π∗ ) , which86 contains the indices of outliers and satisfies |Oπ∗ | = m− n. Naturally , the feature vectors contained87 inX , as well as those vectors fromX # that are not outliers , are called inliers.88 In this formulation , the data generating distribution is defined by the parameters θ # , σ # and π∗ . We89 omit the set of parameters θ and σ , since they are automatically identified using π∗ , θ # and σ # by90 the formula ( θi , σi ) = ( θ # π∗ ( i ) , σ # π∗ ( i ) ) for i ∈ [ n ] . Since our goal is to match the feature vectors , we91 focus our attention on the problem of estimating the parameter π∗ only , considering θ # and σ # as92 nuisance parameters . In what follows , we denote by Pθ # , σ # , π∗ the probability distribution of the93 sequence ( X1 , . . . , Xn , X # 1 , . . . , X # m ) defined by ( 1 ) under condition ( 2 ) with S = [ n ] .94 We are interested in designing estimators that have an expected error smaller than a prescribed level95 α under the weakest possible conditions on the nuisance parameter θ # and noise level σ # . Clearly,96 the problem of matching becomes more difficult with hardly distinguishable features . To quantify97 this phenomenon , we introduce the normalized separation distance κ̄in-in = κ̄in-in ( θ # , σ # , π∗ ) and98 the normalized outlier separation distance κ̄in-out = κ̄in-out ( θ # , σ # , π∗ ) , which measure the minimal99 distance-to-noise ratio between inliers and the minimal distance-to-noise ratio between inliers and100 outliers , respectively . The precise definitions read as101 κ̄in-in , min i , j 6∈Oπ∗ , j 6=i ‖θ # i − θ # j ‖ ( σ # i 2 + σ # j 2 ) 1/2 , κ̄in-out , min i 6∈Oπ∗ , j∈Oπ∗ ‖θ # i − θ # j ‖ ( σ # i 2 + σ # j 2 ) 1/2 . ( 3 ) Notice that κ̄in-in can be rewritten as102 κ̄in-in = min i , j∈ [ n ] i 6=j ‖θi − θj‖ ( σ2i + σ 2 j ) 1/2 . Clearly , if κ̄in-in = 0 or , κ̄in-out = 0 , there are two identical feature vectors in X # . In such a103 situation , assuming σi ’ s are all equal , the parameter π∗ is nonidentifiable , in the sense that there104 exist two different permutations π∗1 and π ∗ 2 such that the distributions Pθ # , σ # , π∗1 and Pθ # , σ # , π∗2105 coincide . Therefore , to ensure the existence of consistent estimators of π∗ it is necessary to impose106 the conditions κ̄in-in > 0 and κ̄in-out > 0 . Moreover , good estimators are those consistently estimating107 π∗ even if either κ̄in-in or κ̄in-out are small . We are interested here in finding the detection boundary in108 terms of the order of magnitude of ( κ̄in-in , κ̄in-out ) . More precisely , for any given α ∈ ( 0 , 1 ) we wish109 to find a regionRαn , m , d in R2 such that:110 • There is an estimator π̂n , m of π∗ satisfying Pθ # , σ # , π∗ ( π̂ 6= π∗ ) ≤ α for every ( θ # , σ # , π∗ ) 111 lying in the detection region , i.e. , for which ( κ̄in-in , κ̄in-out ) ∈ Rαn , m , d.112 • There is a constant C < 1 such that for any estimator π̄n , m of π∗ , we can find a parameter value113 ( θ # , σ # , π∗ ) in the region { ( θ # , σ # , π∗ ) : ( κ̄in-in , κ̄in-out ) ∈ CRαn , m , d } such that π̄ fails to detect114 π∗ with a probability larger than α.115 Let us make two remarks . First , note that in the outlier-free case considered in ( Collier and Dalalyan,116 2016 ) , κ̄in-out is meaningless and , therefore , the detection region is one-dimensionalRαn , m , d . Thus,117 it is necessarily a half-line and is proven to be of the form κ̄in-in ≥ C ( log n/α ) 1/2 ∨ ( d log n/α ) 1/4118 for some universal constant C. Second , the aforementioned definition of the detection region119 Rαn , m , d does not guarantee its uniqueness ( even up to a scaling by a universal constant ) . This is120 in contrast with the outlier-free case . To overcome this difficulty , we look for Rαn , m , d of the form121 [ tin-in , +∞ ) × [ tin-out , +∞ ) with the smallest possible threshold tin-out for the normalized inlier-outlier122 distance κ̄in-out.123 3 Main theoretical results124 In this section , we have collected the main theoretical findings of the paper . When the noise is125 homoscedastic , i.e. , when all σ ’ s are equal , the results obtained by Collier and Dalalyan ( 2016 ) in126 the outlier-free setting can be easily extended to the setting with outliers . Therefore , in the present127 paper , we focus on the heteroscedastic case . For the sake of clarity of exposition , we will present128 the results in the case of known variances σ , σ # prior to investigating the more interesting case of129 unknown variances.130 The detection regions we study below are based on the profile maximum likelihood estimator . The131 model presented in ( 1 ) has the parameter Ξ = ( θ # , σ # , π ) , while the observations are the sequences132 of feature vectorsX andX # . The negative log-likelihood of this model is given by133 ` n ( Ξ ; { X , X # } ) = n∑ i=1 ( ‖Xi − θ # π ( i ) ‖22 2σ # 2π ( i ) + 1 2 log ( σ # 2π ( i ) ) ) + m∑ j=1 ( ‖X # j − θ # j ‖22 2σ # 2j + 1 2 log ( σ # 2j ) ) . The profile negative log-likelihood is then defined as the minimum with respect to ( θ # , σ # ) of the134 log-likelihood ` n ( Ξ ; { X , X # } ) .135

Summary: This paper considers the problem of matching two sets of vectors where the second set consists of a noisy version of the first set (inliers) together with outliers. Under a statistical framework, the paper develops theoretical bounds on the inlier-inlier and inlier-outlier separation that enable correct matching for a class of estimators including the Least-Sum-of-Logarithms (LSL). The approach follows Collier and Dalalyan, 2016, who considered the same problem without outliers. The LSL estimator is also evaluated using synthetic data, as well as in a semi-synthetic computer vision experiment using the IMC-PT-2020 real dataset for the task of feature matching towards obtaining point correspondences from two views.