Patent ID: 11961244
Assignee: NANJING UNIVERSITY OF SCIENCE AND TECHNOLOGY
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 3:
4. The method according to claim 1, wherein the step 3 comprises the substeps of:
(1) finding 2D matching points of adjacent images by assuming that two adjacent 3D frames Fram1 and Fram2 have their corresponding 2D texture maps I1 and I2 and 3D data (xC1,yC1,zC1) and (xC2,yC2,zC2) under the coordinate system of the first camera; assuming that the current 3D frame is Fram2 and the previous 3D frame is Fram1; finding and matching the feature points of the two frames of texture maps I1 and I2 through a scale-invariant feature transformation SIFT method; and then using an Euclidean distance to eliminate some wrong 2D matching points;
(2) optimizing 2D match points by using 3D information corresponding to 2D matching points to eliminate the wrong matching points, comprising the steps of:
(a) calculating a distance from each matching point in Fram1 and Fram2 to all other matching points respectively, wherein the 3D matching point set corresponding to the 2D matching points searched by SIFT is recorded as:

P={P1,P2, . . . ,Pi, . . . ,Pn}

Q={Q1,Q2, . . . ,Q1, . . . ,Qn}

where P and Q represent the 3D point sets corresponding to the 2D matching points in Fram1 and Fram2, respectively, and n is the number of matching point pairs, i ∈ [1,n]; and
computing a distance between each Pi and all points in the point set P:

Hi(Pi)={hi(Pi,P1),hi(Pi,P2), . . . ,hi(Pi,Pj), . . . ,hi(Pi,Pn)}

where Hi(Pi) represents the set of distances between Pi and all points in the point set P, hi(Pi,Pj) represents the distance between Pi and Pj, j∈ [1,n]; and the distance set of Qi is denoted as Hi(Qi);
(b) calculating a difference between the corresponding elements between the two distance sets obtained in the step (1), wherein a difference between the corresponding elements in Hi(Pi) and Hi(Qi) is then obtained by:

Di={Di,1, . . . ,Di,j, . . . ,Di,n}

={hi(Pi,P1)=hi(Qi,Q1), . . . ,hi(Pi,Pj)−hi(Qi,Qj), . . . ,hi(pi,Pn)−hi(Qi,Qn)}

where Di represents a set of differences between corresponding elements in Hi(Pi) and Hi(Qi), and Di,j represents a difference between hi(Pi,Pj) and hi(Qi,Qj);
(c) further excluding wrong 2D matching points by comparing
the absolute value of each element Di,j in Di with a size of a threshold Dthre wherein in response to the absolute value of Di being greater than the number of elements in Dthre and greater than half of the number of matching points n/2, the point pair C and Qi is considered unreliable and is discarded, such that false matching point pairs are eliminated;
(3) obtaining an object rotation and translation matrix between adjacent 3D frames after identifying the 2D matching points by:
(a) converting a problem of solving the rotation and translation matrix into a PnP problem;
(b) using an EPnP method to solve the PnP problem and obtaining the rotation and translation matrices; and
(4) quantifying the amount of motion in each coarse registration, and keeping results of an amount of motion between 30-50.