Patent ID: 11948320
Assignee: MAGIC LEAP, INC.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 0:
1. A method for estimating depth of features in a scene from multi-view images, the method comprising:
obtaining multi-view images, including an anchor image of the scene and a set of reference images of the scene;
passing the anchor image and reference images through a shared RGB encoder and descriptor decoder which (1) outputs a respective descriptor field of descriptors for the anchor image and each reference image, (ii) detects sparse interest points in the anchor image in conjunction with relative poses to determine a search space in the reference images from alternate view-points, and (iii) outputs intermediate feature maps, wherein in the descriptor decoder utilizes a learning architecture configured to train descriptors;
sampling the respective descriptors in the search space of each reference image to determine descriptors in the search space and matching the identified descriptors with descriptors for the sparse interest points in the anchor image, such matched descriptors referred to as matched keypoints;
triangulating the matched keypoints using singular value decomposition (SVD) to output 3D points;
passing the 3D points through a sparse depth encoder to create a sparse depth image from the 3D points and output feature maps; and
a depth decoder generating a dense depth image based on the output feature maps for the sparse depth encoder and the intermediate feature maps from the RGB encoder.