Patent ID: 11954886
Assignee: INTRINSIC INNOVATION LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 17:
18. A system for computing a pose of a deformable object, the system comprising:
a camera system configured to capture images of a scene from multiple viewpoints;
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a plurality of images depicting the deformable object from multiple viewpoints;
computing a class of the deformable object depicted in the images;
loading a 3-D model corresponding to the class of the deformable object;
aligning the 3-D model to the deformable object depicted in the plurality of images to compute a pose of the deformable object, including:
refining a plurality of vertex positions of a plurality of vertices of the 3-D model to match appearances of the deformable object in the images captured from the multiple viewpoints,
wherein refining the plurality of vertex positions is performed using a loss function E comprising a first component E1 and a second component E2,
wherein E1 represents pixel-level differences between an appearance of the deformable object in the images and an appearance of the deformable object in renderings of the 3-D model from viewpoints corresponding to the images, and

wherein E2 represents a degree of alignment of a silhouette of the deformable object in the images and a silhouette of the deformable object in the renderings of the 3-D model from the viewpoints corresponding to the images; and
outputting the pose of the deformable object.