Patent ID: 11960294
Assignee: NAVER LABS CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A system, comprising:
a depth module including an encoder and a decoder and configured to:
receive a first image from a first time from a camera; and
based on the first image and using the encoder and the decoder, generate a depth map including depths between the camera and objects in the first image;

a pose module configured to:
generate a first pose of the camera based on the first image;
generate a second pose of the camera for a second time based on a second image received from the camera before the first image; and
generate a third pose of the camera for a third time based on a third image received from the camera after the first image;

a motion module configured to:
determine a first motion of the camera between the second time and the first time based on the first pose and the second pose; and
determine a second motion of the camera between the second time and the third time based on the second pose and the third pose,

wherein the depth module includes attention mechanisms configured to, based on the first image, generate an attention map including attention coefficients indicative of amounts of attention to attribute to the objects in the first image;
a first reconstruction module configured to reconstruct the second image using the attention map, the depth map, and the first motion between the second time and the first time to produce a reconstructed second image;
a second reconstruction module configured to reconstruct the third image using the attention map, the depth map, and the second motion between the second time and the third time to produce a reconstructed third image; and
a training module configured to, based on at least one of the reconstructed second image and the reconstructed third image, selectively adjust at least one parameter of at least one of depth module, the pose module, and the motion module.