Patent ID: 11875521
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G  B  H | IPC B  G  H

Claim 14:
15. A system for self-supervised depth and ego-motion estimation, the system comprising:
a multi-camera photometric loss module to determine a multi-camera photometric loss associated with a multi-camera rig of an ego vehicle;
a self-occlusion masking module to generate a self-occlusion mask by manually segmenting self-occluded areas of images captured by the multi-camera rig of the ego vehicle and to multiply the multi-camera photometric loss with the self-occlusion mask to form a self-occlusion masked photometric loss;
a depth and ego motion training module to train a depth estimation model and an ego-motion estimation model according to the self-occlusion masked photometric loss by leveraging cross-camera temporal contexts via spatio-temporal photometric constraints to increase an amount of overlap between cameras of the multi-camera rig using a predicted ego-motion of the ego vehicle; and
a full surround mono-depth (FSM) point cloud module to predict a 360° point cloud of a scene surrounding the ego vehicle according to the depth estimation model and the ego-motion estimation model.