Patent ID: 11922640
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A system for 3D object tracking, the system comprising:
a 3D object detector module to infer first 2D semantic keypoints of a 3D object within a sparsely annotated video stream, to link the first 2D semantic keypoints of the 3D object within a current frame of the sparsely annotated video stream into a 2D structured object geometry corresponding to a category of the 3D object, and to embed descriptors in the first 2D semantic keypoints of the 2D structured object geometry;
a keypoint descriptor matching module to match the first 2D semantic keypoints of a current frame with second 2D semantic keypoints in a next frame of the sparsely annotated video stream using the embedded descriptors in the first 2D semantic keypoints of the 2D structured object geometry within the current frame and the next frame;
a keypoint flow model to warp the first 2D semantic keypoints to the second 2D semantic keypoints to form warped 2D semantic keypoints of the 2D structured object geometry in the next frame, and to lift the warped 2D semantic keypoints of the 2D structured object geometry in the next frame to a 3D structured object geometry; and
a 3D object tracking module to label a 3D bounding box of the 3D object in the next frame according to the lifted, warped 2D semantic keypoints of the 3D structured object geometry in the next frame, and to track the 3D object in subsequent frames of the sparsely annotated video stream using the labeled 3D bounding box of the 3D object.