Patent ID: 11922640
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A non-transitory computer-readable medium having program code recorded thereon for 3D object tracking, the program code being executed by a processor and comprising:
program code to infer first 2D semantic keypoints of a 3D object within a sparsely annotated video stream by program code to link the first 2D semantic keypoints of the 3D object within a current frame of the sparsely annotated video stream into a 2D structured object geometry corresponding to a category of the 3D object, and program code to embed descriptors in the first 2D semantic keypoints of the 2D structured object geometry;
program code to match the first 2D semantic keypoints of a current frame with second 2D semantic keypoints in a next frame of the sparsely annotated video stream using the embedded descriptors in the first 2D semantic keypoints of the 2D structured object geometry within the current frame and the next frame;
program code to warp the first 2D semantic keypoints to the second 2D semantic keypoints to form warped 2D semantic keypoints in the next frame;
program code to lift the warped 2D semantic keypoints of the 2D structured object geometry in the next frame to a 3D structured object geometry;
program code to label a 3D bounding box of the 3D object in the next frame according to the lifted, warped 2D semantic keypoints of the 3D structured object geometry in the next frame; and
program code to track the 3D object in subsequent frames of the sparsely annotated video stream using the labeled 3D bounding box of the 3D object.