Patent ID: 11948380
Assignee: APPLE INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A device comprising:
a camera;
a non-transitory memory; and
one or more processors to:
obtain a point cloud of a physical environment including a plurality of points, wherein each of the plurality of points is associated with a set of three-dimensional coordinates in a three-dimensional coordinate system of the physical environment, wherein the plurality of points includes a first cluster of points associated with a first semantic label;
obtain, via the camera, a two-dimensional image of the physical environment associated with a point-of-view set of three-dimensional coordinates in the three-dimensional coordinate system of the physical environment;
detect, in the two-dimensional image of the physical environment, a representation of a first object corresponding to the first semantic label;
determine a first plurality of sets of two-dimensional coordinates in a two-dimensional coordinate system of the two-dimensional image of the physical environment corresponding to the representation of the first object;
determine, from the first cluster of points associated with the first semantic label, a first plurality of sets of three-dimensional coordinates in the three-dimensional coordinate system of the physical environment respectively corresponding to the first plurality of sets of two-dimensional coordinates in the two-dimensional coordinate system of the two-dimensional image of the physical environment; and
determine a pose of the camera based on an estimate of the point-of-view set of three-dimensional coordinates based on the first plurality of two-dimensional coordinates and the first plurality of three-dimensional coordinates.