Patent ID: 11948380
Assignee: APPLE INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method comprising:
at an electronic device including one or more processors and non-transitory memory:
obtaining a point cloud of a physical environment including a plurality of points, wherein each of the plurality of points is associated with a set of three-dimensional coordinates in a three-dimensional coordinate system of the physical environment, wherein the plurality of points includes a first cluster of points associated with a first semantic label;
obtaining, via a camera, a two-dimensional image of the physical environment associated with a point-of-view set of three-dimensional coordinates in the three-dimensional coordinate system of the physical environment;
detecting, in the two-dimensional image of the physical environment, a representation of a first object corresponding to the first semantic label;
determining a first plurality of sets of two-dimensional coordinates in a two-dimensional coordinate system of the two-dimensional image of the physical environment corresponding to the representation of the first object;
determining, from the first cluster of points associated with the first semantic label, a first plurality of sets of three-dimensional coordinates in the three-dimensional coordinate system of the physical environment respectively corresponding to the first plurality of sets of two-dimensional coordinates in the two-dimensional coordinate system of the two-dimensional image of the physical environment; and
determining a pose of the camera based on an estimate of the point-of-view set of three-dimensional coordinates based on the first plurality of two-dimensional coordinates and the first plurality of three-dimensional coordinates.