Patent ID: 11893207
Assignee: APPLE INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 17:
18. A device comprising:
one or more processors;
a non-transitory memory;
one or more displays; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
obtain environmental data corresponding to a physical environment;
determine, based on the environmental data, a bounding surface of the physical environment;
detect a physical element located within the physical environment based on the environmental data;
determine a semantic label for the physical element based on at least a portion of the environmental data corresponding to the physical element;
generate a semantic construction of the physical environment based on the environmental data, wherein the semantic construction of the physical environment includes a representation of the bounding surface, a representation of the physical element and the semantic label for the physical element; and
provide the semantic construction of the physical environment to a virtual intelligent agent (VIA) that generates actions for a graphical object that represents the VIA.