Patent ID: 11868583
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 11:
12. A computing system, comprising:
a head mounted computing device, including:
a camera capturing image data of an ambient environment;
a display device;
at least one processing device; and
a memory storing instructions that, when executed by the at least one processing device, cause the computing system to perform operations, including:
capturing, by the camera, image data of the ambient environment;
identifying, in the image data, a physical object in the ambient environment;
detecting an input to designate the physical object as a physical user interface apparatus;
triggering display, by the display device, of a virtual user interface as a virtual object overlaid on and attached to the physical object;
mapping at least one physical feature of the physical object to at least one virtual feature of the virtual user interface to enable coordinated movement of the virtual user interface and the physical object;
detecting a physical manipulation of the physical object designated as the physical user interface apparatus;
associating the physical manipulation of the physical object with a corresponding user interaction with the virtual user interface attached to the physical object; and
triggering a change in a control algorithm associated with the virtual user interface in response to the physical manipulation of the physical object and the corresponding user interaction with the virtual user interface.