Patent ID: 8633946

Claim:
A method of interacting with a mixed reality, comprising: providing a mobile device having a physical interface; using the mobile device to electronically capture image data relating to one or more real-world objects; using the image data to identify attributes associated with the one or more real-world objects; using the attributes to interact with software being operated at least in part remote to the mobile device to obtain a first command set for a first real-world object and a second command set for a second real-world object from a distal database storing a plurality of command sets for multiple, mixed reality objects, where the command sets comprise network API addresses corresponding to commands that control the first and the second real-world objects, respectively; presenting a virtual interface comprising the first and the second command sets to a user via the physical interface, where at least one of the command sets comprising a non-visible interactive command set is superimposed on an image of a corresponding real-world interface part of the corresponding real-world object and is configured to stay in proper relation to the corresponding real-world object within a mixed reality view; and allowing execution of at least one command from the non-visible interactive command set upon the user interacting directly with the image of the corresponding real-world interface part by issuing a call over a network from the mobile device to a corresponding network API address of the at least one command targeting the corresponding real-world object.