Patent ID: 9690457
Date: 2017-06-27
CPC Classifications: G06F,G06T

Claim:
1. A method performed by a computing system having a processor, the method comprising: digitizing a vision of a scene received from an image capture device to provide a digitized scene; detecting multiple objects in the digitized scene, wherein detecting the multiple objects comprises recognizing digitized objects in the digitized scene; receiving contextual information associated with a user, wherein the contextual information comprises first contextual information and second contextual information; identifying installed applications based on the detected objects, wherein the identifying includes identifying the installed applications based on one of the first contextual information or the second contextual information, further wherein the identifying includes searching a list of attributes for an attribute associated with one or more of the detected objects, the list of attributes provided by the user and associated with at least one of the installed applications; identifying an object from the detected objects based on another one of the first contextual information or the second contextual information to provide the identified object; placing an icon that represents at least one of the identified installed applications proximate to the identified object in a display of the digitized scene, wherein the at least one of the identified installed applications is selected based on the identified object; receiving a user selection of the icon; responsive to the reception of the user selection of the icon, selecting the identified object; determining that the identified object can be interacted with, wherein the determination comprises querying a database about the identified object; obtaining at least one specification regarding an interaction with the identified object, wherein the at least one specification includes information regarding commands to interact with the identified object and information regarding user interface elements; and providing a user interface to interact with the identified object based on the at least one specification in accordance with the user interface elements.