METHOD AND SYSTEM FOR APPLICATION EXECUTION BASED ON OBJECT RECOGNITION FOR MOBILE DEVICES

Embodiments of the present invention enable mobile devices to behave as a dedicate remote control for different target devices through camera detection of a particular target device and autonomous execution of applications linked to the detected target device. Also, when identical target devices are detected, embodiments of the present invention may be configured to use visual identifiers and/or positional data associated with the target device for purposes of distinguishing the target device of interest. Additionally, embodiments of the present invention are capable of being placed in a surveillance mode in which camera detection procedures are constantly performed to locate target devices. Embodiments of the present invention may also enable users to engage this surveillance mode by pressing a button located on the mobile device. Furthermore, embodiments of the present invention may be trained to recognize target devices.

FIELD OF THE INVENTION

Embodiments of the present invention are generally related to the field of devices capable of image capture.

BACKGROUND OF THE INVENTION

Conventional mobile devices, such as smartphones, include the technology to perform a number of different functions. For example, a popular function available on most conventional mobile devices is the ability to use the device to control other electronic devices from a remote location. However, prior to enabling this functionality, most conventional mobile devices require users to perform a number of preliminary steps, such as unlocking the device, supplying a password, searching for the application capable of remotely controlling the target device, etc.

As such, conventional mobile devices require users to “explain” what function they wish to perform with the electronic device they wish to control. Using these conventional devices may prove to be especially cumbersome for users who wish to use their mobile devices to control a number of electronic devices, which may require users to execute a number of different applications. Accordingly, users may become weary of having to perform preliminary steps for each application and frustrated at not being able to efficiently utilize the remote control features of their mobile device.

SUMMARY OF THE INVENTION

Accordingly, a need exists for a solution that enables users to control remote electronic devices (“target devices”) using their mobile devices in a more efficient manner. Embodiments of the present invention enable mobile devices to behave as a dedicated remote controls for different target devices through camera detection of recognized target devices and autonomous execution of applications linked to those devices. Also, when identical target devices are detected, embodiments of the present invention may be configured to use visual identifiers and/or positional data associated with the target device for purposes of distinguishing the target device of interest. Additionally, embodiments of the present invention are capable of being placed in a surveillance mode in which camera detection procedures are constantly performed to locate target devices. Embodiments of the present invention may also enable users to engage this surveillance mode by pressing a button located on the mobile device. Furthermore, embodiments of the present invention may be trained to recognize target devices.

More specifically, in one embodiment, the present invention is implemented as a method of executing an application using a computing device. The method includes associating a first application with a first object located external to the computing device. Additionally, the method includes detecting the first object within a proximal distance of the computing device using a camera system. In one embodiment, the associating further includes training the computing device to recognize the first object using the camera system. In one embodiment, the detecting further includes detecting the first object using a set of coordinates associated with the first object. In one embodiment, the detecting further includes detecting the first object using signals emitted from the first object. In one embodiment, the detecting further includes configuring the computing device to detect the first object during a surveillance mode, in which the surveillance mode is engaged by a user using a button located on the computing device.

Furthermore, the method includes automatically executing the first application upon detection of the first object, in which the first application is configured to execute upon determining a valid association between the first object and the first application and detection of the first object. In one embodiment, the valid association is a mapped relationship between the first application and the first object, in which the mapped relationship is stored in a data structure resident on the computing device.

In one embodiment, the method further includes associating a second application with a second object located external to the computing device. In one embodiment, the method includes detecting the second object within a proximal distance of the computing device using a camera system. In one embodiment, the method includes automatically executing the second application upon detection of the second object, in which the second application is configured to execute upon determining a valid association between the second object and the second application and detection of the second object.

In one embodiment, the present invention is implemented as a system for executing an application using a computing device. The system includes an association module operable to associate the application with an object located external to the computing device. In one embodiment, the associating module is further operable to configure the computing device to recognize the object using machine learning procedures.

Also, the system includes a detection module operable to detect the object within a proximal distance of the computing device using a camera system. In one embodiment, the associating module is further operable to train the computing device to recognize the object using the camera system. In one embodiment, the detection module is further operable to detect the object using a set of coordinates associated with the object. In one embodiment, the detection module is further operable to detect the object using signals emitted from the object. In one embodiment, the detection module is further operable to detect the object during a surveillance mode, in which the surveillance mode is engaged by a user using a button located on the computing device.

Furthermore, the system includes an execution module operable to execute the application upon detection of the object, in which the execution module is operable to determine a valid association between the object and the application, in which the application is configured to automatically execute responsive to the valid association and said detection. In one embodiment, the valid association is a mapped relationship between the application and the object, in which the mapped relationship is stored in a data structure resident on the computing device.

In one embodiment, the present invention is implemented as a method of executing a computer-implemented system process using a computing device. The method includes associating the computer-implemented system process with an object located external to the computing device. In one embodiment, the associating further includes configuring the computing device to recognize visual identifiers located on the object responsive to a detection of similar looking objects.

The method also includes detecting the object within a proximal distance of the computing device using a camera system. In one embodiment, the associating further includes training the computing device to recognize the object using the camera system. In one embodiment, the detecting process further includes detecting the object using a set of coordinates associated with the object. In one embodiment, the detecting further includes detecting the object using signals emitted from the object. In one embodiment, the detecting further includes configuring the computing device to detect the object during a surveillance mode, in which the surveillance mode is engaged by a user using a button located on the computing device.

Furthermore, the method includes automatically executing the computer-implemented system process upon detection of the object, in which the computer-implemented system process is configured to execute upon determining a valid association between the object and the computer-implemented system process and detection of the object. In one embodiment, the valid association is a mapped relationship between the computer-implemented system process and the object, in which the mapped relationship is stored in a data structure resident on the computing device.

DETAILED DESCRIPTION

Portions of the detailed description that follow are presented and discussed in terms of a process. Although operations and sequencing thereof are disclosed in a figure herein (e.g.,FIG. 4,FIG. 5,FIG. 6,FIG. 7) describing the operations of this process, such operations and sequencing are exemplary. Embodiments are well suited to performing various other operations or variations of the operations recited in the flowchart of the figure herein, and in a sequence other than that depicted and described herein.

As used in this application the terms controller, module, system, and the like are intended to refer to a computer-related entity, specifically, either hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a module can be, but is not limited to being, a process running on a processor, an integrated circuit, an object, an executable, a thread of execution, a program, and or a computer. By way of illustration, both an application running on a computing device and the computing device can be a module. One or more modules can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers. In addition, these modules can be executed from various computer readable media having various data structures stored thereon.

Exemplary System in Accordance with Embodiments of the Present Invention

As presented inFIG. 1, an exemplary system100upon which embodiments of the present invention may be implemented is depicted. System100can be implemented as, for example, a digital camera, cell phone camera, portable electronic device (e.g., entertainment device, handheld device, etc.), webcam, video device (e.g., camcorder) and the like. Components of system100may comprise respective functionality to determine and configure respective optical properties and settings including, but not limited to, focus, exposure, color or white balance, and areas of interest (e.g., via a focus motor, aperture control, etc.). Furthermore, components of system100may be coupled via internal communications bus and may receive/transmit image data for further processing over such communications bus.

Embodiments of the present invention may be capable recognizing triggering objects within a proximal distance of system100that trigger the execution of a system process and/or application resident on system100. Triggering objects (e.g., triggering object135) may be objects located external to system100. In one embodiment, triggering objects may be electronic devices capable of sending and/or receiving commands from system100which may include, but are not limited to, entertainment devices (e.g., televisions, DVD players, set-top boxes, etc.), common household devices (e.g., kitchen appliances, thermostats, garage door openers, etc.), automobiles (e.g., car ignition/door opening devices, etc.) and the like. In one embodiment, triggering objects may also be objects (e.g., non-electronic devices) captured from scenes external to system100using a camera system (e.g., image capture of the sky, plants, animals, etc.).

Additionally, applications residing on system100may be configured to execute autonomously upon recognition of a triggering object by system100. For example, with reference to the embodiment depicted inFIG. 1, application236may be configured by the user to initialize or perform a function upon recognition of triggering object135by system100. As such, the user may capable of executing application236by focusing system100in a direction relative to triggering object135. In one embodiment, the user may be prompted by system100to confirm execution of application236. Also, in one embodiment, one triggering object may be linked to multiple applications. As such, the user may be prompted to select which application to execute by system100. Furthermore, users may be capable of linking applications to triggering objects through calibration or setup procedures using system100.

According to one embodiment of the present invention, system100may be capable of detecting triggering objects using a camera system (e.g., camera system101). As illustrated by the embodiment depicted inFIG. 1, system100may capture scenes (e.g., scene140) through lens125, which may be coupled to image sensor145. According to one embodiment, image sensor145may comprise an array of pixel sensors operable to gather image data from scenes external to system100using lens125. Image sensor145may include the functionality to capture and convert light received via lens125into a signal (e.g., digital or analog). Additionally, lens125may be placed in various positions along lens focal length115. In this manner, system100may be capable of adjusting the angle of view of lens125, which may impact the level of scene magnification for a given photographic position. In one embodiment, image sensor145may use lens125to capture images at high speed (e.g., 20 fps, 24 fps, 30 fps, or higher). Images captured may be operable for use as preview images and full resolution capture images or video. Furthermore, image data gathered from these scenes may be stored within memory150for further processing by image processor110and/or other components of system100.

Although system100depicts only lens125in theFIG. 1illustration, embodiments of the present invention may support multiple lens configurations and/or multiple cameras (e.g., stereo cameras). According to one embodiment, system100may include the functionality to use well-known object detection procedures (e.g., edge detection, greyscale matching, etc.) to detect the presence of potential triggering objects within a given scene.

According to one embodiment, users may perform calibration or setup procedures using system100which associate (“link”) applications to a particular triggering object. For example, in one embodiment, users may perform calibration or setup procedures using camera system101to capture images for use as triggering objects. As such, according to one embodiment, image data associated with these triggering objects may be stored in object data structure166. Furthermore, triggering objects captured during these calibration or setup procedures may then be subsequently linked or mapped to system process and/or an application resident on system100. In one embodiment, a user may use a system tool or linking program residing on system100to link image data associated with a triggering object (e.g., triggering object135) to a particular system process and/or application (e.g., application236) residing in memory150.

Furthermore, for identical or similar looking triggering objects, embodiments of the present invention may also be configured to recognize visual identifiers or markers to resolve which trigging object is of interest to an application. For example, visual identifiers may be unique identifiers associated with a particular triggering object. For instance, unique visual identifiers may include, but are not limited to, serial numbers, barcodes, logos, etc. In one embodiment, visual identifiers may not be unique. For instance, visual identifiers may be generic labels (e.g., stickers) affixed to a trigging object by the user for purposes of training system100to distinguish similar looking triggering objects. Furthermore, data used by system100to recognize visual identifiers may be predetermined using a priori data loaded in memory resident on system101in factory. In one embodiment, users may perform calibration or setup procedures using camera system101to identify visual identifiers or markers. According to one embodiment, the user may be prompted to resolve multiple triggering objects detected within a given scene. For instance, in one embodiment, system100may prompt the user via the display device111of system100(e.g., viewfinder of a camera device) to select a particular triggering object among a number of recognized triggering objects detected within a given scene. In one embodiment, the user may make selections using touch control options (e.g., “touch-to-focus”, “touch-to-record”) made available by the camera system.

According to one embodiment, system100may be configured to recognize triggering objects using machine-learning procedures. For example, in one embodiment, system100may gather data that correlates application execution patterns with objects detected by system100using camera system101. Based on the data gathered, system100may learn to associate certain applications with certain objects and store the learned relationship in a data structure (e.g., object data structure166).

Object data structure166may include the functionality to store data mapping the relationship between triggering objects and their respective applications. For example, in one embodiment, object data structure166may be a data structure capable of storing mapping data indicating the relationship between various differing triggering objects and their respective applications. Object recognition module165may include the functionality to receive and compare image data gathered by camera system101to image data associated with recognized triggering objects stored in object data structure166.

For instance, according to one embodiment, image data stored in object data structure166may consist of pixel values (e.g., RGB values) associated with various triggering objects recognized (e.g., through training or calibration) by system100. As such, object recognition module165may compare the pixel values of interesting objects detected using camera system101(e.g., from image data gathered via image sensor145) to the pixel values of recognized triggering objects stored within object data structure166. In one embodiment, if the pixel values of an interesting object are within a pixel value threshold of a recognized triggering object stored within object data structure166, object recognition module165may make a determination that the interesting object detected is the recognized triggering object and then may proceed to perform a lookup of any applications linked to the recognized triggering object detected. It should be appreciated that embodiments of the present invention are not limited by the manner in which pixel values are selected and/or calculating for analysis by object recognition module165(e.g., averaging RGB values for selected groups of pixels).

Embodiments of the present invention may also be capable of detecting triggering objects based on information concerning the current relative position of system100with respect to the current location of a triggering object. With further reference to the embodiment depicted inFIG. 1, system100may be capable of detecting triggering objects using orientation module126and/or GPS module125. Orientation module126may include the functionality to determine the orientation of system100. According to one embodiment, orientation module126may use geomagnetic field sensors and/or accelerometers (not pictured) coupled to system100to determine the orientation of system100. Additionally, GPS module125may include the functionality to gather coordinate data (e.g., latitude, longitude, elevation, etc.) associated with system100at a current position using conventional global positioning system technology. In one embodiment, GPS module125may be configured to use coordinates provided by a user that indicate the current location of the triggering object so that system100may gauge its position with respect to the triggering object.

According to one embodiment, object recognition module165may include the functionality to receive and compare coordinate data gathered by orientation module126and/or GPS module125to coordinate data associated with recognized triggering objects stored in object data structure166. For instance, according to one embodiment, data stored in object data structure166may include 3 dimensional coordinate data (e.g., latitude, longitude, elevation) associated with various triggering objects recognized by system100(e.g., coordinate data provided by a user). As such, object recognition module165may compare coordinate data calculated by orientation module126and/or GPS module125providing the current relative position of system100to coordinate data associated with recognized triggering objects stored within object data structure166. In one embodiment, if the values calculated by orientation module126and/or GPS module125place system100within a proximal distance threshold of a recognized triggering object stored within object data structure166, object recognition module165may make a determination that system100is in proximity to that particular triggering object detected and then may proceed to perform a lookup of any applications linked to the triggering object detected. It should be appreciated that embodiments of the present invention are not limited by the manner in which orientation module126and/or GPS module125calculates the current relative position of system100.

In one embodiment, users may perform calibration or setup procedures using orientation module126and/or GPS module125to determine locations for potential triggering objects. For instance, in one embodiment, a user may provide latitude, longitude, and/or elevation data concerning various triggering objects to system100for use in subsequent triggering object detection procedures. Furthermore, triggering objects locations determined during these calibration or setup procedures may then be subsequently mapped to an application resident on system100by a user.

According to one embodiment, system100may use data gathered from a camera system coupled to system100as well as any positional and/or orientation information associated with system100for purposes of accelerating the triggering object recognition process. For example, according to one embodiment, coordinate data associated with recognized triggering objects may be used in combination with camera system101to accelerate the recognition of triggering objects. As such, similar looking triggering objects located in different regions of a given area (e.g., similar looking televisions placed in different rooms of a house) may be distinguished by embodiments of the present invention in a more efficient manner.

Exemplary Methods of Application Execution Based on Object Recognition in Accordance with Embodiments of the Present Invention

FIG. 2Adepicts an exemplary triggering object detection process using a camera system in accordance with embodiments of the present invention. As described herein, system100may be capable of detecting potential triggering objects using a camera system (e.g., camera system101). As illustrated inFIG. 2A, system100may be placed in a surveillance mode in which camera system101surveys scenes external to system100for potential triggering objects (e.g., detected objects134-1,134-2,134-3). In one embodiment, system100may be engaged in this surveillance mode by pressing object recognition button103. Object recognition button103may be implemented as various types of buttons including, but not limited to, capacitive touch buttons, mechanical buttons, virtual buttons, etc. In one embodiment, system100may be configured to operate in a mode in which system100is constantly surveying scenes external to system100for potential triggering objects and, thus, may not require user intervention for purposes of engaging system100in a surveillance mode.

FIG. 2Bdepicts an exemplary triggering object recognition process in accordance with embodiments of the present invention. As described herein, applications mapped in object data structure166may be configured to execute autonomously immediately upon recognition of their respective triggering objects by object recognition module165. As illustrated inFIG. 2B, camera system101may also be capable of providing object recognition module165with image data associated with detected objects134-1,134-2, and/or134-3(e.g., captured via image sensor145). As such, object recognition module165may be operable to compare the image data received from camera system101(e.g., image data associated with detected objects134-1,134-2,134-3) to the image data values of recognized triggering objects stored in object data structure166. As illustrated inFIG. 2B, after performing comparison operations, object recognition module165may determine that detected object134-2is triggering object135-1.

FIG. 2Cdepicts an exemplary data structure capable of storing mapping data associated with triggering objects and their respective applications in accordance with embodiments of the present invention. As illustrated inFIG. 2C, each triggering object (e.g., triggering objects135-1,135-2,135-3,135-4, etc.) may be mapped to an application (e.g., applications236-1,236-2,236-3,236-4, etc.) in memory resident on system100(e.g., memory locations150-1,150-2,150-3,150-4, etc.). With further reference toFIG. 2B, object recognition module165may scan object data structure166and determine that triggering object135-1is mapped to application236-1.

Accordingly, as illustrated inFIG. 2D, application236-1, depicted as a television remote control application, may be executed in an autonomous manner upon recognition of triggering object135-1by object recognition module165. As such, the user may be able to engage triggering object135-1(depicted as a television) in a manner consist with triggering object135-1's capabilities. For example, the user may be able to use application236-1to turn on triggering object135-1, change triggering object135-1's channels, adjust triggering object135-1's volume, etc.

Although a single application is depicted as being executed by system100inFIG. 2D, embodiments of the present invention are not limited as such. For instance, in one embodiment, system100may be operable to detect multiple triggering objects and execute multiple actions simultaneously in response to their detection (e.g., control several external devices simultaneously). For example, with reference to the embodiment depicted inFIG. 2D, in addition to detecting the triggering object135-1, system100may be configured to simultaneously recognize a DVD triggering object also present in the scene. As such, system100may be configured to execute each triggering object's respective application simultaneously (e.g., execute both a television remote control application and a DVD remote control application at the same time). Furthermore, embodiments of the present invention may be configured to execute a configurable joint action between two detected triggering objects in a given scene. For example, in one embodiment, upon detection of both a television triggering object (e.g., triggering object135-1) and a DVD triggering object, system100may be configured to prompt the user to perform a pre-configured joint action using both objects in which system100may be configured to turn on both the television triggering object and the DVD triggering object and execute a movie (e.g., the television triggering object may be pre-configured to take the DVD triggering object as a source).

FIG. 2Edepicts an exemplary triggering object recognition process in which non-electronic devices are recognized in accordance with embodiments of the present invention. As described herein, triggering objects may also be non-electronic devices captured from scenes external to system100using a camera system. For instance, as illustrated inFIG. 2E, triggering objects captured by system using camera system101may include objects such as the sky (e.g., scene134-4). In a manner similar to the various embodiments described herein, object recognition module165may compare the image data received from camera system101(e.g., image data associated with scene134-4) to the image data values of recognized triggering objects stored in object data structure166. Furthermore, as illustrated inFIG. 2E, after performing comparison operations, object recognition module165may determine that scene134-4is a recognized triggering object and may correspondingly execute application236-3(depicted as a weather application) in an autonomous manner.

FIG. 3Adepicts an exemplary data structure capable of storing coordinate data associated with triggering objects, along with their respective application mappings, in accordance with embodiments of the present invention. As illustrated inFIG. 3A, data stored in object data structure166may consist of 3 dimensional coordinate data (e.g., latitude, longitude, elevation) associated with triggering objects recognized by system100. Furthermore, as illustrated inFIG. 3A, each triggering object may be mapped to an application (applications236-1,236-2,236-3,236-4, etc.) in memory (e.g., memory locations150-1,150-2,150-3,150-4, etc.). In this manner, object recognition module165may use orientation module126and/or GPS module125to determine whether a triggering object is within a proximal distance of system100.

According to one embodiment, a user may provide object recognition module165(e.g., via GUI displayed on display device111) with coordinate data indicating the current location of triggering objects (e.g., coordinate data for triggering objects135-1,135-2,135-3,135-4) so that system100may gauge its position with respect to a particular triggering object at any given time. In this manner, using real-time calculations performed by orientation module126and/or GPS module125regarding the current position of system100, object recognition module165may be capable of determining whether a particular triggering object (or objects) is within a proximal distance of system100and may correspondingly execute an application mapped to that triggering object.

FIG. 3Bdepicts an exemplary triggering object recognition process using spatial systems in accordance with embodiments of the present invention. As illustrated inFIG. 3B, object recognition module165may use real-time calculations performed by orientation module126and/or GPS module125to determine the current position of system100. As depicted inFIG. 3B, orientation module126and/or GPS module125may calculate system100's current position (e.g., latitude, longitude, elevation) as coordinates (a,b,c). Upon the completion of these calculations, object recognition module165may compare the coordinates calculated to coordinate data stored in object data structure166. As illustrated inFIG. 3B, object recognition module165may scan the mapping data stored in object data structure166and execute application236-1, which was linked to triggering object135-1(see object data structure166ofFIG. 3A), after recognizing system100being within a proximal distance of triggering object135-1. According to one embodiment, in a manner similar to the embodiment depicted inFIG. 2Adescribed supra, system100may be placed in a surveillance mode in which triggering objects are constantly searched for using orientation module126and/or GPS module125based on the coordinate data associated with recognized triggering objects stored in object data structure166. In this manner, according to one embodiment, this surveillance may be performed independent of a camera system (e.g., camera system101).

FIG. 3Cdepicts an exemplary triggering object recognition process using signals emitted from a triggering object in accordance with embodiments of the present invention. As illustrated by the embodiment depicted inFIG. 3C, triggering object135-1may be a device (e.g., television) capable of emitting signals that may be detected by a receiver (e.g., antenna106) coupled to system100. Furthermore, as illustrated inFIG. 3C, object recognition module165may compare data received from signals captured via antenna106to signal data associated with recognized triggering objects stored in object data structure166. According to one embodiment, signal data may include positional information, time and/or other information associated with triggering objects. Additionally, in one embodiment, signal data stored in object data structure166may include data associated with signal amplitudes, frequencies, or other characteristics capable of distinguishing signals received from multiple triggering objects. Also, according one embodiment, system100may notify the user that signals were received from multiple triggering objects and may prompt the user to confirm execution of applications mapped those triggering objects detected.

As illustrated inFIG. 3C, object recognition module165may scan the mapping data stored in object data structure166and then correspondingly execute application236-1after recognizing the signal data received by system100as being associated with triggering object135-1(see object data structure166ofFIG. 3A). In one embodiment, system100may be capable of converting signals received from triggering objects into a digital signal using known digital signal conversion processing techniques. Furthermore, signals may be transmitted through wired network connections as well as wireless network connections, including, but not limited to, infrared technology, Bluetooth technology, Wi-Fi networks, the Internet, etc.

AlthoughFIGS. 2A through 3Cdepict various embodiments using different triggering object—application pairings, embodiments of the present invention may not be limited as such. For example, according to one embodiment, applets resident on system100may also be configured to execute in response to detection of a triggering object linked to the applet. Also, in one embodiment, system functions and/or processes associated with an operating system running on system100may be configured to execute responsive to a detection of a recognized triggering object. Furthermore, applications used to process telephonic events performed on system100(e.g., receiving/answering a phone call) may be linked to triggering objects.

FIG. 4provides a flow chart depicting an exemplary application execution process based on the detection of a recognized triggering object in accordance with embodiments of the present invention.

At step405, using a data structure resident on a mobile device, applications are mapped to a triggering object in which each mapped application is configured to execute autonomously upon a recognition of its respective triggering object.

At step410, during a surveillance mode, the mobile device detects objects located external to the mobile device using a camera system.

At step415, image data gathered by the camera system at step410is fed to the object recognition module to determine if any of the objects detected are triggering objects.

At step420, a determination is made as to whether any of the objects detected during step410are triggering objects recognized by the mobile device (e.g., triggering objects mapped to an application in the data structure of step405). If a detected object is a triggering object recognized by the mobile device, then the object recognition module performs a lookup of mapped applications stored in the data structure to determine which applications are linked to the recognized triggering object determined at step420, as detailed in step425. If any of the objects detected are not determined to be a triggering object recognized by the mobile device, then the mobile device continues to operate in the surveillance mode described in step410.

At step425, a detected object is a triggering object recognized by the mobile device and, therefore, the object recognition module performs a lookup of mapped applications stored in the data structure to determine which applications are linked to the recognized triggering object determined at step420.

At step430, applications determined to be linked to the recognized triggering object determined at step420are autonomously executed by the mobile device.

FIG. 5provides a flow chart depicting an exemplary application execution process based on the detection of multiple recognized triggering objects in accordance with embodiments of the present invention.

At step505, using a data structure resident on a mobile device, applications are mapped to a triggering object in which each mapped application is configured to execute autonomously upon a recognition of its respective triggering object.

At step510, during a surveillance mode, the mobile device detects objects located external to the mobile device using a camera system.

At step515, image data gathered by the camera system at step510is fed to the object recognition module to determine if any of the objects detected are triggering objects.

At step520, a determination is made as to whether any of the objects detected during step510are triggering objects recognized by the mobile device (e.g., triggering objects mapped to an application in the data structure of step505). If at least one detected object is a triggering object recognized by the mobile device, then a determination is made as to whether there are multiple triggering objects recognized during step520, as detailed in step525. If any of the objects detected are not determined to be a triggering object recognized by the mobile device, then the mobile device continues to operate in the surveillance mode described in step510.

At step525, at least one detected object is a triggering object recognized by the mobile device and, therefore, a determination is made as to whether there are multiple triggering objects recognized during step520. If multiple triggering objects were recognized during step520, then the mobile device searches for visual identifiers and/or positional information associated with the objects detected at step510to distinguish the recognized triggering objects detected, as detailed in step530. If multiple objects were not recognized during step520, then the object recognition module performs a lookup of mapped applications stored in the data structure to determine which applications are linked to a triggering object recognized during step520, as detailed in step535.

At step530, multiple triggering objects were recognized during step520and, therefore, the mobile device searches for visual identifiers and/or positional information associated with the objects detected at step510to distinguish the recognized triggering objects detected. Furthermore, the object recognition module performs a lookup of mapped applications stored in the data structure to determine which applications are linked to a triggering object recognized during step520, as detailed in step535.

At step535, the object recognition module performs a lookup of mapped applications stored in the data structure to determine which applications are linked to a triggering object recognized during step520.

At step540, applications determined to be linked to a triggering object recognized during step520are autonomously executed by the mobile device.

FIG. 6provides a flow chart depicting an exemplary application execution process based on the detection of a recognized triggering object using the GPS module and/or the orientation module in accordance with embodiments of the present invention.

At step605, using a data structure resident on a mobile device, applications are mapped to a triggering object in which each mapped application is configured to execute autonomously upon a recognition of its respective triggering object.

At step610, during a surveillance mode, the mobile device detects recognized triggering objects located external to the mobile device using the GPS module and/or the orientation module.

At step615, data gathered by the GPS module and/or the orientation module at step610is fed to the object recognition module.

At step620, the object recognition module performs a lookup of mapped applications stored in the data structure to determine which applications are linked to the recognized triggering objects detected at step610.

At step625, applications determined to be linked to the recognized triggering objects detected at step610are autonomously executed by the mobile device.

FIG. 7provides a flow chart depicting an exemplary system process (e.g., operating system process) executed based on the detection of a recognized triggering object in accordance with embodiments of the present invention.

At step705, using a data structure resident on a mobile device, system processes are mapped to a triggering object in which each mapped system process is configured to execute autonomously upon recognition of its respective triggering object.

At step710, during a surveillance mode, the mobile device detects objects located external to the mobile device using a camera system.

At step715, image data gathered by the camera system at step710is fed to the object recognition module to determine if any of the objects detected are triggering objects.

At step720, a determination is made as to whether any of the objects detected during step710are triggering objects recognized by the mobile device (e.g., triggering objects mapped to a system process in the data structure of step705). If a detected object is a triggering object recognized by the mobile device, then the object recognition module performs a lookup of mapped system processes stored in the data structure to determine which processes are linked to the recognized triggering object detected at step720, as detailed in step725. If any of the objects detected are not determined to be a triggering object recognized by the mobile device, then the mobile device continues to operate in the surveillance mode described in step710.

At step725, a detected object is a triggering object recognized by the mobile device and, therefore, the object recognition module performs a lookup of mapped system processes stored in the data structure to determine which processes are linked to the recognized triggering object detected at step720.

At step730, system processes determined to be linked to the recognized triggering object detected at step720are autonomously executed by the mobile device.

While various embodiments have been described and/or illustrated herein in the context of fully functional computing systems, one or more of these example embodiments may be distributed as a program product in a variety of forms, regardless of the particular type of computer-readable media used to actually carry out the distribution. The embodiments disclosed herein may also be implemented using software modules that perform certain tasks. These software modules may include script, batch, or other executable files that may be stored on a computer-readable storage medium or in a computing system. These software modules may configure a computing system to perform one or more of the example embodiments disclosed herein. One or more of the software modules disclosed herein may be implemented in a cloud computing environment. Cloud computing environments may provide various services and applications via the Internet. These cloud-based services (e.g., software as a service, platform as a service, infrastructure as a service) may be accessible through a Web browser or other remote interface. Various functions described herein may be provided through a remote desktop environment or any other cloud-based computing environment.

Embodiments according to the invention are thus described. While the present disclosure has been described in particular embodiments, it should be appreciated that the invention should not be construed as limited by such embodiments, but rather construed according to the below claims.