Motion activated user interface for mobile communications device

Disclosed is a system for interpreting motion of a mobile communications device as input to the mobile communications device. The system includes a processor for executing the various software components, a display, a microphone, a speaker, storage means for storing one or more motions and associated actions, a motion detecting device, and a sensing and interpretation application. The sensing and interpretation application detects motion of the mobile communications device via the motion detecting device. It then determines the current mode of the mobile communications device and compares the detected motion against a database of motions. Each stored motion is associated with a mode and an action to be performed by the mobile communications device. Upon finding a match between the detected motion and a motion in the storage means, the action associated with the detected motion and the current mode of the mobile communications device is performed.

SUMMARY

Disclosed is a method and system for interpreting motion of a mobile communications device as input to the mobile communications device. The system includes a processor for executing the various software components of the mobile communications device, a display coupled with the processor, a microphone coupled with the processor for recording audio, a speaker coupled with the processor for outputting audio, storage means coupled with the processor for storing one or more motions and associated actions, a motion detecting device such as, for instance, an accelerometer coupled with the processor, ad a sensing and interpretation application coupled with the processor.

The sensing and interpretation application detects a specific type of motion with respect to the mobile communications device via the motion detecting device. It then determines the current mode of the mobile communications device and compares the detected motion against a database of motions in the storage means. Each stored motion is associated with a mode of the mobile communications device and an action to be performed by the mobile communications device. Upon finding a match, within tolerable limits, between the detected motion and a motion in the storage means, the action associated with the detected motion and the current mode of the mobile communications device is performed.

The sensing and interpretation application can also train the mobile communications device to recognize a specific motion. It does this by selecting a mode to be associated with a specific motion and selecting an action to be performed within the selected mode. Next, the sensing and interpretation application detects a user defined motion of the mobile communications device via the motion detecting device. The detected motion is then associated with the selected mode and the selected action and stored away.

The system and method can also utilize a text-to-speech engine to convert text strings that can be associated with an action to audible output that can be output by the speaker.

The mode of the mobile communications device can include, among others, one of a phone mode, a picture mode, a music mode, a game mode, camera mode, a contacts mode, a settings mode, and a null mode. The null mode acts as a superseding mode such that any motions and actions associated within the null mode supersede motions and actions in other modes.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1is an illustration of some of the internal components of a mobile communications device10. Within the mobile communications device10there are included several components that operate to practice the embodiments of the invention. Not all the components that comprise a mobile communications device10are illustrated, however.

A processor20serves to coordinate and control the operations and interactions among the various components. Among the various components, there is shown a motion sensing and interpretation application30, other mobile communications device applications40(e.g., voice recognition, contacts, games, etc.), internal (and/or removable) storage50, motion sensing devices60, a text-to-speech engine70, a microphone80, and a speaker90.

The motion sensing and interpretation application30includes the software and software interfaces for managing sensed motion and the responses thereto. The motion sensing and interpretation application30further includes training responses to be associated with specified motions.

Other mobile communications device applications40generally include the other applications of the mobile communications device that can be influenced by or operated on by the motion sensing and interpretation application30. For instance, the application that answers the phone can be altered to accept sensed motion as a means for answering and disconnecting a phone call. In another example, an application for displaying pictures can be made to display the next picture of a series (e.g., slideshow) using a shaking motion such as a flick of one's wrist. There are numerous other examples involving other mobile communications device applications40that can be integrated to accept motion as a source of input.

The internal (and/or removable) storage50serves to store data associated with the motion sensing and interpretation application30such as, for instance, a library of stored motions that are linked with mobile communications device modes and operations/tasks. The motion sensing devices60can include at least one or more accelerometer devices used to detect motion of the mobile communications device. The text-to-speech engine70can be included to convert text data to verbal output. Text data can be associated with a sensed motion and converted to speech upon recognition of the motion. The converted speech can then be output via speaker90.

The microphone80can be used to record words or phrases that can be associated with sensed motion and output via speaker90when the associated motion is sensed and identified. For instance, if the mobile communications device were to be dropped and hit the floor, it could output the word “ouch!” upon or just after impact if the motion sensed was defined and indicative of a dropped mobile communications device.

FIG. 2is a sample screen shot describing aspects of embodiments according to the present invention. In this illustration, the mobile communications device10is displaying95three of the functions (motion training, editing, and deleting) available to the motion sensing and interpretation application30. A user can scroll and select from among the list shown95.

FIG. 3is a flow chart or logic diagram describing motion training aspects of embodiments according to the present invention. One of the functions of the motion sensing and interpretation application30is to learn motions and associate them with actions or outputs. This can sometimes be referred to as a training function.

Motion training begins300when is shown the several modes305of the mobile communications device. Modes can include, but are not limited to, phone mode, camera mode (if the mobile communications device has an integrated camera), music mode (if the mobile communications device has an integrated music player, game mode, picture mode, contacts mode, settings mode, and a null mode, etc.

The null mode can include more irreverent actions such as “phone drop” or “phone toss”. While the mobile communications device is in a particular mode, sensed motion will have a meaning specific to that mode. This allows for the same sensed motion to be used in different modes. However, if an action is defined for a null mode the action will be performed when the specified motion is detected regardless of the current mode. Thus, actions associated with the null mode should not be associated with any other mode as null mode acts as a superseding mode with respect to detected motion. Examples of motions for the null mode include dropping the mobile communications device and tossing the mobile communications device into the air. Examples of actions associated with dropping the mobile communications device or tossing the mobile communications device in the air include an audible “ouch” or an audible “wheeee” respectively.

Once the mobile communications device processes a user's mode selection310, a list of outputs/actions is generated for that mode315. Phone mode, for instance, can include outputs/actions such as “answer”, “hang up”, “mute”, “volume up/down”, “call waiting answer”, etc. Other modes will have outputs/actions that apply to their mode. A user will select an output/action320. The mobile communications device will then prompt the user to create a motion that is to be associated with the mode action/output pairing325. The mobile communications device then processes the motion supplied by the user in response to the prompt330. The motion is then associated with the mode action/output pairing335and stored in a database of motion definitions340. The user is prompted whether to train another motion345. If the response is “no” training ends350. Otherwise, control is returned to process305and the user is shown the list of modes.

It is also anticipated that several motions may have pre-defined or canned mode and output/action associations stored in memory. If so, these can be edited by the user if desired according to the procedures set out below.

FIG. 4is a flow chart or logic diagram describing motion editing aspects of embodiments according to the present invention. The user may also edit an existing mode/motion pairing by selecting from a displayed list of modes for the mobile communications device410. The mobile communications device will process the selection420and display a list of outputs/actions for the selected mode430obtained from the database of defined motions. The user selects from the list and the mobile communications device will process the selection of the action/output440by allowing the user to change450the output/action for the selected motion.

FIG. 5is a flow chart or logic diagram describing motion deleting aspects of embodiments according to the present invention. The user may also delete an existing mode/motion pairing by selecting from a displayed list of modes for the mobile communications device510. The mobile communications device will process the selection520and display a list of outputs/actions for the selected mode530obtained from the database of defined motions. The user selects from the list and the mobile communications device will process the selection of the action/output by deleting the output/action for the selected motion540.

FIG. 6is a flow chart or logic diagram describing operational aspects of embodiments according to the present invention. When the motion sensing and interpretation application30is active, motion is continuously being sensed610and interpreted620based on the current mode of the mobile communications device. When motion for a mode is sensed that has a defined output/action stored340, the mobile communications device will cause the defined output/action to occur630. Upon completion of the output/action, the motion sensing and interpretation application30returns to its vigilant state where it processes subsequent motion searching for matches based on mode and motion.

The types of actions and/or outputs that can be associated with detected motion can include, but are not limited to, audible output via the speaker, visual output via the display, mechanical output such as vibration, and launching an application and/or performing a command within an application. Audible output can be based on pre-recorded sounds, words, phrases as well as links to other audio files such as music files. Audible output can also include text-to-speech conversions of text data. Visual output can include, but is not limited to, graphical imagery on the display such as color and design bursts and links to image files that can be displayed. Visual output can also include events not associated with the display such as flashing the lights associated with the keys of the keypad as well as any other lights visible on the mobile communications device that are not associated with the display.

To help illustrate the scope of the present invention, several illustrative embodiment examples are presented that indicate some, but not all of, the capabilities of the present invention.

FIG. 7illustrates a flip-type of motion that can be defined and recognized by embodiments according to the present invention. In this example, an mobile communications device is shown in five consecutive states (a)-(e) to indicate a flip-type motion. In state (a) the mobile communications device is shown face forward. In state (b) the mobile communications device is shown rotated (or flipped) 90° such that it is in a profile mode. In state (c) the mobile communications device is shown face down having been rotated another 90°. In state (d) the mobile communications device is shown rotated (or flipped) another 90° such that it is in a second opposite profile mode. Lastly, in state (e) the mobile communications device is shown rotated (or flipped) 90° again such that it is returned to face front. This sequence or progression can be associated with an output/action for one or more modes. Moreover, the motion can be broken down into 90° intervals such that each quarter turn can have its own associated mode/motion definition.

FIG. 8illustrates a shaking motion that can be defined and recognized by embodiments according to the present invention. The arrows between the mobile communications device's in this illustration indicate a back and forth motion between state (a) and state (b). This back and forth motion can be termed “shaking” and can be associated with an output/action for one or more modes.

FIG. 9illustrates a spinning motion that can be defined and recognized by embodiments according to the present invention. In this example, the mobile communications device starts out oriented face front with a standard top/bottom orientation. As it is rotated (spun), the mobile communications device's orientation is continuously changing as it traverses an arc and eventually comes full circle. The speed of the rotation can be varied.

The motion sensing and interpretation application30can be used for functional and personalization applications. Functional applications include using motion as a user input device to answer the phone, for instance. Personalization applications can include having the phone make quirky sounds or change display characteristics based on sensed motions. For instance, if the user drops his phone it can be made to say “Ouch!” upon or just after impact while the display can be made to show an explosion of some sort.

WhileFIGS. 7-9have described types of motion that can be defined and used in the embodiments of the present invention, it is important to note that the embodiments of the present invention are not limited to just these motions. They are merely exemplary to help describe aspects of the present invention. For instance, motions can be detected and interpreted in two and three dimensions. Moreover, motion such as knocking the mobile communications device against a hard surface can be detected and interpreted similar to knocking one's hand on a door. Thus, there are numerous types of motion that can be detected and interpreted by the embodiments of the present invention and the examples described herein are not intended to be limiting.

An example of a game or game mode application could be the game of Roulette. Roulette is a casino game in which a large numbered and colored slotted disk is spun around while a small metal ball bounces around the disk until the disk comes to rest and the ball rests within one of the numbered/colored slots. The object of the game is to guess the number and/or color of the slot in which the ball will come to rest.

The mobile communications device can be made to simulate the game by spinning the mobile communications device on a table top. The spinning motion, in this particular mode, will cause a bouncing ball sound and rapid click sound indicative of the sounds made in Roulette. As the spinning begins to ebb, the sounds will do the same. Once the motion stops, the final action/output is to have the mobile communications device randomly select one of the Roulette numbers and its associated color. The selection can be displayed by the mobile communications device using the color as a background for the number.

In picture mode, the mobile communications device can use detected motion to perform various functions. For instance, if the user shakes the mobile communications device once it could indicate “display the next picture”. Two shakes could indicate “zoom in”. Flipping the mobile communications device over to its back could indicate “exit picture mode”.

Another example of personalization could be a phone toss. The motion associated with an mobile communications device hurtling through the air can be detected and associated with a pre-recorded audio output such as “Wheeeeee”.

If the mobile communications device includes a text-to-speech engine, written data can be converted to audible output and associated with a specific motion. For instance, suppose the motion detected is indicative of a sudden or violent change. The mobile communications device can be programmed with a text question such as “Are you alright?” If the mobile communications device further contains a voice recognition (VR) engine and software, it can process the user's response if it is simple enough. If the user responds “No.”, the mobile communications device can interpret and then ask, “Should I dial 9-1-1?” If the user responds, “Yes” the mobile communications device can initiate the emergency phone call.

The foregoing are a small sampling of the types of motion and the associated responses thereto that can be implemented for these and other examples under the various embodiments of the present invention.

Any prompts associated with the present invention may be presented and responded to via a graphical user interface (GUI) presented on the display of the mobile communications device or the like. Prompts may also be audible, vibrating, etc.