TASK SELECTIONS ASSOCIATED WITH TEXT INPUTS

A system and machine-implemented method for performing tasks associated with text inputs, the method including providing a text input mechanism on an electronic device, receiving, at the electronic device, an input by a user using the text input mechanism, determining if the input corresponds to a text selection or task selection, wherein a text selection corresponds to the user entering an actual text input through the text input mechanism and a task selection corresponds to the user requesting to perform a task related to text entered at the device, registering a key corresponding to the input if the input corresponds to a text selection and performing a task corresponding to the input if the input corresponds to a task selection.

BACKGROUND

As electronic devices equipped with touchscreens have become increasingly popular, virtual keyboards have also become popular. Typing on virtual keyboards often corresponds to various tasks. However, performing these tasks may require that a user switch from the virtual keyboard interface to a different non-keyboard user interface to make the selection. The switching of interfaces can often impede the user experience in inputting additional words or phrases with the virtual keyboard.

SUMMARY

The disclosed subject matter relates to a machine-implemented method for performing tasks associated with text inputs, the method comprising providing a text input mechanism on an electronic device. The method further comprising receiving, at the electronic device, an input by a user using the text input mechanism. The method further comprising determining if the input corresponds to a text selection or task selection, wherein a text selection corresponds to the user entering an actual text input through the text input mechanism and a task selection corresponds to the user requesting to perform a task related to text entered at the device. The method further comprising registering a key corresponding to the input if the input corresponds to a text selection and performing a task corresponding to the input if the input corresponds to a task selection.

The disclosed subject matter also relates to a system for performing tasks associated with text inputs, the system comprising one or more processors and a machine-readable medium comprising instructions stored therein, which when executed by the processors, cause the processors to perform operations. The operations comprising receiving, at an electronic device, an input by a user using a text input mechanism. The operations further comprising determining according to one or more criteria if the input corresponds to a text selection or task selection, wherein a text selection corresponds to the user entering an actual text input through the text input mechanism and a task selection corresponds to the user requesting to perform a task, wherein the one or more criteria include characteristics of the input and context of the input. The operations further comprising identifying a key corresponding to the input if the input corresponds to a text selection and identifying a task corresponding to the input if the input corresponds to a task selection.

The disclosed subject matter also relates to a machine-readable medium comprising instructions stored therein, which when executed by a machine, cause the machine to perform operations comprising providing a text input mechanism on an electronic device, the text input mechanism comprising a virtual mechanism for inputting text. The operations further comprising receiving, at the electronic device, an input by a user at the text input mechanism. The operations further comprising determining based on information regarding the input if the input corresponds to a text selection or task selection, wherein a text selection corresponds to the user entering an actual text input through the text input mechanism and a task selection corresponds to the user requesting to perform a task related to text. The operations further comprising registering a key corresponding to the input if the input corresponds to a text selection and performing a task corresponding to the input if the input corresponds to a task selection.

DETAILED DESCRIPTION

Often a user keyboard entry corresponds to and/or is associated with one or more selection tasks (e.g., menu navigation or selection, text field navigation or selection, word prediction navigation or selection, etc.). Traditionally, the mechanism for text entry (e.g., a keyboard) and the mechanism for selection (e.g., touch, cursor, mouse, or other selection mechanism) have been distinct. This means that when the user wishes to select a selection task related to a text entry, the user has to switch between two input mechanisms (e.g., from a keyboard to a selector). In certain instances (e.g., devices where a limited display is available or a single input is selectable at a time such as devices with touch screens, UI keyboards, virtual keyboards, etc.) the user has to switch between input mechanisms, use another UI and/or close one input mechanism (e.g., the text input mechanism), when performing a task relating to a text input.

According to various aspects of the subject technology, systems and methods are provided for allowing a user to select tasks associated with text inputs in a quick and efficient manner. In some aspects, scrubbing and selection gestures by the user can be entered and detected on the text input mechanism (e.g., a virtual keyboard, layout of key or their text input user interface (‘UI”). The detected gestures may be translated to selections, which would otherwise be entered using a separate selection mechanism. The determination as to whether an input received at the text input mechanism is a text input or task input is based on various criteria that differentiate between such inputs. Once it is determined that the user wishes to perform a task, rather than entering text, through the text input mechanism, the system recognizes the gesture (e.g., based on the specific set of related tasks available) and translates the input at the text input mechanism to a task input. The task input then causes a task to be performed that would otherwise be performed by the user directly through a separate selection mechanism.

The tasks may be in response to items being displayed in association with the text and/or corresponding to the text being entered using the text input mechanism. For example, in some implementations, the related task may include a navigation through and/or selection of a text suggestion being displayed to the user in response to the user entering text (e.g., using the text input mechanism). In one example, a text suggestion may include a correction (e.g., autocorrect) or completion (e.g., autocomplete) of the text being entered. For example, the text input may include a first portion of a word or phrase, and a text suggestion may include a second portion of the word or phrase. Alternatively, the text input may include a word or phrase having an error, and the suggestion may include the word or phrase without the error. The error may, for example, include a grammatical, spelling, punctuation, and linguistic error.

In some implementations, the related task may be related to a menu being displayed, for example, in response to text being entered using the text input mechanism. For example, contextual menus or other menus (e.g., providing autocomplete suggestions, text suggestions, options for filling out forms or similar options) may be displayed in display area101of device100. In some implementations, the related task may involve moving from one text entry field to another text entry field (e.g., field or page).

In one example, the related tasks may include a selection of one of a plurality of options (e.g., text suggestions, options in the menu, or text fields). In one example the plurality of options are arranged along one or more axis (e.g., X, Y), and the input (e.g., swipe gesture) is substantially parallel to at least one of the axis.

By allowing the user to perform gestures relating to tasks on the text input mechanism (e.g., virtual keyboard), the user is able to perform related tasks without switching between different user interfaces. In this manner the text input mechanism (e.g., virtual keyboard) is the singular point of entry for the user, and the user can easily switch between text input and task inputs and/or quickly continue inputting additional words or phrases after selecting to perform a specific task (e.g., navigating text suggestions, selecting a text suggestion, navigating a menu, selecting a menu item, navigating a page or fields of a page, or selecting an item or field in a page).

FIG. 1illustrates an example of a client device for implementing various aspects of the subject disclosure. The device100is illustrated as a mobile device equipped with touchscreen101. In some implementations, the touch screen101includes a virtual keyboard102and a display area103. Virtual keyboard102provides a text input mechanism for the device100and may be implemented using touchscreen101. Display area103provides for display of content (e.g., menus) at the device100. Device100may further include a selection mechanism (e.g., through touch, or pen) for selection of items displayed within display area103of touch screen101.

Although device100is illustrated as a smartphone, it is understood the subject technology is applicable to other devices that may implement text input and/or selection mechanism as described herein (e.g., devices having touch capability), such as personal computers, laptop computers, tablet computers (e.g., including e-book readers), video game devices, and the like. Although touchscreen101is described as including both input and display capability, in one example, the device100may include and/or be communicationally coupled to a separate display for displaying items. In one example, the touchscreen101may be implemented using any device providing an input mechanism providing for text input (e.g., through a virtual keyboard) and/or selection (e.g., through touch or pen).

As shown inFIG. 1, the keys of virtual keyboard102include alphabet characters and are laid out according to the QWERTY format. However, virtual keyboard102is not limited to keys that pertain only to alphabet characters, but can include keys that pertain to other non-alphabet characters, such as numbers, symbols, punctuation, and/or other special characters. According to certain aspects, a user may perform a gesture (e.g., tapping and holding onto a particular key) to display keys that pertain to other non-alphabet characters. In this regard, the keys that are initially provided by virtual keyboard102may be referred to as primary keys, while the keys that are provided after the user performs a gesture and subsequently displayed may be referred to as secondary keys.

Although virtual keyboard102is described herein as being a user interface that is displayed to the user, the subject technology is equally applicable to keyboards that are not displayed to users (e.g., keyboards that do not have any keys visible to the user). For example, a touchpad, track pad, or touch screen may be used as a platform for a virtual keyboard. The touchpad, track pad, or touch screen may be blank and may not necessarily provide any indication of where keys would be. Nevertheless, a user familiar with the QWERTY format may still be able to type as if the keyboard were still there. In this regard, the input from the user may still be detected in accordance with various aspects of the subject technology. In some aspects, a menu or any other suitable mechanism may be used to show the user which keys the user may select. For example, a menu may be displayed to show the user which keys the user may select.

A user may perform a gesture (e.g., a tap or a swipe) at the virtual keyboard in an attempt to select a particular key. In addition the user may perform a gesture at the virtual keyboard102to perform a task relating to the text entry. For example, tasks relating the text entry may be displayed within display area103of touch screen101(e.g., a menu, text recommendations, text fields, etc.). In one example, when the user performs a gesture, mobile device may determine if the gesture is to select a particular key or to perform a task. The determination may be based on a number of criteria that distinguish a text input and a task input on the keyboard102. In one example, the criteria may include velocity, direction, context, and/or other similar criteria. In one example, the context may include whether a task is available for selection. In one example, the context includes a combination of criteria including the text entered, the tasks available and/or displayed, velocity of selection, direction of selection, duration of selection, historical information regarding user selection and/or preferences, and/or other criteria that may distinguish a text entry and task input at the virtual keyboard102. The device100may determine the selection type and perform a task in response to the determination.

In one example, where it is determined that the user performed a gesture (e.g., a tap or a swipe) in an attempt to select a particular key, device100may detect the gesture and determine which key to register as the intended text input from the user. For example, if the user taps a point on touchscreen101corresponding to the “S” key of virtual keyboard102, device100may detect the tap at that point, and determine that the tap corresponds to the “S” key. Device100may therefore register the “S” key as the input from the user. Device100may then display the letter “S” in the display area103, for example in a text field, thereby providing an indication to the user that the “S” key was registered as the actual input.

In some examples, when it is determined that the user performed a gesture (e.g., a tap or swipe) in an attempt to perform a task, device100may detect the gesture and determine the task being performed. In one example, the device100may determine the task based on the tasks available and/or being displayed to the use. For example, where text recommendations are provided to a user, and, for example, in relation with text, the user performs a swipe, the device102may determine that the desired task is to move to and/or select the text recommendation in accordance with the swipe (e.g., shape and/or direction of the swipe). In one example, where a menu is being displayed, and the user performs a swipe, the device102may determine that the task being performed is to navigate and/or select an option of the options in the menu. In another example, where the page includes text fields, a swipe or touch by the user may be detected as a desire to move to a different text field on the page. Once the task to be performed is detected, the related task is performed (e.g., as if the task was performed using the appropriate selection mechanism such as a touch or pen).

In one example, the input may be continuous after the previous input (e.g., by continuing from the termination location of the previous input such as the location of key of a text input or the ending location of a task input) and/or may be initiated as a separate gesture (e.g., by lifting off the touchscreen after entering the input and again tapping the touchscreen to initiate the input).

In some examples, when it is determined that the performed gesture corresponds to a task input (e.g., rather than a text entry input), the device100may determine one or more key entries detected during the gesture (e.g., the point of initiation of the entered gesture, one or more middle points or the point of termination of the gestures) and discard the one or more entries as key selection(s). For example, where the input is initiated independently (e.g., not continuous from the last input), the point of initiation may correspond to a key on the virtual keyboard102and may be discarded as a key entry.

FIG. 2illustrates an example of system200for allowing text entry inputs and task inputs on a text input mechanism, in accordance with various aspects of the subject technology. System200, for example, may be part of device100. System200comprises input module201, type detection module202, text selection module203and task selection module204. These modules may be in communication with one another. In one example, the modules201,202,203and204are coupled through a communication bus205. In one example, the input module201is configured to receive an input at a text input mechanism (e.g., virtual keyboard). In one example, the input mechanism201provides the input to type detection module202, which determines if the input corresponds to a text input or a task input. If the type detection module202determines that the input corresponds to a text selection, the text selection module203determines the key being selected and registers the text input. Otherwise, the task selection module204receives the input and determines a task corresponding to the input and performs the task. In one example, the task selection module sends a request to perform the determined task at the device.

In some aspects, the modules may be implemented in software (e.g., subroutines and code). In some aspects, some or all of the modules may be implemented in hardware (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other suitable devices) and/or a combination of both. Additional features and functions of these modules according to various aspects of the subject technology are further described in the present disclosure.

FIG. 3illustrates an example flow diagram of a process300for facilitating select tasks associated with text inputs. System200, for example, may be used to implement method300. However, method300may also be implemented by systems having other configurations. In step301, an indication of a user input is received. The input, for example, may be a tap or swipe or other gesture performed on a text input mechanism (e.g., virtual keyboard102).

In step302, the user input is analyzed to determine if the user input corresponds to a text selection or a task selection. The determination, as described above, may be based on different criteria including the context of the user input as well as the characteristics of the user input. For example, in one example, input characteristics such as duration, velocity, position (e.g., starting and/or ending position), and/or direction may be used to determine if the user input corresponds to a text or task selection. In some implementations, context information such as items provided for display at the device (or a coupled device), previous text inputs, previous user activity and behavior, user preferences and/or user and/or system settings may be taken into account when making the determination in step302.

If, in step302, it is determined that the user input corresponds to a text selection, the process continues to step303. In step303, the key associated with the user input is registered as the input. The user input may be analyzed to determine which key to register as the intended input from the user. In one example, an indication of the key being registered as the input is provided for display to the user (e.g., displayed in the display area103).

Otherwise, if it is determined that the user input corresponds to a task input in step302, in step304, the task associated with the input is determined. In one example, the device100may determine the task based on the items being displayed to the user. In some examples, criteria described above, including the characteristics of the user input and/or context of the user input may be used to determine the task associated with the input. In step305, the task determined in step304is performed. The task may include menu navigation and/or selection, text field and/or page navigation and/or selection, text recommendation navigation and/or selection or other similar activity.

FIG. 4Aillustrates an example in which a user input corresponding to a text selection is entered using a virtual keyboard, in accordance with various aspects of the subject technology. As shown inFIG. 4, the index finger of hand401of the user taps touchscreen101on the “T” key. A determination is made (e.g., at the selection type detection module202) as to the type of input according to the methods described and it is determined that the tap refers to an actual text input. Thus, the “T” key is registered as the user input (e.g., at the text selection module204). The letter “T” is provided for display in the text field402, thereby providing an indication to the user that the “T” key was registered as the input.

FIGS. 4B, illustrates an example in which a user input corresponding to a task selection is entered using a virtual keyboard, in accordance with various aspects of the subject technology. As shown inFIGS. 4A and 4B, a set of text recommendations are provided to a user in text recommendation area403of the display area103. The text recommendations may be generated according to different techniques and provided for display at the device100. The finger of hand401may make a gesture404by moving in the right direction across the virtual keyboard102. In one example, the gesture may be continuous after the text selection shown inFIG. 4Aor may be initiated as a separate gesture (e.g., by lifting the finger of hand401off the touchscreen after entering the last text selection and again tapping the touchscreen to initiate the input). According to characteristics of gesture404and the context of the gesture404it is determined that the user wishes to move across the text recommendations. Accordingly, as shown inFIG. 4B, the text recommendation moves from the center (e.g., default) recommendation “Unit” to the right recommendation “United.” As shown inFIG. 4B, an indication of the task being performed is shown to the user.

FIGS. 5A-5D, illustrate other examples in which user inputs corresponding to text and task selections are entered using a virtual keyboard, in accordance with various aspects of the subject technology. As shown inFIGS. 5A-5D, a form is being displayed on display area103. The form may include one or more text entry fields, including text entry field501and502. As shown inFIG. 5A, the “address” text field501is currently selected, and text is entered into text field501using the virtual keyboard102. For example, the index finger of hand401of the user taps touchscreen101on the “T” key. A determination is made (e.g., at the selection type detection module202) as to the type of input according to the methods described and it is determined that the tap refers to an actual text input. Thus, the “T” key is registered as the user input (e.g., at the text selection module204). The letter “T” is provided for display in the text field402, thereby providing an indication to the user that the “T” key was registered as the input.

Next, as shown inFIG. 5B, the finger of hand401may make a gesture503by moving down the virtual keyboard102. In one example, the gesture may be continuous after the text selection shown inFIG. 5Aor may be initiated as a separate gesture (e.g., by lifting the finger of hand401off the touchscreen after entering the last text selection and again tapping the touchscreen to initiate the input). According to characteristics of gesture503and the context of the gesture503it is determined that the user wishes to move to the next text filed, the “state” text field502. Accordingly, as shown inFIG. 5B, the next text field502is selected in response to gesture503. An indication of the recommendation is shown to the user, for example, by highlighting the text field502or moving the text entry cursor to the text field502.

As shown inFIG. 5C, a menu504is provided for display, in association with text field502, showing the options for the “state” text field. In one example, the menu may be displayed automatically as a result of performing the text field navigation in response to gesture503. In another example, the user may make a separate gesture such as beginning to input text or making another gesture (e.g., holding down on the virtual keyboard for a long duration or other gesture indicating a desire to see the menu).

A gesture505may be entered at virtual keyboard102by the user while the menu304is being displayed, as shown inFIG. 5D. For example, the finger of hand401may make gesture505by moving down the virtual keyboard102. In one example, the gesture may be continuous after the last gesture or text selection, or may be initiated as a separate gesture (e.g., by lifting the finger of hand401off the touchscreen and again tapping the touchscreen to initiate the input). According to characteristics of gesture505and the context of the gesture505it is determined that the user wishes to move down menu504. Accordingly, as shown inFIG. 5D, the next text field502is selected. An indication of the recommendation is shown to the user, for example, by highlighting the next option on the menu504.

In this manner, the user is able to perform tasks associated with text inputs in a quick and efficient manner using the text input mechanism. Accordingly, the user is not required to switch input mechanisms and/or discard the text input when performing tasks related to the text input.

FIG. 6conceptually illustrates an electronic system with which some implementations of the subject technology are implemented. Electronic system600can be a server, computer, phone, PDA, laptop, tablet computer, television with one or more processors embedded therein or coupled thereto, or any other sort of electronic device. Such an electronic system includes various types of computer readable media and interfaces for various other types of computer readable media. Electronic system600includes a bus608, processing unit(s)612, a system memory604, a read-only memory (ROM)610, a permanent storage device602, an input device interface614, an output device interface606, and a network interface616.

Bus608collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of electronic system600. For instance, bus608communicatively connects processing unit(s)612with ROM610, system memory604, and permanent storage device602.

ROM610stores static data and instructions that are needed by processing unit(s)612and other modules of the electronic system. Permanent storage device602, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when electronic system600is off Some implementations of the subject disclosure use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as permanent storage device602.

Other implementations use a removable storage device (such as a floppy disk, flash drive, and its corresponding disk drive) as permanent storage device602. Like permanent storage device602, system memory604is a read-and-write memory device. However, unlike storage device602, system memory604is a volatile read-and-write memory, such a random access memory. System memory604stores some of the instructions and data that the processor needs at runtime. In some implementations, the processes of the subject disclosure are stored in system memory604, permanent storage device602, and/or ROM610. For example, the various memory units include instructions for facilitating entry of text and performing of tasks through inputs entered at a text input mechanism according to various embodiments. From these various memory units, processing unit(s)612retrieves instructions to execute and data to process in order to execute the processes of some implementations.

Bus608also connects to input and output device interfaces614and606. Input device interface614enables the user to communicate information and select commands to the electronic system. Input devices used with input device interface614include, for example, alphanumeric keyboards and pointing devices (also called “cursor control devices”). Output device interfaces606enables, for example, the display of images generated by the electronic system600. Output devices used with output device interface606include, for example, printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some implementations include devices such as a touchscreen that functions as both input and output devices.

All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.