CONTEXT SENSITIVE INPUT TOOLS

A computer-implemented method can include receiving an input from a user. The input can include one or more characters in a first script representative of text in a particular language. The input can be received in association with a document. The method can include determining a context of the input based on one or more semantic topics of the document associated with the input. One or more candidates for the input can be determined based on (i) the input, (ii) the context of the input, and (iii) a language model. The candidates can include one or more characters in a second script representative of the text in the particular language. The language model can express a probability of occurrence of the one or more candidates in the particular language. The method can further include outputting a list of the one or more candidates for display to the user.

FIELD

The present disclosure relates to text input to computing devices and, more particularly, to techniques for utilizing the context of an input to assist a user that is inputting text to a computing device.

BACKGROUND

A user may provide a text input to a computing device by interacting with one or more peripherals, such as a keyboard, keypad or touch display. In some instances, a user may utilize an Input Method Editor (“IME”) that receives text in a first script and a representation of the text in a second script. For example only, a user may wish to input Chinese text in Hanzi characters through the use of a Latin or Roman keyboard, e.g., by entering a Pinyin representation of the text. Alternatively or in addition, a computing device may facilitate text input from a user by suggesting candidate words or characters in the same script as the text input, which is sometimes referred to as “autocorrect” and/or “autocomplete” functionality. In each of these examples, the computing device attempts to determine what text the user is intending to input. It would be desirable to increase the accuracy and speed of this determination.

SUMMARY

In some embodiments of the present disclosure, a computer-implemented method is described. The method can include receiving, at a computing device having one or more processors, an input from a user. The input can include one or more characters in a first script representative of text in a particular language. Further, the input can be received in association with a document. The method can also include determining, at the computing device, a context of the input based on one or more semantic topics of the document associated with the input. Additionally, the method can include determining, at the computing device, one or more candidates for the input based on (i) the input, (ii) the context of the input, and (iii) a language model. The candidates can include one or more characters in a second script representative of the text in the particular language. The language model can express a probability of occurrence of the one or more candidates in the particular language. The method can further include outputting, from the computing device, a list of the one or more candidates for display to the user.

In some embodiments, the context of the input can be determined from text of the document. Additionally, the method can further include determining, at the computing device, a probability for each candidate of the one or more candidates based on the context of the input and the language model. The probability for each particular candidate can be based on a likelihood that the particular candidate is representative of the input in the second script. A ranked order of the one or more candidates can be determined based on the determined probabilities, and the list can be output in the ranked order.

In various embodiments, determining the one or more candidates for the input can include retrieving, at the computing device, a topic-specific dictionary based on the context of the input, and comparing, at the computing device, the input with entries in the topic-specific dictionary. Additionally or alternatively, determining the one or more candidates for the input can include utilizing, at the computing device, the input and the language model to generate (i) the one or more candidates for the input, and (ii) a probability for each candidate of the one or more candidates, and utilizing, at the computing device, the context of the input to adjust the probability for each candidate of the one or more candidates. The probability for each particular candidate can be based on a likelihood that the particular candidate is representative of the input in the second script.

The document can be an email and the context of the input can be determined from previously entered text in the email. Further, the document can be a web page and the context of the input can be determined from text of the web page. In some embodiments, the first and second script can be identical scripts. Additionally, the method can also include receiving, at the computing device, a selection of a particular candidate from the list of one or more candidates, and updating, at the computing device, the context of the input based on the particular candidate selected.

In some embodiments of the present disclosure, a computer system is described. The computer system can include one or more processors and a non-transitory, computer readable medium storing instructions that, when executed by the one or more processors, cause the computer system to perform operations. The operations performed by the computer system can include any one or more of the operations described above in regard to the disclosed computer-implemented method.

DETAILED DESCRIPTION

A user may input text to a computing device, e.g., in order to draft an email or other electronic message, to interact with a web page (enter a search query, provide a “user comment”), or to compose a newspaper article, book or research paper. In some situations, the computing device can provide assistance to a user that is providing input text.

As mentioned above, an Input Method Editor (“IME”) can provide assistance to a user that wishes to input text in a script that is different from the script provided to the user for selection. For example, a user may utilize a Latin keyboard to input Chinese text in Hanzi characters utilizing a Pinyin IME. Further, the computing device can include autocorrect and/or autocomplete functionality that provides candidates (words/syllables/phrases/etc.) to the user based on an incorrect and/or partial input.

The present disclosure relates to a technique for utilizing the context of the input to assist a user inputting text. The context of the input, as well as a language model, can increase the accuracy and speed of the assistance tools of the computing device in order to determine the text that is intended by the user based on the input.

Referring now toFIG. 1, an example computing device100is shown. The computing device100is illustrated as a mobile device (such as a mobile phone, a tablet computer or “phablet” computer), although it should be appreciated that the term “computing device” can include any form of computer(s) or computer system(s) that include one or more processors for executing instructions. For example only, a computing device can take the form of a desktop computer, a laptop computer, a mobile device (such as a mobile phones, tablets, phablets, and wearable computers), and a server or other distributed computer system, as well as two or more of these example computing devices working together to perform an operation.

The illustrated computing device100includes a display104, such as a touch display as shown. The computing device100may additionally or alternatively include a physical keyboard (not shown). The touch display104may display information to, and receive input from, a user108. A “soft” keyboard114may be provided on the display104through which the user108can provide text input. The illustrated keyboard is a Latin keyboard providing Latin alphabet characters, as well as other input options (numbers, a space bar, symbols, etc.). The user108may input text to the computing device100via the touch display104and/or keyboard114using one or more fingers112.

Referring now toFIG. 2, a functional block diagram of the example computing device100is shown. The computing device100can include a processor200and a communication device204. The term “processor” as used herein refers to both a single processor, as well as two or more processors operating together, e.g., in a parallel or distributed architecture, to perform operations of the computing device100. The computing device100can further include a language model208and a context model212. While shown and described herein as separate components of the computing device100, one or both of the language model208and the context model212can be implemented by the processor200. It should be appreciated that the computing device100can include additional computing components that are not illustrated inFIG. 2, such as memory, a microphone, a speaker, one or more buttons and the like.

The processor200controls most operations of the computing device100. For example, the processor200may perform tasks such as, but not limited to, loading/controlling the operating system of the computing device100, loading/configuring communication parameters for the communication device204, controlling IME parameters, and controlling memory storage/retrieval operations, e.g., for loading of the various parameters. Further, the processor200can control communication with the user108via the touch display104of the computing device100.

The processor200may provide the user108with various different character input configurations via the touch display104. For example, the processor200may provide the user108with a form of the standard Latin “QWERTY” keyboard as shown. Alternatively, the processor200may provide the user108with a standard 12-key configuration, also known as a T9-input based character configuration, or other keyboard configuration.

The processor200may receive input from the user108, e.g., via the provided character input configuration. The processor200, however, may also provide various IMEs, e.g., a Pinyin IME, which allow the user108to input text to the computing device100in a first script to obtain text in a different script. The processor200, therefore, may also convert the input received from the user108to one or more desired scripts, e.g., Chinese Hanzi, by converting the user text input in a different script, e.g., in Pinyin. For example, the processor200may use the language model208, in conjunction with a context model212, when interpreting the user text input (described in detail below).

The communication device204controls communication between the computing device100and other devices/networks. For example only, the communication device204may provide for communication between the computing device100and other computing devices associated and/or the Internet. The computing device100may typically communicate via one or more of three communication mediums: a computing network250, e.g., the Internet (hereinafter “the network250”), a mobile telephone network254, and a satellite network258. Other communication mediums may also be implemented. For example, the communication device204may be configured for both wired and wireless network connections, e.g., radio frequency (RF) communication.

Referring now toFIG. 3, another example computing device160for providing context sensitive input tools to a user108is illustrated. The computing device160is in communication with a computing device180of the user108via the network250(such as the Internet). The computing device180is illustrated as a desktop computer, but it should be appreciated that the computing device180could be any computer or computer system, such as the computing device100illustrated inFIGS. 1-2. Additionally, the computing device160will be described in the context of operating as a server, but the computing device160can also be any other type of computer or computer system.

Similar to the computing device100described above, the computing device160can include a processor300and a communication device304, which can operate in a manner similar to the processor200and the communication device204, respectively, described above. The computing device160can further include a language model308and a context model312, which can operate in a manner similar to the language model308and a context model312, respectively, described above. Further, it should be appreciated that, while shown and described herein as separate components of the computing device160, one or both of the language model308and the context model312can be implemented by the processor300. The computing device160can communicate with the computing device180of the user108via the network250.

The techniques described herein can be performed by any of the computing devices100,160,180working alone or in conjunction with one another. For the sake of simplicity, however, the description below will primarily refer to various operations of the computing device100. It should be appreciated that the operations can be performed by one or more specific components of the computing device100(such as the processor200or the communication device204), the computing device160or180and/or specific components thereof, or a combination of these elements.

As mentioned above, the user108can provide input to the computing device100via any one or more input devices, such as the display104, the soft keyboard114, a physical keyboard (not shown), or a microphone (not shown). The input can be a keyboard entry, a handwritten stroke or strokes (for handwriting-to-text functionality), or a voice input (for speech-to-text functionality), although other forms of inputs could be utilized. The input can include one or more characters (or portions of a character) in a first script representative of text in a particular language. For example only, in the case of a Pinyin IME, the user108can provide text input in Latin script that is representative of text in the Chinese language.

The computing device100can receive the input from the user108directly (from the user108interacting with the computing device100) or indirectly (e.g., the computing device160can receive the input from the user108via another computing device100,180). The input can be received in association with a document. A document can be any textual record to which the input is to be added, including, but not limited to, an email or other electronic message, a web page, and a document being created/edited by the user108. Other types of documents include, e.g., an email string to which the user108is replying, and one or more previous electronic messages that have been sent to or received from the intended recipient of the electronic message being created by the user108.

In order to provide text input assistance, the computing device100can determine the context of the input, e.g., based on one or more semantic topics of the document associated with the input. An input of text to a document can be expected to be at least somewhat related to the semantic meaning or topic(s) of the document. Thus, the context of the input may be selectively utilized as a signal to assist in determining one or more candidates (characters, words, phrases, etc.) for the input. For example only, if a document is describing a war or a battle of armies and a user108provides the input text “piece,” it may be advantageous to provide the word “peace” as a candidate option for the user108. In this example, the candidate “peace” is an example of autocorrect functionality as it is a spelling correction of the “piece” input of the user108.

The use of the context of the input as described herein is distinct from the utilization of a language model208,308. A language model208,308can express a probability of occurrence of one or more tokens (e.g., words) in a particular language. For example, a language model208,308can describe the probability of a specific token appearing after a given sequence of previously input tokens. Language models are typically described in relation to n-grams, which refer to the probability of a particular token based on the previous (n-1) tokens (n=1 is a unigram model, n=2 is a bigram model, etc.). In contrast to a language model208,308, a context model212,312can be utilized to describe longer distance relations between tokens. For example only, referring to the example of “war” and “piece/peace” above, if the token “war” is outside of the previous n tokens in the document, an n-gram language model208,308will not capture any relation between “war” and “piece/peace” as described. A context model212,312that is utilized to determine the context of the input (e.g., one or more semantic topics of the document associated with the input), however, may be able to capture such a relation between “war” and “piece/peace” if these tokens relate to the same semantic topic(s).

As mentioned above, the context of the input can be determined based on one or more semantic topics of the document associated with the input. The semantic topics are a set of topics or concepts related to the text (words, phrases, etc.) of the document. A semantic analysis of the text of a document can be performed to extract the semantic topics.

In some embodiments of the present disclosure, the semantic topics can be extracted from the document by performing Latent Semantic Analysis, Latent Dirichlet Allocation, Replicated Softmax Model, Deep Boltzmann Machine, or a combination of these (or other) techniques. Additionally or alternatively, for a document that is a web page, the semantic topics could be extracted based on keywords associated with the web page. For other types of documents, the semantic topics can be based on the text that has already been entered by the user preceding the current input. It should be appreciated that other techniques for determining the context of the input may be utilized in addition, or as an alternative, to the techniques described above.

A context model212can be created and utilized by the computing device100to determine the context of the input. For example only, the context model212can be generated by a supervised machine learning algorithm that utilizes labeled training data to infer a relationship between documents and semantic topics. Alternatively, the context model212can be generated by an unsupervised machine learning algorithm, a semi-supervised machine learning algorithm, or a combination of all three of these types of algorithms.

In each case, the context model212can include a context identifier for each known text element (words, phrases, etc.). The context model212further includes a plurality of semantic topics, as well as a score for each known text element in relation to each of the semantic topics. Each of the scores is indicative of the correlation between the text element and its associated semantic topic, e.g., the probability that a particular text element is correlated with a particular semantic topic. The context model212can be used to identify the semantic topics, as well as the scores, based on the context identifier(s) of a particular document, as described more fully below.

The context of the input can be determined by identifying the text elements (words, phrases, etc.) of the document associated with the input being received. The context identifier for each of these text elements can be determined from the context model212. Based on the determined context identifiers, the semantic topics and scores for each of the identified text elements of the document can be determined. The scores can be combined to determine which semantic topic or semantic topics are probable for the document. The context model212can determine the probability of occurrence of further text elements (such as the input) based on the correlation between text elements and the determined semantic topics. The probability of occurrence can be utilized in conjunction with the language model208to identify probable candidates for the input of the user.

Additionally, the computing device100can determine a probability for each candidate of the one or more identified candidates. The probability for a particular candidate can be based on a likelihood that the particular candidate is representative of the input. The probability can be based on the context of the input from the context model212and the language model208.

As described above, both the context model212and the language model208can provide a probability for a particular candidate. In some embodiments, the individual probabilities from each of the language model208and the context model212can be combined to determine a combined probability for each of the one or more candidates. The combination of the probabilities from the language model208and the context model212for a particular candidate can be determined based on the equation:

where w is the particular candidate, history is the information upon which the candidate is based (e.g., for the language model208, the history can be the known n-grams, and for the context model212, the history can be the context of the input), P(w|history) is the combined probability, Plangmod(w|history) is the probability from the language model, Pcntxtmod(w|history) is the probability from the context model, and α is a parameter determined to provide the best fit to training data. In some embodiments, α is selected to equal 0.3, although other values could be utilized. The combined probability can be utilized, e.g., to determine a ranked order of the one or more candidates.

In some embodiments, the computing device100can utilize the input and the language model208to generate the one or more candidates for the input and a probability for each of the candidates. The computing device100can then utilize context of the input (from the context model212) to adjust the probability for each of the candidates, e.g., by determining a combined probability for each candidate. In this manner, the context of the input is utilized to assist in the determination of the most probable candidates, rather than assisting in the determination of possible candidates.

In some embodiments, the context of the input can be utilized to retrieve a topic-specific dictionary. A topic-specific dictionary is a listing of text elements (words, phrases, etc.) that are associated with a particular semantic topic. The topic-specific dictionary can include unique words that are not present in the standard language model208. Upon determining the context of the input, a topic-specific dictionary corresponding to the identified semantic topics of the document can be retrieved. The input can then be compared to the entries of the topic-specific dictionary to determine one or more candidates for the input.

Once the one or more candidates have been determined, the computing device100can output a list of the one or more candidates (or a subset of the one or more candidates) for display to the user108. For the computing device100that includes a display104, the outputting of the list of candidates can include displaying the candidates. For the computing device160, the outputting of the list of candidates can include providing the list of candidates to another computing device100,180for display by the other computing device100,180. In some embodiments, the list of candidates can be output in the ranked order, e.g., determined based on the combined probability described above.

Once the list of candidates is output to the user108, the user108can select a particular candidate as representative of the input intended by the user108. The computing device100can receive the selection of the particular candidate for inclusion in the document. Further, the computing device100can update the context of the input based on the particular candidate selected. That is, once the user108has selected a particular candidate for inclusion in the document, that particular candidate becomes a portion of the document. The context of the updated document, which now includes the selected candidate, can then be determined and utilized for determining one or more candidates for a further input by the user108.

Referring now toFIG. 4, an example technique400for assisting a user108in providing text input to a computing device100is illustrated. While described as being performed by the computing device100, it should be appreciated that the operations can be performed by one or more specific components of the computing device100(such as the processor200or the communication device204), the computing device160or180and/or specific components thereof, or a combination of these elements. Further, the technique400can be implemented by a computer system that includes: (i) one or more processors, and (ii) a non-transitory, computer readable medium storing instructions that, when executed by the one or more processors, cause the computer system to perform the operations of the technique400.

At404, the computing device100receives an input from the user108. The input can include one or more characters in a first script that is representative of text in a particular language. Further, the input can be received in association with a document that is being created/edited by the user108. At408, the computing device100can determine a context of the input based on one or more semantic topics of the document associated with the input. A context model212can be utilized to determine the context of the input from the document (e.g., the text of the document) in any of the manners described above.

One or more candidates for the input can be determined at412. The one or more candidates can be determined based on (i) the input, (ii) the context of the input, and (iii) a language model208. As described above, the language model208can express a probability of occurrence of the one or more candidates in the particular language. The candidates can include one or more characters in a second script representative of the text in the particular language. In the situation where the computing device100is providing autocorrect and/or autocomplete functionality, the first and second scripts can be identical. In the situation where the computing device100is providing an IME functionality (alone or in combination with autocorrect and/or autocomplete), the first and second scripts can be different. For example only, the user108may provide the input in the Latin alphabet to input Chinese text in Hanzi characters utilizing a Pinyin IME.

At416, a probability for each candidate of the one or more candidates can be determined based on the context of the input (from the context model212) and the language model208. A ranked order of the candidates can be determined at420. The ranked order can be based on the probability for each candidate. At424, the list of the one or more candidates can be output for display to the user108. In some embodiments, the list can be output in the ranked order determined at420. A selection of a particular candidate from the list of the one or more candidates can be received at428. Based on the particular candidate selected, at432the context of the input can be updated. The technique400may then end or return to404for one or more additional cycles.

As used herein, the terms module or device may refer to, be part of, or include an Application Specific Integrated Circuit (ASIC); an electronic circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code, or a process executed by a distributed network of processors and storage in networked clusters or datacenters; other suitable components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip. The terms module or device may include memory (shared, dedicated, or group) that stores code executed by the one or more processors.