Method of and system for processing a user-generated input command

A method of processing a user-generated input command executable at a computing apparatus includes, for each one of the plurality of machine-executable instructions, receiving a plurality of example expressions of the machine-executable instruction, generating a plurality of features for the plurality of example expressions, and analyzing the plurality of features associated with the plurality of example expressions to determine a plurality of feature patterns. The plurality of feature patterns is used to convert the user-generated input command into a machine executable output instruction, the machine executable output instruction being one of the plurality of machine-executable instructions. Systems for processing a user-generated command are also disclosed.

The present application claims priority to Russian Patent Application No. 2014135305, filed Aug. 29, 2014, entitled “METHOD OF AND SYSTEM FOR PROCESSING A USER-GENERATED INPUT COMMAND” the entirety of which is incorporated herein.

FIELD

The present technology relates to methods and systems for processing a user-generated input command.

BACKGROUND

Natural language processing (NLP) is used in many different applications where a user interacts with a machine via speech (i.e. by issuing one or more spoken commands). An example of such an application includes voice-activated control systems for various kinds of devices such as smartphones, televisions, household appliances, and the like. Automated assistance at call centers and switch boards, provide another example of application for NLP. The NLP application receives user-generated speech as input, converts the speech into text, and then determines the part of the received text that includes commands or operations that the user would like to have executed, and then accordingly generates machine-executable instructions.

Improving the performance of the NLP application depends on improving the accuracy and efficiency of processing the user-generated speech and the text derived therefrom. The received user-generated text is tagged with features or tags to help determine the meaning and commands embedded in the text. The tags or features are generated by using some large body of text (“text corpus”) in which each word has been manually assigned with features. In order to improve accuracy of the NLPA, the size of the tagged text corpus is increased. This is very resource intensive, and cannot be done quickly or easily.

There is therefore a need for improving the ability to process user-generated speech more efficiently and less resource intensively.

SUMMARY

Embodiments of the present technology have been developed based on inventors' appreciating that there exists at least one problem or an area for improvement associated with the prior art solutions.

As such, according to a first broad aspect of the present technology, there is provided a method of processing a user-generated input command executable at a computing apparatus includes, for each one of the plurality of machine-executable instructions, receiving a plurality of example expressions of the machine-executable instruction, generating a respective feature for the plurality of example expressions, and analyzing the plurality of features associated with the plurality of example expressions to determine a plurality of feature patterns. The plurality of features includes the feature for each of the plurality of example expressions. The plurality of feature patterns is used to convert the user-generated input command into a machine executable output instruction, the machine executable output instruction being one of the plurality of machine-executable instructions.

In another implementation, said analyzing includes obtaining an occurrence count for each of the plurality of features, and analyzing the plurality of features after discarding therefrom, a low occurrence feature of the plurality of features, the low occurrence feature having an occurrence count being smaller than a threshold occurrence count.

In another implementation, said analyzing comprises analyzing using a Charm Bitset algorithm.

In another implementation, said generating the plurality of features for each one of the plurality of example expressions includes generating the plurality of features based on a predetermined rule.

In another implementation, said generating the plurality of features for each one of the plurality of example expressions includes generating the plurality of features based on a statistical model.

In another implementation, the statistical model is one of: a Hidden Markov Model; and a conditional random field model.

In another implementation, the method includes generating, by the computing apparatus, an additional example expression for a command of the plurality of commands; and including the generated additional example expression in the plurality of example expressions for each machine-executable instruction of the plurality of machine-executable instructions.

In another implementation, the method includes, before including the generated additional example expression in the plurality of example expressions, presenting the generated additional example expression to a training administrator. Responsive to said presenting, a validation is received from the training administrator, for the generated additional example expression, the validation being indicative of a validity of the generated additional example expression.

In another implementation, said including the generated additional example expression in the plurality of example expressions is further responsive to the validation being indicative of the generated additional example expression being a valid expression.

In another implementation, said including the generated additional example expression in the plurality of example expressions comprises associating the validation received for the generated additional example expression with the generated additional example expression.

In another implementation, said generating of the additional example expression includes accessing an external resource communicatively coupled with the computing apparatus.

In another implementation, the external resource is a network resource communicatively coupled with the computing apparatus, and the method includes crawling the network resource to obtain information for generating the additional example expression.

In another implementation, the method includes receiving the user-generated input command and using the plurality of feature patterns to convert said user generated input command into the machine-executable output instruction.

In another implementation, the feature associated with one of the plurality of expressions is indicative of one of: a grammar element, a root of a word related to the example expression, a normalized form of the word, a punctuation element, a word separator, and a command.

In another implementation, a feature weight is assigned to each feature of the plurality of features.

According to another broad aspect of the present technology, there is provided a system for processing a user-generated input command. The system includes a communication interface for receiving a plurality of example expressions of a machine-executable instruction of a plurality of machine executable instructions, and a training module communicatively coupled to the communication interface and having a processor configured to generate a plurality of features for the plurality of example expressions of the machine-executable instruction, and analyze the plurality of features associated with the plurality of example expressions of the machine-executable instruction to determine a plurality of feature patterns for the of the machine-executable instruction. The plurality of feature patterns is used to convert a user-generated input command into a machine executable output command, the machine executable output instruction being one of the plurality of machine executable instructions.

In another implementation, the training module is communicatively coupled to an external resource, and the processor is further configured to generate an additional example expression for a machine-executable instruction of the plurality of machine-executable instructions, and include the generated additional example expression in the plurality of example expressions.

In another implementation, the training module is coupled to an external resource, the generated additional example expression being generated by accessing the external resource.

In another implementation, the external resource is a network resource, and the training module is coupled to a crawling module configured to crawl the network resource.

In the context of the present specification, a “network resource” is any data or collection of data that can be provided by a publisher over a network and that is associated with a network resource address. Non-limiting examples of network resources include HTML pages, documents, images, video, feed sources, as well as pluralities of files such as the foregoing. Network resources may include content, such as words, phrases, pictures, and so on, and/or embedded information such as metadata, hyperlinks and/or embedded instructions (such as JavaScript scripts).

In the context of the present specification, “client device” or “electronic device” is any computer hardware that is capable of running software appropriate to the relevant task at hand. Thus, some (non-limiting) examples of client devices include personal computers (desktops, laptops, netbooks, etc.), smartphones, and tablets, as well as network equipment such as routers, switches, and gateways. It should be noted that a device acting as a client device in the present context is not precluded from acting as a server to other client devices. The use of the expression “a client device” does not preclude multiple client devices being used in receiving/sending, carrying out or causing to be carried out any task or request, or the consequences of any task or request, or steps of any method described herein.

DETAILED DESCRIPTION

With reference toFIG. 1, there is shown a schematic diagram of a network environment100suitable for implementing non-limiting embodiments of the present technology. It is to be expressly understood that the network environment100is depicted merely as an illustrative implementation of the present technology. Thus, the description thereof that follows is intended to be only a description of illustrative examples of the present technology. This description is not intended to define the scope or set forth the bounds of the present technology.

In some cases, what are believed to be helpful examples of modifications to the network environment100may also be set forth below. The modifications are described merely as an aid to understanding, and again, not to define the scope or set forth the bounds of the present technology. These modifications are not an exhaustive list, and, as a person skilled in the art would understand, other modifications are likely possible. Further, where this has not been done (i.e. where no examples of modifications have been set forth), it should not be interpreted that no modifications are possible and/or that what is described is the sole manner of implementing that element of the present technology. As a person skilled in the art would understand, this is likely not the case. In addition it is to be understood that the network environment100may provide in certain instances simple implementations of the present technology, and that where such is the case they have been presented in this manner as an aid to understanding. As persons skilled in the art would understand, various implementations of the present technology may be of a greater complexity.

The network environment100includes an electronic device102coupled to a communications network110via a communication link103.

The electronic device102is typically associated with a user (not depicted) and, as such, can sometimes be referred to as a “client device”. In the illustrated embodiment, the electronic device102is a smartphone102. The implementation of the electronic device102is however not limited to a smartphone102. As an example, the electronic device102may be implemented as a personal computer (desktops, laptops, netbooks, etc.), a wireless electronic device (a tablet and the like), as well as network equipment (a router, a switch, or a gateway). The general implementation of the electronic device102is known in the art and, as such, will not be described here at much length.

Although the present description is made with reference to the network environment100having one electronic device102, it should be understood that the network environment100could include more than one electronic device102. The number of electronic devices is not limited to the two depicted herein.

The electronic device102includes a user input interface (such as a keyboard, a mouse, a touch pad, a touch screen, microphone, and the like) for receiving user inputs. The electronic device102includes a user output interface (such as a screen, a speaker, a printer and the like) for providing visual, auditory or tactile outputs to the user. The electronic device102includes a network communication interface (such as a modem, a network card and the like) for two-way communication over the communications network110via the communication link103. The electronic device102also includes a processor coupled to the user input interface, the user output interface and the network communication interface. The processor is configured to execute various methods, including those described herein below. To that end the processor may store or have access to computer readable commands which, when executed, cause the processor to execute the various methods described herein. The electronic device102comprises hardware and/or software and/or firmware, as is known in the art, to execute various applications. The electronic device102has some applications, such as a dictation application150, which are configured to receive user-generated speech as input and execute or cause execution of one or more machine-executable instructions responsive to the received user-generated speech. The electronic device102also has applications configured to receive and transmit information via the communication network110. Examples of such applications include, the dictation application150, a browser application, a music streaming application, a photo sharing application, and the like.

In the illustrated embodiment of the present technology, the communications network110is implemented as the Internet. In other embodiments of the present technology, the communications network110can be implemented differently, such as in the form of a wide-area communications network, a local-area communications network, a private communications network and the like.

The communication link103can also have various non-limiting implementations, and the particular implementation(s) of the communication link103for the electronic device102will depend on how the electronic device102is implemented. In the illustrated embodiment, the communication link103for the electronic device102can be either wireless (such as the Wireless Fidelity, or WiFi® for short, Bluetooth® or the like) or wired (such as a Universal Serial Bus or USB-based connection). Furthermore, the communication link103coupling the electronic device102to the communication network110could also include more than one type of link. For example, the smartphone102could be coupled to the network110via wireless as a well as a wired connection.

It should be expressly understood that implementations for the electronic device102, the communication link103, and the communications network110are provided for illustration purposes only. As such, those skilled in the art will easily appreciate other specific implementational details for the electronic device102, the communication link103, and the communications network110. The examples provided herein are not meant to limit the scope of the present technology.

Also coupled to the communications network110is an application server120. The application server120is connected to a subset of the electronic devices connected to the communication network110. For example, in the illustrated embodiment of the network environment100, the application server120is coupled to the electronic device102. The application server120could also be connected to other servers, such as but not limited to network resource servers, application servers, and other network configuration servers, via the communication network110.

The application server120can be implemented as a conventional computer server. In an example of an embodiment of the present technology, the application server120can be implemented as a Dell™ PowerEdge™ Server running the Microsoft™ Windows Server™ operating system. The application server120could also be implemented as other kinds of network equipment such as, but not limited to, a router, a switch, or a gateway, a base station and the like. The application server120could be implemented in any suitable hardware and/or software and/or firmware, or a combination thereof. In the depicted non-limiting embodiment of present technology, the application server120is a single server. In alternative non-limiting embodiments of the present technology, the functionality of the application server120could be distributed and may be implemented via multiple servers. The implementation of the application server120is well known. However, briefly speaking, the application server120comprises a communication interface (not shown) structured and configured to communicate with the electronic device102, and other devices coupled to the communications network110. The application server120further comprises at least one computer processor (not shown) operationally connected with the communication interface and structured and configured to execute various processes to be described herein.

The network environment100also includes a network resource server130hosting three network resources130a,130b,130cthat can be accessed by connecting to the network resource server130via the communication network110. The network resource server130could be implemented as a conventional computer server. In an example of an embodiment of the present technology, the network resource server130could be implemented as a Dell™ PowerEdge™ Server running the Microsoft™ Windows Server™ operating system. The network resource server130could also be implemented in any other suitable hardware and/or software and/or firmware or a combination thereof. In the depicted non-limiting embodiment of present technology, the network resource server130is a single server. In alternative non-limiting embodiments of the present technology, the functionality of the network resource server130may be distributed and may be implemented via multiple servers. It should also be understood that the network resource server130and the network resources130a,130b,130chosted by the network resource server130as presented herein is exemplary. The network environment100could include any number and kind of network resource servers130and each network resource server130could host any number and kind of network resources.

In the illustrated embodiment, for convenience and simplification of description of the present technology, the network resource server130is assumed to be a web-resource server and the network resources130a,130b,130c, hosted by the network resource servers130are assumed to be web resources (or websites). It should however be understood, that the present technology is no way limited to web resource servers and web resources.

Natural Language Processing Application

In the context of the present description, a natural language processing application is an application which converts a user-generated commands300(FIG. 2) into a machine-executable instruction310, and vice versa. The description below is provided with reference to a dictation application150, in which user-generated speech received as input is converted to text and processed to detect a command300embedded within the text and convert the commands to a machine executable instruction310.

With reference toFIG. 1, the dictation application150is provided on the electronic device102to enable the user (not shown) of the electronic device102to input speech via a microphone of the electronic device. The text input by the user of the electronic device102comprises dictated text content that the user wants transcribed into a document. The input speech also includes one or more user-generated input commands300to be executed or operated on the dictated text content.

While using the dictation application150, a user may want to perform operations such as, but not limited to: add to a document, edit sentence, delete a word, save a document, continue a paragraph, begin on the next page, open a saved document, create a new document, and the like. The desired operations include a command (edit, add, save, create, etc. . . . ) and objects on which the operations are to be performed (word, sentence, document, page, etc. . . . ) One of the challenges for the dictation application150is to identify the user-generated commands300embedded in the speech input by the user. The other challenge is to resolve ambiguities in the speech input by the user.

For each of the operations, a request for execution of the operation can be expressed in many different ways by the user of the application. For example, the operation “add” can be requested by using commands such as “put in”, “combine”, “insert”, and other such word(s). In order to execute the requested operation, the processor requires instruction(s) in a specific form that are specific to the particular programming language in which the operating system for the processor is implemented. The dictation application150thus needs to convert the user-generated commands300embedded in the input received from the user to machine-executable instructions310. The dictation application150therefore needs to learn different natural language expressions for each machine-executable instruction310.

In general, dictation application is implemented by a natural language processing module (not shown). In this implementation, the natural language processing module is integrated with the processor of the electronic device102. In some other alternative implementations, the natural language processing module is integrated with a server, such as the application server120. In some other embodiments, a portion of the natural language processing module can be partly integrated with the electronic device102, and another portion of the natural language processing module is integrated with the application server120. The natural language processing module includes a combination of hardware and/or software and/or firmware for operation of the dictation application150. The processor includes a speech recognition unit that detects and converts detected speech into text data, an analyzer for analyzing the text data as will be described below, and a parser for converting the analyzed text data to one or more machine executable instructions320.

The dictation application receives as input speech which may include one or more commands to perform operations such as, but not limited to: add, edit, delete, save, continue a document, open a saved document, create a new document, and the like. As mentioned above, the input speech is converted to text data by a speech-to-text converter. The dictation application150then analyzes the text data. The analysis of the text data comprises assigning features400(FIG. 4A) to each of the words in the text data.

The features400are descriptors for the words, and include part-of speech descriptors such as verb, noun, singular, plural, subjunctive, infinitive, and the like. The features may also include descriptors such as “command” or “not command” for identifying whether the word is an acceptable form of a command.

In some implementations, different types of features400are assigned to each word. Examples of types of features include the word as is, the normalized form of the word, a grammar element or part-of speech, a common root of the word, punctuation, word separator, command, not a command, and the like. Each type of feature can be represented symbolically, for example, by a number or a hash value. In some implementations, the features are assigned weights which are indicative of the importance of the feature type for processing the user-generated input command300. In some implementations, the weights are assigned manually, and in other implementations, the weights are assigned automatically with the help of machine learning routines.

For example, the word as is could be represented by “1”, and assigned a weight of 5.0; the normalized form of the word could be represented by “2”, and assigned a weight of “5.0” the feature type “grammar element” could be represented by the number “3” and assigned a weight of 1.0, the feature type “command or not command” could be represented by the number “9” and assigned a weight of 3.0, and so on and so forth. Each word is also assigned a value for each type of feature400. Thus, for the word “delete”, feature types assigned could be [1, 2, 3, 3, 9] with respective values [delete, delete, verb, present, command] and respective weights [5, 5, 1, 1, 3].

The features400are assigned to a word based on a morphological and lexicological analysis of the word, and taking into account the neighboring words. In some implementations, if a particular word(s) could have more than one meaning, then the dictation application150looks at the features400of the neighbouring words in order to assign features400to that word.

The input text data is analyzed by comparing to a training set of data. The comparison is made by determining an intersection between the features400of the words of the input text and the feature patterns410stored in a feature pattern database provided by a training module220, and that of the input text data. The accuracy of the dictation application150increases with the size of the feature pattern database, and/or with its ability to use the available feature patterns to correlate a particular machine-executable instruction310with different possible user-generated commands300.

With reference toFIG. 2, the dictation application150is operatively connected to the training module220for learning different user-generated commands300associated with a machine-executable instruction310, and to learn different feature patterns410associated with each machine-executable instruction310as will be described below.

In the illustrated non-limiting embodiment, the training module220is integrated with the dictation application150. Thus in the illustrated embodiment, the training module220is integrated with the electronic device102. In some other alternative embodiments, the training module220is integrated with a server, such as the application server120. In some other embodiments, a portion of the training module220is integrated with the electronic device102, and another portion of the training module220is integrated with the application server120. The training module220includes a combination of hardware and/or software and/or firmware for the execution of various tasks related to training of the dictation application150as will be described below. The training module220is also operatively coupled to resources330for accessing information. The resources330could be external resources such as network resources such as web-pages, databases, and the like. The resources230could also be locally stored or integrated with the training module220. Fir example, a dictionary, or a thesaurus could be stored locally on a memory storage device of the training module220. The training module220is also coupled to a communication interface240by which can communicate with a training administrator.

The training administrator is a human operator that interacts with the training module220for the creation of the feature patterns410and example expressions320associated with different machine-executable instructions310.

Although, the description herein will be provided with reference to the dictation application150, it should however be understood that the methods and systems of the present technology are not limited to the dictation application150, but could be applied to any natural language processing application which processes natural language. For example, the natural language processing application could be an automated call directing application implemented that receives speech input by a caller, processes the received input speech to determine the caller's request, and accordingly directs the phone call to the appropriate location. As another example, the natural language processing application could be a voice-activated control system for a television, a text-to-speech application for converting text to speech, and the like.

Method

A method600of the present technology for processing a user-generated input command300will now be described with reference toFIGS. 3A to 5.

As mentioned above, the processing of the user-generated input command300can be used in the context of a natural language processing application, such as the dictation application150which converts a user-generated input command300received as input to a machine-executable instruction310. The machine executable instruction310is output to an entity that will execute or cause execution of the machine executable instruction310. The dictation application150converts a user-generated input command300received as input to a machine executable instruction310which is output to the processor of the electronic device102.

The method600will be described in the context of the dictation application150, however, as mentioned above the method600is not limited to a dictation application150, but can be used with any natural language processing application that causes execution of one or more machine executable instructions310in response to receiving one or more user-generated input commands300.

The method600is executed by the training module220. The method600begins at step610when the training module220receives a plurality of example expressions320for each machine-executable instruction310of a plurality of machine-executable instructions. In some implementations, the plurality of example expressions320is received from a training administrator. Each example expression320is a word or a phrase that can be used to express machine-executable instruction310in normal human language. In other words, each example expression320is a possible user-generated input command300.

For example, for the machine-executable instruction310“delete”, the training administrator enters a list of example expressions320as shown inFIG. 3A. Each of the example expressions320describes the operation “delete” in normal human language. Each example expression320could have one word, or a plurality of words. Thus, each example expression320could be a word or a phrase.

Similarly, the training administrator can enter a list of example expressions320for a plurality of machine-executable instructions310that are executable by the computing apparatus. As mentioned above, the list of machine-executable instructions310could include instructions such as “add”, “show”, and the like. The particular list of machine-executable instructions310for which the training module220receives example expressions320could be particular to the particular implementation of the natural language processing application. For example the list of machine-executable instructions310could be different for a call center application compared to the dictation application150.

At step620, features400are generated for each of the plurality of example expressions received at step610. The features400have been discussed above. In some implementations, the features400are generated using predetermined rules. For example, if the word “book” is preceded by “a”, the word book is assigned a value “noun” for the feature type “grammar element”. In some implementations, the features400are generated using algorithms based on statistical models such as Hidden Markov Model and/or a conditional random field (CRF), and the like. Each example expression320is associated with a set of features400. For example, with reference toFIG. 4A, the word “remove” is assigned a set of features [123339].

In order to assign features400, each example expression320is divided into units comprising a word or a group of words, and then the features400are assigned to each unit of the example expression. For example, in some implementations of the method600, the example expression “cut the sentence”, is divided into units “cut”, “the” and “sentence” and respectively assigned features400[122], [48], and [346]. In some other implementations of the method600, the example expression could be divided into units “cut” and “the sentence” and assigned features400[122] and [3468].

At step630, the features400associated with the example expressions320for a particular machine-executable instruction310are analyzed to determine feature pattern(s)410(FIG. 4B) for that machine-executable instruction310. The features400of the example expressions320are analyzed using pattern mining algorithms, such as, but not limited to, the Charm Bitset algorithm. It is contemplated that any suitable pattern mining algorithm could be used to determine feature patterns410associated with each machine-executable instruction310. Thus, as step630, each of the plurality of machine executable instructions300is associated with a plurality of feature patterns410. The feature patterns410associated with each command are indicative of the occurrence of particular features400in combination with other features400for that machine-executable instruction310.

By analysing the features400associated with the example expressions320for the plurality of machine-executable instructions310and obtaining feature patterns410associated with each machine executable instruction310, the training module220learns the rules for the occurrence of particular features400in combination with other features400. The size of the database of feature patterns410available to the dictation application150can be increased by adding additional examples320of usage for a particular machine-executable instruction(s)310, and then pattern mining the features400associated with the example expressions320for that particular machine-executable instruction(s)310without having to pattern mine an entire text corpus, which is a much more resource intensive task.

In some implementations, before mining the plurality of features400associated with the plurality of example expressions320to determine feature patterns410for a machine-executable instruction310, low occurrence features400are discarded from the plurality of features400associated with plurality of example expressions320for that machine-executable instruction310. Thus, features400that appear rarely in the example expressions320for a particular machine-executable instruction310are discarded before mining the plurality of features400associated with the example expressions320to determine feature patterns410associated with that machine-executable instruction310. Thus, in some implementations, the number of occurrences (occurrence count) of a feature400in the plurality of features400associated with the plurality of example expressions320for a machine-executable instruction310is counted. If the occurrence count of a feature400is lower than a threshold occurrence count, that feature400is discarded or deleted form the plurality of features400assigned for the features400associated with the plurality of example expressions320that machine executable instruction310.

In some implementations, one or more additional example expressions330generated by the training module220for one or more of the machine-executable instructions310. The additional example expressions330generated for a particular machine-executable instructions310are then included in the plurality of example expressions320for that machine-executable instruction310.

The additional example expressions330are generated by obtaining words and phrases having a similar meaning as the machine-executable instruction310, or one of the example expressions320thereof. The training module220accesses one or more resources230such as a dictionary, a thesaurus, a networked resource such as the web resources130a,130b,130c, and the like in order to generate the additional example expressions330. For the example scenario ofFIGS. 3A and 3B, the training module220could access a thesaurus to obtain synonyms for the word “erase” and thereby generate additional example expressions “eliminate”, “annul” and “expunge” and “blot” for the machine-executable instruction310“delete”. It is contemplated that a resource230could be stored locally in the training module220, and/or coupled to the training module220via the communication network110. In some implementations, the training module220is coupled to a web-crawling module to obtain information therefrom. In some implementations, the generation of additional example expressions330by the training module220could be initiated by the occurrence of a web-crawling event operation.

In some implementations, a generated additional example expression330is presented to the training administrator for validation. The generated additional example expression330is presented to the training administrator via the communication interface240. The training administrator then evaluates the generated example expression330and rates it as valid or invalid. It is contemplated that the validation for the generated additional example expression330could be provided in the form of a validity rating on a scale having more than two possible values.

In some implementations of the method600, the additional example expression330generated by the training module220for a particular machine-executable instruction310is included in the plurality of example expressions320received therefor (from the training administrator) responsive to the validation received from the training administrator for that generated additional example expression330. Thus, in the exemplary scenario for the machine executable instruction310“delete”, the generated additional example expressions330“eliminate”, “annul” and “expunge” and “blot” are presented to the training administrator. In this exemplary scenario, the validation received from training administrator for the generated example expression330“eliminate” could be indicative of its validity, or indicative of it being above a threshold validity level. Responsive to the receipt of the positive validation from the training administrator, the generated example expression330would be included in the plurality of expressions320associated with the machine-executable instruction310“delete”.

In some implementations, all of the additional example expression330generated by the training module220for a particular machine-executable instruction310are included in the plurality of example expressions320along with the corresponding validation thereof. Thus, each additional example expression330is included with the plurality of example expressions320regardless of its validation being positive or negative, or the validity rating. In some implementations, the validation of the additional example expression330is associated the additional example expression330. The validation of the additional example expression330may be included as a feature400for the additional example expression330. Thus, the plurality of example expressions, in this implementation, could include example expressions320that are invalid or inoperative for describing the corresponding machine executable instruction. This information related to invalid or inoperative example expressions is used to increase the accuracy of the processing of a user-generated command300to a machine-executable instruction310. The inclusion of the invalid or inoperative example expression reduces errors in the conversion of a user-generated command300to a machine-executable instruction310by providing the dictation application150with instances of inoperative or invalid usages of the machine-executable instruction310.

In some implementations, the plurality of example expressions320for a machine executable instruction310is sorted in descending order of proximity to the corresponding machine-executable instruction310. Thus, in some implementations, a proximity parameter is determined for each example expression with respect to the corresponding machine-executable instruction310.

It should be understood that although the descriptions was provided with reference to English language examples, it should be understood that the methods and systems described above are not limited to being implemented for any one language. The methods and systems described above can be implemented in any language and can also be implemented in more than one language.