Patent Description:
Often, natural language user interface mechanisms are supported by an existing, pre-trained natural language model. Such an existing natural language model can already understand various idioms of human speech and can recognize, for example, that the concept of population can be linguistically expressed utilizing phrasing such as "the number of people who live in", "the population of", "how many people are in", and other like phrasing. However, such existing natural language models still need to be primed in order to provide access to the specific functionality offered by specific application programs. For example, an application program providing home automation control can receive different types of natural language commands than an application program providing geographic information. Typically, to prime an existing natural language model, the developer of an application program can provide multiple examples of natural language input that invokes a specific function of the application program. A user natural language input is then compared to the provided examples utilizing the existing natural language model as a basis for the comparison. Whichever example is closest, within the context of the existing natural language model, to the received user natural language input, determines which functionality of the application program is associated with the received user natural language input.

Unfortunately, the examples provided are often suboptimal since the existing natural language model is typically developed independently of the application program seeking to utilize such an existing natural language model in order to enable the application program to receive and process natural language input. Thus, developers of application programs often provide natural language input examples that provide a suboptimal priming, which can, in turn, degrade the overall natural language input processing performance, since users' natural language inputs will be compared against those suboptimal examples. Additionally, examples from one domain can overlap, and cause ambiguity, with examples from another domain, which can be especially significant in instances where application programs are constructed from reusable components, which may, themselves, be developed independently of the application programs within which they are utilized. For example, one component can provide its own examples which can conflict with the examples provided by another, different component, and those examples can both be in conflict with the examples provided by an application developer of the overall application program that utilized both components. Such overlap, ambiguity and sub-optimality is exacerbated by the use of casual and informal language in examples. In particular, casual and informal language can be imprecise and ambiguous. For example, in informal language, it can be common to use similar terminology to describe the weather as to describe a human's emotional state.

<CIT> relates to systems and methods for receiving a query created by a user, receiving output data of at least one function to retrieve data related to the query and analyzing the output data of the at least one function to retrieve data related to the query. The systems and methods further provide for generating at least one dynamic knowledge graph associated with the output data of the at least one function, wherein the at least one dynamic knowledge graph comprises data from the output data of the at least one function and indicates relationships between the data, analyzing the at least one dynamic knowledge graph to determine data relevant to the query generated by the user, and generating a response to the query based on the data relevant in the at least one dynamic knowledge graph.

It is the object of the present invention to enhance access to programmatic functionality through natural language input.

To optimize the utilization of an existing natural language model in providing natural language input functionality to independently developed application programs, the natural language model can be primed utilizing optimized natural language input examples generated from a labeled knowledge graph corresponding to an independently developed application program. The knowledge graph can be labeled by developers of the independently developed application program and can be provided to an example generator, which can be designed to parse the labeled knowledge graph, and generate therefrom, natural language input examples that will most effectively prime the existing natural language model such that subsequent comparisons of user natural language input to such examples will provide increased accuracy in invoking application program functionality through natural language user input. Parsing of the labeled knowledge graph can include the identification of triples, comprising a source node, a destination node, and a link between them, each of which can be labeled. One or more natural language input examples can be generated from an individual triple by concatenating the natural language words or phrases utilized to label the source node in the link. Such natural language input examples can then be associated with the triple and can provide a source of comparison for subsequent natural language user input. Determinations that subsequently received natural language user input is similar to the generated examples can result in an identification of the triple, which can, in turn, trigger the performance of a function associated with the destination node of the triple. Labels can include preferred labels and alternative labels, and various permutations thereof can be concatenated to generate alternative natural language input examples.

Additional features and advantages will be made apparent from the following detailed description that proceeds with reference to the accompanying drawings.

The following detailed description may be best understood when taken in conjunction with the accompanying drawings, of which:.

The following description relates to optimizing the utilization of an existing natural language model to provide natural language input functionality to independently developed application programs. Such optimization includes the priming of the natural language model utilizing optimized natural language input examples generated from a labeled knowledge graph that corresponds to an independently developed application program. The knowledge graph can be labeled by developers of the independentlydeveloped application program and can be provided to an example generator, which can be designed to parse the labeled knowledge graph, and generate therefrom, natural language input examples that will most effectively prime the existing natural language model such that subsequent comparisons of user natural language input to such examples will provide increased accuracy in invoking application program functionality through natural language user input. Parsing of the labeled knowledge graph can include the identification of triples, comprising a source node, a destination node, and a link between them, each of which can be labeled. One or more natural language input examples can be generated from an individual triple by concatenating the natural language words or phrases utilized to label the source node in the link. Such natural language input examples can then be associated with the triple, and can provide a source of comparison for subsequent natural language user input. Determinations that subsequently received natural language user input is similar to the generated examples can result in an identification of the triple, which can, in turn, trigger the performance of a function associated with the destination node of the triple. Labels can include preferred labels and alternative labels, and various permutations thereof can be concatenated to generate alternative natural language input examples.

Although not required, the description below will be in the general context of computer-executable instructions, such as program modules, being executed by a computing device. More specifically, the description will reference acts and symbolic representations of operations that are performed by one or more computing devices or peripherals, unless indicated otherwise. As such, it will be understood that such acts and operations, which are at times referred to as being computer-executed, include the manipulation by a processing unit of electrical signals representing data in a structured form. This manipulation transforms the data or maintains it at locations in memory, which reconfigures or otherwise alters the operation of the computing device or peripherals in a manner well understood by those skilled in the art. The data structures where data is maintained are physical locations that have particular properties defined by the format of the data.

Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the computing devices need not be limited to conventional personal computers, and include other computing configurations, including servers, hand-held devices, programmable consumer electronics, network PCs, Internet of Things (IoT), and the like. Similarly, the computing devices need not be limited to stand-alone computing devices, as the mechanisms are also practicable in distributed computing environments where tasks are performed by one or more remote processing devices, working in either series or parallel, that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

With reference to <FIG>, an exemplary system <NUM> is illustrated, providing context for the descriptions below. The exemplary system <NUM> of <FIG> illustrates a user <NUM> utilizing a computing device <NUM> to invoke functionality <NUM> offered by the computing device <NUM>. For example, the computing device <NUM> can be an information presenting device which can provide answers to user queries. As another example, the computing device <NUM> can be a home control and automation device. Other like computing devices are equally utilizable with the mechanisms described herein.

For ease of user interaction, the computing device <NUM> can offer natural language input processing so that the user <NUM> can invoke functionality of the computing device <NUM> utilizing natural language input. For example, the user <NUM> can speak a natural language input question or command, such as the exemplary user speech <NUM>, to which the computing device <NUM> can meaningfully respond. Alternatively, or in addition, the user can type, write, or otherwise input natural language phrasing and wording into the computing device <NUM>. For example, the user <NUM> may seek to learn the population of New York City. Accordingly, the user <NUM> can phrase a natural language input question, such as in the form of: "how many people live in New York City?" As another example, the user <NUM> may seek to have the computing device <NUM> control a heating device, such as by phrasing a natural language input command in the form of "turn up the heat.

According to one aspect, rather than creating natural language input processing for each application program, application program developers can leverage a pre-existing natural language model. Such an existing natural language model can be independently created and curated, and can represent functionality and capability that would be difficult or impractical for an application program developer to create on their own. For example, an existing language model can be based on vast quantities of natural language input in order to recognize equivalence among colloquial expressions, common phrasing, and other like natural language subtleties. Such vast quantities of natural language input can include, for example, hundreds of thousands, or even millions of, newspaper articles, published speeches, encyclopedic entries, or other like natural language input. Typically, machine learning can be applied to such vast quantities of natural language input in order to derive some or all of the existing natural language model. In such a manner, an existing natural language model can understand various idioms of human speech and can recognize, for example, that the concept of population can be linguistically expressed utilizing phrasing such as "the number of people who live in", "the population of", "how many people are in", and other like phrasing.

Turning back to the exemplary system <NUM> of <FIG>, application programs executing on a computing device, such as the exemplary computing device <NUM>, can utilize existing natural language models, such as the exemplary pre-trained language model <NUM>, through network communications with one or more server computing devices, such as the exemplary server computing device <NUM>, which can be communicationally coupled to the exemplary computing device <NUM> via the network <NUM>. An application program executing on the exemplary computing device <NUM> can include a natural language user input module, such as the exemplary natural language user input module <NUM>, which can interface with hardware and/or software mechanisms by which the exemplary computing device <NUM> can receive natural language user input. Natural language user input can be provided through a microphone that can receive the voice input <NUM>, a keyboard that can receive typed input, a touchpad and stylus that can receive written input, and other like hardware and/or software mechanisms.

Upon receiving the natural language user input, the natural language user input module <NUM> can communicate the natural language input <NUM> to a natural language decoding component, such as the exemplary natural language decoding component <NUM>. The natural language decoding component <NUM> can utilize an existing natural language model, such as the pre-trained language model <NUM>, to analyze the natural language input <NUM>. More specifically, and as will be detailed further below, the pre-trained language model <NUM> can form a basis by which the natural language input <NUM> is compared with examples, such as the examples <NUM>, that correspond to specific functions provided by the application program functionality <NUM>.

Once the natural language input <NUM> is determined, by the natural language decoding component <NUM>, to be a request for a particular function provided by the application program functionality <NUM>, an unambiguous reference <NUM> to such an application program function can be returned, such as to the natural language user input module <NUM>. The natural language user input module <NUM> can then invoke the appropriate function, from among the application program functionality <NUM>, as illustrated by the invocation action <NUM>.

Although illustrated in <FIG> as executing on specific computing devices, various aspects and components of the exemplary system <NUM> can execute on computing devices different than those illustrated without departing from the context of the descriptions provided herein. For example, aspects of the application program functionality <NUM> can be executed on one or more remote computing devices and can work in concert with processes executing locally on the computing device <NUM>. As another example, aspects of the natural language decoding <NUM>, or the entirety of the natural language decoding <NUM>, can be performed locally on the computing device <NUM>.

Additionally, the exemplary computing device <NUM> can be an appliance, or specific-purpose, computing device, such as a smart thermostat, a voice-controlled light switch, a programmable remote, or other like specific-purpose computing device. In such an instance, the application program functionality <NUM> can be provided by preinstalled application programs. Conversely, the exemplary computing device <NUM> can be a general-purpose computing device, such as a desktop computing device, a laptop computing device, a tablet computing device, smart phone computing device, and other like general-purpose computing devices. In such an instance, the application program functionality <NUM> can generally represent the functionality provided by one or more application programs, including user-installed application programs. The natural language user input module <NUM> can be a component of individual application programs, such that one application program can have its own natural language user input module that can be independent of the natural language user input module of another, different application program. Alternatively, or in addition, the natural language user input module <NUM> can be an operating system component, or other like shared component that can be equally utilized by different applications or different processes executing on the exemplary computing device <NUM>.

As indicated previously, the natural language decoding component <NUM> can process natural language input, such as the exemplary natural language input <NUM>, based on an existing natural language model, such as the exemplary pre-trained language model <NUM>, which can form a basis by which the natural language input <NUM> is compared with examples, such as the examples <NUM>. More specifically, statistical analysis, or other like forms of comparison, can be utilized by the natural language decoding component <NUM> to evaluate the statistical equivalence, closeness in a multidimensional data space, or other like measure of similarity correspondence between a received natural language input, such as the exemplary natural language input <NUM>, and the examples <NUM>, in view of the pre-trained language model <NUM>. By way of a simple example, a natural language input of the form "how many people are in New York?" can be determined, by the natural language decoding component <NUM>, to be more similar to, and, thus, correspond more to, an example of the form "what is the population of New York?" than to an example of the form "how many people are in Rochester?", based upon the pre-trained language model <NUM>. In such a simple example, the pre-trained language model <NUM> can comprise information that the phrase "how many people are in" carries an equivalent meaning to the phrase "what is the population of. " As such, even though the natural language input "how many people are in New York?" has more words that are similar to the example "how many people are in Rochester?" than to the example "what is the population of New York?", the natural language decoding component <NUM> can utilize the above identified semantical equivalence, provided by the pre-trained language model <NUM>, to determine that the natural language input "how many people are in New York?" is more similar to, and, thus, correspond more to, the example "what is the population of New York?" than to the example "how many people are in Rochester?" Based upon such a determination, the natural language decoding component <NUM> can provide an unambiguous reference <NUM> to the function, or aspect, of the application program functionality <NUM> that is associated with the example "what is the population of New York?" For example, the natural language decoding component <NUM> can provide an unambiguous reference <NUM> to an aspect of the application program functionality <NUM> that can cause the device <NUM> to display the information "<NUM>,<NUM>,<NUM>" as a response to the natural language user input of "how many people are in New York?".

As can be seen from the above description, the ability of the natural language decoding component <NUM> to accurately correlate natural language user input to a specific function or aspect of the application program functionality is dependent upon the degree to which the examples <NUM>, provided for each function or aspect of the application program functionality that is to be invocable utilizing natural language user input, match up with the knowledge or structure of the pre-trained language model <NUM>. As such, examples formed without reference to the pre-trained language model <NUM> can introduce confusion or ambiguity in the decoding performed by the natural language decoding component <NUM>. For example, if multiple examples are provided for a single function in order to provide alternative phrasing by which such a function might be invoked by a natural language input, but such multiple examples use alternative phrasing that is in conflict with alternative phrasing deemed by the pre-trained language model <NUM> to convey the same concept, then such multiple examples can potentially result in confusion and even an inability of the natural language decoding component <NUM> to accurately generate an unambiguous reference to that function for certain types of natural language user input that the user <NUM> would utilize to invoke that function. As a result, the user <NUM> may experience dissatisfaction or annoyance at the computing device <NUM> not correctly invoking the function the user intended with the natural language user input.

Moreover, as indicated, examples from different domains can overlap with one another and cause further ambiguity and/or sub-optimality. Such can be especially significant in instances where application programs are constructed from reusable components, which may, themselves, be developed independently of the application programs within which they are utilized. As such, one component can provide its own examples which can conflict with the examples provided by another, different component, and those examples can both be in conflict with the examples provided by an application developer of the overall application program that utilizes both components. Such overlap, ambiguity and sub-optimality is exacerbated by the use of casual and informal language in examples. In particular, casual and informal language can be imprecise and ambiguous.

To provide examples, such as the examples <NUM>, which can act as references to which the natural language decoding component <NUM> compares natural language user input in order to determine which function, or aspect of the application program functionality <NUM>, should be invoked in response to the user's natural language input, an example generator can be utilized that can be tuned in accordance with the pre-trained language model <NUM> such that the examples generated by such an example generator can optimally be utilized with the pre-trained language model <NUM> and not cause confusion, but rather enable the natural language decoding component <NUM>, utilizing the pre-trained language model <NUM>, to more easily and more accurately correlate natural language user input to a single example, and, thus, to a single function, or aspect of the application program functionality <NUM>. Moreover, such an example generator can avoid ambiguity by both centralizing the generation of examples, thereby avoiding conflict and ambiguity introduced when examples are generated independently and within differing domains, and by utilizing more precise terminology inherent in the labeled knowledge graph which the example generator consumes as input. In such a manner, development of application programs that can accept natural language user input can be facilitated and made more efficient. Additionally, user experience with such application programs can be improved, and user interaction with such application programs can be made more accurate and more efficient.

Turning to <FIG>, the exemplary system <NUM> shown therein illustrates an exemplary example generator component <NUM> that can generate examples <NUM> that can be optimized to work with the pre-trained language model <NUM> being utilized by the natural language decoding component <NUM>. According to one aspect, the example generator <NUM> can generate the examples <NUM> based on a labeled knowledge graph, such as the exemplary labeled knowledge graph <NUM>, which can be provided as input <NUM> to the example generator <NUM>. More specifically, a knowledge graph can represent the information and functionality, of which an application program is comprised, linked together through links that indicate correspondences between such information and functionality. Knowledge graphs represent a common and often used paradigm by which application program developers organize, arrange, or otherwise delineate the features, information and functionality of their application programs. The relationship between a knowledge graph, such as the exemplary knowledge graph <NUM>, and application program functionality, such as the exemplary application program functionality <NUM>, is visually represented by the arrow <NUM>.

The example generator <NUM> can generate the natural language input examples <NUM> from a labeled knowledge graph, such as the exemplary labeled knowledge graph <NUM>. In a labeled knowledge graph, the nodes and links between the nodes can be labeled with one or more natural language words or phrases. As indicated previously, application program developers often rely on knowledge graphs to delineate the functionality of their application programs. Accordingly, such application program developers are optimally qualified to apply labels, utilizing natural language words or phrases, to the nodes and links of such knowledge graphs. Indeed, often the natural language words or phrases utilized to label the nodes and links of such knowledge graphs will be the natural language words or phrases that the application program developers were already using to nominate aspects of the knowledge graph. As such, it is expected that the labeling of a knowledge graph, such as to generate the labeled a knowledge graph <NUM>, can be more efficient for application program developers to perform than the direct generation of the examples <NUM>.

The utilization of the more precise labels, terms and keywords from the formal model of knowledge that are captured in the graph structure is a meaningful mechanism by which the problems of ambiguity, overlap and sub-optimality identified above are solved by the mechanisms described below. More specifically, the labels of a knowledge graph tend to be significantly less ambiguous due to the reason and purpose for their utilization in the first place. For example, a knowledge graph conveying geometric concepts would be labeled utilizing terms such as "surface area" and "volume", rather than imprecise terms such as "size". The utilization of such more precise terminology in the labeling enables the example generator to generate more distinct, and less conflicting, examples, such as in the manner detailed below. Additionally, the Uniform Resource Indicator (URI) structures utilized within knowledge graphs are, by definition, "unique". The mechanisms described below utilize such uniqueness, and the formalized composition provided by a labeled knowledge graph and apply it to natural language input modeling to solve the problems identified above.

According to one aspect, the example generator <NUM> can generate the natural language input examples <NUM> from the labels applied to nodes and links of the labeled knowledge graph <NUM>. As indicated previously, the generation of the natural language input examples <NUM> can be in coordination with knowledge or functionality that is already part of the pre-trained language model <NUM>. Such coordination is visually represented in <FIG> by the arrow <NUM>. For example, a pre-trained language model <NUM>, based on large quantities of natural language that was provided as input to machine learning processes that generated the pre-trained language model <NUM>, can comprise an extensive database or other like knowledge of colloquialisms, phrasings and terminology that express equivalent concepts. Accordingly, the coordination represented by the arrow <NUM> can entail the example generator <NUM> minimizing filler words or other transition phrasing when generating the natural language examples <NUM>. In response, the example generator <NUM> can generate the natural language examples <NUM> by concatenating two or more natural language labels from the labeled knowledge graph <NUM> with a minimum of additional wording. For example, the example generator <NUM> can generate the natural language examples <NUM> by concatenating the words or phrases from two or more natural language labels with no intermediate words or phrases. Such a concatenation can include appending a word or phrase used to label a link to the end of a word or phrase used to label a preceding node, prepending the word or phrase used to label the link to the beginning of the word or phrase used to label the preceding node, or other like concatenations.

Each of the natural language examples <NUM> generated by the example generator <NUM> can correspond to a specific aspect of the application program functionality <NUM>. In such a manner, when a subsequent natural language user input is obtained, and determined, by the natural language decoding component <NUM>, to be most similar to a particular one of the natural language examples <NUM>, the corresponding function or aspect of the application program functionality can be identified and, ultimately, invoked in order to respond to the user's natural language input. Such a correspondence between the given natural language example, of the natural language examples <NUM>, and a specific aspect of the application program functionality <NUM>, can be explicitly indicated, such as through an identifier or other like explicit identification of a unique aspect of the application program functionality <NUM>. Alternatively, such a correspondence can be implicitly indicated, such as through an extrapolation based upon the labels of the labeled knowledge graph <NUM>, or other like implicit indicator. The correspondence between a specific one of the natural language examples <NUM>, and a specific aspect of the application program functionality is visually illustrated in <FIG> by the arrow <NUM>.

Turning to <FIG>, the exemplary system <NUM> shown therein depicts an illustratively simple exemplary labeled knowledge graph that can provide context for descriptions of the operation of the example generator detailed above. As can be seen, the illustratively simple knowledge graph shown in <FIG> can be a portion of a knowledge graph of an application program providing information to users, such as, in the specific example illustrated, information regarding the population of various cities. Accordingly, an exemplary node <NUM> can represent, within the context of the functionality of the application program, the concept of New York City. Such a node <NUM> can be connected through a link <NUM> to a subsequent node <NUM>, representing the value eight million. The link <NUM> can signify the relationship between New York City and the quantity eight million, the relationship being that the population of New York City is eight million people. Analogously, a link <NUM>, representing the concept of population, can connect the node <NUM>, representing the concept of the city of Buffalo, to the node <NUM>, representing the quantity <NUM>,<NUM> and a link <NUM>, also representing the concept of population, can connect the node <NUM>, representing the concept of the city of Rochester, to the node <NUM>, representing the quantity <NUM>,<NUM>.

A further node, such as the exemplary node <NUM>, can represent the concept of the state of New York. Each of the links <NUM>, <NUM> and <NUM> can then, in turn, signify the relationship between the concept of the state of New York, as represented by the node <NUM>, and the concepts of the cities of New York City, represented by the node <NUM>, of Buffalo, represented by the node <NUM>, and of Rochester, represented by the node <NUM>, respectively. The links <NUM>, <NUM> and <NUM> can, thereby, represent the concept of a city relationship, namely between New York State and the enumerated cities. Typically, the knowledge graph can be expressed in the form of triples, with each triple comprising a source node, a destination node, and a link between them. For example, the exemplary triple <NUM> comprises a source node <NUM>, representing the concept of the city of Rochester, a destination node <NUM>, representing a value of <NUM>,<NUM>, and a link between them, namely the link <NUM>, representing the population relationship between the city of Rochester and the value of <NUM>,<NUM>. Accordingly, the exemplary triple <NUM> can represent the information that the population of the city of Rochester is <NUM>,<NUM>.

As indicated previously, a knowledge graph, such as the exemplary knowledge graph shown in <FIG>, can have its nodes and edges labeled with natural language labels identifying the nodes and edges. For example, the link <NUM>, representing the relationship between the nodes <NUM> and <NUM>, can be labeled with the natural language word "population", since the relationship indicated by the link can be a population relationship. According to one aspect, such a label can itself be a node in the knowledge graph, such as the exemplary node <NUM>, and can be linked to the link <NUM> via a separate link <NUM> representing that the node <NUM> is a label of the link <NUM>. In an analogous manner, the link <NUM> can also be labeled "population", as represented by the node <NUM> and the connecting link <NUM>, and the link <NUM> can, likewise, also be labeled "population", as represented by the node <NUM>, and the connecting link <NUM>. Links <NUM>, <NUM> and <NUM> can be labeled with the natural language word "city", since the relationship indicated by those links can be a city relationship. Accordingly, the link <NUM> can be labeled "city", as represented by the node <NUM> and the connecting link <NUM>, the link <NUM> can be labeled "city", as represented by the node <NUM> and the connecting link <NUM>, and the link <NUM> can be labeled "city", as represented by the node <NUM> and the connecting link <NUM>. The node <NUM>, representing the concept of the city of Rochester, can be labeled with the natural language name "Rochester", as illustrated by the node <NUM> and the connecting link <NUM>. Similarly, the node <NUM>, representing the concept of the city of Buffalo, can be labeled with the natural language name "Buffalo", as illustrated by the node <NUM> and the connecting link <NUM>.

In some instances, a natural language label can include both a preferred, or primary, label, and one or more alternative, or secondary, labels. Such labels can be alternative names, such as nicknames, alternative phrasing, or other like alternatives. For example, the label applied to the node <NUM>, representing the concept of the city of New York City, can include both the formal name "New York City" as well as one or more alternative names, or nicknames, such as the "Big Apple" or "Gotham". As such, the node <NUM>, representing the labels applied to the node <NUM>, as represented by the labeling link <NUM>, can comprise the natural language words "New York City", together with an implicit or explicit indication that such natural language words represent a preferred, or primary, label, along with the natural language words "Big Apple" and "Gotham", together with an implicit or explicit indication that such natural language words represent an alternative, or secondary, label for the node <NUM>. In a similar manner, the node <NUM>, representing the concept of the state of New York State, can be labeled with the natural language words "New York State" as a preferred label, and the natural language words "Empire State" as an alternative label, such as illustrated by the node <NUM>, which can be connected to the node <NUM> via the labeling link <NUM>.

Utilizing a labeled knowledge graph, such as the exemplary labeled knowledge graph <NUM> shown in <FIG>, a natural language input example generator can generate examples of natural language inputs that are to correspond to specific aspects of the functionality offered by an application program represented by the labeled knowledge graph <NUM>. As indicated previously, a labeled knowledge graph can be parsed into discrete triples, such as the exemplary triple <NUM>. According to one aspect, a natural language input example can be generated by concatenating the labels applied to a source node of a triple and to a link of that triple. Thus, for example, a natural language input example of the form "Rochester population" can be generated corresponding to the triple <NUM>. As can be seen, the natural language input example "Rochester population" can be generated by concatenating the natural language words of the label applied to the source node <NUM> of the triple <NUM>, namely the natural language word "Rochester", as contained in the label node <NUM> and the natural language words of the label applied to the link <NUM> of the triple <NUM>, namely the natural language word "population", as contained in the label node <NUM>. Such a concatenation can be the appending of the natural language word or phrase utilized to label the link of a triple onto the end of the natural language word or phrase utilized to label the source node of the same triple. Alternatively, or in addition, such a concatenation can be the prepending of the natural language word or phrase utilized to label the link of a triple onto the beginning of the natural language word or phrase utilized to label the source node of the same triple. Thus, for example, a natural language input example of the form "population Rochester" could also be generated for the exemplary triple <NUM>.

As indicated previously, the generation of natural language input examples can be tuned based on the knowledge or construct of the pre-existing natural language model. For example, if the pre-existing natural language model has substantial familiarity with colloquialisms, filler words, and other like natural language constructs relevant to the concepts embodied by the functionality offered by an application program, simpler examples that avoid alternatives of such colloquialisms filler words and other like natural language constructs can avoid duplication and/or confusion. Thus, for example, to generate an optimized natural language input example corresponding to application program functionality for returning the population of the city of Rochester, a simple example form from the concatenation of labels, such as the "Rochester population" and/or "population Rochester" natural language input examples whose generation was detailed above, would be optimal.

According to one aspect, a correlation between generated natural language input examples and the corresponding functionality can be explicitly provided. In such an instance, functionality associated with the destination node of the triple, such as the exemplary destination node <NUM> of the exemplary triple <NUM>, can be identified or otherwise correlated to the generated natural language input examples. In the example shown in <FIG>, and detailed herein, the functionality associated with the node <NUM> can include the presentation, to a user, of the value <NUM>,<NUM>. Such functionality can be associated with a specific subroutine that can have a unique identifier; in which case such a unique identifier can be explicitly provided with the generated natural language input examples. Implicit identifications can also be utilized, which can be based on the already existing labels, and their association with the generated natural language input examples. Ultimately, the natural language input examples can be implicitly or explicitly linked to the triple <NUM> and/or the functionality associated with the node <NUM> such that, if a particular natural language user input is determined to be closest to the generated natural language input examples associated with the triple <NUM> and/or the functionality associated with the node <NUM>, such functionality can be invoked to respond to that particular natural language user input. Thus, for example, if a user were to ask "how many people live in Rochester?", then the determination that such a natural language input is closest to the generated natural language input example "Rochester population", can result in an invocation of the functionality associated with the node <NUM>, which can result in the application program presenting the user with the numeric value "<NUM>,<NUM>" in response to the user's natural language input. From the user's perspective, the natural language question "how many people live in Rochester?" was answered with the output "<NUM>,<NUM>.

As indicated previously, labels assigned to nodes or links can include both preferred and alternative natural language labels. In such an instance, multiple natural language input examples can be generated to account for the alternative natural language labels. For example, as detailed above, the node <NUM> can include alternative natural language labels for the node <NUM>, representing the concept of New York City, including the alternative natural language labels "Big Apple" and "Gotham. " Accordingly, in addition to generating a natural language input example of the form "New York City population" for the triple comprising the source node <NUM>, the link <NUM> and the destination node <NUM>, the example generator can also generate, for that same triple, another natural language input example of the form "Big Apple population" and/or another natural language input example of the form "Gotham population. " As can be seen, such multiple examples can represent concatenations of the preferred label of the node <NUM> with the preferred label of the link <NUM> as well as concatenations of one or more of the alternative labels of the node <NUM> with the preferred label of the link <NUM>. Still further natural language examples can be generated by concatenating the labels of the node <NUM> and the link <NUM> by prepending the label of the link <NUM> onto the front of the preferred and alternative labels of the node <NUM>. Thus, further natural language input examples can be generated that are, in the present example, of the form "population Big Apple" and/or "population Gotham.

In instances where multiple nodes and/or links, whose labels are being concatenated to form the natural language input examples, each comprise both preferred and alternative natural language labels, the natural language input example generator can concatenate specific ones, or all, of the permutations of the preferred and alternative natural language labels. In shorthand, if one node and/or link comprised a preferred label "A" and alternative labels "B" and "C", and another node and/or link comprised a preferred label "X" and an alternative label "Y", then the natural language input example generator could generate a preferred natural language input example of the form "AX" and/or "XA" and could further generate one or more alternative natural language input examples of the form "AY", "BX", "BY", "CX, "CY", "YA", "XB", "YB", "XC" and/or "YC. " The selection of which one or more such alternative natural language input examples to generate can be part of the tuning process such as was represented by the arrow <NUM> in <FIG> and described in detail above.

While the exemplary labeled knowledge graph <NUM> shown in <FIG> illustrates an exemplary operation of a natural language input example generator within the context of an information presentation application program, such as an application program that can provide geographic information, equivalent mechanisms can be utilized to generate natural language input examples within the context of application programs that control external devices, or otherwise perform like actions. For example, the exemplary labeled knowledge graph <NUM> shown in <FIG> depicts an illustratively simple labeled knowledge graph for an application program that could control external devices, such as a home automation application program, which could be an embedded program on a home automation computing device, or can be a standalone application program executing on a general-purpose computing device.

As can be seen, the exemplary labeled knowledge graph <NUM> comprises a node <NUM> that can represent the concept of a specific room, a node <NUM> that can represent the concept of a light-producing device, and a link <NUM> between the node <NUM> and the node <NUM>, with the link representing the concept of a switch, such that the node <NUM> represents a type of switch in the room represented by the node <NUM>. In a similar manner, the node <NUM> can represent the concept of a Heating Ventilation Air Conditioning (HVAC) device, and the link <NUM> between the node <NUM> and <NUM> can represent the concept of a thermostat, such that the node <NUM> represents a type of thermostat in the room represented by the node <NUM>. One or more of the nodes <NUM>, <NUM> and <NUM> can have labels applied to them, as can the links <NUM> and <NUM>. For example, the node <NUM> can have a label of the form "Family Room" applied to it, as illustrated by the label node <NUM> and the labeling link <NUM> which associates the label of node <NUM> to the concept of a room represented by the node <NUM>. Similarly, the node <NUM> can have a label of the form "Reading Lamp" applied to it, as illustrated by the label node <NUM> and the corresponding labeling link <NUM>, and the node <NUM> can have a label of the form "Electric Fireplace" applied to it, as illustrated by the label node <NUM> and the corresponding labeling link <NUM>.

Still further nodes, such as the exemplary nodes <NUM>, <NUM>, <NUM> and <NUM>, can represent the functionality of instructing an external device. For example, the node <NUM> can represent the concept of sending an "on" command to an external device, and the node <NUM> can represent the concept of sending an "off" command to such a device. Similarly, as another example, the node <NUM> can represent the concept of sending a "heat increase" commands to an external device, and the node <NUM> can represent the concept of sending a "heat decrease" command to that same device. Nodes <NUM> and <NUM> can be linked to a particular type of device, namely the concept of an HVAC device represented by the node <NUM>. More specifically, the link <NUM> can connect the concept of a "heat increase" command, represented by the node <NUM>, to the concept of an HVAC device, represented by the node <NUM> and, similarly, the link <NUM> can connect the concept of a "heat decrease" command, represented by the node <NUM>, to the same concept of an HVAC device, represented by the node <NUM>. In an analogous manner, nodes <NUM> and <NUM> can also be linked to a particular type of device, namely the concept of a light-producing device represented by the node <NUM>. As such, the link <NUM> can connect the concept of an "on" command, represented by the node <NUM>, to the concept of the light-producing device, represented by the node <NUM> and, similarly, the link <NUM> can connect the concept of an "of" command, represented by the node <NUM>, to the concept of the light-producing device represented by the node <NUM>.

In the exemplary labeled knowledge graph <NUM>, each of the links <NUM>, <NUM>, <NUM> and <NUM> can also be labeled. For example, the link <NUM> can have a label of the form "turn on", as represented by the label node <NUM>, which is applied to, or otherwise associated with, the link <NUM> via the labeling link <NUM>, and, similarly, the link <NUM> can have a label of the form "turn off', as represented by the label node <NUM>, which is applied to the link <NUM> via the labeling link <NUM>. In an analogous manner the link <NUM> can have a label of the form "turn up", as represented by the label node <NUM>, which is applied to the link <NUM> via the labeling link <NUM>, and, similarly, the link <NUM> can have a label of the form "turn down", as represented by the label node <NUM>, which is applied to the link <NUM> via the labeling link <NUM>. As detailed above, certain labels can include both preferred labels and alternative labels. Thus, for example, the labels applied to the link <NUM> can include a preferred label of the form "turn up", and an alternative label of the form "raise the temperature", and, likewise, the labels applied to the link <NUM> can include a preferred label of the form "turn down", and an alternative label of the form "lower the temperature.

As indicated previously, the generation of natural language input examples can entail the concatenation of labels from a source node of a triple and from a link of the triple. Thus, for example, with reference to the exemplary triple <NUM>, comprising a source node <NUM>, a destination node <NUM>, and a link <NUM> connecting the source node <NUM> the destination node <NUM>, a natural language input example for the triple <NUM> can be of the form "Reading Lamp turn on. " Another example can be generated by the concatenation of the same labels, except in reverse order, such that, for the triple <NUM>, a natural language input example of the form "turn on Reading Lamp" can also be generated.

According to one aspect, the generation of natural language input examples can comprise the further concatenation of labels of preceding elements in the knowledge graph to which the triple, for which the natural language input examples are being generated, is linked. Thus, for example, for the exemplary triple <NUM>, shown in <FIG>, a natural language input example can be generated by concatenating the labels of one or more of the preceding node <NUM> and preceding link <NUM>, including, for example, natural language input examples of the form "Family Room Reading Lamp turn off", "turn off Family Room Reading Lamp", "Family Room turn off Reading Lamp", and/or other like permutations. Again, utilizing shorthand for ease of explanation, a triple having a source node with a label "A", a link with a label "B", and preceding elements in the knowledge graph having a label "X", can result in natural language input examples of the form "ABX", "AXB", "XAB", "BXA", "BAX" and/or "XBA". The selection of which one or more such alternative natural language input examples to generate can be part of the tuning process such as was represented by the arrow <NUM> in <FIG> and described in detail above. Moreover, to the extent that one or more of the labels include both preferred and alternative labels, alternative combinations and permutations of the labels, as detailed above, can be the basis of generating still further natural language input examples corresponding to a triple, such as the exemplary triple <NUM>.

Turning to <FIG>, the flow diagram <NUM> shown therein illustrates an exemplary series of steps by which one or more natural language input examples can be generated in order to enable a natural language processing component to utilize pre-existing natural language models as a basis for comparing a user's natural language input to the provided examples, and, thereby, enable user natural language input to invoke and access corresponding functionality of an application program. Initially, as illustrated in <FIG>, the exemplary flow diagram <NUM> can commence with the obtaining or receiving of a labeled knowledge graph at step <NUM>. Subsequently, at step <NUM> a triple within the knowledge graph of step <NUM> can be identified. At step <NUM> a natural language label of a source node of the triple of step <NUM> can be obtained and, at step <NUM>, a natural language label of the link of the triple of step <NUM> can be obtained. Subsequently at step <NUM>, a natural language input example can be generated by concatenating the obtained labels of steps <NUM> and <NUM>. As indicated previously, such concatenation, at step <NUM>, can entail the appending of the natural language label of the link, obtained at step <NUM>, onto the end of the natural language label of the source node, obtained at step <NUM>. Alternatively, or in addition, such concatenation, at step <NUM>, can entail the prepending of the natural language label of the link, obtained at step <NUM>, to the beginning of the natural language label of the source node, obtained at step <NUM>. As also indicated previously, the generation of natural language input examples can further comprise the concatenation of other nodes and links, such as, specifically, the nodes and links that precede the triple identified at step <NUM>. In such an instance, in addition to steps <NUM> and <NUM>, other steps can be performed to obtain the natural language labels of the preceding nodes and links.

At step <NUM>, after the generation of natural language input examples at step <NUM>, a determination can be made as to whether any of the labels, obtained at steps <NUM> and <NUM>, or any other like steps, contain alternative labels in addition to preferred labels. If no such alternative labels exist, processing can proceed to step <NUM>. However, if such alternative labels are identified at step <NUM>, then processing can proceed to step <NUM> and additional natural language input examples can be generated from permutations and combinations of preferred and alternative labels, such as in the manner detailed above. At step <NUM>, then, a correspondence between the generated natural language input examples and the triple identified at step <NUM> can be assigned. As indicated previously, such a correspondence can be explicitly indicated, such as with identifiers of the functionality of the destination node of the triple identified at step <NUM>, identifiers of the triple itself, identifiers of relevant function calls, subroutines, or other like explicit identifications. Alternatively, as also indicated previously, such a correspondence can be implicitly indicated such as by the implicit relationship between the natural language input examples generated and the labels of the corresponding nodes.

At step <NUM>, the generated natural language input examples, and any explicitly indicated correspondence with aspects of the labeled knowledge graph and/or functionality of the corresponding application program, can be transmitted, or otherwise provided, such as to a natural language processing component. At step <NUM>, processing can revert back to step <NUM> to identify a new triple, and then proceed as before, or, alternatively, if all appropriate, delineated, or otherwise indicated, triples have already been identified and processed, the relevant processing can end at step <NUM>.

Turning to <FIG>, the flow diagram <NUM> shown therein illustrates an exemplary series of steps by which the natural language input examples generated and provided by the mechanisms detailed above, including, for example, in the exemplary flow diagram <NUM>, can be utilized to facilitate application programs to accept and respond to natural language user input. More specifically, and as illustrated in <FIG>, initially, at step <NUM>, a natural language user input can be received, or otherwise obtained. At step <NUM>, a comparison can be made between the natural language input of step <NUM> and the aforementioned natural language input examples. As indicated previously, the comparison, at step <NUM>, can be based on an existing natural language model, which can provide the basis by which determinations are made as to the closeness of the natural language input to one or more of the natural language input examples. As also indicated previously, such a comparison can be performed utilizing a number of statistical, analytical, machine learning, and other like comparative mechanisms. At step <NUM>, the results of the comparison of step <NUM> can determine a natural language input example that is deemed to be most similar, or otherwise "closest" to the natural language input of step <NUM>. Subsequently, at step <NUM>, an identification can be made of a corresponding triple, or other like implicit or explicit correspondence between the natural language input example of step <NUM> and an application program's labeled knowledge graph, functionality, or the like. Based upon such a correspondence, identified at step <NUM>, the identified function of the application program can be invoked, or otherwise performed, at step <NUM>. In such a manner, the application program can be responsive, through the performance of the function at step <NUM>, to the user's natural language input, received at step <NUM>. The relevant processing can then end at step <NUM>.

Turning to <FIG>, an exemplary computing device <NUM> is illustrated which can perform some or all of the mechanisms and actions described above. The exemplary computing device <NUM> includes, but is not limited to, one or more central processing units (CPUs) <NUM>, a system memory <NUM>, and a system bus <NUM> that couples various system components including the system memory to the processing unit <NUM>. The system bus <NUM> may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The computing device <NUM> can optionally include graphics hardware, including, but not limited to, a graphics hardware interface <NUM> and a display device <NUM>, which includes display devices capable of receiving touch-based user input, such as a touch-sensitive, or multi-touch capable, display device. Depending on the specific physical implementation, one or more of the CPUs <NUM>, the system memory <NUM> and other components of the computing device <NUM> can be physically co-located, such as on a single chip. In such a case, some or all of the system bus <NUM> can be nothing more than silicon pathways within a single chip structure and its illustration in <FIG> can be nothing more than notational convenience for the purpose of illustration.

The computing device <NUM> also typically includes computer readable media, which includes any available media that can be accessed by computing device <NUM> and includes both volatile and nonvolatile media and removable and non-removable media. Computer storage media includes media implemented in any method or technology for storage of content such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired content and which can be accessed by the computing device <NUM>. Computer storage media, however, does not include communication media. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any content delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.

A basic input/output system <NUM> (BIOS), containing the basic routines that help to transfer content between elements within computing device <NUM>, such as during start-up, is typically stored in ROM <NUM>. By way of example, and not limitation, <FIG> illustrates operating system <NUM>, other program modules <NUM>, and program data <NUM>.

The computing device <NUM> may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only, <FIG> illustrates a hard disk drive <NUM> that reads from or writes to non-removable, nonvolatile magnetic media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used with the exemplary computing device include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and other computer storage media as defined and delineated above. The hard disk drive <NUM> is typically connected to the system bus <NUM> through a non-volatile memory interface such as interface <NUM>.

The drives and their associated computer storage media discussed above and illustrated in <FIG>, provide storage of computer readable instructions, data structures, program modules and other data for the computing device <NUM>. In <FIG>, for example, hard disk drive <NUM> is illustrated as storing operating system <NUM>, other program modules <NUM>, and program data <NUM>. Note that these components can either be the same as or different from operating system <NUM>, other program modules <NUM> and program data <NUM>. Operating system <NUM>, other program modules <NUM> and program data <NUM> are given different numbers hereto illustrate that, at a minimum, they are different copies.

The computing device <NUM> may operate in a networked environment using logical connections to one or more remote computers. The computing device <NUM> is illustrated as being connected to the general network connection <NUM> (to a network <NUM>) through a network interface or adapter <NUM>, which is, in turn, connected to the system bus <NUM>. In a networked environment, program modules depicted relative to the computing device <NUM>, or portions or peripherals thereof, may be stored in the memory of one or more other computing devices that are communicatively coupled to the computing device <NUM> through the general network connection <NUM>. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between computing devices may be used.

Although described as a single physical device, the exemplary computing device <NUM> can be a virtual computing device, in which case the functionality of the above-described physical components, such as the CPU <NUM>, the system memory <NUM>, the network interface <NUM>, and other like components can be provided by computer-executable instructions. Such computer-executable instructions can execute on a single physical computing device, or can be distributed across multiple physical computing devices, including being distributed across multiple physical computing devices in a dynamic manner such that the specific, physical computing devices hosting such computer-executable instructions can dynamically change over time depending upon need and availability. In the situation where the exemplary computing device <NUM> is a virtualized device, the underlying physical computing devices hosting such a virtualized computing device can, themselves, comprise physical components analogous to those described above, and operating in a like manner. Furthermore, virtual computing devices can be utilized in multiple layers with one virtual computing device executing within the construct of another virtual computing device. The term "computing device", therefore, as utilized herein, means either a physical computing device or a virtualized computing environment, including a virtual computing device, within which computer-executable instructions can be executed in a manner consistent with their execution by a physical computing device. Similarly, terms referring to physical components of the computing device, as utilized herein, mean either those physical components or virtualizations thereof performing the same or equivalent functions.

The descriptions above include, as a first embodiment a method of providing user access to application program functionality through user natural language input, the method comprising: generating natural language input examples from a labeled knowledge graph of an application program providing the application program functionality, the generating comprising: identifying a first triple within the labeled knowledge graph, the first triple comprising a first source node, a first link, and a first destination node, wherein: the first destination node is a node in the labeled knowledge graph corresponding to a first function of the application program functionality; the first link connects the first source node to the first destination node and delineates a programmatic relationship within the application program between the first source node and the first destination node; the first source node is labeled with a first label comprising a first preferred natural language word or phrase; and the first link is labeled with a second label comprising a second preferred natural language word or phrase; and generating a first natural language input example by concatenating the first preferred natural language word or phrase and the second preferred natural language word or phrase; providing the generated natural language input examples including the first natural input example to a natural language processor; receiving a first user natural language input; determining that the first user natural language input corresponds more to the first natural language input example than to any other of the generated natural language input examples, wherein the natural language processor utilizes an existing natural language model to determine similarities between the first user language input and the provided natural language input examples including the first natural input example; identifying the first triple based on the determination of the first natural language input example; and providing an identification of the first triple to the application program, the application program responding to the first user natural language input by performing the first function based on receiving the provided identification of the first triple, the application program thereby providing the user access to the application program functionality through the natural language input.

A second embodiment is the method of the first embodiment, wherein the generating the natural language input examples further comprises: traversing the labeled knowledge graph to identify additional triples within the labeled knowledge graph; and repeating the generating with the labels of source node and links of the identified additional triples.

A third embodiment is the method of the first embodiment, wherein the generating the first natural language input example comprises prepending the first preferred natural language word or phrase directly to the second preferred natural language word or phrase.

A fourth embodiment is the method of the first embodiment, wherein the generating the first natural language input example comprises appending the first preferred natural language word or phrase directly to the second preferred natural language word or phrase.

A fifth embodiment is the method of the first embodiment, wherein the generating the first natural language input example comprises: concatenating a third preferred natural language word or phrase with the concatenation of the first preferred natural language word or phrase and the second preferred natural language word or phrase; wherein: a second triple within the labeled knowledge graph comprises a second source node, a second link, and a second destination node; the first source node of the first triple is the second destination node of the second triple; and either the second source node or the second link is labeled with a third label comprising the third preferred natural language word or phrase.

A sixth embodiment is the method of the first embodiment, wherein the generating the natural language input examples further comprises: generating a second natural language input example by concatenating a first alternative natural language word or phrase and the second preferred natural language word or phrase; wherein the first label further comprises the first alternative natural language word or phrase as an alternative to the first preferred natural language word or phrase.

A seventh embodiment is the method of the sixth embodiment, wherein the generating the natural language input examples further comprises: generating a third natural language input example by concatenating either: (<NUM>) the first preferred natural language word or phrase and a second alternative natural language word or phrase or (<NUM>) the first alternative natural language word or phrase and the second alternative natural language word or phrase; wherein the second label further comprises the second alternative natural language word or phrase as an alternative to the second preferred natural language word or phrase.

An eighth embodiment is the method of the sixth embodiment, further comprising: associating the first natural language input example with a first priority; and associating the second natural language input example with a second priority that is lower than the first priority.

A ninth embodiment is the method of the first embodiment, wherein the receiving, the determining, the identifying, and the providing the identification are performed during runtime of the application program; and wherein further the generating the natural language input examples is performed prior to the runtime of the application program.

A tenth embodiment is the method of the first embodiment, wherein the first user natural language input is a question; and wherein further the first function comprises a first information that is responsive to the question.

An eleventh embodiment is a method of priming an existing natural language model to provide user access to application program functionality through user natural language input, the method comprising: obtaining a labeled knowledge graph of the application program, the labeled knowledge graph comprising: a first destination node corresponding to a first function of the application program functionality; a first link connecting a first source node to the first destination node, the first link delineating a programmatic relationship within the application program between the first source node and the first destination node; a first label applied to the first source node, the first label comprising a first preferred natural language word or phrase; and a second label applied to the first link, the second label comprising a second preferred natural language word or phrase; generating natural language input examples (<NUM>) from the labeled knowledge graph, the generating comprising: identifying a first triple within the labeled knowledge graph, the first triple comprising the first source node, the first link, and the first destination node; generating a first natural language input example by concatenating the first preferred natural language word or phrase and the second preferred natural language word or phrase; and providing the generated natural language input examples including the first natural language input example to a natural language processor, the user access to the application program functionality through the user natural language input being provided, at least in part, by the natural language processor utilizing the existing natural language model to determine similarities between the user natural language input and the provided natural language input examples, including the first natural language input example.

A twelfth embodiment is the method of the eleventh embodiment, further comprising: traversing the labeled knowledge graph to identify additional triples within the labeled knowledge graph; and repeating the generating and the providing for the identified additional triples.

A thirteenth embodiment is the method of the eleventh embodiment, wherein the generating the first natural language input example comprises prepending the first preferred natural language word or phrase directly to the second preferred natural language word or phrase.

A fourteenth embodiment is the method of the eleventh embodiment, wherein the generating the first natural language input example comprises appending the first preferred natural language word or phrase directly to the second preferred natural language word or phrase.

A fifteenth embodiment is the method of the eleventh embodiment, wherein the generating the first natural language input example comprises: concatenating a third preferred natural language word or phrase with the concatenation of the first preferred natural language word or phrase and the second preferred natural language word or phrase; wherein: the labeled knowledge graph further comprises a second triple, the second triple comprising: a second source node, a second link, and a second destination node; the first source node of the first triple is the second destination node of the second triple; and either the second source node or the second link is labeled with a third label comprising the third preferred natural language word or phrase.

A sixteenth embodiment the method of the eleventh embodiment, further comprising: generating a second natural language input example from the concatenation of the first and second labels, the second natural language input example comprising the second preferred natural language word or phrase appended to a first alternative natural language word or phrase; and providing the second natural language input example to the natural language processor; wherein the first label further comprises the first alternative natural language word or phrase.

A seventeenth embodiment is the method of the sixteenth embodiment, wherein the providing the generated first natural language input example to the natural language processor comprises identifying, to the natural language processor, that the generated first natural language input example is associated with a first priority; and wherein the providing the generated second natural language input example to the natural language processor comprises identifying, to the natural language processor, that the generated second natural language input example is associated with a second priority that is lower than the first priority.

An eighteenth embodiment is a method of the eleventh embodiment, wherein the providing the generated first natural language input example to the natural language processor further comprises identifying the generated first natural language input example as corresponding to the identified first triple.

A nineteenth embodiment is the method of the eleventh embodiment, wherein the first user natural language input is a question; and wherein further the first function comprises a first information that is responsive to the question.

A twentieth embodiment is one or more computer-readable storage media comprising computer-executable instructions, which, when executed by one or more processing units, cause one or more computing devices, in aggregate, to: obtain a labeled knowledge graph of an application program, the labeled knowledge graph comprising: a first destination node corresponding to a first function of the application program; a first link connecting a first source node to the first destination node, the first link delineating a programmatic relationship within the application program between the first source node and the first destination node; a first label applied to the first source node, the first label comprising a first preferred natural language word or phrase; and a second label applied to the first link, the second label comprising a second preferred natural language word or phrase; generating natural language input examples (<NUM>) from the labeled knowledge graph, the generating comprising: identify a first triple within the labeled knowledge graph, the first triple comprising the first source node, the first link, and the first destination node; generate a first natural language input example by concatenating the first preferred natural language word or phrase and the second preferred natural language word or phrase; and provide the generated natural language input examples including the first natural language input example to a natural language processor that utilizes an existing natural language model to determine similarities between user natural language input and natural language input examples, including the generated first natural language input example, thereby enabling user access to application program functionality through user natural language input.

Claim 1:
A method of providing user access to application program functionality (<NUM>) through user natural language input (<NUM>), the method comprising:
generating natural language input examples (<NUM>) from a labeled knowledge graph (<NUM>) of an application program providing the application program functionality, the generating comprising:
identifying (<NUM>) a first triple within the labeled knowledge graph, the first triple (<NUM>, <NUM>) comprising a first source node (<NUM>, <NUM>), a first link (<NUM>, <NUM>), and a first destination node (<NUM>, <NUM>), wherein:
the first destination node is a node in the labeled knowledge graph corresponding to a first function of the application program functionality;
the first link connects the first source node to the first destination node and delineates a programmatic relationship within the application program between the first source node and the first destination node;
the first source node is labeled with a first label comprising a first preferred natural language word or phrase; and
the first link is labeled with a second label comprising a second preferred natural language word or phrase; and
generating (<NUM>) a first natural language input example by concatenating the first preferred natural language word or phrase and the second preferred natural language word or phrase;
providing (<NUM>) the generated natural language input examples including the first natural input example to a natural language processor;
receiving (<NUM>) a first user natural language input;
determining (<NUM>) that the first user natural language input corresponds more to the first natural language input example than to any other of the generated natural language input examples, wherein the natural language processor utilizes an existing natural language model to determine similarities between the first user language input and the provided natural language input examples including the first natural input example;
identifying (<NUM>) the first triple based on the determination of the first natural language input example; and
providing an identification of the first triple to the application program, the application program responding to the first user natural language input by performing (<NUM>) the first function based on receiving the provided identification of the first triple, the application program thereby providing the user access to the application program functionality through the natural language input.