INTENTION IDENTIFICATION IN DIALOGUE SYSTEM

Embodiments of the present disclosure relate to question answering. A computer-implemented method includes determining a plurality of intention candidates of a user from the user's question; determining a set of entities and attributes associated with the set of entities from the plurality of intention candidates; constructing a decision tree from the set of entities and the attributes associated with the set of entities, wherein each node of the decision tree is associated with a respective one of the attributes and represents a respective subset of the plurality of intention candidates, and wherein the respective subset of the plurality of intention candidates are split based on the entities associated with the respective one of the attributes; and generating a question corresponding to a node of the decision tree to determine the user's intention.

BACKGROUND

The present disclosure relates to data processing, and more specifically to conducting a natural language dialogue between a dialogue system and a user of a client computing device.

A dialogue system is a computer system intended to converse with a human. Dialog systems have employed text, speech, graphics, haptics, gestures, and other modes for communication on both the input and output channel. In speech or text-based dialogue systems, such as automated customer service systems, users communicate with the system through spoken utterances or short text messages. Once a user input (spoken utterance or text input) is received, the automated system attempts to process/analyze the user utterance to reduce it to a computer understandable form. Given this unambiguous interpretation of the utterance, the system can perform tasks or produce a response, such as an answer to a question asked by the user. However, some user utterances, text inputs, or portions thereof, may be ambiguous to the dialogue system. For example, the user's question is “How to withdraw my pension?” The user may intend to withdraw pension in location A but the user's intention may be interpreted by mistake as how to withdraw pension in location B. As a result, the correct answer to the user's question is not accurately recalled and recommended.

BRIEF SUMMARY

According to one embodiment of the present disclosure, there is provided a computer-implemented method. A computer-implemented method for conducting a natural language dialogue between a dialogue system and a user of a client computing device, comprising receiving, by the one or more processing units, a user question from a client computing device. Determining, by one or more processing units, a set of entities and attributes associated with the set of entities from the plurality of intention candidates. Constructing, by one or more processing units, a decision tree from the set of entities and the attributes associated with the set of entities, wherein each node of the decision tree is associated with a respective one of the attributes and represents a respective subset of the plurality of intention candidates, and wherein the respective subset of the plurality of intention candidates are split based on the entities associated with the respective one of the attributes. Generating, by one or more processing units, a question corresponding to a node of the decision tree to determine the user's intention. Transmitting, by the one or more processing units, the question to the client computing device. Receiving, by one or more processing units, the user's answer to the question from the client computing device. Determining, by one or more processing units, the user's intention from the decision tree based on the user's answer to the question. Determining and transmitting, by one or more processing units, a response to the user's question based on the user's intention.

According to another embodiment of the present disclosure, there is provided a device. The device comprises a processor; and a memory having instructions stored thereon for execution by the processor, the instructions, when executed by the processor, cause the device to perform acts for conducting a natural language dialogue between a dialogue system and a user of a client computing device, the acts comprising. Receiving a user question from a client computing device. Determining a plurality of intention candidates of a user from the user's question; determining a set of entities and attributes associated with the set of entities from the plurality of intention candidates; constructing a decision tree from the set of entities and the attributes associated with the set of entities, wherein each node of the decision tree is associated with a respective one of the attributes and represents a respective subset of the plurality of intention candidates, and wherein the respective subset of the plurality of intention candidates are split based on the entities associated with the respective one of the attributes; and generating a question corresponding to a node of the decision tree to determine the user's intention. Transmitting the question to the client computing device and receiving the user's answer to the question from the client computing device. Determining the user's intention from the decision tree based on the user's answer to the question. Determining and transmitting a response to the user's question based on the user's intention.

According to another embodiment of the present disclosure, there is provided and comprising machine-executable instructions, the instructions, when executed on a device, causing the device to perform acts for conducting a natural language dialogue between a dialogue system and a user of a client computing device, the acts comprising. Receiving a user question from a client computing device. Determining a plurality of intention candidates of a user from the user's question; determining a set of entities and attributes associated with the set of entities from the plurality of intention candidates; constructing a decision tree from the set of entities and the attributes associated with the set of entities, wherein each node of the decision tree is associated with a respective one of the attributes and represents a respective subset of the plurality of intention candidates, and wherein the respective subset of the plurality of intention candidates are split based on the entities associated with the respective one of the attributes; and generating a question corresponding to a node of the decision tree to determine the user's intention. Transmitting the question to the client computing device and receiving the user's answer to the question from the client computing device. Determining the user's intention from the decision tree based on the user's answer to the question. Determining and transmitting a response to the user's question based on the user's intention.

DETAILED DESCRIPTION

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

The present embodiments provide mechanisms for determining an intention of a user. An intention is a purpose/goal/reason behind an utterance, either textual or audible, exchanged with a dialogue system. The first thing you need to do when you get the utterance from the user, is to understand what the user wants, and this is the intention classification or intention detection. For example, an intention may express whether the user wants the dialogue system to check the value of some variable against some reference amount or retrieve the amount of a variable or explain a variable. As noted above, in dialogue systems, some user speech utterances, text input, or portions thereof, may be ambiguous to the dialogue system and may reduce the accuracy of the generated responses, leading to user frustration. Determining a likely intention of a user, e.g., complaining about a bill, questioning a rate increase, etc., may assist in resolving such ambiguities. Therefore, intention detection in a dialogue system is a decisive step to drive the conversation in a dialog system in the right direction. It would be beneficial to have a mechanism for determining the likely intention of the user in a dialogue system.

With reference now toFIG.4, a method400conducting a natural language dialogue between a dialogue system and a user of a client computing device in accordance with some embodiments of the present disclosure is shown. A user may ask a question in a natural language form through the client computing device to expect an answer to that question from the dialogue system. For example, the user's question may be a speech input in a natural language form or a keyboard input. For example, the user may ask a question such as “How can I withdraw my fund?” The question may be ambiguous with respect to defining the user's intention. For example, the user may intend to withdraw his pension in the city of Beijing, rather than housing fund in the city of Xi'an. But the user's question does not specify the particular city and/or particular kind of fund in the question.

At block402, the method400may determine a plurality of intention candidates for the user from the user's question. For example, one or more models may be configured to determine the user's intention candidates from the user's question. Each of the models may be a machine learning model for natural language processing. The machine learning model may be a text classification model, for example, a neural network. Each model may determine one or more intention candidates from the user's question, where the intention candidates may each have a higher probability than a predefined threshold. The probability may be the probability of being determined as true intention. Alternatively, some of candidates in a group of invention candidates may be of higher probability than the others in the group. Each model may determine a predefined number of intention candidates from the user's question that may have higher probability than the others. For example, each model may generate the top-5 results as the intention candidates.

FIG.5is a schematic diagram illustrating a method500for obtaining an intention set including a plurality of intention candidates in accordance with some embodiments. As shown inFIG.5, the user's question502is provided to each of models504-1,504-2, and504-3(collectively referred to as models504). The models504-1,504-2, and504-3may determine respective intention candidates506-1,506-2, and506-3(collectively referred to as intention candidates506) from the user's question502. For example, the model504-1may determine the intention candidates506-1from the question502. The intention candidates506-1include intentions1to5, which are the top-5 results of the model504-1from the highest probability to the lowest probability. The model504-2may determine the intention candidates506-2from the question502. The intention candidates506-2include intentions3,6,1,5, and9, which are the top-5 results of the model504-2from the highest probability to the lowest probability. The model504-3may determine the intention candidates506-3from the question502. The intention candidates506-3include intentions8,7,3,6, and10, which are the top-5 results of the model504-3from the highest probability to the lowest probability.

As shown inFIG.5, the intention candidates506may be merged or aggregated to obtain a group of intention candidates. The group of intention candidates may be deduplicated to obtain an intention set508. The intention set508may include only one instance for each intention candidate. In this way, the number of intention candidates in the intention set508may be different from the total number of the intention candidates506. For example, the intention candidates506include three instances of Intention3while the intention set508includes only one instance of Intention3. The number of intention candidates in the intention set508is ten (10) while the total number of the intention candidates506is fifteen (15).

The intention candidates506have a relatively high probability but not the maximum probability to ensure a high recall rate. Recall rate, also known as recall, is the fraction of relevant instances that were retrieved and is the measure for how many true positives get predicted out of all the positives in the dataset. In this case, the intention candidates506may be referred to as recall intention candidate. The user's intention may be selected from the intention set508to disambiguate the user's question. By including the recall intention candidates, the intention set508may ensure or guarantee a high recall rate for intention classification, especially when more than one model is used for determining the intention candidates.

FIG.6shows an example of the intention set508in accordance with some embodiments of the present disclosure. For example, Intention1may be “How to withdraw housing fund in Shanghai”, Intention2may be “How to withdraw housing fund in Xi'an”, Intention3may be “How to withdraw housing fund in Beijing”, Intention4may be “How to withdraw pension in Beijing”, Intention5may be “How to withdraw pension in Shanghai”, and so on. For ease of discussion, reference is made to Intentions1to5in the following description.

Returning toFIG.4, at block404, the method400may determine a set of entities and attributes associated with the set of entities from the plurality of intention candidates. An entity is an object in the real world with an independent existence that can be differentiated from other objects. For example, an entity may be a named entity. A named entity is a real-world object that can be denoted with a proper name. For example, the intention candidates may be segmented to obtain a lexicon or vocabulary of the user's intention. The lexicon or vocabulary may include the entities and associated attributes. The entities associated with an attribute may be the values of the attribute. The attributes may also be referred to as tags of the entities. In some embodiments, Named Entity Recognition (NER) may be applied to the intention set508to obtain the vocabulary of the user's intention, as depicted inFIG.7. For example, the entity recognition may use Bidirectional Long Short-Term Memory—Conditional Random Field (BiLSTM-CRF).

FIG.7shows an example vocabulary700in accordance with some embodiments of the present disclosure. The vocabulary700includes a set of entities702-1,702-2,702-3,702-4, and702-5(collectively referred to as entities702) and attributes704-1,704-2,704-3,704-4, and704-5(collectively referred to as attributes704) associated with the set of entities702-1,702-2,702-3,702-4, and702-5, respectively. The entity702-1is “Shanghai” and its attribute704-1is “City”; the entity702-2is “Xi'an” and its attribute704-2is “City”; the entity702-3is “Beijing” and its attribute704-3is “City”; the entity702-4is “housing fund” and its attribute704-4is “funding”; and the entity702-5is “pension” and its attribute704-5is “funding”.

At block406, the method400may construct a decision tree from the set of entities and the attributes associated with the set of entities. Each node of the decision tree is associated with a respective one of the attributes and represents a respective subset of the plurality of intention candidates. For example, a root node of the decision tree may represent all of the plurality of intention candidates. The respective subset of the plurality of intention candidates may be split based on the entities associated with the respective one of the attributes. The intention candidates are examples for training to construct the decision tree. For example, the decision tree may be constructed from the vocabulary700by any training method currently known or to be developed in the future.

In some embodiments, the decision tree may be constructed from top to bottom. The “best” attribute may be first identified or selected to construct the root node of the decision tree. Information gain is an example of the information metric to select an attribute. In this instance, the “best” attribute is the attribute that has the highest information gain. Alternative information metrics may include information gain ratio, Gini index, Chi-square statistic, or the like. Then, the training examples may be partitioned based on the values of the attribute represented by the root node. The values of the attribute may be the entities associated with the attribute. An example method for constructing a decision tree will be described below with reference toFIG.8. The method may be recursively applied to each partition to construct the decision tree. For example, the example algorithms for training the decision tree may be Top-Down Induction of Decision Trees (TDIDD), including ID3, C4.5 or the like.

FIG.8illustrates an example decision tree800in accordance with some embodiments of the present disclosure. The decision tree800includes a root node802and leaf nodes804-1and804-2, where the leaf nodes804-1and804-2are child nodes of the root node802. The root node802represents the attribute of “funding” which has the highest information gain. The root node802may represent all of the intention candidates which may be split based on the entities associated with the attribute of “funding”. The values of the attribute represented by the root node802may include housing fund and pension. In other words, the entities associated with the attribute of “funding” are housing fund and pension. The intention candidates may be partitioned based on the entities associated with the attribute represented by the root node802or the values of the attribute represented by the root node802. For example, the first subset corresponds to housing fund and the second subset corresponds to pension. The “best” attribute for the first subset is selected to be “city” represented by the node804-1, while the “best” attribute for the second subset is also selected to be “city” represented by the node804-2. Further, the respective subset of the plurality of intention candidates may be split based on the entities associated with the respective one of the attributes. For example, the values of the node804-1lead to the intention candidates806-1,806-2and806-3. Specifically, the value “Shanghai” of the node804-1leads to the intention candidate806-1(also shown as Intention1), the value “Xi'an” of the node804-1leads to the intention candidate806-2(also shown as Intention2), and the value “Beijing” of the node804-1leads to the intention candidate806-3(also shown as Intention3). The value “Beijing” of the node804-2leads to the intention candidate806-4(also shown as Intention4) and the value “Shanghai” of the node804-2leads to the intention candidate806-5(also shown as Intention5).

Each of the nodes in the decision tree may correspond to a question for interaction with the user. For example, the root node802may correspond to a question “which funding?” For example, the nodes804may correspond to a question “which city?” The method400may determine the user's intention based on the question(s) for interaction with the user, and then determine the answer to the user's question. Returning toFIG.4, at block408, the method400may generate a question corresponding to a node of the decision tree to determine the user's intention. For example, a question may be generated corresponding to the root node of the decision tree. As shown inFIG.8, the question corresponding to the root node802may be “which funding?” The question may alternatively include some choices for the user. For example, the question may be “Housing fund or pension?”

At block410, the method400may receive the user's answer to the generated question from the client computing device. For example, the method400may provide the generated question to the user and then the user may answer the question in response. For example, the user may answer “pension”.

At block412, the method400may determine the user's intention from the decision tree based on the user's answer to the question. For example, if the user's answer is “pension,” the user in effect selects the second subset of the decision tree800by answering the question. If the root node802is also a leaf node, the user's intention may be determined to be the intention candidate selected by the user's answer. InFIG.8, the root node802is not a leaf node but has two child nodes. In this example, a child node of the root node802may be determined based on the user's answer. For example, if the user's answer is “pension”, the child node of the root node802is determined to be the node804-2. Another question may be generated corresponding to the node804-2, for example, “which city?” For example, if the user's answer to this question is “Beijing”, the candidate intention806-4may be determined to be the user's intention, considering that the node804-2is a leaf node of the decision tree. If the node804-2is not a leaf node but has one or more child nodes, the answer “Beijing” may lead to a child node of the node804-2and a further question may be generated corresponding to that child node. Further interactions with the user may be needed to determine the user's intention.

The decision tree800has a depth of two, as shown inFIG.8. It is noted that the decision tree in another embodiment may have a different depth than the decision tree800. For example, the decision tree may have a depth of one, i.e., the decision tree includes the root node only. The decision tree may alternatively have a depth of more than two. The recall intention candidates may include only highly relevant intention candidates. Hence, the recall intention candidates may include only a small number of intention candidates. By ranking the small number of recall intention candidates, the decision tree may not be too deep, for example, less than three. For example, the depth of the decision tree may be less than three for most cases and no more than two questions are provided to the user. Moreover, the questions may be multiple-choice questions to relieve the user's burden. QA accuracy may be improved without causing any substantial burden for the user.

At block414, the method400may determine a response to the user's question based on the user's intention. For example, if the candidate intention806-4may be determined to be the user's intention, it can be determined that the user intends to ask a question “How to withdraw pension in Beijing?”. The method400may then query a database for storing question-answer pairs to search for an answer to the disambiguated question. In this way, the method400may disambiguate the user's question that may be considered ambiguous and provide a more accurate response to the user. The determined response based on the user's intention is transmitted to the user.

It should be noted that the processing of conducting a natural language dialogue according to embodiments of this disclosure could be implemented by computer system/server12ofFIG.1.