Patent Description:
As the artificial intelligence technology evolves rapidly, man-machine interaction systems are in wide adoption. For example, a smart assistant has become one of the most important applications on an existing smart terminal. Common smart assistant products in the market include Apple Siri, Google Assistant, Amazon Alexa and Huawei HiVoice. The foregoing smart assistant products have respective features, but one of core functions of the smart assistant is to help a user complete a specific task through voice or text interaction, for example, making a call, setting a reminder, playing music, querying a flight status, and booking a restaurant. The foregoing task is usually initiated by the user and completed by one or more rounds of interaction with the smart assistant. By interacting with the user, the smart assistant gradually understands and confirms the user's intention and requirement, and usually completes the task by querying a database, and invoking an application programming interface (Application Programming Interface, API), or the like. Each task is usually performed independently and does not affect or depend on each other.

A task-oriented spoken dialog system (Task-oriented spoken dialog system) is one of the core technologies of the smart assistant. The task-based spoken dialog system (hereinafter referred to as "dialog system") is mostly based on a slot-filling (Slot-Filling) mode. A core technology of the dialog system is to define several slots (Slot) based on a task, and continuously identify the user's intention and extract related slot information during a dialog with the user. After the slot information is determined, the task can be completed. For example, in an air ticket booking task, a slot may be defined as: a departure location, a destination, a departure time, and a flight number. After the information is confirmed, the smart assistant may help a user complete the air ticket booking task.

Most of the existing smart assistants are built based on tasks. Each task has an independent slot, which can be considered as an independent dialog system. Different dialog systems run independently of each other. Generally, at an upper layer of the dialog system, a central control system is responsible for distributing a user to a specific task based on user input, and then starting a dialog for the task. In this case, only the dialog between the user and the smart assistant is involved, and each task is performed independently.

<CIT> describes a voice interaction method.

<CIT> describes a human-machine dialogue method.

The present invention is defined by the indpendent claims.

As shown in <FIG>, this application provides a man-machine interaction system. The man-machine interaction system mainly includes a central control module, a task engine module, a task memory, and a task controller.

The central control module <NUM> is configured to recognize an intention of a dialog request, determine a task, and distribute the task to a corresponding task engine.

The task engine module <NUM> includes a plurality of task engines. Each task engine is mainly responsible for a dialog task, and parses dialog request information to obtain key information (key-value) that meets a condition. For example, for an air ticket booking task engine, key information that meets an air ticket booking task may be extracted, such as departure location information, destination information, and time information. In addition, the task engine may store a corresponding parsing result in the task memory.

The task memory <NUM> is configured to store task status information, and may be accessed by a subsequent dialog, to determine an initial status and a behavior of a subsequent task. In a neural network-based dialog system, the task memory may be implemented by using a memory network (Memory Network), to encode task status information of each historical task, control the subsequent dialog to access related historical task status information by using an attention mechanism, and participate in determining a behavior and an output of a current dialog. Using the memory network to implement the task memory can better memorize historical task information generated a long time ago. In addition, because the attention mechanism is used to access the task memory, the system is enabled to obtain background knowledge most related to a current task.

In this embodiment of the present invention, the task status information includes the key information of the task, and the key information is each slot of the task and a value of each slot. The task status information may further include other information, for example, a name or an identifier of the task, whether the task is completed, or other dialog information in a task dialog process. For example, status information of a restaurant booking task is as follows:.

The task controller <NUM> is configured to control sequential execution of a plurality of tasks, and determine a next possible task based on historical task status information. Optionally, the task controller may further determine the next possible task based on a dialog between the man-machine interaction system and a user in the current task and current environment information.

When predicting that the next task is not empty, the man-machine interaction system actively initiates a dialog with the user, and determines a behavior and an output of the dialog by accessing the status information stored in the task memory. When predicting that the next task is empty, the man-machine interaction system does not perform subsequent operations and waits for the user to proactively trigger a next dialog.

In an embodiment of the present invention, the task controller is implemented by using a recurrent neural network (Recurrent Neural Network, RNN). To be specific, the RNN is used to predict the next task based on the historical task status, the current dialog, and the current environment information. It is readily figured out that the task controller in the solutions of the present invention is not limited to being implemented by using the RNN, and a person skilled in the art may use another machine learning method to predict the dialog task. In this embodiment of the present invention, the task controller may be an independent module, or the central control module may implement a function of the task controller, namely, the task controller and the central control module are one module.

In this embodiment of the present invention, after the task engine module obtains the key information that meets the condition, the task engine module may execute a corresponding task based on the key information. Alternatively, the central control module may execute a corresponding task based on the key information. Alternatively, an intelligent terminal may execute a corresponding task based on the key information. Alternatively, in the man-machine interaction system, a new module is developed to execute a corresponding task based on the key information. In this application, an entity for executing the corresponding task based on the key information is not specifically limited.

It should be noted that a function of the man-machine interaction system may be implemented by a server, or may be implemented by a terminal device, or may be jointly implemented by the server and the terminal device.

In addition, the man-machine interaction system provided in this embodiment of the present invention uses the task memory, for example, the memory network, and the task controller, for example, the recurrent neural network RNN, to facilitate the entire system to perform deep learning training.

Based on the man-machine interaction system shown in <FIG>, an embodiment of the present invention provides a multi-task processing method, as shown in <FIG>. It should be noted that, in this embodiment of the present invention, a smart assistant is used as an example of the man-machine interaction. For ease of description, "smart assistant" is used in some descriptions to replace "man-machine interaction system". The following describes the multi-task processing method shown in <FIG> with reference to the example of a multi-task processing scenario in the embodiment of the present invention shown in <FIG>.

Step S201: Determine a first task based on request information entered by a user.

In this embodiment of this application, the request information may be voice information, text information, image information, or the like. The user may input the request information to an intelligent terminal, and the intelligent terminal may forward the request information to a server. In this embodiment of this application, this step may be completed by the central control module in the man-machine interaction system shown in <FIG>. The central control module may recognize an intention of the request information to determine the first task.

In the example of the multi-task processing scenario shown in <FIG>, in a dialog <NUM>, the intelligent terminal receives a request message "I want to book an air ticket to Shanghai" entered by the user. The central control module in the man-machine interaction system determines that the first task is an "air ticket booking" task by recognizing the intention of the request message.

Step S202: Obtain key information corresponding to the first task, and execute the first task.

In this embodiment of this application, different slots may be disposed in a task engine corresponding to each task, the slot may be specifically a variable, and a value of the slot may be specifically key information corresponding to the slot. The slot may also be referred to as an information slot, and the key information corresponding to the slot may also be referred to as slot information. The man-machine interaction system extracts the key information corresponding to each slot by using the request information and/or one or more rounds of dialogs between the smart assistant and the user. For example, the key information of the task may be obtained by a task engine module.

In the example of the multi-task processing scenario shown in <FIG>, a task engine corresponding to the "air ticket booking" task includes slots such as a "flight number slot", a "departure location slot", a "destination slot", a "departure time slot", and an "arrival time slot". The smart assistant extracts the key information corresponding to each slot through rounds of dialogs with the user, and invokes an air ticket booking API (Application Programming Interface, application programming interface) based on the key information to execute the air ticket booking task.

Step S203: Store task status information of the first task, where the task status information includes the key information.

In this embodiment of the present invention, the task status information may be stored in a task memory, for example, a memory network. For example, after the key information corresponding to the first task is obtained or after the first task is executed, the task status information of the first task is stored. The task status information includes the key information, and optionally may further include other information such as a task name and a task completion status.

In the example of the multi-task processing scenario shown in <FIG>, air ticket booking task status information <NUM> includes the key information of the air ticket booking task, where the key information of the air ticket booking task is stored in the task memory.

Step S204: Predict and initiate a second task based on the task status information of the first task.

In this embodiment of the present invention, the second task may be predicted based on the task status information of the first task by using a task controller, for example, an RNN neural network. Optionally, in addition to the task status information of the first task, a predicted input may further include environment information in which the user is located, for example, information such as a time and a geographical location. After the second task is predicted, the task controller or the central control module may initiate the second task.

In the example of the multi-task processing scenario shown in <FIG>, after completing the "air ticket booking" task, the task controller predicts that a next task is hotel booking based on task status information corresponding to the "air ticket booking" task, and the smart assistant actively initiates a "hotel booking" task.

After the second task is initiated, the task engine module needs to obtain key information of the second task. In this embodiment of the present invention, by accessing the task status information of the air ticket booking task in the task memory, information most related to the second task, namely, the hotel booking task, may be calculated according to the attention mechanism. For example, destination information and arrival time information in the air ticket booking task. Based on the information, the smart assistant actively initiates a dialog interaction with the user. For example, a dialog <NUM> in <FIG> is initiated to gradually determine information such as a city, a hotel name, a check-in time, a check-out time, and a room type, namely, information of each slot of a hotel task engine, and complete the hotel booking through a hotel booking API.

In this embodiment of the present invention, task status information of the second task is stored, and is used as an input for predicting a next task. In the example of the multi-task processing scenario shown in <FIG>, hotel booking task status information <NUM> is stored in the task memory. The task controller predicts that a next task is empty based on the hotel booking task status information <NUM>. In other words, the next task does not need to be initiated.

In this embodiment of the present invention, status information of each task can be shared and used. The man-machine interaction system may predict a next task based on the stored task status information, and actively initiate the predicted task. This improves intelligence and efficiency of multi-task processing by the man-machine interaction system.

<FIG> and <FIG> show another example of a multi-task processing scenario according to an embodiment of the present invention. This example describes multi-task processing after the scenario shown in <FIG>. In this example, both the air ticket booking task and the hotel booking task are completed, and corresponding air ticket booking task status information <NUM> and hotel booking task status information <NUM> are stored in a task memory. Specific implementation has been described in the foregoing embodiment.

As shown in <FIG> and <FIG>, in a dialog <NUM>, a user actively initiates a "restaurant booking" task, and performs rounds of dialogs with a smart assistant, to gradually determine key information of the restaurant booking task, including information such as a city, a restaurant, a date, a time, a quantity of guests, and a confirmation status. When determining the key information of the restaurant booking task, a man-machine interaction system may access the task status information of the air ticket booking task and the hotel booking task, and obtain information related to the restaurant booking task. For example, information such as the city and the time.

Then, the man-machine interaction system stores task status information <NUM> related to the restaurant booking task in the task memory. A task controller predicts that a next task is "a dialog with a third party (restaurant)" based on the task status information of the restaurant booking task.

As described in the foregoing embodiment, during task prediction, environment information may be further used as an input.

Subsequently, the smart assistant actively initiates a dialog <NUM> with the third party (restaurant) by making a phone call. In this dialog, the smart assistant accesses the task status information of the restaurant booking task to gradually determine the key information of restaurant booking and complete the restaurant booking.

After the restaurant is booked, the man-machine interaction system updates the task status information of the restaurant booking task in the task memory, and changes the confirmation status information in the task status information from "no" to "yes", to obtain updated task status information <NUM>. Then, the task controller predicts that a next task is "confirming a meal booking result with the user". The smart assistant initiates a dialog <NUM> with the user to notify the user that the restaurant has been booked. After "confirming the meal booking result with the user" is completed, the corresponding task status information does not need to be updated. In this case, task status information <NUM> is consistent with the task status information <NUM>. Then, the task controller predicts that a next task is "booking a vehicle" based on the stored task status information. Key information obtaining and task execution of the vehicle booking task are similar to those of the foregoing tasks.

In this embodiment, the man-machine interaction system may predict a next task based on the stored task status information, and actively initiate a dialog with a third party. This improves intelligence and efficiency of multi-task processing by the man-machine interaction system.

In the method provided in this embodiment of the present invention, the stored task status information may be accessed by a subsequent task. Therefore, the man-machine interaction system may further understand the user's intention with the assistance of historical task status information, for example, semantic disambiguation on a current dialog statement. As shown in <FIG> is an example of performing semantic disambiguation on a current task statement based on historical task status information. In this example, a current task is a restaurant booking task, and a man-machine interaction system performs the semantic disambiguation on a restaurant booking task dialog <NUM> by using an attention mechanism and based on task status information <NUM> of a historical air ticket booking task adjacent to the current task.

In the dialog <NUM>, when initiating the restaurant booking task, a user directly says booking a dinner on the 26th. The man-machine interaction system obtains that a current month is April based on departure time and arrival time information in the task status information of the air ticket booking task. Therefore, the man-machine interaction system understands that a specific date expected by the user is April 26th. Subsequently, the user requests that a restaurant location be close to an airport. The man-machine interaction system infers that an organization name after the disambiguation is Shanghai Pudong Airport based on destination information "Shanghai Pudong" in the task status information of the air ticket booking task.

In this embodiment of the present invention, the man-machine interaction system understands a user intention by accessing stored task status information. This improves intelligence and working efficiency of the man-machine interaction system.

The foregoing embodiment of the multi-task processing method describes the man-machine interaction system. For example, a task engine module in the man-machine interaction system may access the task status information stored in a task memory, and determine information related to the current task according to the attention mechanism, and further generate an action of a current dialog and a subsequent statement. The following describes in detail with reference to an example of accessing historical task status information in the embodiment of the present invention shown in <FIG>.

In the example shown in <FIG>, task status information <NUM> of an air ticket booking task is stored in a task memory in a form of key-value (key information), where the key represents a slot, and value represents a specific value of the slot. The task status information <NUM> of the air ticket booking task includes slots such as a "flight number slot", a "departure location slot", a "destination slot", a "departure time slot", and an "arrival time slot", and values of the slots. In a memory network, the key information is represented as an embedding (embedding) vector.

In a current hotel booking task, a man-machine interaction system calculates a correlation between each slot in the task status information of the air ticket booking task and the current task by using an attention mechanism. In other words, an attention weight vector of each slot is calculated. For example, the attention weight vector may be calculated according to a formula Att=softmax(WKT)V. Att represents the attention weight vector, softmax represents an exponential normalization function, W represents a model parameter, K is a vector representation of key, and V is a vector representation of value.

As shown in <FIG>, in the task status information of the air ticket booking task, a slot that is related to a hotel booking task is an arrival location and an arrival time determines a city where a hotel is located and a check-in time to some extent. The man-machine interaction system determines an action <NUM> of a current dialog: Inform(Task=Hotel, Date=<NUM>-<NUM>-<NUM>, City=Shanghai) based on the attention weight vector and specific key-value information. The action indicates asking a user whether to book a hotel in Shanghai on April <NUM>, <NUM>. Then, the man-machine interaction system may generate a natural language by using a corresponding module, for example, a language generation module, to initiate a dialog <NUM>: "Would you like to book your hotel in Shanghai on April <NUM>?" The prior art may be used for natural language generation in the man-machine interaction system.

In this embodiment, the man-machine interaction system confirms information related to the current task in the historical task status information by using the attention mechanism, therefore the man-machine interaction system is more focused and more efficient in using the historical task status information.

The foregoing embodiment of the multi-task processing method describes that a task controller may predict a next task based on stored task status information. Optionally, the task controller may further perform prediction with reference to environment information. <FIG> shows an example of predicting a task by a task controller according to an embodiment of the present invention.

In this example, the task controller is implemented by using a recurrent neural network RNN. For each task, task status information of the task xt and environment information in which a user is located zt are input into the recurrent neural network. A current implicit status vector ht is calculated based on a historical hidden status vector ht-<NUM>, and then a next task is predicted based on the current hidden status vector ht, and so on.

In an example, the implicit status vector ht may be calculated according to a formula ht = f(Wxxt + Wzzt + Whht-<NUM> + b). f is a transformation function, for example, a sigmoid function or a ReLU function, Wx, Wz and Wh are parameter matrices, and are respectively multiplied by the task status information xt, the environment information zt, and the historical implicit status vector ht-<NUM>, and b is a parameter vector.

The foregoing embodiment has described in detail how the man-machine interaction system shown in <FIG> completes the multi-task processing method shown in <FIG>. A person skilled in the art can understand that a structure of the man-machine interaction system shown in <FIG> is an example. For example, division into the modules is merely logical function division and may be another division in actual implementation. For example, functional modules described in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module, or one or more modules are integrated into another device. The foregoing modules may be implemented in a form of hardware, or may be implemented in a form of a software functional module.

When the modules are implemented in the form of a software functional module and sold or used as an independent product, the modules may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the present invention essentially, or the part contributing to the prior art, or all or some of the technical solutions may be implemented in a form of a software product. The software product is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to perform all or some of the steps in the methods described in the embodiments of the present invention. The foregoing storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc.

<FIG> is a schematic diagram of a hardware structure of a man-machine interaction system according to an embodiment of the present invention. The man-machine interaction system shown in <FIG> includes a memory <NUM>, a processor <NUM>, a communications interface <NUM>, and a bus <NUM>. A communication connection between the memory <NUM>, the processor <NUM>, and the communications interface <NUM> is implemented through the bus <NUM>.

The memory <NUM> may be a read-only memory (Read-only Memory, ROM), a static storage device, a dynamic storage device, or a random access memory (Random Access Memory, RAM). The memory <NUM> may store a program. When the program stored in the memory <NUM> is executed by the processor <NUM>, the processor <NUM> and the communications interface <NUM> are configured to perform the steps in the foregoing method embodiments.

In an example, the processor <NUM> may use a general-purpose central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (Application-specific Integrated Circuit, ASIC), a graphics processing unit (Graphics Processing Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), or one or more integrated circuits. The processor <NUM> is configured to execute a related program, to implement modules in the man-machine interaction system provided in the foregoing embodiments, for example, a central control module, a task engine module, a task memory, and a task controller, and a function that needs to be executed, or perform steps in the foregoing multi-task processing method embodiments, for example, step S201 to step S203.

In another example, the processor <NUM> may alternatively be an integrated circuit chip and has a signal processing capability. In an implementation process, steps of the multi-task processing method provided in the foregoing embodiments may be completed by using a hardware integrated logic circuit in the processor <NUM> or an instruction in a form of software.

The communications interface <NUM> uses a transceiver apparatus, for example, but not limited to, a transceiver, to implement communication between the man-machine interaction system and another device or a communications network.

The bus <NUM> may include a path for transmitting information between components of the man-machine interaction system.

Claim 1:
A multi-task processing method in a man-machine interaction system, comprising:
determining (S201), based on request information entered by a user, a first task;
obtaining (S202) key information corresponding to the first task and executing the first task, wherein the key information comprises one or more slots and values of the one or more slots;
storing (S203) task status information of the first task, wherein the task status information comprises the key information; and
predicting and initiating (S204), based on the task status information of the first task, a second task;
wherein the predicting and initiating, based on the task status information of the first task, a second task comprises: inputting the task status information of the first task into a recurrent neural network, predicting the second task, and initiating the second task;
wherein the inputting the task status information of the first task into a recurrent neural network, predicting the second task comprises:
inputting the task status information of the first task into the recurrent neural network to obtain an implicit status vector ht through calculation, and predicting, based on the implicit status vector, the second task, wherein ht = f(Wxxt + Wzzt + Whht-<NUM> + b), f is a transformation function, xt is a task status information vector, zt is an environment information vector, ht-<NUM> is a historical implicit status vector, Wx, Wz and Wh are parameter matrices, and b is a parameter vector.