Patent Document ID: 9911413
Application ID: 15392718
Patent Status: 1

Claim One:
1. A method for configuring a natural language (NL) understanding system, the NL understanding system including at least an intent classifier, the intent classifier comprising a neural network configurable with configuration data including neural network weights to distinguish a plurality of intents based on a linguistic input, the method comprising: configuring the intent classifier with first configuration data, the first configuration data having been determined from first collected data comprising input data items each annotated with a specific intent of a plurality of intents, the neural network weights of the first configuration data having been determined to best match the input data items and the annotated specific intents; processing a second plurality of user inputs to the system, wherein the processing includes, for each of the second plurality of user inputs, using the intent classifier configured with the first configuration data to determine a recognized intent corresponding to said input, and causing determination of a corresponding response to the input based on the recognized intent; storing second collected data, the second collected data comprising, for each of the second plurality of user inputs, a representation of the user input and a corresponding response; receiving manual annotation for the second collected data, an annotation for each item of the second collected data indicating whether the corresponding response is consistent with the user input, thereby forming second annotated data without requiring annotating a correct intent for user inputs for which the caused action does not sufficiently match the user input; determining second configuration data for the intent classifier, the second configuration data being determined to distinguish the plurality of intents by computing the second configuration data to match the second annotated data, including treating correct intents as latent variables that are not represented in the second annotated data, the determining including incrementally updating the neural network weights of the second configuration data; and configuring the intent classifier with the second configuration data.