Patent ID: 11934952
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 5:
6. A system for training a natural language processing (NLP) classifier, comprising:
a non-transitory memory; and
one or more processor coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
receiving, via a data interface, a training dataset of data samples that correspond to a data probability distribution;
generating, for at least one data sample from the training dataset, a respective noise sample according to a noise probability distribution;
inputting a data sample and the respective noise sample to the NLP classifier;
encoding the respective data sample into an encoded data sample representation;
encoding the respective noise sample into an encoded noise sample representation;
generating, by the NLP classifier, a first classification output corresponding to the encoded data sample representation and a second classification output corresponding to the encoded noise sample representation;
computing a first energy term based at least in part on the first classification output and the encoded data sample representation according to an energy function selected from the group consisting of a scalar function, a hidden function, and a sharp-hidden function;
computing a second energy term based at least in part on the second classification output and the encoded noise sample representation according to the energy function;
computing a noise contrastive estimation (NCE) loss objective based at least in part on the first energy term and the second energy term; and
training the NLP classifier based at least in part on a combination of the NCE loss objective and a cross-entropy loss computed based on the first classification output conditioned on a respective data input sample.