Patent ID: 11922305
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 10:
11. A system for policy improvement in task-oriented learning, the system comprising:
a data interface configured to receive a training dataset comprising a plurality of dialogue rollouts generated by a latent stochastic behavior policy, wherein each rollout includes a time series of observations representing information of a respective dialogue at a plurality of dialogue turns;
a memory configured to store a neural model;
a processor configured to:
generate, by the neural model, a first predicted action distribution based on a current state of the respective dialogue according to a target policy;
compute a first discounted sum of future reward based on a discount parameter and a reward function of actions and states of the respective dialogue according to the latent behavior policy;
compute a first loss objective based on a first expectation of the first discounted sum of future reward and the first predicted action distribution, wherein the first expectation is taken over a probability distribution of the states and the actions according to the latent stochastic behavior policy; and
update the neural model by minimizing at least the first loss objective subject to a condition that a KL-divergence between the latent stochastic behavior policy and the target policy conditioned on the current state of the respective dialogue is less than a pre-defined hyperparameter.