Patent ID: 11934935
Assignee: DEEPMIND TECHNOLOGIES LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 8:
9. The method of claim 1, wherein the training further comprises:
obtaining a different context input;
processing the different context input using the trained autoregressive generative neural network to obtain, for each of the plurality of generation time steps, a respective different autoregressive output; and
determining a fourth gradient with respect to the feedforward parameters to maximize a contrastive loss that depends at least in part on, for each of the generation time steps, a second divergence from the likelihood distribution defined by the different autoregressive output for the generation time step and the likelihood distribution for the generation time step defined by the training feedforward output, and wherein determining the update to the current values of the feedforward parameters comprises determining the update based at least in part on the fourth gradient.