Patent ID: 11947935
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 16:
17. A system, comprising:
a processor and a memory; wherein the memory includes instructions that when executed on the processor perform actions that: provide, in a model space, a pre-trained deep learning model trained to generate source code, wherein the pre-trained deep learning model includes a plurality of model parameters;
generate, in a user space, a custom tuning dataset to tune the pre-trained deep learning model for a target source code generation task, wherein the custom tuning dataset includes a plurality of input sequences, an input sequence of the plurality of input sequences including a prefix and a source code sample, wherein the prefix includes at least one trainable parameter;
apply, in the model space, the tuning dataset to the pre-trained deep learning model to generate a custom model for the target source code generation task, wherein the pre-trained deep learning model learns to optimize the prefix with the plurality of model parameters frozen;
compute, in the user space, an error loss from output of the application of one of the plurality of input sequences and a ground truth output;
backpropagate, to the model space, the error loss to the pre-trained deep learning model;
backpropagate, to the user space, gradients computed from the model space;
update, in the user space, embeddings of the prefix based on the backpropagated gradients; and
output the custom model, wherein the user space and the model space are in separate execution environments.