Patent ID: 11875128
Assignee: ADA SUPPORT INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 2:
3. The method of claim 1, further comprising fine-tuning values of the learnable parameters of the pre-trained language model by:
receiving a conversation dataset comprising data samples, each data sample of the conversation dataset having words;
tokenizing each data sample of the conversation dataset to generate a tokenized data sample comprising tokens;
masking at least one token of the tokenized data sample of the conversation dataset to generate a tokenized data sample with masked tokens;
generating, using the pre-trained language model, a predicted token for the at least one masked token;
computing a masked language model loss for the pre-trained language model based on the predicted token and a respective token of the data sample of the conversation dataset; and
back propagating the masked language model loss through the pre-trained language model to adjust values of learnable parameters of the pre-trained language model.