Patent ID: 11893346
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. The computer-implemented method of claim 1, wherein the training comprises:
initializing a set of parameters of the token embedding portion to a base set of token embedding parameters;
initializing a set of parameters of the token self-attention portion to a base set of token self-attention parameters;
first training the encoder model, the first training comprising adjusting a set of parameters of the relativity embedding portion and a set of parameters of the metadata self-attention portion while the set of parameters of the token embedding portion is set to the base set of token embedding parameters and the set of parameters of the token self-attention portion is set to the base set of token self-attention parameters, the first training generating a partially trained encoder model; and
second training the partially trained encoder model, the second training comprising adjusting the set of parameters of the partially trained encoder model, the second training generating the trained encoder model.