Patent ID: 11886446
Assignee: BAIDU USA LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A system comprising:
one or more processors; and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
responsive to a stop condition not being reached, performing steps comprising:
given a relevance ranking batch of cross-lingual query and document pairs, in which the relevance ranking batch comprises (1) at least one positive sample cross-lingual query and document pair, in which the query and the document are in different languages and the document is relevant to the query and (2) at least one negative sample cross-lingual query and document pair, in which the query and the document are in different languages and the document is not relevant to the query, using a cross-lingual language model with a relevance ranking model head to predict a relevance ranking that ranks whether a document is relevant to a corresponding query;
updating the cross-lingual language model using known relevance rankings and predicted relevance rankings;
given a query language modeling batch of cross-lingual query and document pairs, in which the query and the document are in different languages and the document is relevant to the query, inputting the query, in which a portion of tokens in the query are masked tokens, and the corresponding document into the cross-lingual language model to predict the masked tokens of the query; and
updating the cross-lingual language model based upon the cross-lingual language model's prediction compared to the masked tokens; and

responsive to a stop condition being reached, outputting the cross-lingual language model, which has been updated, as a pretrained cross-lingual language model.