Patent ID: 11928854
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 1:
2. The method of claim 1, wherein for one or more of the query embeddings, obtaining the query embedding comprises:
obtaining a text sequence that describes a category of object; and
processing the text sequence using a text encoding subnetwork of the object detection neural network to generate the query embedding;
wherein the image encoding subnetwork and the text encoding subnetwork are pre-trained, wherein the pre-training includes repeatedly performing operations comprising:
obtaining: (i) a training image, (ii) a positive text sequence, wherein the positive text sequence characterizes the training image, and (iii) one or more negative text sequences, wherein the negative text sequences do not characterize the training image;
generating an embedding of the training image using the image encodin subnetwork, comprising:
processing the training image using the image encoding subnetwork to generate a set of object embeddings for the training image; and
processing the object embeddings using an embedding neural network to generate the embedding of the training image;

generating respective embeddings of the positive text sequence and each of the negative text sequences using the text encoding subnetwork; and
jointly training the image encoding subnetwork and the text encoding subnetwork to encourage: (i) greater similarity between the embedding of the training image and the embedding of the positive text sequence, (ii) lesser similarity between the embedding of the training image and the embeddings of the negative text sequences, comprising:
jointly training the image encoding subnetwork and the text encoding subnetwork to optimize an objective function that includes a contrastive loss term.