Patent ID: 11960843
Assignee: ADOBE INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 8:
9. A machine learning system, comprising:
an input device configured to receive a plurality of training datasets comprising an image classification dataset and a natural language dataset;
a neural network model for performing a referring expression task, including an image processing sub-module with a first plurality of neural network layers for performing an object detection task and a natural language sub-module with a second plurality of neural network layers for performing a phrase parsing task, wherein the object detection task and phrase parsing task are combined to obtain the referring expression task of the neural network model;
a sub-module determination engine configured to:
receive the plurality of training datasets, wherein the plurality of training datasets comprises a first training dataset comprising images and a second training dataset comprising natural language phrases;
determine a data format associated with the first training dataset is an object detection task;
determine a data format associated with the second training dataset is a phrase parsing task;
identifying, using a lookup table, the image processing sub-module as being associated with the object detection task;
identifying, using the lookup table, the natural language sub-module as being associated with the phrase parsing task;
training the first plurality of neural network layers of the image processing sub-module using the image classification dataset, wherein the image processing sub-module is trained to perform the object detection task; and
training the second plurality of neural network layers of the natural language sub-module using the natural language dataset, wherein the natural language sub-module is trained to perform the phrase parsing task; and

an output device configured to output the referring expression task of the neural network model based on the image processing sub-module or the natural language sub-module.