Patent ID: 11875267
Assignee: QUANTUM-SI INCORPORATED
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A method for generating a prediction for a prediction task using a multi-modal statistical model configured to receive input data from multiple modalities including input data from a first modality and input data from a second modality different from the first modality, the method comprising:
using at least one computer hardware processor to perform:
obtaining information specifying the multi-modal statistical model including values of parameters of multiple components of the multi-modal statistical model, the multiple components including first and second encoders for processing input data for the first and second modalities, first and second modality embeddings, a joint-modality representation, and a predictor, wherein:
the first and second modality embeddings comprise a plurality of weights and vectors for projecting the joint-modality representation into respective first and second modality spaces;

obtaining first input data for the first data modality;
providing the first input data to the first encoder to generate a first feature vector;
identifying a second feature vector using the joint-modality representation, the first modality embedding and the first feature vector; and
generating the prediction for the prediction task using the predictor and the second feature vector.