Patent ID: 11915123
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 18:
19. A method comprising:
receiving, by a computing device, multi-modal data, the multi-modal data comprising data in different formats from two or more modalities, including at least a first data set and a second data set, the first data set having a first modality and the second data set having a second modality that differs from the first modality;
separately processing the first and second data sets, including encoding the first data set into first encodings comprising one or more first vectors and encoding the second data set into second encodings comprising one or more second vectors;
analyzing the processed multi-modal data, including exploring inter-relation of information from at least two temporal sequences of a same or different sampling frequency, enabling direct incorporation of semantic information learned from the first modality of the first data feed to every time step reasoning on behaviors of the second modality of the second data feed, fusing the first and second encodings, the fusing comprising combining the first and second vectors representing correlated temporal behavior between performance behavior of data in the modalities included in the multi-modal data; and
returning the fused vectors encoding common behaviors.