Patent ID: 11921782
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 6:
7. A method, comprising:
acquiring, from a conversational artificial intelligence session with a user, a user query;
performing a single-modal data fusing processing containing a similarity matching calculation on the user query and each single-modal data in each multi-modal tuple of plurality of the multi-modal tuples, and generating a plurality of single-modal fused data, wherein the single-modal fused data contains a first similarity information on a similarity between the user query and the single-modal data;
performing a multi-modal data fusing processing on a plurality of single-modal fused data corresponding to each multi-modal tuple of the plurality of multi-modal tuples, and generating a multi-modal fused data, wherein the multi-modal fused data contains similarity information on a similarity between the user query and the multi-modal tuples;
performing ranking on the similarity information on the similarity between the user query and the multi-modal tuples, and determining one or more multi-modal tuples for generating a multi-modal response according to a result of the ranking on the similarity information;
generating the multi-modal response according to the determined one or more multi-modal tuples,
generating summary information using the determined one or more multi-modal tuples; and
causing the multi-modal response to be output on a display device of the user as part of a response to the user query in the conversational artificial intelligence session, the output including one or more video clips corresponding to the one or more multi-modal tuples and including the summary information.