Patent ID: 11921782
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. The non-transitory, machine-readable medium according to claim 14, wherein the plurality of multi-modal tuples have a context sequential relationship therebetween,
wherein the operations of performing the multi-modal data fusing processing on the plurality of single-modal fused data corresponding to each multi-modal tuple of the plurality of multi-modal tuples and generating the multi-modal fused data comprises:
performing a first multi-modal data fusing processing on a plurality of single-modal fused data corresponding to each multi-modal tuple and generating a first multi-modal fused data, wherein the first multi-modal fused data contains a first similarity information on a similarity between the user query and the multi-modal tuples; and
performing a second multi-modal data fusing processing on a plurality of first multi-modal fused data according to a context sequential relationship and generating a plurality of second multi-modal fused data, the second multi-modal fused data contains a second similarity information on a similarity between the user query and the multi-modal tuples after the context sequential relationship is introduced; and
wherein the operations of performing the ranking on the similarity information further comprises ranking on the second similarity information on a similarity between the user query and the multi-modal tuples, and determine one or more multi-modal tuples for generating a multi-modal response according to the result of the ranking on the second similarity information.