Patent ID: 11877016
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 8:
9. The method according to claim 8, wherein
the calculating a plurality of bidirectional attention data between the feature data of each modality and the feature data of each of other modalities and the feature data of the candidate text with respect to each modality, and superimposing the plurality of bidirectional attention data together, to obtain cross-attention data between the modality and each of other modalities and the comment paragraph, and the calculating bidirectional attention data between the feature data of the candidate text and the feature data of each modality with respect to the candidate text and superimposing the plurality of bidirectional attention data together to obtain cross-attention data between the candidate text and each modality further comprises:
performing self-attention calculation processing on the feature data of each modality and the feature data of the candidate text, and generating self-attention data of each modality and self-attention data of the candidate text;
calculating a plurality of bidirectional attention data between self-attention data of each modality and the self-attention data of other modalities and the self-attention data of the candidate text with respect to each modality, and superimposing the plurality of bidirectional attention data together, to obtain cross-attention data between the modality and other various modalities and candidate texts; and
calculating bidirectional attention data between the self-attention data of the candidate text and the self-attention data of each modality with respect to the candidate text, and superimposing the plurality of bidirectional attention data together, to obtain cross-attention data between the candidate text and each modality.