Patent ID: 11963771
Assignee: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES
Field: Computer technology (Electrical engineering)
Classification: CPC A  G | IPC A  G

Claim 3:
4. The method according to claim 1, wherein calculating the in-depth audio features and the in-depth video features by using multi-head attention mechanism so as to obtain attention audio features and attention video features includes:
inputting the in-depth audio features and in-depth video features, respectively, into a multi-head attention module to obtain in-depth audio feature representations with varied weight distribution and in-depth video feature representations with varied weight distribution; and
normalizing the in-depth audio feature representations with varied weight distribution and the in-depth video feature representations with varied weight distribution by using a softmax function to obtain valued degrees with respect to different features, and the attention audio features and the attention video features.