# Learning Better Representations for Audio-Visual Emotion Recognition with Common Information
## 5 Experiments
### Experimental Results
#### 5.3.2 Multimodal Performance  
We report the performance of audio-visual emotion recognition on the eNTERFACE'05, BAUM-1s, and RAVDESS datasets in order to further demonstrate the effectiveness of our system. In the meanwhile, we make a comparison with previous works, which also conduct experiments on these original datasets. Table 3 summarizes the results. To implement our architecture, we set \(\alpha\) equal to 1 and make the fusion network have two fully connected layers on the eNTERFACE'05 dataset, set \(\alpha\) equal to 1 and make the fusion network have one fully connected layer on the BAUM-1s dataset, set \(\alpha\) equal to 0.1 and make the fusion network have one fully connected layer on the RAVDESS dataset. From Table 3, we can find that our method is competitive with the compared works. To be specific, on the eNTERFACE'05 dataset, the performance of our method is at least 1% higher than that of the previous methods [20; 21; 22; 27; 29; 95; 97; 98]. On the BAUM-1s dataset, we improve the performance of the method [27] from 60.05% to 67.59%. On the RAVDESS dataset, our method performs much better than the methods [99; 100] by more than 13%. It is worth noting that the method [22] combines the audio and visual features that are based on CCA and the method [27] utilizes a similar approach with low-rank representation. Our method performs better than these two methods, which show that the correlation loss we propose can learn the discriminative feature representations more effectively. Besides, most of the above methods are based on DNN models, while our method achieves better performance, which shows that the features learned from our deep learning model has more powerful expressiveness. To sum up, our deep learning method can efficiently extract the discriminative feature representations with common information in order to achieve the highest accuracies among all the compared methods. Figure 5 shows the classification confusion matrices using our method on the eNTERFACE'05, BAUM-1s, and RAVDESS datasets. It can be found that, on the eNTERFACE'05 dataset, \"fear\" and \"surprise\" are harder to be recognized when compared to other emotions. This indicates audio-visual cues of these two emotions contain less emotional information. On the BAUM-1s dataset, \"happiness\" achieves the highest recognition accuracy among the six emotions. However, \"anger\" and \"fear\" have lower recognition accuracies. This may be due to the small number of samples of \"anger\" and \"fear\" on the BAUM-1s dataset, which results in our model being unable to fully learn the features of \"anger\" and \"fear\" for emotion classification. On the RAVDESS dataset, both \"anger\" and \"happiness\" can be identified with 100% accuracy. Similar to the eNTERFACE'05 dataset, \"fear\" and \"surprise\" are more difficult to be recognized. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Method** & **Accuracy(\%)** \\ \hline \multirow{8}{*}{eNTERFACE’05} & [PERSON] et al. [22], CCA & 71 \\  & [PERSON] et al. [97], SVM & 85.23 \\  & [PERSON] et al. [98], PCA + LDA + Rules & 86.67 \\  & [PERSON] et al. [95], Sparse Kernel Reduced-Rank Regression & 87.46 \\  & [PERSON] et al. [20], Attention Mechanism & 79.2 \\  & [PERSON] et al. [29], HGR Maximal Correlation + Autoencoder & 85.43 \\  & [PERSON] et al. [21], Extreme Learning Machine + SVM & 86.4 \\  & [PERSON] et al. [27], Low-rank Representation & 86.98 \\  & Ours & **88.66** \\ \hline \multirow{2}{*}{BAUM-1s} & [PERSON] et al. [27], Low-rank Representation & 60.05 \\  & Ours & **67.59** \\ \hline \multirow{2}{*}{RAVDESS} & [PERSON] et al. [99], Incremental Learning & 67.70 \\  & [PERSON] et al. [100], Spiking Neural Networks & 83.60 \\ \multicolumn{2}{c}{Ours} & **97.57** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The performance comparison of audio-visual emotion recognition with previous works on the eNTERFACE’05, BAUM-1s, and RAVDESS datasets.