# Multimodal Emotion Recognition Using Visual, Vocal and Physiological Signals: A Review
## 4 Human Emotional Expression Recognition
### Multimodal Emotion Recognition
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Ref** & **Year** & **Multimodal Database** & **Elicitation** & **Features** & **Classifier** & **Average Accuracy** & **Fusion Method** & **Modalities** \\ \hline
[223] & 2018 & \begin{tabular}{c} IEMOCAP, \\ EmotiW \\ \end{tabular} & - & \
\begin{tabular}{c} BiGRU, \\ attention \\ layer \\ \end{tabular} & CNN & 72.7\%, WFI = 0.726 &
\begin{tabular}{c} Word-level \\ feature-level \\ fusion, CNN \\ \end{tabular} & Audio/text \\
[164] & 2015 & eNTERFACE & - & - & (SVM), ELM & Acc: 88\% &
\begin{tabular}{c} Feature-level \\ fusion \\ \end{tabular} & Audio/facial/ text \\ \hline
[224] & 2021 & \begin{tabular}{c} RAVDESS \\ \end{tabular} & \
\begin{tabular}{c} Acted \\ bi-LSTM \\ \end{tabular} & - & Acc: 86.70\% &
\begin{tabular}{c} Decision- \\ using \\ multinomial \\ logistic \\ regression \\ \end{tabular} & Audio/facial \\
[225] & 2024 & \begin{tabular}{c} WESAD \\ \end{tabular} & \
\begin{tabular}{c} MobileNetV2 \\ spectrogram/(2D \\ CNN with a \\ federated \\ learning \\ concept) \\ \end{tabular} & - &
\begin{tabular}{c} Decision- \\ level fusion \\ using \\ averaging \\ probability \\ voting \\ \end{tabular} & Audio/facial \\
[226] & 2023 & \begin{tabular}{c} WESAD \\ \end{tabular} & - & \begin{tabular}{c} temporal \\ convolution- \\ based \\ modality- \\ specific \\ encoders \\ \end{tabular} & FC & \begin{tabular}{c} Acc: 84.81\% \\ val: 63.29, ar: \\ fusion using \\ val: 64.07, ar: \\ 50.42 \\ \end{tabular} & \
\begin{tabular}{c} Feature-level \\ fusion using \\ a transformer \\ \end{tabular} &
\begin{tabular}{c} EDA, BVP, \\ TEMP \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Multimodal emotion recognition using deep learning features. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Ref** & **Year** & **Multimodal Database** & **Elicitation** & **Features** & **Classifier** & **Average Accuracy** & **Fusion Method** & **Modalities** \\ \hline
[227] & 2022 & In-house dataset & Induced & CBAM and ResNet34 & MLP & Acc: 78.32\% (subject-dependent) & Data-level fusion & EEG/facial \\  & & & ConvLSTM2D & & & & \\
[228] & 2022 & RAVDESS & & and CNN (MFCCs + MS + SC + TZ) CNN & MLP & Acc: 86\% (subject-dependent) & Feature-level fusion & Audio/video \\
[229] & 2022 & SAVEE & - & CNN and bi-LSTM & MLP & Acc: 99.75\% (subject-dependent) & Feature-level fusion & Audio/video \\  & & RML & & (2C, EN, EN) CNN & & Acc: 99.23\% & & \\  & & & & ENE) CNN & & & \\  & & & AlexNet with contrastive adversarial & & & & \\  & & & & learning & & & \\  & & & (facial) & & & & \\  & & IEMOCAP & & MFCC, velocity and acceleration + & Acc: & Adaptive decision-level fusion & Facial/audio \\
[230] & 2024 & (fa- & - & velocity and acceleration + VGGNet & - & Acc: 66.20 & Optimization-based model fusion & Facial/audio/ GSR \\
[231] & 2023 & ASCERTAIN & Induced & optimized DDQ & - & Acc: 66.20 & \begin{tabular}{c} Optimization-based model \\ fusion \\ \end{tabular}   & Facial/audio/ GSR \\
[232] & & & & & & & \\  & & &