# Multimodal Emotion Recognition Using Visual, Vocal and Physiological Signals: A Review
## 4 Human Emotional Expression Recognition
### Single Modality
#### 4.1.3 Physiological Modality
The classification methods that have been used on handcrafted features for EEG-based emotion recognition are detailed in Table 11.  
Previous work has been on EEG-based deep learning of emotion recognition. [PERSON] et al. [201] packed multiple channels of EEG signals into 3D data of segments of 2D signals within a specific period. Then, they used a 3D CNN to classify the 3D data for emotion state recognition. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Ref** & **Year** & **Database** & **Elicitation** & **Features** & **Classifier** & **Accuracy** \\ \hline \hline \multirow{4}{*}{[190]} & \multirow{4}{*}{2014} & \multirow{4}{*}{GAMEEMO} & \multirow{4}{*}{Acted} & PS & & 87.53\% \\  & & & wavelet & & 78.41\% \\  & & & ENT & SVM & 65.12\% \\  & & & HE & & 71.38\% \\  & & & FD & & 70.63\% \\
* & & DEAP & Induced & Sample entropy & SVM & val: 80.43\%, ar: \\  & & & & & 79.11\% \\
* & & DEAP & Induced & Sample entropy of & SVM & 93.20\% \\  & & & & IMFs & & \\
* & & & FD, 5 statistics and 4 & & & \\
* & & & & band powers (\(\theta,\alpha,\beta\), & SVM & 35.76\% \\  & & & & \(\theta/\beta\) ratio) & & \\
* & & in-house dataset & & & kNN & 83.04\% \\  & & induced by & & Wavelet & LDA & 80.52\% \\
* & & audio-visual stimuli & & Entropy from & MLPs combined & ar: 87.43, val: 88.74\% \\
* & & DEAP & Induced & Temporal window & through DST & 79.95\% \\
* & & SEED & & Differential entropy & DGCNN & val: 86.23\%, ar: \\  & & & & & 84.54\%, dom: 85.02\% \\
* & & & & Differential entropy & CNN & 91.45\% \\
* & & & and brain network & & 79.16-91.39\% \\
* & & & & NDE & SVM & 81.66–85.39\% \\
* & & & Hybrid of time, & kNN & 82\% \\
* & & frequency, & SVM & 83\% \\  & & time–frequency and & & ANN & 96\% \\  & & & location features & & \\ \hline \hline \end{tabular}
\end{table} 
Table 11: EEG-based emotion recognition methods. The benefits of exploiting fewer EEG channels are less intrusiveness for the participants and a smaller cost for data processing. Two channels of EEG signals were used for a bispectral analysis to obtain the bispectrum features [202]. In addition, the energy bands and FD features were extracted from two channels of EEG signals [203]. EEG signals are always affected by noise and artifacts (the signal-to-noise ratio of EEG signals is low), which leads to poor robustness of the emotion recognition model and affects the emotion recognition accuracy. Because there is a limitation on available resources, or because the specific bio-signal is extremely noisy [204; 205], data augmentations (for example, employing GAN) are used to generate high-quality simulated data and to enhance the model [206; 207]. A preprocessing step is also undertaken to attenuate noise and remove artifacts. Some filtering (noise cancellation) approaches, such as surface Laplacian (SL) filtering [194], are also used to attenuate noise and remove artifacts. Even given the number of studies related to emotion recognition using EEG signals' features, with their respective advantages and drawbacks, the main limitation is related to the availability and quality of data [204; 205]. ECG-based emotion recognition also uses handcrafted features. A Bayesian NN was used to analyze the heartbeat data obtained by a fitness tracker to identify the emotional state of the user wearing the tracker [208]. Heartbeat data were obtained as a photoplethysmograph. With ECG data, a neural network was used for emotion recognition. Two NNs were proposed, one for signal transformation recognition and another for identifying emotions. The first signal transformation recognition network was trained using an extensive dataset to identify specific transformations in the input signal. The second network identified the emotions associated with each transformation successfully. The two-network model performed better than the traditional approaches, and the authors reported accuracies close to 98% [209]. Canonical correlation relating these signals to emotion alone could predict the correct emotions from physiological data with over 85% efficiency [210].