# Review of Studies on Emotion Recognition and Judgment Based on Physiological Signals
## 4 Research on and Application of Multimodal Physiological Signals
### Multiple Physiological Signals
They obtained a set of optimal features from the physiological EDA and pulse wave signals, reduced the dimensionality features, chose features with a significance of 0.01 according to the relationship between the features and emotional states, and finally built a model of emotion recognition and judgment as shown in Figure 3, where BpNN50 means percentage of main wave interval > 50 ms, 1 dmean is the mean value of first-order difference, HVLA is high valence-low arousal, LVHA is low valence-high arousal, HVHA is high valence-high arousal and LVLA is low valence-low arousal. [PERSON] et al. [83] built a real-time emotional biofeedback system based on EDA, ECG, blood volume pulse and skin temperature signals and presented the experimental results for the system based on the above physiological signals. It was found that the system could help improve emotional self-awareness by decreasing error rate by 16.673% and 3.333% for men and women, respectively. [PERSON] et al. [84] presented a multi-stage multimodal dynamical fusion network based on EEG, EMG and electrooculography signals to solve the problem of cross-modal interaction appearing in one-stage fusion. They obtained the joint representation according to cross-modal correlation using the presented network, which was verified on a multimodal benchmark. The experimental results showed that the method of multi-stage, multi-modal dynamical fusion was better than the related one-stage, multi-modal methods. [PERSON] et al. [85] performed an experiment to explore subjects' emotional experience of virtual reality by comparing feelings and ECG, EDA and skin temperature signals. The results showed that a higher arousal level was obtained for virtual reality than that for a two-dimensional (2D) case. Eye fatigue occurred easily in both the virtual reality and 2D environments, while the former was more likely to lead to symptoms of dizziness and vertigo. Subjects had higher excitement  
Figure 3: Process of emotion judgment model [82]. levels in the virtual reality environment as well as more frequent and stronger emotional fluctuations. [PERSON] and [PERSON] [86] presented an integrated system of emotion evaluation. The system, which detected and analyzed variations in human objective and emotional states in an integrative way, consisted of subsystems of real-time subjective and objective emotionality evaluation based on EEG, ECG, EDA and skin temperature. They declared that the presented system would be conducive to the construction of a human emotion evaluation index by assembling the measured data into a database. [PERSON] et al. [87] proposed an assisted robotic system for children with hearing disabilities to improve emotion identification ability based on EDA, blood volume pulse and skin temperature signals. Situations concerned with a conventional setup, a tablet, and robot + tablet were assessed with 16 subjects with cochlear implants or hearing aids. The results indicated that the collected physiological signals could be divided, and the positive and negative emotions could be more effectively distinguished when subjects interacted with the robot than in the other two setups. [PERSON] and [PERSON] [88] identified a preferable framework of artificial intelligent ensemble feature selection and model of heterogeneous ensemble classification. Three base classifiers were applied to make decisions aimed at improving the classification performance of health-related messages. The results showed that the integration of the above framework and the model presented the highest accuracy in emotion classification based on the signal features of ECG, EDA, respiration and skin temperature. [PERSON] and [PERSON] [89] studied the effectiveness of physiological signals, i.e., EEG, ECG, EMG and EDA, in detecting negative emotions in 43 subjects driving in different scenarios. They extracted 58 features from the physiological signals and calculated the assessment criteria for sensitivity, accuracy, specificity and precision.