# Electroglottograph-Based Speech Emotion Recognition via Cross-Modal Distillation
## 3 Experiments and Results
### Evaluations and Results  
In this section, we will further evaluate and discuss the results of our framework, especially from the aspect of the different performance between different kinds of emotions. Figures 8 and 9 show the confusion matrix of our student model and the model with CMED, respectively. The horizontal axis is our predicted emotion label and the vertical axis is the true label. Through the contrast of these two confusion matrices, we can find that the validation accuracy of almost every emotion has been consistently improved with the aid of cross-modal emotion distillation. To be specific, for anger, happiness, and neutrality, the validation accuracy has been improved by 15%, 3%, and 17%, respectively, which reflects the superiority of CMED in improving the performance of SER when enough utterances can be acquired. For the emotions that lack utterances, such as fear and disgust, the performance can also be improved by 11% and 7%. Better performance on these types of emotions indicates that with CMED, the ability to distinguish emotions has been improved significantly, regardless of the lack of utterances.  
From Figure 9, it can be seen that our framework can classify sadness, surprise, happiness and neutrality better than other emotions. This is because, as can be seen in Table 1, these emotions occupy a larger set of utterances than the others. For example, as there are only 28 utterances of fear in the training set and 7 in the validation set, clearly recognizing fear is a much harder challenge than others, as every mismatch influences the results greatly.  
When considering the relationship between emotions, it can be noted that some emotions are more similar to each other and more likely to be mismatched, such as sadness and neutrality, anger, surprise and happiness. This can be explained from the aspect of the characteristics of different emotions. For example, from the dimensional structure theories [53], sadness and neutrality both demonstrate low activation, while anger, surprise and happiness demonstrate high activation. The different levels of activation highly correspond to the average value of the fundamental frequency, which is our model's input.  
To further discover the distribution of our 512-dimension deep emotion feature in the emotion feature map, we visualized the feature of the 4-layer Bi-LSTM output using the t-SNE technique [54], which carries out a feature dimension reduction from the original dimension to two-dimensional space. To clearly visualize the distribution, we selected three emotions which are difficult to classify: surprise, happiness and neutrality. Figures 10 and 11 illustrate the distributions of the deep emotion features of the student model and the proposed model with CMED, respectively. The horizontal and vertical axes indicate the x and y axes in two-dimensional space.  
Figure 7: Experiments on different proportions of cross-modal emotion distillation in the total loss.  
From Figures 10 and 11, we can intuitively observe that with the aid of cross-modal emotion distillation, the deep emotion features are projected to different clusters clearly and separated from each other. Visualization experiments proved that the cross-modal emotion distillation helps to capture more emotion characteristics and results in a better distribution on the deep emotion feature map.  
Figure 8: Confusion matrix of our student model.  
Figure 9: Confusion matrix of our framework with CMED.