# Speech Emotion Recognition Using Deep Learning Transfer Models and Explainable Techniques
## 2 Related Work  
### Studies Applying Deep Learning to Spectrogram Images  
This section presents studies conducted on spectrograms in which a one-dimensional (1D) signal was converted into two-dimensional (2D) data by applying deep learning to obtain more information and applications. Each study used preprocessing methods and network designs for speech research. [PERSON] [8] added specific values to insufficient areas to reduce confusion in training and classification when dividing speech data containing imbalanced time. This was achieved by using the IEMOCAP dataset. Padded data were trained and classified by a model that used a convolutional neural network (CNN) and long short-term memory (LSTM). The model was evaluated by weighted accuracy (WA), which assigns weights according to the number of samples when data are imbalanced. It was also evaluated using unweighted accuracy (UA), which reflects equally regardless of the number of samples. The evaluation of the accuracy of the emotion classification model by WA and UA showed that WA was evaluated at 71.45%, and UA was evaluated at 64.22% when specific values were padded. When the length of speech data was imbalanced, WA reached 68.86%, and UA was at 57.45%, showing slight variations. [PERSON] [9] constructed a model using a CNN and a fully connected layer. The model's performance was compared with that of the fine-tuned, pretrained AlexNet using seven-emotion data contained in the Berlin dataset for effective speech emotion recognition (SER). AlexNet showed an imbalance in that the accuracy was high in certain classes but low in others; therefore, the mean accuracy was 56.19%. The proposed model was confirmed to be even more effective than AlexNet, with a mean accuracy of 56.38%. [PERSON] [10] used speech data containing MUOC classroom atmosphere information to check and observe the atmospheric and environmental characteristics of a classroom containing many people. Filter bank, Mel-frequency Cepstral Coefficients (MFCC), and spectrogram features were collected to improve performance and reduce loss. Each feature was processed using an independently trained hybrid neural network combining CNN and LSTM models. The information collected from each feature map, independently processed by the network, was classified into six types of classroom atmospheres. When all the collected features were combined and classified, the lowest loss rate was 24.64%. [PERSON] [11] conducted a study to confirm an effective data type when recognizing emotions using the Berlin dataset speech data. Speech data in 1D audio type was converted to a 2D image type by MFCC and used for training support vector machine (SVM), 1D-CNN, and 2D-CNN models. Each model classified seven emotions for each gender: SVM, using 1D audio-type data had the lowest performance, whereas the 2D-CNN model using 2D image-type data from the MFCC showed the highest performance, achieving 92.5% accuracy in all classes. Thus, transforming speech data into 2D image-type data is effective when classifying speech data. [PERSON] [12] proposed training NAO robots to respond according to a recognized user's emotion using CNN and random forest (RF) models with the CASIA emotion dataset from China. The data that the robots collected in everyday environments were mixed with noise and had lower purity than the data used for training. Therefore, to improve the generalization performance compared to those of existing methods, features of the speech data were extracted using the CNN model and the extracted feature map was classified using the RF classifier for six emotions. When classified through CNN, a performance of 81.43% was achieved, and when using RF together with CNN, the performance was increased to 84.68%, confirming that using CNN and RF together was effective in the learning process for NAO robots. [PERSON] [13] converted speech data to spectrograms to extract time-frequency features from the IEMOCAP dataset and divided each spectrogram into certain sizes.