Speech Emotion Recognition Using Deep Sparse Auto-Encoder Extreme Learning Machine with a New Weighting Scheme and Spectro-Temporal Features Along with Classical Feature Selection and A New Quantum-Inspired Dimension Reduction Method

In today's world, affective computing is very important in the relationship between man and machine. In this paper, a multi-stage system for speech emotion recognition (SER) based on speech signal is proposed, which uses new techniques in different stages of processing. The system consists of three stages: feature extraction, feature selection/dimension reduction, and finally feature classification. In the first stage, a complex set of long-termstatistics features is extracted from both the speech signal and the glottal-waveform signal using a combination of new and diverse features such as prosodic features, spectral features, and spectro-temporal features. One of the challenges of the SER systems is to distinguish correlated emotions. These features are good discriminators for speech emotions and increase the SER's ability to recognize similar and different emotions. The data augmentation technique is also used to increase the number of training samples. This feature vector with a large number of dimensions naturally has redundancy. In the second stage, using classical feature selection techniques as well as a new quantum-inspired technique to reduce the feature vector dimensionality (proposed by the authors), the number of feature vector dimensions is reduced. In the third stage, the optimized feature vector is classified by a weighted deep sparse extreme learning machine (ELM) classifier. The classifier performs classification in three steps: sparse random feature learning, orthogonal random projection using the singular value decomposition (SVD) technique, and discriminative classification in the last step using the generalized Tikhonov regularization technique. Also, many existing emotional datasets suffer from the problem of data imbalanced distribution, which in turn increases the classification error and decreases system performance. In this paper, a new weighting method has also been proposed to deal with class imbalance, which is more efficient than existing weighting methods. The proposed method is evaluated on three standard emotional databases EMODB, SAVEE, and IEMOCAP. According to our latest information, the system proposed in this paper is more accurate in recognizing emotions than the latest state-of-the-art methods. 1Introduction Recognition of emotions from speech signal or Speech Emotion Recognition (SER) is one of the research fields in affective computing. The purpose of the SER is to analyze human speech signal and to extract the emotional state of the person (fear, joy, sadness, etc). The SER systems usually have three stages: feature extraction, dimension reduction/feature selection, and classification. In the feature extraction stage, prosodic and spectral features are usually used. These features alone are not able to discriminate unstable transitions in the speech signal in different emotions. For this purpose, in this paper, spectro-temporal features such as Gabor filter bank (GBFB) and separate Gabor filter bank (SGBFB) features [1] have been used, which have more discriminative power. New spectral features such as constant-Q cepstral coefficient (CQCC) [2], Single frequency cepstral coefficient (SFCC) [3] and IIR-CQT Mel-frequency cepstral coefficient (ICMC) [4] that have not previously been used to identify 1 S.J. Kabudian Department of Computer Engineering and Information Technology, Razi University, Kermanshah, Iran. e-mail: Kabudian@razi.ac. ir Corresponding Author