# A Novel Heterogeneous Parallel Convolution Bi-LSTM for Speech Emotion Recognition
## 2 Related Work
### Emotion Feature Extraction  
The feature extraction in SER can be challenging because of the high nonlinearity of input speech data that contains noise from various sources ranging from data collection to data preprocessing. The two most challenging problems in SER feature extraction are probably the extraction of frame-based high-level feature representations and the construction of utterance-level features [12, 13]. Since speech signals are considered to be approximately stationary in small frames, those acoustic features extracted from short frames, which include pitch, mel-frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), and prosodic features, are believed to be influenced by emotions. The frame-based features, often referred to as low-level features, can provide detailed emotionally relevant local information [14]. So far, a variety of Low-Level Descriptor (LLD) features have been employed for SER. For example, [PERSON] et al. proposed a bag-of-audio-words (BoAW) approach, which was created from MFCCs and energy Low Level Descriptors (LLDs), to build the feature vectors in SER and employed support vector regressions (SVR) to predict the emotion [15]. Furthermore, based on the frame-based low-level features, neural networks are utilized to extract neural representations frame by frame, which are referred to as frame-based high-level feature representations [16]. Recently, SER has made great progress by introducing neural networks to extract high-level neural hidden representations. For example, the authors in [16, 17] applied neural networks on the low-level features, e.g., pitch, energy, to learn high-level features, i.e., the neural network outputs. [PERSON] et al. proposed an end-to-end model composed of a convolution neural network (CNN) architecture used to extract features before feeding a bi-directional long short-term memory (BLSTM) to model the temporal dynamics in the data [18]. [PERSON] et al. proposed an attentive convolution neural network (ACNN) that combines CNNs with the attention mechanism [19]. However, the emotion recognition at utterance-level requires a global feature representation, which contains both detailed local information and global characteristics related to emotion. Based on the frame-based high-level features learned, various methods are used to construct the utterance-level features. [PERSON] et al. proposed to use extreme learning machines (ELM) upon the utterance-level statistical features [17]. The utterance-level features are the statistics of segment-level deep neural network (DNN) output probabilities, where each segment is a stack of neighboring frames. [PERSON] et al. introduced recurrent neural networks to increase the ability of the model to capture temporal information [20]. However, since only the final states of recurrent layers were used for classification, it would lead to the loss of detailed information in the following emotion classification, because all information is stored in the fixed-size final states. [PERSON] et al. explored pooling utterance-level features from the high-level features output by CNNs with attention weights, because not all regions in the spectrogram contain information useful for emotion recognition [21], in which using pooling can also avoid squeezing all of the information into one fixed-size vector. In this study, we propose a new multi-feature fusion representation method for discrete emotion recognition. Unlike most of the previous studies in the literature, the method of our feature fusion was inspired by the way that conventional speech features (e.g., Mel-Frequency Cepstral Coefficients (MFCCs)) are computed. That is, 32D Low-Level Descriptor (LLD) features, including 12D Chroma [22] and 20D MFCC [23], are extracted. The High-Level Statistical Functions (HSF), such as the mean of Chroma and the mean, variance, and maximum of MFCC, are calculated accordingly. Totally, 72 D acoustic features are used as the input for the following machine learning model for speech emotion recognition.