# Multi-Level Attention-Based Categorical Emotion Recognition Using Modulation-Filtered Cochleagram
## 1 Introduction  
The Internet of Everything (IoE) presents a plethora of opportunities for human-robot interaction (HRI), and incorporating emotion information can significantly improve the robot's ability to comprehend human intentions during HRI. Emotion information can be obtained through various means, such as speech, facial expressions, gestures, and electroencephalography (EEG) [1]. Among these, speech is the most natural and convenient communication mode between humans and robots. Therefore, speech emotion recognition (SER) holds immense potential for diverse applications in HRI, such as intelligent driving, service robotics, online education, telemedicine, and criminal investigations [2]. The extraction of emotional features is one of the key technologies in SER. The commonly used emotional features mainly include: hand-crafted low-level descriptor (LLD) and its high-level statistical features (HSF) [3], Mel filterbank features [4], spectrogram [5, 6], etc. However, researchers have not identified the best speech features for SER and still explore the effective features that can represent emotional states [7]. Humans can easilyperceive emotional information and its changes through the auditory system. Sounds reach the auditory cortex after passing through several auditory signal processing stages, which then perceives differences in intensity and tone to produce varying psychological responses. Therefore, identifying emotions from the perspective of auditory perception could be an effective approach. However, the human auditory system is highly complex, and the mechanisms of signal processing are not yet fully understood. To address this, researchers have developed functional models of the auditory system based on its characteristics, such as the models of the cochlear basilar membrane, the inner hair cell, the nerve conduction, and the auditory center. These models are mainly applied in a cochlear implant, hearing aid, sound source positioning, speech enhancement [8], and other areas, yet limited studies have explored auditory perception and understanding. Psychoacoustic research reveals that speech signals are decomposed into spectral-temporal components in the cochlea and are subject to spectral-temporal modulation through the auditory pathway, generating a modulation spectrum [9]. This modulation spectrum plays an essential role in speech perception and understanding [10,11]. Several studies have used statistical functions on the modulation spectrum to obtain modulation spectral features (MSF) for SER tasks [12]. [PERSON] et al. [13] proposed a feature-pooling scheme for dimensional emotion recognition using a combined MSF and 3D modulation spectrum representation. Recent research shows that cochleagrams have more advantages in SER than features such as spectrograms [14,15]. [PERSON] et al. [16] proposed a modulation-filtered cochleagram (MCG) feature to extract high-level auditory representations for dimensional emotion recognition. The experimental results showed excellent performance in terms of arousal and valence prediction, but the effectiveness of this feature in categorical emotion recognition requires further improvement. In order to extract high-level feature representations from speech features, deep learning methods, such as convolutional neural network (CNNs), recurrent neural network (RNNs), and transformers, are mainly used for the SER task [17,18]. CNNs are often used to extract high-level speech feature representation due to their scale and rotation invariance [19]. RNNs, including long short-term memory (LSTM) [20], are commonly employed to capture sequence dependencies owing to their ability to handle long-term dependencies in the speech sequence [21]. Recently, attention mechanisms have been incorporated into deep learning methods to automatically capture salient emotion features in speech sequences. [PERSON] et al. [4] proposed an attentive CNN (ACNN) based on the attention model to identify emotions from the log-Mel filterbank features. [PERSON] et al.