# HyFusER: Hybrid Multimodal Transformer for Emotion Recognition Using Dual Cross Modal Attention
## 1 Introduction  
Emotion recognition has emerged as a critical element in the field of human-computer interaction (HCI), enabling precise understanding of user emotions and appropriate responses. Recently, with the widespread adoption of non-face-to-face environments, conversational AI systems have become essential across various industries, including remote education, healthcare, and emotional labor support. These systems actively interact with users, provide information, solve problems, and deliver personalized experiences tailored to individual needs and preferences, thereby enhancing user satisfaction. In the field of remote education, emotion recognition plays a vital role in understanding learners' emotional states and motivations in real time, enabling the provision of the most suitable learning environment. By analyzing student focus or engagement levels, it becomes possible to offer personalized learning content, maximizing learning outcomes. A study developed instructional design principles incorporating multimodal elements to improve student concentration in remote learning contexts [1]. Another study explored changes in teachers'perceptions of online classes following the COVID-19 pandemic, systematically identifying the strengths and weaknesses of online education and seeking new educational possibilities based on these insights [2]. In healthcare, the potential applications of emotion recognition technology are significant. By continuously monitoring patients' psychological and emotional states, this technology can propose optimized healthcare and treatment strategies tailored to individuals. Research on IoT-based smart healthcare systems has demonstrated their ability to collect and analyze personalized health data, enabling customized medical care and status monitoring [3]. Additionally, AI-based well-being support systems for the elderly have been shown to induce psychological stability, alleviate depression, and improve quality of life in older adults [4]. Emotion recognition also plays a crucial role in systems supporting emotional labor. Due to the inherently emotionally demanding nature of their work, emotional laborers require appropriate feedback to manage stress and fatigue effectively. One study introduced an AI-powered smart healthcare exercise management system that monitors individuals' health in real time and provides solutions to reduce fatigue [5]. Research on personalized health management systems for home patients has also highlighted their effectiveness in monitoring daily health conditions and facilitating medical intervention when necessary, creating a more efficient environment for both patients and healthcare providers [6]. These prior studies underscore the importance of emotion recognition models capable of accurately capturing user emotions. The model proposed in this study reflects this necessity and is expected to contribute significantly to enhancing user-centric services across various domains, including remote education, healthcare, and emotional labor support. Existing studies on multimodal emotion recognition, which simultaneously utilize text and speech, have aimed to enhance the accuracy and reliability of emotion recognition by complementing subtle emotional signals that are difficult to capture with a single modality. For example, [PERSON]'s study [7] extracted features such as MFCC (mel-frequency cepstral coefficients), energy, and pitch from speech data, while text was processed as transcriptions using a Bag-of-Words approach, with feature vectors from both modalities simply concatenated. [PERSON]'s study [8] converted speech signals into spectrograms, extracting features through CNN, while the text modality employed word embeddings processed via a CNN-LSTM architecture to learn sequential information. By averaging the output values of the two models, the study effectively combined time-frequency domain features with linguistic contextual information to predict emotions.