Deep Learning for Multimodal Emotion RecognitionAttentive Residual Disconnected RNN Human communicates using verbal and nonverbal cues. One of the most essential elements that complements the understanding of communication is emotion. Emotion is expressed not only in words, but also facial expressions, body language, tone, etc. Therefore, we formulate the emotion recognition as a multimodal task.Emotions are usually described in a sequence along with the utterances. In recent years, RNNbased models have been known to be good at modeling the entire sequence and capturing longterm dependencies. However, it lacks the ability to extract local key patterns and positioninvariant features. Hence, we adopt Deep Attentive Residual Disconnected RNN model which incorporates the concept from both RNN and CNN to enhance the ability to capture spatial and temporal features.We utilize CMU MOSEI dataset comprising of language, visual, and acoustic modalities for training and evaluating our model. The results show that Deep Attentive Residual Disconnected RNN model outperforms the baseline. Besides, the use of multimodal approach also solidifies the recognition better compared to those of single modalities.

A Continuous Facial Expression Recognition Model based on Deep Learning Method Effective facial expression recognition is a very important part of perceiving the useru0027s emotions and designing a successful humancomputer interaction system in all fields. Many studies also applied computer vision, machine learning, and deep learning methods on affective computing. However, they mainly focus on the facial expression identification by a single static image, but didnu0027t consider the continuous facial expression of human emotions maybe is for a period of times. Therefore, to recognize the pattern of continuous facial expression and mood changes for further application is still a challenging task. In this study, we propose a continuous facial expression recognition model based on deep learning approach, which combined convolutional neural networks (CNN) and recurrent neural networks (RNN) to analyze and identify continuous facial expressions in a period of time to improve the traditional image recognition method.