Learning Perceived Emotion Using Affective and Deep Features for Mental Health Applications Virtual agents are being increasingly used in the areas of healthcare, treatments, and therapy. Virtual agents with emotional intelligence have shown the potential to be applicable to deliver mental health intervention and facilitate therapies for children on the autism spectrum. To build an emotionally intelligent agent, automatic recognition of emotions is an essential building block. For mental health therapies and treatments, detecting rapidly changing emotions of individuals can be useful to know whether the patients are showing appropriate emotional response or not. To this end, we present a new datadriven approach to identify the perceived emotions of individuals based on their walking styles. We extract an individualxe2x80x99s walking gait in the form of a sequence of 3D poses given an RGB video of himher walking. We leverage the gait features to classify the perceived emotional state of the individual into one of four categories: happy, sad, angry, or neutral. First, we use an LSTM network to extract deep features of the gait using labeled emotion datasets. Next, we compute the affective features of the gaits using posture and movement cues. We combine these affective features with the deep features and classify them using a Random Forest Classifier. We observe that this approach provides an accuracy of 80:07% in identifying the perceived emotions. Additionally, we present a new dataset consisting of videos of walking individuals, their extracted gaits, and perceived emotion labels associated with each gait. We refer to this dataset as the xe2x80x98EWalk (Emotion Walk)xe2x80x9d dataset.

Different Performances of Speech and Natural Gait in Identifying Anxiety and Depression. Recognition of individualxe2x80x99s emotions using the data of speech and natural gait has been proved effective by previous studies. This study establishes a supervised regression model based on individualxe2x80x99s speech and natural gait data to predict the scores of depression and anxiety. The results show that the performance of the predictive model based on gait features is better than that of speech features. The basic processes are as follows: First, we recruited 88 participants and collected their speech data and natural gait data in laboratory; Second, we required each participant to finish the anxiety and depression scales (PHQ9, CESD, TAI, and GAD7), to get the anxiety and depression scores as the tagging data for modeling; Third, we extracted 6125 features from each participantxe2x80x99s speech data and 1602 features from the gait data; Finally, we used a variety of machine learning algorithms to train and test the predictive models. The results show that the max Pearson Correlation Coefficient between the predicted scores and the questionnaire scores of the speechbased model is 0.43, and the max Pearson Correlation Coefficient of the gaitbased model is 0.73, which is much better than the speechbased model.