Accuracy improvement of facial expression recognition in speech acts: Confirmation of effectiveness of information around a mouth and GANbased data augmentation With the growth of the social robot market, much research has been undertaken on facial expression recognition, which is an important function of a social robot. Facial expression recognition models have shown good performance in a facial expression image dataset that expresses emotion without considering speaking effect. However, in reality, humans often express emotions by speaking and moving the muscles around the mouth. Therefore, the lack of consideration of speech leads to unsatisfactory emotion recognition results. In this paper, we investigated two points to be considered in learning a facial expression recognition model. First, we confirmed whether the information around a mouth induces the recognition model in speech act to misrecognition like the case of a facial expression recognition in nonspeech acts or it has valid information for facial expression recognition. Second, Generative Adversarial Network (GAN)based data augmentation has been performed to cover the problem in which the accuracy of the recognition model in speech acts is low because of the relatively small variance about the subject in RML dataset. The results showed that the information around the mouth made facial expression recognition in speech acts exhibit higher performance, unlike the case of facial expression recognition in nonspeech acts. In addition, the GANbased data augmentation alleviated the accuracy degradation in facial expression recognition because of the low variance of the dataset.

Facial Expression Recognition Using Robust Local Directional Strength Pattern Features and Recurrent Neural Network This work proposes a novel facial expression recognition approach to contribute to better humanmachine interactions. To do that, edge features in facial expression images are combined with a recurrent neural network (RNN) to classify different facial expressions. Robust edge features are first obtained by using Local Directional Strength Pattern (LDSP) and applied with RNN. This LDSPRNN approach achieves superior recognition performance than other conventional approaches on a randomly distributed training and testing datasets obtained from a public dataset. The proposed approach should be useful for various practical applications such as a robot analyzing and understanding different human emotions from facial expressions based on robotic vision.