Automatic recognition of facial expressions
Creators: name: Lajevardi, Seyed
Publisher: name: RMIT University
Year: 2024

description: Facial expression is a visible manifestation of the affective state, cognitive activity, intention, personality and psychopathology of a person; it not only expresses our expressions, but also provides important communicative cues during social interaction. Expression recognition can be embedded into a face recognition system to improve its robustness. In a real-time face recognition system where a series of images of an individual are captured, facial expression recognition (FER) module picks the one which is most similar to a neutral expression for recognition, because normally a face recognition system is trained using neutral expression images. In the case where only one image is available, the estimated expression can be used either to decide which classifier to choose or to add some kind of compensation. In a human-computer interaction (HCI), expression is an input of great potential in terms of communicative cues. This is especially true in voice-activated control systems. This implies an FER module can markedly improve the performance of such systems. Customer's facial expressions can also be collected by service providers as implicit user feedback to improve their service. Compared with a conventional questionnaire-based method, this should be more reliable and furthermore, has virtually no cost.<br><br>The main challenge for FER system is to attain the highest possible classification rate for the recognition of six expressions (Anger, Disgust, Fear, Happy, Sad and Surprise). The other challenges are the illumination variation, rotation and noise.<br>In this thesis, several innovative methods based on image processing and pattern recognition theory have been devised and implemented. The main contributions of algorithms and advanced modelling techniques are summarized as follows. 1) A new feature extraction approach called HLAC-like (higher-order local autocorrelation-like) features has been presented to detect and to extract facial features from face images. 2) An innovative design is introduced with the ability to detect cases using face feature extraction method based on orthogonal moments for images with noise and/or rotation. Using this technique, the expression from face images with high levels of noise and even rotation has been recognized properly. 3) A facial expression recognition system is designed based on the combination region. In this system, a method called hybrid face regions (HFR) according to the combined part of an image is presented. Using this method, the features are extracted from the components of the face (eyes, nose and mouth) and then the expression is identified based on these features. 4) A novel classification methodology has been proposed based on structural similarity algorithm in facial expression recognition scenarios. 5) A new methodology for expression recognition is presented using colour facial images based on multi-linear image analysis. In this scenario, the colour images are unfolded to two dimensional (2-D) matrix based on multi-linear algebra and then classified based on multi-linear discriminant analysis (LDA) classifier. Furthermore, the colour effect on facial images of various resolutions is studied for FER system. The addressed issues are challenging problems and are substantial for developing a facial expression recognition system. <br>