Extraction of facial informative regions and discriminative shared space for facial expression recognition

Recognition of human’s emotion through facial expressions has many important applications ranging from behaviour recognition, human-computer interaction, security, psychology, and so on. Recognition of facial expressions from non-frontal faces, and recognition from different views are two important research challenges. As different views of a facial expression are just different manifestations of the expression, the information embedded in different views can be effectively utilized for facial expression recognition (FER). Motivated from the above mentioned facts, we proposed to extract facial informative regions and discriminative shared space for facial expression recognition. Extraction of discriminative features for different facial expressions is a key step in facial expression recognition. However, most discriminative facial features can be extracted from the informative regions of a face. In this view, the importance of different facial sub-regions are investigated, and subsequently the facial sub-regions which have significant contributions in different facial expressions are only considered for feature extraction. Furthermore, a weightedprojection based local binary pattern (WPLBP) feature is proposed. For this, texture features are extracted from informative regions and they are weighted on the basis of their importance. Finally, an efficient face model is derived from the informative regions of a face. The proposed face model has several advantages, and it gives better performance than other existing face models. Next, we proposed an Uncorrelated Multi-view Discriminant Locality Preserving Projection (UMvDLPP) analysis to recognize expressions from multi-view face images. The proposed UMvDLPP first transforms expressions of different views to a common uncorrelated discriminative subspace, and then classification is performed. One of the major advantages of our proposed scheme is that classifiers need not be learned separately for all the views. Moreover, it can effectively handle multi-modal characteristics of multi-view data than the existing learning-based methods. Discriminative shared Gaussian process latent variable model (DS-GPLVM) [1] can give better performance in the multi-view FER than the existing multi-view linear and non-linear learning-based methods. Laplacian-based prior used in DS-GPLVM only captures topological structure of data space without the inter-class separability of the data, and hence, the obtained latent space is not optimal. So, an efficient prior is proposed, which not only depends on the topological structure of the intra-class data, but also on the local-between-class-scatter-matrix of the data onto the latent manifold. The proposed approach employs a hierarchical framework, which is termed as multi-level uncorrelated DS-GPLVM (ML-UDSGPLVM). In this framework, expressions are first divided into three sub-categories. Subsequently, each of the sub-categories are further classified to identify the constituent basic expressions. Hence, first level of MLUDSGPLVM i.e., 1-UDSGPLVM is learned for classification of different categories, and then a separate 2-UDSGPLVM is learned for recognizing constituent expressions of each of the categories. Extensive experiments on a standard dataset show that our proposed method performs better than the existing multi-view FER methods. This improvement is due to the fact that the proposed method enhances the discrimination between the classes more effectively, and classifies expressions category-wise followed by classification of the basic expressions embedded in each of the sub-categories (hierarchical approach).