Analysis of Facial Ageing Using Dynamic Features

Facial ageing analysis based on 4D data (3D plus time) is much more robust to pose changes and illumination variations than using 2D image and video. The purpose of this investigation was to measure the effects of age and gender related facial changes using dynamic 3D facial scans. Experiments were carried out on the subjects, who were divided into two groups by age (15-30 years and 31-60 years). Each group was further subdivided by gender. 3D scans of the subjects were processed to extract facial features which were tracked through the duration of the data capture. Subsequently, a set of dynamic features were computed from these facial features, as well as static features for comparison. Two-way multivariate analysis of variance (MANOVA) of these features demonstrated that statistically significant age and gender related differences could be detected. We show that 3D facial dynamics provide more useful information than static features for the characterisation of smiles. Khtam Al-Meyah et al. / Procedia Computer Science 112 (2017) 790–799 791 * * face characteristics, is an essential element of facial analysis. Although there has been a great deal of progress in face analysis in the last few years, many problems remain unsolved. The design of algorithms that are effective in discriminating between males and females, or classifying faces into different age categories is still a major area of research. Moreover, research on analysis of facial dynamics encounters many challenging problems, especially in the context of ageing. However, gender differences in age-related change in the 3D dynamics of smile have been, to date, unexplored. A lack of suitable datasets in particular proving to be the limiting factor. To address this issue we have performed a study using 3D videos of closed mouth smiles for 80 subjects to determine whether ageing affects smile expressions. We have investigated and shown that the lips becoming less elastic and mobile with increased age is a factor, and we investigated whether gender played a role in the smile dynamics. 2. Related Work In recent years, researchers have attempted to quantify facial ageing effects through various strategies. Ageing has been considered as the fourth dimension. The human face undergoes many skeletal and soft tissue cellular changes, due to age progression which consequently affects its functional behaviour . Studies have shown that lips become less elastic and less mobile with ageing. Oral structures such as teeth and periodontium also change with age. Due to these changes the smile can be affected. Moreover, facial movements differ between the genders, especially in adulthood. Recent psychological studies show that men and women have different smile behaviour. However, in many studies on smile aesthetics, these results were not statistically tested. Geld and Desai et al recently studied age related changes in smiles using videos. In this work, gender differences in age related changes in smiles were not explicitly identified. Due to the lack of information concerning gender and age differences, Chetan et al. proposed a cross-sectional study to determine the trends and patterns between the different ages groups, and to determine if gender plays an important role. The findings from this study were: • There is a difference between male and female smiles, and smiles also change with age. • As individuals age, there is a loss of muscle tone which leads to a reduction in the height of smiles. • During a smile, females exhibit greater horizontal movement and males exhibit greater vertical movement. Facial dynamics yields further information allowing for more detailed analysis. This allows for greater discrimination across larger age differences. In the last decade, dynamic features of smiles (such as duration, speed, and amplitude of smiles) have received attention as opposed to morphometric cues to discriminate smiles. Cohn et al. analysed correlations between lip-corner displacements, head rotations, and eye motion during spontaneous smiles. In another study, Cohn and Schmidt reported that spontaneous smiles have a smaller onset amplitude of lip corner movement, but a more stable relation between amplitude and duration. Furthermore, the maximum speed of the smile onset is higher in posed samples, and posed eyebrow raises have higher maximum speed and larger amplitude, but have shorter duration than spontaneous ones. Linear discriminant classifier were proposed to distinguish between spontaneous and deliberate enjoyment smiles using duration, amplitude, and duration amplitude measures of smile onsets. They analysed the significance of the proposed features and showed that the amplitude of the lip corner movement is a strong linear function of duration in spontaneous smiles, but not in deliberate ones. Krumhuber et al. studied the effects of dynamic attributes of smiles in human and synthetic faces, and whether facial dynamics tell us something about the genuineness of an emotion. The findings demonstrate that the participants who showed an authentic smile were perceived as more likeable, attractive and trustworthy than those who showed a fake smile or a neutral expression. Recently, Dibekliolu et al. proposed a method to use facial expression dynamics, and produced a database to explore the effect of dynamic features for age estimation. In another study the dynamic features are used for age estimation using a person’s smile and it was shown that the facial expression dynamics with appearance information are much more reliable for group classification tasks. Furthermore, experiments were carried out on disgust expression to evaluate the effectiveness of the proposed method in this study on a different expression and the results significantly improve the age estimation accuracy. In addition, Dibekliolu et al. proposed using the dynamics of eyelid, cheek, and lip corner movements to distinguish between spontaneous and posed smiles, where distance-based and angular features are defined in terms of changes in eye aperture. 792 Khtam Al-Meyah et al. / Procedia Computer Science 112 (2017) 790–799 Fig. 1. The Unified Texture Map (UTM) and Cleaned Mesh 3. Data Acquisition and Preprocessing 3.1. Data Collection Over the last eight years we have collected a 3D longitudinal database of 3D videos for our research. A 3dMD face capture system is used, which uses structured light to capture a textured 3D mesh at a rate of 60 frames per second. The meshes contain around 55,000 triangles, and capture the front and sides (beyond the ears) of the subjects’ faces. In this work, a set of 80 subjects who performed closed mouth smiles was used. The set was subdivided into two groups according to age: young people (Group 1: ages 15-30 years) and adults (Group 2: ages 31-60 years). Both groups contain 40 subjects equally split between males and females. The mesh data undergoes several preprocessing steps that are necessary to remove artifacts and improve subsequent processing steps. A fully automatic, robust preprocessing pipeline is used that cleans the 3D meshes and creates of a single-view texture map (UTM) from the raw data provided by the capture system which produces multi-view texture maps and meshes with duplicate vertices. The cleaning steps include removing non-manifold edges and vertices, along with small components and isolated vertices, and the holes created from the removal of these imperfections are identified and filled. The UTM as shown in Fig. 1 allows for the modification of vertices without incurring texture issues. After applying this processing, tracking and registration are carried out to create corresponding meshes and texture maps. The tracking and registration approaches are described in the following sections. 3.2. Facial Tracking and Registration The tracking approach we use requires the user to annotate a single mesh from the sequence, clicking on the ordered sequence of 41 facial feature points: eye corners, centre of upper eyelids, cheek centre, nose tip, lip corners, and contour of face. These points are then automatically tracked following the approach proposed by Vandeventer, and enable us to subsequently analyse the facial dynamics as shown in Fig 2. In each frame of a sequence, the face should to be aligned before feature extraction to eliminate any effects of head movement. This is carried out using Procrustes analysis to determine the rigid transformation. 3.3. Temporal Segmentation A smile can be defined as the upward movement of the lip corners, which corresponds to Action Unit 12 in the facial action coding system (FACS). Khtam Al-Meyah et al. / Procedia Computer Science 112 (2017) 790–799 793 Fig. 2. Tracked Smile Sequence Frames Most facial expressions are composed of three non-overlapping temporal phases, namely: the onset, apex, and offset. Onset is the initial phase of a facial expression and it defines the duration from neutral to expressive state. Apex phase is the stable peak period (which may be short in duration) of the expression between onset and offset. Offset is the final phase from expressive to neutral state. Following the registration step, the onset, apex, and offset phases of the smile are detected using the approach proposed by Dibeklioğlu et al. Smile amplitude is estimated as the mean amplitude of the right and left lip corners, normalized by the length of the lip. Let Dlip(t) be the value of the mean amplitude signal of the lip corners in frame t.