# Real-Time Analysis of Facial Expressions for Mood Estimation
## 3 Vision-Based Mood Estimation
### Identification of the Activation of Action Units  
As described in the previous section, AUs can be used for the interpretation of facial signals of mood. The former results led us to consider the use of this mapping in the estimation of facial expressions of mood. The motivation behind this is that it would be useful to know not only the emotions that the person is experiencing in an instant of time, but also their mood during certain interval of time. This could open a wide range of new possibilities for human-computer interaction (HCI) applications that deal with the affective state of the user. [PERSON] et al. developed a generic person-independent facial tracker that exhibits a high degree of robustness towards variations frequently encountered in real images [31]. We use this tracker to capture the motion of facial points, see Figure 2. This method uses a parametric shape model, which captures the statistical variation of a set of landmarks on the facial image. This model is built using Principal Component Analysis of training images. Next, for each landmark, a function returns a response map indicating the probability that the landmark is located in a certain position based on the result of a previous iteration  
Figure 1: Flowchart with the steps of processing. or an initialization value. To estimate the final position of the landmarks, an adjustment is performed that jointly searches for the maxima in the response maps, maintaining the constraints given by the shape model. This centrally characterizes the problem, which is why the authors define it as a constrained local model (CLM). The advantage over other proposals of facial features tracking is that this tracker robustly captures the motion of 66 facial points. Similarly to [7], from the tracked points we develop a set of distance-based rules to recognize the AUs. First, we define a set of facial parameters based on distances between tracked points corresponding to facial features, see Table 2.  
In this case, our strength is that, in comparison with the classical used data sets such as the CK+ and the MMI, we work with long sequences (more than 90 s), typical of a real study of spontaneous expressions. Therefore, the facial parameters for the AUs recognition rules are automatically learned from the tracked points positions in the first seconds of each sequence, when the user rests its face in neutral position. That is, let \(m\) be the number of frames used for learning (typically 25 frames, i.e., 1 s) and \(FPi=\{FPi^{(1)},\ldots,FPi^{(m)}\}\) the samples for the i-th facial parameter. We compute the maximum likelihood values for the mean, \(\mu_{i}\), and the variance, \(\sigma_{i}^{2}\), of a probabilistic gaussian density function, \(\mathbf{p}(FPi)=G(\mu_{i},\sigma_{i}^{2})\), which models the rest values of the facial parameters, by means of Equations (1) and (2). \[\mu_{i}=\frac{1}{m}\cdot\sum_{j=1}^{m}FPi^{(j)}. \tag{1}\]  
\[\sigma_{i}^{2}=\frac{1}{m}\cdot\sum_{j=1}^{m}(FPi^{(j)}-\mu_{i})^{2}. \tag{2}\] \begin{table}
\begin{tabular}{c c} \hline \hline
**Facial Parameter** & **Description** \\ \hline FP1 & Inner eyes distance (SCALE) \\ FP2 & Mean eyebrow–eveline distance (right) \\ FP3 & Inner eyebrow–eveline distance (right) \\ FP4 & Mean eyebrow–eveline distance (left) \\ FP5 & Inner eyebrow–eveline distance (left) \\ FP6 & Open eye distance (right) \\ FP7 & Open eye distance (left) \\ FP8 & Horizontal mouth width \\ FP9 & Horizontal mouth height \\ FP10 & Vertical distance between mouth and mouse \\ FP11 & Vertical distance between chin and mouse \\ FP12 & Lips distance \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Facial parameters extracted from the tracked facial points. Figure 2: (a) Facial landmarks. (b) Face tracker. This model is used to detect that a significant change was produced in the facial parameter if \(\textbf{p}(FPi)<0.05\), where \(\textbf{p}(FP_{i})\) is computed from Equation (3).