# Real-Time Analysis of Facial Expressions for Mood Estimation
## 3 Vision-Based Mood Estimation
### Mood Estimation  
In order to build a robust mood estimation, we define each mood pattern as a standard probability distribution associated with the activation of the AUs associated with each mood (see Table 5). Let \(p_{ij}\in[0,1]\) be the contribution of each AU \(j\) to the mood \(i\), where if an AU is highly determinant, it is assigned the value 1, whereas if it is not important for a certain mood, it is assigned the value 0. Then, a pattern, \(\mathbf{p}_{i}\), is constructed for each mood with the following values as the AU contributions for the i-th mood:  
\[\mathbf{p}_{i}=C_{p}\cdot(p_{i1},p_{i2},...,p_{in}), \tag{4}\]  
where \(C_{p}\) is a normalization constant for imposing the condition that \(\sum_{j}p_{ij}=1\), and \(n\) is the number of AUs. From a new video sequence of \(W\) images, which are consecutive in the sequence, the activated AUs can be determined by means of the previously described method (see Section 3.1). By repeating that comparison with all the images, it is possible to determine if an AU has been activated in one or in several frames. In other words, an occurrence or \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**AU** & **TP** & **FP** & **TN** & **FN** & **Spec** & **Acc** \\ \hline AU1 & 31 & 669 & 5338 & 4 & **88.42** & **88.42** \\ AU2 & 27 & 871 & 5171 & 3 & **85.58** & **85.61** \\ AU4 & 168 & 306 & 5525 & 73 & **94.75** & **93.76** \\ AU5 & 58 & 238 & 5761 & 15 & **96.03** & **95.83** \\ AU6 & 252 & 732 & 4981 & 107 & **87.19** & **86.18** \\ AU10 & 243 & 1306 & 4518 & 5 & **77.58** & **78.41** \\ AU12 & 434 & 1128 & 4435 & 75 & **79.72** & **80.19** \\ AU15 & 7 & 676 & 5367 & 22 & **88.81** & **88.50** \\ AU25 & 775 & 1139 & 3792 & 366 & **76.90** & **75.21** \\ AU26 & 216 & 1251 & 4433 & 172 & **77.99** & **76.56** \\ AU43 & 272 & 344 & 5361 & 95 & **93.97** & **92.77** \\ \hline Total & 2483 & 8690 & 54,682 & 937 & **86.29** & **85.59** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Action Units detection evaluation results. \begin{table}
\begin{tabular}{c c} \hline \hline
**Mood** & **Action Units (AUs)** \\ \hline
**Exuberant** & AU6, AU5, AU12, AU25, AU26 \\
**Bored** & AU1, AU2, AU4, AU15, AU43 \\
**Docile** & AU1, AU2, AU12, AU43 \\
**Hostile** & AU4, AU10, AU5, AU15, AU25, AU26 \\
**Anxious** & AU1, AU2, AU4, AU5, AU15, AU25, AU26 \\
**Relaxed** & AU6, AU12, AU43 \\
**Dependent** & AU1, AU2, AU5, AU12, AU25, AU26 \\
**Disdainful** & AU4, AU15, AU43 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: AUs describing facial expressions of mood (from [PERSON] et al. [29]). relevance value, can be obtained for each AU. Each of those occurrence values for each AU\({}_{j}\) is referred to as \(q_{j}\), which is calculated by means of the expression of Equation (5). \[q_{j}=\frac{C_{q}}{W}\cdot\sum_{k=1}^{W}s_{kj}. \tag{5}\]  
Equation (5) is a normalized mean of the AU\({}_{j}\) ocurrence in the complete video. That is, \(s_{kj}\) represents the activation or non-activation of the corresponding AU\({}_{j}\) in the image corresponding to frame \(k\). If the AU\({}_{j}\) has been activated, \(s_{kj}\) is assigned the value \(s_{kj}=1\), whereas if it has not been activated, \(s_{kj}=0\). Finally, \(C_{q}\) is a normalization constant for imposing the condition that \(\sum_{j=1}^{n}q_{j}=1\). For determining the similarity between the estimated AU\({}_{j}\) distribution, \(q_{j}\), and each mood pattern, \(\mathbf{p}_{i}\), the Bhattacharyya coefficient [35], \(D_{i}\), is used for each mood \(i\) according to Equation (6). \[D_{i}=\sum_{j=1}^{n}(p_{ij}\cdot q_{j})^{1/2}. \tag{6}\]  
Finally, the estimated mood is the maximum of the computed similarities, \(M=\max_{i}D_{i}\). A reliable and robust mood recognition method is thereby achieved, where image analysis is performed in sequences captured by the camera, which allow for evaluating dynamically the contribution of the AUs to the mood of the subject.