# Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
## 5 Experimental Results
### Assessment on TD Children
Figure 5: Scores computed for the production of Anger expressions by TD children. upper face part instead). These happened since, in correspondence to the requests of sad expression, some children occluded the mouth but without affecting the upper face part. Similar conclusions can be drawn for some spots corresponding to requests of fear expressions (Figure 4) whereas this problem was never encountered during requests of anger expression (Figure 5). Some clippings in children's faces corresponding to the aforementioned situations, in which the system was not able to produce reliable results in landmark positioning, are shown in Figure 6. In the reported patches, the landmarks were wrong positioned due to the presence of the hands in front of the mouth. In particular, in Figure 6a,b, two examples in which the child affected the positioning of landmarks for both upper and lower face (for happy and sad expression respectively) are shown. Please observe as the child's fingers press on his cheeks largely changing his facial features. In Figure 6c,d two examples in which the occlusions by the hands affected only the lower part of the face are reported instead. To better evaluate the automatic classification of the ability in producing facial expressions, a quantitative comparison with the annotations provided by professionals was performed. It easy to understand that the classification of videos in input, as containing the expected expression or not, depends on the decision threshold on gathered numerical scores for lower and upper face parts. As a consequence, a study related to this crucial parameter of the automatic classification model was preliminary made. Figure 7 reports, in the topmost graphs, the precision and recall curves for the classification of expressions while varying decision threshold. Each figure is related to a different expression and, in each of them, bottom graphs represent the corresponding curves for the F1-score. The best value for the decision threshold, i.e., the value that maximized the related F1-score, is indicated on the \(x\)-axis. Figure 6: Some situations that generated miss matches between the outcomes of the system and the annotations of experts: (**a**) the child affected the positioning of landmarks for both upper and lower face parts while performing a happy expression; (**b**) the child affected the positioning of landmarks for both upper and lower face parts while performing a sad expression; (**c**) the child affected the positioning of landmarks for only the lower part of the face while performing an anger expression; (**d**) the child affected the positioning of landmarks for only the lower part of the face while performing a fear expression. According to the above, Table 8 reports classification performance for TD children when the best decision threshold (in terms of F1-score) for each expression was used. Values in Table 8 demonstrate that the proposed pipeline provides very affordable outcomes, i.e., that computed scores are strictly correlated with annotations provided by professionals. It is very encouraging to observe the overall performance in terms of F1-score (0.84), precision (0.85) and recall (0.83) considering the situational difficulties due to non-collaborative behaviors of children. Moreover, in the evaluation of the system, the occurrence of some subtle executions should be considered in which even the psychologists made a decision for the annotations only after an inter-judge agreement given the initial divergence in judgment. This adds value to the automatic classification system as it highlights its usefulness in the specific application context. This aspect will be further discussed later in Section 7.