# Viewpoint Robustness of Automated Facial Action Unit Detection Systems
## 3 Results  
To evaluate the robustness of the automatic AU detection systems to changes in angle, biserial correlations between manual FACS coding and the machine output for each facial image were calculated, as stated above. In the ANOVA model, angle was the within-group factor (0\({}^{\circ}\), 15\({}^{\circ}\), 30\({}^{\circ}\), and 45\({}^{\circ}\)) and machine type was the between-group factor (FaceReader, OpenFace, and Py-Feat), as also stated above. The number of images differed among angles because FaceReader sometimes failed to fit or compute facial data (0\({}^{\circ}\): 72 images, 15\({}^{\circ}\): 71 images, 30\({}^{\circ}\): 69 images, and 45\({}^{\circ}\): 60 images). The ANOVA (Figure 2) showed a main effect of system (\(F\)(2, 536) = 22.07, \(\eta^{2}_{p}\) = 0.08, \(p\) < 0.001). Multiple comparisons using [PERSON]'s modified sequentially rejective Bonferroni procedure indicated that the correlation coefficient for OpenFace was higher than that for the other two systems (_ts_ > 3.46, \(p\)s < 0.001). The correlation coefficient was also higher for FaceReader than Py-Feat (\(t\)(268) = 2.93, \(p\) < 0.004, _Hedge's_ \(g\) = 0.25, 95% _CI_ = 0.08-0.42). Machine performance significantly differed by facial angle (\(F\)(3, 268) = 9.82, \(\eta^{2}_{p}\) = 0.10, \(p\) < 0.001). Performance at 45\({}^{\circ}\) was significantly lower than at other angles (_ts_ > 3.69, \(p\)s < 0.001), while no significant performance difference was observed at any other angle (_ts_ < 1.23, \(p\)s > 0.66). Figure 2: Average biserial correlation values between manually coded and machine-predicted action units (AUs) by angle. Error bars represent standard error. There was also a significant interaction effect between machine and angle (\(F(6,536)=8.37\), \(\eta^{2}_{p}\) = 0.09, \(p\) < 0.001). For OpenFace and FaceReader, there was a significant main effect of angle (\(F\)s > 12.51, \(\eta^{2}_{p}\)s > 0.12, \(p\)s < 0.001) but not for Py-feat (\(F(3,268)=0.48\), \(\eta^{2}_{p}\) = 0.01, \(p\) = 0.70). For OpenFace, there was no significant difference in performance between 0\({}^{\circ}\) and 15\({}^{\circ}\), or between 30\({}^{\circ}\) and 45\({}^{\circ}\) (ts < 2.17, \(p\)s > 0.06). However, in the other comparisons, the correlation coefficient gradually decreased from 0\({}^{\circ}\) to 45\({}^{\circ}\) (ts > 2.74, \(p\)s < 0.02). For FaceReader, the performance at 45\({}^{\circ}\) was significantly lower than that at the other angles (ts > 5.17, \(p\)s < 0.001), but there were no other significant differences (ts < 0.19, \(p\)s = 1.00). To confirm that the facial particularities of the individual being tested affected the results, the analysis was repeated with each participant removed. The result showed a significant effect similar to the above (main effect of system \(F\)s > 15.74, \(p\)s < 0.000 and facial angle \(F\)s > 7.60, \(p\)s < 0.000; interaction effect \(F\)s > 6.86, \(p\)s < 0.000). The estimated performance accuracy for each individual's data is depicted visually in Supplementary Materials. Figure 3 shows the average AUC values by AU. The receiver operating characteristic curves are available online ([[https://bit.ly/2](https://bit.ly/2) ZKaT22]([https://bit.ly/2](https://bit.ly/2) ZKaT22) (accessed on 25 November 2021)). The AUC values for the facial angles varied among the AUs. A sharp reduction in AUC values was noted at 45\({}^{\circ}\) for FaceReader. The variation by angle was relatively low for AUs 7 (lid tightened), 12 (lip corner pulled), and 25 (lips parted), excluding at 45\({}^{\circ}\) for FaceReader. Regardless of angle, all three systems detected AUs 12 and 25 accurately but had difficulty in detecting AU 7. For AU 9 (nose wrinkled), OpenFace outperformed Py-feat and FaceReader. Py-feat had a significantly lower AUC value for AU 4 (brow lowered) and higher scores for AUs 5 (upper lid raised) and 24 (lip pressed) than the other two systems. Importantly, in contrast to OpenFace and FaceReader, AUs in the frontal view (0\({}^{\circ}\)) were not easily detected by Py-Feat.