## 3 Results  
### Test-Retest Reliability  
#### 3.1.1 Test-Retest Differences between First and Second Test Data Series  
As a first approach to analyzing test-retest reliability, we compared data pairwise from the first (test\({}_{1}\)) and second (test\({}_{2}\)) EFE FR8 analyses of the same videorecording for each subject. Since the recordings differed in length, the number of data points generated by EFE analyses also differed between subjects. For any given subject, we had \(<\) length in seconds \(>\) times \(<\) frames/second \(>\) times \(<\) emotions: 8 \(>\) data points for both test\({}_{1}\) and test\({}_{2}\). In contrast with the widespread belief reflected by the literature that algorithmic AEFEA has total test-retest reliability, in 12 of 60 of our cases (subjects), the data series gained from test\({}_{1}\) and test\({}_{2}\) were nonidentical. In these cases, test-retest differences affected a considerable proportion of data points. We calculated the ratio of differing data points between test\({}_{1}\) and test\({}_{2}\) outputs for each subject; Figure 1 shows the resulting distribution.  
For exploratory purposes, we calculated the mean differences for each subject and emotion, based on individual test\({}_{1}\)-test\({}_{2}\) difference distributions, and then calculated group-level descriptive statistics on each set of individual means; see Appendix A Table A2. Overall, the means of the individual test\({}_{1}\)-test\({}_{2}\) differences were very small, on the order of magnitude of \(10^{-3}\)-\(10^{-7}\) for the eight emotions. The SD values were on a similar order of magnitude--\(10^{-3}\)-\(10^{-6}\).  
To explore the individual patterns of the magnitudes of differences between the test\({}_{1}\) and test\({}_{2}\) output data series, for each subject with nonidentical test\({}_{1}\)-test\({}_{2}\) datasets and for each emotion, we calculated the individual distributions of these different magnitudes. A total of 12 (subjects) \(\times\) 8 (emotions) = 96 frequency distributions were thus obtained. For illustration, such difference distributions are shown for two subjects in Appendix A Tables A3 and A4. In these two cases, similarly to other subjects showing test\({}_{1}\)-test\({}_{2}\) differences, the means of the individual emotion difference distributions were low.  
For the 12 subjects with nonidentical test\({}_{1}\)-test\({}_{2}\) results, we checked the number of FIND_FAILED (for the failure of finding the face on a video frame) and FIT_FAILED (for the failure of fitting the face model onto the face on a video frame) error signals in the two  
Figure 1: Histogram on the 12 subjects with nonidentical data series when comparing test\({}_{1}\) and test\({}_{2}\) datasets, along differing data points, as a percentage of all data points.  
subsequent test runs. Importantly, these error signals--and consequently the number of valid data points--are always fully correlated across the eight emotion dimensions in any dataset.  
On the subsequent runs (tests), these error indices were almost totally stable for each of the 12 subjects. The differences in their absolute frequencies were not above 3 in any of the 12 cases (cf. the mean total number of data points = 26,139). The FIND_FAILED index was more stable than the FIT_FAILED index (showing difference only in one case in the twelve-case cohort, and only in one data point). This clearly shows that imperfect test-retest reliability did not arise from different distributions of these error signals in the corresponding datasets.  
Clearly, these error signals were perfectly stable for each of the 48 subjects with identical test\({}_{1}\)-test\({}_{2}\) datasets.