###### Abstract  
Automated emotional facial expression analysis (AEFEA) is used widely in applied research, including the development of screening/diagnostic systems for atypical human neurodevelopmental conditions. The validity of AEFEA systems has been systematically studied, but their test-retest reliability has not been researched thus far. We explored the test-retest reliability of a specific AEFEA software, Noldus FaceReader 8.0 (FR8; by Noldus Information Technology). We collected intensity estimates for 8 repeated emotions through FR8 from facial video recordings of 60 children: 31 typically developing children and 29 children with autism spectrum disorder. Test-retest reliability was imperfect in 20% of cases, affecting a substantial proportion of data points; however, the test-retest differences were small. This shows that the test-retest reliability of FR8 is high but not perfect. A proportion of cases which initially failed to show perfect test-retest reliability reached it in a subsequent analysis by FR8. This suggests that repeated analyses by FR8 can, in some cases, lead to the \"stabilization\" of emotion intensity datasets. Under ANOVA, the test-retest differences did not influence the pattern of cross-emotion and cross-group effects and interactions. Our study does not question the validity of previous results gained by AEFEA technology, but it shows that further exploration of the test-retest reliability of AEFEA systems is desirable. a 2022 2022 2022 2022 2022 2022 2022 2022 2022 202 numerous attempts to utilize AEFEA technologies in applied fields, such as researching consumer behavior, user experience, and marketing [5; 8] and in diagnostic technologies in psychiatry [6; 9; 10]. While their validity has been widely investigated, no study has been published on their test-retest reliability; the latter is an equally important property of any tool for collecting data on human behavior. Thus, the present study empirically explored the test-retest reliability of a specific AEFEA system. As AEFEA technology seems to have great potential in screening and diagnostic technologies for neurodevelopmental conditions, we used data from typically developing children and children with autism spectrum disorder (ASD). Our study suggests that the test-retest reliability of Noldus FaceReader 8.0 (FR8; by Noldus Information Technology; Wageningen, The Netherlands) is very high but not perfect. Our findings do not question the validity of previous results gained by AEFEA technology, but they demonstrate that further exploration is needed to gain a better understanding of this key characteristic of AEFEA systems. Test-retest reliability showed highly similar patterns in the groups of children with ASD and with neurotypical development. [PERSON]'s Facial Action Coding System (FACS) [11] is a widely used framework for identifying emotional facial expressions (EFEs) and assessing their intensities by human coders based on the anatomical features of the face. It also serves as the foundation for making emotion estimations in several AEFEA technologies [12], as it is highly valid and reliable when used by human coders [13; 14]. It can be elaborated precisely enough to be implemented in computational algorithms [7; 15]. It defines action units (AUs) anatomically, that is, through different muscle movements of the face, to identify specific EFEs as constituted by their specific combinations. Intensity estimations for any emotional expression are derived from the intensities of their component AUs, that is, from the amplitudes of the relevant muscle movements. At present, a few commercially available versions of AEFEA technology exist, based on the FACS, including the AFFDEX (developed by Affectiva, Boston, MA, USA, distributed by Affectiva and iMotions, Copenhagen, Denmark) [16; 17; 18]; the FACET (developed by Emotient, San Diego, CA, USA, distributed by iMotions, Copenhagen, Denmark) [19]; and the Noldus FaceReader (developed by VicarVision, Amsterdam, The Netherlands, distributed by Noldus Information Technology) [20].