Contact: Confidential This work is supported by: AN INTERFACE WITH COMPUTER-GENERATED FACIAL EXPRESSIONS AS AN ALTERNATIVE FOR MOOD SELF-REPORTS IN AN EMA CONTEXT Hristo Valev[a,b], Tim Leufkens[a], Joyce Westerink[a,c], Corina Sas[b] Philips Research [a] Lancaster University [b] Technical University Eindhoven [c] hristo.valev@philips.com h.valev@lancaster.ac.uk Abstract Traditionally numerical scales, colors, abstract representations or animated characters have been for reporting one’s own mood. While working in practice, they do not provide a universal stable representation between participants. Abstract representations or colors rely on subjective interpretation, while numerical or animated characters require mapping the subjective feeling of mood onto a scale, which is also differentially interpretable. The described approach relies on computer-generated facial expressions to span a scale, which can be used for mood self-reports. Generating facial expressions [1] O. Langner, R. Dotsch, G. Bijlstra, D. H. J. Wigboldus, S. T. Hawk, and A. van Knippenberg, “Presentation and validation of the radboud faces database,” Cogn. Emot., vol. 24, no. 8, pp. 1377–1388, 2010. Participants 37 healthy participants in the Netherlands (20 male and 17 female) with an mean age of 30 years old were recruited. All participants provided an informed consent form prior to their recruitment. Relationship between type of assessment, oscillations and assessment duration On the left hand side you see all participants (N=37) that took part in the study. The amount of assessments is represented by the thickness of the lines. Each provided assessment with the facial expression interface and the slider is depicted in the first band. For each assessment we captured the ‘oscillations’ or turning points in the pattern of their selection, represented in the second band. The final band depicts the time it took to provide an assessment, where the label describes the higher bound of the duration except for the highest value. Graph has been generated with raw-graphs ("How to make an alluvial diagram", by RAWGraphs Team. Licensed under CC BY-NC-SA 4.0. Accessed: May 12, 2020, from https://rawgraphs.io/learning/how-to-make-an-alluvial-diagram/) Results This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska- Curie grant agreement No 722022. Discussion • We have recruited healthy participants, which is why as expected the distribution is heavily represented in the happiness dimension. This may have unforeseen implications for sadness. • The high correlation between both assessments gives us the insight that both assessment methods can be used interchangeably with high certainty that they will yield comparative results. • Albeit not conclusive, the significantly higher variance captured by the facial expression interface could mean that it describes better the modality of mood. Furthermore, two-thirds of the assessments provided with the facial expression interface featured more than one turning points in their trace. This is indicative of the desire of the participants to be more particular in their input. • The quality of the generated facial expressions was good as they were easily understood, even though the only real facial expressions used to train the model were the expressions for extreme happiness (i.e. 200), extreme sadness (i.e. 0) and the neutral expression. Considerations • The happiest or saddest facial expressions might not necessarily be representative of their respective extremes, which could explain the offset between the distributions for the facial expression and slider assessments. • Timestamps were only recorded during the interaction with the assessment elements. This might have reduced the time required to provide an assessment with the slider as a single tap would have a 0 seconds duration. AffecTech: Personal Technologies for Affective Health Distribution plot of a