Making Simulated Faces Come Alive (2016)

Our behaviour emerges as the result of many systems interacting at different scales, from low level biology to high level social interaction. Is it possible to create naturalistic explanatory models which can integrate these factors? We describe the general approach and design of a framework to create autonomous expressive embodied models of behaviour based on affective and cognitive neuroscience theories, sharing similar goals with current research in developmental robotics and embodied cognition, but with emphasis on computer graphics and face to face interaction. We aim to create a 'large functioning sketch' of several fundamental aspects of behaviour affecting the face to explore emergence from interaction of low level and high level systems in a top-down bottom up approach.A key feature of the approach is (given the constraints of the medium), to create as naturalistic models as possible in order to elicit and respond to the appropriate behaviours from the user, involving both sensing and synthesis of visual and auditory stimuli. The models are intended to be as autonomous as possible, so interactions are unscripted and co-created. In order to develop an overall generative model of behaviour that, where possible, remains grounded in biologically plausible models and thus has more explanatory power than statistical approaches, we are developing BL, a modular Lego-like neural network modelling language, sensing and visual simulation framework, to facilitate integration of a range of emerging and established models from diverse sources such as cognitive science, developmental psychology, computational neuroscience and physiology with high quality interactive computer graphics. As a result of combining many diverse models, the neurobehavioural architectures created in BL often form large, complex networks and simulations form complex dynamic systems justifying a visual computing approach in which systems and subsystems can be explored visually as the simulation proceeds. As the users behaviour as an interactive partner is a key factor in social learning, it is important to capture and visualize the effect of internal and external events on the networks.We describe progress along this path with an example, 'BabyX', a psychobiological model of a virtual infant we are developing. While we are exploring drivers of facial behaviour, we also expect this foundational approach has the potential for more 'human' computer interfaces.We also describe our work on a face simulator designed to provide a more realistic face, and an increased biological basis, to existing conversational agents. For HCI tasks (and also HRI), we are building on the same underlying computational platform the 'Auckland Face Simulator' to produce highly realistic avatars capable of real-time interaction. These faces are designed to be both used as stimuli for psychological research and also to provide a realistic inferface for third party virtual agent and AI applications. BL allows internal variables of the avatar's nervous system to be controlled at any level, e.g. from muscles to affective circuits.A live demonstration of 'BabyX' and the 'Auckland Face Simulator' will be given during the Keynote presentation.