From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent
This paper describes the results of a research project aimed at implementing a realistic 3D Embodied Agent that can be animated in real-time and is believable and expressive: that is, able to coherently communicate complex information through the combination and the tight synchronisation of verbal and nonverbal signals. We describe, in particular, how we animate this Agent (that we called Greta) so as to enable her to manifest the affective states that are dynamically activated and de-activated in her mind during the dialog with the user. The system is made up of three tightly interrelated components: A representation of the Agent Mind: this includes long and short-term affective components (personality and emotions) and simulates how emotions are triggered and decay over time according to the Agent's personality and to the context, and how several emotions may overlap. Dynamic belief networks with weighting of goals is the formalism we employ to this purpose. A mark-up language to denote the communicative meanings that may be associated with dialog moves performed by the Agent. A translation of the Agent's tagged move into a face expression, that combines appropriately the available channels (gaze direction, eyebrow shape, head direction and movement etc). The final output is a 3-D facial model that respects the MPEG-4 standard and uses MPEG-4 Facial Animation Parameters to produce facial expressions.
embodied agent communication modelling nonverbal signals emotion triggering