The Great Deceivers: Virtual Agents and Believable Lies

The Great Deceivers: Virtual Agents and Believable Lies Jo˜ao Dias (joao.dias@gaips.inesc-id.pt) INESC-ID and IST, Universidade T´ecnica de Lisboa, Portugal Ruth Aylett (r.s.aylett@hw.ac.uk) Heriot-Watt University, Edinburgh, Scotland Henrique Reis and Ana Paiva (henrique.reis@ist.utl.pt, ana.paiva@inesc-id.pt) INESC-ID and IST, Universidade T´ecnica de Lisboa, Portugal Abstract This paper proposes a model giving Theory of Mind (ToM) capabilities to artificial agents to allow them to carry out de- ceptive behaviours. It describes a model supporting an N-level Theory of Mind and reports a study to assess whether equip- ping agents with a two-level ToM results in them being per- ceived as more socially intelligent than agents with a single- level ToM. A deception game being developed for intercultural training of children, used for this study, is described. Finally, we report results from this study consistent with the hypothe- sis that a two-level Theory of Mind better supports agents in deceptive behaviour. Keywords: Virtual Agents; Theory of Mind; Deception Introduction The work reported in this paper arises from the use of syn- thetic graphical characters interacting in rich virtual worlds. These may be required for interactive drama applications (Mateas & Stern, 2003), or for story-based education and training applications (Paiva et al., 2004) (Swartout et al., 2006). A key criterion for success is that such agents be be- lievable, that is lead a user, or viewer, to feel that they have an inner life of their own, with goals, motivations and emo- tions, and are in some sense ’alive’ (Bates, 1994). Thus inter- action between such characters must display features related to human-human interaction; whether the actions they carry out, their emotional expressions, ability to exhibit empathy, or non-verbal as well as verbal communications. Such features must be contextually appropriate, and in order to achieve this, characters may be driven by an architecture uniting cognitive and affective models, for example using a cognitive appraisal approach (Dias & Paiva, 2005) (Marsella & Gratch, 2009). Computationally implemented cognitive appraisal models are often naive, assuming entirely open behaviour, sometimes referred to as meeting the sincerity condition (Searle, 1976). However, this is unusual in everyday human-human commu- nication where deception often occurs. This may be as sim- ple as masking anger in front of a social superior or fear in front of a child on a dark night (Rosis, Pelachaud, Poggi, Carofiglio, & Carolis, 2003), (Prendinger & Ishizuka, 2001), or as complex as deliberately misleading or lying to another person in order to gain an advantage. Deceptive behaviour includes not only the generation of false beliefs in others but also the claiming of desired identities, the exchange of non-existent emotions, and the communication of false pref- erences or opinions (Wyer & Epstein, 1996). Thus decep- tion can be seen as a human-like characteristic that would enhance the believability of synthetic characters portrayed in real world social situations. A Theory of Mind (ToM) process allows an agent to at- tribute an artificial mental state to another agent and reason about it. In a single-level ToM, agent A can represent only its belief about what an agent B is thinking; an agent C that can not only model what B is thinking but can also model what agent B thinks about C has a two-level ToM. In this paper we investigate the hypothesis that an agent with a single-level ToM will be less successful in believable deception then an agent with a two-level ToM. Deception cannot be investigated in abstract but requires a concrete scenario. Our work uses an interactive game played by and with autonomous graphi- cal characters. This is based on the popular game Mafia, or Werewolf, described below, in which deception is fundamen- tal to successful play. The characters are implemented with a cognitive appraisal-based architecture (Dias & Paiva, 2005) that includes a deliberative mechanism and has been extended to support an N-level ToM mechanism. Background and Related Work We define a “lie” as a direct communicative act that an agent performs to deceive another agent. We consider deception through verbal mechanisms - speech acts - though deception may also be achieved through non-verbal mechanisms. De- ception has been widely studied in AI, though usually with disembodied software agents. GOLEM (Castelfranchi & deRosis, 1998) is based on the blocks world of AI planning research. Goals conflict, since agents aim to build different structures from the same avail- able blocks. Agents can achieve goals through their own ac- tions or by asking for “help” from others. Agents have task delegation and adoption preferences and different capabili- ties, used to plan their actions based on their knowledge of other agents. Deception is instrumental, resulting only from goal conflicts, though it extends to deception about capabil- ities, goals or personality. However, agents in GOLEM can only produce lies within this limited scope. They cannot for example lie about the requests they have made or plan to make. This would require second order reasoning about the reasoning of other agents, which is not present here. De Rosis and Carofiglio (deRosis, F; Carofiglio, V; Gras- sano & Castelfranchi, 2003) focus on the communicative per-