# BAAP: A Behavioral Animation Authoring Platform for Emotion Driven 3D Virtual Characters

Ling Li, Gengdai Liu, Mingmin Zhang, Zhigeng Pan, Edwin Song

► To cite this version:

Ling Li, Gengdai Liu, Mingmin Zhang, Zhigeng Pan, Edwin Song. BAAP: A Behavioral Animation Authoring Platform for Emotion Driven 3D Virtual Characters. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.350-357, 10.1007/978-3-642-15399-0_37. hal-01055625

HAL Id: hal-01055625

https://inria.hal.science/hal-01055625

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
BAAP: A Behavioral Animation Authoring Platform for
Emotion Driven 3D Virtual Characters

Ling Li<fnref n="1" />, Gengdai Liu², Mingmin Zhang<fnref n="1" />,<fnref n="1" />, Zhigeng Pan<fnref n="1" /> and Edwin, Song³,

¹ State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China
{liling, zmm, zgpan}@cad.zju.edu.cn

² Colllege of Computer Science, Xidian University, Xi’an, China
gdliu@xidian.edu.cn

³ Intel Asia-Pacific Research & Development Ltd. Shanghai, China
edwin song@intel.com

**Abstract.** Emotion, as an important aspect of human intelligence, has been playing a significant role in virtual characters. We propose an improved three-level structure of affective model as “personality-emotion-mood” for intelligent and emotional virtual characters. We also present the emotion state space, as well as the emotion updating functions, to generate authentic and expressive emotions. In order to achieve the complexity and variety of behaviors, we bring forward a behavior organizing structure as the behavior tree, which defines four kinds of behavior organizations as well as the behavior tag and behavior message, to manage virtual characters’ behaviors. At the end, we achieve an experimental platform BAAP, which prove our emotion model and behavior organizing structure to be effective and practical in generating intelligent and emotional behavioral animations.

**Keywords:** virtual character, affective computing, behavior tree, behavioral animation, authoring platform

# 1 Introduction

In previous work [1], we proposed a computational emotion model which can be incorporated into physiological and social components of emotions. We improve it in this paper and present an experimental platform BAAP. We mainly focus on two subjects: emotion modeling and behavior organizing. Based on the OCEAN model [2] and the OCC model [3], we propose an improved three-level affective structure as “personality-emotion-mood”. The emotion state space, as well as the emotion updating functions, is defined for generating authentic and expressive emotions.

To organize behaviors of virtual characters, we bring forward a structure as the behavior tree similar to Halo2 [4]. There are four kinds of organizations for executing behaviors: “Sequence”, “Selector”, “Repeat” and “Parallel”. By designing behavior

<footnote n="1">¹ Corresponding Author</footnote>
tags and behavior messages, we succeed to generate intelligent and emotional
behavioral animations.

## 2 Related work

Picard restates the definition of affective computing [5]. Information of personality and emotion can influence virtual characters over probability of choosing behaviors. Egges [6] presents the personality-emotion influence matrix to explain how each personality factor influences each emotion factor. Chittaro [7] defines an OCEAN-based model where behavior sequences are chosen from an animation library based on personality. The EMA model [8] combined a plan-based appraisal model with a detailed problem-focused model. However, one limitation among the previous work is the lack of interactions between physiology, cognition and social interaction.

Inspired by the psychological emotion models, researchers recognize the utility of computational models. Loyall [9] described an innovative system for authoring expressive, autonomous interactive characters. However, it focuses on building specific and unique believable characters, where the goal is artistic abstraction of reality, not biologically plausible behavior. Egges describe a generic model [10] and generate emotional, communicative and multiple character animations [11]. Perlin [12] create the autonomous digital actor to act out scenes using procedural animation techniques. All these works seek to create believable, emotional or social characters.

Behavior modeling and organizing are significant aspects of generating intelligent and emotional behavioral animations. Diller [13] focuses on behavior generations relevant for training applications. In the CreatureSmarts Architecture [14], robust, reactive, adaptable, honest, expressive, sensible and scalable creatures are sought to be built. Kallmann [15] presents a feature modeling approach to define behavioral information, paying special attention to the capabilities of interaction. Isla [4] uses hierarchical state machines and DAG [16] in Halo 2, where the use of state machines is regarded as less feasible for the long term and even proving difficult in generation games. The major problems are the complexity of state machines grow quickly when the behavior of a character becomes slightly more sophisticated.

## 3 Overview

<figure><img src="image_2.png" /></figure>
<caption>Fig. 1. Overview of BAAP.</caption>

Figure 1 shows the architecture of BAAP. In the first instance, a dialog system is provided for users to customize the virtual environment, for example, importing personal data (i.e. personality and emotion, which are described in detail in section 4) of virtual characters, customizing behavior tree based on the motion databases, and so on. Personal data of virtual characters is dealt with by the character controller.

After initialization, the animation generator begins to take up the control. It first setup the virtual environment (i.e., locate virtual characters and objects, read in behavior data of the customized behavior tree and set initial emotion states and behaviors of virtual characters), afterwards, it carries out the execution of the behavior tree, which traverses leaf and non-leaf nodes of the behavior tree and executes behaviors recorded on the leaf nodes to generate behavioral animations. The particular introduction of behavior tree is described in section 5.

# 4 Emotion modeling

## 4.1 Personality

Personality is closely related with emotion and mood and is modeled using vector p, with each element corresponding to a component of the OCEAN model [2] and restricted within the interval (0, 1):

$$p^T = \langle a_1, a_2, a_3, a_4, a_5 \rangle, \forall i \in \{1, 2, \dots, 5\}, a_i \in (0, 1) \quad (1)$$

where $a_i$ refers to each element of the OCEAN model.

## 4.2 Emotion

Emotion is formalized with an emotion state set of six elements (which are, happiness, sadness, anger, fear, surprise and disgust [17]) and specified using vector $e_t$ at time t with each element restricted within the interval (0, 1):

$$e_t^T = \langle b_1, b_2, b_3, b_4, b_5, b_6 \rangle, \forall i \in \{1, 2, \dots, 6\}, b_i \in (0, 1), t > 0 \quad (2)$$

where $b_i$ refers to each element of the emotion state set.

## 4.3 Emotion state Space

<figure><img src="image_3.png" /></figure>
<caption>Fig. 2. Emotion state Space.</caption>

The emotion state space is designed in BAAP. As shown in Figure 2, six components are divided into three groups (namely, {happiness, sadness}, {anger, fear} and {surprise, disgust}), with the former of each represents the active component and the latter represents the passive component.

The two components within the same group are independent of each other; however, they both hold either positive or negative relationships with components of other groups. Components of the same kind hold positive relationships with each other; while of different kinds hold negative relationships. For example, the active component “happiness” is independent of the passive component “sadness”, holds positive relationships with the active components “anger” and “surprise”, and negative relationships with the passive components “fear” and “disgust”.

## 4.4 Emotion updating

Different from the relative permanence of personality, emotion may change over time under rapid enhancing function $f_e$ and slow weakening function $f_w$:

$$f_e(b_i, u): b_i \leftarrow f(f^1(b_i)+u), i \in \{1, 2, ..., 6\}, u>0 \quad (3)$$

$$f_w(b_i, v): b_i \leftarrow b_i * (1 - e^{-1/vt}), i \in \{1, 2, ..., 6\}, v>0, t>0 \quad (4)$$

where u and v are the enhancing and weakening parameters respectively, $f^1$ is the inverse function of f described as follows:

$$f(x): x \leftarrow e^{-k_1/(x-k_2)}, k_1, k_2 > 0 \quad (5)$$

where $k_1$ and $k_2$ are parameters to determine the gradient of function f.

## 4.5 Mood

Mood is virtual character's subjective experience. In this paper, only active mood and passive mood are considered. Mood is described with vector $m$, with each component restricted within the interval $(0, 1)$:

$$m^T = \langle m_a, m_p \rangle, m_a, m_p \in (0, 1) \quad (6)$$

where

$$m_i = \log_3(\sum_3 3^{(b_i-1)}), i \in \{a, p\} \quad (7)$$

$m_a$ and $m_p$ denote the active and passive mood intensity respectively. $b_a$ denotes the active components; while $b_p$ denotes the passive components.

# 5 Behavior organization

<table><caption>Table 1. The organizations of behaviors in BAAP.</caption><thead><tr><th>organizations</th><th>descriptions</th></tr></thead><tbody><tr><td>Sequence</td><td>All the direct children should execute in turn.</td></tr><tr><td>Selector</td><td>At most one direct child is chosen to execute.</td></tr><tr><td>Repeat</td><td>All the direct children should execute in turn for several times.</td></tr><tr><td>Parallel</td><td>All the direct children should start execution at the same time.</td></tr></tbody></table>
Behavior is the primary approach to represent emotion. The organization, formulation and execution of behavior are elaborated in this paper. The behavior DAG [16] (directed acyclic graph), namely, behavior tree, is implemented in BAAP. The non-leaf node of a behavior tree is designed to determine how to execute its subtrees; while the leaf node is the pure behavior to achieve some essential work. There are four kinds of non-leaf nodes, defining four kinds of organizations shown in Table 1.

## 5.1 Behavior cinquain

There are five factors of behavior taken into account, namely *inflictor*, *sufferer*, *trigger*, *aim* and *state*, which compose behavior cinquain *bhv*:

$$bhv = <inf, suf, tri, aim, state> \qquad (8)$$

<figure><img src="image_4.png" /><figcaption>Fig. 3. Example Behavior tree.</figcaption></figure>

Figure 3 shows an example behavior tree, with one and only root node. The pink rectangles indicate leaf and non-leaf nodes; while the blue ones contain elements of pure behaviors. Moreover, the words recorded in each top blue rectangle (i.e. the leftmost “Bob”) denote the inflictor; while the second ones (i.e. the leftmost “Steve”) denote the sufferer; such words as “Tri: 2#” indicate that there are two components recorded in the tri element.

## 5.2 Behavior tag

It is difficult to prepare everything in detail when customizing a behavior tree, for the complexity of solving behavior factors will grow implausible if the tree becomes massive. Luckily, we have the behavior tag. The idea is not to keep all these specific factors in the behavior cinquain but encode some of them as a tag, which are ignored when customizing the behavior tree. Not until the behavior is executing will the tag be computed, in this case, all the details of the behavior can be obtained.

## 5.3 Behavior message

There is another problem, that is, once a behavior finishes execution, the situation of the virtual environment may have changed. Then, how to inform the changes?

Our method is through the behavior message, which does not appear in a static behavior tree structure, but is instead dynamically generated by the message handler
to a specific behavior node during the behavior tree is executing. Concretely, a
behavior calls the message handler after executing; the message handler, which
conserves current environment information, picks up this call and releases a message
containing the updated information; virtual characters will update their information
accordingly and respectively once receiving the message.

# 6 Results and Discussions

Two contrastive experiments, where virtual characters are customized with different personalities and emotions, are prepared in BAAP. To obtain convincing results, we make them share the same behavior tree; unfortunately, we are only able to affix two small segments of the tree because of its large scale, as shown in Figure 4.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Segments of the behavior tree built in BAAP.</figcaption></figure>

In Experiment 1, the personality of Bob is customized to be high in neuroticism, while Jane in openness and extraversion, Steve in agreeableness and Halley in conscientiousness. As for initial emotions, we customize Bob to be high in fear, surprise and disgust, Jane to be in anger, Steve to be in surprise and Halley to be neutral. The reason for Halley to be neutral is we intend to build a reference.

As shown in Figure 5, a segment of behavioral animations is generated through the execution of the behavior tree. The general process is: 1). Halley keeps on barking at the cake, which attracts Steve. 2). Steve fails to reach the cake and runs to Bob. 3). Steve asks Bob for help. 4). Bob refuses Steve's request. 5). Steve does not give up and runs to Jane. 6). Steve asks Jane for help. 7). Jane gets angry with Bob. 8). Bob feels dejected and walks to the cake. 9). Steve feels pleased for Bob's help.

<figure><img src="image_6.png" /><figcaption>Fig. 5. Procedure of Experiment 1.</figcaption></figure>

In Experiment 2, the personality of Bob is customized to be high in openness and agreeableness, while Jane in extraversion, Steve in agreeableness and Halley in conscientiousness. In contrast with Experiment 1, we customize Bob to be high in
happiness, Jane to be in anger, Steve to be in fear and Halley still to be neutral.

As shown in Figure 6, a contrastive segment of behavioral animations is generated through the execution of the behavior tree. The general process is: 1). Halley keeps on barking at the cake, which attracts Steve. 2). Steve fails to reach the cake and runs to Bob for help. 3). Bob approves Steve's request. 4). Bob feels happy and walks to the cake; Steve follows Bob with pleasure and excitement. 5). Bob takes the cake. 6). Steve feels satisfied with Bob's help.

<figure><img src="image_7.png" /><figcaption>Fig. 6. Procedure of Experiment 2.</figcaption></figure>

We achieve two contrastive segments of behavioral animations. With neurotic personality and disgusted emotion, Bob will refuse Steve's request (in Experiment 1), however, it will approve Steve's request if customized to be open and agreeable (in Experiment 2). Similarly, Jane and Steve will also behave differently with different personalities and emotions. Driven by personalities and emotions, our method enables virtual characters to present authentic and expressive behaviors.

Figure 7 shows the mood curves of virtual characters in Experiment 1 (the left picture) and Experiment 2 (the right picture). In the left picture, we are informed that Bob goes through an emotional process with twists and turns (the red curve); Jane keeps a mood curve with a sudden rise (the blue curve); Steve's mood curve arises right after Bob's first rise (the green curve). We get the opposite result in the right picture where Bob goes through a flat emotional process (the red curve) and Steve looks with twists and turns (the green curve).

<figure><img src="image_8.png" /><figcaption>Fig. 7. Mood curves of virtual characters in the two contrastive experiments.</figcaption></figure>

# 7 Conclusions and Future work

In this paper, an improved three-level structure of affective model as “personality-emotion-mood”, as well as a new behavior organizing structure behavior tree, is brought forward to generate intelligent and emotional behavioral. An interactive platform named BAAP is also constructed to allow users to customize virtual characters to achieve authentic and expressive emotions as well as behaviors. Our future work will focus on facial expressions and learnable abilities of virtual
characters. In addition, we believe that better behavioral animations will be generated
with better technology of motion controllers and physical simulations.

Acknowledgements

This research is co-funded by INTEL-University cooperation project and the 863
project (Grant NO: 2006AA01Z303).

References

1. Yang, H., Pan, Z., Zhang. M.: Modeling emotional action for social characters. Knowledge Engineering Review, 23 (4): 321--337 (2008)

2. Larstone, R., Jang, K., Livesley, W., Vernon, P.: The relationship between Eysenck's P-E-N model of personality, the five-factor model of personality, and traits delineating personality dysfunction. Personality and Individual Differences, 33: 25--37(2002)

3. Ortony, A., Clore, G., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press (1988)

4. Isla, D.: Handling Complexity in the Halo 2 AI. In Proceedings of Game Developers Conference (GDC) (2005)

5. Picard, R.: Affective computing: challenges. International Journal of Human Computer Studies, 59(1-2): 55--64 (2003)

6. Egges, A., Kshirsagar, S., Thalmann, N.: Generic personality and emotion simulation for conversational agents. CAVW, 15(1): 1--13 (2004)

7. Chittaro, L., Serra, M.: Behavioral programming of autonomous characters based on probabilistic automata and personality. CAVW, 15 (34): 319--326 (2004)

8. Marsella, S., Gratch, J.: EMA: A computational model of appraisal dynamics. European Meeting on Cybernetics and Systems Research (2006)

9. Loyall, A., Reilly, W., Bates, J., Weyhrauch, P.: System for authoring highly interactive, personality-rich interactive characters. Proceedings of the 2004 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, pp. 59--68 (2004)

10. Egges, A., Kshirsagar, S., Thalmann, N.: A Model for Personality and Emotion Simulation. In Proceedings of Knowledge-Based Intelligent Information & Engineering Systems (KES2003), Lecture Notes in AI. Springer-Verlag: Berlin (2003)

11. Egges, A., Thalmann, N.: Emotional communicative body animation for multiple characters. In Proceedings of the First International Workshop on Crowd Simulation, pp. 31--40 (2005)

12. Perlin, K., Seidman, G.: Autonomous Digital Actors, Motion in Games (2008)

13. Diller, D., Ferguson, W., Leung, W., Benyo, A., Foley, D.: Behavior modeling in commercial games. In BRIMS'04: Proceedings of the 2004 Behavior Representation in Modeling and Simulation Conference (2004)

14. Burke, R., Isla, D., Downie, M., Ivanov, Y., Blumberg, B.: CreatureSmarts: The Art and Architecture of a Virtual Brain. In Proceedings of the Game Developers Conference (2001)

15. Greisemer, J., Butcher, C.: The Integration of AI and Level Design in Halo, In proceedings of the Game Developers Conference, San Jose, CA (2002)

16. Kallmann, M., Thalmann, D.: Modeling Behaviors of Interactive Objects for Real-Time Virtual Environments. Visual Languages and Computing, vol. 13, pp. 177--195 (2002)

17. Ekman, P.: An argument for basic emotions. Cognition and Emotion (1992)