Research Note

Prototyping Results of Driving the Behavior of AI Agents
ストレスでAIエージェントの行動を駆りたてることのプロトタイピング

BRANDSE Michael

ブランセ マイケル

Digital Hollywood University, Assistant Professor
デジタルハリウッド大学 助教

AI in games has come a long way since its inception, but is still woefully lacking when it comes to more emotive AIagents. In this paper, we describe a system that will serve as the base for more emotive AI, by using stress response todrive agent behavior. By using stressors to initiate agent behavior, we not only enable anything to become a driver forbehavior, but we create a more natural approach to AI, which will allow for emotions to be designed as well. However,prototyping has shown that our initial use of behavior trees was lacking. Behavior trees did not allow us the dynamic
behavior we required and were hard to maintain as well. Therefore, further development of our system is required.
Keywords: Game Design, Interaction Design, Artificial Intelligence

1. Introduction

specifically in certain situations is impossible and would require

Development of AI for games is often limited to improving ways

a specifically trained AI agent for that very situation. As such,in which AI can attack the player in a believable way, such asmachine learning is at present not suited for game development,

path finding or making an AI agent fumble aggressive behavior

where oftentimes AI needs to behave according to the wishes ofin a natural manner to give players more of a chance to retaliate

the designer. We wish to focus on a solution that can be used ineffectively. Unfortunately, approaches that would give the AI

the average game development process, and as such, machine

agent a more emotional behavior are relatively non-existent andlearning is not an option for this research.

as a result, the manners in which AI agents can show emotion

2. Approachto the player have been limited to rather rudimentary methods ofinteraction. For instance, in the game Persona 5[1]

2.1 Behavior Tree

, interaction

between the player and the AI agent is done through scripted

In order to develop our AI agent, we decided to use the Unreal

text sequences, where the player is given a number of choices of

4 engine. The benefits of using Unreal 4 is that it is easy towhich only a few will improve the relationship between the player

quickly set up a game project, which allowed us to focus ourand the AI agent. As such, the building of relationships is aresources on developing the AI agent. Furthermore, Unreal

linear progress, where a particular variable just needs to reach a

4 uses an event based behavior tree for its AI, which initially

specific value for the relationship to improve. Persona 5 is hardly

showed compatibility for our stress response system.

alone in this, with many other games following similar tendencies

2.2 Stressors

in not allowing an AI agent a more dynamic emotional response.

Our system consists out of two elements:

In this paper, we aim to create a basis for more dynamic
emotional interactions between player and NPC. For this, we(1) Stressors.

propose a system based on the human stress response, using(2) Stress response.

stress as a means of driving agent behavior. In particular, weStressors are events that cause stress to the agent and havewill be using acute stress, which is not necessarily detrimental
to mental health [2]. As such, our implementation only deals withtwo properties.

(almost) instantaneous reactions and won’t deal with long term

(1) They have varying levels of intensity.

concepts such as chronic stress.

(2) Due to the human body trying to maintain homeostasis,stressors degrade over time.

1.1 Emotional AI and machine learning

To calculate degradation, we use a basic half-life formula.

Machine learning has already given us a number of applications
Equation: N(t) = N 0e −λ tfor emotional AI. However, while machine learning has its merits,
our focus is on creating a solution that is usable by gamedesigners. Machine learning however requires that designers

Where N 0 is the initial quantity of stress that will decay, N(t) istrain an AI agent, which takes time and resources. In the casethe stress that has not yet decayed after time t , and λ is thean AI agent does not exhibit the desired traits the designers aredecay constant of the decaying stress. We identify two types ofaiming for, retraining becomes necessary. Furthermore, once

intensity; eustress (positive stress) and distress (negative stress).

an AI agent has been trained, making the AI agent react very

To mimic the human fast and slow response to stress [3], we divide©︎2020 Digital Hollywood University54

DHU JOURNAL Vol.07 2020

stressors into two types; instant stressors and stressors with aneffect over a period of time. All stressors are added to a single
pool of stress. Initially, we used the average of all stressors inthis pool as the output stress level, but in our initial prototyping
we found that this made AI agent response time unnaturally
fast, to the point of it being instant. To solve this problem, weinstead used the average of this pool as the target stress level,
and interpolated from the current level of stress to the target
level of stress using interpolation functions found within theUnreal 4 engine. This gave us a slight delay in stress activation,
eliminating the instant reactions and giving the AI something
resembling human reaction time.

2.3 Stress Response
Out of available stress response models, we found that thecognitive-mediational theory [4] was the most effective to use
for our own proposed model. According to this theory, thesubject uses a two-step appraisal process to analyze stress anddetermine a manner of coping based on the type of stress. The
first step is to determine whether there is a perceived threat ornot. When a threat has been perceived, a secondary appraisal isdone to determine how to cope with the threat. We based coping
mechanisms on the four fear responses; freeze, flight, fight andfright [5], though we decided not to incorporate fright (referredto as“playing dead”in early literature). In our implementation,
we found that fright is better suited as an extension to the freezemechanism rather than a separate coping mechanism.
Freeze, also known as hyper vigilance, is used by the AI
agent to gather information. Flight is then used to escape fromthe stressor. If while fleeing there is no escape path, the fightFigure 1: Stress response systemmechanism will be used to engage the stressor until escapeoptions can be confirmed.

(2) A simple system reduces the chance of bugs appearing that

Initially, the AI agent’s reaction time due to stress level

could hinder our AI agent’s credibility, especially consideringinterpolation still caused a few unnatural moments, where agents

this system is meant as a base.

seemed to not react to new stressors in time. To remedy this,(3) Ease of integration is important. Since our system has

we introduced an“instinctual reaction”to the process, wherebeen developed as a tool for game designers, having too many

designers can potentially add quick and simple actions, such

options may make it hard to integrate into (existing) gameas rotating to face the new stressor (Figure 1). What separates

projects. Introducing too many variables at this stage would only

our model from more classical approaches to game AI is thatmake it harder and more time consuming to integrate.

our model assumes that flight is the end goal due to fleeing
giving organisms a better chance at survival rather than simply

3. Prototype

fighting the stressor. Furthermore, since our system assumes

3.1 Prototype Developmentthat anything can become a stressor, plenty of behavior patterns

The prototype environment was developed in Unreal 4. To

become possible.

speed up development, we opted to use functions and assets
already present in the engine wherever possible.

2.4 System SimplicityDue to the default character in Unreal 4 not having any ability toSince we developed this system specifically for use in games,

exhibit facial expressions, nor having any option to add those,

we tried to keep it as simple as possible. The reasoning for this

we instead opted to show the AI agent’s mental state throughis threefold.

the use of icons in text balloons (Figure 3).

(1) Performance is important. Since the system is meant for
in-game use, we cannot put too much strain on the system,

3.2 Prototype Game Rules

especially since this model will serve as the basis for much more

For the prototype, we made a simple game where the goal iscomplicated systems.

for the player to surprise the AI agent present within the gamemap. One game map has been constructed, with one AI agent
and a number of debris objects on higher elevations the player
can push down (Figure 2).

©︎2020 Digital Hollywood University55

DHU JOURNAL Vol.07 2020

prototype also suggested that behavior trees lack the ability todynamically react to different kinds of stimuli. For instance, wefound that our AI agent tended to react very slowly to the player
trying to surprise the agent after the agent had been assailed
with falling debris. A potential solution to this problem could be
the use of utility AI [6] instead of behavior trees. Using utility AI,
the agent is given a number of tasks, which are scored based on
an interval. The higher the score, the more likely it is for the AI tostart a specific task.

4.2 Emotional Expressions
Due to this research being meant to create the basis of ourFigure 2: The game map used for the prototype. The spheres
function as debris. When the game starts, the player isspawned on the upper level and the AI agent is spawned on
the lower level.

more emotional approach to AI, we were limited in how complex
the behavior of our AI agent could be. As such, for the prototype
we could only create very basic implementations of emotions
such as surprise, fear and annoyance.

If a debris object is dropped near the AI agent, the AI agent will
experience an increase in distress. The player has an additional

5. Conclusions and Future Workaction, in the form of trying to surprise the AI agent, which canIn this paper, we have presented a system where AI agents basehave various effects depending on the current stress level of theagent. The agent has roughly four types of behavior;

their decisions on the distress and eustress they are experiencing.

(1) If too much time passes with nothing happening (i.e. if noUsing this system, we have found a much more natural way ofdebris drops nears the agent or the player doesn’t attempt todriving behavior, which has allowed for the creation of a basis thatsurprise the agent), the agent will move to a different random

allows for more emotive AI agents. Furthermore, due to anything

location within a certain radius.

being able to become a stressor, the system is very versatile in what(2) If the AI agent becomes stressed due to falling debris, thecan cause behaviors to happen.

agent will first panic and freeze in place (freeze response) andHowever, we also found that in conjunction with behavior trees,

later move to a safe location (flight response).

our system was not able to perform at its full potential and as(3) If the player tries to surprise the AI agent, but the agent issuch, we wish to see whether our system could work better withnot sufficiently stressed, the agent gets annoyed and moves

something based on utility AI instead.

away from the player.

For our future work, we wish to further develop this system, as well(4) If the player tries to surprise the AI agent when it has built up

as broaden the range of emotions that our system can output.

a sufficient amount of distress, the AI agent will be surprised andfall over.

References

Using these four behaviors, the goal for the player is to simply[1] P-Studio,“Persona 5”Sega (2016).

successfully make the AI agent fall over in surprise.

[2] Elizabeth D. Kirby et al.“Acute stress enhances adult rat
hippocampal neurogenesis and activation of newborn neurons
via secreted astrocytic FGF2”Elife 2(2013).
[3] Bryan Kolb, Ian Q. Whishaw. “Fundamentals of humanneuropsychology”Macmillan (2009).
[4] Richard S. Lazarus.“Stress and emotion: A new synthesis”
Spring Publishing (2006).
[5] H. Stefan Bracha: ”Freeze, Flight, Fight, Fright, Faint:Adaptationist perspectives on the acute stress responsespectrum”CNS Spectrums (2004), p679-685.
[6] Gamasutra:“Are Behavior Trees a Thing of the Past?”
h t t p s : / / w w w. g a m a s u t r a . c o m / b l o g s / J a k o b R a s m u s s e n /20160427/271188/Are_Behavior_Trees_a_Thing_of_the_Past.
php (Accessed 2020-07-20).

Figure 3: The player tries to surprise the AI agent, but due tolacking distress levels, the agent becomes annoyed instead.
Emotion is shown through text balloons.

4. Limitations
4.1 Behavior Tree Limitation
While initially it seemed behavior trees are a good way of
implementing our system, we quickly found that our system will
create behavior trees that are relatively hard to prototype andmaintain, which is something we wish to avoid. Developing the©︎2020 Digital Hollywood University

56

DHU JOURNAL Vol.07 2020