JAIST Repository
https://dspace.jaist.ac.jp/Title

Production of Emotion‑based Behaviors for a
Human‑like Computer Game Player [課題研究報告書]

Author(s)Temsiririrkkul, Sila

CitationIssue Date

2015‑03

TypeThesis or Dissertation

Text version

author

URL

http://hdl.handle.net/10119/12957

RightsDescription

Supervisor: 池田

心, 情報科学研究科, 修士

Japan Advanced Institute of Science and Technology

Abstract
For a long while, the main goal of academic research in computer games was to make asuitable computer opponent for humans. To reach this goal, the first and major step wasto improve the strength of the computer players. In the recent years, the computer gameplayers have improved significantly due to the advance of search algorithms and computer
technology. The computer players are now strong enough to win against average humanplayers in classical board games such as Chess or Othello, but also in modern computer
games such as Starcraft. It is perhaps time to devote more attention to other issues ofcomputer players such as their usage for education or entertainment purposes.
The current performance of computer game players is very promising and efficient interms of strength, but the behavior of such strong computer players is not so promising
for the entertainment of players. For example, in a famous video of a computer player
for Infinite Mario Bros, Mario character shows very precise movements, and no hesitationin the decisions. Such behavior looks very mechanical, in other words too strong andthen not so entertaining.Furthermore, in the case of multiplayer games, since the computer player may be a partner or an opponent of the human player, such too strong or
unnatural behaviors can make human players suspect that they are being cheated, andthe entertainment of the game will be harmed. Hence, the production of behavior thatlooks natural to humans, called human-like, is essential to make computer players able toentertain humans.
Many approaches were proposed to produce human-like behavior. Schrum presented ahuman-like computer player based on a neural network for First-Person Shooter games.
The computer player learning was supervised by human-player data recorded from pastgames. In 2013 Fujii et al. showed a new approach to produce human-like behavior withhuman-like mistakes. The biological constraints, including sensory error, perceptual andmotion delay, and physical fatigue, were introduced into the path finding algorithm A*
and the reinforcement learning Q-Learning. This research tried to imitate the behaviorof human players by using a single behavior model, so the transition between different
behaviors was not explicitly implemented, though such transition is common for humanplayers. Then one of our goal in this research is to produce such transition between
multiple behaviors.
In order to conduct a research on computer players, a suitable test-bed platform isneeded. The classical platform of board games such as Chess or Go was the main target
over the last 30 years. In order to play well at these board games, good reasoning andplanning is required. In modern video games, other capacities such as pattern recognition,navigation and decision making in a short period are required. Furthermore, many games
have imperfect information and they are played not only by one or two players but by
three or even more. Thus it is a very challenging target. Recently, Sergey et al. publishedthe platform called “Mario AI Benchmark” to evaluate computer players in a side-scrolling
video game. The platform provides an API which allows developers to implement theirown computer player. Some competitions were held using this platform, targeting notonly strength, but also human-likeness and entertainment. Then we also employ thesame test-bed platform in this research.
In many modern video games, multiple goals are given to the players. Sometimes aplayer might choose to suspend the main goal of the game and challenge himself foranother goal. For example in Super Mario Bros, the main goal of the game is to clear thestage within a limited time. In the beginning, the player tries to reach the goal as fast
as possible. But after he finds some coins, the player might ignore the main mission andtry to collect coins, which is a sub-objective of the game. He might be inspired by hisgreed or need of enjoyment. Or when he encounters many enemies at once, he might stopmoving or collecting the coins and run away from the risk of being killed. Such behavior
might be inspired by his fear. Such changes of behavior are inspired by human feelings
or emotions. Hence, we believe that the production of behavior transition is importantto obtain a human-like behavior.
In this study, we propose a design of human-like computer players with five emotional
behaviors : “Safety”, “Hurry”, “Greedy”, “Enjoy” and “Habit”. “Safety” reflects anxiousness and fear of the player when he is on guard. “Hurry” shows speedy less carefulactions, when the player is anxious about the remaining time. “Greedy” reflects theenjoyment of humans when they find rewards. “Enjoy” reflects the enjoyment and interest such as killing enemies continuously. “Habit” reflects unintended behavior such aspressing repeatedly the jump button.
And also we propose a hand-coded rule-based transition model of these behaviors.
As a preliminary experiment, we carefully investigate the human-likeness of the “Safety”,“Hurry” and “Greedy” models, by comparing play videos from these models, from theA* algorithm and from human beginners. We conclude that our proposed models cansuccessfully produce very natural behavior specific to each purpose. In the near future,
automatic switching between the behaviors will be investigated with the proposed transition model. The individual performance of the behavior models and the total performanceof the transition model with behavior switching will be evaluated by both of a Turing test
and an entertainment test.

2