EUROSIS-ETI BELIEVABILITY ASSESSMENT FOR FIGHTING GAME AI

We describe two methods of analyzing human and AI play style patterns in an arcade fighting game. The first is the application of a Turing test to study game characters’ behavior. The second is the calculation of a cosine similarity between “behavior fingerprints” consisting of sequences of individual actions or combo chains. The main goal of this study is aimed to find an approach that helps to determine the believability of game AI. Our experiments with Universal Fighting Engine environment and its built-in AI system demonstrated that both people and AI agents exhibit different play styles, and AI agents are virtually indistinguishable from human-controlled characters. INTRODUCTION Most types of computer games implement AI logic in some form. According to Dill, the purpose of game AI is to support a certain player experience (Dill, 2013)). AI plays the key role in supporting players’ entertainment; too weak or too strong AI can reduce the overall quality of a game. In certain game genres an AI system is supposed to imitate human behavior. This ability is especially important if AI controls human-like characters or replaces real players. One of the problems of designing human-like AI is to find actual criteria of human-likeness that can be used to distinguish characters controlled by human players and by AI. One of the possible solutions is to adapt Turing test (Turing, 1950) for game AI assessment, i.e., to rely on human evaluation of the believability (Livingstone, 2006), (Gorman, Thurau, Bauckhage, & Humphrys, 2006) . Alternatively, it is possible to test human-likeness by comparing behavior patterns of an AI system with those of human players, as shown in (Tencé & Buche, 2008) and (Mozgovoy & Umarov, 2010). We discuss both approaches for analyzing play styles and believability of game characters in an arcade fighting game. The first approach is a Turing test-inspired series of player evaluations performed by people watching pre-recorded game clips. The aim of this approach is to investigate human ability to discern individual players and to separate AIcontrolled and human-controlled characters. The second approach uses an automated evaluation algorithm that builds a “behavior fingerprint” for each game character. The fingerprints are then compared to reveal similarities and differences between the players and between humanand AIcontrolled characters. The experiments show that each player in our game possesses recognizable behavior traits, but separating humans from AI agents is difficult. In our experiments, we use a publicly available fighting game engine called Universal Fighting Engine (UFE). During game sessions, players can operate game characters by controlling six attack buttons and four direction keys. In addition, the players can make game characters perform special actions such as fireball and uppercut by using key combos. UFE contains a built-in AI system called Fuzzy AI Add-on that uses fuzzy logic to evaluate the information of the scene and calculate the desirability of each given action, translating the AI decisions directly into user input. In the experiments we use three AI-controlled characters, based on different Fuzzy AI settings: 1) very easy; 2) normal; 3) impossible. TURING TEST FOR PLAY STYLE ANALYSIS To verify whether human players show unique play styles and whether human players are distinguishable from AIcontrolled characters, we prepared two types of a Turing test. Matching game clips test We asked a group of testers to watch five game clips, each showing a match between a player A-E and a random opponent. Players A-C were controlled by three different persons, while Player D and Player E were controlled by the Fuzzy AI system set to a very easy and normal modes respectively. Next, we asked the testers to watch five more clips showing the same players A-E playing against random opponents. Finally, the testers had to accomplish the following assignments: 1. To identify pairs of clips showing the same players A-E. 2. To identify whether each character A-E in the latter five clips is controlled by a person or by an AI system. A tester gets one point for each correct pair or answer, and we perform the experiment twice. Therefore, the best possible score is 10 for each question, and the total number of clips each tester has to watch is 20. Grouping game clips test For this test, we prepared 15 clips showing each of the players A-E fighting against a random opponent three times. Players A-C were controlled by three different persons, while Player D and Player E were controlled by the Fuzzy AI system set to normal and impossible modes respectively. We have showed these clips to the testers, and asked them to: 1. Group together three clips of to the same player A-E. THIS IS AN EXAMPLE OF HOW YOUR PAPER SHOULD LOOK FROM PAGE 2 ONWARDS © EUROSIS-ETI 2. Clasify the players into the “human” and “AI” groups. The tester gets two points for each correct group of three clips belonging to the same character and one point for an incomplete group of two correct and one wrong clips. Therefore, one may score up to 10 points in this task. When the tester correctly marks a group of three AIcontrolled charcters as “AI group”, we add three points to the score. When the marked group contains only two AIcontrolled characters, we add two points. Therefore, the tester can score up to 15 points in the second assignment.