Copyright Material – Provided by Taylor & Francis

3
Logging Visualization in
FINAL FANTASY XVMatthew W. Johnson, Fabien Gravot, Shintaro Minamino,
Ingimar Hólm Guðmundsson, Hendrik Skubch,
and Youichiro Miyake

3.1
3.2
3.3

Introduction
Architecture
Statistical Analysis

3.4
Spatial Analysis
3.5
Conclusion
References

3.1 Introduction
Data analytics have had a variety of uses in the gaming industry, most commonly in
online games and the mobile market. However, in the development of FINAL FANTASY
XV, an action style Role Playing Game (RPG), we have found an additional use for data
logging combined with visualization. One challenge we face with a large team, measuring in the hundreds working across the globe, is ensuring quality game data. In an ideal
scenario, the game’s design and tools would be well defined, and accurate data would
be ensured at the time of authoring. However, games often have changing requirements,
new features, and new fun and exciting ideas introduced during the development process.
These aspects can make it difficult to know precise boundaries of still-developing areasof a game. Nevertheless, ensuring quality game data is still an important problem that
all teams address with varying techniques. Furthermore as the project scale continues togrow, additional quality assurance challenges are introduced.
We address these issues in FINAL FANTASY XV through a comprehensive logging
and data analysis toolset. We log various aspects of the game in a database and use a
web-based front end for analysis. This enables us to investigate movement issues, monitor event distributions in both time and space, and observe changes in the navigation
mesh over time. Conceptually, our approach enables similar analysis options as doneby Thorzen for The Witcher 3 (Thorzen 2015). However, Thorzen directly populated a21

Copyright Material – Provided by Taylor & Francis

database with content information in order to find problematic data, while we are logging gameplay information to a database and analyzing that data.
In this chapter, we will describe the architecture we used and the reasons behind the
choices we made, will look at the three main uses of our tool, and will discuss some of the
future work we plan. By leveraging data visualization, we found that identifying potential
issues, even in large worlds with lots of data, was easy and convenient for designers anddevelopers. Additionally, the logging system was not intrusive and had no adverse effects
to the gameplay experience and provided meaningful insight to our team.

3.2 Architecture
The architecture for this system was influenced by two main goals: (1) minimize impact
on game performance at all costs and (2) allow both designers and developers capabilities to analyze the data via web browser. Since we log a lot of data, effectively searching
through it is important. Conveniently, these goals can be achieved with off-the-shelf
database and web technologies. Our implementation employs MongoDB as our database and NodeJS as our web server. These meet the requirements while also allowing
substantial flexibility during the development of our visualization tool (MongoDB 2016,
NodeJS 2016).
3.2.1 Logging
In order to minimize the logging’s impact on performance, it needed to be fast and nonblocking. A logging system that affects frame rate or causes gameplay issues is more harmful than it is helpful. We implemented a nonintrusive logging system by doing most of theheavy computation via a process that can run on a different computer, which is important when playtesting on a console. In our case, the theoretical maximum bandwidth was
80 Mbps, which is a data volume limitation, but all the data described in this chapter did
not reach this limit.
In Figure 3.1, there are three main components. The game and application logger makeup the first component, which executes on the gaming console. The next component is thelog aggregator, which we will cover momentarily. The final component is the client side
view, for which we support a couple of options.
We will come back to Figure 3.1, but for now let us focus on the logging system, which iscompiled into the game’s runtime. As the player experiences the game, a variety of different events contain log statements that save data to the logging system. Figure 3.2 shows

Log aggregator
Application
Game
code

Application
loggerReceiverSortedbinarylog

Databaseprinter
Real-time
printer
CSV
printer

DatabaseserverWeb
serverClientwebbrowser

Figure 3.1
Data flow from game to web browser visualization.

22

3. Logging Visualization in FINAL FANTASY XVCopyright Material – Provided by Taylor & Francis

Thread 1(1) LogBuilder log(”ActorPosition”);(2) log.Log(”ActorId”, actorId);(3) log.Log(”Position”, position);(4) log.Finish();Thread N(5) LogBuilder log(”PQS”);(6) log.Log(”ActorId”, actorId);(7) log.Log(”Position”, position);(8) log.Log(”QueryId”, queryId);(9) log.Finish();Double buffer
(8)

(9)

(3)

(4)

Adaptive sleep

Current write buffer
Swap
(1)

(5)

(2)

(7)

(6)

Sentinel thread
TransmitLog
aggregator

Figure 3.2
Logging library implementation pipeline shows multithreaded logging process.
Table 3.1 Elementary Data Chunk for Logging
Context Id
Action Id
Name Id
Type Id
Data

Unique Id for one log
Identify the chunk role
String to identify the data inside the database
Type of the binary data
Array of the binary data

the logging system’s architecture. A single log entry can be arbitrarily complex and iscomposed of multiple elementary data chunks described in Table 3.1. Each chunk inthe same log entry shares the same context ID, which is generated by atomic increment.
Figure 3.2 shows two log entries, comprised of nine total chunks. As an example, when aplayer takes damage or an NPC activates a script, the system generates a context ID forthat event and tags the relevant data chunks in the log. Since these events can happen
simultaneously, the log system executes on multiple threads and writes to a double buffer.
When a buffer fills, it is swapped out and sent to the sentinel. The sentinel accumulates
data and adaptively sends it to the log aggregator over TCP/IP. The sentinel adapts to
the rate of data commits to the aggregator, which is a function of how much data arecoming in from the buffer. Based on the data-logging rate, the sentinel will send data
between once per second to ten times per second. Furthermore, the sentinel also sends
its own statistics to the log aggregator, so that system can also be monitored and tuned
for performance.
Writing a chunk is a three-step process:1. Reserve: Increase the number of users of the buffer, and reserve the memory
needed for writing the whole chunk. This is normally a nonlocking function
because it only increments the buffer’s atomic variables. If there is not enoughmemory, then we have to block until the next buffer swap.
2. Write: Save the data.
3. Finish: Reduce the number of users of the buffer. The sentinel will wait until thebuffer is unused before sending the data via TCP/IP.
3.2 Architecture23

Copyright Material – Provided by Taylor & Francis

The sentinel is working on its own thread and regularly wakes up to swap the bufferand send the logged data to the log aggregator via TPC/IP. The update frequency varies
based on the rate of data sent. What is more, the statistics of this adaptive sleep are also
logged and can be used to further tune the algorithm.
3.2.2 The Log Aggregator
Returning to Figure 3.1, the next component is the log aggregator. Since the logging system receives chunks for different log entries from multiple threads, essentially nothing isin a desirable order. The log aggregator’s role is to fix this issue so that data are easier tostore, manipulate, visualize, and understand. The aggregator performs another importanttask, which is offloading parts of the processing done by the runtime in order to minimize
performance impact. Once the chunks are ordered per log entry, they can be used by oneof the several printers.
We currently have three different printers:•••

The CSV printer: Parses binary data to a CSV file. Useful for debugging or local
tests.
The database printer: The most useful of the three, it stores data inside a global
database. It parses the binary to JSON and makes it ready for later analysis. This
printer also has a local buffer, which is used to reduce network congestion on the
database server.
The real-time printer: Similar to the database printer since its output format is
identical (JSON), uses a different protocol, based on web server push notification
technology (i.e., “web socket”). This printer provides streaming data, which provides nearly real-time updates on the client web browser. With this printer, we areable to display the player and enemies on a 2D map while playing the game, which
is useful for analyzing and tuning the AI.

3.2.3 The Data Format
Although the logging system allows an arbitrarily complex log, we still need to be able toanalyze it. Consequently, we defined several standard log headers. A log header is generated with a class wrapper around the data, which will generate the common elements.
One of these shared data formats is the session header. The session identifies agame play experience. Each session is stored in a different database collection. The
session header is explained in Table 3.2 and allows filtering for which data collection
Table 3.2 Common Headers Used for Logging Sessions
Machine name
Session Id
UserStart time
Binary name
Binary version
Configuration
Platform

24(Computer Machine ID—physical hardware)
Unique ID for this execution of the game
(Windows login of user)
Time stamp of session play starting
Vs—Solution name
Change list number from Versioning Software(Visual studio Build Configuration—Release/Debug/etc.)
PS4, Xbox, PC

3. Logging Visualization in FINAL FANTASY XVCopyright Material – Provided by Taylor & Francis

will be used for further analysis. For example, if a developer was interested in looking at data generated from a specific tester at a specific time that a particular artifact
occurred, then they can more easily investigate the parameters logged around thatissue.
In addition, all logs share a common header with the game time, a position (used for
spatial analysis), and the corresponding agent Id. These common headers offer format
consistency for the front-end methods.
3.2.4 Data Analysis
Data analysis is done through the database printer and accessed through a web front end
written in JavaScript. A separate front end interacts directly with the log aggregator for
real-time updates via the real-time printer. Accessing the database is a two-step process:first determine the sessions that are to be analyzed, then retrieve the data you are interested in and aggregate the session results.
Our visualization toolset supports a variety of visualizations, which can be split into
two broad categories: spatial analysis and statistical analysis. Spatial analysis visualizations project the data on a 2D map via Leaflet and provides a Google map-like user
experience for exploring spatial data in the world of FINAL FANTASY XV (Leaflet 2016).
Statistical analysis uses more traditional visualization techniques, such as bar charts andhistograms, to allow us to investigate nonspatial characteristics in our data, for example,
temporal distributions of particular events.

3.3 Statistical Analysis
Different views were developed for displaying event information, which we were interested in tracking. The web client was implemented in JavaScript and data-driven documents (D3) (D3 2016). The first view we made available is a histogram of data gatheredfrom our point querying system described in later chapters of this book. The team wasinterested in tracking the amount of use the system had and in which sections of thegame. Figure 3.3 shows the different play sessions logged on the left-hand side, withthe corresponding histogram on the right. In this example session, we can see that the
Ambient Spawn Query, a query responsible for spawning NPCs, is occurring nearly
700 times during this play session. In this example, it is easy to recognize the effects
of entering a large town with a high number of agents being spawned into the scene.
This type of visualization makes it convenient to recognize potential bugs related to ourpoint querying system.
We developed a separate view for monitoring the distribution of executed character
dialogues. When certain events happen in the game, one of many dialogue events can betriggered, and these events are separated into groups. In Figure 3.4, we see six different
dialogue groups, each with multiple dialogues. By presenting this information in a bar
chart ordered by group and frequency, we can easily monitor potential issues that may
exist in our dialogue system. With this grouped histogram view, it is easy to see that one
script in our first group occurs more often than intended, and the parameters for that
script may need review. Visualization makes identifying these cases easy and convenientfor designers and developers.
3.3 Statistical Analysis

25

Copyright Material – Provided by Taylor & Francis

SID_5755232dec64381d7898d95b

600

SID_5755229b9b36a728a8dec575
SID_57551dfdf606f2297435e50b

500

SID_57551a8fec64381d788501d0

400

SID_575519539b36a728a8d767be
SID_5755186071d7922100543b7f

300

SID_575517daf606f2297435de76
SID_575515646d21381690a37947

200

SID_575513116d21381690a072b4

_Q
UE
RY
RY
_S
_ST
AT
IC_
BIE
ON
N
LY
T_
DY
ST
STO
RO
P_
LL_
AI_
T:S
AI_
QU
TA
HO
TA
ER
CT
RT
CT
Y
IC A
-FR
IC A
IEN
L_Q
L_Q
D:N
UE
UE
R
OC
RY
YB
BU
T
U
I
DD
S
DD
Y_S
Y_
TR
TA
OL
LK
LI N
AI_
_A
PPR
TA
G
CT
OA
IC A
CH
_H
L_Q
AI_
EA
UE
VY
AI_
TA
RY
CT
TA
BU
CT
IC A
DD
IC A
L_Q
Y_W
L_Q
UE
AR
AI_
RY
UE
P
MO
RY
TA
MO
CT
N_
IC A
N_
OR
Z IG
BIT
L_Q
_SC
ZA
UE
G2
RY
_C
MO
LO
N_
SE_
Z IG
L_S
ZA
C
G2
_C
LO
SE_
R_S
C
UE

_Q

RY
AM
UE

RY
BU
D

_S
PAW
N
TA
CT

IC A

L_Q

UE

NT
NT

BIE

AI_

TA
CT

AI_

L_Q

UE

_Q

_S
PAW
N

NT

BIE

BIE

RY
AM

UE

L_Q
IC A

RY
AM
UE

L_Q

IC A
TA
CT
AI_

IC A

RY

RY
_M
UE

_Q

_Q

_S
PAW
N

P_S
PAW
N

NT

BIE

RY
AM

UE

RY
AM

UE
L_Q

IC A

TA
CT

AI_

TA
CT

L_Q
IC A

TA
CT
AI_

_S
PAW
N

RY
UE

UE

UE
_Q

L_Q
OL

GR
OU

A_
TR

_ IN
NT

BIE

RY
AM

UE
L_Q

IC A

TA
CT

P_S
TR

P_S
PAW
N
NT

NT
BIE

BIE

RY
AM
UE

L_Q
IC A
AI_

TA
CT
AI_

AI_

TA
CT

IC A

L_Q

UE

RY
AM

SID_575507e5f606f2297434a201

SID_5755052bf606f2297432c33e_G_G

SID_57550890f606f22974350100

RO
U

RO
U

SID_575508dcec64381d78772325

SID_57550625f606f22974337981

RY

0

SID_57550b5271d79221004f529e

AI_

SID_57550bfa9b36a728a8cb45bf

RY

100

SID_57551022f606f229743575a9

SID_5755071af606f2297433e092

Count

SID_5755243f9b36a728a8e501c8

SID_575503b26d213816909c706b
SID_5754fd89ec64381d7861789c
Figure 3.3
A histogram of our point querying system logging all queries and their frequencies.

3.4 Spatial Analysis
The statistical analyses are interesting, but the 2D map is arguably the more exciting use ofthis system. We used Leaflet, an open-source JavaScript map library to create an interactive 2D map of the FINAL FANTASY XV world. The library supports features like drawing
points, adding tool tips, displaying heat maps, and zoom levels. The range of possibilities
is large, and we continue to find new uses.
The first step is to load the appropriate data into Leaflet in order to create our 2D map.
The basic process is to generate a series of top–down orthographic camera screen shots.
The camera moves along a grid and captures the necessary images. Next, high-level zoom
images are generated by downscaling and merging those screen shots. By performing thisprocess, we enable mouse-wheel zoom on the world through a web browser. Once the dataloads, we can begin exploring the world.
26

3. Logging Visualization in FINAL FANTASY XVCopyright Material – Provided by Taylor & Francis

7

SCENE_ID_ACCIDENT_COLLIDE_G_02 | SCRIPT_GROUP_ACCIDENT_COLLIDE_GROUP

2SCENE_ID_ACCIDENT_COLLIDE_G_03 | SCRIPT_GROUP_ACCIDENT_COLLIDE_GROUP

2SCENE_ID_ACCIDENT_COLLIDE_I_02 | SCRIPT_GROUP_ACCIDENT_COLLIDE_GROUP

1SCENE_ID_ACCIDENT_COLLIDE_P_02 | SCRIPT_GROUP_ACCIDENT_COLLIDE_GROUP

1SCENE_ID_ACCIDENT_COLLIDE_P_03 | SCRIPT_GROUP_ACCIDENT_COLLIDE_GROUP

1SCENE_ID_ACCIDENT_COLLIDE_NP_02 | SCRIPT_GROUP_ACCIDENT_COLLIDE_GROUP

2

4SCENE_ID_DRIVE_LESTALLUM_ARRIVAL_01 | SCRIPT_GROUP_DRIVE_CAR_STOP_GROUP

4SCENE_ID_TOWN_IN_DINER_CONV | SCRIPT_GROUP_DRIVE_CAR_STOP_GROUPSCENE_ID_CAR_DOWN_LONG_01 | SCRIPT_GROUP_DRIVE_CAR_STOP_GROUP

1SCENE_ID_TOWN_ID_LEST_DAY_01 | SCRIPT_GROUP_DRIVE_CAR_STOP_GROUP

1SCENE_ID_CAR_DOWN_FIELD_08 | SCRIPT_GROUP_DRIVE_CAR_STOP_GROUP

1SCENE_ID_CAR_DOWN_CARDIRT_CONV | SCRIPT_GROUP_DRIVE_CAR_STOP_GROUP
4SCENE_ID_AC_GAL_NPC_TALK100103 | SCRIPT_GROUP_GROUP_GAL_AMBIENTCHATTER

2SCENE_ID_AC_GAL_NPC_TALK020005 | SCRIPT_GROUP_GROUP_GAL_AMBIENTCHATTER

2SCENE_ID_AC_GAL_NPC_TALK100104 | SCRIPT_GROUP_GROUP_GAL_AMBIENTCHATTER

2SCENE_ID_AC_GAL_NPC_TALK990003 | SCRIPT_GROUP_GROUP_GAL_AMBIENTCHATTER

2SCENE_ID_AC_GAL_NPC_TALK990009 | SCRIPT_GROUP_GROUP_GAL_AMBIENTCHATTER

2SCENE_ID_AC_GAL_NPC_TALK990011 | SCRIPT_GROUP_GROUP_GAL_AMBIENTCHATTER

2SCENE_ID_NICE_ATTACK_PN_02 | SCRIPT_GROUP_NICE_ATTACK_GROUP

2SCENE_ID_NICE_ATTACK_IN_03 | SCRIPT_GROUP_NICE_ATTACK_GROUP

3SCENE_ID_NICE_ATTACK_PN_04 | SCRIPT_GROUP_NICE_ATTACK_GROUP

2SCENE_ID_NICE_ATTACK_IN_05 | SCRIPT_GROUP_NICE_ATTACK_GROUP

1SCENE_ID_NICE_ATTACK_IN_04 | SCRIPT_GROUP_NICE_ATTACK_GROUP
32

SCENE_ID_LIGHT_ON_A_01 | SCRIPT_GROUP_LIGHT_ON_GROUPSCENE_ID_LIGHT_ON_CONV_B_01 | SCRIPT_GROUP_LIGHT_ON_GROUP

1SCENE_ID_LIGHT_ON_P_G | SCRIPT_GROUP_LIGHT_ON_GROUP

1SCENE_ID_LIGHT_ON_P_I | SCRIPT_GROUP_LIGHT_ON_GROUP

1SCENE_ID_ACCIDENT_NEARHIT_IP_01 | SCRIPT_GROUP_ACCIDENT_NEARHIT_GROUP

1SCENE_ID_ACCIDENT_NEARHIT_IG_02 | SCRIPT_GROUP_ACCIDENT_NEARHIT_GROUP

1SCENE_ID_ACCIDENT_NEARHIT_SIMPLE_XG | SCRIPT_GROUP_ACCIDENT_NEARHIT_GROUP

2SCENE_ID_ACCIDENT_NEARHIT_NI_01 | SCRIPT_GROUP_ACCIDENT_NEARHIT_GROUP

Figure 3.4
A grouped histogram view showing one script executing more often than intended.

3.4.1 Two-Dimensional Navigation Mesh Map
Any type of screenshot could be used in this process, but rather than using game screenshots, we used images of the game’s navigation mesh. This provided us with several benefits. First, for quality assurance purposes when our navigation mesh data are being built, wewere also generating the 2D map screenshots. In addition to this, the process is automatic,
making it convenient to use. Next, by using the navigation mesh, we can ignore geometrythat may obstruct a top–down view, such as trees or buildings. Finally, the navigation meshcovers all the areas in which NPCs and the player can travel allowing us to focus on areasthat are contained in the game play portions of the world. We export not one but several
maps at the same time, shown in Figure 3.5.
In Figure 3.5, there are six different uses of the leaflet visualization. These are: (a) connectivity mesh, (b) large monster navigation mesh, (c) nonfiltered mesh, (d) filtered navigation mesh, (e) difference image, and (f) zoomed in section of map shown in (d). We willhandle them out of order, starting with the nonfiltered map.
3.4 Spatial Analysis

27

Copyright Material – Provided by Taylor & Francis

(a)

(b)

(c)

(d)

(e)

(f )

Figure 3.5
Leaflet map visualization of the navigation mesh of the FINAL FANTASY XV world. (a) connectivity mesh, (b) large monster navigation mesh, (c) nonfiltered mesh, (d) filtered navigation
mesh, (e) difference image, and (f) zoomed in section of map shown in (d).

3.4.1.1 Nonfiltered Mesh

Figure 3.5c shows the whole set of auto-generated navigation meshes. In the live tool, themap’s colors have meaning, which represent polygon flags such as water for sea monsters
or ceiling heights for flying creatures. The other figures show subsets of this set.
3.4.1.2 Connectivity Mesh

Figure 3.5a shows the navigation mesh’s walking connectivity. For adjacent regions withdiffering patterns in the image, it means agents are incapable of path planning between

28

3. Logging Visualization in FINAL FANTASY XVCopyright Material – Provided by Taylor & Francis

them because the regions are not connected. In the web tool, these regions are depictedwith differing colors. This allows us to quickly check for invisible walls or unintentionalholes in the mesh. We can also confirm whether an agent should be able to walk between
two points or not. Navigation mesh generation is an automated process, based on a variety
of rules (angle of a slope, step-height of a character, etc.). Depending on how the world isedited, otherwise-connected regions can become disconnected and vice versa. This toollets us confirm mesh accuracy as the world map evolves.
3.4.1.3 Filtered Meshes

The generated mesh has navigation mesh above the sea, over roof tops, and even inside big
rocks, anywhere it is possible to for an agent to stand. To avoid this, a common solution is
to use a seed point. Only areas connected to the seed point will be kept. By using the walkconnectivity shown in Figure 3.5a, we can generate Figure 3.5d, which is the filtered meshthat is used in the game for humanoid characters.
Figure 3.5b is similar to Figure 3.5d but with a different archetype. It is really useful
to be able to check the map for the different archetypes. Different characters have different sizes, different step heights, and so on, and so Figure 3.5d is the navigation mesh,which is available for humans. Figure 3.5b on the other hand is for large creatures, namely
the Behemoth. The Behemoth is one of FINAL FANTASY XV’s largest creatures, and it
requires a lot of space in order to walk about, making its navigable area smaller than ahuman’s. The white area in these images is where agents, whether human or behemoth,
can go.
3.4.1.4 Difference Image

Lastly, Figure 3.5e is a difference map showing the difference between two subsequentversions of the world’s navigation mesh. White regions in this image are the same as previous versions. Dark areas are places where changes occurred. This allows designers anddevelopers to monitor changes over time, which can make it easy to confirm errors when
AI behavior suddenly or unexpectedly changes in one area of the map.
Figure 3.5f depicts a zoomed version of a small region shown in the bottom of 3.5d, theinterior of a large building.
3.4.2 Heat Map Visualization
Heat maps are an effective and convenient method for displaying density information overa 2D map. One example of heat map use was tracking the player’s position to monitor map
coverage by the quality assurance team. Here, however, we describe an interesting feature
related to data packaging.
In FINAL FANTASY XV, we have numerous animations and data variations depending
on a character’s age, culture, and other characteristics. As we cannot load all possible animations into memory, we must know which ones to preload and where. We used the log
data to display a heat map of animation IDs. This allowed designers to check for inconsistencies and rarely used animations, which helped to package animations into the relevant
areas. Figure 3.6 shows an example of such a heat map.

3.4 Spatial Analysis

29

Copyright Material – Provided by Taylor & Francis

Figure 3.6
Heat map visualization of events being executed by NPCs in a small town.

3.5 Conclusion
Interactive visualization techniques like the ones described here have improved the efficiency of developing large-scale video games. A number of characteristics from the gameare collected and analyzed during the development process, which has made debugging
and investigation of data issues more efficient. Map tools, like Leaflet, also enable us tomonitor spatial issues that might arise in the game. These types of tools are particularly
exciting because their future potential is just starting to be explored.

References
Data Driven Documents [Computer Software]. (2016). Retrieved from https://d3js.org/
Leaflet [Computer Software]. (2016). Retrieved from https://leafletjs.com/
MongoDB [Computer Software]. (2016). Retrieved from https://www.mongodb.com
NodeJS [Computer Software]. (2016). Retrieved from https://nodejs.org/en/
Thorzen, M. (2015). Content optimization pipeline for an open world game, GDC 2015.

30

3. Logging Visualization in FINAL FANTASY XV