  As modern games continue growing both in size and complexity, it has become
more challenging to ensure that all the relevant content is tested and that any
potential issue is properly identified and fixed. Attempting to maximize
testing coverage using only human participants, however, results in a tedious
and hard to orchestrate process which normally slows down the development
cycle. Complementing playtesting via autonomous agents has shown great promise
accelerating and simplifying this process. This paper addresses the problem of
automatically exploring and testing a given scenario using reinforcement
learning agents trained to maximize game state coverage. Each of these agents
is rewarded based on the novelty of its actions, thus encouraging a curious and
exploratory behaviour on a complex 3D scenario where previously proposed
exploration techniques perform poorly. The curious agents are able to learn the
complex navigation mechanics required to reach the different areas around the
map, thus providing the necessary data to identify potential issues. Moreover,
the paper also explores different visualization strategies and evaluates how to
make better use of the collected data to drive design decisions and to
recognize possible problems and oversights.
