Abstract-This paper surveys research on applying neuroevolution (NE) to games. In neuroevolution, artificial neural networks are trained through evolutionary algorithms, taking inspiration from the way biological brains evolved. We analyze the application of NE in games along five different axes, which are the role NE is chosen to play in a game, the different types of neural networks used, the way these networks are evolved, how the fitness is determined and what type of input the network receives. The paper also highlights important open research challenges in the field. Index Terms-Evolutionary algorithms, neural networks, neuroevolution. T HE field of artificial and computational intelligence in games is now an established research field, but still growing and rapidly developing. 1 In this field, researchers study how to automate the playing, design, understanding or adaptation of games using a wide variety of methods drawn from computational intelligence (CI) and artificial intelligence (AI) [17] , [76] , [78] . One of the more common techniques, which is applicable to a wide range of problems within this research field, is neuroevolution (NE) [30] , [147] . Neuroevolution refers to the generation of artificial neural networks (their connection weights and/or topology) using evolutionary algorithms. This technique has been used successfully for tasks as diverse as robot control [83] This paper surveys the use of neuroevolution in games (Fig. 1) . A main motivation for writing it is that neuroevolution is an important method that has seen continued popularity Furthermore, work in this field is published in a number of conferences and journals in neighbouring fields. Overview. An important distinction between NE approaches is the role that NE plays in a game, which is tightly coupled to the input the evolved neural network receives (e.g., angle sensors) and what type of output it produces (e.g., a request to turn). NE's role also directly influences the type of fitness evaluation. Different evolutionary algorithms support different network types and some methods can be more or less appropriate for different types of input representations. since its inception around two decades ago, and that there are numerous existing applications in games and even more potential applications. The researcher or practitioner seeking to apply neuroevolution in a game application could therefore use a guide to the state of the art. Another main motivation is that games are excellent testbeds for neuroevolution research (and other AI research) that have many advantages over existing testbeds, such as mobile robotics. This paper is therefore meant to also be useful for the neuroevolution researcher seeking to use games as a testbed. In writing this paper, we have sought a broad and representative coverage of all kinds of neuroevolution to most kinds of games. While it is not possible to be exhaustive, we attempt to cover all of the main directions in this field and the most important papers. We only cover work where neuroevolution has in some way been applied to a game problem. By neuroevolution, we mean techniques where evolutionary computation or similar bio-inspired stochastic search/optimisation algorithms are applied to artificial neural networks (ANNs). We take an inclusive view of neural networks, including both weighted sums, self-organizing maps and multi-layer perceptrons, but not 1943-068X
Recently, there has been a renewed push in the use of evolutionary algorithms to offer alternatives for (Deep) Reinforcement Learning REF .
Neuroevolution in Games: State of the Art and Open Challenges
IEEE Transactions on Computational Intelligence and AI in Games
IEEE Transactions on Computational Intelligence and AI in Games
Computer Science