Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning Game testing has been long recognized as a notoriously challenging task, which mainly relies on manual playing and scripting based testing in game industry. Even until recently, automated game testing still remains to be largely untouched niche. A key challenge is that game testing often requires to play the game as a sequential decision process. A bug may only be triggered until completing certain difficult intermediate tasks, which requires a certain level of intelligence. The recent success of deep reinforcement learning (DRL) sheds light on advancing automated game testing, without human competitive intelligent support. However, the existing DRLs mostly focus on winning the game rather than game testing. To bridge the gap, in this paper, we first perform an indepth analysis of 1349 real bugs from four realworld commercial game products. Based on this, we propose four oracles to support automated game testing, and further propose Wuji, an onthefly game testing framework, which leverages evolutionary algorithms, DRL and multiobjective optimization to perform automatic game testing. Wuji balances between winning the game and exploring the space of the game. Winning the game allows the agent to make progress in the game, while space exploration increases the possibility of discovering bugs. We conduct a largescale evaluation on a simple game and two popular commercial games. The results demonstrate the effectiveness of Wuji in exploring space and detecting bugs. Moreover, Wuji found 3 previously unknown bugs, which have been confirmed by the developers, in the commercial games.

Tactical Reward Shaping: Bypassing Reinforcement Learning with StrategyBased Goals Deep Reinforcement Learning (DRL) has shown its promising capabilities to learn optimal policies directly from trial and error. However, learning can be hindered if the goal of the learning, defined by the reward function, is not optimal. We demonstrate that by setting the goaltarget of competition in a counterintuitive but intelligent way, instead of heuristically trying solutions through many hours, the DRL simulation can quickly converge into a winning strategy. The ICRADJI RoboMaster AI Challenge is a game of cooperation and competition between robots in a partially observable environment, quite similar to the CounterStrike game. Unlike the traditional approach to games, where the reward is given at winning the match or hitting the enemy, our DRL algorithm rewards our robots when in a geometricstrategic advantage, which implicitly increases the winning chances. Furthermore, we use Deep Q Learning (DQL) to generate multiagent paths for moving, which improves the cooperation between two robots by avoiding collision. Finally, we implement a variant A* algorithm with the same implicit geometric goal as DQL and compare results. We conclude that a wellset goal can put in question the need for learning algorithms, with geometricbased searches outperforming DQL in many orders of magnitude.