# Training a Minesweeper Agent Using a Convolutional Neural Network
## 1 Introduction
Two representative algorithms are the Single-Point (SP) algorithm and the Constraint Satisfaction Problem (CSP) algorithm [3,4]. The SP algorithm is based on the fundamental Minesweeper technique: if a cell shows 8, then each adjacent cell must contain a mine, and similar rules apply to other numbers. The CSP algorithm treats Minesweeper as a more general constraint satisfaction problem, where the revealed numbers provide constraints on the distribution of mines. Essentially, CSP is similar to SP, as both are based on constraint-driven numerical logic, but while SP only considers individual 3 \(\times\) 3 grids, CSP considers the correlations between adjacent 3 \(\times\) 3 grids or even larger areas. As a result, CSP has higher complexity and a higher win rate. The win rates for SP are 75.28%, 42.07%, and 1.28% for beginner, intermediate, and expert levels, respectively. For CSP, the win rates are 91.25%, 75.94%, and 32.90% [5]. It should be noted that classifying these algorithms as \"intelligent\" is debatable, as true intelligence involves not only high win rates but also the ability to learn the rules and techniques of Minesweeper through training. Nevertheless, SP and CSP algorithms serve as benchmarks for evaluating other algorithms, as they effectively reflect the highest level of human player performance;
2. Learning-based algorithms mimic the process by which human players acquire Minesweeper skills. The agent is trained through repeated gameplay, typically involving exploration and trial and error. The key is to design a selection mechanism that evaluates whether the agent's actions are moving toward the goal. If progress is made, the agent is encouraged to continue searching in the same direction; if no progress is made, the current path is abandoned to avoid blind searching. The Q-learning algorithm selects actions by evaluating the cumulative rewards of each game. As long as the reward function is well-designed, this algorithm is theoretically a general approach. Reference [6] shows that, under a 4 \(\times\) 4 grid with three mines, the algorithm's win rate can reach 70% after 1 million training episodes. However, due to its reliance on table-based state-action value storage, the algorithm has significant limitations in tasks with large state spaces (i.e., the state explosion problem). Reference [6] points out that in a 6 \(\times\) 6 grid experiment, the algorithm is impractical on the hardware platform used due to excessive computation time. By replacing the traditional Q-learning table with a deep neural network, the Deep Q-Network (DQN) can effectively address the computational challenges of large state spaces. In Reference [7], the DQN model achieved a win rate of 90.2% in a 6 \(\times\) 6 grid with four mines, but its training steps exceeded 17 million, resulting in prolonged computation time. Although the DQN model shows great potential for complex decision-making tasks, there is still room for improvement in sample efficiency and training stability. The combination of deep learning and reinforcement learning is a popular research direction, enabling the creation of highly complex models and achieving impressive results in various fields, such as large language models such as ChatGPT [8], autonomous driving [9,10], and intelligent robots [11]. Computer games, such as those on the OpenAI Gym platform, have played a key role in the development of artificial intelligence technologies dominated by deep neural networks [12,13], providing an ideal validation and testing environment that drives algorithmic breakthroughs. The rich data and complex environments of games promote the optimization of deep learning algorithms. For instance, Google DeepMind successfully trained agents surpassing human performance in 49 Atari games with varying levels of complexity, action spaces, and reward structures using DQN [14], demonstrating the method's effectiveness and robustness in complex environments.