# Training a Minesweeper Agent Using a Convolutional Neural Network
## 1 Introduction  
Minesweeper was first released in 1992 on the Windows operating system, aiming to train users in balancing the use of the left and right mouse buttons, improving mouse movement speed and click accuracy, thereby making the operation of PCs more natural and fluid. This number-based logic game became highly popular, providing an engaging way to pass time, and eventually became a standard game in subsequent versions of the Windows operating system [1]. The game interface consists of a minefield, a mine counter, and a timer, as shown in Figure 1. The minefield is rectangular, with mines randomly placed within it. Players can customize the size of the minefield and the number of mines, ranging from a minimum grid size of \(8\times 8\) to a maximum of 30 \(\times\) 24 (with 30 being the maximum length and 24 the maximum width). The number of mines can vary between 10 and 667, with the maximum being 667 in the configuration of 29 \(\times\) 23. The objective of Minesweeper is for the player to uncover all non-mine cells as quickly as possible to complete the game and achieve victory. If the player uncovers a cell containing a mine, it will trigger an explosion and result in a loss. Typically, Minesweeper is designed to ensure that the player's first move does not trigger a mine. Minesweeper is a game that combines practicality, fun, and competitiveness, testing the player's reaction speed, logical reasoning, and adaptability. Additionally, Minesweeper possesses several characteristics that make it an ideal benchmark for testing intelligent algorithms [2]:  
1. Partially Observable Environment: In Minesweeper, players do not have access to the full minefield information and can only see the uncovered cells. This partial observability requires algorithms to make decisions with incomplete information, similar to the challenges faced in many real-world scenarios;
2. Probabilistic Reasoning: The game involves uncertainty, with players inferring the positions of mines based on displayed numbers. Intelligent algorithms must handle this probabilistic reasoning to make optimal decisions, making Minesweeper an ideal tool for evaluating algorithms with probabilistic reasoning capabilities;
3. Sequential Decision Making: Every move in Minesweeper is related to previous actions, requiring optimization of a series of decisions. This characteristic mirrors many real-world tasks, where planning and executing a sequence of actions is essential;
4. Large State Space: Although the rules are simple, the state space of Minesweeper is vast due to the numerous configurations of mines and uncovered cells. Testing algorithms in such a complex state space helps assess their generalization ability and efficient exploration capabilities;
5. Reward-based Feedback: The game has clear win/loss outcomes, along with intermediate feedback provided by the revealed numbers. This reward structure is highly suitable for reinforcement learning algorithms, which learn by maximizing rewards. In summary, Minesweeper is a logical, uncertain, and complex game that requires moderate storage and computational resources, making it an ideal platform for testing intelligent algorithms. The process followed by a Minesweeper agent is as follows: first, the Minesweeper grid is encoded to create an observable state. Then, based on the current state and the selected algorithmic model, the optimal action is determined. The current state, the chosen action, and the rules of Minesweeper collectively determine the next state, forming a sequential decision-making process. Depending on the type of algorithmic model, intelligent algorithms for Minesweeper can be divided into two main categories:  
Figure 1: Minesweeper game interface on Windows XP system (beginner level, 9 \(\times\) 9 grid, 10 mines). 1. Deterministic Algorithms: These algorithms formalize human players' experience and techniques into an expert system, requiring no further training.