# Training a Minesweeper Agent Using a Convolutional Neural Network
## 5 Discussion  
### Mechanisms of Two Algorithms in the Minesweeper Task  
Compared to the DQN model, supervised learning not only converged more rapidly but also exhibited enhanced training stability when applied to the training of a Minesweeper agent. This section provides an analysis of the mechanisms underlying both algorithms in the Minesweeper task, aiming to elucidate the reasons for their performance differences.  
The DQN model is a classic deep reinforcement learning algorithm. It follows a reinforcement learning framework and trains the agent through trial and error: the network repeatedly plays the Minesweeper game, selecting a cell each time. If the cell is not a mine, back-propagation is used to inform the network that the probability of the cell being safe should be higher. If it is a mine, back-propagation similarly updates the network to increase the probability of the cell being a mine. The amount by which the probability should be increased or decreased depends on the set learning rate. Additionally, the DQN model uses deep learning as a computational tool by incorporating artificial neural networks, significantly expanding the range of problems that reinforcement learning can address. For example, even for a simple 6 \(\times\) 6 grid with four mines, Monte Carlo simulations show that the number of possible minefield states exceeds 5.3 million. Traditional Q-learning requires generating a table that includes all possible states and performing a search to map states to actions. This table-driven approach is computationally and storage-intensive,  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model/Algorithm DQN** & **SL** & **SP** & **CSP** & **Human Players** \\ \hline Win Rate & 93.30\% \(\pm\) & 91.2\% \(\pm\) & 77.99\% \(\pm\) & 84–90\% & 87\% \(\pm\) 4.5\% \\  & 0.77\% & 0.9\% & 1.30\% & & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of win rates among different models or algorithms.  
Figure 7: Training curves of the two methods (smoothing parameter = 0.6).  
making it impractical on current hardware. The DQN model replaces the table with a neural network, allowing state-to-action mapping through simple network calculations. The artificial neural network used in this study contains 896 nodes and 676,804 parameters, achieving a two-order-of-magnitude compression of the minefield state space. In summary, the DQN model is a network-driven approach that integrates the feedback mechanism with data compression, and after approximately 8 h of training, it produced a network capable of playing Minesweeper.  
Supervised learning, on the other hand, uses the true labels of the minefield to train the model, predicting the probability of each cell being a mine. The agent progressively uncovers the minefield by selecting the cell with the lowest mine probability, similar to a human player's review and backtracking process. This method is intuitive and efficient, helping to improve the agent's decision-making quality and training stability. The algorithmic structure of supervised learning is similar to that of DQN, and both methods in this study employ the same deep neural network. The key difference is that supervised learning uses simple minefield labels instead of the complex reward mechanism in DQN, significantly simplifying the feedback process and accelerating training. In about 1.8 h of training, supervised learning could produce a network capable of playing Minesweeper.  
An apt analogy for the relationship between DQN and supervised learning would be comparing electric vehicles to gasoline vehicles: while both share similar overall structures, the electric vehicle eliminates more complex components such as the engine and transmission system, replacing them with simpler components such as the power battery system, electric drive, and control systems, leading to superior acceleration and lower maintenance costs.