# Imitating Human Go Players via Vision Transformer
## 3 Materials and Methods
### Vision Transformer for Go  
In this work, we aim to demonstrate the capability of the Vision Transformer (ViT) to imitate the moves of human experts in the game of Go. This problem is formulated as a classification task, where each possible move on the board is treated as a distinct class. Given the 19 \(\times\) 19 grid of the Go board, this results in a total of 361 possible classes, corresponding to all board positions, excluding the pass move. Each class represents a unique location on the board where a stone can be placed, and the model's goal is to predict the next move based on the current board configuration, mimicking the decisions made by expert players. This section provides an overview of the key components involved in applying the Vision Transformer (ViT) to the task of imitating expert-level gameplay in Go. For a comprehensive theoretical background on the ViT, readers are encouraged to refer to [6; 16]. Our methodology draws inspiration from [6], particularly for Equations (1) and (2), which outline the mathematical foundations of the ViT. These equations have been adapted to align with the specific requirements of our task, demonstrating how the ViT can be effectively applied to analyze board positions and predict the next move in Go. An overview of our method is presented in Figure 5. In the ViT, the input features must be reshaped into a sequence of feature patches. The features from Table 1 have the shape \(\mathbb{R}^{10\times 19\times 19}\), which cannot be reshaped directly because 19 is a prime number. Therefore, the features are first padded with zeros to form \(\mathbb{R}^{10\times 21\times 21}\). We set the patch size to 7; accordingly, the features are broken down into 9 patches, each of shape 7 \(\times\) 7. These patches are then flattened to form a sequence of 2D patches \(p\in\mathbb{R}^{9\times(10\cdot 7^{2})}\). We use a learnable linear projection to project each patch into \(\mathbb{R}^{768}\), resulting in projected patches \(p_{proj}\in\mathbb{R}^{9\times 768}\). After prepending an additional class token \(x_{class}\), positional embeddings are added to incorporate spatial information, which is crucial for identifying long-range patterns on the board. The resulting tensor is used as input for the Transformer encoder, denoted by \(z_{0}\). Formally, we define:  
With the patch size set to 7, the padded features are divided into 9 patches, each of size 7 \(\times\) 7. These patches are then flattened into a sequence of 2D patches represented as \(p\in\mathbb{R}^{9\times(10\cdot 7^{2})}\). A learnable linear projection is applied to each patch, projecting the \begin{table}
\begin{tabular}{c c c} \hline
**Feature** & **Shape** & **Description** \\ \hline Black & (1, 19, 19) & The positions of black stones are marked with 1. \\ White & (1, 19, 19) & The rest are marked with 0. \\ Invalid & (1, 19, 19) & The positions of invalid moves are marked with 1. \\ Turn & (1, 19, 19) & The rest are marked with 0. \\ Ones & (1, 19, 19) & An entire plane is marked with 1. \\ Empty & (1, 19, 19) & The positions that are not occupied by a stone are marked with 1. The rest are marked with 0. \\  & & The features that store the most recent 4 moves. \\ Recent moves & (4, 19, 19) & The ith plane of this feature stores the feature of the ith most recent move. The position of the move is marked with 1. The rest are marked with 0. \\ \hline \end{tabular}
\end{table} 
Table 1: The features extracted from raw positions. patches into a higher-dimensional space of \(\mathbb{R}^{9\times 768}\). This step produces the projected patches \(p_{proj}\in\mathbb{R}^{9\times 768}\), which are now suitable for processing by the ViT. Before passing the patches into the Transformer encoder, an additional learnable class token \(x_{class}\) is prepended to the sequence. Positional embeddings are added to the sequence to encode spatial information, which is critical for recognizing long-range dependencies and patterns on the Go board.