# Imitating Human Go Players via Vision Transformer
## 3 Materials and Methods
### Vision Transformer for Go
The resulting tensor, denoted as \(z_{0}\), serves as the input to the Transformer encoder. Formally,  
\[z_{0}=[x_{class};p_{1}L;p_{2}L\ldots;p_{1}0L]+E_{pos},\ L\in\mathbb{R}^{(10 \cdot 7^{2})\times 768},E_{pos}\in\mathbb{R}^{(10+1)\times 768} \tag{1}\]  
This formulation ensures that the input data are structured in a way that fully utilizes the ViT's capability to model the complex spatial and strategic relationships inherent in Go gameplay. The Transformer encoder blocks used in our study are illustrated in Figure 3. To evaluate the impact of model depth on performance, we experimented with three different configurations of Vision Transformer (ViT) by varying the number of encoder blocks: 4, 8, and 12 blocks. Each encoder block comprises two key components: a multi-head attention (MA) module and a multi-layer perceptron (MLP) module. These components work together to model both local and global dependencies within the input data, enabling the network to capture the complex patterns inherent in Go gameplay. The operations within each encoder block are described mathematically as follows:  
1. **Multi-head Attention Module**: The output of the multi-head attention module is computed as follows: \[z_{l}^{\prime}=MSA(LN(z_{l-1}))+z_{l-1},\ l=1\ldots L\] (2)  
Here, \(z_{l-1}\) is the input to the \(l\)-th encoder block, LN denotes layer normalization, and MSA represents the multi-head self-attention mechanism. The residual connection (\(+z_{l-1}\)) ensures stability during training by mitigating issues such as vanishing gradients. Figure 5: Vision Transformer for Go. It should be noted that the images of Go are for illustrative purposes. The actual data are the features from Table 1. The illustration of the figure was inspired by [6]. 2. **Multi-layer Perceptron Module**:  
The output of the MLP module is calculated as follows:  
\[z_{l}=MLP\big{(}LN\big{(}z_{l}^{\prime}\big{)}\big{)}+z_{l}^{\prime},\ l=1\dots L \tag{3}\]  
where \(z_{l}^{\prime}\) is the intermediate representation from the MA module, and the residual connection (\(+z_{l}^{\prime}\)) further enhances training stability. In this study, \(L\) is set to 4, 8, and 12 to compare the performance of Vision Transformer (ViT) models with varying depths. By experimenting with these different configurations, we aim to evaluate the trade-offs between model complexity and predictive accuracy in the task of mimicking expert-level Go gameplay. After processing the input through all \(L\) encoder blocks, the output from the first embedding (corresponding to the class token) of the last encoder block undergoes layer normalization. This normalized representation serves as the basis for generating the final score for each possible move on the Go board. These scores are then converted into a probability distribution over all potential moves using the softmax function. The softmax operation ensures that the scores are transformed into a normalized probability distribution, where the probabilities of all possible moves sum to 1. Mathematically, the softmax function is defined as follows:  
\[softmax(z)_{i}=\frac{e^{z_{j}}}{\sum_{j=1}^{n}e^{z_{j}}} \tag{4}\]  
where \(\text{z}=[z_{1},z_{2},\dots,z_{n}]\) represents the input logits, and \(softmax(z)_{i}\) is the probability assigned to the \(i\)-th class (or move). Here, \(n\) corresponds to the total number of classes, which, in the context of a 19 \(\times\) 19 Go board, is 361 (excluding the pass move). This final probability distribution indicates the model's confidence for each possible move and is used to select the next move during inference, thereby enabling the model to imitate the decision-making process of expert Go players.