# Official International Mahjong: A New Playground for AI Research
## 5 AI Algorithms
### Supervised Learning  
Supervised learning was also widely used, especially in the second Mahjong AI competition, probably because of the availability of high-quality datasets. The basic idea of supervised learning is to train a policy model to predict the action to choose under the current observation from the match dataset in advance, and directly use the model to compute an action during the gameplay. Since the policy model is learned directly to clone the behavior in the match dataset, the performance of the model depends on the quality of the dataset. Despite the low quality of the dataset provided in the first competition, which consists of matches of human players of different levels in another Mahjong variant, agents trained by supervised learning still outperform those using heuristic methods, indicating the huge potential of deep learning algorithms on this game. More participants turned to supervised learning in the second competition, making up most of the top 16 agents. The supervised models of different teams differ mainly in three components: the design of the features, the design of the network model, and the data preprocessing scheme. Most teams designed the features as all visible information from the current player, including the current hand, the exposed medals and their types for each player, the discarded tiles of each player, the prevalent wind and the seat wind, and the number of unseen tiles. Some teams missed some of the features or caused information loss by combining some of the features, such as ignoring the order of the discarded tiles. Meanwhile, these features were encoded differently as tensors by different teams. The most basic method is to treat all of these features as vectors and stack fully connected layers as the network model. While such encoding can retain all useful information, it overlooks the spatial relationship between the Mahjong tiles, since most of the scoring patterns involve special patterns of tiles such as pumps and shifted chows. Suphx [12], an AI system built by MSRA to play Richi Mahjong, encodes the hand as 4 channels of images of \(34\times 1\), and uses 1-dimensional convolutional neural networks (CNN) to extract high-level features from the multi-channel images. Inspired by and improved from the design of Suphx, many teams in the competition encoded the tiles as 4 channels of images of \(4\times 9\) to better capture the spatial relationship between suited tiles of the same number. The encoding of both schemes is illustrated in Figure 5.  
While the designs of features differ in most of the AI agents using supervised learning, the action space of different agents is quite similar, including winning, discarding, and different types of melding. Most of the teams used variants of CNN as the network model to predict the action, a method that is widely used in the field of computer vision to extract features from images due to its strong capability to capture spatial patterns. Specifically, the agents in the first Mahjong AI competition mainly use Resnet [31] as the CNN model, while other variants of CNN such as ResNeXt [32] can also be seen in the agents submitted in the second competition. These variants of CNN adopt deeper layers by adding shortcut connections, or use kernels with more complex structures to increase the ability to extract high-level features. Figure 5: The encoding of features in Suphx and some AI agents submitted by the competitors. (**a**) Tiles encoded as 4 channels of images of \(34\times 1\). Image from [12]. (**b**) Tiles encoded as 4 channels of images of \(4\times 9\). Image from [30]. Before training the network, the given dataset of matches is first preprocessed into millions of state-action pairs. The schemes of data preprocessing of these teams are also different from each other. Some teams simply took all of the actions made by each player and the corresponding observation of the player as the states in these matches.