Transformer Reversi AI
This Reversi AI is a Transformer-based model that uses Mixture-of-Experts architecture for decision-making instead of CNNs.
Github - rsu-Suba/ReversiGPT
Architecture
- Parameters: 1.68M
- Model size: 20.6MB
- Input:
(8, 8, 2)- Own stones
- Opponent stones
- Board representation: 64 tokens
- Embedding dimension: 128
- Transformer blocks: 4
- Attention heads: 4
- MHA experts: 2
- FFN experts: 2
- Routing: softmax-based dynamic routing
- Iterative routing steps: 2
- Normalization: Pre-LN LayerNorm
- FFN activation: GELU
Diagram
Input (8×8×2)
↓
64 Tokens
↓
Dense 128
↓
Token + Row + Column + Move Embeddings
↓
DynamicAssembly × 4
├─ MHA Expert × 2
└─ FFN Expert × 2
↓
├─ Policy Head → 64 moves
└─ Value Head → Win-rate
DynamicAssembly
Each DynamicAssembly block contains a shared pool of MHA and FFN experts. At each iteration step, the mean-pooled board representation is combined with a step embedding and passed to a router.
The router produces a probability distribution over the expert pool. All experts are evaluated and their outputs are combined using the router probabilities.
Policy Head
The policy head predicts a probability distribution over the 64 board positions.
Value Head
The value head predicts a scalar value in [-1, 1], representing
the estimated game outcome.
Training
This model was trained using self-play game records generated by a previous CNN-based Othello AI.
Self-Play data generation
- MCTS simulations: 600
- MCTS batch size: 100
- PUCT constant: 2.3
- Total games: 10,000
Model Training
- Optimizer: AdamW
- Clipnorm: 1.0
- Weight decay: 0.05
- Learning rate: 6.0e-5
- Learning rate scheduler: ReduceLROnPlateau
- Training batch size: 256
- Epochs: 75
Evaluation
A 63M-parameter ResNet CNN model was used as the previous model.
Win rate
- vs Random : 100% (100 / 100 games)
- vs Prev model : 63% (63 / 100 games)
Average final stone count
- vs Random : 58
- vs Prev model : 47
Usage / Quick start
import numpy as np
import tensorflow as tf
from huggingface_hub import hf_hub_download
from model import TokenAndPositionEmbedding, MHA, FFN, DynamicAssembly
model_path = hf_hub_download(repo_id="rsu/Reversi-Transformer-1", filename="M1.h5", repo_type="model")
custom_objects = {
"TokenAndPositionEmbedding": TokenAndPositionEmbedding,
"MHA": MHA,
"FFN": FFN,
"DynamicAssembly": DynamicAssembly,
}
model = tf.keras.models.load_model(model_path, custom_objects=custom_objects, compile=False)
board = np.zeros((1, 8, 8, 2), dtype=np.float32)
board[0, 3, 3, 1] = 1.0
board[0, 3, 4, 1] = 1.0
board[0, 3, 3, 1] = 1.0
board[0, 4, 4, 1] = 1.0
policy, value = model(board, training=False)
print("Policy (64 moves prob): ", policy.numpy())
print("Value (win rate est [-1, 1]): ", value.numpy().item())