YdkGen: Transformer-Based Yu-Gi-Oh! Deck Generator (Pure PyTorch)

A PyTorch-native causal transformer model purpose-built for generating valid Yu-Gi-Oh! deck lists (.ydk files). This model is adapted to the unique constraints of trading card game (TCG) deck building—departing from standard causal language models to support unordered, repeatable card collections with Yu-Gi-Oh!-specific format and banlist rules. Github.

Model Architecture

YdkGen implements a causal transformer optimized for unordered, repeatable class collections, with core design modifications to address the fundamental differences between TCG deck generation and traditional language modeling. Unlike standard causal LMs, deck generation requires handling:

• Unordered card sequences (deck order does not impact play)
• Repeatable cards (0–3 copies per card, per Yu-Gi-Oh! rules)
• Effective duplicate ID restrictions
• Fixed main/extra deck card count limits
• Banlist (Limited/Forbidden) card restrictions
• Continuously growing vocabulary (new Yu-Gi-Oh! cards are regularly released)
• Ambiguous "correctness" (no single optimal deck build for a meta/archetype)
• Under-specified deck building logic (context-dependent play patterns)

Key Design Choices

To resolve these challenges, the model uses a minimal causal transformer backbone paired with heavy train/test-time data augmentation and beam search inference. Critical architectural optimizations include:

1. Prior Knowledge Embedding: Injects all card metadata (from cards.cdb, strings.conf, and Lua scripts) directly into the embedding layer—eliminating hard-coded card ID dependencies in the embedding space and reducing vocabulary overfitting.
2. Unseen Card Compatibility: Dropout applied directly to card ID inputs enables theoretical support for unseen cards (with unmapped IDs) by falling back to non-ID metadata pathways (untested).
3. Batch Construction: Decks are shuffled and zero-padded to form fixed-length batches, with causal training used as a flexible train-all-length strategy for variable deck sizes.
4. Multi-Mode Loss Function: Three specialized loss modes tailor the model’s prediction objective to different deck generation strategies (details below).

Loss Modes

The model supports three distinct prediction loss modes, each optimizing for a different deck distribution forecasting strategy:

• MODE 0: Predicts the full deck card distribution (normalized softmax) at every token step. The full deck can theoretically be inferred from the first token; greedy sampling of the highest-confidence card (iteratively) improves generation accuracy (analogous to diffusion model sampling).
• MODE 1: Predicts the remaining deck card distribution (normalized softmax) at every token step (outputs class 0 if the remaining deck is empty).
• MODE 2: Predicts logits for any remaining card (all valid remaining cards score equally). Probabilities are aggregated via logsumexp—this mode exhibits poor performance and is provided only as a baseline.

Inference

Inference compensates for the selected training loss mode and leverages beam search to enhance generation quality and adherence to Yu-Gi-Oh! deck rules.

Prerequisites

Dependencies

Install required Python packages via pip:

pip install -r requirements.txt

Required Data Files

The model relies on Yu-Gi-Oh! card/format metadata and training deck data—place these files in the project root with the following structure (copy from YGOPro or it's successors):

• deck/*.ydk or deck/*/*.ydk: Training data (valid Yu-Gi-Oh! deck lists)
• cards.cdb: Comprehensive card metadata (stats, archetypes, types, etc.)
• lflist.conf: Official Yu-Gi-Oh! banlist (Limited/Forbidden/Restricted; uses the latest version by default)
• strings.conf: Card set name metadata (currently unused in the current implementation)
• script/c*.lua: Card effect/implementation logic (for advanced metadata extraction)

Training

Initiate training with the default configuration using the following command:

python train.py

Custom Training Configuration

All hyperparameters and training settings are configurable directly in train.py for custom runs. Below are the key tunable parameters with their default values and descriptions:

# Model Checkpoint Configuration
FILE = 'model0'     # Auto-saves/loads checkpoints to/from {FILE}{N}.pth
# Hardware/Precision
DEV = 'cuda'        # Use 'cpu' for systems without a CUDA-enabled GPU
AMP = torch.float16 # Use torch.float32 if DEV = 'cpu' (no mixed precision)
# Transformer Architecture
DIM = 256           # Core model embedding/hidden dimension
LAYER = 5           # Number of transformer encoder/decoder layers
HDIM = 16           # Dimension per attention head
MLP = 4/3           # MLP hidden dimension scale factor (relative to DIM)
DROP = 0.1          # Global dropout rate (applied to embeddings/attention/MLP)
# Optimization
LR = 0.0007         # AdamW learning rate
WD = 1e-4           # AdamW weight decay (regularization)
BS = 16             # Training batch size (adjust for GPU memory)
# Training Loop
EPOCH = 1000        # Number of checkpoint save iterations
ITER = 20000        # Training steps per checkpoint save
# Loss Mode (match inference mode)
MODE = 0            # 0: Full deck distribution (softmax) at each token
                    # 1: Remaining deck distribution (softmax) at each token
                    # 2: Remaining card logits (logsumexp aggregation; baseline)

Pretrained Checkpoints

Pretrained model checkpoints and tokenizer data are available via multiple mirrors (Hugging Face, HF Mirror, ModelScope) for immediate inference/finetuning. All checkpoints adhere to Yu-Gi-Oh! format rules and are trained on the provided YDK dataset.

Name	Mirror 1 (Hugging Face)	Mirror 2 (HF Mirror)	Mirror 3 (ModelScope)	Description
model.i	model.i	model.i	Tokenizer data (binary numpy.int32 array format)
model0.pth	model0.pth	model0.pth	Model checkpoint (MODE 0; PyTorch .pth format)
model1.pth	model1.pth	model1.pth	Model checkpoint (MODE 1; PyTorch .pth format)
model2.pth	model2.pth	model2.pth	Model checkpoint (MODE 2; PyTorch .pth format) – poor performance (baseline)

Dataset

The model is trained on a curated small dataset of valid Yu-Gi-Oh! .ydk deck lists, available via the same mirrors as the pretrained checkpoints:

Name	Mirror 1 (Hugging Face Datasets)	Mirror 2 (HF Mirror Datasets)	Mirror 3 (ModelScope Datasets)	Description
ydk	ydk	ydk	Curated Yu-Gi-Oh! YDK dataset

Inference & Evaluation

Run the script to generate Yu-Gi-Oh! decks with the pretrained model (or custom checkpoints) and evaluate generation quality. Generated decks are saved as .ydk files for direct use in Yu-Gi-Oh! simulators.

Run Inference/Evaluation

python eval.py

Custom Inference Configuration

Tune inference parameters in eval.py for custom deck generation—ensure the MODE matches the checkpoint’s training loss mode for valid results:

FILE = 'model0'  # Loads checkpoint from {FILE}.pth; saves generated decks to deck/{FILE}.ydk
DEV = 'cpu'      # CPU is sufficient for inference (CUDA not required)
MODE = 0         # Training loss mode of the checkpoint (MANDATORY MATCH)
NUM = 55         # Total cards in generated deck (follows Yu-Gi-Oh! official format rules)
BEAM = 256       # Beam search width (higher = better generation, slower inference; reduce for resource-constrained systems)