eoinf/trigram_components_016_2b_schedule_200m_dmodel256

Transformer experiment grid

Tiny causal transformers with d_model values 256, 1 seed(s) per width, vocabulary size 4096, context length 256, and a nominal schedule of 1,999,998,976 training tokens. Actual per-model horizons (including early stops) are recorded in grid_summary.json and result.json.

Dataset: see configuration.json; tokenizer: eoinf/pile_tokenizer_4096.

Evaluation uses training data; there is no held-out evaluation.

Model architecture and loading code are in the synthetic_trigrams repository. Each model directory contains state-dict checkpoints, the run definition, checkpoint history, training-update log, and result metadata. Optimizer resume states and training data are not included.