eoinf/empirical_information_redistributed_spread_200m_dmodel2_4_6

Transformer experiment grid

Tiny causal transformers with d_model values 2, 4, 6, 1 seed(s) per width, vocabulary size 4096, context length 256, and a nominal schedule of 1,999,998,976 training tokens. Actual per-model horizons (including early stops) are recorded in grid_summary.json and result.json.

Dataset: see configuration.json; tokenizer: see configuration.json.

Evaluation settings are recorded in configuration.json.

Model architecture and loading code are in the synthetic_trigrams repository. Each model directory contains state-dict checkpoints, the run definition, checkpoint history, training-update log, and result metadata. Optimizer resume states and training data are not included.