dtokens-checkpoints

Pretrained checkpoints for the run archive of "A modified language model and its observable behavioural signatures" (the d-token toy-scale experimental programme, §6 of the paper). These are the from-scratch base checkpoints that the reported runs start from; they let you re-run or regenerate the experiments.

You do not need these weights to audit the paper's numbers: every quantitative claim is recomputed from the JSON run logs in the archive by verify_numbers.py and the block verifiers, none of which load a checkpoint. The weights are required only to reproduce a run from its starting point.

• Models: 18 small from-scratch transformers (TinyGPT, ~2.73 M params each)
• Format: raw PyTorch state_dict blobs (not 🤗 Transformers models — see Loading)
• Total size: ~189 MB (≈11 MB per file)
• Pinned commit: ab36ea93fdae0fc3b88ddf829289f67932bd4191

Model description

Each checkpoint is a single torch.save dict with four keys:

key	contents
cfg	TinyGPT config: vocab=39, ctx=256, d=192, n_layer=6, n_head=4, dropout=0.0, tie=False
sd	the model state_dict (77 tensors)
rho	the synthetic-corpus conformity rate used in pretraining
seed	the pretraining seed

The architecture is a compact decoder-only transformer (code/model.py in the archive), trained from scratch on a synthetic, rho-controlled corpus — not a fine-tune or derivative of any pretrained model. The values inside each file (cfg, rho, seed) are authoritative.

Loading

These are raw state dicts, so load them directly with the archive's model.py rather than via from_pretrained:

import torch
from model import TinyGPT, Config   # from the run archive's code/ directory

ckpt = torch.load("ckpt_g192_r09_s0.pt", map_location="cpu", weights_only=False)
cfg   = Config(**ckpt["cfg"])
model = TinyGPT(cfg)
model.load_state_dict(ckpt["sd"])
model.eval()

weights_only=False is needed because the file stores the small cfg/rho/seed metadata alongside the tensors. Load only checkpoints you trust (these); the SHA-256 table below lets you confirm you have exactly the published bytes.

Checkpoints

file	rho	seed	role
ckpt_g192_r09_s0.pt	0.9	0	Primary demo base — blocks A/A′ (P1 race), D/D′ (write-channel persistence & ablation), R (R10p/R11p pin), frame-conditioned controls
ckpt_inert_2M_s0.pt	0.9	0	Block-E P2 instrument (qualified pool); B2 = s0 qualification PASS
ckpt_inert_2M_s1.pt	0.9	1	Inert-2M instrument pool (qualification)
ckpt_inert_2M_s2.pt	0.9	2	Block-E P2 instrument (qualified pool); E4 frozen arm
ckpt_inert_2M_r05_s0.pt	0.5	0	Block C rho-precondition map
ckpt_inert_2M_r05_s1.pt	0.5	1	Block C rho-precondition map
ckpt_inert_2M_r05_s2.pt	0.5	2	Block C rho-precondition map
ckpt_inert_2M_r07_s0.pt	0.7	0	Block C rho-precondition map
ckpt_inert_2M_r07_s1.pt	0.7	1	Block C rho-precondition map
ckpt_inert_2M_r07_s2.pt	0.7	2	Block C rho-precondition map
ckpt_inert_r09_s0.pt	0.9	0	Block F paired-pretrain — inert arm
ckpt_inert_r09_s1.pt	0.9	1	Block F paired-pretrain — inert arm
ckpt_inert_r09_s2.pt	0.9	2	Block F paired-pretrain — inert arm
ckpt_nodemo_r09_s0.pt	0.9	0	Block F paired-pretrain — nodemo arm
ckpt_nodemo_r09_s1.pt	0.9	1	Block F paired-pretrain — nodemo arm
ckpt_nodemo_r09_s2.pt	0.9	2	Block F paired-pretrain — nodemo arm
ckpt_g192_r09_s1.pt	0.9	1	Round-3 seed-robustness independent init
ckpt_g192_r09_s2.pt	0.9	2	Round-3 seed-robustness independent init

The two g192_r09_s1/s2 files belong in the archive's round3_seed_robustness/checkpoints/; the round-3 seed-0 init is bit-identical to ckpt_g192_r09_s0.pt and is not duplicated.

Retrieval and integrity

The run archive ships a checkpoint_kit/ that downloads these files, verifies each one's SHA-256 fail-closed, and restores them to their canonical locations:

cd checkpoint_kit
pip install huggingface_hub        # optional; a urllib fallback is built in
python3 fetch_checkpoints.py --all # downloads, SHA-checks, restores to checkpoints/ and
                                   #   round3_seed_robustness/checkpoints/

Or pull a single file directly:

from huggingface_hub import hf_hub_download
p = hf_hub_download("tmbgreaves/dtokens-checkpoints", "ckpt_g192_r09_s0.pt",
                    revision="ab36ea93fdae0fc3b88ddf829289f67932bd4191")

The SHA-256 of each file is the integrity anchor. These digests were computed from the original archive bytes and equal Hugging Face's own recomputed git-LFS oids at upload:

file	bytes	sha256
ckpt_g192_r09_s0.pt	10,962,279	bbd16937209bb558cf39a869c5cfb03a7e9c48c6f263a73d31100767131fecd9
ckpt_inert_2M_r05_s0.pt	10,962,611	0f7cee3a2e56862a5ffb11b9cdb1cc4abb688017ce47394fb71ca9fe69809fba
ckpt_inert_2M_r05_s1.pt	10,962,611	d00df20a821d1fd89bb736a5e8e3032b0054befd1bb5bbdb8f40e0c5b1875624
ckpt_inert_2M_r05_s2.pt	10,962,611	f5897111752bb91e962a31d12e5c8299d92ad318c43cf54e79d8444d01f88dff
ckpt_inert_2M_r07_s0.pt	10,962,611	010647d3db5bd3771aa980aea7046a87c621bf54ff98702f417830360696b37f
ckpt_inert_2M_r07_s1.pt	10,962,611	31961b5b61011235aa2c3c2ce37ae72108b0ea217f9019ca82a7a7d7d15503e1
ckpt_inert_2M_r07_s2.pt	10,962,611	118145f516fb693eaed8f0b445eb67f9d3814b8755aefae663cd473ff28f1ac9
ckpt_inert_2M_s0.pt	10,962,279	f136b2b7915267bee7a0261784c02de19bf034965c80d36c7d8803a3e0728bf6
ckpt_inert_2M_s1.pt	10,962,279	5fa22b25a27b01430469bff620d727588c3d19134a18ad7b4f367776c5a0337c
ckpt_inert_2M_s2.pt	10,962,279	bae7039f2b25029b95a3dfff41d6208b533c9cd9a3ac233d562bb08262e4111a
ckpt_inert_r09_s0.pt	10,962,362	882fa418e8275abcb7dfe06189d794b57e33152e2e5651eca59c6a9f3445ae85
ckpt_inert_r09_s1.pt	10,962,362	5ad49f7be51d460f499c32083b37e5f4a592eab8234a587af3181f48670f6006
ckpt_inert_r09_s2.pt	10,962,362	f3654c4398533b68268a80cbe6785e4711fef0c537684694798dc53b2267f4bb
ckpt_nodemo_r09_s0.pt	10,962,445	562fa8c4e5763f59fee5bd32dd7f8774f0b5dc1160c0a61eea1ba451436fbb8f
ckpt_nodemo_r09_s1.pt	10,962,445	faa7e85dbbde64aa9b953d2efaa3a6df4e7484265e939d78b0f0c3dd062c3de2
ckpt_nodemo_r09_s2.pt	10,962,445	b0e75f91bf08515bd361c6e06a60bcf524b8816a46a1bf7fdb144f78f2337b5b
ckpt_g192_r09_s1.pt	10,962,279	00098b669d96c98ea1f433007a24b98718c93e96d1e45355a541c690dc790a25
ckpt_g192_r09_s2.pt	10,962,279	4ce5201619c211653515f2b43e477769c87bb93921f5d5aad4b5666a77ca9c76

Intended use and limitations

These are research artifacts for reproducibility, not general-purpose language models. They are tiny models (vocab 39, 2.73 M params) trained on a synthetic symbolic corpus; they have no natural-language capability and no use outside reproducing or extending the d-token experiments. Pretraining is bf16/GPU-sensitive: a regenerated checkpoint reproduces the short pretrain bit-exactly on the same hardware, but long online runs are trajectory- dependent and are not guaranteed to reproduce bit-for-bit across machines (see the archive's RESULTS_seed_robustness.md). For that reason, a regenerated checkpoint must be re-qualified (run-plan v5–v6 protocol) before its conformity instrument is trusted.

Provenance and authorship

The paper and its analysis were authored by Claude model instances (Sonnet, Opus, Fable). No human is an author or takes responsibility for the content; a human collaborator contributed the seed idea (the d-token) and orchestrated the sessions — running commands and relaying run plans, critiques, and output between instances — but judged none of the results. Both execution and adjudication were performed by model instances against pre-registered decision tables, and every quantitative claim is independently recomputed from raw logs by the archive's verifiers. See the run archive's README.md and the paper's authorship note and §2 for the full account.

Citation

@misc{dtokens_checkpoints,
  title  = {A modified language model and its observable behavioural signatures
            (d-token run-archive checkpoints)},
  author = {Claude model instances (Sonnet, Opus, Fable)},
  year   = {2026},
  howpublished = {Hugging Face Hub: tmbgreaves/dtokens-checkpoints},
  note   = {Commit ab36ea93fdae0fc3b88ddf829289f67932bd4191}
}

License

Released under Apache-2.0. These models are trained from scratch on a synthetic corpus and are not a derivative of any third-party pretrained model, so no upstream license applies; Apache-2.0 is a standard permissive choice for model weights and adds an explicit patent grant. (MIT or CC-BY-4.0 would be reasonable alternatives if you prefer a shorter or attribution-style license — change the license: field above to match if you switch.)