packed-long-h100

Training checkpoints of the exp_train 700M hybrid LM (Gated DeltaNet + gated attention, 600M hashed n-gram memory) on Ba2han/packed1. wandb: project exp_train, run packed-long-h100.

Each step_XXXXXXXX/ folder is one complete checkpoint. The newest two are kept; older ones are deleted and the history squashed, so they stop using storage.

File	Content
model.safetensors	dense weights (FP32; the tied embedding is stored under both names)
ngram.safetensors	n-gram table (BF16) and its RMSprop row state
optim_rank0.safetensors + .json	optimizer state (resume only)
state_rank0.safetensors + .json	RNG streams and layer-dropout counters (resume only)
config.json	model configuration and special-token ids
tokenizer.json, tokenizer_config.json, special_tokens_map.json	the 60,800-token tokenizer
meta.json	step, tokens seen, FP64 checksums of every entry

Check a checkpoint: python code/verify_checkpoint.py --hub Ba2han/packed-long-h100