YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
GPT-2 Checkpoint Archive
This repository contains the selected checkpoints from the legacy-RoPE, attention-position, and Music Transformer experiments.
This upload phase also contains the selected trajectory snapshots.
Selection
- Adam full and Muon legacy-RoPE BF16 trajectories, seeds 42/43/44
- trajectory snapshots every 1,000 steps through 10,000, plus each run's
last_checkpoint.pt - available historical legacy-RoPE BF16/FP32 comparison trajectories
- ALiBi and sinusoidal-PE last checkpoints at the requested learning rates
- Music Transformer Adam last checkpoint and Muon last checkpoint with PE trained by Muon
- no NorMuon checkpoints
There are 99 repository checkpoint paths backed by 93 unique
local files (122.22 GiB logical, 115.07 GiB unique). The duplicate
paths are intentional: last_checkpoint.pt can point to the same snapshot as step 10,000.
The old Muon seed42 BF16/FP32 comparison directories had already been cleared and are
listed under missing_requested_families in selection.json. The available Muon FP32
seed43 tail (steps 7000-10000) is included and marked partial.
See manifest.csv or manifest.json for exact paths and metadata. These PyTorch
checkpoints can contain optimizer and RNG state; only load pickle-based files from
sources you trust.