Archived intermediate verl checkpoints
Selected historical intermediate checkpoints from JaeWooShin's verl research experiments, archived on 2026-09-28 to reclaim local storage. The latest checkpoint of every affected run remains on the source server. These archives preserve the exact original bytes, including all optimizer, RNG, tokenizer, and configuration files that were present. Some older checkpoints did not contain optimizer state; archiving does not add missing training state or convert weights.
Archives under archives/data/ and archives/data1/ mirror the paths relative to
the original /data/jae/ and /data1/jae/ roots. Each .tar.zst has a corresponding
JSON manifest under manifests/, containing its SHA-256, per-file SHA-256 hashes,
sizes, original paths, and the latest checkpoint preserved locally. Each archive
stores its files beneath a checkpoint/ prefix. Manifests may include historical
local paths needed to reproduce the training setup.
No new license is granted by this archival copy. The original base-model and dataset terms continue to apply. These are historical research artifacts, not validated releases or general-purpose inference models. Load serialized PyTorch files only if you trust their source.
Restore an offloaded checkpoint
The source directory keeps an HF_ARCHIVE.json marker with the immutable Hugging
Face commit and archive location. With Python 3.12, huggingface_hub, and zstd
available, download restore.py from this repository and run:
python restore.py --marker /absolute/path/to/global_step_50/HF_ARCHIVE.json
Use --destination /another/checkpoint/path to restore elsewhere. The helper
verifies the downloaded archive and every restored file, and refuses to overwrite
different existing files. Allocate space for both the compressed download and the
restored checkpoint. The download cache can be removed after verifying restoration.
Every source checkpoint is compressed, fully decompressed and checked against its
per-file hashes, uploaded, and checked against the Hub's committed file checksum
before original files are removed. The source retains an independent audit log
and manifests at claude_results/storage_hf_cleanup_20260928 in the verl checkout.