REAM-288 Rank-32 3M-token adapter

Rank-32 LoRA continuation for the REAM-288 pruned mixed-precision base model. The frozen expert FC1/FC2 tensors used native FlashInfer grouped NVFP4 kernels; non-expert matrices remained BF16 and the PLE remained higher precision.

Pinned inputs

Base: jakeatx/slimder-qwen38-ream288-depth32-agentic-ngram50-mixed-nvfp4-v2 at revision 76585f07af88aaba3649723a150b28dd8c2a24dc
Dataset: jakeatx/august-2026-frontier-traces at revision cf4ca93f3be1cbf59514ec508dee69a04c5d978f
Eligible rows: 12,558, with a fixed 5% validation split
Objective: assistant/reasoning supervised tokens only; dataset rows were not trimmed or altered for this run

Training result

Final supervised-token cursor: 3,001,072
Final validation loss (32 windows): 0.6171666597
Validation progression: 0.6332411821 at 1.20M, 0.6297578955 at 1.50M, 0.6188944065 at 2.00M, 0.6211808663 at 2.50M, and 0.6171666597 at 3.00M
Final continuation throughput: 42.13 supervised tok/s and 98.49 rendered tok/s
LoRA rank/alpha: 32/64
Adapter file size: approximately 156 MiB

Training losses are naturally noisy across differently sized traces. The held-out validation loss finished at its best measured checkpoint, improving about 2.54% relative to the 1.20M checkpoint.

Files

adapter_model.safetensors and adapter_config.json: inference adapter
training/: optimizer, RNG, and trainer state needed for exact continuation
evidence/: resolved configuration, full JSONL training log, source checkpoint hashes, and a concise summary
SHA256SUMS: hashes for every published payload file

The custom REAM/PLE architecture and mixed-NVFP4 runtime are supplied by the pinned base repository; generic Transformers loading may not exercise the native grouped-MoE path.

Qwen3.8 Perian project lineage

This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Final rank-32 QLoRA adapter repository. The milestone at wall token 9,336,692 is the adapter merged into the Perian GGUF release.

The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:

depth: 48 to 32 transformer layers;
routed-expert width: 384 to 288 experts per layer;
PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.

Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.