ryan-0608/MoS-DFlash-Qwen3-4B-Training-Archive

MoS DFlash Qwen3-4B Training Archive

This repository contains the Qwen3-4B scale-validation checkpoint trajectories: the matched-volume generalist, D0-initialized MoS, and generalist-initialized MoS. It does not contain Qwen3.5 artifacts.

Pinned dependencies

• Target: Qwen/Qwen3-4B@1cfa9a7208912126459214e8b04321603b3df60c
• D0: z-lab/Qwen3-4B-DFlash-b16@b74e3a329c4d963783143b1e970d95b002be72bd
• The target tokenizer, embedding, LM head, and full LM weights are not redistributed.

Layout and lineage

dflash_legacy.py
dflash_router.py
experiments/
  generalist/matched-volume/
  joint-mos/d0-init/
  joint-mos/generalist-init/
releases/
registry/
schema/
provenance/original-code/<sha256>/

The generalist-initialized run starts from the retained generalist trajectory snapshot at step 33,750. It is not total-volume matched to the D0-initialized arm; the artifact manifest records both continuation and cumulative exposure.

Legacy checkpoints use root code SHA-256 08625e679c8a.... The final integrated request-level-router checkpoints use root code SHA-256 8c046f3dbbfa.... Always load a pinned revision with trust_remote_code=True and the exact manifest load_subfolder.

Resume and data boundary

Resume state is stored in the separate public Storage Bucket and bound by SHA-256. The two MoS runs used accumulation_steps=2, so their complete state is classified stateful-non-bitwise, not bitwise exact.

No raw prompts, prepared datasets, hidden-state features, activations, or per-sample model outputs are included. Reviewed aggregate evidence is published separately.

License

The released draft weights and MoS code are MIT. Qwen3-4B is an Apache-2.0 runtime dependency and is not redistributed. See LICENSE and THIRD_PARTY_NOTICES.md.