Quasar-230K — Model Lineage and Audit Record
This repository is the public technical disclosure for the private
silx-ai/Quasar-230K weight repository. It records the checkpoint’s
initialization, transformations, later optimizer work, evaluation limitations,
and reproducible audit results.
Model lineage
Quasar-230K uses inclusionAI/Ling-mini-base-2.0-20T as its disclosed
sparse-MoE/base initialization. Quasar adds a custom hybrid-attention system
using Quasar, Raven, and GLA branches.
This lineage must not be interpreted as from-scratch or independently sourced pretraining. Weight inspection can establish stored differences and functional changes, but it cannot authenticate a historical step count or training-token count by itself.
Early coordinate conversion
An early checkpoint-conversion experiment expressed inherited tensors in a different, functionally equivalent coordinate basis. The recorded operations included:
- paired RMSNorm/projection sign changes;
- residual and intermediate-channel permutations;
- group-local physical expert-storage permutations with preserved logical routing;
- common per-layer router-bias shifts that preserved top-k ordering.
The experiment explicitly referenced inclusionAI/Ling-mini-base-2.0-20T and
used absolute raw weight-cosine targets of 0.02 for selected tensors and
0.2 for RoPE-compatible Q/K signs. These transformations were designed to
preserve behavior. They are not additional pretraining and are not evidence of
provenance independence.
The exact historical sign vectors, permutations, seeds, router shifts, source hashes, and expert-storage maps are retained in the sealed provenance evidence.
Canonical expert storage
The final packed expert tensors were physically rewritten into canonical logical order. Logical router expert ID now equals physical expert index, so a runtime expert permutation map is no longer required.
Canonicalization performed no optimizer update. It rewrote 38 packed expert tensors across 19 shards and verified:
- all 38 canonical expert tensors against the parent’s logical experts;
- 523 non-expert tensors in rewritten shards as bit-exact to the parent;
- the twentieth weight shard as unchanged;
- absence of the runtime expert-storage permutation contract.
The four-GPU EP4 ten-token sentinel generated token ID 1594 at all ten steps
both before and after canonicalization. Top-two token identities matched on all
ranks at every step. Eight of ten steps had identical reported BF16 top-two
values. The other two differed by at most 0.875; an unchanged-parent control
rerun showed the same two-step variation and maximum delta under the same
distributed grouped-GEMM kernel.
Expert optimizer work
The final experts are not merely reordered or sign-flipped copies of the initial experts. An exhaustive audit covering all 19 × 256 = 4,864 experts, after applying the documented historical inverse alignment, reported:
| Measurement | Result |
|---|---|
| Aggregate magnitude cosine | 0.685631116 |
| Highest recovered expert cosine | 0.753024142 |
| Relative Frobenius error | 0.856687561 |
| Exact inverse-aligned element fraction | 0.124332% |
| Fully exact expert components | 0 |
Part of the expert work used an explicitly disclosed Ling-referenced, inverse-aligned weight-space objective. The final fast expert stage did not perform a language-model/data forward pass. It must be described as source-aware expert weight-space optimization, not independent pretraining.
Quasar-specific branch training
The Quasar-specific branch system contains 298,310,800 parameter slots
(1.802% of the checkpoint). All 107 large branch matrices received optimizer
gradients during the documented branch-training stage.
| Measurement | Result |
|---|---|
| Aggregate magnitude cosine versus branch starting state | 0.886760701 |
| Relative Frobenius error versus branch starting state | 0.553353038 |
| Exact unchanged element fraction | 0.304765% |
| Held-out causal loss with branches active | 7.180161 |
| Held-out causal loss with only the GQA path | 7.899890 |
| Loss increase when branches were removed | 0.719729 |
These movement, gradient-coverage, and ablation measurements are evidence of targeted branch training and functional branch use. They do not establish that the entire inherited checkpoint was independently pretrained.
ARC-E disclosure
The final ARC-E experiment directly exposed ARC-E test questions and answer labels during optimization. All resulting ARC-E scores are contaminated, in-sample measurements. They are not clean and must not be used as leaderboard- comparable ARC-E results.
Claims supported by the evidence
The available evidence supports the following description:
Quasar-230K is a Ling-initialized sparse-MoE checkpoint with Quasar hybrid- attention architecture, materially modified experts, trained Quasar-specific branches, and fully disclosed function-preserving coordinate conversions.
The evidence does not establish from-scratch pretraining, independent model origin, a clean ARC-E score after label exposure, or a historical 230,000-step claim solely from the stored weights.
Sealed revision and evidence hashes
The canonical expert-storage revision was produced from parent revision
8bcf01098510825472aefe7215cd33db808f7825 and published as revision
383c90e8c214cad159db6c825e5aad62d0911c39 before the repository became
unavailable to the current authenticated API session.
| Artifact | SHA-256 |
|---|---|
| Canonical runtime config | d77b6ee29bca3f92a47876fafe41e88feae33555d1bdafcbf8d7ca4d9cf9096b |
| Full transformation provenance | 26c7c41aef131da6d55f15e4534e6eb2f6c30598e032ea50a32f42e89fa93657 |
| Canonicalization report | 833a908f21cfb7f6adf89aa58ad73d391efbd83ca6f6b8be955f5787985d5743 |
| Canonicalization equivalence report | 6bbd89354268da593584871969e7b786ec78eaab2cc3517afe780912f04fec5f |
| Remote verification report | 7ec94481ae6e634a58f98eca95210202edd59a8fecacb274e1b75aa6123c6471 |
The private weight repository should retain a minimal pointer to this disclosure:
"provenance_url": "https://huggingface.co/silx-ai/Clear-Quasar-230k"