Soulfate24/AutoRound-ASHQ1-Remix_Double-Quantization_Suite

AutoRound & ASHQ1-Remix Suite

Version 2.1.1 — Activation-aware GGUF quantization whose every ratio, floor, and cap traces to a measured experiment. Plain-BF16-native first; AutoRound lineage supported with explicit saturation bounds.

---

🌟 Highlights

┌────────────────────────┐
│ Safetensors (Raw/BF16) │
└────────────┬───────────┘
             │ 00_SAFETENSORS-to-AutoRound-BF16-GGUF.py (optional, int4 sources)
             ▼
┌────────────────────────┐
│ BF16 GGUF + provenance │
└────────────┬───────────┘
             │ 01_create-calibration-dataset-and-imatrix.py (lineage-aware)
             ▼
┌─────────────────────────────────────────────────┐
│ 02_BF16-GGUF-to-ASHQ1.py                        │
│  ASHQ1-Remix optimizer (knapsack + floors)      │
│  + ASHQ1-mmproj.py (vision tower, imatrix-free) |
└────────────┬────────────────────────────────────┘
             ▼
┌──────────────────────────────┐
│ Tiers: Nano 27 · Mini 30 ·   │
|  Compact 33 · Quality 36 (%) │
└────────────┬─────────────────┘
             ▼
┌──────────────────────────────────────────────┐
│ 03_perplexity_test.py  +  90_attribution-    │
│ probe.py          PPL/KLD/RMS/top-p · probes │
└──────────────────────────────────────────────┘

• Measurement-backed: the Calibration Ledger L0–L6 documents six laws, each rooted in knockout probes archived in attribution-results.csv.
• Four standardized tiers on the arithmetic 27/30/33/36 ladder (+Fidelity opt-in), strictly monotone in KLD/RMS/top-p on every validated family.
• Cross-family proven: qwen35 hybrids (4B & 9B) and dense archs (SmolLM3 3B, llama 1B).
• Honest failures: floors never lie — impossible targets warn instead of silently degrading safety classes.
• Stock-defended: every tier wins its duel against the size-matched uniform quant (L6); archival ratios defer to stock Q8_0.

---

📊 Tiers & Selection Policy

Tier	Plain ratio	Role
Nano	27%	Edge cases / maximum compression (blocked by default on int4 lineage)
Mini	30%	Minimum for ≥9B serving
Compact	33%	Minimum for 3–4B; balanced deployment
Quality	36%	Minimum for ~1B; near-lossless general deployment

Minimum-tier policy (measured, not folklore — see Charter §7). Smaller models hit the constructibility wall sooner: a 1B below 33% stops differentiating adjacent tiers, while a 9B tolerates 30% comfortably. AutoRound-int4 lineage: practical ceiling is Compact (information ceiling at equal size); Nano excluded unless ASHQ1_INCLUDE_NANO=1 after validation.

---

📈 Release Benchmarks (wiki.test.raw, symmetric FA-auto reference)

Ornith-1.5-9B

Tier	Size	PPL	KLD	RMS Δp	top-p
Quality-36pc	6170 MiB	9.3677	0.0365	4.97%	93.2%
Compact-33pc	5650 MiB	9.6526	0.0505	5.83%	91.7%
Mini-30pc	5225 MiB	9.8564	0.0649	6.67%	90.3%
Nano-27pc	4625 MiB	10.0184	0.0856	7.70%	88.1%

Qwen3.8-4B-Distill

Tier	Size	PPL	KLD	RMS Δp	top-p
Quality-36pc	2902 MiB	9.1124	0.0095	2.56%	95.1%
Compact-33pc	2661 MiB	9.1963	0.0160	3.31%	93.7%
Mini-30pc	2420 MiB	9.2973	0.0258	4.26%	92.1%
Nano-27pc	2179 MiB	9.4305	0.0417	5.56%	89.1%

TwIL-LM3 (dense SmolLM3-arch, tied token_embd, 128k vocab, L7-fixed Nano)

Tier	Size	PPL	KLD	RMS Δp	top-p
Quality-36pc	2121 MiB	9.5785	0.0163	3.35%	92.4%
Compact-33pc	1945 MiB	9.6292	0.0235	3.94%	91.2%
Mini-30pc	1769 MiB	9.6890	0.0304	4.42%	90.2%
Nano-27pc	1593 MiB	9.9084	0.0499	5.75%	88.1%

Dense ladder at Quality-36: KLD 0.0095 (4B) → 0.0163 (3B) → 0.0555 (1B); at Nano-27 the tied-readout rule (Charter L7) holds every family inside the solid band — SmolLM3-arch models pick up the Q6_K token_embd floor automatically, at zero footprint cost.

MiniCPM5-1B (dense llama-arch)

Tier	Size	PPL	KLD	RMS Δp	top-p
Quality-36pc	749 MiB	27.2456	0.0555	5.16%	86.3%
Compact-33pc	687 MiB	28.1717	0.0997	6.69%	81.9%
Mini-30pc	663 MiB	28.5485	0.1153	7.16%	80.4%
Nano-27pc	563 MiB	28.8736	0.1382	7.98%	78.4%

---

🛠️ Suite Components

Script	Purpose
00_SAFETENSORS-to-AutoRound-BF16-GGUF.py	Optional int4 reconditioning + provenance sidecars.
00b_BF16-GGUF-MTP-extract.py	Split the MTP draft head off the trunk.
01_create-calibration-dataset-and-imatrix.py	Multi-domain corpus + lineage-aware imatrix.
01b_BF16-GGUF-modules-fusion.py	Reattach quantized modules (MTP/mmproj).
02_BF16-GGUF-to-ASHQ1.py	Batch tier orchestration (auto-runs mmproj).
03_perplexity_test.py	KL reference base + PPL/KLD/RMS/top-p/per-chunk sweep.
ASHQ1.py	Core optimizer (single target CLI + tier runner).
ASHQ1-mmproj.py	Vision-tower tiers, deliberately imatrix-free.
90_attribution-probe.py	Knockout attribution harness (methodology instrument).

---

⚡ Quick Start

pip install gguf numpy huggingface_hub safetensors         # + auto-round/torchvision for step 00
python 00_SAFETENSORS-to-AutoRound-BF16-GGUF.py            # create BF16 gguf files from safetensors project, create AutoRound version with `--autoround`
python 01_create-calibration-dataset-and-imatrix.py        # imatrix.gguf (plain lineage)
python 02_BF16-GGUF-to-ASHQ1.py                            # all tiers + mmproj
python 03_perplexity_test.py                               # evaluate
python 90_attribution-probe.py --list                      # optional: inspect probes

Windows/NTFS: compact /c /exe:xpress8k <file> shrinks KLD logits ~85% and BF16 GGUFs ~18% without touching results.

---

📜 Citation & Credits

• ASHQ1 (Autonomous Selective Hybrid Quantization) by wepiqx — priority-queue knapsack formulation, tied-group activation hashing, MSE scheduling.
• Empero AI (Qwen3.8-27B-Ridge) — GDN state preservation (ssm_alpha/ssm_beta @ Q8_0) and native MTP draft heads.
• Intel AutoRound — sign-gradient low-bit optimization with Hessian compensation.
• llama.cpp by Georgi Gerganov & ggml contributors — GGUF/GGML runtime and tools.
• Calibration recipes inspired by Bartowski; multi-imatrix max-combination per community practice (cHunter789 KV-cache recipe referenced by the orchestrator).

License: apache-2.0