justinbalexander/LFM2-8B-A1B-STQ2.0-patchset

LFM2-8B-A1B STQ2.0 patchset

Simulated quantization builds of LiquidAI/LFM2-8B-A1B, used to measure what the STQ encoder formats do to model quality before any quantization kernel or storage format is committed.

Files

file	format	weight coding	expert tensors patched	sha256-16
SIM-LV7-f16.gguf	levels7	7 uniform levels, 3 bits	66	0f32f3e36b48e5bd
SIM-STQ2_0-f16.gguf	stq2_0	4-lane groups, one forced zero, 2.0 bpw	66	f0e1dbfb21f4796d
manifests/levels7.recipe.json	levels7	the build inputs for this format	—	—
manifests/stq2_0.recipe.json	stq2_0	the build inputs for this format	—	—

Both model files are 16,685,562,560 bytes.

What a SIM file is

A SIM file is an F16 container whose expert tensors hold the format's reconstructed values. The builder reads each expert tensor, applies the format's own encoder, applies its decoder, and writes the result back as F16.

• Patched: the 66 expert tensors, blk.2 to blk.23, namely ffn_gate_exps.weight, ffn_down_exps.weight and ffn_up_exps.weight — 22 layers x 3 tensors, 32 experts per layer.
• Untouched: the other 190 tensors. Attention, embeddings, norms and the output head stay F16 or F32.

The source model holds 256 tensors. The file size does not change: F16 in, F16 out.

Why we build them this way

1. The measurement belongs to the format. The file runs on the standard F16 code path. No new kernel takes part, so a change in perplexity or in a prompt battery comes from the format's arithmetic alone.
2. The arithmetic is exact. The builder applies the same encode then decode pair that a real deployment applies. Nothing is approximated or modelled by a stand-in.
3. The tensor scope matches the deployment. A deployment of these formats quantizes only the expert bank and leaves attention at F16. A SIM file has the same scope, so the measurement isolates the part of the model that the format exists to compress.
4. The cost is low. A build takes about 8 to 30 minutes and needs no quantization kernel. A format can therefore be judged before storage work is committed to it.

What a SIM file is not

• Not a size result. The file is 16.7 GB because storage stays F16.
• Not a whole-model quantization. Attention is not quantized.
• Not a deployable quantized model.

Framing rule. Report a result from these files as "expert bank at N bpw + F16 attention". Never as a whole-model result. A comparison against Q4_K_M needs the same note, because Q4_K_M quantizes attention as well.

The two formats

levels7 (LV7)

• 7 uniform levels: -d, -2d/3, -d/3, 0, +d/3, +2d/3, +d
• 3 bits per weight
• one scale per 256 weights, fitted by least squares and weighted by the importance matrix
• encoder: quantize_row_levels7_im

stq2_0 (STQ2.0)

• one group of 4 lanes is coded in one byte
• exactly one lane in the group is forced to zero (the 3:4 layout)
• the other three lanes take {+/-d/2, +/-d}
• 4 x 4^3 = 256 patterns per group, so 2.0 bpw
• one scale per 256 weights; the zero lane is chosen by incremental cost
• encoder: quantize_row_stq2_0_im

Recipe

Each format records its inputs:

{
 "mode": "levels7",
 "source_gguf": "models/LFM2-8B-A1B-F16.gguf",
 "source_sha1mib": "f7a0eca7197c29b8",
 "imatrix": "models/lfm2_q4_400.imatrix",
 "imatrix_sha1mib": "00a64d607c40ef36",
 "encoder_lib": "tests_stq/libenc.so",
 "encoders_used": {"all": "quantize_row_levels7_im"},
 "tensors_patched": 66,
 "created_utc": "..."
}

• mode selects the encoder in the builder.
• source_sha1mib is the SHA-256 of the first MiB of the source GGUF.
• imatrix supplies the per-weight importance that the scale fit and the zero placement use.
• tensors_patched counts the expert tensors rewritten. It is 66 in every build here.

Rebuild

Toolchain: github.com/justinbalexander/llama.cpp, branch stq1_0-cpu-moe.

1. Build the encoder library tests_stq/libenc.so from that branch.
2. Run the builder: python3 tests_stq/patch_c.py <source.gguf> <out.gguf> <imatrix> <mode>
3. Compare the SHA-256 of the output with the value in the table.

The SHA-256 is the identity of a published file. A rebuild reproduces it only when the encoder commit, the source model and the importance matrix are the same.

Artifacts

Large artifacts live in this repository, not on a laptop and not on a rented pod. Pods are deleted and local disks are limited. The model files here were uploaded from a GPU pod by pod_upload.py, which uses HfApi.upload_large_folder.