groxaxo/Qwopus3.5-9B-Coder-Flash-Heretic-MLX

Qwopus3.5-9B-Coder-Flash-Heretic — MLX quantizations

One repository containing the original 8-bit, oMLX selective6-bit/5-bit, and additional OptiQ target6/5 BPW MLX versions of groxaxo/Qwopus3.5-9B-Coder-Flash-Heretic, pinned to source revision 85fa2083f4c1dff753119c01b0894dba0bb345d8.

Downloads and measured KL divergence

KL direction is KL(BF16 ‖ quantized), reported in nats per scored next-token position. Lower is closer to BF16. Same-top-prediction rate compares the quantized model with BF16; it is not coding task accuracy.

Variant folder	Weight size	Overall KL	Code-only KL	Same top prediction as BF16
8bit	10.717 GB	0.00143659	0.00129513	98.03%
6bit-selective	8.501 GB	0.00608873	0.00584129	96.39%
optiq-6bit	8.993 GB	0.02307562	0.02320652	94.10%
5bit-selective	7.343 GB	0.02383329	0.02501866	92.74%
optiq-5bit	8.192 GB	0.03713321	0.03366119	92.17%

The comparison uses 64 fixed examples and 7,672 next-token positions: 32 OpenAI HumanEval prompt-plus-canonical-solution examples, 16 English Stanford Alpaca instruction/answer examples, and 16 authored Spanish explanations. Context length is limited to 128 input tokens. It includes all 248,320 vocabulary tokens, temperature 1, teacher forcing, and no top-k/top-p filtering. Float64 arithmetic is used for normalization and KL reduction.

See the five-variant evaluation report, machine-readable results, and per-token numeric metrics. The report includes corpus breakdowns, bootstrap confidence intervals, and paired comparisons.

OptiQ variants

optiq-5bit and optiq-6bit use mlx-optiq 0.8.2, a separate converter from oMLX oQ. One native exact KL sensitivity sweep (8x512 calibration tokens, 4/8-bit candidates, block floor 2) is re-targeted to both size budgets. The calibration reference is a uniform4-bit model built from this exact source for the 24 GB Mac; all final comparison KL numbers use the true BF16 reference. Targets are allocator averages, not uniform matrix widths or exact whole-model BPW.

These variants use native OptiQ vision and MTP sidecars. Vision remains BF16; the native MTP sidecar has4-bit projections and BF16 protected tensors. Full image input requires OptiQ or a sidecar-compatible runtime. Strict mlx-vlm loading with oMLX's declared-sidecar loader passed the same three inference checks on both. For text-only use:

hf download groxaxo/Qwopus3.5-9B-Coder-Flash-Heretic-MLX --include "optiq-6bit/*" --local-dir ./qwopus-mlx

from mlx_lm import load, generate
model, tokenizer = load("./qwopus-mlx/optiq-6bit")
print(generate(model, tokenizer, prompt="Write a Python add(a, b) function.", max_tokens=128))

Download the complete variant, including its optiq/ folder, for vision/MTP support. See the comparison and OptiQ reproduction notes.

Quantization

• Affine MLX quantization, group size 64, with BF16 floating-point residuals.
• 8bit: 257 eligible matrices at 8-bit.
• 6bit-selective: 222 matrices at 6-bit and 35 at 8-bit.
• 5bit-selective: 233 matrices at 5-bit, 15 at 6-bit, and 9 at 8-bit.
• The vision tower is retained in BF16. Sensitive unquantized state is preserved, and the source's restored MTP weights are retained.
• Selective allocation uses measured decoder-layer sensitivity from 32 code/multilingual samples of 128 tokens on the 8-bit model, followed by the oMLX byte-budgeted allocation recipe.

Each variant is self-contained, with its own tokenizer, processors, configuration, provenance, checksums, and evaluation receipts. The folder names describe the base matrix precision; retained BF16 tensors and higher-precision selections mean these are not uniform whole-model bits-per-weight figures.

Download one variant

hf download groxaxo/Qwopus3.5-9B-Coder-Flash-Heretic-MLX \
  --include "6bit-selective/*" --local-dir ./qwopus-mlx

The model path is then ./qwopus-mlx/6bit-selective. Choose 8bit/* or 5bit-selective/* to download another variant. The repository root groups variants and is not itself a loadable model directory.

Use with MLX VLM

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("./qwopus-mlx/6bit-selective")
prompt = apply_chat_template(
    processor, model.config,
    prompt="Write a Python function that returns the square of its argument.",
    num_images=0, enable_thinking=False,
)
result = generate(model, processor, prompt=prompt, max_tokens=256)
print(result.text)

The original three variants passed strict loading with stock mlx-vlm, an arithmetic check, an AST-checked Python function response, and a simple image-color check. Conversion/evaluation used oMLX 0.7.0.dev4, MLX 0.32.2, mlx-lm 0.32.0, and mlx-vlm 0.7.1. Reproduction instructions include the pinned evaluation data sources and scripts.

Evaluation limits

KL measures short-context text distribution drift. This evaluation does not measure HumanEval pass@1, generated-program correctness, long-context behavior, vision-conditioned KL, or MTP draft acceptance/performance. MTP weights are retained, while MTP use requires a compatible runtime. Source-model claims and training details are attributed to the original model card.

License and provenance

Model weights inherit Apache-2.0 from the source model. See LICENSE and the original source card included in each variant. Evaluation scripts, authored Spanish fixtures, and numeric reports are supplied under Apache-2.0. Third-party HumanEval/Alpaca corpus text and token IDs are not redistributed; the reproducer downloads the original pinned sources, whose licenses and terms apply. See evaluation data attribution.

The SHA-256 manifest covers the published payload. No BF16 weight files or raw full-vocabulary logit cache are included.