ghazni101/Apodex-1.1-mini-MQ4RP

Apodex-1.1-mini — MQ4R & MQ4RP quants for hipfire

Pre-quantized Apodex-1.1-mini (Qwen3.5-MoE hybrid-attention architecture, 256-expert MoE) in hipfire's .mq4r container, in two recipes — file names reflect the recipe:

File	Recipe	Tensor mix	Size
apodex-1.1-mini-mq4r.mq4r	MQ4R — uniform MQ4 everywhere	20901 MQ4 / 1 Q8 / 191 F16	18.70 GB
apodex-1.1-mini-mq4rp.mq4r	MQ4RP — uniform MQ4 + protected fixed tier	20671 MQ4 / 231 Q8 / 191 F16	18.78 GB

Both share: embed_tokens at Q8F16; norms, A_log, dt_bias at F16; routed experts (gate_up_proj, down_proj) at uniform MQ4.

• Upstream model: apodex/Apodex-1.1-mini (Apache-2.0)
• Engine: hipfire — a Rust-native LLM inference engine for AMD RDNA GPUs

Architecture (matches upstream config.json): 40 layers (10 full-attention + 30 DeltaNet linear-attention), 256 experts top-8, hidden_size 2048, head_dim 256, 262144 context, vocab 248320. Loaded by hipfire's arch_id=6 (qwen3_5_moe) path; the .mq4r wire extension triggers the automatic Redline PM4 route on single-GPU gfx1100/1151/1201 loads.

Note: the upstream checkpoint carries a vision tower and an MTP draft head; both are intentionally absent from these quants — hipfire's qwen3_5_moe text path does not consume them, which keeps the files at ~18.7 GB.

The two recipes

MQ4R is the plain recipe: uniform 4.25-bit MQ4 across every weight class — attention, router, shared experts, conv1d, lm_head, routed experts.

MQ4RP ("+Protected") additionally pins the precision-critical fixed tiers at Q8F16 so every token passes through full-precision gating:

Tier	Tensors	Precision
Routed experts (gate_up_proj, down_proj) + attention + lm_head	20671	MQ4 uniform
Router (mlp.gate.weight, mlp.shared_expert_gate.weight) + shared expert (mlp.shared_expert.{gate,up,down}_proj) + linear_attn.conv1d (every-token dense path) + embed_tokens	231	Q8F16 (protected)

The suffix parallels upstream hipfire's mq4p grammar.

Verification

Both files were validated on RX 7900 XTX (gfx1100, RDNA3) with hipfire build 3307ccf6 (master), served via a dedicated container with the Redline PM4 route and q8 KV. Smoke tests: /v1/models listing, thinking-disabled completions (clean stop), thinking-enabled generations with reasoning traces, and a 400-token generation measured at 226 tok/s end-to-end for MQ4R. The Ornith reference builds of the same recipes measured ~235 tok/s (MQ4R) vs ~212 tok/s (MQ4RP) under a 24-request benchmark protocol; expect a similar ~10% RP decode penalty here.

Known issue (uniform-MQ4 routed experts on this model family): when reasoning is enabled, if the token budget ends inside an open think span the daemon fails closed with open think span at end of generation. Disable thinking for throughput runs or give the budget enough headroom to close the span.

Usage

hipfire serve ~/.hipfire/models/apodex-1.1-mini-mq4rp.mq4r --kv-mode q8
hipfire serve ~/.hipfire/models/apodex-1.1-mini-mq4r.mq4r --kv-mode q8

Requires an AMD RDNA GPU (gfx1100/1151/1201 class) with ROCm; the .mq4r extension selects the PM4 route automatically on single-GPU loads. With q8 KV the full context fits a 24 GB card (~22 GB VRAM class).

Quantization format

All hipfire MagnumQuant formats are FWHT-rotated (incoherence processing) with an 8-byte affine or fp16-codebook group header at group size 256. MQ4 = uniform 4.25-bits-per-weight scalar quantization. Reproduced with hipfire-quantize:

• MQ4R: --format mq4g256 --no-kmap --no-q8-router --no-q8-conv1d --fixed-tier embed:q8
• MQ4RP: --format mq4g256 --no-kmap with HIPFIRE_Q8_PARTIAL=router,shared_expert

Attribution

• Base model: apodex/Apodex-1.1-mini — Apache-2.0 license. All credit for the base model belongs to its authors; this repo redistributes only a lossy quantization of it.
• Quantization format & engine: hipfire — Rust-native, no-Python-hot-path inference engine for consumer + datacenter AMD GPUs.
• For reference SKUs of other models (including graded mixed-precision builds and E8-lattice .mfp4), see ghazni101/Ornith-1.5-35B-A3B-MQ4RP.

Quantized locally on an AMD Ryzen 9 host from the upstream safetensors; validated on RDNA3 (gfx1100).