VisualInference/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit

Qwen3.8-27B-AEON-Ultimate-Uncensored — Multimodal MLX 6-bit

A 6-bit MLX quantization of AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 (revision 8f76e82), AEON's abliterated release of Qwen/Qwen3.8-27B, for local inference on Apple Silicon.

The vision tower is fully preserved (333/333 tensors — AEON's release keeps it hash-identical to stock Qwen3.8), so image and video understanding work through mlx-vlm. The model's native MTP head is published separately as a drafter for lossless speculative decoding: intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-MLX-MTP-Drafter — pairing it typically speeds decode 1.4–1.9× at identical output quality.

Requirements

• Apple Silicon Mac with MLX support; pip install mlx-vlm (converted with mlx-vlm 0.6.13 / mlx 0.32.0).
• Unified memory: ~21 GB for weights, with an observed runtime peak of ~25 GiB at short context and ~30 GiB at 13k-token context. Comfortable on 48 GB+ machines, workable on 36 GB; not recommended below that.
• Because decode is memory-bandwidth-bound, tokens/sec scales roughly linearly with the chip's memory bandwidth (see measured numbers below).

Usage

# text / vision
python -m mlx_vlm generate \
  --model intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit \
  --prompt "Describe this image." --image photo.jpg

# faster decode with the MTP drafter (lossless speculative decoding)
python -m mlx_vlm generate \
  --model intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit \
  --draft-model intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-MLX-MTP-Drafter \
  --draft-kind mtp --draft-block-size 3 \
  --prompt "Write a short story about a lighthouse keeper."

Tips:

• Draft block size 3 is the all-round sweet spot; 4 edges ahead on code; ≥5 regresses.
• When serving (mlx_vlm.server), --prefill-step-size 512 cuts peak prefill memory by ~6 GB at no measured speed cost.
• Recommended sampling (from the Qwen3.8 card): thinking temp=1.0, top_p=0.95, top_k=20; non-thinking temp=0.7, top_p=0.8, presence_penalty=1.5.

Conversion recipe

python -m mlx_vlm convert \
  --hf-path AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 \
  -q --q-bits 6 --q-group-size 64 \
  --mlx-path Qwen3.8-27B-AEON-Ultimate-Uncensored-mlx-6bit

Affine mode, group size 64, no calibration (RTN is near-lossless at 6-bit). Language model and vision tower are both quantized at the global setting; mtp.* tensors are excluded by design — mlx-vlm loads the MTP drafter as a separate model (see the drafter repo for the split recipe).

Measured performance

Test machine: Mac mini M4 Pro, 48 GB unified memory (~273 GB/s). Higher-bandwidth chips (M-series Max/Ultra) scale decode proportionally — serial decode on this quantization measures at ~100% of the device's streaming-bandwidth roofline.

Workload	Config	Decode
Coding (temp 0.2)	MTP block 4	21.5 tok/s
Document QA @ 13k ctx (temp 0.7)	MTP block 3	16.9 tok/s
Creative prose (temp 0.7)	MTP block 3	15.9 tok/s
Serial (no drafter)	—	11.4 tok/s

Prefill measured ~105–110 tok/s, flat with context length up to the 13k tested. Draft acceptance is workload-dependent: ~46% on open-ended prose, substantially higher on code and grounded QA.

Validation

• Text, vision, and long-context (13k) generation smoke-tested after conversion.
• Two-phase coding gate (spec research → implementation, executed against 28 hidden edge-case asserts across an SSE-parser task and a stack-VM task): 28/28.
• Speculative decoding verified lossless (drafter rejections fall back to the target model's own tokens; the sampling distribution is unchanged by construction).

Provenance & responsibility

Qwen/Qwen3.8-27B → AEON-7 SSM-conv1d repair + abliterix abliteration (BF16, vision and MTP untouched — see their card for methodology and KL evidence) → this repo (6-bit MLX quantization, nothing else changed).

This is an abliterated, refusal-removed model. As the upstream card puts it: the model does not decide whether to comply — you do. Outputs are the responsibility of the operator; use within the law of your jurisdiction. Apache-2.0, inherited from base.