Qwen3.8-27B-heretic-MLX-8bit

8-bit quantized MLX conversion of trohrbaugh/Qwen3.8-27B-heretic-ara, for Apple Silicon.

What this is

The base model is an abliterated ("heretic") derivative of Qwen/Qwen3.8-27B: its refusal behaviour has been surgically removed, so it will answer prompts a safety-tuned model would decline. This repo only changes the format and precision — it adds no alignment, and removes none either. Evaluate it before putting it in front of users, and apply your own filtering where your use case needs it.

Lineage: Qwen/Qwen3.8-27B → trohrbaugh/Qwen3.8-27B-heretic-ara → this MLX build.

Text-only

The base model is multimodal (image-text-to-text). This conversion is text-only. mlx-lm converts the language tower only: the config carries no vision_config, and no tensor belongs to the vision tower. Image and video input will not work.

Quantization

Affine quantization, group size 64, 8.501 effective bits per weight.

The full set

All four builds were converted from the same source revision and measured the same way.

Build	Size	Bits/weight	Generation	Peak memory
4-bit	15.1 GB	4.501	37.9 tok/s	15.5 GB
6-bit	21.9 GB	6.501	27.9 tok/s	22.2 GB
8-bit	28.6 GB	8.501	22.2 tok/s	28.9 GB
bf16	50.0 GB	16	12.7 tok/s	54.1 GB

Measured on a Mac Studio M3 Ultra (512 GB unified memory), macOS 26.5.2, mlx-lm 0.31.3, 68-token prompt, 120 generated tokens. Single run, one machine, one prompt — an order-of-magnitude guide, not a benchmark.

This repo is the 8-bit build: 28.6 GB, 6 shards, 22.2 tok/s, 28.9 GB peak memory.

Usage

pip install mlx-lm

mlx_lm.generate --model donedynamics/Qwen3.8-27B-heretic-MLX-8bit \\
  --prompt "Introduce yourself briefly." --max-tokens 256

from mlx_lm import load, generate

model, tokenizer = load("donedynamics/Qwen3.8-27B-heretic-MLX-8bit")
messages = [{"role": "user", "content": "Introduce yourself briefly."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Reasoning mode

The chat template supports enable_thinking and reasoning_effort. Thinking is on by default and consumes tokens before the answer begins, so a small max_tokens budget may return reasoning only. Pass enable_thinking=False to apply_chat_template for direct answers.

Conversion

mlx_lm.convert --hf-path trohrbaugh/Qwen3.8-27B-heretic-ara -q --q-bits 8 \\
  --mlx-path Qwen3.8-27B-heretic-MLX-8bit

Converted with mlx-lm 0.31.3 from the bf16 weights at revision a67ae100d933c0d17af3232bda35825979fc63ce. Before conversion the source was verified: 7 shards, 1199 tensors, all safetensors headers parsed, no missing files.

License and attribution

Apache-2.0, inherited through the lineage above. Credit for the model goes to the Qwen team at Alibaba Group's Tongyi Lab, and for the abliteration to trohrbaugh. This repo contributes only the MLX conversion.