Qwen3.8-27B-RANA-abliterated-MLX

MLX builds of abliterant/Qwen3.8-27B-RANA-abliterated, a refusal-ablated Qwen/Qwen3.8-27B, for research with Apple Silicon and mlx-vlm.

Quick start · Evaluation · Available files · Abliterant models

This is a safety-alignment-removed research model. Read Intended use and Limitations before using it. Method, full evaluation and release gates are on the BF16 card.

At a glance

Field	Value
Base checkpoint	Qwen/Qwen3.8-27B
Conversion source	Abliterant BF16 release
Release type	Refusal-ablated vision-language model, converted to MLX
Weights	Root 4-bit; 4-bit/, 5-bit/, 6-bit/, 8-bit/, bf16/; BF16 MTP drafter in mtp/
Vision	Included in every model build; vision tower remains BF16
Recorded runtime	mlx-vlm 0.7.3, MLX 0.32.2
Test platform	MLX CUDA on Linux, not Apple Silicon
Context validation	No MLX context-length validation reported; quick-start checks used --max-tokens 4096
License	Apache-2.0, inherited from Qwen

Quick start

Download one precision folder, then load it by local path:

pip install -U mlx-vlm    # mlx-vlm >= 0.7.3, mlx >= 0.32.2
hf download abliterant/Qwen3.8-27B-RANA-abliterated-MLX --include "4-bit/*" \
  --local-dir ./Qwen3.8-27B-RANA-abliterated-MLX

python -m mlx_vlm generate --model ./Qwen3.8-27B-RANA-abliterated-MLX/4-bit \
  --enable-thinking --max-tokens 32768 --temperature 1.0 --top-p 0.95 --top-k 20 \
  --prompt "Explain how a hash map handles collisions."

Another precision, plus the MTP drafter (repeat --include for each folder):

hf download abliterant/Qwen3.8-27B-RANA-abliterated-MLX --include "8-bit/*" --include "mtp/*" \
  --local-dir ./Qwen3.8-27B-RANA-abliterated-MLX

python -m mlx_vlm generate --model ./Qwen3.8-27B-RANA-abliterated-MLX/8-bit \
  --draft-model ./Qwen3.8-27B-RANA-abliterated-MLX/mtp --draft-kind mtp \
  --enable-thinking --max-tokens 32768 --temperature 1.0 --top-p 0.95 --top-k 20 \
  --prompt "Explain how a hash map handles collisions."

• Don't pass the repository ID itself to --model. mlx-vlm then downloads every folder (about 160 GB), not just the 4-bit files at the root. The root repeats the 4-bit/ files for apps that open a repository's top level.
• Add --image path/to/image.png for vision input. The drafter is optional; it only speeds up generation.
• An OpenAI-compatible server: python -m mlx_vlm server --model ./Qwen3.8-27B-RANA-abliterated-MLX/8-bit (add --draft-model …/mtp --draft-kind mtp for speculative decoding). It supports tool calls and returns the reasoning separately.
• Thinking is on by default. Qwen's thinking-mode sampling: temperature 1.0, top_p 0.95, top_k 20.
• Long technical requests can need 20–50k tokens of reasoning; allow a large --max-tokens.

Release checks used mlx-vlm 0.7.3 and MLX 0.32.2 on Linux with MLX's CUDA backend. Apple Silicon performance was not measured.

What changed

• Source: the published BF16 weights, converted with mlx_vlm.convert (mlx-vlm 0.7.3, MLX 0.32.2). Quantized builds use affine round-to-nearest quantization with group size 64; the vision tower stays in BF16. bf16/ is the same conversion without quantization. The MTP head is split into mtp/ with --mtp and kept in BF16. This is a conversion and quantization of the refusal-ablated weights, not additional training.
• Layout: in every folder, tensor names, shapes and dtypes and config.json are identical to the matching mlx-community or lmstudio-community build of Qwen/Qwen3.8-27B (4-bit/ and 8-bit/: mlx-community -4bit / -8bit; 5-bit/ and 6-bit/: lmstudio-community -MLX-5bit / -MLX-6bit; bf16/: mlx-community -bf16; mtp/: mlx-community -MTP-bf16). It loads wherever those load.

The RANA abliteration method, evaluation and release gates are documented on the BF16 card. The MLX conversion record does not specify an exact source BF16 revision.

Evaluation

All recorded checks ran with MLX 0.32.2's CUDA backend on Linux, using files sha256-identical to the evaluated release artifacts. Raw outputs are in results/.

KLD is the mean KL divergence of each build's next-token distribution from the bf16/ build on wiki.test.raw, using the same protocol as the GGUF card: 100 chunks of 512 tokens, with the second half of each chunk scored. Lower KLD is closer to BF16. Each build has 25,500 scored tokens. This measures quantization drift, not refusal behavior.

Build	KLD vs. BF16 (lower is closer)	Same top token	Perplexity (lower is better)
Root / 4-bit/	0.0468	90.0%	6.945
5-bit/	0.0129	94.8%	6.863
6-bit/	0.0042	96.9%	6.788
8-bit/	0.0013	98.3%	6.779
bf16/	reference	–	6.778

• Files: every release file was hashed after upload (sha256 for weights) and matched the conversion output.
• Function: with mlx_vlm server, reading text and a shape from an image and a three-turn tool-calling exchange passed on 4-bit/, 8-bit/ and bf16/.
• MTP speculative decoding: with mtp/, greedy decoding and 512 tokens of a coding answer, mean accepted tokens per round were: 4-bit 2.49 (74% of drafted), 5-bit 2.57 (79% of drafted), 6-bit 2.56 (78% of drafted), 8-bit 2.57 (79% of drafted), bf16 2.57 (79% of drafted).
• Quick-start checks: the release commands were run with --max-tokens 4096, including the repeated---include download of 8-bit/ and mtp/.

Available files

The root is the 4-bit build; other precisions are in subfolders, and mtp/ holds the drafter for MTP speculative decoding. Vision is included in every model build.

Folder	Precision	Size	Runtime / role
(root)	same files as 4-bit/	16.08 GB	MLX, top-level loading
4-bit/	4-bit, 4.695 bits/weight	16.08 GB	MLX / mlx-vlm
5-bit/	5-bit, 5.678 bits/weight	19.44 GB	MLX / mlx-vlm
6-bit/	6-bit, 6.661 bits/weight	22.80 GB	MLX / mlx-vlm
8-bit/	8-bit, 8.627 bits/weight	29.53 GB	MLX / mlx-vlm
bf16/	BF16, unquantized	54.74 GB	MLX / mlx-vlm; evaluation reference
mtp/	MTP drafter, BF16	0.87 GB	Optional speculative decoding

Other formats: BF16 (Transformers), FP8 (vLLM / SGLang), GGUF (llama.cpp).

Limitations and intended use

Limitations

• Not tested on Apple Silicon in the recorded release checks. The checks above ran MLX's CUDA backend on Linux (same MLX version, same model code, same files); speed on a Mac is not measured here.
• Refusal behavior was not re-measured on the MLX builds. KLD shows how closely each build tracks the BF16 weights; refusal results for BF16, FP8 and a 4-bit GGUF are on their cards. Lower-bit builds drift further from BF16.
• Everything listed under Limitations on the BF16 card applies here too.
• Reduced refusal does not establish greater safety, accuracy, capability retention or universally compliant behavior.

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behavior.
• Not for public or end-user deployment without a separate moderation layer. The model's own refusals have been largely removed, so any safety filtering has to happen outside it.
• You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen, and the terms of any platform where outputs are used.

Provenance and license

These MLX artifacts derive from the Abliterant BF16 release of Qwen/Qwen3.8-27B. The release evidence is recorded at revision 8a647c2e5e329dfdd399c9541b8058fa5447c228.

The weights retain the Apache-2.0 license inherited from Qwen. Conversion, quantization and refusal ablation do not remove the upstream license obligations.

Credits:

• Qwen team: base model Qwen/Qwen3.8-27B.
• Arditi et al., 2024: "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim): prior work on norm-preserving abliteration.
• Apple MLX team and mlx-vlm (Prince Canuma): MLX and the conversion and runtime tooling.