Mini Qwen 3.8
Collection
All shrunken down versions of Qwen 3.8 we "made". • 2 items • Updated
mac-qwen3.8-27b-abliterated (MLX)
⚠️ Untested — use at your own risk.
Basic Intruction for use:
When running the model add the systemprompt "You are a helpful assistant." to assure that the model actually executes its tasks as it should.
Why does this exist?
This is the Apple Silicon optimized version of our gleraTech/mini-qwen3.8-abliterated-27B GGUF release.
While the GGUF works on any platform, MLX is Apple's own machine learning framework built specifically for unified memory architectures. On Apple Silicon, MLX can access the full memory bandwidth directly without copying data between CPU and GPU — meaning faster inference, lower latency, and better memory efficiency compared to running a GGUF through llama.cpp on the same hardware.
If you're on a Mac with Apple Silicon, this is the version you want.
This model runs on 16GB systems, but don't expect a smooth experience. If you really want Qwen3.8-27B on limited hardware and are okay with your machine sweating, this is for you. Manage your expectations accordingly.
Model Details
- Base model: Qwen3.8-27B
- Abliteration: OBLITERATUS V3 (iterative SVD + LEACE, -2.1pp MMLU)
- Quantization: Q3_K_S → MLX
- Size: ~11.7 GB
- Recommended RAM: 16GB or more
Usage
pip install mlx-lm
mlx_lm.generate \
--model gleraTech/mac-qwen3.8-27b-abliterated \
--prompt "Your prompt here"
Credits
- Qwen Team for Qwen3.8-27B
- OBLITERATUS for the abliteration
- gleraTech for quantization & MLX conversion
License
Apache 2.0
Hardware compatibility
Log In to add your hardware
Quantized
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support