gemma-4-12B-it-abliterated-mlx-4bit

MLX 4-bit conversion of OpenYourMind/gemma-4-12B-it-abliterated-uncensored for native Apple Silicon inference.

This is an uncensored (abliterated) variant of Google's Gemma 4 12B instruction-tuned model, converted to MLX 4-bit quantization for fast, private, on-device inference on M-series Macs.

Model details

Property	Value
Base model	google/gemma-4-12B-it
Abliteration source	OpenYourMind/gemma-4-12B-it-abliterated-uncensored
Format	MLX 4-bit (q-bits=4, group-size=64)
Size	~6.77 GB
Parameters	~11.95B
Vision	✅ Preserved (encoder-free architecture)
Context window	256K tokens
Languages	140+

Requirements

• Apple Silicon Mac (M1 or later)
• Unified memory: 16GB minimum, 24GB recommended
• mlx-vlm installed

pip install mlx-vlm

Usage

Text only

from mlx_vlm import load, generate

model, processor = load("IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit")
output = generate(model, processor, prompt="Explain quantum computing simply.", max_tokens=512)
print(output)

Vision (image input)

from mlx_vlm import load, generate

model, processor = load("IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit")
output = generate(
    model,
    processor,
    prompt="Describe what you see in this image.",
    image="path/to/image.jpg",
    max_tokens=256
)
print(output)

CLI

mlx_vlm.generate \
  --model IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit \
  --prompt "Describe this image." \
  --image path/to/image.jpg \
  --max-tokens 256

LM Studio

Search IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit in the LM Studio model browser (MLX filter enabled).

Ollama

ollama run hf.co/IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit

Conversion details

Converted on Apple Silicon M4 Pro (24GB) using:

python3 -m mlx_vlm convert \
  --hf-path OpenYourMind/gemma-4-12B-it-abliterated-uncensored \
  --mlx-path ./gemma4-12b-abliterated-mlx-4bit \
  -q --q-bits 4 --q-group-size 64

Note: mlx-vlm is required (not mlx-lm) to preserve the vision tower. Using mlx-lm would silently drop the vision and audio embedders.

Performance (M4 Pro 24GB)

Context	TG (tok/s)
1k	~24
4k	~24

Disclaimer

This model has its refusal mechanisms removed via abliteration. It may generate sensitive or controversial content. Use responsibly and in accordance with applicable laws and the Gemma license.