tpls/gemma-4-12B-coder-fable5-composer2.5-v1-steer

Gemma-4 12B Coder — activation-steered (GGUF + control vector, deprecated)

⚠️ Deprecated — do not use for new work. control-vector steering proved unreliable — the strength that removes refusals also destabilises tool-calling and coherence, and the repo mixes a base GGUF with the vector. Use the abliterated line (uncensored, stable) or SFT v5 (the tool-calling winner).

Replaced by tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-GGUF.

Uncensored gemma-4 12B coder for local, agentic tool use — GGUF quantizations for llama.cpp / Ollama.

Run it: llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-steer:Q5_K_M --jinja (full commands below).

At a glance

Type	GGUF quantizations · llama.cpp / Ollama
Techniques	activation-steering → imatrix-quant
Tool-calling	native token preserved (no shim) — but see deprecation
Status	⚠️ Deprecated → tpls/gemma-4-12B-coder-fable5-composer2.5-v1-sft-v5-GGUF
Use	llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-steer:Q5_K_M --jinja

Use it

# llama.cpp (server) — tool-calling needs the recovery shim, see below
llama-server -hf tpls/gemma-4-12B-coder-fable5-composer2.5-v1-steer:Q5_K_M --jinja --ctx-size 16384

# Ollama
ollama run hf.co/tpls/gemma-4-12B-coder-fable5-composer2.5-v1-steer:Q5_K_M

Files

Sizes and a one-click loader are in the file browser / Quantizations widget above; the note says which quant to reach for.

Quant	Notes
Q5_K_M	higher quality, ~9.5 GB
coder

Intended use & limitations

Built for code generation and agentic tool use; serve locally via llama.cpp / Ollama, or use as a base to fine-tune / merge / quantize. Outputs can be wrong or fabricated — validate tool arguments before executing, and keep a human in the loop for anything consequential.

⚠️ Uncensored. For this variant a control vector suppresses refusals at inference — safety guardrails are substantially removed and it will attempt requests a stock model would refuse. You are responsible for what you generate and how it's used; not suitable where refusal behaviour is itself a safety requirement.

Where this sits in the family

• base (upstream) — yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1
  • SFT v5 (weights)
    • SFT v5 + abliterated (weights)
      • SFT v5 + abliterated (GGUF)
    • SFT v5 (GGUF)

---

Provenance & reproduction

How this model was built — technique chain, training mix, and the exact knobs/pins, so the result is reproducible without any of our tooling.

Mechanics applied

Step	Technique	What it does	Provenance
1	activation-steering	a control vector uncensors at inference WITHOUT editing the weights	yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1
2	imatrix-quant	llama.cpp quantization with an importance matrix (imatrix)	—

1. activation-steering

no weight edit → the canonical tool token survives, so bare --jinja works (no shim).

2. imatrix-quant

• calibration: code + tool-call markup
• embed/output: kept at f16 (protects tool-call logits)
• eog_patch: tokens 105/106 → EOG (bounds the <|turn> runaway)

Quantization environment

The GGUF bytes depend on the quantizer build, not just the weights — a different llama.cpp release rounds tensors differently and can change the convert mapping. Pins the toolchain these quants were produced with:

Step	Tool / setting
quantizer	llama.cpp tools image ghcr.io/ggml-org/llama.cpp:full
convert	convert_hf_to_gguf.py → f16 GGUF
imatrix	llama-imatrix over the calibration set (CPU forward pass)
quantize	llama-quantize --imatrix, token-embeddings + output tensor kept at f16

The image is the rolling :full tag, not a digest — for byte-exact reproduction pin the image digest you build with. The imatrix-quant step above lists the calibration set and the EOG patch this build applied.

---

Part of the Gemma-4 12B Coder — archive (superseded) collection.

Something not right, or a request? Open a discussion — happy to help.