Voice

Gemma 4 · 26B MoE · StyleTune · GGUF Q8_0

Your model, in your voice. One file to run—nothing extra to set up.

A full model is 26GB. A voice is 1.4GB — smaller models even less. You download the voice, not the model.

This is a tool — no .gguf is included. You create the voiced model locally with voice cast.

---

Two steps to start

One-time setup, then two commands. One model, one voice:

python3 voice.py path  # once — adds voice to PATH (then use voice directly)
voice get Gryphe/Gemma-4-26B-A4B-StyleTune-V2
# Name it styletune-v2 when prompted
voice cast ./model.gguf styletune-v2 --out ./voiced/

Run it:

llama serve -m ./voiced/model.gguf

We tried it on Orion: We cast Gryphe/Gemma-4-26B-A4B-StyleTune-V2 voice onto TheDrummer/Orion-26B-A4B-v1.4 — phrasing was 98% different (1.6% shared at 1.0, 3.2% at greedy). Same cast on its abliterated sibling llmfan46/gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic looped; re-cast as delta stopped looping and kept the voice. We basically got StyleTune on the QAT version with just voice cast delta — one file out.

No adapter at runtime. voice cast bakes the voice into a complete .gguf — the head at Q8_0, everything else untouched. Just llama serve -m.

Example voice shown. Any compatible safetensors voice works the same way. Names are lowercased on disk—styletune-v2 is the stored form. Model and voice names here are examples; use any compatible pair that shares the same architecture and vocabulary.

---

Not just Gemma — any compatible model

Voice is not tied to Gemma.

If the architecture and vocabulary match, the voice transfers — Qwen, Gemma, Llama, any family. Take the voice from a finetune and cast it onto the base you already have. The same file you run for reasoning keeps its reasoning; the voice just changes how it sounds.

For example, a voice from Wiself/Holodeck-Lounge-MTP casts onto any qwen3.5-9b base, ReadyArt/For-Her-Darkside-12B-v1.4 onto any gemma4 12b base — even mixed with ReadyArt/Serenity-12B to see what happens. Gemma 4 26B MoE above is just one illustration. Mix and match within the same family: keep the more performant base, keep the better voice.

Qwen, Llama, Gemma, GLM, DeepSeek — if it has a voice, you can move it.

---

How it works

Two benefits, no extras to learn:

• Get what you need: Light and fast — pulled with range requests in seconds, saved as one tensor (voice.safetensors + voice.json). voice list to see, voice info to inspect.
• Run fused: Cast writes a complete model you can run anywhere. No extra setup.

You will see each step as it happens and get a ready-to-run file at the end.

Take the voice of any compatible model or finetune and cast it onto the file you already have. Mix and match. Keep the stronger model for reasoning and the better voice for style. You choose—no extra steps at runtime.

When you need a delta

Direct cast is the default. If an abliterated target loops, try delta — it blends only the difference, so the MoE router stays calm.

Why a delta helps when looping

A delta carries only the difference from its base, so it blends more gently with the target. On some abliterated targets, a tiny shift can change behavior. MoE routing is discrete, so a small change in the residual stream can tip which expert fires next. If direct casting loops, the delta path often avoids it.

---

Commands

Seven verbs. One idea each:

Command	Description
voice get <org/model>	Get a voice from Hugging Face and save it to your library
voice delta [name]	Make a portable style you can reuse across compatible models
voice cast <model.gguf> <voice>	Fuse a voice into your model and write a complete, ready-to-run file
voice list	Show your saved voices—add base to list cached bases
voice info <voice>	Show details for a saved voice or delta
voice remove <voice>	Remove a voice from your library
voice path	Add voice to your PATH

Example:

voice get Gryphe/Gemma-4-26B-A4B-StyleTune-V2
# Name it styletune-v2 when prompted
voice list
#   Name               Source                                  Type
#   ────────────────── ─────────────────────────────────────── ────────
#   styletune-v2       Gryphe/Gemma-4-26B-A4B-StyleTune-V2      voice
#   delta-styletune-v2 Gryphe/Gemma-4-26B-A4B-StyleTune-V2      delta

voice info styletune-v2
voice list base
voice cast ./model.gguf styletune-v2 --out ./voiced/

delta- is reserved. Voice names cannot start with it. Deltas always do. Full CLI reference: DOCS.md (generated from voice --help, 7 commands). Names are lowercased when saved.

What StyleTune showed

Gryphe reports for StyleTune-V2—on their private 200-prompt roleplay eval (private regex, certified cliché-free, 1–20 turns, greedy 0.0)—52% fewer clichés per 100 words (1.141→0.551) and 19.9% shared trigram vocabulary vs base instruct.[^1]

[^1]: Source: Gryphe/Gemma-4-26B-A4B-StyleTune-V2 model card and discussion #1. We cite their numbers, not ours. No model-index is published here—this is a private metric, not a reproducible Hub leaderboard.

How Gryphe tested

• Single epoch, 100% narrative data; prompts certified cliché-free and not seen in training.
• 200 prompts sampled at intervals 1 to 20 turns, each ending on a user turn.
• Run at temperature 0.0 greedy deterministic.
• Clichés via their private regex (for example, calculating and predatory) counted per 100 words; phrasing via shared trigram vocabulary.

Our quick check—heuristic, n=10, not comparable

Small sample, heuristic, not comparable—direction only. With that caveat, the check suggests distinct phrasing.

We used 10 roleplay prompts in chat. Each was run at temp 1.0 and at greedy 0.0 (max_tokens 2000, seed 42). This is a small-sample heuristic using our own 50-phrase list, not Gryphe's private regex. Absolute per-100 numbers are not comparable to Gryphe's.

With that caveat, overall trigram overlap vs base was 1.6% at 1.0 and 3.2% at greedy. Every prompt was under 5%. We saw 5–5.7k words total. Thinking tokens averaged about 2.6k. There were no loops and no empties. The voice was clearly distinct. Greedy was slightly lower on clichés than 1.0. Direction and distinctness align with Gryphe's report.

Details, per-prompt tables, and harness: benchmark.md.

---

A smarter base: quantization-aware training

Google trains a second kind of checkpoint for on-device use—Quantization-Aware Training (QAT). Instead of shrinking the model after it is done, QAT simulates low-precision math during training so the weights learn to survive it.

In Google's recipe for Gemma 3 and Gemma 4, that is about 5,000 steps with a fake-quantized forward pass and KL divergence to the BF16 teacher as the target. The model stays in high precision, but it computes as if it were already quantized, learning to be friendly to compression while it can still adapt.

The gain, per Google, is meaningful at the size you actually run: 54% less perplexity drop at Q4_0 vs standard post-training quantization (Google Developers Blog, April 18, 2025).

StyleTune-V2 is built from the regular gemma-4-26B-A4B-it, not the QAT variant. Love StyleTune-V2 but want Google's QAT base? Keep the QAT model — cast any voice onto it. No QAT finetune needed. Any finetune from gemma-4-26B-A4B or gemma-4-26B-A4B-it can now lend its voice—not the whole model, just the voice.

---

---

Evaluation

Gryphe (author, n=200, greedy 0.0, private regex): 1.141→0.551 clichés/100w (−52%), 19.9% shared vocabulary—private regex, certified cliché-free prompts, 1–20 turn intervals. Source: Gryphe model card + discussion #1. Private metric for citation only—not a Hub leaderboard.

Ours (independent, heuristic, n=10, not comparable—direction only):

Condition	Base clichés/100	Voiced clichés/100	Shared trigram (Jaccard)	Notes
temp 1.0	0.18 (10 hits / 5463w)	0.14 (8 hits / 5563w)	1.62% overall (176/10877), per-prompt 0.2–1.9%	All <5%, distinct phrasing
greedy 0.0	0.12 (6 hits / 5085w)	0.10 (6 hits / 5723w)	3.23% overall (336/10406), per-prompt 1.0–4.8%	Slight canonical convergence, still far below 19.9%

Full tables, sampler-vs-voice dominance, and per-prompt first-80: benchmark.md (deep, 4 subagents). Harness via POST /v1/chat/completions with Gemma 4 chat template and <|channel>thought parsing.

For official leaderboard submission: When you have Hub dataset IDs, add YAML files under .eval_results/ per Evaluation Results docs. This card cites Gryphe's private eval—it does not publish a leaderboard entry.

---

Limitations

• Heuristic mismatch: Our 50-phrase list misses Gryphe's private regex (calculating xx, predatory xx, and others), so absolute 0.10–0.18 vs 1.141→0.551 is not comparable—only direction and low shared vocabulary are signal.
• Sample size and prompts: 10 single-turn prompts vs Gryphe's 200 multi-turn certified set (intervals 1–20, ends on user turn). Small sample, higher variance.
• Thinking overhead: Gemma 4 thinking mode (<|channel>thought) adds about 2.6k thinking tokens on average. Gryphe's eval predates thinking configs and did not report this split.
• Determinism: Greedy 0.0 reduces variance but still yields more canonical phrasing (shared 1.6%→3.2%)—expected, not a failure.
• If you see looping, use the delta path (voice delta → voice cast … delta-…). Tiny shifts can affect behavior on abliterated targets. See “Why a delta helps when looping” above.
• Scope: Style transfer changes phrasing and voice, not knowledge or factuality. Not evaluated for safety, bias, or instruction following. See Gemma 4 model card Limitations.

---

References

• Gemma 4 Technical Report—Gemma Team, 2026. arXiv:2607.02770—base model google/gemma-4-26B-A4B-it (MoE 26B A4B, 256K context, 262K vocab).
• Gemma 3 Quantization-Aware Training—Google Developers Blog, Apr 18, 2025: State-of-the-art AI to consumer GPUs—QAT recipe (5k steps, fake-quant + KL to BF16 teacher, 54% less perplexity drop at Q4_0).
• Gryphe StyleTune-V2—Model card and discussion #1—200-prompt eval, private regex, 1.141→0.551 and 19.9% shared vocabulary.

---

Use with Hugging Face

GGUF (this repo—local with llama.cpp)

This is the native path for the fused model above—no Python, no adapter at runtime:

llama serve -m ./voiced/model.gguf --jinja
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"voiced","messages":[{"role":"user","content":"Write a tavern scene, no clichés."}],"temperature":1.0}'

Transformers (HF-native snippet—for the base or other safetensors voices)

The Voice tool extracts the lm_head from safetensors models. For any HF transformers model (including the base below), the canonical snippet is:

from transformers import pipeline

# Base model—valid, lowercased HF ID (link verified, HF is case-insensitive)
# https://huggingface.co/google/gemma-4-26b-a4b-it
pipe = pipeline("text-generation", model="google/gemma-4-26b-a4b-it")
print(pipe("Write a short tavern scene with varied diction:")[0]["generated_text"])

# For a StyleTune finetune example (illustrative):
# pipe = pipeline("text-generation", model="gryphe/gemma-4-26b-a4b-styletune-v2")

Base-model note: google/gemma-4-26b-a4b-it is the lowercased form of the real HF ID google/gemma-4-26B-A4B-it (verified live, Apache-2.0 via Gemma license). gryphe/gemma-4-26b-a4b-styletune-v2 is the lowercased illustrative voice source—replace with any compatible safetensors model that shares architecture and vocab. Relation: base_model_relation: quantized (fused GGUF head quantized to Q8_0, remainder preserved).

---

Demo

Local demo—same one-file run: llama serve -m ./voiced/model.gguf.

A Hugging Face Space that loads the model directly from this repo will be linked here when published (so downloads cross-link on the model page).

---

Choosing a style

Pick the voice you like. Cast it directly. That is the default. Make a delta only when you need it:

• Full voice: The exact voice from the source model. Use it when you want that model's voice, exactly. Two steps: get and cast.
• Delta: Make once — casts onto any compatible target of that family. One style, all your bases.

Both fuse the same way into the output file. Small, sharp, and complete.

If you are not sure, start direct:

voice cast ./model.gguf styletune-v2 --out ./voiced/

If a model loops, try delta:

voice delta styletune-v2
voice cast ./model.gguf delta-styletune-v2 --out ./voiced/

---

Compatibility

Runs with compatible models that share the same architecture and vocabulary. Check shape before you cast:

Works with:

• Hugging Face safetensors models that expose an output projection — lm_head.weight, output.weight, or tied embed_tokens.weight auto-found
• GGUF models of the same architecture and vocab size—the rest of the model stays exactly as it was
• Transposed heads and BF16/F16/F32 — handled, no flag needed
• llama.cpp and anything that reads GGUF—run with llama serve -m <voiced.gguf>

Does not work with:

• Mismatched architectures or vocab sizes—shape must match
• Missing output projection
• Heads where the last dimension is not a multiple of 32—required for the Q8_0 fused output

Your voiced GGUF is standalone. No extra files. No extra steps to run.

Split—quantized vs full precision: The source safetensors voice is full precision (BF16/F16/F32), full shape. The fused GGUF keeps all tensors at their original quantization except the output head, which is written at Q8_0 (quantized). For deltas, the math is abliterated_head (dequantized to F32) + delta (F16→F32) then quantized to Q8_0. The Q8_0 block requires the last dimension % 32 == 0 (true for Gemma 4 26B: hidden 2816, vocab 262144). All other tensors and metadata (tokenizer, vocab, arch) are byte-copied.

Disclosures: get and delta fetch from Hugging Face and need network access. cast writes a new GGUF to disk, so ensure you have free space for a full model copy.

---

Your files, where they are

Your library lives at ~/.voices/:

~/.voices/
  styletune-v2/voice.safetensors      # your voice, single tensor
  styletune-v2/voice.json             # metadata for that voice
  delta-styletune-v2/                 # portable style for reuse
  delta-styletune-v2/voice.safetensors
  delta-styletune-v2/voice.json
  base/<org>/<model>.safetensors      # cached base head
  base/<org>/<model>.json             # metadata for the base

Notes:

• Names are lowercased when saved. Input StyleTune-V2 is stored as styletune-v2.
• voice list shows voices and deltas. voice list base shows cached bases. voice info <name> shows details for any entry.

---

Troubleshooting

Voice not found:

Use voice list to see what is saved. Names match case-insensitively. StyleTune-V2 and styletune-v2 resolve to the same entry.

Shape mismatch. Does not fit this model:

The voice and the target must share the same head shape. A 26B head will not fit a 12B model. Try a voice from the same architecture and vocab size.

Tied weights:

Some models tie the output projection to the embedding. Voice finds the correct tensor automatically. You do not need to specify it.

Could not reach the model:

Check the org/model name and your connection. Voice fetches the header first and will tell you if the model is not accessible.

Output folder not writable:

Pass --out to a folder you own or to a full file path:

voice cast ./model.gguf styletune-v2 --out ./voiced/
voice cast ./model.gguf styletune-v2 --out ./voiced/model.gguf

Add -v to any command for detailed progress. You can pass a direct .safetensors path to cast instead of a saved name when needed.

---

Details

What is inside the files

• Voice: single-tensor safetensors, lm_head.weight or output.weight, original dtype (BF16/F16/F32), full shape.
• Delta: single-tensor safetensors, F16, same shape, voice − base.
• Voiced GGUF: copy of your model with the output head replaced and quantized to Q8_0. All other tensors and metadata are preserved. Vocab, tokenizer, and architecture stay exactly as they were.

How it works under the hood:

• get downloads only the head bytes with range requests, not the whole file, then saves to ~/.voices/<name>/.
• delta subtracts the base head from the voice head (voice − base) and saves an F16 delta as delta-<name>. The base head is cached once at ~/.voices/base/<org>/<model>.safetensors.
• cast replaces the head and writes a new GGUF with the head at Q8_0 (delta to Q8_0 for deltas, direct fuse for full voices). The result is a complete model.

A voice lives in the last layer. Changing that layer changes how the model sounds. Voice saves that layer and casts it onto a compatible model you already have—fused, standalone, and ready to run. Transposition between [vocab, hidden] and [hidden, vocab] is handled automatically. So is BF16 to F32 for the math and F32 to Q8_0 for the result. The last dimension must be a multiple of 32 for Q8_0.

Scope: This card documents the Voice tool (adapter extraction) and its quantized GGUF output (head at Q8_0). Base weights remain under their source license; tool code is Apache-2.0—see License.

---

Collection

This voice builds on Gemma 4. Find all Gemma 4 models in the official collection:

• Google Gemma 4 Collection: https://huggingface.co/collections/google/gemma-4—includes google/gemma-4-26B-A4B-it (this card's base_model) and google/gemma-4-26B-A4B.

Related StyleTune voices (examples, any compatible safetensors works):

• StyleTune: 12B · 26B · 26B V2 · 31B

---

Credits

Gryphe proved one tensor changes everything. Anthracite and Latitude helped make it happen. The MythoMax lineage found that lm_head carries style. Hugging Face made the format portable. llama.cpp made it runnable.

StyleTune: 12B · 26B · 26B V2 · 31B

All StyleTune links are examples. Replace with any compatible voice you prefer.

---

License

Voice tooling in this repository is Apache-2.0—see LICENSE. This covers the voice code and packaging only.

Voices, deltas, and base heads keep their source model license. When you run voice get or voice delta, the saved voice.json records the source license for your reference. For example, google/gemma-4-26b-a4b-it (display: google/gemma-4-26B-A4B-it) remains under its Gemma license (Apache-2.0 with Gemma terms—see google/gemma-4-26B-A4B-it and ai.google.dev/gemma/docs/gemma_4_license). Check the Hugging Face model card before you share a voiced model. The google/gemma-4-26b-a4b-it name in the frontmatter is the lowercased, verifiable HF ID (link lives, case-insensitive); gryphe/gemma-4-26b-a4b-styletune-v2 and other StyleTune names are illustrative examples—use any compatible model you have access to.