gemma-cvantic

multimodal SFT fine-tune of Google's gemma-4-E4B-it · GGUF

---

What is this?

An experimental instruction fine-tune of google/gemma-4-E4B-it — the 4.5B-effective-parameter omni-modal Gemma 4 model (128K context, text + image + audio input).

Trained with SFT (13,000 examples) to strengthen math, coding, physics/astronomy reasoning and Russian-language instructions. Quantized to GGUF with llama.cpp for fast local inference.

---

Quantization lineup

File	Quant	Size	Quality / speed
gemma-cvantic.Q8_0.gguf	Q8_0	~7.6 GB	near-lossless, best quality
gemma-cvantic.Q5_K_S.gguf	Q5_K_S	~5.4 GB	great balance ★ recommended
gemma-cvantic.IQ4_XS.gguf	iQ4_XS (imatrix)	~4.8 GB	smallest, fast, slightly less accurate
gemma-cvantic.BF16-mmproj.gguf	vision projector	~0.9 GB	required for image input

mmproj is the multimodal (vision) projector — pass it with --mmproj to enable image understanding. All quants are BF16/FP16 conversions of the same merged weights, so any main-file + mmproj combo works.

---

Training recipe

Parameter	Value
Base model	google/gemma-4-E4B-it
Method	SFT (DoRA / LoRA-style adapter, then merged)
Trainable params	~36.7M (0.61%)
Examples	13,000
Context	128K (inherited)

Dataset mix (MIT / Apache-2.0 only):

Dataset	Split	Rows	Domain
HuggingFaceH4/ultrachat_200k	train_sft	5,000	general chat
theblackcat102/evol-codealpaca-v1	train	3,000	coding
qwedsacf/competition_math (MATH)	train	2,000	math
HuggingFaceTB/cosmopedia	openstax · physics/astronomy	2,000	physics & astronomy
openai/gsm8k	main/train	1,000	grade-school math

---

Quick start

llama.cpp (text)

llama-cli \
  -m gemma-cvantic.Q5_K_S.gguf \
  -p "Реши задачу: если цена товара выросла на 20% и составила 480 руб., какой была исходная цена?"

Multimodal (vision)

llama-cli \
  -m gemma-cvantic.Q5_K_S.gguf \
  --mmproj gemma-cvantic.BF16-mmproj.gguf \
  -i

> what's in this photo?

llama-server (OpenAI-compatible API)

llama-server \
  -m gemma-cvantic.Q8_0.gguf \
  --mmproj gemma-cvantic.BF16-mmproj.gguf \
  --port 8080

import openai

client = openai.OpenAI(base_url="http://localhost:8080/v1", api_key="local")
resp = client.chat.completions.create(
    model="gemma-cvantic",
    messages=[{"role": "user", "content": "Расскажи про эффект Доплера на пальцах"}],
)
print(resp.choices[0].message.content)

---

Picking a file

• CPU-only, want quality → Q5_K_S (fits ~8 GB RAM/VRAM, sweet spot)
• 16 GB+ / strong GPU → Q8_0
• Tiny footprint / speed first → iQ4_XS (fits ~6 GB)
• Vision → always add the BF16-mmproj

Sizes are approximate; VRAM usage depends on context length.

---

Notes

• Working-name "cvantic" — experimental build, quality varies per domain. Think of it as a field test of Gemma 4 E4B fine-tuning.
• Text-only SFT; vision/audio behavior inherited from the base model and not specifically tuned.
• Base model: Google DeepMind · License: Apache 2.0 · Gemma 4 docs

---

Quantized with llama.cpp (BF16 base + imatrix for iQ4_XS).