kerasformers/gemma-4-12b

See our collection for all Gemma 4 sizes and variants.

Run Gemma 4 with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/gemma-4-12b

Pure-Keras 3 conversion of google/gemma-4-12B for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX. This is the 12B variant, served here as image + audio + text -> text via Gemma4UnifiedProcessor; weights are stored in bfloat16.

For model details, license, and usage terms, see Google's model card.

Gemma 4 family

Property	E2B	E4B	12B Unified	31B Dense
Total Parameters	2.3B effective (5.1B with embeddings)	4.5B effective (8B with embeddings)	11.95B	30.7B
Layers	35	42	48	60
Sliding Window	512 tokens	512 tokens	1024 tokens	1024 tokens
Context Length	128K tokens	128K tokens	256K tokens	256K tokens
Vocabulary Size	262K	262K	262K	262K
Supported Modalities	Text, Image, Audio	Text, Image, Audio	Text, Image, Audio	Text, Image
Vision Encoder Parameters	~150M	~150M	-	~550M
Audio Encoder Parameters	~300M	~300M	-	No Audio

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.gemma4_unified import Gemma4UnifiedGenerate, Gemma4UnifiedProcessor

model = Gemma4UnifiedGenerate.from_weights("kerasformers/gemma-4-12b")
processor = Gemma4UnifiedProcessor.from_weights("kerasformers/gemma-4-12b")

inputs = processor(conversation=[
    {"role": "user", "content": [
        {"type": "image", "image": Image.open("cat.jpg")},
        {"type": "audio", "path": "clip.wav"},
        {"type": "text", "text": "Describe the image and what you hear."},
    ]}
])
outputs = model.generate(**inputs, max_new_tokens=64)
print(processor.decode(outputs[0]))

Load any Gemma 4 variant the same way with from_weights("kerasformers/<variant>"):

Variant	Hub
gemma-4-12b	kerasformers/gemma-4-12b
gemma-4-12b-it	kerasformers/gemma-4-12b-it
gemma-4-26b-a4b	kerasformers/gemma-4-26b-a4b
gemma-4-26b-a4b-it	kerasformers/gemma-4-26b-a4b-it
gemma-4-31b	kerasformers/gemma-4-31b
gemma-4-31b-it	kerasformers/gemma-4-31b-it
gemma-4-e2b	kerasformers/gemma-4-e2b
gemma-4-e2b-it	kerasformers/gemma-4-e2b-it
gemma-4-e4b	kerasformers/gemma-4-e4b
gemma-4-e4b-it	kerasformers/gemma-4-e4b-it

Tips

• Set KERAS_BACKEND before importing Keras / kerasformers.
• Loads in bfloat16 by default. Pass load_dtype="float32" for full precision, or quantization="int8" to shrink further.
• See the Gemma 4 docs.
• Community / upstream weights still work via the hf: prefix: Gemma4UnifiedGenerate.from_weights("hf:google/gemma-4-12B").

Special Thanks

A huge thank you to the Google Gemma authors for creating and releasing these models.

License: Apache 2.0.