Instructions to use talisma/Gemma-4-E4B-Reasoning with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use talisma/Gemma-4-E4B-Reasoning with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Use Docker

docker model run hf.co/talisma/Gemma-4-E4B-Reasoning:Q4_K_M

LM Studio
Jan

vLLM

How to use talisma/Gemma-4-E4B-Reasoning with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "talisma/Gemma-4-E4B-Reasoning"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "talisma/Gemma-4-E4B-Reasoning",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Ollama
How to use talisma/Gemma-4-E4B-Reasoning with Ollama:
```
ollama run hf.co/talisma/Gemma-4-E4B-Reasoning:Q4_K_M
```

Unsloth Studio

How to use talisma/Gemma-4-E4B-Reasoning with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for talisma/Gemma-4-E4B-Reasoning to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for talisma/Gemma-4-E4B-Reasoning to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for talisma/Gemma-4-E4B-Reasoning to start chatting

How to use talisma/Gemma-4-E4B-Reasoning with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "talisma/Gemma-4-E4B-Reasoning:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use talisma/Gemma-4-E4B-Reasoning with Docker Model Runner:
```
docker model run hf.co/talisma/Gemma-4-E4B-Reasoning:Q4_K_M
```

Lemonade

How to use talisma/Gemma-4-E4B-Reasoning with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Run and chat with the model

lemonade run user.Gemma-4-E4B-Reasoning-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use talisma/Gemma-4-E4B-Reasoning with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use talisma/Gemma-4-E4B-Reasoning with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf talisma/Gemma-4-E4B-Reasoning:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "talisma/Gemma-4-E4B-Reasoning:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

🧠 Gemma‑4‑E4B‑Reasoning

A pocket‑sized reasoner. Quantized for the real world.

📄 Google's Official Model Card · 📘 Technical Report · 💬 Talisma AI Studio

🍿 TL;DR — what even is this? (for the non‑techie reader)

Imagine a very well‑read, very fast‑thinking assistant that Google DeepMind built (called Gemma 4), and Talisma's AI team took its E4B "effective 4‑billion parameter" version — small enough to run on a laptop, GPU server, or even a beefy phone — and repackaged it into several compressed file sizes (called "quantizations") so you can pick exactly how much RAM/VRAM you want to spend versus how sharp you want the model to be.

Think of it like exporting the same photo as a tiny thumbnail, a web‑res JPEG, or a full‑resolution RAW file — same picture, different file sizes for different needs. That's what the Q2_K → f16 files below are.

This particular checkpoint is tuned for reasoning — step‑by‑step thinking before it answers — while still understanding text, images, and audio.

✨ Why this model is actually exciting

🧩 Thinks before it speaks

Built‑in "thinking mode" — the model reasons internally, then gives you a clean final answer. Great for math, logic, and multi‑step problems.

👀 Sees & hears too

Not just text — it reads documents, screenshots, charts, handwriting, and even transcribes/translates speech.

🪶 Light enough for edge devices

The "E" stands for effective parameters — clever engineering (Per‑Layer Embeddings) makes an 8B‑parameter model feel and run like a ~4.5B one.

🌍 Speaks 140+ languages

Pretrained across 140+ languages, with strong out‑of‑the‑box support for 35+.

🛠️ Built for agents

Native function/tool‑calling support — plug it into workflows, apps, and automations (yes, it plays nicely with tools like n8n).

📚 Remembers a lot at once

128K‑token context window — feed it long documents, transcripts, or chat histories without losing the thread.

🎯 Who is this for & what can you build with it?

🖱️ Click to expand real‑world use cases

If you are a...	You can use it to...
🧑‍💻 Developer	Build local, private chatbots and coding assistants that don't call an external API
🏢 Enterprise / CXM team	Power customer‑support agents, ticket summarizers, and document‑parsing pipelines (like Talisma's own CXM/CRM stack)
🎓 Student / Researcher	Get step‑by‑step explanations for math, science, and logic problems
📄 Ops / Back‑office teams	Extract data from scanned PDFs, invoices, forms, and handwritten notes (OCR + document understanding)
🌐 Global product teams	Localize content or build multilingual support bots across 140+ languages
🎙️ Voice‑app builders	Transcribe and translate speech (ASR + speech translation) — no separate speech model needed
🤖 Automation engineers	Wire it into agentic workflows using native function‑calling

🔬 Under the hood — E4B specifications

Extracted specifically for the E4B variant from Google DeepMind's official Gemma 4 model card.

Property	Gemma‑4‑E4B
Total Parameters	4.5B effective (8B with embeddings)
Architecture	Dense, decoder‑only transformer with hybrid attention (local sliding window + global)
Layers	42
Sliding Window	512 tokens
Context Length	128K tokens
Vocabulary Size	262K
Modalities Supported	Text, Image, Audio
Vision Encoder	~150M params (variable aspect ratio & resolution)
Audio Encoder	~300M params (up to 30s audio input)
Training Data Cutoff	January 2025
License	Apache 2.0

📎 Architecture notes (click to expand)

Gemma 4 uses a hybrid attention mechanism that interleaves local sliding‑window attention with full global attention — the final layer is always global. This gives it the speed and low memory footprint of a lightweight model, without losing awareness on long‑context tasks. Global attention layers use unified Keys/Values and Proportional RoPE (p‑RoPE) to keep memory in check on long sequences.

The "E" in E4B stands for effective parameters: Per‑Layer Embeddings (PLE) give each decoder layer its own compact token embedding used purely for fast lookups. These tables inflate the total parameter count (8B) without inflating actual compute cost — so it runs like a ~4.5B model.

📊 Benchmark performance (Gemma‑4‑E4B, instruction‑tuned)

Benchmark	Score	What it measures
MMLU Pro	69.4%	Broad academic/professional knowledge
AIME 2026 (no tools)	42.5%	Competition‑level math reasoning
LiveCodeBench v6	52.0%	Real‑world coding ability
Codeforces ELO	940	Competitive programming skill
GPQA Diamond	58.6%	Graduate‑level science reasoning
Tau2 (avg of 3)	42.2%	Multi‑turn agentic tool‑use
BigBench Extra Hard	33.1%	Hard, diverse reasoning tasks
MMMLU	76.6%	Multilingual knowledge
MMMU Pro (vision)	52.6%	College‑level visual reasoning
OmniDocBench 1.5 (↓ better)	0.181	Document parsing accuracy
MATH‑Vision	59.5%	Math problems presented as images
MedXPertQA MM	28.7%	Medical multimodal reasoning
CoVoST (audio)	35.54	Speech translation quality
FLEURS (↓ better)	0.08	Speech recognition error rate
MRCR v2 @128K (long context)	25.4%	Needle‑in‑haystack recall at long context

For comparison, the previous generation Gemma 3 27B scored 67.6% on MMLU Pro and only 20.8% on AIME 2026 — E4B beats a 6x‑larger predecessor on reasoning‑heavy tasks while running far lighter.

📦 Available quantizations

We took the Gemma‑4‑E4B base model and compressed it into multiple GGUF quantization levels, so you can trade off file size, RAM usage, and output quality depending on your hardware.

File	Size	Quality	Speed	Recommended for
`Gemma-4-E4B-Q2_K.gguf`	4.4 GB	⭐⭐	⚡⚡⚡⚡⚡	Extremely constrained devices, testing only
`Gemma-4-E4B-Q3_K_M.gguf`	4.85 GB	⭐⭐⭐	⚡⚡⚡⚡	Low‑RAM laptops, quick prototyping
`Gemma-4-E4B-Q4_K_M.gguf`	5.34 GB	⭐⭐⭐⭐	⚡⚡⚡⚡	✅ Best all‑round balance — most users start here
`Gemma-4-E4B-Q5_K_M.gguf`	5.76 GB	⭐⭐⭐⭐	⚡⚡⚡	Better quality, still light
`Gemma-4-E4B-Q6_K.gguf`	6.22 GB	⭐⭐⭐⭐⭐	⚡⚡⚡	Near full‑precision quality
`Gemma-4-E4B-Q8_0.gguf`	8.03 GB	⭐⭐⭐⭐⭐	⚡⚡	Max quality quantization, workstation GPUs
`Gemma-4-E4B-f16-Base_model.gguf`	15.1 GB	🏆 Full precision	⚡	Research, fine‑tuning, maximum fidelity

💡 Rule of thumb: allocate roughly file size + 2 GB of RAM/VRAM for comfortable inference (extra headroom for context and KV‑cache at longer sequence lengths).

🤔 Not sure which one to pick? Click here

Just exploring / on a laptop with 8–16 GB RAM → go with Q4_K_M
Have a decent GPU (12 GB+ VRAM) and want the best answers → go with Q6_K or Q8_0
Fine‑tuning, research, or need bit‑for‑bit fidelity → use f16-Base_model
Running on something really small (Raspberry Pi‑class, phones) → Q2_K or Q3_K_M, but expect lower reasoning quality

🚀 Quick start

🦙 Using llama.cpp

# download your preferred quantization
huggingface-cli download talisma/Gemma-4-E4B-Reasoning Gemma-4-E4B-Q4_K_M.gguf --local-dir .

# run it
./llama-cli -m Gemma-4-E4B-Q4_K_M.gguf -p "Explain photosynthesis to a 10 year old." -cnv

🐍 Using Python (transformers, base model)

from transformers import AutoProcessor, AutoModelForMultimodalLM

MODEL_ID = "google/gemma-4-E4B-it"

processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(
    MODEL_ID, dtype="auto", device_map="auto"
)

messages = [
    {"role": "system", "content": "<|think|>You are a helpful assistant."},
    {"role": "user", "content": "Write a short joke about saving RAM."},
]

inputs = processor.apply_chat_template(
    messages, tokenize=True, return_dict=True, return_tensors="pt",
    add_generation_prompt=True, enable_thinking=True
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=False))

💻 Using Ollama

ollama run hf.co/talisma/Gemma-4-E4B-Reasoning:Q4_K_M

⚙️ Recommended settings

Setting	Value
`temperature`	1.0
`top_p`	0.95
`top_k`	64
Thinking mode	Add `<\|think\|>` at the start of the system prompt to enable step‑by‑step reasoning
Multimodal input order	Images before text · Audio after text

🧵 Multi‑turn tip: when building conversation history, only keep the model's final answers — strip out prior "thinking" content before the next user turn (except during tool‑call turns, where thinking should be preserved).

🧬 Provenance & attribution

Base model:      google/gemma-4-E4B-it   (Google DeepMind)
Quantized by:    Talisma AI Studio
Format:          GGUF
Method:          llama.cpp quantization
License:         Apache 2.0 (inherited from base model)

Talisma AI Studio evaluates and benchmarks open‑weight models (Gemma, Qwen, Llama, Mistral, Phi, and embedding models) to power Talisma's enterprise CXM/CRM and Agentic AI products. This repository packages Google DeepMind's Gemma‑4‑E4B for efficient, on‑premise, and edge deployment.

⚠️ Good to know

Limitations & responsible use

Like all LLMs, this model can produce incorrect or outdated factual statements — verify important outputs.
Performance depends heavily on training data scope; niche or highly specialized domains may be handled less reliably.
Audio input is capped at 30 seconds and video at ~60 seconds (processed as 1 frame/sec).
Google DeepMind applies rigorous safety filtering (including CSAM and sensitive‑data filtering) during training — see the official responsible AI guidelines for more.

📖 Citation

@misc{gemmateam2026gemma4,
      title={Gemma 4 Technical Report},
      author={Gemma Team},
      year={2026},
      eprint={2607.02770},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.02770},
}

Made with 🧡 by Talisma AI Studio · Base model by Google DeepMind

🤗 Talisma on Hugging Face · 📄 License: Apache 2.0