Instructions to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Use Docker

docker model run hf.co/shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shafire/OpenZero-Gemma4-E2B-Agentic-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shafire/OpenZero-Gemma4-E2B-Agentic-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Ollama
How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with Ollama:
```
ollama run hf.co/shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M
```

Unsloth Studio

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for shafire/OpenZero-Gemma4-E2B-Agentic-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for shafire/OpenZero-Gemma4-E2B-Agentic-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for shafire/OpenZero-Gemma4-E2B-Agentic-GGUF to start chatting

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with Docker Model Runner:
```
docker model run hf.co/shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M
```

Lemonade

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.OpenZero-Gemma4-E2B-Agentic-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use shafire/OpenZero-Gemma4-E2B-Agentic-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "shafire/OpenZero-Gemma4-E2B-Agentic-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

OpenZero Gemma4 E2B Agentic — Standalone GGUF

ONE FILE. NO ADAPTER. NO BASE-MODEL HUNT.

OpenZero Gemma4 E2B Agentic is a small local GGUF research, coding and tool-workflow specialist. Choose the verified Q4_K_M, Q8_0 or F16 build; no PEFT archive or separate base model is required to run it.

What is included

File	Purpose
`OpenZero-Gemma4-E2B-Agentic-Q4_K_M.gguf`	Recommended balance of size and quality
`OpenZero-Gemma4-E2B-Agentic-Q8_0.gguf`	Higher fidelity, larger download
`OpenZero-Gemma4-E2B-Agentic-F16.gguf`	Reference full-precision GGUF

Standalone model: yes
Separate adapter required: no
Separate base model required: no
Base architecture: google/gemma-4-E2B-it
Fine-tuning: 2,606 training examples; 137 held-out evaluation examples
Final held-out loss: 1.520900
Held-out token accuracy: 0.657464
GGUF conversion and CPU load test: passed for all three files with llama.cpp
Modality in this repository: text-only; no mmproj vision/audio projector is included

Run with llama.cpp

hf download shafire/OpenZero-Gemma4-E2B-Agentic-GGUF OpenZero-Gemma4-E2B-Agentic-Q4_K_M.gguf --local-dir .

llama-cli -m OpenZero-Gemma4-E2B-Agentic-Q4_K_M.gguf --jinja -c 8192 -t 8 --temp 0.6 --top-p 0.95

Positioning

Built for private local inference, practical code work, evidence-aware research and agent runtimes. Tool execution belongs to the surrounding runtime; validate outputs before acting on them.

Verified release

Training adapter: QLoRA specialist run, 82/82 steps
Train loss: 2.261954
Held-out evaluation loss: 1.520900
Held-out token accuracy: 0.657464
Q4_K_M — 3,416,119,872 bytes — SHA-256 9a7e717d13208526782c5fa5074bbcd7f445c6ac720e88d0e0e3ae6d87ebdeef
Q8_0 — 4,947,414,592 bytes — SHA-256 b612e7a6ffc2b9707f3b68e7e94fc8f8764e0975fefa4d1256717356f18af19f
F16 — 9,273,527,872 bytes — SHA-256 51e25487e2193df69b9271ae8ad7bdaffb6197a861b65f6e27c70e2a556fbbb0

Provenance and reproducibility

The V12 LoRA adapter was trained for 82 steps (one epoch) on 2,606 OpenZero instruction examples. A separate 137-row held-out set was used only for final evaluation. Training used unsloth/gemma-4-E2B-it-unsloth-bnb-4bit; the adapter was merged into google/gemma-4-E2B-it revision 3e22461f65e89153144f8adb70e3b8c2cc9845a7. The merged model was converted with llama.cpp b10333 / commit 08659901c43b51de735740f1cf61bb82fbe0c4e4; Q8_0 and Q4_K_M were independently quantized from F16. A narrowly asserted converter compatibility shim derived global_head_dim=512 from the preserved per-layer Gemma configuration. Every file passed a bounded one-shot CPU text load test (-c 128 -n 1 -ngl 0 --no-conversation --single-turn --simple-io --no-warmup) and remote size/LFS SHA verification.

This is a practical specialist fine-tune, not a claim of superiority over the base model. Tool calls are text emitted for an agent runtime to validate and execute. This repository does not include Gemma's multimodal projector.

This model is an independent fine-tune based on Gemma and is not affiliated with or endorsed by Google. Preserve Google Gemma attribution and applicable upstream terms. OpenZero training materials and resulting community release are subject to the OpenZero Community Source terms; do not describe this release as OSI-approved open source. Review both upstream and OpenZero terms before redistribution or commercial use.