Instructions to use CQSystems/Google-Gemma-4-E4B-It-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Use Docker

docker model run hf.co/CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

LM Studio
Jan
Ollama
How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with Ollama:
```
ollama run hf.co/CQSystems/Google-Gemma-4-E4B-It-GGUF:F16
```
Unsloth Desktop

How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "CQSystems/Google-Gemma-4-E4B-It-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with Docker Model Runner:
```
docker model run hf.co/CQSystems/Google-Gemma-4-E4B-It-GGUF:F16
```

Lemonade

How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Run and chat with the model

lemonade run user.Google-Gemma-4-E4B-It-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use CQSystems/Google-Gemma-4-E4B-It-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf CQSystems/Google-Gemma-4-E4B-It-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "CQSystems/Google-Gemma-4-E4B-It-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

CQ-Gemma-E4B-IT (GGUF)

This repository contains quantized GGUF formats of the Google Gemma 4 E4B Instruction-Tuned model, published by CQ Systems.

These files are optimized for local inference on consumer hardware, particularly Apple Silicon (M-series Macs) and standard GPUs, using tools like llama.cpp, LM Studio, and Ollama.

Available Files

We provide three different versions of the model to suit different memory and performance requirements.

Filename	Size	Quantization	Recommendation / Notes
CQ_Gemma4_E4B_f16_Q4_K.gguf	5.0 GB	Q4_K_M	🏆 Recommended for most users. Best balance of speed, size, and quality. Easily fits in 8GB+ unified memory (e.g., M1/M2/M3/M4 Macs).
cq_gemma4_e4b_q8.gguf	7.5 GB	Q8_0	High quality 8-bit integer quantization. Nearly indistinguishable from the F16 baseline. Requires ~10GB+ RAM/VRAM.
CQ_Gemma4_E4B_f16.gguf	14.0 GB	F16	Baseline 16-bit float. Highest fidelity, highest memory footprint. Useful for re-quantization or research purposes.

How to Use

Using llama.cpp

You can run this model directly in your terminal using the compiled llama.cpp CLI tool.

# Basic chat launch with the recommended Q4_K model
./llama-cli -m CQ_Gemma4_E4B_f16_Q4_K.gguf \
-c 8192 \
-n 1024 \
-p "You are a helpful assistant.\n\nUser: Write a short joke about saving RAM.\nAssistant:"

Using LM Studio / Ollama

Download the .gguf file of your choice (we recommend CQ_Gemma4_E4B_f16_Q4_K.gguf).
LM Studio: Drag and drop the downloaded file into your LM Studio model folder, or use the local import feature.
Ollama: Create a Modelfile with the line FROM ./CQ_Gemma4_E4B_f16_Q4_K.gguf, then run ollama create cq-gemma-e4b -f Modelfile.

About the Base Model (Gemma 4 E4B)

Gemma 4 is a family of open, multimodal models built by Google DeepMind. The E4B variant is explicitly optimized for on-device processing, making it incredibly powerful for local deployment on laptops and mobile devices.

Key Capabilities

Thinking Mode: Features a built-in reasoning mode that lets the model think step-by-step before answering.
Massive Context: Supports a context window of up to 128K tokens.
Parameter Efficiency: "E4B" stands for 4.5B effective parameters (8B total with Per-Layer Embeddings). This allows it to perform well above its weight class while keeping active memory lookups fast.
System Prompts: Gemma 4 uses standard system, assistant, and user roles natively.

(Note: While the base Gemma 4 E4B supports audio and image inputs natively, these GGUF files primarily support text generation out-of-the-box via llama.cpp. Multimodal capabilities in GGUF require an accompanying mmproj vision/audio encoder file).

Thinking Mode Configuration

To enable the built-in reasoning capabilities, include the <|think|> token at the start of your system prompt. The model will then output its internal reasoning inside <|channel>thought\n ... <channel|> tags before providing the final response.

License & Acknowledgements

Base Model: Google DeepMind
License: Gemma 4 License
Quantization: CQ Systems using llama.cpp

For full benchmark details, evaluations, and ethical considerations, please refer to the original Google Gemma 4 E4B documentation.

Downloads last month: 91

GGUF

Model size

8B params

Architecture

gemma4

Hardware compatibility

16-bit

View +1 variant

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for CQSystems/Google-Gemma-4-E4B-It-GGUF

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Quantized

(386)

this model