Cerberus-E4B: Multi-Task Reasoning & Agentic Model

Cerberus-E4B is an open-weights, multi-task reasoning and agentic language model fine-tuned on top of Google's gemma-4-E4B-it architecture (~7.5B total parameters with frozen vision and audio towers, 4B active language backbone).

Cerberus is designed for advanced bilingual step-by-step reasoning (English & Russian), precise function calling via native Gemma 4 tool schemas, and versatile dual-channel generation.

Looking for quantized GGUF weights?
👉 Cerberus-E4B-GGUF

---

Highlights & Capabilities

• Dual-Channel Generation: Seamlessly toggles between long, deliberate step-by-step thought traces (<|channel>thought\n...<channel|>) and concise, direct responses (instant).
• Advanced Mathematical Reasoning: Shows a +16.7% accuracy improvement over the Google Gemma 4 base model on challenging competition-level mathematics (MATH-500).
• Native Tool Calling (Agentic): Trained with strict adherence to Gemma 4's native function calling protocol (<|turn>model\n<|tool_call>...<|call:function_name{...}>), ensuring structured execution for multi-turn agents.
• Deep Russian Language Alignment: Native-grade Russian math and logical reasoning without falling back to clumsy machine translation.
• Multimodal Architecture Compatibility: Trained with LoRA targeting the language reasoning backbone while keeping vision (ViT SigLIP, 280 soft tokens) and audio (Conformer, 16 kHz) encoders intact for drop-in GGUF mmproj support.

---

Benchmark Results (vs. Base Model)

All evaluations were conducted in greedy / deterministic mode comparing Cerberus-E4B directly against google/gemma-4-E4B-it:

Benchmark	Base Model (Gemma-4-E4B-it)	Cerberus-E4B	Delta (vs. Base)	Generation Mode
MATH-500	43.3%	60.0%	+16.7%	Thinking Mode (<thought>)
MATH-500	35.0%	48.0%	+13.0%	Instant Mode (Direct calculation)
MMLU-Redux	62.0%	63.0%	+1.0%	Multi-discipline academic knowledge
IFEval	90.8%	90.5%	Parity (~90%)	Complex system constraint following

---

Quickstart (Hugging Face Transformers)

Cerberus-E4B is released as an official PEFT LoRA adapter for google/gemma-4-E4B-it.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "google/gemma-4-E4B-it"
adapter_model_id = "rrlabs-ai/Cerberus-E4B"

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Apply Cerberus LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)

# 1. Thinking Mode Prompt (Step-by-step reasoning)
prompt = "Find the number of positive integers n <= 1000 such that gcd(n, 36) = 1."
formatted_input = f"<|turn>user\n{prompt}<turn|>\n<|turn>model\n<|channel>thought\n"

inputs = tokenizer(formatted_input, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.8,
    top_p=0.95,
    do_sample=True
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=False)
print(response)

---

Recommended Generation Presets

To ensure optimal generation quality across different tasks:

• For Deliberate Reasoning (Thinking mode):
  • Sampling: do_sample=True, temperature=0.8, top_p=0.95, top_k=64.
  • For llama.cpp / GGUF: Enable the dynamic DRY sampler (--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2) to eliminate n-gram loops during long thinking traces.
• For Direct Answers, Code & Function Calling (Instant mode):
  • Sampling: Greedy decoding (do_sample=False, temperature=0.0) for maximum determinism and syntax reliability.

---

GGUF & Quantized Models

Looking for standalone quantized GGUF weights for local execution on llama.cpp, LM Studio, or Ollama? Pre-quantized GGUF files (including Q4_K_M, Q5_K_M, Q8_0, BF16, and multimodal mmproj) are available at: 👉 rrlabs-ai/Cerberus-E4B-GGUF

---

📱 Recommended Web & Mobile Interface: rrchat

Cerberus-E4B pairs seamlessly with rrchat — an open-source, mobile-ready AI chat interface:

Use your local PC's llama.cpp from your phone. Clean mobile-ready AI chat interface with vision, web search, and privacy.

• 🌐 Project Repository: github.com/leftik1/rrchat
• 📷 Camera & Vision Ready: Upload photos or snapshots directly from your mobile camera and get instant multimodal visual analysis from Cerberus-E4B.
• 🔍 Web Search & Privacy: Built-in web search capabilities while keeping your inference and chat history 100% private and hosted on your local hardware.

---

Dataset Attribution & Licensing

Cerberus was trained on a carefully curated, decontaminated multi-source dataset. In accordance with dataset terms and open research best practices, we gratefully acknowledge the following data creators:

1. NVIDIA Nemotron:
   • Contains synthetic data derived from nvidia/Nemotron-Post-Training-Dataset-v2 and nvidia/Nemotron-SFT-Agentic-v2.
   • Attribution Notice: This work contains content derived from NVIDIA Nemotron Post-Training and SFT Agentic datasets, developed by NVIDIA Corporation and licensed under the NVIDIA Open Synthetic Data License.
2. Hugging Face Open-R1:
   • Mathematical reasoning traces sourced from open-r1/OpenR1-Math-220k (Apache-2.0 License).
3. Hugging Face SmolTalk:
   • General dialogue and structured conversation samples from HuggingFaceTB/smoltalk2 (Apache-2.0 License).
4. Vikhr Models (GrandMaster):
   • Russian-language STEM and complex instructions sourced from Vikhrmodels/GrandMaster-PRO-MAX (Apache-2.0 License).
5. Base Model:
   • Built on google/gemma-4-E4B-it, developed by Google DeepMind and governed by the Gemma Terms of Use.

---

License

This model adapter is released under the Apache-2.0 License. Use of the underlying base model remains subject to the Gemma Terms of Use.