offmonreal/Ornith-1.0-35B-MaxQuality-MTP-GGUF

Ornith-1.0-35B MaxQuality MTP GGUF

⚡ Speed at a glance

Variant	Context	Generation speed	CPU offload	Fits entirely in 16 GB VRAM
Q2_K-AllGPU MTP	10K	~192 tok/s	none — whole model + MTP head resident in VRAM	✅
Q3_K_S-MaxSpeed MTP	20K	~95 tok/s	13 expert layers on CPU	❌
Q3_K_M MTP	10K	~90 tok/s	20 expert layers on CPU	❌
Q4_K_M MTP	10K	~72 tok/s	20 expert layers on CPU	❌

The Q2_K-AllGPU file is the one to reach for on a single 16 GB consumer card: routed experts quantized to Q2_K (with the more error-sensitive down projection bumped one step to Q3_K to claw back quality), everything else the same iMatrix-guided recipe as the rest of this repo. Zero CPU offload means zero PCIe round-trips per token — that's most of the ~2x jump over the CPU-offloaded files above, not just the smaller weights.

Tested hardware and software

• Linux
• NVIDIA GeForce RTX 5060 Ti, 16 GB VRAM
• AMD Ryzen 9 5950X, 16 cores / 32 threads
• 64 GB system RAM
• TheTom/llama-cpp-turboquant, commit c26cbdffc (ships working --spec-type draft-mtp support for this architecture)

Speed climbs with context rather than falling on the CPU-offloaded files — the first tokens of a response are typically slower than steady-state generation once the draft head has a longer window of prior tokens to work from. Q2_K-AllGPU was measured at ~10K context.

:::warning Read before assuming a speedup MTP's benefit depends heavily on how much of the model is offloaded to CPU. When most of the model fits in VRAM, the draft-then-verify cycle is nearly free and the numbers above hold. Push the CPU-offload range wide enough (long context on a tight VRAM budget, many expert layers moved to system RAM), and the opposite can happen: we measured a ~50% slowdown on a sibling model at the same architecture, same heavy-CPU-offload 256K configuration, despite a 90%+ draft-acceptance rate — the extra CPU/PCIe round trips per speculative step outweighed the tokens saved. Always benchmark on your own -ot/offload profile before trusting a number measured on someone else's. :::

Same quality-first GGUF quants as Ornith-1.0-35B-MaxQuality-iMatrix-GGUF, with one addition: a Multi-Token Prediction (MTP) draft head grafted in, giving self-contained speculative decoding — no separate draft-model file needed, everything is in one GGUF.

The head was not trained for Ornith. Ornith is a fine-tune of Qwen/Qwen3.6-35B-A3B, and that base model ships a real, trained MTP head that Ornith's own release doesn't include. We extracted that head (verbatim, no retraining) and grafted it in as an extra transformer block, using only the ~900 MiB of donor weights that layer needs — not the full base model. Every other tensor in these files is untouched Ornith.

Files

File	Size	BPW	Recipe	Intended use
Ornith-1.0-35B_Q4_K_M.gguf	20.55 GiB	4.88	Same as Q4_K_M iMatrix	Higher quality, MTP for a speed bonus
Ornith-1.0-35B_Q3_K_M.gguf	16.89 GiB	3.98	Same as Q3 iMatrix	Balanced quality/size, MTP for a speed bonus
Ornith-1.0-35B_Q3_K_S-MaxSpeed.gguf	15.50 GiB	~3.7	Q3 iMatrix, routed-expert down projection relaxed from Q4_K to Q3_K	MaxSpeed: maximum throughput with a reasonable quality trade-off — the routed-expert down tensors give up one step of precision so the file, MTP head included, still comes in smaller than the plain Q3 with no head at all
Ornith-1.0-35B_Q2_K-AllGPU.gguf	13.09 GiB	3.03 (trunk)	Routed-expert gate/up at Q2_K, down bumped to Q3_K, embeddings/output at Q5_K, attention at Q4_K	AllGPU: the whole model, MTP head included, fits in a 16 GB card with zero CPU offload — the fastest file in this repo, at a real quality cost from the aggressive routed-expert Q2_K

All are calibrated with the same iMatrix as the base release (calibration_datav5.txt, 802 chunks). Ornith-1.0-35B_Q3_K_S-MaxSpeed.gguf and Ornith-1.0-35B_Q2_K-AllGPU.gguf are the only files with a changed base recipe; the Q4_K_M and Q3_K_M files are byte-identical to the non-MTP release except for the added MTP block.

MTP quantization

The grafted block (blk.40 in the GGUF — the model's own layer count plus one) is copied verbatim from the donor at its native precision: Q8_0 for the routed/shared expert and attention projections, BF16 for the router, F32 for norms. It is not requantized to match the trunk's Q4/Q3/Q2 precision — the head is small (under 1 GiB) and precision there disproportionately affects acceptance rate, so we left it alone.

Recommended launch commands (256K context)

Each command is self-contained — one file, --spec-type draft-mtp turns on the grafted head, no --model-draft needed. -ot ranges are wider than the equivalent non-MTP release because the MTP context itself needs extra VRAM (roughly 1 GiB at full 256K context) on top of the trunk.

Q2_K-AllGPU MTP — everything on GPU, no CPU offload

llama-server \
  --jinja --host 0.0.0.0 --port 8080 \
  -m ~/models/gguf/Ornith-1.0-35B_Q2_K-AllGPU.gguf \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  --n-gpu-layers 99 --n-cpu-moe 0 \
  --ctx-size 131072 --parallel 1 \
  --flash-attn on \
  --cache-type-k turbo3 --cache-type-v turbo3 \
  --batch-size 32768 --ubatch-size 1024 --cache-reuse 256

No -ot line at all — the entire trunk plus MTP head is small enough to sit in VRAM outright. Context is capped at 131072 (half of the other files' 256K) to leave headroom for the KV cache on GPU; push past that and the server falls back to CPU offload like the other variants.

Q4_K_M MTP

llama-server \
  --jinja --host 0.0.0.0 --port 8080 \
  -m ~/models/gguf/Ornith-1.0-35B_Q4_K_M.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --n-gpu-layers 99 --n-cpu-moe 0 \
  -ot "blk\.(2[0-9]|3[0-9])\.ffn_.*_exps\.weight=CPU" \
  --ctx-size 262144 --parallel 1 \
  --flash-attn on \
  --cache-type-k turbo3 --cache-type-v turbo3 \
  --batch-size 262144 --ubatch-size 1024 --cache-reuse 256

Q3_K_M MTP

llama-server \
  --jinja --host 0.0.0.0 --port 8080 \
  -m ~/models/gguf/Ornith-1.0-35B_Q3_K_M.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --n-gpu-layers 99 --n-cpu-moe 0 \
  -ot "blk\.(2[5-9]|3[0-9])\.ffn_.*_exps\.weight=CPU" \
  --ctx-size 262144 --parallel 1 \
  --flash-attn on \
  --cache-type-k turbo3 --cache-type-v turbo3 \
  --batch-size 262144 --ubatch-size 1024 --cache-reuse 256

Q3 MTP (MaxSpeed)

The lighter trunk needs fewer CPU-offloaded layers than the two files above — 13 instead of 20, more of the model stays on GPU.

llama-server \
  --jinja --host 0.0.0.0 --port 8080 \
  -m ~/models/gguf/Ornith-1.0-35B_Q3_K_S-MaxSpeed.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --n-gpu-layers 99 --n-cpu-moe 0 \
  -ot "blk\.(2[7-9]|3[0-9])\.ffn_.*_exps\.weight=CPU" \
  --ctx-size 262144 --parallel 1 \
  --flash-attn on \
  --cache-type-k turbo3 --cache-type-v turbo3 \
  --batch-size 262144 --ubatch-size 1024 --cache-reuse 256

Upstream model card

This release is based on the original Ornith-1.0-35B model card reproduced in full below. Its original license declaration and complete model card are preserved unchanged.

library_name: transformers license: mit license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/blob/main/LICENSE pipeline_tag: text-generation

Ornith-1.0-35B

Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.

Highlights:

• State-of-the-Art Coding Agents: Available in 9B-Dense, 31B-Dense, 35B-MoE, and 397B-MoE (post-trained on top of Gemma 4 and Qwen 3.5), achieving state-of-the-art performance among open-source models of comparable size on coding benchmarks such as Terminal-Bench 2.1, SWE-Bench, NL2Repo and OpenClaw.
• Self-Improving Training Framework: Ornith-1.0 employs RL to learn to generate not only solution rollouts, but also the scallfold that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model discovers better search trajectories and generates higher-quality solutions.
• Licence: MIT licensed, globally accessible, and free from regional limitations.

Ornith 1.0 35B

This model card documents Ornith-1.0-35B, the lightweight member of the Ornith family, designed for efficient single-GPU deployment.

Benchmarks

	Ornith-1.0-35B	Qwen3.5-35B	Qwen3.6-35B	Gemma4-31B	Qwen3.5-397B
Agentic Coding
Terminal-Bench 2.1 (Terminus-2)	64.2	41.4	52.5	42.1	53.5
Terminal-Bench 2.1 (Claude Code)	62.8	38.9	49.2	-	48.6
SWE-bench Verified	75.6	70	73.4	52	76.4
SWE-bench Pro	50.4	44.6	49.5	35.7	51.6
SWE-bench Multilingual	69.3	60.3	67.2	51.7	69.3
NL2Repo	34.6	20.5	29.4	15.5	36.8
Claw-eval Avg	69.8	65.4	68.7	48.5	70.7
SWE Atlas - QnA	37.1	13.2	15.5	-	20.4
SWE Atlas - RF	29.7	10.2	11.4	-	18.4
SWE Atlas - TW	27.8	9.8	13.3	-	18.5

* Terminal-Bench 2.1 (Terminus-2): We evaluate Terminal-Bench 2.1 using the Harbor/Terminus-2 framework with parser=json, temperature=1.0, top_p=1.0, and a 128K context window. Each run uses a 4-hour timeout with 32 CPU cores and 48GB RAM, and results are averaged over 5 runs. We adjust the Qwen chat template to ensure consistency between training and inference (https://huggingface.co/deepreinforce-ai/Ornith-1.0-397B/blob/main/chat_template.jinja), and modify Harbor to align with vLLM's reasoning_content key.
* Terminal-Bench 2.1 (Claude Code): We evaluate Terminal-Bench 2.1 using Claude Code 2.1.126 with parser=json, temperature=1.0, top_p=1.0, max_new_tokens=131072. Results are averaged over 5 runs. Again, Qwen chat template needs to be modified.
* SWE-Bench Verified, Pro and Multilingual: using OpenHands harness with temp=1.0, top_p=0.95, 256k context window.
* SWE Atlas QnA, RF, TW: using mini SWE agent harness with temp=1.0, top_p=0.95, 128K context window. Results are averaged over 5 runs.
* NL2Repo: with temperature=1.0, top_p=1.0, 400K context, 48K output and anti-hacking filters.
* ClawEval: An agentic code benchmark over real-user task distributions; temp=0.6 and 256K context.

Quickstart

📝 NOTE

Ornith-1.0-35B is a reasoning model: by default the assistant turn opens with a <think> … </think> block before the final answer. The serving recipes below enable a reasoning parser so the chain-of-thought is returned in a separate reasoning_content field, and a tool-call parser so the model's <tool_call> blocks are surfaced as OpenAI-style tool_calls.

Serving Ornith-1.0-35B requires recent runtimes:

• Transformers ≥ 5.8.1
• vLLM ≥ 0.19.1
• SGLang ≥ 0.5.9

Serving Ornith-1.0-35B

The two recipes below stand up an OpenAI-compatible server on a single 8×80GB GPU node (tensor-parallel 8). Adjust --tensor-parallel-size / --tp to the number of GPUs you have.

vLLM

vllm serve deepreinforce-ai/Ornith-1.0-35B \
    --served-model-name Ornith-1.0-35B \
    --tensor-parallel-size 8 \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.90 \
    --enable-prefix-caching \
    --enable-auto-tool-choice --tool-call-parser qwen3_xml \
    --reasoning-parser qwen3 \
    --trust-remote-code

SGLang

python -m sglang.launch_server \
    --model-path deepreinforce-ai/Ornith-1.0-35B \
    --served-model-name Ornith-1.0-35B \
    --tp 8 \
    --host 0.0.0.0 --port 8000 \
    --context-length 262144 \
    --mem-fraction-static 0.85 \
    --tool-call-parser qwen3_coder \
    --reasoning-parser qwen3

Hugging Face Transformers

For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-35B requires transformers >= 5.8.1.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "deepreinforce-ai/Ornith-1.0-35B"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a Python function is_prime(n). Keep it short."}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
generated = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
)
output_ids = generated[0][inputs.input_ids.shape[1]:]

# The reply contains a <think> ... </think> reasoning block followed by the answer.
content = tokenizer.decode(output_ids, skip_special_tokens=True)
print(content)

To split the reasoning trace from the final answer, parse on the </think> marker:

text = tokenizer.decode(output_ids, skip_special_tokens=True)
if "</think>" in text:
    reasoning, answer = text.split("</think>", 1)
    reasoning = reasoning.replace("<think>", "").strip()
    answer = answer.strip()
else:
    reasoning, answer = "", text.strip()

Using Ornith-1.0-35B via the Chat Completions API

Once a vLLM or SGLang server is running, talk to it with any OpenAI-compatible client.

Basic Usage

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",  # any non-empty string works for a local server
)

response = client.chat.completions.create(
    model="Ornith-1.0-35B",
    messages=[
        {"role": "user", "content": "Write a one-line Python lambda that squares a number."}
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=1024,
)

message = response.choices[0].message
# reasoning_content holds the <think> trace; content holds the final answer.
print("reasoning:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

You can also stream tokens, or hand the model tools — Ornith-1.0-35B emits well-formed function calls that the server parses into the standard tool_calls field:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="Ornith-1.0-35B",
    messages=[{"role": "user", "content": "What is the weather in Paris right now?"}],
    tools=tools,
    tool_choice="auto",
    temperature=0.6,
    max_tokens=2048,
)

tool_call = response.choices[0].message.tool_calls[0]
print(tool_call.function.name, tool_call.function.arguments)
# -> get_weather {"city": "Paris"}

You can point any OpenAI-compatible SDK (Python, Node.js, etc.) or curl at the same /v1/chat/completions endpoint.

Agentic Usage

Ornith-1.0-35B excels in tool-calling and agentic coding capabilities.

Agent Frameworks

Because Ornith-1.0-35B exposes an OpenAI-compatible endpoint with tool calling, it works out of the box with standard agent frameworks. Below is a minimal example that connects Ornith-1.0-35B to tools through an MCP server.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.getenv("OPENAI_BASE_URL", "http://localhost:8000/v1"),
    api_key=os.getenv("OPENAI_API_KEY", "EMPTY"),
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "run_shell",
            "description": "Run a shell command and return its output.",
            "parameters": {
                "type": "object",
                "properties": {
                    "command": {"type": "string", "description": "The command to run"}
                },
                "required": ["command"],
            },
        },
    }
]

messages = [{"role": "user", "content": "List the Python files in the current directory."}]

response = client.chat.completions.create(
    model="deepreinforce-ai/Ornith-1.0-35B",
    messages=messages,
    tools=tools,
    temperature=0.6,
    top_p=0.95,
)
print(response.choices[0].message)

Examples of using Ornith with agent harness:

Hermes Agent

# Hermes talks to any OpenAI-compatible endpoint — point it at your Ornith server.
export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"
export MODEL="deepreinforce-ai/Ornith-1.0-35B"

Atomic.chat/ Ollama / llama.cpp

# Both runtimes load a GGUF build of Ornith (publish one at deepreinforce-ai/Ornith-1.0-35B-GGUF).

# llama.cpp — serve an OpenAI-compatible API on port 8000.
llama-server -hf deepreinforce-ai/Ornith-1.0-35B-GGUF --port 8000 -c 262144

# Ollama — pull and chat with the same GGUF straight from Hugging Face.
ollama run hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF

OpenClaw

# OpenClaw talks to any OpenAI-compatible endpoint — point it at your Ornith server.
export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"
export OPENAI_MODEL="deepreinforce-ai/Ornith-1.0-35B"

Unsloth Studio

pip install unsloth

# Load Ornith for fast local inference or fine-tuning (Python):
#   from unsloth import FastLanguageModel
#   model, tokenizer = FastLanguageModel.from_pretrained(
#       "deepreinforce-ai/Ornith-1.0-35B",
#       max_seq_length=262144,
#       load_in_4bit=True,
#   )

OpenHands

pip install openhands-ai

# OpenHands routes through LiteLLM; the "openai/" prefix selects the OpenAI-compatible path.
export LLM_MODEL="openai/deepreinforce-ai/Ornith-1.0-35B"
export LLM_BASE_URL="http://localhost:8000/v1"
export LLM_API_KEY="EMPTY"

# Launch the CLI (or run the official OpenHands Docker image with the same env vars).
openhands

Coding CLIs

Ornith-1.0-35B is optimized for terminal-based coding agents. Point any OpenAI-compatible coding CLI at your Ornith-1.0-35B endpoint (set OPENAI_BASE_URL and OPENAI_API_KEY) to understand large codebases, automate tedious work, and ship faster.

OpenCode

# Register your local Ornith endpoint as a provider in ~/.config/opencode/opencode.json:
#
# {
#   "$schema": "https://opencode.ai/config.json",
#   "provider": {
#     "ornith": {
#       "npm": "@ai-sdk/openai-compatible",
#       "name": "Ornith (local)",
#       "options": { "baseURL": "http://localhost:8000/v1", "apiKey": "EMPTY" },
#       "models": { "deepreinforce-ai/Ornith-1.0-35B": { "name": "Ornith-1.0-35B" } }
#     }
#   }
# }

opencode

Citation

If you find our work helpful, feel free to give us a cite.

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}