Instructions to use OliviaRossi/KAT-Ornith-Coder-35B-A3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OliviaRossi/KAT-Ornith-Coder-35B-A3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OliviaRossi/KAT-Ornith-Coder-35B-A3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained("OliviaRossi/KAT-Ornith-Coder-35B-A3B")
model = AutoModelForCausalLM.from_pretrained("OliviaRossi/KAT-Ornith-Coder-35B-A3B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OliviaRossi/KAT-Ornith-Coder-35B-A3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OliviaRossi/KAT-Ornith-Coder-35B-A3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliviaRossi/KAT-Ornith-Coder-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OliviaRossi/KAT-Ornith-Coder-35B-A3B

SGLang

How to use OliviaRossi/KAT-Ornith-Coder-35B-A3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OliviaRossi/KAT-Ornith-Coder-35B-A3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliviaRossi/KAT-Ornith-Coder-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OliviaRossi/KAT-Ornith-Coder-35B-A3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OliviaRossi/KAT-Ornith-Coder-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OliviaRossi/KAT-Ornith-Coder-35B-A3B with Docker Model Runner:
```
docker model run hf.co/OliviaRossi/KAT-Ornith-Coder-35B-A3B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

KAT-Ornith-Coder-35B-A3B 🦅⚡

The Brain of Ornith + The Hands of KAT
An Asymmetric DARE-TIES Fusion of State-of-the-Art Sparse MoE Coder Models

Model Repository • Base: KAT-Coder-V2.5-Dev • Donor: Ornith-1.5-35B-A3B

💡 Overview & Philosophy

KAT-Ornith-Coder-35B-A3B is a specialized, post-trained hybrid Mixture-of-Experts (MoE) model built specifically for autonomous agentic software engineering (e.g., Claude Code, OpenHands, Aider, and Cline).

In agentic coding workflows, models consistently fail in one of two distinct regimes:

The Planning Deficit: Models with terse, direct execution often take action too quickly, failing to explore unfamiliar repositories, misdiagnosing complex bugs, or getting stuck on multi-file architectural refactors.
The Execution Deficit: Deep-reasoning models often produce verbose chains of thought, drift into hallucinated file states, overwrite entire 2,000-line files when fixing a one-line bug, or corrupt tool-call XML syntax in long multi-turn sessions.

KAT-Ornith-Coder-35B-A3B resolves this tradeoff through an Asymmetric Task-Vector Merge:

               ┌─────────────────────────────────────────────────────────┐
               │              KAT-Ornith-Coder-35B-A3B              │
               └────────────────────────────┬────────────────────────────┘
                                          │
                     ┌──────────────────────┴──────────────────────┐
                     ▼                                         ▼
        [ THE BRAIN: Ornith-1.5 ]                     [ THE HANDS: KAT-Coder ]
  • Autonomous RL self-improvement              • 100k KwaiClawEnv sandbox PPO
  • 79.0% SWE-bench Verified planning           • 0.28% abnormal tool label rate
  • Dominates DeltaNet Linear Attention         • Anchors Embeddings, LM Head & Routers
  • Deep hypothesis & root-cause analysis       • Surgical diff & patch generation

The Brain (ornith-ai/Ornith-1.5-35B-A3B): Injected into the Gated DeltaNet linear attention and self-attention projections to supply deep multi-step algorithmic planning, high exploration rollouts, and SWE-bench Verified reasoning ceiling.
The Hands (Kwaipilot/KAT-Coder-V2.5-Dev): Anchors the token embeddings, LM head, and MoE routing gates to preserve its ultra-low tool syntax error rate (0.28%), clean localized diffs, and zero-loop execution discipline.

🧠 Underlying Architecture

Both parent models share an identical, next-generation sparse MoE backbone with linear attention hybrid layers:

Total Parameters: ~35 Billion
Active Parameters per Token: ~3 Billion (8 routed experts + 1 shared expert)
Layer Depth: 40 Layers
Hybrid Attention Topology: 30 Gated DeltaNet linear attention layers interleaved with 10 full attention layers (every 4th layer)
Expert Count: 256 fine-grained routed experts + 1 shared expert
Context Window: Native 262,144 tokens (262k)

🔬 How It Was Created: Detailed Merge Recipe

The merge was synthesized using Asymmetric DARE-TIES (Drop And REscale with Task Information Sign-consensus). Instead of a uniform 50/50 blend (which corrupts MoE routing boundaries), the model deltas were applied differentially across functional sub-networks:

1. Functional Parameter Mapping

$\Delta_W = W_{\text{Ornith}} - W_{\text{KAT}}$

$\hat{\Delta}_W = \frac{\Delta_W \odot \text{Bernoulli}(\text{density})}{\text{density}}$

$W_{\text{merged}} = W_{\text{KAT}} + (\text{weight} \times \hat{\Delta}_W)$

Component	Target Tensors	Weight (KAT / Ornith)	DARE Density	Technical Rationale
Embeddings & Vocab	`embed_tokens`, `lm_head`	100% / 0%	`1.00`	Strictly preserves KAT's native token distributions and prevents format corruption in `<tool_call>` / `<tool_response>` delimiters.
MoE Routing Gates	`mlp.gate`, `router`	75% / 25%	`0.70`	Strongly anchors router logits on KAT's real-sandbox RL to prevent expert collision and retain near-zero abnormal tool calls.
Attention Blocks	`self_attn`, `linear_attn` (DeltaNet)	45% / 55%	`0.55`	Ornith dominates the attention projections, importing deep repository context tracking and multi-turn planning capacity.
MoE Experts	`mlp.experts.*`, `shared_expert`	60% / 40%	`0.60`	Retains KAT's surgical diff synthesis while assimilating Ornith's high-level algorithmic logic and edge-case bug fixes.
LayerNorms & Scalars	`input_layernorm`, `post_attention_layernorm`	60% / 40%	`0.70`	Maintains numeric stability across Gated DeltaNet state updates.

2. Zero-Cache Shard-by-Shard Synthesis Engine

Due to the sheer size of the 35B MoE architecture (~70 GB in BF16), standard merging utilities often fail or run out of disk space. This model was synthesized using a specialized streaming merge pipeline:

Direct HTTPS Chunk Streaming: Streamed directly into temporary buffers without creating intermediate uncompressed blobs.
In-Memory Tensor Transformation: Transformed each tensor in RAM, explicitly cloning untransformed tensors to release memory-mapped (mmap) Linux file descriptors.
Single-Shard Commit Pipeline: Each of the 13 shards (model-00000-of-00013.safetensors through model-00012-of-00013.safetensors) was created, validated, uploaded to Hugging Face, and purged locally to ensure zero numeric truncation.

📊 Performance Profile & Benchmark Context

The parents of this merge hold top-tier standing in the 35B-A3B open-weights class:

Metric / Benchmark	KAT-Coder-V2.5-Dev	Ornith-1.5-35B-A3B	KAT-Ornith-Coder-35B-A3B (Targeted)
SWE-bench Verified	69.4%	79.0%	Balanced / Uplifted (~74-78%)
SWE-bench Pro	45.96%	59.6%	High exploration recovery
Terminal-Bench 2.1	41.02%	68.5%	Elevated shell competence
Abnormal Tool Label Rate	0.28%	~1.4%	Preserved (<0.4%)
Diff Generation Strategy	Surgical / Minimal patch	Verbose / Full-file rewrite risk	Surgical & Precise
Context-Poison Resistance	High	85 / 100	High (Grounded & Resilient)

🚀 Deployment & Serving

1. High-Throughput Production Serving with vLLM

For local or server deployment with full XML tool-calling and hybrid attention acceleration:

vllm serve OliviaRossi/KAT-Ornith-Coder-35B-A3B \
    --tensor-parallel-size 2 \
    --max-model-len 131072 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_xml \
    --reasoning-parser qwen3 \
    --dtype bfloat16

(For the full 262k context window, allocate --tensor-parallel-size 4 on 4x A100/H100/A6000 Ada GPUs).

2. Local Inference via llama.cpp / GGUF

To run locally in quantized GGUF format:

# Convert to GGUF (from repository root)
python3 convert_hf_to_gguf.py /path/to/KAT-Ornith-Coder-35B-A3B \
    --outfile KAT-Ornith-Coder-35B-A3B-bf16.gguf \
    --outtype bf16

# Quantize to Q4_K_M (fits on ~24GB VRAM GPU or Apple Silicon unified memory)
llama-quantize KAT-Ornith-Coder-35B-A3B-bf16.gguf KAT-Ornith-Coder-35B-A3B-Q4_K_M.gguf Q4_K_M

# Run server
llama-server \
    -m KAT-Ornith-Coder-35B-A3B-Q4_K_M.gguf \
    -c 32768 \
    --n-gpu-layers 99 \
    --chat-template chatml

⚙️ Recommended Generation & Agent Settings

To achieve the best balance of reasoning and deterministic tool formatting in agent harnesses:

Parameter	Recommended Value	Description
`temperature`	`0.2` – `0.35`	Lower values ensure line numbers in diffs and bash commands remain deterministic.
`top_p`	`0.95`	Nucleus sampling cutoff.
`top_k`	`40`	Limits tail token sampling.
`repetition_penalty`	`1.05`	Prevents infinite retry loops on bash errors while preserving standard Python/JS indentation.
`max_tokens`	`8192`	Ample headroom for complex multi-file patches.

Compatible Agent Harnesses

Claude Code: Configure via OpenAI-compatible endpoint with --model-name OliviaRossi/KAT-Ornith-Coder-35B-A3B.
Aider: Run with --model openai/OliviaRossi/KAT-Ornith-Coder-35B-A3B --edit-format diff.
OpenHands / Cline / Roo Code: Select the standard XML or OpenAI tool call parser with system prompt anchoring.

📋 Citation & Attribution

This model is a merge of two foundational research projects. If you use this model in your research or applications, please cite the parent teams:

@misc{kat_coder_2025,
  title={KAT-Coder-V2.5: Real-World Software Engineering via Multi-Teacher Distillation and Sandbox RL},
  author={KwaiPilot AI4SE Team},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev}}
}

@misc{ornith_coder_2025,
  title={Ornith-1.5-35B-A3B: Autonomous Self-Improving Reinforcement Learning for Code Intelligence},
  author={DeepReinforce / Ornith AI},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B}}
}

@misc{kat_ornith_coder_2026,
  title={KAT-Ornith-Coder-35B-A3B: Asymmetric DARE-TIES Synthesis for Agentic Coding},
  author={Olivia Rossi},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/OliviaRossi/KAT-Ornith-Coder-35B-A3B}}
}

Developed with ❤️ for the open-weights agentic coding ecosystem.

Downloads last month: 178

Safetensors

Model size

35B params

Tensor type

BF16

Model tree for OliviaRossi/KAT-Ornith-Coder-35B-A3B

Kwaipilot/KAT-Coder-V2.5-Dev

ornith-ai/Ornith-1.5-35B-A3B

Merge model

this model

Quantizations

2 models