# 03 — Benchmarks

Reported at **effort=0.99** (~2026-07-14). Harnesses differ; treat as directional. Primary tables live in `reference/hub/MODEL_CARD.md`.

---

## How to read Inkling’s scorecard

| Strong among open | Weaker / mid |
|---|---|
| Agentic coding / tools | HLE pure knowledge vs closed |
| Instruction following (IFBench) | SimpleQA / parametric fact recall |
| Omni vision+audio | Video OOB (path exists, light eval) |
| FORTRESS adversarial (open set they show) | Still needs app-layer guards |

**Product framing:** broad customizable base — not “wins every leaderboard.”

---

## Snapshot (Inkling)

### Reasoning
| | |
|---|---|
| HLE text | 29.7% |
| HLE + tools | 46.0% |
| AIME 2026 | 97.1% |
| GPQA Diamond | 87.2% |

### Agentic
| | | |
|---|---|---|
| SWE-bench Verified | 77.6% | bash-only harness (Inkling) |
| SWE-bench Pro | 54.3% | |
| Terminal Bench 2.1 | 63.8 | contaminated search → 0 |
| MCP Atlas | 74.1% | |
| Tau 3 Banking | 23.7% | |

### Factuality / chat
| | |
|---|---|
| BrowseComp | 77.1% |
| SimpleQA Verified | 43.9% |
| IFBench | 79.8% |
| Global-MMLU-Lite | 88.7% |

### Multimodal
| | |
|---|---|
| MMMU Pro | ~73.5% |
| CharXiv RQ | 78.1% |
| CharXiv + python | 82.0% |
| Audio MC | 56.6% |
| MMAU | 77.2% |
| VoiceBench | 91.4% |

### Safety
| | |
|---|---|
| FORTRESS adv. | 78.0% |
| FORTRESS benign | 95.9% |
| StrongREJECT | 98.6% |

---

## Vibe patterns (HF blog)

- **Vision:** OCR/describe → structure → answer; medium effort (~0.7) often best token trade  
- **Audio:** transcribe first; effort helps *reasoning* more than *hearing*  
- Prompt quality dominates token spend  

---

## Practical takeaway

Use Inkling when you need **open multimodal + agentic base + fine-tune path**.  
Don’t expect it to replace the strongest closed models on pure knowledge / hardest HLE without tools and domain adaptation.

Next: [04-inference.md](04-inference.md)