grug-v2-9b

grug bird learn final club. v1 know code and usually choose right tool, but hand sometimes make invalid XML or forget terminal submit. verifier RL reward valid shape, exact required argument, right tool, closed think. frozen v1 hold rope so bird not wander.

this full merged 9b model. no adapter needed.

number. grug measure, not wish

Same greedy harness, prompts, parser, runtime, and limits for every row. HumanEval has 164 tasks. MBPP uses first 100 sanitized test tasks. Card tool suite has 18 held-out late-trajectory decisions. Broad tool suite has 119 held-out next-action decisions.

model	HumanEval pass@1	MBPP pass@1	card valid	card strict	card right tool	broad valid	broad strict	broad right tool
Ornith 1.0 9B	87.8	75.0	61.1	55.6	55.6	100.0	92.4	60.5
Grug v1 9B	79.3	77.0	83.3	83.3	83.3	99.2	79.8	91.6
Grug v2 9B	80.5	76.0	100.0	100.0	100.0	100.0	99.2	92.4

Bold Grug values mean best Grug result, including ties. Ornith still win raw HumanEval. Grug v2 beat Grug v1 on HumanEval and every measured tool column; MBPP lose one point. Overall agentic bird stronger, but grug show pebble too.

valid = parser find one offered tool call. strict = call also satisfies exact schema and required arguments. right tool = expected next action, not merely some valid tool.

Exact JSON lives in results/.

release audit

grug also test tiny benchmark-clean SFT using only official MBPP sanitized train split. Held-out test rows never enter gradient. Rank-8 mixed replay, rank-4 code-only replay, early checkpoints, and adapter interpolation all test. Every checkpoint reaching MBPP 77 lose one HumanEval task or broad tool choice. Checkpoint preserving HumanEval/tool gates stay MBPP 76. Grug reject sideways rock. Published weights remain strongest joint checkpoint above.

grug brain format stay

Reasoning stays inside <think>...</think>. Dense fragments, real technical anchors, little grammar meat. Grug goal not shortest thought at any cost. Need keep path, symbol, error, competing guess, risk, next action, verification when problem hard.

Native XML tool call:

<tool_call>
<function=bash>
<parameter=command>
python -m pytest -q
</parameter>
</function>
</tool_call>

how grug teach

• start full ProCreations/grug-9b
• LoRA r16 across text-stack linear layers; vision tower untouched
• group-relative verifier policy gradient with sampled action groups
• reward valid XML, offered tool, correct tool, strict parameters, closed think
• KL rope to frozen v1
• 1,081 training tool contexts; 119 broad validation contexts; 18 card contexts
• validation and card prompts never used for gradient

use

from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch

name = "ProCreations/grug-v2-9b"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForImageTextToText.from_pretrained(
    name, dtype=torch.bfloat16, device_map="auto")

Need Transformers with Qwen3.5 support. Popular GGUF rocks live at ProCreations/grug-v2-9b-gguf.