Instructions to use ProCreations/grug-27b-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProCreations/grug-27b-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ProCreations/grug-27b-v2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProCreations/grug-27b-v2")
model = AutoModelForMultimodalLM.from_pretrained("ProCreations/grug-27b-v2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProCreations/grug-27b-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProCreations/grug-27b-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ProCreations/grug-27b-v2

SGLang

How to use ProCreations/grug-27b-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProCreations/grug-27b-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProCreations/grug-27b-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ProCreations/grug-27b-v2 with Docker Model Runner:
```
docker model run hf.co/ProCreations/grug-27b-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Grug 27B v2

Grug 27B v2 is a fresh post-training run from Qwen 3.8 27B for compact caveman-style reasoning, coding, and tools. It includes a Grug-tuned native MTP head in this checkpoint. No separate draft model is needed. GGUF downloads also include the head in every text quantization.

The release improves measured coding and tool results over Grug v1.1 and fixes its session-title regression on the tested suite. It does not establish that higher effort always improves accuracy, that reasoning stays perfectly Grug-style on every hard case, or that all loops are eliminated. Read the effort tables and remaining limitations below.

Measured quality

Matched H200/vLLM runs use temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05, seed 42, and the same task subsets and output caps. These are executable-test and official AST-checker measurements on fixed subsets, not full leaderboard scores.

Model	Medium HumanEval (132)	Medium MBPP (225)	Medium BFCL subset (200)
Qwen 3.8 27B	99.2%	93.3%	89.0%
Grug v1.1	91.7%	80.4%	81.0%
Grug v2	97.0%	90.7%	88.0%

On the 12-issue SWE-bench Verified Django/SymPy subset, medium effort with the original limits resolved 10/12 for v2 and 8/12 for v1.1. In the separate matched xhigh run with retained reasoning, 65,536-token context, and 16,384 output tokens per tool turn, v2 resolved 11/12, v1.1 7/12, and Qwen 10/12. The harness uses a shared pinned environment, not the official per-instance Docker images. A single case changes the score by 8.3 points; these small runs do not establish broad statistical superiority.

V2 completed 90/90 session-title checks across low, medium, and xhigh without tool-call leaks. Real vLLM HTTP tests passed 24/24, including titles with tools present, streamed calls, and thinking enabled/disabled. These tests address the reported title problem; the OpenCode desktop application itself was not tested.

With reasoning retained in tool history and the original per-effort limits, v2 resolved 9 / 10 / 11 of 12 at low/medium/xhigh, compared with v1.1's 8 / 9 / 7. This is a positive effort-scaling result on the small agentic subset, not a guarantee across tasks.

Compared with foundation Qwen, compactness still costs some core accuracy. For example, medium MATH on the 150-case subset is 84.7% for v2 versus 95.3% for Qwen. Full comparisons, original and larger output budgets, MTP accuracy, quantized smoke tests, and repetition-penalty-1.0 reference runs are in results.md. The evaluation protocol and machine-readable results describe scoring corrections, completion rates, and limitations.

Reasoning effort and Grug style

Use low, medium, or xhigh through chat_template_kwargs.reasoning_effort. Medium is the default. All three retain the native effort instructions and are trained for terse Grug reasoning; final answers and code can use ordinary language.

Mean HumanEval reasoning lengths were 192 / 266 / 656 tokens for v2 at low/medium/xhigh, versus 483 / 617 / 1214 for Qwen. More effort therefore spends more reasoning on this coding subset while remaining more compact than the foundation model.

Accuracy does not rise monotonically: several coding and math scores decline at xhigh, even when its output limit is doubled. Long traces sometimes drift into ordinary English. A real unfinished loop remains in the low-effort MATH audit, so the reported looping issue is mitigated, not completely solved. The saved template also normalizes historical reasoning once; clients still need the correct reasoning/tool parsers.

Integrated MTP

All 15 native draft-head tensors are indexed in model-mtp.safetensors inside this repository. Only the head was trained during its tuning stage; the main model stayed frozen. Held-out draft/verifier agreement improved from 90.44% to 90.90%, with distillation loss improving from 0.3502 to 0.2965.

On 18 serial H200 prompts after warmup, measured output throughput was 68.02 tokens/sec without MTP, 112.77 with one draft token, and 142.44 with two. Actual draft-token acceptance was 86.87% and 77.20%. These are workload-specific results. MTP-off/on greedy token sequences matched on 10/18 prompts; functional coding/tool accuracy is measured separately in the results.

Enable the packaged head with the tested vLLM 0.29.0 options:

vllm serve ProCreations/grug-27b-v2 \
  --served-model-name grug-v2 --max-model-len 24576 \
  --max-num-seqs 8 --gpu-memory-utilization 0.85 \
  --reasoning-parser qwen3 --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Generation defaults are temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05. Set them explicitly when clients override model defaults. See usage.md for requests, effort controls, history normalization, and llama.cpp commands.

Build and provenance

The main LoRA run used 3,466 training examples, task-group-disjoint validation, and verified coding/math/repair material plus inherited tool replay and new title examples. The best validation checkpoint was merged into the foundation model. The vision encoder is preserved; this release does not establish vision quality or full configured-context performance.

All model generation, training, merging, MTP tuning, quantization, and runtime validation ran on HF Jobs. The local GPU was not used. The conservative estimate for this build, including failed and canceled attempts, is $120.92, below the authorized $170 cap; this is not an invoice. See training.md, release provenance, HF Jobs cost ledger, and SHA256SUMS.

Apache-2.0 license, following the foundation checkpoint. New benchmark test cases were excluded from the training mixture; inherited replay and foundation pretraining may still contain unrecognized overlap. Training receipts and evaluation source are included; raw inherited replay remains private.