Instructions to use ProCreations/grug-v1.1-qwen-3.8-27b-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProCreations/grug-v1.1-qwen-3.8-27b-mtp with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ProCreations/grug-v1.1-qwen-3.8-27b-mtp")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProCreations/grug-v1.1-qwen-3.8-27b-mtp")
model = AutoModelForMultimodalLM.from_pretrained("ProCreations/grug-v1.1-qwen-3.8-27b-mtp", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProCreations/grug-v1.1-qwen-3.8-27b-mtp with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProCreations/grug-v1.1-qwen-3.8-27b-mtp"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-v1.1-qwen-3.8-27b-mtp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ProCreations/grug-v1.1-qwen-3.8-27b-mtp

SGLang

How to use ProCreations/grug-v1.1-qwen-3.8-27b-mtp with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProCreations/grug-v1.1-qwen-3.8-27b-mtp" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-v1.1-qwen-3.8-27b-mtp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProCreations/grug-v1.1-qwen-3.8-27b-mtp" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-v1.1-qwen-3.8-27b-mtp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ProCreations/grug-v1.1-qwen-3.8-27b-mtp with Docker Model Runner:
```
docker model run hf.co/ProCreations/grug-v1.1-qwen-3.8-27b-mtp
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

grug-v1.1-qwen-3.8-27b-mtp

same grug as ProCreations/grug-v1.1-qwen-3.8-27b, plus a draft head that guess ahead, tuned on grug own output.

Qwen3.8 ship native MTP (multi-token prediction) head. it predict token t+2 so engine can guess ahead and verify many token in one pass. speculative decoding only go fast when draft head agree with real model. grug retune that head on grug distribution.

what change

metric	Qwen3.8 native head	after grug tuning
t+2 agreement with verifier	90.04%	95.37%
gold top-1	85.50%	88.20%
eval loss	0.3802	0.1649

disagree rate go from 1-in-10 to 1-in-21. old grug v1 mtp sit at 89.3%.

honest part: native head was already 90.0% before any training. grug not rescue broken head — grug sharpen already-good head. and gold top-1 go up too (85.5 -> 88.2), so head get genuinely better at real token, not just better at copying verifier mistake.

also honest: 95.37% is top-1 agreement with the verifier on grug spans — the quantity that decide whether a speculative draft get accepted. it is NOT a measured wall-clock speedup. real tok/s depend on your engine, batch size and draft length.

how grug train it

backbone frozen. only the 425M-param mtp.* module move.
teacher = grug-v1.1-qwen-3.8-27b own greedy next-token choice (self-distillation). this target acceptance directly: acceptance IS agreement with verifier.
data = ProCreations/grug-27b-v2-corrective — agent trajectory, tool call, grug think, code, math. the distribution grug actually make in real use.
4.0M token seen, 1.25M supervised, 489 step, 26 minute on one RTX PRO 6000.
verified model output unchanged — same weight, same score as v1.1. draft head only make it faster, never change answer.

full receipt in mtp_training.json.

use grug

engine that know Qwen3.8 MTP (vLLM, sglang) pick head up from config.json (mtp_num_hidden_layers: 1) and model-mtp.safetensors.

vllm serve ProCreations/grug-v1.1-qwen-3.8-27b-mtp \
  --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}' \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

plain transformers ignore mtp.* and just run the normal model — safe, only no speedup.

GGUF with draft head: ProCreations/grug-v1.1-qwen-3.8-27b-mtp-gguf. no draft head, smaller: ProCreations/grug-v1.1-qwen-3.8-27b-gguf.

benchmark score, effort table and chart live on the main card — this variant score identical, it is the same verified weight.

apache-2.0.