Instructions to use armand0e/Granite-4.2-30B-Fable-Distill with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use armand0e/Granite-4.2-30B-Fable-Distill with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="armand0e/Granite-4.2-30B-Fable-Distill")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("armand0e/Granite-4.2-30B-Fable-Distill")
model = AutoModelForCausalLM.from_pretrained("armand0e/Granite-4.2-30B-Fable-Distill", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use armand0e/Granite-4.2-30B-Fable-Distill with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "armand0e/Granite-4.2-30B-Fable-Distill"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "armand0e/Granite-4.2-30B-Fable-Distill",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/armand0e/Granite-4.2-30B-Fable-Distill

SGLang

How to use armand0e/Granite-4.2-30B-Fable-Distill with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "armand0e/Granite-4.2-30B-Fable-Distill" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "armand0e/Granite-4.2-30B-Fable-Distill",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "armand0e/Granite-4.2-30B-Fable-Distill" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "armand0e/Granite-4.2-30B-Fable-Distill",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use armand0e/Granite-4.2-30B-Fable-Distill with Docker Model Runner:
```
docker model run hf.co/armand0e/Granite-4.2-30B-Fable-Distill
```

Granite-4.2-30B-Fable-Distill

ibm-granite/granite-4.2-30b fine-tuned on agentic coding traces and chat distilled from Claude Fable 5, with a multilingual slice. Merged bf16 weights (~55 GB), drop-in for the base model: same architecture, same tokenizer, same chat template. It is primarily a tool-use tune — three-quarters of the supervised tokens are tool calls — and it trains Granite's native <think> reasoning.

The LoRA adapter this was merged from is published separately as armand0e/Granite-4.2-30B-Fable-Distill-LoRA.

Serving with vLLM

Granite 4.2 uses <think>…</think> reasoning and an XML tool-call format, and vLLM (≤ 0.28) has no parser named for it. Two existing parsers match its wire format exactly:

reasoning: deepseek_r1 — Granite's chat template prefills the opening <think> and the model only emits </think>; this parser is built for that shape (the granite parser targets Granite 3.x's prose markers and does not work).
tools: qwen3_coder — Granite renders <tool_call><function=NAME><parameter=X>…</parameter></function></tool_call>, the same format as the Qwen3-Coder parser (the granite4 parser expects a different one).

vllm serve armand0e/Granite-4.2-30B-Fable-Distill \
  --served-model-name granite-fable \
  --reasoning-parser deepseek_r1 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --default-chat-template-kwargs '{"enable_thinking": true}' \
  --max-model-len 40960 --dtype bfloat16

On a single 64–80 GB GPU the bf16 weights leave little room for KV cache; add --quantization bitsandbytes for in-flight 4-bit, or quantize to AWQ/GPTQ INT4 for the best decode throughput.

Keep thinking enabled. With thinking off, Granite's template prefills an empty <think></think> and the model emits no tag at all, so deepseek_r1 — which treats everything before a </think> as reasoning — returns the whole reply in the reasoning field with empty content, and tool calls are not parsed. Send chat_template_kwargs: {"enable_thinking": true} (or set it as the server default as above). enable_thinking is the template's kwarg name; low_effort: true is also available for shorter reasoning.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("armand0e/Granite-4.2-30B-Fable-Distill")
model = AutoModelForCausalLM.from_pretrained("armand0e/Granite-4.2-30B-Fable-Distill", torch_dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "Explain what a race condition is in two sentences."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0, ids.shape[1]:]))

Training data

773 examples / 8.78M tokens after rendering at a 40,960-token context (trim_followups; median 1.5k tokens, p90 35.6k). Prepared with teich ≥ 0.3.6 (Granite 4.2 support), reasoning kept and supervised, per-row auto mode: 652 non-thinking / 121 thinking rows.

source	description
Fable chat (multilingual)	353 English + 106 translated chats (es/fr/de/pt/ja/zh)
`armand0e/claude-fable-5-claude-code`	Claude Code agent traces
`yellowbeeblackbee/claude-traces`	Claude Code agent traces
`TeichAI/Fable-5-Cursor-Traces`	244 Cursor agent traces (English)
personal-fable / glint-fable	raw Claude Code sessions, plus ~30% with user/assistant text translated

Supervised text by span: tool_call 73%, final_answer 20%, reasoning 8%. Language mix by tokens: English 90%, es/fr/de/pt/ja/zh ~10% combined.

Training

QLoRA (NF4 base), LoRA r=32 / α=32 / dropout 0 on q,k,v,o,gate,up,down_proj — 281M trainable parameters, merged back into the bf16 base weights. 40,960-token context, LR 8e-5 linear, warmup 5, 2 epochs (194 steps), batch 1 × grad-accum 8, paged_adamw_8bit, max_grad_norm 0.3. Single 64 GB GPU, ~24 h. Base-model loss on this data was 0.90 at step 1; per-step loss settled in the 0.69–0.75 band through epoch 1 and averaged 0.75 (median 0.73) across epoch 2.

Trained with Unsloth + TRL; masking and span audit by teich. Note for anyone reproducing on teich ≤ 0.3.7: Granite supervised spans could extend past their own assistant turn into the following user turn (24 real cases in this data); the training script clamps spans at a genuine turn boundary (<|im_end|> followed by <|im_start|>), while leaving quoted <|im_start|> strings inside code untouched.

Limitations

Not recommended for coding. Despite the agentic/tool-use training data, this model — like the base Granite 4.2 30B — is weak at writing and editing code. Use it for tool orchestration, extraction, chat and multilingual tasks; reach for a coding-focused model (e.g. a Qwen3 coder variant) when the task is primarily code.

The non-English share is ~10% of tokens; this is a lighter multilingual signal than the earlier Gemma-based Fable tunes. Supervising reasoning was chosen deliberately for a thinking-native model, but it has not been benchmarked against an answers-only variant of the same run. The adapter was trained against NF4-quantized base weights and merged into bf16; this is standard QLoRA practice but means the merged model is not bit-identical to what training saw.

Downloads last month: -

Safetensors

Model size

29B params

Tensor type

BF16

Model tree for armand0e/Granite-4.2-30B-Fable-Distill

Base model

ibm-granite/granite-4.1-30b-base

Finetuned

ibm-granite/granite-4.2-30b

Finetuned

(3)

this model