Instructions to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF",
	filename="Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Ollama
How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with Ollama:
```
ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
```

Unsloth Studio

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF to start chatting

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with Docker Model Runner:
```
docker model run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
```

Lemonade

How to use AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M

List all available models

lemonade list

🪶 Parable-Granite-3B v2 — trained on genuine Claude Fable 5 agent traces

A tiny local model that thinks before it answers — planning, reasoning, and terminal instincts distilled from real agent sessions.

~3 GB of RAM is all you need. Laptop, old GPU, Raspberry-Pi-class boxes with swap — the Q4 build runs anywhere. One command and you have a private, offline reasoning model on your machine:
ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

📊 The headline — v2 is a different model

v2 is a full retrain: 13× more genuine Fable 5 trace data (11,574 sessions, 16.8M tokens — corpus published) and a rebuilt recipe (completion-masked loss, replay mixing, benchmark-gated checkpoints, seed-averaged weights).

same harness, greedy, Q4_K_M	v1	v2 (this release)
Dev pass-rate (MBPP subset, n=50) — base: 0.68	—	0.82
Agent-artifact leakage (JSON blobs, phantom turns)	6/34	0/34
Strict 34-prompt coding qual — base: 27/34	~18/34	25/34
HumanEval / HumanEval+	62.8 / 57.9	70.1 / 65.9

Clean answers, structured reasoning, agent instincts — and the transcript artifacts that leaked into v1's replies are gone. One trade, made on purpose: raw HumanEval-style function synthesis stays the base model's turf (81.7 vs 70.1) — v2 spends that capacity on agent behavior instead, and spends half as much as v1 did. Measurement notes below. 👇

🚀 Announcements

📌 Same links, new model. v2 replaces v1 in place — every existing Ollama command, script, and bookmark now serves v2. No migration, nothing to change.

🔮 v3 is already training. Rejection-sampled SFT: thousands of candidate solutions generated against executable tests, only verified passers enter the corpus. The goal is simple — above-base agent capability, not just clean behavior. Follow AnkitAI for the drop.

📦 Full family. This 3B is the smallest Parable. Need more headroom? 8B Granite, 8B Qwen, 4B Qwen — same recipe, no matter your hardware.

📦 Pick your size

File	Size	Fits in	Notes
Q4_K_M	2.1 GB	~3 GB RAM/VRAM	⭐ Recommended — best size/quality balance
Q5_K_M	2.4 GB	~3.5 GB	Higher quality
Q6_K	2.8 GB	~4 GB	Near-lossless
Q8_0	3.6 GB	~5 GB	Maximum quality
F16	6.8 GB	~8 GB	Full precision, for re-quantizing

Full-precision safetensors (vLLM, transformers, further fine-tuning): Parable-Granite-4.1-3B-Claude-Fable-5

🚀 How to run it

Ollama (chat template ships inside the GGUF — zero config):

ollama run parable/granite4.1-fable:3b
# or straight from this repo:
ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

llama.cpp:

llama-cli -m Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf --jinja \
  -p "Write a bash one-liner to find the 10 largest files in a directory tree."

LM Studio / Jan / Open WebUI: search "parable" in-app, or paste this repo URL.

Python (llama-cpp-python):

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF",
    filename="*Q4_K_M.gguf", n_ctx=8192,
)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Write a Python function that retries an HTTP request with exponential backoff."}],
    max_tokens=3000, temperature=0.7,
)
print(out["choices"][0]["message"]["content"])

🧠 Thinking mode

Every answer opens with a <think>...</think> reasoning block — that's the Fable 5 heritage. llama.cpp's --jinja mode separates it automatically; strip it before showing replies to end users. Sampling: temperature 0.7, top_p 0.95, and budget max_tokens generously (2500+) — trace-trained models think at length before answering.

🔬 Measurement notes

All numbers: identical llama.cpp harness, greedy decoding, Q4_K_M, base model measured on the same instrument. We train multiple seeds and ship the weight-average — single-run scores at 3B swing ±3 points on GPU nondeterminism alone, so most cards report their luckiest run; we ship the average and report the shipped weights' own numbers. Raw eval outputs live in this repo.

Which model should you use? Pure single-function code completion → the base model is genuinely strong there. Explanations, debugging, terminal workflows, structured reasoning, agent-style tasks → that's what Parable is trained on, and where v2 shines.

📚 What's new in v2 (training)

The recipe follows our ongoing tech report (in preparation):

Completion-only loss masking (Hermes 3, Tülu 3) — loss on assistant tokens only, so the model learns to answer, not to imitate transcripts
30% replay mix of general instruction data (Luo et al., Biderman et al.) — the anti-forgetting lever
Session re-segmentation + sanitization — why v1 sometimes leaked agent JSON into normal chat, and v2 never does (0/34)
Benchmark-gated checkpoints (Dong et al.) instead of fixed epochs
Seed-averaged weights (model soups, Wortsman et al.) — we ship the average of multiple runs, not the lottery winner

With Claude Fable 5 now retired, genuine self-authored Fable traces are a fixed, non-renewable corpus. Unlike most models in this niche, our full training corpus is public: AnkitAI/parable-corpus-v2 — deduplicated, quality-gated, provenance-tagged.

⚠️ Good to know

Fine-tuned at 2,048-token sequences; the base 128K context stays available, fine-tuned behavior is strongest in the opening turns.
Not trained for: multi-file repo navigation, vision, non-English.
Inherits Granite-4.1-3B's knowledge cutoff. Treat generated commands as drafts to review.

📚 Base & license

Weights: Apache-2.0 (inherited from ibm-granite/granite-4.1-3b). Training data: Fable-5-traces AGPL-3.0, gpt5.5-terminal MIT — since traces originate from third-party assistants, their terms may apply to downstream training; check before commercial distillation.

🪶 Get Parable

Platform
Ollama	`ollama run parable/granite4.1-fable:3b` · parable namespace
Hugging Face	full collection
LM Studio	search "parable" in-app
ModelScope	Parable on ModelScope

🙏 Acknowledgements

Glint-Research & Roman1111111 for the open trace data · IBM Granite for the base · empero-ai whose Qwable recipe inspired the series · llama.cpp

🗂 Version history

v2 (2026-07-16) — this release. 13× corpus, rebuilt recipe, seed-averaged weights, zero leakage.
v1 (2026-07) — initial release, 857-row corpus. Preserved as repo revision history.

🪶 Three gigabytes. Real Fable 5 reasoning. Yours, offline, right now.

ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M

Downloads last month: 322

GGUF

Model size

3B params

Architecture

granite

Hardware compatibility

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF

Base model

ibm-granite/granite-4.1-3b

Finetuned

(15)

this model

Datasets used to train AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF

Collection including AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF

Parable

Collection

Open models trained on real Claude Fable 5 and GPT-5.5 agent traces. Eval-gated releases. Also on Ollama: ollama run parable/fable • 9 items • Updated 2 days ago • 1

Papers for AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

Paper • 2308.08747 • Published Jan 5, 2025

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition

Paper • 2310.05492 • Published Oct 9, 2023 • 2