Instructions to use dev-sandakelum/fable-mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dev-sandakelum/fable-mini with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="dev-sandakelum/fable-mini", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("dev-sandakelum/fable-mini", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use dev-sandakelum/fable-mini with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dev-sandakelum/fable-mini" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dev-sandakelum/fable-mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/dev-sandakelum/fable-mini
- SGLang
How to use dev-sandakelum/fable-mini with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "dev-sandakelum/fable-mini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dev-sandakelum/fable-mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "dev-sandakelum/fable-mini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dev-sandakelum/fable-mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use dev-sandakelum/fable-mini with Docker Model Runner:
docker model run hf.co/dev-sandakelum/fable-mini
fable-mini
A 3,285,504-parameter decoder-only transformer, trained from scratch as an educational miniature of the "Fable" assistant style (see the companion GitHub repo gghsff07-sudo/my_claude for the full study, design, training and evaluation write-ups).
This is a small educational model, not a real assistant, and not equivalent to Fable 5.2 or any production LLM. It was built to demonstrate a complete, honest, from-scratch pipeline on a CPU budget. It did not learn to answer questions or produce coherent sentences (see "Evaluation" below).
Model description
- Architecture: pre-LN decoder-only transformer β 4 layers, hidden size 256, 4 attention heads (head dim 64), FFN 768, GELU, learned absolute positions, bias-free fused QKV, tied input/output embeddings. Context length 512.
- Tokenizer: byte-level BPE (vocab 2,048 = 256 bytes + 1789 learned merges + 3 specials), trained from scratch on the training split only.
- Weights: run2 final checkpoint (step 749), fp32 safetensors. The tied LM head
weight is stored once (
tok_emb.weight) and re-tied on load. - Generation:
model.generate()supportstemperature,top_p,top_k,max_new_tokens,repetition_penaltywith a KV cache.
Special tokens: <|pad|>=256, <|eos|>=257, <|doc|>=258 (document separator,
doubling as BOS). No <|unk|> β byte-level coverage means every byte has an id.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("<user>/fable-mini", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("<user>/fable-mini", trust_remote_code=True)
messages = [
{"role": "system", "content": "You are a concise, warm assistant."},
{"role": "user", "content": "Hi!"},
]
inputs = tok.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt"
)
out = model.generate(
inputs.input_ids,
max_new_tokens=64,
do_sample=True,
temperature=0.8,
top_p=0.9,
top_k=50,
repetition_penalty=1.2,
pad_token_id=256,
eos_token_id=257,
)
text = tok.decode(out[0][inputs.input_ids.shape[1]:])
print(text)
The model's chat format (what the chat template renders) is plain-text turns:
<system>You are a concise, warm assistant.</system>
User: Hi!
Assistant: ...
Training data
Trained on an 18-document in-repo corpus (90/10 document-level split, seed 42):
- The "Claude Fable 5.1" reference document (~270 KB)
- 5 research/design documents written during the project
- 11 short synthetic Fable-style dialogue files (User:/Assistant: turns)
run2 upweighted the dialogue files 6Γ to reinforce the chat register. The total corpus is a few hundred KB of English text β far too small for general language ability, by design. No web scrape, no external datasets.
Training: AdamW, peak lr 3e-4, 100-step warmup, cosine decay to 10%, gradient clip 1.0, batch 8Γ512 tokens, CPU only, ~750 steps in total across two runs (600 plain + 150 resumed with dialogue upweighting).
Evaluation (honest numbers)
| Split | Loss | Perplexity |
|---|---|---|
| train | 3.6209 | 37.37 |
| val | 4.1600 | 64.07 |
Perplexity is on this corpus only β a corpus-fit number, not a general-language number. Caveats, stated plainly:
- run2's val split shares genre and phrasing patterns with the upweighted training dialogues, so val ppl 64.07 flatters the model somewhat.
- The train/val gap (3.62 vs 4.16) shows substantial memorization.
- Sample generations are register-imitation, not coherent answers. A verbatim example (temperature 0.8): "I's this is story is that't leave as the one-15" and you to beeny and hurborry is usually search β segoy, or I'm the user't render it't being matter how you need to were food and a ca- Clalated β your holarning are mode?"
- The chat scaffold (User:/Assistant: turns,
<|eos|>stops) works mechanically β short turns, clean stops β but the content is word soup.
The full evaluation with sample tables is [EVAL.md in the GitHub repo] (https://github.com/gghsff07-sudo/my_claude/blob/main/EVAL.md).
Intended use
Education and research: studying a complete from-scratch LLM pipeline (tokenizer training, transformer implementation, training loop, evaluation, Hugging Face packaging) at a size that trains in minutes on a laptop CPU. Good for: unit-testing HF integration code, experimenting with decoding strategies, teaching.
Limitations and safety
- Not a chatbot. Output is incoherent word salad imitating the training register. It cannot answer questions, follow instructions, or hold a topic.
- No safety behavior exists. The model has no refusal capability, no calibration, and no alignment training. Do not use it for anything real.
- Not equivalent to Fable 5.2 or any production assistant; the name refers to the style study, not a capability claim.
- English-only training data; 512-token context; 2K vocab.
Acknowledging the source style
The "Fable" reference documents analyzed during this project are Anthropic-style assistant-writing material; this model is an independent educational miniature with no connection to, or endorsement by, Anthropic.
- Downloads last month
- 29