Instructions to use Bayhakl1/MiniGrimm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Bayhakl1/MiniGrimm with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Bayhakl1/MiniGrimm")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Bayhakl1/MiniGrimm") model = AutoModelForCausalLM.from_pretrained("Bayhakl1/MiniGrimm", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Bayhakl1/MiniGrimm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Bayhakl1/MiniGrimm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/MiniGrimm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Bayhakl1/MiniGrimm
- SGLang
How to use Bayhakl1/MiniGrimm with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Bayhakl1/MiniGrimm" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/MiniGrimm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Bayhakl1/MiniGrimm" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Bayhakl1/MiniGrimm", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Bayhakl1/MiniGrimm with Docker Model Runner:
docker model run hf.co/Bayhakl1/MiniGrimm
MiniGrimm (48M SFT)
MiniGrimm is a tiny 48.29M-parameter (42.00M non-embedding) decoder-only Transformer trained from scratch and fine-tuned with Supervised Fine-Tuning (SFT).
This is a small-scale experimental model built to test a deep-and-thin Llama-style architecture trained with the Muon optimizer. Because it is under 50M parameters and has a 512-token context window, keep your expectations realistic—it is not a general-purpose assistant and will hallucinate facts or stumble on complex logic. That said, it can hold short casual chats and do basic step-by-step arithmetic surprisingly well for its size.
Benchmarks
Evaluated after SFT on ArithMark 3.0 (arithmetic reasoning):
| Benchmark | Metric | Score |
|---|---|---|
| ArithMark 3.0 | Raw Accuracy | 38.1% |
| ArithMark 3.0 | Normalized Accuracy | 38.2% |
(Not bad for a 42M non-embedding model, though it will still make plenty of mistakes on multi-step word problems or larger numbers.)
Other Benchmarks:
| Benchmark | Accuracy (Raw) | Accuracy (Norm) | Notes |
|---|---|---|---|
| SciQ | 70.7% (±1.44) | 63.0% (±1.53) | Science QA |
| PIQA | 61.7% (±1.13) | 62.0% (±1.13) | Physical commonsense |
| Winogrande | 52.4% (±1.40) | — | Pronoun resolution |
| BoolQ | 48.6% (±0.87) | — | Yes/No reading comprehension |
| ARC-Easy | 48.6% (±1.03) | 43.8% (±1.02) | Grade-school science |
| HellaSwag | 29.1% (±0.45) | 31.7% (±0.46) | Sentence completion |
| OpenBookQA | 19.2% (±1.76) | 30.0% (±2.05) | Elementary science facts |
| ARC-Challenge | 22.3% (±1.22) | 27.2% (±1.30) | Harder science reasoning |
| LAMBADA (OpenAI) | 26.9% (±0.62) | — | Perplexity: 83.15 (±3.71) |
Architecture & Training Summary
Instead of a wide, shallow network, MiniGrimm uses a tall, narrow layout (28 layers at d_model=384) paired with modern stability tweaks:
| Spec | Detail |
|---|---|
| Parameters | 48.29M total (~42.00M non-embedding, tied embeddings) |
| Layers / Hidden Dim | 28 layers, d_model = 384, ffn_hidden = 960 (SwiGLU) |
| Attention | Grouped-Query Attention (6 query heads, 2 KV heads) + QK-Norm |
| Positional Encoding | RoPE (theta = 10000.0) |
| Logit Soft-Capping | 30.0 |
| Context Window | 512 tokens |
| Tokenizer | Custom 16,384-vocab Byte-Level BPE (with single-digit number splitting) |
| Pretraining | ~12.55B tokens (FineWeb-Edu, Cosmopedia-v2, Wikipedia, CoT/math, chat) using Muon + AdamW |
| SFT | Assistant-only loss masking across GSM8K, SmolTalk, UltraChat-200k, Tülu-3, and No Robots |
Usage
Because MiniGrimm uses custom modeling code (for QK-Norm and logit soft-capping), you need to pass trust_remote_code=True when loading the model.
Quickstart with transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "Bayhakl1/MiniGrimm"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
trust_remote_code=True,
).to(device)
model.eval()
# Format the prompt using MiniGrimm's special chat tokens
prompt = (
"<|system|>You are a helpful assistant.<|end|>"
"<|user|>If I have 3 apples and buy 5 more, then eat 2, how many apples do I have?<|end|>"
"<|assistant|>"
)
inputs = tokenizer(prompt, return_tensors="pt").to(device)
# Stop when the model emits either <|end|> or <|endoftext|>
stop_token_ids = [
tokenizer.convert_tokens_to_ids("<|end|>"),
tokenizer.eos_token_id,
]
with torch.no_grad():
output_ids = model.generate(
**inputs,
max_new_tokens=150,
do_sample=True,
temperature=0.6,
top_p=0.9,
eos_token_id=stop_token_ids,
pad_token_id=tokenizer.pad_token_id,
)
# Slice off the prompt tokens so only the assistant's reply is printed
reply_ids = output_ids[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(reply_ids, skip_special_tokens=True))
Important Usage Notes
- Chat Template Format: Always wrap turns using the exact single-token delimiters
<|system|>,<|user|>,<|assistant|>, and<|end|>:
(The<|system|>Optional system prompt.<|end|><|user|>First message<|end|><|assistant|>First reply<|end|><|user|>Follow-up<|end|><|assistant|><|system|>...<|end|>block is optional—you can start directly with<|user|>.) - 512-Token Context Limit: The model was trained with a maximum sequence length of 512 tokens (prompt + generated tokens combined). Keep prompts short and
max_new_tokensaround100–250so the total length stays within 512 tokens. - Greedy vs. Sampling: For math questions,
do_sample=False(greedy decoding) usually gives cleaner step-by-step arithmetic. For casual chat,do_sample=Truewithtemperature=0.6–0.7helps avoid repetitive loops.
Limitations
- Small Model Scale (~48M): Limited factual recall—it will confidently make up dates, names, or trivia.
- Short Memory: Anything past 512 tokens is outside its training window.
- English Only: Trained almost exclusively on English text and basic math.
- Downloads last month
- 215