Instructions to use GautierP/MiniLLM-70M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GautierP/MiniLLM-70M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="GautierP/MiniLLM-70M", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("GautierP/MiniLLM-70M", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use GautierP/MiniLLM-70M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GautierP/MiniLLM-70M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GautierP/MiniLLM-70M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/GautierP/MiniLLM-70M

SGLang

How to use GautierP/MiniLLM-70M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "GautierP/MiniLLM-70M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GautierP/MiniLLM-70M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "GautierP/MiniLLM-70M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GautierP/MiniLLM-70M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use GautierP/MiniLLM-70M with Docker Model Runner:
```
docker model run hf.co/GautierP/MiniLLM-70M
```

Mini-LLM Assistant 70M

A ~70M parameter conversational assistant trained entirely from scratch with PyTorch — no pretrained weights, no distillation. Tokenizer, pretraining and supervised fine-tuning all belong to the Mini-LLM project.

This repository hosts both the SFT assistant (repository root) and the base model it was fine-tuned from (base/).

Model details


Parameters	69,975,680 (input embedding shared with the output head)
Architecture	Llama-style decoder: RMSNorm, RoPE, causal attention, SwiGLU
Context length	512 tokens
Vocabulary	16,384 (BPE)
Pretraining	~1.44B tokens (FineWeb-Edu + Wikipedia), 22,000 steps — base val loss 3.030
Fine-tuning	2 epochs of SFT on Dolly-15k, No Robots, OpenAssistant oasst2, alpaca-cleaned and identity/small-talk examples — val loss 2.718

Base model

The base/ subfolder contains the pretrained checkpoint (mini-llm-base-70M-22000steps.pt) before SFT: 69,975,680 parameters, 16,384-token BPE vocabulary, ~1.44B tokens (FineWeb-Edu + Wikipedia), 22,000 steps — base val loss 3.030. It is a raw completion model, not an assistant: base/ ships without a chat template and without the assistant's sampling defaults.

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "GautierP/MiniLLM-70M"
tok = AutoTokenizer.from_pretrained(repo, subfolder="base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, subfolder="base", trust_remote_code=True)

inputs = tok("The capital of France is", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=64, do_sample=True, temperature=0.8, top_k=50)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "GautierP/MiniLLM-70M"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True)

messages = [{"role": "user", "content": "What is the capital of France?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=128)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

trust_remote_code=True is required — the architecture ships with the repository (modeling_mini_llm.py, configuration_mini_llm.py). Sampling defaults (temperature 0.5, top_k 40, repetition penalty 1.15) live in generation_config.json.

The training format is single-turn and fixed; chat_template.jinja implements it exactly:

User: <question>
Assistant: <answer>

Multi-turn history is never sent to the model: it anchors on its own previous answers and degrades when it sees them, so each question is answered independently.

Limitations

Very small model trained on a modest budget: answers are often fluent and in the right format, but factual errors and hallucinations are common. Do not use it for factual lookup or any production task. English only.

Collection including GautierP/MiniLLM-70M

Mini-LLM

Collection

2 items • Updated 3 days ago

GautierP
/

MiniLLM-70M