Instructions to use regnant-io/kw5-109M-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use regnant-io/kw5-109M-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="regnant-io/kw5-109M-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("regnant-io/kw5-109M-instruct")
model = AutoModelForCausalLM.from_pretrained("regnant-io/kw5-109M-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use regnant-io/kw5-109M-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "regnant-io/kw5-109M-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "regnant-io/kw5-109M-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/regnant-io/kw5-109M-instruct

SGLang

How to use regnant-io/kw5-109M-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "regnant-io/kw5-109M-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "regnant-io/kw5-109M-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "regnant-io/kw5-109M-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "regnant-io/kw5-109M-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use regnant-io/kw5-109M-instruct with Docker Model Runner:
```
docker model run hf.co/regnant-io/kw5-109M-instruct
```

KW5-Lite Instruct

A 109.5M-parameter Swahili instruction-following model, LoRA fine-tuned from regnant-io/kw5-v1-base.

Built by Regnant. Training code: kw5-lite and another 88K examples.

If you used a previous revision of this repo, please re-pull. The chat template shipped before this revision inserted a second <s>[INST] when a system message was present and appended a trailing space at generation time. On the same weights that cost roughly half the model's quality — see What changed.

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("regnant-io/kw5-v1-it")
tok = AutoTokenizer.from_pretrained("regnant-io/kw5-v1-it")

messages = [{"role": "user", "content": "Orodhesha majina ya miji mitano ya Tanzania."}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# add_special_tokens=False: the template already emits <s>.
inputs = tok(prompt, return_tensors="pt", add_special_tokens=False)
out = model.generate(**inputs, max_new_tokens=256)   # generation_config carries good defaults
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

1. Dar es Salaam
2. Dodoma
3. Arusha
4. Tanga
5. Zanzibar

Decoding settings matter a lot at this size

This is a 110M model. It is coherent at low temperature with a repetition penalty and degenerates at the transformers defaults. The shipped generation_config.json already sets:

Parameter	Value
`do_sample`	`true`
`temperature`	`0.2`
`top_p`	`0.9`
`repetition_penalty`	`1.3`

temperature 0.1–0.2 with repetition_penalty 1.3–1.4 is the usable band. Above ~0.7, or with no repetition penalty, output quality falls off sharply.

Prompt format

<s>[INST] {user} [/INST] {assistant}</s>[INST] {user2} [/INST] {assistant2}</s>

with an optional system block inside the first [INST] only:

<s>[INST] <<SYS>>
{system}
<</SYS>>

{user} [/INST]

apply_chat_template produces exactly this. Two rules if you build prompts by hand:

Tokenize with add_special_tokens=False. The template emits <s> itself; letting the tokenizer add another gives a doubled BOS the model never saw. add_bos_token is pinned to false in tokenizer_config.json for this reason.
Do not use <|system|> / <|user|> / <|assistant|> (token ids 4–6). They were reserved when the tokenizer was trained but never appear in the pretraining corpus, and LoRA left embed_tokens frozen — so those embedding rows are still at initialisation. Prompting with them returns noise:
```
<|user|>\nOrodhesha majina ya miji...   ->  'KANIWA AMUAAAAAAAAAAAAAHAAAAAAAAATI...'
<s>[INST] Orodhesha majina ya miji... [/INST]  ->  '1. Dar es Salaam\n2. Dodoma\n...'
```

Model details


Parameters	109.5M (tied input/output embeddings)
Architecture	Llama-compatible decoder-only transformer
Layers / hidden / FFN	12 / 768 / 2048
Attention heads	12 query, 12 key-value (standard MHA — not GQA)
Normalization / activation	RMSNorm (pre-norm) / SwiGLU
Position encoding	RoPE, theta 10000
Context length	2048
Vocabulary	32,000 SentencePiece BPE, NFC, byte-fallback
Precision	FP16
File size	219 MB

Fine-tuning


Base checkpoint	`kw5-v1-base` step 6,150 — 1.41B tokens, exactly 2 epochs over the corpus
Method	LoRA, merged into the base weights
Rank / alpha	16 / 32
Adapted modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Frozen	`embed_tokens` / `lm_head` (tied), all norms
Training data	~107,000 Swahili instruction and conversation examples
Framework	Unsloth + Transformers
Hardware	1× NVIDIA T4 (16 GB)

The rank and adapted-module list above are recovered directly from the published weights: the singular-value spectrum of (merged − base) cuts off sharply after 16 for every adapted projection, and the non-adapted tensors match the base checkpoint bit-for-bit.

Training data composition

~107,000 examples spanning instruction-following, multi-turn dialogue, Tanzanian factual and cultural content (methali, mila, utamaduni), Swahili grammar (sarufi), Sheng code-switching, safety and refusal patterns, and identity responses. Loss was masked to assistant turns only — prompts do not contribute to the objective.

Evaluation

Generation-based instruction-following, 16 prompts across five categories, graded by keyword and structure checks (scripts/eval_sft.py in the training repo), at temperature 0.2 / repetition penalty 1.3:

Category	Score
List generation	4 / 4
Explanation	3 / 3
Instruction following	3 / 3
Conversation	2 / 3
Factual question answering	0 / 3
Total	12 / 16 (75%)

This is a small, non-standard suite; treat it as a smoke test, not a benchmark.

Read the failure column. Factual recall is the clear weak point — the model does not reliably know that the capital of Tanzania is Dodoma, or that Lake Victoria is Africa's largest lake. At 110M parameters trained on 1.41B tokens, that is expected. Use this model for fluent Swahili generation, formatting and conversational structure; do not use it as a knowledge source.

What changed in this revision

Two things, both of which affected everyone using this model:

1. The chat template was wrong. Measured on identical weights and prompts:

Prompt path	Score
Previous `chat_template.jinja`	6 / 16 (37.5%)
Previous template with a system message	3 / 16 (18.8%)
Corrected template (this revision)	10 / 16 (62.5%)

The old template emitted <s>[INST] before every user turn rather than only the first, so a system message produced <s>[INST] <<SYS>>…<</SYS>>\n\n<s>[INST] {user} [/INST] — a second BOS in the middle of the prompt — and multi-turn produced </s><s>[INST] where training used </s>[INST]. add_generation_prompt also appended a bare space, which changes how the first generated word tokenizes.

2. The weights are now the model that actually matches this repo's stated base. The previous revision was fine-tuned from an unpublished base checkpoint (step 8,176) that is past the point where pretraining began overfitting, on ~22,500 examples. This revision is fine-tuned from the published kw5-v1-base (step 6,150, exactly 2 epochs) on ~107,000 examples. Non-LoRA tensors now match kw5-v1-base exactly, so the two repos are finally consistent.

Also in this revision: embeddings re-tied (536 MB → 219 MB, numerically identical — lm_head was a byte-for-byte duplicate of embed_tokens), pad_token_id corrected to 3, and useful generation defaults shipped.

Limitations

Factual reliability is poor. See the evaluation section. Do not use for factual lookup, medical, legal or financial advice.
Count adherence is weak. Asked for five items it will sometimes produce ten.
Primarily Tanzanian Swahili. Coverage of Kenyan, Ugandan and Congolese varieties is thinner.
2048-token context, no tool use, no code, English capability is incidental and untested.
Safety tuning is minimal — a few hundred refusal examples. Not suitable for unsupervised deployment to end users.
Trained on web-scraped text (FineWeb-2) and so carries its biases.

Citation

@misc{kw5lite2026,
  title  = {KW5-Lite: a 110M-parameter Swahili language model trained on a single T4},
  author = {Regnant},
  year   = {2026},
  url    = {https://huggingface.co/regnant-io/kw5-v1-it}
}

Apache 2.0.

Downloads last month: 3,986

Safetensors

Model size

0.1B params

Tensor type

F16

Model tree for regnant-io/kw5-109M-instruct

Base model

regnant-io/kw5-109M

Adapter

(1)

this model