Instructions to use OpenSI-SI/OpenSI-Haiku with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenSI-SI/OpenSI-Haiku with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenSI-SI/OpenSI-Haiku")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenSI-SI/OpenSI-Haiku")
model = AutoModelForCausalLM.from_pretrained("OpenSI-SI/OpenSI-Haiku", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenSI-SI/OpenSI-Haiku with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenSI-SI/OpenSI-Haiku"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenSI-SI/OpenSI-Haiku",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenSI-SI/OpenSI-Haiku

SGLang

How to use OpenSI-SI/OpenSI-Haiku with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenSI-SI/OpenSI-Haiku" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenSI-SI/OpenSI-Haiku",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenSI-SI/OpenSI-Haiku" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenSI-SI/OpenSI-Haiku",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenSI-SI/OpenSI-Haiku with Docker Model Runner:
```
docker model run hf.co/OpenSI-SI/OpenSI-Haiku
```

OpenSI-Haiku

An open conversational model (SFT from the OpenSI 0.3B base).

Qwen2 architecture · 113.9M params · 2048 context · Interactive (low-latency) served profile · MIT License

⚠️ Name note: the project name uses "Haiku" to denote its smallest/fastest real-time variant. Actual parameter count: 113,916,672 (113.9M), not "0.3B". Treat as a ~0.11B chat model.

Quick facts


Parameters	113,916,672 (113.9M)
Layers / hidden	12 / 768
Heads (GQA)	12 query, 4 KV
FFN (SwiGLU)	2048
Vocab	50,000 (ByteLevel BPE)
Context	2048
Embeddings	tied (`lm_head.weight` = `embed_tokens.weight`)
Base ckpt (seed)	`runs/opensi-0.3B-seed/opensi-0.3B.ckpt.pt` (step 9536, train EMA loss 3.0499)
SFT method	Full-parameter SFT (no LoRA), masked CE on assistant tokens only
Training steps	120 (3 epochs)
Best val loss	2.7959 (ppl ~16.4)
Tokens (SFT)	~10.4M train tokens packed into 5,091 windows, val 196k (94 windows)
Hardware	NVIDIA GB10, BF16 training, eval/serve in FP32
Dtype exported	float32 (numerically verified)

SFT data

Mixture (kept after dedup and length filtering, max 2048):

databricks/databricks-dolly-15k (CC BY-SA 3.0, English)
shibing624/alpaca-zh (CC BY 4.0, Chinese, Alpaca-GPT4)

SFT mirrors the seed architecture exactly; tokenizer unchanged (vocab 50000). No new special tokens were introduced.

Chat format (canonical)

Format is plain-text role markers with EOS terminator. This matches the single-source-of-truth implementation:

### System:\n{system}\n
### User:\n{user}\n
### Assistant:\n{content}</s> (assistant turns end with existing </s> id=2)
If no system is provided, a default system is auto-inserted during training/serving (see below).

Default system:
You are OpenSI-Haiku, a helpful and concise AI assistant.

The HF chat_template is bit-exact equivalent to src/chat_format.py (verified by chat_format.self_check against the tokenizer). The assistant generation prompt ends with ### Assistant:\n and the model is trained to emit </s> to terminate.

Inference

Two generation endpoints (OpenAI-compatible):

POST /v1/chat/completions — multi-turn with system/user/assistant. Auto-inserts default system if missing. Supports temperature, top_p, top_k, repetition_penalty, stop, max_tokens, min_tokens, seed, request_id, logprobs (shape/format consistent with completions path). Returns OpenAI-style chat.completion. stream and n>1 are not supported by this static-batch engine.
POST /v1/completions — continuation (also works on chat models).

Serving modes: low-latency interactive profile (--batch-window-ms 0) is recommended for real-time chat. Deterministic greedy behavior is reproducible under fixed seeds in practice (FP32).

Verification

HF export numerical alignment: max abs error 4.18e-5, cosine similarity 1.0000000000, argmax agreement 100.00% (vs custom GPT).
Template consistency: chat_format.self_check passes for representative cases.
Endpoint checks: 14/14 pass via scripts/serve/verify_chat.py (including correct 400s for invalid requests).
Base backward compatibility: /health and /v1/* semantics preserved for the existing base service (restarted with updated, backward-compatible server).

Limitations (important)

This is a 113.9M model trained on a small, mostly instruction-style SFT set (~55k examples mixed EN/ZH). Expect:

Fluent, format-correct chat responses
Good at short, structured tasks and English commonsense patterns
Limited factual recall / domain knowledge (will confidently hallucinate on niche facts)
Weaker multi-hop reasoning and arithmetic than larger models
May echo or ramble on very long prompts near context; prefer concise instructions

License

MIT (as inherited from the OpenSI seed/base line). SFT data sources retain their original licenses (CC BY-SA 3.0 / CC BY 4.0).

Acknowledgements

Built on the OpenSI codebase; adapted to chat via masked SFT with EOS termination and a Jinja-equivalent canonical format. The HF export produces a Qwen2ForCausalLM-compatible directory (tokenizer_config.json includes the chat template).

Downloads last month: 114