Instructions to use nkthebass/tinybrainbot-320mV2-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nkthebass/tinybrainbot-320mV2-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nkthebass/tinybrainbot-320mV2-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-320mV2-instruct")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-320mV2-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nkthebass/tinybrainbot-320mV2-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nkthebass/tinybrainbot-320mV2-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-320mV2-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-320mV2-instruct

SGLang

How to use nkthebass/tinybrainbot-320mV2-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nkthebass/tinybrainbot-320mV2-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-320mV2-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nkthebass/tinybrainbot-320mV2-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-320mV2-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nkthebass/tinybrainbot-320mV2-instruct with Docker Model Runner:
```
docker model run hf.co/nkthebass/tinybrainbot-320mV2-instruct
```

TinyBrainBot 320M V2 — Instruct

A ~326M-parameter decoder-only language model, trained from scratch on ~10B tokens and then supervised-fine-tuned for chat / instruction following. This is the instruct model.

Base model: tinybrainbot-320mV2-base (full pretraining details there).
Successor to the 303M V2 instruct.

TL;DR: A compact general-purpose assistant trained from scratch on ~10B tokens. It matches/beats Pythia-410M on general benchmarks (on far fewer training tokens) and sweeps GPT-2-124M, follows instructions, and has 2–3-digit arithmetic far stronger than its GSM8K reasoning score would suggest (94–99% on 2–3-digit addition) — though it hits a clear length-generalization wall beyond 3 digits. It is not math-specialized.

Model details


Parameters	325,899,264 (~326M)
Architecture	Decoder-only transformer, pre-norm, RMSNorm, SwiGLU MLP, RoPE
Hidden size	1024
Layers	26
Attention heads	16 (query) / 4 KV heads (grouped-query attention)
FFN size	2816
Context length	1024
Vocabulary	32,000
Tokenizer	`tbb-32k-v2` — 32k BPE (67% English / 20% code / 13% math), with reserved `<think>`/`</think>` special tokens
Precision	trained in fp16 with an fp32 master copy (autocast)

Usage

Prompt with the chat format:

<|user|>
{user message}
<|end|>
<|assistant|>
{assistant reply}
<|end|>

Example (greedy):

<|user|>
What is the capital of France?
<|end|>
<|assistant|>
Paris.
<|end|>

The model gives concise direct answers and shows worked steps for arithmetic.

Training procedure

Pretraining (base)

Pretrained from scratch on 10.03B tokens (51,000 steps, WSD schedule, peak LR 6e-4, two-phase broad → quality-anneal, final loss 1.436) across 2× Tesla V100-PCIE-16GB with PyTorch DDP (gloo), fp16 autocast + fused AdamW. Full pretraining details and the 13-source data mix are documented on the base model card.

Supervised fine-tuning (this model)

Hyperparameter	Value
Steps	~1,433 (resumed from base @ 51,000)
Tokens	~94M
LR	1.5e-5, constant
Warmup	100 steps
Global batch	4 micro × 8 grad-accum × 2 GPUs × 1024 seq
Loss masking	assistant-only
Best checkpoint	step 52,000 (selected on eval, not last)

SFT data mix (sampling weights; sum = 9.5; math ≈ 15.8% of the mixture):

Source	Weight
longdef-sft (~16k long / multi-step answers)	2.0
smoltalk	2.0
math-v2 (~70k verified worked examples)	1.5
qa-distill	1.0
lamini-instructions	1.0
soda-dialogues	1.0
greetings	0.5
reasoning-distill	0.5

math-v2 is ~70k programmatically-generated, tolerance-verified arithmetic worked examples (multiplication / division / decimals, with the final answer checked). It appears in both pretraining and SFT — see the note below.

On where the arithmetic comes from: the base and instruct models score almost identically on the GPT-3 Arithmetic suite (aggregate 31.4% vs 31.2%). That base/instruct parity suggests most arithmetic computation was acquired during pretraining, while SFT mainly shaped instruction-following and response format (the one sub-task where SFT clearly helps is composite / order-of-operations, 7.0% vs 3.7%). Pretraining also delivered far more math tokens in absolute terms — on the order of ~215M vs ~15M — despite SFT's higher math proportion.

Evaluation

Measured on our own log-likelihood MC harness (lm-eval style, fixed seed). Headline metric = acc_norm for HellaSwag/ARC/OpenBookQA, acc for WinoGrande/MMLU. Reference values are published lm-eval approximations — treat gaps under ~±2 points as ties.

vs the previous 303M instruct (full test sets)

Benchmark	n	320M V2 instruct	303M instruct
HellaSwag	10042	34.5	30.7
ARC-Easy (acc_norm)	2376	49.3	47.6
ARC-Easy (raw acc)	2376	57.0	51.0
ARC-Challenge	1172	27.6	27.6
OpenBookQA	500	31.8	29.0
WinoGrande	1267	53.3	52.2
MMLU	14042	28.0	27.1

→ 5 wins, 1 tie, 0 losses over the previous generation.

vs reference models (headline metric)

Benchmark	320M V2 instruct	GPT-2-124M	Pythia-410M	SmolLM-360M
HellaSwag	34.5	31	34	54
ARC-Easy	49.3 (57 raw)	44	52	70
ARC-Challenge	27.6	22	24	37
OpenBookQA	31.8	29	30	42
WinoGrande	53.3	52	53	57
MMLU	28.0	26	25	34

→ Sweeps GPT-2-124M; ~5 wins + 1 draw vs Pythia-410M. SmolLM-360M (trained on ~600B aggressively-filtered tokens) remains the frontier for this size.

Training efficiency. These results come from ~10B pretraining tokens — roughly an order of magnitude fewer than the Pythia suite's ~300B. The Pythia-410M parity is therefore best read as a token-efficiency result (curated data + quality anneal) rather than a scale win.

Math — computation vs reasoning

The model was trained on arithmetic computation, not word-problem reasoning — the two benchmarks below show that split clearly.

GSM8K (grade-school word problems, full 1319-problem test, zero-shot chain-of-thought):

Model	GSM8K
GPT-2-124M	~0%
320M V2 instruct	0.53%
Pythia-410M	~1–2%
SmolLM2-360M-Instruct	~3–5%

→ At the floor for a general-purpose model of this size and training mix; stronger sub-1B math-specialized models can score substantially higher. GSM8K rewards multi-step semantic reasoning, which this recipe did not target.

GPT-3 Arithmetic (Brown et al. 2020 protocol, exact-match, n=300/sub-task):

Sub-task	Accuracy
2-digit addition	99.0%
3-digit addition	94.0%
2-digit subtraction	49.3%
3-digit subtraction	42.7%
4-digit addition / subtraction	0.3% / 0.3%
5-digit addition / subtraction	0.0% / 0.0%
2-digit multiplication	21.7%
single-digit composite (order of ops)	7.0%
Aggregate (all 10 sub-tasks)	31.4%

→ Strong through 3 digits, then a hard wall. 2–3-digit addition is near-solved (94–99%, with correct carrying), but 4+-digit accuracy collapses to 0%: the model executes a fixed **3-column** addition routine and silently drops the higher place values — a length-generalization limit tied to the training distribution (math-v2 operands are ≤3 digits), not truncation (generations complete normally and end with a stated answer). Subtraction sits ~42–50% — it handles a−b when a>b but drops the sign on negative results. 2-digit multiplication ~22%; single-digit composite (order of operations) ~7%.

How to read this: the GPT-3 Arithmetic suite mainly probes exact symbolic computation and short-range algorithmic generalization; it should not be interpreted as evidence of strong mathematical reasoning (see GSM8K above). The two results together are the point: strong at computing, weak at reasoning.

Intended use & limitations

Intended use: a capable general chat assistant at ~326M scale, on-device / low-resource deployment, research on small-model SFT, and arithmetic computation.

Limitations:

Math reasoning (word problems, GSM8K/MATH) is at the floor — the model computes but does not reason through multi-step problems.
Negative-result subtraction is unreliable (drops the sign).
WinoGrande and MMLU sit near the random floor — consistent with the capacity and data limits of a ~326M model under this training recipe.
Trained predominantly on English; 1024-token context; no RLHF/safety tuning — outputs may be incorrect or inappropriate and should not be relied upon unchecked.

Hardware & framework

2× NVIDIA Tesla V100-PCIE-16GB · Windows · PyTorch DDP (gloo) · fp16 autocast (fp32 master) · fused AdamW · custom TinyBrainBot trainer.

Downloads last month: -

Safetensors

Model size

0.3B params

Tensor type

F16

Model tree for nkthebass/tinybrainbot-320mV2-instruct

Base model

nkthebass/tinybrainbot-320mV2-base

Finetuned

(1)

this model