Model Card for Toffee 3b β

We've finally completed Toffee β, a series of language models trained to act as helpful assistants. Toffee 3B β is the first model in the series, a refined version of mistralai/Ministral-3-3B-Base-2512, which was post-trained on high-quality instructional datasets.

In fact, we were trying to achieve the title of "The Gold Standard of Small Chat-models" and you know, we succeeded, the model scored excellent results on benchmarks for its number of parameters.

Model description

Model: maxzt/toffee-3b-beta
Language(s) (NLP): Primarily English
License: MIT
Finetuned from model: mistralai/Ministral-3-3B-Base-2512

📊 MMLU & HellaSwag benchmarks results

Model	Params	MMLU	HellaSwag
Toffee-3B-β	3B	63.10%	72.04%
HuggingFaceH4/zephyr-7b-beta	7B	61.07%	84.36%
Qwen2.5-3B-Instruct	3B	65.40%	71.56%
Llama 3.2 3B-Instruct	3B	63.40%	69.80%
Falcon3-3B-Instruct	3B	56.90%	—
Gemma 3 4B PT	4B	59.60%	77.20%
Phi-3.5-mini-Instruct	3.8B	69.00%	69.40%

Toffee-3B-β: MMLU 63.10%, HellaSwag 72.04%.

Results for the other models are taken from published model cards / benchmark reports. Evaluation settings may differ between models (shots, prompt format, evaluator version), so the table is an approximate comparison rather than a strictly controlled leaderboard.

Technical Specifications

Property	Value
Parameters	3B
Architecture	Ministral 3 / Mistral-family
Context Length	8,192 tokens
Vocabulary Size	131,072
Hidden Size	3,072
Layers	26
Attention Heads	32
KV Heads	8
Precision	BF16
Base Model	Ministral-3-3B-Base-2512
License	MIT

Training Details

Parameter	Value
Base model	`mistralai/Ministral-3-3B-Base-2512`
Fine-tuning	Supervised Fine-Tuning (SFT) + LoRA
Training examples	Up to 80,000
Train / eval split	98% / 2%
Max sequence length	8,192 tokens
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.05
LoRA targets	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Epochs	1
Learning rate	1e-4
LR scheduler	Cosine
Warmup ratio	0.05
Weight decay	0.01
Per-device batch size	4
Gradient accumulation	8
Effective batch size*	32
Optimizer	AdamW (`adamw_torch`)
Precision	BF16, fallback to FP16
Packing	Enabled
Loss	Assistant-only
Random seed	42
Framework	Unsloth + Hugging Face

Usage

import torch

from transformers import (
    Mistral3ForConditionalGeneration,
    MistralCommonBackend,
)

model_id = "maxzt/toffee-3b-beta"

tokenizer = MistralCommonBackend.from_pretrained(
    model_id,
)

model = Mistral3ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

user_message = "Explain why the sky is blue."

prompt = f"[INST] {user_message} [/INST] "

inputs = tokenizer.encode(
    prompt,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
    )

response = tokenizer.decode(
    outputs[0][inputs.shape[-1]:]
)

print(response.strip())

Training Data

Toffee-3B-β was trained on a curated mixture of instruction-following, mathematical reasoning, and programming data.

Dataset	Target examples	Purpose
HuggingFaceH4/ultrachat_200k	35,000	General instruction following and conversation
nvidia/OpenMathInstruct-2	25,000	Mathematical reasoning and problem solving
m-a-p/CodeFeedback-Filtered-Instruction	20,000	Programming and code instruction following
Total	80,000

The datasets were converted into a unified prompt/completion format, shuffled with a fixed seed (42), and split into 98% training / 2% evaluation data.

Examples longer than 8,192 tokens were removed. During SFT, loss was applied only to assistant completion tokens.

Citation

@misc{toffee3bbeta,
  title={Toffee 3B β},
  author={Max ZT},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/maxzt/toffee-3b-beta}
}