Surogate-3.5-2B

Surogate is a Romanian-first bilingual (RO/EN) assistant built by Invergent (Bucharest, Romania). It is a dual-mode reasoner: with thinking enabled it reasons step-by-step in the language of the prompt (Romanian prompts → Romanian chain-of-thought, including math); with thinking disabled it answers directly. Trained with the open-source Surogate training framework.

Highlights

• The cleanest Romanian at its size: invented word-forms at 0.95 per 1k words — ~5× cleaner than comparable general-purpose models — with English leakage at just 0.81 per 1k, measured on a 13,963-prompt greedy benchmark (dexonline-authoritative scorer over a 1.52M-form lexicon).
• Dual-mode reasoning: 100% prompt-language reasoning (RO and EN probes), 0% <think> leakage in no-think mode (1,392 deterministic probes).
• The compact sibling of Surogate-3.5-4B — the 4B extends the same qualities with stronger instruction following, math, and translation.

Benchmarks

Benchmark	Surogate-3.5-2B
Romanian text quality
RO invented word-forms / 1k	0.95
RO English-leak / 1k	0.81
RO missing diacritics / 1k	1.59
Knowledge & STEM
MMLU-Pro	63.2
MMLU-Redux	75.6
GPQA	49.0
SuperGPQA	35.6
Instruction following
RO IFEval (prompt / instruction strict)	0.108 / 0.339
EN IFEval (prompt strict)	0.518
IFEval	62.1
IFBench	32.6
MultiChallenge	26.6
Math & reasoning
RO GSM8K (strict)	0.447
EN GSM8K (strict)	0.660
HMMT Feb 25 / Nov 25	~21.8 / ~18.6
PolyMATH	24.8
Agents & long context
BFCL-V4	41.4
TAU2-Bench	46.4
LongBench v2	36.8
Multilingual & translation
Translation EN→RO / RO→EN (chrF2)	45.9 / 52.8
WMT24++	42.6
MMMLU	59.9
MMLU-ProX	49.7
INCLUDE	52.6
Global PIQA	65.8

Usage

vLLM

vllm serve surogate/Surogate-3.5-2B --reasoning-parser qwen3

The repository ships a generation_config.json (temperature 0.6, top_p 0.95, top_k 20) that vLLM applies automatically — these are the recommended sampling settings. For orthography-critical use, greedy decoding (temperature=0) gives the best results.

Thinking control (chat template)

tok.apply_chat_template(messages, add_generation_prompt=True,
                        enable_thinking=True)   # step-by-step reasoning in the prompt's language
                        # enable_thinking=False -> direct answer

Notes

• Surogate-3.5-2B is optimized Romanian-first at compact scale; for maximum capability across tasks, use Surogate-3.5-4B.
• As with any compact model, verify critical outputs.

License & attribution

Apache 2.0. Built by Invergent using the Surogate training framework.