stefaniancu/rost-126m-sft

126M parameter nanochat GPT (sft stage), research tag ro30, checkpoint step 000600.

Part of the rost model zoo — the full set of trained variants behind the rost research series, published for reproducibility. This is a research model, not a product.

architecture	nanochat GPT, 8 layers, 512 embed, 2048 context
tokenizer	included under tokenizer/ (32,768 vocab)
training mixture	Romanian SFT mixture (OpenLLM-Ro) with 25% English (--english-frac 0.25)
stage	sft
val bpb (own split)	0.48868

The best bilingual conversational model of the rost small series (research tag ro30, experiment 004d): English in, English out; Romanian in, Romanian out. Its sibling trained without the English fraction answered everything in Romanian — the 'SFT language decides output language' finding.

Validation bits-per-byte is measured on this arm's own validation split and is not comparable across arms — cross-arm comparisons in the rost write-ups are always cross-evaluated on identical text.

Licence

CC-BY-NC-4.0, non-commercial, inherited from the most restrictive component of the training data.