Dariachup/dormouse-mt-uk-en

dormouse MT — Ukrainian → English (fine-tuned, v0.7)

Fine-tune of Helsinki-NLP/opus-mt-uk-en (76M params, runs on CPU) on real Ukrainian chat. Serves as the cloud-target translator inside the dormouse-ua library, so a Ukrainian user can talk to an English-speaking LLM without the model ever having to generate Cyrillic.

Training corpus (v0.7) — 285 784 pairs

source	pairs	what it is
v06	158 482	real Ukrainian chat (Telegram, Threads) — human
native	96 738	native English chat (hh-rlhf, oasst1), UA side MT
cs	20 924	customer-support bitext, UA side MT
surzhyk	9 640	synthetic surzhyk injected into real sentences

Base: Helsinki-NLP/opus-mt-uk-en, mixed with general OPUS/flores pairs to limit catastrophic forgetting.

Metrics (v0.7, 1000 held-out pairs, sacrebleu)

slice	n	base BLEU	ft BLEU	base chrF	ft chrF
overall	1000	23.20	38.46	45.49	58.80
cs *	250	24.82	58.99	47.94	75.70
native *	250	34.01	47.77	57.53	67.63
v06	250	17.43	31.32	38.82	53.16
surzhyk	250	16.12	25.79	37.97	47.36

Read the starred slices with suspicion

* — on the cs and native slices the Ukrainian source side was itself produced by gemini-2.5-flash, not written by a human. A 47-59 BLEU there means "the fine-tune round-trips Gemini's Ukrainian well", which is what it was trained to do. It is not evidence of human-level quality.

The honest slices are v06 (real Ukrainian chat with human references) and surzhyk. Read those first: 31.32 BLEU / 53.16 chrF, up from 17.43 / 38.82 on the base model.

Known failure modes

Trained on chat, so it drifts on commercial nouns outside that domain: реквізити → refunds, гравіювання імені → name-playing order, застібка → zip. A glossary layer or a domain retrain fixes this; the model alone does not.

Usage

from transformers import MarianMTModel, MarianTokenizer
m = MarianMTModel.from_pretrained("Dariachup/dormouse-mt-uk-en")
t = MarianTokenizer.from_pretrained("Dariachup/dormouse-mt-uk-en")
ids = t("та шо там по замовленню", return_tensors="pt")
print(t.decode(m.generate(**ids, num_beams=4)[0], skip_special_tokens=True))

Or through the library, which also handles surzhyk normalisation:

from dormouse.mt_translator import get_translator
print(get_translator("uk-en").translate("та шо там по замовленню"))