Haidass-Translate-143M

English | 中文

A 143M-parameter bidirectional Chinese↔English translation model, instruction-tuned on the Haidass1.5-143M base — the strongest zh⇄en translator at this scale among general chat-architecture models.

Live demo: Haidass Translate on Hugging Face Spaces.

FLORES-200 dev

Model	Params	Arch	en→zh BLEU	en→zh chrF++	zh→en BLEU	zh→en chrF++
HY-MT1.5-1.8B	1800M	LLM	44.65	30.98	27.68	57.96
Qwen3-0.6B	600M	LLM	30.94	21.10	20.21	48.62
OPUS-MT en-zh	78M	Seq2Seq	30.88	21.80	-	-
OPUS-MT zh-en	78M	Seq2Seq	-	-	22.99	51.03
Qwen2.5-0.5B-Instruct	500M	LLM	28.96	19.65	18.09	45.85
M2M-100-418M	418M	Seq2Seq	28.04	20.53	20.58	48.79
Haidass-Translate-143M	143M	LLM	27.56	19.27	17.17	43.20
NLLB-200-distilled-600M	600M	Seq2Seq	22.44	16.74	25.71	52.28
Drafter-143M*	143M	LLM	12.04	9.43	5.47	27.31

*Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining.

OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve.

FLORES+ devtest

The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev):

Model	Params	Arch	en→zh BLEU	en→zh chrF++	zh→en BLEU	zh→en chrF++
HY-MT1.5-1.8B	1800M	LLM	37.36	26.08	20.33	51.48
OPUS-MT en-zh	78M	Seq2Seq	32.23	22.40	-	-
OPUS-MT zh-en	78M	Seq2Seq	-	-	23.06	51.03
Qwen3-0.6B	600M	LLM	31.76	21.48	19.66	48.14
Qwen2.5-0.5B-Instruct	500M	LLM	29.32	19.95	18.04	46.00
M2M-100-418M	418M	Seq2Seq	28.29	20.60	19.52	47.87
Haidass-Translate-143M	143M	LLM	28.48	19.45	17.01	42.69
NLLB-200-distilled-600M	600M	Seq2Seq	23.07	16.94	24.30	51.48
Drafter-143M*	143M	LLM	10.93	9.00	5.82	26.83

devtest sentences do not overlap with dev. This model's scores on the new split are essentially unchanged (en→zh 27.56→28.48, zh→en 17.17→17.01), indicating that the results reflect translation ability rather than memorization of a specific benchmark.

Decontamination

To verify that the scores contain no test-set leakage, we audited all 15.83M training samples: every sentence is cut into consecutive fragments (8 words for English, 10 characters for Chinese), and any training sample sharing any fragment with any test sentence is counted as a hit. Results: 1,147 hits (0.0072%) against FLORES-200 dev, 1,788 (0.0113%) against FLORES+ devtest. Manual inspection shows the hits are common-phrase-level fragment overlaps rather than full-sentence leakage — i.e., the reported scores are not inflated by leakage. Audit report (top-50 overlapping samples included for inspection): audit_report.json (devtest audit: audit_floresplus_devtest.json in the same repo).

Training recipe

• Base: Haidass1.5-143M (Qwen3 architecture: 30 layers, hidden 576, GQA 9/3, vocab 64,000)
• Data: 7.837M cleaned zh↔en parallel sentence pairs (15.67M samples bidirectional, translation-only, no general-domain data)
• Packing: official MindSpeed-LLM --pack --neat-pack (607,622 full 2048-token sequences with inter-document attention-mask isolation)
• Training: 16×Ascend 910C, GBS=256, lr 3e-5 cosine over a 5-epoch schedule; released checkpoint at epoch 4 (iteration 9,496, ~5.0B tokens, loss 1.671) — epoch-wise ablation showed epoch 4 as the sweet spot (epoch 5 added no gain)
• Framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("DALabCommunity/Haidass-Translate-143M", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("DALabCommunity/Haidass-Translate-143M")

msgs = [{"role": "user", "content": "将以下文本翻译为英文：光子甚至比构成原子的物质还要小！"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# Even photons are smaller than the stuff that makes up atoms!

Note: the training data follows the qwen3 chat template (an empty <think></think> block precedes the assistant turn). Always use the model's built-in chat_template at inference; do not hand-craft prompts.

Translation samples (spot check on FLORES-200 dev)

en→zh:

Src: Water is another example. The compound water is made up of two hydrogen atoms and one oxygen atom. Out: 水是另一个例子。化合物水是由两个氢原子和一个氧原子组成的。(sentence chrF++ 71.7)

Src: They are listed on the UNESCO World Heritage List. Out: 它们被列入联合国教科文组织世界遗产名录。(sentence chrF++ 69.4)

zh→en:

Src: 它们被列入了联合国教科文组织世界遗产名录。 Out: They are listed in the UNESCO World Heritage List. (sentence chrF++ 89.2)

Src: 光子甚至比构成原子的物质还要小！ Out: Even photons are smaller than the stuff that makes up atoms! (sentence chrF++ 84.1)

Known limitations

• zh→en gains come mainly from longer training (2→4 epochs: 14.65→17.17); at 2 epochs, more same-distribution parallel data (1M/4M/8M) plateaued at ~14.4
• Typical residual errors: entity mix-ups (e.g., "斯洛伐克" → Slovenia), occasional omission of numeric details
• Optimized for zh⇄en translation only; not a general chat model

Evaluation

• Metrics: sacreBLEU corpus BLEU (tokenize=zh for Chinese targets, tokenize=13a for English) + chrF++ (word_order=2); prompts byte-identical to the training chat template
• Benchmark sources: openlanguagedata/flores_plus (FLORES+, the maintained version; gated — auto-approved after accepting terms; Simplified Chinese now cmn_Hans); facebook/flores (original archive, unmaintained); login-free mirror facebookresearch/flores
• Raw predictions: the eval/ directory in this repo contains per-sentence prediction jsonl ({direction, src, ref, hyp}, one file per model per benchmark) for every model in the tables above — all scores can be recomputed with sacreBLEU
• Full results & reproduction: the 16-model comparison table, per-sentence predictions, decontamination audits and evaluation scripts live in the companion dataset umeiko/Haidass-Translate-143M-eval

_{Note: all scores are measured on the FLORES Chinese–English subset (eng_Latn ↔ zho_Hans), bidirectional (997 sentences for dev, 1,012 for devtest), with greedy decoding.}