Haidass-Translate-143M-Instruction

English | 中文

The instruction-tolerant sibling of DALabCommunity/Haidass-Translate-143M: same 143M zh⇄en translation training, plus 9.1% cleaned general-domain data (STEP_FUN ShareGPT) mixed in. Translation scores are within 0.3 BLEU of the pure-translation version, and the model retains limited general instruction-following ability that the pure-translation version does not have.

Live demo: Haidass Translate Instruction on Hugging Face Spaces.

FLORES-200 dev

Model	Params	Arch	en→zh BLEU	en→zh chrF++	zh→en BLEU	zh→en chrF++
HY-MT1.5-1.8B	1800M	LLM	44.65	30.98	27.68	57.96
Qwen3-0.6B	600M	LLM	30.94	21.10	20.21	48.62
OPUS-MT en-zh	78M	Seq2Seq	30.88	21.80	-	-
OPUS-MT zh-en	78M	Seq2Seq	-	-	22.99	51.03
Qwen2.5-0.5B-Instruct	500M	LLM	28.96	19.65	18.09	45.85
M2M-100-418M	418M	Seq2Seq	28.04	20.53	20.58	48.79
Haidass-Translate-143M	143M	LLM	23.89	17.55	14.65	40.76
Haidass-Translate-143M-Instruction	143M	LLM	23.64	17.21	14.35	40.13
NLLB-200-distilled-600M	600M	Seq2Seq	22.44	16.74	25.71	52.28
Drafter-143M*	143M	LLM	12.04	9.43	5.47	27.31

*Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining.

OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve.

FLORES+ devtest

The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev):

Model	Params	Arch	en→zh BLEU	en→zh chrF++	zh→en BLEU	zh→en chrF++
HY-MT1.5-1.8B	1800M	LLM	37.36	26.08	20.33	51.48
OPUS-MT en-zh	78M	Seq2Seq	32.23	22.40	-	-
OPUS-MT zh-en	78M	Seq2Seq	-	-	23.06	51.03
Qwen3-0.6B	600M	LLM	31.76	21.48	19.66	48.14
Qwen2.5-0.5B-Instruct	500M	LLM	29.32	19.95	18.04	46.00
M2M-100-418M	418M	Seq2Seq	28.29	20.60	19.52	47.87
Haidass-Translate-143M	143M	LLM	25.06	17.72	13.68	40.02
Haidass-Translate-143M-Instruction	143M	LLM	24.74	17.77	13.95	39.93
NLLB-200-distilled-600M	600M	Seq2Seq	23.07	16.94	24.30	51.48
Drafter-143M*	143M	LLM	10.93	9.00	5.82	26.83

Instruction following

Mixing 9.1% general-domain data into the translation SFT gives this variant a limited ability to respond to general instructions (short poems, simple summaries, paraphrases) that the pure-translation sibling essentially lacks. This ability is fragile: prompts outside the translation domain may produce echoes of the input, partial answers, or drifting continuations. The model is still primarily a translator; treat general instruction following as a bonus, not a guarantee.

Training recipe

• Base: Haidass1.5-143M (Qwen3 architecture: 30 layers, hidden 576, GQA 9/3, vocab 64,000)
• Data: 7.837M cleaned zh↔en parallel sentence pairs (15.67M samples bidirectional) + 157,593 cleaned STEP_FUN ShareGPT conversations (9.1% of training tokens)
• Packing: official MindSpeed-LLM --pack --neat-pack (683,227 full 2048-token sequences with inter-document attention-mask isolation)
• Training: 16×Ascend 910C, GBS=256, lr 3e-5 cosine, 2 epochs = 5,337 steps (~2.8B tokens), final loss 1.788
• Framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("umeiko/Haidass-Translate-143M-Instruction", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("umeiko/Haidass-Translate-143M-Instruction")

msgs = [{"role": "user", "content": "将以下文本翻译为英文：光子甚至比构成原子的物质还要小！"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# Even photons are smaller than the stuff that makes up atoms!

Note: the training data follows the qwen3 chat template (an empty <think></think> block precedes the assistant turn). Always use the model's built-in chat_template at inference; do not hand-craft prompts.

Known limitations

• zh→en BLEU plateaus at ~14 across data scales; more same-distribution parallel data does not push this direction further
• Typical residual errors: entity mix-ups, occasional omission of numeric details; rare-entity-dense zh→en inputs can trigger repetitive looping under greedy decoding (use repetition_penalty / no_repeat_ngram_size as mitigation)
• General instruction following is limited and unstable (see above)
• Optimized for zh⇄en; not a general chat model

Evaluation

• Metrics: sacreBLEU corpus BLEU (tokenize=zh for Chinese targets, tokenize=13a for English) + chrF++ (word_order=2); prompts byte-identical to the training chat template
• Decontamination: all 15.83M training samples audited against both benchmarks by n-gram overlap (8-word English / 10-character Chinese fragments): 1,147 hits (0.0072%) vs FLORES-200 dev, 1,788 (0.0113%) vs FLORES+ devtest; the STEP_FUN-only subset shows 6 hits (0.0038%) vs devtest. Manual inspection shows common-phrase-level overlap, not full-sentence leakage
• Raw predictions, audit reports and evaluation scripts for every model in the tables above: umeiko/Haidass-Translate-143M-eval

_{Note: all scores are measured on the FLORES Chinese–English subset (eng_Latn ↔ zho_Hans), bidirectional (997 sentences for dev, 1,012 for devtest), with greedy decoding.}