amnmt-en-am-base

A from-scratch PyTorch Transformer for bidirectional English ↔ Amharic translation. One 60.5M-parameter model serves both directions via a target-language tag (<2am> / <2en>). Code: english-amharic-transformer-nmt · Demo: Space.

Usage

pip install "amnmt[tokenization,hub] @ git+https://github.com/Bekafi01/english-amharic-transformer-nmt.git"
amnmt translate -m hf://Bekafi/amnmt-en-am-base -d en-am "Coffee originated in Ethiopia."

from amnmt.inference.translator import Translator

tr = Translator.from_checkpoint("hf://Bekafi/amnmt-en-am-base")
tr.translate(["The children are playing in the garden."], "en-am")  # ['ልጆች በአትክልቱ ውስጥ እየተጫወቱ ነው።']
tr.translate(["ቡና መነሻው ኢትዮጵያ ነው።"], "am-en", beam_size=4)

Inputs are normalized the same way as the training data (NFC, Ethiopic homophone folding, Ge'ez numerals → digits), so raw Amharic spelling variants are accepted.

Files

file
model.pt	fp16 weights + model config, ~120 MB. torch.load → {"model", "config", "vocab_size", "pad_id", "export"}
tokenizer.json	joint 32k BPE (HF tokenizers), byte fallback; specials <pad>=0 <s>=1 </s>=2 <unk>=3 <2am>=4 <2en>=5

Model

Architecture	Pre-LN encoder–decoder, 6+6 layers, d_model 512, 8 heads, d_ff 2048, dropout 0.1, sinusoidal positions
Embeddings	one 32k × 512 matrix tied across source, target and output projection
Parameters	60,524,544
Max length	256 positions (trained on ≤ 128 per side)
Decoding	beam search, GNMT length penalty 1.0

Training data

4,429,739 sentence pairs after cleaning, both directions (8.86M training samples):

source	license	pairs after dedup
NLLB mined bitext (OPUS), LASER score ≥ 1.068	CC-BY-NC-4.0	3.5M
MT560 en–am slice	CC-BY-4.0	628k
CCAligned (OPUS)	CC (web crawl)	259k
Tanzil (OPUS)	Tanzil	35k
OPUS-100 am–en	mixed	3k

Preprocessing: NFC, Ethiopic homophone folding (ሐ/ኀ→ሀ, ሠ→ሰ, ዐ→አ, ፀ→ጸ), Ge'ez numerals → digits, ፡ → space, Moses detokenization; length/ratio/script/URL/markup filters; exact dedup on a normalized key. FLORES-200 dev and devtest were removed from training by normalized-key match.

Training

54,000 steps (7.1 epochs) on a single Colab T4, fp16 + GradScaler, 8,000 padded tokens × 3 accumulation per step, AdamW (β 0.9/0.98), peak LR 7e-4 with 4k warmup and inverse-sqrt decay, label smoothing 0.1, grad clip 1.0. ≈ 15 GPU-hours over four resumed sessions. Stopped when FLORES dev loss plateaued (perplexity 7.9).

Evaluation — FLORES-200 devtest (1,012 sentences), sacreBLEU 2.6

direction	beam	BLEU	chrF++	spBLEU	BLEU vs raw refs
en → am	4	14.51	37.74	28.36	12.24
en → am	1	13.42	36.90	–	11.32
am → en	4	24.39	49.27	24.95	24.29
am → en	1	23.53	48.39	–	23.45

"vs raw refs" scores the folded output against the unfolded FLORES references — the −2.3 BLEU en→am gap is the cost of homophone folding, not translation errors.

Limitations

• Amharic output uses folded spellings; words written with ሐ, ሠ, ዐ, ፀ in standard orthography come out with ሀ, ሰ, አ, ጸ.
• Trained mostly on religious, web-crawled and mined text; news/encyclopedic input (FLORES) works well, conversational and technical text less so.
• Sentences longer than ~120 tokens are truncated. No document context.
• 60M parameters: expect occasional dropped clauses and tense errors on long sentences.

License

Weights: CC-BY-NC-4.0 (inherited from the NLLB bitext). Code: MIT.