Amrouche-2-600M

A 615M-parameter (615,073,792) bidirectional machine translation model for Kabyle (Taqbaylit, kab_Latn), English (eng_Latn), French (fra_Latn), and Modern Standard Arabic (arb_Arab), fine-tuned from facebook/nllb-200-distilled-600M on 544,965 verified parallel sentence pairs from the AƔBALU parallel corpus.

Amrouche-2-600M outperforms base NLLB-600M across all six translation directions on FLORES+ devtest, reaching 34.88 macro chrF++ (+3.91 gain) and 11.09 macro BLEU (+20.7% relative gain). On the held-out gold test splits, it reaches 50.73 macro chrF++ / 30.75 macro BLEU on the 384-sentence panel and 48.58 macro chrF++ / 27.98 macro BLEU on the full 868-sentence test set, delivering over $2\times$ faster generation throughput at less than half the disk and memory footprint of Amrouche-2-1.3B (1.23 GB vs. 2.74 GB).

Unlike historical vocabulary-trimmed models, Amrouche-2-600M preserves the complete 256,206-token SentencePiece vocabulary. This prevents <unk> substitutions on loanwords or unseen entities and maintains direct compatibility with standard Hugging Face pipelines, CTranslate2, and ONNX runtimes.

For resource-constrained edge environments or CPU-only microservices, see the quantized release Amrouche-2-600M-CT2-int8 (622.6 MB, 155 ms single-sentence latency, 4.9× faster on CPU with less than 0.15 chrF++ quantization delta).

Results

All models are evaluated on FLORES+ devtest (1,012 sentences per direction, 6,072 total) and the held-out gold test splits using sacrebleu 2.6.0 (chrF++ with word order 2 and BLEU with 13a tokenization):

1. FLORES+ devtest (1,012 sentences per direction)

Evaluated against the orthographically corrected kab_Latn reference benchmark:

Direction	Base NLLB-600M	Amrouche-2-600M (Native)	Amrouche-2-600M (Hub Balanced)	Relative BLEU Gain
kab → eng	39.53 / 17.82	42.63 / 20.32	42.53 / 20.35	+14.2%
kab → fra	37.38 / 15.20	39.90 / 17.26	39.95 / 17.21	+13.2%
eng → kab	27.51 / 5.90	33.09 / 8.45	33.24 / 8.56	+45.1%
fra → kab	26.18 / 4.51	31.36 / 6.45	31.67 / 6.67	+47.9%
arb → kab	25.74 / 4.67	30.94 / 6.48	31.06 / 6.51	+39.4%
kab → arb	29.49 / 7.05	30.75 / 7.20	30.83 / 7.22	+2.4%
6-Direction Macro	30.97 / 9.19	34.78 / 11.03	34.88 / 11.09	+20.7%

2. Held-Out Gold Test Splits (test.arrow)

Evaluated on decontaminated human-authored pairs partitioned by transitive content groups (zero document or sentence overlap):

Evaluation Split	Support	chrF++	Normalized chrF++	BLEU	Copy Count	Empty Count
Held-Out Test Panel (panel_test_384)	384 sentences (64/dir)	50.73	51.33	30.75	0 / 384	0 / 384
Full Held-Out Gold Test (full_test_868)	868 sentences	48.58	49.29	27.98	0 / 868	0 / 868
Development Panel (panel_dev_384)	384 sentences (64/dir)	51.01	51.58	28.91	0 / 384	0 / 384

• Distillation Efficiency: At 615M parameters, Amrouche-2-600M delivers 34.88 macro chrF++ on FLORES+ and 50.73 macro chrF++ on the held-out test split, operating within a lean 1.23 GB BF16 storage footprint and 1,690 MiB peak VRAM.
• Target-Side Generation: Gains are heavily concentrated when translating into Kabyle (+5.73 chrF++ on eng → kab, +5.49 on fra → kab, and +5.32 on arb → kab), rectifying the severe morphological and grammatical deficits of the base model.
• Bidirectional Arabic Support: Full bidirectional translation with Modern Standard Arabic (arb_Arab), lifting arb → kab by +5.32 chrF++ and kab → arb by +1.34 chrF++.
• Zero Empty / Copy Failures: Evaluated across thousands of sentences, the model produced zero degenerate empty strings and zero untranslated source copies.

Comparison: Amrouche-2-600M vs. Amrouche-2-1.3B

Amrouche-2 is provided in two complementary parameter tiers sharing the exact same verified bitext curriculum, tokenizer, and standalone inference architecture:

Attribute	Amrouche-2-600M	Amrouche-2-1.3B	Trade-off / Delta
Parameters	615,073,792 (615M)	1,370,638,336 (1.37B)	-55.1% parameters
Model Size (BF16)	1.23 GB	2.74 GB	-55.1% disk / VRAM
Model Size (CT2 int8)	622.6 MB	1.38 GB	-54.9% storage
Layers (Enc / Dec)	12 / 12	24 / 24	Half depth
FFN Intermediate Dim	4,096	8,192	2x width
CPU Latency (batch=1, 8 threads)	155 ms (CT2) / 631 ms (PT)	206 ms (CT2) / 958 ms (PT)	+33% lower latency
CPU Throughput (batch=8, 8 threads)	17.05 sent/s (233.6 tok/s)	12.93 sent/s (178.7 tok/s)	+31% faster throughput
CUDA Peak VRAM (Batch 16)	1,690.5 MiB	3,333.0 MiB	Fits on consumer GPUs
FLORES+ Devtest Macro chrF++	34.88	38.22	-3.34 chrF++
FLORES+ Devtest Macro BLEU	11.09	13.80	-2.71 BLEU
Held-Out Gold Test (test_868)	48.58 chrF++ / 27.98 BLEU	52.36 chrF++ / 32.26 BLEU	-3.78 chrF++
Held-Out Panel (test_384)	50.73 chrF++ / 30.75 BLEU	54.72 chrF++ / 35.46 BLEU	-3.99 chrF++

When to Choose Each Model

• Choose Amrouche-2-600M for: Real-time interactive UI translation, chat translation, subtitle synchronization, client-side/edge deployment (Apple Silicon, ONNX, mobile devices), or high-throughput batch translation of large corpora where speed and memory efficiency are primary.
• Choose Amrouche-2-1.3B for: Publishing-grade literary translation, complex multi-clause syntax, nuanced morphosyntactic agreement on long sentences, and server deployments where maximum translation fidelity is required regardless of compute cost.

Orthographic Consistency

Hypotheses and references were measured both raw and after passing through the versioned AƔBALU normalizer (1.3.0+rules1.0.0):

Evaluation Condition	Raw chrF++	Normalized chrF++	Orthography Gap
eng → kab	47.80	48.54	0.74
fra → kab	43.44	44.28	0.84
arb → kab	47.28	48.32	1.04

The small gap confirms that Amrouche-2-600M consistently emits standard Kabyle Latin characters (ɣ ɛ ḥ ḍ ṣ ṭ ẓ ṛ č ǧ ţ) without homoglyph corruption.

Intended Use

• Primary applications: Fast, bidirectional sentence, paragraph, and document translation between Kabyle and English, French, or Modern Standard Arabic on resource-constrained devices, edge environments, and interactive applications.
• Downstream integration: Bilingual corpus alignment, subtitle and UI localization, cross-lingual information retrieval, and pivot translation (X → eng → kab or kab → eng → X).

Not suitable for:

• Safety-critical medical, clinical, or legally binding decisions without qualified human review.
• Sibling Amazigh languages such as Tashelhit (shi), Central Atlas Tamazight (tzm), Tarifit (rif), Shawiya (shy), or Tamasheq (taq), which have distinct lexicons and morphosyntax.
• Unstandardized Arabizi or Neo-Tifinagh inputs without prior conversion through Boulifa-48M or Juba-27M.

Usage

Amrouche-2-600M can be used via custom generation presets (trust_remote_code=True), standard Hugging Face pipeline, or native M2M100ForConditionalGeneration:

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "agbalu/Amrouche-2-600M"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
).eval()

# Batch translation (English -> Kabyle)
print(
    model.translate(
        ["The house is big.", "I speak Kabyle.", "Water is life."],
        src_lang="en",
        tgt_lang="kab",
        tokenizer=tokenizer,
    )
)
# ['Meqqer uxxam-nni.', 'Heddreɣ taqbaylit.', 'Aman d tudert.']

# Modern Standard Arabic -> Kabyle
print(
    model.translate(
        "هل بإمكانك أن تفتح لي الباب؟",
        src_lang="ar",
        tgt_lang="kab",
        tokenizer=tokenizer,
    )
)
# 'Tzemreḍ ad iyi-d-teldiḍ tawwurt?'

# Kabyle -> French
print(
    model.translate(
        "Aman d tudert, taqbaylit d tutlayt-nneɣ.",
        src_lang="kab",
        tgt_lang="fr",
        tokenizer=tokenizer,
    )
)
# "L'eau est la vie, le kabyle est notre langue."

Using the Hugging Face pipeline API:

from transformers import pipeline

translator = pipeline(
    "translation",
    model="agbalu/Amrouche-2-600M",
    trust_remote_code=True,
    src_lang="eng_Latn",
    tgt_lang="kab_Latn",
)
print(translator("When did you graduate?"))
# [{'translation_text': 'Melmi i d-tewwiḍ agerdas?'}]

Using native M2M100ForConditionalGeneration without remote code:

import torch
from transformers import AutoTokenizer, M2M100ForConditionalGeneration

model_id = "agbalu/Amrouche-2-600M"
tokenizer = AutoTokenizer.from_pretrained(model_id, src_lang="eng_Latn")
model = M2M100ForConditionalGeneration.from_pretrained(model_id, dtype=torch.bfloat16).eval()

inputs = tokenizer(["Water is life."], return_tensors="pt", padding=True)
with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids("kab_Latn"),
        num_beams=4,
        max_new_tokens=128,
    )
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
# ['Aman d tudert.']

Standalone Inference Features (trust_remote_code=True)

When loaded with trust_remote_code=True, model.translate(...) activates the standalone AƔBALU inference engine:

• Arbitrary Document Length: Automatically chunks multi-paragraph texts along sentence boundaries and reassembles them with exact layout preservation, bypassing NLLB's 512-token context limit.
• In-Flight Normalisation & Typography Folding: Standardises Kabyle orthography and folds smart typography (« » “ ” —) that base NLLB maps to <unk>, preventing token fragmentation.
• Decoding Presets:
  • preset="fast": 2-beam low-latency decoding.
  • preset="balanced" (default): 4-beam search with confidence validation.
  • preset="quality": Multi-candidate generation with Minimum Bayes Risk (chrF++) and round-trip noisy-channel reranking.
• Failure Detection & Retry: Automatically detects repetition loops, untranslated copies, and off-script drift, retrying affected segments with contrastive guidance.
• Zero Remote Code Alternative: For standard production environments (vLLM, CTranslate2, ONNX), use native M2M100ForConditionalGeneration directly (trust_remote_code=False).

Architecture

Parameter	Specification
Parameters, trained	615,073,792
Parameters, published	615,073,792
Base Model	facebook/nllb-200-distilled-600M
Backbone	12 encoder layers, 12 decoder layers, 1,024 hidden dimension, 16 attention heads, 4,096 intermediate (ReLU)
Context Window	512 tokens per segment (1,024 maximum positional embeddings; unbounded document length via automatic segmentation)
Position Embeddings	Sinusoidal positional embeddings
Vocabulary	256,206 tokens (native NLLB SentencePiece BPE, tied input/output embeddings)
Precision	BF16

The model preserves the exact parameter structure and tied embedding matrix (model.shared.weight) of M2M100ForConditionalGeneration. No vocabulary trimming or adapter layers are used, allowing direct conversion to CTranslate2 and ONNX runtimes.

Training Recipe

Parameter	Specification
Corpus Size	544,965 parallel pairs → 1,089,930 directional examples (Stage 1: 544,965 pairs; Stage 2: 539,494 gold pairs)
Optimizer	Fused AdamW, $\beta=(0.9, 0.98)$, $\epsilon=10^{-8}$, weight decay 0.01, gradient norm clipping 1.0
Learning Rate	Stage 1: $1.0\times 10^{-4} \to 1.0\times 10^{-5}$; Stage 2: $2.0\times 10^{-5} \to 2.0\times 10^{-6}$ (cosine decay)
Effective Batch Size	32,768 target tokens per update (4,096 micro-batch tokens, max 16 sequences per micro-step)
Training Steps	1,560 total updates (Stage 1: 1,091 steps; Stage 2: 469 steps)
Precision	bfloat16

• Two-Stage Curriculum: Stage 1 adapted the base checkpoint on 544,965 parallel pairs using a 60/40 sampling mix of gold human translations and multi-sentence web alignments capped at 3 segments. Stage 2 reinitialized the optimizer at a $5\times$ lower learning rate and trained exclusively on 539,494 verified human gold pairs across all six translation directions.
• Exclusion of Mined Bitext: Approximately 90.1% of publicly available Kabyle parallel data is web-mined by NLLB with unverified precision and severe sibling-Berber confusion. All 440,750 unverified mined records were excluded to prevent distilling base-model errors.
• Full Parameter Fine-Tuning: Full parameter fine-tuning in bfloat16 was used over parameter-efficient methods to ensure complete adaptation across all encoder and decoder layers.

Decontamination

All candidate rows were filtered through exact-string and 13-word n-gram overlap checks against FLORES+ (dev and devtest, raw and normalized kab_Latn, eng_Latn, fra_Latn, and arb_Arab), SIB-200, and local evaluation references. Overlapping records were quarantined prior to training. Multi-sentence records and shared translation hubs were grouped transitively via union-find ContentGroups, guaranteeing zero sentence or document overlap across training, validation, and test splits.

Limitations

• Dialect & Script Scope: Trained and evaluated on standard Kabyle Latin orthography (kab_Latn). Informal keyboard spellings, Arabizi numerals (3, 7, 9), and Neo-Tifinagh script require upstream normalization before translation.
• Domain Coverage: Because 76.6% of gold training pairs originate from Tatoeba community translations and 4.2% from biblical text, literary and conversational sentences are better covered than specialized legal, clinical, or industrial terminology.
• Arabic Supervision Share: Direct arb_Arab ↔ kab_Latn gold training data comprises 4,135 pairs per direction (0.77% of Stage 2 training examples). On technical Arabic prose, pivoting through English (arb_Arab → eng_Latn → kab_Latn) can yield higher lexical precision.
• Human Adequacy Evaluation: Reported metrics (chrF++, BLEU) are reference-based automated measurements; formal human adequacy and fluency evaluations have not yet been conducted.

Files

File	Size	Description
model.safetensors	1.23 GB	Published BF16 model weights (615,073,792 parameters)
tokenizer.json	32.2 MB	Fast NLLB-200 subword tokenizer (256,206 tokens)
sentencepiece.bpe.model	4.85 MB	Native SentencePiece BPE model (CTranslate2 compatible)
modeling_amrouche.py	40.5 KB	Standalone AmroucheForConditionalGeneration and MBR reranker
preprocessing_amrouche.py	20.4 KB	Kabyle character normalizer, typography folder, and segmenter
pipeline_amrouche.py	4.3 KB	Standalone AmroucheTranslationPipeline implementation
tokenizer_config.json	3.7 KB	Language-tag registry and tokenizer configuration
special_tokens_map.json	3.5 KB	Special token definitions (<s>, <pad>, </s>, <unk>)
config.json	1.1 KB	Architecture hyperparameters, auto_map, and custom_pipelines
generation_config.json	184 B	Default decoder generation parameters and token IDs
export.stats.json	2.0 KB	Checksum manifest and staging verification report

Reproduction

Fine-tuned from facebook/nllb-200-distilled-600M across 544,965 verified parallel pairs (1,089,930 directional examples) using fused AdamW with effective batch size 32,768 target tokens over a two-stage curriculum (1,560 total steps). Evaluated across all six translation directions on the decontaminated 384-sentence held-out test split using sacrebleu 2.6.0. All reported metrics reproduce deterministically from the published checkpoint.

The Name

Taos Amrouche (1913–1976) sang exclusively in Kabyle and wrote in French. She published Jacinthe noire in 1947, the first novel by a Kabyle woman; from 1936 she collected and performed the Kabyle songs her mother had preserved; and she co-founded the Académie berbère in 1966. One of her albums is Chants sauvés de l'oubli — songs saved from oblivion.

A life spent carrying Kabyle into another language without ever surrendering it is what a translation model is for. Her mother, Fadhma Aït Mansour Amrouche, who from 1930 began writing down the songs and tales inherited from her ancestors, gives her name to the speech recognition model — the two halves of one family's work, split the way the two models are.

The naming is homage and implies no endorsement by anyone.

Citation

@software{agbalu_amrouche_2_600m_2026,
  title  = {Amrouche-2-600M: Lightweight Bidirectional Kabyle, English, French, and Arabic Machine Translation},
  author = {AƔBALU},
  year   = {2026},
  url    = {https://huggingface.co/agbalu/Amrouche-2-600M},
  note   = {615,073,792 parameters; 51.34 macro chrF++ across 6 held-out translation directions}
}

Licence

Apache-2.0 on weights and code. Base model weights derive from facebook/nllb-200-distilled-600M (CC-BY-NC-4.0).

The training text composition over the curated parallel training corpus (1,089,930 directional examples) is:

Redistribution	Share
Permissive	88.8%
Unclear	10.9%
Share-alike	0.3%
Non-commercial	0.0%

Part of AƔBALU, a Kabyle and Amazigh corpus and model collection.