Pantogloss 500-to-English Compact

pantogloss-500-en-compact is the recommended compact model for Pantogloss. It translates approximately 500 languages into English using TensorFlow/Keras. The 538 MiB artifact retains FP16 transformer weights while storing the large source embedding as symmetric per-row INT5 with 2,800 exact FP16 override rows.

The artifact is Apache License 2.0. See LICENSE and NOTICE.

Use

Install Pantogloss 0.16 or newer with the appropriate accelerator extra:

python -m pip install "pantogloss[metal]>=0.16"  # Apple Silicon
python -m pip install "pantogloss[cuda]>=0.16"   # NVIDIA CUDA

from pantogloss import Translator

translator = Translator.from_pretrained("pantogloss-500-en-compact")
print(translator.translate("Comment allez-vous ?"))

The Pantogloss runtime selects the validated five-shard UINT8 lookup automatically. On Apple Metal it uses eager packed row lookup, a compiled five-layer encoder prefix, four eager encoder layers, and a compiled cached decoder. CUDA retains the complete compiled path.

Validation

The frozen confirmation evaluation covered 8,250 examples across 50 languages with zero failures. Against the released FP16 baseline, BLEU changed by -0.0460, chrF by -0.0403, and the worst per-language chrF delta was -0.802. Controlled batch-1 and batch-16 throughput and process-memory gates passed on an NVIDIA RTX 3080 Ti Laptop GPU and an Apple M3 Max using TensorFlow Metal.

See the Pantogloss repository for detailed methodology, limitations, language quality tiers, and reproducible evaluation tooling: https://github.com/chrismattmann/pantogloss

Provenance

This model is derived from the original RTG many-to-English checkpoint and the released chrismattmann/pantogloss-500-en-fp16 conversion. Detailed immutable source hashes and conversion metadata are recorded in manifest.json and packed.weights.json.

Downloads last month: -