luigi000/envit5-translation-ct2-int8

envit5-translation — CTranslate2 int8

An int8 CTranslate2 conversion of VietAI/envit5-translation (275M T5, English ↔ Vietnamese).

Quality (measured on this int8 artifact)

FLORES-200 devtest, 300 sentences, CPU, beam 1: vi→en chrF2 64.9, en→vi chrF2 60.9. Note: greedy decoding (beam 1) scores higher than beam 4 on chrF for this model (beam search drifts paraphrastic); COMET-22 rates the two beams equal, so beam 1 is the recommended operating point.

Usage

Inputs take a vi: / en: source-language prefix; the model may echo a target prefix which should be stripped:

import ctranslate2, sentencepiece as spm, re

translator = ctranslate2.Translator("model_dir", compute_type="int8")
sp = spm.SentencePieceProcessor("model_dir/spiece.model")

tokens = sp.encode("vi: Cuộc sống của một nhân viên ngân hàng rất khó khăn.", out_type=str) + ["</s>"]
result = translator.translate_batch([tokens], beam_size=1)
text = sp.decode(result[0].hypotheses[0])
print(re.sub(r"^(?:\S{1,3}\s+)?(en|vi)\s*:\s*", "", text))

Attribution & license

Original model by VietAI (paper: MTet: Multi-domain Translation for English and Vietnamese), released under an OpenRAIL license; the same license and its use restrictions apply to this quantized redistribution.