Code-Dev/nllb-200-distilled-1.3B-ct2-int8

NLLB-200 Distilled 1.3B - CTranslate2 INT8 Quantization

This is an optimized INT8 quantized version of Meta's facebook/nllb-200-distilled-1.3B generated using ctranslate2.

License

This model inherits the original upstream license: CC-BY-NC-4.0 (Non-Commercial Use Only).

How to use

import ctranslate2
import transformers

translator = ctranslate2.Translator("Code-Dev/nllb-200-distilled-1.3B-ct2-int8", device="cpu") # or "cuda"
tokenizer = transformers.AutoTokenizer.from_pretrained("Code-Dev/nllb-200-distilled-1.3B-ct2-int8")

source = tokenizer.convert_ids_to_tokens(tokenizer.encode("Hello world!"))
results = translator.translate_batch([source], target_prefix=[["spa_Latn"]])
output_tokens = results.hypotheses[0][1:] # Strip language prefix

print(tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens)))