Code-Dev/nllb-200-distilled-600M-ct2-int8

NLLB-200 Distilled 600M - CTranslate2 INT8 Quantization

This is an optimized, ultra-lightweight INT8 quantized version of Meta's facebook/nllb-200-distilled-600M compiled using ctranslate2. It brings the processing footprint down to ~600MB for edge compute deployments.

License

This model inherits the original upstream license terms: CC-BY-NC-4.0 (Non-Commercial Use Only).

Quickstart Execution Script

import ctranslate2
import transformers

# Pull and initialize directly from the hub profile path
translator = ctranslate2.Translator("Code-Dev/nllb-200-distilled-600M-ct2-int8", device="cpu") # Switch to "cuda" if using GPU
tokenizer = transformers.AutoTokenizer.from_pretrained("Code-Dev/nllb-200-distilled-600M-ct2-int8")

source = tokenizer.convert_ids_to_tokens(tokenizer.encode("Hello world!"))
results = translator.translate_batch([source], target_prefix=[["spa_Latn"]])

output_tokens = results.hypotheses[1:]
print(tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens)))