|
--- |
|
license: apache-2.0 |
|
pipeline_tag: translation |
|
inference: false |
|
--- |
|
|
|
Quantization of [madlad400-10b-mt](https://huggingface.co/google/madlad400-10b-mt) using [Ctranslate2](https://github.com/OpenNMT/CTranslate2) for running on CPU or Nvidia GPU. |
|
|
|
Example usage: |
|
|
|
```python |
|
import ctranslate2, transformers |
|
from huggingface_hub import snapshot_download |
|
|
|
model_path = snapshot_download("zenoverflow/madlad400-10b-mt-ct2-int8-float16") |
|
print("\n", end="") |
|
|
|
translator = ctranslate2.Translator(model_path, device="auto") |
|
tokenizer = transformers.T5Tokenizer.from_pretrained(model_path) |
|
|
|
target_lang_code = "ja" |
|
|
|
source_text = "This sentence has no meaning." |
|
|
|
input_text = f"<2{target_lang_code}> {source_text}" |
|
input_tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(input_text)) |
|
results = translator.translate_batch([input_tokens]) |
|
output_tokens = results[0].hypotheses[0] |
|
output_text = tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens)) |
|
|
|
print(output_text) |
|
|
|
``` |
