zenoverflow commited on
Commit
95c35d5
1 Parent(s): a246cc5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -0
README.md CHANGED
@@ -1,3 +1,32 @@
1
  ---
2
  license: apache-2.0
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: translation
4
  ---
5
+
6
+ Quantization of [madlad400-3b-mt](https://huggingface.co/google/madlad400-3b-mt) using [Ctranslate2](https://github.com/OpenNMT/CTranslate2) for running on CPU.
7
+
8
+ Example usage:
9
+
10
+ ```python
11
+ import ctranslate2, transformers
12
+ from huggingface_hub import snapshot_download
13
+
14
+ model_path = snapshot_download("zenoverflow/madlad400-3b-mt-int8-float32")
15
+ print("\n", end="")
16
+
17
+ translator = ctranslate2.Translator(model_path, device="cpu")
18
+ tokenizer = transformers.T5Tokenizer.from_pretrained(model_path)
19
+
20
+ target_lang_code = "ja"
21
+
22
+ source_text = "This sentence has no meaning."
23
+
24
+ input_text = f"<2{target_lang_code}> {source_text}"
25
+ input_tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(input_text))
26
+ results = translator.translate_batch([input_tokens])
27
+ output_tokens = results[0].hypotheses[0]
28
+ output_text = tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens))
29
+
30
+ print(output_text)
31
+
32
+ ```