Update README.md

README.md

@@ -103,9 +103,59 @@ language:
 - zu
 license: mit
 tags:
-  - m2m100-12B
+- ctranslate2
 ---
 
+# Fast-Inference with Ctranslate2
+Speedup inference by 2x-8x using int8 inference in C++
+quantized version of facebook/m2m100_1.2B
+
+pip install hf_hub_ctranslate2>=1.0.3 ctranslate2>=3.13.0
+
+```python
+from hf_hub_ctranslate2 import MultiLingualTranslatorCT2fromHfHub
+
+model = MultiLingualTranslatorCT2fromHfHub(
+    model_name_or_path="michaelfeil/ct2fast-m2m100_PARAMS", device="cpu", compute_type="int8",
+    tokenizer=AutoTokenizer.from_pretrained(f"facebook/m2m100_418M")
+)
+
+outputs = model.generate(
+    ["How do you call a fast Flamingo?", "Wie geht es dir?"],
+    src_lang=["en", "de"],
+    tgt_lang=["de", "fr"]
+)
+```
+
+compute_type=int8_float16 for device="cuda"
+compute_type=int8 for device="cpu"
+
+Converted 5/13/23 to Ctranslate2
+```bash
+export ORG="facebook"
+export NAME="m2m100_PARAMS"
+ct2-transformers-converter --model "$ORG/$NAME" --copy_files .gitattributes README.md generation_config.json sentencepiece.bpe.model  special_tokens_map.json tokenizer_config.json vocab.json --quantization float16
+```
+
+Alternative
+
+```python
+import ctranslate2
+import transformers
+
+translator = ctranslate2.Translator("m2m100_PARAMS")
+tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/m2m100_PARAMS")
+tokenizer.src_lang = "en"
+
+source = tokenizer.convert_ids_to_tokens(tokenizer.encode("Hello world!"))
+target_prefix = [tokenizer.lang_code_to_token["de"]]
+results = translator.translate_batch([source], target_prefix=[target_prefix])
+target = results[0].hypotheses[0][1:]
+
+print(tokenizer.decode(tokenizer.convert_tokens_to_ids(target)))
+```
+
+
 # M2M100 12B
 
 M2M100 is a multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation.