Update README.md

README.md

@@ -106,15 +106,38 @@ tags:
 - ctranslate2
 ---
 
+# Fast-Inference with Ctranslate2
+Speedup inference by 2x-8x using int8 inference in C++
+quantized version of facebook/m2m100_1.2B
+
+pip install hf_hub_ctranslate2>=1.0.3 ctranslate2>=3.13.0
+
+```python
+from hf_hub_ctranslate2 import MultiLingualTranslatorCT2fromHfHub
+
+model = MultiLingualTranslatorCT2fromHfHub(
+    model_name_or_path="michaelfeil/ct2fast-m2m100_PARAMS", device="cpu", compute_type="int8",
+    tokenizer=AutoTokenizer.from_pretrained(f"facebook/m2m100_418M")
+)
+
+outputs = model.generate(
+    ["How do you call a fast Flamingo?", "Wie geht es dir?"],
+    src_lang=["en", "de"],
+    tgt_lang=["de", "fr"]
+)
+```
+
+compute_type=int8_float16 for device="cuda"
+compute_type=int8 for device="cpu"
+
 Converted 5/13/23 to Ctranslate2
 ```bash
 export ORG="facebook"
 export NAME="m2m100_PARAMS"
 ct2-transformers-converter --model "$ORG/$NAME" --copy_files .gitattributes README.md generation_config.json sentencepiece.bpe.model  special_tokens_map.json tokenizer_config.json vocab.json --quantization float16
 ```
-Fast-Inference with Ctranslate2
-Speedup inference by 2x-8x using int8 inference in C++
-quantized version of facebook/m2m100_1.2B
+
+Alternative
 
 ```python
 import ctranslate2
@@ -132,33 +155,7 @@ target = results[0].hypotheses[0][1:]
 print(tokenizer.decode(tokenizer.convert_tokens_to_ids(target)))
 ```
 
-Alternative:
-pip install hf_hub_ctranslate2>=1.0.0 ctranslate2>=3.13.0
-
-Checkpoint compatible to ctranslate2 and hf-hub-ctranslate2
 
-compute_type=int8_float16 for device="cuda"
-compute_type=int8 for device="cpu"
-```python
-from hf_hub_ctranslate2 import TranslatorCT2fromHfHub, GeneratorCT2fromHfHub
-
-model_name = "michaelfeil/ct2fast-m2m100_PARAMS"
-model = TranslatorCT2fromHfHub(
-        # load in int8 on CUDA
-        model_name_or_path=model_name, 
-        device="cuda",
-        compute_type="int8_float16"
-)
-model.tokenizer = AutoTokenizer.from_pretrained("facebook/m2m100_PARAMS")
-outputs = model.generate(
-    text=["Translate to german: How are you doing?"],
-    min_decoding_length=24,
-    max_decoding_length=32,
-    max_input_length=512,
-    beam_size=5
-)
-print(outputs)
-```
 # Original: M2M100 418M
 
 M2M100 is a multilingual encoder-decoder (seq-to-seq) model trained for Many-to-Many multilingual translation.