ikeno-ada
/

m2m100_418-mt-bitsandbytes-4bit

text2text-generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

ikeno-ada commited on Apr 3

Commit

cbda76b

•

1 Parent(s): 10ba63b

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -123,11 +123,12 @@ To install `sentencepiece` run `pip install sentencepiece bitsandbytes`
 ```python
 from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
 hi_text = "जीवन एक चॉकलेट बॉक्स की तरह है।"
 chinese_text = "生活就像一盒巧克力。"
-model = M2M100ForConditionalGeneration.from_pretrained("ikeno-ada/m2m100_418-mt-bitsandbytes-4bit")
 tokenizer = M2M100Tokenizer.from_pretrained("ikeno-ada/m2m100_418-mt-bitsandbytes-4bit")
 # translate Hindi to French

 ```python
 from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
+import torch
 hi_text = "जीवन एक चॉकलेट बॉक्स की तरह है।"
 chinese_text = "生活就像一盒巧克力。"
+model = M2M100ForConditionalGeneration.from_pretrained("ikeno-ada/m2m100_418-mt-bitsandbytes-4bit",device_map= "auto",torch_dtype=torch.bfloat16)
 tokenizer = M2M100Tokenizer.from_pretrained("ikeno-ada/m2m100_418-mt-bitsandbytes-4bit")
 # translate Hindi to French