ethzanalytics
/

dolly-v2-7b-sharded

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

pszemraj commited on Apr 23, 2023

Commit

4aed4c8

•

1 Parent(s): e0d26bd

Update README.md

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -35,5 +35,7 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
 model_name = "ethzanalytics/dolly-v2-7b-sharded"
 tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True)
 ```

 model_name = "ethzanalytics/dolly-v2-7b-sharded"
 tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+          model_name, load_in_8bit=True, device_map="auto",
+        )
 ```