PrunaAI
/

recurrentgemma-2b-bnb-4bit-smashed

Text Generation

recurrent_gemma

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

johnrachwanpruna commited on Apr 10

Commit

1b9d2bf

•

1 Parent(s): 5a5dea1

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -57,7 +57,7 @@ You can run the smashed model with these steps:
       from transformers import AutoTokenizer, AutoModelForCausalLM
       tokenizer = AutoTokenizer.from_pretrained("PrunaAI/recurrentgemma-2b-bnb-4bit-smashed")
-      model = AutoModelForCausalLM.from_pretrained("PrunaAI/recurrentgemma-2b-bnb-4bit-smashed", device_map="auto")
       input_text = "Write me a poem about Machine Learning."
       input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

       from transformers import AutoTokenizer, AutoModelForCausalLM
       tokenizer = AutoTokenizer.from_pretrained("PrunaAI/recurrentgemma-2b-bnb-4bit-smashed")
+      model = AutoModelForCausalLM.from_pretrained("PrunaAI/recurrentgemma-2b-bnb-4bit-smashed")
       input_text = "Write me a poem about Machine Learning."
       input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")