iproskurina
/

bloom-1b7-gptq-4bit

Text Generation

text-generation-inference

Model card Files Files and versions Community

iproskurina commited on Jan 21

Commit

bf125eb

•

1 Parent(s): 2b8b9ca

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -101,4 +101,4 @@ tokenizer = AutoTokenizer.from_pretrained(pretrained_model_dir, use_fast=True)
 model = AutoGPTQForCausalLM.from_quantized(pretrained_model_dir, device="cuda:0", model_basename="model")
 pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer)
 print(pipeline("auto-gptq is")[0]["generated_text"])
-```b

 model = AutoGPTQForCausalLM.from_quantized(pretrained_model_dir, device="cuda:0", model_basename="model")
 pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer)
 print(pipeline("auto-gptq is")[0]["generated_text"])
+```