neuralmagic
/

mpt-7b-chat-pruned50-quant-ds

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mgoin commited on Oct 5, 2023

Commit

261e43f

•

1 Parent(s): 6a8055c

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -7,7 +7,7 @@ tags:
 Chat-aligned MPT 7b model pruned to 50% and quantized for inference with DeepSparse
 ```python
-import deepsparse
-model = deepsparse.TextGeneration(model="hf:mgoin/mpt-7b-chat-50pruned-quant")
 model("Tell me a joke.", max_new_tokens=50)
 ```

 Chat-aligned MPT 7b model pruned to 50% and quantized for inference with DeepSparse
 ```python
+from deepsparse import TextGeneration
+model = TextGeneration(model="hf:neuralmagic/mpt-7b-chat-pruned50-quant")
 model("Tell me a joke.", max_new_tokens=50)
 ```