File size: 379 Bytes
2943a4e 8e482d3 170b852 a1b59e5 170b852 24856d0 261e43f c0152d0 24856d0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 |
---
tags:
- deepsparse
---
# Sparse MPT-7B-Chat - DeepSparse
[Chat-aligned MPT 7b model](https://huggingface.co/mosaicml/mpt-7b-chat) pruned to 50% and quantized using SparseGPT for inference with DeepSparse
```python
from deepsparse import TextGeneration
model = TextGeneration(model="hf:neuralmagic/mpt-7b-chat-pruned50-quant")
model("Tell me a joke.", max_new_tokens=50)
``` |