open_llama_3b_v2-ds

https://huggingface.co/openlm-research/open_llama_3b_v2

Inference

import deepsparse

model = deepsparse.TextGeneration(model="hf:mgoin/open_llama_3b_v2-ds")
model("Tell me a joke.")

Export

git clone https://huggingface.co/openlm-research/open_llama_3b_v2
sparseml.transformers.export_onnx --model_path ./open_llama_3b_v2 --task text-generation --sequence_length 512