text-generation-inference

/

Mixtral-8x7B-Instruct-v0.1-medusa

like 16

Follow

Text Generation Inference 109

Hello everyone, I wanted to try this model on a sagemaker endpoint but I get the following error:
"huggingface_hub.utils._errors.LocalEntryNotFoundError: File model-00001-of-00019.safetensors of model mistralai/Mixtral-8x7B-Instruct-v0.1 not found in /tmp. Please run text-generation-server download-weights mistralai/Mixtral-8x7B-Instruct-v0.1 first."
What would it be the correct way to download the weights of mixtral during the deployement of the medusa version? thank you very much