Deployment details
- Needed some information on how to deploy this model in AWS Sagemaker or an ec2 instance and use it through an API
@thekiran Let me know if this tutorial helps: https://medium.com/@joaopcmoura/lora-serving-on-amazon-sagemaker-serve-100s-of-fine-tuned-llms-for-the-price-of-1-85034ef889c5
thanks. will check it out!
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/models/mistralai/Mistral-7B-Instruct-v0.1
Cannot access gated repo for url https://huggingface.co/api/models/mistralai/Mistral-7B-Instruct-v0.1.
i get this error when i deploy the model
not sure how to add my hugging face token details, could you please help
@thekiran Have you requested access to Mistral 7B Instruct?
yes. im not sure where to add the HF token in the above medium code. could you please help me out with that
You can just set the HUGGING_FACE_HUB_TOKEN environment variable to your HF token and you should be good to go