SageMaker deployment problem
#4
by
PAPAT4
- opened
Thanks for sharing a way to deploy llava to SageMaker. I’m still having difficulties, though. Following deploy_llava.ipynb, when I make a request from the predictor, I receive an error that env needs to be set in the HuggingFaceModel. I tried setting env={‘HF_MODEL_ID':'anymodality/llava-v1.5-7b', ‘HF_TASK’:'visual-question-answering'}, but that produces an error:
"code": 400,
"type": “InternalServerException",
"message": "\u0027llava\u0027"
Do you know what could be causing this problem?
env={‘HF_MODEL_ID':'anymodality/llava-v1.5-7b', ‘HF_TASK’:'visual-question-answering'} this one is not able to use because the model is not in HUB.
I believe there is some other reason for your issue.