tiiuae

/

falcon-7b-instruct

like 899

I'm trying to deploy the model to Amazon SageMaker via the HuggingFaceModel class. When I try to use the code snippet provided with this model, I can deploy the endpoint, however the predict query fails with the following error:

"{
  "code": 400,
  "type": "InternalServerException",
  "message": "Loading /.sagemaker/mms/models/tiiuae__falcon-7b-instruct requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code\u003dTrue` to remove this error."
}"

I tried passing myself the trust_remote_code=True in the Pipeline code with a custom inference python script, but nothing changes - which makes me think that it just gets ignored by the Multi-Model Server in SageMaker.

Any suggestions on how to solve it?