safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer

#35
by saivineetha - opened

I have done SFT training on the model with base model as facebook opt-350m. Later saved the trained model and trying to deploy on aws sagemaker. But I'm getting this error "safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer"

How can this error be solved in deployment

import json
from sagemaker.huggingface import HuggingFaceModel

sagemaker config

instance_type = "ml.g5.4xlarge"
number_of_gpu = 1
health_check_timeout = 300

Define Model and Endpoint configuration parameter

config = {
'HF_MODEL_ID': "/opt/ml/model", # path to where sagemaker stores the model
'SM_NUM_GPUS': json.dumps(number_of_gpu), # Number of GPU used per replica
'MAX_INPUT_LENGTH': json.dumps(1024), # Max length of input text
'MAX_TOTAL_TOKENS': json.dumps(2048), # Max length of the generation (including input text)

'HF_MODEL_QUANTIZE': "bitsandbytes",# Comment in to quantize

}

create HuggingFaceModel with the image uri

llm_model = HuggingFaceModel(
role=role,
image_uri=llm_image,
model_data=s3_model_uri,
env=config
)

Deploy model to an endpoint

https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.model.Model.deploy

llm = llm_model.deploy(
initial_instance_count=1,
instance_type=instance_type,

volume_size=400, # If using an instance with local SSD storage, volume_size must be None, e.g. p4 but not p3

container_startup_health_check_timeout=health_check_timeout, # 10 minutes to be able to load the model
)

I've tried this code for deploying. But getting the error "safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer". How can I solve it

I am not familiar with the sagemaker deployment, but google gives some information, and one is

https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10199

Thanks @ydshieh for the help.

Sign up or log in to comment