Model not compiled warning

#2
by charlescearl - opened

Hi,

I have followed the commands given in https://huggingface.co/blog/text-generation-inference-on-inferentia2 but I am still getting the message

WARNING:sagemaker:Your model is not compiled. Please compile your model before using Inferentia.

when attempting to run the deploy line:

llm = llm_model.deploy(
        initial_instance_count=1,
        instance_type=instance_type,
        container_startup_health_check_timeout=health_check_timeout,
    )

Sign up or log in to comment