mistralai

/

Codestral-22B-v0.1

like 1.05k

The Deploy to SageMaker instructions has

predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g5.2xlarge",
    container_startup_health_check_timeout=300,
  )

But it seems to run out of memory on "ml.g5.2xlarge".

What is the correct minimum instance type to get the model up and running in sagemaker?