deepseek-ai

/

deepseek-coder-33b-instruct

like 411

Hi, I tried to deploy this model in Inference Endpoints using the Text Generation Inference , however it fails to start. When I see the logs, I found that it detects this model as PEFT model and try to load model with the AutoPEFT class. As we know this model doesn’t have any adapter weights.

Error logs

2023/12/23 17:15:48 ~ {"timestamp":"2023-12-23T10:15:48.081726Z","level":"INFO","fields":{"message":"Peft model detected.\n"},"target":"text_generation_launcher"}
2023/12/23 17:15:48 ~ {"timestamp":"2023-12-23T10:15:48.081759Z","level":"INFO","fields":{"message":"Loading the model it might take a while without feedback\n"},"target":"text_generation_launcher"}
2023/12/23 17:15:48 ~ {"timestamp":"2023-12-23T10:15:48.612560Z","level":"ERROR","fields":{"message":"Download encountered an error: Traceback (most recent call last):\n\n File \"/opt/conda/lib/python3.10/site-packages/peft/utils/config.py\", line 117, in from_pretrained\n config_file = hf_hub_download(\n\n File \"/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py\", line 110, in _inner_fn\n validate_repo_id(arg_value)\n\n File \"/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py\", line 164, in validate_repo_id\n raise HFValidationError(\n\nhuggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: '/repository'.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File \"/opt/conda/lib/python3.10/site-packages/text_generation_server/utils/peft.py\", line 16, in download_and_unload_peft\n model = AutoPeftModelForCausalLM.from_pretrained(\n\n File \"/opt/conda/lib/python3.10/site-packages/peft/auto.py\", line 69, in from_pretrained\n peft_config = PeftConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)\n\n File \"/opt/conda/lib/python3.10/site-packages/peft/utils/config.py\", line 121, in from_pretrained\n raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'\")\n\nValueError: Can't find 'adapter_config.json' at '/repository'\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File \"/opt/conda/lib/python3.10/site-packages/peft/utils/config.py\", line 117, in from_pretrained\n config_file = hf_hub_download(\n\n File \"/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py\", line 110, in _inner_fn\n validate_repo_id(arg_value)\n\n File \"/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py\", line 164, in validate_repo_id\n raise HFValidationError(\n\nhuggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: '/repository'.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n File \"/opt/conda/bin/text-generation-server\", line 8, in <module>\n sys.exit(app())\n\n File \"/opt/conda/lib/python3.10/site-packages/text_generation_server/cli.py\", line 204, in download_weights\n utils.download_and_unload_peft(\n\n File \"/opt/conda/lib/python3.10/site-packages/text_generation_server/utils/peft.py\", line 24, in download_and_unload_peft\n model = AutoPeftModelForSeq2SeqLM.from_pretrained(\n\n File \"/opt/conda/lib/python3.10/site-packages/peft/auto.py\", line 69, in from_pretrained\n peft_config = PeftConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)\n\n File \"/opt/conda/lib/python3.10/site-packages/peft/utils/config.py\", line 121, in from_pretrained\n raise ValueError(f\"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'\")\n\nValueError: Can't find 'adapter_config.json' at '/repository'\n\n"},"target":"text_generation_launcher","span":{"name":"download"},"spans":[{"name":"download"}]}
2023/12/23 17:15:48 ~ Error: DownloadError

Note: I already changed the container config for Quantization as None