Cannot access gated repo for url https://huggingface.co/api/models/mistralai/Mixtral-8x7B-Instruct-v0.1.
Hi everyone,
I'm running mistralai/Mixtral-8x7B-Instruct-v0.1 via TGI, until yesterday everything was working fine. But today I've received this error when executed the docker run command:
Repo model mistralai/Mixtral-8x7B-Instruct-v0.1 is gated. You must be authenticated to access it.
Somebody can help me to fix this issue?
What does it mean? If I have to login into Huggingface hub, How can I do it using TGI? Is any information shared aside user info?
I have the same question when i ma trying to pull the model from vllm.
At this end, I have
" File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/models/mistralai/Mixtral-8x7B-Instruct-v0.1"
They seem to have locked these files behind authentication. It is possible to retrieve if you add a HF token to your request. See more here:
https://huggingface.co/docs/hub/en/security-tokens
https://huggingface.co/docs/transformers.js/en/guides/private
when i get this, I just download them manually via web. But ya, doing it tokens is more efficient
You will need to go back to the model page. Click "Agree and access repository".
Somehow, it still doesn't work, despite agreeing.
accessing other private models does work so it's not the code. Is there some delay or something?
-- Update: it works now, just had to wait a few minutes
I solved the problem generating the HF tokes as @JaUg said then I just added the token as a variable environment in the docker run command.
docker run -d -e HF_TOKEN='XXXXXX' --gpus all --shm-size 1g -p 8080:80 -v /data:/data ghcr.io/huggingface/text-generation-inference:latest --model-id mistralai/Mixtral-8x7B-Instruct-v0.1 ...