OSError cached file and config.json
I am using Ubuntu 20.04
I followed self deployment on Mistral website: https://docs.mistral.ai/self-deployment/vllm/
I did set HF_TOKEN and then logged in to huggingface cli using command:
$ export HF_TOKEN=<token>
¢ huggingface-cli login
_| _| _| _| _|_|_| _|_|_| _|_|_| _| _| _|_|_| _|_|_|_| _|_| _|_|_| _|_|_|_|
_| _| _| _| _| _| _| _|_| _| _| _| _| _| _| _|
_|_|_|_| _| _| _| _|_| _| _|_| _| _| _| _| _| _|_| _|_|_| _|_|_|_| _| _|_|_|
_| _| _| _| _| _| _| _| _| _| _|_| _| _| _| _| _| _| _|
_| _| _|_| _|_|_| _|_|_| _|_|_| _| _| _|_|_| _| _| _| _|_|_| _|_|_|_|
A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.
Setting a new token will erase the existing one.
To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .
Token:
Add token as git credential? (Y/n) n
Token is valid (permission: write).
Your token has been saved to /home/cloud-user/.cache/huggingface/token
Login successful
I ran command:
$ docker run --gpus all \
-e HF_TOKEN=$HF_TOKEN -p 8000:8000 \
ghcr.io/mistralai/mistral-src/vllm:latest \
--host 0.0.0.0 \
--model mistralai/Mistral-7B-Instruct-v0.2
I am getting error as below
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like mistralai/Mistral-7B-Instruct-v0.2 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'
@shivrajhug everything looks good from a HF-authentication point of view. Can it be that the docker settings don't allow the docker container to access your network? (and therefore doesn't have access to the internet?) Only a guess here, hope that can help.
i have the same problem , @shivrajhug have you get the solution of this problem or not yet ?
Guys i have the same error, how to solve it?
same problem, how to solve it?
Me have the same
I dont know dude i cant figure out this error please help
maybe you can try creating a new access token in write type, in this https://huggingface.co/settings/tokens