OSError cached file and config.json

#38
by shivrajhug - opened

I am using Ubuntu 20.04

I followed self deployment on Mistral website: https://docs.mistral.ai/self-deployment/vllm/

I did set HF_TOKEN and then logged in to huggingface cli using command:

$ export HF_TOKEN=<token>

¢ huggingface-cli login

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|
    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|
    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|
    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|
    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.
    Setting a new token will erase the existing one.
    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .
Token:
Add token as git credential? (Y/n) n
Token is valid (permission: write).
Your token has been saved to /home/cloud-user/.cache/huggingface/token
Login successful

I ran command:

$ docker run --gpus all \
    -e HF_TOKEN=$HF_TOKEN -p 8000:8000 \
    ghcr.io/mistralai/mistral-src/vllm:latest \
    --host 0.0.0.0 \
    --model mistralai/Mistral-7B-Instruct-v0.2

I am getting error as below

OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like mistralai/Mistral-7B-Instruct-v0.2 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'

@shivrajhug everything looks good from a HF-authentication point of view. Can it be that the docker settings don't allow the docker container to access your network? (and therefore doesn't have access to the internet?) Only a guess here, hope that can help.

i have the same problem , @shivrajhug have you get the solution of this problem or not yet ?

Guys i have the same error, how to solve it?

same problem, how to solve it?

Me have the same

I dont know dude i cant figure out this error please help

maybe you can try creating a new access token in write type, in this https://huggingface.co/settings/tokens

Sign up or log in to comment