nvidia

/

NV-Embed-v1

like 349

nvm it's not working

Also getting this error.

Thank you for reporting this issue. Can you try to install the required packages as below?

pip uninstall -y transformer-engine
pip install torch==2.2.0
pip install transformers --upgrade
pip install flash-attn==2.2.0
pip install sentence-transformers==2.7.0

Try loading the model like this:

        text_config = {"_name_or_path": "mistralai/Mistral-7B-v0.1"}
        self.model = AutoModel.from_pretrained(
            "nvidia/NV-Embed-v1",
            trust_remote_code=True,
            text_config=text_config,
        )

I have the same issue.

        _hf_model = AutoModel.from_pretrained('nvidia/NV-Embed-v1',
                                              trust_remote_code=True,
                                              text_config=text_config,
                                              use_auth_token=_HF_TOKEN)

results in this error:

Traceback (most recent call last):
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 69, in <module>
    main()
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 60, in main
    embedding = text_embedding_hf(args.text, args.device)
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 39, in text_embedding_hf
    model = _get_hf_model(device)
  File "/dev-volume/vitruvius-backend/shared/room_transformer/text_embeddings.py", line 29, in _get_hf_model
    _hf_model = AutoModel.from_pretrained('nvidia/NV-Embed-v1',
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
    return model_class.from_pretrained(
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3626, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/dev-volume/data/.huggingface/modules/transformers_modules/nvidia/NV-Embed-v1/497fec6bd98c747cc5dad1c5ff5ef3d94dc0de8c/modeling_nvembed.py", line 326, in __init__
    self.tokenizer = AutoTokenizer.from_pretrained(config.text_config._name_or_path) if config.text_config is not None else None
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 837, in from_pretrained
    config = AutoConfig.from_pretrained(
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict
    resolved_config_file = cached_file(
  File "/dev-volume/.cache/pypoetry/virtualenvs/vitruvius-backend-CzLWYIoL-py3.10/lib/python3.10/site-packages/transformers/utils/hub.py", line 417, in cached_file
    raise EnvironmentError(
OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/mistralai/Mistral-7B-v0.1.
401 Client Error. (Request ID: Root=1-66686085-49367655425d427772650e58;2819bcda-07ae-4c58-969a-e6124e1ae416)

Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-v0.1/resolve/main/config.json.
Access to model mistralai/Mistral-7B-v0.1 is restricted. You must be authenticated to access it.

You need to (1) to accept the conditions of mistralai/Mistral-7B-v0.1 and (2) set the HF_TOKEN environment variable.

I'm getting the same access error.
I have access to mistralai/Mistral-7B-v0.1 and can manually download the config.json file.
HF_TOKEN is set

I find the similar issue in mistralai: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1/discussions/2
Here are two possible solutions for this issue.

1. installing transformers from source as follows

pip install --upgrade git+https://github.com/huggingface/transformers

2. Try to updated Transformers and renamed "mistral" to "llama" in config.json file.