Transformers fail to load deepseek-ai/DeepSeek-V3 with vllm
Author: ajayvohra2005
Pull request: no
Labels: name: bug

### System Info

**transformers_version:** 4.51.2 
**platform**: ubuntu 24.04
**python_version**:  3.12

**Stacktrace:**

```
Failed to initialize Python stub: TypeError: unsupported operand type(s) for /: 'PosixPath' and 'NoneType'

At:
/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py(411): get_cached_module_file 
/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py(558): get_class_from_dynamic_module
/usr/local/lib/python3.12/dist-packages/transformers/models/auto/configuration_auto.py(1123): from_pretrained/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/config.py(315): get_config\n  /usr/local/lib/python3.12/dist-packages/vllm/config.py(423): __init__
/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py(1042): create_model_config  /usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py(1154): create_engine_config
/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py(166): build_async_engine_client_from_engine_args
/usr/lib/python3.12/contextlib.py(211): __aenter__
/opt/tritonserver/backends/vllm/model.py(289): _run_llm_engine
/usr/lib/python3.12/asyncio/events.py(103): _run
/usr/lib/python3.12/asyncio/base_events.py(1988): _run_once
/usr/lib/python3.12/asyncio/base_events.py(649): run_forever
/usr/lib/python3.12/asyncio/base_events.py(687): run_until_complete
/usr/lib/python3.12/asyncio/runners.py(126): run
/usr/lib/python3.12/asyncio/runners.py(193): run
/usr/lib/python3.12/threading.py(1014): run
/usr/lib/python3.12/threading.py(1077): _bootstrap_inner
/usr/lib/python3.12/threading.py(1030): _bootstrap"
```

**Root cause analysis:**

In the code below, `get_cached_module_file` is always being called with `_commit_hash=None` , leading to the error, as the called function, `get_cached_module_file`, does not protect against `_commit_hash` being `None`.  Maybe set `_commit_hash=revision` in `get_cached_module_file`, when `_commit_hash` is passed as `None`.

```
def get_class_from_dynamic_module(
    class_reference: str,
    pretrained_model_name_or_path: Union[str, os.PathLike],
    cache_dir: Optional[Union[str, os.PathLike]] = None,
    force_download: bool = False,
    resume_download: Optional[bool] = None,
    proxies: Optional[dict[str, str]] = None,
    token: Optional[Union[bool, str]] = None,
    revision: Optional[str] = None,
    local_files_only: bool = False,
    repo_type: Optional[str] = None,
    code_revision: Optional[str] = None,
    **kwargs,
) -> type:
    """
    Extracts a class from a module file, present in the local folder or repository of a model.

    <Tip warning={true}>

    Calling this function will execute the code in the module file found locally or downloaded from the Hub. It should
    therefore only be called on trusted repos.

    </Tip>



    Args:
        class_reference (`str`):
            The full name of the class to load, including its module and optionally its repo.
        pretrained_model_name_or_path (`str` or `os.PathLike`):
            This can be either:

            - a string, the *model id* of a pretrained model configuration hosted inside a model repo on
              huggingface.co.
            - a path to a *directory* containing a configuration file saved using the
              [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`.

            This is used when `class_reference` does not specify another repo.
        module_file (`str`):
            The name of the module file containing the class to look for.
        class_name (`str`):
            The name of the class to import in the module.
        cache_dir (`str` or `os.PathLike`, *optional*):
            Path to a directory in which a downloaded pretrained model configuration should be cached if the standard
            cache should not be used.
        force_download (`bool`, *optional*, defaults to `False`):
            Whether or not to force to (re-)download the configuration files and override the cached versions if they
            exist.
        resume_download:
            Deprecated and ignored. All downloads are now resumed by default when possible.
            Will be removed in v5 of Transformers.
        proxies (`Dict[str, str]`, *optional*):
            A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',
            'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.
        token (`str` or `bool`, *optional*):
            The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
            when running `huggingface-cli login` (stored in `~/.huggingface`).
        revision (`str`, *optional*, defaults to `"main"`):
            The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
            git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
            identifier allowed by git.
        local_files_only (`bool`, *optional*, defaults to `False`):
            If `True`, will only try to load the tokenizer configuration from local files.
        repo_type (`str`, *optional*):
            Specify the repo type (useful when downloading from a space for instance).
        code_revision (`str`, *optional*, defaults to `"main"`):
            The specific revision to use for the code on the Hub, if the code leaves in a different repository than the
            rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based system for
            storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git.

    <Tip>

    Passing `token=True` is required when you want to use a private model.

    </Tip>

    Returns:
        `typing.Type`: The class, dynamically imported from the module.

    Examples:

    ```python
    # Download module `modeling.py` from huggingface.co and cache then extract the class `MyBertModel` from this
    # module.
    cls = get_class_from_dynamic_module("modeling.MyBertModel", "sgugger/my-bert-model")

    # Download module `modeling.py` from a given repo and cache then extract the class `MyBertModel` from this
    # module.
    cls = get_class_from_dynamic_module("sgugger/my-bert-model--modeling.MyBertModel", "sgugger/another-bert-model")
    ```"""
    use_auth_token = kwargs.pop("use_auth_token", None)
    if use_auth_token is not None:
        warnings.warn(
            "The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.",
            FutureWarning,
        )
        if token is not None:
            raise ValueError("`token` and `use_auth_token` are both specified. Please set only the argument `token`.")
        token = use_auth_token

    # Catch the name of the repo if it's specified in `class_reference`
    if "--" in class_reference:
        repo_id, class_reference = class_reference.split("--")
    else:
        repo_id = pretrained_model_name_or_path
    module_file, class_name = class_reference.split(".")

    if code_revision is None and pretrained_model_name_or_path == repo_id:
        code_revision = revision
    # And lastly we get the class inside our newly created module
    final_module = get_cached_module_file(
        repo_id,
        module_file + ".py",
        cache_dir=cache_dir,
        force_download=force_download,
        resume_download=resume_download,
        proxies=proxies,
        token=token,
        revision=code_revision,
        local_files_only=local_files_only,
        repo_type=repo_type,  
    )
    return get_class_in_module(class_name, final_module, force_reload=force_download)
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run `deepseek-ai/DeepSeek-V3` model with vLLM, with loading of model directly from hub.

### Expected behavior

No transformers library crash.

Comments:
Hi @ajayvohra2005, can you give us a sample of the code you ran to get this? We generally don't see this error in `transformers`, so I think it might be an issue either in `vLLM` or your code

I don't have any code to share. It is a straight use of vLLM. You can try to load this model with vLLM using openai server, or triton inference server. The reason you may not see this error is because code takes very different pathways depending on how the model is being loaded. 

However, as you may see in the code, there is no check against `_commit_hash` being None, if the code path way vLLM takes is exercised, and it will always cause this error.

Hi @ajayvohra2005, I'm still not sure I see the bug, and other vLLM users haven't reported it! When I inspect our code, it seems like instances of `revision=None` or `_commit_hash=None` are correctly caught and default to the `main` branch of the model repo.

Can you give me a minimal reproducer script or something we can run to see the error?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.