Repository: vllm-project/vllm

Issue:
Feature request: Enable activation of `hf_transfer` if available.
A suggestion is to accelerate the usage of `from huggingface_hub import snapshot_download` by using `pip install hf_transfer` https://github.com/huggingface/hf_transfer

hf_transfer is a pretty lightweight binary, that uses rust to download files where python is compule / gil bound while downloading files.

Benefits: download from 200Mbit -> 2GBit (my experience in gcp, us-central) + nice to have if you have to do multiple concurrent downloads (lora)

```python
try:
    # enable hf hub transfer if available
    import hf_transfer  # type: ignore # noqa
    import huggingface_hub.constants  # type: ignore

    # can also be set  via env  var HF_HUB_ENABLE_HF_TRANSFER 
    # I would not suggest doing so, as its unclear if any venv will have 
    # pip install hf_transfer
    huggingface_hub.constants.HF_HUB_ENABLE_HF_TRANSFER = True
except ImportError:
    pass
```

Let me know if this proposal is acceptable, if so, I can open a PR.


Discussion:
Are there any negatives to `hf_transfer` over `snapshot_download`?
https://github.com/huggingface/huggingface_hub/blob/f386b2ae74bf18443836936941ae8bd1bfd40903/docs/source/en/guides/download.md?plain=1#L191

@rib-2 Its a feature flag of snapshot_download

## Faster downloads

If you are running on a machine with high bandwidth,
you can increase your download speed with [`hf_transfer`](https://github.com/huggingface/hf_transfer),
a Rust-based library developed to speed up file transfers with the Hub.
To enable it:

1. Specify the `hf_transfer` extra when installing `huggingface_hub`
   (e.g. `pip install huggingface_hub[hf_transfer]`).
2. Set `HF_HUB_ENABLE_HF_TRANSFER=1` as an environment variable.

`hf_transfer` is a power user tool!
It is tested and production-ready,
but it lacks user-friendly features like advanced error handling or proxies.
For more details, please take a look at this [section](https://huggingface.co/docs/huggingface_hub/hf_transfer).

@robertgshaw2-neuralmagic Should I open a PR?
Update, hf_transfer has been added to #3031 #3008 - does it make sense to use it if available? Using the hook above?

Resolving patch:
diff --git a/vllm/model_executor/weight_utils.py b/vllm/model_executor/weight_utils.py
--- a/vllm/model_executor/weight_utils.py
+++ b/vllm/model_executor/weight_utils.py
@@ -8,6 +8,7 @@
 from typing import Any, Iterator, List, Optional, Tuple
 
 import filelock
+import huggingface_hub.constants
 import numpy as np
 import torch
 from huggingface_hub import HfFileSystem, snapshot_download
@@ -29,6 +30,21 @@
     'TEMP') or os.environ.get('TMP') or "/tmp/"
 
 
+def enable_hf_transfer():
+    """automatically activates hf_transfer
+    """
+    if "HF_HUB_ENABLE_HF_TRANSFER" not in os.environ:
+        try:
+            # enable hf hub transfer if available
+            import hf_transfer  # type: ignore # noqa
+            huggingface_hub.constants.HF_HUB_ENABLE_HF_TRANSFER = True
+        except ImportError:
+            pass
+
+
+enable_hf_transfer()
+
+
 class Disabledtqdm(tqdm):
 
     def __init__(self, *args, **kwargs):


Test patch:
diff --git a/tests/model_executor/weight_utils.py b/tests/model_executor/weight_utils.py
new file mode 100644
--- /dev/null
+++ b/tests/model_executor/weight_utils.py
@@ -0,0 +1,26 @@
+import os
+
+import huggingface_hub.constants
+import pytest
+
+from vllm.model_executor.weight_utils import enable_hf_transfer
+
+
+def test_hf_transfer_auto_activation():
+    if "HF_HUB_ENABLE_HF_TRANSFER" in os.environ:
+        # in case it is already set, we can't test the auto activation
+        pytest.skip(
+            "HF_HUB_ENABLE_HF_TRANSFER is set, can't test auto activation")
+    enable_hf_transfer()
+    try:
+        # enable hf hub transfer if available
+        import hf_transfer  # type: ignore # noqa
+        HF_TRANFER_ACTIVE = True
+    except ImportError:
+        HF_TRANFER_ACTIVE = False
+    assert (huggingface_hub.constants.HF_HUB_ENABLE_HF_TRANSFER ==
+            HF_TRANFER_ACTIVE)
+
+
+if __name__ == "__main__":
+    test_hf_transfer_auto_activation()