Reqwest Error: HTTP status client error (429 Too Many Requests)
Author: yanan1116
State: open

### Describe the bug



full error message:
```
Traceback (most recent call last):
  File "/home/yanan/miniconda3/bin/hf", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/cli/hf.py", line 56, in main
    app()
    ~~~^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/typer/main.py", line 327, in __call__
    raise e
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/typer/main.py", line 310, in __call__
    return get_command(self)(*args, **kwargs)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/typer/core.py", line 803, in main
    return _main(
        self,
    ...<6 lines>...
        **extra,
    )
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/typer/core.py", line 192, in _main
    rv = self.invoke(ctx)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/typer/main.py", line 691, in wrapper
    return callback(**use_params)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/cli/download.py", line 188, in download
    _print_result(run_download())
                  ~~~~~~~~~~~~^^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/cli/download.py", line 149, in run_download
    return snapshot_download(
        repo_id=repo_id,
    ...<10 lines>...
        dry_run=dry_run,
    )
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/_snapshot_download.py", line 451, in snapshot_download
    thread_map(
    ~~~~~~~~~~^
        _inner_hf_hub_download,
        ^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        tqdm_class=tqdm_class,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/tqdm/contrib/concurrent.py", line 69, in thread_map
    return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/tqdm/contrib/concurrent.py", line 51, in _executor_map
    return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs))
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
               ^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ~~~~~~~~~~^^^^^^^^^
  File "/home/yanan/miniconda3/lib/python3.13/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/yanan/miniconda3/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/yanan/miniconda3/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/_snapshot_download.py", line 431, in _inner_hf_hub_download
    hf_hub_download(  # type: ignore
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
        repo_id,
        ^^^^^^^^
    ...<14 lines>...
        dry_run=dry_run,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 986, in hf_hub_download
    return _hf_hub_download_to_local_dir(
        # Destination
    ...<16 lines>...
        dry_run=dry_run,
    )
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1390, in _hf_hub_download_to_local_dir
    _download_to_tmp_and_move(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        incomplete_path=paths.incomplete_path(etag),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<8 lines>...
        tqdm_class=tqdm_class,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 1791, in _download_to_tmp_and_move
    xet_get(
    ~~~~~~~^
        incomplete_path=incomplete_path,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        tqdm_class=tqdm_class,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/yanan/miniconda3/lib/python3.13/site-packages/huggingface_hub/file_download.py", line 571, in xet_get
    download_files(
    ~~~~~~~~~~~~~~^
        xet_download_info,
        ^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        progress_updater=[progress_updater],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
RuntimeError: Data processing error: CAS service error : Reqwest Error: HTTP status client error (429 Too Many Requests), domain: https://cas-server.xethub.hf.co/reconstructions/04b8a4667b84b3b874a6a2f070cec88920f6289e71185d69fa87e3cf29834710
```


### Steps to reproduce the bug

my command 
```bash
hf download nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim --repo-type dataset --include "single_panda_gripper.CoffeePressButton/**" --local-dir /home/yanan/robotics/Isaac-GR00T/gr00t_dataset_official/
```

### Expected behavior

expect the data can be downloaded without any issue

### Environment info

huggingface_hub          1.1.4

Comments: the dataset repo: `https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim`

Hi @yanan1116,

Thanks for the detailed report! However, this issue was filed in the wrong repository. This is a `huggingface_hub` issue, not a `datasets` issue.

Looking at your traceback, you're using the `hf download` CLI command (from `huggingface_hub`), and the error occurs in `huggingface_hub/file_download.py` at line 571 in the `xet_get` function. The `datasets` library is not involved in this download at all.

The 429 error means the CAS (Content Addressable Storage) service at `https://cas-server.xethub.hf.co` is rate-limiting your requests. The `huggingface_hub` library currently doesn't have automatic retry logic for 429 errors from the CAS service.

Please reopen this issue at: https://github.com/huggingface/huggingface_hub/issues