Adapt and test huggingface_hub v1.0.0
Author: Wauplin
Pull request: yes

Test as part of https://github.com/huggingface/huggingface_hub/issues/3340

**The main changes in `huggingface_hub` impacting transformers are:**
- `Repository` removed
- `HfFolder` removed
- migrated to `httpx` instead of `requests`

---

**List of changes made in this PR:**
-  removed all imports that were commented with `# for backward compatibility`
- replaced `requests` exceptions by `httpx` ones in testing utils
- `Trainer` returns a `CommitInfo` (which is a subclass of `str`) => was already the case before but not type annotated
- `allow_redirects` => renamed `follow_redirects` in httpx
- `proxies` => argument is now ignored. Proxies must be set globally via env variable, not in script. If user passes a proxies, a warning is emitted and the process continues (ignoring the proxy)
- all `requests.HTTPError` are replaced by some `huggingface_hub.HfHubHTTPError` (which inherits from `httpx.HTTPError`)
- removed `test_dynamic_saving_from_local_repo` as it was commented as "# to be removed when huggingface_hub v1 comes out"
- removed all `HfFolder` occurrences from the tests => they were not relevant anyway
- migrated most "Repository clone from" by `snapshot_download` and "Repository push_to_hub" by `upload_folder`
- removed `ModelFilter` from `utils/update_tiny_models.py` (to be fair it has been removed since quite some time so this script is currently broken on `main`)
- switched a lot of `requests.get` into `httpx.get` in the code. However:
   - it'd be very beneficial to define a single client for them to benefit from connection pooling => that's out of scope for this PR
   - a LOT of `requests` calls is happening in `./models/` and `test_models/` so I won't change them => it'd be good in `transformers` v5 to get rid of `requests` dependency entirely => also out of scope for this PR
---

**TODO: (later?)**
- [x] adapt `utils/create_dummy_models.py` => currently using `Repository` https://github.com/huggingface/transformers/pull/42380
- [x] ~remove back manual install of huggingface_hub 1.0.0.rc0 once `tokenizers` is out~
- [x] ~what to do with KerasCallback? => got deleted~
- [x] in v5, all `use_auth_token` logic must be removed (currently throws warnings) https://github.com/huggingface/transformers/pull/41666



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40889). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> python -m pytest -v tests/models/zoedepth/test_image_processing_zoedepth.py::ZoeDepthImageProcessingTest::test_slow_fast_equivalence


The following failure is related (see below)

>      dummy_image = Image.open(
            httpx.get("http://images.cocodataset.org/val2017/000000039769.jpg", follow_redirects=True).content
        )

Is `httpx` works well with ` Image.open` or in general with image?

The same failure happens many times in the job `tests_processors`

https://app.circleci.com/pipelines/github/huggingface/transformers/146796/workflows/8cffc93e-ca8e-4fd3-b01b-984033ee4ac1/jobs/1938513/parallel-runs/7?filterBy=FAILED

```
python -m pytest -v tests/models/zoedepth/test_image_processing_zoedepth.py::ZoeDepthImageProcessingTest::test_slow_fast_equivalence

============================================================================================ FAILURES =============================================================================================
_____________________________________________________________________ ZoeDepthImageProcessingTest.test_slow_fast_equivalence ______________________________________________________________________

self = <tests.models.zoedepth.test_image_processing_zoedepth.ZoeDepthImageProcessingTest testMethod=test_slow_fast_equivalence>

    @require_vision
    @require_torch
    def test_slow_fast_equivalence(self):
        if not self.test_slow_image_processor or not self.test_fast_image_processor:
            self.skipTest(reason="Skipping slow/fast equivalence test")

        if self.image_processing_class is None or self.fast_image_processing_class is None:
            self.skipTest(reason="Skipping slow/fast equivalence test as one of the image processors is not defined")

>       dummy_image = Image.open(
            httpx.get("http://images.cocodataset.org/val2017/000000039769.jpg", follow_redirects=True).content
        )

tests/test_image_processing_common.py:184: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.9/site-packages/PIL/Image.py:3466: in open
    filename = os.path.realpath(os.fspath(fp))
/usr/local/lib/python3.9/posixpath.py:392: in realpath
    path, ok = _joinrealpath(filename[:0], filename, strict, {})
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

path = b''
rest = b'\\)\xa4\x81\x17\xb5\xac\x97\xe9\x96\x98\nZh\x01U@*6\xb1\xdc\x1ew\x00\x0e\x9c\xbfLX\x12\x1ac\x08\xc9\x1d\xf0T\x86\x0b...f\x00\xd3Ff \x10v\xdb\xad\x85\xaf\xd0\x0e\x9b\xe2\x8d\xa7\xd4v``\xba\xbd\xd6s\xfe\xa3\xb3\xb6\xa5\xc3Z\xea\xbf\xff\xd9'
strict = False, seen = {}

    def _joinrealpath(path, rest, strict, seen):
        if isinstance(path, bytes):
            sep = b'/'
            curdir = b'.'
            pardir = b'..'
        else:
            sep = '/'
            curdir = '.'
            pardir = '..'
            getcwd = os.getcwd
        if strict is ALLOW_MISSING:
            ignored_error = FileNotFoundError
        elif strict:
            ignored_error = ()
        else:
            ignored_error = OSError

        maxlinks = None

        if isabs(rest):
            rest = rest[1:]
            path = sep

        while rest:
            name, _, rest = rest.partition(sep)
            if not name or name == curdir:
                # current dir
                continue
            if name == pardir:
                # parent dir
                if path:
                    path, name = split(path)
                    if name == pardir:
                        path = join(path, pardir, pardir)
                else:
                    path = pardir
                continue
            newpath = join(path, name)
            try:
>               st = os.lstat(newpath)
E               ValueError: lstat: embedded null character in path

/usr/local/lib/python3.9/posixpath.py:436: ValueError
===================================================================================== short test summary info =====================================================================================
FAILED tests/models/zoedepth/test_image_processing_zoedepth.py::ZoeDepthImageProcessingTest::test_slow_fast_equivalence - ValueError: lstat: embedded null character in path

```

```
python -m pytest -v tests/utils/test_tokenization_utils.py::TokenizerUtilTester::test_cached_files_are_used_when_internet_is_down

python -m pytest -v tests/utils/test_tokenization_utils.py::TokenizerUtilTester::test_cached_files_are_used_when_internet_is_down_missing_files
```
are also related to this PR.

```
============================================================================================ FAILURES =============================================================================================
______________________________________________________________ TokenizerUtilTester.test_cached_files_are_used_when_internet_is_down _______________________________________________________________

self = <tests.utils.test_tokenization_utils.TokenizerUtilTester testMethod=test_cached_files_are_used_when_internet_is_down>

    def test_cached_files_are_used_when_internet_is_down(self):
        # A mock response for an HTTP head request to emulate server down
        response_mock = mock.Mock()
        response_mock.status_code = 500
        response_mock.headers = {}
        response_mock.raise_for_status.side_effect = httpx.HTTPError("failed")
        response_mock.json.return_value = {}

        # Download this model to make sure it's in the cache.
        _ = BertTokenizer.from_pretrained("hf-internal-testing/tiny-random-bert")

        # Under the mock environment we get a 500 error when trying to reach the tokenizer.
        with mock.patch("httpx.Client.request", return_value=response_mock) as mock_head:
>           _ = BertTokenizer.from_pretrained("hf-internal-testing/tiny-random-bert")

tests/utils/test_tokenization_utils.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/transformers/tokenization_utils_base.py:2043: in from_pretrained
    for template in list_repo_templates(
src/transformers/utils/hub.py:166: in list_repo_templates
    return [
src/transformers/utils/hub.py:166: in <listcomp>
    return [
/usr/local/lib/python3.9/site-packages/huggingface_hub/hf_api.py:3083: in list_repo_tree
    for path_info in paginate(path=tree_url, headers=headers, params={"recursive": recursive, "expand": expand}):
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_pagination.py:37: in paginate
    hf_raise_for_status(r)
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_http.py:560: in hf_raise_for_status
    response.raise_for_status()
/usr/local/lib/python3.9/unittest/mock.py:1092: in __call__
    return self._mock_call(*args, **kwargs)
/usr/local/lib/python3.9/unittest/mock.py:1096: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
/usr/local/lib/python3.9/unittest/mock.py:1151: in _execute_mock_call
    raise effect
src/transformers/utils/hub.py:477: in cached_files
    hf_hub_download(
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py:89: in _inner_fn
    return fn(*args, **kwargs)
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:937: in hf_hub_download
    return _hf_hub_download_to_cache_dir(
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:998: in _hf_hub_download_to_cache_dir
    (url_to_download, etag, commit_hash, expected_size, xet_file_data, head_call_error) = _get_metadata_or_catch_error(
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:1453: in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py:89: in _inner_fn
    return fn(*args, **kwargs)
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:1379: in get_hf_file_metadata
    response = _httpx_follow_relative_redirects(method="HEAD", url=url, headers=hf_headers, timeout=timeout)
/usr/local/lib/python3.9/site-packages/huggingface_hub/file_download.py:266: in _httpx_follow_relative_redirects
    hf_raise_for_status(response)
/usr/local/lib/python3.9/site-packages/huggingface_hub/utils/_http.py:560: in hf_raise_for_status
    response.raise_for_status()
/usr/local/lib/python3.9/unittest/mock.py:1092: in __call__
    return self._mock_call(*args, **kwargs)
/usr/local/lib/python3.9/unittest/mock.py:1096: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Mock name='mock.raise_for_status' id='140314207904624'>, args = (), kwargs = {}, effect = HTTPError('failed')

    def _execute_mock_call(self, /, *args, **kwargs):
        # separate from _increment_mock_call so that awaited functions are
        # executed separately from their call, also AsyncMock overrides this method

        effect = self.side_effect
        if effect is not None:
            if _is_exception(effect):
>               raise effect
E               httpx.HTTPError: failed

/usr/local/lib/python3.9/unittest/mock.py:1151: HTTPError
======================================================================================== warnings summary =========================================================================================
<frozen importlib._bootstrap>:228
  <frozen importlib._bootstrap>:228: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:228
  <frozen importlib._bootstrap>:228: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================================== short test summary info =====================================================================================
FAILED tests/utils/test_tokenization_utils.py::TokenizerUtilTester::test_cached_files_are_used_when_internet_is_down - httpx.HTTPError: failed
================================================================================== 1 failed, 2 warnings in 0.91s ==================================================================================
```

> python -m pytest -v  tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_no_dist

This pass when I ssh into a CircleCI runner. Could ignore for now and will check later

There are some

> AttributeError: module 'huggingface_hub.hf_api' has no attribute 'HfFolder'

for example

> python -m pytest -v examples/pytorch/test_accelerate_examples.py::ExamplesTestsNoTrainer::test_run_swag_no_trainer


But I'm not able to reproduce when I ssh into a CircleCI runner. Could check later.

Thanks a lot @ydshieh! :heart:  This was very helpful. It looks like some errors have been resolved by themselves (I did a rebase from `main` so maybe it helped^^). For the rest, I addressed them in https://github.com/huggingface/transformers/pull/40889/commits/b45ebc764efcfe92ce08437dde3d92b37949cbc8 https://github.com/huggingface/transformers/pull/40889/commits/14e99075c4c1ceda6836809c9b1668e42c59e6a9 and https://github.com/huggingface/transformers/pull/40889/commits/e7c94d58764ad63d3b8c192443b866d1b07c18f8.

And now all tests are passing! :heavy_check_mark: :tada: 
<img width="943" height="205" alt="image" src="https://github.com/user-attachments/assets/f96a4517-e415-46c9-b2ed-8d2f3b3db896" />


Documentation is still failing but I've opened https://github.com/huggingface/kernels/pulls which should fix it. For the 2 other checks, I don't know if they are relevant.

Hey @huggingface/transformers-core-maintainers , this PR is finally ready for review! All tests are now passing. Please read PR description to find a list of all changes made in this PR which will prepare for transformers v5 and huggingface_hub v1. 

Regarding the failing docs CI, it requires a new version of https://github.com/huggingface/kernels to be released on PyPI now that https://github.com/huggingface/kernels/pull/151 has been merged. @danieldk @LysandreJik Would it be possible to make a patch release for it? :pray: 

Other than that, let me know if you have any questions!

@Wauplin Would you like me to trigger a complete CI on Github Actions?

If so, could you check 

> .github/workflows/model_jobs.yml

before the step

>       - name: Run all tests on GPU

what command(s) to bed added in order to test against this `huggingface_hub` RC ?

I'm also try to switch to python 3.10 for CI (CircleCI). Might be good to check this PR with the newly build docker images.

Will let you know tomorrow

> @Wauplin Would you like me to trigger a complete CI on Github Actions?
> If so, could you check .github/workflows/model_jobs.yml
> what command(s) to bed added in order to test against this `huggingface_hub` RC ?

I suppose the complete CI runs even more tests, correct? In that case, then yes maybe it's a good idea. Since the workflow CI is installing transformers, and then only datasets/accelerate there is a big chance that it will override `huggingface_hub`'s version. Therefore I think it'd be good to add a

```yaml
      - name: Update / Install some packages (for Past CI)
        if: ${{ contains(inputs.docker, '-past-') }}
        working-directory: /transformers
        run: |
          python3 -m pip install "huggingface_hub==1.0.0.rc0"
```

after the datasets/accelerate ones.

> nothing looks wrong here, IDK when you want this to be merged, we can enforce RC0 in the mean time on main!

@ArthurZucker  Great! I would suggest merging this PR asap with rc0 enforced. We can do that as soon as a new release of kernels is done (see https://github.com/huggingface/transformers/pull/40889#issuecomment-3318593126 - otherwise it'll break documentation CI).

And once @ydshieh last run of tests is passing! Or at least if they don't pass, it'd be good to have an idea of what's failing. I suppose we'll have minor things to adapt in datasets/accelerate.

> I suppose the complete CI runs even more tests, correct? 

Yes.  I will trigger, no harm anyway.

workflow run

https://github.com/huggingface/transformers/actions/runs/17923228517

will check reports tomorrow

There are some failing tests (in the complete CI runs) that seems irrelevant at a first glance but actually related surprisingly.

> HF_HOME=/mnt/cache RUN_SLOW=1 python3 -m pytest -v tests/generation/test_continuous_batching.py::ContinuousBatchingTest::test_continuous_batching_parity_gemma_eager

fails with 1.0 rc but pass with 0.35.

Add `HF_TOKEN=hf_XXX` works with 1.0 rc. Could you check this ?

same for

> tests/models/gemma/test_modeling_gemma.py::GemmaIntegrationTest::test_model_2b_bf16_dola 


> tests/peft_integration/test_peft_integration.py::PeftIntegrationTester::test_peft_pipeline_no_warning

also fails with this PR 
```
        from transformers import pipeline

        ADAPTER_PATH = "peft-internal-testing/tiny-OPTForCausalLM-lora"
        BASE_PATH = "hf-internal-testing/tiny-random-OPTForCausalLM"

        # Input text for testing
        text = "Who is a Elon Musk?"

        model = AutoModelForCausalLM.from_pretrained(
            BASE_PATH,
            device_map="auto",
        )
        tokenizer = AutoTokenizer.from_pretrained(BASE_PATH)

>       lora_model = PeftModel.from_pretrained(
            model,
            ADAPTER_PATH,
            device_map="auto",
        )

tests/peft_integration/test_peft_integration.py:865:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.10/dist-packages/peft/peft_model.py:441: in from_pretrained
    PeftConfig._get_peft_type(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'peft.config.PeftConfig'>, model_id = 'peft-internal-testing/tiny-OPTForCausalLM-lora', hf_hub_download_kwargs = {'cache_dir': None, 'revision': None, 'subfolder': None, 'token': None, ...}, subfolder = None, path = 'peft-internal-testing/tiny-OPTForCausalLM-lora'

    @classmethod
    def _get_peft_type(
        cls,
        model_id: str,
        **hf_hub_download_kwargs,
    ):
        subfolder = hf_hub_download_kwargs.get("subfolder", None)

        path = os.path.join(model_id, subfolder) if subfolder is not None else model_id

        if os.path.isfile(os.path.join(path, CONFIG_NAME)):
            config_file = os.path.join(path, CONFIG_NAME)
        else:
            try:
                config_file = hf_hub_download(
                    model_id,
                    CONFIG_NAME,
                    **hf_hub_download_kwargs,
                )
            except Exception:
>               raise ValueError(f"Can't find '{CONFIG_NAME}' at '{model_id}'")
E               ValueError: Can't find 'adapter_config.json' at 'peft-internal-testing/tiny-OPTForCausalLM-lora'

/usr/local/lib/python3.10/dist-packages/peft/config.py:268: ValueError

```

> tests/pipelines/test_pipelines_text_to_audio.py::TextToAudioPipelineTests::test_csm_model_pt

same error as the first and second one

if you need to see the job run pages


https://github.com/huggingface/transformers/actions/runs/17923228517/job/50963407968

The CI (circleci) against python 3.10 is fine 🎉 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

Yay! thanks for confirming! :tada: 

I will work on the remaining TODOs in separate PRs.