Repository: huggingface/huggingface_hub
Pull request #1174 (2022-11-09 10:38:57)

Warn user if output is truncated in `list_models`
If output is truncated in `list_models` but not on user's request, raise a warning to suggest upgrading `huggingface_hub`.

This will be useful when pagination will be mandatory on Hub's side. To be removed when pagination is handled client-side as well.

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_1174). All of your documentation changes will be reflected on that endpoint.
codecov[bot]: # [Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/1174?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Base: **84.77**% // Head: **84.82**% // Increases project coverage by **`+0.05%`** :tada:
> Coverage data is based on head [(`a8a8716`)](https://codecov.io/gh/huggingface/huggingface_hub/pull/1174?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) compared to base [(`4b086f7`)](https://codecov.io/gh/huggingface/huggingface_hub/commit/4b086f70bed8b73b22ddd632b5c42a4b8731ffa6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1174      +/-   ##
==========================================
+ Coverage   84.77%   84.82%   +0.05%     
==========================================
  Files          42       42              
  Lines        4196     4210      +14     
==========================================
+ Hits         3557     3571      +14     
  Misses        639      639              
```


| [Impacted Files](https://codecov.io/gh/huggingface/huggingface_hub/pull/1174?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Coverage Δ | |
|---|---|---|
| [src/huggingface\_hub/hf\_api.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/1174/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9oZl9hcGkucHk=) | `88.01% <100.00%> (+0.21%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/1174?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).


Patch:
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index 0c9df24b..15651aa9 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -809,8 +809,14 @@ class HfApi:
             params.update({"cardData": True})
         r = requests.get(path, params=params, headers=headers)
         hf_raise_for_status(r)
-        d = r.json()
-        res = [ModelInfo(**x) for x in d]
+        items = [ModelInfo(**x) for x in r.json()]
+
+        # If pagination has been enabled server-side, older versions of `huggingface_hub`
+        # are deprecated as output is truncated.
+        _warn_if_truncated(
+            items, total_count=r.headers.get("X-Total-Count"), limit=limit
+        )
+
         if emissions_thresholds is not None:
             if cardData is None:
                 raise ValueError(
@@ -818,8 +824,9 @@ class HfApi:
                     " `cardData=True`."
                 )
             else:
-                return _filter_emissions(res, *emissions_thresholds)
-        return res
+                return _filter_emissions(items, *emissions_thresholds)
+
+        return items
 
     def _unpack_model_filter(self, model_filter: ModelFilter):
         """
@@ -1009,8 +1016,15 @@ class HfApi:
             params.update({"full": True})
         r = requests.get(path, params=params, headers=headers)
         hf_raise_for_status(r)
-        d = r.json()
-        return [DatasetInfo(**x) for x in d]
+        items = [DatasetInfo(**x) for x in r.json()]
+
+        # If pagination has been enabled server-side, older versions of `huggingface_hub`
+        # are deprecated as output is truncated.
+        _warn_if_truncated(
+            items, total_count=r.headers.get("X-Total-Count"), limit=limit
+        )
+
+        return items
 
     def _unpack_dataset_filter(self, dataset_filter: DatasetFilter):
         """
@@ -1149,8 +1163,15 @@ class HfApi:
             params.update({"models": models})
         r = requests.get(path, params=params, headers=headers)
         hf_raise_for_status(r)
-        d = r.json()
-        return [SpaceInfo(**x) for x in d]
+        items = [SpaceInfo(**x) for x in r.json()]
+
+        # If pagination has been enabled server-side, older versions of `huggingface_hub`
+        # are deprecated as output is truncated.
+        _warn_if_truncated(
+            items, total_count=r.headers.get("X-Total-Count"), limit=limit
+        )
+
+        return items
 
     @validate_hf_hub_args
     def model_info(
@@ -3360,6 +3381,38 @@ def _parse_revision_from_pr_url(pr_url: str) -> str:
     return f"refs/pr/{re_match[1]}"
 
 
+def _warn_if_truncated(
+    items: List[Any], limit: Optional[int], total_count: Optional[str]
+) -> None:
+    # TODO: remove this once pagination is properly implemented in `huggingface_hub`.
+    if total_count is None:
+        # Total count header not implemented
+        return
+
+    try:
+        total_count_int = int(total_count)
+    except ValueError:
+        # Total count header not implemented properly server-side
+        return
+
+    if len(items) == total_count_int:
+        # All items have been returned => not truncated
+        return
+
+    if limit is not None and len(items) == limit:
+        # `limit` is set => truncation is expected
+        return
+
+    # Otherwise, pagination has been enabled server-side and the output has been
+    # truncated by server => warn user.
+    warnings.warn(
+        "The list of repos returned by the server has been truncated. Listing repos"
+        " from the Hub using `list_models`, `list_datasets` and `list_spaces` now"
+        " requires pagination. To get the full list of repos, please consider upgrading"
+        " `huggingface_hub` to its latest version."
+    )
+
+
 api = HfApi()
 
 set_access_token = api.set_access_token


Test patch:
diff --git a/tests/test_hf_api.py b/tests/test_hf_api.py
index c28b9ca8..91df91d0 100644
--- a/tests/test_hf_api.py
+++ b/tests/test_hf_api.py
@@ -55,6 +55,7 @@ from huggingface_hub.hf_api import (
     ModelSearchArguments,
     RepoFile,
     SpaceInfo,
+    _warn_if_truncated,
     erase_from_credential_store,
     read_from_credential_store,
     repo_type_and_id_from_hf_id,
@@ -2170,3 +2171,26 @@ class HfApiTokenAttributeTest(unittest.TestCase):
         self, mock_build_hf_headers: Mock, expected_value: str
     ) -> None:
         self.assertEqual(mock_build_hf_headers.call_args[1]["token"], expected_value)
+
+
+class WarnIfTruncatedTest(unittest.TestCase):
+    def test_warn_if_truncated(self) -> None:
+        # Can't tell if output is truncated
+        _warn_if_truncated([1, 2, 3], limit=None, total_count=None)
+
+        # Can't tell if output is truncated
+        _warn_if_truncated([1, 2, 3], limit=None, total_count="foo")
+
+        # All items returned
+        _warn_if_truncated([1, 2, 3], limit=None, total_count="3")
+
+        # Output is truncated (no limit, received 3)
+        with self.assertWarns(UserWarning):
+            _warn_if_truncated([1, 2, 3], limit=None, total_count="5")
+
+        # Output is truncated (limit is 4, received 3)
+        with self.assertWarns(UserWarning):
+            _warn_if_truncated([1, 2, 3], limit=4, total_count="5")
+
+        # Output is truncated by the user
+        _warn_if_truncated([1, 2, 3], limit=3, total_count="5")