Repository: huggingface/huggingface_hub
Pull request #1143 (2022-10-31 13:32:05)

Deprecate output in list_models
Fix https://github.com/huggingface/huggingface_hub/issues/1087 and prepare future pagination of `list_models`, `list_datasets` and `list_spaces` (cc @Pierrci @julien-c - cf [slack thread](https://huggingface.slack.com/archives/C03V11RNS7P/p1664283907577299) (internal link)).

The idea is to return a custom list on which any method that is specific to lists (e.g. almost everything except `for item in list_models(): ...`) is deprecated. This should encourage users to process the output as an iterator instead of a list. In particular, `len(list_models())` is deprecated (as it will not be accurate anymore when the api will be paginated.  Listing discussions from a repo is already paginated (python generator in `hfh`) which is good for consistency.

Implementation-wise, I'm using a metaclass to decorate all methods (so a bit python automagic) but I didn't see another way of doing it. For the record, I couldn't use the `__getattribute__` method as it is not called for magic methods like `__getitem__` or `__len__`.

Wauplin: (Internal discussion [here](https://huggingface.slack.com/archives/C03UMD3NYHL/p1667228814641129))

**Note:** there are 2 different topics here:
1. Deprecate the `list` return value to prepare for the future `iterator` return value (which will iterate over pages)
2. Handle pagination right now even if it will be implemented server side later (e.g. paginate over all pages, concatenate in a single list and output => suboptimal but future versions of `hfh` will not return a list anyway). Also possible that server-side we check the user-agent and don't paginate on older versions of `hfh` (=> would not break any existing library)

This PR currently handles 1. but not 2.
codecov[bot]: # [Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/1143?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Base: **84.61**% // Head: **84.69**% // Increases project coverage by **`+0.07%`** :tada:
> Coverage data is based on head [(`5a6f035`)](https://codecov.io/gh/huggingface/huggingface_hub/pull/1143?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) compared to base [(`a02f2b9`)](https://codecov.io/gh/huggingface/huggingface_hub/commit/a02f2b9520e97e56a6a8fb0af2aa7710c317dba9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).
> Patch coverage: 94.44% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1143      +/-   ##
==========================================
+ Coverage   84.61%   84.69%   +0.07%     
==========================================
  Files          41       41              
  Lines        4096     4130      +34     
==========================================
+ Hits         3466     3498      +32     
- Misses        630      632       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/huggingface_hub/pull/1143?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Coverage Δ | |
|---|---|---|
| [src/huggingface\_hub/utils/\_deprecation.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/1143/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi91dGlscy9fZGVwcmVjYXRpb24ucHk=) | `97.72% <93.75%> (-2.28%)` | :arrow_down: |
| [src/huggingface\_hub/hf\_api.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/1143/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9oZl9hcGkucHk=) | `88.11% <100.00%> (+0.04%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/1143?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).


Patch:
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index 934f5440..8d1732ae 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -64,7 +64,11 @@ from .utils import (  # noqa: F401 # imported for backward compatibility
     validate_hf_hub_args,
     write_to_credential_store,
 )
-from .utils._deprecation import _deprecate_method, _deprecate_positional_args
+from .utils._deprecation import (
+    _deprecate_list_output,
+    _deprecate_method,
+    _deprecate_positional_args,
+)
 from .utils._typing import Literal, TypedDict
 from .utils.endpoint_helpers import (
     AttributeDictionary,
@@ -662,6 +666,7 @@ class HfApi:
         d = r.json()
         return DatasetTags(d)
 
+    @_deprecate_list_output(version="0.14")
     @validate_hf_hub_args
     def list_models(
         self,
@@ -679,7 +684,7 @@ class HfApi:
         token: Optional[Union[bool, str]] = None,
     ) -> List[ModelInfo]:
         """
-        Get the public list of all the models on huggingface.co
+        Get the list of all the models on huggingface.co
 
         Args:
             filter ([`ModelFilter`] or `str` or `Iterable`, *optional*):
@@ -720,7 +725,10 @@ class HfApi:
                 or [`~huggingface_hub.login`]), token will be retrieved from the cache.
                 If `False`, token is not sent in the request header.
 
-        Returns: List of [`huggingface_hub.hf_api.ModelInfo`] objects
+        Returns:
+            `List[ModelInfo]`: a list of [`huggingface_hub.hf_api.ModelInfo`] objects.
+             To anticipate future pagination, please consider the return value to be a
+             simple iterator.
 
         Example usage with the `filter` argument:
 
@@ -874,6 +882,7 @@ class HfApi:
         query_dict["filter"] = tuple(filter_list)
         return query_dict
 
+    @_deprecate_list_output(version="0.14")
     @validate_hf_hub_args
     def list_datasets(
         self,
@@ -889,7 +898,7 @@ class HfApi:
         token: Optional[str] = None,
     ) -> List[DatasetInfo]:
         """
-        Get the public list of all the datasets on huggingface.co
+        Get the list of all the datasets on huggingface.co
 
         Args:
             filter ([`DatasetFilter`] or `str` or `Iterable`, *optional*):
@@ -920,6 +929,11 @@ class HfApi:
                 or [`~huggingface_hub.login`]), token will be retrieved from the cache.
                 If `False`, token is not sent in the request header.
 
+        Returns:
+            `List[DatasetInfo]`: a list of [`huggingface_hub.hf_api.DatasetInfo`] objects.
+             To anticipate future pagination, please consider the return value to be a
+             simple iterator.
+
         Example usage with the `filter` argument:
 
         ```python
@@ -1052,6 +1066,7 @@ class HfApi:
         d = r.json()
         return [MetricInfo(**x) for x in d]
 
+    @_deprecate_list_output(version="0.14")
     @validate_hf_hub_args
     def list_spaces(
         self,
@@ -1105,7 +1120,9 @@ class HfApi:
                 If `False`, token is not sent in the request header.
 
         Returns:
-            `List[SpaceInfo]`: a list of [`huggingface_hub.hf_api.SpaceInfo`] objects
+            `List[SpaceInfo]`: a list of [`huggingface_hub.hf_api.SpaceInfo`] objects.
+             To anticipate future pagination, please consider the return value to be a
+             simple iterator.
         """
         path = f"{self.endpoint}/api/spaces"
         headers = self._build_hf_headers(token=token)
diff --git a/src/huggingface_hub/utils/_deprecation.py b/src/huggingface_hub/utils/_deprecation.py
index 6d1582f9..68a14b11 100644
--- a/src/huggingface_hub/utils/_deprecation.py
+++ b/src/huggingface_hub/utils/_deprecation.py
@@ -1,7 +1,7 @@
 import warnings
 from functools import wraps
 from inspect import Parameter, signature
-from typing import Iterable, Optional
+from typing import Generator, Iterable, Optional
 
 
 def _deprecate_positional_args(*, version: str):
@@ -131,3 +131,103 @@ def _deprecate_method(*, version: str, message: Optional[str] = None):
         return inner_f
 
     return _inner_deprecate_method
+
+
+def _deprecate_list_output(*, version: str):
+    """Decorator to deprecate the usage as a list of the output of a method.
+
+    To be used when a method currently returns a list of objects but is planned to return
+    an generator instead in the future. Output is still a list but tweaked to issue a
+    warning message when it is specifically used as a list (e.g. get/set/del item, get
+    length,...).
+
+    Args:
+        version (`str`):
+            The version when output will start to be an generator.
+    """
+
+    def _inner_deprecate_method(f):
+        @wraps(f)
+        def inner_f(*args, **kwargs):
+            list_value = f(*args, **kwargs)
+            return DeprecatedList(
+                list_value,
+                warning_message=(
+                    "'{f.__name__}' currently returns a list of objects but is planned"
+                    " to be a generator starting from version {version} in order to"
+                    " implement pagination. Please avoid to use"
+                    " `{f.__name__}(...).{attr_name}` or explicitly convert the output"
+                    " to a list first with `list(iter({f.__name__})(...))`.".format(
+                        f=f,
+                        version=version,
+                        # Dumb but working workaround to render `attr_name` later
+                        # Taken from https://stackoverflow.com/a/35300723
+                        attr_name="{attr_name}",
+                    )
+                ),
+            )
+
+        return inner_f
+
+    return _inner_deprecate_method
+
+
+def _empty_gen() -> Generator:
+    # Create an empty generator
+    # Taken from https://stackoverflow.com/a/13243870
+    return
+    yield
+
+
+# Build the set of attributes that are specific to a List object (and will be deprecated)
+_LIST_ONLY_ATTRS = frozenset(set(dir([])) - set(dir(_empty_gen())))
+
+
+class DeprecateListMetaclass(type):
+    """Metaclass that overwrites all list-only methods, including magic ones."""
+
+    def __new__(cls, clsname, bases, attrs):
+        # Check consistency
+        if "_deprecate" not in attrs:
+            raise TypeError(
+                "A `_deprecate` method must be implemented to use"
+                " `DeprecateListMetaclass`."
+            )
+        if list not in bases:
+            raise TypeError(
+                "Class must inherit from `list` to use `DeprecateListMetaclass`."
+            )
+
+        # Create decorator to deprecate list-only methods, including magic ones
+        def _with_deprecation(f, name):
+            @wraps(f)
+            def _inner(self, *args, **kwargs):
+                self._deprecate(name)  # Use the `_deprecate`
+                return f(self, *args, **kwargs)
+
+            return _inner
+
+        # Deprecate list-only methods
+        for attr in _LIST_ONLY_ATTRS:
+            attrs[attr] = _with_deprecation(getattr(list, attr), attr)
+
+        return super().__new__(cls, clsname, bases, attrs)
+
+
+class DeprecatedList(list, metaclass=DeprecateListMetaclass):
+    """Custom List class for which all calls to a list-specific method is deprecated.
+
+    Methods that are shared with a generator are not deprecated.
+    See `_deprecate_list_output` for more details.
+    """
+
+    def __init__(self, iterable, warning_message: str):
+        """Initialize the list with a default warning message.
+
+        Warning message will be formatted at runtime with a "{attr_name}" value.
+        """
+        super().__init__(iterable)
+        self._deprecation_msg = warning_message
+
+    def _deprecate(self, attr_name: str) -> None:
+        warnings.warn(self._deprecation_msg.format(attr_name=attr_name), FutureWarning)


Test patch:
diff --git a/tests/test_hf_api.py b/tests/test_hf_api.py
index dd36c304..a27fcc93 100644
--- a/tests/test_hf_api.py
+++ b/tests/test_hf_api.py
@@ -1176,6 +1176,7 @@ class HfApiPublicTest(unittest.TestCase):
         _api = HfApi(endpoint=ENDPOINT_STAGING)
         _ = _api.list_models()
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_list_models(self):
         _api = HfApi()
@@ -1183,6 +1184,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(models), 100)
         self.assertIsInstance(models[0], ModelInfo)
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_list_models_author(self):
         _api = HfApi()
@@ -1192,6 +1194,7 @@ class HfApiPublicTest(unittest.TestCase):
         for model in models:
             self.assertTrue(model.modelId.startswith("google/"))
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_list_models_search(self):
         _api = HfApi()
@@ -1204,6 +1207,7 @@ class HfApiPublicTest(unittest.TestCase):
             # (and changes it in the future) but for now it should do the trick.
             self.assertTrue("bert" in model.modelId.lower())
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_list_models_complex_query(self):
         # Let's list the 10 most recent models
@@ -1295,13 +1299,15 @@ class HfApiPublicTest(unittest.TestCase):
         _api = HfApi(endpoint=ENDPOINT_STAGING)
         _ = _api.list_datasets()
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
-    def test_list_datasets(self):
+    def test_list_datasets_no_filter(self):
         _api = HfApi()
         datasets = _api.list_datasets()
         self.assertGreater(len(datasets), 100)
         self.assertIsInstance(datasets[0], DatasetInfo)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_author_and_name(self):
         _api = HfApi()
@@ -1311,6 +1317,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertTrue("huggingface" in datasets[0].author)
         self.assertTrue("DataMeasurementsFiles" in datasets[0].id)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_benchmark(self):
         _api = HfApi()
@@ -1319,6 +1326,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 0)
         self.assertTrue("benchmark:raft" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_language_creator(self):
         _api = HfApi()
@@ -1327,6 +1335,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 0)
         self.assertTrue("language_creators:crowdsourced" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_language(self):
         _api = HfApi()
@@ -1341,6 +1350,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertTrue("languages:en" in datasets[0].tags)
         self.assertTrue("languages:fr" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_multilinguality(self):
         _api = HfApi()
@@ -1349,6 +1359,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 0)
         self.assertTrue("multilinguality:multilingual" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_size_categories(self):
         _api = HfApi()
@@ -1357,6 +1368,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 0)
         self.assertTrue("size_categories:100K<n<1M" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_task_categories(self):
         _api = HfApi()
@@ -1365,6 +1377,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 0)
         self.assertTrue("task_categories:audio-classification" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_filter_datasets_by_task_ids(self):
         _api = HfApi()
@@ -1373,6 +1386,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 0)
         self.assertTrue("task_ids:automatic-speech-recognition" in datasets[0].tags)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_list_datasets_full(self):
         _api = HfApi()
@@ -1382,6 +1396,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertIsInstance(dataset, DatasetInfo)
         self.assertTrue(any(dataset.cardData for dataset in datasets))
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_list_datasets_author(self):
         _api = HfApi()
@@ -1389,6 +1404,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(datasets), 1)
         self.assertIsInstance(datasets[0], DatasetInfo)
 
+    @expect_deprecation("list_datasets")
     @with_production_testing
     def test_list_datasets_search(self):
         _api = HfApi()
@@ -1465,6 +1481,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertIsInstance(metrics[0], MetricInfo)
         self.assertTrue(any(metric.description for metric in metrics))
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_filter_models_by_author(self):
         _api = HfApi()
@@ -1473,6 +1490,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(models), 0)
         self.assertTrue("muellerzr" in models[0].modelId)
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_filter_models_by_author_and_name(self):
         # Test we can search by an author and a name, but the model is not found
@@ -1481,6 +1499,7 @@ class HfApiPublicTest(unittest.TestCase):
         models = _api.list_models(filter=f)
         self.assertTrue("facebook/bart-base" in models[0].modelId)
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_failing_filter_models_by_author_and_model_name(self):
         # Test we can search by an author and a name, but the model is not found
@@ -1489,6 +1508,7 @@ class HfApiPublicTest(unittest.TestCase):
         models = _api.list_models(filter=f)
         self.assertEqual(len(models), 0)
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_filter_models_with_library(self):
         _api = HfApi()
@@ -1499,6 +1519,7 @@ class HfApiPublicTest(unittest.TestCase):
         models = _api.list_models(filter=f)
         self.assertGreater(len(models), 0)
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_filter_models_with_task(self):
         _api = HfApi()
@@ -1510,6 +1531,7 @@ class HfApiPublicTest(unittest.TestCase):
         models = _api.list_models(filter=f)
         self.assertGreater(1, len(models))
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_filter_models_by_language(self):
         _api = HfApi()
@@ -1521,6 +1543,7 @@ class HfApiPublicTest(unittest.TestCase):
 
         assert len(res_fr) != len(res_en)
 
+    @expect_deprecation("list_models")
     @with_production_testing
     def test_filter_models_with_complex_query(self):
         _api = HfApi()
@@ -1609,6 +1632,7 @@ class HfApiPublicTest(unittest.TestCase):
             )
         )
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_full(self):
         _api = HfApi()
@@ -1618,6 +1642,7 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertIsInstance(space, SpaceInfo)
         self.assertTrue(any(space.cardData for space in spaces))
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_author(self):
         _api = HfApi()
@@ -1629,6 +1654,7 @@ class HfApiPublicTest(unittest.TestCase):
             )
         )
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_search(self):
         _api = HfApi()
@@ -1636,6 +1662,7 @@ class HfApiPublicTest(unittest.TestCase):
         space = spaces[0]
         self.assertTrue("wikipedia" in space.id.lower())
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_sort_and_direction(self):
         _api = HfApi()
@@ -1648,18 +1675,21 @@ class HfApiPublicTest(unittest.TestCase):
             spaces_ascending_likes[-2].likes, spaces_ascending_likes[-1].likes
         )
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_limit(self):
         _api = HfApi()
         spaces = _api.list_spaces(limit=5)
         self.assertEqual(len(spaces), 5)
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_with_models(self):
         _api = HfApi()
         spaces = _api.list_spaces(models="bert-base-uncased")
         self.assertTrue("bert-base-uncased" in getattr(spaces[0], "models", []))
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_spaces_with_datasets(self):
         _api = HfApi()
@@ -1740,16 +1770,19 @@ class HfApiPrivateTest(HfApiCommonTestWithLogin):
             )
             self.assertIsInstance(dataset_info, DatasetInfo)
 
+    @expect_deprecation("list_datasets")
     def test_list_private_datasets(self):
         orig = len(self._api.list_datasets(use_auth_token=False))
         new = len(self._api.list_datasets(use_auth_token=self._token))
         self.assertGreater(new, orig)
 
+    @expect_deprecation("list_models")
     def test_list_private_models(self):
         orig = len(self._api.list_models(use_auth_token=False))
         new = len(self._api.list_models(use_auth_token=self._token))
         self.assertGreater(new, orig)
 
+    @expect_deprecation("list_spaces")
     @with_production_testing
     def test_list_private_spaces(self):
         orig = len(self._api.list_spaces(use_auth_token=False))
diff --git a/tests/test_utils_deprecation.py b/tests/test_utils_deprecation.py
index ccaf6f78..9dadfc8b 100644
--- a/tests/test_utils_deprecation.py
+++ b/tests/test_utils_deprecation.py
@@ -5,6 +5,7 @@ import pytest
 
 from huggingface_hub.utils._deprecation import (
     _deprecate_arguments,
+    _deprecate_list_output,
     _deprecate_method,
     _deprecate_positional_args,
 )
@@ -127,3 +128,61 @@ class TestDeprecationUtils(unittest.TestCase):
             "'dummy_deprecated' (from 'tests.test_utils_deprecation') is deprecated"
             " and will be removed from version 'xxx'. This is a custom message.",
         )
+
+    def test_deprecate_list_output(self) -> None:
+        """Test test_deprecate_list_output throw warnings."""
+
+        @_deprecate_list_output(version="xxx")
+        def dummy_deprecated() -> None:
+            return [1, 2, 3]
+
+        output = dummy_deprecated()
+
+        # Still a list !
+        self.assertIsInstance(output, list)
+
+        # __getitem__
+        with pytest.warns(FutureWarning) as record:
+            self.assertEqual(output[0], 1)
+
+        # (check real message once)
+        self.assertEqual(
+            record[0].message.args[0],
+            "'dummy_deprecated' currently returns a list of objects but is planned to"
+            " be a generator starting from version xxx in order to implement"
+            " pagination. Please avoid to use `dummy_deprecated(...).__getitem__` or"
+            " explicitly convert the output to a list first with"
+            " `list(iter(dummy_deprecated)(...))`.",
+        )
+
+        # __setitem__
+        with pytest.warns(FutureWarning):
+            output[0] = 10
+
+        # __delitem__
+        with pytest.warns(FutureWarning):
+            del output[1]
+
+        # `.append` deprecated (as .index, .pop, .insert, .remove, .sort,...)
+        with pytest.warns(FutureWarning):
+            output.append(5)
+
+        # Magic __len__ deprecated (as __add__, __mul__, __contains__,...)
+        with pytest.warns(FutureWarning):
+            self.assertEqual(len(output), 3)
+
+        with pytest.warns(FutureWarning):
+            self.assertTrue(3 in output)
+
+        # List has been modified
+        # Iterating over items is NOT deprecated !
+        for item, expected in zip(output, [10, 3, 5]):
+            self.assertEqual(item, expected)
+
+        # If output is converted to list, no warning is raised.
+        output_as_list = list(iter(dummy_deprecated()))
+        self.assertEqual(output_as_list[0], 1)
+        del output_as_list[1]
+        output_as_list.append(5)
+        self.assertEqual(len(output_as_list), 3)
+        self.assertTrue(3 in output_as_list)