Repository: huggingface/huggingface_hub
Pull request #994 (2022-08-16 16:16:28)

Allowlist and denylist when uploading a folder
Fix https://github.com/huggingface/huggingface_hub/issues/982 and https://github.com/huggingface/huggingface_hub/issues/836.

Implements a `filter_repo_objects` helper to filter objects based on their path and a allowlist/denylist, using [fnmatch](https://docs.python.org/3/library/fnmatch.html). Used in `snapshot_download` and `upload_folder`. Use of `allow_regex`/`ignore_regex` is now deprecated in favor of `allow_patterns`/`ignore_patterns`. Backward compatibility is planned until version 0.12.

**EDIT:** now also implemented in `push_to_hub` mixin and `push_to_hub_keras`.

codecov[bot]: # [Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/994?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
> Merging [#994](https://codecov.io/gh/huggingface/huggingface_hub/pull/994?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) (c885410) into [main](https://codecov.io/gh/huggingface/huggingface_hub/commit/880408a148dc79825c995de136f5c2a8eaa31741?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) (880408a) will **decrease** coverage by `18.21%`.
> The diff coverage is `24.24%`.

```diff
@@             Coverage Diff             @@
##             main     #994       +/-   ##
===========================================
- Coverage   47.61%   29.40%   -18.22%     
===========================================
  Files          29       30        +1     
  Lines        3316     3336       +20     
===========================================
- Hits         1579      981      -598     
- Misses       1737     2355      +618     
```


| [Impacted Files](https://codecov.io/gh/huggingface/huggingface_hub/pull/994?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Coverage Δ | |
|---|---|---|
| [src/huggingface\_hub/\_snapshot\_download.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9fc25hcHNob3RfZG93bmxvYWQucHk=) | `18.18% <16.66%> (+0.43%)` | :arrow_up: |
| [src/huggingface\_hub/utils/paths.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi91dGlscy9wYXRocy5weQ==) | `24.00% <24.00%> (ø)` | |
| [src/huggingface\_hub/hf\_api.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9oZl9hcGkucHk=) | `27.44% <50.00%> (-5.15%)` | :arrow_down: |
| [src/huggingface\_hub/keras\_mixin.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9rZXJhc19taXhpbi5weQ==) | `19.17% <0.00%> (-71.92%)` | :arrow_down: |
| [src/huggingface\_hub/fastai\_utils.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9mYXN0YWlfdXRpbHMucHk=) | `21.56% <0.00%> (-57.85%)` | :arrow_down: |
| [src/huggingface\_hub/hub\_mixin.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9odWJfbWl4aW4ucHk=) | `28.73% <0.00%> (-57.48%)` | :arrow_down: |
| [src/huggingface\_hub/repository.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9yZXBvc2l0b3J5LnB5) | `13.59% <0.00%> (-45.27%)` | :arrow_down: |
| [src/huggingface\_hub/utils/\_subprocess.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi91dGlscy9fc3VicHJvY2Vzcy5weQ==) | `62.50% <0.00%> (-25.00%)` | :arrow_down: |
| [src/huggingface\_hub/file\_download.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9maWxlX2Rvd25sb2FkLnB5) | `26.51% <0.00%> (-20.17%)` | :arrow_down: |
| [src/huggingface\_hub/utils/tqdm.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi91dGlscy90cWRtLnB5) | `45.45% <0.00%> (-13.64%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/huggingface/huggingface_hub/pull/994/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface)

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_994). All of your documentation changes will be reflected on that endpoint.
Wauplin: Thanks for the review @LysandreJik. I made the small changes in https://github.com/huggingface/huggingface_hub/pull/994/commits/420f6078079bebd4132f26bd471af2e8fbb2794f and will merge once CI is happy.

Patch:
diff --git a/docs/source/how-to-downstream.mdx b/docs/source/how-to-downstream.mdx
index eb76638c..6e33b8d7 100644
--- a/docs/source/how-to-downstream.mdx
+++ b/docs/source/how-to-downstream.mdx
@@ -85,28 +85,28 @@ already know the file names you need.
 
 However, you don't always want to download the contents of an entire repository with
 [`snapshot_download`]. Even if you don't know the file name, you can download specific
-files if you know the file type with `allow_regex` and `ignore_regex`. Use the
-`allow_regex` and `ignore_regex` arguments to specify which files to download. These
-parameters accept either a single regex or a list of regexes. 
+files if you know the file type with `allow_patterns` and `ignore_patterns`. Use the
+`allow_patterns` and `ignore_patterns` arguments to specify which files to download. These
+parameters accept either a single pattern or a list of patterns.
 
-The regex matching is based on
-[`fnmatch`](https://docs.python.org/3/library/fnmatch.html), which provides support for
-Unix shell-style wildcards.
+Patterns are Standard Wildcards (globbing patterns) as documented
+[here](https://tldp.org/LDP/GNU-Linux-Tools-Summary/html/x11655.htm). The pattern
+matching is based on [`fnmatch`](https://docs.python.org/3/library/fnmatch.html).
 
-For example, you can use `allow_regex` to only download JSON configuration files:
+For example, you can use `allow_patterns` to only download JSON configuration files:
 
 ```python
 >>> from huggingface_hub import snapshot_download
->>> snapshot_download(repo_id="lysandre/arxiv-nlp", allow_regex="*.json")
+>>> snapshot_download(repo_id="lysandre/arxiv-nlp", allow_patterns="*.json")
 ```
 
-On the other hand, `ignore_regex` can exclude certain files from being downloaded. The
+On the other hand, `ignore_patterns` can exclude certain files from being downloaded. The
 following example ignores the `.msgpack` and `.h5` file extensions:
 
 ```python
 >>> from huggingface_hub import snapshot_download
->>> snapshot_download(repo_id="lysandre/arxiv-nlp", ignore_regex=["*.msgpack", "*.h5"])
+>>> snapshot_download(repo_id="lysandre/arxiv-nlp", ignore_patterns=["*.msgpack", "*.h5"])
 ```
 
-Passing a regex can be especially useful when repositories contain files that are never
+Passing a pattern can be especially useful when repositories contain files that are never
 expected to be downloaded by [`snapshot_download`].
diff --git a/docs/source/how-to-upstream.mdx b/docs/source/how-to-upstream.mdx
index 3361b142..77bf123c 100644
--- a/docs/source/how-to-upstream.mdx
+++ b/docs/source/how-to-upstream.mdx
@@ -39,10 +39,11 @@ Specify the path of the file to upload, where you want to upload the file to in
 ```py
 >>> from huggingface_hub import HfApi
 >>> api = HfApi()
->>> api.upload_file(path_or_fileobj="/path/to/local/folder/README.md", 
-...                 path_in_repo="README.md", 
-...                 repo_id="username/test-dataset",
-...                 repo_type="dataset",
+>>> api.upload_file(
+...     path_or_fileobj="/path/to/local/folder/README.md",
+...     path_in_repo="README.md",
+...     repo_id="username/test-dataset",
+...     repo_type="dataset",
 ... )
 ```
 
@@ -50,13 +51,19 @@ Specify the path of the file to upload, where you want to upload the file to in
 
 Use the [`upload_folder`] function to upload a local folder to an existing repository. Specify the path of the local folder to upload, where you want to upload the folder to in the repository, and the name of the repository you want to add the folder to. Depending on your repository type, you can optionally set the repository type as a `dataset`, `model`, or `space`.
 
+Use the `allow_patterns` and `ignore_patterns` arguments to specify which files to upload. These parameters accept either a single pattern or a list of patterns.
+Patterns are Standard Wildcards (globbing patterns) as documented [here](https://tldp.org/LDP/GNU-Linux-Tools-Summary/html/x11655.htm).
+If both `allow_patterns` and `ignore_patterns` are provided, both constraints apply. By default, all files from the folder are uploaded.
+
 ```py
 >>> from huggingface_hub import HfApi
 >>> api = HfApi()
->>> api.upload_folder(folder_path="/path/to/local/folder",
-...                   path_in_repo="my-dataset/train",
-...                   repo_id="username/test-dataset",
-...                   repo_type="dataset",
+>>> api.upload_folder(
+...     folder_path="/path/to/local/folder",
+...     path_in_repo="my-dataset/train",
+...     repo_id="username/test-dataset",
+...     repo_type="dataset",
+...     ignore_patterns="**/logs/*.txt",
 ... )
 ```
 
diff --git a/src/huggingface_hub/_snapshot_download.py b/src/huggingface_hub/_snapshot_download.py
index 05fbce41..d17af73a 100644
--- a/src/huggingface_hub/_snapshot_download.py
+++ b/src/huggingface_hub/_snapshot_download.py
@@ -1,43 +1,22 @@
 import os
-from fnmatch import fnmatch
 from pathlib import Path
 from typing import Dict, List, Optional, Union
 
 from .constants import DEFAULT_REVISION, HUGGINGFACE_HUB_CACHE, REPO_TYPES
 from .file_download import REGEX_COMMIT_HASH, hf_hub_download, repo_folder_name
 from .hf_api import HfApi, HfFolder
-from .utils import logging
+from .utils import filter_repo_objects, logging
+from .utils._deprecation import _deprecate_arguments
 
 
 logger = logging.get_logger(__name__)
 
 
-def _filter_repo_files(
-    *,
-    repo_files: List[str],
-    allow_regex: Optional[Union[List[str], str]] = None,
-    ignore_regex: Optional[Union[List[str], str]] = None,
-) -> List[str]:
-    allow_regex = [allow_regex] if isinstance(allow_regex, str) else allow_regex
-    ignore_regex = [ignore_regex] if isinstance(ignore_regex, str) else ignore_regex
-    filtered_files = []
-    for repo_file in repo_files:
-        # if there's an allowlist, skip download if file does not match any regex
-        if allow_regex is not None and not any(
-            fnmatch(repo_file, r) for r in allow_regex
-        ):
-            continue
-
-        # if there's a denylist, skip download if file does matches any regex
-        if ignore_regex is not None and any(
-            fnmatch(repo_file, r) for r in ignore_regex
-        ):
-            continue
-
-        filtered_files.append(repo_file)
-    return filtered_files
-
-
+@_deprecate_arguments(
+    version="0.12",
+    deprecated_args={"allow_regex", "ignore_regex"},
+    custom_message="Please use `allow_patterns` and `ignore_patterns` instead.",
+)
 def snapshot_download(
     repo_id: str,
     *,
@@ -54,6 +33,8 @@ def snapshot_download(
     local_files_only: Optional[bool] = False,
     allow_regex: Optional[Union[List[str], str]] = None,
     ignore_regex: Optional[Union[List[str], str]] = None,
+    allow_patterns: Optional[Union[List[str], str]] = None,
+    ignore_patterns: Optional[Union[List[str], str]] = None,
 ) -> str:
     """Download all files of a repo.
 
@@ -98,10 +79,10 @@ def snapshot_download(
         local_files_only (`bool`, *optional*, defaults to `False`):
             If `True`, avoid downloading the file and return the path to the
             local cached file if it exists.
-        allow_regex (`list of str`, `str`, *optional*):
-            If provided, only files matching this regex are downloaded.
-        ignore_regex (`list of str`, `str`, *optional*):
-            If provided, files matching this regex are not downloaded.
+        allow_patterns (`List[str]` or `str`, *optional*):
+            If provided, only files matching at least one pattern are downloaded.
+        ignore_patterns (`List[str]` or `str`, *optional*):
+            If provided, files matching any of the patterns are not downloaded.
 
     Returns:
         Local folder path (string) of repo snapshot
@@ -119,7 +100,6 @@ def snapshot_download(
 
     </Tip>
     """
-
     if cache_dir is None:
         cache_dir = HUGGINGFACE_HUB_CACHE
     if revision is None:
@@ -151,6 +131,13 @@ def snapshot_download(
         cache_dir, repo_folder_name(repo_id=repo_id, repo_type=repo_type)
     )
 
+    # TODO: remove these 4 lines in version 0.12
+    #       Deprecated code to ensure backward compatibility.
+    if allow_regex is not None:
+        allow_patterns = allow_regex
+    if ignore_regex is not None:
+        ignore_patterns = ignore_regex
+
     # if we have no internet connection we will look for an
     # appropriate folder in the cache
     # If the specified revision is a commit hash, look inside "snapshots".
@@ -181,10 +168,10 @@ def snapshot_download(
     repo_info = _api.repo_info(
         repo_id=repo_id, repo_type=repo_type, revision=revision, token=token
     )
-    filtered_repo_files = _filter_repo_files(
-        repo_files=[f.rfilename for f in repo_info.siblings],
-        allow_regex=allow_regex,
-        ignore_regex=ignore_regex,
+    filtered_repo_files = filter_repo_objects(
+        items=[f.rfilename for f in repo_info.siblings],
+        allow_patterns=allow_patterns,
+        ignore_patterns=ignore_patterns,
     )
     commit_hash = repo_info.sha
     snapshot_folder = os.path.join(storage_folder, "snapshots", commit_hash)
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index dec2f424..6e324073 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -51,7 +51,7 @@ from .constants import (
     REPO_TYPES_URL_PREFIXES,
     SPACES_SDK_TYPES,
 )
-from .utils import logging
+from .utils import filter_repo_objects, logging
 from .utils._deprecation import _deprecate_positional_args
 from .utils._errors import (
     _raise_convert_bad_request,
@@ -2215,6 +2215,8 @@ class HfApi:
         revision: Optional[str] = None,
         create_pr: Optional[bool] = None,
         parent_commit: Optional[str] = None,
+        allow_patterns: Optional[Union[List[str], str]] = None,
+        ignore_patterns: Optional[Union[List[str], str]] = None,
     ):
         """
         Upload a local folder to the given repo. The upload is done
@@ -2224,6 +2226,13 @@ class HfApi:
         The structure of the folder will be preserved. Files with the same name
         already present in the repository will be overwritten, others will be left untouched.
 
+        Use the `allow_patterns` and `ignore_patterns` arguments to specify which files
+        to upload. These parameters accept either a single pattern or a list of
+        patterns. Patterns are Standard Wildcards (globbing patterns) as documented
+        [here](https://tldp.org/LDP/GNU-Linux-Tools-Summary/html/x11655.htm). If both
+        `allow_patterns` and `ignore_patterns` are provided, both constraints apply. By
+        default, all files from the folder are uploaded.
+
         Uses `HfApi.create_commit` under the hood.
 
         Args:
@@ -2259,7 +2268,10 @@ class HfApi:
                 If specified and `create_pr` is `True`, the pull request will be created from `parent_commit`.
                 Specifying `parent_commit` ensures the repo has not changed before committing the changes, and can be
                 especially useful if the repo is updated / committed to concurrently.
-
+            allow_patterns (`List[str]` or `str`, *optional*):
+                If provided, only files matching at least one pattern are uploaded.
+            ignore_patterns (`List[str]` or `str`, *optional*):
+                If provided, files matching any of the patterns are not uploaded.
 
         Returns:
             `str`: A URL to visualize the uploaded folder on the hub
@@ -2284,6 +2296,7 @@ class HfApi:
         ...     repo_id="username/my-dataset",
         ...     repo_type="datasets",
         ...     token="my_token",
+        ...     ignore_patterns="**/logs/*.txt",
         ... )
         # "https://huggingface.co/datasets/username/my-dataset/tree/main/remote/experiment/checkpoints"
 
@@ -2312,7 +2325,12 @@ class HfApi:
             else f"Upload {path_in_repo} with huggingface_hub"
         )
 
-        files_to_add = _prepare_upload_folder_commit(folder_path, path_in_repo)
+        files_to_add = _prepare_upload_folder_commit(
+            folder_path,
+            path_in_repo,
+            allow_patterns=allow_patterns,
+            ignore_patterns=ignore_patterns,
+        )
 
         pr_url = self.create_commit(
             repo_type=repo_type,
@@ -3232,9 +3250,16 @@ class HfFolder:
 
 
 def _prepare_upload_folder_commit(
-    folder_path: str, path_in_repo: str
+    folder_path: str,
+    path_in_repo: str,
+    allow_patterns: Optional[Union[List[str], str]] = None,
+    ignore_patterns: Optional[Union[List[str], str]] = None,
 ) -> List[CommitOperationAdd]:
-    """Generate the list of Add operations for a commit to upload a folder."""
+    """Generate the list of Add operations for a commit to upload a folder.
+
+    Files not matching the `allow_patterns` (allowlist) and `ignore_patterns` (denylist)
+    constraints are discarded.
+    """
     folder_path = os.path.normpath(os.path.expanduser(folder_path))
     if not os.path.isdir(folder_path):
         raise ValueError(f"Provided path: '{folder_path}' is not a directory")
@@ -3252,7 +3277,15 @@ def _prepare_upload_folder_commit(
                     ).replace(os.sep, "/"),
                 )
             )
-    return files_to_add
+
+    return list(
+        filter_repo_objects(
+            files_to_add,
+            allow_patterns=allow_patterns,
+            ignore_patterns=ignore_patterns,
+            key=lambda x: x.path_in_repo,
+        )
+    )
 
 
 def _parse_revision_from_pr_url(pr_url: str) -> str:
diff --git a/src/huggingface_hub/hub_mixin.py b/src/huggingface_hub/hub_mixin.py
index b207fdbc..757e98c5 100644
--- a/src/huggingface_hub/hub_mixin.py
+++ b/src/huggingface_hub/hub_mixin.py
@@ -2,7 +2,7 @@ import json
 import os
 import tempfile
 from pathlib import Path
-from typing import Dict, Optional, Union
+from typing import Dict, List, Optional, Union
 
 import requests
 from huggingface_hub import hf_api
@@ -268,6 +268,8 @@ class ModelHubMixin:
         token: Optional[str] = None,
         branch: Optional[str] = None,
         create_pr: Optional[bool] = None,
+        allow_patterns: Optional[Union[List[str], str]] = None,
+        ignore_patterns: Optional[Union[List[str], str]] = None,
         # TODO (release 0.12): signature must be the following
         # repo_id: str,
         # *,
@@ -278,10 +280,15 @@ class ModelHubMixin:
         # branch: Optional[str] = None,
         # create_pr: Optional[bool] = None,
         # config: Optional[dict] = None,
+        # allow_patterns: Optional[Union[List[str], str]] = None,
+        # ignore_patterns: Optional[Union[List[str], str]] = None,
     ) -> str:
         """
         Upload model checkpoint to the Hub.
 
+        Use `allow_patterns` and `ignore_patterns` to precisely filter which files
+        should be pushed to the hub. See [`upload_folder`] reference for more details.
+
         Parameters:
             repo_id (`str`, *optional*):
                 Repository name to which push.
@@ -304,6 +311,10 @@ class ModelHubMixin:
                 Defaults to `False`.
             config (`dict`, *optional*):
                 Configuration object to be saved alongside the model weights.
+            allow_patterns (`List[str]` or `str`, *optional*):
+                If provided, only files matching at least one pattern are pushed.
+            ignore_patterns (`List[str]` or `str`, *optional*):
+                If provided, files matching any of the patterns are not pushed.
 
         Returns:
             The url of the commit of your model in the given repository.
@@ -334,6 +345,8 @@ class ModelHubMixin:
                     commit_message=commit_message,
                     revision=branch,
                     create_pr=create_pr,
+                    allow_patterns=allow_patterns,
+                    ignore_patterns=ignore_patterns,
                 )
 
         # If the repo id is None, it means we use the deprecated version using Git
diff --git a/src/huggingface_hub/keras_mixin.py b/src/huggingface_hub/keras_mixin.py
index f3199cdb..eb1f1cac 100644
--- a/src/huggingface_hub/keras_mixin.py
+++ b/src/huggingface_hub/keras_mixin.py
@@ -5,7 +5,7 @@ import tempfile
 import warnings
 from pathlib import Path
 from shutil import copytree, rmtree
-from typing import Any, Dict, Optional, Union
+from typing import Any, Dict, List, Optional, Union
 from urllib.parse import quote
 
 import yaml
@@ -329,6 +329,8 @@ def push_to_hub_keras(
     repo_id: Optional[str] = None,  # optional only until 0.12
     branch: Optional[str] = None,
     create_pr: Optional[bool] = None,
+    allow_patterns: Optional[Union[List[str], str]] = None,
+    ignore_patterns: Optional[Union[List[str], str]] = None,
     **model_save_kwargs,
     # TODO (release 0.12): signature must be the following
     # model,
@@ -341,6 +343,8 @@ def push_to_hub_keras(
     # branch: Optional[str] = None,
     # create_pr: Optional[bool] = None,
     # config: Optional[dict] = None,
+    # allow_patterns: Optional[Union[List[str], str]] = None,
+    # ignore_patterns: Optional[Union[List[str], str]] = None,
     # log_dir: Optional[str] = None,
     # include_optimizer: Optional[bool] = False,
     # tags: Optional[Union[list, str]] = None,
@@ -351,6 +355,9 @@ def push_to_hub_keras(
     Upload model checkpoint or tokenizer files to the Hub while synchronizing a
     local clone of the repo in `repo_path_or_name`.
 
+    Use `allow_patterns` and `ignore_patterns` to precisely filter which files should be
+    pushed to the hub. See [`upload_folder`] reference for more details.
+
     Parameters:
         model (`Keras.Model`):
             The [Keras
@@ -377,6 +384,10 @@ def push_to_hub_keras(
             Defaults to `False`.
         config (`dict`, *optional*):
             Configuration object to be saved alongside the model weights.
+        allow_patterns (`List[str]` or `str`, *optional*):
+            If provided, only files matching at least one pattern are pushed.
+        ignore_patterns (`List[str]` or `str`, *optional*):
+            If provided, files matching any of the patterns are not pushed.
         log_dir (`str`, *optional*):
             TensorBoard logging directory to be pushed. The Hub automatically
             hosts and displays a TensorBoard instance if log files are included
@@ -438,7 +449,12 @@ def push_to_hub_keras(
             #       duplicate code from `upload_folder`. We are not directly using
             #       `upload_folder` since we want to add delete operations to the
             #       commit as well.
-            operations += _prepare_upload_folder_commit(saved_path, path_in_repo="")
+            operations += _prepare_upload_folder_commit(
+                saved_path,
+                path_in_repo="",
+                allow_patterns=allow_patterns,
+                ignore_patterns=ignore_patterns,
+            )
             pr_url = api.create_commit(
                 repo_type="model",
                 repo_id=repo_id,
diff --git a/src/huggingface_hub/utils/__init__.py b/src/huggingface_hub/utils/__init__.py
index 0f0da23b..9267b25d 100644
--- a/src/huggingface_hub/utils/__init__.py
+++ b/src/huggingface_hub/utils/__init__.py
@@ -22,6 +22,7 @@ from ._errors import (
     RepositoryNotFoundError,
     RevisionNotFoundError,
 )
+from ._paths import filter_repo_objects
 from ._subprocess import run_subprocess
 from .tqdm import (
     are_progress_bars_disabled,
diff --git a/src/huggingface_hub/utils/_deprecation.py b/src/huggingface_hub/utils/_deprecation.py
index 5958432c..19bd9753 100644
--- a/src/huggingface_hub/utils/_deprecation.py
+++ b/src/huggingface_hub/utils/_deprecation.py
@@ -1,7 +1,7 @@
 import warnings
 from functools import wraps
 from inspect import Parameter, signature
-from typing import Set
+from typing import Optional, Set
 
 
 def _deprecate_positional_args(*, version: str):
@@ -49,7 +49,9 @@ def _deprecate_positional_args(*, version: str):
     return _inner_deprecate_positional_args
 
 
-def _deprecate_arguments(*, version: str, deprecated_args: Set[str]):
+def _deprecate_arguments(
+    *, version: str, deprecated_args: Set[str], custom_message: Optional[str] = None
+):
     """Decorator to issue warnings when using deprecated arguments.
 
     TODO: could be useful to be able to set a custom error message.
@@ -59,6 +61,9 @@ def _deprecate_arguments(*, version: str, deprecated_args: Set[str]):
             The version when deprecated arguments will result in error.
         deprecated_args (`List[str]`):
             List of the arguments to be deprecated.
+        custom_message (`str`, *optional*):
+            Warning message that is raised. If not passed, a default warning message
+            will be created.
     """
 
     def _inner_deprecate_positional_args(f):
@@ -77,12 +82,14 @@ def _deprecate_arguments(*, version: str, deprecated_args: Set[str]):
 
             # Warn and proceed
             if len(used_deprecated_args) > 0:
-                warnings.warn(
+                message = (
                     f"Deprecated argument(s) used in '{f.__name__}':"
                     f" {', '.join(used_deprecated_args)}. Will not be supported from"
-                    f" version '{version}'.",
-                    FutureWarning,
+                    f" version '{version}'."
                 )
+                if custom_message is not None:
+                    message += "\n\n" + custom_message
+                warnings.warn(message, FutureWarning)
             return f(*args, **kwargs)
 
         return inner_f
diff --git a/src/huggingface_hub/utils/_paths.py b/src/huggingface_hub/utils/_paths.py
new file mode 100644
index 00000000..3a496696
--- /dev/null
+++ b/src/huggingface_hub/utils/_paths.py
@@ -0,0 +1,122 @@
+# coding=utf-8
+# Copyright 2022-present, the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Contains utilities to handle paths in Huggingface Hub."""
+from fnmatch import fnmatch
+from pathlib import Path
+from typing import Callable, Generator, Iterable, List, Optional, TypeVar, Union
+
+
+T = TypeVar("T")
+
+
+def filter_repo_objects(
+    items: Iterable[T],
+    *,
+    allow_patterns: Optional[Union[List[str], str]] = None,
+    ignore_patterns: Optional[Union[List[str], str]] = None,
+    key: Optional[Callable[[T], str]] = None,
+) -> Generator[T, None, None]:
+    """Filter repo objects based on an allowlist and a denylist.
+
+    Input must be a list of paths (`str` or `Path`) or a list of arbitrary objects.
+    In the later case, `key` must be provided and specifies a function of one argument
+    that is used to extract a path from each element in iterable.
+
+    Patterns are Unix shell-style wildcards which are NOT regular expressions. See
+    https://docs.python.org/3/library/fnmatch.html for more details.
+
+    Args:
+        items (`Iterable`):
+            List of items to filter.
+        allow_patterns (`str` or `List[str]`, *optional*):
+            Patterns constituing the allowlist. If provided, item paths must match at
+            least one pattern from the allowlist.
+        ignore_patterns (`str` or `List[str]`, *optional*):
+            Patterns constituing the denylist. If provided, item paths must not match
+            any patterns from the denylist.
+        key (`Callable[[T], str]`, *optional*):
+            Single-argument function to extract a path from each item. If not provided,
+            the `items` must already be `str` or `Path`.
+
+    Returns:
+        Filtered list of objects, as a generator.
+
+    Raises:
+        :class:`ValueError`:
+            If `key` is not provided and items are not `str` or `Path`.
+
+    Example usage with paths:
+    ```python
+    >>> # Filter only PDFs that are not hidden.
+    >>> list(filter_repo_objects(
+    ...     ["aaa.PDF", "bbb.jpg", ".ccc.pdf", ".ddd.png"],
+    ...     allow_patterns=["*.pdf"],
+    ...     ignore_patterns=[".*"],
+    ... ))
+    ["aaa.pdf"]
+    ```
+
+    Example usage with objects:
+    ```python
+    >>> list(filter_repo_objects(
+    ... [
+    ...     CommitOperationAdd(path_or_fileobj="/tmp/aaa.pdf", path_in_repo="aaa.pdf")
+    ...     CommitOperationAdd(path_or_fileobj="/tmp/bbb.jpg", path_in_repo="bbb.jpg")
+    ...     CommitOperationAdd(path_or_fileobj="/tmp/.ccc.pdf", path_in_repo=".ccc.pdf")
+    ...     CommitOperationAdd(path_or_fileobj="/tmp/.ddd.png", path_in_repo=".ddd.png")
+    ... ],
+    ... allow_patterns=["*.pdf"],
+    ... ignore_patterns=[".*"],
+    ... key=lambda x: x.repo_in_path
+    ... ))
+    [CommitOperationAdd(path_or_fileobj="/tmp/aaa.pdf", path_in_repo="aaa.pdf")]
+    ```
+    """
+    if isinstance(allow_patterns, str):
+        allow_patterns = [allow_patterns]
+
+    if isinstance(ignore_patterns, str):
+        ignore_patterns = [ignore_patterns]
+
+    if key is None:
+
+        def _identity(item: T) -> str:
+            if isinstance(item, str):
+                return item
+            if isinstance(item, Path):
+                return str(item)
+            raise ValueError(
+                f"Please provide `key` argument in `filter_repo_objects`: `{item}` is"
+                " not a string."
+            )
+
+        key = _identity  # Items must be `str` or `Path`, otherwise raise ValueError
+
+    for item in items:
+        path = key(item)
+
+        # Skip if there's an allowlist and path doesn't match any
+        if allow_patterns is not None and not any(
+            fnmatch(path, r) for r in allow_patterns
+        ):
+            continue
+
+        # Skip if there's a denylist and path matches any
+        if ignore_patterns is not None and any(
+            fnmatch(path, r) for r in ignore_patterns
+        ):
+            continue
+
+        yield item


Test patch:
diff --git a/.github/workflows/python-tests.yml b/.github/workflows/python-tests.yml
index 779b9f19..be9614ff 100644
--- a/.github/workflows/python-tests.yml
+++ b/.github/workflows/python-tests.yml
@@ -5,8 +5,12 @@ on:
     branches:
       - main
       - ci_*
+    paths-ignore:
+      - "docs/**"
   pull_request:
     types: [assigned, opened, synchronize, reopened]
+    paths-ignore:
+      - "docs/**"
 
 env:
   HUGGINGFACE_CO_STAGING: yes
diff --git a/tests/test_snapshot_download.py b/tests/test_snapshot_download.py
index 89714ad3..a5b2ef0e 100644
--- a/tests/test_snapshot_download.py
+++ b/tests/test_snapshot_download.py
@@ -12,7 +12,11 @@ from huggingface_hub.hf_api import HfFolder
 from huggingface_hub.utils import logging
 
 from .testing_constants import ENDPOINT_STAGING, TOKEN, USER
-from .testing_utils import retry_endpoint, set_write_permission_and_retry
+from .testing_utils import (
+    expect_deprecation,
+    retry_endpoint,
+    set_write_permission_and_retry,
+)
 
 
 logger = logging.get_logger(__name__)
@@ -340,15 +344,23 @@ class SnapshotDownloadTests(unittest.TestCase):
             # folder name contains the revision's commit sha.
             self.assertTrue(self.second_commit_hash in storage_folder)
 
+    # TODO: from version 0.12, replace `regex` by 'patterns` and remove expected
+    #       deprecation. Could be possible to already do tests with `allow_patterns` and
+    #       `ignore_patterns` but current test implementation is convenient since it
+    #       covers both old and new naming + check deprecation.
+    @expect_deprecation("snapshot_download")
     def test_download_model_with_allow_regex(self):
         self.check_download_model_with_regex("*.txt")
 
+    @expect_deprecation("snapshot_download")
     def test_download_model_with_allow_regex_list(self):
         self.check_download_model_with_regex(["dummy_file.txt", "dummy_file_2.txt"])
 
+    @expect_deprecation("snapshot_download")
     def test_download_model_with_ignore_regex(self):
         self.check_download_model_with_regex(".gitattributes", allow=False)
 
+    @expect_deprecation("snapshot_download")
     def test_download_model_with_ignore_regex_list(self):
         self.check_download_model_with_regex(["*.git*", "*.pt"], allow=False)
 
diff --git a/tests/test_utils_deprecation.py b/tests/test_utils_deprecation.py
index 0f74acd7..63ff0f32 100644
--- a/tests/test_utils_deprecation.py
+++ b/tests/test_utils_deprecation.py
@@ -68,3 +68,41 @@ class TestDeprecationUtils(unittest.TestCase):
 
         with pytest.warns(FutureWarning):
             dummy_b_c_deprecated("A", b="B", c="C")
+
+    def test_deprecate_arguments_with_default_warning_message(self) -> None:
+        """Test default warning message when deprecating arguments."""
+
+        @_deprecate_arguments(version="xxx", deprecated_args={"a"})
+        def dummy_deprecated_default_message(a: str = "a") -> None:
+            pass
+
+        # Default message
+        with pytest.warns(FutureWarning) as record:
+            dummy_deprecated_default_message(a="a")
+        self.assertEqual(len(record), 1)
+        self.assertEqual(
+            record[0].message.args[0],
+            "Deprecated argument(s) used in 'dummy_deprecated_default_message': a. Will"
+            " not be supported from version 'xxx'.",
+        )
+
+    def test_deprecate_arguments_with_custom_warning_message(self) -> None:
+        """Test custom warning message when deprecating arguments."""
+
+        @_deprecate_arguments(
+            version="xxx",
+            deprecated_args={"a"},
+            custom_message="This is a custom message.",
+        )
+        def dummy_deprecated_custom_message(a: str = "a") -> None:
+            pass
+
+        # Custom message
+        with pytest.warns(FutureWarning) as record:
+            dummy_deprecated_custom_message(a="a")
+        self.assertEqual(len(record), 1)
+        self.assertEqual(
+            record[0].message.args[0],
+            "Deprecated argument(s) used in 'dummy_deprecated_custom_message': a. Will"
+            " not be supported from version 'xxx'.\n\nThis is a custom message.",
+        )
diff --git a/tests/test_utils_paths.py b/tests/test_utils_paths.py
new file mode 100644
index 00000000..8c968ad0
--- /dev/null
+++ b/tests/test_utils_paths.py
@@ -0,0 +1,103 @@
+import unittest
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Callable, List, Optional, Union
+
+from huggingface_hub.utils import filter_repo_objects
+
+
+@dataclass
+class DummyObject:
+    path: Path
+
+
+DUMMY_FILES = ["not_hidden.pdf", "profile.jpg", ".hidden.pdf", ".hidden_picture.png"]
+DUMMY_PATHS = [Path(path) for path in DUMMY_FILES]
+DUMMY_OBJECTS = [DummyObject(path=path) for path in DUMMY_FILES]
+
+
+class TestPathsUtils(unittest.TestCase):
+    def test_get_all_pdfs(self) -> None:
+        """Get all PDFs even hidden ones."""
+        self._check(
+            items=DUMMY_FILES,
+            expected_items=["not_hidden.pdf", ".hidden.pdf"],
+            allow_patterns=["*.pdf"],
+        )
+
+    def test_get_all_pdfs_except_hidden(self) -> None:
+        """Get all PDFs except hidden ones."""
+        self._check(
+            items=DUMMY_FILES,
+            expected_items=["not_hidden.pdf"],
+            allow_patterns=["*.pdf"],
+            ignore_patterns=[".*"],
+        )
+
+    def test_get_all_pdfs_except_hidden_using_single_pattern(self) -> None:
+        """Get all PDFs except hidden ones, using single pattern."""
+        self._check(
+            items=DUMMY_FILES,
+            expected_items=["not_hidden.pdf"],
+            allow_patterns="*.pdf",  # not a list
+            ignore_patterns=".*",  # not a list
+        )
+
+    def test_get_all_images(self) -> None:
+        """Get all images."""
+        self._check(
+            items=DUMMY_FILES,
+            expected_items=["profile.jpg", ".hidden_picture.png"],
+            allow_patterns=["*.png", "*.jpg"],
+        )
+
+    def test_get_all_images_except_hidden_from_paths(self) -> None:
+        """Get all images except hidden ones, from Path list."""
+        self._check(
+            items=DUMMY_PATHS,
+            expected_items=[Path("profile.jpg")],
+            allow_patterns=["*.png", "*.jpg"],
+            ignore_patterns=".*",
+        )
+
+    def test_get_all_images_except_hidden_from_objects(self) -> None:
+        """Get all images except hidden ones, from object list."""
+        self._check(
+            items=DUMMY_OBJECTS,
+            expected_items=[DummyObject(path="profile.jpg")],
+            allow_patterns=["*.png", "*.jpg"],
+            ignore_patterns=".*",
+            key=lambda x: x.path,
+        )
+
+    def test_filter_objects_key_not_provided(self) -> None:
+        """Test ValueError is raised if filtering non-string objects."""
+        with self.assertRaisesRegex(ValueError, "Please provide `key` argument"):
+            list(
+                filter_repo_objects(
+                    items=DUMMY_OBJECTS,
+                    allow_patterns=["*.png", "*.jpg"],
+                    ignore_patterns=".*",
+                )
+            )
+
+    def _check(
+        self,
+        items: List[Any],
+        expected_items: List[Any],
+        allow_patterns: Optional[Union[List[str], str]] = None,
+        ignore_patterns: Optional[Union[List[str], str]] = None,
+        key: Optional[Callable[[Any], str]] = None,
+    ) -> None:
+        """Run `filter_repo_objects` and check output against expected result."""
+        self.assertListEqual(
+            list(
+                filter_repo_objects(
+                    items=items,
+                    allow_patterns=allow_patterns,
+                    ignore_patterns=ignore_patterns,
+                    key=key,
+                )
+            ),
+            expected_items,
+        )