Repository: huggingface/huggingface_hub
Pull request #1142 (2022-10-31 13:13:15)

Add Hub verification token to evaluation metadata
This PR adds a new `verifyToken` field to the evaluation metadata schema to enable the Hub to verify whether evaluation results come from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) vs self-reported.

This is need to enable the following PRs:

* https://github.com/huggingface/moon-landing/pull/4037
* https://github.com/huggingface/autotrain-backend/pull/823

Note: I'm not sure what's the best practice to handle camelCase fields in Python codebases, so please let me know if I should refactor variable names :)

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_1142). All of your documentation changes will be reflected on that endpoint.
osanseviero: Is the `huggingface_hub` library the best place to add support for these more internal use cases? Except us, nobody else should use this (and one could argue the same for the previous `verified` parameter). Looking at the PRs, I was wondering if doing something like 

```
card.data.eval_results[0].metrics_token = "token"
```

or similar would make more sense, rather than supporting setting verified tokens out of the box

lewtun: > Is the `huggingface_hub` library the best place to add support for these more internal use cases? Except us, nobody else should use this (and one could argue the same for the previous `verified` parameter). Looking at the PRs, I was wondering if doing something like
> 
> ```
> card.data.eval_results[0].metrics_token = "token"
> ```
> 
> or similar would make more sense, rather than supporting setting verified tokens out of the box

We currently rely on the `metadata_update()` function in AutoTrain to create Hub PRs for evaluation (see [here](https://github.com/huggingface/autotrain-backend/blob/618a80613e6a2c3950f0b802896e941e632dbb81/zeus/zeus/run/common.py#L294)), which means metadata like `verifyToken` is excluded unless it's included in the `huggingface_hub` schema. 

If desired, we could roll our own version of this function using the commit API functions in `huggingface_hub`, although that's a bit more work on our side.

cc @abhishekkrthakur 
julien-c: hmm yeah i think this should live in internal code (wdyt also @coyotte508 @allendorf?)
coyotte508: IMO `verified` / `verifyToken` should at least be in the schema if only to be read? I mean they will be in model cards, I don't see why they shouldn't be in the Metrics type.

Take this with a grain of salt, I'm only a little familiar with the hub library and not at all with how metadata is used in its scope.
Wauplin: > IMO verified / verifyToken should at least be in the schema if only to be read? I mean they will be in model cards / in the JSON from the hub API (at least verified will be), I don't see why they shouldn't be in the metrics type.

I would be of @coyotte508 's opinion, at least if I understand correctly what this token is. For what I have read in https://github.com/huggingface/moon-landing/issues/3263 (and more precisely https://github.com/huggingface/moon-landing/issues/3263#issuecomment-1239320855) (internal links), we want to generate a token that proves the metrics have been verified by HF and we want this token to be public so that everyone can check it is a valid one, right ? If that's the case, I'd definitely be ok to have them in `hfh`, maybe with more explanations in the docstring part on what this token is.


lewtun: > we want to generate a token that proves the metrics have been verified by HF and we want this token to be public so that everyone can check it is a valid one, right ? If that's the case, I'd definitely be ok to have them in `hfh`, maybe with more explanations in the docstring part on what this token is.

Yes, `verifyToken` is a signed JWT that enables HF to verify if a model evaluation came from our evaluation service. I'm happy to clarify this with a better docstring (I'm also equally happy to move this logic to AutoTrain if desired)
julien-c: Ok i hadn't actually read the code so i thought the signing code was here (which wouldn't make sense IMO)

Yes the types in `repocard_data.py` could be here indeed (ie. the actual types from the API) but `metrics_token` not sure if it's useful to add it here (especially confusing as it's not the name from the API)

Up to you

Wauplin: Agree that `metrics_token` is misleading as the only `token` we have at the moment is the User Access Token. But well documented I guess it should be good. I don't expect a lot of users to use this feature anyway so we can expect the one interested in it will take the time to understand how it words/read the docstring.
lewtun: Thanks for the feedback @julien-c @Wauplin ! I'll update this PR accordingly
Wauplin: (@lewtun I have just merged main in your branch to make the CI pass. You need to do a `git pull` locally before making any new commits)
codecov[bot]: # [Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/1142?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Base: **84.61**% // Head: **84.63**% // Increases project coverage by **`+0.01%`** :tada:
> Coverage data is based on head [(`5bc4f53`)](https://codecov.io/gh/huggingface/huggingface_hub/pull/1142?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) compared to base [(`a02f2b9`)](https://codecov.io/gh/huggingface/huggingface_hub/commit/a02f2b9520e97e56a6a8fb0af2aa7710c317dba9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1142      +/-   ##
==========================================
+ Coverage   84.61%   84.63%   +0.01%     
==========================================
  Files          41       41              
  Lines        4096     4100       +4     
==========================================
+ Hits         3466     3470       +4     
  Misses        630      630              
```


| [Impacted Files](https://codecov.io/gh/huggingface/huggingface_hub/pull/1142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Coverage Δ | |
|---|---|---|
| [src/huggingface\_hub/repocard.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/1142/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9yZXBvY2FyZC5weQ==) | `93.85% <100.00%> (+0.06%)` | :arrow_up: |
| [src/huggingface\_hub/repocard\_data.py](https://codecov.io/gh/huggingface/huggingface_hub/pull/1142/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9yZXBvY2FyZF9kYXRhLnB5) | `98.41% <100.00%> (+0.02%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/huggingface/huggingface_hub/pull/1142?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).

lewtun: @Wauplin thanks for the reviews - I think this should be good to go once the CI runs. Let me know if you want any more changes :)
Wauplin: @lewtun sorry I just added a last comment in https://github.com/huggingface/huggingface_hub/pull/1142#discussion_r1012858629 and then we should be good to go I think. Thanks in advance :pray: 

Patch:
diff --git a/docs/source/how-to-manage.mdx b/docs/source/how-to-manage.mdx
index 312f36d2..b9157236 100644
--- a/docs/source/how-to-manage.mdx
+++ b/docs/source/how-to-manage.mdx
@@ -45,7 +45,7 @@ By default, [`create_repo`] creates a model repository. But you can use the `rep
 ```py
 >>> from huggingface_hub import create_repo
 >>> create_repo("lysandre/test-dataset", repo_type="dataset")
-'https://huggingface.co/lysandre/test-dataset'
+'https://huggingface.co/datasets/lysandre/test-dataset'
 ```
 
 When you create a repository, you can set your repository visibility with the `private` parameter. For example, if you want to create a private repository:
diff --git a/src/huggingface_hub/_login.py b/src/huggingface_hub/_login.py
index 72ca67d4..f4968077 100644
--- a/src/huggingface_hub/_login.py
+++ b/src/huggingface_hub/_login.py
@@ -17,14 +17,25 @@ from getpass import getpass
 from typing import List, Optional
 
 from .commands._cli_utils import ANSI
+from .commands.delete_cache import _ask_for_confirmation_no_tui
 from .hf_api import HfApi
-from .utils import HfFolder, is_google_colab, is_notebook, logging, run_subprocess
+from .utils import (
+    HfFolder,
+    is_google_colab,
+    is_notebook,
+    list_credential_helpers,
+    logging,
+    run_subprocess,
+    set_git_credential,
+    unset_git_credential,
+)
+from .utils._deprecation import _deprecate_method
 
 
 logger = logging.get_logger(__name__)
 
 
-def login(token: Optional[str] = None) -> None:
+def login(token: Optional[str] = None, add_to_git_credential: bool = True) -> None:
     """Login the machine to access the Hub.
 
     The `token` is persisted in cache and set as a git credential. Once done, the machine
@@ -50,6 +61,12 @@ def login(token: Optional[str] = None) -> None:
     Args:
         token (`str`, *optional*):
             User access token to generate from https://huggingface.co/settings/token.
+        add_to_git_credential (`bool`, defaults to `False`):
+            If `True`, token will be set as git credential. If no git credential helper
+            is configured, a warning will be displayed to the user. If `token` is `None`,
+            the value of `add_to_git_credential` is ignored and will be prompted again
+            to the end user.
+
 
     Raises:
         [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError)
@@ -61,7 +78,13 @@ def login(token: Optional[str] = None) -> None:
             If running in a notebook but `ipywidgets` is not installed.
     """
     if token is not None:
-        _login(token)
+        if not add_to_git_credential:
+            print(
+                "Token will not been saved to git credential helper. Pass"
+                " `add_to_git_credential=True` if you want to set the git"
+                " credential as well."
+            )
+        _login(token, add_to_git_credential=add_to_git_credential)
     elif is_notebook():
         notebook_login()
     else:
@@ -78,7 +101,7 @@ def logout() -> None:
         print("Not logged in!")
         return
     HfFolder.delete_token()
-    HfApi.unset_access_token()
+    unset_git_credential()
     print("Successfully logged out.")
 
 
@@ -108,7 +131,10 @@ def interpreter_login() -> None:
     To login, `huggingface_hub` now requires a token generated from https://huggingface.co/settings/tokens .
     """
     )
-    _login(token=getpass("Token: "))
+    token = getpass("Token: ")
+    add_to_git_credential = _ask_for_confirmation_no_tui("Add token as git credential?")
+
+    _login(token=token, add_to_git_credential=add_to_git_credential)
 
 
 ###
@@ -159,12 +185,16 @@ def notebook_login() -> None:
     )
 
     token_widget = widgets.Password(description="Token:")
+    git_checkbox_widget = widgets.Checkbox(
+        value=True, description="Add token as git credential?"
+    )
     token_finish_button = widgets.Button(description="Login")
 
     login_token_widget = widgets.VBox(
         [
             widgets.HTML(NOTEBOOK_LOGIN_TOKEN_HTML_START),
             token_widget,
+            git_checkbox_widget,
             token_finish_button,
             widgets.HTML(NOTEBOOK_LOGIN_TOKEN_HTML_END),
         ],
@@ -175,10 +205,11 @@ def notebook_login() -> None:
     # On click events
     def login_token_event(t):
         token = token_widget.value
+        add_to_git_credential = git_checkbox_widget.value
         # Erase token and clear value to make sure it's not saved in the notebook.
         token_widget.value = ""
         clear_output()
-        _login(token)
+        _login(token, add_to_git_credential=add_to_git_credential)
 
     token_finish_button.on_click(login_token_event)
 
@@ -188,34 +219,58 @@ def notebook_login() -> None:
 ###
 
 
-def _login(token: str) -> None:
+def _login(token: str, add_to_git_credential: bool) -> None:
     hf_api = HfApi()
     if token.startswith("api_org"):
         raise ValueError("You must use your personal account token.")
     if not hf_api._is_valid_token(token=token):
         raise ValueError("Invalid token passed!")
-    hf_api.set_access_token(token)
+    print("Token is valid.")
+
+    if add_to_git_credential:
+        if _is_git_credential_helper_configured():
+            set_git_credential(token)
+            print(
+                "Your token has been saved in your configured git credential helpers"
+                f" ({','.join(list_credential_helpers())})."
+            )
+        else:
+            print("Token has not been saved to git credential helper.")
+
     HfFolder.save_token(token)
-    print("Login successful")
     print("Your token has been saved to", HfFolder.path_token)
+    print("Login successful")
+
+
+def _is_git_credential_helper_configured() -> bool:
+    """Check if a git credential helper is configured.
+
+    Warns user if not the case (except for Google Colab where "store" is set by default
+    by `huggingface_hub`).
+    """
+    helpers = list_credential_helpers()
+    if len(helpers) > 0:
+        return True  # Do not warn: at least 1 helper is set
 
     # Only in Google Colab to avoid the warning message
     # See https://github.com/huggingface/huggingface_hub/issues/1043#issuecomment-1247010710
     if is_google_colab():
         _set_store_as_git_credential_helper_globally()
-
-    helpers = _currently_setup_credential_helpers()
-
-    if "store" not in helpers:
-        print(
-            ANSI.red(
-                "Authenticated through git-credential store but this isn't the helper"
-                " defined on your machine.\nYou might have to re-authenticate when"
-                " pushing to the Hugging Face Hub. Run the following command in your"
-                " terminal in case you want to set this credential helper as the"
-                " default\n\ngit config --global credential.helper store"
-            )
+        return True  # Do not warn: "store" is used by default in Google Colab
+
+    # Otherwise, warn user
+    print(
+        ANSI.red(
+            "Cannot authenticate through git-credential as no helper is defined on your"
+            " machine.\nYou might have to re-authenticate when pushing to the Hugging"
+            " Face Hub.\nRun the following command in your terminal in case you want to"
+            " set the 'store' credential helper as default.\n\ngit config --global"
+            " credential.helper store\n\nRead"
+            " https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage for more"
+            " details."
         )
+    )
+    return False
 
 
 def _set_store_as_git_credential_helper_globally() -> None:
@@ -236,18 +291,8 @@ def _set_store_as_git_credential_helper_globally() -> None:
         raise EnvironmentError(exc.stderr)
 
 
-def _currently_setup_credential_helpers(directory=None) -> List[str]:
-    try:
-        output = run_subprocess(
-            "git config --list".split(),
-            directory,
-        ).stdout.split("\n")
-
-        current_credential_helpers = []
-        for line in output:
-            if "credential.helper" in line:
-                current_credential_helpers.append(line.split("=")[-1])
-    except subprocess.CalledProcessError as exc:
-        raise EnvironmentError(exc.stderr)
-
-    return current_credential_helpers
+@_deprecate_method(
+    version="0.14", message="Please use `list_credential_helpers` instead."
+)
+def _currently_setup_credential_helpers(directory: Optional[str] = None) -> List[str]:
+    return list_credential_helpers(directory)
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index ec61e391..934f5440 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -15,7 +15,6 @@
 import json
 import os
 import re
-import subprocess
 import warnings
 from dataclasses import dataclass, field
 from pathlib import Path
@@ -52,17 +51,20 @@ from .constants import (
     REPO_TYPES_URL_PREFIXES,
     SPACES_SDK_TYPES,
 )
-from .utils import HfFolder  # noqa: F401 # imported for backward compatibility
-from .utils import (
+from .utils import (  # noqa: F401 # imported for backward compatibility
+    HfFolder,
     HfHubHTTPError,
     build_hf_headers,
+    erase_from_credential_store,
     filter_repo_objects,
     hf_raise_for_status,
     logging,
     parse_datetime,
+    read_from_credential_store,
     validate_hf_hub_args,
+    write_to_credential_store,
 )
-from .utils._deprecation import _deprecate_positional_args
+from .utils._deprecation import _deprecate_method, _deprecate_positional_args
 from .utils._typing import Literal, TypedDict
 from .utils.endpoint_helpers import (
     AttributeDictionary,
@@ -558,88 +560,6 @@ class DatasetSearchArguments(AttributeDictionary):
         self["author"] = author_dict
 
 
-def write_to_credential_store(username: str, password: str):
-    with subprocess.Popen(
-        "git credential-store store".split(),
-        stdin=subprocess.PIPE,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.STDOUT,
-    ) as process:
-        input_username = f"username={username.lower()}"
-        input_password = f"password={password}"
-
-        assert (
-            process.stdin is not None
-        ), "subprocess must be opened with subprocess.PIPE"
-        process.stdin.write(
-            f"url={ENDPOINT}\n{input_username}\n{input_password}\n\n".encode("utf-8")
-        )
-        process.stdin.flush()
-
-
-def read_from_credential_store(
-    username=None,
-) -> Tuple[Union[str, None], Union[str, None]]:
-    """
-    Reads the credential store relative to huggingface.co. If no `username` is
-    specified, will read the first entry for huggingface.co, otherwise will read
-    the entry corresponding to the username specified.
-
-    The username returned will be all lowercase.
-    """
-    with subprocess.Popen(
-        "git credential-store get".split(),
-        stdin=subprocess.PIPE,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.STDOUT,
-    ) as process:
-        standard_input = f"url={ENDPOINT}\n"
-
-        if username is not None:
-            standard_input += f"username={username.lower()}\n"
-
-        standard_input += "\n"
-
-        assert (
-            process.stdin is not None
-        ), "subprocess must be opened with subprocess.PIPE"
-        assert (
-            process.stdout is not None
-        ), "subprocess must be opened with subprocess.PIPE"
-        process.stdin.write(standard_input.encode("utf-8"))
-        process.stdin.flush()
-        output = process.stdout.read().decode("utf-8")
-
-    if len(output) == 0:
-        return None, None
-
-    username, password = [line for line in output.split("\n") if len(line) != 0]
-    return username.split("=")[1], password.split("=")[1]
-
-
-def erase_from_credential_store(username=None):
-    """
-    Erases the credential store relative to huggingface.co. If no `username` is
-    specified, will erase the first entry for huggingface.co, otherwise will
-    erase the entry corresponding to the username specified.
-    """
-    with subprocess.Popen(
-        "git credential-store erase".split(),
-        stdin=subprocess.PIPE,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.STDOUT,
-    ) as process:
-        standard_input = f"url={ENDPOINT}\n"
-
-        if username is not None:
-            standard_input += f"username={username.lower()}\n"
-
-        standard_input += "\n"
-
-        process.stdin.write(standard_input.encode("utf-8"))
-        process.stdin.flush()
-
-
 class HfApi:
     def __init__(
         self, endpoint: Optional[str] = None, token: Optional[str] = None
@@ -692,6 +612,13 @@ class HfApi:
             return False
 
     @staticmethod
+    @_deprecate_method(
+        version="0.14",
+        message=(
+            "`HfApi.set_access_token` is deprecated as it is very ambiguous. Use"
+            " `login` or `set_git_credential` instead."
+        ),
+    )
     def set_access_token(access_token: str):
         """
         Saves the passed access token so git can correctly authenticate the
@@ -704,6 +631,13 @@ class HfApi:
         write_to_credential_store(USERNAME_PLACEHOLDER, access_token)
 
     @staticmethod
+    @_deprecate_method(
+        version="0.14",
+        message=(
+            "`HfApi.unset_access_token` is deprecated as it is very ambiguous. Use"
+            " `login` or `unset_git_credential` instead."
+        ),
+    )
     def unset_access_token():
         """
         Resets the user's access token.
diff --git a/src/huggingface_hub/repocard.py b/src/huggingface_hub/repocard.py
index 7fb536ed..95c898ad 100644
--- a/src/huggingface_hub/repocard.py
+++ b/src/huggingface_hub/repocard.py
@@ -27,6 +27,7 @@ from huggingface_hub.utils import is_jinja_available, yaml_dump
 
 from .constants import REPOCARD_NAME
 from .utils import EntryNotFoundError, validate_hf_hub_args
+from .utils._deprecation import _deprecate_positional_args
 from .utils.logging import get_logger
 
 
@@ -79,7 +80,22 @@ class RepoCard:
 
         </Tip>
         """
+
+        # Set the content of the RepoCard, as well as underlying .data and .text attributes.
+        # See the `content` property setter for more details.
         self.content = content
+
+    @property
+    def content(self):
+        """The content of the RepoCard, including the YAML block and the Markdown body."""
+        line_break = _detect_line_ending(self._content) or "\n"
+        return f"---{line_break}{self.data.to_yaml(line_break=line_break)}{line_break}---{line_break}{self.text}"
+
+    @content.setter
+    def content(self, content: str):
+        """Set the content of the RepoCard."""
+        self._content = content
+
         match = REGEX_YAML_BLOCK.search(content)
         if match:
             # Metadata found in the YAML block
@@ -101,8 +117,7 @@ class RepoCard:
         self.data = self.card_data_class(**data_dict)
 
     def __str__(self):
-        line_break = _detect_line_ending(self.content) or "\n"
-        return f"---{line_break}{self.data.to_yaml(line_break=line_break)}{line_break}---{line_break}{self.text}"
+        return self.content
 
     def save(self, filepath: Union[Path, str]):
         r"""Save a RepoCard to a file.
@@ -533,7 +548,9 @@ def metadata_save(local_path: Union[str, Path], data: Dict) -> None:
         readme.close()
 
 
+@_deprecate_positional_args(version="0.12")
 def metadata_eval_result(
+    *,
     model_pretty_name: str,
     task_pretty_name: str,
     task_id: str,
@@ -547,6 +564,7 @@ def metadata_eval_result(
     dataset_config: Optional[str] = None,
     dataset_split: Optional[str] = None,
     dataset_revision: Optional[str] = None,
+    metrics_verification_token: Optional[str] = None,
 ) -> Dict:
     """
     Creates a metadata dict with the result from a model evaluated on a dataset.
@@ -572,9 +590,7 @@ def metadata_eval_result(
             The name of the metric configuration used in `load_metric()`.
             Example: bleurt-large-512 in `load_metric("bleurt", "bleurt-large-512")`.
         metrics_verified (`bool`, *optional*, defaults to `False`):
-            If true, indicates that evaluation was generated by Hugging Face (vs. self-reported).
-            If a user tries to push self-reported metric results with verified=True, the push
-            will be rejected.
+            Indicates whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Automatically computed by Hugging Face, do not set.
         dataset_config (`str`, *optional*):
             Example: fr. The name of the dataset configuration used in `load_dataset()`.
         dataset_split (`str`, *optional*):
@@ -582,6 +598,8 @@ def metadata_eval_result(
         dataset_revision (`str`, *optional*):
             Example: 5503434ddd753f426f4b38109466949a1217c2bb. The name of the dataset dataset revision
             used in `load_dataset()`.
+        metrics_verification_token (`bool`, *optional*):
+            A JSON Web Token that is used to verify whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not.
 
     Returns:
         `dict`: a metadata dict with the result from a model evaluated on a dataset.
@@ -649,6 +667,7 @@ def metadata_eval_result(
                     dataset_type=dataset_id,
                     metric_config=metrics_config,
                     verified=metrics_verified,
+                    verify_token=metrics_verification_token,
                     dataset_config=dataset_config,
                     dataset_split=dataset_split,
                     dataset_revision=dataset_revision,
diff --git a/src/huggingface_hub/repocard_data.py b/src/huggingface_hub/repocard_data.py
index 213fe37a..46b7f55c 100644
--- a/src/huggingface_hub/repocard_data.py
+++ b/src/huggingface_hub/repocard_data.py
@@ -51,7 +51,9 @@ class EvalResult:
         metric_args (`Dict[str, Any]`, *optional*):
             The arguments passed during `Metric.compute()`. Example for `bleu`: max_order: 4
         verified (`bool`, *optional*):
-            If true, indicates that evaluation was generated by Hugging Face (vs. self-reported).
+            Indicates whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Automatically computed by Hugging Face, do not set.
+        verify_token (`str`, *optional*):
+            A JSON Web Token that is used to verify whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not.
     """
 
     # Required
@@ -113,9 +115,12 @@ class EvalResult:
     # Example for `bleu`: max_order: 4
     metric_args: Optional[Dict[str, Any]] = None
 
-    # If true, indicates that evaluation was generated by Hugging Face (vs. self-reported).
+    # Indicates whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Automatically computed by Hugging Face, do not set.
     verified: Optional[bool] = None
 
+    # A JSON Web Token that is used to verify whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not.
+    verify_token: Optional[str] = None
+
 
 @dataclass
 class CardData:
@@ -416,6 +421,7 @@ def model_index_to_eval_results(
                 metric_args = metric.get("args")
                 metric_config = metric.get("config")
                 verified = metric.get("verified")
+                verify_token = metric.get("verifyToken")
 
                 eval_result = EvalResult(
                     task_type=task_type,  # Required
@@ -432,6 +438,7 @@ def model_index_to_eval_results(
                     metric_args=metric_args,
                     metric_config=metric_config,
                     verified=verified,
+                    verify_token=verify_token,
                 )
                 eval_results.append(eval_result)
     return name, eval_results
@@ -521,6 +528,7 @@ def eval_results_to_model_index(
                     "config": result.metric_config,
                     "args": result.metric_args,
                     "verified": result.verified,
+                    "verifyToken": result.verify_token,
                 }
                 for result in results
             ],
diff --git a/src/huggingface_hub/utils/__init__.py b/src/huggingface_hub/utils/__init__.py
index ae13c26c..75c08f9b 100644
--- a/src/huggingface_hub/utils/__init__.py
+++ b/src/huggingface_hub/utils/__init__.py
@@ -39,6 +39,14 @@ from ._errors import (
     hf_raise_for_status,
 )
 from ._fixes import yaml_dump
+from ._git_credential import (
+    erase_from_credential_store,
+    list_credential_helpers,
+    read_from_credential_store,
+    set_git_credential,
+    unset_git_credential,
+    write_to_credential_store,
+)
 from ._headers import build_hf_headers, get_token_to_send
 from ._hf_folder import HfFolder
 from ._http import http_backoff
@@ -63,7 +71,7 @@ from ._runtime import (
     is_tf_available,
     is_torch_available,
 )
-from ._subprocess import run_subprocess
+from ._subprocess import run_interactive_subprocess, run_subprocess
 from ._validators import (
     HFValidationError,
     smoothly_deprecate_use_auth_token,
diff --git a/src/huggingface_hub/utils/_git_credential.py b/src/huggingface_hub/utils/_git_credential.py
new file mode 100644
index 00000000..1c5239d7
--- /dev/null
+++ b/src/huggingface_hub/utils/_git_credential.py
@@ -0,0 +1,186 @@
+# coding=utf-8
+# Copyright 2022-present, the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Contains utilities to manage Git credentials."""
+import subprocess
+from typing import List, Optional, Tuple, Union
+
+from ..constants import ENDPOINT
+from ._deprecation import _deprecate_method
+from ._subprocess import run_interactive_subprocess, run_subprocess
+
+
+def list_credential_helpers(folder: Optional[str] = None) -> List[str]:
+    """Return the list of git credential helpers configured.
+
+    See https://git-scm.com/docs/gitcredentials.
+
+    Credentials are saved in all configured helpers (store, cache, macos keychain,...).
+    Calls "`git credential approve`" internally. See https://git-scm.com/docs/git-credential.
+
+    Args:
+        folder (`str`, *optional*):
+            The folder in which to check the configured helpers.
+    """
+    try:
+        output = run_subprocess("git config --list", folder=folder).stdout
+        # NOTE: If user has set an helper for a custom URL, it will not we caught here.
+        #       Example: `credential.https://huggingface.co.helper=store`
+        #       See: https://github.com/huggingface/huggingface_hub/pull/1138#discussion_r1013324508
+        return sorted(  # Sort for nice printing
+            {  # Might have some duplicates
+                line.split("=")[-1].split()[0]
+                for line in output.split("\n")
+                if "credential.helper" in line
+            }
+        )
+    except subprocess.CalledProcessError as exc:
+        raise EnvironmentError(exc.stderr)
+
+
+def set_git_credential(
+    token: str, username: str = "hf_user", folder: Optional[str] = None
+) -> None:
+    """Save a username/token pair in git credential for HF Hub registry.
+
+    Credentials are saved in all configured helpers (store, cache, macos keychain,...).
+    Calls "`git credential approve`" internally. See https://git-scm.com/docs/git-credential.
+
+    Args:
+        username (`str`, defaults to `"hf_user"`):
+            A git username. Defaults to `"hf_user"`, the default user used in the Hub.
+        token (`str`, defaults to `"hf_user"`):
+            A git password. In practice, the User Access Token for the Hub.
+            See https://huggingface.co/settings/tokens.
+        folder (`str`, *optional*):
+            The folder in which to check the configured helpers.
+    """
+    with run_interactive_subprocess("git credential approve", folder=folder) as (
+        stdin,
+        _,
+    ):
+        stdin.write(
+            f"url={ENDPOINT}\nusername={username.lower()}\npassword={token}\n\n"
+        )
+        stdin.flush()
+
+
+def unset_git_credential(
+    username: str = "hf_user", folder: Optional[str] = None
+) -> None:
+    """Erase credentials from git credential for HF Hub registry.
+
+    Credentials are erased from the configured helpers (store, cache, macos
+    keychain,...), if any. If `username` is not provided, any credential configured for
+    HF Hub endpoint is erased.
+    Calls "`git credential erase`" internally. See https://git-scm.com/docs/git-credential.
+
+    Args:
+        username (`str`, defaults to `"hf_user"`):
+            A git username. Defaults to `"hf_user"`, the default user used in the Hub.
+        folder (`str`, *optional*):
+            The folder in which to check the configured helpers.
+    """
+    with run_interactive_subprocess("git credential reject", folder=folder) as (
+        stdin,
+        _,
+    ):
+        standard_input = f"url={ENDPOINT}\n"
+        if username is not None:
+            standard_input += f"username={username.lower()}\n"
+        standard_input += "\n"
+
+        stdin.write(standard_input)
+        stdin.flush()
+
+
+@_deprecate_method(
+    version="0.14",
+    message=(
+        "Please use `huggingface_hub.set_git_credential` instead as it allows"
+        " the user to chose which git-credential tool to use."
+    ),
+)
+def write_to_credential_store(username: str, password: str) -> None:
+    with run_interactive_subprocess("git credential-store store") as (stdin, _):
+        input_username = f"username={username.lower()}"
+        input_password = f"password={password}"
+        stdin.write(f"url={ENDPOINT}\n{input_username}\n{input_password}\n\n")
+        stdin.flush()
+
+
+@_deprecate_method(
+    version="0.14",
+    message=(
+        "Please open an issue on https://github.com/huggingface/huggingface_hub if this"
+        " a useful feature for you."
+    ),
+)
+def read_from_credential_store(
+    username: Optional[str] = None,
+) -> Union[Tuple[str, str], Tuple[None, None]]:
+    """
+    Reads the credential store relative to huggingface.co.
+
+    Args:
+        username (`str`, *optional*):
+            A username to filter to search. If not specified, the first entry under
+            `huggingface.co` endpoint is returned.
+
+    Returns:
+        `Tuple[str, str]` or `Tuple[None, None]`: either a username/password pair or
+        None/None if credential has not been found. The returned username is always
+        lowercase.
+    """
+    with run_interactive_subprocess("git credential-store get") as (stdin, stdout):
+        standard_input = f"url={ENDPOINT}\n"
+        if username is not None:
+            standard_input += f"username={username.lower()}\n"
+        standard_input += "\n"
+
+        stdin.write(standard_input)
+        stdin.flush()
+        output = stdout.read()
+
+    if len(output) == 0:
+        return None, None
+
+    username, password = [line for line in output.split("\n") if len(line) != 0]
+    return username.split("=")[1], password.split("=")[1]
+
+
+@_deprecate_method(
+    version="0.14",
+    message=(
+        "Please use `huggingface_hub.unset_git_credential` instead as it allows"
+        " the user to chose which git-credential tool to use."
+    ),
+)
+def erase_from_credential_store(username: Optional[str] = None) -> None:
+    """
+    Erases the credential store relative to huggingface.co.
+
+    Args:
+        username (`str`, *optional*):
+            A username to filter to search. If not specified, all entries under
+            `huggingface.co` endpoint is erased.
+    """
+    with run_interactive_subprocess("git credential-store erase") as (stdin, _):
+        standard_input = f"url={ENDPOINT}\n"
+        if username is not None:
+            standard_input += f"username={username.lower()}\n"
+        standard_input += "\n"
+
+        stdin.write(standard_input)
+        stdin.flush()
diff --git a/src/huggingface_hub/utils/_subprocess.py b/src/huggingface_hub/utils/_subprocess.py
index ab57ad69..e3b95d28 100644
--- a/src/huggingface_hub/utils/_subprocess.py
+++ b/src/huggingface_hub/utils/_subprocess.py
@@ -13,10 +13,12 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License
+"""Contains utilities to easily handle subprocesses in `huggingface_hub`."""
 import os
 import subprocess
+from contextlib import contextmanager
 from pathlib import Path
-from typing import List, Optional, Union
+from typing import IO, Generator, List, Optional, Tuple, Union
 
 from .logging import get_logger
 
@@ -65,3 +67,52 @@ def run_subprocess(
         cwd=folder or os.getcwd(),
         **kwargs,
     )
+
+
+@contextmanager
+def run_interactive_subprocess(
+    command: Union[str, List[str]],
+    folder: Optional[Union[str, Path]] = None,
+    **kwargs,
+) -> Generator[Tuple[IO[str], IO[str]], None, None]:
+    """Run a subprocess in an interactive mode in a context manager.
+
+    Args:
+        command (`str` or `List[str]`):
+            The command to execute as a string or list of strings.
+        folder (`str`, *optional*):
+            The folder in which to run the command. Defaults to current working
+            directory (from `os.getcwd()`).
+        kwargs (`Dict[str]`):
+            Keyword arguments to be passed to the `subprocess.run` underlying command.
+
+    Returns:
+        `Tuple[IO[str], IO[str]]`: A tuple with `stdin` and `stdout` to interact
+        with the process (input and output are utf-8 encoded).
+
+    Example:
+    ```python
+    with _interactive_subprocess("git credential-store get") as (stdin, stdout):
+        # Write to stdin
+        stdin.write("url=hf.co\nusername=obama\n".encode("utf-8"))
+        stdin.flush()
+
+        # Read from stdout
+        output = stdout.read().decode("utf-8")
+    ```
+    """
+    if isinstance(command, str):
+        command = command.split()
+
+    with subprocess.Popen(
+        command,
+        stdin=subprocess.PIPE,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+        encoding="utf-8",
+        cwd=folder or os.getcwd(),
+        **kwargs,
+    ) as process:
+        assert process.stdin is not None, "subprocess is opened as subprocess.PIPE"
+        assert process.stdout is not None, "subprocess is opened as subprocess.PIPE"
+        yield process.stdin, process.stdout


Test patch:
diff --git a/tests/test_cache_layout.py b/tests/test_cache_layout.py
index da095c89..e0da8245 100644
--- a/tests/test_cache_layout.py
+++ b/tests/test_cache_layout.py
@@ -9,7 +9,7 @@ from huggingface_hub.utils import logging
 from huggingface_hub.utils._errors import EntryNotFoundError
 
 from .testing_constants import ENDPOINT_STAGING, TOKEN, USER
-from .testing_utils import repo_name, with_production_testing
+from .testing_utils import expect_deprecation, repo_name, with_production_testing
 
 
 logger = logging.get_logger(__name__)
@@ -315,6 +315,7 @@ class ReferenceUpdates(unittest.TestCase):
     _api = HfApi(endpoint=ENDPOINT_STAGING, token=TOKEN)
 
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_fastai_integration.py b/tests/test_fastai_integration.py
index c0e4acb7..70ca049f 100644
--- a/tests/test_fastai_integration.py
+++ b/tests/test_fastai_integration.py
@@ -15,7 +15,7 @@ from huggingface_hub.utils import (
 )
 
 from .testing_constants import ENDPOINT_STAGING, TOKEN, USER
-from .testing_utils import repo_name, set_write_permission_and_retry
+from .testing_utils import expect_deprecation, repo_name, set_write_permission_and_retry
 
 
 WORKING_REPO_SUBDIR = f"fixtures/working_repo_{__name__.split('.')[-1]}"
@@ -65,6 +65,7 @@ else:
 @require_fastai_fastcore
 class TestFastaiUtils(TestCase):
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_hf_api.py b/tests/test_hf_api.py
index 713213d6..dd36c304 100644
--- a/tests/test_hf_api.py
+++ b/tests/test_hf_api.py
@@ -34,7 +34,7 @@ from huggingface_hub._commit_api import (
     CommitOperationDelete,
     fetch_upload_modes,
 )
-from huggingface_hub._login import _login
+from huggingface_hub._login import _login, _set_store_as_git_credential_helper_globally
 from huggingface_hub.community import DiscussionComment, DiscussionWithDetails
 from huggingface_hub.constants import (
     REPO_TYPE_DATASET,
@@ -85,6 +85,7 @@ from .testing_utils import (
     DUMMY_MODEL_ID,
     DUMMY_MODEL_ID_REVISION_ONE_SPECIFIC_COMMIT,
     SAMPLE_DATASET_IDENTIFIER,
+    expect_deprecation,
     repo_name,
     require_git_lfs,
     retry_endpoint,
@@ -112,13 +113,16 @@ class HfApiCommonTest(unittest.TestCase):
 
 
 class HfApiLoginTest(HfApiCommonTest):
+    @expect_deprecation("erase_from_credential_store")
     def setUp(self) -> None:
         erase_from_credential_store(USERNAME_PLACEHOLDER)
 
     @classmethod
+    @expect_deprecation("set_access_token")
     def tearDownClass(cls) -> None:
         cls._api.set_access_token(TOKEN)
 
+    @expect_deprecation("read_from_credential_store")
     def test_login_git_credentials(self):
         self.assertTupleEqual(
             read_from_credential_store(USERNAME_PLACEHOLDER), (None, None)
@@ -133,6 +137,7 @@ class HfApiLoginTest(HfApiCommonTest):
             read_from_credential_store(USERNAME_PLACEHOLDER), (None, None)
         )
 
+    @expect_deprecation("read_from_credential_store")
     def test_login_cli(self):
         self._api.set_access_token(TOKEN)
         self.assertTupleEqual(
@@ -144,7 +149,8 @@ class HfApiLoginTest(HfApiCommonTest):
             read_from_credential_store(USERNAME_PLACEHOLDER), (None, None)
         )
 
-        _login(token=TOKEN)
+        _set_store_as_git_credential_helper_globally()
+        _login(token=TOKEN, add_to_git_credential=True)
         self.assertTupleEqual(
             read_from_credential_store(USERNAME_PLACEHOLDER),
             (USERNAME_PLACEHOLDER, TOKEN),
@@ -158,11 +164,12 @@ class HfApiLoginTest(HfApiCommonTest):
         with pytest.raises(
             ValueError, match="You must use your personal account token."
         ):
-            _login(token="api_org_dummy_token")
+            _login(token="api_org_dummy_token", add_to_git_credential=True)
 
 
 class HfApiCommonTestWithLogin(HfApiCommonTest):
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
@@ -1773,6 +1780,7 @@ class HfFolderTest(unittest.TestCase):
 @require_git_lfs
 class HfLargefilesTest(HfApiCommonTest):
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_hubmixin.py b/tests/test_hubmixin.py
index e11b7662..6da2c523 100644
--- a/tests/test_hubmixin.py
+++ b/tests/test_hubmixin.py
@@ -67,6 +67,7 @@ class HubMixingTest(HubMixingCommonTest):
         )
 
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_keras_integration.py b/tests/test_keras_integration.py
index d68ce6f7..378a60e1 100644
--- a/tests/test_keras_integration.py
+++ b/tests/test_keras_integration.py
@@ -85,6 +85,7 @@ class CommonKerasTest(unittest.TestCase):
         )
 
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_repocard.py b/tests/test_repocard.py
index 1cda3d7e..87745620 100644
--- a/tests/test_repocard.py
+++ b/tests/test_repocard.py
@@ -49,7 +49,12 @@ from .testing_constants import (
     TOKEN,
     USER,
 )
-from .testing_utils import repo_name, retry_endpoint, set_write_permission_and_retry
+from .testing_utils import (
+    expect_deprecation,
+    repo_name,
+    retry_endpoint,
+    set_write_permission_and_retry,
+)
 
 
 SAMPLE_CARDS_DIR = Path(__file__).parent / "fixtures/cards"
@@ -223,6 +228,7 @@ class RepocardMetadataUpdateTest(unittest.TestCase):
     _api = HfApi(endpoint=ENDPOINT_STAGING, token=TOKEN)
 
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
@@ -606,7 +612,7 @@ class RepoCardTest(TestCaseWithCapLog):
             metrics="acc",
         )
         # Mock what RepoCard.from_template does so we can test w/o Jinja2
-        content = f"{card_data.to_yaml()}\n\n# MyModel\n\nHello, world!"
+        content = f"---\n{card_data.to_yaml()}\n---\n\n# MyModel\n\nHello, world!"
         card = RepoCard(content)
 
         url = f"{ENDPOINT_STAGING_BASIC_AUTH}/{repo_id}/resolve/main/README.md"
@@ -637,7 +643,7 @@ class RepoCardTest(TestCaseWithCapLog):
             metrics="acc",
         )
         # Mock what RepoCard.from_template does so we can test w/o Jinja2
-        content = f"{card_data.to_yaml()}\n\n# MyModel\n\nHello, world!"
+        content = f"---\n{card_data.to_yaml()}\n---\n\n# MyModel\n\nHello, world!"
         card = RepoCard(content)
 
         url = f"{ENDPOINT_STAGING_BASIC_AUTH}/api/models/{repo_id}/discussions"
@@ -656,6 +662,14 @@ class RepoCardTest(TestCaseWithCapLog):
         card = RepoCard.load(card_path)
         self.assertIn("\r\n", str(card))
 
+    def test_updating_text_updates_content(self):
+        sample_path = SAMPLE_CARDS_DIR / "sample_simple.md"
+        card = RepoCard.load(sample_path)
+        card.text = "Hello, world!"
+        self.assertEqual(
+            card.content, f"---\n{card.data.to_yaml()}\n---\nHello, world!"
+        )
+
 
 class ModelCardTest(TestCaseWithCapLog):
     def test_model_card_with_invalid_model_index(self):
diff --git a/tests/test_repocard_data.py b/tests/test_repocard_data.py
index 514345b5..3f57e279 100644
--- a/tests/test_repocard_data.py
+++ b/tests/test_repocard_data.py
@@ -91,6 +91,8 @@ class ModelCardDataTest(unittest.TestCase):
                             {
                                 "type": "acc",
                                 "value": 0.9,
+                                "verified": True,
+                                "verifyToken": 1234,
                             }
                         ],
                     },
@@ -106,6 +108,8 @@ class ModelCardDataTest(unittest.TestCase):
         self.assertEqual(eval_results[1].metric_value, 0.9)
         self.assertEqual(eval_results[2].task_type, "image-classification")
         self.assertEqual(eval_results[2].dataset_type, "beans")
+        self.assertEqual(eval_results[2].verified, True)
+        self.assertEqual(eval_results[2].verify_token, 1234)
 
     def test_card_data_requires_model_name_for_eval_results(self):
         with pytest.raises(
diff --git a/tests/test_repository.py b/tests/test_repository.py
index 93533513..41d237f2 100644
--- a/tests/test_repository.py
+++ b/tests/test_repository.py
@@ -65,6 +65,7 @@ class RepositoryCommonTest(unittest.TestCase):
 
 class RepositoryTest(RepositoryCommonTest):
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
@@ -1455,6 +1456,7 @@ class RepositoryOfflineTest(RepositoryCommonTest):
         files = os.listdir(current_head_repo.local_dir)
         self.assertIn("file.txt", files)
 
+    @expect_deprecation("set_access_token")
     def test_repo_user(self):
         api = HfApi(endpoint=ENDPOINT_STAGING)
         token = TOKEN
@@ -1483,6 +1485,7 @@ class RepositoryOfflineTest(RepositoryCommonTest):
         self.assertEqual(username, user["fullname"])
         self.assertEqual(email, user["email"])
 
+    @expect_deprecation("set_access_token")
     def test_repo_passed_user(self):
         api = HfApi(endpoint=ENDPOINT_STAGING)
         token = TOKEN
@@ -1513,6 +1516,7 @@ class RepositoryOfflineTest(RepositoryCommonTest):
         self.assertEqual(username, "RANDOM_USER")
         self.assertEqual(email, "EMAIL@EMAIL.EMAIL")
 
+    @expect_deprecation("_currently_setup_credential_helpers")
     def test_correct_helper(self):
         subprocess.run(
             ["git", "config", "--global", "credential.helper", "get"],
@@ -1616,6 +1620,7 @@ class RepositoryOfflineTest(RepositoryCommonTest):
 
 class RepositoryDatasetTest(RepositoryCommonTest):
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_snapshot_download.py b/tests/test_snapshot_download.py
index 8de98e74..fb9231ff 100644
--- a/tests/test_snapshot_download.py
+++ b/tests/test_snapshot_download.py
@@ -25,6 +25,7 @@ class SnapshotDownloadTests(unittest.TestCase):
     _api = HfApi(endpoint=ENDPOINT_STAGING, token=TOKEN)
 
     @classmethod
+    @expect_deprecation("set_access_token")
     def setUpClass(cls):
         """
         Share this valid token in all tests below.
diff --git a/tests/test_utils_git_credentials.py b/tests/test_utils_git_credentials.py
new file mode 100644
index 00000000..476ae923
--- /dev/null
+++ b/tests/test_utils_git_credentials.py
@@ -0,0 +1,70 @@
+import time
+import unittest
+from pathlib import Path
+
+import pytest
+
+from huggingface_hub.constants import ENDPOINT
+from huggingface_hub.utils import run_interactive_subprocess, run_subprocess
+from huggingface_hub.utils._git_credential import (
+    list_credential_helpers,
+    set_git_credential,
+    unset_git_credential,
+)
+
+
+STORE_AND_CACHE_HELPERS_CONFIG = """
+[credential]
+    helper = store
+    helper = cache --timeout 30000
+"""
+
+
+@pytest.mark.usefixtures("fx_cache_dir")
+class TestGitCredentials(unittest.TestCase):
+    cache_dir: Path
+
+    def setUp(self):
+        """Initialize and configure a local repo.
+
+        Avoid to configure git helpers globally on a contributor's machine.
+        """
+        run_subprocess("git init", folder=self.cache_dir)
+        with (self.cache_dir / ".git" / "config").open("w") as f:
+            f.write(STORE_AND_CACHE_HELPERS_CONFIG)
+
+    def test_list_credential_helpers(self) -> None:
+        helpers = list_credential_helpers(folder=self.cache_dir)
+        self.assertIn("cache", helpers)
+        self.assertIn("store", helpers)
+
+    def test_set_and_unset_git_credential(self) -> None:
+        username = "hf_test_user_" + str(round(time.time()))  # make username unique
+
+        # Set credentials
+        set_git_credential(
+            token="hf_test_token", username=username, folder=self.cache_dir
+        )
+
+        # Check credentials are stored
+        with run_interactive_subprocess(
+            "git credential fill", folder=self.cache_dir
+        ) as (stdin, stdout):
+            stdin.write(f"url={ENDPOINT}\nusername={username}\n\n")
+            stdin.flush()
+            output = stdout.read()
+        self.assertIn("password=hf_test_token", output)
+
+        # Unset credentials
+        unset_git_credential(username=username, folder=self.cache_dir)
+
+        # Check credentials are NOT stored
+        # Cannot check with `git credential fill` as it would hang forever: only
+        # checking `store` helper instead.
+        with run_interactive_subprocess(
+            "git credential-store get", folder=self.cache_dir
+        ) as (stdin, stdout):
+            stdin.write(f"url={ENDPOINT}\nusername={username}\n\n")
+            stdin.flush()
+            output = stdout.read()
+        self.assertEqual("", output)