The Hugging Face Hub is the go-to place for sharing machine learning
models, demos, datasets, and metrics.
huggingface_hub library helps you interact with
the Hub without leaving your development environment. You can create and manage
repositories easily, download and upload files, and get useful model and dataset
metadata from the Hub.
To get started, you should install the
huggingface_hub library. You can install the
pip install huggingface_hub
conda install -c conda-forge huggingface_hub
Repositories on the Hub are git version controlled, and users can download a single file or the whole repository. You can use the hf_hub_download() function to download files. This function will download and cache a file on your local disk. The next time you need that file, it will load from your cache, so you don’t need to re-download it.
You will need the repository id and the filename of the file you want to download. For example, to download the Pegasus model configuration file:
from huggingface_hub import hf_hub_download hf_hub_download(repo_id="google/pegasus-xsum", filename="config.json")
To download a specific version of the file, use the
revision parameter to specify the
branch name, tag, or commit hash. If you choose to use the commit hash, it must be the
full-length hash instead of the shorter 7-character commit hash:
from huggingface_hub import hf_hub_download hf_hub_download( repo_id="google/pegasus-xsum", filename="config.json", revision="4d33b01d79672f27f001f6abade33f22d993b151" )
For more details and options, see the API reference for hf_hub_download().
In a lot of cases, you must be logged in with a Hugging Face account to interact with the Hub: download private repos, upload files, create PRs,… Create an account if you don’t already have one, and then sign in to get your User Access Token from your Settings page. The User Access Token is used to authenticate your identity to the Hub.
Once you have your User Access Token, run the following command in your terminal:
Or if you prefer to work from a Jupyter or Colaboratory notebook, then login with
from huggingface_hub import notebook_login notebook_login()
You can also provide your token to the functions and methods. This way you don’t need to store your token anywhere.
Once you are logged in, all requests to the Hub will use your access token by default.
If you want to disable implicit use of your token, you should set the
HF_HUB_DISABLE_IMPLICIT_TOKEN environment variable.
Once you’ve registered and logged in, create a repository with the create_repo() function:
from huggingface_hub import HfApi api = HfApi() api.create_repo(repo_id="super-cool-model")
If you want your repository to be private, then:
from huggingface_hub import HfApi api = HfApi() api.create_repo(repo_id="super-cool-model", private=True)
Private repositories will not be visible to anyone except yourself.
To create a repository or to push content to the Hub, you must provide a User Access
Token that has the
write permission. You can choose the permission when creating the
token in your Settings page.
Use the upload_file() function to add a file to your newly created repository. You need to specify:
- The path of the file to upload.
- The path of the file in the repository.
- The repository id of where you want to add the file.
from huggingface_hub import HfApi api = HfApi() api.upload_file(path_or_fileobj="/home/lysandre/dummy-test/README.md", path_in_repo="README.md", repo_id="lysandre/test-model", )
To upload more than one file at a time, take a look at this guide which will introduce you to several methods for uploading files (with or without git).
huggingface_hub library provides an easy way for users to interact with the Hub
with Python. To learn more about how you can manage your files and repositories on the
Hub, we recommend reading our how-to guides for how to: