Hub Python Library documentation

Filesystem API

You are viewing v0.19.3 version. A newer version v0.21.3 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Filesystem API

The HfFileSystem class provides a pythonic file interface to the Hugging Face Hub based on fsspec.

HfFileSystem

HfFileSystem is based on fsspec, so it is compatible with most of the APIs that it offers. For more details, check out our guide and fsspec’s API Reference.

class huggingface_hub.HfFileSystem

< >

( *args **kwargs )

Parameters

  • endpoint (str, optional) — The endpoint to use. If not provided, the default one (https://huggingface.co) is used.
  • token (str, optional) — Authentication token, obtained with HfApi.login method. Will default to the stored token.

Access a remote Hugging Face Hub repository as if were a local file system.

Usage:

>>> from huggingface_hub import HfFileSystem

>>> fs = HfFileSystem()

>>> # List files
>>> fs.glob("my-username/my-model/*.bin")
['my-username/my-model/pytorch_model.bin']
>>> fs.ls("datasets/my-username/my-dataset", detail=False)
['datasets/my-username/my-dataset/.gitattributes', 'datasets/my-username/my-dataset/README.md', 'datasets/my-username/my-dataset/data.json']

>>> # Read/write files
>>> with fs.open("my-username/my-model/pytorch_model.bin") as f:
...     data = f.read()
>>> with fs.open("my-username/my-model/pytorch_model.bin", "wb") as f:
...     f.write(data)

__init__

< >

( *args endpoint: typing.Optional[str] = None token: typing.Optional[str] = None **storage_options )

resolve_path

< >

( path: str revision: typing.Optional[str] = None )

ls

< >

( path: str detail: bool = True refresh: bool = False revision: typing.Optional[str] = None **kwargs )

List the contents of a directory.