You are viewing v0.17.1 version.
A newer version
v0.27.0 is available.
Filesystem API
The HfFileSystem class provides a pythonic file interface to the Hugging Face Hub based on fssepc.
HfFileSystem
HfFileSystem is based on fsspec, so it is compatible with most of the APIs that it offers. For more details, check out our guide and the fsspec’s API Reference.
class huggingface_hub.HfFileSystem
< source >( *args **kwargs )
Parameters
-
endpoint (
str, optional) — The endpoint to use. If not provided, the default one (https://huggingface.co) is used. -
token (
str, optional) — Authentication token, obtained withHfApi.loginmethod. Will default to the stored token.
Access a remote Hugging Face Hub repository as if were a local file system.
Usage:
>>> from huggingface_hub import HfFileSystem
>>> fs = HfFileSystem()
>>> # List files
>>> fs.glob("my-username/my-model/*.bin")
['my-username/my-model/pytorch_model.bin']
>>> fs.ls("datasets/my-username/my-dataset", detail=False)
['datasets/my-username/my-dataset/.gitattributes', 'datasets/my-username/my-dataset/README.md', 'datasets/my-username/my-dataset/data.json']
>>> # Read/write files
>>> with fs.open("my-username/my-model/pytorch_model.bin") as f:
... data = f.read()
>>> with fs.open("my-username/my-model/pytorch_model.bin", "wb") as f:
... f.write(data)__init__
< source >( *args endpoint: typing.Optional[str] = None token: typing.Optional[str] = None **storage_options )
ls
< source >( path: str detail: bool = True refresh: bool = False revision: typing.Optional[str] = None **kwargs )
List the contents of a directory.