Hugging Face's logo

Hub Python Library documentation

Utilities

You are viewing v0.10.1 version. A newer version v0.26.2 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Utilities

Controlling the logging of huggingface_hub

The huggingface_hub package exposes a logging utility to control the logging level of the package itself. You can import it as such:

from huggingface_hub import logging

Then, you may define the verbosity in order to update the amount of logs you’ll see:

from huggingface_hub import logging

logging.set_verbosity_error()
logging.set_verbosity_warning()
logging.set_verbosity_info()
logging.set_verbosity_debug()

logging.set_verbosity(...)

The levels should be understood as follows:

  • error: only show critical logs about usage which may result in an error or unexpected behavior.
  • warning: show logs that aren’t critical but usage may result in unintended behavior. Additionally, important informative logs may be shown.
  • info: show most logs, including some verbose logging regarding what is happening under the hood. If something is behaving in an unexpected manner, we recommend switching the verbosity level to this in order to get more information.
  • debug: show all logs, including some internal logs which may be used to track exactly what’s happening under the hood.

huggingface_hub.utils.logging.get_verbosity

< >

( )

Return the current level for the HuggingFace Hub’s root logger.

HuggingFace Hub has following logging levels:

  • huggingface_hub.logging.CRITICAL, huggingface_hub.logging.FATAL
  • huggingface_hub.logging.ERROR
  • huggingface_hub.logging.WARNING, huggingface_hub.logging.WARN
  • huggingface_hub.logging.INFO
  • huggingface_hub.logging.DEBUG

huggingface_hub.utils.logging.set_verbosity

< >

( verbosity: int )

Parameters

  • verbosity (int) — Logging level, e.g., huggingface_hub.logging.DEBUG and huggingface_hub.logging.INFO.

Sets the level for the HuggingFace Hub’s root logger.

huggingface_hub.utils.logging.set_verbosity_info

< >

( )

Sets the verbosity to logging.INFO.

huggingface_hub.utils.logging.set_verbosity_debug

< >

( )

Sets the verbosity to logging.DEBUG.

huggingface_hub.utils.logging.set_verbosity_warning

< >

( )

Sets the verbosity to logging.WARNING.

huggingface_hub.utils.logging.set_verbosity_error

< >

( )

Sets the verbosity to logging.ERROR.

huggingface_hub.utils.logging.disable_propagation

< >

( )

Disable propagation of the library log outputs. Note that log propagation is disabled by default.

huggingface_hub.utils.logging.enable_propagation

< >

( )

Enable propagation of the library log outputs. Please disable the HuggingFace Hub’s default handler to prevent double logging if the root logger has been configured.

Repo-specific helper methods

The methods exposed below are relevant when modifying modules from the huggingface_hub library itself. Using these shouldn’t be necessary if you use huggingface_hub and you don’t modify them.

huggingface_hub.utils.logging.get_logger

< >

( name: typing.Optional[str] = None )

Parameters

  • name (str, optional) — The name of the logger to get, usually the filename

Returns a logger with the specified name. This function is not supposed to be directly accessed by library users.

Example:

>>> from huggingface_hub import get_logger

>>> logger = get_logger(__file__)
>>> logger.set_verbosity_info()

Handling HTTP errors

huggingface_hub defines its own HTTP errors to refine the HTTPError raised by requests with additional information sent back by the server.

Raise for status

hf_raise_for_status() is meant to be the central method to “raise for status” from any request made to the Hub. It wraps the base requests.raise_for_status to provide additional information. Any HTTPError thrown is converted into a HfHubHTTPError.

import requests
from huggingface_hub.utils import hf_raise_for_status, HfHubHTTPError

response = requests.post(...)
try:
    hf_raise_for_status(response)
except HfHubHTTPError as e:
    print(str(e)) # formatted message
    e.request_id, e.server_message # details returned by server

    # Complete the error message with additional information once it's raised
    e.append_to_message("\n`create_commit` expects the repository to exist.")
    raise

huggingface_hub.utils.hf_raise_for_status

< >

( response: Response endpoint_name: typing.Optional[str] = None )

Parameters

  • response (Response) — Response from the server.
  • endpoint_name (str, optional) — Name of the endpoint that has been called. If provided, the error message will be more complete.

Internal version of response.raise_for_status() that will refine a potential HTTPError. Raised exception will be an instance of HfHubHTTPError.

This helper is meant to be the unique method to raise_for_status when making a call to the Hugging Face Hub.

Example:

    import requests
    from huggingface_hub.utils import hf_raise_for_status, HfHubHTTPError

    response = requests.post(...)
    try:
        hf_raise_for_status(response)
    except HfHubHTTPError as e:
        print(str(e)) # formatted message
        e.request_id, e.server_message # details returned by server

        # Complete the error message with additional information once it's raised
        e.append_to_message("
ate_commit` expects the repository to exist.")
        raise

Raises when the request has failed:

  • RepositoryNotFoundError If the repository to download from cannot be found. This may be because it doesn’t exist, because repo_type is not set correctly, or because the repo is private and you do not have access.
  • RevisionNotFoundError If the repository exists but the revision couldn’t be find.
  • EntryNotFoundError If the repository exists but the entry (e.g. the requested file) couldn’t be find.
  • BadRequestError If request failed with a HTTP 400 BadRequest error.
  • HfHubHTTPError If request failed for a reason not listed above.

HTTP errors

Here is a list of HTTP errors thrown in huggingface_hub.

HfHubHTTPError

HfHubHTTPError is the parent class for any HF Hub HTTP error. It takes care of parsing the server response and format the error message to provide as much information to the user as possible.

class huggingface_hub.utils.HfHubHTTPError

< >

( message: str response: typing.Optional[requests.models.Response] )

HTTPError to inherit from for any custom HTTP Error raised in HF Hub.

Any HTTPError is converted at least into a HfHubHTTPError. If some information is sent back by the server, it will be added to the error message.

Added details:

  • Request id from “X-Request-Id” header if exists.
  • Server error message if we can found one in the response body.

Example:

    import requests
    from huggingface_hub.utils import hf_raise_for_status, HfHubHTTPError

    response = requests.post(...)
    try:
        hf_raise_for_status(response)
    except HfHubHTTPError as e:
        print(str(e)) # formatted message
        e.request_id, e.server_message # details returned by server

        # Complete the error message with additional information once it's raised
        e.append_to_message("
ate_commit` expects the repository to exist.")
        raise

append_to_message

< >

( additional_message: str )

Append additional information to the HfHubHTTPError initial message.

RepositoryNotFoundError

class huggingface_hub.utils.RepositoryNotFoundError

< >

( message: str response: typing.Optional[requests.models.Response] )

Raised when trying to access a hf.co URL with an invalid repository name, or with a private repo name the user does not have access to.

Example:

>>> from huggingface_hub import model_info
>>> model_info("<non_existent_repository>")
(...)
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: PvMw_VjBMjVdMz53WKIzP)

Repository Not Found for url: https://huggingface.co/api/models/%3Cnon_existent_repository%3E.
Please make sure you specified the correct `repo_id` and `repo_type`.
If the repo is private, make sure you are authenticated.
Invalid username or password.

RevisionNotFoundError

class huggingface_hub.utils.RevisionNotFoundError

< >

( message: str response: typing.Optional[requests.models.Response] )

Raised when trying to access a hf.co URL with a valid repository but an invalid revision.

Example:

>>> from huggingface_hub import hf_hub_download
>>> hf_hub_download('bert-base-cased', 'config.json', revision='<non-existent-revision>')
(...)
huggingface_hub.utils._errors.RevisionNotFoundError: 404 Client Error. (Request ID: Mwhe_c3Kt650GcdKEFomX)

Revision Not Found for url: https://huggingface.co/bert-base-cased/resolve/%3Cnon-existent-revision%3E/config.json.

EntryNotFoundError

class huggingface_hub.utils.EntryNotFoundError

< >

( message: str response: typing.Optional[requests.models.Response] )

Raised when trying to access a hf.co URL with a valid repository and revision but an invalid filename.

Example:

>>> from huggingface_hub import hf_hub_download
>>> hf_hub_download('bert-base-cased', '<non-existent-file>')
(...)
huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: 53pNl6M0MxsnG5Sw8JA6x)

Entry Not Found for url: https://huggingface.co/bert-base-cased/resolve/main/%3Cnon-existent-file%3E.

BadRequestError

class huggingface_hub.utils.BadRequestError

< >

( message: str response: typing.Optional[requests.models.Response] )

Raised by hf_raise_for_status when the server returns a HTTP 400 error.

Example:

>>> resp = requests.post("hf.co/api/check", ...)
>>> hf_raise_for_status(resp, endpoint_name="check")
huggingface_hub.utils._errors.BadRequestError: Bad request for check endpoint: {details} (Request ID: XXX)

LocalEntryNotFoundError

class huggingface_hub.utils.LocalEntryNotFoundError

< >

( message: str )

Raised when trying to access a file that is not on the disk when network is disabled or unavailable (connection issue). The entry may exist on the Hub.

Note: ValueError type is to ensure backward compatibility. Note: LocalEntryNotFoundError derives from HTTPError because of EntryNotFoundError even when it is not a network issue.

Example:

>>> from huggingface_hub import hf_hub_download
>>> hf_hub_download('bert-base-cased', '<non-cached-file>',  local_files_only=True)
(...)
huggingface_hub.utils._errors.LocalEntryNotFoundError: Cannot find the requested files in the disk cache and outgoing traffic has been disabled. To enable hf.co look-ups and downloads online, set 'local_files_only' to False.

Validators

huggingface_hub include custom validators to validate method arguments automatically. Validation is inspired by the work done in Pydantic to validate type hints but with more limited features.

Generic decorator

validate_hf_hub_args() is a generic decorator to encapsulate methods that have arguments following huggingface_hub’s naming. By default, all arguments that has a validator implemented will be validated.

If an input is not valid, a HFValidationError is thrown. Only the first non-valid value throws an error and stops the validation process.

Usage:

>>> from huggingface_hub.utils import validate_hf_hub_args

>>> @validate_hf_hub_args
... def my_cool_method(repo_id: str):
...     print(repo_id)

>>> my_cool_method(repo_id="valid_repo_id")
valid_repo_id

>>> my_cool_method("other..repo..id")
huggingface_hub.utils._validators.HFValidationError: Cannot have -- or .. in repo_id: 'other..repo..id'.

>>> my_cool_method(repo_id="other..repo..id")
huggingface_hub.utils._validators.HFValidationError: Cannot have -- or .. in repo_id: 'other..repo..id'.

validate_hf_hub_args

huggingface_hub.utils.validate_hf_hub_args

< >

( fn: CallableT )

Raises

HFValidationError

Validate values received as argument for any public method of huggingface_hub.

The goal of this decorator is to harmonize validation of arguments reused everywhere. By default, all defined validators are tested.

Validators:

  • validate_repo_id(): repo_id must be "repo_name" or "namespace/repo_name". Namespace is a username or an organization.

Example:

>>> from huggingface_hub.utils import validate_hf_hub_args

>>> @validate_hf_hub_args
... def my_cool_method(repo_id: str):
...     print(repo_id)

>>> my_cool_method(repo_id="valid_repo_id")
valid_repo_id

>>> my_cool_method("other..repo..id")
huggingface_hub.utils._validators.HFValidationError: Cannot have -- or .. in repo_id: 'other..repo..id'.

>>> my_cool_method(repo_id="other..repo..id")
huggingface_hub.utils._validators.HFValidationError: Cannot have -- or .. in repo_id: 'other..repo..id'.

HFValidationError

class huggingface_hub.utils.HFValidationError

< >

( )

Generic exception thrown by huggingface_hub validators.

Inherits from ValueError.

Argument validators

Validators can also be used individually. Here is a list of all arguments that can be validated.

repo_id

huggingface_hub.utils.validate_repo_id

< >

( repo_id: str )

Validate repo_id is valid.

This is not meant to replace the proper validation made on the Hub but rather to avoid local inconsistencies whenever possible (example: passing repo_type in the repo_id is forbidden).

Rules:

  • Between 1 and 96 characters.
  • Either “repo_name” or “namespace/repo_name”
  • [a-zA-Z0-9] or ”-”, ”_”, ”.”
  • ”—” and ”..” are forbidden

Valid: "foo", "foo/bar", "123", "Foo-BAR_foo.bar123"

Not valid: "datasets/foo/bar", ".repo_id", "foo--bar", "foo.git"

Example:

>>> from huggingface_hub.utils import validate_repo_id
>>> validate_repo_id(repo_id="valid_repo_id")
>>> validate_repo_id(repo_id="other..repo..id")
huggingface_hub.utils._validators.HFValidationError: Cannot have -- or .. in repo_id: 'other..repo..id'.

Discussed in https://github.com/huggingface/huggingface_hub/issues/1008. In moon-landing (internal repository):

←Inference API Discussions and Pull Requests→