Logging methods

datasets tries to be very transparent and explicit about its inner working, but this can be quite verbose at times.

A series of logging methods let you easily adjust the level of verbosity of the whole library.

Currently the default verbosity of the library is WARNING.

To change the level of verbosity, just use one of the direct setters. For instance, here is how to change the verbosity to the INFO level.

import datasets
datasets.logging.set_verbosity_info()

You can also use the environment variable DATASETS_VERBOSITY to override the default verbosity. You can set it to one of the following: debug, info, warning, error, critical. For example:

DATASETS_VERBOSITY=error ./myprogram.py

All the methods of this logging module are documented below, the main ones are datasets.logging.get_verbosity() to get the current level of verbosity in the logger and datasets.logging.set_verbosity() to set the verbosity to the level of your choice. In order (from the least verbose to the most verbose), those levels (with their corresponding int values in parenthesis) are:

Functions

datasets.logging.get_verbosity() → int

Return the current level for the HuggingFace datasets library’s root logger. :returns: Logging level, e.g., datasets.logging.DEBUG and datasets.logging.INFO.

Note

HuggingFace datasets library has following logging levels: - datasets.logging.CRITICAL, datasets.logging.FATAL - datasets.logging.ERROR - datasets.logging.WARNING, datasets.logging.WARN - datasets.logging.INFO - datasets.logging.DEBUG

datasets.logging.set_verbosity(verbosity: int) → None

Set the level for the HuggingFace datasets library’s root logger. :param verbosity: Logging level, e.g., datasets.logging.DEBUG and datasets.logging.INFO.

datasets.logging.set_verbosity_info()

Set the level for the HuggingFace datasets library’s root logger to INFO.

This will display most of the logging information and tqdm bars.

Shortcut to datasets.logging.set_verbosity(datasets.logging.INFO)

datasets.logging.set_verbosity_warning()

Set the level for the HuggingFace datasets library’s root logger to WARNING.

This will display only the warning and errors logging information (no tqdm bars).

Shortcut to datasets.logging.set_verbosity(datasets.logging.WARNING)

datasets.logging.set_verbosity_debug()

Set the level for the HuggingFace datasets library’s root logger to DEBUG.

This will display all the logging information and tqdm bars.

Shortcut to datasets.logging.set_verbosity(datasets.logging.DEBUG)

datasets.logging.set_verbosity_error()

Set the level for the HuggingFace datasets library’s root logger to ERROR.

This will display only the errors logging information (no tqdm bars).

Shortcut to datasets.logging.set_verbosity(datasets.logging.ERROR)

datasets.logging.disable_default_handler() → None

Disable the default handler of the HuggingFace datasets library’s root logger.

datasets.logging.enable_default_handler() → None

Enable the default handler of the HuggingFace datasets library’s root logger.

datasets.logging.disable_propagation() → None

Disable propagation of the library log outputs. Note that log propagation is disabled by default.

datasets.logging.enable_propagation() → None

Enable propagation of the library log outputs. Please disable the HuggingFace datasets library’s default handler to prevent double logging if the root logger has been configured.

datasets.logging.get_logger(name: Optional[str] = None) → logging.Logger

Return a logger with the specified name. This function can be used in dataset and metrics scripts.

Levels

datasets.logging.CRITICAL = 50

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.DEBUG = 10

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.ERROR = 40

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.FATAL = 50

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.INFO = 20

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.NOTSET = 0

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.WARN = 30

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

datasets.logging.WARNING = 30

int(x=0) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4