Logging methods
🤗 Datasets strives to be transparent and explicit about how it works, but this can be quite verbose at times. We have included a series of logging methods which allow you to easily adjust the level of verbosity of the entire library. Currently the default verbosity of the library is set to WARNING
.
To change the level of verbosity, use one of the direct setters. For instance, here is how to change the verbosity to the INFO
level:
import datasets
datasets.logging.set_verbosity_info()
You can also use the environment variable DATASETS_VERBOSITY
to override the default verbosity, and set it to one of the following: debug
, info
, warning
, error
, critical
:
DATASETS_VERBOSITY=error ./myprogram.py
All the methods of this logging module are documented below. The main ones are:
- datasets.logging.get_verbosity() to get the current level of verbosity in the logger
- datasets.logging.set_verbosity() to set the verbosity to the level of your choice
In order from the least to the most verbose (with their corresponding int
values):
datasets.logging.CRITICAL
ordatasets.logging.FATAL
(int value, 50): only report the most critical errors.datasets.logging.ERROR
(int value, 40): only report errors.datasets.logging.WARNING
ordatasets.logging.WARN
(int value, 30): only reports error and warnings. This the default level used by the library.datasets.logging.INFO
(int value, 20): reports error, warnings and basic information.datasets.logging.DEBUG
(int value, 10): report all information.
By default, tqdm
progress bars will be displayed during dataset download and preprocessing. datasets.logging.disable_progress_bar() and datasets.logging.enable_progress_bar() can be used to suppress or unsuppress this behavior.
Functions
Return the current level for the HuggingFace datasets library’s root logger.
HuggingFace datasets library has following logging levels:
- datasets.logging.CRITICAL, datasets.logging.FATAL
- datasets.logging.ERROR
- datasets.logging.WARNING, datasets.logging.WARN
- datasets.logging.INFO
- datasets.logging.DEBUG
Set the level for the HuggingFace datasets library’s root logger.
Set the level for the HuggingFace datasets library’s root logger to INFO.
This will display most of the logging information and tqdm bars.
Shortcut to datasets.logging.set_verbosity(datasets.logging.INFO)
Set the level for the HuggingFace datasets library’s root logger to WARNING.
This will display only the warning and errors logging information and tqdm bars.
Shortcut to datasets.logging.set_verbosity(datasets.logging.WARNING)
Set the level for the HuggingFace datasets library’s root logger to DEBUG.
This will display all the logging information and tqdm bars.
Shortcut to datasets.logging.set_verbosity(datasets.logging.DEBUG)
Set the level for the HuggingFace datasets library’s root logger to ERROR.
This will display only the errors logging information and tqdm bars.
Shortcut to datasets.logging.set_verbosity(datasets.logging.ERROR)
Disable propagation of the library log outputs. Note that log propagation is disabled by default.
Enable propagation of the library log outputs. Please disable the HuggingFace datasets library’s default handler to prevent double logging if the root logger has been configured.
Return a logger with the specified name. This function can be used in dataset and metrics scripts.
Enable tqdm progress bar.
Enable tqdm progress bar.
Return a boolean indicating whether tqdm progress bars are enabled.
Levels
datasets.logging.CRITICAL
datasets.logging.CRITICAL = 50
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.DEBUG
datasets.logging.DEBUG = 10
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.ERROR
datasets.logging.ERROR = 40
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.FATAL
datasets.logging.FATAL = 50
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.INFO
datasets.logging.INFO = 20
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.NOTSET
datasets.logging.NOTSET = 0
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.WARN
datasets.logging.WARN = 30
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
datasets.logging.WARNING
datasets.logging.WARNING = 30
int([x]) -> integer
int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.int(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ’+’ or ’-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4