Logging methods¶
🤗 Datasets strives to be transparent and explicit about how it works, but this can be quite verbose at times. We have included a series of logging methods which allow you to easily adjust the level of verbosity of the entire library. Currently the default verbosity of the library is set to WARNING
.
To change the level of verbosity, use one of the direct setters. For instance, here is how to change the verbosity to the INFO
level:
import datasets
datasets.logging.set_verbosity_info()
You can also use the environment variable DATASETS_VERBOSITY
to override the default verbosity, and set it to one of the following: debug
, info
, warning
, error
, critical
:
DATASETS_VERBOSITY=error ./myprogram.py
All the methods of this logging module are documented below. The main ones are:
datasets.logging.get_verbosity()
to get the current level of verbosity in the loggerdatasets.logging.set_verbosity()
to set the verbosity to the level of your choice
In order from the least to the most verbose (with their corresponding int
values):
datasets.logging.CRITICAL
ordatasets.logging.FATAL
(int value, 50): only report the most critical errors.datasets.logging.ERROR
(int value, 40): only report errors.datasets.logging.WARNING
ordatasets.logging.WARN
(int value, 30): only reports error and warnings. This the default level used by the library.datasets.logging.INFO
(int value, 20): reports error, warnings and basic information.datasets.logging.DEBUG
(int value, 10): report all information.
Functions¶
-
datasets.logging.
get_verbosity
() → int¶ Return the current level for the HuggingFace datasets library’s root logger. :returns: Logging level, e.g.,
datasets.logging.DEBUG
anddatasets.logging.INFO
.Note
HuggingFace datasets library has following logging levels: -
datasets.logging.CRITICAL
,datasets.logging.FATAL
-datasets.logging.ERROR
-datasets.logging.WARNING
,datasets.logging.WARN
-datasets.logging.INFO
-datasets.logging.DEBUG
-
datasets.logging.
set_verbosity
(verbosity: int) → None¶ Set the level for the HuggingFace datasets library’s root logger. :param verbosity: Logging level, e.g.,
datasets.logging.DEBUG
anddatasets.logging.INFO
.
-
datasets.logging.
set_verbosity_info
()¶ Set the level for the HuggingFace datasets library’s root logger to INFO.
This will display most of the logging information and tqdm bars.
Shortcut to
datasets.logging.set_verbosity(datasets.logging.INFO)
-
datasets.logging.
set_verbosity_warning
()¶ Set the level for the HuggingFace datasets library’s root logger to WARNING.
This will display only the warning and errors logging information and tqdm bars.
Shortcut to
datasets.logging.set_verbosity(datasets.logging.WARNING)
-
datasets.logging.
set_verbosity_debug
()¶ Set the level for the HuggingFace datasets library’s root logger to DEBUG.
This will display all the logging information and tqdm bars.
Shortcut to
datasets.logging.set_verbosity(datasets.logging.DEBUG)
-
datasets.logging.
set_verbosity_error
()¶ Set the level for the HuggingFace datasets library’s root logger to ERROR.
This will display only the errors logging information and tqdm bars.
Shortcut to
datasets.logging.set_verbosity(datasets.logging.ERROR)
-
datasets.logging.
disable_propagation
() → None¶ Disable propagation of the library log outputs. Note that log propagation is disabled by default.
-
datasets.logging.
enable_propagation
() → None¶ Enable propagation of the library log outputs. Please disable the HuggingFace datasets library’s default handler to prevent double logging if the root logger has been configured.
-
datasets.logging.
get_logger
(name: Optional[str] = None) → logging.Logger¶ Return a logger with the specified name. This function can be used in dataset and metrics scripts.
Levels¶
-
datasets.logging.
CRITICAL
= 50¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
DEBUG
= 10¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
ERROR
= 40¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
FATAL
= 50¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
INFO
= 20¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
NOTSET
= 0¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
WARN
= 30¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4
-
datasets.logging.
WARNING
= 30¶ int(x=0) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4