Package: uologging
UOLogging is a solution for configuring Python's built-in logging module, including a utility for tracing multithreaded downloads.Installpip install uologging[all]ℹ Use the '[all]suffix to download the (†very small)humanizepackage as well.humanizemakes logs human-readable when dealing with large numbers.† Thehumanizepackage is 78.5 kB as of version 4.9.0.Enable console loggingSimply call "init_console()" to initializing Python's root logger to log to console:# ⚠ Inadvisable: Enable logging for ALL python packages/modules
uologging.init_console()⚠ WARNING: It is inadvisable to "init" the overall root logger except for debugging.
Why? The console can getvery noisywhen using 3rd party libraries (that use Pythonloggingmodule).In general, you will want to specify your package name. To enable logging within your package only, you can provide your package name.The handy invocation of__name__.split('.')[0]will provide your package's name fromanywhere within your package.# ✅ Best Practice: Enable logging only for your package.
my_package_name = __name__.split('.')[0]
uologging.init_console(my_package_name)Enable (Linux) syslog loggingSimilarly, you can call "init_syslog()":# Best Practice: Enable logging for your python package
my_package_name = __name__.split('.')[0]
uologging.init_syslog(my_package_name)

# Inadvisable: Enable logging for ALL python packages/modules
uologging.init_syslog()Set Logging VerbosityPer Python logging suggestion: WARNING, ERROR, and CRITICAL messages are all logged by default.If you are interested in seeing the DEBUG and INFO log messages, you'll need to update the logging verbosity in your application.
We provide the method set_verbosity() for this purpose.
Higher number means more logging.Choices are [0,2].
Default is 0. Default will captures WARNING, ERROR, and CRITICAL logs.
Provide 1 to also capture INFO logs.
Provide 2 to also capture DEBUG logs.# Enable maximum logging for your python package
my_package_name = __name__.split('.')[0]
uologging.set_verbosity(2, args.verbosity_flag, my_package_name)

# Enable maximum logging for ALL python packages/modules
uologging.set_verbosity(2)argparse 'verbosity flag'For CLI tools, we provide an integration with argparse to set the logging verbosity.
This integration enables the tool's user to add-vvat the command-line for maximum logging verbosity.-vwill enable INFO messages, but not DEBUG.The verbosity_flag can be gathered via argparse using "add_verbosity_flag(parser)":import uologging
import argparse

parser = argparse.ArgumentParser()
uologging.add_verbosity_flag(parser)

args = parser.parse_args(['-vv'])
# args.verbosity_flag == 2Now, simply call "set_verbosity()" withargs.verbosity_flagfor your package:my_package_name = __name__.split('.')[0]
uologging.set_verbosity(args.verbosity_flag, my_package_name)Example: Configuring CLI tool with console & syslog loggingLet's imagine you have a package "examplepkg" with a CLI tool in the "mytool" module.# my_cli_tool.py
import argparse
import uologging

# Parse CLI arguments, '-vv' will result in maximum logging verbosity.
parser = argparse.ArgumentParser()
uologging.add_verbosity_flag(parser)
args = parser.parse_args()

# Initialize logging
my_package_name = __name__.split('.')[0]
uologging.init_console(my_package_name)
uologging.init_syslog(my_package_name)
uologging.set_verbosity(args.verbosity_flag, my_package_name)Logging messages formatThe formatting for log messages is specified in the (private)uologging._logging_formatvariable.Here are a couple of lines showing what you can expect your logs to looks like:2022-01-07 15:40:09 DEBUG    Some simle message for you [hello.py:10]
2022-01-07 15:40:09 DEBUG    Finished: example.hello:hello((),{}) [hello.py:10] 
2022-01-07 15:40:09 DEBUG    example.hello:hello((),{}) execution time: 0.00 sec [hello.py:10]Tracing a functionThere is a simpletracedecorator you can use in your python modules to log the 'execution time' of any of your functions.The trace decorator logs at DEBUG severity.
So, callset_verbosity(>=2)to see the trace messages in your logs.# hello.py
import logging
import uologging

logger = logging.getLogger(__name__)

@uologging.trace(logger)
def hello():
    print('hello!')

hello()Expect the following messages to be logged:2022-01-07 15:40:09 DEBUG    Starting: example.hello:hello((),{}) [hello.py:10]
hello!
2022-01-07 15:40:09 DEBUG    Finished: example.hello:hello((),{}) [hello.py:10] 
2022-01-07 15:40:09 DEBUG    example.hello:hello((),{}) execution time: 0.00 sec [hello.py:10]Tracing Concurrent DownloadsAssume you have some service, and you need to makeseveral http requeststo downloadlots of gobs of data.
You want to do it concurrently, to save on time.ℹ 'Downloading data via several http requests' is an example of an 'embarrassingly parallel' problem.
I.e. for each concurrent worker we add, we gain 1x speedup Lots of speedup.
Ex. using8 workers will give 8x speedup.For this big, concurrent download, you would like to see:download progress indications, to know that the download is indeed happening, andwhen the download is completed,how much in total was downloaded.There is theuologging.DownloadTracerfor that!The download tracer logs a message for every 1 MB downloaded by default (customizable via 'threshold_bytes').Trace a concurrent downloads using theDownloadTracerwithpythonrequestspackagelike the following:fromconcurrent.futuresimportThreadPoolExecutorimportrequestsimportuologgingdownload_tracer=DownloadTracer('MyService',threshold_bytes=750000)ALL_MY_URLS=['http://ex1.io','http://ex2.io','http://ex3.io']# ... Replace with your actual list of URLsdefhttp_get(url):respons=requests.get(url)download_tracer.trace(len(response.content))returnresponsewithThreadPoolExecutor(max_workers=4)asexecutor:map(http_get,ALL_MY_URLS)loggingBest PracticesUse the Python logging package per the following best practices:logger = logging.getLogger(__name__)to get the logger for each module/script.Then, uselogger.debug(),logger.info(),logger.warning(), etc to add tracing to your Python modules/scripts.ExampleA trivial example demonstrating best practices:# hello.py
import logging

logger = logging.getLogger(__name__)

def hello():
    logger.debug('About to say "hello!"')
    print('hello!')
    logger.debug('Said "hello!"')