Package: logging-opentracing
opentracing-logging-pythonOpenTracing handler for the Python logging libraryInformation: This library is currently in a beta stateInstallationInstall the package with pippip install logging-opentracingUsageWe use the mock tracer for the follwing examples but you can also use other OpenTracing compatible tracers.An compatible tracer would be, for instance,JaegerSimpleIn the first example we initialize theOpenTracingHandlerforloggingand create an active span with the namehello-world.
In this active span we make make an info-log and in the end we have a look if this log was forwarded to OpenTracing.importloggingfromopentracing.mocktracerimportMockTracerfromlogging_opentracingimportOpenTracingHandler# initialize a mock tracertracer=MockTracer()# prepare the loggerlogger=logging.getLogger('mylogger')logger.setLevel(logging.INFO)# create a new OpenTracing handler for the logging packagehandler=OpenTracingHandler(tracer=tracer)logger.addHandler(handler)# start an active spanwithtracer.start_active_span('hello-world'):# this log will be propagated tologger.info('Hello World from Python logging to OpenTracing')# retrieve the finished spanfinished_span=tracer.finished_spans()[0]# get the log line fromlog=finished_span.logs[0]# print the key_values of the logprint(log.key_values)# {'event': 'info', 'message': 'Hello World from Python logging to OpenTracing'}Here some additional explanation# initialize a mock tracertracer=MockTracer()Initialize the mock tracer from the OpenTracing library.
As mentioned before, instead you can use any OpenTracing compatible tracer.# prepare the loggerlogger=logging.getLogger('mylogger')logger.setLevel(logging.INFO)Prepare a logger from the Pythonloggingpackage.
Set its logging level toINFOsuch that logs with the severityINFOare also captured.# create a new OpenTracing handler for the logging packagehandler=OpenTracingHandler(tracer=tracer)logger.addHandler(handler)First, initialize the OpenTracing handlerOpenTracingHandlerforlogging.
It needs an OpenTracing tracer as parameter.
Then, add the handler to the logger.# start an active spanwithtracer.start_active_span('hello-world'):# this log will be propagated tologger.info('Hello World from Python logging to OpenTracing')Start a new active span with the namehello-world.
Within this active span, make a log with the severity info.
It is expected that this log will be captured by our handler for OpenTracing which should forward the log to our tracer.# retrieve the finished spanfinished_span=tracer.finished_spans()[0]# get the log line fromlog=finished_span.logs[0]# print the key_values of the logprint(log.key_values)# {'event': 'info', 'message': 'Hello World from Python logging to OpenTracing'}These lines are only used to check if the log has been successfully forwarder to out tracer.Custom FormatterThe previous example showed how logs are forwarded to OpenTracing with the default formating option for logs.
Thereby, the defalt format is{'info': <log severity>, 'message': <log format>}.In the case that a different formatting is required, in the constructor ofOpenTracingHandlercan be adjusted.
To do so, set the parameterkv_format.
It expects a dictionary, where each key-value pair represents a key value pair forwarded to the methodlog_kv()of
OpenTracing. Thereby,keyis the key which will be directly used as key in the OpenTracing logvalueis a string which can contain placeholders for %-stype formatting of the logging package. (See alsoFormatfor more details)For each key-value pair a new formatterlogging.Formatterwill be created.When we replace from the previoussimple examplethe lines# create a new OpenTracing handler for the logging packagehandler=OpenTracingHandler(tracer=tracer)with the following lines# create a new formatter with a custom formatformatter=OpenTracingFormatter(kv_format={'event':'%(levelname_lower)s','message':'%(message)s','source':'%(filename)s:L%(lineno)d',})# create a new OpenTracing handler which uses the custom formatterhandler=OpenTracingHandler(tracer=tracer,formatter=formatter)we initialize a handler with a formatter with a custom format.The expected output of the modified example is{'event': 'info', 'message': 'Hello World from Python logging to OpenTracing', 'source': 'custom_formatter.py:L26'}See the full examplecustom_formatter.pyManually pass a spanThe OpenTracing logging handler tries to retrieve a span by accessing the current scopescope = tracer.scope_manager.activeand an the case that a scope is available accessing its current spanspan = scope.span.
However, such a scope is only available when a scope has been indirectly created through starting an active spantracer.start_active_span()or when a scope has been directly activatedtracer.scope_manager.activate().Therefore it is also possible to pass a span to a log with theextraparameter.# start a spanwithtracer.start_span('hello-world')asspan:# the span is directly passed to the log with the "extra" parameterlogger.info('A span has been directly passed',extra={'span':span})The default key which is used to check if a span has been passed is'span'.
However, it can be customized in the instantiation of the handler.handler=OpenTracingHandler(tracer=tracer,span_key='sp')# ...# start a spanwithtracer.start_span('hello-world')asspan:# the span is directly passed to the log with the "extra" parameter# this time we have to use the key "sp" because we set span_key='sp' in the constructorlogger.info('A span has been directly passed',extra={'sp':span})See the full examplespan_passed.pyExceptionThe OpenTracing handler can also be used to log exceptions.
To do so, just log with the the levelexception.try:logger.info('This will be difficult')# this statement will cause a ZeroDivisionError1/0exceptZeroDivisionError:logger.exception('Oh no we have a ZeroDivision Error')This would result in the logs{'event': 'info', 'message': 'This will be difficult'}
{'event': 'error', 'message': 'Oh no we have a ZeroDivision Error', 'error.object': ZeroDivisionError('division by zero'), 'error.kind': <class 'ZeroDivisionError'>, 'stack': '  File \"<path_suffix>/logging_opentracing/examples/exception.py\" line 23, in <module>\\n    1 / 0\\n'}where the same formatting is used like OpenTracing uses when an uncaught exception is created.See the full exampleexception.pyAdditional key-value pairsTo each logging call extra key-value pairs can be passed which should be included in a OpenTracing log.
Pass a dictionary with the key-value pairs to be added to the keykvof the extra parameter of a logging call.# add additional key-value pairs to the log by providing a dict to the key "kv" of the "extra" parameterlogger.info('Here we pass additional arguments to the log',extra={'kv':{'key a':[1,2,3],'key b':'foo'}})which results in a log{'event': 'info', 'message': 'Here we pass additional arguments to the log', 'key a': [1, 2, 3], 'key b': 'foo'}Per default these additional key-value pairs are expected to have the keykv.
However, this can be customized by setting the parameterextra_kv_keyof the constructor ofOpenTracingHandler.OpenTracingHandler(tracer=tracer,extra_kv_key='properties')# ...# add additional key-value pairs to the log by providing a dict to the customized key "properties" of the "extra" parameterlogger.info('Here we pass additional arguments to the log',extra={'properties':{'key a':[1,2,3],'key b':'foo'}})See the full exampleextra_kv.pyFormatThis library useslogging.Formatter(fmt=fmt).format(logging_LogRecord)for getting information from alogging.LogRecord, wherefmtis the format specified in the
values of the parameterkv_formatin the constructor ofOpenTracingHandler.logging_LogRecordis the variable which hold alogging.logRecord.Therefore, the format offmtfollows the formatting specification ofLogRecord attributes.Following, an excerpt of the official Python docs; use the format placeholders specified in the columnFormat.Attribute nameFormatDescriptionasctime%(asctime)sHuman-readable time when the LogRecord was created. By default this is of the form ‘2003-07-08 16:49:45,896’ (the numbers after the comma are millisecond portion of the time).created%(created)fTime when the LogRecord was created (as returned by time.time()).filename%(filename)sFilename portion ofpathname.funcName%(funcName)sName of function containing the logging call.levelname%(levelname)sText logging level for the message ('DEBUG','INFO','WARNING','ERROR','CRITICAL').levelname_lower%(levelname_lower)sLower case text logging level for the message ('debug','info','warning','error','critical'). This is a non-default attribute which will be automatically added by the OpenTracingFormatter.levelno%(levelno)sNumeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL).lineno%(lineno)dSource line number where the logging call was issued (if available).message%(message)sThe logged message. This is set when Formatter.format() is invoked.module%(module)sModule (name portion offilename).msecs%(msecs)dMillisecond portion of the time when the LogRecord was created.name%(name)sName of the logger used to log the call.pathname%(pathname)sFull pathname of the source file where the logging call was issued (if available).process%(process)dProcess ID (if available).processName%(processName)sProcess name (if available).relativeCreated%(relativeCreated)dTime in milliseconds when the LogRecord was created, relative to the time the logging module was loaded.thread%(thread)dThread ID (if available).threadName%(threadName)sThread name (if available).