Package: mflog
mflogStatus (master branch)What is it ?It is an opinionated python (structured) logging library built onstructlogfor theMetWork Framework(but it can be used in any context).Structured logging means that you don’t write hard-to-parse and hard-to-keep-consistent prose in your logs but that you log events that happen in a context instead.https://www.structlog.org/en/stable/why.htmlExample:frommflogimportget_logger# Get a loggerlog=get_logger("foo.bar")# Bind some attributes to the logger depending on the contextlog=log.bind(user="john")log=log.bind(user_id=123)# [...]# Log somethinglog.warning("user logged in",happy=True,another_key=42)Onstderr, you will get:2019-01-28T07:52:42.903067Z  [WARNING] (foo.bar#7343) user logged in {another_key=42 happy=True user=john user_id=123}Onjson output file, you will get:{"timestamp":"2019-01-28T08:16:40.047710Z","level":"warning","name":"foo.bar","pid":29317,"event":"user logged in","another_key":42,"happy":true,"user":"john","user_id":123}If thepython/rich libraryis installed (this is not a
mandatory requirement) and if the output is a real terminal (and not a redirection or a pipe),
the library will automatically configure a fancy color output (of course you can disable it if
you don't like):With following demo python program:importmflog# Get a loggerlogger=mflog.get_logger("foobar")# Bind two context variables to this loggerlogger=logger.bind(user_id=1234,is_logged=True)# Log somethinglogger.info("This is an info message",special_value="foo")logger.critical("This is a very interesting critical message")# Let's play with exceptiontry:# Just set a variable to get a demo of locals variable dumpvar={"key1":[1,2,3],"key2":"foobar"}1/0exceptException:logger.exception("exception raised (a variables dump should follow)")You will get this color ouput:(opinionated) Choices and Featureswe use main ideas fromstructloglibrarywe log[DEBUG]and[INFO]messages onstdout(in a human friendly way)we log[WARNING],[ERROR]and[CRITICAL]onstderr(in a human friendly way)(and optionally) we log all messages (worse than a minimal configurable level) in a configurable file inJSON(for easy automatic parsing)(and optionally) we send all messages (worse than a minimal configurable level) to an UDP syslog server (in JSON or in plain text)we can configure a global minimal level to ignore all messages belowwe reconfigure automatically python standard logging library to usemflogUnicode and Bytes messages are supported (in Python2 and Python3)good support for exceptions (with backtraces)override easily minimal levels (for patterns of logger names) programmatically or with plain text configuration filesif thepython/rich libraryis installed (this is not a mandatory requirement) and if the output is a real terminal (and not a redirection), the library will automatically configure a fancy color output (can be really useful but of course you can disable this feature if you don't like it)How to use ?Amfloglogger can be used as a standardlogginglogger.For example:# Importfrommflogimportget_logger# Get a loggerx=get_logger("foo.bar")# Usagex.warning("basic message")x.critical("message with templates:%i,%s",2,"foo")x.debug("message with key/values",foo=True,bar="string")try:1/0exceptException:x.exception("we catched an exception with automatic traceback")x=x.bind(context1="foo")x=x.bind(context2="bar")x.info("this is a contexted message",extra_var=123)How to configure ?In pythonimportmflog# Configuremflog.set_config(minimal_level="DEBUG",json_minimal_level="WARNING",json_file="/foo/bar/my_output.json")# Get a loggerx=mflog.get_logger("foo.bar")# [...]With environment variables$exportMFLOG_MINIMAL_LEVEL="DEBUG"$exportMFLOG_JSON_MINIMAL_LEVEL="WARNING"$exportMFLOG_JSON_FILE="/foo/bar/my_output.json"$python

>>>importmflog
>>>
>>># Get a logger>>>x=mflog.get_logger("foo.bar")>>>
>>># [...]NoteWhen you get amfloglogger, if default configuration is applied automatically
if not set manually before.How to override minimal level for a specific loggerIf you have a "noisy" specific logger, you can override its minimal log level.The idea is to configure this in a file like this:# lines beginning with # are comments

# this line say 'foo.bar' logger will have a minimal level of WARNING
foo.bar => WARNING

# this line say 'foo.*' loggers will have a minimal level of DEBUG
# (see python fnmatch for accepted wildcards)
foo.* => DEBUG

# The first match winsThen, you can use# yes we use a list here because you can use several files# (the first match wins)mflog.set_config([...],override_files=["/full/path/to/your/override.conf"])or# if you want to provide multiple files, use ';' as a separator
export MFLOG_MINIMAL_LEVEL_OVERRIDE_FILES=/full/path/to/your/override.confLink with standard python logging libraryWhen you get amfloglogger or when you callset_config()function,
the standard pythonlogginglibrary is reconfigured to usemflog.Example:importloggingimportmflog# standard use of logging libraryx=logging.getLogger("standard.logger")print("<output of the standard logging library>")x.warning("foo bar")print("</output of the standard logging library>")# we set the mflog configurationmflog.set_config()# now logging library use mflogprint()print("<output of the standard logging library through mflog>")x.warning("foo bar")print("</output of the standard logging library through mflog>")Output:<output of the standard logging library>
foo bar
</output of the standard logging library>

<output of the standard logging library through mflog>
2019-01-29T09:32:37.093240Z  [WARNING] (standard.logger#15809) foo bar
</output of the standard logging library through mflog>mflog loggers API.debug(message, *args, **kwargs)Log the given message as[DEBUG].*argscan be used for placeholders (to format the given message)**kwargscan be used for key/values (log context).Examples:frommflogimportget_loggerx=get_logger('my.logger')x.debug("my debug message with placeholders:%sand%i","foo",123,key1="value1, key2=True, key5=123).info(message, *args, **kwargs)Same as.debugbut with[INFO]severity level..warning(message, *args, **kwargs)Same as.debugbut with[WARNING]severity level..error(message, *args, **kwargs)Same as.debugbut with[ERROR]severity level..critical(message, *args, **kwargs)Same as.debugbut with[CRITICAL]severity level..exception(message, *args, **kwargs)Same as.error(so with[ERROR]severity level) but we automatically add
the current stacktrace in the message through special key/values..bind(**new_values)Return a new logger with**new_valuesadded to the existing ones
(see examples at the beginning)..unbind(*keys)Return a new logger with*keysremoved from the context.
It raisesKeyErrorif the key is not part of the context..try_unbind(*keys)Like.unbindbut best effort:  missing keys are ignored..die(optional_message, *args, **kwargs)Same as.exception()but also do a.dump_locals()call and exit the program
withsys.exit(1)..dump_locals()Dump locals variables onstderr(for debugging).mflog.*All previous loggers method are also available inmflogmodule.Example:importmflogmflog.warning("this is a warning message",context1="foobar",user_id=123)FAQIf I want to use mflog inside my library ?If you write a library and if you want to usemflog, usemflognormally.
You just should avoid to callset_config()inside your library.Do you have "thread local context mode" ?This mode is explainedhere.You have to understand what you are doing.If you want to use it, just addthread_local_context=Trueto yourset_config()call. And you can use.new(**new_values)on mflog loggers to clear context
and binds some initial values.Can I globally add an extra context to each log line ?If you addextra_context_func=your_python_functo yourset_config()call,
and ifyour_python_funcreturns a dict of key/values as strings when called
with no argument, these key/values will be added to your log context.Another way to do that without even callingset_config()is to define
an environment variable calledMFLOG_EXTRA_CONTEXT_FUNCcontaining the
full path to your python func.Full example:# in shellexportMFLOG_EXTRA_CONTEXT_FUNC="mflog.unittests.extra_context"then, in your python interpreter:>>>frommflogimportget_logger>>>get_logger("foo").info("bar")2019-04-11T07:32:53.517260Z[INFO](foo#15379) bar {extra_context_key1=extra_context_value1 extra_context_key2=extra_context_value2}Here is the code ofmflog.unittests.extra_context:defextra_context():return{"extra_context_key1":"extra_context_value1","extra_context_key2":"extra_context_value2"}Can I filter some context keys in stdout/stderr output (but keep them in json output) ?Yes, addjson_only_keys=["key1", "key2"]to yourset_config()call or useMFLOG_JSON_ONLY_KEYS=key1,key2environment variable.What about if I don't want to redirect standard pythonloggingtomflog?You can addstandard_logging_redirect=Falsein yourset_config()call
of setMFLOG_STANDARD_LOGGING_REDIRECT=0environment variable.Can I silent a specific noisy logger?You can useoverride_filesfeature to do that or you can also use themflog.add_overridefunction.For example:importmflog# for all mylogger.* loggers (fnmatch pattern), the minimal level is CRITICALmflog.add_override("mylogger.*",CRITICAL)# Not very interesting but this call will be ignoredmflog.get_logger("mylogger.foo").warning("foo")How can I use syslog logging?You can configure it with these keyword arguments duringset_config()call:syslog_minimal_level:WARNING,CRITICAL...syslog_address:null(no syslog (defaut)),127.0.0.1:514(send packets to 127.0.0.1:514),/dev/log(unix socket)...syslog_format:msg_only(default) orjsonor with corresponding env vars:MFLOG_SYSLOG_MINIMAL_LEVELMFLOG_SYSLOG_ADDRESSMFLOG_SYSLOG_FORMATHow to disable the fancy color output?This feature is automatically enabled when:python/richlibrary is installedthe corresponding output (stdout, stderr) is a real terminal (and not a redirection to a file)But you can manually disable it by addingfancy_output=Falseto yourset_config().CoverageSeeCoverage reportContributing guideSeeCONTRIBUTING.mdfile.Code of ConductSeeCODE_OF_CONDUCT.mdfile.Sponsors(If you are officially paid to work on MetWork Framework, please contact us to add your company logo here!)