Package: logfmter
Python LogfmterUsing the stdlib logging module and without changing a single logging call, logfmter supportsglobal(first and third party)logfmtstructured logging.>logging.warn("user created",extra=user)at=WARNINGmsg="user created"first_name=Johnlast_name=Doeage=25Table of ContentsWhyInstallUsageIntegrationConfigurationExtensionGuidesDevelopmentRequired SoftwareGetting StartedPublishingWhyenables both human and computer readable logs,recommended as a "best practice" by Splunkformats all first and third party logs, you never have to worry about a library using a different logging formatsimple to integrate into any existing application, requires no changes to existing log statements i.e.structlogInstall$pipinstalllogfmterUsageThis package exposes a singleLogfmterclass that can be integrated into
thestandard library logging systemlike anylogging.Formatter.IntegrationbasicConfigimportloggingfromlogfmterimportLogfmterhandler=logging.StreamHandler()handler.setFormatter(Logfmter())logging.basicConfig(handlers=[handler])logging.error("hello",extra={"alpha":1})# at=ERROR msg=hello alpha=1logging.error({"token":"Hello, World!"})# at=ERROR token="Hello, World!"dictConfigIf you are usingdictConfig, you need to consider your setting
ofdisable_existing_loggers. It is enabled by default, and causes
any third party module loggers to be disabled.importlogging.configlogging.config.dictConfig({"version":1,"formatters":{"logfmt":{"()":"logfmter.Logfmter",}},"handlers":{"console":{"class":"logging.StreamHandler","formatter":"logfmt"}},"loggers":{"":{"handlers":["console"],"level":"INFO"}},})logging.info("hello",extra={"alpha":1})# at=INFO msg=hello alpha=1Notice, you can configure theLogfmterby providing keyword arguments as dictionary
items after"()":..."logfmt":{"()":"logfmter.Logfmter","keys":[...],"mapping":{...}}...fileConfigUsing logfmter via fileConfig is not supported, because fileConfig does not support custom formatter initialization. There may be some hacks to make this work in the future. Let me know if you have ideas or really need this.ConfigurationkeysBy default, theat=<levelname>key/value will be included in all log messages. These
default keys can be overridden using thekeysparameter. If the key you want to include
in your output is represented by a different attribute on the log record, then you can
use themappingparameter to provide that key/attribute mapping.Reference the Pythonlogging.LogRecordDocumentationfor a list of available attributes.importloggingfromlogfmterimportLogfmterformatter=Logfmter(keys=["at","processName"])handler=logging.StreamHandler()handler.setFormatter(formatter)logging.basicConfig(handlers=[handler])logging.error("hello")# at=ERROR processName=MainProceess msg=hellomappingBy default, a mapping of{"at": "levelname"}is used to allow theatkey to reference
the log record'slevelnameattribute. You can override this parameter to provide your
own mappings.importloggingfromlogfmterimportLogfmterformatter=Logfmter(keys=["at","process"],mapping={"at":"levelname","process":"processName"})handler=logging.StreamHandler()handler.setFormatter(formatter)logging.basicConfig(handlers=[handler])logging.error("hello")# at=ERROR process=MainProceess msg=hellodatefmtIf you request theasctimeattribute (directly or through a mapping), then the date format
can be overridden through thedatefmtparameter.importloggingfromlogfmterimportLogfmterformatter=Logfmter(keys=["at","when"],mapping={"at":"levelname","when":"asctime"},datefmt="%Y-%m-%d")handler=logging.StreamHandler()handler.setFormatter(formatter)logging.basicConfig(handlers=[handler])logging.error("hello")# at=ERROR when=2022-04-20 msg=helloExtensionYou can subclass the formatter to change its behavior.importloggingfromlogfmterimportLogfmterclassCustomLogfmter(Logfmter):"""Provide a custom logfmt formatter which formatsbooleans as "yes" or "no" strings."""@classmethoddefformat_value(cls,value):ifisinstance(value,bool):return"yes"ifvalueelse"no"returnsuper().format_value(value)handler=logging.StreamHandler()handler.setFormatter(CustomLogfmter())logging.basicConfig(handlers=[handler])logging.error({"example":True})# at=ERROR example=yesGuidesDefault Key/Value PairsInstead of providing key/value pairs at each log call, you can override
the log record factory to provide defaults:_record_factory=logging.getLogRecordFactory()defrecord_factory(*args,**kwargs):record=_record_factory(*args,**kwargs)record.trace_id=123returnrecordlogging.setLogRecordFactory(record_factory)This will cause all logs to have thetrace_id=123pair regardless of includingtrace_idin keys or manually addingtrace_idto theextraparameter or themsgobject.DevelopmentRequired SoftwareRefer to the links provided below to install these development dependencies:direnvgitpyenvGetting StartedSetup$<runtimes.txtxargs-n1pyenvinstall-s
$direnvallow
$pipinstall-rrequirements/dev.txt
$pre-commitinstall
$pipinstall-e.TestsRun the test suite against the active python environment.$pytestRun the test suite against the active python environment and
watch the codebase for any changes.$ptwRun the test suite against all supported python versions.$toxPublishingCreateUpdate the version number inlogfmter/__init__.py.Add an entry inHISTORY.md.Commit the changes, tag the commit, and push the tags:$gitcommit-am"v<major>.<minor>.<patch>"$gittagv<major>.<minor>.<patch>
$gitpushoriginmain--tagsConvert the tag to a release in GitHub with the history
entry as the description.Build$python-mbuildUpload$ twine upload dist/*