Package: contextlogger
Context LoggerDescriptionRequirementsInstallationUsageTutorialLogging configurationLogging without contextLogging with static contextLogging with dynamic contextLogging with context in multiple modulesStructured loggingVersion UpdatesContributionsDescriptionA simple logger that uses thecontextvarslibrary to inject contextual details in your logs.Source Code:https://github.com/kolitiri/contextloggerRequirementsPython 3.7+InstallationpipinstallcontextloggerUsageThis is a bare minimum example to get you started. (Check the tutorial below for some more realistic scenarios)""" my_app.py """importasyncioimportloggingfromuuidimportuuid4fromcontextloggerimportCLogVars,CLogVar,getCLogger# Create and configure a CLogger instanceclogger=getCLogger(__name__)console_handler=logging.StreamHandler()clogger.addHandler(console_handler)clogger.setLevel('DEBUG')# Create a CLogVars container with static or dynamic context variablesclogger.clogvars=CLogVars(static=CLogVar(name='static'),request_id=CLogVar(name='request_id',setter=lambda:str(uuid4())),)asyncdefmy_func():# Set the value of your static variableclogger.setvar('static',value=1)# Set the value of your dynamic variableclogger.setvar('request_id')clogger.info(f"Hello World!")asyncdefmain():awaitasyncio.gather(my_func())if__name__=='__main__':asyncio.run(main())Output{'static': 1, 'request_id': '7e643fe2-bc7a-498c-a0fe-66ae58c671da'} Hello World!TutorialThis should be as simple as it gets!Let's assume that we have a package with the following structure:my_app/
│
├── my_app/
│   ├── __init__.py
│   ├── runner.py
├── main.pywhere yourmain.pyis simply running the two tasks defined in your runner.py module.""" main.py """importasynciofrommy_app.runnerimporttask1,task2asyncdefmain():awaitasyncio.gather(task1(),task2(),)asyncio.run(main())""" runner.py """asyncdeftask1():passasyncdeftask2():passLogging configurationIn the__init__.pymodule of your project setup your logger.
This will be very similar to the way you would normally configure a regular logger from the standard library.""" __init__.py """importloggingfromlogging.handlersimportTimedRotatingFileHandlerimportosimportcontextlogger# Create a CLogger instanceclogger=contextlogger.getCLogger(__name__)# The the logging levelclogger.setLevel('DEBUG')# Create a logging formatterlogging_format="%(asctime)s%(levelname)s%(name)s%(message)s"formatter=logging.Formatter(logging_format)# Create handlers for console loggerconsole_handler=logging.StreamHandler()console_handler.setFormatter(formatter)clogger.addHandler(console_handler)# Create handlers for file loggerLOG_DIR='logs'APP='MY-APP'ifnotos.path.exists(LOG_DIR):os.makedirs(LOG_DIR)file_handler=TimedRotatingFileHandler(f"{LOG_DIR}/{APP}.log",when="midnight",interval=1)file_handler.setFormatter(formatter)file_handler.suffix="%Y%m%d"clogger.addHandler(file_handler)So far, the only thing that we have done differently is that instead of using thegetLoggerfunction of the standard logging library, we used thegetCLoggerfunction from the contextlogger library.Logging without contextOnce your logger configuration is set, you can use your logger in yourrunner.pyfile (or whatever file you choose as your entry point)""" runner.py """frommy_appimportcloggerasyncdeftask1():clogger.info(f"Hello from{task1.__name__}")asyncdeftask2():clogger.info(f"Hello from{task2.__name__}")As expected, if you run yourmain.pythe output of the clogger will be:2020-12-06 18:07:27,008 INFO my_app Hello from task1
2020-12-06 18:07:27,009 INFO my_app Hello from task2Logging with static contextNow lets add some context to our logging.We can do that by adding a custom UserDict (CLogVars) ofCLogVar(context log variables) to our logger.Let's add a 'static' attribute... Not very useful but why not!""" runner.py """fromcontextloggerimportCLogVars,CLogVarfrommy_appimportcloggerclogger.clogvars=CLogVars(static=CLogVar(name='static'),)asyncdeftask1():# Set our 'static' value for task1clogger.setvar('static',value=1)clogger.info(f"Hello from{task1.__name__}")asyncdeftask2():# Set our 'static' value for task2clogger.setvar('static',value=2)clogger.info(f"Hello from{task2.__name__}")And voila! Now the output of the clogger includes our static values:2020-12-06 18:12:07,505 INFO my_app {'static': 1} Hello from task1
2020-12-06 18:12:07,505 INFO my_app {'static': 2} Hello from task2Logging with dynamic contextOk but that was not very handy, right? Let's do something more realistic. Let's pretend that this is our FastApi application and we want to add a 'request_id' to every request we receive.Now, things get interesting! Our CLogVar can also accept a 'setter' argument which is a function that generates a new uuid every time we enter a new context. Every time we call thesetvarwithout a value, it will try to find asetterto do the job!fromuuidimportuuid4fromcontextloggerimportCLogVars,CLogVarfrommy_appimportcloggerclogger.clogvars=CLogVars(static=CLogVar(name='static'),request_id=CLogVar(name='request_id',setter=lambda:str(uuid4())),)asyncdeftask1():# Set our 'static' value for task1clogger.setvar('static',value=1)# Set our request_id value for task1clogger.setvar('request_id')clogger.info(f"Hello from{task1.__name__}")asyncdeftask2():# Set our 'static' value for task2clogger.setvar('static',value=2)# Set our 'request_id' value for task2clogger.setvar('request_id')clogger.info(f"Hello from{task2.__name__}")And here we are, with something a lot more useful than just a static value:2020-12-06 18:21:17,626 INFO my_app {'request_id': 'd3961bd9-f701-4222-ad32-f204e9eb968a', 'static': 1} Hello from task1
2020-12-06 18:21:17,626 INFO my_app {'request_id': '6d4cdab2-e24b-481b-b54b-12c6ee9bcc1b', 'static': 2} Hello from task2Logging with context in multiple modulesFinally, let's add an extra module just for the sake of it.Now our directory will look like:my_app/
│
├── my_app/
│   ├── __init__.py
│   ├── runner.py
│   ├── another_module.py
├── main.pywhere ouranother_module.pysimply imports the logger and uses it in a function:frommy_appimportcloggerdeftest():clogger.info(f"Hello from another_module")Now, if we call our test function inside the task1 of ourrunner.py:fromuuidimportuuid4fromcontextloggerimportCLogVars,CLogVarfrommy_appimportcloggerfrommy_app.another_moduleimporttestclogger.clogvars=CLogVars(static=CLogVar(name='static'),request_id=CLogVar(name='request_id',setter=lambda:str(uuid4())),)asyncdeftask1():# Set our 'static' value for task1clogger.setvar('static',value=1)# Set our request_id value for task1clogger.setvar('request_id')clogger.info(f"Hello from{task1.__name__}")test()asyncdeftask2():# Set our 'static' value for task2clogger.setvar('static',value=2)# Set our 'request_id' value for task2clogger.setvar('request_id')clogger.info(f"Hello from{task2.__name__}")we should see that the log line that is printed inside ouranother_module.pyhas the same request_id and static values with the log line that is printed in task1. And this is expected since they belong to the same context.2020-12-06 18:33:36,634 INFO my_app {'request_id': '1eff5e40-4b05-4cd1-bd9c-edbee85d2995', 'static': 1} Hello from task1
2020-12-06 18:33:36,635 INFO my_app {'request_id': '1eff5e40-4b05-4cd1-bd9c-edbee85d2995', 'static': 1} Hello from another_module
2020-12-06 18:33:36,635 INFO my_app {'request_id': 'ec68779f-46f6-4ea0-a003-9ddb053887e1', 'static': 2} Hello from task2Structured loggingIf you prefer structured logging, you can create thecloggerinstance using the 'structured' argument, which will cause the message to be printed in a structured format.Then, just change your formatter accordingly.""" __init__.py """importloggingfromlogging.handlersimportTimedRotatingFileHandlerimportosimportcontextlogger# Create a CLogger instanceclogger=contextlogger.getCLogger(__name__,structured=True)# Create a logging formatterlogging_format="{'time': '%(asctime)s', 'level': '%(levelname)s', 'name': '%(name)s',%(message)s}"formatter=logging.Formatter(logging_format)# Create handlers for console loggerconsole_handler=logging.StreamHandler()console_handler.setFormatter(formatter)clogger.addHandler(console_handler)# Create handlers for file loggerLOG_DIR='logs'APP='MY-APP'ifnotos.path.exists(LOG_DIR):os.makedirs(LOG_DIR)file_handler=TimedRotatingFileHandler(f"{LOG_DIR}/{APP}.log",when="midnight",interval=1)file_handler.setFormatter(formatter)file_handler.suffix="%Y%m%d"clogger.addHandler(file_handler)The output result will become:{'time': '2020-12-11 15:52:11,487', 'level': 'INFO', 'name': 'my_app', 'msg': 'Hello from task1', 'static': '1', 'request_id': 'cc75cb8f-f267-4406-b49c-fc2196a686c6'}
{'time': '2020-12-11 15:52:11,487', 'level': 'INFO', 'name': 'my_app', 'msg': 'Hello from another_module', 'static': '1', 'request_id': 'cc75cb8f-f267-4406-b49c-fc2196a686c6'}
{'time': '2020-12-11 15:52:11,487', 'level': 'INFO', 'name': 'my_app', 'msg': 'Hello from task2', 'static': '2', 'request_id': '7117cdb4-a0dd-4e12-89a5-756a03d7f8b1'}Version Updates1.0.0: Introduces breaking changes. CLogger.clogvars has been converted from a List to a custom UserDict to maintain consistency between get/set functionality.ContributionsIf you want to contribute to the package, please have a look at the CONTRIBUTING.md file for some basic instructions.
Feel free to reach me in my email or my twitter account, which you can find in my github profile!LicenseThis project is licensed under the terms of the MIT license.AuthorsChristos Liontos