Package: isentia-common-python
Isentia common tools for PythonThis library collects tools and code that are shared across repositories and projects within Isentia.Instructions to use this libraryInstallation is simple:pipinstallisentia-common-pythonRight now the library only contains json logging code. We might add more features later.JSON loggingAuthors: Neville Tummon, Toney Thomas, Koen DouterloignePlace the following somewhere near the start of your app.fromisentia_common.logimportinit_json_logginginit_json_logging("my_namespace")Now use logging just like you always do.importlogginglogging.info("Hello world!")logging.warning("Hello world!",extra={"http_code":418,"reply":"I'm a teapot"})This will print{"asctime": "2020-07-06 13:01:46,859", "name": "root", "levelname": "INFO", "message": "Hello world!"}
{"asctime": "2020-07-06 13:01:46,859", "name": "root", "levelname": "WARNING", "message": "Hello world!", "my_namespace": {"http_code": 418, "reply": "I'm a teapot"}}As you see, it structures the output as JSON and adds the typical fields. You can use the "extra" parameter when logging objects. All objects logged like this will be put into the namespace that you specified in the init call, when you didinit_json_logging("my_namespace").The namespacing makes it very easy to search through your logs in tools like ElasticSearch / Kibana. In particular, it prevents conflicts with other teams or projects that want to use the same field names.By default, the logging level is set to INFO. You can override this just like normal, usinglogging.getLogger().setLevel(logging.DEBUG)More fieldsBy default we only print the time, logger name (usually root), level, message and extra fields. If you need more visibility, you can enableextendedmode.init_json_logging("my_namespace",extended=True)logging.info("Hello world!")Enablingextendedwill also include process id, filename, lineno and funcname in the JSON:{"asctime": "2020-07-06 13:48:38,015", "name": "root", "levelname": "INFO", "message": "Hello world!", "process": 8015, "filename": "inspect_logging.py", "lineno": 8, "funcName": "<module>"}Module-level loggersAll Python logging will output JSON. In particular, module-level loggers will also log to JSON. (As an aside, code typically does not require module level loggers)module_lvl_logger=logging.getLogger("special.module")module_lvl_logger.setLevel(logging.DEBUG)module_lvl_logger.debug("an info message",extra={"http_status_code":200,"endpoint":"/api/no-problemo"})will print{"asctime": "2020-07-06 13:01:46,859", "name": "special.module", "levelname": "DEBUG", "message": "an info message", "my_namespace": {"http_status_code": 200, "endpoint": "/api/no-problemo"}}Only the "name" field changes.Global overrideNow you may be thinking: Great, all fields are namespaced, but what if I want to track a field across projects / apps? Like traceID or summary_id? I still want my apps to have different namespaces, but I also want to be able to log some fields that are global, i.e. that live outside of the namespace.For this usecase we've made the extra field "global" a special field, which gets moved out of the "extra". Look at the following example:importlogginglogging.info("Hello world!",extra={"http_code":418,"reply":"I'm a teapot","global":{"request_id":50}})This will print{"asctime": "2020-07-06 13:01:46,859", "name": "root", "levelname": "INFO", "message": "info with global", "global": {"request_id": 50}, "my_app_name": {"http_code": 418, "reply": "I'm a teapot"}}In short, the field "global" is like a shared namespace. Only put in there what you need, as you might clash with other teams.ConstantsIf you wish a global variable to be included with every log message (e.g. a trace_id or request_id)
you can set this in the init call.init_json_logging('my_namespace',constants={'trace_id':123})logging.info("Hello world!",extra={"http_code":418,"reply":"I'm a teapot","global":{"request_id":50}})This will print{"asctime": "2020-07-06 13:01:46,859", "name": "root", "levelname": "INFO", "message": "info with global", "global": {"trace_id":123, "request_id": 50}, "my_app_name": {"http_code": 418, "reply": "I'm a teapot"}}And every other message will include the trace_id in global namespace.Low-level detailsThis library is built on top ofpython-json-logger. We extended python-json-logger via monkey-patching to add the concept of namespacing and global overrides.