Package: powerflex-logging-utilities
powerflex-logging-utilitiesHelpful code for logging in Python by PowerFlex.ModuleDescriptionforbid_toplevel_loggingDisable logging with the top-level root logging functions such aslogging.info.log_slow_callbacksEither warn or info log when an async callback runs for too long.init_loggersA function for easily setting up logging to a file and to stdout.ClassDescriptionJsonFormatterA JSON log formatter to enable structured logging. It depends on thepython-json-loggerpackage.TraceLoggerA Python Logger subclass that adds a TRACE logging levelAsyncNatsLogLevelListenerA NATS interface for changing the program's log level by sending a NATS requestInstallationYou can install fromPyPidirectly:pip install powerflex-logging-utilitiesSample usageInitializing LoggersSetupall Loggersto output JSON to stdout and to a file:importloggingimportsysfrompowerflex_logging_utilitiesimport(JsonFormatter,init_loggers,TraceLogger,)LOG_LEVEL="DEBUG"FILE_LOG_LEVEL="TRACE"LOG_FILE="./logs/trace.log"MAX_LOG_FILE_MB=10MAX_TOTAL_LOG_FILE_MB=10000root_logger=logging.getLogger()# Log warnings with the py.warnings loggerlogging.captureWarnings(True)# Fix iPython autocompletelogging.getLogger("parso").propagate=Falseinit_loggers.init_loggers([root_logger],log_level=LOG_LEVEL,file_log_level=FILE_LOG_LEVEL,filename=LOG_FILE,max_bytes=1000*1000*MAX_LOG_FILE_MB,backup_count=MAX_TOTAL_LOG_FILE_MB//MAX_LOG_FILE_MB,stream=sys.stdout,formatter=JsonFormatter,info_logger=root_logger,)# Either use logging.getLogger or don't initialize a logger until your root logger is configured.logging.setLoggerClass(TraceLogger)logger=logging.getLogger(__name__)This uses Python's logger propagation feature.
We only need to configure the root Logger in order to make sure all other Loggers output in the desired format.You can passformatter_kwargsto enable logging with a different JSON serializer.To use:logger = logging.getLogger(__name__)
logger.info("hello world")Explicitly listing loggersYou can also list the loggers you'd like to configure instead of configuring
the root logger.This could be useful if you configure your package's main loggerlogging.getLogger("package"). You can then use Python's logger propagation by callinglogging.getLogger("package.submodule.a.b.c")to get Logger instances for all
other submodules.importloggingfrompowerflex_logging_utilitiesimport(JsonFormatter,init_loggers,)logger=logging.getLogger("your_package_name")# Log warnings with the py.warnings loggerlogging.captureWarnings(True)init_loggers.init_loggers([logger,"asyncio","py.warnings"],log_level="DEBUG",file_log_level="TRACE",filename="./logs/trace-no-root.log",formatter=JsonFormatter,info_logger=logger,)NOTICE: if you use this method, any loggers you do not explicitly list will have non-JSON output.Using several other utilitiesimportloggingfrompowerflex_logging_utilitiesimport(forbid_toplevel_logging,log_slow_callbacks,)logger=logging.getLogger(__name__)# Log slow async callbacks with two log levelslog_slow_callbacks.log_slow_callbacks(logger)# Forbid functions such as logging.info since they implicitly use the root loggerforbid_toplevel_logging.forbid_logging_with_logging_toplevel()Using the JSON formatterimportloggingimportsysfrompowerflex_logging_utilitiesimportJsonFormatterlog_handler=logging.StreamHandler(stream=sys.stdout)log_handler.setLevel("DEBUG")log_handler.setFormatter(JsonFormatter())logger=logging.getLogger(__name__)logger.addHandler(log_handler)logger.setLevel("DEBUG")logger.info("hello world",extra={"data":["log structured data",":D"],1:"handles non string key",}){
  "message": "hello world",
  "name": "__main__",
  "module": "<ipython-input-10-b016ce80d46f>",
  "lineno": 1,
  "funcName": "<cell line: 1>",
  "filename": "<ipython-input-10-b016ce80d46f>",
  "asctime": "2022-05-12 01:04:16,824",
  "data": [
    "log structured data",
    ":D"
  ],
  "severity": "INFO",
  "1": "handles non string key"
}Using pipenvRunmake setup-with-pipenvto install all dependencies.
Make sure you have the version of Python specified in.tool-versionsor simply change this file to your Python version (must be 3.8+).Runpipenv shellor run the followingmakecommands withpipenv run make ....
You could also aliaspmaketopipenv run makefor convenience.TestsThere is 100% code coverage.make test-unitTo test in several versions of Python, run:toxTo download several versions of Python, usepyenvorasdfTo usepyenv, install ithereand run the following script:./install_python_versions_pyenv.shTo useasdf, install the core partshereand run the following commands:./install_python_versions_asdf.shTesting the code in this READMEmake test-readmeChecking code qualityThe Github Actions will run all of the following checks on the code.Code formattingmake format-fixLintingmake lintType checkingmake type-check-strictReleasing toPyPi.orgMake sure all code checks have passed withmake commitready.Make sure you commit all code you wish to release withgit commit.Set the version in./src/powerflex_monitoring/VERSIONPlease attempt to followsemantic versioning.Runmake bump-versionto commit the change to theVERSIONfile.Runmake releaseto upload the package to pypi.org and to push a new git tag