Package: loggate
LoggateThe complex logging system with support of log metadata and delivery toGrafana Loki.
This library supports threading & asyncio modules.Simple stdout/stderr colorized outputOne example is more than a thousand words.fromloggateimportsetup_logging,get_loggersetup_logging(level='DEBUG')logger=get_logger('component',meta={'version':'1.0.0'})logger.debug('Loading resources for the component')logger.info('Initialize of the component')logger.warning('The component is not ready')logger.error('The component failed.',meta={'inputs':{'A':1,'B':2}})logger.critical('The component unexpected failed.',meta={'attrs':{'A':1,'B':2}})Console output:Exceptionsfromloggateimportsetup_logging,get_loggersetup_logging()logger=get_logger('component',meta={'version':'1.0.0'})try:raiseException('Some error')exceptExceptionasex:logger.error('The component failed.',exc_info=True)Console output:Advanced configurationThe Loggate supports a declarative configuration alike aslogging.config.
But this support profiles as well. It's mean we can declare many logging profiles and switch between them. Default profile is calleddefault.We can use yaml file as configuration file (see next):profiles:default:# Default profilefilters:warning:# This is a filter for stdout logger, that enable only logs with level lower than WARNING.# For logs with WARNING and higher we use stderr logger.class:loggate.LowerLogLevelFilterlevel:WARNINGformatters:colored:# This is stdout/sterr formatter.class:loggate.LogColorFormatterloki:# This is formatter of loki messages.class:loggate.loki.LokiLogFormatterhandlers:stdout:# This is a stdout handlerclass:logging.StreamHandlerstream:ext://sys.stdoutformatter:coloredfilters:-warningstderr:# This is a stderr handlerclass:logging.StreamHandlerstream:ext://sys.stderrformatter:coloredlevel:WARNINGloki:# This is a loki handlerclass:loggate.loki.LokiThreadHandler# for asyncio use loggate.loki.LokiHandlerformatter:lokiurls:-"http://loki1:3100/loki/api/v1/push"-"http://loki2:3100/loki/api/v1/push"-"http://loki3:3100/loki/api/v1/push"meta:# loggate handlers accept metadata as well. Standard logging handlers do not!stage:devip:192.168.1.10loggers:root:level:INFOhandlers:-stdout-stderr-loki'urllib3.connectionpool':level:WARNINGimportyamlfromloggateimportsetup_logging,get_loggerdefget_yaml(filename):withopen(filename,'r+')asfd:returnyaml.safe_load(fd)schema=get_yaml('test.yaml')setup_logging(profiles=schema.get('profiles'))logger=get_logger('component')logger.debug('Loading resources for the component')logger.info('Initialize of the component')logger.warning('The component is not ready')The console output is the same as above, but now we send logs to Loki as well.Loki output:DescriptionMethodsget_loggername- Return logger for this name. Empty name returns root logger.meta- Metadata (dict), which are sent only by this logger.getLogger- only alias forget_loggersetup_logging- init setup of application logging.profiles- Profiles (dict) of logging profiles. When we do not set this parameter, application use predefined profile with logINFOlevel (this level can be set by parameterlevel).default_profile- name of the default profile (default:default)level- This is special parameter for situation when  application use predefined profile (defaultINFO).FiltersClassloggate.LowerLogLevelFilterThis filters out all logs which are higher thanlevel.level- log levelFormattersClassloggate.LogColorFormatterColorized formatter for stdout/stderr.fmt- message format (default:%(LEVEL_COLOR)s%(asctime)s\t [%(levelname)s] %(name)s:%(COLOR_RESET)s %(message)s)datefmt- datetime format (default:%Y-%m-%d %H:%M:%S)style- style of templating (default:%).validate- validate the input format (default: True)INDENTATION_TRACEBACK- default:\t\t\tINDENTATION_METADATA- default:\t\t\t\tCOLOR_DEBUG, ...,COLOR_CRITICAL- set color of this levels (e.g.\x1b[1;31m, seemore colors).COLOR_METADATA- set color metadataCOLOR_TRACEBACK- set color of tracebacksCOLOR_...- set custom colorClassloggate.loki.LokiLogFormatterThis is special loki formatter, this converts log records to jsons.HandlersClassloggate.loki.LokiHandlerThis handler send log records to Loki server. This is blocking implementation of handler.
It means, when we call log method (debug, ...critical) the message is sent in the same thread. We should use
this only for tiny scripts where other ways have a big overhead.level- This handler sends only log records with log level equal or higher than this (default: all =logging.NOTSET).urls- List of loki entrypoints.strategy- Deploy strategy (default:random).random- At the beginning the handler choose random Loki server and others are fallbacks.fallbacks- The handler uses the first Loki server and others are fallbacks.all- The handler send the log record to all loki servers.auth- The Loki authentication, the list with two items (username,password).timeout- Timeout for one delivery try (default: 5s).ssl_verify- Enable ssl verify (default: True).loki_tags- the list of metadata keys, which are sent to Loki server as label (defailt: [logger,level]).meta- Metadata (dict), which are sent only by this handler.Classloggate.loki.LokiAsyncioHandlerThis is non-bloking extending of LokiHandler. We register an extra asyncio task for sending messages to the Loki server.
Parameters are the same asloggate.loki.LokiHandler. This handler usesurllib.requestsmodule in default (aiohttpas optional).
Unfortunatelyurllib.requestsmodule does not support asyncio, it means the sending itself is blocking.
Theloggate.loki.Loki AsyncioHandlercan use the optional dependencyaiohttpfor non-bloking sending.Classloggate.loki.LokiThreadHandlerThis is non-bloking extending of LokiHandler. We register and start an extra thread for sending messages to the Loki server.
Parameters are the same asloggate.loki.LokiHandler.ProfilesThe structure of profiles (parameterprofilesofsetup_logging).<profile_name>:filters:<filter_name>:class:<filter_class><filter_attribute_name>:<filter_attribute_value>formatters:<formatter_name>:class:<formatter_class><formatter_attribute_name>:<formatter_attribute_value>handlers:<handler_name>:class:<handler_class><handler_attribute_name>:<handler_attribute_value>loggers:<logger_name>|root:# definition of root loggerlevel:<log_level>handlers:-<name_of_handler>|<definition_of_handler>disabled:True|False# default: Falsepropagate:True|False# default: Truemeta:<logger_metadata>