Package: pygogo
pygogo: a Python logger with superpowersIndexIntroduction|Requirements|Motivation|Usage|Installation|Project Structure|Design Principles|Structured Logging|Formatters|Handlers|Scripts|Contributing|LicenseIntroductionpygogo is a Python logginglibraryandcommand-line interfacewith super powers.
pygogo leverages the standard Pythonlogging moduleunder the hood, so there’s
no need to learn yet-another logging library. The default implementation sends
all messages tostdout, and any messages at levelWARNINGor above also tostderr.With pygogo, you canLog via different handlers depending on the event severityFormat log messages as plain text, csv, json, and more..Send logs to stdout, stderr, file, email, sockets, and more..Inter-operate with the standard python logging moduleand much more…Requirementspygogo has been tested and is known to work on Python 3.7, 3.8, and 3.9; and PyPy3.7.MotivationThe standard logging module is great, but requires a ton of boilerplate before
you can do anything really interesting with it. I designed pygogo to provide
many useful logging use-cases out of the box. A reimplementation ofUsing LoggerAdapters to impart contextual informationis shown below:importpygogoasgogologger=gogo.Gogo(__name__).get_structured_logger(connid='1234')logger.info('log message')# Prints the following to stdout{"message":"log message","connid":"1234"}Usagepygogo is intended to be used either directly as a Pythonlibraryor from
the terminal via thecommand-line interface.LibraryExamplesHello Worldfrompygogoimportloggerlogger.debug('hello world')logger.error('hello error')# Prints the following to `stdout`helloworldhelloerror# Prints the following to `stderr`helloerrorLog based debuggingimportpygogoasgogodefmain(verbose=False):logger=gogo.Gogo(__name__,verbose=verbose).loggerlogger.debug('I will log to `stdout` only if `verbose` is True')logger.info('I will log to `stdout` always')logger.warning('I will log to both `stdout` and `stderr` always')Disabled dual loggingimportpygogoasgogologger=gogo.Gogo(monolog=True).loggerlogger.debug('debug message')logger.info('info message')logger.warning('warning message')logger.error('error message')logger.critical('critical message')# Prints the following to `stdout.log` (all messages at level `INFO` or below):debugmessageinfomessage# Prints the following to `stderr` (messages at level `WARNING` or above):warningmessageerrormessagecriticalmessageCustom formatter[1]importloggingimportpygogoasgogolog_format='%(asctime)s-%(name)s-%(levelname)s-%(message)s'formatter=logging.Formatter(log_format)logger=gogo.Gogo('examples.fmt',low_hdlr=gogo.handlers.file_hdlr('custom_fmt.log'),low_formatter=formatter,high_level='error',high_formatter=formatter).logger# Now let's log something!logger.debug('debug message')logger.info('info message')logger.warn('warn message')logger.error('error message')logger.critical('critical message')# Prints the following to `custom_fmt.log` (all messages):2015-12-1818:51:30,416-examples.fmt.base-DEBUG-debugmessage2015-12-1818:51:30,416-examples.fmt.base-INFO-infomessage2015-12-1818:51:30,416-examples.fmt.base-WARNING-warnmessage2015-12-1818:51:30,416-examples.fmt.base-ERROR-errormessage2015-12-1818:51:30,416-examples.fmt.base-CRITICAL-criticalmessage# Prints the following to `stderr` (messages at level `ERROR` or above):2015-12-1818:51:30,416-examples.fmt.base-ERROR-errormessage2015-12-1818:51:30,416-examples.fmt.base-CRITICAL-criticalmessageStructured logging[2]importpygogoasgogoformatter=gogo.formatters.structured_formatterkwargs={'low_level':'info','low_formatter':formatter}logger=gogo.Gogo('examples.structured',**kwargs).loggerextra={'set_value':set([1,2,3]),'snowman':'☃'}logger.info('log message',extra=extra)# doctest: +ELLIPSIS# Prints the following to `stdout`:{"snowman":"\u2603","name":"examples.structured.base","level":"INFO","message":"log message","time":"2015-12-18 18:52:39","msecs":58.973073959350586,"set_value":[1,2,3]}Using Filters to impart contextual information[3]importloggingimportpygogoasgogolevels=('DEBUG','INFO','WARNING','ERROR','CRITICAL')log_frmt=('%(asctime)-4s%(name)-5s%(levelname)-8sIP:%(ip)-15sUser: ''%(user)-8s%(message)s')formatter=logging.Formatter(log_frmt)going=gogo.Gogo('a',low_formatter=formatter)a1=going.get_logger('b.c',ip='123.231.231.123',user='fred')a2=going.get_logger('e.f',ip='192.168.0.1',user='sheila')# Now let's log something!a1.debug('A debug message')a1.info('An info%s','message')forlevelin[getattr(logging,l)forlinlevels]:name=logging.getLevelName(level)a2.log(level,'A%smsg',name)# Prints the following to `stdout` (all messages):2015-12-1910:12:24,479a.b.cDEBUGIP:123.231.231.123User:fredAdebugmessage2015-12-1910:12:24,479a.b.cINFOIP:123.231.231.123User:fredAninfomessage2015-12-1910:12:24,479a.e.fDEBUGIP:192.168.0.1User:sheilaADEBUGmsg2015-12-1910:12:24,479a.e.fINFOIP:192.168.0.1User:sheilaANINFOmsg2015-12-1910:12:24,479a.e.fWARNINGIP:192.168.0.1User:sheilaAWARNINGmsg2015-12-1910:12:24,479a.e.fERRORIP:192.168.0.1User:sheilaANERRORmsg2015-12-1910:12:24,479a.e.fCRITICALIP:192.168.0.1User:sheilaACRITICALmsg# Prints the following to `stderr` (messages at level `WARNING` or above):2015-12-1910:12:24,479a.e.fWARNINGIP:192.168.0.1User:sheilaAWARNINGmsg2015-12-1910:12:24,479a.e.fERRORIP:192.168.0.1User:sheilaANERRORmsg2015-12-1910:12:24,479a.e.fCRITICALIP:192.168.0.1User:sheilaACRITICALmsgMultiple loggers[4]importpygogoasgogogoing=gogo.Gogo('examples.lggrs',low_hdlr=gogo.handlers.file_hdlr('multi_lggrs.log'),low_formatter=gogo.formatters.fixed_formatter,high_level='info',high_formatter=gogo.formatters.console_formatter)root=going.loggerlogger1=going.get_logger('area1')logger2=going.get_logger('area2')# Now let's log something!root.info('Jackdaws love my big sphinx.')logger1.debug('Quick zephyrs blow, daft Jim.')logger1.info('How daft jumping zebras vex.')logger2.warning('Jail zesty vixen who grabbed pay.')logger2.error('The five boxing wizards jump.')# Prints the following to `multi_lggrs.log` (all messages):2015-12-1817:21:37.417examples.lggrs.baseINFOJackdawslovemybigsphinx.2015-12-1817:21:37.417examples.lggrs.area1DEBUGQuickzephyrsblow,daftJim.2015-12-1817:21:37.417examples.lggrs.area1INFOHowdaftjumpingzebrasvex.2015-12-1817:21:37.417examples.lggrs.area2WARNINGJailzestyvixenwhograbbedpay.2015-12-1817:21:37.417examples.lggrs.area2ERRORThefiveboxingwizardsjump.# Prints the following to `stderr` (messages at level `INFO` or above):examples.lggrs.base:INFOJackdawslovemybigsphinx.examples.lggrs.area1:INFOHowdaftjumpingzebrasvex.examples.lggrs.area2:WARNINGJailzestyvixenwhograbbedpay.examples.lggrs.area2:ERRORThefiveboxingwizardsjump.Notes[1]https://docs.python.org/2/howto/logging-cookbook.html#multiple-handlers-and-formatters[2]https://docs.python.org/2/howto/logging-cookbook.html#implementing-structured-logging[3]https://docs.python.org/2/howto/logging-cookbook.html#using-filters-to-impart-contextual-information[4]https://docs.python.org/2/howto/logging-cookbook.html#logging-to-multiple-destinationsCommand-line InterfaceExamplesBasic Usagegogo[options]<message>show helpgogo-hCLI usageusage: gogo [options] <message>description: Logs a given messagepositional arguments:message               The message to log (defaults to reading from stdin).optional arguments:-h,--helpshow this help message and exit-lLEVEL,--msg-levelLEVELThe level to log the message (default: info).
Must be one of: critical, error, warning, info, debug.-nNAME,--nameNAMEThe logger name (default: pygogo)-DHANDLER,--high-hdlrHANDLERThe high pass log handler (default: stderr).
Must be one of: buffered, email, file, fileobj,
socket, stderr, stdout, syslog, webhook.-dHANDLER,--low-hdlrHANDLERThe low pass log handler (default: stdout).
Must be one of: buffered, email, file, fileobj,
socket, stderr, stdout, syslog, webhook.-LLEVEL,--high-levelLEVELMin level to log to the high pass handler
(default: warning).
Must be one of: buffered, email, file, fileobj,
socket, stderr, stdout, syslog, webhook.-eLEVEL,--low-levelLEVELMin level to log to the low pass handler
(default: debug).
Must be one of: buffered, email, file, fileobj,
socket, stderr, stdout, syslog, webhook.-FFORMAT,--high-formatFORMATHigh pass handler log format (default: basic).
Must be one of: basic, bom, console, csv,
fixed, json, structured.-oFORMAT,--low-formatFORMATLow pass handler log format (default: basic).
Must be one of: basic, bom, console, csv,
fixed, json, structured.-m,--monologLog high level events only to high pass handler.-fFILENAME,--filenameFILENAMEThe filename to log to.
Required for the follow handlers: file.-sSUBJECT,--subjectSUBJECTThe log subject (default: You’ve got mail).
Used in the follow handlers: email.-uURL,--urlURLThe log url. Required for the follow handlers: webhook.-HHOST,--hostHOSTThe host.
Used in the follow handlers: socket and syslog.-pNUM,--portNUMThe port number.
Used in the follow handlers: socket and syslog.-t,--tcpUse TCP instead of UDP.
Used in the follow handlers: socket and syslog.-g,--getUse a GET request instead of POST.
Used in the follow handlers: webhook.-v,--versionShow version and exit.-V,--verboseIncrease output verbosity.Hello Worldgogo'hello world'Log based debugginggogo'default info level will log to `stdout`'gogo--level=debug"debug won't log"gogo--level=debug-V'verbose will log to `stdout`'gogo--level=info'info will log to `stdout`'gogo--level=warning'warning will log to both `stdout` and `stderr`'# Prints the following to `stdout`:defaultinfolevelwilllogto`stdout`verbosewilllogto`stdout`infowilllogto`stdout`warningwilllogtoboth`stdout`and`stderr`# Prints the following to `stderr`:warningwilllogtoboth`stdout`and`stderr`Disable dual logginggogo--level=debug-V'debug message'gogo--level=info'info message'gogo--level=warning-m'warning message'gogo--level=error-m'error message'gogo--level=critical-m'critical message'# Prints the following to `stdout.log` (all messages at level `INFO` or below):debugmessageinfomessage# Prints the following to `stderr` (messages at level `WARNING` or above):warningmessageerrormessagecriticalmessageStructured logginggogo--low-format=json'log message'# Prints the following to `stdout`:{"time":"2015-12-19 11:26:53.776","name":"pygogo.runner","level":"INFO","message":"log message"}Alternate handlergogo--low-hdlr=file'log message'# Prints the following to `pygogo.log` in the current dir (assuming the current dir is named `pygogo`):{"time":"2015-12-19 11:26:53.776","name":"pygogo.runner","level":"INFO","message":"log message"}Installation(You are using avirtualenv, right?)At the command line, install pygogo using eitherpip(recommended)pipinstallpygogooreasy_installeasy_installpygogoPlease see theinstallation docfor more details.Project Structure┌──bin│└──gogo├──docs│├──AUTHORS.rst│├──CHANGES.rst│├──INSTALLATION.rst│└──TODO.rst├──helpers│├──check-stage│├──clean│├──pippy│├──srcdist│└──wheel├──pygogo│├──__init__.py│├──formatters.py│├──handlers.py│├──main.py│└──utils.py├──tests│├──__init__.py│├──standard.rc│├──test.py│└──test_main.py├──CONTRIBUTING.rst├──LICENSE├──MANIFEST.in├──Makefile├──README.rst├──dev-requirements.txt├──examples.py├──manage.py├──py2-requirements.txt├──setup.cfg├──setup.py└──tox.iniDesign Principlesthe built-inloggingmodule isn’t broken so don’t reinvent the wheelprefer functions over objectskeep the API as simple as possibleStructured LoggingThere are severals ways to get structured (machine readable) log messages using pygogo.
Each method makes a different customization/complexity trade-off which is
outlined below:SetupThe following methods make use of these variables.importpygogoasgogokwargs={'contextual':True}extra={'additional':True}Methodsbasic structured loggerThe simplest to use. Useful if you don’t need message metadata, i.e., log level,
log name, and log time.logger=gogo.Gogo('basic').get_structured_logger('base',**kwargs)logger.debug('message',extra=extra)# Prints the following to `stdout`:{"additional":true,"contextual":true,"message":"message"}structured formatterRequires an additional step of specifying a formatter. Useful if you need
message metadata, i.e., log level, log name, and log time.formatter=gogo.formatters.structured_formatterlogger=gogo.Gogo('struct',low_formatter=formatter).get_logger(**kwargs)logger.debug('message',extra=extra)# Prints the following to `stdout`:{"additional":true,"contextual":true,"level":"DEBUG","message":"message","msecs":760.5140209197998,"name":"struct.base","time":"2015-12-19 14:25:58"}JSON formatterRequires an additional step of specifying a formatter. Useful if you require
millisecond precision in the date. If you are ok with having the milliseconds
in a separate field, consider thestructured formattersince it supports
theextrakeyword and contextual information.formatter=gogo.formatters.json_formatterlogger=gogo.Gogo('json',low_formatter=formatter).get_logger(**kwargs)logger.debug('message',extra=extra)# Prints the following to `stdout`:{"level":"DEBUG","message":"message","name":"json.base","time":"2015-12-19 14:25:58.760"}# Note that both `extra` and `kwargs` were ignoredcustom loggerThe most complex and customizable. Useful if you need a custom
log or date format not provided by the above methods. However, even though this
method supports theextrakeyword when logging, it is static (unlike thestructured loggerorstructured formatter). This is because the log
format must be specified at the time of the log’s creation and therefore can’t
adapt to log messages with differingextraparameters.logfmt=('{"time": "%(asctime)s.%(msecs)d", "name": "%(name)s", "level":'' "%(levelname)s", "message": "%(message)s", ''"contextual": "%(contextual)s", "additional": "%(additional)s"}')fmtr=logging.Formatter(logfmt,datefmt=gogo.formatters.DATEFMT)logger=gogo.Gogo('custom',low_formatter=fmtr).get_logger(**kwargs)logger.debug('message',extra=extra)# Prints the following to `stdout`:{"additional":"True","contextual":"True","level":"DEBUG","message":"message","name":"custom.logger","time":"2015-12-19 14:25:58.760"}SummaryThe following table can help make sense of the different methods:structured loggerstructured formatterjson formattercustom loggercontextual information✔✔✔extraparam support✔✔✔dynamicextrasupport✔✔message metadata✔✔✔available via the command line✔✔msecsfield✔milliseconds in time field✔✔custom date format✔custom log format✔Formatterspygogo has several builtin formatters and also supports anylogging.Formatterinstance.Examplesbuiltin CSV format in pythonimportpygogoasgogoformatter=gogo.formatters.csv_formattergogo.Gogo('csv',low_formatter=formatter).logger.debug('message')# Prints the following to `stdout`:2015-12-1917:03:48.99,csv.base,DEBUG,"message"logging.Formatterinstance in pythonimportloggingimportpygogoasgogodatefmt=gogo.formatters.DATEFMTformatter=logging.Formatter(gogo.formatters.CSV_FORMAT,datefmt=datefmt)gogo.Gogo('csv',low_formatter=formatter).get_logger('custom').debug('message')# Prints the following to `stdout`:2015-12-1917:03:48.99,csv.custom,DEBUG,"message"builtin CSV format via CLIgogo--low-format=csv'message'# Prints the following to `stdout`:2015-12-1915:51:32.16,pygogo.runner,INFO,"message"SummaryThe following table can help make sense of the different builtin formatters:namemessagebasicmessagebommessageconsolename: INFO     messagecsv2015-12-19 15:51:32.16,name,INFO,”message”fixed2015-12-19 15:51:32.16 name INFO     messagejson{“level”: “INFO”, “message”: “message”, “name”: “name”, “time”: “2015-12-19 15:51:32.16”}structured{“level”: “INFO”, “message”: “message”, “msecs”: 16.5140209197998, “name”: “name”, “time”: “2015-12-19 15:51:32”}Handlerspygogo has several builtin handlers and also supports any instance from thelogging.handlersmodule.Examplesbuiltin stdout handler in pythonimportpygogoasgogohdlr=gogo.handlers.stdout_hdlr()gogo.Gogo('stdout',low_hdlr=hdlr).logger.debug('message')# Prints 'message' to `stdout`logging.StreamHandlerinstance in pythonimportloggingimportsysimportpygogoasgogohdlr=logging.StreamHandler(sys.stdout)gogo.Gogo('stdout',low_hdlr=hdlr).get_logger('custom').debug('message')# Prints 'message' to `stdout`builtin CSV format via CLIgogo--low-hdlr=stdout'message'# Prints 'message' to `stdout`SummaryThe following table can help make sense of the different builtin handlers:namedescriptionbufferedHolds log in memory until it reaches its
capacity, or it logs a message with a
level at or above the flush levelemailEmails log to a given email addressfileWrites log to a given filenamefileobjWrites log to a given file-like objectsocketWrites log to a given network socketstderrWrites log to standard errorstdoutWrites log to standard outputsyslogWrites log to syslogwebhookPOSTs log to a urlScriptspygogo comes with a built in task managermanage.pySetuppipinstall-rdev-requirements.txtExamplesRun python linter and nose testsmanagelintmanagetestContributingPlease mimic the coding style/conventions used in this repo.
If you add new classes or functions, please add the appropriate doc blocks with
examples. Also, make sure the python linter and nose tests pass.Please see thecontributing docfor more details.Licensepygogo is distributed under theMIT License.Changelog%%version%% (unreleased)NewAdd changelog.Add quick logger instantiation.BugfixesDon’t parse args unless running as script.Fix CLI usage text.Fix examples.Fix packaging scripts.Update docs, fix requirements, and remove sphinx.v0.8.13 (2016-01-01)BugfixesFix new year bug.v0.8.10 (2015-12-30)BugfixesFix capitalization.Fix optional dependency parsing.Fix spacing and remove unneeded marker.v0.8.9 (2015-12-30)BugfixesFix conditional dependency configuration.Fix requirements.v0.8.8 (2015-12-29)BugfixesFix project name.v0.8.7 (2015-12-29)BugfixesAdd pygogo to script path.Add pygogo to test path.v0.8.6 (2015-12-29)BugfixesFix doccheck errors.v0.8.5 (2015-12-29)BugfixesAdd missing helper script.v0.8.4 (2015-12-29)BugfixesFix setup.py (again!)v0.8.3 (2015-12-29)BugfixesFix setup.py.v0.8.2 (2015-12-29)BugfixesFix travis.v0.8.1 (2015-12-29)BugfixesFix tox config and tests.Change helper script name.Fix spacing.Fix makefile error.Only clean once.v0.8.0 (2015-12-25)BugfixesFix sphinx.Fix tox.Fix lint errors.Fix tests.v0.7.0 (2015-12-23)BugfixesFix lint error.v0.6.3 (2015-12-22)BugfixesFix pylint errors.Fix pylint and make check optional.Don’t overwrite fmtrs argument.v0.5.0 (2015-12-20)BugfixesFix duplicate handler/filter bug.Fix tests.Fix assertion arg orders.v0.4.0 (2015-12-19)NewAdd BOM formatter.BugfixesFix spelling and spacing.Fix lint errors.Fix example.Fix lint errors and add msecs to formatter.Remove ‘ascitime’ from StructuredFormatter.Use port if given.Fix docblocks.v0.2.2 (2015-12-07)BugfixesFix typo.