Package: python-logging-rabbitmq
python-logging-rabbitmqLogging handler to ships logs to RabbitMQ. Compatible with Django.InstallationInstall using pip.pipinstallpython_logging_rabbitmqVersionsVersionDependency>= 2.xPika == 0.13<= 1.1.1Pika <= 0.10HandlersThis package has two built-in handlers that you can import as follows:frompython_logging_rabbitmqimportRabbitMQHandleror (thanks to@wallezhang)frompython_logging_rabbitmqimportRabbitMQHandlerOneWayHandlerDescriptionRabbitMQHandlerBasic handler for sending logs to RabbitMQ. Every record will be delivered directly to RabbitMQ using the exchange configured.RabbitMQHandlerOneWayHigh throughput handler. Initializes an internal queue where logs are stored temporarily. A thread is used to deliver the logs to RabbitMQ using the exchange configured. Your app doesn't need to wait until the log is delivered. Notice that if the main thread dies you might lose logs.Standalone pythonTo use with python first create a logger for your app, then create an instance of the handler and add it to the logger created.importloggingfrompython_logging_rabbitmqimportRabbitMQHandlerlogger=logging.getLogger('myapp')logger.setLevel(logging.DEBUG)rabbit=RabbitMQHandler(host='localhost',port=5672)logger.addHandler(rabbit)logger.debug('test debug')As result, a similar message as follows will be sent to RabbitMQ:{"relativeCreated":280.61580657958984,"process":13105,"args":[],"module":"test","funcName":"<module>","host":"albertomr86-laptop","exc_text":null,"name":"myapp","thread":140032818181888,"created":1482290387.454017,"threadName":"MainThread","msecs":454.01692390441895,"filename":"test.py","levelno":10,"processName":"MainProcess","pathname":"test.py","lineno":11,"msg":"test debug","exc_info":null,"levelname":"DEBUG"}Sending logsBy default, logs will be sent to RabbitMQ using the exchange'log', this should be oftype topic. Therouting keyused is formed by concatenating thelogger nameand thelog level. For example:importloggingfrompython_logging_rabbitmqimportRabbitMQHandlerlogger=logging.getLogger('myapp')logger.setLevel(logging.DEBUG)logger.addHandler(RabbitMQHandler(host='localhost',port=5672))logger.info('test info')logger.debug('test debug')logger.warning('test warning')The messages will be sent using the following routing keys:myapp.INFOmyapp.DEBUGmyapp.WARNINGFor an explanation about topics and routing keys go tohttps://www.rabbitmq.com/tutorials/tutorial-five-python.htmlWhen create the handler, you're able to specify different parameters in order to connect to RabbitMQ or configure the handler behavior.Overriding routing-key creationIf you wish to override routing-key format entirely, you can passrouting_key_formatterfunction which takesLogRecordobjects and returns
routing-key. For example:RabbitMQHandler(host='localhost',port=5672,routing_key_formatter=lambdar:('some_exchange_prefix.{}'.format(r.levelname.lower())))ConfigurationThese are the configuration allowed:ParameterDescriptionDefaulthostRabbitMQ Server hostname or ip address.localhostportRabbitMQ Server port.5672usernameUsername for authentication.NonepasswordProvide a password for the username.NoneexchangeName of the exchange to publish the logs. This exchange is considered of type topic.logdeclare_exchangeWhether or not to declare the exchange.Falseremove_requestIf True (default), remove request & exc info.Truerouting_key_formatCustomize how messages are routed to the queues.{name}.{level}routing_key_formatterCustomize how routing-key is constructed.Noneconnection_paramsAllow extra params to connect with RabbitMQ.NoneformatterUse custom formatter for the logs.python_logging_rabbitmq.JSONFormatterclose_after_emitClose the active connection after send a log. A new connection is open for the next log.FalsefieldsDict to add as a field in each logs send to RabbitMQ. This is useful when you want fields in each log but without pass them every time.Nonefields_under_rootWhen is True, each key in parameter 'fields' will be added as an entry in the log, otherwise they will be logged under the key 'fields'.Truemessage_headersA dictionary of headers to be published with the message.Nonerecord_fieldsA set of attributes that should be preserved from the record object.Noneexclude_record_fieldsA set of attributes that should be ignored from the record object.NoneheartbeatLower bound for heartbeat timeout.60content_typeThe format of the message sent to the queue.text/plainExamplesRabbitMQ Connectionrabbit=RabbitMQHandler(host='localhost',port=5672,username='guest',password='guest',connection_params={'virtual_host':'/','connection_attempts':3,'socket_timeout':5000})Custom fieldsrabbit=RabbitMQHandler(host='localhost',port=5672,fields={'source':'MyApp','env':'production'},fields_under_root=True)Custom formatterBy default, python_logging_rabbitmq implements a custom JSONFormatter; but if you prefer to format your own message you could do it as follow:importloggingfrompython_logging_rabbitmqimportRabbitMQHandlerFORMAT='%(asctime)-15s%(message)s'formatter=logging.Formatter(fmt=FORMAT)rabbit=RabbitMQHandler(formatter=formatter)For a custom JSON Formatter take a look athttps://github.com/madzak/python-json-loggerDjangoTo use with Django add the handler in thelogging config.LOGGING={'version':1,'disable_existing_loggers':False,'handlers':{'rabbit':{'level':'DEBUG','class':'python_logging_rabbitmq.RabbitMQHandler','host':'localhost'}},'loggers':{'myapp':{'handlers':['rabbit'],'level':'DEBUG','propagate':False}}}ConfigurationSame as when use it with standalone python, you could configure the handle directly when declaring it in the config:LOGGING={'version':1,'disable_existing_loggers':False,'handlers':{'rabbit':{'level':'DEBUG','class':'python_logging_rabbitmq.RabbitMQHandler','host':'localhost','port':5672,'username':'guest','password':'guest','exchange':'log','declare_exchange':False,'connection_params':{'virtual_host':'/','connection_attempts':3,'socket_timeout':5000},'fields':{'source':'MainAPI','env':'production'},'fields_under_root':True}},'loggers':{'myapp':{'handlers':['rabbit'],'level':'DEBUG','propagate':False}}}Custom formatterLOGGING={'version':1,'disable_existing_loggers':False,'formatters':{'standard':{'format':'%(levelname)-8s[%(asctime)s]:%(message)s'}},'handlers':{'rabbit':{'level':'DEBUG','class':'python_logging_rabbitmq.RabbitMQHandler','host':'localhost','formatter':'standard'}},'loggers':{'myapp':{'handlers':['rabbit'],'level':'DEBUG','propagate':False}}}JSON formatterpipinstallpython-json-loggerLOGGING={'version':1,'disable_existing_loggers':False,'formatters':{'json':{'()':'pythonjsonlogger.jsonlogger.JsonFormatter','fmt':'%(name)s%(levelname) %(asctime)s%(message)s'}},'handlers':{'rabbit':{'level':'DEBUG','class':'python_logging_rabbitmq.RabbitMQHandler','host':'localhost','formatter':'json'}},'loggers':{'myapp':{'handlers':['rabbit'],'level':'DEBUG','propagate':False}}}ReleasesCHANGELOGSimilar effortsMQHandler (https://github.com/ziXiong/MQHandler)http://stackoverflow.com/a/25479008/2716524LogBook (http://pydoc.net/Python/Logbook/0.12.5/logbook.queues/)https://pypi.python.org/pypi/py-amqp-logging/0.1