Package: pylogbeat
PyLogBeatPyLogBeat is a simple, incomplete implementation of the Beats protocol
used by Elastic Beats and Logstash. For more information about Beats seehttps://www.elastic.co/products/beatsandhttps://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html.With this library it is possible to send log messages or any data to
Logstash' beats input plugin or any other service which implements
the Beats protocol.The main difference to other transport mechanisms like direct TCP
or UDP transfer is that with the Beats protocol there is a higher
reliability of the data transfer, especially since the server
acknowledges the data it received so the client knows whether and
what to resend.InstallationThe easiest method is to install directly from pypi using pip:pip install pylogbeatIf you prefer, you can download PyLogBeat fromhttps://github.com/eht16/pylogbeatand install it directly from source:python setup.py installDue to a bug (https://github.com/logstash-plugins/logstash-input-beats/pull/342)
in some Logstash versions, you will need Logstash >= 5.6.12 or Logstash >= 6.4.0.Get the SourceThe source code is available athttps://github.com/eht16/pylogbeat/.UsageSimple usemessage={'@timestamp':'2018-01-02T01:02:03','@version':'1','message':'hello world'}client=PyLogBeatClient('localhost',5959,ssl_enable=False)client.connect()client.send([message])client.close()Using a context managerwithPyLogBeatClient('localhost',5959,ssl_enable=False)asclient:client.send([message])Using a SSL connectionwithPyLogBeatClient('localhost',5959,ssl_enable=True,ssl_verify=True,keyfile='certificate.key',certfile='certificate.crt',ca_certs='ca.crt')asclient:client.send([message])For details regarding the SSL certificates and how to configure the
Logstash input for SSL, seehttps://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html.Message FormatPyLogBeatClient.send()accepts a sequence (list, tuple, set) of "messages".
The messages itself can either be adictobject representing the final
message to be sent to Logstash or abytesorstringobject which must
contain properly formattedJSON.
If adictis passed as element, it is converted toJSONusingjson.dumps().Example messageThe following example is a message asJSON:{"@timestamp":"2018-01-02T01:02:03","@version":"1","extra":{"application":"django_example","django_version":"2.1.0","environment":"production"},"host":"my-local-host","level":"INFO","logsource":"my-local-host","message":"foo bar","pid":65534,"program":"example.py","type":"python-logstash"}This is the standard Logstash message format in JSON.LoggingPyLogBeat uses a logger named "pylogbeat" to log some debug messages
and warnings in case of errors. By default, the logger's log level
is set toWarningso you will not see any debug log messages.
If necessary simply change the log level of the logger to see the debug
messages. For example:importlogginglogging.getLogger('pylogbeat').setLevel(logging.DEBUG)It is important to make this changeafteryou imported
thepylogbeatmodule.Furthermore, PyLogBeatClient's constructor method takes ause_loggingargument which should be a boolean indicating whether the logging
subsystem should be used at all. The argument defaults toFalse,
i.e. if you want any logging, you need to passTrue.
If PyLogBeat is used itself as part of the logging system (e.g.
as the transport of a handler), it is important to not emit any new
log messages once the logging subsystem has been shutdown or is in the
process of shutting down. In this case,use_loggingmust beFalsein order to suppress generating log messages.Protocol SupportThe implemented Beats protocol is not yet officially specified and
documented, unfortunately. Hopefully the Beats developers will
provide a specification in the future.
So far, sending the data and waiting for the ACK from the server is
implemented. But there might some details from the protocol missing
in the implementation.Future MaintenanceIf you are interested in the code, want to improve it and/or
complete the protocol support, please feel free to send PRs.
I would be happy if someone likes to continue developing this library
and would also take full maintainership for future development and
releases.ContributingFound a bug or got a feature request? Please report it athttps://github.com/eht16/pylogbeat/issues.CreditsThis code is based onhttps://github.com/brxie/PyLumberjackand
adopted to support version 2 of the protocol.
Thanks to brxie for the initial code.ChangeLog2.0.1 / 2023-07-30Load certificate chain only if a certificate was specified2.0.0 / 2020-10-04Remove "six" dependencyRequire Python >= 3.61.0.5 / 2020-10-04Setpython_requiresto Python >=2.7 or Python >= 3.6 for
smooth upgrade to upcoming Python3 only.
This way Python2 only users will stay at this release.1.0.4 / 2020-05-06Validate input data and fix documentation about accepted values (#2)1.0.3 / 2020-04-22Add note about required Logstash versions (#1)Improve unit tests1.0.2 / 2018-12-31Add badges to README1.0.1 / 2018-12-31Fix typo in setup.pyUse distribution "trusty" for Travis builds1.0.0 / 2018-12-31Initial releaseLicensePyLogBeat is licensed under the Apache License 2.0.AuthorEnrico Trögerenrico.troeger@uvena.de