tangods-graylogger 1.0.0
A tango device server to catch logs on the old log4tango API and send them to Graylogger using GELF and HTTP.

# The Graylogger Tango Device Server

A Tango device server to catch log messages generated by other device servers,
using the traditional log4tango implementation, and redirect them to Graylog
using GELF HTTP.

## Repository content

* Graylogger tangoDS: a minimalistic tango device that can be used as logging
target. This tangoDS outputs to Graylog HTTP, using graylog GELF format.
To test, set the `logging_target` property of the log producer to
log to `device::my/gray/logger`, where `my/gray/logger` is the tango name
of your Graylogger tangoDS.  
In the Graylogger device, you must set the hostname and port properties
to point to your Graylog server.  
In graylog you must set up an input that processes GELF HTTP requests.
The default port is `12201`.

* Producer tangoDS: a software tango device that has commands for
producing log messages using the various `*_stream` methods
(`self.debug_stream()`, `self.error_stream()` etc...).  
It also has a wrapper for exceptions that catches a generic exception, alters
state and status, sends to `self.error_stream()` and re-raises them.

* A docker compose file to spin a graylog container. To start it, start
systemd `docker.service` then navigate to the `docker` directory and run
`docker-compose -f docker-compose.yml`. Stop with `<Ctlr-C>`. The messages
and settings are stored on disk.  
To reverto to a blank installation run `docker-compose down -v` when not
running. To log in, go to `localhost:9000` and login as `admin` password
`password12345`.

* A docker configuration file: `config/docker.conf`. This is mostly to tune
the container performance.

## Observations and annotations

Graylog was chosen because it is very convenient for log analysis. The GELF
protocol was chosen for its simplicity (only a few items are mandatory) and
because it is possible to extend it with other tags, the user defined tags
simply start with undescore. Http was chosen because it's simple to implement
and because the server replies to post request, making it possible to
understand if the log was sent succesfully. It is also possible to switch to
https for enhanced security, even if that is not implemented yet.

A good place for tango logging documentation is the
[RFC](https://tango-controls.readthedocs.io/projects/rfc/en/latest/14/Logging.html).

Tango logging has room for improvement. The amount of information and format is
limited and not all features are implemented. Exceptions are not added to the
logs automatically, not even if changing the `logging_level`: changes to the
code are necessary. The easiest way to do so in python is to use wrappers.
See the example in `Producer`.

The methods that are named `debug_stream()` and similar are not really streams,
they are *loggers*.

If you start a tango device with `-v3` or `-v5` the logging behavior changes
completely: `logging_target` and `logging_level` are overridden. -v5 adds extra
corba logs to the stream. The corba logs are not documented (AFAIK).

It is not entirely possible to set the logging target in jive, as multi stream
requires a multi line property that is not possible to input in jive. One way
to do so, is by using pytango `Database()` class.

`itango` example:
`Database().put_device_property('my/log/producer', {"logging_target":
['console', 'file::/home/me/test.log', 'device::my/gray/logger']})`.

This adds 3 logging targets, one to console, one to file and one to another
tango device.

If the logging target tangoDS is restarterd, the whole logging will stop.
It is not possible to initialize the devices in order, they have to be
restarted, because the logging is set in a stage before the init itself.
The tango device which act as a log target needs to be started first, then all
the other devices can start. There is no way around this and this is an
important limitation. It could be worth to look into file based approaches.

To work around these limitations, the `Graylogger` tango device server has
commands that allow it to reset the logging mechanism on the other devices.

Sardana seems to do things differently, it does not seem to respond to
`set_logging_target()` and `set_logging_level()`. It uses standard python
logging. I was unable to use a device or console target.