Package: ondewo-logging
ONDEWO LoggingThis is the logging package for ONDEWO products. It allows for easy integration with our EFK stack, and adds some useful features to the base python logging package (such as timing and exception handling), and handles GRPC error messages nicely.UseageTo use this library, first pip install it:pip install ondewo-loggingthen import it into your project like so:from ondewo.logging.logger import logger_consoleNoteIn order for logger to logmodule_name,docker_image_name, andgit_repo_nameone has to pass them as environment variables to the container where the logged service is running.DecoratorsA couple of decorators are included:from ondewo.logging.decorators import Timer, timing, exception_handling, exception_silencingThe Timer class can be used as a context manager:with Timer() as t:
  sleep(1)or as a decorator:@Timer()
def sleeptime():
  sleep(1)and can be used with different messages or logging levels:Logging level:@Timer(logger=logger_console.info)Message:@Timer(message="MESSAGE WITH TIME {} {}"),@Timer(message="SIMPLER MESSAGE WITHOUT TIME")Disable argument logging:@Timer(log_arguments=False)Enable exception suppression:@Timer(supress_exceptions=True)See the tests for detailed examples of how these work.Timing is just an instance of the Timer class:timing = Timer()for backwards compatibility.The exception_handling function is a decorator which will log errors nicely using the ondewo logging syntax (below). It will also log the inputs and outputs of the function. The exception_silencing function just shows the inputs and outputs and gets rid of the stacktrace, it can be useful for debugging. Finally, log_arguments will dump the inputs and outputs of a function into the logs.Ondewo log formatThe structure of the logs looks like this:message: Dict[str, Any] = {
  "message": f"Here is the normal log, including relevant information such the magic number: {magic number}. These values are also added seperately below, either just with the variable name or some other relevant name. Finally, there are some tags to help with searching through the logs.",
  "magic_number": magic_number,
  "tags": ["magic", "number"]
}Note on tags:The tags allow for easy searching and grouping in kibana. They can be added in a somewhat ad-hoc manner by the programmer on the ground, though some (like 'timing') are standardised. Please talk to your project team lead for details.FluentdQuickstartgit clonehttps://github.com/ondewo/ondewo-logging-pythonmakeedit the fluentd config with the url and password of your elasticsearch host:sed -i 's/<PASSWORD>/my_password/' './fluentd/conf/fluent.conf'
sed -i 's/<HOST>/my_elasticsearch_host/' './fluentd/conf/fluent.conf'run fluentddocker-compose -f fluentd/docker-compose.yaml up -dYou now have a fluentd message handler running on your machine. If you use the ondewo.logging library, your logs will be shipped to your elasticsearch server.Fluentd ConfigPer thefluentd/docker-compose.yaml, we map the configuration files and the logs into the fluentd image and open some ports.We also need tochown -R 100:"$GID" fluentd/log. That command should allow both you and fluentd to read the logs.Beyond that, it is just a question of formatting the logs wherever they come from. Here is the example from the fluentd config that sends stuff to the fluentd stdout, so you can see the logs from all your images in the same place.<source>
  @type forward
  port 24224
</source>

# py.console logging gets piped to stdout
<match py.console.**>
  @type stdout
  <format>
      @type ltsv
      delimiter_pattern :
      label_delimiter =
  </format>
</match>In this conf, we recieve imput over a tcp connection, then dumps the output to stdout, so you can use that stream to watch log output via fluentd. The config is also set up to save all the logs locally, and ship them to a remote server.Automatic Release ProcessThe entire process is automated to make development easier. The actual steps are simple:TODOs in Pull Request before the release:Update the Version number inside the Makefile! : Major and Minor Version Number must be the same for Client and API at all timesexample: API 2.9.0 --> Client 2.9.XCheck if RELEASE.md is up-to-dateUpdate the Version number inside the setup.py by using:makeupdate_setupTODOs after Pull Request was merged in:Checkout master:gitcheckoutmasterPull the new stuff:gitpullRelease:makeondewo_releaseThemake ondewo_releasecommand can be divided into 5 steps:cloning the devops-accounts repository and extracting the credentialscreating and pushing the release branchcreating and pushing the release tagcreating the GitHub releasecreating and pushing the new PyPi releaseThe variables for the GitHub Access Token, PyPi Username and Password are all inside
of the Makefile, but the values are overwritten duringmake ondewo_release, because
they are passed from the devops-accounts repo as arguments to the actualreleasecommand.