benoitc/gunicorn — pull request #672 by alq666 (2013-12-23 22:18:30+00:00)
Title: Add runtime instrumentation via statsd

# What is this pull request?

[statsd](https://github.com/etsy/statsd) is the *de facto* low-overhead instrumentation of many web stacks. I have come across a number of different ways to instrument `gunicorn`, none of which are part of the core and rely on external scripts being run out-of-band to guess the number of requests processed or the amount of cpu spent.

I am hereby proposing that gunicorn optionally submits metrics to a local (or remote) statsD instance with the use of `--statsd=host:port` option. Being that statsD uses UDP as transport, the receiving end being up or down does not affect the performance of the gunicorn application (unlike other stat collection schemes).

Here is an example of such metric being graphed.

![metric_explorer___datadog-2](https://f.cloud.github.com/assets/19755/1803522/e8a24f14-6c1f-11e3-8097-28281a776c30.png)

# How does this work?

The arbiter and the base worker use the `SIGALRM` facility to submit metrics every 5 seconds (the delay is currently hardcoded). All such code should be surrounded by `try: ... except` so as to not affect runtime.

# What is this STAT logging about?

When statsD instrumentation is turned on, each worker also emits an `INFO` log line with the STAT prefix in it, with a copy of the metrics. This is arguably redundant but may be useful in forensics situation. If you feel that it is clogging the logs I am happy to make logging optional or move it down to `DEBUG`.

# Who is behind this?

[I](https://github.com/alq666) use gunicorn at [Datadog](http://www.datadoghq.com) and have found statsD an indispensable aid to measure the performance of high-traffic applications. There is no company-specific bits in the pull request of course.