DataDog/dd-agent — pull request #1015 by wisq (2014-07-03 18:30:39+00:00)
Title: Additional derived Redis metrics

We run a bunch of Redis DBs, and some of them are meant for ephemeral data, over 99.9% of which should have a TTL set. There are several thousand keys that do not, but these are a tiny portion of the database. Should that portion ever increase, we'd like to alert, because it indicates that something is wrong and we're filling our ephemeral store up with non-ephemeral data.

While the current Redis monitoring does let us track the total number of keys and the number of keys with TTLs, Datadog does not (to our knowledge) currently let us alert on a mathematically derived figure, such as the total number of *persistent* keys (i.e. total keys minus keys with TTLs) or the percentage of keys that are persistent / will expire.

This PR adds three new figures:

* `redis.persist` — `redis.keys` minus `redis.expires`
* `redis.persist.percent` — `redis.persist` as a percentage of `redis.keys`
* `redis.expires.percent` — `redis.expires` as a percentage of `redis.keys`

If there's a way to alert on these derived values, we'd love to hear it; similarly, if this PR is accepted and then alerting on derived values is later added, this can of course be reverted. But for now, this is the code we're using, and we're happy to contribute it back if you think it's useful.