basho/riak_pipe — pull request #81 by uwiger (2013-12-05 15:09:22+00:00)
Title: Integrate the Exometer metrics package

This PR is part of a set of PRs aimed at integrating the [Exometer](https://github.com/Feuerlabs/exometer) metrics package into Riak.

(From basho/riak_core#465)

From our measurements so far, Exometer offers both better throughput and lower footprint than the previous metrics management, and at the same time offers more flexible and uniform handling and better extensibility.

In addition to maintaining the console command `riak-admin status` and the HTTP JSON report (which aim to be backwards-compatible), a new console command, `riak-admin stat ` has been added, for selective reporting of statistics as well as some management (ability to enable/disable metrics on the fly).

Other noteworthy changes:

* Exometer entry names are always lists. In riak, only atoms and numbers should be used as list elements.
* A top-level 'prefix' (default: `riak`) has been added, in order to differentiate between stats from different riak-style products (e.g. when reporting stats to collectd). The function `riak_core_stat:prefix()` is generated as a constant expression through the parse transform `riak_core_stat_xform`, which in its turn checks the OS env variable `RIAK_CORE_STAT_PREFIX`. A typical entry would thus be e.g. `[riak,riak_kv,node,gets,siblings]`.
* Internally in riak, stats are referred to symbolically using the same (tuple-based) names as before. These often refer to more than one low-level metric, so it seemed reasonable to keep this naming scheme.
* Exometer provides similar functionality as 'sidejob' and the riak_core stat cache, so these are no longer used for stats management (although sidejob still maintains some stats on its own, which are accessible via Exometer). Other apps still register with `riak_core_stat`, but need not provide callbacks in other to query the stats. The query style of exometer is the same as that of riak_core_stat.