basho/riak_core — pull request #158 by russelldb (2012-03-23 14:37:55+00:00)
Title: Restructure stats to be managed by riak_core

Jon's original requirement email said the following-

The current stats implementation has a number of problems
- structure wise the modules that implement stats calculation are in
riak_core, and modules that use them are in riak_core_stats, riak_kv_stats and
riak_repl_stats
- riak_kv_stat process has been a bottleneck in the past due to the amount of
garbage it creates and how much data goes through it.
- SNMP/JMX apps are tied to riak_kv_stat implementation and a pain to extend.

New stats subsystem
- when apps register with riak_core, provide a list of the stats that
the app provides (e.g. for RiakKV put FSM time, number of gossip events
number of handoff events etc)
- for each registered stat provide a specification (probably good to use a proplist
so we could add oid's later if we make SNMP generic)
- how to handle it - e.g. a gauge to count the rate of events, a duration for
request times (fsm/vnode etc) (maybe module names for stats mods perhaps
with a shorthand name)
- what it means - human readable description
- what to display (e.g. for get/put FSM times we'd like to display a min/mean/95/99/max
time over the last minute and total).
- riak_core will be responsible for maintaining all state for the statistics
- we'd quite like to move to an erlang process-per-stat, could use the registered
name mechanism if stats were unique.

This PR addresses the base of that. When we're happy with it I'll work on some further features for display/JMX etc. Will raise separate PR for Repl and KV