histogram: move to common, add unit tests

Signed-off-by: Sage Weil <sage@inktank.com>
histogram: add decay

Signed-off-by: Sage Weil <sage@inktank.com>
osd/osd_types: add cache policy fields to pg_pool_t

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: allow new pool policy fields to be set

Signed-off-by: Sage Weil <sage@inktank.com>
osd: add pg_pool_t::get_pg_num_divisor

A PG is not always an equally sized fraction of the total pool size due to
the use of ceph_stable_mod.  Add a helper to return the fraction
(denominator) of a given pg based on the current pg_num value.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: rename test/test_osd_types.cc -> test/osd/types.cc

Signed-off-by: Sage Weil <sage@inktank.com>
mon: include dirty stats in 'ceph df detail'

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: add on_finish to OpContext

Add a callback hook for whenever an OpContext completes or cancels.  We
are pretty sloppy here about the return values because our initial user
will not care, and it is unclear if future users will.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: factor clone check out of evict op code

Move the check for clones into a helper so that we will be able to use in
other places where we need to evict.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/HitSet: add HitSetRef

Signed-off-by: Sage Weil <sage@inktank.com>
osd: agent worker thread

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: basic flush and evict agent functionality

This is very basic flush and evict functionality for the tiering agent.

The flush policy is very simple: if we are above the threshold and the
object is dirty, and not super young, flush it.  This is not too braindead
of a policy (although we could clearly do something smarter).

The evict policy is pretty simple: evict the object if it is clean and
we are over our full threshold.  If we are in the middle mode, try to
estimate how cold the object is based on an accumulated histogram of
objects we have examined so far, and decide to evict based on our
position in that histogram relative to our "effort" level.

Caveats:
 * the histograms are not refreshed
 * we aren't taking temperature into consideration yet, although some of
   the infrastructure is there.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: decay tier agent histograms over time

Make decisions based on recent observations of object age distributions,
not all time history.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: test dirty stats in ceph df detail

Signed-off-by: Loic Dachary <loic@dachary.org>
histogram: fix histogram::get_position_micro overflow

Convert the return values to uint64_t

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: add 'notieragent' flag to OSDMap

This will pause tiering agent work.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: observe 'notieragent' osdmap flag

Pause/unpause the agent thread accordingly.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: only enable tier agent when osd is in active state

Signed-off-by: Sage Weil <sage@inktank.com>
osd: add pg_pool_t::get_random_pg_position()

Return a hash position somewhere inside a given pg.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: initial agent to random hash position inside pg

When the agent starts, start at a random offset to ensure we get a more
uniform distribution of attention to all objects in the PG.  Otherwise, we
will disproportionately examine objects at the "beginning" of the PG if we
are interrupted by peering or restarts or some other activity.

Note that if the agent_state is preserved, we do not forget our position,
which is also nice.

We *could* persist this position in the pg_info_t somewhere, but I am not
sure it is worth the effort.

Signed-off-by: Sage Weil <sage@inktank.com>