histogram: calculate bin position of a value in the histrogram

Generate a lower and upper bound.

Signed-off-by: Sage Weil <sage@inktank.com>
histogram: rename set -> set_bin

Signed-off-by: Sage Weil <sage@inktank.com>
histogram: move to common, add unit tests

Signed-off-by: Sage Weil <sage@inktank.com>
histogram: add decay

Signed-off-by: Sage Weil <sage@inktank.com>
osd/osd_types: add cache policy fields to pg_pool_t

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: allow new pool policy fields to be set

Signed-off-by: Sage Weil <sage@inktank.com>
osd: add pg_pool_t::get_pg_num_divisor

A PG is not always an equally sized fraction of the total pool size due to
the use of ceph_stable_mod.  Add a helper to return the fraction
(denominator) of a given pg based on the current pg_num value.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: rename test/test_osd_types.cc -> test/osd/types.cc

Signed-off-by: Sage Weil <sage@inktank.com>
mon: include dirty stats in 'ceph df detail'

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: add on_finish to OpContext

Add a callback hook for whenever an OpContext completes or cancels.  We
are pretty sloppy here about the return values because our initial user
will not care, and it is unclear if future users will.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: factor clone check out of evict op code

Move the check for clones into a helper so that we will be able to use in
other places where we need to evict.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/HitSet: add HitSetRef

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: fix finish_flush

Make sure we reallocate a pgbackend transaction at the time when we are
initiating new work.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: agent worker thread

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: basic flush and evict agent functionality

This is very basic flush and evict functionality for the tiering agent.

The flush policy is very simple: if we are above the threshold and the
object is dirty, and not super young, flush it.  This is not too braindead
of a policy (although we could clearly do something smarter).

The evict policy is pretty simple: evict the object if it is clean and
we are over our full threshold.  If we are in the middle mode, try to
estimate how cold the object is based on an accumulated histogram of
objects we have examined so far, and decide to evict based on our
position in that histogram relative to our "effort" level.

Caveats:
 * the histograms are not refreshed
 * we aren't taking temperature into consideration yet, although some of
   the infrastructure is there.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: decay tier agent histograms over time

Make decisions based on recent observations of object age distributions,
not all time history.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: test dirty stats in ceph df detail

Signed-off-by: Loic Dachary <loic@dachary.org>
histogram: fix histogram::get_position_micro overflow

Convert the return values to uint64_t

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: add 'notieragent' flag to OSDMap

This will pause tiering agent work.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: observe 'notieragent' osdmap flag

Pause/unpause the agent thread accordingly.

Signed-off-by: Sage Weil <sage@inktank.com>