osd: introduce simple sleep during scrub

This option is similar to osd_snap_trim_sleep: simply inject an optional
sleep in the thread that is doing scrub work.  This is a very kludgey and
coarse knob for limiting the impact of scrub on the cluster, but can help
until we have a more robust and elegant solution.

Only sleep if we are in the NEW_CHUNK state to avoid delaying processing of
an in-progress chunk.  In this state nothing is blocked on anything.
Conveniently, chunky_scrub() requeues itself for each new chunk.

Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c4e8451cc5b4ec5ed07e09c08fb13221e31a7ac6)
common/io_priority: wrap ioprio_set() and gettid()

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 705713564bebd84ad31cc91698311cf2fbd51a48)

Conflicts:
	src/common/Makefile.am
common/Thread: allow io priority to be set for a Thread

Ideally, set this before starting the thread.  If you set it after, we
could potentially race with create() itself.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 01533183e7455b713640e001962339907fb6f980)
common/WorkQueue: allow io priority to be set for wq

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 5e4b3b1f1cb870f39fc7cfb3adeae93e078d9057)

Conflicts:
	src/common/WorkQueue.cc
Merge pull request #2014 from ceph/wip-scrub-dumpling

osd: scrub priority updates for dumpling

Reviewed-by: Loic Dachary <loic@dachary.org>