mon: OSDMonitor: limit tier-specific pool set/get on non-tier pools

Fixes: 8696

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
osd: pg_pool_t: clear tunables on clear_tier()

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: OSDMonitor: be scary about inconsistent pool tier ids

We may not crash your cluster, but you'll know that this is not something
that should have happened.  Big letters makes it obvious.  We'd make them
red too if we bothered to look for the ANSI code.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits: cephtool: split get/set on tier pools from get/set tests

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits: cephtool: test get/set on both tier and non-tier pools

Make sure gets and sets of tiering-specific variables succeed on tier
pools and fail on non-tier pools.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits: cephtool: test erroneous 'tier remove'

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits: cephtool: pool needs to be a tier to be used as such

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits: cephtool: cleanup after pool creation

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits: cephtool: adjust pool name where missing as it has changed

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Merge pull request #2086 from ceph/wip-8696

mon: OSDMonitor: limit tier-specific pool set/get on non-tier pools

Reviewed-by: Sage Weil <sage@redhat.com>