Objecter: rename Op::oloc -> Op::base_oloc

We want to be able to target other pools for caching and tiering, so
we need to take an oloc from the client and translate it into an
actual target. Rename oloc to base_oloc to make clear which one it is.

Signed-off-by: Greg Farnum <greg@inktank.com>
Objecter: add an Op::target_oloc and use it instead of base_oloc in send_op()

For now we simply set target_oloc = base_oloc in recalc_op_target(), but
we will shortly be doing more interesting things with it there.

Signed-off-by: Greg Farnum <greg@inktank.com>
Objecter: be careful about precalculated pgids

The only current user of the precalc_pgid field is list_objects. That's
fine, but we don't want new users to inadvertently appear and somehow
break the caching/tiering stuff by forcing us to go to the base pool
when we should be talking to somebody else. Add an assert to catch
these cases.

Signed-off-by: Greg Farnum <greg@inktank.com>
Objecter: respect read_tier & write_tier for initial op submission

We overwrite target_oloc.pool with the appropriate [read|write]_tier.
write_tier wins if it matches both.
We don't handle any sort of redirect yet.

Signed-off-by: Greg Farnum <greg@inktank.com>
osd_types: note that write_tier wins if read_tier is different

For pg_pool_t.

Signed-off-by: Greg Farnum <greg@inktank.com>
workunits: add a test for caching redirects

This may need to change since it exploits some of the loose
consistency we currently have with caching pools, but for now
it checks that the Objecter does what we want.

Signed-off-by: Greg Farnum <greg@inktank.com>
Merge pull request #560 from ceph/wip-6032-cache-objecter

Wip 6032 cache objecter

Reviewed-by: Sage Weil <sage@inktank.com>