PG: always clear agent_state in clear_primary_state

We aren't actually active between activate() and all_activated_committed().
We'd have to suspend agent_work during that period which seems like too much
complexity for too little work saved.

Fixes: #7904
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: hit_set_setup, agent_setup, skip if !active

We call these in on_activate and on_pool_change.  In the former, we are
necessarily active.  In the latter, we only want to do anything if we are
active (otherwise, it will be taken care of when we eventually do become
active).

Fixes: #7904
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: improve get_object_context debugging

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd: add 'osd debug reject backfill probability' option

This will make the OSD randomly reject backfill reservation requests.  This
exercises the failure code paths but does not break overall behavior
because the primary will back off and retry later.

This should help us reproduce #7922.

Signed-off-by: Sage Weil <sage@inktank.com>
ReplicatedPG: handle FLAG_OMAP on promote and copyfrom

Fixes: #7967
Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: track num_objects_omap in pg stats

Fixes: #7831
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::agent_choose_mode: exclude omap objects for ec base pool

Fixes: #7831
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::agent_choose_mode: use num_user_objects for target_max_bytes calc

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::agent_choose_mode: improve debugging

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::_delete_oid: adjust num_object_clones

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd: Send REJECT to all previously acquired reservations

When getting a REJECT from a backfill target, tell already GRANTed targets to
go back to RepNotRecovering state by sending a REJECT to them.

Fixes: #7922

Signed-off-by: David Zafman <david.zafman@inktank.com>
Merge pull request #1600 from ceph/wip-7922

Wip 7922

Passes my manual testing and the new teuthology test case.

Reviewed-by: Sage Weil <sage@inktank.com>
ReplicatedPG: fix CEPH_OSD_OP_CREATE on cache pools

The following

./ceph osd pool create data-cache 8 8
./ceph osd tier add data data-cache
./ceph osd tier cache-mode data-cache writeback
./ceph osd tier set-overlay data data-cache

./rados -p data create foo
./rados -p data stat foo

results in

  error stat-ing data/foo: No such file or directory

even though foo exists in the data-cache pool, as it should.  STAT
checks for (exists && !is_whiteout()), but the whiteout flag isn't
cleared on CREATE as it is on WRITE and WRITEFULL.  The problem is
that, for newly created 0-sized cache pool objects, CREATE handler in
do_osd_ops() doesn't get a chance to queue OP_TOUCH, and so the logic
in prepare_transaction() considers CREATE to be a read and therefore
doesn't clear whiteout.  Fix it by allowing CREATE handler to queue
OP_TOUCH at all times, mimicking WRITE and WRITEFULL behaviour.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
Merge pull request #1602 from ceph/wip-cache-create-fix

ReplicatedPG: fix CEPH_OSD_OP_CREATE on cache pools

Reviewed-by: Samuel Just <sam.just@inktank.com>
mon: MonCommands.h: have 'auth' read-only operations require 'x' cap

This reintroduces the same semantics that were in place in dumpling prior
to the refactoring of the cap/command matching code.

We haven't added this requirement to auth read-write operations as that
would have the potential to break a lot of well-configured keyrings once
the users upgraded, without any significant gain -- we assume that if
they have set 'rw' caps on a given entity, they are indeed expecting said
entity to be sort-of-privileged entities with regard to monitor access.

Fixes: #7919

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
doc/release-notes: note about emperor backport of mon auth fix

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: do not hit_set_persist while potentially backfilling hit_set_*

The hit_set transactions may include both a modify of the new hit_set and
deletion of an old one, spanning the backfill boundary, and we may end up
sending a backfill target a blank transaction that does not correctly
remove the old object.  Later it will notice the stray object and
throw an assertion.

Fix this by skipping hit_set_persist() if any of the backfill targets are
still working on the very first hash value in the PG (which is where all
of the hit_set objects live).  This is coarse but simple.

Another solution would be to send separate ops for the trim/deletion and
new hit_set update, but that is a bit more complex and a bit more
runtime overhead (twice the messages).

Fixes: #7983
Signed-off-by: Sage Weil <sage@inktank.com>
ceph-post-file: fix installation of ssh key files

Fixes: #7992
Signed-off-by: Sage Weil <sage@inktank.com>
Merge pull request #1604 from ceph/wip-7992

ceph-post-file: fix installation of ssh key files
Merge pull request #1603 from ceph/wip-7983

osd/ReplicatedPG: do not hit_set_persist while potentially backfilling hit_set_*

Reviewed-by: Samuel Just <sam.just@inktank.com>