Bug fix in the way crush_type is extracted from osd crush dump

Signed-off-by: Babu Shanmugam <anbu@enovance.com>
While generating crush_types, avoiding item parsing, and calculating type count by just iterating thorugh buckets list

Signed-off-by: Babu Shanmugam <anbu@enovance.com>
Fetching the date from ceph osd dump as that is more reliable across ceph versions

Signed-off-by: Babu Shanmugam <anbu@enovance.com>
Included the total cluster size in components_count object

Signed-off-by: Babu Shanmugam <anbu@enovance.com>
Merge branch 'master' of https://github.com/enovance/ceph-brag into firefly
osd_types: pg_t: add get_ancestor() method

Give us the ancestor for when the pool had a past value for pg_num.

Signed-off-by: Sage Weil <sage@inktank.com>
ReplicatedPG::mark_all_unfound_lost: delete local copy if necessary

There might be a local copy for an EC pool in the DELETE case.  The replica
copies should be already handled by merge_log.

Fixes: #7439
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::hit_set_persist: clean up degraded check

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd: track the number of hit_set archive objects in a pg

Also, use this value in agent_choose_mode instead of the max
number.

Related: #8124
Signed-off-by: Samuel Just <sam.just@inktank.com>
ECMsgTypes::ECSubWrite: fix at_version indentation

Signed-off-by: Samuel Just <sam.just@inktank.com>
encoding: use unqualified name for encode/decode in boost::optional encoding

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: propogate hit_set history with repop

We don't actually send the whole info on each repop, just the log
entries, updated stats, and a few other bits.  For hit_set ops, we need
to also communicate the new hit_set history status atomically with the
log entries and the transaction.  Thus, we add a channel for an optional
pg_hit_set_history_t field in PGBackend::submit_transaction interface
and associated messages and implementations to update the hit_set info
field along with the log entries.

This also means that hit_set_(persist|trim) update an
updated_hit_set_history field on the OpContext instead of directly
modifying the info field.

Fixes: #8124
Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,PGLog: update hit_set during peering

Fixes: #8124
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::agent_load_hit_sets: take ondisk_read_lock

Otherwise, the hit_set might be not yet written due to a recently
completed recovery.

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: do not use shard for hit_set object names

Signed-off-by: Samuel Just <sam.just@inktank.com>
qa/workunits/cephtool/test.sh: make set pg_num test non-racy

Loop while the pool is creating.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Merge pull request #1705 from ceph/wip-8124

Wip 8124

Reviewed-by: Sage Weil <sage@inktank.com>
ReplicatedPG,rados: add CEPH_OSD_[COPY_FROM]_MAP_SNAP_TO_CLONE

When promoting a clone, we want to use the provided snapid to specify
specify the clone id directly.

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: do not create whiteout clones

First, make_writeable treats whiteout heads like snapdir for
cloning purposes.  Second, to ensure that we send the correct
deletes on flush to the backing pool, we instead use oi.snaps
on any clone we are flushing to infer the snaps during which
head did not exist and send a delete as appropriate prior to
the copy_from.

Normally, we'd have a problem if the delete and the copy_from
completed, but an interval change intervened before the dirty
flag was cleared since we'd end up re-deleting the object.
To avoid that, we use the CEPH_OSD_FLAG_ORDERSNAP flag.

Additionally, we will use the correct snap_seq on the delete
or flush as appropriate to ensure that the previous clone
gets created with the same clone id as in the cache pool.

Fixes: #7942
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: if we get ENOENT on clone, remove clone from snapset

Fixes: #7916
Signed-off-by: Samuel Just <sam.just@inktank.com>