documentation: revise placement group number guide

When a cluster has few OSDs (less than 50) propose a preselection of
values: as long as the number of placement groups is not too small nor
too large, it won't make much of a difference anyway.

Users of small clusters tend to blindly apply the (OSD*100)/(pool size)
formula and worry about chosing a wrong value because they do not
understand the tradeoffs. The preselection will hopefully save them from
this uncertainty.

Add an explanation of how placement groups relate to OSDs, CRUSH and
pools to help understand the tradeoffs. Explain the
tradeoffs (durability, distribution and resource usages) with examples.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Reviewed-by: Gerben Meijer <infernix@gmail.com>
Reviewed-by: Laurent Guerby <laurent@guerby.net>
Merge pull request #2557 from ceph/wip-mon-fix-checks

ceph-mon: check fs stats just before preforking

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2417 from dachary/wip-placement-group

doc: revise placement group number guide.

Reviewed-by: John Wilkins <jowilkin@redhat.com>
Merge pull request #2414 from dachary/wip-cephtool-test

tests: make qa/workunits/cephtool/test.sh more robust

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Merge pull request #2561 from athanatos/wip-9293

Wip 9293

Reviewed-by: Sage Weil <sage@redhat.com>
PG: check full ratio again post-reservation

Otherwise, we might queue 30 pgs for backfill at 0.80 fullness
and then never check again filling the osd after pg 11.

Fixes: #9574
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
rgw: PutObjMetadata, clear bufferlist before appending into it

Fixes: #9576
Backport: firefly, dumpling

We need to completely rewrite the bufferlist, not append into it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
rgw: rados->set_attrs() updates bucket index

Fixes: #5595
Backport: dumpling, firefly
We need to update the bucket index when updating object attrs, otherwise
we're missing meta changes that need to be registered. It also
solves issue of bucket index not knowing about object acl changes,
although this one still requires some more work.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
osd: initialize purged_snap on backfill start; restart backfill if change

If we backfill a PG to a new OSD, we currently neglect to initialize
purged_snaps.  As a result, the first time the snaptrimmer runs it has to
churn through every deleted snap for all time, and to make matters worse
does so in one go with the PG lock held.  This leads to badness on any
cluster with a significant number of removed snaps that experiences
backfill.

Resolve this by initializing purged_snaps when we finish backfill.  The
backfill itself will clear out any stray snaps and ensure the object set
is in sync with purged_snaps.  Note that purged_snaps on the primary
that is driving backfill will not change during this period as the
snaptrimmer is not scheduled unless the PG is clean (which it won't be
during backfill).

If we by chance to interrupt backfill, go clean with other OSDs,
purge snaps, and then let this OSD rejoin, we will either restart
backfill (non-contiguous log) or the log will include the result of
the snap trim (the events that remove the trimmed snap).

Fixes: #9487
Backfill: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
ReplicatedPG: don't move on to the next snap immediately

If we have a bunch of trimmed snaps for which we have no
objects, we'll spin for a long time.  Instead, requeue.

Fixes: #9487
Backport: dumpling, firefly, giant
Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge remote-tracking branch 'origin/giant'
Merge pull request #2506 from dachary/wip-9304-unintended-implicit-ruleset

erasure-code: pool create must not always create a ruleset

Reviewed-by: João Eduardo Luís <joao@redhat.com>
tests: use memcmp to compare binary buffers

instead of strncmp because it will stop at the first \0

http://tracker.ceph.com/issues/6697 Fixes: #6697

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2567 from dachary/wip-6697-strncmp-vs-memcmp

tests: use memcmp to compare binary buffers

Reviewed-by: Sage Weil <sage@redhat.com>
Merge remote-tracking branch 'origin/giant'
ReplicatedPG: clean out completed trimmed objects as we go

Also, explicitely maintain a max number of concurrently trimming
objects.

Fixes: 9113
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
For pgls OP, get/put budget on per list session basis, instead of per OP basis, which could lead to deadlock.

Signed-off-by: Guang Yang (yguang@yahoo-inc.com)
osd: Remove unused PG functions queue_notify(), queue_info(), queue_log()

Signed-off-by: David Zafman <dzafman@redhat.com>
mailmap: Thorsten Glaser affiliation

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Thorsten Glaser <tg@mirbsd.de>
mailmap: Yan Zheng affiliation

Also adding Yan Zheng to .peoplemap to track org. change

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>