osd: OSDMonitor: ignore pgtemps from removed pool

There's a window in-between receiving an MOSDPGTemp message from an OSD
and actually handling it that may lead to the pool the pg temps refer to
no longer existing. This may happen if the MOSDPGTemp message is queued
pending dispatching due to an on-going proposal (maybe even the pool
removal).

This patch fixes such behavior in two steps:

1. Check if the pool exists in the osdmap upon preprocessing
 - if pool does not exist in the osdmap, then the pool must have been
   removed prior to handling the message, but after the osd sent it.
 - safe to ignore the pg update
2. If all pg updates in the message have been ignored, ignore the whole
   message.  Otherwise, let prepare handle the rest.

3. Recheck if pool exists in the osdmap upon prepare
 - We may have ignored this pg back in preprocess, but other pgs in the
   message may have led the message to be passed on to prepare; ignore
   pg update once more.
4. Check if pool is pending removal and ignore pg update if so.

We delegate checking the pending value to prepare_pgtemp() because in this
case we should only ignore the update IFF the pending value is in fact
committed.  Otherwise we should retry the message.  prepare_pgtemp() is
the appropriate place to do so.

Fixes: 7116

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Merge pull request #1153 from ceph/wip-7116-joao

osd: OSDMonitor: ignore pgtemps from removed pool

Reviewed-by: Sage Weil <sage@inktank.com>
rgw: fix multipart upload listing

Fixes: #7169

A separate fix has been created for dumpling.

Previously we read the entire list of parts, disregarding the actual
marker and the requested max parts. This fix refactors the way we read
the list of parts (doing it in parts, using marker).

Create new upload-id format that is used to identify uploads with sorted
omap entries. Make sure we're backward compatible and handle correctly
mixed-versions rgw uploads.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: fix multipart min part size

As part of the fix for wip-7169 it turned out that we removed
min_part_size. Looking back, the original implementation was broken
anyway and didn't do anything. This fixes it and makes it configurable.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
OSD: don't assume we have the pool in handle_pg_create

The pool may have been removed since the creation message
was sent.  Previously, role would end up as -1 and this
path would be avoided.

Fixes: 7190
Introduced in 268ae82ac3fe7b541c450aae168dcf9fa7294508
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #1154 from ceph/wip-7169-2

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Merge pull request #1155 from ceph/wip-7190

OSD: don't assume we have the pool in handle_pg_create
Reviewed-by: Greg Farnum <greg@inktank.com>
OSDMap: fix deepish_copy_from

Start with a shallow copy!

Signed-off-by: Sage Weil <sage@inktank.com>
cls_user: init cls_user_stats fields in ctor

Signed-off-by: Sage Weil <sage@inktank.com>
cls_user: init cls_user_list_buckets_op fields

Signed-off-by: Sage Weil <sage@inktank.com>
civetweb: fix warning

Signed-off-by: Sage Weil <sage@inktank.com>
osd: disable omap if the pool is not replicated

If one of the operations in MOSDOp is CEPH_OSD_OP_OMAP* and the pool is
not replicated, return EINVAL.

http://tracker.ceph.com/issues/6895 fixes #6895

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: add osd pool set no_omap

so omap support can be set on a per pool basis (unless it is erasure
coded in which case it is always disabled).

Signed-off-by: Loic Dachary <loic@dachary.org>