rgw: support swift storage policy api

Fixes: #9932

When creating a bucket, set the placement rule according to the
provided X-Storage-Policy http header.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
PG looking-up/checking does not need to hold the osd_lock since it is being protected by pg_map_lock, remove the assertion.

Signed-off-by: Guang Yang <yguang@yahoo-inc.com>
Move the sched_scrub to a new tick timer which does not need to hold the osd_lock.

Signed-off-by: Guang Yang <yguang@yahoo-inc.com>
WBThrottle: Call posix_fadvise to free page cache if nocache set in clear().

Because FileStore will sync fs and call clear() to clear all pending
ghobjct. So in clear(), the pending object can free the page cache if
nocache is set.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
os: add filestore_fadvise to control whether use posix_fadvise.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
librados: drop useless ver arg to _notify_ack

Signed-off-by: Sage Weil <sage@redhat.com>
librados: define updated watch/notify interface

- new notify callback with the correct values:
  - notify_id
  - watch handle
  - payload
- new notify_ack call
  - not implicit when the callback returns (for new api only)
  - optional payload
- new watch2 call
  - that provides the new callback
- new notify2 call
  - with the right arguments, and optional timeout

A couple refactors in here:
- IoCtx notify_ack is now called unlocked (Note: this will soon change
  with pending Objecter locking changes)
- Objecter notify_ack takes a buffer

TODO:
- no timeout on the individual watch, yet...

Signed-off-by: Sage Weil <sage@redhat.com>
osd: implement notify ack payloads

If the notified send back reply payloads, pass them back to the notifier.

Note that we have changed the on-wire behavior of the watch completion
message a bit: instead of sending the original notify payload back to the
notifier, we send the map of notified to replies.  Note that only users of
the new API will know what to do with the notify acknowledgement
information.  At the same time, we stop sending the original payload.
However, the old API users never saw that data; we were uselessly sending
it over the wire.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: update notify2 API to accept reply payloads

Allow the notify2 callers to provide bufferlists or char ** pointers
so that we can pass the reply buffer back to them.

Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_rados_api_watch_notify: test notify2 api

Signed-off-by: Sage Weil <sage@redhat.com>
librados: include notifier's gid in notify

The notify replies include the identify of everyone who acked the
notify.  For symmetry, inform the notified client who sent the notify.

Fixes: #9198
Signed-off-by: Sage Weil <sage@redhat.com>
osd: allow multiple watchers/notify replies per client

It is perfectly valid to register multiple watchers via the same
librados client.  Make the notify_replies a multimap to capture all
of the acks.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: unconditionally pass WATCH_NOTIFY to legacy callback

This was always WATCH_NOTIFY; just pass that explicitly.  No need to
pay attention to what's in the message.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: define CEPH_OSD_WATCH_OP_*

Signed-off-by: Sage Weil <sage@redhat.com>
librados: use new watch op codes; simplify Objecter helpers

- drop the useless add_watch() helper; do it explicitly
- drop the unused var arg everywhere
- make a separate notify member of the union that excludes
  the other unused fields

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: fix assert_src_version

Don't use the add_watch() helper; do it like assert_version
does it.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: define RECONNECT and PING watch ops

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: handle PING and RECONNECT watch ops

The ping will essentially assert that a watch is still valid.

A reconnect will reestablish session state *only* if the watch is
still persistent.  If not, it will fail (and the client will know it
may have missed something).

Note that the only difference here is that a PING is a bit lighter
weight; it will not reestablish the session state (which should already
be established).  We could use a single op here but the unique op
code makes the messages easier to understand and simplifies the code
path a bit for PING.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: remove crufty declaration

Signed-off-by: Sage Weil <sage@redhat.com>
librados: define error callback

This is the async notification of a watch error (failure to reconnect to
the object's OSD after a peering event or network disconnect).

Signed-off-by: Sage Weil <sage@redhat.com>