librados: avoid memcopy for rados_getxattr.

When call rados_getxattr, pass user buff to libardos to avoid memcopy.
This operation like rados_read.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
librados: Avoid copy data from librados to caller buff
when using rados_read_op_read.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
osdc/Objecter: do notify completion callback in fast-dispatch context

The notify completion has exactly one user, the librados caller which
does nothing but take a local (inner) lock and signal a Cond.  Do this
in the fast-dispatch context for simplicity.

Notably, this makes the notify completion (and timeout) trigger a
notify2() return (with ETIMEDOUT) even when the finisher queue that
normally delivers notify is busy.. for example with a notify that is
being very slow.  In our case, the unit test is doing a sleep(3) to
test timeouts but also prevented the ETIMEDOUT notification from
being delivered to the caller.  This patch resolves that.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: add rados_watch_flush() call

Add a call so that callers can make sure all queued callbacks have
completed before shutting down the ioctx.  This avoids a segv triggered
by the LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1
test due to the ioctx being destroyed when the in-progress callback
does a notify_ack.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: watch_flush() on shutdown

Users can easily forget this. It makes shutdown potentially block, but if
they have racing callbacks they get what they ask for.

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: initialize new_backfill in ctor

*** CID 1260213:  Uninitialized scalar field  (UNINIT_CTOR)
/osd/ReplicatedPG.cc: 1242 in ReplicatedPG::ReplicatedPG(OSDService *,
std::tr1::shared_ptr<const OSDMap>, const PGPool &, spg_t)()
1236       snap_trimmer_machine(this)
1237     {
1238       missing_loc.set_backend_predicates(
1239         pgbackend->get_is_readable_predicate(),
1240         pgbackend->get_is_recoverable_predicate());
1241       snap_trimmer_machine.initiate();
>>>     CID 1260213:  Uninitialized scalar field  (UNINIT_CTOR)
>>>     Non-static class member "new_backfill" is not initialized in this
constructor nor in any functions that it calls.
1242     }
1243
1244     void ReplicatedPG::get_src_oloc(const object_t& oid, const
object_locator_t& oloc, object_locator_t& src_oloc)
1245     {
1246       src_oloc = oloc;
1247       if (oloc.key.empty())

Signed-off-by: Sage Weil <sage@redhat.com>
remove unmatched op code comparasion

Signed-off-by: Dong Lei <leidong@yahoo-inc.com>
osd: accumulate authoritative peers during recovery

When PGBackend::be_compare_scrubmaps finds multiple good peers, it only
keeps the last one. This is fine for replication but erasure coding
needs to know all good peers for recovery.

PGBackend::be_compare_scrubmaps is modified to accumulate all good peers
and return them to PGBackend::be_compare_scrubmaps and indirectly to
PG::scrub_compare_maps.

PG::scrub_compare_maps will dispatch the good peers to authmap and
good_peers. In the case of authmap, the data structure is not modified
and only the last good peer is set. The ReplicatedPG::_scrub uses
authmap in a non trivial way and it should probably be modified to use
information from multiple good peers instead of just the last one. This
could be the focus of another change.

The scrubber.authoritative data structure is changed to include a list
of pair<ScrubMap::object, pg_shard_t> instead of a single
pair<ScrubMap::object, pg_shard_t> to pass to PG::repair_object and
allow it to add all good peers to the missing_loc locations if the
primary has a missing object. It could be just a list of pg_shard_t
instead because the ScrubMap::object is not used but makes more sense to
keep both and it will presumably be useful when / if the logic changes.

http://tracker.ceph.com/issues/10017 Fixes: #10017

Signed-off-by: Loic Dachary <ldachary@redhat.com>
erasure-code: test repair when file is removed

Add tests for when files disappear from the file system :

  * file is missing from the primary OSD
  * file is missing from an OSD that is not the primary
  * files are missing from two OSDs that are not the primary
  * files are missing from two OSDs, one of which is the primary

http://tracker.ceph.com/issues/10017 Refs: #10017

Signed-off-by: Loic Dachary <ldachary@redhat.com>
Merge pull request #3229 from ceph/wip-osd-ctor

osd/ReplicatedPG: initialize new_backfill in ctor

Reviewed-by: Samuel Just <sjust@redhat.com>
librados: warn about rados_watch_flush() prior to ioctx shutdown

Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #3222 from ceph/wip-watch-notify-2

librados: fix watch/notify event flushing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Formatter: add TableFormatter class for more human readable and shell parsable output

Signed-off-by: Andreas Peters <andreas.joachim.peters@cern.ch>
Formatting

Signed-off-by: Andreas Peters <andreas.joachim.peters@cern.ch>
librados: adding RadosWhereis class to provide an object location API

Signed-off-by: Andreas Peters <andreas.joachim.peters@cern.ch>
rados tool: adding 'whereis [--dns]' command and '--format table[-kv]' for table output format

Signed-off-by: Andreas Peters <andreas.joachim.peters@cern.ch>
Merge pull request #3200 from boydc2014/master

fix unmatched op code

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Merge remote-tracking branches 'dachary/wip-10017-erasure-code-repair', 'dachary/wip-rados-whereis-cmd', 'guangyy/wip-10183', 'majianpeng/fix1', 'mslovy/optimazation_wbthrottle' and 'upstream/master' into wip-sam-testing