debian: fix ceph-fs-common-dbg depends

Signed-off-by: James Page <james.page@ubuntu.com>
rgw: don't override subuser perm mask if perm not specified

Bug #2650. We were overriding subuser perm mask whenever subuser
was modified, even if perm mask was not passed.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
mon: purge removed osds from [near]full sets

The [near]full sets are volatile state.  Remove removed (or created)
osds from the set when we process a map.

Fixes: #2779
Signed-off-by: Sage Weil <sage@inktank.com>
mon: use single helper for [near]full sets

Use a single helper to add/remove osds from the [near]full sets.  This
keeps the logic in a single place, and simplifies the code somewhat.

Signed-off-by: Sage Weil <sage@inktank.com>
qa: download tests from specified branch

These python tests aren't installed, so they need to be downloaded

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
PG: merge_log always use stats from authoritative replica

If the osd recieving the log has divergent entries, it will
also have a "divergent" stat structure.  In general, it suffices
to simply trust the stat structure shipped with the authoritative
log and info since merge_log is only used to merge an authoritative
log.

Probably fixes #2769.

In cases like #2769, this bug can result in a primary with a stat
structure which double counts an operation: once for the
divergent operation, and once for the replay.  It turned up
in a regression suite run as a scrub stat mismatch.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: send_still_alive when we get a reply if we reported failure

When we get a ping reply, remove the peer from the failure_queue
and send a still alive message if the peer is in the failure_pending
map.

Otherwise, the monitor could slowly accumulate sporadic failure reports
leading to an osd being incorrectly marked out.

This bug may have been contributing to the wrongly-marked-down
thrashing observed on some systems.

Signed-off-by: Samuel Just <sam.just@inktank.com>
qa: download tests from specified branch

These python tests aren't installed, so they need to be downloaded

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
doc: cookbooks issue resolved, so changed 'ceph-cookbooks' back to 'ceph.'

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
doc: remove last reference to ceph-cookbooks.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
crushtool: allow information generated during testing to be dumped
to a set of CSV files for off-line analysis.

Signed-off-by: caleb miles <caleb.miles@inktank.com>
OSD: add config options to fake missed pings

In order to test monitor and osd failure detection and false
positive correction, this patch adds the following options:

 1. osd_debug_drop_ping_probability: probability of dropping
    a string of pings from a client upon ping recipt.
 2. osd_debug_drop_ping_duration: number of pings to drop in
    a row.

This should help with replicating some wrongly-marked-down
thrashing cases.

Signed-off-by: Samuel Just <sam.just@inktank.com>
vstart: use absolute path for keyring

Stores absolute path to the generated keyring so that tests running in
other directories (e.g. src/java/test) can simply reference the
generated ceph.conf.

Signed-off-by: Noah Watkins <jawhawk@cs.ucsc.edu>
osd: simplify helper usage for misdirected ops

Make the helper exclusively for the PG != NULL cases, and open-code the
one PG == NULL caller.  This is simpler, and lets us include more useful
information in the log message.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: based misdirected op role calc on acting set

We want to look at the acting set here, nothing else.  This was causing us
to erroneously queue ops for later (wasting memory) and to erroneously
print out a 'misdrected op' message in the cluster log (confusion and
incorrect [but ignored] -ENXIO reply).

Fixes: #2022
Signed-off-by: Sage Weil <sage@inktank.com>
osdmap: drop useless and unused get_pg_role() method

Users probably want get_pg_acting_rank().  If they don't, they can probably
have the mapping and can calculate the rank themselves.  Having this here
is asking for bugs like #2022.

Signed-off-by: Sage Weil <sage@inktank.com>
filestore: dump open fds when we hit EMFILE

Use a helper to dump /proc/self/fd when we hit EMFILE in the filestore.
Ideally, we should trigger this in other appropriate places, but it is
not immediately clear that there is a sane way to do that.

Fixes: #2330
Signed-off-by: Sage Weil <sage@inktank.com>
mon/MonitorStore: always O_TRUNC when writing states

It is possible for a .new file to already exist, potentially with a
larger size.  This would happen if:

 - we were proposing a different value
 - we crashed (or were stopped) before it got renamed into place
 - after restarting, a different value was proposed and accepted.

This isn't so unlikely for the log state machine, where we're
aggregating random messages.  O_TRUNC ensure we avoid getting the tail
end of some previous junk.

I observed #2593 and found that a logm state value had a larger size on
one mon (after slurping) than the others, pointing to put_bl_sn_map().

While we are at it, O_TRUNC put_int() too; the same type of bug is
possible there, too.

Fixes: #2593
Signed-off-by: Sage Weil <sage@inktank.com>
mon: remove osds from [near]full sets when their stats are removed from pgmap

Greg points out that we could have a situation like:

 - mon recovers..
 - goes through osdmaps, notes an osd was removed and removes from
   full/nearfull
 - goes through pgmaps, and re-adds it when it encounters some osd_stat_ts.

Fix this by removing the osd from the full/nearfull set when we remove
the osd_stat_t from the pgmap.  Any osd removal is always followed by
an osd_stat_rm[] record when the primary processes the new osdmap and
proposed the appropriate pgmap updates.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge remote-tracking branch 'gh/bugfix-2779'

Reviewed-by: Greg Farnum <greg@inktank.com>